2024-05-30T15:40:13.038 INFO:root:teuthology version: 0.0.1.dev296+ge1979f0 2024-05-30T15:40:13.038 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name cbodley-2024-05-30_15:09:56-rgw-wip-cbodley-testing-distro-default-smithi --description rgw/verify/{0-install accounts$/{main-tenant} clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} --owner scheduled_cbodley@teuthology --archive /home/teuthworker/archive/cbodley-2024-05-30_15:09:56-rgw-wip-cbodley-testing-distro-default-smithi/7734120 -- /home/teuthworker/archive/cbodley-2024-05-30_15:09:56-rgw-wip-cbodley-testing-distro-default-smithi/7734120/orig.config.yaml 2024-05-30T15:40:13.085 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-30T15:40:13.169 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/cbodley-2024-05-30_15:09:56-rgw-wip-cbodley-testing-distro-default-smithi/7734120 branch: wip-cbodley-testing description: rgw/verify/{0-install accounts$/{main-tenant} clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} email: null first_in_suite: false job_id: '7734120' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: cbodley-2024-05-30_15:09:56-rgw-wip-cbodley-testing-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-cbodley-testing ceph: conf: client: debug rgw: 20 debug rgw notification: 20 rgw bucket counters cache: true rgw crypt require ssl: false rgw crypt s3 kms backend: testing rgw crypt s3 kms encryption keys: testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= rgw d3n l1 datacache persistent path: /tmp/rgw_datacache/ rgw d3n l1 datacache size: 10737418240 rgw d3n l1 local datacache enabled: true rgw enable ops log: true rgw max chunk size: 4194304 rgw obj stripe size: 6291456 rgw s3 auth use sts: true rgw sts key: abcdefghijklmnop rgw torrent flag: true rgw user counters cache: true setgroup: ceph setuser: ceph global: mon client directed command retry: 5 ms inject socket failures: 5000 osd heartbeat grace: 40 osd_max_pg_log_entries: 10 osd_min_pg_log_entries: 10 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon osd crush smoke test: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd fast shutdown: false osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) - \(POOL_APP_NOT_ENABLED\) - not have an application enabled - \(OSD_SLOW_PING_TIME sha1: da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 rgw: client.0: valgrind: - --tool=memcheck - --max-threads=1024 compression type: random datacache: true datacache_path: /tmp/rgw_datacache ec-data-pool: true frontend: beast storage classes: LUKEWARM, FROZEN s3tests: accounts: iam alt root: RGW99999999999999999 iam root: RGW88888888888888888 s3 main: RGW11111111111111111 s3 tenant: RGW33333333333333333 force-branch: ceph-master slow_backend: true storage classes: LUKEWARM, FROZEN thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-cbodley-testing sha1: da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 owner: scheduled_cbodley@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 5740 sha1: da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 sleep_before_teardown: 0 suite: rgw suite_branch: wip-cbodley-testing suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 targets: smithi006.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDR4d1KWa9GgYfcJ1JEAH57mEqxzqJIgppsOibV5cfLNbiYeju7wgfxNRm48qMwhqyJPw6XxF1ttJs/fHDVfprA= smithi204.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNpvmHyUurLgakidLXapfD9qXq/0T3au9KSzbm3sB6iRmJXVzk4rKyJJhMLRMArWvT9zkezUQnALjIQNsGkPe38= tasks: - install: extra_system_packages: deb: - s3cmd - maven rpm: - s3cmd - maven - ceph: null - openssl_keys: null - rgw: client.0: null - tox: - client.0 - workunit: clients: client.0: - rgw/run-datacache.sh env: RGW_DATACACHE_PATH: /tmp/rgw_datacache - workunit: clients: client.0: - rgw/run-bucket-check.sh - workunit: clients: client.0: - cls/test_cls_lock.sh - cls/test_cls_log.sh - cls/test_cls_refcount.sh - cls/test_cls_rgw.sh - cls/test_cls_rgw_gc.sh - cls/test_cls_rgw_stats.sh - cls/test_cls_cmpomap.sh - cls/test_cls_2pc_queue.sh - cls/test_cls_user.sh - rgw/test_rgw_gc_log.sh - rgw/test_rgw_obj.sh - rgw/test_librgw_file.sh - rgw/test_awssdkv4_sig.sh - workunit: clients: client.0: - rgw/test_rgw_s3_mp_reupload.sh - ragweed: client.0: default-branch: ceph-master rgw_server: client.0 stages: prepare,check - workunit: clients: client.0: - rgw/run-reshard.sh - s3tests-java: client.0: force-branch: ceph-master force-repo: https://github.com/ceph/java_s3tests.git - s3tests: client.0: rgw_server: client.0 - workunit: clients: client.0: - rgw/run-versioning.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: e1979f064b0becde2020282c7e87cf3c5e98026c timestamp: 2024-05-30_15:09:56 tube: smithi user: cbodley verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-05-30T15:40:13.169 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/qa; will attempt to use it 2024-05-30T15:40:13.170 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/qa/tasks 2024-05-30T15:40:13.170 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-30T15:40:13.171 INFO:teuthology.task.internal:Checking packages... 2024-05-30T15:40:13.192 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2' 2024-05-30T15:40:13.192 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-30T15:40:13.192 INFO:teuthology.packaging:ref: None 2024-05-30T15:40:13.192 INFO:teuthology.packaging:tag: None 2024-05-30T15:40:13.192 INFO:teuthology.packaging:branch: wip-cbodley-testing 2024-05-30T15:40:13.192 INFO:teuthology.packaging:sha1: da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 2024-05-30T15:40:13.193 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-cbodley-testing 2024-05-30T15:40:13.437 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-4011-gda98024b-1jammy 2024-05-30T15:40:13.438 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-30T15:40:13.444 INFO:teuthology.task.internal:no buildpackages task found 2024-05-30T15:40:13.444 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-30T15:40:13.490 INFO:teuthology.task.internal:Saving configuration 2024-05-30T15:40:13.506 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-30T15:40:13.525 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-30T15:40:13.548 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi006.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/cbodley-2024-05-30_15:09:56-rgw-wip-cbodley-testing-distro-default-smithi/7734120', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-30 15:34:18.240122', 'locked_by': 'scheduled_cbodley@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDQIDMeYdsQeEl3N4HuxI4iaxaimy+5R2upK3XYe2yG8'} 2024-05-30T15:40:13.572 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/cbodley-2024-05-30_15:09:56-rgw-wip-cbodley-testing-distro-default-smithi/7734120', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-30 15:34:18.238667', 'locked_by': 'scheduled_cbodley@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICWL/Tvwais4NNAnV4j099Y5RaWUjWFZJhvWzSirzL15'} 2024-05-30T15:40:13.572 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-30T15:40:13.577 INFO:teuthology.task.internal:roles: ubuntu@smithi006.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-05-30T15:40:13.577 INFO:teuthology.task.internal:roles: ubuntu@smithi204.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-05-30T15:40:13.577 INFO:teuthology.run_tasks:Running task console_log... 2024-05-30T15:40:13.661 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb4b1ee3ac0>, signals=[15]) 2024-05-30T15:40:13.661 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-30T15:40:13.666 INFO:teuthology.task.internal:Opening connections... 2024-05-30T15:40:13.667 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi006.front.sepia.ceph.com 2024-05-30T15:40:13.668 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-30T15:40:13.747 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi204.front.sepia.ceph.com 2024-05-30T15:40:13.748 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-30T15:40:13.826 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-30T15:40:13.831 DEBUG:teuthology.orchestra.run.smithi006:> uname -m 2024-05-30T15:40:13.837 INFO:teuthology.orchestra.run.smithi006.stdout:x86_64 2024-05-30T15:40:13.837 DEBUG:teuthology.orchestra.run.smithi006:> cat /etc/os-release 2024-05-30T15:40:13.885 INFO:teuthology.orchestra.run.smithi006.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-05-30T15:40:13.885 INFO:teuthology.orchestra.run.smithi006.stdout:NAME="Ubuntu" 2024-05-30T15:40:13.885 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION_ID="22.04" 2024-05-30T15:40:13.885 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-05-30T15:40:13.885 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION_CODENAME=jammy 2024-05-30T15:40:13.885 INFO:teuthology.orchestra.run.smithi006.stdout:ID=ubuntu 2024-05-30T15:40:13.886 INFO:teuthology.orchestra.run.smithi006.stdout:ID_LIKE=debian 2024-05-30T15:40:13.886 INFO:teuthology.orchestra.run.smithi006.stdout:HOME_URL="https://www.ubuntu.com/" 2024-05-30T15:40:13.886 INFO:teuthology.orchestra.run.smithi006.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-05-30T15:40:13.886 INFO:teuthology.orchestra.run.smithi006.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-05-30T15:40:13.886 INFO:teuthology.orchestra.run.smithi006.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-05-30T15:40:13.886 INFO:teuthology.orchestra.run.smithi006.stdout:UBUNTU_CODENAME=jammy 2024-05-30T15:40:13.887 INFO:teuthology.lock.ops:Updating smithi006.front.sepia.ceph.com on lock server 2024-05-30T15:40:13.908 DEBUG:teuthology.orchestra.run.smithi204:> uname -m 2024-05-30T15:40:13.913 INFO:teuthology.orchestra.run.smithi204.stdout:x86_64 2024-05-30T15:40:13.913 DEBUG:teuthology.orchestra.run.smithi204:> cat /etc/os-release 2024-05-30T15:40:13.959 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-05-30T15:40:13.959 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="Ubuntu" 2024-05-30T15:40:13.959 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="22.04" 2024-05-30T15:40:13.959 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-05-30T15:40:13.959 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_CODENAME=jammy 2024-05-30T15:40:13.959 INFO:teuthology.orchestra.run.smithi204.stdout:ID=ubuntu 2024-05-30T15:40:13.960 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE=debian 2024-05-30T15:40:13.960 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://www.ubuntu.com/" 2024-05-30T15:40:13.960 INFO:teuthology.orchestra.run.smithi204.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-05-30T15:40:13.960 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-05-30T15:40:13.960 INFO:teuthology.orchestra.run.smithi204.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-05-30T15:40:13.960 INFO:teuthology.orchestra.run.smithi204.stdout:UBUNTU_CODENAME=jammy 2024-05-30T15:40:13.960 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2024-05-30T15:40:13.983 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-30T15:40:13.992 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-30T15:40:14.040 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-30T15:40:14.041 DEBUG:teuthology.orchestra.run.smithi006:> test '!' -e /home/ubuntu/cephtest 2024-05-30T15:40:14.043 DEBUG:teuthology.orchestra.run.smithi204:> test '!' -e /home/ubuntu/cephtest 2024-05-30T15:40:14.048 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-30T15:40:14.066 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-30T15:40:14.066 DEBUG:teuthology.orchestra.run.smithi006:> test -z $(ls -A /var/lib/ceph) 2024-05-30T15:40:14.090 DEBUG:teuthology.orchestra.run.smithi204:> test -z $(ls -A /var/lib/ceph) 2024-05-30T15:40:14.100 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-30T15:40:14.154 INFO:teuthology.run_tasks:Running task kernel... 2024-05-30T15:40:14.167 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-05-30T15:40:14.167 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-05-30T15:40:14.168 DEBUG:teuthology.orchestra.run.smithi006:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-30T15:40:14.168 DEBUG:teuthology.orchestra.run.smithi204:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-30T15:40:14.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-30T15:40:14.173 DEBUG:teuthology.orchestra.run.smithi006:> uname -r 2024-05-30T15:40:14.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-30T15:40:14.174 DEBUG:teuthology.orchestra.run.smithi204:> uname -r 2024-05-30T15:40:14.219 INFO:teuthology.orchestra.run.smithi204.stdout:5.15.0-56-generic 2024-05-30T15:40:14.219 INFO:teuthology.task.kernel:Running kernel on smithi204: 5.15.0-56-generic 2024-05-30T15:40:14.220 DEBUG:teuthology.orchestra.run.smithi204:> sudo apt-get clean 2024-05-30T15:40:14.222 INFO:teuthology.orchestra.run.smithi006.stdout:5.15.0-56-generic 2024-05-30T15:40:14.222 INFO:teuthology.task.kernel:Running kernel on smithi006: 5.15.0-56-generic 2024-05-30T15:40:14.222 DEBUG:teuthology.orchestra.run.smithi006:> sudo apt-get clean 2024-05-30T15:40:14.351 DEBUG:teuthology.orchestra.run.smithi204:> sudo apt-get update 2024-05-30T15:40:14.354 DEBUG:teuthology.orchestra.run.smithi006:> sudo apt-get update 2024-05-30T15:40:14.493 INFO:teuthology.orchestra.run.smithi204.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-05-30T15:40:14.530 INFO:teuthology.orchestra.run.smithi006.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-30T15:40:14.541 INFO:teuthology.orchestra.run.smithi006.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-05-30T15:40:14.603 INFO:teuthology.orchestra.run.smithi204.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-30T15:40:14.631 INFO:teuthology.orchestra.run.smithi006.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-05-30T15:40:14.641 INFO:teuthology.orchestra.run.smithi006.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-05-30T15:40:14.685 INFO:teuthology.orchestra.run.smithi204.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-05-30T15:40:14.740 INFO:teuthology.orchestra.run.smithi204.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,475 kB] 2024-05-30T15:40:14.779 INFO:teuthology.orchestra.run.smithi006.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,686 kB] 2024-05-30T15:40:14.884 INFO:teuthology.orchestra.run.smithi204.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [471 kB] 2024-05-30T15:40:14.891 INFO:teuthology.orchestra.run.smithi204.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [254 kB] 2024-05-30T15:40:14.896 INFO:teuthology.orchestra.run.smithi204.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-05-30T15:40:14.897 INFO:teuthology.orchestra.run.smithi204.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-05-30T15:40:14.898 INFO:teuthology.orchestra.run.smithi204.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,876 kB] 2024-05-30T15:40:14.916 INFO:teuthology.orchestra.run.smithi006.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [633 kB] 2024-05-30T15:40:14.936 INFO:teuthology.orchestra.run.smithi006.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [313 kB] 2024-05-30T15:40:14.942 INFO:teuthology.orchestra.run.smithi204.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [318 kB] 2024-05-30T15:40:14.946 INFO:teuthology.orchestra.run.smithi006.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-05-30T15:40:14.947 INFO:teuthology.orchestra.run.smithi006.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-05-30T15:40:14.948 INFO:teuthology.orchestra.run.smithi006.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,933 kB] 2024-05-30T15:40:14.951 INFO:teuthology.orchestra.run.smithi204.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-05-30T15:40:14.951 INFO:teuthology.orchestra.run.smithi204.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [604 kB] 2024-05-30T15:40:14.966 INFO:teuthology.orchestra.run.smithi204.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [854 kB] 2024-05-30T15:40:14.981 INFO:teuthology.orchestra.run.smithi204.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [165 kB] 2024-05-30T15:40:14.983 INFO:teuthology.orchestra.run.smithi204.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-05-30T15:40:14.991 INFO:teuthology.orchestra.run.smithi204.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-05-30T15:40:14.991 INFO:teuthology.orchestra.run.smithi204.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-05-30T15:40:14.998 INFO:teuthology.orchestra.run.smithi006.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [328 kB] 2024-05-30T15:40:15.004 INFO:teuthology.orchestra.run.smithi006.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-05-30T15:40:15.005 INFO:teuthology.orchestra.run.smithi006.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [703 kB] 2024-05-30T15:40:15.012 INFO:teuthology.orchestra.run.smithi204.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-05-30T15:40:15.024 INFO:teuthology.orchestra.run.smithi006.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,076 kB] 2024-05-30T15:40:15.036 INFO:teuthology.orchestra.run.smithi204.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-05-30T15:40:15.039 INFO:teuthology.orchestra.run.smithi006.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [246 kB] 2024-05-30T15:40:15.042 INFO:teuthology.orchestra.run.smithi006.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-05-30T15:40:15.045 INFO:teuthology.orchestra.run.smithi006.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-05-30T15:40:15.046 INFO:teuthology.orchestra.run.smithi204.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-05-30T15:40:15.046 INFO:teuthology.orchestra.run.smithi006.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-05-30T15:40:15.046 INFO:teuthology.orchestra.run.smithi006.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-05-30T15:40:15.066 INFO:teuthology.orchestra.run.smithi006.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-05-30T15:40:15.084 INFO:teuthology.orchestra.run.smithi006.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-05-30T15:40:15.084 INFO:teuthology.orchestra.run.smithi006.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.3 kB] 2024-05-30T15:40:15.085 INFO:teuthology.orchestra.run.smithi006.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-05-30T15:40:15.086 INFO:teuthology.orchestra.run.smithi006.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,475 kB] 2024-05-30T15:40:15.090 INFO:teuthology.orchestra.run.smithi006.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-05-30T15:40:15.091 INFO:teuthology.orchestra.run.smithi006.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [16.0 kB] 2024-05-30T15:40:15.091 INFO:teuthology.orchestra.run.smithi006.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [27.2 kB] 2024-05-30T15:40:15.091 INFO:teuthology.orchestra.run.smithi006.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-05-30T15:40:15.092 INFO:teuthology.orchestra.run.smithi006.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-05-30T15:40:15.188 INFO:teuthology.orchestra.run.smithi204.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,686 kB] 2024-05-30T15:40:15.486 INFO:teuthology.orchestra.run.smithi006.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [471 kB] 2024-05-30T15:40:15.491 INFO:teuthology.orchestra.run.smithi204.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [633 kB] 2024-05-30T15:40:15.504 INFO:teuthology.orchestra.run.smithi006.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [254 kB] 2024-05-30T15:40:15.513 INFO:teuthology.orchestra.run.smithi006.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-05-30T15:40:15.514 INFO:teuthology.orchestra.run.smithi006.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,876 kB] 2024-05-30T15:40:15.528 INFO:teuthology.orchestra.run.smithi204.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [313 kB] 2024-05-30T15:40:15.542 INFO:teuthology.orchestra.run.smithi204.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-05-30T15:40:15.544 INFO:teuthology.orchestra.run.smithi204.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-05-30T15:40:15.545 INFO:teuthology.orchestra.run.smithi204.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,933 kB] 2024-05-30T15:40:15.607 INFO:teuthology.orchestra.run.smithi006.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-05-30T15:40:15.608 INFO:teuthology.orchestra.run.smithi204.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [328 kB] 2024-05-30T15:40:15.608 INFO:teuthology.orchestra.run.smithi006.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [318 kB] 2024-05-30T15:40:15.619 INFO:teuthology.orchestra.run.smithi204.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-05-30T15:40:15.620 INFO:teuthology.orchestra.run.smithi006.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-05-30T15:40:15.620 INFO:teuthology.orchestra.run.smithi006.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [854 kB] 2024-05-30T15:40:15.646 INFO:teuthology.orchestra.run.smithi006.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [604 kB] 2024-05-30T15:40:15.784 INFO:teuthology.orchestra.run.smithi204.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,076 kB] 2024-05-30T15:40:15.823 INFO:teuthology.orchestra.run.smithi006.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [165 kB] 2024-05-30T15:40:16.241 INFO:teuthology.orchestra.run.smithi006.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-05-30T15:40:16.247 INFO:teuthology.orchestra.run.smithi006.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-05-30T15:40:16.247 INFO:teuthology.orchestra.run.smithi006.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-05-30T15:40:16.261 INFO:teuthology.orchestra.run.smithi006.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-05-30T15:40:16.264 INFO:teuthology.orchestra.run.smithi006.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-05-30T15:40:16.417 INFO:teuthology.orchestra.run.smithi204.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [703 kB] 2024-05-30T15:40:16.455 INFO:teuthology.orchestra.run.smithi204.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [246 kB] 2024-05-30T15:40:16.468 INFO:teuthology.orchestra.run.smithi204.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-05-30T15:40:16.469 INFO:teuthology.orchestra.run.smithi204.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-05-30T15:40:16.469 INFO:teuthology.orchestra.run.smithi204.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-05-30T15:40:16.471 INFO:teuthology.orchestra.run.smithi204.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-05-30T15:40:16.472 INFO:teuthology.orchestra.run.smithi204.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-05-30T15:40:16.472 INFO:teuthology.orchestra.run.smithi204.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.3 kB] 2024-05-30T15:40:16.499 INFO:teuthology.orchestra.run.smithi204.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-05-30T15:40:16.503 INFO:teuthology.orchestra.run.smithi204.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-05-30T15:40:16.667 INFO:teuthology.orchestra.run.smithi204.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-05-30T15:40:16.750 INFO:teuthology.orchestra.run.smithi204.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [27.2 kB] 2024-05-30T15:40:16.857 INFO:teuthology.orchestra.run.smithi204.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [16.0 kB] 2024-05-30T15:40:16.913 INFO:teuthology.orchestra.run.smithi204.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-05-30T15:40:16.944 INFO:teuthology.orchestra.run.smithi204.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-05-30T15:40:20.977 INFO:teuthology.orchestra.run.smithi006.stdout:Fetched 13.8 MB in 2s (6,768 kB/s) 2024-05-30T15:40:21.232 INFO:teuthology.orchestra.run.smithi204.stdout:Fetched 13.8 MB in 3s (5,451 kB/s) 2024-05-30T15:40:22.494 INFO:teuthology.orchestra.run.smithi006.stdout:Reading package lists... 2024-05-30T15:40:22.513 DEBUG:teuthology.orchestra.run.smithi006:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-05-30T15:40:22.581 INFO:teuthology.orchestra.run.smithi006.stdout:Reading package lists... 2024-05-30T15:40:22.597 INFO:teuthology.orchestra.run.smithi204.stdout:Reading package lists... 2024-05-30T15:40:22.614 DEBUG:teuthology.orchestra.run.smithi204:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-05-30T15:40:22.677 INFO:teuthology.orchestra.run.smithi204.stdout:Reading package lists... 2024-05-30T15:40:22.795 INFO:teuthology.orchestra.run.smithi006.stdout:Building dependency tree... 2024-05-30T15:40:22.795 INFO:teuthology.orchestra.run.smithi006.stdout:Reading state information... 2024-05-30T15:40:22.887 INFO:teuthology.orchestra.run.smithi204.stdout:Building dependency tree... 2024-05-30T15:40:22.888 INFO:teuthology.orchestra.run.smithi204.stdout:Reading state information... 2024-05-30T15:40:22.982 INFO:teuthology.orchestra.run.smithi006.stdout:The following packages were automatically installed and are no longer required: 2024-05-30T15:40:22.982 INFO:teuthology.orchestra.run.smithi006.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-30T15:40:22.983 INFO:teuthology.orchestra.run.smithi006.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-30T15:40:22.983 INFO:teuthology.orchestra.run.smithi006.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-30T15:40:22.983 INFO:teuthology.orchestra.run.smithi006.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-05-30T15:40:22.983 INFO:teuthology.orchestra.run.smithi006.stdout: python2.7-minimal 2024-05-30T15:40:22.983 INFO:teuthology.orchestra.run.smithi006.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-30T15:40:22.985 INFO:teuthology.orchestra.run.smithi006.stdout:The following additional packages will be installed: 2024-05-30T15:40:22.985 INFO:teuthology.orchestra.run.smithi006.stdout: linux-generic linux-headers-5.15.0-107 linux-headers-5.15.0-107-generic 2024-05-30T15:40:22.985 INFO:teuthology.orchestra.run.smithi006.stdout: linux-headers-generic linux-image-5.15.0-107-generic 2024-05-30T15:40:22.986 INFO:teuthology.orchestra.run.smithi006.stdout: linux-modules-5.15.0-107-generic linux-modules-extra-5.15.0-107-generic 2024-05-30T15:40:22.987 INFO:teuthology.orchestra.run.smithi006.stdout:Suggested packages: 2024-05-30T15:40:22.987 INFO:teuthology.orchestra.run.smithi006.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-05-30T15:40:22.987 INFO:teuthology.orchestra.run.smithi006.stdout:Recommended packages: 2024-05-30T15:40:22.987 INFO:teuthology.orchestra.run.smithi006.stdout: thermald 2024-05-30T15:40:23.040 INFO:teuthology.orchestra.run.smithi006.stdout:The following NEW packages will be installed: 2024-05-30T15:40:23.040 INFO:teuthology.orchestra.run.smithi006.stdout: linux-headers-5.15.0-107 linux-headers-5.15.0-107-generic 2024-05-30T15:40:23.040 INFO:teuthology.orchestra.run.smithi006.stdout: linux-image-5.15.0-107-generic linux-modules-5.15.0-107-generic 2024-05-30T15:40:23.041 INFO:teuthology.orchestra.run.smithi006.stdout: linux-modules-extra-5.15.0-107-generic 2024-05-30T15:40:23.042 INFO:teuthology.orchestra.run.smithi006.stdout:The following packages will be upgraded: 2024-05-30T15:40:23.043 INFO:teuthology.orchestra.run.smithi006.stdout: linux-generic linux-headers-generic linux-image-generic 2024-05-30T15:40:23.060 INFO:teuthology.orchestra.run.smithi204.stdout:The following packages were automatically installed and are no longer required: 2024-05-30T15:40:23.061 INFO:teuthology.orchestra.run.smithi204.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-30T15:40:23.061 INFO:teuthology.orchestra.run.smithi204.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-30T15:40:23.061 INFO:teuthology.orchestra.run.smithi204.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-30T15:40:23.061 INFO:teuthology.orchestra.run.smithi204.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-05-30T15:40:23.062 INFO:teuthology.orchestra.run.smithi204.stdout: python2.7-minimal 2024-05-30T15:40:23.062 INFO:teuthology.orchestra.run.smithi204.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-30T15:40:23.063 INFO:teuthology.orchestra.run.smithi204.stdout:The following additional packages will be installed: 2024-05-30T15:40:23.063 INFO:teuthology.orchestra.run.smithi204.stdout: linux-generic linux-headers-5.15.0-107 linux-headers-5.15.0-107-generic 2024-05-30T15:40:23.063 INFO:teuthology.orchestra.run.smithi204.stdout: linux-headers-generic linux-image-5.15.0-107-generic 2024-05-30T15:40:23.064 INFO:teuthology.orchestra.run.smithi204.stdout: linux-modules-5.15.0-107-generic linux-modules-extra-5.15.0-107-generic 2024-05-30T15:40:23.065 INFO:teuthology.orchestra.run.smithi204.stdout:Suggested packages: 2024-05-30T15:40:23.065 INFO:teuthology.orchestra.run.smithi204.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-05-30T15:40:23.065 INFO:teuthology.orchestra.run.smithi204.stdout:Recommended packages: 2024-05-30T15:40:23.065 INFO:teuthology.orchestra.run.smithi204.stdout: thermald 2024-05-30T15:40:23.114 INFO:teuthology.orchestra.run.smithi204.stdout:The following NEW packages will be installed: 2024-05-30T15:40:23.114 INFO:teuthology.orchestra.run.smithi204.stdout: linux-headers-5.15.0-107 linux-headers-5.15.0-107-generic 2024-05-30T15:40:23.114 INFO:teuthology.orchestra.run.smithi204.stdout: linux-image-5.15.0-107-generic linux-modules-5.15.0-107-generic 2024-05-30T15:40:23.115 INFO:teuthology.orchestra.run.smithi204.stdout: linux-modules-extra-5.15.0-107-generic 2024-05-30T15:40:23.116 INFO:teuthology.orchestra.run.smithi204.stdout:The following packages will be upgraded: 2024-05-30T15:40:23.116 INFO:teuthology.orchestra.run.smithi204.stdout: linux-generic linux-headers-generic linux-image-generic 2024-05-30T15:40:23.223 INFO:teuthology.orchestra.run.smithi006.stdout:3 upgraded, 5 newly installed, 0 to remove and 320 not upgraded. 2024-05-30T15:40:23.223 INFO:teuthology.orchestra.run.smithi006.stdout:Need to get 113 MB of archives. 2024-05-30T15:40:23.223 INFO:teuthology.orchestra.run.smithi006.stdout:After this operation, 583 MB of additional disk space will be used. 2024-05-30T15:40:23.223 INFO:teuthology.orchestra.run.smithi006.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-107-generic amd64 5.15.0-107.117 [22.7 MB] 2024-05-30T15:40:23.295 INFO:teuthology.orchestra.run.smithi204.stdout:3 upgraded, 5 newly installed, 0 to remove and 320 not upgraded. 2024-05-30T15:40:23.296 INFO:teuthology.orchestra.run.smithi204.stdout:Need to get 113 MB of archives. 2024-05-30T15:40:23.296 INFO:teuthology.orchestra.run.smithi204.stdout:After this operation, 583 MB of additional disk space will be used. 2024-05-30T15:40:23.296 INFO:teuthology.orchestra.run.smithi204.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-107-generic amd64 5.15.0-107.117 [22.7 MB] 2024-05-30T15:40:24.719 INFO:teuthology.orchestra.run.smithi204.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-107-generic amd64 5.15.0-107.117 [11.5 MB] 2024-05-30T15:40:24.760 INFO:teuthology.orchestra.run.smithi006.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-107-generic amd64 5.15.0-107.117 [11.5 MB] 2024-05-30T15:40:25.074 INFO:teuthology.orchestra.run.smithi204.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-107-generic amd64 5.15.0-107.117 [63.9 MB] 2024-05-30T15:40:25.148 INFO:teuthology.orchestra.run.smithi006.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-107-generic amd64 5.15.0-107.117 [63.9 MB] 2024-05-30T15:40:27.295 INFO:teuthology.orchestra.run.smithi006.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.107.107 [1,694 B] 2024-05-30T15:40:27.295 INFO:teuthology.orchestra.run.smithi006.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.107.107 [2,496 B] 2024-05-30T15:40:27.296 INFO:teuthology.orchestra.run.smithi006.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-107 all 5.15.0-107.117 [12.3 MB] 2024-05-30T15:40:27.340 INFO:teuthology.orchestra.run.smithi204.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.107.107 [1,694 B] 2024-05-30T15:40:27.340 INFO:teuthology.orchestra.run.smithi204.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.107.107 [2,496 B] 2024-05-30T15:40:27.341 INFO:teuthology.orchestra.run.smithi204.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-107 all 5.15.0-107.117 [12.3 MB] 2024-05-30T15:40:27.747 INFO:teuthology.orchestra.run.smithi204.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-107-generic amd64 5.15.0-107.117 [2,887 kB] 2024-05-30T15:40:27.828 INFO:teuthology.orchestra.run.smithi204.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.107.107 [2,354 B] 2024-05-30T15:40:28.067 INFO:teuthology.orchestra.run.smithi006.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-107-generic amd64 5.15.0-107.117 [2,887 kB] 2024-05-30T15:40:28.187 INFO:teuthology.orchestra.run.smithi006.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.107.107 [2,354 B] 2024-05-30T15:40:28.253 INFO:teuthology.orchestra.run.smithi204.stdout:Fetched 113 MB in 5s (24.1 MB/s) 2024-05-30T15:40:28.433 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package linux-modules-5.15.0-107-generic. 2024-05-30T15:40:28.589 INFO:teuthology.orchestra.run.smithi006.stdout:Fetched 113 MB in 5s (22.1 MB/s) 2024-05-30T15:40:28.765 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package linux-modules-5.15.0-107-generic. 2024-05-30T15:40:30.446 INFO:teuthology.orchestra.run.smithi204.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-05-30T15:40:30.450 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../0-linux-modules-5.15.0-107-generic_5.15.0-107.117_amd64.deb ... 2024-05-30T15:40:30.514 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking linux-modules-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-30T15:40:30.776 INFO:teuthology.orchestra.run.smithi006.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-05-30T15:40:30.781 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../0-linux-modules-5.15.0-107-generic_5.15.0-107.117_amd64.deb ... 2024-05-30T15:40:30.871 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking linux-modules-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-30T15:40:32.657 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package linux-image-5.15.0-107-generic. 2024-05-30T15:40:32.678 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../1-linux-image-5.15.0-107-generic_5.15.0-107.117_amd64.deb ... 2024-05-30T15:40:32.771 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking linux-image-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-30T15:40:33.056 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package linux-image-5.15.0-107-generic. 2024-05-30T15:40:33.077 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../1-linux-image-5.15.0-107-generic_5.15.0-107.117_amd64.deb ... 2024-05-30T15:40:33.160 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking linux-image-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-30T15:40:33.218 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-107-generic. 2024-05-30T15:40:33.239 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-107-generic_5.15.0-107.117_amd64.deb ... 2024-05-30T15:40:33.285 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking linux-modules-extra-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-30T15:40:33.651 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-107-generic. 2024-05-30T15:40:33.673 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-107-generic_5.15.0-107.117_amd64.deb ... 2024-05-30T15:40:33.709 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking linux-modules-extra-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-30T15:40:40.169 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../3-linux-generic_5.15.0.107.107_amd64.deb ... 2024-05-30T15:40:40.219 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../3-linux-generic_5.15.0.107.107_amd64.deb ... 2024-05-30T15:40:40.295 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking linux-generic (5.15.0.107.107) over (5.15.0.56.54) ... 2024-05-30T15:40:40.346 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking linux-generic (5.15.0.107.107) over (5.15.0.56.54) ... 2024-05-30T15:40:40.603 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.107.107_amd64.deb ... 2024-05-30T15:40:40.696 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.107.107_amd64.deb ... 2024-05-30T15:40:40.757 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking linux-image-generic (5.15.0.107.107) over (5.15.0.56.54) ... 2024-05-30T15:40:40.800 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking linux-image-generic (5.15.0.107.107) over (5.15.0.56.54) ... 2024-05-30T15:40:41.069 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package linux-headers-5.15.0-107. 2024-05-30T15:40:41.091 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../5-linux-headers-5.15.0-107_5.15.0-107.117_all.deb ... 2024-05-30T15:40:41.103 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package linux-headers-5.15.0-107. 2024-05-30T15:40:41.114 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../5-linux-headers-5.15.0-107_5.15.0-107.117_all.deb ... 2024-05-30T15:40:41.135 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking linux-headers-5.15.0-107 (5.15.0-107.117) ... 2024-05-30T15:40:41.153 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking linux-headers-5.15.0-107 (5.15.0-107.117) ... 2024-05-30T15:40:46.635 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package linux-headers-5.15.0-107-generic. 2024-05-30T15:40:46.671 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../6-linux-headers-5.15.0-107-generic_5.15.0-107.117_amd64.deb ... 2024-05-30T15:40:46.709 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking linux-headers-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-30T15:40:47.066 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package linux-headers-5.15.0-107-generic. 2024-05-30T15:40:47.104 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../6-linux-headers-5.15.0-107-generic_5.15.0-107.117_amd64.deb ... 2024-05-30T15:40:47.148 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking linux-headers-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-30T15:40:49.321 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.107.107_amd64.deb ... 2024-05-30T15:40:49.447 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking linux-headers-generic (5.15.0.107.107) over (5.15.0.56.54) ... 2024-05-30T15:40:49.805 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up linux-headers-5.15.0-107 (5.15.0-107.117) ... 2024-05-30T15:40:49.927 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up linux-headers-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-30T15:40:50.403 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.107.107_amd64.deb ... 2024-05-30T15:40:50.794 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up linux-headers-generic (5.15.0.107.107) ... 2024-05-30T15:40:50.936 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking linux-headers-generic (5.15.0.107.107) over (5.15.0.56.54) ... 2024-05-30T15:40:50.941 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up linux-image-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-30T15:40:51.287 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up linux-headers-5.15.0-107 (5.15.0-107.117) ... 2024-05-30T15:40:51.408 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up linux-headers-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-30T15:40:51.534 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up linux-headers-generic (5.15.0.107.107) ... 2024-05-30T15:40:51.638 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up linux-image-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-30T15:40:52.270 INFO:teuthology.orchestra.run.smithi204.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-05-30T15:40:52.270 INFO:teuthology.orchestra.run.smithi204.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-05-30T15:40:52.270 INFO:teuthology.orchestra.run.smithi204.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-107-generic 2024-05-30T15:40:52.271 INFO:teuthology.orchestra.run.smithi204.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-107-generic 2024-05-30T15:40:52.960 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up linux-modules-extra-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-30T15:40:53.078 INFO:teuthology.orchestra.run.smithi006.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-05-30T15:40:53.079 INFO:teuthology.orchestra.run.smithi006.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-05-30T15:40:53.079 INFO:teuthology.orchestra.run.smithi006.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-107-generic 2024-05-30T15:40:53.079 INFO:teuthology.orchestra.run.smithi006.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-107-generic 2024-05-30T15:40:53.232 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up linux-modules-extra-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-30T15:40:54.369 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up linux-modules-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-30T15:40:54.666 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up linux-modules-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-30T15:40:55.754 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up linux-image-generic (5.15.0.107.107) ... 2024-05-30T15:40:55.880 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up linux-generic (5.15.0.107.107) ... 2024-05-30T15:40:56.743 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up linux-image-generic (5.15.0.107.107) ... 2024-05-30T15:40:56.757 INFO:teuthology.orchestra.run.smithi204.stdout:Processing triggers for linux-image-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-30T15:40:56.828 INFO:teuthology.orchestra.run.smithi204.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-05-30T15:40:56.828 INFO:teuthology.orchestra.run.smithi204.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-107-generic 2024-05-30T15:40:56.844 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up linux-generic (5.15.0.107.107) ... 2024-05-30T15:40:56.945 INFO:teuthology.orchestra.run.smithi006.stdout:Processing triggers for linux-image-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-30T15:40:57.013 INFO:teuthology.orchestra.run.smithi006.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-05-30T15:40:57.013 INFO:teuthology.orchestra.run.smithi006.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-107-generic 2024-05-30T15:41:10.777 INFO:teuthology.orchestra.run.smithi204.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-05-30T15:41:10.777 INFO:teuthology.orchestra.run.smithi204.stdout:Sourcing file `/etc/default/grub' 2024-05-30T15:41:10.800 INFO:teuthology.orchestra.run.smithi204.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-30T15:41:10.822 INFO:teuthology.orchestra.run.smithi204.stdout:Generating grub configuration file ... 2024-05-30T15:41:11.063 INFO:teuthology.orchestra.run.smithi204.stdout:Found linux image: /boot/vmlinuz-5.15.0-107-generic 2024-05-30T15:41:11.078 INFO:teuthology.orchestra.run.smithi204.stdout:Found initrd image: /boot/initrd.img-5.15.0-107-generic 2024-05-30T15:41:11.373 INFO:teuthology.orchestra.run.smithi204.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-30T15:41:11.378 INFO:teuthology.orchestra.run.smithi204.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-30T15:41:11.443 INFO:teuthology.orchestra.run.smithi204.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-30T15:41:11.448 INFO:teuthology.orchestra.run.smithi204.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-30T15:41:11.622 INFO:teuthology.orchestra.run.smithi204.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-30T15:41:11.622 INFO:teuthology.orchestra.run.smithi204.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-05-30T15:41:11.623 INFO:teuthology.orchestra.run.smithi204.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-30T15:41:11.657 INFO:teuthology.orchestra.run.smithi204.stdout:done 2024-05-30T15:41:11.678 INFO:teuthology.orchestra.run.smithi006.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-05-30T15:41:11.678 INFO:teuthology.orchestra.run.smithi006.stdout:Sourcing file `/etc/default/grub' 2024-05-30T15:41:11.695 INFO:teuthology.orchestra.run.smithi006.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-30T15:41:11.725 INFO:teuthology.orchestra.run.smithi006.stdout:Generating grub configuration file ... 2024-05-30T15:41:12.120 DEBUG:teuthology.orchestra.run.smithi204:> dpkg -s linux-image-generic 2024-05-30T15:41:12.131 INFO:teuthology.orchestra.run.smithi006.stdout:Found linux image: /boot/vmlinuz-5.15.0-107-generic 2024-05-30T15:41:12.140 INFO:teuthology.orchestra.run.smithi204.stdout:Package: linux-image-generic 2024-05-30T15:41:12.140 INFO:teuthology.orchestra.run.smithi204.stdout:Status: install ok installed 2024-05-30T15:41:12.141 INFO:teuthology.orchestra.run.smithi204.stdout:Priority: optional 2024-05-30T15:41:12.141 INFO:teuthology.orchestra.run.smithi204.stdout:Section: kernel 2024-05-30T15:41:12.141 INFO:teuthology.orchestra.run.smithi204.stdout:Installed-Size: 21 2024-05-30T15:41:12.141 INFO:teuthology.orchestra.run.smithi204.stdout:Maintainer: Ubuntu Kernel Team 2024-05-30T15:41:12.141 INFO:teuthology.orchestra.run.smithi204.stdout:Architecture: amd64 2024-05-30T15:41:12.141 INFO:teuthology.orchestra.run.smithi204.stdout:Source: linux-meta 2024-05-30T15:41:12.141 INFO:teuthology.orchestra.run.smithi204.stdout:Version: 5.15.0.107.107 2024-05-30T15:41:12.141 INFO:teuthology.orchestra.run.smithi204.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-107), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-05-30T15:41:12.141 INFO:teuthology.orchestra.run.smithi204.stdout:Depends: linux-image-5.15.0-107-generic, linux-modules-extra-5.15.0-107-generic, linux-firmware, intel-microcode, amd64-microcode 2024-05-30T15:41:12.141 INFO:teuthology.orchestra.run.smithi204.stdout:Recommends: thermald 2024-05-30T15:41:12.141 INFO:teuthology.orchestra.run.smithi204.stdout:Description: Generic Linux kernel image 2024-05-30T15:41:12.141 INFO:teuthology.orchestra.run.smithi204.stdout: This package will always depend on the latest generic kernel image 2024-05-30T15:41:12.141 INFO:teuthology.orchestra.run.smithi204.stdout: available. 2024-05-30T15:41:12.142 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-107-generic 2024-05-30T15:41:12.142 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-30T15:41:12.142 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-30T15:41:12.142 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-05-30T15:41:12.142 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi204.front.sepia.ceph.com, path=None, version=distro) 2024-05-30T15:41:12.142 DEBUG:teuthology.orchestra.run.smithi204:> sudo apt-get clean 2024-05-30T15:41:12.152 INFO:teuthology.orchestra.run.smithi006.stdout:Found initrd image: /boot/initrd.img-5.15.0-107-generic 2024-05-30T15:41:12.330 DEBUG:teuthology.orchestra.run.smithi204:> sudo apt-get update 2024-05-30T15:41:12.507 INFO:teuthology.orchestra.run.smithi204.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-30T15:41:12.507 INFO:teuthology.orchestra.run.smithi204.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-05-30T15:41:12.535 INFO:teuthology.orchestra.run.smithi204.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-05-30T15:41:12.553 INFO:teuthology.orchestra.run.smithi006.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-30T15:41:12.560 INFO:teuthology.orchestra.run.smithi006.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-30T15:41:12.571 INFO:teuthology.orchestra.run.smithi204.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-05-30T15:41:12.633 INFO:teuthology.orchestra.run.smithi006.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-30T15:41:12.639 INFO:teuthology.orchestra.run.smithi006.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-30T15:41:12.807 INFO:teuthology.orchestra.run.smithi006.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-30T15:41:12.807 INFO:teuthology.orchestra.run.smithi006.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-05-30T15:41:12.807 INFO:teuthology.orchestra.run.smithi006.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-30T15:41:12.853 INFO:teuthology.orchestra.run.smithi006.stdout:done 2024-05-30T15:41:13.320 DEBUG:teuthology.orchestra.run.smithi006:> dpkg -s linux-image-generic 2024-05-30T15:41:13.338 INFO:teuthology.orchestra.run.smithi006.stdout:Package: linux-image-generic 2024-05-30T15:41:13.338 INFO:teuthology.orchestra.run.smithi006.stdout:Status: install ok installed 2024-05-30T15:41:13.338 INFO:teuthology.orchestra.run.smithi006.stdout:Priority: optional 2024-05-30T15:41:13.338 INFO:teuthology.orchestra.run.smithi006.stdout:Section: kernel 2024-05-30T15:41:13.338 INFO:teuthology.orchestra.run.smithi006.stdout:Installed-Size: 21 2024-05-30T15:41:13.338 INFO:teuthology.orchestra.run.smithi006.stdout:Maintainer: Ubuntu Kernel Team 2024-05-30T15:41:13.338 INFO:teuthology.orchestra.run.smithi006.stdout:Architecture: amd64 2024-05-30T15:41:13.338 INFO:teuthology.orchestra.run.smithi006.stdout:Source: linux-meta 2024-05-30T15:41:13.338 INFO:teuthology.orchestra.run.smithi006.stdout:Version: 5.15.0.107.107 2024-05-30T15:41:13.338 INFO:teuthology.orchestra.run.smithi006.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-107), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-05-30T15:41:13.338 INFO:teuthology.orchestra.run.smithi006.stdout:Depends: linux-image-5.15.0-107-generic, linux-modules-extra-5.15.0-107-generic, linux-firmware, intel-microcode, amd64-microcode 2024-05-30T15:41:13.339 INFO:teuthology.orchestra.run.smithi006.stdout:Recommends: thermald 2024-05-30T15:41:13.339 INFO:teuthology.orchestra.run.smithi006.stdout:Description: Generic Linux kernel image 2024-05-30T15:41:13.339 INFO:teuthology.orchestra.run.smithi006.stdout: This package will always depend on the latest generic kernel image 2024-05-30T15:41:13.339 INFO:teuthology.orchestra.run.smithi006.stdout: available. 2024-05-30T15:41:13.339 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-107-generic 2024-05-30T15:41:13.339 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-30T15:41:13.339 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-30T15:41:13.339 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-05-30T15:41:13.339 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi006.front.sepia.ceph.com, path=None, version=distro) 2024-05-30T15:41:13.339 DEBUG:teuthology.orchestra.run.smithi006:> sudo apt-get clean 2024-05-30T15:41:13.536 DEBUG:teuthology.orchestra.run.smithi006:> sudo apt-get update 2024-05-30T15:41:13.666 INFO:teuthology.orchestra.run.smithi006.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-30T15:41:13.666 INFO:teuthology.orchestra.run.smithi006.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-05-30T15:41:13.681 INFO:teuthology.orchestra.run.smithi006.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-05-30T15:41:13.700 INFO:teuthology.orchestra.run.smithi006.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-05-30T15:41:13.915 INFO:teuthology.orchestra.run.smithi204.stdout:Reading package lists... 2024-05-30T15:41:13.933 DEBUG:teuthology.orchestra.run.smithi204:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-05-30T15:41:13.986 INFO:teuthology.orchestra.run.smithi204.stdout:Reading package lists... 2024-05-30T15:41:14.164 INFO:teuthology.orchestra.run.smithi204.stdout:Building dependency tree... 2024-05-30T15:41:14.165 INFO:teuthology.orchestra.run.smithi204.stdout:Reading state information... 2024-05-30T15:41:14.325 INFO:teuthology.orchestra.run.smithi204.stdout:linux-image-generic is already the newest version (5.15.0.107.107). 2024-05-30T15:41:14.325 INFO:teuthology.orchestra.run.smithi204.stdout:The following packages were automatically installed and are no longer required: 2024-05-30T15:41:14.325 INFO:teuthology.orchestra.run.smithi204.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-30T15:41:14.325 INFO:teuthology.orchestra.run.smithi204.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-30T15:41:14.326 INFO:teuthology.orchestra.run.smithi204.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-30T15:41:14.326 INFO:teuthology.orchestra.run.smithi204.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-30T15:41:14.326 INFO:teuthology.orchestra.run.smithi204.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-30T15:41:14.326 INFO:teuthology.orchestra.run.smithi204.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-30T15:41:14.364 INFO:teuthology.orchestra.run.smithi204.stdout:0 upgraded, 0 newly installed, 0 to remove and 320 not upgraded. 2024-05-30T15:41:14.366 DEBUG:teuthology.orchestra.run.smithi204:> dpkg -s linux-image-generic 2024-05-30T15:41:14.421 INFO:teuthology.orchestra.run.smithi204.stdout:Package: linux-image-generic 2024-05-30T15:41:14.421 INFO:teuthology.orchestra.run.smithi204.stdout:Status: install ok installed 2024-05-30T15:41:14.422 INFO:teuthology.orchestra.run.smithi204.stdout:Priority: optional 2024-05-30T15:41:14.422 INFO:teuthology.orchestra.run.smithi204.stdout:Section: kernel 2024-05-30T15:41:14.422 INFO:teuthology.orchestra.run.smithi204.stdout:Installed-Size: 21 2024-05-30T15:41:14.422 INFO:teuthology.orchestra.run.smithi204.stdout:Maintainer: Ubuntu Kernel Team 2024-05-30T15:41:14.422 INFO:teuthology.orchestra.run.smithi204.stdout:Architecture: amd64 2024-05-30T15:41:14.422 INFO:teuthology.orchestra.run.smithi204.stdout:Source: linux-meta 2024-05-30T15:41:14.422 INFO:teuthology.orchestra.run.smithi204.stdout:Version: 5.15.0.107.107 2024-05-30T15:41:14.422 INFO:teuthology.orchestra.run.smithi204.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-107), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-05-30T15:41:14.422 INFO:teuthology.orchestra.run.smithi204.stdout:Depends: linux-image-5.15.0-107-generic, linux-modules-extra-5.15.0-107-generic, linux-firmware, intel-microcode, amd64-microcode 2024-05-30T15:41:14.422 INFO:teuthology.orchestra.run.smithi204.stdout:Recommends: thermald 2024-05-30T15:41:14.422 INFO:teuthology.orchestra.run.smithi204.stdout:Description: Generic Linux kernel image 2024-05-30T15:41:14.422 INFO:teuthology.orchestra.run.smithi204.stdout: This package will always depend on the latest generic kernel image 2024-05-30T15:41:14.423 INFO:teuthology.orchestra.run.smithi204.stdout: available. 2024-05-30T15:41:14.423 DEBUG:teuthology.orchestra.run.smithi204:> mktemp 2024-05-30T15:41:14.467 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.sMRFoWshfp 2024-05-30T15:41:14.467 DEBUG:teuthology.orchestra.run.smithi204:> sudo cp /boot/grub/grub.cfg /tmp/tmp.sMRFoWshfp 2024-05-30T15:41:14.523 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.sMRFoWshfp 2024-05-30T15:41:14.636 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.sMRFoWshfp is 10KB 2024-05-30T15:41:14.649 DEBUG:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.sMRFoWshfp 2024-05-30T15:41:14.654 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-05-30T15:41:14.707 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-30T15:41:14.707 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-05-30T15:41:14.707 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-05-30T15:41:14.775 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-107-generic 2024-05-30T15:41:14.775 DEBUG:teuthology.orchestra.run.smithi204:> sudo update-grub 2024-05-30T15:41:15.083 INFO:teuthology.orchestra.run.smithi006.stdout:Reading package lists... 2024-05-30T15:41:15.101 DEBUG:teuthology.orchestra.run.smithi006:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-05-30T15:41:15.164 INFO:teuthology.orchestra.run.smithi006.stdout:Reading package lists... 2024-05-30T15:41:15.379 INFO:teuthology.orchestra.run.smithi006.stdout:Building dependency tree... 2024-05-30T15:41:15.379 INFO:teuthology.orchestra.run.smithi006.stdout:Reading state information... 2024-05-30T15:41:15.579 INFO:teuthology.orchestra.run.smithi006.stdout:linux-image-generic is already the newest version (5.15.0.107.107). 2024-05-30T15:41:15.579 INFO:teuthology.orchestra.run.smithi006.stdout:The following packages were automatically installed and are no longer required: 2024-05-30T15:41:15.579 INFO:teuthology.orchestra.run.smithi006.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-30T15:41:15.579 INFO:teuthology.orchestra.run.smithi006.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-30T15:41:15.579 INFO:teuthology.orchestra.run.smithi006.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-30T15:41:15.580 INFO:teuthology.orchestra.run.smithi006.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-30T15:41:15.580 INFO:teuthology.orchestra.run.smithi006.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-30T15:41:15.580 INFO:teuthology.orchestra.run.smithi006.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-30T15:41:15.624 INFO:teuthology.orchestra.run.smithi006.stdout:0 upgraded, 0 newly installed, 0 to remove and 320 not upgraded. 2024-05-30T15:41:15.626 DEBUG:teuthology.orchestra.run.smithi006:> dpkg -s linux-image-generic 2024-05-30T15:41:15.642 INFO:teuthology.orchestra.run.smithi006.stdout:Package: linux-image-generic 2024-05-30T15:41:15.643 INFO:teuthology.orchestra.run.smithi006.stdout:Status: install ok installed 2024-05-30T15:41:15.643 INFO:teuthology.orchestra.run.smithi006.stdout:Priority: optional 2024-05-30T15:41:15.643 INFO:teuthology.orchestra.run.smithi006.stdout:Section: kernel 2024-05-30T15:41:15.643 INFO:teuthology.orchestra.run.smithi006.stdout:Installed-Size: 21 2024-05-30T15:41:15.643 INFO:teuthology.orchestra.run.smithi006.stdout:Maintainer: Ubuntu Kernel Team 2024-05-30T15:41:15.643 INFO:teuthology.orchestra.run.smithi006.stdout:Architecture: amd64 2024-05-30T15:41:15.643 INFO:teuthology.orchestra.run.smithi006.stdout:Source: linux-meta 2024-05-30T15:41:15.643 INFO:teuthology.orchestra.run.smithi006.stdout:Version: 5.15.0.107.107 2024-05-30T15:41:15.643 INFO:teuthology.orchestra.run.smithi006.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-107), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-05-30T15:41:15.643 INFO:teuthology.orchestra.run.smithi006.stdout:Depends: linux-image-5.15.0-107-generic, linux-modules-extra-5.15.0-107-generic, linux-firmware, intel-microcode, amd64-microcode 2024-05-30T15:41:15.643 INFO:teuthology.orchestra.run.smithi006.stdout:Recommends: thermald 2024-05-30T15:41:15.644 INFO:teuthology.orchestra.run.smithi006.stdout:Description: Generic Linux kernel image 2024-05-30T15:41:15.644 INFO:teuthology.orchestra.run.smithi006.stdout: This package will always depend on the latest generic kernel image 2024-05-30T15:41:15.644 INFO:teuthology.orchestra.run.smithi006.stdout: available. 2024-05-30T15:41:15.644 DEBUG:teuthology.orchestra.run.smithi006:> mktemp 2024-05-30T15:41:15.689 INFO:teuthology.orchestra.run.smithi006.stdout:/tmp/tmp.IH30iIwxld 2024-05-30T15:41:15.689 DEBUG:teuthology.orchestra.run.smithi006:> sudo cp /boot/grub/grub.cfg /tmp/tmp.IH30iIwxld 2024-05-30T15:41:15.745 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 0666 /tmp/tmp.IH30iIwxld 2024-05-30T15:41:15.864 DEBUG:teuthology.orchestra.remote:smithi006:/tmp/tmp.IH30iIwxld is 10KB 2024-05-30T15:41:15.878 DEBUG:teuthology.orchestra.run.smithi006:> rm -fr /tmp/tmp.IH30iIwxld 2024-05-30T15:41:15.883 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-05-30T15:41:15.937 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T15:41:15.937 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-05-30T15:41:15.937 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-05-30T15:41:15.950 INFO:teuthology.orchestra.run.smithi204.stderr:Sourcing file `/etc/default/grub' 2024-05-30T15:41:15.952 INFO:teuthology.orchestra.run.smithi204.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-30T15:41:15.955 INFO:teuthology.orchestra.run.smithi204.stderr:Generating grub configuration file ... 2024-05-30T15:41:16.016 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-107-generic 2024-05-30T15:41:16.016 DEBUG:teuthology.orchestra.run.smithi006:> sudo update-grub 2024-05-30T15:41:16.146 INFO:teuthology.orchestra.run.smithi204.stderr:Found linux image: /boot/vmlinuz-5.15.0-107-generic 2024-05-30T15:41:16.153 INFO:teuthology.orchestra.run.smithi204.stderr:Found initrd image: /boot/initrd.img-5.15.0-107-generic 2024-05-30T15:41:16.385 INFO:teuthology.orchestra.run.smithi204.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-30T15:41:16.390 INFO:teuthology.orchestra.run.smithi204.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-30T15:41:16.429 INFO:teuthology.orchestra.run.smithi204.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-30T15:41:16.435 INFO:teuthology.orchestra.run.smithi204.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-30T15:41:16.574 INFO:teuthology.orchestra.run.smithi204.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-30T15:41:16.574 INFO:teuthology.orchestra.run.smithi204.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-05-30T15:41:16.574 INFO:teuthology.orchestra.run.smithi204.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-30T15:41:16.592 INFO:teuthology.orchestra.run.smithi204.stderr:done 2024-05-30T15:41:16.600 DEBUG:teuthology.orchestra.run.smithi204:> sudo shutdown -r now 2024-05-30T15:41:17.299 INFO:teuthology.orchestra.run.smithi006.stderr:Sourcing file `/etc/default/grub' 2024-05-30T15:41:17.300 INFO:teuthology.orchestra.run.smithi006.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-30T15:41:17.302 INFO:teuthology.orchestra.run.smithi006.stderr:Generating grub configuration file ... 2024-05-30T15:41:17.551 INFO:teuthology.orchestra.run.smithi006.stderr:Found linux image: /boot/vmlinuz-5.15.0-107-generic 2024-05-30T15:41:17.558 INFO:teuthology.orchestra.run.smithi006.stderr:Found initrd image: /boot/initrd.img-5.15.0-107-generic 2024-05-30T15:41:17.892 INFO:teuthology.orchestra.run.smithi006.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-30T15:41:17.897 INFO:teuthology.orchestra.run.smithi006.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-30T15:41:17.940 INFO:teuthology.orchestra.run.smithi006.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-30T15:41:17.945 INFO:teuthology.orchestra.run.smithi006.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-30T15:41:18.066 INFO:teuthology.orchestra.run.smithi006.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-30T15:41:18.066 INFO:teuthology.orchestra.run.smithi006.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-05-30T15:41:18.066 INFO:teuthology.orchestra.run.smithi006.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-30T15:41:18.093 INFO:teuthology.orchestra.run.smithi006.stderr:done 2024-05-30T15:41:18.102 DEBUG:teuthology.orchestra.run.smithi006:> sudo shutdown -r now 2024-05-30T15:41:46.626 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-30T15:41:46.626 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-05-30T15:41:46.627 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-30T15:41:48.106 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-30T15:41:48.107 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-05-30T15:41:48.107 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-30T15:42:21.215 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2024-05-30T15:42:22.750 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.6 2024-05-30T15:42:30.223 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-05-30T15:42:30.224 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-30T15:42:31.753 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-05-30T15:42:31.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-30T15:42:50.142 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.6 2024-05-30T15:43:02.149 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-05-30T15:43:02.150 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-30T15:43:05.214 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.6 2024-05-30T15:43:20.228 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-05-30T15:43:20.229 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-30T15:43:20.522 DEBUG:teuthology.orchestra.run.smithi006:> true 2024-05-30T15:43:21.128 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-05-30T15:43:21.128 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-107-generic"... 2024-05-30T15:43:21.129 DEBUG:teuthology.orchestra.run.smithi006:> uname -r 2024-05-30T15:43:21.174 INFO:teuthology.orchestra.run.smithi006.stdout:5.15.0-107-generic 2024-05-30T15:43:21.174 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-107-generic vs 5.15.0-107-generic 2024-05-30T15:43:21.174 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-30T15:43:21.174 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-05-30T15:43:22.175 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-30T15:43:22.175 DEBUG:teuthology.orchestra.run.smithi006:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-30T15:43:22.253 INFO:teuthology.orchestra.run.smithi006.stdout:ttyS1 2024-05-30T15:43:22.283 DEBUG:teuthology.parallel:result is None 2024-05-30T15:43:30.226 DEBUG:teuthology.orchestra.remote:timed out 2024-05-30T15:43:42.229 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-05-30T15:43:42.230 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-30T15:43:42.520 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-05-30T15:43:43.083 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-05-30T15:43:43.084 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-107-generic"... 2024-05-30T15:43:43.084 DEBUG:teuthology.orchestra.run.smithi204:> uname -r 2024-05-30T15:43:43.128 INFO:teuthology.orchestra.run.smithi204.stdout:5.15.0-107-generic 2024-05-30T15:43:43.128 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-107-generic vs 5.15.0-107-generic 2024-05-30T15:43:43.128 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-30T15:43:43.128 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-05-30T15:43:44.129 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-30T15:43:44.129 DEBUG:teuthology.orchestra.run.smithi204:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-30T15:43:44.213 INFO:teuthology.orchestra.run.smithi204.stdout:ttyS1 2024-05-30T15:43:44.238 DEBUG:teuthology.parallel:result is None 2024-05-30T15:43:44.239 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-30T15:43:44.247 INFO:teuthology.task.internal:Creating test directory... 2024-05-30T15:43:44.247 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-30T15:43:44.250 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-30T15:43:44.255 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-30T15:43:44.289 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-30T15:43:44.327 INFO:teuthology.task.internal:Creating archive directory... 2024-05-30T15:43:44.327 DEBUG:teuthology.orchestra.run.smithi006:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-30T15:43:44.330 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-30T15:43:44.353 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-30T15:43:44.366 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-30T15:43:44.366 DEBUG:teuthology.orchestra.run.smithi006:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-30T15:43:44.391 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-30T15:43:44.408 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-30T15:43:44.413 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-30T15:43:44.417 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-30T15:43:44.422 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-30T15:43:44.423 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-30T15:43:44.432 INFO:teuthology.task.internal:Configuring sudo... 2024-05-30T15:43:44.432 DEBUG:teuthology.orchestra.run.smithi006:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-30T15:43:44.467 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-30T15:43:44.479 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-30T15:43:44.488 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-30T15:43:44.489 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-30T15:43:44.523 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-30T15:43:44.528 DEBUG:teuthology.orchestra.run.smithi006:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-30T15:43:44.574 DEBUG:teuthology.orchestra.run.smithi006:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-30T15:43:44.622 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T15:43:44.622 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-30T15:43:44.687 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-30T15:43:44.693 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-30T15:43:44.741 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-30T15:43:44.741 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-30T15:43:44.796 DEBUG:teuthology.orchestra.run.smithi006:> sudo service rsyslog restart 2024-05-30T15:43:44.799 DEBUG:teuthology.orchestra.run.smithi204:> sudo service rsyslog restart 2024-05-30T15:43:44.868 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-30T15:43:44.875 INFO:teuthology.task.internal:Starting timer... 2024-05-30T15:43:44.875 INFO:teuthology.run_tasks:Running task pcp... 2024-05-30T15:43:45.031 INFO:teuthology.run_tasks:Running task selinux... 2024-05-30T15:43:45.068 DEBUG:teuthology.task.selinux:Excluding smithi006: OS 'ubuntu' does not support SELinux 2024-05-30T15:43:45.068 DEBUG:teuthology.task.selinux:Excluding smithi204: OS 'ubuntu' does not support SELinux 2024-05-30T15:43:45.068 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-30T15:43:45.068 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-05-30T15:43:45.068 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-30T15:43:45.068 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-30T15:43:45.149 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-30T15:43:45.159 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-30T15:43:45.160 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi006.front.sepia.ceph.com,smithi204.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-30T15:50:49.794 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi006.front.sepia.ceph.com'), Remote(name='ubuntu@smithi204.front.sepia.ceph.com')] 2024-05-30T15:50:49.795 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-05-30T15:50:49.796 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-30T15:50:49.879 DEBUG:teuthology.orchestra.run.smithi006:> true 2024-05-30T15:50:49.954 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-05-30T15:50:49.955 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-05-30T15:50:49.955 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-30T15:50:50.040 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-05-30T15:50:50.110 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-05-30T15:50:50.110 INFO:teuthology.run_tasks:Running task clock... 2024-05-30T15:50:50.120 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-30T15:50:50.120 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-30T15:50:50.121 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-30T15:50:50.123 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-30T15:50:50.123 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-30T15:50:50.152 INFO:teuthology.orchestra.run.smithi006.stdout:30 May 15:50:50 ntpd[17508]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-05-30T15:50:50.152 INFO:teuthology.orchestra.run.smithi006.stdout:30 May 15:50:50 ntpd[17508]: Command line: ntpd -gq 2024-05-30T15:50:50.152 INFO:teuthology.orchestra.run.smithi006.stdout:30 May 15:50:50 ntpd[17508]: ---------------------------------------------------- 2024-05-30T15:50:50.152 INFO:teuthology.orchestra.run.smithi006.stdout:30 May 15:50:50 ntpd[17508]: ntp-4 is maintained by Network Time Foundation, 2024-05-30T15:50:50.152 INFO:teuthology.orchestra.run.smithi006.stdout:30 May 15:50:50 ntpd[17508]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-05-30T15:50:50.152 INFO:teuthology.orchestra.run.smithi006.stdout:30 May 15:50:50 ntpd[17508]: corporation. Support and training for ntp-4 are 2024-05-30T15:50:50.152 INFO:teuthology.orchestra.run.smithi006.stdout:30 May 15:50:50 ntpd[17508]: available at https://www.nwtime.org/support 2024-05-30T15:50:50.152 INFO:teuthology.orchestra.run.smithi006.stdout:30 May 15:50:50 ntpd[17508]: ---------------------------------------------------- 2024-05-30T15:50:50.153 INFO:teuthology.orchestra.run.smithi006.stdout:30 May 15:50:50 ntpd[17508]: proto: precision = 0.045 usec (-24) 2024-05-30T15:50:50.153 INFO:teuthology.orchestra.run.smithi006.stdout:30 May 15:50:50 ntpd[17508]: basedate set to 2022-02-04 2024-05-30T15:50:50.153 INFO:teuthology.orchestra.run.smithi006.stdout:30 May 15:50:50 ntpd[17508]: gps base set to 2022-02-06 (week 2196) 2024-05-30T15:50:50.153 INFO:teuthology.orchestra.run.smithi006.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-30T15:50:50.154 INFO:teuthology.orchestra.run.smithi006.stdout:30 May 15:50:50 ntpd[17508]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-30T15:50:50.154 INFO:teuthology.orchestra.run.smithi006.stdout:30 May 15:50:50 ntpd[17508]: restrict ::: KOD does nothing without LIMITED. 2024-05-30T15:50:50.154 INFO:teuthology.orchestra.run.smithi006.stderr:restrict ::: KOD does nothing without LIMITED. 2024-05-30T15:50:50.155 INFO:teuthology.orchestra.run.smithi006.stdout:30 May 15:50:50 ntpd[17508]: Listen and drop on 0 v6wildcard [::]:123 2024-05-30T15:50:50.155 INFO:teuthology.orchestra.run.smithi006.stdout:30 May 15:50:50 ntpd[17508]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-05-30T15:50:50.155 INFO:teuthology.orchestra.run.smithi006.stdout:30 May 15:50:50 ntpd[17508]: Listen normally on 2 lo 127.0.0.1:123 2024-05-30T15:50:50.155 INFO:teuthology.orchestra.run.smithi006.stdout:30 May 15:50:50 ntpd[17508]: Listen normally on 3 ens1f0 172.21.15.6:123 2024-05-30T15:50:50.155 INFO:teuthology.orchestra.run.smithi006.stdout:30 May 15:50:50 ntpd[17508]: Listen normally on 4 lo [::1]:123 2024-05-30T15:50:50.155 INFO:teuthology.orchestra.run.smithi006.stdout:30 May 15:50:50 ntpd[17508]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:15e4%4]:123 2024-05-30T15:50:50.155 INFO:teuthology.orchestra.run.smithi006.stdout:30 May 15:50:50 ntpd[17508]: Listening on routing socket on fd #22 for interface updates 2024-05-30T15:50:50.180 INFO:teuthology.orchestra.run.smithi204.stdout:30 May 15:50:50 ntpd[17577]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-05-30T15:50:50.181 INFO:teuthology.orchestra.run.smithi204.stdout:30 May 15:50:50 ntpd[17577]: Command line: ntpd -gq 2024-05-30T15:50:50.181 INFO:teuthology.orchestra.run.smithi204.stdout:30 May 15:50:50 ntpd[17577]: ---------------------------------------------------- 2024-05-30T15:50:50.181 INFO:teuthology.orchestra.run.smithi204.stdout:30 May 15:50:50 ntpd[17577]: ntp-4 is maintained by Network Time Foundation, 2024-05-30T15:50:50.181 INFO:teuthology.orchestra.run.smithi204.stdout:30 May 15:50:50 ntpd[17577]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-05-30T15:50:50.181 INFO:teuthology.orchestra.run.smithi204.stdout:30 May 15:50:50 ntpd[17577]: corporation. Support and training for ntp-4 are 2024-05-30T15:50:50.181 INFO:teuthology.orchestra.run.smithi204.stdout:30 May 15:50:50 ntpd[17577]: available at https://www.nwtime.org/support 2024-05-30T15:50:50.181 INFO:teuthology.orchestra.run.smithi204.stdout:30 May 15:50:50 ntpd[17577]: ---------------------------------------------------- 2024-05-30T15:50:50.182 INFO:teuthology.orchestra.run.smithi204.stdout:30 May 15:50:50 ntpd[17577]: proto: precision = 0.050 usec (-24) 2024-05-30T15:50:50.182 INFO:teuthology.orchestra.run.smithi204.stdout:30 May 15:50:50 ntpd[17577]: basedate set to 2022-02-04 2024-05-30T15:50:50.182 INFO:teuthology.orchestra.run.smithi204.stdout:30 May 15:50:50 ntpd[17577]: gps base set to 2022-02-06 (week 2196) 2024-05-30T15:50:50.182 INFO:teuthology.orchestra.run.smithi204.stdout:30 May 15:50:50 ntpd[17577]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-30T15:50:50.182 INFO:teuthology.orchestra.run.smithi204.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-30T15:50:50.182 INFO:teuthology.orchestra.run.smithi204.stderr:restrict ::: KOD does nothing without LIMITED. 2024-05-30T15:50:50.183 INFO:teuthology.orchestra.run.smithi204.stdout:30 May 15:50:50 ntpd[17577]: restrict ::: KOD does nothing without LIMITED. 2024-05-30T15:50:50.184 INFO:teuthology.orchestra.run.smithi204.stdout:30 May 15:50:50 ntpd[17577]: Listen and drop on 0 v6wildcard [::]:123 2024-05-30T15:50:50.184 INFO:teuthology.orchestra.run.smithi204.stdout:30 May 15:50:50 ntpd[17577]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-05-30T15:50:50.184 INFO:teuthology.orchestra.run.smithi204.stdout:30 May 15:50:50 ntpd[17577]: Listen normally on 2 lo 127.0.0.1:123 2024-05-30T15:50:50.184 INFO:teuthology.orchestra.run.smithi204.stdout:30 May 15:50:50 ntpd[17577]: Listen normally on 3 enp3s0f1 172.21.15.204:123 2024-05-30T15:50:50.184 INFO:teuthology.orchestra.run.smithi204.stdout:30 May 15:50:50 ntpd[17577]: Listen normally on 4 lo [::1]:123 2024-05-30T15:50:50.184 INFO:teuthology.orchestra.run.smithi204.stdout:30 May 15:50:50 ntpd[17577]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:77ab%5]:123 2024-05-30T15:50:50.184 INFO:teuthology.orchestra.run.smithi204.stdout:30 May 15:50:50 ntpd[17577]: Listening on routing socket on fd #22 for interface updates 2024-05-30T15:50:51.182 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:51 ntpd[17577]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:51.182 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:51 ntpd[17577]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:51.182 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:51 ntpd[17577]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:51.182 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:51 ntpd[17577]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:51.182 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:51 ntpd[17577]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:51.182 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:51 ntpd[17577]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:51.182 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:51 ntpd[17577]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-30T15:50:51.182 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:51 ntpd[17577]: can't open /var/log/ntpstats/peerstats.20240530: Permission denied 2024-05-30T15:50:51.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:51 ntpd[17577]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:51.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:51 ntpd[17577]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:51.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:51 ntpd[17577]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-30T15:50:51.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:51 ntpd[17577]: can't open /var/log/ntpstats/peerstats.20240530: Permission denied 2024-05-30T15:50:53.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:53 ntpd[17508]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:53.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:53 ntpd[17508]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:53.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:53 ntpd[17508]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:53.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:53 ntpd[17508]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:53.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:53 ntpd[17508]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-30T15:50:53.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:53 ntpd[17508]: can't open /var/log/ntpstats/peerstats.20240530: Permission denied 2024-05-30T15:50:53.182 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:53 ntpd[17577]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:53.182 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:53 ntpd[17577]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:53.182 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:53 ntpd[17577]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:53.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:53 ntpd[17577]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:53.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:53 ntpd[17577]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:53.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:53 ntpd[17577]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:53.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:53 ntpd[17577]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:53.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:53 ntpd[17577]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:53.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:53 ntpd[17577]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-30T15:50:53.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:53 ntpd[17577]: can't open /var/log/ntpstats/peerstats.20240530: Permission denied 2024-05-30T15:50:53.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:53 ntpd[17577]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:53.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:53 ntpd[17577]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:53.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:53 ntpd[17577]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-30T15:50:53.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:53 ntpd[17577]: can't open /var/log/ntpstats/peerstats.20240530: Permission denied 2024-05-30T15:50:53.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:53 ntpd[17577]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:53.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:53 ntpd[17577]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:53.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:53 ntpd[17577]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-30T15:50:53.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:53 ntpd[17577]: can't open /var/log/ntpstats/peerstats.20240530: Permission denied 2024-05-30T15:50:54.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:54 ntpd[17508]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:54.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:54 ntpd[17508]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:54.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:54 ntpd[17508]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:54.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:54 ntpd[17508]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:54.154 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:54 ntpd[17508]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:54.154 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:54 ntpd[17508]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:54.154 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:54 ntpd[17508]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-30T15:50:54.154 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:54 ntpd[17508]: can't open /var/log/ntpstats/peerstats.20240530: Permission denied 2024-05-30T15:50:54.154 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:54 ntpd[17508]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:54.154 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:54 ntpd[17508]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:54.154 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:54 ntpd[17508]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-30T15:50:54.154 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:54 ntpd[17508]: can't open /var/log/ntpstats/peerstats.20240530: Permission denied 2024-05-30T15:50:55.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:55 ntpd[17508]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:55.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:55 ntpd[17508]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:55.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:55 ntpd[17508]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:55.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:55 ntpd[17508]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:55.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:55 ntpd[17508]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:55.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:55 ntpd[17508]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:55.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:55 ntpd[17508]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-30T15:50:55.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:55 ntpd[17508]: can't open /var/log/ntpstats/peerstats.20240530: Permission denied 2024-05-30T15:50:55.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:55 ntpd[17577]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:55.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:55 ntpd[17577]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:55.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:55 ntpd[17577]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:55.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:55 ntpd[17577]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:55.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:55 ntpd[17577]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:55.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:55 ntpd[17577]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:55.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:55 ntpd[17577]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:55.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:55 ntpd[17577]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:55.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:55 ntpd[17577]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:55.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:55 ntpd[17577]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:55.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:55 ntpd[17577]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-30T15:50:55.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:55 ntpd[17577]: can't open /var/log/ntpstats/peerstats.20240530: Permission denied 2024-05-30T15:50:55.184 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:55 ntpd[17577]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:55.184 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:55 ntpd[17577]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:55.184 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:55 ntpd[17577]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-30T15:50:55.184 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:55 ntpd[17577]: can't open /var/log/ntpstats/peerstats.20240530: Permission denied 2024-05-30T15:50:55.184 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:55 ntpd[17577]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:55.184 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:55 ntpd[17577]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:55.184 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:55 ntpd[17577]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-30T15:50:55.184 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:55 ntpd[17577]: can't open /var/log/ntpstats/peerstats.20240530: Permission denied 2024-05-30T15:50:56.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:56 ntpd[17508]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:56.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:56 ntpd[17508]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:56.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:56 ntpd[17508]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:56.154 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:56 ntpd[17508]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:56.154 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:56 ntpd[17508]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:56.154 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:56 ntpd[17508]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:56.154 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:56 ntpd[17508]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-30T15:50:56.154 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:56 ntpd[17508]: can't open /var/log/ntpstats/peerstats.20240530: Permission denied 2024-05-30T15:50:56.154 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:56 ntpd[17508]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:56.154 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:56 ntpd[17508]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:56.154 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:56 ntpd[17508]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-30T15:50:56.154 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:56 ntpd[17508]: can't open /var/log/ntpstats/peerstats.20240530: Permission denied 2024-05-30T15:50:57.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:57 ntpd[17508]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:57.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:57 ntpd[17508]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:57.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:57 ntpd[17508]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:57.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:57 ntpd[17508]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:57.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:57 ntpd[17508]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-30T15:50:57.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:57 ntpd[17508]: can't open /var/log/ntpstats/peerstats.20240530: Permission denied 2024-05-30T15:50:57.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:57 ntpd[17577]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:57.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:57 ntpd[17577]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:57.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:57 ntpd[17577]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:57.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:57 ntpd[17577]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:57.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:57 ntpd[17577]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:57.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:57 ntpd[17577]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:57.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:57 ntpd[17577]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:57.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:57 ntpd[17577]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:57.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:57 ntpd[17577]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-30T15:50:57.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:57 ntpd[17577]: can't open /var/log/ntpstats/peerstats.20240530: Permission denied 2024-05-30T15:50:57.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:57 ntpd[17577]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:57.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:57 ntpd[17577]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:57.183 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:57 ntpd[17577]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-30T15:50:57.184 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:57 ntpd[17577]: can't open /var/log/ntpstats/peerstats.20240530: Permission denied 2024-05-30T15:50:57.184 INFO:teuthology.orchestra.run.smithi204.stdout:30 May 15:50:57 ntpd[17577]: ntpd: time slew +0.001868 s 2024-05-30T15:50:57.184 INFO:teuthology.orchestra.run.smithi204.stdout:ntpd: time slew +0.001868s 2024-05-30T15:50:57.184 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:57 ntpd[17577]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-05-30T15:50:57.184 INFO:teuthology.orchestra.run.smithi204.stderr:30 May 15:50:57 ntpd[17577]: can't open /var/log/ntpstats/loopstats.20240530: Permission denied 2024-05-30T15:50:57.239 INFO:teuthology.orchestra.run.smithi204.stdout: remote refid st t when poll reach delay offset jitter 2024-05-30T15:50:57.239 INFO:teuthology.orchestra.run.smithi204.stdout:============================================================================== 2024-05-30T15:50:57.239 INFO:teuthology.orchestra.run.smithi204.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-30T15:50:57.239 INFO:teuthology.orchestra.run.smithi204.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-30T15:50:57.239 INFO:teuthology.orchestra.run.smithi204.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-30T15:50:57.239 INFO:teuthology.orchestra.run.smithi204.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-30T15:50:58.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:58 ntpd[17508]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:58.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:58 ntpd[17508]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:58.154 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:58 ntpd[17508]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:58.154 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:58 ntpd[17508]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:58.154 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:58 ntpd[17508]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:58.154 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:58 ntpd[17508]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:58.154 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:58 ntpd[17508]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-30T15:50:58.154 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:58 ntpd[17508]: can't open /var/log/ntpstats/peerstats.20240530: Permission denied 2024-05-30T15:50:58.154 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:58 ntpd[17508]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:58.154 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:58 ntpd[17508]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:58.154 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:58 ntpd[17508]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-30T15:50:58.154 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:58 ntpd[17508]: can't open /var/log/ntpstats/peerstats.20240530: Permission denied 2024-05-30T15:50:59.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:59 ntpd[17508]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:59.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:59 ntpd[17508]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:59.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:59 ntpd[17508]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-30T15:50:59.153 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:59 ntpd[17508]: can't open /var/log/ntpstats/rawstats.20240530: Permission denied 2024-05-30T15:50:59.154 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:59 ntpd[17508]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-30T15:50:59.154 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:59 ntpd[17508]: can't open /var/log/ntpstats/peerstats.20240530: Permission denied 2024-05-30T15:50:59.154 INFO:teuthology.orchestra.run.smithi006.stdout:30 May 15:50:59 ntpd[17508]: ntpd: time slew +0.001890 s 2024-05-30T15:50:59.154 INFO:teuthology.orchestra.run.smithi006.stdout:ntpd: time slew +0.001890s 2024-05-30T15:50:59.154 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:59 ntpd[17508]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-05-30T15:50:59.154 INFO:teuthology.orchestra.run.smithi006.stderr:30 May 15:50:59 ntpd[17508]: can't open /var/log/ntpstats/loopstats.20240530: Permission denied 2024-05-30T15:50:59.201 INFO:teuthology.orchestra.run.smithi006.stdout: remote refid st t when poll reach delay offset jitter 2024-05-30T15:50:59.201 INFO:teuthology.orchestra.run.smithi006.stdout:============================================================================== 2024-05-30T15:50:59.201 INFO:teuthology.orchestra.run.smithi006.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-30T15:50:59.201 INFO:teuthology.orchestra.run.smithi006.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-30T15:50:59.201 INFO:teuthology.orchestra.run.smithi006.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-30T15:50:59.201 INFO:teuthology.orchestra.run.smithi006.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-30T15:50:59.202 INFO:teuthology.run_tasks:Running task install... 2024-05-30T15:50:59.211 DEBUG:teuthology.task.install:project ceph 2024-05-30T15:50:59.211 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2'}} 2024-05-30T15:50:59.211 DEBUG:teuthology.task.install:config {'extra_system_packages': {'deb': ['s3cmd', 'maven'], 'rpm': ['s3cmd', 'maven']}, 'flavor': 'default', 'sha1': 'da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2'} 2024-05-30T15:50:59.211 INFO:teuthology.task.install:Using flavor: default 2024-05-30T15:50:59.216 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-30T15:50:59.216 INFO:teuthology.task.install:extra packages: [] 2024-05-30T15:50:59.217 DEBUG:teuthology.orchestra.run.smithi006:> sudo apt-key list | grep Ceph 2024-05-30T15:50:59.217 DEBUG:teuthology.orchestra.run.smithi204:> sudo apt-key list | grep Ceph 2024-05-30T15:50:59.277 INFO:teuthology.orchestra.run.smithi204.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-05-30T15:50:59.297 INFO:teuthology.orchestra.run.smithi006.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-05-30T15:50:59.304 INFO:teuthology.orchestra.run.smithi204.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-05-30T15:50:59.304 INFO:teuthology.orchestra.run.smithi204.stdout:uid [ unknown] Ceph.com (release key) 2024-05-30T15:50:59.305 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-05-30T15:50:59.305 INFO:teuthology.task.install.deb:Installing system (non-project) packages: s3cmd, maven on remote deb x86_64 2024-05-30T15:50:59.306 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 2024-05-30T15:50:59.324 INFO:teuthology.orchestra.run.smithi006.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-05-30T15:50:59.324 INFO:teuthology.orchestra.run.smithi006.stdout:uid [ unknown] Ceph.com (release key) 2024-05-30T15:50:59.325 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-05-30T15:50:59.325 INFO:teuthology.task.install.deb:Installing system (non-project) packages: s3cmd, maven on remote deb x86_64 2024-05-30T15:50:59.325 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 2024-05-30T15:50:59.527 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default/ 2024-05-30T15:50:59.528 INFO:teuthology.task.install.deb:Package version is 19.0.0-4011-gda98024b-1jammy 2024-05-30T15:50:59.542 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default/ 2024-05-30T15:50:59.543 INFO:teuthology.task.install.deb:Package version is 19.0.0-4011-gda98024b-1jammy 2024-05-30T15:50:59.698 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-30T15:50:59.698 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-05-30T15:50:59.712 DEBUG:teuthology.orchestra.run.smithi204:> sudo apt-get update 2024-05-30T15:50:59.722 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T15:50:59.722 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-05-30T15:50:59.736 DEBUG:teuthology.orchestra.run.smithi006:> sudo apt-get update 2024-05-30T15:50:59.891 INFO:teuthology.orchestra.run.smithi204.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-05-30T15:50:59.917 INFO:teuthology.orchestra.run.smithi006.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-05-30T15:50:59.999 INFO:teuthology.orchestra.run.smithi204.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-30T15:51:00.009 INFO:teuthology.orchestra.run.smithi204.stdout:Ign:3 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy InRelease 2024-05-30T15:51:00.028 INFO:teuthology.orchestra.run.smithi006.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-30T15:51:00.038 INFO:teuthology.orchestra.run.smithi006.stdout:Ign:3 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy InRelease 2024-05-30T15:51:00.048 INFO:teuthology.orchestra.run.smithi204.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-05-30T15:51:00.077 INFO:teuthology.orchestra.run.smithi006.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-05-30T15:51:00.081 INFO:teuthology.orchestra.run.smithi204.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-05-30T15:51:00.085 INFO:teuthology.orchestra.run.smithi204.stdout:Ign:6 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy Release.gpg 2024-05-30T15:51:00.111 INFO:teuthology.orchestra.run.smithi006.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-05-30T15:51:00.116 INFO:teuthology.orchestra.run.smithi006.stdout:Ign:6 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy Release.gpg 2024-05-30T15:51:00.124 INFO:teuthology.orchestra.run.smithi204.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,409 B] 2024-05-30T15:51:00.152 INFO:teuthology.orchestra.run.smithi006.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,409 B] 2024-05-30T15:51:00.162 INFO:teuthology.orchestra.run.smithi204.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-05-30T15:51:00.166 INFO:teuthology.orchestra.run.smithi204.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-05-30T15:51:00.192 INFO:teuthology.orchestra.run.smithi006.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-05-30T15:51:00.195 INFO:teuthology.orchestra.run.smithi006.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-05-30T15:51:00.299 INFO:teuthology.orchestra.run.smithi204.stdout:Fetched 29.1 kB in 0s (71.2 kB/s) 2024-05-30T15:51:00.339 INFO:teuthology.orchestra.run.smithi006.stdout:Fetched 29.1 kB in 0s (69.7 kB/s) 2024-05-30T15:51:01.253 INFO:teuthology.orchestra.run.smithi204.stdout:Reading package lists... 2024-05-30T15:51:01.269 DEBUG:teuthology.orchestra.run.smithi204:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-4011-gda98024b-1jammy cephadm=19.0.0-4011-gda98024b-1jammy ceph-mds=19.0.0-4011-gda98024b-1jammy ceph-mgr=19.0.0-4011-gda98024b-1jammy ceph-common=19.0.0-4011-gda98024b-1jammy ceph-fuse=19.0.0-4011-gda98024b-1jammy ceph-test=19.0.0-4011-gda98024b-1jammy ceph-volume=19.0.0-4011-gda98024b-1jammy radosgw=19.0.0-4011-gda98024b-1jammy python3-rados=19.0.0-4011-gda98024b-1jammy python3-rgw=19.0.0-4011-gda98024b-1jammy python3-cephfs=19.0.0-4011-gda98024b-1jammy python3-rbd=19.0.0-4011-gda98024b-1jammy libcephfs2=19.0.0-4011-gda98024b-1jammy libcephfs-dev=19.0.0-4011-gda98024b-1jammy librados2=19.0.0-4011-gda98024b-1jammy librbd1=19.0.0-4011-gda98024b-1jammy rbd-fuse=19.0.0-4011-gda98024b-1jammy 2024-05-30T15:51:01.331 INFO:teuthology.orchestra.run.smithi204.stdout:Reading package lists... 2024-05-30T15:51:01.338 INFO:teuthology.orchestra.run.smithi006.stdout:Reading package lists... 2024-05-30T15:51:01.355 DEBUG:teuthology.orchestra.run.smithi006:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-4011-gda98024b-1jammy cephadm=19.0.0-4011-gda98024b-1jammy ceph-mds=19.0.0-4011-gda98024b-1jammy ceph-mgr=19.0.0-4011-gda98024b-1jammy ceph-common=19.0.0-4011-gda98024b-1jammy ceph-fuse=19.0.0-4011-gda98024b-1jammy ceph-test=19.0.0-4011-gda98024b-1jammy ceph-volume=19.0.0-4011-gda98024b-1jammy radosgw=19.0.0-4011-gda98024b-1jammy python3-rados=19.0.0-4011-gda98024b-1jammy python3-rgw=19.0.0-4011-gda98024b-1jammy python3-cephfs=19.0.0-4011-gda98024b-1jammy python3-rbd=19.0.0-4011-gda98024b-1jammy libcephfs2=19.0.0-4011-gda98024b-1jammy libcephfs-dev=19.0.0-4011-gda98024b-1jammy librados2=19.0.0-4011-gda98024b-1jammy librbd1=19.0.0-4011-gda98024b-1jammy rbd-fuse=19.0.0-4011-gda98024b-1jammy 2024-05-30T15:51:01.418 INFO:teuthology.orchestra.run.smithi006.stdout:Reading package lists... 2024-05-30T15:51:01.526 INFO:teuthology.orchestra.run.smithi204.stdout:Building dependency tree... 2024-05-30T15:51:01.527 INFO:teuthology.orchestra.run.smithi204.stdout:Reading state information... 2024-05-30T15:51:01.620 INFO:teuthology.orchestra.run.smithi006.stdout:Building dependency tree... 2024-05-30T15:51:01.621 INFO:teuthology.orchestra.run.smithi006.stdout:Reading state information... 2024-05-30T15:51:01.712 INFO:teuthology.orchestra.run.smithi204.stdout:The following packages were automatically installed and are no longer required: 2024-05-30T15:51:01.712 INFO:teuthology.orchestra.run.smithi204.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-30T15:51:01.713 INFO:teuthology.orchestra.run.smithi204.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-30T15:51:01.713 INFO:teuthology.orchestra.run.smithi204.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-30T15:51:01.713 INFO:teuthology.orchestra.run.smithi204.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-30T15:51:01.713 INFO:teuthology.orchestra.run.smithi204.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-30T15:51:01.713 INFO:teuthology.orchestra.run.smithi204.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-30T15:51:01.714 INFO:teuthology.orchestra.run.smithi204.stdout:The following additional packages will be installed: 2024-05-30T15:51:01.714 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-05-30T15:51:01.714 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-05-30T15:51:01.715 INFO:teuthology.orchestra.run.smithi204.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-05-30T15:51:01.715 INFO:teuthology.orchestra.run.smithi204.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-05-30T15:51:01.716 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-05-30T15:51:01.716 INFO:teuthology.orchestra.run.smithi204.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-05-30T15:51:01.716 INFO:teuthology.orchestra.run.smithi204.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-05-30T15:51:01.716 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-05-30T15:51:01.716 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-05-30T15:51:01.716 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-05-30T15:51:01.716 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-05-30T15:51:01.716 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-05-30T15:51:01.716 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-05-30T15:51:01.716 INFO:teuthology.orchestra.run.smithi204.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-05-30T15:51:01.716 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-05-30T15:51:01.716 INFO:teuthology.orchestra.run.smithi204.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-05-30T15:51:01.717 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-05-30T15:51:01.717 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-05-30T15:51:01.717 INFO:teuthology.orchestra.run.smithi204.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-05-30T15:51:01.717 INFO:teuthology.orchestra.run.smithi204.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-05-30T15:51:01.717 INFO:teuthology.orchestra.run.smithi204.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-05-30T15:51:01.717 INFO:teuthology.orchestra.run.smithi204.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-05-30T15:51:01.717 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-05-30T15:51:01.717 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet zip 2024-05-30T15:51:01.717 INFO:teuthology.orchestra.run.smithi204.stdout:Suggested packages: 2024-05-30T15:51:01.718 INFO:teuthology.orchestra.run.smithi204.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-05-30T15:51:01.718 INFO:teuthology.orchestra.run.smithi204.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-05-30T15:51:01.718 INFO:teuthology.orchestra.run.smithi204.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-05-30T15:51:01.718 INFO:teuthology.orchestra.run.smithi204.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-05-30T15:51:01.718 INFO:teuthology.orchestra.run.smithi204.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-05-30T15:51:01.718 INFO:teuthology.orchestra.run.smithi204.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-05-30T15:51:01.718 INFO:teuthology.orchestra.run.smithi204.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-05-30T15:51:01.718 INFO:teuthology.orchestra.run.smithi204.stdout:Recommended packages: 2024-05-30T15:51:01.718 INFO:teuthology.orchestra.run.smithi204.stdout: btrfs-tools 2024-05-30T15:51:01.818 INFO:teuthology.orchestra.run.smithi204.stdout:The following NEW packages will be installed: 2024-05-30T15:51:01.819 INFO:teuthology.orchestra.run.smithi204.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-05-30T15:51:01.819 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-05-30T15:51:01.819 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-05-30T15:51:01.819 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-05-30T15:51:01.819 INFO:teuthology.orchestra.run.smithi204.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-05-30T15:51:01.819 INFO:teuthology.orchestra.run.smithi204.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-05-30T15:51:01.820 INFO:teuthology.orchestra.run.smithi204.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-05-30T15:51:01.820 INFO:teuthology.orchestra.run.smithi204.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-05-30T15:51:01.820 INFO:teuthology.orchestra.run.smithi204.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-05-30T15:51:01.820 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-05-30T15:51:01.820 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-05-30T15:51:01.820 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-05-30T15:51:01.820 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-05-30T15:51:01.820 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-05-30T15:51:01.821 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-05-30T15:51:01.821 INFO:teuthology.orchestra.run.smithi204.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-05-30T15:51:01.821 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-05-30T15:51:01.821 INFO:teuthology.orchestra.run.smithi204.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-05-30T15:51:01.821 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-05-30T15:51:01.821 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-05-30T15:51:01.821 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-05-30T15:51:01.821 INFO:teuthology.orchestra.run.smithi204.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-05-30T15:51:01.821 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-05-30T15:51:01.821 INFO:teuthology.orchestra.run.smithi204.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-05-30T15:51:01.821 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-05-30T15:51:01.821 INFO:teuthology.orchestra.run.smithi204.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-05-30T15:51:01.826 INFO:teuthology.orchestra.run.smithi006.stdout:The following packages were automatically installed and are no longer required: 2024-05-30T15:51:01.826 INFO:teuthology.orchestra.run.smithi006.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-30T15:51:01.826 INFO:teuthology.orchestra.run.smithi006.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-30T15:51:01.827 INFO:teuthology.orchestra.run.smithi006.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-30T15:51:01.827 INFO:teuthology.orchestra.run.smithi006.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-30T15:51:01.827 INFO:teuthology.orchestra.run.smithi006.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-30T15:51:01.827 INFO:teuthology.orchestra.run.smithi006.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-30T15:51:01.827 INFO:teuthology.orchestra.run.smithi006.stdout:The following additional packages will be installed: 2024-05-30T15:51:01.827 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-05-30T15:51:01.828 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-05-30T15:51:01.829 INFO:teuthology.orchestra.run.smithi006.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-05-30T15:51:01.829 INFO:teuthology.orchestra.run.smithi006.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-05-30T15:51:01.829 INFO:teuthology.orchestra.run.smithi006.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-05-30T15:51:01.829 INFO:teuthology.orchestra.run.smithi006.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-05-30T15:51:01.829 INFO:teuthology.orchestra.run.smithi006.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-05-30T15:51:01.829 INFO:teuthology.orchestra.run.smithi006.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-05-30T15:51:01.829 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-05-30T15:51:01.829 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-05-30T15:51:01.829 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-05-30T15:51:01.830 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-05-30T15:51:01.830 INFO:teuthology.orchestra.run.smithi006.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-05-30T15:51:01.830 INFO:teuthology.orchestra.run.smithi006.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-05-30T15:51:01.830 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-05-30T15:51:01.830 INFO:teuthology.orchestra.run.smithi006.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-05-30T15:51:01.830 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-05-30T15:51:01.830 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-05-30T15:51:01.830 INFO:teuthology.orchestra.run.smithi006.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-05-30T15:51:01.830 INFO:teuthology.orchestra.run.smithi006.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-05-30T15:51:01.830 INFO:teuthology.orchestra.run.smithi006.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-05-30T15:51:01.830 INFO:teuthology.orchestra.run.smithi006.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-05-30T15:51:01.830 INFO:teuthology.orchestra.run.smithi006.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-05-30T15:51:01.830 INFO:teuthology.orchestra.run.smithi006.stdout: xmlstarlet zip 2024-05-30T15:51:01.831 INFO:teuthology.orchestra.run.smithi006.stdout:Suggested packages: 2024-05-30T15:51:01.831 INFO:teuthology.orchestra.run.smithi006.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-05-30T15:51:01.831 INFO:teuthology.orchestra.run.smithi006.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-05-30T15:51:01.831 INFO:teuthology.orchestra.run.smithi006.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-05-30T15:51:01.831 INFO:teuthology.orchestra.run.smithi006.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-05-30T15:51:01.831 INFO:teuthology.orchestra.run.smithi006.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-05-30T15:51:01.831 INFO:teuthology.orchestra.run.smithi006.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-05-30T15:51:01.831 INFO:teuthology.orchestra.run.smithi006.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-05-30T15:51:01.831 INFO:teuthology.orchestra.run.smithi006.stdout:Recommended packages: 2024-05-30T15:51:01.831 INFO:teuthology.orchestra.run.smithi006.stdout: btrfs-tools 2024-05-30T15:51:01.884 INFO:teuthology.orchestra.run.smithi204.stdout:0 upgraded, 118 newly installed, 0 to remove and 307 not upgraded. 2024-05-30T15:51:01.884 INFO:teuthology.orchestra.run.smithi204.stdout:Need to get 189 MB of archives. 2024-05-30T15:51:01.884 INFO:teuthology.orchestra.run.smithi204.stdout:After this operation, 863 MB of additional disk space will be used. 2024-05-30T15:51:01.885 INFO:teuthology.orchestra.run.smithi204.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-05-30T15:51:01.938 INFO:teuthology.orchestra.run.smithi006.stdout:The following NEW packages will be installed: 2024-05-30T15:51:01.938 INFO:teuthology.orchestra.run.smithi006.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-05-30T15:51:01.939 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-05-30T15:51:01.939 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-05-30T15:51:01.940 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-05-30T15:51:01.940 INFO:teuthology.orchestra.run.smithi006.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-05-30T15:51:01.940 INFO:teuthology.orchestra.run.smithi006.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-05-30T15:51:01.940 INFO:teuthology.orchestra.run.smithi006.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-05-30T15:51:01.940 INFO:teuthology.orchestra.run.smithi006.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-05-30T15:51:01.940 INFO:teuthology.orchestra.run.smithi006.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-05-30T15:51:01.941 INFO:teuthology.orchestra.run.smithi006.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-05-30T15:51:01.941 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-05-30T15:51:01.941 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-05-30T15:51:01.941 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-05-30T15:51:01.941 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-05-30T15:51:01.941 INFO:teuthology.orchestra.run.smithi006.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-05-30T15:51:01.941 INFO:teuthology.orchestra.run.smithi006.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-05-30T15:51:01.941 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-05-30T15:51:01.941 INFO:teuthology.orchestra.run.smithi006.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-05-30T15:51:01.941 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-05-30T15:51:01.941 INFO:teuthology.orchestra.run.smithi006.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-05-30T15:51:01.941 INFO:teuthology.orchestra.run.smithi006.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-05-30T15:51:01.941 INFO:teuthology.orchestra.run.smithi006.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-05-30T15:51:01.941 INFO:teuthology.orchestra.run.smithi006.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-05-30T15:51:01.942 INFO:teuthology.orchestra.run.smithi006.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-05-30T15:51:01.942 INFO:teuthology.orchestra.run.smithi006.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-05-30T15:51:01.942 INFO:teuthology.orchestra.run.smithi006.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-05-30T15:51:01.991 INFO:teuthology.orchestra.run.smithi204.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-05-30T15:51:01.993 INFO:teuthology.orchestra.run.smithi204.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-05-30T15:51:02.010 INFO:teuthology.orchestra.run.smithi006.stdout:0 upgraded, 118 newly installed, 0 to remove and 307 not upgraded. 2024-05-30T15:51:02.010 INFO:teuthology.orchestra.run.smithi006.stdout:Need to get 189 MB of archives. 2024-05-30T15:51:02.010 INFO:teuthology.orchestra.run.smithi006.stdout:After this operation, 863 MB of additional disk space will be used. 2024-05-30T15:51:02.011 INFO:teuthology.orchestra.run.smithi006.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-05-30T15:51:02.014 INFO:teuthology.orchestra.run.smithi204.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-05-30T15:51:02.046 INFO:teuthology.orchestra.run.smithi204.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-4011-gda98024b-1jammy [3,568 kB] 2024-05-30T15:51:02.083 INFO:teuthology.orchestra.run.smithi204.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-05-30T15:51:02.084 INFO:teuthology.orchestra.run.smithi006.stdout:Get:2 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-4011-gda98024b-1jammy [3,568 kB] 2024-05-30T15:51:02.089 INFO:teuthology.orchestra.run.smithi204.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-05-30T15:51:02.102 INFO:teuthology.orchestra.run.smithi204.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-05-30T15:51:02.106 INFO:teuthology.orchestra.run.smithi204.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-05-30T15:51:02.107 INFO:teuthology.orchestra.run.smithi204.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-05-30T15:51:02.108 INFO:teuthology.orchestra.run.smithi204.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-05-30T15:51:02.117 INFO:teuthology.orchestra.run.smithi204.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-05-30T15:51:02.119 INFO:teuthology.orchestra.run.smithi204.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-05-30T15:51:02.121 INFO:teuthology.orchestra.run.smithi204.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-05-30T15:51:02.123 INFO:teuthology.orchestra.run.smithi204.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-05-30T15:51:02.134 INFO:teuthology.orchestra.run.smithi006.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-05-30T15:51:02.138 INFO:teuthology.orchestra.run.smithi006.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-05-30T15:51:02.148 INFO:teuthology.orchestra.run.smithi204.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-05-30T15:51:02.150 INFO:teuthology.orchestra.run.smithi204.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-05-30T15:51:02.152 INFO:teuthology.orchestra.run.smithi204.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-05-30T15:51:02.154 INFO:teuthology.orchestra.run.smithi204.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-05-30T15:51:02.155 INFO:teuthology.orchestra.run.smithi204.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-05-30T15:51:02.155 INFO:teuthology.orchestra.run.smithi204.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-05-30T15:51:02.156 INFO:teuthology.orchestra.run.smithi204.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-05-30T15:51:02.166 INFO:teuthology.orchestra.run.smithi006.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-05-30T15:51:02.169 INFO:teuthology.orchestra.run.smithi204.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-05-30T15:51:02.170 INFO:teuthology.orchestra.run.smithi204.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-05-30T15:51:02.170 INFO:teuthology.orchestra.run.smithi204.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-05-30T15:51:02.190 INFO:teuthology.orchestra.run.smithi204.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-05-30T15:51:02.190 INFO:teuthology.orchestra.run.smithi204.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-05-30T15:51:02.191 INFO:teuthology.orchestra.run.smithi204.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-05-30T15:51:02.196 INFO:teuthology.orchestra.run.smithi204.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-05-30T15:51:02.197 INFO:teuthology.orchestra.run.smithi204.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-05-30T15:51:02.197 INFO:teuthology.orchestra.run.smithi204.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-05-30T15:51:02.198 INFO:teuthology.orchestra.run.smithi204.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-05-30T15:51:02.215 INFO:teuthology.orchestra.run.smithi204.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-05-30T15:51:02.216 INFO:teuthology.orchestra.run.smithi204.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-05-30T15:51:02.217 INFO:teuthology.orchestra.run.smithi204.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-05-30T15:51:02.244 INFO:teuthology.orchestra.run.smithi204.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-05-30T15:51:02.245 INFO:teuthology.orchestra.run.smithi204.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-05-30T15:51:02.245 INFO:teuthology.orchestra.run.smithi006.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-05-30T15:51:02.246 INFO:teuthology.orchestra.run.smithi204.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-05-30T15:51:02.252 INFO:teuthology.orchestra.run.smithi006.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-05-30T15:51:02.255 INFO:teuthology.orchestra.run.smithi204.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-05-30T15:51:02.255 INFO:teuthology.orchestra.run.smithi204.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-05-30T15:51:02.256 INFO:teuthology.orchestra.run.smithi204.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-05-30T15:51:02.257 INFO:teuthology.orchestra.run.smithi204.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-05-30T15:51:02.268 INFO:teuthology.orchestra.run.smithi006.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-05-30T15:51:02.272 INFO:teuthology.orchestra.run.smithi006.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-4011-gda98024b-1jammy [3,229 kB] 2024-05-30T15:51:02.274 INFO:teuthology.orchestra.run.smithi006.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-05-30T15:51:02.274 INFO:teuthology.orchestra.run.smithi006.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-05-30T15:51:02.275 INFO:teuthology.orchestra.run.smithi006.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-05-30T15:51:02.281 INFO:teuthology.orchestra.run.smithi204.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-05-30T15:51:02.284 INFO:teuthology.orchestra.run.smithi204.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-05-30T15:51:02.286 INFO:teuthology.orchestra.run.smithi204.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-05-30T15:51:02.286 INFO:teuthology.orchestra.run.smithi006.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-05-30T15:51:02.289 INFO:teuthology.orchestra.run.smithi006.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-05-30T15:51:02.291 INFO:teuthology.orchestra.run.smithi006.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-05-30T15:51:02.293 INFO:teuthology.orchestra.run.smithi006.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-05-30T15:51:02.306 INFO:teuthology.orchestra.run.smithi204.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-05-30T15:51:02.308 INFO:teuthology.orchestra.run.smithi204.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-05-30T15:51:02.312 INFO:teuthology.orchestra.run.smithi204.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-05-30T15:51:02.313 INFO:teuthology.orchestra.run.smithi204.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-05-30T15:51:02.314 INFO:teuthology.orchestra.run.smithi204.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-05-30T15:51:02.318 INFO:teuthology.orchestra.run.smithi204.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-05-30T15:51:02.327 INFO:teuthology.orchestra.run.smithi006.stdout:Get:17 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-4011-gda98024b-1jammy [715 kB] 2024-05-30T15:51:02.332 INFO:teuthology.orchestra.run.smithi006.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-05-30T15:51:02.336 INFO:teuthology.orchestra.run.smithi006.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-05-30T15:51:02.341 INFO:teuthology.orchestra.run.smithi006.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-05-30T15:51:02.347 INFO:teuthology.orchestra.run.smithi006.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-05-30T15:51:02.348 INFO:teuthology.orchestra.run.smithi006.stdout:Get:22 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-4011-gda98024b-1jammy [356 kB] 2024-05-30T15:51:02.348 INFO:teuthology.orchestra.run.smithi006.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-05-30T15:51:02.348 INFO:teuthology.orchestra.run.smithi006.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-05-30T15:51:02.355 INFO:teuthology.orchestra.run.smithi006.stdout:Get:25 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-4011-gda98024b-1jammy [32.8 kB] 2024-05-30T15:51:02.356 INFO:teuthology.orchestra.run.smithi006.stdout:Get:26 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-4011-gda98024b-1jammy [180 kB] 2024-05-30T15:51:02.362 INFO:teuthology.orchestra.run.smithi006.stdout:Get:27 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-4011-gda98024b-1jammy [67.4 kB] 2024-05-30T15:51:02.363 INFO:teuthology.orchestra.run.smithi006.stdout:Get:28 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-4011-gda98024b-1jammy [334 kB] 2024-05-30T15:51:02.369 INFO:teuthology.orchestra.run.smithi006.stdout:Get:29 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-4011-gda98024b-1jammy [7,895 kB] 2024-05-30T15:51:02.391 INFO:teuthology.orchestra.run.smithi006.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-05-30T15:51:02.391 INFO:teuthology.orchestra.run.smithi006.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-05-30T15:51:02.392 INFO:teuthology.orchestra.run.smithi006.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-05-30T15:51:02.393 INFO:teuthology.orchestra.run.smithi006.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-05-30T15:51:02.409 INFO:teuthology.orchestra.run.smithi204.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-05-30T15:51:02.411 INFO:teuthology.orchestra.run.smithi204.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-05-30T15:51:02.415 INFO:teuthology.orchestra.run.smithi204.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-05-30T15:51:02.416 INFO:teuthology.orchestra.run.smithi204.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-05-30T15:51:02.418 INFO:teuthology.orchestra.run.smithi204.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-05-30T15:51:02.446 INFO:teuthology.orchestra.run.smithi006.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-05-30T15:51:02.447 INFO:teuthology.orchestra.run.smithi006.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-05-30T15:51:02.447 INFO:teuthology.orchestra.run.smithi006.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-05-30T15:51:02.453 INFO:teuthology.orchestra.run.smithi006.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-05-30T15:51:02.516 INFO:teuthology.orchestra.run.smithi204.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-05-30T15:51:02.517 INFO:teuthology.orchestra.run.smithi006.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-05-30T15:51:02.517 INFO:teuthology.orchestra.run.smithi006.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-05-30T15:51:02.520 INFO:teuthology.orchestra.run.smithi204.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-05-30T15:51:02.523 INFO:teuthology.orchestra.run.smithi204.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-05-30T15:51:02.524 INFO:teuthology.orchestra.run.smithi006.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-05-30T15:51:02.526 INFO:teuthology.orchestra.run.smithi006.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-05-30T15:51:02.527 INFO:teuthology.orchestra.run.smithi006.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-05-30T15:51:02.528 INFO:teuthology.orchestra.run.smithi006.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-05-30T15:51:02.531 INFO:teuthology.orchestra.run.smithi006.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-05-30T15:51:02.531 INFO:teuthology.orchestra.run.smithi006.stdout:Get:45 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-4011-gda98024b-1jammy [112 kB] 2024-05-30T15:51:02.532 INFO:teuthology.orchestra.run.smithi006.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-05-30T15:51:02.533 INFO:teuthology.orchestra.run.smithi006.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-05-30T15:51:02.533 INFO:teuthology.orchestra.run.smithi006.stdout:Get:48 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-4011-gda98024b-1jammy [470 kB] 2024-05-30T15:51:02.540 INFO:teuthology.orchestra.run.smithi006.stdout:Get:49 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-4011-gda98024b-1jammy [27.9 MB] 2024-05-30T15:51:02.554 INFO:teuthology.orchestra.run.smithi006.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-05-30T15:51:02.554 INFO:teuthology.orchestra.run.smithi006.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-05-30T15:51:02.556 INFO:teuthology.orchestra.run.smithi204.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-05-30T15:51:02.556 INFO:teuthology.orchestra.run.smithi204.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-05-30T15:51:02.558 INFO:teuthology.orchestra.run.smithi204.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-05-30T15:51:02.558 INFO:teuthology.orchestra.run.smithi204.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-05-30T15:51:02.559 INFO:teuthology.orchestra.run.smithi204.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-05-30T15:51:02.560 INFO:teuthology.orchestra.run.smithi204.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-05-30T15:51:02.561 INFO:teuthology.orchestra.run.smithi204.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-05-30T15:51:02.565 INFO:teuthology.orchestra.run.smithi204.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-05-30T15:51:02.570 INFO:teuthology.orchestra.run.smithi204.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-05-30T15:51:02.572 INFO:teuthology.orchestra.run.smithi006.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-05-30T15:51:02.573 INFO:teuthology.orchestra.run.smithi006.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-05-30T15:51:02.576 INFO:teuthology.orchestra.run.smithi006.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-05-30T15:51:02.580 INFO:teuthology.orchestra.run.smithi006.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-05-30T15:51:02.582 INFO:teuthology.orchestra.run.smithi006.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-05-30T15:51:02.585 INFO:teuthology.orchestra.run.smithi006.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-05-30T15:51:02.586 INFO:teuthology.orchestra.run.smithi006.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-05-30T15:51:02.593 INFO:teuthology.orchestra.run.smithi204.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-05-30T15:51:02.596 INFO:teuthology.orchestra.run.smithi204.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-05-30T15:51:02.597 INFO:teuthology.orchestra.run.smithi204.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-05-30T15:51:02.602 INFO:teuthology.orchestra.run.smithi204.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-05-30T15:51:02.607 INFO:teuthology.orchestra.run.smithi204.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-05-30T15:51:02.608 INFO:teuthology.orchestra.run.smithi204.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-05-30T15:51:02.609 INFO:teuthology.orchestra.run.smithi204.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-05-30T15:51:02.609 INFO:teuthology.orchestra.run.smithi006.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-05-30T15:51:02.610 INFO:teuthology.orchestra.run.smithi006.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-05-30T15:51:02.612 INFO:teuthology.orchestra.run.smithi006.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-05-30T15:51:02.616 INFO:teuthology.orchestra.run.smithi204.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-05-30T15:51:02.622 INFO:teuthology.orchestra.run.smithi204.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-05-30T15:51:02.627 INFO:teuthology.orchestra.run.smithi204.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-05-30T15:51:02.633 INFO:teuthology.orchestra.run.smithi006.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-05-30T15:51:02.660 INFO:teuthology.orchestra.run.smithi204.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-05-30T15:51:02.661 INFO:teuthology.orchestra.run.smithi204.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-05-30T15:51:02.662 INFO:teuthology.orchestra.run.smithi204.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-05-30T15:51:02.665 INFO:teuthology.orchestra.run.smithi204.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-05-30T15:51:02.666 INFO:teuthology.orchestra.run.smithi204.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-05-30T15:51:02.680 INFO:teuthology.orchestra.run.smithi204.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-05-30T15:51:02.681 INFO:teuthology.orchestra.run.smithi204.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-05-30T15:51:02.681 INFO:teuthology.orchestra.run.smithi204.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-05-30T15:51:02.715 INFO:teuthology.orchestra.run.smithi204.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-05-30T15:51:02.716 INFO:teuthology.orchestra.run.smithi204.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-05-30T15:51:02.785 INFO:teuthology.orchestra.run.smithi006.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-05-30T15:51:02.787 INFO:teuthology.orchestra.run.smithi006.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-05-30T15:51:02.790 INFO:teuthology.orchestra.run.smithi006.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-05-30T15:51:02.790 INFO:teuthology.orchestra.run.smithi006.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-05-30T15:51:02.792 INFO:teuthology.orchestra.run.smithi006.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-05-30T15:51:02.830 INFO:teuthology.orchestra.run.smithi006.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-05-30T15:51:02.834 INFO:teuthology.orchestra.run.smithi006.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-05-30T15:51:02.834 INFO:teuthology.orchestra.run.smithi006.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-05-30T15:51:02.857 INFO:teuthology.orchestra.run.smithi006.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-05-30T15:51:02.858 INFO:teuthology.orchestra.run.smithi006.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-05-30T15:51:02.858 INFO:teuthology.orchestra.run.smithi006.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-05-30T15:51:02.859 INFO:teuthology.orchestra.run.smithi006.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-05-30T15:51:02.859 INFO:teuthology.orchestra.run.smithi006.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-05-30T15:51:02.860 INFO:teuthology.orchestra.run.smithi006.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-05-30T15:51:02.865 INFO:teuthology.orchestra.run.smithi006.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-05-30T15:51:02.866 INFO:teuthology.orchestra.run.smithi006.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-05-30T15:51:02.874 INFO:teuthology.orchestra.run.smithi006.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-05-30T15:51:02.885 INFO:teuthology.orchestra.run.smithi006.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-05-30T15:51:02.887 INFO:teuthology.orchestra.run.smithi006.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-05-30T15:51:02.888 INFO:teuthology.orchestra.run.smithi006.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-05-30T15:51:02.894 INFO:teuthology.orchestra.run.smithi006.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-05-30T15:51:02.899 INFO:teuthology.orchestra.run.smithi006.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-05-30T15:51:02.902 INFO:teuthology.orchestra.run.smithi006.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-05-30T15:51:02.902 INFO:teuthology.orchestra.run.smithi006.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-05-30T15:51:02.909 INFO:teuthology.orchestra.run.smithi006.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-05-30T15:51:02.910 INFO:teuthology.orchestra.run.smithi006.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-05-30T15:51:02.917 INFO:teuthology.orchestra.run.smithi006.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-05-30T15:51:02.927 INFO:teuthology.orchestra.run.smithi006.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-05-30T15:51:02.927 INFO:teuthology.orchestra.run.smithi006.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-05-30T15:51:02.928 INFO:teuthology.orchestra.run.smithi006.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-05-30T15:51:02.931 INFO:teuthology.orchestra.run.smithi006.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-05-30T15:51:02.934 INFO:teuthology.orchestra.run.smithi006.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-05-30T15:51:02.944 INFO:teuthology.orchestra.run.smithi006.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-05-30T15:51:02.946 INFO:teuthology.orchestra.run.smithi006.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-05-30T15:51:02.947 INFO:teuthology.orchestra.run.smithi006.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-05-30T15:51:02.952 INFO:teuthology.orchestra.run.smithi006.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-05-30T15:51:02.952 INFO:teuthology.orchestra.run.smithi006.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-05-30T15:51:03.233 INFO:teuthology.orchestra.run.smithi006.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-4011-gda98024b-1jammy [5,094 kB] 2024-05-30T15:51:03.310 INFO:teuthology.orchestra.run.smithi006.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-4011-gda98024b-1jammy [244 kB] 2024-05-30T15:51:03.313 INFO:teuthology.orchestra.run.smithi006.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-4011-gda98024b-1jammy [124 kB] 2024-05-30T15:51:03.315 INFO:teuthology.orchestra.run.smithi006.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-4011-gda98024b-1jammy [1,547 kB] 2024-05-30T15:51:03.329 INFO:teuthology.orchestra.run.smithi006.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-4011-gda98024b-1jammy [6,264 kB] 2024-05-30T15:51:03.449 INFO:teuthology.orchestra.run.smithi006.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-4011-gda98024b-1jammy [22.8 MB] 2024-05-30T15:51:03.737 INFO:teuthology.orchestra.run.smithi204.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-4011-gda98024b-1jammy [3,229 kB] 2024-05-30T15:51:04.003 INFO:teuthology.orchestra.run.smithi006.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-4011-gda98024b-1jammy [14.0 kB] 2024-05-30T15:51:04.004 INFO:teuthology.orchestra.run.smithi006.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-4011-gda98024b-1jammy [871 kB] 2024-05-30T15:51:04.017 INFO:teuthology.orchestra.run.smithi006.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-4011-gda98024b-1jammy [2,444 kB] 2024-05-30T15:51:04.052 INFO:teuthology.orchestra.run.smithi006.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-4011-gda98024b-1jammy [792 kB] 2024-05-30T15:51:04.062 INFO:teuthology.orchestra.run.smithi006.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-4011-gda98024b-1jammy [149 kB] 2024-05-30T15:51:04.063 INFO:teuthology.orchestra.run.smithi006.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-4011-gda98024b-1jammy [2,261 kB] 2024-05-30T15:51:04.086 INFO:teuthology.orchestra.run.smithi006.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-4011-gda98024b-1jammy [8,625 kB] 2024-05-30T15:51:04.263 INFO:teuthology.orchestra.run.smithi006.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-4011-gda98024b-1jammy [14.2 kB] 2024-05-30T15:51:04.264 INFO:teuthology.orchestra.run.smithi006.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-4011-gda98024b-1jammy [54.8 MB] 2024-05-30T15:51:04.382 INFO:teuthology.orchestra.run.smithi204.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-4011-gda98024b-1jammy [715 kB] 2024-05-30T15:51:04.495 INFO:teuthology.orchestra.run.smithi204.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-4011-gda98024b-1jammy [356 kB] 2024-05-30T15:51:04.553 INFO:teuthology.orchestra.run.smithi204.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-4011-gda98024b-1jammy [32.8 kB] 2024-05-30T15:51:04.554 INFO:teuthology.orchestra.run.smithi204.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-4011-gda98024b-1jammy [180 kB] 2024-05-30T15:51:04.588 INFO:teuthology.orchestra.run.smithi204.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-4011-gda98024b-1jammy [67.4 kB] 2024-05-30T15:51:04.618 INFO:teuthology.orchestra.run.smithi204.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-4011-gda98024b-1jammy [334 kB] 2024-05-30T15:51:04.656 INFO:teuthology.orchestra.run.smithi204.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-4011-gda98024b-1jammy [7,895 kB] 2024-05-30T15:51:05.601 INFO:teuthology.orchestra.run.smithi204.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-4011-gda98024b-1jammy [112 kB] 2024-05-30T15:51:05.610 INFO:teuthology.orchestra.run.smithi204.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-4011-gda98024b-1jammy [470 kB] 2024-05-30T15:51:05.628 INFO:teuthology.orchestra.run.smithi204.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-4011-gda98024b-1jammy [27.9 MB] 2024-05-30T15:51:05.735 INFO:teuthology.orchestra.run.smithi006.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-4011-gda98024b-1jammy [124 kB] 2024-05-30T15:51:05.738 INFO:teuthology.orchestra.run.smithi006.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-4011-gda98024b-1jammy [39.0 kB] 2024-05-30T15:51:05.739 INFO:teuthology.orchestra.run.smithi006.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-4011-gda98024b-1jammy [15.7 MB] 2024-05-30T15:51:06.109 INFO:teuthology.orchestra.run.smithi006.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-4011-gda98024b-1jammy [91.1 kB] 2024-05-30T15:51:06.533 INFO:teuthology.orchestra.run.smithi204.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-4011-gda98024b-1jammy [5,094 kB] 2024-05-30T15:51:06.597 INFO:teuthology.orchestra.run.smithi204.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-4011-gda98024b-1jammy [244 kB] 2024-05-30T15:51:06.600 INFO:teuthology.orchestra.run.smithi204.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-4011-gda98024b-1jammy [124 kB] 2024-05-30T15:51:06.601 INFO:teuthology.orchestra.run.smithi204.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-4011-gda98024b-1jammy [1,547 kB] 2024-05-30T15:51:06.621 INFO:teuthology.orchestra.run.smithi204.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-4011-gda98024b-1jammy [6,264 kB] 2024-05-30T15:51:06.700 INFO:teuthology.orchestra.run.smithi006.stdout:Fetched 189 MB in 4s (45.5 MB/s) 2024-05-30T15:51:06.725 INFO:teuthology.orchestra.run.smithi204.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-4011-gda98024b-1jammy [22.8 MB] 2024-05-30T15:51:06.830 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-05-30T15:51:06.931 INFO:teuthology.orchestra.run.smithi006.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-05-30T15:51:06.937 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-05-30T15:51:07.012 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-30T15:51:07.176 INFO:teuthology.orchestra.run.smithi204.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-4011-gda98024b-1jammy [14.0 kB] 2024-05-30T15:51:07.176 INFO:teuthology.orchestra.run.smithi204.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-4011-gda98024b-1jammy [871 kB] 2024-05-30T15:51:07.185 INFO:teuthology.orchestra.run.smithi204.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-4011-gda98024b-1jammy [2,444 kB] 2024-05-30T15:51:07.209 INFO:teuthology.orchestra.run.smithi204.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-4011-gda98024b-1jammy [792 kB] 2024-05-30T15:51:07.217 INFO:teuthology.orchestra.run.smithi204.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-4011-gda98024b-1jammy [149 kB] 2024-05-30T15:51:07.228 INFO:teuthology.orchestra.run.smithi204.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-4011-gda98024b-1jammy [2,261 kB] 2024-05-30T15:51:07.255 INFO:teuthology.orchestra.run.smithi204.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-4011-gda98024b-1jammy [8,625 kB] 2024-05-30T15:51:07.332 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-05-30T15:51:07.360 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-05-30T15:51:07.398 INFO:teuthology.orchestra.run.smithi204.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-4011-gda98024b-1jammy [14.2 kB] 2024-05-30T15:51:07.398 INFO:teuthology.orchestra.run.smithi204.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-4011-gda98024b-1jammy [54.8 MB] 2024-05-30T15:51:07.424 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-30T15:51:07.760 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-05-30T15:51:07.789 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-05-30T15:51:07.827 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-30T15:51:08.222 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-05-30T15:51:08.249 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-30T15:51:08.288 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-30T15:51:08.656 INFO:teuthology.orchestra.run.smithi204.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-4011-gda98024b-1jammy [124 kB] 2024-05-30T15:51:08.658 INFO:teuthology.orchestra.run.smithi204.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-4011-gda98024b-1jammy [39.0 kB] 2024-05-30T15:51:08.659 INFO:teuthology.orchestra.run.smithi204.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-4011-gda98024b-1jammy [15.7 MB] 2024-05-30T15:51:08.683 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-05-30T15:51:08.711 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-30T15:51:08.750 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-30T15:51:09.013 INFO:teuthology.orchestra.run.smithi204.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-4011-gda98024b-1jammy [91.1 kB] 2024-05-30T15:51:09.103 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-05-30T15:51:09.130 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-30T15:51:09.169 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-30T15:51:09.506 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-05-30T15:51:09.522 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-05-30T15:51:09.556 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-30T15:51:09.575 INFO:teuthology.orchestra.run.smithi204.stdout:Fetched 189 MB in 7s (26.4 MB/s) 2024-05-30T15:51:09.740 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-05-30T15:51:09.840 INFO:teuthology.orchestra.run.smithi204.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-05-30T15:51:09.842 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package librados2. 2024-05-30T15:51:09.846 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-05-30T15:51:09.870 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../007-librados2_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:09.909 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking librados2 (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:09.938 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-30T15:51:10.234 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-05-30T15:51:10.260 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-05-30T15:51:10.292 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-30T15:51:10.362 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package librbd1. 2024-05-30T15:51:10.389 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../008-librbd1_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:10.428 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking librbd1 (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:10.628 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-05-30T15:51:10.655 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-05-30T15:51:10.695 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-30T15:51:10.856 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libcephfs2. 2024-05-30T15:51:10.883 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:10.922 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libcephfs2 (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:11.140 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-05-30T15:51:11.167 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-30T15:51:11.208 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-rados. 2024-05-30T15:51:11.215 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-30T15:51:11.224 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../010-python3-rados_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:11.258 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-rados (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:11.576 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-05-30T15:51:11.578 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-05-30T15:51:11.603 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-30T15:51:11.605 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-4011-gda98024b-1jammy_all.deb ... 2024-05-30T15:51:11.634 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-30T15:51:11.644 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-ceph-argparse (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:11.897 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-cephfs. 2024-05-30T15:51:11.912 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:11.955 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-cephfs (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:11.996 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-05-30T15:51:12.022 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-30T15:51:12.062 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-30T15:51:12.275 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-ceph-common. 2024-05-30T15:51:12.302 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-4011-gda98024b-1jammy_all.deb ... 2024-05-30T15:51:12.341 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-ceph-common (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:12.424 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-05-30T15:51:12.451 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-05-30T15:51:12.491 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-30T15:51:12.661 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-wcwidth. 2024-05-30T15:51:12.690 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-05-30T15:51:12.727 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-30T15:51:12.810 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package librados2. 2024-05-30T15:51:12.837 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../007-librados2_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:12.868 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking librados2 (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:13.005 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-prettytable. 2024-05-30T15:51:13.032 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-05-30T15:51:13.071 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-05-30T15:51:13.313 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package librbd1. 2024-05-30T15:51:13.340 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../008-librbd1_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:13.349 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-rbd. 2024-05-30T15:51:13.365 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:13.379 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking librbd1 (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:13.424 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-rbd (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:13.769 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-05-30T15:51:13.796 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-05-30T15:51:13.824 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libcephfs2. 2024-05-30T15:51:13.835 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-30T15:51:13.851 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:13.890 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libcephfs2 (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:14.113 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-05-30T15:51:14.129 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-05-30T15:51:14.171 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-05-30T15:51:14.176 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-rados. 2024-05-30T15:51:14.192 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../010-python3-rados_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:14.235 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-rados (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:14.449 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-05-30T15:51:14.476 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-05-30T15:51:14.516 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-05-30T15:51:14.521 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-05-30T15:51:14.548 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-4011-gda98024b-1jammy_all.deb ... 2024-05-30T15:51:14.587 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-ceph-argparse (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:14.840 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-cephfs. 2024-05-30T15:51:14.855 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:14.869 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package lua5.1. 2024-05-30T15:51:14.896 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-05-30T15:51:14.898 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-cephfs (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:14.935 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-05-30T15:51:15.218 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-ceph-common. 2024-05-30T15:51:15.244 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-4011-gda98024b-1jammy_all.deb ... 2024-05-30T15:51:15.255 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package lua-any. 2024-05-30T15:51:15.282 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-05-30T15:51:15.292 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-ceph-common (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:15.321 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking lua-any (27ubuntu1) ... 2024-05-30T15:51:15.541 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package zip. 2024-05-30T15:51:15.557 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-05-30T15:51:15.591 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking zip (3.0-12build2) ... 2024-05-30T15:51:15.638 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-wcwidth. 2024-05-30T15:51:15.664 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-05-30T15:51:15.704 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-30T15:51:15.894 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package unzip. 2024-05-30T15:51:15.921 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-05-30T15:51:15.960 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-05-30T15:51:16.007 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-prettytable. 2024-05-30T15:51:16.034 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-05-30T15:51:16.073 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-05-30T15:51:16.326 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-rbd. 2024-05-30T15:51:16.341 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:16.356 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package luarocks. 2024-05-30T15:51:16.376 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-rbd (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:16.383 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-05-30T15:51:16.422 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-05-30T15:51:16.721 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-05-30T15:51:16.748 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-05-30T15:51:16.788 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-30T15:51:16.813 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package librgw2. 2024-05-30T15:51:16.840 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../025-librgw2_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:16.880 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking librgw2 (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:17.073 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-05-30T15:51:17.089 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-05-30T15:51:17.132 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-05-30T15:51:17.443 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-05-30T15:51:17.470 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-05-30T15:51:17.474 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-rgw. 2024-05-30T15:51:17.490 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:17.509 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-05-30T15:51:17.532 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-rgw (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:17.813 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package lua5.1. 2024-05-30T15:51:17.829 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-05-30T15:51:17.860 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package liboath0:amd64. 2024-05-30T15:51:17.863 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-05-30T15:51:17.887 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-05-30T15:51:17.926 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-05-30T15:51:18.263 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libradosstriper1. 2024-05-30T15:51:18.290 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:18.329 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libradosstriper1 (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:18.416 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package lua-any. 2024-05-30T15:51:18.442 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-05-30T15:51:18.483 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking lua-any (27ubuntu1) ... 2024-05-30T15:51:18.727 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package zip. 2024-05-30T15:51:18.753 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-05-30T15:51:18.793 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking zip (3.0-12build2) ... 2024-05-30T15:51:18.958 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package ceph-common. 2024-05-30T15:51:18.985 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../029-ceph-common_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:19.083 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking ceph-common (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:19.089 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package unzip. 2024-05-30T15:51:19.115 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-05-30T15:51:19.147 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-05-30T15:51:19.534 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package luarocks. 2024-05-30T15:51:19.560 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-05-30T15:51:19.600 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-05-30T15:51:20.017 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package librgw2. 2024-05-30T15:51:20.044 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../025-librgw2_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:20.083 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking librgw2 (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:20.605 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package ceph-base. 2024-05-30T15:51:20.632 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../030-ceph-base_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:20.681 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking ceph-base (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:20.719 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-rgw. 2024-05-30T15:51:20.745 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:20.786 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-rgw (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:21.081 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package liboath0:amd64. 2024-05-30T15:51:21.096 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-05-30T15:51:21.130 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-05-30T15:51:21.251 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-05-30T15:51:21.278 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-05-30T15:51:21.317 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-05-30T15:51:21.408 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libradosstriper1. 2024-05-30T15:51:21.434 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:21.475 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libradosstriper1 (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:21.620 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-cheroot. 2024-05-30T15:51:21.647 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-05-30T15:51:21.686 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-30T15:51:21.886 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-common. 2024-05-30T15:51:21.913 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../029-ceph-common_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:21.953 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-common (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:22.031 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-05-30T15:51:22.058 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-05-30T15:51:22.097 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-05-30T15:51:22.350 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-jaraco.text. 2024-05-30T15:51:22.366 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-05-30T15:51:22.400 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-05-30T15:51:22.686 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-05-30T15:51:22.714 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-05-30T15:51:22.753 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-05-30T15:51:23.056 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-tempora. 2024-05-30T15:51:23.083 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-05-30T15:51:23.122 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-05-30T15:51:23.434 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-portend. 2024-05-30T15:51:23.461 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-05-30T15:51:23.467 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-base. 2024-05-30T15:51:23.493 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../030-ceph-base_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:23.501 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-05-30T15:51:23.534 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-base (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:24.028 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-05-30T15:51:24.055 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-05-30T15:51:24.094 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-05-30T15:51:24.188 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-05-30T15:51:24.214 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-05-30T15:51:24.254 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-05-30T15:51:24.398 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-cherrypy3. 2024-05-30T15:51:24.425 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-05-30T15:51:24.464 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-05-30T15:51:24.558 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-cheroot. 2024-05-30T15:51:24.583 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-05-30T15:51:24.624 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-30T15:51:24.784 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-natsort. 2024-05-30T15:51:24.800 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-05-30T15:51:24.834 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-05-30T15:51:24.943 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-05-30T15:51:24.970 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-05-30T15:51:25.010 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-05-30T15:51:25.146 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-logutils. 2024-05-30T15:51:25.173 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-05-30T15:51:25.212 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-05-30T15:51:25.297 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-jaraco.text. 2024-05-30T15:51:25.323 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-05-30T15:51:25.355 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-05-30T15:51:25.540 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-markupsafe. 2024-05-30T15:51:25.567 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-05-30T15:51:25.607 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-05-30T15:51:25.607 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-05-30T15:51:25.634 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-05-30T15:51:25.674 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-05-30T15:51:25.910 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-mako. 2024-05-30T15:51:25.937 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-05-30T15:51:25.976 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-30T15:51:26.044 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-tempora. 2024-05-30T15:51:26.070 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-05-30T15:51:26.110 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-05-30T15:51:26.262 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-simplegeneric. 2024-05-30T15:51:26.279 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-05-30T15:51:26.321 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-05-30T15:51:26.397 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-portend. 2024-05-30T15:51:26.423 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-05-30T15:51:26.463 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-05-30T15:51:26.590 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-singledispatch. 2024-05-30T15:51:26.617 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-05-30T15:51:26.657 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-05-30T15:51:26.716 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-05-30T15:51:26.731 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-05-30T15:51:26.766 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-05-30T15:51:26.960 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-webob. 2024-05-30T15:51:26.987 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-05-30T15:51:27.026 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-05-30T15:51:27.069 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-cherrypy3. 2024-05-30T15:51:27.095 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-05-30T15:51:27.135 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-05-30T15:51:27.387 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-waitress. 2024-05-30T15:51:27.414 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-05-30T15:51:27.457 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-30T15:51:27.464 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-natsort. 2024-05-30T15:51:27.491 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-05-30T15:51:27.522 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-05-30T15:51:27.773 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-tempita. 2024-05-30T15:51:27.800 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-logutils. 2024-05-30T15:51:27.801 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-05-30T15:51:27.827 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-05-30T15:51:27.840 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-05-30T15:51:27.867 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-05-30T15:51:28.110 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-paste. 2024-05-30T15:51:28.126 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-05-30T15:51:28.159 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-05-30T15:51:28.195 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-markupsafe. 2024-05-30T15:51:28.221 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-05-30T15:51:28.261 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-05-30T15:51:28.471 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-05-30T15:51:28.498 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-05-30T15:51:28.537 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-05-30T15:51:28.694 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-mako. 2024-05-30T15:51:28.721 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-05-30T15:51:28.806 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-30T15:51:28.815 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-pastedeploy. 2024-05-30T15:51:28.842 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-05-30T15:51:28.873 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-05-30T15:51:29.152 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-webtest. 2024-05-30T15:51:29.179 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-05-30T15:51:29.209 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-simplegeneric. 2024-05-30T15:51:29.218 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-05-30T15:51:29.236 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-05-30T15:51:29.267 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-05-30T15:51:29.529 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-pecan. 2024-05-30T15:51:29.537 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-singledispatch. 2024-05-30T15:51:29.556 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-05-30T15:51:29.563 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-05-30T15:51:29.596 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-05-30T15:51:29.603 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-05-30T15:51:29.898 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-webob. 2024-05-30T15:51:29.925 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-05-30T15:51:29.959 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-certifi. 2024-05-30T15:51:29.965 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-05-30T15:51:29.986 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-05-30T15:51:30.025 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-05-30T15:51:30.318 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-waitress. 2024-05-30T15:51:30.344 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-05-30T15:51:30.353 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-idna. 2024-05-30T15:51:30.381 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-05-30T15:51:30.387 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-30T15:51:30.420 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-05-30T15:51:30.696 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-tempita. 2024-05-30T15:51:30.722 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-05-30T15:51:30.739 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-urllib3. 2024-05-30T15:51:30.762 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-05-30T15:51:30.766 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-05-30T15:51:30.806 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-30T15:51:31.015 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-paste. 2024-05-30T15:51:31.030 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-05-30T15:51:31.065 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-05-30T15:51:31.084 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-requests. 2024-05-30T15:51:31.112 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-05-30T15:51:31.142 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-30T15:51:31.393 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-05-30T15:51:31.409 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-05-30T15:51:31.412 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-werkzeug. 2024-05-30T15:51:31.439 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-05-30T15:51:31.452 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-05-30T15:51:31.478 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-05-30T15:51:31.704 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pastedeploy. 2024-05-30T15:51:31.720 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-05-30T15:51:31.754 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-05-30T15:51:31.756 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-05-30T15:51:31.772 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-4011-gda98024b-1jammy_all.deb ... 2024-05-30T15:51:31.806 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking ceph-mgr-modules-core (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:32.083 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-webtest. 2024-05-30T15:51:32.109 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-05-30T15:51:32.149 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-05-30T15:51:32.178 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-05-30T15:51:32.205 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:32.244 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:32.460 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pecan. 2024-05-30T15:51:32.487 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-05-30T15:51:32.527 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-05-30T15:51:32.605 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-bcrypt. 2024-05-30T15:51:32.633 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-05-30T15:51:32.672 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-05-30T15:51:32.899 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-certifi. 2024-05-30T15:51:32.925 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-05-30T15:51:32.933 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-openssl. 2024-05-30T15:51:32.961 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-05-30T15:51:32.965 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-05-30T15:51:32.991 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-05-30T15:51:33.328 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package ceph-mgr. 2024-05-30T15:51:33.355 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:33.394 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking ceph-mgr (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:33.947 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package ceph-mon. 2024-05-30T15:51:33.974 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:34.055 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking ceph-mon (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:34.110 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-idna. 2024-05-30T15:51:34.137 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-05-30T15:51:34.177 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-05-30T15:51:34.455 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-urllib3. 2024-05-30T15:51:34.481 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-05-30T15:51:34.521 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-30T15:51:34.609 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package ceph-osd. 2024-05-30T15:51:34.636 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:34.667 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking ceph-osd (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:34.833 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-requests. 2024-05-30T15:51:34.859 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-05-30T15:51:34.899 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-30T15:51:35.194 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-werkzeug. 2024-05-30T15:51:35.221 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-05-30T15:51:35.260 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-05-30T15:51:35.530 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-05-30T15:51:35.546 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-4011-gda98024b-1jammy_all.deb ... 2024-05-30T15:51:35.588 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mgr-modules-core (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:35.968 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-05-30T15:51:35.995 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:36.035 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:36.279 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-bcrypt. 2024-05-30T15:51:36.295 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-05-30T15:51:36.337 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-05-30T15:51:36.396 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package ceph. 2024-05-30T15:51:36.423 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../066-ceph_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:36.462 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking ceph (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:36.626 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-openssl. 2024-05-30T15:51:36.652 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-05-30T15:51:36.692 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-05-30T15:51:36.773 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package ceph-fuse. 2024-05-30T15:51:36.801 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:36.839 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking ceph-fuse (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:36.996 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mgr. 2024-05-30T15:51:37.022 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:37.079 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mgr (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:37.176 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package ceph-mds. 2024-05-30T15:51:37.204 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:37.242 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking ceph-mds (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:37.457 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mon. 2024-05-30T15:51:37.483 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:37.524 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mon (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:37.595 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package cephadm. 2024-05-30T15:51:37.623 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../069-cephadm_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:37.662 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking cephadm (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:37.990 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-asyncssh. 2024-05-30T15:51:38.017 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-05-30T15:51:38.056 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-05-30T15:51:38.144 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-osd. 2024-05-30T15:51:38.171 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:38.210 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-osd (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:38.375 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python-babel-localedata. 2024-05-30T15:51:38.392 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-05-30T15:51:38.433 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-30T15:51:39.481 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph. 2024-05-30T15:51:39.505 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-babel. 2024-05-30T15:51:39.507 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../066-ceph_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:39.533 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-05-30T15:51:39.539 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:39.572 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-05-30T15:51:39.867 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-fuse. 2024-05-30T15:51:39.894 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:39.933 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-jinja2. 2024-05-30T15:51:39.934 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-fuse (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:39.961 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-05-30T15:51:39.999 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-05-30T15:51:40.295 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mds. 2024-05-30T15:51:40.322 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:40.328 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-05-30T15:51:40.355 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-4011-gda98024b-1jammy_all.deb ... 2024-05-30T15:51:40.361 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mds (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:40.385 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking ceph-mgr-cephadm (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:40.706 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-repoze.lru. 2024-05-30T15:51:40.733 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-05-30T15:51:40.756 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package cephadm. 2024-05-30T15:51:40.764 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-05-30T15:51:40.784 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../069-cephadm_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:40.823 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking cephadm (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:41.042 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-routes. 2024-05-30T15:51:41.069 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-05-30T15:51:41.108 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-05-30T15:51:41.151 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-asyncssh. 2024-05-30T15:51:41.178 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-05-30T15:51:41.217 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-05-30T15:51:41.453 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-05-30T15:51:41.480 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-4011-gda98024b-1jammy_all.deb ... 2024-05-30T15:51:41.519 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking ceph-mgr-dashboard (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:41.537 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python-babel-localedata. 2024-05-30T15:51:41.560 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-05-30T15:51:41.603 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-30T15:51:42.642 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-babel. 2024-05-30T15:51:42.670 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-05-30T15:51:42.709 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-05-30T15:51:42.749 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-05-30T15:51:42.766 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-05-30T15:51:42.807 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-30T15:51:43.020 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-jinja2. 2024-05-30T15:51:43.047 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-05-30T15:51:43.086 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-05-30T15:51:43.330 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-joblib. 2024-05-30T15:51:43.358 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-05-30T15:51:43.396 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-05-30T15:51:43.448 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-05-30T15:51:43.475 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-4011-gda98024b-1jammy_all.deb ... 2024-05-30T15:51:43.514 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mgr-cephadm (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:43.791 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-05-30T15:51:43.819 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-05-30T15:51:43.851 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-repoze.lru. 2024-05-30T15:51:43.857 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-05-30T15:51:43.879 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-05-30T15:51:43.918 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-05-30T15:51:44.177 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-sklearn. 2024-05-30T15:51:44.196 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-routes. 2024-05-30T15:51:44.205 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-05-30T15:51:44.211 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-05-30T15:51:44.245 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-05-30T15:51:44.302 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-30T15:51:44.599 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-05-30T15:51:44.626 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-4011-gda98024b-1jammy_all.deb ... 2024-05-30T15:51:44.665 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mgr-dashboard (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:45.077 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-05-30T15:51:45.106 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-4011-gda98024b-1jammy_all.deb ... 2024-05-30T15:51:45.237 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:45.896 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-05-30T15:51:45.912 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-05-30T15:51:45.954 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-30T15:51:46.207 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-cachetools. 2024-05-30T15:51:46.235 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-05-30T15:51:46.274 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-05-30T15:51:46.450 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-joblib. 2024-05-30T15:51:46.477 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-05-30T15:51:46.516 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-05-30T15:51:46.535 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-pyasn1. 2024-05-30T15:51:46.563 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-05-30T15:51:46.593 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-05-30T15:51:46.878 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-05-30T15:51:46.905 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-05-30T15:51:46.905 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-05-30T15:51:46.933 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-05-30T15:51:46.944 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-05-30T15:51:46.971 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-05-30T15:51:47.258 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-rsa. 2024-05-30T15:51:47.263 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-sklearn. 2024-05-30T15:51:47.285 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-05-30T15:51:47.290 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-05-30T15:51:47.324 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-rsa (4.8-1) ... 2024-05-30T15:51:47.330 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-30T15:51:47.628 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-google-auth. 2024-05-30T15:51:47.655 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-05-30T15:51:47.694 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-05-30T15:51:48.014 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-05-30T15:51:48.042 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-05-30T15:51:48.080 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-30T15:51:48.156 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-05-30T15:51:48.184 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-4011-gda98024b-1jammy_all.deb ... 2024-05-30T15:51:48.308 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:48.325 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-websocket. 2024-05-30T15:51:48.353 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-05-30T15:51:48.391 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-05-30T15:51:48.703 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-kubernetes. 2024-05-30T15:51:48.731 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-05-30T15:51:48.778 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-30T15:51:49.254 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-cachetools. 2024-05-30T15:51:49.281 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-05-30T15:51:49.320 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-05-30T15:51:49.392 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-05-30T15:51:49.409 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-4011-gda98024b-1jammy_all.deb ... 2024-05-30T15:51:49.450 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:49.720 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libonig5:amd64. 2024-05-30T15:51:49.748 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-05-30T15:51:49.786 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-30T15:51:49.832 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pyasn1. 2024-05-30T15:51:49.859 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-05-30T15:51:49.898 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-05-30T15:51:50.139 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libjq1:amd64. 2024-05-30T15:51:50.167 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-05-30T15:51:50.206 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-30T15:51:50.244 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-05-30T15:51:50.271 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-05-30T15:51:50.310 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-05-30T15:51:50.492 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package jq. 2024-05-30T15:51:50.508 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-05-30T15:51:50.550 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-05-30T15:51:50.621 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-rsa. 2024-05-30T15:51:50.648 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-05-30T15:51:50.688 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-rsa (4.8-1) ... 2024-05-30T15:51:50.760 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package socat. 2024-05-30T15:51:50.777 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-05-30T15:51:50.810 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-05-30T15:51:50.975 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-google-auth. 2024-05-30T15:51:51.002 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-05-30T15:51:51.042 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-05-30T15:51:51.072 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package xmlstarlet. 2024-05-30T15:51:51.088 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-05-30T15:51:51.122 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-05-30T15:51:51.370 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-05-30T15:51:51.397 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-05-30T15:51:51.436 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-30T15:51:51.476 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package ceph-test. 2024-05-30T15:51:51.493 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../097-ceph-test_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:51.534 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking ceph-test (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:51.748 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-websocket. 2024-05-30T15:51:51.775 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-05-30T15:51:51.814 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-05-30T15:51:52.152 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-kubernetes. 2024-05-30T15:51:52.179 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-05-30T15:51:52.218 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-30T15:51:52.932 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-05-30T15:51:52.960 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-4011-gda98024b-1jammy_all.deb ... 2024-05-30T15:51:52.998 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:53.293 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libonig5:amd64. 2024-05-30T15:51:53.321 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-05-30T15:51:53.351 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-30T15:51:53.671 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libjq1:amd64. 2024-05-30T15:51:53.698 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-05-30T15:51:53.737 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-30T15:51:53.889 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package ceph-volume. 2024-05-30T15:51:53.905 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-4011-gda98024b-1jammy_all.deb ... 2024-05-30T15:51:53.947 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking ceph-volume (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:53.999 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package jq. 2024-05-30T15:51:54.014 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-05-30T15:51:54.057 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-05-30T15:51:54.259 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libcephfs-dev. 2024-05-30T15:51:54.275 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:54.308 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libcephfs-dev (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:54.318 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package socat. 2024-05-30T15:51:54.334 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-05-30T15:51:54.368 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-05-30T15:51:54.654 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package xmlstarlet. 2024-05-30T15:51:54.670 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-05-30T15:51:54.780 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-05-30T15:51:54.803 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package lua-socket:amd64. 2024-05-30T15:51:54.832 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-05-30T15:51:54.869 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-05-30T15:51:55.193 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-test. 2024-05-30T15:51:55.207 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package lua-sec:amd64. 2024-05-30T15:51:55.221 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../097-ceph-test_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:55.236 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-05-30T15:51:55.259 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-test (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:55.274 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-05-30T15:51:55.619 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package nvme-cli. 2024-05-30T15:51:55.648 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-05-30T15:51:55.686 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-05-30T15:51:56.100 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package pkg-config. 2024-05-30T15:51:56.128 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-05-30T15:51:56.166 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-05-30T15:51:56.410 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-05-30T15:51:56.427 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-05-30T15:51:56.468 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-05-30T15:51:56.880 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-iniconfig. 2024-05-30T15:51:56.897 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-05-30T15:51:56.930 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-05-30T15:51:57.183 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-pastescript. 2024-05-30T15:51:57.211 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-05-30T15:51:57.249 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-05-30T15:51:57.586 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-pluggy. 2024-05-30T15:51:57.614 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-05-30T15:51:57.652 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-05-30T15:51:57.705 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-volume. 2024-05-30T15:51:57.733 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-4011-gda98024b-1jammy_all.deb ... 2024-05-30T15:51:57.780 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-volume (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:57.972 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-psutil. 2024-05-30T15:51:58.000 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-05-30T15:51:58.038 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-05-30T15:51:58.118 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libcephfs-dev. 2024-05-30T15:51:58.134 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:51:58.184 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libcephfs-dev (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:51:58.366 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-py. 2024-05-30T15:51:58.395 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-05-30T15:51:58.433 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-py (1.10.0-1) ... 2024-05-30T15:51:58.571 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package lua-socket:amd64. 2024-05-30T15:51:58.599 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-05-30T15:51:58.645 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-05-30T15:51:58.736 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-pygments. 2024-05-30T15:51:58.764 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-05-30T15:51:58.802 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-05-30T15:51:59.008 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package lua-sec:amd64. 2024-05-30T15:51:59.036 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-05-30T15:51:59.083 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-05-30T15:51:59.265 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-pyinotify. 2024-05-30T15:51:59.294 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-05-30T15:51:59.331 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-05-30T15:51:59.513 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package nvme-cli. 2024-05-30T15:51:59.540 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-05-30T15:51:59.587 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-05-30T15:51:59.934 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-toml. 2024-05-30T15:51:59.963 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-05-30T15:52:00.000 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-05-30T15:52:00.026 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package pkg-config. 2024-05-30T15:52:00.055 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-05-30T15:52:00.284 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-05-30T15:52:00.303 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-pytest. 2024-05-30T15:52:00.332 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-05-30T15:52:00.369 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-05-30T15:52:00.512 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-05-30T15:52:00.528 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-05-30T15:52:00.562 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-05-30T15:52:00.723 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-simplejson. 2024-05-30T15:52:00.751 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-05-30T15:52:00.790 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-05-30T15:52:00.982 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-iniconfig. 2024-05-30T15:52:00.998 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-05-30T15:52:01.032 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-05-30T15:52:01.067 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-05-30T15:52:01.084 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-05-30T15:52:01.125 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-05-30T15:52:01.319 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pastescript. 2024-05-30T15:52:01.346 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-05-30T15:52:01.385 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-05-30T15:52:01.689 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pluggy. 2024-05-30T15:52:01.716 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-05-30T15:52:01.755 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-05-30T15:52:01.846 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package radosgw. 2024-05-30T15:52:01.874 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../116-radosgw_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:52:01.904 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking radosgw (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:02.066 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-psutil. 2024-05-30T15:52:02.094 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-05-30T15:52:02.132 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-05-30T15:52:02.478 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-py. 2024-05-30T15:52:02.505 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-05-30T15:52:02.552 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-py (1.10.0-1) ... 2024-05-30T15:52:02.816 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package rbd-fuse. 2024-05-30T15:52:02.845 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:52:02.882 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking rbd-fuse (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:02.906 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pygments. 2024-05-30T15:52:02.934 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-05-30T15:52:02.972 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-05-30T15:52:03.196 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-05-30T15:52:03.419 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pyinotify. 2024-05-30T15:52:03.447 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-05-30T15:52:03.453 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-30T15:52:03.476 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-05-30T15:52:03.563 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-05-30T15:52:03.763 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-toml. 2024-05-30T15:52:03.791 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-05-30T15:52:03.822 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-05-30T15:52:03.829 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-05-30T15:52:04.132 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pytest. 2024-05-30T15:52:04.160 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-05-30T15:52:04.197 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-05-30T15:52:04.198 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-05-30T15:52:04.519 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-simplejson. 2024-05-30T15:52:04.547 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-05-30T15:52:04.585 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-05-30T15:52:04.612 INFO:teuthology.orchestra.run.smithi006.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-05-30T15:52:04.681 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up cephadm (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:04.871 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-05-30T15:52:04.888 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-05-30T15:52:04.929 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-05-30T15:52:05.265 INFO:teuthology.orchestra.run.smithi006.stdout:Adding system user cephadm....done 2024-05-30T15:52:05.421 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-30T15:52:05.676 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package radosgw. 2024-05-30T15:52:05.680 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-05-30T15:52:05.703 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../116-radosgw_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:52:05.741 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking radosgw (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:05.939 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-05-30T15:52:06.066 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-05-30T15:52:06.325 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-05-30T15:52:06.585 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-05-30T15:52:06.653 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package rbd-fuse. 2024-05-30T15:52:06.669 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-4011-gda98024b-1jammy_amd64.deb ... 2024-05-30T15:52:06.710 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-py (1.10.0-1) ... 2024-05-30T15:52:06.711 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking rbd-fuse (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:06.995 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-05-30T15:52:07.007 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-05-30T15:52:07.258 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-30T15:52:07.348 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-05-30T15:52:07.393 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-05-30T15:52:07.616 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-05-30T15:52:07.653 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-05-30T15:52:07.908 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-05-30T15:52:07.993 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-05-30T15:52:08.168 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-05-30T15:52:08.419 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-05-30T15:52:08.441 INFO:teuthology.orchestra.run.smithi204.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-05-30T15:52:08.512 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up cephadm (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:08.679 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-05-30T15:52:08.921 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-ceph-argparse (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:09.044 INFO:teuthology.orchestra.run.smithi204.stdout:Adding system user cephadm....done 2024-05-30T15:52:09.164 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-30T15:52:09.193 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-30T15:52:09.290 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-05-30T15:52:09.416 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-05-30T15:52:09.461 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-05-30T15:52:09.542 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-05-30T15:52:09.720 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-05-30T15:52:09.818 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-05-30T15:52:09.846 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-05-30T15:52:10.081 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-05-30T15:52:10.182 INFO:teuthology.orchestra.run.smithi006.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-05-30T15:52:10.215 INFO:teuthology.orchestra.run.smithi006.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-05-30T15:52:10.271 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-30T15:52:10.366 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-05-30T15:52:10.372 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-05-30T15:52:10.493 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-py (1.10.0-1) ... 2024-05-30T15:52:10.732 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-30T15:52:10.802 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-05-30T15:52:10.858 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-05-30T15:52:11.142 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-05-30T15:52:11.163 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-05-30T15:52:11.397 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-05-30T15:52:11.401 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-05-30T15:52:11.648 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-05-30T15:52:11.711 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up zip (3.0-12build2) ... 2024-05-30T15:52:11.812 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-05-30T15:52:11.867 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-05-30T15:52:12.135 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-05-30T15:52:12.394 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-05-30T15:52:12.514 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-05-30T15:52:12.653 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-ceph-argparse (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:12.773 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-05-30T15:52:12.899 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-05-30T15:52:12.904 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-30T15:52:13.014 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-05-30T15:52:13.025 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-30T15:52:13.115 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-05-30T15:52:13.216 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-05-30T15:52:13.334 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-05-30T15:52:13.475 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-05-30T15:52:13.719 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-05-30T15:52:13.876 INFO:teuthology.orchestra.run.smithi204.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-05-30T15:52:13.906 INFO:teuthology.orchestra.run.smithi204.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-05-30T15:52:13.953 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-05-30T15:52:13.962 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-30T15:52:14.063 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-05-30T15:52:14.355 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-30T15:52:14.423 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-30T15:52:14.532 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-05-30T15:52:14.648 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-05-30T15:52:14.791 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-05-30T15:52:15.008 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-05-30T15:52:15.059 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-05-30T15:52:15.369 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up zip (3.0-12build2) ... 2024-05-30T15:52:15.495 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-05-30T15:52:15.534 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-05-30T15:52:15.660 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-ceph-common (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:15.953 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-05-30T15:52:16.164 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-05-30T15:52:16.270 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-30T15:52:16.423 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-05-30T15:52:16.549 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-05-30T15:52:16.675 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-30T15:52:16.977 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-05-30T15:52:17.362 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-05-30T15:52:17.493 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-05-30T15:52:17.596 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-05-30T15:52:17.728 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-30T15:52:17.837 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-toml (0.10.2-1) ... 2024-05-30T15:52:17.998 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-30T15:52:18.180 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-30T15:52:18.299 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-05-30T15:52:18.306 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-05-30T15:52:18.432 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-30T15:52:18.633 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-05-30T15:52:18.733 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-05-30T15:52:18.885 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-05-30T15:52:18.992 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-05-30T15:52:19.011 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-ceph-common (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:19.277 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-30T15:52:19.320 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-05-30T15:52:19.378 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-05-30T15:52:19.630 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-30T15:52:19.679 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-05-30T15:52:19.939 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-05-30T15:52:20.206 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-05-30T15:52:20.466 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-05-30T15:52:20.725 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-05-30T15:52:20.788 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-05-30T15:52:20.993 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-30T15:52:21.031 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-30T15:52:21.094 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-05-30T15:52:21.140 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-toml (0.10.2-1) ... 2024-05-30T15:52:21.370 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-30T15:52:21.383 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-30T15:52:21.487 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-30T15:52:21.509 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-05-30T15:52:21.635 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-30T15:52:21.764 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-05-30T15:52:21.938 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-05-30T15:52:22.073 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-05-30T15:52:22.196 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-05-30T15:52:22.333 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-05-30T15:52:22.439 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-30T15:52:22.458 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-05-30T15:52:22.565 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-05-30T15:52:22.584 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-05-30T15:52:22.699 INFO:teuthology.orchestra.run.smithi006.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-05-30T15:52:22.858 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-05-30T15:52:22.936 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-30T15:52:23.062 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-05-30T15:52:23.101 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-05-30T15:52:23.360 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-05-30T15:52:23.464 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-05-30T15:52:23.612 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-05-30T15:52:23.740 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up lua-any (27ubuntu1) ... 2024-05-30T15:52:23.863 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-05-30T15:52:23.866 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-portend (3.0.0-1) ... 2024-05-30T15:52:24.106 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-30T15:52:24.133 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-30T15:52:24.215 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-05-30T15:52:24.259 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-05-30T15:52:24.491 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-30T15:52:24.577 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-05-30T15:52:24.617 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-30T15:52:24.878 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-30T15:52:24.902 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-05-30T15:52:25.171 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-05-30T15:52:25.203 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-05-30T15:52:25.297 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-rsa (4.8-1) ... 2024-05-30T15:52:25.438 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-05-30T15:52:25.564 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-05-30T15:52:25.573 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-05-30T15:52:25.715 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-05-30T15:52:25.786 INFO:teuthology.orchestra.run.smithi204.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-05-30T15:52:25.849 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-30T15:52:26.050 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-30T15:52:26.176 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-05-30T15:52:26.202 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-05-30T15:52:26.569 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-05-30T15:52:26.587 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-05-30T15:52:26.829 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up lua-any (27ubuntu1) ... 2024-05-30T15:52:26.881 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-05-30T15:52:26.930 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-portend (3.0.0-1) ... 2024-05-30T15:52:27.181 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-30T15:52:27.250 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-30T15:52:27.290 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-05-30T15:52:27.359 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up librados2 (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:27.460 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libsqlite3-mod-ceph (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:27.561 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-05-30T15:52:27.599 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-05-30T15:52:27.884 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-30T15:52:27.897 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libcephfs2 (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:28.024 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libradosstriper1 (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:28.150 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-05-30T15:52:28.152 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-05-30T15:52:28.278 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-rsa (4.8-1) ... 2024-05-30T15:52:28.443 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up librbd1 (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:28.538 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-05-30T15:52:28.569 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up ceph-mgr-modules-core (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:28.670 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up ceph-fuse (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:28.797 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-30T15:52:28.874 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-30T15:52:28.874 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-30T15:52:29.159 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-05-30T15:52:29.363 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libcephfs-dev (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:29.481 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-rados (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:29.569 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-05-30T15:52:29.606 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up librgw2 (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:29.732 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-30T15:52:29.845 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-05-30T15:52:30.165 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-30T15:52:30.266 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up librados2 (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:30.367 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libsqlite3-mod-ceph (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:30.493 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-05-30T15:52:30.977 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-rbd (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:31.103 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up rbd-fuse (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:31.171 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libcephfs2 (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:31.229 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-rgw (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:31.298 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libradosstriper1 (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:31.358 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-cephfs (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:31.432 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-05-30T15:52:31.484 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up ceph-common (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:31.692 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up librbd1 (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:31.818 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mgr-modules-core (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:31.944 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-fuse (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:31.985 INFO:teuthology.orchestra.run.smithi006.stdout:Adding group ceph....done 2024-05-30T15:52:32.147 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-30T15:52:32.148 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-30T15:52:32.345 INFO:teuthology.orchestra.run.smithi006.stdout:Adding system user ceph....done 2024-05-30T15:52:32.514 INFO:teuthology.orchestra.run.smithi006.stdout:Setting system user ceph properties....done 2024-05-30T15:52:32.531 INFO:teuthology.orchestra.run.smithi006.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-05-30T15:52:32.646 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libcephfs-dev (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:32.679 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-05-30T15:52:32.772 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-rados (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:32.898 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up librgw2 (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:33.008 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-30T15:52:33.066 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-05-30T15:52:33.535 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up ceph-test (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:33.636 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up radosgw (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:34.096 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-30T15:52:34.096 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-30T15:52:34.211 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-rbd (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:34.312 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up rbd-fuse (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:34.422 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-rgw (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:34.538 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up ceph-base (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:34.548 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-cephfs (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:34.674 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-common (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:34.896 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-05-30T15:52:35.200 INFO:teuthology.orchestra.run.smithi204.stdout:Adding group ceph....done 2024-05-30T15:52:35.364 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up ceph-mds (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:35.551 INFO:teuthology.orchestra.run.smithi204.stdout:Adding system user ceph....done 2024-05-30T15:52:35.591 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-30T15:52:35.591 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-30T15:52:35.722 INFO:teuthology.orchestra.run.smithi204.stdout:Setting system user ceph properties....done 2024-05-30T15:52:35.735 INFO:teuthology.orchestra.run.smithi204.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-05-30T15:52:35.870 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-05-30T15:52:36.083 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up ceph-mgr (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:36.215 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-05-30T15:52:36.311 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-30T15:52:36.311 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-30T15:52:36.692 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-test (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:36.784 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up ceph-osd (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:36.818 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up radosgw (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:37.056 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-30T15:52:37.056 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-30T15:52:37.268 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-30T15:52:37.268 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-30T15:52:37.569 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up ceph-mgr-k8sevents (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:37.674 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:37.728 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-base (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:37.822 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up ceph-mon (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:38.043 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-30T15:52:38.043 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-30T15:52:38.054 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-05-30T15:52:38.531 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up ceph-mgr-cephadm (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:38.547 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mds (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:38.674 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up ceph (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:38.794 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-30T15:52:38.794 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-30T15:52:38.816 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up ceph-mgr-dashboard (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:38.968 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up ceph-volume (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:39.257 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mgr (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:39.345 INFO:teuthology.orchestra.run.smithi006.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-05-30T15:52:39.438 INFO:teuthology.orchestra.run.smithi006.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-05-30T15:52:39.474 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-30T15:52:39.474 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-30T15:52:39.565 INFO:teuthology.orchestra.run.smithi006.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-30T15:52:39.950 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-osd (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:40.243 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-30T15:52:40.243 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-30T15:52:40.658 INFO:teuthology.orchestra.run.smithi006.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-05-30T15:52:40.735 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mgr-k8sevents (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:40.845 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:40.979 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mon (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:41.207 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-30T15:52:41.207 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-30T15:52:41.366 INFO:teuthology.orchestra.run.smithi006.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-30T15:52:41.369 DEBUG:teuthology.orchestra.run.smithi006:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install s3cmd maven 2024-05-30T15:52:41.425 INFO:teuthology.orchestra.run.smithi006.stdout:Reading package lists... 2024-05-30T15:52:41.631 INFO:teuthology.orchestra.run.smithi006.stdout:Building dependency tree... 2024-05-30T15:52:41.631 INFO:teuthology.orchestra.run.smithi006.stdout:Reading state information... 2024-05-30T15:52:41.698 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mgr-cephadm (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:41.824 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:41.828 INFO:teuthology.orchestra.run.smithi006.stdout:The following packages were automatically installed and are no longer required: 2024-05-30T15:52:41.828 INFO:teuthology.orchestra.run.smithi006.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-30T15:52:41.828 INFO:teuthology.orchestra.run.smithi006.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-30T15:52:41.828 INFO:teuthology.orchestra.run.smithi006.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-30T15:52:41.828 INFO:teuthology.orchestra.run.smithi006.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-30T15:52:41.829 INFO:teuthology.orchestra.run.smithi006.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-30T15:52:41.829 INFO:teuthology.orchestra.run.smithi006.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-30T15:52:41.830 INFO:teuthology.orchestra.run.smithi006.stdout:The following additional packages will be installed: 2024-05-30T15:52:41.830 INFO:teuthology.orchestra.run.smithi006.stdout: libaopalliance-java libapache-pom-java libatinject-jsr330-api-java 2024-05-30T15:52:41.830 INFO:teuthology.orchestra.run.smithi006.stdout: libcdi-api-java libcommons-cli-java libcommons-io-java libcommons-lang3-java 2024-05-30T15:52:41.830 INFO:teuthology.orchestra.run.smithi006.stdout: libcommons-parent-java libgeronimo-annotation-1.3-spec-java 2024-05-30T15:52:41.831 INFO:teuthology.orchestra.run.smithi006.stdout: libgeronimo-interceptor-3.0-spec-java libguava-java libguice-java 2024-05-30T15:52:41.831 INFO:teuthology.orchestra.run.smithi006.stdout: libhawtjni-runtime-java libjansi-java libjansi-native-java libjsr305-java 2024-05-30T15:52:41.831 INFO:teuthology.orchestra.run.smithi006.stdout: libmaven-parent-java libmaven-resolver-java libmaven-shared-utils-java 2024-05-30T15:52:41.831 INFO:teuthology.orchestra.run.smithi006.stdout: libmaven3-core-java libplexus-cipher-java libplexus-classworlds-java 2024-05-30T15:52:41.831 INFO:teuthology.orchestra.run.smithi006.stdout: libplexus-component-annotations-java libplexus-interpolation-java 2024-05-30T15:52:41.831 INFO:teuthology.orchestra.run.smithi006.stdout: libplexus-sec-dispatcher-java libplexus-utils2-java libsisu-inject-java 2024-05-30T15:52:41.831 INFO:teuthology.orchestra.run.smithi006.stdout: libsisu-plexus-java libslf4j-java libwagon-file-java 2024-05-30T15:52:41.831 INFO:teuthology.orchestra.run.smithi006.stdout: libwagon-http-shaded-java libwagon-provider-api-java python3-magic 2024-05-30T15:52:41.832 INFO:teuthology.orchestra.run.smithi006.stdout:Suggested packages: 2024-05-30T15:52:41.832 INFO:teuthology.orchestra.run.smithi006.stdout: libaopalliance-java-doc libatinject-jsr330-api-java-doc libel-api-java 2024-05-30T15:52:41.832 INFO:teuthology.orchestra.run.smithi006.stdout: libcommons-io-java-doc libcommons-lang3-java-doc libasm-java libcglib-java 2024-05-30T15:52:41.832 INFO:teuthology.orchestra.run.smithi006.stdout: libjsr305-java-doc libmaven-shared-utils-java-doc liblogback-java 2024-05-30T15:52:41.832 INFO:teuthology.orchestra.run.smithi006.stdout: libplexus-classworlds-java-doc libplexus-sec-dispatcher-java-doc 2024-05-30T15:52:41.832 INFO:teuthology.orchestra.run.smithi006.stdout: libplexus-utils2-java-doc testng libcommons-logging-java liblog4j1.2-java 2024-05-30T15:52:41.873 INFO:teuthology.orchestra.run.smithi006.stdout:The following NEW packages will be installed: 2024-05-30T15:52:41.873 INFO:teuthology.orchestra.run.smithi006.stdout: libaopalliance-java libapache-pom-java libatinject-jsr330-api-java 2024-05-30T15:52:41.873 INFO:teuthology.orchestra.run.smithi006.stdout: libcdi-api-java libcommons-cli-java libcommons-io-java libcommons-lang3-java 2024-05-30T15:52:41.873 INFO:teuthology.orchestra.run.smithi006.stdout: libcommons-parent-java libgeronimo-annotation-1.3-spec-java 2024-05-30T15:52:41.873 INFO:teuthology.orchestra.run.smithi006.stdout: libgeronimo-interceptor-3.0-spec-java libguava-java libguice-java 2024-05-30T15:52:41.874 INFO:teuthology.orchestra.run.smithi006.stdout: libhawtjni-runtime-java libjansi-java libjansi-native-java libjsr305-java 2024-05-30T15:52:41.874 INFO:teuthology.orchestra.run.smithi006.stdout: libmaven-parent-java libmaven-resolver-java libmaven-shared-utils-java 2024-05-30T15:52:41.874 INFO:teuthology.orchestra.run.smithi006.stdout: libmaven3-core-java libplexus-cipher-java libplexus-classworlds-java 2024-05-30T15:52:41.874 INFO:teuthology.orchestra.run.smithi006.stdout: libplexus-component-annotations-java libplexus-interpolation-java 2024-05-30T15:52:41.874 INFO:teuthology.orchestra.run.smithi006.stdout: libplexus-sec-dispatcher-java libplexus-utils2-java libsisu-inject-java 2024-05-30T15:52:41.874 INFO:teuthology.orchestra.run.smithi006.stdout: libsisu-plexus-java libslf4j-java libwagon-file-java 2024-05-30T15:52:41.875 INFO:teuthology.orchestra.run.smithi006.stdout: libwagon-http-shaded-java libwagon-provider-api-java maven python3-magic 2024-05-30T15:52:41.875 INFO:teuthology.orchestra.run.smithi006.stdout: s3cmd 2024-05-30T15:52:41.925 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mgr-dashboard (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:42.057 INFO:teuthology.orchestra.run.smithi006.stdout:0 upgraded, 35 newly installed, 0 to remove and 307 not upgraded. 2024-05-30T15:52:42.058 INFO:teuthology.orchestra.run.smithi006.stdout:Need to get 10.4 MB of archives. 2024-05-30T15:52:42.058 INFO:teuthology.orchestra.run.smithi006.stdout:After this operation, 13.9 MB of additional disk space will be used. 2024-05-30T15:52:42.058 INFO:teuthology.orchestra.run.smithi006.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libapache-pom-java all 18-1 [4,720 B] 2024-05-30T15:52:42.076 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-volume (19.0.0-4011-gda98024b-1jammy) ... 2024-05-30T15:52:42.140 INFO:teuthology.orchestra.run.smithi006.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libatinject-jsr330-api-java all 1.0+ds1-5 [5,348 B] 2024-05-30T15:52:42.140 INFO:teuthology.orchestra.run.smithi006.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libgeronimo-interceptor-3.0-spec-java all 1.0.1-4fakesync [8,616 B] 2024-05-30T15:52:42.170 INFO:teuthology.orchestra.run.smithi006.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcdi-api-java all 1.2-3 [54.3 kB] 2024-05-30T15:52:42.339 INFO:teuthology.orchestra.run.smithi006.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-cli-java all 1.4-2 [55.8 kB] 2024-05-30T15:52:42.415 INFO:teuthology.orchestra.run.smithi006.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-parent-java all 43-1 [10.8 kB] 2024-05-30T15:52:42.421 INFO:teuthology.orchestra.run.smithi204.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-05-30T15:52:42.423 INFO:teuthology.orchestra.run.smithi006.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-io-java all 2.11.0-2 [297 kB] 2024-05-30T15:52:42.514 INFO:teuthology.orchestra.run.smithi204.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-05-30T15:52:42.563 INFO:teuthology.orchestra.run.smithi006.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-lang3-java all 3.11-1 [526 kB] 2024-05-30T15:52:42.658 INFO:teuthology.orchestra.run.smithi006.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libgeronimo-annotation-1.3-spec-java all 1.3-1 [11.2 kB] 2024-05-30T15:52:42.658 INFO:teuthology.orchestra.run.smithi006.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjsr305-java all 0.1~+svn49-11 [27.0 kB] 2024-05-30T15:52:42.662 INFO:teuthology.orchestra.run.smithi006.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libguava-java all 29.0-6 [2,418 kB] 2024-05-30T15:52:42.666 INFO:teuthology.orchestra.run.smithi204.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-30T15:52:42.970 INFO:teuthology.orchestra.run.smithi006.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libaopalliance-java all 20070526-6 [9,084 B] 2024-05-30T15:52:43.053 INFO:teuthology.orchestra.run.smithi006.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libguice-java all 4.2.3-2 [1,434 kB] 2024-05-30T15:52:43.575 INFO:teuthology.orchestra.run.smithi006.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libhawtjni-runtime-java all 1.17-1 [28.8 kB] 2024-05-30T15:52:43.577 INFO:teuthology.orchestra.run.smithi006.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjansi-native-java all 1.8-1 [23.8 kB] 2024-05-30T15:52:43.578 INFO:teuthology.orchestra.run.smithi006.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjansi-java all 1.18-1 [56.8 kB] 2024-05-30T15:52:43.583 INFO:teuthology.orchestra.run.smithi006.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-parent-java all 31-2 [5,140 B] 2024-05-30T15:52:43.584 INFO:teuthology.orchestra.run.smithi006.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-utils2-java all 3.3.0-1 [250 kB] 2024-05-30T15:52:43.601 INFO:teuthology.orchestra.run.smithi006.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-provider-api-java all 3.3.4-1 [48.5 kB] 2024-05-30T15:52:43.604 INFO:teuthology.orchestra.run.smithi006.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-resolver-java all 1.4.2-3build1 [555 kB] 2024-05-30T15:52:43.636 INFO:teuthology.orchestra.run.smithi006.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libmaven-shared-utils-java all 3.3.0-1ubuntu0.22.04.1 [148 kB] 2024-05-30T15:52:43.643 INFO:teuthology.orchestra.run.smithi006.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-cipher-java all 1.8-2 [15.1 kB] 2024-05-30T15:52:43.726 INFO:teuthology.orchestra.run.smithi204.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-05-30T15:52:43.807 INFO:teuthology.orchestra.run.smithi006.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-classworlds-java all 2.6.0-1 [49.4 kB] 2024-05-30T15:52:44.071 INFO:teuthology.orchestra.run.smithi006.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-component-annotations-java all 2.1.0-1 [6,564 B] 2024-05-30T15:52:44.077 INFO:teuthology.orchestra.run.smithi006.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-interpolation-java all 1.26-1 [76.8 kB] 2024-05-30T15:52:44.173 INFO:teuthology.orchestra.run.smithi006.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-sec-dispatcher-java all 1.4-4 [28.1 kB] 2024-05-30T15:52:44.202 INFO:teuthology.orchestra.run.smithi006.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libslf4j-java all 1.7.32-1 [141 kB] 2024-05-30T15:52:44.282 INFO:teuthology.orchestra.run.smithi006.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libsisu-inject-java all 0.3.4-2 [347 kB] 2024-05-30T15:52:44.371 INFO:teuthology.orchestra.run.smithi204.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-30T15:52:44.374 DEBUG:teuthology.orchestra.run.smithi204:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install s3cmd maven 2024-05-30T15:52:44.381 INFO:teuthology.orchestra.run.smithi006.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libsisu-plexus-java all 0.3.4-3 [181 kB] 2024-05-30T15:52:44.411 INFO:teuthology.orchestra.run.smithi006.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven3-core-java all 3.6.3-5 [1,535 kB] 2024-05-30T15:52:44.430 INFO:teuthology.orchestra.run.smithi204.stdout:Reading package lists... 2024-05-30T15:52:44.537 INFO:teuthology.orchestra.run.smithi006.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-file-java all 3.3.4-1 [7,928 B] 2024-05-30T15:52:44.537 INFO:teuthology.orchestra.run.smithi006.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-http-shaded-java all 3.3.4-1 [1,855 kB] 2024-05-30T15:52:44.605 INFO:teuthology.orchestra.run.smithi006.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/universe amd64 maven all 3.6.3-5 [17.8 kB] 2024-05-30T15:52:44.620 INFO:teuthology.orchestra.run.smithi204.stdout:Building dependency tree... 2024-05-30T15:52:44.621 INFO:teuthology.orchestra.run.smithi204.stdout:Reading state information... 2024-05-30T15:52:44.770 INFO:teuthology.orchestra.run.smithi006.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-magic all 2:0.4.24-2 [12.6 kB] 2024-05-30T15:52:44.789 INFO:teuthology.orchestra.run.smithi204.stdout:The following packages were automatically installed and are no longer required: 2024-05-30T15:52:44.789 INFO:teuthology.orchestra.run.smithi204.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-30T15:52:44.789 INFO:teuthology.orchestra.run.smithi204.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-30T15:52:44.789 INFO:teuthology.orchestra.run.smithi204.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-30T15:52:44.789 INFO:teuthology.orchestra.run.smithi204.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-30T15:52:44.790 INFO:teuthology.orchestra.run.smithi204.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-30T15:52:44.790 INFO:teuthology.orchestra.run.smithi204.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-30T15:52:44.791 INFO:teuthology.orchestra.run.smithi204.stdout:The following additional packages will be installed: 2024-05-30T15:52:44.791 INFO:teuthology.orchestra.run.smithi204.stdout: libaopalliance-java libapache-pom-java libatinject-jsr330-api-java 2024-05-30T15:52:44.791 INFO:teuthology.orchestra.run.smithi204.stdout: libcdi-api-java libcommons-cli-java libcommons-io-java libcommons-lang3-java 2024-05-30T15:52:44.791 INFO:teuthology.orchestra.run.smithi204.stdout: libcommons-parent-java libgeronimo-annotation-1.3-spec-java 2024-05-30T15:52:44.791 INFO:teuthology.orchestra.run.smithi204.stdout: libgeronimo-interceptor-3.0-spec-java libguava-java libguice-java 2024-05-30T15:52:44.791 INFO:teuthology.orchestra.run.smithi204.stdout: libhawtjni-runtime-java libjansi-java libjansi-native-java libjsr305-java 2024-05-30T15:52:44.791 INFO:teuthology.orchestra.run.smithi204.stdout: libmaven-parent-java libmaven-resolver-java libmaven-shared-utils-java 2024-05-30T15:52:44.791 INFO:teuthology.orchestra.run.smithi204.stdout: libmaven3-core-java libplexus-cipher-java libplexus-classworlds-java 2024-05-30T15:52:44.791 INFO:teuthology.orchestra.run.smithi204.stdout: libplexus-component-annotations-java libplexus-interpolation-java 2024-05-30T15:52:44.791 INFO:teuthology.orchestra.run.smithi204.stdout: libplexus-sec-dispatcher-java libplexus-utils2-java libsisu-inject-java 2024-05-30T15:52:44.791 INFO:teuthology.orchestra.run.smithi204.stdout: libsisu-plexus-java libslf4j-java libwagon-file-java 2024-05-30T15:52:44.792 INFO:teuthology.orchestra.run.smithi204.stdout: libwagon-http-shaded-java libwagon-provider-api-java python3-magic 2024-05-30T15:52:44.792 INFO:teuthology.orchestra.run.smithi204.stdout:Suggested packages: 2024-05-30T15:52:44.792 INFO:teuthology.orchestra.run.smithi204.stdout: libaopalliance-java-doc libatinject-jsr330-api-java-doc libel-api-java 2024-05-30T15:52:44.792 INFO:teuthology.orchestra.run.smithi204.stdout: libcommons-io-java-doc libcommons-lang3-java-doc libasm-java libcglib-java 2024-05-30T15:52:44.792 INFO:teuthology.orchestra.run.smithi204.stdout: libjsr305-java-doc libmaven-shared-utils-java-doc liblogback-java 2024-05-30T15:52:44.793 INFO:teuthology.orchestra.run.smithi204.stdout: libplexus-classworlds-java-doc libplexus-sec-dispatcher-java-doc 2024-05-30T15:52:44.793 INFO:teuthology.orchestra.run.smithi204.stdout: libplexus-utils2-java-doc testng libcommons-logging-java liblog4j1.2-java 2024-05-30T15:52:44.830 INFO:teuthology.orchestra.run.smithi204.stdout:The following NEW packages will be installed: 2024-05-30T15:52:44.831 INFO:teuthology.orchestra.run.smithi204.stdout: libaopalliance-java libapache-pom-java libatinject-jsr330-api-java 2024-05-30T15:52:44.831 INFO:teuthology.orchestra.run.smithi204.stdout: libcdi-api-java libcommons-cli-java libcommons-io-java libcommons-lang3-java 2024-05-30T15:52:44.831 INFO:teuthology.orchestra.run.smithi204.stdout: libcommons-parent-java libgeronimo-annotation-1.3-spec-java 2024-05-30T15:52:44.831 INFO:teuthology.orchestra.run.smithi204.stdout: libgeronimo-interceptor-3.0-spec-java libguava-java libguice-java 2024-05-30T15:52:44.831 INFO:teuthology.orchestra.run.smithi204.stdout: libhawtjni-runtime-java libjansi-java libjansi-native-java libjsr305-java 2024-05-30T15:52:44.831 INFO:teuthology.orchestra.run.smithi204.stdout: libmaven-parent-java libmaven-resolver-java libmaven-shared-utils-java 2024-05-30T15:52:44.831 INFO:teuthology.orchestra.run.smithi204.stdout: libmaven3-core-java libplexus-cipher-java libplexus-classworlds-java 2024-05-30T15:52:44.831 INFO:teuthology.orchestra.run.smithi204.stdout: libplexus-component-annotations-java libplexus-interpolation-java 2024-05-30T15:52:44.831 INFO:teuthology.orchestra.run.smithi204.stdout: libplexus-sec-dispatcher-java libplexus-utils2-java libsisu-inject-java 2024-05-30T15:52:44.831 INFO:teuthology.orchestra.run.smithi204.stdout: libsisu-plexus-java libslf4j-java libwagon-file-java 2024-05-30T15:52:44.831 INFO:teuthology.orchestra.run.smithi204.stdout: libwagon-http-shaded-java libwagon-provider-api-java maven python3-magic 2024-05-30T15:52:44.831 INFO:teuthology.orchestra.run.smithi204.stdout: s3cmd 2024-05-30T15:52:44.852 INFO:teuthology.orchestra.run.smithi006.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/universe amd64 s3cmd all 2.2.0-1 [120 kB] 2024-05-30T15:52:45.013 INFO:teuthology.orchestra.run.smithi204.stdout:0 upgraded, 35 newly installed, 0 to remove and 307 not upgraded. 2024-05-30T15:52:45.013 INFO:teuthology.orchestra.run.smithi204.stdout:Need to get 10.4 MB of archives. 2024-05-30T15:52:45.013 INFO:teuthology.orchestra.run.smithi204.stdout:After this operation, 13.9 MB of additional disk space will be used. 2024-05-30T15:52:45.013 INFO:teuthology.orchestra.run.smithi204.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libapache-pom-java all 18-1 [4,720 B] 2024-05-30T15:52:45.096 INFO:teuthology.orchestra.run.smithi204.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libatinject-jsr330-api-java all 1.0+ds1-5 [5,348 B] 2024-05-30T15:52:45.097 INFO:teuthology.orchestra.run.smithi204.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libgeronimo-interceptor-3.0-spec-java all 1.0.1-4fakesync [8,616 B] 2024-05-30T15:52:45.115 INFO:teuthology.orchestra.run.smithi204.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcdi-api-java all 1.2-3 [54.3 kB] 2024-05-30T15:52:45.283 INFO:teuthology.orchestra.run.smithi204.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-cli-java all 1.4-2 [55.8 kB] 2024-05-30T15:52:45.358 INFO:teuthology.orchestra.run.smithi204.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-parent-java all 43-1 [10.8 kB] 2024-05-30T15:52:45.365 INFO:teuthology.orchestra.run.smithi204.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-io-java all 2.11.0-2 [297 kB] 2024-05-30T15:52:45.503 INFO:teuthology.orchestra.run.smithi204.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-lang3-java all 3.11-1 [526 kB] 2024-05-30T15:52:45.533 INFO:teuthology.orchestra.run.smithi006.stdout:Fetched 10.4 MB in 3s (3,252 kB/s) 2024-05-30T15:52:45.597 INFO:teuthology.orchestra.run.smithi204.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libgeronimo-annotation-1.3-spec-java all 1.3-1 [11.2 kB] 2024-05-30T15:52:45.597 INFO:teuthology.orchestra.run.smithi204.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjsr305-java all 0.1~+svn49-11 [27.0 kB] 2024-05-30T15:52:45.601 INFO:teuthology.orchestra.run.smithi204.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libguava-java all 29.0-6 [2,418 kB] 2024-05-30T15:52:45.622 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libapache-pom-java. 2024-05-30T15:52:45.708 INFO:teuthology.orchestra.run.smithi006.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173906 files and directories currently installed.) 2024-05-30T15:52:45.714 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../00-libapache-pom-java_18-1_all.deb ... 2024-05-30T15:52:45.754 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libapache-pom-java (18-1) ... 2024-05-30T15:52:45.908 INFO:teuthology.orchestra.run.smithi204.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libaopalliance-java all 20070526-6 [9,084 B] 2024-05-30T15:52:45.991 INFO:teuthology.orchestra.run.smithi204.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libguice-java all 4.2.3-2 [1,434 kB] 2024-05-30T15:52:46.007 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libatinject-jsr330-api-java. 2024-05-30T15:52:46.036 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../01-libatinject-jsr330-api-java_1.0+ds1-5_all.deb ... 2024-05-30T15:52:46.074 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libatinject-jsr330-api-java (1.0+ds1-5) ... 2024-05-30T15:52:46.513 INFO:teuthology.orchestra.run.smithi204.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libhawtjni-runtime-java all 1.17-1 [28.8 kB] 2024-05-30T15:52:46.514 INFO:teuthology.orchestra.run.smithi204.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjansi-native-java all 1.8-1 [23.8 kB] 2024-05-30T15:52:46.516 INFO:teuthology.orchestra.run.smithi204.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjansi-java all 1.18-1 [56.8 kB] 2024-05-30T15:52:46.521 INFO:teuthology.orchestra.run.smithi204.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-parent-java all 31-2 [5,140 B] 2024-05-30T15:52:46.521 INFO:teuthology.orchestra.run.smithi204.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-utils2-java all 3.3.0-1 [250 kB] 2024-05-30T15:52:46.538 INFO:teuthology.orchestra.run.smithi204.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-provider-api-java all 3.3.4-1 [48.5 kB] 2024-05-30T15:52:46.542 INFO:teuthology.orchestra.run.smithi204.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-resolver-java all 1.4.2-3build1 [555 kB] 2024-05-30T15:52:46.573 INFO:teuthology.orchestra.run.smithi204.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libmaven-shared-utils-java all 3.3.0-1ubuntu0.22.04.1 [148 kB] 2024-05-30T15:52:46.580 INFO:teuthology.orchestra.run.smithi204.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-cipher-java all 1.8-2 [15.1 kB] 2024-05-30T15:52:46.721 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libgeronimo-interceptor-3.0-spec-java. 2024-05-30T15:52:46.745 INFO:teuthology.orchestra.run.smithi204.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-classworlds-java all 2.6.0-1 [49.4 kB] 2024-05-30T15:52:46.749 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../02-libgeronimo-interceptor-3.0-spec-java_1.0.1-4fakesync_all.deb ... 2024-05-30T15:52:46.786 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libgeronimo-interceptor-3.0-spec-java (1.0.1-4fakesync) ... 2024-05-30T15:52:46.999 INFO:teuthology.orchestra.run.smithi204.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-component-annotations-java all 2.1.0-1 [6,564 B] 2024-05-30T15:52:47.005 INFO:teuthology.orchestra.run.smithi204.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-interpolation-java all 1.26-1 [76.8 kB] 2024-05-30T15:52:47.022 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libcdi-api-java. 2024-05-30T15:52:47.039 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../03-libcdi-api-java_1.2-3_all.deb ... 2024-05-30T15:52:47.080 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libcdi-api-java (1.2-3) ... 2024-05-30T15:52:47.099 INFO:teuthology.orchestra.run.smithi204.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-sec-dispatcher-java all 1.4-4 [28.1 kB] 2024-05-30T15:52:47.126 INFO:teuthology.orchestra.run.smithi204.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libslf4j-java all 1.7.32-1 [141 kB] 2024-05-30T15:52:47.207 INFO:teuthology.orchestra.run.smithi204.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libsisu-inject-java all 0.3.4-2 [347 kB] 2024-05-30T15:52:47.306 INFO:teuthology.orchestra.run.smithi204.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libsisu-plexus-java all 0.3.4-3 [181 kB] 2024-05-30T15:52:47.332 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libcommons-cli-java. 2024-05-30T15:52:47.335 INFO:teuthology.orchestra.run.smithi204.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven3-core-java all 3.6.3-5 [1,535 kB] 2024-05-30T15:52:47.350 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../04-libcommons-cli-java_1.4-2_all.deb ... 2024-05-30T15:52:47.391 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libcommons-cli-java (1.4-2) ... 2024-05-30T15:52:47.462 INFO:teuthology.orchestra.run.smithi204.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-file-java all 3.3.4-1 [7,928 B] 2024-05-30T15:52:47.462 INFO:teuthology.orchestra.run.smithi204.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-http-shaded-java all 3.3.4-1 [1,855 kB] 2024-05-30T15:52:47.532 INFO:teuthology.orchestra.run.smithi204.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/universe amd64 maven all 3.6.3-5 [17.8 kB] 2024-05-30T15:52:47.618 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libcommons-parent-java. 2024-05-30T15:52:47.635 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../05-libcommons-parent-java_43-1_all.deb ... 2024-05-30T15:52:47.668 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libcommons-parent-java (43-1) ... 2024-05-30T15:52:47.696 INFO:teuthology.orchestra.run.smithi204.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-magic all 2:0.4.24-2 [12.6 kB] 2024-05-30T15:52:47.779 INFO:teuthology.orchestra.run.smithi204.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/universe amd64 s3cmd all 2.2.0-1 [120 kB] 2024-05-30T15:52:47.921 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libcommons-io-java. 2024-05-30T15:52:47.938 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../06-libcommons-io-java_2.11.0-2_all.deb ... 2024-05-30T15:52:47.979 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libcommons-io-java (2.11.0-2) ... 2024-05-30T15:52:48.248 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libcommons-lang3-java. 2024-05-30T15:52:48.265 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../07-libcommons-lang3-java_3.11-1_all.deb ... 2024-05-30T15:52:48.306 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libcommons-lang3-java (3.11-1) ... 2024-05-30T15:52:48.432 INFO:teuthology.orchestra.run.smithi204.stdout:Fetched 10.4 MB in 3s (3,288 kB/s) 2024-05-30T15:52:48.514 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libapache-pom-java. 2024-05-30T15:52:48.587 INFO:teuthology.orchestra.run.smithi204.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173906 files and directories currently installed.) 2024-05-30T15:52:48.593 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../00-libapache-pom-java_18-1_all.deb ... 2024-05-30T15:52:48.600 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libgeronimo-annotation-1.3-spec-java. 2024-05-30T15:52:48.618 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../08-libgeronimo-annotation-1.3-spec-java_1.3-1_all.deb ... 2024-05-30T15:52:48.638 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libapache-pom-java (18-1) ... 2024-05-30T15:52:48.659 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libgeronimo-annotation-1.3-spec-java (1.3-1) ... 2024-05-30T15:52:48.878 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libjsr305-java. 2024-05-30T15:52:48.883 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libatinject-jsr330-api-java. 2024-05-30T15:52:48.895 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../09-libjsr305-java_0.1~+svn49-11_all.deb ... 2024-05-30T15:52:48.899 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../01-libatinject-jsr330-api-java_1.0+ds1-5_all.deb ... 2024-05-30T15:52:48.928 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libjsr305-java (0.1~+svn49-11) ... 2024-05-30T15:52:48.933 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libatinject-jsr330-api-java (1.0+ds1-5) ... 2024-05-30T15:52:49.177 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libgeronimo-interceptor-3.0-spec-java. 2024-05-30T15:52:49.180 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libguava-java. 2024-05-30T15:52:49.193 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../02-libgeronimo-interceptor-3.0-spec-java_1.0.1-4fakesync_all.deb ... 2024-05-30T15:52:49.197 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../10-libguava-java_29.0-6_all.deb ... 2024-05-30T15:52:49.235 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libgeronimo-interceptor-3.0-spec-java (1.0.1-4fakesync) ... 2024-05-30T15:52:49.239 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libguava-java (29.0-6) ... 2024-05-30T15:52:49.488 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libcdi-api-java. 2024-05-30T15:52:49.516 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../03-libcdi-api-java_1.2-3_all.deb ... 2024-05-30T15:52:49.580 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libcdi-api-java (1.2-3) ... 2024-05-30T15:52:49.683 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libaopalliance-java. 2024-05-30T15:52:49.700 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../11-libaopalliance-java_20070526-6_all.deb ... 2024-05-30T15:52:49.742 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libaopalliance-java (20070526-6) ... 2024-05-30T15:52:49.841 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libcommons-cli-java. 2024-05-30T15:52:49.857 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../04-libcommons-cli-java_1.4-2_all.deb ... 2024-05-30T15:52:49.891 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libcommons-cli-java (1.4-2) ... 2024-05-30T15:52:49.986 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libguice-java. 2024-05-30T15:52:50.003 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../12-libguice-java_4.2.3-2_all.deb ... 2024-05-30T15:52:50.044 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libguice-java (4.2.3-2) ... 2024-05-30T15:52:50.118 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libcommons-parent-java. 2024-05-30T15:52:50.134 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../05-libcommons-parent-java_43-1_all.deb ... 2024-05-30T15:52:50.168 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libcommons-parent-java (43-1) ... 2024-05-30T15:52:50.381 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libhawtjni-runtime-java. 2024-05-30T15:52:50.396 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libcommons-io-java. 2024-05-30T15:52:50.398 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../13-libhawtjni-runtime-java_1.17-1_all.deb ... 2024-05-30T15:52:50.424 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../06-libcommons-io-java_2.11.0-2_all.deb ... 2024-05-30T15:52:50.439 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libhawtjni-runtime-java (1.17-1) ... 2024-05-30T15:52:50.462 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libcommons-io-java (2.11.0-2) ... 2024-05-30T15:52:50.692 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libjansi-native-java. 2024-05-30T15:52:50.709 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../14-libjansi-native-java_1.8-1_all.deb ... 2024-05-30T15:52:50.723 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libcommons-lang3-java. 2024-05-30T15:52:50.750 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libjansi-native-java (1.8-1) ... 2024-05-30T15:52:50.751 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../07-libcommons-lang3-java_3.11-1_all.deb ... 2024-05-30T15:52:50.790 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libcommons-lang3-java (3.11-1) ... 2024-05-30T15:52:50.945 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libjansi-java. 2024-05-30T15:52:50.962 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../15-libjansi-java_1.18-1_all.deb ... 2024-05-30T15:52:51.003 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libjansi-java (1.18-1) ... 2024-05-30T15:52:51.084 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libgeronimo-annotation-1.3-spec-java. 2024-05-30T15:52:51.100 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../08-libgeronimo-annotation-1.3-spec-java_1.3-1_all.deb ... 2024-05-30T15:52:51.142 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libgeronimo-annotation-1.3-spec-java (1.3-1) ... 2024-05-30T15:52:51.256 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libmaven-parent-java. 2024-05-30T15:52:51.272 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../16-libmaven-parent-java_31-2_all.deb ... 2024-05-30T15:52:51.334 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libmaven-parent-java (31-2) ... 2024-05-30T15:52:51.379 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libjsr305-java. 2024-05-30T15:52:51.406 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../09-libjsr305-java_0.1~+svn49-11_all.deb ... 2024-05-30T15:52:51.437 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libjsr305-java (0.1~+svn49-11) ... 2024-05-30T15:52:51.600 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libplexus-utils2-java. 2024-05-30T15:52:51.628 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../17-libplexus-utils2-java_3.3.0-1_all.deb ... 2024-05-30T15:52:51.666 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libplexus-utils2-java (3.3.0-1) ... 2024-05-30T15:52:51.673 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libguava-java. 2024-05-30T15:52:51.701 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../10-libguava-java_29.0-6_all.deb ... 2024-05-30T15:52:51.740 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libguava-java (29.0-6) ... 2024-05-30T15:52:52.004 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libwagon-provider-api-java. 2024-05-30T15:52:52.021 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../18-libwagon-provider-api-java_3.3.4-1_all.deb ... 2024-05-30T15:52:52.053 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libwagon-provider-api-java (3.3.4-1) ... 2024-05-30T15:52:52.289 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libmaven-resolver-java. 2024-05-30T15:52:52.295 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libaopalliance-java. 2024-05-30T15:52:52.306 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../19-libmaven-resolver-java_1.4.2-3build1_all.deb ... 2024-05-30T15:52:52.311 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../11-libaopalliance-java_20070526-6_all.deb ... 2024-05-30T15:52:52.347 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libmaven-resolver-java (1.4.2-3build1) ... 2024-05-30T15:52:52.352 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libaopalliance-java (20070526-6) ... 2024-05-30T15:52:52.606 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libguice-java. 2024-05-30T15:52:52.621 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../12-libguice-java_4.2.3-2_all.deb ... 2024-05-30T15:52:52.655 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libguice-java (4.2.3-2) ... 2024-05-30T15:52:52.668 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libmaven-shared-utils-java. 2024-05-30T15:52:52.697 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../20-libmaven-shared-utils-java_3.3.0-1ubuntu0.22.04.1_all.deb ... 2024-05-30T15:52:52.735 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libmaven-shared-utils-java (3.3.0-1ubuntu0.22.04.1) ... 2024-05-30T15:52:53.021 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libplexus-cipher-java. 2024-05-30T15:52:53.034 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libhawtjni-runtime-java. 2024-05-30T15:52:53.038 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../21-libplexus-cipher-java_1.8-2_all.deb ... 2024-05-30T15:52:53.051 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../13-libhawtjni-runtime-java_1.17-1_all.deb ... 2024-05-30T15:52:53.079 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libplexus-cipher-java (1.8-2) ... 2024-05-30T15:52:53.092 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libhawtjni-runtime-java (1.17-1) ... 2024-05-30T15:52:53.290 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libplexus-classworlds-java. 2024-05-30T15:52:53.319 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../22-libplexus-classworlds-java_2.6.0-1_all.deb ... 2024-05-30T15:52:53.348 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libplexus-classworlds-java (2.6.0-1) ... 2024-05-30T15:52:53.354 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libjansi-native-java. 2024-05-30T15:52:53.382 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../14-libjansi-native-java_1.8-1_all.deb ... 2024-05-30T15:52:53.420 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libjansi-native-java (1.8-1) ... 2024-05-30T15:52:53.601 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libplexus-component-annotations-java. 2024-05-30T15:52:53.618 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../23-libplexus-component-annotations-java_2.1.0-1_all.deb ... 2024-05-30T15:52:53.659 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libplexus-component-annotations-java (2.1.0-1) ... 2024-05-30T15:52:53.673 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libjansi-java. 2024-05-30T15:52:53.701 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../15-libjansi-java_1.18-1_all.deb ... 2024-05-30T15:52:53.739 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libjansi-java (1.18-1) ... 2024-05-30T15:52:53.912 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libplexus-interpolation-java. 2024-05-30T15:52:53.940 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../24-libplexus-interpolation-java_1.26-1_all.deb ... 2024-05-30T15:52:53.975 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libmaven-parent-java. 2024-05-30T15:52:53.978 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libplexus-interpolation-java (1.26-1) ... 2024-05-30T15:52:53.992 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../16-libmaven-parent-java_31-2_all.deb ... 2024-05-30T15:52:54.034 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libmaven-parent-java (31-2) ... 2024-05-30T15:52:54.247 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libplexus-sec-dispatcher-java. 2024-05-30T15:52:54.264 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../25-libplexus-sec-dispatcher-java_1.4-4_all.deb ... 2024-05-30T15:52:54.286 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libplexus-utils2-java. 2024-05-30T15:52:54.305 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libplexus-sec-dispatcher-java (1.4-4) ... 2024-05-30T15:52:54.314 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../17-libplexus-utils2-java_3.3.0-1_all.deb ... 2024-05-30T15:52:54.353 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libplexus-utils2-java (3.3.0-1) ... 2024-05-30T15:52:54.517 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libslf4j-java. 2024-05-30T15:52:54.534 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../26-libslf4j-java_1.7.32-1_all.deb ... 2024-05-30T15:52:54.575 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libslf4j-java (1.7.32-1) ... 2024-05-30T15:52:54.622 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libwagon-provider-api-java. 2024-05-30T15:52:54.638 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../18-libwagon-provider-api-java_3.3.4-1_all.deb ... 2024-05-30T15:52:54.672 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libwagon-provider-api-java (3.3.4-1) ... 2024-05-30T15:52:54.837 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libsisu-inject-java. 2024-05-30T15:52:54.866 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../27-libsisu-inject-java_0.3.4-2_all.deb ... 2024-05-30T15:52:54.883 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libmaven-resolver-java. 2024-05-30T15:52:54.899 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../19-libmaven-resolver-java_1.4.2-3build1_all.deb ... 2024-05-30T15:52:54.928 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libsisu-inject-java (0.3.4-2) ... 2024-05-30T15:52:54.941 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libmaven-resolver-java (1.4.2-3build1) ... 2024-05-30T15:52:55.223 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libsisu-plexus-java. 2024-05-30T15:52:55.252 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../28-libsisu-plexus-java_0.3.4-3_all.deb ... 2024-05-30T15:52:55.271 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libmaven-shared-utils-java. 2024-05-30T15:52:55.289 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libsisu-plexus-java (0.3.4-3) ... 2024-05-30T15:52:55.299 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../20-libmaven-shared-utils-java_3.3.0-1ubuntu0.22.04.1_all.deb ... 2024-05-30T15:52:55.337 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libmaven-shared-utils-java (3.3.0-1ubuntu0.22.04.1) ... 2024-05-30T15:52:55.543 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libmaven3-core-java. 2024-05-30T15:52:55.560 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../29-libmaven3-core-java_3.6.3-5_all.deb ... 2024-05-30T15:52:55.592 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libmaven3-core-java (3.6.3-5) ... 2024-05-30T15:52:55.598 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libplexus-cipher-java. 2024-05-30T15:52:55.626 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../21-libplexus-cipher-java_1.8-2_all.deb ... 2024-05-30T15:52:55.666 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libplexus-cipher-java (1.8-2) ... 2024-05-30T15:52:55.893 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libplexus-classworlds-java. 2024-05-30T15:52:55.909 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../22-libplexus-classworlds-java_2.6.0-1_all.deb ... 2024-05-30T15:52:55.942 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libplexus-classworlds-java (2.6.0-1) ... 2024-05-30T15:52:55.964 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libwagon-file-java. 2024-05-30T15:52:55.992 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../30-libwagon-file-java_3.3.4-1_all.deb ... 2024-05-30T15:52:56.030 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libwagon-file-java (3.3.4-1) ... 2024-05-30T15:52:56.179 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libplexus-component-annotations-java. 2024-05-30T15:52:56.206 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../23-libplexus-component-annotations-java_2.1.0-1_all.deb ... 2024-05-30T15:52:56.245 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libplexus-component-annotations-java (2.1.0-1) ... 2024-05-30T15:52:56.282 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libwagon-http-shaded-java. 2024-05-30T15:52:56.299 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../31-libwagon-http-shaded-java_3.3.4-1_all.deb ... 2024-05-30T15:52:56.341 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libwagon-http-shaded-java (3.3.4-1) ... 2024-05-30T15:52:56.523 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libplexus-interpolation-java. 2024-05-30T15:52:56.551 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../24-libplexus-interpolation-java_1.26-1_all.deb ... 2024-05-30T15:52:56.589 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libplexus-interpolation-java (1.26-1) ... 2024-05-30T15:52:56.786 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package maven. 2024-05-30T15:52:56.814 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../32-maven_3.6.3-5_all.deb ... 2024-05-30T15:52:56.850 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libplexus-sec-dispatcher-java. 2024-05-30T15:52:56.852 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking maven (3.6.3-5) ... 2024-05-30T15:52:56.866 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../25-libplexus-sec-dispatcher-java_1.4-4_all.deb ... 2024-05-30T15:52:56.900 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libplexus-sec-dispatcher-java (1.4-4) ... 2024-05-30T15:52:57.103 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libslf4j-java. 2024-05-30T15:52:57.119 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../26-libslf4j-java_1.7.32-1_all.deb ... 2024-05-30T15:52:57.203 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libslf4j-java (1.7.32-1) ... 2024-05-30T15:52:57.223 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package python3-magic. 2024-05-30T15:52:57.252 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../33-python3-magic_2%3a0.4.24-2_all.deb ... 2024-05-30T15:52:57.289 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking python3-magic (2:0.4.24-2) ... 2024-05-30T15:52:57.465 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libsisu-inject-java. 2024-05-30T15:52:57.482 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../27-libsisu-inject-java_0.3.4-2_all.deb ... 2024-05-30T15:52:57.540 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libsisu-inject-java (0.3.4-2) ... 2024-05-30T15:52:57.592 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package s3cmd. 2024-05-30T15:52:57.621 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../34-s3cmd_2.2.0-1_all.deb ... 2024-05-30T15:52:57.659 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking s3cmd (2.2.0-1) ... 2024-05-30T15:52:57.835 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libsisu-plexus-java. 2024-05-30T15:52:57.863 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../28-libsisu-plexus-java_0.3.4-3_all.deb ... 2024-05-30T15:52:57.901 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libsisu-plexus-java (0.3.4-3) ... 2024-05-30T15:52:58.047 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libslf4j-java (1.7.32-1) ... 2024-05-30T15:52:58.163 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libmaven3-core-java. 2024-05-30T15:52:58.173 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libplexus-utils2-java (3.3.0-1) ... 2024-05-30T15:52:58.179 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../29-libmaven3-core-java_3.6.3-5_all.deb ... 2024-05-30T15:52:58.213 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libmaven3-core-java (3.6.3-5) ... 2024-05-30T15:52:58.298 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libplexus-classworlds-java (2.6.0-1) ... 2024-05-30T15:52:58.424 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libjsr305-java (0.1~+svn49-11) ... 2024-05-30T15:52:58.576 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libwagon-file-java. 2024-05-30T15:52:58.584 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libaopalliance-java (20070526-6) ... 2024-05-30T15:52:58.603 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../30-libwagon-file-java_3.3.4-1_all.deb ... 2024-05-30T15:52:58.642 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libwagon-file-java (3.3.4-1) ... 2024-05-30T15:52:58.701 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libcommons-cli-java (1.4-2) ... 2024-05-30T15:52:58.802 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libplexus-component-annotations-java (2.1.0-1) ... 2024-05-30T15:52:58.895 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libwagon-http-shaded-java. 2024-05-30T15:52:58.903 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libplexus-cipher-java (1.8-2) ... 2024-05-30T15:52:58.923 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../31-libwagon-http-shaded-java_3.3.4-1_all.deb ... 2024-05-30T15:52:58.961 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libwagon-http-shaded-java (3.3.4-1) ... 2024-05-30T15:52:59.030 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libgeronimo-annotation-1.3-spec-java (1.3-1) ... 2024-05-30T15:52:59.156 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libgeronimo-interceptor-3.0-spec-java (1.0.1-4fakesync) ... 2024-05-30T15:52:59.281 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libapache-pom-java (18-1) ... 2024-05-30T15:52:59.407 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libatinject-jsr330-api-java (1.0+ds1-5) ... 2024-05-30T15:52:59.431 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package maven. 2024-05-30T15:52:59.459 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../32-maven_3.6.3-5_all.deb ... 2024-05-30T15:52:59.498 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking maven (3.6.3-5) ... 2024-05-30T15:52:59.559 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libplexus-interpolation-java (1.26-1) ... 2024-05-30T15:52:59.685 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libplexus-sec-dispatcher-java (1.4-4) ... 2024-05-30T15:52:59.759 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-magic. 2024-05-30T15:52:59.775 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../33-python3-magic_2%3a0.4.24-2_all.deb ... 2024-05-30T15:52:59.794 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libwagon-http-shaded-java (3.3.4-1) ... 2024-05-30T15:52:59.818 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-magic (2:0.4.24-2) ... 2024-05-30T15:52:59.895 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up python3-magic (2:0.4.24-2) ... 2024-05-30T15:53:00.079 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package s3cmd. 2024-05-30T15:53:00.107 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../34-s3cmd_2.2.0-1_all.deb ... 2024-05-30T15:53:00.137 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking s3cmd (2.2.0-1) ... 2024-05-30T15:53:00.138 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libcdi-api-java (1.2-3) ... 2024-05-30T15:53:00.264 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libhawtjni-runtime-java (1.17-1) ... 2024-05-30T15:53:00.390 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libwagon-provider-api-java (3.3.4-1) ... 2024-05-30T15:53:00.516 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libmaven-parent-java (31-2) ... 2024-05-30T15:53:00.524 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libslf4j-java (1.7.32-1) ... 2024-05-30T15:53:00.650 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libcommons-parent-java (43-1) ... 2024-05-30T15:53:00.651 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libplexus-utils2-java (3.3.0-1) ... 2024-05-30T15:53:00.776 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libsisu-inject-java (0.3.4-2) ... 2024-05-30T15:53:00.777 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libplexus-classworlds-java (2.6.0-1) ... 2024-05-30T15:53:00.894 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up s3cmd (2.2.0-1) ... 2024-05-30T15:53:00.903 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libjsr305-java (0.1~+svn49-11) ... 2024-05-30T15:53:01.029 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libaopalliance-java (20070526-6) ... 2024-05-30T15:53:01.138 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libcommons-cli-java (1.4-2) ... 2024-05-30T15:53:01.187 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libsisu-plexus-java (0.3.4-3) ... 2024-05-30T15:53:01.239 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libplexus-component-annotations-java (2.1.0-1) ... 2024-05-30T15:53:01.313 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libmaven-resolver-java (1.4.2-3build1) ... 2024-05-30T15:53:01.340 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libplexus-cipher-java (1.8-2) ... 2024-05-30T15:53:01.439 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libguava-java (29.0-6) ... 2024-05-30T15:53:01.466 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libgeronimo-annotation-1.3-spec-java (1.3-1) ... 2024-05-30T15:53:01.573 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libcommons-lang3-java (3.11-1) ... 2024-05-30T15:53:01.592 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libgeronimo-interceptor-3.0-spec-java (1.0.1-4fakesync) ... 2024-05-30T15:53:01.708 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libjansi-native-java (1.8-1) ... 2024-05-30T15:53:01.718 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libapache-pom-java (18-1) ... 2024-05-30T15:53:01.809 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libwagon-file-java (3.3.4-1) ... 2024-05-30T15:53:01.844 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libatinject-jsr330-api-java (1.0+ds1-5) ... 2024-05-30T15:53:01.910 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libcommons-io-java (2.11.0-2) ... 2024-05-30T15:53:01.970 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libplexus-interpolation-java (1.26-1) ... 2024-05-30T15:53:02.019 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libguice-java (4.2.3-2) ... 2024-05-30T15:53:02.105 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libplexus-sec-dispatcher-java (1.4-4) ... 2024-05-30T15:53:02.206 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libwagon-http-shaded-java (3.3.4-1) ... 2024-05-30T15:53:02.305 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libjansi-java (1.18-1) ... 2024-05-30T15:53:02.431 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libmaven-shared-utils-java (3.3.0-1ubuntu0.22.04.1) ... 2024-05-30T15:53:02.557 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libmaven3-core-java (3.6.3-5) ... 2024-05-30T15:53:02.666 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up maven (3.6.3-5) ... 2024-05-30T15:53:02.732 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-magic (2:0.4.24-2) ... 2024-05-30T15:53:02.883 INFO:teuthology.orchestra.run.smithi006.stdout:update-alternatives: using /usr/share/maven/bin/mvn to provide /usr/bin/mvn (mvn) in auto mode 2024-05-30T15:53:02.954 INFO:teuthology.orchestra.run.smithi006.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-30T15:53:03.035 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libcdi-api-java (1.2-3) ... 2024-05-30T15:53:03.161 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libhawtjni-runtime-java (1.17-1) ... 2024-05-30T15:53:03.286 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libwagon-provider-api-java (3.3.4-1) ... 2024-05-30T15:53:03.412 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libmaven-parent-java (31-2) ... 2024-05-30T15:53:03.539 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libcommons-parent-java (43-1) ... 2024-05-30T15:53:03.640 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libsisu-inject-java (0.3.4-2) ... 2024-05-30T15:53:03.741 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up s3cmd (2.2.0-1) ... 2024-05-30T15:53:04.026 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libsisu-plexus-java (0.3.4-3) ... 2024-05-30T15:53:04.152 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libmaven-resolver-java (1.4.2-3build1) ... 2024-05-30T15:53:04.261 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libguava-java (29.0-6) ... 2024-05-30T15:53:04.359 INFO:teuthology.orchestra.run.smithi006.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-30T15:53:04.365 DEBUG:teuthology.parallel:result is None 2024-05-30T15:53:04.387 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libcommons-lang3-java (3.11-1) ... 2024-05-30T15:53:04.521 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libjansi-native-java (1.8-1) ... 2024-05-30T15:53:04.631 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libwagon-file-java (3.3.4-1) ... 2024-05-30T15:53:04.732 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libcommons-io-java (2.11.0-2) ... 2024-05-30T15:53:04.833 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libguice-java (4.2.3-2) ... 2024-05-30T15:53:04.959 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libjansi-java (1.18-1) ... 2024-05-30T15:53:05.085 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libmaven-shared-utils-java (3.3.0-1ubuntu0.22.04.1) ... 2024-05-30T15:53:05.228 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libmaven3-core-java (3.6.3-5) ... 2024-05-30T15:53:05.353 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up maven (3.6.3-5) ... 2024-05-30T15:53:05.610 INFO:teuthology.orchestra.run.smithi204.stdout:update-alternatives: using /usr/share/maven/bin/mvn to provide /usr/bin/mvn (mvn) in auto mode 2024-05-30T15:53:05.658 INFO:teuthology.orchestra.run.smithi204.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-30T15:53:07.073 INFO:teuthology.orchestra.run.smithi204.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-30T15:53:07.077 DEBUG:teuthology.parallel:result is None 2024-05-30T15:53:07.078 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 2024-05-30T15:53:07.219 DEBUG:teuthology.orchestra.run.smithi006:> dpkg-query -W -f '${Version}' ceph 2024-05-30T15:53:07.238 INFO:teuthology.orchestra.run.smithi006.stdout:19.0.0-4011-gda98024b-1jammy 2024-05-30T15:53:07.239 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-4011-gda98024b-1jammy 2024-05-30T15:53:07.239 INFO:teuthology.task.install:The correct ceph version 19.0.0-4011-gda98024b-1jammy is installed. 2024-05-30T15:53:07.241 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 2024-05-30T15:53:07.375 DEBUG:teuthology.orchestra.run.smithi204:> dpkg-query -W -f '${Version}' ceph 2024-05-30T15:53:07.394 INFO:teuthology.orchestra.run.smithi204.stdout:19.0.0-4011-gda98024b-1jammy 2024-05-30T15:53:07.395 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-4011-gda98024b-1jammy 2024-05-30T15:53:07.395 INFO:teuthology.task.install:The correct ceph version 19.0.0-4011-gda98024b-1jammy is installed. 2024-05-30T15:53:07.397 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-30T15:53:07.397 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T15:53:07.397 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-30T15:53:07.411 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-30T15:53:07.411 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-30T15:53:07.449 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-30T15:53:07.449 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T15:53:07.449 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/daemon-helper 2024-05-30T15:53:07.466 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-30T15:53:07.521 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-30T15:53:07.521 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/daemon-helper 2024-05-30T15:53:07.534 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-30T15:53:07.587 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-30T15:53:07.587 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T15:53:07.587 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-30T15:53:07.601 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-30T15:53:07.657 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-30T15:53:07.657 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-30T15:53:07.671 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-30T15:53:07.727 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-30T15:53:07.727 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T15:53:07.727 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/stdin-killer 2024-05-30T15:53:07.741 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-30T15:53:07.798 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-30T15:53:07.798 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/stdin-killer 2024-05-30T15:53:07.811 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-30T15:53:07.866 INFO:teuthology.run_tasks:Running task ceph... 2024-05-30T15:53:07.942 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-30T15:53:07.943 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 777 /var/log/ceph 2024-05-30T15:53:07.945 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 777 /var/log/ceph 2024-05-30T15:53:07.959 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-30T15:53:07.959 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-30T15:53:07.998 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-30T15:53:08.011 INFO:tasks.ceph:Creating extra log directories... 2024-05-30T15:53:08.011 DEBUG:teuthology.orchestra.run.smithi006:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-30T15:53:08.055 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-30T15:53:08.078 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-30T15:53:08.078 INFO:tasks.ceph:config {'conf': {'client': {'debug rgw': 20, 'debug rgw notification': 20, 'rgw bucket counters cache': True, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw d3n l1 datacache persistent path': '/tmp/rgw_datacache/', 'rgw d3n l1 datacache size': 10737418240, 'rgw d3n l1 local datacache enabled': True, 'rgw enable ops log': True, 'rgw max chunk size': 4194304, 'rgw obj stripe size': 6291456, 'rgw s3 auth use sts': True, 'rgw sts key': 'abcdefghijklmnop', 'rgw torrent flag': True, 'rgw user counters cache': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000, 'osd heartbeat grace': 40, 'osd_max_pg_log_entries': 10, 'osd_min_pg_log_entries': 10}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd crush smoke test': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled', '\\(OSD_SLOW_PING_TIME'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-30T15:53:08.078 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/cbodley-2024-05-30_15:09:56-rgw-wip-cbodley-testing-distro-default-smithi/7734120', 'branch': 'wip-cbodley-testing', 'description': 'rgw/verify/{0-install accounts$/{main-tenant} clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}', 'email': None, 'first_in_suite': False, 'job_id': '7734120', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'cbodley-2024-05-30_15:09:56-rgw-wip-cbodley-testing-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-cbodley-testing'}, 'ceph': {'conf': {'client': {'debug rgw': 20, 'debug rgw notification': 20, 'rgw bucket counters cache': True, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw d3n l1 datacache persistent path': '/tmp/rgw_datacache/', 'rgw d3n l1 datacache size': 10737418240, 'rgw d3n l1 local datacache enabled': True, 'rgw enable ops log': True, 'rgw max chunk size': 4194304, 'rgw obj stripe size': 6291456, 'rgw s3 auth use sts': True, 'rgw sts key': 'abcdefghijklmnop', 'rgw torrent flag': True, 'rgw user counters cache': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000, 'osd heartbeat grace': 40, 'osd_max_pg_log_entries': 10, 'osd_min_pg_log_entries': 10}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd crush smoke test': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled', '\\(OSD_SLOW_PING_TIME'], 'sha1': 'da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2'}}, 'rgw': {'client.0': {'valgrind': ['--tool=memcheck', '--max-threads=1024']}, 'compression type': 'random', 'datacache': True, 'datacache_path': '/tmp/rgw_datacache', 'ec-data-pool': True, 'frontend': 'beast', 'storage classes': 'LUKEWARM, FROZEN'}, 's3tests': {'accounts': {'iam alt root': 'RGW99999999999999999', 'iam root': 'RGW88888888888888888', 's3 main': 'RGW11111111111111111', 's3 tenant': 'RGW33333333333333333'}, 'force-branch': 'ceph-master', 'slow_backend': True, 'storage classes': 'LUKEWARM, FROZEN'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-cbodley-testing', 'sha1': 'da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2'}}, 'owner': 'scheduled_cbodley@teuthology', 'priority': 80, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 5740, 'sha1': 'da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2', 'sleep_before_teardown': 0, 'suite': 'rgw', 'suite_branch': 'wip-cbodley-testing', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2', 'targets': {'smithi006.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDR4d1KWa9GgYfcJ1JEAH57mEqxzqJIgppsOibV5cfLNbiYeju7wgfxNRm48qMwhqyJPw6XxF1ttJs/fHDVfprA=', 'smithi204.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNpvmHyUurLgakidLXapfD9qXq/0T3au9KSzbm3sB6iRmJXVzk4rKyJJhMLRMArWvT9zkezUQnALjIQNsGkPe38='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_system_packages': {'deb': ['s3cmd', 'maven'], 'rpm': ['s3cmd', 'maven']}, 'flavor': 'default', 'sha1': 'da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2'}}, {'ceph': None}, {'openssl_keys': None}, {'rgw': {'client.0': None}}, {'tox': ['client.0']}, {'workunit': {'clients': {'client.0': ['rgw/run-datacache.sh']}, 'env': {'RGW_DATACACHE_PATH': '/tmp/rgw_datacache'}}}, {'workunit': {'clients': {'client.0': ['rgw/run-bucket-check.sh']}}}, {'workunit': {'clients': {'client.0': ['cls/test_cls_lock.sh', 'cls/test_cls_log.sh', 'cls/test_cls_refcount.sh', 'cls/test_cls_rgw.sh', 'cls/test_cls_rgw_gc.sh', 'cls/test_cls_rgw_stats.sh', 'cls/test_cls_cmpomap.sh', 'cls/test_cls_2pc_queue.sh', 'cls/test_cls_user.sh', 'rgw/test_rgw_gc_log.sh', 'rgw/test_rgw_obj.sh', 'rgw/test_librgw_file.sh', 'rgw/test_awssdkv4_sig.sh']}}}, {'workunit': {'clients': {'client.0': ['rgw/test_rgw_s3_mp_reupload.sh']}}}, {'ragweed': {'client.0': {'default-branch': 'ceph-master', 'rgw_server': 'client.0', 'stages': 'prepare,check'}}}, {'workunit': {'clients': {'client.0': ['rgw/run-reshard.sh']}}}, {'s3tests-java': {'client.0': {'force-branch': 'ceph-master', 'force-repo': 'https://github.com/ceph/java_s3tests.git'}}}, {'s3tests': {'client.0': {'rgw_server': 'client.0'}}}, {'workunit': {'clients': {'client.0': ['rgw/run-versioning.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e1979f064b0becde2020282c7e87cf3c5e98026c', 'timestamp': '2024-05-30_15:09:56', 'tube': 'smithi', 'user': 'cbodley', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841'} 2024-05-30T15:53:08.079 DEBUG:teuthology.orchestra.run.smithi006:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-30T15:53:08.110 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-30T15:53:08.116 DEBUG:teuthology.orchestra.run.smithi006:> sudo install -d -m0777 -- /var/run/ceph 2024-05-30T15:53:08.159 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m0777 -- /var/run/ceph 2024-05-30T15:53:08.171 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T15:53:08.172 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/scratch_devs of=/dev/stdout 2024-05-30T15:53:08.218 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-30T15:53:08.218 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_1 2024-05-30T15:53:08.266 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-30T15:53:08.266 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-30T15:53:08.266 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 882 Links: 1 2024-05-30T15:53:08.266 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-30T15:53:08.267 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2024-05-30 15:47:31.707371598 +0000 2024-05-30T15:53:08.267 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-05-30 15:47:31.435374027 +0000 2024-05-30T15:53:08.267 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-05-30 15:47:31.435374027 +0000 2024-05-30T15:53:08.267 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: - 2024-05-30T15:53:08.267 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-30T15:53:08.320 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2024-05-30T15:53:08.320 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2024-05-30T15:53:08.321 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000446713 s, 1.1 MB/s 2024-05-30T15:53:08.322 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-30T15:53:08.371 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_2 2024-05-30T15:53:08.418 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-30T15:53:08.418 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-30T15:53:08.418 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 892 Links: 1 2024-05-30T15:53:08.418 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-30T15:53:08.418 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2024-05-30 15:47:32.175367417 +0000 2024-05-30T15:53:08.418 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-05-30 15:47:31.911369776 +0000 2024-05-30T15:53:08.418 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-05-30 15:47:31.911369776 +0000 2024-05-30T15:53:08.418 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: - 2024-05-30T15:53:08.419 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-30T15:53:08.472 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2024-05-30T15:53:08.473 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2024-05-30T15:53:08.473 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000417093 s, 1.2 MB/s 2024-05-30T15:53:08.474 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-30T15:53:08.523 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_3 2024-05-30T15:53:08.570 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-30T15:53:08.570 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-30T15:53:08.570 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 909 Links: 1 2024-05-30T15:53:08.570 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-30T15:53:08.570 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2024-05-30 15:47:32.695362773 +0000 2024-05-30T15:53:08.570 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-05-30 15:47:32.435365095 +0000 2024-05-30T15:53:08.570 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-05-30 15:47:32.435365095 +0000 2024-05-30T15:53:08.571 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: - 2024-05-30T15:53:08.571 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-30T15:53:08.625 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2024-05-30T15:53:08.625 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2024-05-30T15:53:08.625 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000440783 s, 1.2 MB/s 2024-05-30T15:53:08.626 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-30T15:53:08.675 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_4 2024-05-30T15:53:08.722 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-30T15:53:08.722 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-30T15:53:08.722 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 934 Links: 1 2024-05-30T15:53:08.722 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-30T15:53:08.722 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2024-05-30 15:47:33.215358127 +0000 2024-05-30T15:53:08.722 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-05-30 15:47:32.935360629 +0000 2024-05-30T15:53:08.722 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-05-30 15:47:32.935360629 +0000 2024-05-30T15:53:08.722 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: - 2024-05-30T15:53:08.723 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-30T15:53:08.776 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2024-05-30T15:53:08.776 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2024-05-30T15:53:08.776 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000321867 s, 1.6 MB/s 2024-05-30T15:53:08.778 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-30T15:53:08.827 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-30T15:53:08.827 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-30T15:53:08.827 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/scratch_devs of=/dev/stdout 2024-05-30T15:53:08.833 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-30T15:53:08.833 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_1 2024-05-30T15:53:08.880 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-30T15:53:08.880 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-30T15:53:08.880 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 871 Links: 1 2024-05-30T15:53:08.880 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-30T15:53:08.880 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-05-30 15:47:29.017821342 +0000 2024-05-30T15:53:08.881 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-05-30 15:47:28.753823698 +0000 2024-05-30T15:53:08.881 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-05-30 15:47:28.753823698 +0000 2024-05-30T15:53:08.881 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: - 2024-05-30T15:53:08.881 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-30T15:53:08.934 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-05-30T15:53:08.934 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-05-30T15:53:08.935 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000446111 s, 1.1 MB/s 2024-05-30T15:53:08.936 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-30T15:53:08.985 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_2 2024-05-30T15:53:09.032 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-30T15:53:09.032 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-30T15:53:09.032 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 891 Links: 1 2024-05-30T15:53:09.032 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-30T15:53:09.032 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-05-30 15:47:29.517816882 +0000 2024-05-30T15:53:09.032 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-05-30 15:47:29.261819166 +0000 2024-05-30T15:53:09.032 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-05-30 15:47:29.261819166 +0000 2024-05-30T15:53:09.032 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: - 2024-05-30T15:53:09.033 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-30T15:53:09.086 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-05-30T15:53:09.086 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-05-30T15:53:09.086 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000376789 s, 1.4 MB/s 2024-05-30T15:53:09.087 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-30T15:53:09.137 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_3 2024-05-30T15:53:09.184 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-30T15:53:09.184 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-30T15:53:09.184 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 907 Links: 1 2024-05-30T15:53:09.184 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-30T15:53:09.184 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-05-30 15:47:30.013812457 +0000 2024-05-30T15:53:09.184 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-05-30 15:47:29.737814919 +0000 2024-05-30T15:53:09.184 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-05-30 15:47:29.737814919 +0000 2024-05-30T15:53:09.184 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: - 2024-05-30T15:53:09.185 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-30T15:53:09.238 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-05-30T15:53:09.238 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-05-30T15:53:09.238 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000332487 s, 1.5 MB/s 2024-05-30T15:53:09.239 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-30T15:53:09.289 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_4 2024-05-30T15:53:09.335 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-30T15:53:09.336 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-30T15:53:09.336 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 925 Links: 1 2024-05-30T15:53:09.336 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-30T15:53:09.336 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-05-30 15:47:30.481808282 +0000 2024-05-30T15:53:09.336 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-05-30 15:47:30.253810317 +0000 2024-05-30T15:53:09.337 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-05-30 15:47:30.253810317 +0000 2024-05-30T15:53:09.337 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: - 2024-05-30T15:53:09.337 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-30T15:53:09.390 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-05-30T15:53:09.390 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-05-30T15:53:09.390 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.00036136 s, 1.4 MB/s 2024-05-30T15:53:09.391 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-30T15:53:09.441 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-30T15:53:09.441 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi006.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi204.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-30T15:53:09.442 INFO:tasks.ceph:Generating config... 2024-05-30T15:53:09.443 INFO:tasks.ceph:[client] debug rgw = 20 2024-05-30T15:53:09.443 INFO:tasks.ceph:[client] debug rgw notification = 20 2024-05-30T15:53:09.443 INFO:tasks.ceph:[client] rgw bucket counters cache = True 2024-05-30T15:53:09.443 INFO:tasks.ceph:[client] rgw crypt require ssl = False 2024-05-30T15:53:09.443 INFO:tasks.ceph:[client] rgw crypt s3 kms backend = testing 2024-05-30T15:53:09.443 INFO:tasks.ceph:[client] rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2024-05-30T15:53:09.443 INFO:tasks.ceph:[client] rgw d3n l1 datacache persistent path = /tmp/rgw_datacache/ 2024-05-30T15:53:09.443 INFO:tasks.ceph:[client] rgw d3n l1 datacache size = 10737418240 2024-05-30T15:53:09.443 INFO:tasks.ceph:[client] rgw d3n l1 local datacache enabled = True 2024-05-30T15:53:09.444 INFO:tasks.ceph:[client] rgw enable ops log = True 2024-05-30T15:53:09.444 INFO:tasks.ceph:[client] rgw max chunk size = 4194304 2024-05-30T15:53:09.444 INFO:tasks.ceph:[client] rgw obj stripe size = 6291456 2024-05-30T15:53:09.444 INFO:tasks.ceph:[client] rgw s3 auth use sts = True 2024-05-30T15:53:09.444 INFO:tasks.ceph:[client] rgw sts key = abcdefghijklmnop 2024-05-30T15:53:09.444 INFO:tasks.ceph:[client] rgw torrent flag = True 2024-05-30T15:53:09.444 INFO:tasks.ceph:[client] rgw user counters cache = True 2024-05-30T15:53:09.444 INFO:tasks.ceph:[client] setgroup = ceph 2024-05-30T15:53:09.444 INFO:tasks.ceph:[client] setuser = ceph 2024-05-30T15:53:09.444 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-05-30T15:53:09.444 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-05-30T15:53:09.444 INFO:tasks.ceph:[global] osd heartbeat grace = 40 2024-05-30T15:53:09.444 INFO:tasks.ceph:[global] osd_max_pg_log_entries = 10 2024-05-30T15:53:09.445 INFO:tasks.ceph:[global] osd_min_pg_log_entries = 10 2024-05-30T15:53:09.445 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-30T15:53:09.445 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-30T15:53:09.445 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-30T15:53:09.445 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-30T15:53:09.445 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-30T15:53:09.445 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2024-05-30T15:53:09.445 INFO:tasks.ceph:[osd] bdev async discard = True 2024-05-30T15:53:09.445 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-05-30T15:53:09.445 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-05-30T15:53:09.445 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-30T15:53:09.445 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-30T15:53:09.445 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-05-30T15:53:09.445 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-05-30T15:53:09.446 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-30T15:53:09.446 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-30T15:53:09.446 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-05-30T15:53:09.446 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-30T15:53:09.446 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-30T15:53:09.446 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-30T15:53:09.446 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-30T15:53:09.446 INFO:tasks.ceph:[osd] osd fast shutdown = False 2024-05-30T15:53:09.446 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-30T15:53:09.446 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-05-30T15:53:09.446 INFO:tasks.ceph:Setting up mon.a... 2024-05-30T15:53:09.447 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-30T15:53:09.482 INFO:teuthology.orchestra.run.smithi006.stdout:creating /etc/ceph/ceph.keyring 2024-05-30T15:53:09.498 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-30T15:53:09.579 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-30T15:53:09.633 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.6'), ('mon.c', '[v2:172.21.15.6:3301,v1:172.21.15.6:6790]'), ('mon.b', '172.21.15.204')] 2024-05-30T15:53:09.633 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.6,[v2:172.21.15.6:3301,v1:172.21.15.6:6790],172.21.15.204', 'mon client directed command retry': 5, 'ms inject socket failures': 5000, 'osd heartbeat grace': 40, 'osd_max_pg_log_entries': 10, 'osd_min_pg_log_entries': 10}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon osd crush smoke test': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': True, 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug rgw': 20, 'debug rgw notification': 20, 'rgw bucket counters cache': True, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw d3n l1 datacache persistent path': '/tmp/rgw_datacache/', 'rgw d3n l1 datacache size': 10737418240, 'rgw d3n l1 local datacache enabled': True, 'rgw max chunk size': 4194304, 'rgw obj stripe size': 6291456, 'rgw s3 auth use sts': True, 'rgw sts key': 'abcdefghijklmnop', 'rgw torrent flag': True, 'rgw user counters cache': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-05-30T15:53:09.634 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T15:53:09.634 DEBUG:teuthology.orchestra.run.smithi006:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-30T15:53:09.682 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.6 --addv c '[v2:172.21.15.6:3301,v1:172.21.15.6:6790]' --add b 172.21.15.204 --print /home/ubuntu/cephtest/ceph.monmap 2024-05-30T15:53:09.769 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:53:09.769 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:53:09.769 INFO:teuthology.orchestra.run.smithi006.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-30T15:53:09.770 INFO:teuthology.orchestra.run.smithi006.stdout:monmaptool: generated fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T15:53:09.770 INFO:teuthology.orchestra.run.smithi006.stdout:setting min_mon_release = quincy 2024-05-30T15:53:09.770 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 0 2024-05-30T15:53:09.770 INFO:teuthology.orchestra.run.smithi006.stdout:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T15:53:09.770 INFO:teuthology.orchestra.run.smithi006.stdout:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T15:53:09.770 INFO:teuthology.orchestra.run.smithi006.stdout:created 2024-05-30T15:53:09.772574+0000 2024-05-30T15:53:09.770 INFO:teuthology.orchestra.run.smithi006.stdout:min_mon_release 17 (quincy) 2024-05-30T15:53:09.770 INFO:teuthology.orchestra.run.smithi006.stdout:election_strategy: 1 2024-05-30T15:53:09.770 INFO:teuthology.orchestra.run.smithi006.stdout:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T15:53:09.770 INFO:teuthology.orchestra.run.smithi006.stdout:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T15:53:09.770 INFO:teuthology.orchestra.run.smithi006.stdout:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T15:53:09.770 INFO:teuthology.orchestra.run.smithi006.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-30T15:53:09.771 DEBUG:teuthology.orchestra.run.smithi006:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-30T15:53:09.818 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 87f2315a-4430-477b-ae32-1664164ce9e8... 2024-05-30T15:53:09.819 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-30T15:53:09.863 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-30T15:53:09.892 INFO:teuthology.orchestra.run.smithi006.stdout:[global] 2024-05-30T15:53:09.892 INFO:teuthology.orchestra.run.smithi006.stdout: chdir = "" 2024-05-30T15:53:09.892 INFO:teuthology.orchestra.run.smithi006.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-30T15:53:09.892 INFO:teuthology.orchestra.run.smithi006.stdout: auth supported = cephx 2024-05-30T15:53:09.892 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:53:09.893 INFO:teuthology.orchestra.run.smithi006.stdout: filestore xattr use omap = true 2024-05-30T15:53:09.893 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:53:09.893 INFO:teuthology.orchestra.run.smithi006.stdout: mon clock drift allowed = 1.000 2024-05-30T15:53:09.893 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:53:09.893 INFO:teuthology.orchestra.run.smithi006.stdout: osd crush chooseleaf type = 0 2024-05-30T15:53:09.893 INFO:teuthology.orchestra.run.smithi006.stdout: auth debug = true 2024-05-30T15:53:09.893 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:53:09.893 INFO:teuthology.orchestra.run.smithi006.stdout: ms die on old message = true 2024-05-30T15:53:09.893 INFO:teuthology.orchestra.run.smithi006.stdout: ms die on bug = true 2024-05-30T15:53:09.893 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:53:09.893 INFO:teuthology.orchestra.run.smithi006.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-30T15:53:09.893 INFO:teuthology.orchestra.run.smithi006.stdout: mon pg warn max object skew = 0 2024-05-30T15:53:09.894 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:53:09.894 INFO:teuthology.orchestra.run.smithi006.stdout: # disable pg_autoscaler by default for new pools 2024-05-30T15:53:09.894 INFO:teuthology.orchestra.run.smithi006.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-30T15:53:09.894 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:53:09.894 INFO:teuthology.orchestra.run.smithi006.stdout: osd pool default size = 2 2024-05-30T15:53:09.894 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:53:09.894 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd allow primary affinity = true 2024-05-30T15:53:09.894 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd allow pg remap = true 2024-05-30T15:53:09.894 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on legacy crush tunables = false 2024-05-30T15:53:09.894 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on crush straw calc version zero = false 2024-05-30T15:53:09.894 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on no sortbitwise = false 2024-05-30T15:53:09.894 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on osd down out interval zero = false 2024-05-30T15:53:09.895 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on too few osds = false 2024-05-30T15:53:09.895 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-30T15:53:09.895 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-30T15:53:09.895 INFO:teuthology.orchestra.run.smithi006.stdout: mon_allow_pool_size_one = true 2024-05-30T15:53:09.895 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:53:09.895 INFO:teuthology.orchestra.run.smithi006.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-30T15:53:09.895 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:53:09.895 INFO:teuthology.orchestra.run.smithi006.stdout: osd default data pool replay window = 5 2024-05-30T15:53:09.895 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:53:09.895 INFO:teuthology.orchestra.run.smithi006.stdout: mon allow pool delete = true 2024-05-30T15:53:09.895 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:53:09.895 INFO:teuthology.orchestra.run.smithi006.stdout: mon cluster log file level = debug 2024-05-30T15:53:09.895 INFO:teuthology.orchestra.run.smithi006.stdout: debug asserts on shutdown = true 2024-05-30T15:53:09.896 INFO:teuthology.orchestra.run.smithi006.stdout: mon health detail to clog = false 2024-05-30T15:53:09.896 INFO:teuthology.orchestra.run.smithi006.stdout: mon host = "172.21.15.6,[v2:172.21.15.6:3301,v1:172.21.15.6:6790],172.21.15.204" 2024-05-30T15:53:09.896 INFO:teuthology.orchestra.run.smithi006.stdout: mon client directed command retry = 5 2024-05-30T15:53:09.896 INFO:teuthology.orchestra.run.smithi006.stdout: ms inject socket failures = 5000 2024-05-30T15:53:09.896 INFO:teuthology.orchestra.run.smithi006.stdout: osd heartbeat grace = 40 2024-05-30T15:53:09.896 INFO:teuthology.orchestra.run.smithi006.stdout: osd_max_pg_log_entries = 10 2024-05-30T15:53:09.896 INFO:teuthology.orchestra.run.smithi006.stdout: osd_min_pg_log_entries = 10 2024-05-30T15:53:09.896 INFO:teuthology.orchestra.run.smithi006.stdout: fsid = 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T15:53:09.896 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:53:09.896 INFO:teuthology.orchestra.run.smithi006.stdout:[osd] 2024-05-30T15:53:09.896 INFO:teuthology.orchestra.run.smithi006.stdout: osd journal size = 100 2024-05-30T15:53:09.896 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:53:09.896 INFO:teuthology.orchestra.run.smithi006.stdout: osd scrub load threshold = 5.0 2024-05-30T15:53:09.896 INFO:teuthology.orchestra.run.smithi006.stdout: osd scrub max interval = 600 2024-05-30T15:53:09.896 INFO:teuthology.orchestra.run.smithi006.stdout: osd mclock profile = high_recovery_ops 2024-05-30T15:53:09.897 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:53:09.897 INFO:teuthology.orchestra.run.smithi006.stdout: osd recover clone overlap = true 2024-05-30T15:53:09.897 INFO:teuthology.orchestra.run.smithi006.stdout: osd recovery max chunk = 1048576 2024-05-30T15:53:09.897 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:53:09.897 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug shutdown = true 2024-05-30T15:53:09.897 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug op order = true 2024-05-30T15:53:09.897 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug verify stray on activate = true 2024-05-30T15:53:09.897 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug trim objects = true 2024-05-30T15:53:09.897 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:53:09.897 INFO:teuthology.orchestra.run.smithi006.stdout: osd open classes on start = true 2024-05-30T15:53:09.897 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug pg log writeout = true 2024-05-30T15:53:09.897 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:53:09.897 INFO:teuthology.orchestra.run.smithi006.stdout: osd deep scrub update digest min age = 30 2024-05-30T15:53:09.897 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:53:09.898 INFO:teuthology.orchestra.run.smithi006.stdout: osd map max advance = 10 2024-05-30T15:53:09.898 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:53:09.898 INFO:teuthology.orchestra.run.smithi006.stdout: journal zero on create = true 2024-05-30T15:53:09.898 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:53:09.898 INFO:teuthology.orchestra.run.smithi006.stdout: filestore ondisk finisher threads = 3 2024-05-30T15:53:09.898 INFO:teuthology.orchestra.run.smithi006.stdout: filestore apply finisher threads = 3 2024-05-30T15:53:09.898 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:53:09.898 INFO:teuthology.orchestra.run.smithi006.stdout: bdev debug aio = true 2024-05-30T15:53:09.898 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug misdirected ops = true 2024-05-30T15:53:09.898 INFO:teuthology.orchestra.run.smithi006.stdout: bdev async discard = True 2024-05-30T15:53:09.898 INFO:teuthology.orchestra.run.smithi006.stdout: bdev enable discard = True 2024-05-30T15:53:09.898 INFO:teuthology.orchestra.run.smithi006.stdout: bluestore allocator = bitmap 2024-05-30T15:53:09.899 INFO:teuthology.orchestra.run.smithi006.stdout: bluestore block size = 96636764160 2024-05-30T15:53:09.899 INFO:teuthology.orchestra.run.smithi006.stdout: bluestore fsck on mount = True 2024-05-30T15:53:09.899 INFO:teuthology.orchestra.run.smithi006.stdout: debug bluefs = 1/20 2024-05-30T15:53:09.899 INFO:teuthology.orchestra.run.smithi006.stdout: debug bluestore = 1/20 2024-05-30T15:53:09.899 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2024-05-30T15:53:09.899 INFO:teuthology.orchestra.run.smithi006.stdout: debug osd = 20 2024-05-30T15:53:09.899 INFO:teuthology.orchestra.run.smithi006.stdout: debug rocksdb = 4/10 2024-05-30T15:53:09.899 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd backfillfull_ratio = 0.85 2024-05-30T15:53:09.899 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd full ratio = 0.9 2024-05-30T15:53:09.899 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd nearfull ratio = 0.8 2024-05-30T15:53:09.899 INFO:teuthology.orchestra.run.smithi006.stdout: osd failsafe full ratio = 0.95 2024-05-30T15:53:09.899 INFO:teuthology.orchestra.run.smithi006.stdout: osd fast shutdown = False 2024-05-30T15:53:09.899 INFO:teuthology.orchestra.run.smithi006.stdout: osd objectstore = bluestore 2024-05-30T15:53:09.900 INFO:teuthology.orchestra.run.smithi006.stdout: osd shutdown pgref assert = True 2024-05-30T15:53:09.900 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:53:09.900 INFO:teuthology.orchestra.run.smithi006.stdout:[mgr] 2024-05-30T15:53:09.900 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2024-05-30T15:53:09.900 INFO:teuthology.orchestra.run.smithi006.stdout: debug mgr = 20 2024-05-30T15:53:09.900 INFO:teuthology.orchestra.run.smithi006.stdout: debug mon = 20 2024-05-30T15:53:09.900 INFO:teuthology.orchestra.run.smithi006.stdout: debug auth = 20 2024-05-30T15:53:09.900 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min pgs per osd = 4 2024-05-30T15:53:09.900 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min bytes per osd = 10 2024-05-30T15:53:09.900 INFO:teuthology.orchestra.run.smithi006.stdout: mgr/telemetry/nag = false 2024-05-30T15:53:09.900 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:53:09.900 INFO:teuthology.orchestra.run.smithi006.stdout:[mon] 2024-05-30T15:53:09.900 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2024-05-30T15:53:09.901 INFO:teuthology.orchestra.run.smithi006.stdout: debug mon = 20 2024-05-30T15:53:09.901 INFO:teuthology.orchestra.run.smithi006.stdout: debug paxos = 20 2024-05-30T15:53:09.901 INFO:teuthology.orchestra.run.smithi006.stdout: debug auth = 20 2024-05-30T15:53:09.901 INFO:teuthology.orchestra.run.smithi006.stdout: mon data avail warn = 5 2024-05-30T15:53:09.901 INFO:teuthology.orchestra.run.smithi006.stdout: mon mgr mkfs grace = 240 2024-05-30T15:53:09.901 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min pgs per osd = 4 2024-05-30T15:53:09.901 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd reporter subtree level = osd 2024-05-30T15:53:09.901 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd prime pg temp = true 2024-05-30T15:53:09.901 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min bytes per osd = 10 2024-05-30T15:53:09.901 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:53:09.901 INFO:teuthology.orchestra.run.smithi006.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-30T15:53:09.901 INFO:teuthology.orchestra.run.smithi006.stdout: auth mon ticket ttl = 660 # 11m 2024-05-30T15:53:09.901 INFO:teuthology.orchestra.run.smithi006.stdout: auth service ticket ttl = 240 # 4m 2024-05-30T15:53:09.902 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:53:09.902 INFO:teuthology.orchestra.run.smithi006.stdout: # don't complain about insecure global_id in the test suite 2024-05-30T15:53:09.902 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-30T15:53:09.902 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-30T15:53:09.902 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:53:09.902 INFO:teuthology.orchestra.run.smithi006.stdout: # 1m isn't quite enough 2024-05-30T15:53:09.902 INFO:teuthology.orchestra.run.smithi006.stdout: mon_down_mkfs_grace = 2m 2024-05-30T15:53:09.902 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:53:09.902 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_filestore_osds = false 2024-05-30T15:53:09.902 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd crush smoke test = False 2024-05-30T15:53:09.902 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:53:09.902 INFO:teuthology.orchestra.run.smithi006.stdout:[client] 2024-05-30T15:53:09.902 INFO:teuthology.orchestra.run.smithi006.stdout: rgw cache enabled = true 2024-05-30T15:53:09.903 INFO:teuthology.orchestra.run.smithi006.stdout: rgw enable ops log = True 2024-05-30T15:53:09.903 INFO:teuthology.orchestra.run.smithi006.stdout: rgw enable usage log = true 2024-05-30T15:53:09.903 INFO:teuthology.orchestra.run.smithi006.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-30T15:53:09.903 INFO:teuthology.orchestra.run.smithi006.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-30T15:53:09.903 INFO:teuthology.orchestra.run.smithi006.stdout: debug rgw = 20 2024-05-30T15:53:09.903 INFO:teuthology.orchestra.run.smithi006.stdout: debug rgw notification = 20 2024-05-30T15:53:09.903 INFO:teuthology.orchestra.run.smithi006.stdout: rgw bucket counters cache = True 2024-05-30T15:53:09.903 INFO:teuthology.orchestra.run.smithi006.stdout: rgw crypt require ssl = False 2024-05-30T15:53:09.903 INFO:teuthology.orchestra.run.smithi006.stdout: rgw crypt s3 kms backend = testing 2024-05-30T15:53:09.904 INFO:teuthology.orchestra.run.smithi006.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2024-05-30T15:53:09.904 INFO:teuthology.orchestra.run.smithi006.stdout: rgw d3n l1 datacache persistent path = /tmp/rgw_datacache/ 2024-05-30T15:53:09.904 INFO:teuthology.orchestra.run.smithi006.stdout: rgw d3n l1 datacache size = 10737418240 2024-05-30T15:53:09.904 INFO:teuthology.orchestra.run.smithi006.stdout: rgw d3n l1 local datacache enabled = True 2024-05-30T15:53:09.904 INFO:teuthology.orchestra.run.smithi006.stdout: rgw max chunk size = 4194304 2024-05-30T15:53:09.904 INFO:teuthology.orchestra.run.smithi006.stdout: rgw obj stripe size = 6291456 2024-05-30T15:53:09.904 INFO:teuthology.orchestra.run.smithi006.stdout: rgw s3 auth use sts = True 2024-05-30T15:53:09.904 INFO:teuthology.orchestra.run.smithi006.stdout: rgw sts key = abcdefghijklmnop 2024-05-30T15:53:09.904 INFO:teuthology.orchestra.run.smithi006.stdout: rgw torrent flag = True 2024-05-30T15:53:09.904 INFO:teuthology.orchestra.run.smithi006.stdout: rgw user counters cache = True 2024-05-30T15:53:09.904 INFO:teuthology.orchestra.run.smithi006.stdout: setgroup = ceph 2024-05-30T15:53:09.904 INFO:teuthology.orchestra.run.smithi006.stdout: setuser = ceph 2024-05-30T15:53:09.905 INFO:teuthology.orchestra.run.smithi006.stdout:[mon.a] 2024-05-30T15:53:09.905 INFO:teuthology.orchestra.run.smithi006.stdout:[mon.c] 2024-05-30T15:53:09.905 INFO:teuthology.orchestra.run.smithi006.stdout:[mon.b] 2024-05-30T15:53:09.906 INFO:teuthology.orchestra.run.smithi204.stdout:[global] 2024-05-30T15:53:09.906 INFO:teuthology.orchestra.run.smithi204.stdout: chdir = "" 2024-05-30T15:53:09.906 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-30T15:53:09.907 INFO:teuthology.orchestra.run.smithi204.stdout: auth supported = cephx 2024-05-30T15:53:09.907 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-30T15:53:09.907 INFO:teuthology.orchestra.run.smithi204.stdout: filestore xattr use omap = true 2024-05-30T15:53:09.907 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-30T15:53:09.907 INFO:teuthology.orchestra.run.smithi204.stdout: mon clock drift allowed = 1.000 2024-05-30T15:53:09.907 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-30T15:53:09.907 INFO:teuthology.orchestra.run.smithi204.stdout: osd crush chooseleaf type = 0 2024-05-30T15:53:09.907 INFO:teuthology.orchestra.run.smithi204.stdout: auth debug = true 2024-05-30T15:53:09.907 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-30T15:53:09.907 INFO:teuthology.orchestra.run.smithi204.stdout: ms die on old message = true 2024-05-30T15:53:09.907 INFO:teuthology.orchestra.run.smithi204.stdout: ms die on bug = true 2024-05-30T15:53:09.907 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-30T15:53:09.907 INFO:teuthology.orchestra.run.smithi204.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-30T15:53:09.908 INFO:teuthology.orchestra.run.smithi204.stdout: mon pg warn max object skew = 0 2024-05-30T15:53:09.908 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-30T15:53:09.908 INFO:teuthology.orchestra.run.smithi204.stdout: # disable pg_autoscaler by default for new pools 2024-05-30T15:53:09.908 INFO:teuthology.orchestra.run.smithi204.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-30T15:53:09.908 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-30T15:53:09.908 INFO:teuthology.orchestra.run.smithi204.stdout: osd pool default size = 2 2024-05-30T15:53:09.908 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-30T15:53:09.908 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd allow primary affinity = true 2024-05-30T15:53:09.908 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd allow pg remap = true 2024-05-30T15:53:09.908 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on legacy crush tunables = false 2024-05-30T15:53:09.908 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on crush straw calc version zero = false 2024-05-30T15:53:09.908 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on no sortbitwise = false 2024-05-30T15:53:09.908 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on osd down out interval zero = false 2024-05-30T15:53:09.909 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on too few osds = false 2024-05-30T15:53:09.909 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-30T15:53:09.909 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-30T15:53:09.909 INFO:teuthology.orchestra.run.smithi204.stdout: mon_allow_pool_size_one = true 2024-05-30T15:53:09.909 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-30T15:53:09.909 INFO:teuthology.orchestra.run.smithi204.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-30T15:53:09.909 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-30T15:53:09.909 INFO:teuthology.orchestra.run.smithi204.stdout: osd default data pool replay window = 5 2024-05-30T15:53:09.909 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-30T15:53:09.909 INFO:teuthology.orchestra.run.smithi204.stdout: mon allow pool delete = true 2024-05-30T15:53:09.909 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-30T15:53:09.909 INFO:teuthology.orchestra.run.smithi204.stdout: mon cluster log file level = debug 2024-05-30T15:53:09.909 INFO:teuthology.orchestra.run.smithi204.stdout: debug asserts on shutdown = true 2024-05-30T15:53:09.909 INFO:teuthology.orchestra.run.smithi204.stdout: mon health detail to clog = false 2024-05-30T15:53:09.910 INFO:teuthology.orchestra.run.smithi204.stdout: mon host = "172.21.15.6,[v2:172.21.15.6:3301,v1:172.21.15.6:6790],172.21.15.204" 2024-05-30T15:53:09.910 INFO:teuthology.orchestra.run.smithi204.stdout: mon client directed command retry = 5 2024-05-30T15:53:09.910 INFO:teuthology.orchestra.run.smithi204.stdout: ms inject socket failures = 5000 2024-05-30T15:53:09.910 INFO:teuthology.orchestra.run.smithi204.stdout: osd heartbeat grace = 40 2024-05-30T15:53:09.910 INFO:teuthology.orchestra.run.smithi204.stdout: osd_max_pg_log_entries = 10 2024-05-30T15:53:09.910 INFO:teuthology.orchestra.run.smithi204.stdout: osd_min_pg_log_entries = 10 2024-05-30T15:53:09.910 INFO:teuthology.orchestra.run.smithi204.stdout: fsid = 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T15:53:09.910 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-30T15:53:09.910 INFO:teuthology.orchestra.run.smithi204.stdout:[osd] 2024-05-30T15:53:09.910 INFO:teuthology.orchestra.run.smithi204.stdout: osd journal size = 100 2024-05-30T15:53:09.910 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-30T15:53:09.910 INFO:teuthology.orchestra.run.smithi204.stdout: osd scrub load threshold = 5.0 2024-05-30T15:53:09.910 INFO:teuthology.orchestra.run.smithi204.stdout: osd scrub max interval = 600 2024-05-30T15:53:09.911 INFO:teuthology.orchestra.run.smithi204.stdout: osd mclock profile = high_recovery_ops 2024-05-30T15:53:09.911 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-30T15:53:09.911 INFO:teuthology.orchestra.run.smithi204.stdout: osd recover clone overlap = true 2024-05-30T15:53:09.911 INFO:teuthology.orchestra.run.smithi204.stdout: osd recovery max chunk = 1048576 2024-05-30T15:53:09.911 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-30T15:53:09.911 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug shutdown = true 2024-05-30T15:53:09.911 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug op order = true 2024-05-30T15:53:09.911 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug verify stray on activate = true 2024-05-30T15:53:09.911 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug trim objects = true 2024-05-30T15:53:09.911 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-30T15:53:09.911 INFO:teuthology.orchestra.run.smithi204.stdout: osd open classes on start = true 2024-05-30T15:53:09.911 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug pg log writeout = true 2024-05-30T15:53:09.912 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-30T15:53:09.912 INFO:teuthology.orchestra.run.smithi204.stdout: osd deep scrub update digest min age = 30 2024-05-30T15:53:09.912 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-30T15:53:09.912 INFO:teuthology.orchestra.run.smithi204.stdout: osd map max advance = 10 2024-05-30T15:53:09.912 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-30T15:53:09.912 INFO:teuthology.orchestra.run.smithi204.stdout: journal zero on create = true 2024-05-30T15:53:09.912 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-30T15:53:09.912 INFO:teuthology.orchestra.run.smithi204.stdout: filestore ondisk finisher threads = 3 2024-05-30T15:53:09.912 INFO:teuthology.orchestra.run.smithi204.stdout: filestore apply finisher threads = 3 2024-05-30T15:53:09.912 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-30T15:53:09.912 INFO:teuthology.orchestra.run.smithi204.stdout: bdev debug aio = true 2024-05-30T15:53:09.912 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug misdirected ops = true 2024-05-30T15:53:09.912 INFO:teuthology.orchestra.run.smithi204.stdout: bdev async discard = True 2024-05-30T15:53:09.913 INFO:teuthology.orchestra.run.smithi204.stdout: bdev enable discard = True 2024-05-30T15:53:09.913 INFO:teuthology.orchestra.run.smithi204.stdout: bluestore allocator = bitmap 2024-05-30T15:53:09.913 INFO:teuthology.orchestra.run.smithi204.stdout: bluestore block size = 96636764160 2024-05-30T15:53:09.913 INFO:teuthology.orchestra.run.smithi204.stdout: bluestore fsck on mount = True 2024-05-30T15:53:09.913 INFO:teuthology.orchestra.run.smithi204.stdout: debug bluefs = 1/20 2024-05-30T15:53:09.913 INFO:teuthology.orchestra.run.smithi204.stdout: debug bluestore = 1/20 2024-05-30T15:53:09.913 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-05-30T15:53:09.913 INFO:teuthology.orchestra.run.smithi204.stdout: debug osd = 20 2024-05-30T15:53:09.913 INFO:teuthology.orchestra.run.smithi204.stdout: debug rocksdb = 4/10 2024-05-30T15:53:09.913 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd backfillfull_ratio = 0.85 2024-05-30T15:53:09.913 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd full ratio = 0.9 2024-05-30T15:53:09.913 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd nearfull ratio = 0.8 2024-05-30T15:53:09.913 INFO:teuthology.orchestra.run.smithi204.stdout: osd failsafe full ratio = 0.95 2024-05-30T15:53:09.913 INFO:teuthology.orchestra.run.smithi204.stdout: osd fast shutdown = False 2024-05-30T15:53:09.914 INFO:teuthology.orchestra.run.smithi204.stdout: osd objectstore = bluestore 2024-05-30T15:53:09.914 INFO:teuthology.orchestra.run.smithi204.stdout: osd shutdown pgref assert = True 2024-05-30T15:53:09.914 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-30T15:53:09.914 INFO:teuthology.orchestra.run.smithi204.stdout:[mgr] 2024-05-30T15:53:09.914 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-05-30T15:53:09.914 INFO:teuthology.orchestra.run.smithi204.stdout: debug mgr = 20 2024-05-30T15:53:09.914 INFO:teuthology.orchestra.run.smithi204.stdout: debug mon = 20 2024-05-30T15:53:09.914 INFO:teuthology.orchestra.run.smithi204.stdout: debug auth = 20 2024-05-30T15:53:09.914 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min pgs per osd = 4 2024-05-30T15:53:09.914 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min bytes per osd = 10 2024-05-30T15:53:09.914 INFO:teuthology.orchestra.run.smithi204.stdout: mgr/telemetry/nag = false 2024-05-30T15:53:09.914 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-30T15:53:09.914 INFO:teuthology.orchestra.run.smithi204.stdout:[mon] 2024-05-30T15:53:09.915 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-05-30T15:53:09.915 INFO:teuthology.orchestra.run.smithi204.stdout: debug mon = 20 2024-05-30T15:53:09.915 INFO:teuthology.orchestra.run.smithi204.stdout: debug paxos = 20 2024-05-30T15:53:09.915 INFO:teuthology.orchestra.run.smithi204.stdout: debug auth = 20 2024-05-30T15:53:09.915 INFO:teuthology.orchestra.run.smithi204.stdout: mon data avail warn = 5 2024-05-30T15:53:09.915 INFO:teuthology.orchestra.run.smithi204.stdout: mon mgr mkfs grace = 240 2024-05-30T15:53:09.915 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min pgs per osd = 4 2024-05-30T15:53:09.915 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd reporter subtree level = osd 2024-05-30T15:53:09.915 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd prime pg temp = true 2024-05-30T15:53:09.915 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min bytes per osd = 10 2024-05-30T15:53:09.915 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-30T15:53:09.915 INFO:teuthology.orchestra.run.smithi204.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-30T15:53:09.915 INFO:teuthology.orchestra.run.smithi204.stdout: auth mon ticket ttl = 660 # 11m 2024-05-30T15:53:09.915 INFO:teuthology.orchestra.run.smithi204.stdout: auth service ticket ttl = 240 # 4m 2024-05-30T15:53:09.915 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-30T15:53:09.916 INFO:teuthology.orchestra.run.smithi204.stdout: # don't complain about insecure global_id in the test suite 2024-05-30T15:53:09.916 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-30T15:53:09.916 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-30T15:53:09.916 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-30T15:53:09.916 INFO:teuthology.orchestra.run.smithi204.stdout: # 1m isn't quite enough 2024-05-30T15:53:09.916 INFO:teuthology.orchestra.run.smithi204.stdout: mon_down_mkfs_grace = 2m 2024-05-30T15:53:09.916 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-30T15:53:09.916 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_filestore_osds = false 2024-05-30T15:53:09.916 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd crush smoke test = False 2024-05-30T15:53:09.916 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-30T15:53:09.916 INFO:teuthology.orchestra.run.smithi204.stdout:[client] 2024-05-30T15:53:09.916 INFO:teuthology.orchestra.run.smithi204.stdout: rgw cache enabled = true 2024-05-30T15:53:09.916 INFO:teuthology.orchestra.run.smithi204.stdout: rgw enable ops log = True 2024-05-30T15:53:09.916 INFO:teuthology.orchestra.run.smithi204.stdout: rgw enable usage log = true 2024-05-30T15:53:09.917 INFO:teuthology.orchestra.run.smithi204.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-30T15:53:09.917 INFO:teuthology.orchestra.run.smithi204.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-30T15:53:09.917 INFO:teuthology.orchestra.run.smithi204.stdout: debug rgw = 20 2024-05-30T15:53:09.917 INFO:teuthology.orchestra.run.smithi204.stdout: debug rgw notification = 20 2024-05-30T15:53:09.917 INFO:teuthology.orchestra.run.smithi204.stdout: rgw bucket counters cache = True 2024-05-30T15:53:09.917 INFO:teuthology.orchestra.run.smithi204.stdout: rgw crypt require ssl = False 2024-05-30T15:53:09.917 INFO:teuthology.orchestra.run.smithi204.stdout: rgw crypt s3 kms backend = testing 2024-05-30T15:53:09.917 INFO:teuthology.orchestra.run.smithi204.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2024-05-30T15:53:09.917 INFO:teuthology.orchestra.run.smithi204.stdout: rgw d3n l1 datacache persistent path = /tmp/rgw_datacache/ 2024-05-30T15:53:09.917 INFO:teuthology.orchestra.run.smithi204.stdout: rgw d3n l1 datacache size = 10737418240 2024-05-30T15:53:09.917 INFO:teuthology.orchestra.run.smithi204.stdout: rgw d3n l1 local datacache enabled = True 2024-05-30T15:53:09.917 INFO:teuthology.orchestra.run.smithi204.stdout: rgw max chunk size = 4194304 2024-05-30T15:53:09.917 INFO:teuthology.orchestra.run.smithi204.stdout: rgw obj stripe size = 6291456 2024-05-30T15:53:09.917 INFO:teuthology.orchestra.run.smithi204.stdout: rgw s3 auth use sts = True 2024-05-30T15:53:09.917 INFO:teuthology.orchestra.run.smithi204.stdout: rgw sts key = abcdefghijklmnop 2024-05-30T15:53:09.918 INFO:teuthology.orchestra.run.smithi204.stdout: rgw torrent flag = True 2024-05-30T15:53:09.918 INFO:teuthology.orchestra.run.smithi204.stdout: rgw user counters cache = True 2024-05-30T15:53:09.918 INFO:teuthology.orchestra.run.smithi204.stdout: setgroup = ceph 2024-05-30T15:53:09.918 INFO:teuthology.orchestra.run.smithi204.stdout: setuser = ceph 2024-05-30T15:53:09.918 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.a] 2024-05-30T15:53:09.918 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.c] 2024-05-30T15:53:09.918 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.b] 2024-05-30T15:53:09.918 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-30T15:53:09.918 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-30T15:53:10.004 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-30T15:53:10.004 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T15:53:10.004 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-30T15:53:10.010 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T15:53:10.010 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-30T15:53:10.058 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi006.front.sepia.ceph.com 2024-05-30T15:53:10.058 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T15:53:10.058 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-30T15:53:10.058 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-30T15:53:10.123 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T15:53:10.123 DEBUG:teuthology.orchestra.run.smithi006:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-30T15:53:10.171 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi204.front.sepia.ceph.com 2024-05-30T15:53:10.171 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-30T15:53:10.171 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-30T15:53:10.171 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-30T15:53:10.195 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-30T15:53:10.195 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-30T15:53:10.240 INFO:tasks.ceph:Setting up mon nodes... 2024-05-30T15:53:10.241 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-30T15:53:10.241 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-30T15:53:10.287 INFO:teuthology.orchestra.run.smithi006.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-30T15:53:10.290 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-30T15:53:10.333 INFO:teuthology.orchestra.run.smithi204.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-30T15:53:10.345 INFO:tasks.ceph:Setting up mds nodes... 2024-05-30T15:53:10.345 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-30T15:53:10.346 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-30T15:53:10.381 INFO:teuthology.orchestra.run.smithi006.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-30T15:53:10.392 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-30T15:53:10.425 INFO:teuthology.orchestra.run.smithi204.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-30T15:53:10.436 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-30T15:53:10.436 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi006.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi204.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-30T15:53:10.436 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-30T15:53:10.450 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-30T15:53:10.450 INFO:tasks.ceph:role: osd.0 2024-05-30T15:53:10.450 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi006.front.sepia.ceph.com 2024-05-30T15:53:10.450 DEBUG:teuthology.orchestra.run.smithi006:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-30T15:53:10.512 INFO:teuthology.orchestra.run.smithi006.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-30T15:53:10.512 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-30T15:53:10.512 INFO:teuthology.orchestra.run.smithi006.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-30T15:53:10.512 INFO:teuthology.orchestra.run.smithi006.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-30T15:53:10.512 INFO:teuthology.orchestra.run.smithi006.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-30T15:53:10.512 INFO:teuthology.orchestra.run.smithi006.stdout: = sunit=0 swidth=0 blks 2024-05-30T15:53:10.513 INFO:teuthology.orchestra.run.smithi006.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-30T15:53:10.513 INFO:teuthology.orchestra.run.smithi006.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-30T15:53:10.513 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-30T15:53:10.513 INFO:teuthology.orchestra.run.smithi006.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-30T15:53:10.989 INFO:teuthology.orchestra.run.smithi006.stdout:Discarding blocks...Done. 2024-05-30T15:53:10.994 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi006.front.sepia.ceph.com -o noatime 2024-05-30T15:53:10.994 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-30T15:53:11.142 DEBUG:teuthology.orchestra.run.smithi006:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-30T15:53:11.153 INFO:teuthology.orchestra.run.smithi006.stderr:sudo: /sbin/restorecon: command not found 2024-05-30T15:53:11.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-30T15:53:11.154 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-30T15:53:11.210 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-30T15:53:11.211 INFO:tasks.ceph:role: osd.1 2024-05-30T15:53:11.211 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi006.front.sepia.ceph.com 2024-05-30T15:53:11.211 DEBUG:teuthology.orchestra.run.smithi006:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-30T15:53:11.273 INFO:teuthology.orchestra.run.smithi006.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-30T15:53:11.273 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-30T15:53:11.273 INFO:teuthology.orchestra.run.smithi006.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-30T15:53:11.273 INFO:teuthology.orchestra.run.smithi006.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-30T15:53:11.273 INFO:teuthology.orchestra.run.smithi006.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-30T15:53:11.273 INFO:teuthology.orchestra.run.smithi006.stdout: = sunit=0 swidth=0 blks 2024-05-30T15:53:11.273 INFO:teuthology.orchestra.run.smithi006.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-30T15:53:11.273 INFO:teuthology.orchestra.run.smithi006.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-30T15:53:11.274 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-30T15:53:11.274 INFO:teuthology.orchestra.run.smithi006.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-30T15:53:11.724 INFO:teuthology.orchestra.run.smithi006.stdout:Discarding blocks...Done. 2024-05-30T15:53:11.726 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi006.front.sepia.ceph.com -o noatime 2024-05-30T15:53:11.726 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-30T15:53:11.816 DEBUG:teuthology.orchestra.run.smithi006:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-30T15:53:11.824 INFO:teuthology.orchestra.run.smithi006.stderr:sudo: /sbin/restorecon: command not found 2024-05-30T15:53:11.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-30T15:53:11.825 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-30T15:53:11.879 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-30T15:53:11.879 INFO:tasks.ceph:role: osd.2 2024-05-30T15:53:11.879 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi006.front.sepia.ceph.com 2024-05-30T15:53:11.879 DEBUG:teuthology.orchestra.run.smithi006:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-30T15:53:11.942 INFO:teuthology.orchestra.run.smithi006.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-30T15:53:11.942 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-30T15:53:11.942 INFO:teuthology.orchestra.run.smithi006.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-30T15:53:11.942 INFO:teuthology.orchestra.run.smithi006.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-30T15:53:11.942 INFO:teuthology.orchestra.run.smithi006.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-30T15:53:11.942 INFO:teuthology.orchestra.run.smithi006.stdout: = sunit=0 swidth=0 blks 2024-05-30T15:53:11.942 INFO:teuthology.orchestra.run.smithi006.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-30T15:53:11.942 INFO:teuthology.orchestra.run.smithi006.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-30T15:53:11.942 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-30T15:53:11.943 INFO:teuthology.orchestra.run.smithi006.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-30T15:53:12.397 INFO:teuthology.orchestra.run.smithi006.stdout:Discarding blocks...Done. 2024-05-30T15:53:12.399 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi006.front.sepia.ceph.com -o noatime 2024-05-30T15:53:12.399 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-30T15:53:12.506 DEBUG:teuthology.orchestra.run.smithi006:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-30T15:53:12.517 INFO:teuthology.orchestra.run.smithi006.stderr:sudo: /sbin/restorecon: command not found 2024-05-30T15:53:12.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-30T15:53:12.518 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-30T15:53:12.574 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-30T15:53:12.574 INFO:tasks.ceph:role: osd.3 2024-05-30T15:53:12.574 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi006.front.sepia.ceph.com 2024-05-30T15:53:12.574 DEBUG:teuthology.orchestra.run.smithi006:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-30T15:53:12.638 INFO:teuthology.orchestra.run.smithi006.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-30T15:53:12.638 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-30T15:53:12.638 INFO:teuthology.orchestra.run.smithi006.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-30T15:53:12.638 INFO:teuthology.orchestra.run.smithi006.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-30T15:53:12.638 INFO:teuthology.orchestra.run.smithi006.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-30T15:53:12.639 INFO:teuthology.orchestra.run.smithi006.stdout: = sunit=0 swidth=0 blks 2024-05-30T15:53:12.639 INFO:teuthology.orchestra.run.smithi006.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-30T15:53:12.639 INFO:teuthology.orchestra.run.smithi006.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-30T15:53:12.639 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-30T15:53:12.639 INFO:teuthology.orchestra.run.smithi006.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-30T15:53:13.131 INFO:teuthology.orchestra.run.smithi006.stdout:Discarding blocks...Done. 2024-05-30T15:53:13.133 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi006.front.sepia.ceph.com -o noatime 2024-05-30T15:53:13.133 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-30T15:53:13.289 DEBUG:teuthology.orchestra.run.smithi006:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-30T15:53:13.301 INFO:teuthology.orchestra.run.smithi006.stderr:sudo: /sbin/restorecon: command not found 2024-05-30T15:53:13.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-30T15:53:13.302 DEBUG:teuthology.orchestra.run.smithi006:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-30T15:53:13.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:53:13.410+0000 7f3dd40637c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-30T15:53:13.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:53:13.414+0000 7f3dd40637c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-30T15:53:13.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:53:13.414+0000 7f3dd40637c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-30T15:53:13.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:53:13.414+0000 7f3dd40637c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-30T15:53:13.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:53:13.414+0000 7f3dd40637c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-30T15:53:13.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:53:13.426+0000 7f3dd40637c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-30T15:53:16.416 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-30T15:53:16.430 DEBUG:teuthology.orchestra.run.smithi006:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-30T15:53:16.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:53:16.514+0000 7ff4c6fd57c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-30T15:53:16.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:53:16.518+0000 7ff4c6fd57c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-30T15:53:16.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:53:16.518+0000 7ff4c6fd57c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-30T15:53:16.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:53:16.518+0000 7ff4c6fd57c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-30T15:53:16.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:53:16.518+0000 7ff4c6fd57c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-30T15:53:16.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:53:16.518+0000 7ff4c6fd57c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-30T15:53:19.489 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-30T15:53:19.547 DEBUG:teuthology.orchestra.run.smithi006:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-30T15:53:19.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:53:19.626+0000 7f95d46c87c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-30T15:53:19.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:53:19.630+0000 7f95d46c87c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-30T15:53:19.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:53:19.630+0000 7f95d46c87c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-30T15:53:19.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:53:19.630+0000 7f95d46c87c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-30T15:53:19.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:53:19.630+0000 7f95d46c87c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-30T15:53:19.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:53:19.630+0000 7f95d46c87c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-30T15:53:22.616 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-30T15:53:22.675 DEBUG:teuthology.orchestra.run.smithi006:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-30T15:53:22.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:53:22.762+0000 7f844fcaa7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-30T15:53:22.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:53:22.762+0000 7f844fcaa7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-30T15:53:22.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:53:22.762+0000 7f844fcaa7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-30T15:53:22.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:53:22.762+0000 7f844fcaa7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-30T15:53:22.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:53:22.762+0000 7f844fcaa7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-30T15:53:22.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:53:22.762+0000 7f844fcaa7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-30T15:53:25.762 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-30T15:53:25.819 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-30T15:53:25.833 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-30T15:53:25.833 INFO:tasks.ceph:role: osd.4 2024-05-30T15:53:25.834 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi204.front.sepia.ceph.com 2024-05-30T15:53:25.834 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-30T15:53:25.895 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-30T15:53:25.895 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-30T15:53:25.895 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-30T15:53:25.895 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-30T15:53:25.896 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-30T15:53:25.896 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-05-30T15:53:25.896 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-30T15:53:25.896 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-30T15:53:25.896 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-30T15:53:25.896 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-30T15:53:26.349 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-05-30T15:53:26.350 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-05-30T15:53:26.351 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-30T15:53:26.438 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-30T15:53:26.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-30T15:53:26.449 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: /sbin/restorecon: command not found 2024-05-30T15:53:26.450 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-30T15:53:26.502 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-30T15:53:26.502 INFO:tasks.ceph:role: osd.5 2024-05-30T15:53:26.502 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi204.front.sepia.ceph.com 2024-05-30T15:53:26.503 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-30T15:53:26.568 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-30T15:53:26.568 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-30T15:53:26.568 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-30T15:53:26.569 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-30T15:53:26.569 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-30T15:53:26.569 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-05-30T15:53:26.569 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-30T15:53:26.569 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-30T15:53:26.569 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-30T15:53:26.569 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-30T15:53:27.031 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-05-30T15:53:27.033 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-05-30T15:53:27.033 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-30T15:53:27.193 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-30T15:53:27.204 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: /sbin/restorecon: command not found 2024-05-30T15:53:27.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-30T15:53:27.205 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-30T15:53:27.260 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-30T15:53:27.260 INFO:tasks.ceph:role: osd.6 2024-05-30T15:53:27.260 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi204.front.sepia.ceph.com 2024-05-30T15:53:27.260 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-30T15:53:27.323 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-30T15:53:27.323 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-30T15:53:27.323 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-30T15:53:27.324 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-30T15:53:27.324 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-30T15:53:27.324 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-05-30T15:53:27.324 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-30T15:53:27.324 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-30T15:53:27.324 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-30T15:53:27.324 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-30T15:53:27.796 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-05-30T15:53:27.798 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-05-30T15:53:27.798 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-30T15:53:27.927 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-30T15:53:27.937 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: /sbin/restorecon: command not found 2024-05-30T15:53:27.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-30T15:53:27.938 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-30T15:53:27.989 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-30T15:53:27.989 INFO:tasks.ceph:role: osd.7 2024-05-30T15:53:27.989 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi204.front.sepia.ceph.com 2024-05-30T15:53:27.989 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-30T15:53:28.050 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-30T15:53:28.051 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-30T15:53:28.051 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-30T15:53:28.051 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-30T15:53:28.051 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-30T15:53:28.051 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-05-30T15:53:28.051 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-30T15:53:28.051 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-30T15:53:28.051 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-30T15:53:28.051 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-30T15:53:28.470 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-05-30T15:53:28.472 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-05-30T15:53:28.472 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-30T15:53:28.555 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-30T15:53:28.605 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: /sbin/restorecon: command not found 2024-05-30T15:53:28.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-30T15:53:28.606 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-30T15:53:28.718 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-30T15:53:28.716+0000 7f64f215e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-30T15:53:28.719 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-30T15:53:28.720+0000 7f64f215e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-30T15:53:28.720 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-30T15:53:28.720+0000 7f64f215e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-30T15:53:28.720 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-30T15:53:28.720+0000 7f64f215e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-30T15:53:28.720 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-30T15:53:28.720+0000 7f64f215e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-30T15:53:28.721 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-30T15:53:28.720+0000 7f64f215e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-30T15:53:31.717 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-30T15:53:31.732 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-30T15:53:31.812 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-30T15:53:31.812+0000 7f093987a7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-30T15:53:31.813 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-30T15:53:31.812+0000 7f093987a7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-30T15:53:31.813 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-30T15:53:31.812+0000 7f093987a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-30T15:53:31.813 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-30T15:53:31.812+0000 7f093987a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-30T15:53:31.813 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-30T15:53:31.812+0000 7f093987a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-30T15:53:31.814 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-30T15:53:31.812+0000 7f093987a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-30T15:53:34.781 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-30T15:53:34.837 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-30T15:53:34.919 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-30T15:53:34.916+0000 7f9beada77c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-30T15:53:34.920 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-30T15:53:34.920+0000 7f9beada77c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-30T15:53:34.920 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-30T15:53:34.920+0000 7f9beada77c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-30T15:53:34.920 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-30T15:53:34.920+0000 7f9beada77c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-30T15:53:34.920 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-30T15:53:34.920+0000 7f9beada77c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-30T15:53:34.921 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-30T15:53:34.920+0000 7f9beada77c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-30T15:53:37.898 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-30T15:53:37.952 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-30T15:53:38.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-30T15:53:38.032+0000 7fcc873427c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-30T15:53:38.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-30T15:53:38.032+0000 7fcc873427c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-30T15:53:38.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-30T15:53:38.032+0000 7fcc873427c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-30T15:53:38.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-30T15:53:38.032+0000 7fcc873427c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-30T15:53:38.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-30T15:53:38.032+0000 7fcc873427c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-30T15:53:38.036 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-30T15:53:38.036+0000 7fcc873427c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-30T15:53:41.010 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-30T15:53:41.063 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-30T15:53:41.063 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T15:53:41.063 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-30T15:53:41.078 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T15:53:41.078 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-30T15:53:41.133 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T15:53:41.134 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-30T15:53:41.190 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T15:53:41.190 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-30T15:53:41.245 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T15:53:41.246 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-30T15:53:41.301 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-30T15:53:41.302 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-30T15:53:41.315 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-30T15:53:41.315 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-30T15:53:41.371 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-30T15:53:41.371 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-30T15:53:41.427 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-30T15:53:41.427 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-30T15:53:41.478 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-30T15:53:41.479 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-30T15:53:41.531 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T15:53:41.531 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-30T15:53:41.537 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-30T15:53:41.537 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-30T15:53:41.580 INFO:tasks.ceph:Adding keys to all mons... 2024-05-30T15:53:41.580 DEBUG:teuthology.orchestra.run.smithi006:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-30T15:53:41.583 DEBUG:teuthology.orchestra.run.smithi204:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-30T15:53:41.626 INFO:teuthology.orchestra.run.smithi006.stdout:[mgr.y] 2024-05-30T15:53:41.626 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQDmoFhmGHVUERAA07XbQo71vt61XU3BirRzOw== 2024-05-30T15:53:41.626 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.0] 2024-05-30T15:53:41.626 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQDpoFhm8nKuGBAAqzVhVUh+82yIc+VFWwLr6g== 2024-05-30T15:53:41.626 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.1] 2024-05-30T15:53:41.626 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQDsoFhmCHL+HhAApcoGeI/PEanE1Rg/xCWWIw== 2024-05-30T15:53:41.626 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.2] 2024-05-30T15:53:41.626 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQDvoFhmQ8atJRAAZix6siBqSPO0oBf/XQ9fFw== 2024-05-30T15:53:41.626 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.3] 2024-05-30T15:53:41.627 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQDyoFhmOXWTLRAAu6YYwN0sr+64dywiQAplHw== 2024-05-30T15:53:41.627 INFO:teuthology.orchestra.run.smithi006.stdout:[mgr.x] 2024-05-30T15:53:41.627 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQDmoFhmpwoUFBAA4Pk2J9AA4Dq7QNgwDdDQsw== 2024-05-30T15:53:41.627 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.4] 2024-05-30T15:53:41.627 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQD4oFhmU1cLKxAAeBNqg5c2drLa/hq2WzVetA== 2024-05-30T15:53:41.627 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.5] 2024-05-30T15:53:41.627 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQD7oFhmHuOoMBAACmN/vBMfPk7JsJsLyHQ+ng== 2024-05-30T15:53:41.628 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.6] 2024-05-30T15:53:41.628 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQD+oFhmJHcENxAA8PJq0d1x89FY6OgwYZK8Ww== 2024-05-30T15:53:41.628 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.7] 2024-05-30T15:53:41.628 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQACoVhm2odDAhAA0Mw88omJBE6Y8l85b9Eyhg== 2024-05-30T15:53:41.628 INFO:teuthology.orchestra.run.smithi006.stdout:[client.0] 2024-05-30T15:53:41.628 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQDmoFhmpSP0FhAA+uJegnuT3+AUGcfTkA8QpQ== 2024-05-30T15:53:41.628 INFO:teuthology.orchestra.run.smithi006.stdout:[client.1] 2024-05-30T15:53:41.628 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQDmoFhmTK+UGRAAnmWthYXYUcJfDZo0UMziEg== 2024-05-30T15:53:41.634 INFO:teuthology.orchestra.run.smithi204.stdout:[mgr.y] 2024-05-30T15:53:41.634 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQDmoFhmGHVUERAA07XbQo71vt61XU3BirRzOw== 2024-05-30T15:53:41.634 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.0] 2024-05-30T15:53:41.634 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQDpoFhm8nKuGBAAqzVhVUh+82yIc+VFWwLr6g== 2024-05-30T15:53:41.634 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.1] 2024-05-30T15:53:41.634 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQDsoFhmCHL+HhAApcoGeI/PEanE1Rg/xCWWIw== 2024-05-30T15:53:41.634 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.2] 2024-05-30T15:53:41.635 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQDvoFhmQ8atJRAAZix6siBqSPO0oBf/XQ9fFw== 2024-05-30T15:53:41.635 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.3] 2024-05-30T15:53:41.635 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQDyoFhmOXWTLRAAu6YYwN0sr+64dywiQAplHw== 2024-05-30T15:53:41.635 INFO:teuthology.orchestra.run.smithi204.stdout:[mgr.x] 2024-05-30T15:53:41.635 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQDmoFhmpwoUFBAA4Pk2J9AA4Dq7QNgwDdDQsw== 2024-05-30T15:53:41.635 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.4] 2024-05-30T15:53:41.635 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQD4oFhmU1cLKxAAeBNqg5c2drLa/hq2WzVetA== 2024-05-30T15:53:41.635 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.5] 2024-05-30T15:53:41.635 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQD7oFhmHuOoMBAACmN/vBMfPk7JsJsLyHQ+ng== 2024-05-30T15:53:41.635 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.6] 2024-05-30T15:53:41.635 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQD+oFhmJHcENxAA8PJq0d1x89FY6OgwYZK8Ww== 2024-05-30T15:53:41.636 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.7] 2024-05-30T15:53:41.636 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQACoVhm2odDAhAA0Mw88omJBE6Y8l85b9Eyhg== 2024-05-30T15:53:41.636 INFO:teuthology.orchestra.run.smithi204.stdout:[client.0] 2024-05-30T15:53:41.636 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQDmoFhmpSP0FhAA+uJegnuT3+AUGcfTkA8QpQ== 2024-05-30T15:53:41.636 INFO:teuthology.orchestra.run.smithi204.stdout:[client.1] 2024-05-30T15:53:41.636 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQDmoFhmTK+UGRAAnmWthYXYUcJfDZo0UMziEg== 2024-05-30T15:53:41.637 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-30T15:53:41.675 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-30T15:53:41.728 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T15:53:41.766 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T15:53:41.808 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T15:53:41.851 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T15:53:41.895 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T15:53:41.897 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T15:53:41.980 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T15:53:41.982 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T15:53:42.023 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-30T15:53:42.066 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-30T15:53:42.108 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T15:53:42.151 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T15:53:42.194 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T15:53:42.234 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T15:53:42.279 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T15:53:42.318 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T15:53:42.365 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T15:53:42.402 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T15:53:42.450 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-30T15:53:42.562 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-30T15:53:42.613 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-30T15:53:42.615 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-30T15:53:42.661 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-30T15:53:42.662 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-30T15:53:42.674 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-30T15:53:42.767 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-30T15:53:42.777 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-30T15:53:42.835 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-30T15:53:42.922 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-30T15:53:42.933 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-30T15:53:42.947 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-30T15:53:43.042 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-30T15:53:43.053 DEBUG:teuthology.orchestra.run.smithi006:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-30T15:53:43.055 DEBUG:teuthology.orchestra.run.smithi204:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-30T15:53:43.099 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-30T15:53:43.099 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-30T15:53:43.100 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-30T15:53:43.102 INFO:tasks.ceph.mon.a:Started 2024-05-30T15:53:43.103 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-30T15:53:43.103 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-30T15:53:43.105 INFO:tasks.ceph.mon.c:Started 2024-05-30T15:53:43.105 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-30T15:53:43.105 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-30T15:53:43.144 INFO:tasks.ceph.mon.b:Started 2024-05-30T15:53:43.144 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-30T15:53:43.145 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-30T15:53:43.145 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-30T15:53:43.147 INFO:tasks.ceph.mgr.y:Started 2024-05-30T15:53:43.147 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-30T15:53:43.147 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-30T15:53:43.149 INFO:tasks.ceph.mgr.x:Started 2024-05-30T15:53:43.149 INFO:tasks.ceph:Setting crush tunables to default 2024-05-30T15:53:43.149 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --cluster ceph osd crush tunables default 2024-05-30T15:53:48.730 INFO:tasks.ceph.mgr.y.smithi006.stderr:2024-05-30T15:53:48.730+0000 7ff974642180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-30T15:53:48.784 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-30T15:53:48.784+0000 7fb5f44df180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-30T15:53:49.254 INFO:tasks.ceph.mgr.y.smithi006.stderr:2024-05-30T15:53:49.254+0000 7ff974642180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-30T15:53:49.292 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-30T15:53:49.292+0000 7fb5f44df180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-30T15:53:49.348 INFO:teuthology.orchestra.run.smithi006.stderr:adjusted tunables profile to default 2024-05-30T15:53:49.364 INFO:tasks.ceph:check_enable_crimson: False 2024-05-30T15:53:49.364 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-30T15:53:49.364 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T15:53:49.364 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-30T15:53:49.376 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T15:53:49.376 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-30T15:53:49.428 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T15:53:49.428 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-30T15:53:49.459 INFO:tasks.ceph.mgr.y.smithi006.stderr:2024-05-30T15:53:49.458+0000 7ff974642180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-30T15:53:49.470 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T15:53:49.470 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-30T15:53:49.496 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-30T15:53:49.496+0000 7fb5f44df180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-30T15:53:49.526 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-30T15:53:49.526 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-30T15:53:49.538 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-30T15:53:49.539 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-30T15:53:49.591 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-30T15:53:49.591 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-30T15:53:49.642 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-30T15:53:49.643 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-30T15:53:49.693 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new c9b54fda-6c4d-49b5-b6f7-9baf01edc4fe 0 2024-05-30T15:53:49.949 INFO:teuthology.orchestra.run.smithi204.stdout:0 2024-05-30T15:53:49.965 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 476e6529-66eb-4479-8ca9-b646750fb07c 1 2024-05-30T15:53:50.034 INFO:tasks.ceph.mgr.y.smithi006.stderr:2024-05-30T15:53:50.034+0000 7ff974642180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-30T15:53:50.090 INFO:tasks.ceph.mgr.y.smithi006.stderr:2024-05-30T15:53:50.090+0000 7ff974642180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-30T15:53:50.093 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-30T15:53:50.092+0000 7fb5f44df180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-30T15:53:50.145 INFO:tasks.ceph.mgr.y.smithi006.stderr:2024-05-30T15:53:50.146+0000 7ff974642180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-30T15:53:50.148 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-30T15:53:50.148+0000 7fb5f44df180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-30T15:53:50.162 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-05-30T15:53:50.178 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new c8461074-51a0-4209-aed4-e05343440267 2 2024-05-30T15:53:50.205 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-30T15:53:50.204+0000 7fb5f44df180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-30T15:53:50.227 INFO:tasks.ceph.mgr.y.smithi006.stderr:2024-05-30T15:53:50.226+0000 7ff974642180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-30T15:53:50.281 INFO:tasks.ceph.mgr.y.smithi006.stderr:2024-05-30T15:53:50.282+0000 7ff974642180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-30T15:53:50.286 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-30T15:53:50.284+0000 7fb5f44df180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-30T15:53:50.345 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-30T15:53:50.344+0000 7fb5f44df180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-30T15:53:50.362 INFO:tasks.ceph.mgr.y.smithi006.stderr:2024-05-30T15:53:50.362+0000 7ff974642180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-30T15:53:50.377 INFO:teuthology.orchestra.run.smithi204.stdout:2 2024-05-30T15:53:50.394 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new c4775e06-d8e2-44c0-a0c8-9f4e8e21f5cc 3 2024-05-30T15:53:50.419 INFO:tasks.ceph.mgr.y.smithi006.stderr:2024-05-30T15:53:50.418+0000 7ff974642180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-30T15:53:50.433 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-30T15:53:50.432+0000 7fb5f44df180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-30T15:53:50.490 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-30T15:53:50.488+0000 7fb5f44df180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-30T15:53:50.578 INFO:tasks.ceph.mgr.y.smithi006.stderr:2024-05-30T15:53:50.578+0000 7ff974642180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-30T15:53:50.629 INFO:teuthology.orchestra.run.smithi204.stdout:3 2024-05-30T15:53:50.643 INFO:tasks.ceph.mgr.y.smithi006.stderr:2024-05-30T15:53:50.642+0000 7ff974642180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-30T15:53:50.647 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new bfc9fd12-63b8-4711-a306-23a17a9aaf0f 4 2024-05-30T15:53:50.650 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-30T15:53:50.648+0000 7fb5f44df180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-30T15:53:50.714 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-30T15:53:50.712+0000 7fb5f44df180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-30T15:53:50.830 INFO:teuthology.orchestra.run.smithi204.stdout:4 2024-05-30T15:53:50.846 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new e21db988-15e6-488e-8bc1-e5ba0a2f354b 5 2024-05-30T15:53:51.144 INFO:teuthology.orchestra.run.smithi204.stdout:5 2024-05-30T15:53:51.162 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 85a75312-8faf-4279-9301-134fcb19cbd9 6 2024-05-30T15:53:51.280 INFO:tasks.ceph.mgr.y.smithi006.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-05-30T15:53:51.280 INFO:tasks.ceph.mgr.y.smithi006.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-05-30T15:53:51.280 INFO:tasks.ceph.mgr.y.smithi006.stderr: from numpy import show_config as show_numpy_config 2024-05-30T15:53:51.311 INFO:tasks.ceph.mgr.x.smithi204.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-05-30T15:53:51.311 INFO:tasks.ceph.mgr.x.smithi204.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-05-30T15:53:51.312 INFO:tasks.ceph.mgr.x.smithi204.stderr: from numpy import show_config as show_numpy_config 2024-05-30T15:53:51.354 INFO:teuthology.orchestra.run.smithi204.stdout:6 2024-05-30T15:53:51.369 INFO:tasks.ceph.mgr.y.smithi006.stderr:2024-05-30T15:53:51.366+0000 7ff974642180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-30T15:53:51.370 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new f0550e43-5653-4433-8f78-17af5e82316d 7 2024-05-30T15:53:51.375 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-30T15:53:51.376+0000 7fb5f44df180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-30T15:53:51.548 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-30T15:53:51.548+0000 7fb5f44df180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-30T15:53:51.559 INFO:tasks.ceph.mgr.y.smithi006.stderr:2024-05-30T15:53:51.558+0000 7ff974642180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-30T15:53:51.589 INFO:teuthology.orchestra.run.smithi204.stdout:7 2024-05-30T15:53:51.604 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-30T15:53:51.605 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-30T15:53:51.607 INFO:tasks.ceph.osd.0:Started 2024-05-30T15:53:51.607 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-30T15:53:51.607 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-30T15:53:51.609 INFO:tasks.ceph.osd.1:Started 2024-05-30T15:53:51.610 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-30T15:53:51.610 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-30T15:53:51.612 INFO:tasks.ceph.osd.2:Started 2024-05-30T15:53:51.612 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-30T15:53:51.612 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-30T15:53:51.615 INFO:tasks.ceph.osd.3:Started 2024-05-30T15:53:51.615 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-30T15:53:51.615 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-30T15:53:51.648 INFO:tasks.ceph.osd.4:Started 2024-05-30T15:53:51.649 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-30T15:53:51.649 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-30T15:53:51.651 INFO:tasks.ceph.osd.5:Started 2024-05-30T15:53:51.651 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-30T15:53:51.651 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-30T15:53:51.653 INFO:tasks.ceph.osd.6:Started 2024-05-30T15:53:51.653 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-30T15:53:51.653 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-30T15:53:51.655 INFO:tasks.ceph.osd.7:Started 2024-05-30T15:53:51.655 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T15:53:51.723 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-30T15:53:51.720+0000 7fb5f44df180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-30T15:53:51.761 INFO:tasks.ceph.mgr.y.smithi006.stderr:2024-05-30T15:53:51.758+0000 7ff974642180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-30T15:53:51.787 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-30T15:53:51.784+0000 7fb5f44df180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-30T15:53:51.847 INFO:tasks.ceph.mgr.y.smithi006.stderr:2024-05-30T15:53:51.846+0000 7ff974642180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-30T15:53:51.862 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:53:51.863 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":10,"fsid":"87f2315a-4430-477b-ae32-1664164ce9e8","created":"2024-05-30T15:53:48.501791+0000","modified":"2024-05-30T15:53:51.589084+0000","last_up_change":"0.000000","last_in_change":"2024-05-30T15:53:51.589084+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c9b54fda-6c4d-49b5-b6f7-9baf01edc4fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"476e6529-66eb-4479-8ca9-b646750fb07c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c8461074-51a0-4209-aed4-e05343440267","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c4775e06-d8e2-44c0-a0c8-9f4e8e21f5cc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"bfc9fd12-63b8-4711-a306-23a17a9aaf0f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e21db988-15e6-488e-8bc1-e5ba0a2f354b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"85a75312-8faf-4279-9301-134fcb19cbd9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f0550e43-5653-4433-8f78-17af5e82316d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T15:53:51.881 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-30T15:53:51.881 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-30T15:53:51.921 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-30T15:53:51.920+0000 7fb5f44df180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-30T15:53:52.000 INFO:tasks.ceph.mgr.y.smithi006.stderr:2024-05-30T15:53:51.998+0000 7ff974642180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-30T15:53:52.239 INFO:tasks.ceph.osd.0.smithi006.stderr:2024-05-30T15:53:52.238+0000 7f93d705b7c0 -1 Falling back to public interface 2024-05-30T15:53:52.246 INFO:tasks.ceph.osd.1.smithi006.stderr:2024-05-30T15:53:52.246+0000 7f061659a7c0 -1 Falling back to public interface 2024-05-30T15:53:52.250 INFO:tasks.ceph.osd.2.smithi006.stderr:2024-05-30T15:53:52.250+0000 7f1ebb93d7c0 -1 Falling back to public interface 2024-05-30T15:53:52.251 INFO:tasks.ceph.osd.3.smithi006.stderr:2024-05-30T15:53:52.250+0000 7fe0c49d77c0 -1 Falling back to public interface 2024-05-30T15:53:52.256 INFO:tasks.ceph.osd.6.smithi204.stderr:2024-05-30T15:53:52.256+0000 7f05bebd27c0 -1 Falling back to public interface 2024-05-30T15:53:52.257 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-05-30T15:53:52.256+0000 7f31897607c0 -1 Falling back to public interface 2024-05-30T15:53:52.276 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-05-30T15:53:52.276+0000 7fa0b7ebf7c0 -1 Falling back to public interface 2024-05-30T15:53:52.280 INFO:tasks.ceph.osd.7.smithi204.stderr:2024-05-30T15:53:52.280+0000 7f2c6bf547c0 -1 Falling back to public interface 2024-05-30T15:53:52.554 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-30T15:53:52.552+0000 7fb5f44df180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-30T15:53:52.631 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-30T15:53:52.628+0000 7fb5f44df180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-30T15:53:52.671 INFO:tasks.ceph.mgr.y.smithi006.stderr:2024-05-30T15:53:52.670+0000 7ff974642180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-30T15:53:52.687 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-30T15:53:52.752 INFO:tasks.ceph.mgr.y.smithi006.stderr:2024-05-30T15:53:52.750+0000 7ff974642180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-30T15:53:52.813 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-30T15:53:52.812+0000 7fb5f44df180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-30T15:53:52.871 INFO:teuthology.misc.health.smithi006.stdout: 2024-05-30T15:53:52.871 INFO:teuthology.misc.health.smithi006.stdout:{"epoch":10,"fsid":"87f2315a-4430-477b-ae32-1664164ce9e8","created":"2024-05-30T15:53:48.501791+0000","modified":"2024-05-30T15:53:51.589084+0000","last_up_change":"0.000000","last_in_change":"2024-05-30T15:53:51.589084+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c9b54fda-6c4d-49b5-b6f7-9baf01edc4fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"476e6529-66eb-4479-8ca9-b646750fb07c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c8461074-51a0-4209-aed4-e05343440267","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c4775e06-d8e2-44c0-a0c8-9f4e8e21f5cc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"bfc9fd12-63b8-4711-a306-23a17a9aaf0f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e21db988-15e6-488e-8bc1-e5ba0a2f354b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"85a75312-8faf-4279-9301-134fcb19cbd9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f0550e43-5653-4433-8f78-17af5e82316d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T15:53:52.886 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-30T15:53:52.888 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-30T15:53:52.888+0000 7fb5f44df180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-30T15:53:52.947 INFO:tasks.ceph.mgr.y.smithi006.stderr:2024-05-30T15:53:52.946+0000 7ff974642180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-30T15:53:53.024 INFO:tasks.ceph.mgr.y.smithi006.stderr:2024-05-30T15:53:53.022+0000 7ff974642180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-30T15:53:53.181 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-30T15:53:53.180+0000 7fb5f44df180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-30T15:53:53.244 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-30T15:53:53.244+0000 7fb5f44df180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-30T15:53:53.337 INFO:tasks.ceph.mgr.y.smithi006.stderr:2024-05-30T15:53:53.338+0000 7ff974642180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-30T15:53:53.402 INFO:tasks.ceph.mgr.y.smithi006.stderr:2024-05-30T15:53:53.402+0000 7ff974642180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-30T15:53:54.293 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-30T15:53:54.292+0000 7fb59b120640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-30T15:53:54.295 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-30T15:53:54.292+0000 7fb59b120640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-30T15:53:55.934 INFO:tasks.ceph.osd.7.smithi204.stderr:2024-05-30T15:53:55.932+0000 7f2c6bf547c0 -1 osd.7 0 log_to_monitors true 2024-05-30T15:53:56.032 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-05-30T15:53:56.032+0000 7f31897607c0 -1 osd.4 0 log_to_monitors true 2024-05-30T15:53:56.089 INFO:tasks.ceph.osd.0.smithi006.stderr:2024-05-30T15:53:56.086+0000 7f93d705b7c0 -1 osd.0 0 log_to_monitors true 2024-05-30T15:53:56.112 INFO:tasks.ceph.osd.2.smithi006.stderr:2024-05-30T15:53:56.110+0000 7f1ebb93d7c0 -1 osd.2 0 log_to_monitors true 2024-05-30T15:53:56.141 INFO:tasks.ceph.osd.6.smithi204.stderr:2024-05-30T15:53:56.140+0000 7f05bebd27c0 -1 osd.6 0 log_to_monitors true 2024-05-30T15:53:56.185 INFO:tasks.ceph.osd.3.smithi006.stderr:2024-05-30T15:53:56.186+0000 7fe0c49d77c0 -1 osd.3 0 log_to_monitors true 2024-05-30T15:53:56.206 INFO:tasks.ceph.osd.1.smithi006.stderr:2024-05-30T15:53:56.206+0000 7f061659a7c0 -1 osd.1 0 log_to_monitors true 2024-05-30T15:53:56.228 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-05-30T15:53:56.228+0000 7fa0b7ebf7c0 -1 osd.5 0 log_to_monitors true 2024-05-30T15:53:59.427 INFO:tasks.ceph.osd.3.smithi006.stderr:2024-05-30T15:53:59.426+0000 7fe0c0980640 -1 osd.3 0 waiting for initial osdmap 2024-05-30T15:53:59.441 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-05-30T15:53:59.440+0000 7f3185709640 -1 osd.4 0 waiting for initial osdmap 2024-05-30T15:53:59.461 INFO:tasks.ceph.osd.6.smithi204.stderr:2024-05-30T15:53:59.460+0000 7f05bb38d640 -1 osd.6 0 waiting for initial osdmap 2024-05-30T15:53:59.462 INFO:tasks.ceph.osd.3.smithi006.stderr:2024-05-30T15:53:59.462+0000 7fe0b87b4640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-30T15:53:59.462 INFO:tasks.ceph.osd.0.smithi006.stderr:2024-05-30T15:53:59.462+0000 7f93d3004640 -1 osd.0 0 waiting for initial osdmap 2024-05-30T15:53:59.463 INFO:tasks.ceph.osd.7.smithi204.stderr:2024-05-30T15:53:59.460+0000 7f2c67efd640 -1 osd.7 0 waiting for initial osdmap 2024-05-30T15:53:59.468 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-05-30T15:53:59.468+0000 7f317d53d640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-30T15:53:59.474 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-05-30T15:53:59.472+0000 7fa0b3e68640 -1 osd.5 0 waiting for initial osdmap 2024-05-30T15:53:59.482 INFO:tasks.ceph.osd.2.smithi006.stderr:2024-05-30T15:53:59.482+0000 7f1eb78e6640 -1 osd.2 0 waiting for initial osdmap 2024-05-30T15:53:59.484 INFO:tasks.ceph.osd.6.smithi204.stderr:2024-05-30T15:53:59.484+0000 7f05b29af640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-30T15:53:59.484 INFO:tasks.ceph.osd.7.smithi204.stderr:2024-05-30T15:53:59.484+0000 7f2c5fd31640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-30T15:53:59.487 INFO:tasks.ceph.osd.1.smithi006.stderr:2024-05-30T15:53:59.486+0000 7f0612d55640 -1 osd.1 0 waiting for initial osdmap 2024-05-30T15:53:59.502 INFO:tasks.ceph.osd.0.smithi006.stderr:2024-05-30T15:53:59.502+0000 7f93cae38640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-30T15:53:59.504 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-05-30T15:53:59.504+0000 7fa0abc9c640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-30T15:53:59.509 INFO:tasks.ceph.osd.1.smithi006.stderr:2024-05-30T15:53:59.510+0000 7f060a377640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-30T15:53:59.518 INFO:tasks.ceph.osd.2.smithi006.stderr:2024-05-30T15:53:59.518+0000 7f1eaf71a640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-30T15:53:59.692 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-30T15:53:59.985 INFO:teuthology.misc.health.smithi006.stdout: 2024-05-30T15:53:59.986 INFO:teuthology.misc.health.smithi006.stdout:{"epoch":12,"fsid":"87f2315a-4430-477b-ae32-1664164ce9e8","created":"2024-05-30T15:53:48.501791+0000","modified":"2024-05-30T15:53:57.295043+0000","last_up_change":"0.000000","last_in_change":"2024-05-30T15:53:51.589084+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c9b54fda-6c4d-49b5-b6f7-9baf01edc4fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"476e6529-66eb-4479-8ca9-b646750fb07c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c8461074-51a0-4209-aed4-e05343440267","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c4775e06-d8e2-44c0-a0c8-9f4e8e21f5cc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"bfc9fd12-63b8-4711-a306-23a17a9aaf0f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e21db988-15e6-488e-8bc1-e5ba0a2f354b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"85a75312-8faf-4279-9301-134fcb19cbd9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f0550e43-5653-4433-8f78-17af5e82316d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T15:54:00.002 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-30T15:54:06.808 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-30T15:54:07.114 INFO:teuthology.misc.health.smithi006.stdout: 2024-05-30T15:54:07.114 INFO:teuthology.misc.health.smithi006.stdout:{"epoch":16,"fsid":"87f2315a-4430-477b-ae32-1664164ce9e8","created":"2024-05-30T15:53:48.501791+0000","modified":"2024-05-30T15:54:03.531712+0000","last_up_change":"2024-05-30T15:54:00.488066+0000","last_in_change":"2024-05-30T15:53:51.589084+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T15:54:01.322096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9b54fda-6c4d-49b5-b6f7-9baf01edc4fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":177258275},{"type":"v1","addr":"172.21.15.6:6801","nonce":177258275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":177258275},{"type":"v1","addr":"172.21.15.6:6803","nonce":177258275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":177258275},{"type":"v1","addr":"172.21.15.6:6807","nonce":177258275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":177258275},{"type":"v1","addr":"172.21.15.6:6805","nonce":177258275}]},"public_addr":"172.21.15.6:6801/177258275","cluster_addr":"172.21.15.6:6803/177258275","heartbeat_back_addr":"172.21.15.6:6807/177258275","heartbeat_front_addr":"172.21.15.6:6805/177258275","state":["exists","up"]},{"osd":1,"uuid":"476e6529-66eb-4479-8ca9-b646750fb07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":1897362715},{"type":"v1","addr":"172.21.15.6:6809","nonce":1897362715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1897362715},{"type":"v1","addr":"172.21.15.6:6811","nonce":1897362715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1897362715},{"type":"v1","addr":"172.21.15.6:6815","nonce":1897362715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1897362715},{"type":"v1","addr":"172.21.15.6:6813","nonce":1897362715}]},"public_addr":"172.21.15.6:6809/1897362715","cluster_addr":"172.21.15.6:6811/1897362715","heartbeat_back_addr":"172.21.15.6:6815/1897362715","heartbeat_front_addr":"172.21.15.6:6813/1897362715","state":["exists","up"]},{"osd":2,"uuid":"c8461074-51a0-4209-aed4-e05343440267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":2704410641},{"type":"v1","addr":"172.21.15.6:6817","nonce":2704410641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2704410641},{"type":"v1","addr":"172.21.15.6:6820","nonce":2704410641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":2704410641},{"type":"v1","addr":"172.21.15.6:6827","nonce":2704410641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2704410641},{"type":"v1","addr":"172.21.15.6:6824","nonce":2704410641}]},"public_addr":"172.21.15.6:6817/2704410641","cluster_addr":"172.21.15.6:6820/2704410641","heartbeat_back_addr":"172.21.15.6:6827/2704410641","heartbeat_front_addr":"172.21.15.6:6824/2704410641","state":["exists","up"]},{"osd":3,"uuid":"c4775e06-d8e2-44c0-a0c8-9f4e8e21f5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":1141406589},{"type":"v1","addr":"172.21.15.6:6821","nonce":1141406589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6823","nonce":1141406589},{"type":"v1","addr":"172.21.15.6:6826","nonce":1141406589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1141406589},{"type":"v1","addr":"172.21.15.6:6831","nonce":1141406589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1141406589},{"type":"v1","addr":"172.21.15.6:6829","nonce":1141406589}]},"public_addr":"172.21.15.6:6821/1141406589","cluster_addr":"172.21.15.6:6826/1141406589","heartbeat_back_addr":"172.21.15.6:6831/1141406589","heartbeat_front_addr":"172.21.15.6:6829/1141406589","state":["exists","up"]},{"osd":4,"uuid":"bfc9fd12-63b8-4711-a306-23a17a9aaf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":986196749},{"type":"v1","addr":"172.21.15.204:6808","nonce":986196749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":986196749},{"type":"v1","addr":"172.21.15.204:6811","nonce":986196749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":986196749},{"type":"v1","addr":"172.21.15.204:6815","nonce":986196749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":986196749},{"type":"v1","addr":"172.21.15.204:6813","nonce":986196749}]},"public_addr":"172.21.15.204:6808/986196749","cluster_addr":"172.21.15.204:6811/986196749","heartbeat_back_addr":"172.21.15.204:6815/986196749","heartbeat_front_addr":"172.21.15.204:6813/986196749","state":["exists","up"]},{"osd":5,"uuid":"e21db988-15e6-488e-8bc1-e5ba0a2f354b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2677861390},{"type":"v1","addr":"172.21.15.204:6817","nonce":2677861390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2677861390},{"type":"v1","addr":"172.21.15.204:6819","nonce":2677861390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2677861390},{"type":"v1","addr":"172.21.15.204:6824","nonce":2677861390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2677861390},{"type":"v1","addr":"172.21.15.204:6821","nonce":2677861390}]},"public_addr":"172.21.15.204:6817/2677861390","cluster_addr":"172.21.15.204:6819/2677861390","heartbeat_back_addr":"172.21.15.204:6824/2677861390","heartbeat_front_addr":"172.21.15.204:6821/2677861390","state":["exists","up"]},{"osd":6,"uuid":"85a75312-8faf-4279-9301-134fcb19cbd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3687190393},{"type":"v1","addr":"172.21.15.204:6801","nonce":3687190393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3687190393},{"type":"v1","addr":"172.21.15.204:6803","nonce":3687190393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3687190393},{"type":"v1","addr":"172.21.15.204:6809","nonce":3687190393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3687190393},{"type":"v1","addr":"172.21.15.204:6805","nonce":3687190393}]},"public_addr":"172.21.15.204:6801/3687190393","cluster_addr":"172.21.15.204:6803/3687190393","heartbeat_back_addr":"172.21.15.204:6809/3687190393","heartbeat_front_addr":"172.21.15.204:6805/3687190393","state":["exists","up"]},{"osd":7,"uuid":"f0550e43-5653-4433-8f78-17af5e82316d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":3156311527},{"type":"v1","addr":"172.21.15.204:6825","nonce":3156311527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3156311527},{"type":"v1","addr":"172.21.15.204:6827","nonce":3156311527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3156311527},{"type":"v1","addr":"172.21.15.204:6831","nonce":3156311527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3156311527},{"type":"v1","addr":"172.21.15.204:6829","nonce":3156311527}]},"public_addr":"172.21.15.204:6825/3156311527","cluster_addr":"172.21.15.204:6827/3156311527","heartbeat_back_addr":"172.21.15.204:6831/3156311527","heartbeat_front_addr":"172.21.15.204:6829/3156311527","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:57.123142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:57.210132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:57.107006+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:57.217713+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:57.079193+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:57.281733+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:57.111373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:56.985566+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T15:54:07.133 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-30T15:54:07.133 INFO:tasks.ceph:Creating RBD pool 2024-05-30T15:54:07.133 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --cluster ceph osd pool create rbd 8 2024-05-30T15:54:07.554 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'rbd' created 2024-05-30T15:54:07.579 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-05-30T15:54:08.557 INFO:teuthology.orchestra.run.smithi006.stderr:enabled application 'rbd' on pool 'rbd' 2024-05-30T15:54:08.586 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-30T15:54:08.587 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-30T15:54:08.587 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-30T15:54:08.587 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-30T15:54:08.587 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-30T15:54:08.935 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:54:08.954 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2105440097},{"type":"v1","addr":"172.21.15.204:6833","nonce":2105440097}]},"active_addr":"172.21.15.204:6833/2105440097","active_change":"2024-05-30T15:53:53.269649+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4103,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.204:0","nonce":217484880}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.204:0","nonce":2614062754}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.204:0","nonce":2628201073}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.204:0","nonce":2237984916}]}]} 2024-05-30T15:54:08.958 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-30T15:54:08.958 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-30T15:54:08.958 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T15:54:09.257 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:54:09.257 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":18,"fsid":"87f2315a-4430-477b-ae32-1664164ce9e8","created":"2024-05-30T15:53:48.501791+0000","modified":"2024-05-30T15:54:08.557335+0000","last_up_change":"2024-05-30T15:54:00.488066+0000","last_in_change":"2024-05-30T15:53:51.589084+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T15:54:01.322096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-30T15:54:07.446935+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9b54fda-6c4d-49b5-b6f7-9baf01edc4fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":177258275},{"type":"v1","addr":"172.21.15.6:6801","nonce":177258275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":177258275},{"type":"v1","addr":"172.21.15.6:6803","nonce":177258275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":177258275},{"type":"v1","addr":"172.21.15.6:6807","nonce":177258275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":177258275},{"type":"v1","addr":"172.21.15.6:6805","nonce":177258275}]},"public_addr":"172.21.15.6:6801/177258275","cluster_addr":"172.21.15.6:6803/177258275","heartbeat_back_addr":"172.21.15.6:6807/177258275","heartbeat_front_addr":"172.21.15.6:6805/177258275","state":["exists","up"]},{"osd":1,"uuid":"476e6529-66eb-4479-8ca9-b646750fb07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":1897362715},{"type":"v1","addr":"172.21.15.6:6809","nonce":1897362715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1897362715},{"type":"v1","addr":"172.21.15.6:6811","nonce":1897362715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1897362715},{"type":"v1","addr":"172.21.15.6:6815","nonce":1897362715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1897362715},{"type":"v1","addr":"172.21.15.6:6813","nonce":1897362715}]},"public_addr":"172.21.15.6:6809/1897362715","cluster_addr":"172.21.15.6:6811/1897362715","heartbeat_back_addr":"172.21.15.6:6815/1897362715","heartbeat_front_addr":"172.21.15.6:6813/1897362715","state":["exists","up"]},{"osd":2,"uuid":"c8461074-51a0-4209-aed4-e05343440267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":2704410641},{"type":"v1","addr":"172.21.15.6:6817","nonce":2704410641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2704410641},{"type":"v1","addr":"172.21.15.6:6820","nonce":2704410641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":2704410641},{"type":"v1","addr":"172.21.15.6:6827","nonce":2704410641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2704410641},{"type":"v1","addr":"172.21.15.6:6824","nonce":2704410641}]},"public_addr":"172.21.15.6:6817/2704410641","cluster_addr":"172.21.15.6:6820/2704410641","heartbeat_back_addr":"172.21.15.6:6827/2704410641","heartbeat_front_addr":"172.21.15.6:6824/2704410641","state":["exists","up"]},{"osd":3,"uuid":"c4775e06-d8e2-44c0-a0c8-9f4e8e21f5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":1141406589},{"type":"v1","addr":"172.21.15.6:6821","nonce":1141406589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6823","nonce":1141406589},{"type":"v1","addr":"172.21.15.6:6826","nonce":1141406589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1141406589},{"type":"v1","addr":"172.21.15.6:6831","nonce":1141406589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1141406589},{"type":"v1","addr":"172.21.15.6:6829","nonce":1141406589}]},"public_addr":"172.21.15.6:6821/1141406589","cluster_addr":"172.21.15.6:6826/1141406589","heartbeat_back_addr":"172.21.15.6:6831/1141406589","heartbeat_front_addr":"172.21.15.6:6829/1141406589","state":["exists","up"]},{"osd":4,"uuid":"bfc9fd12-63b8-4711-a306-23a17a9aaf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":986196749},{"type":"v1","addr":"172.21.15.204:6808","nonce":986196749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":986196749},{"type":"v1","addr":"172.21.15.204:6811","nonce":986196749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":986196749},{"type":"v1","addr":"172.21.15.204:6815","nonce":986196749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":986196749},{"type":"v1","addr":"172.21.15.204:6813","nonce":986196749}]},"public_addr":"172.21.15.204:6808/986196749","cluster_addr":"172.21.15.204:6811/986196749","heartbeat_back_addr":"172.21.15.204:6815/986196749","heartbeat_front_addr":"172.21.15.204:6813/986196749","state":["exists","up"]},{"osd":5,"uuid":"e21db988-15e6-488e-8bc1-e5ba0a2f354b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2677861390},{"type":"v1","addr":"172.21.15.204:6817","nonce":2677861390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2677861390},{"type":"v1","addr":"172.21.15.204:6819","nonce":2677861390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2677861390},{"type":"v1","addr":"172.21.15.204:6824","nonce":2677861390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2677861390},{"type":"v1","addr":"172.21.15.204:6821","nonce":2677861390}]},"public_addr":"172.21.15.204:6817/2677861390","cluster_addr":"172.21.15.204:6819/2677861390","heartbeat_back_addr":"172.21.15.204:6824/2677861390","heartbeat_front_addr":"172.21.15.204:6821/2677861390","state":["exists","up"]},{"osd":6,"uuid":"85a75312-8faf-4279-9301-134fcb19cbd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3687190393},{"type":"v1","addr":"172.21.15.204:6801","nonce":3687190393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3687190393},{"type":"v1","addr":"172.21.15.204:6803","nonce":3687190393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3687190393},{"type":"v1","addr":"172.21.15.204:6809","nonce":3687190393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3687190393},{"type":"v1","addr":"172.21.15.204:6805","nonce":3687190393}]},"public_addr":"172.21.15.204:6801/3687190393","cluster_addr":"172.21.15.204:6803/3687190393","heartbeat_back_addr":"172.21.15.204:6809/3687190393","heartbeat_front_addr":"172.21.15.204:6805/3687190393","state":["exists","up"]},{"osd":7,"uuid":"f0550e43-5653-4433-8f78-17af5e82316d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":3156311527},{"type":"v1","addr":"172.21.15.204:6825","nonce":3156311527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3156311527},{"type":"v1","addr":"172.21.15.204:6827","nonce":3156311527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3156311527},{"type":"v1","addr":"172.21.15.204:6831","nonce":3156311527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3156311527},{"type":"v1","addr":"172.21.15.204:6829","nonce":3156311527}]},"public_addr":"172.21.15.204:6825/3156311527","cluster_addr":"172.21.15.204:6827/3156311527","heartbeat_back_addr":"172.21.15.204:6831/3156311527","heartbeat_front_addr":"172.21.15.204:6829/3156311527","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:57.123142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:57.210132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:57.107006+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:57.217713+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:57.079193+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:57.281733+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:57.111373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:56.985566+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T15:54:09.276 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-30T15:54:09.277 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T15:54:09.573 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:54:09.573 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":18,"fsid":"87f2315a-4430-477b-ae32-1664164ce9e8","created":"2024-05-30T15:53:48.501791+0000","modified":"2024-05-30T15:54:08.557335+0000","last_up_change":"2024-05-30T15:54:00.488066+0000","last_in_change":"2024-05-30T15:53:51.589084+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T15:54:01.322096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-30T15:54:07.446935+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9b54fda-6c4d-49b5-b6f7-9baf01edc4fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":177258275},{"type":"v1","addr":"172.21.15.6:6801","nonce":177258275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":177258275},{"type":"v1","addr":"172.21.15.6:6803","nonce":177258275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":177258275},{"type":"v1","addr":"172.21.15.6:6807","nonce":177258275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":177258275},{"type":"v1","addr":"172.21.15.6:6805","nonce":177258275}]},"public_addr":"172.21.15.6:6801/177258275","cluster_addr":"172.21.15.6:6803/177258275","heartbeat_back_addr":"172.21.15.6:6807/177258275","heartbeat_front_addr":"172.21.15.6:6805/177258275","state":["exists","up"]},{"osd":1,"uuid":"476e6529-66eb-4479-8ca9-b646750fb07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":1897362715},{"type":"v1","addr":"172.21.15.6:6809","nonce":1897362715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1897362715},{"type":"v1","addr":"172.21.15.6:6811","nonce":1897362715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1897362715},{"type":"v1","addr":"172.21.15.6:6815","nonce":1897362715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1897362715},{"type":"v1","addr":"172.21.15.6:6813","nonce":1897362715}]},"public_addr":"172.21.15.6:6809/1897362715","cluster_addr":"172.21.15.6:6811/1897362715","heartbeat_back_addr":"172.21.15.6:6815/1897362715","heartbeat_front_addr":"172.21.15.6:6813/1897362715","state":["exists","up"]},{"osd":2,"uuid":"c8461074-51a0-4209-aed4-e05343440267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":2704410641},{"type":"v1","addr":"172.21.15.6:6817","nonce":2704410641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2704410641},{"type":"v1","addr":"172.21.15.6:6820","nonce":2704410641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":2704410641},{"type":"v1","addr":"172.21.15.6:6827","nonce":2704410641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2704410641},{"type":"v1","addr":"172.21.15.6:6824","nonce":2704410641}]},"public_addr":"172.21.15.6:6817/2704410641","cluster_addr":"172.21.15.6:6820/2704410641","heartbeat_back_addr":"172.21.15.6:6827/2704410641","heartbeat_front_addr":"172.21.15.6:6824/2704410641","state":["exists","up"]},{"osd":3,"uuid":"c4775e06-d8e2-44c0-a0c8-9f4e8e21f5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":1141406589},{"type":"v1","addr":"172.21.15.6:6821","nonce":1141406589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6823","nonce":1141406589},{"type":"v1","addr":"172.21.15.6:6826","nonce":1141406589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1141406589},{"type":"v1","addr":"172.21.15.6:6831","nonce":1141406589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1141406589},{"type":"v1","addr":"172.21.15.6:6829","nonce":1141406589}]},"public_addr":"172.21.15.6:6821/1141406589","cluster_addr":"172.21.15.6:6826/1141406589","heartbeat_back_addr":"172.21.15.6:6831/1141406589","heartbeat_front_addr":"172.21.15.6:6829/1141406589","state":["exists","up"]},{"osd":4,"uuid":"bfc9fd12-63b8-4711-a306-23a17a9aaf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":986196749},{"type":"v1","addr":"172.21.15.204:6808","nonce":986196749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":986196749},{"type":"v1","addr":"172.21.15.204:6811","nonce":986196749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":986196749},{"type":"v1","addr":"172.21.15.204:6815","nonce":986196749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":986196749},{"type":"v1","addr":"172.21.15.204:6813","nonce":986196749}]},"public_addr":"172.21.15.204:6808/986196749","cluster_addr":"172.21.15.204:6811/986196749","heartbeat_back_addr":"172.21.15.204:6815/986196749","heartbeat_front_addr":"172.21.15.204:6813/986196749","state":["exists","up"]},{"osd":5,"uuid":"e21db988-15e6-488e-8bc1-e5ba0a2f354b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2677861390},{"type":"v1","addr":"172.21.15.204:6817","nonce":2677861390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2677861390},{"type":"v1","addr":"172.21.15.204:6819","nonce":2677861390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2677861390},{"type":"v1","addr":"172.21.15.204:6824","nonce":2677861390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2677861390},{"type":"v1","addr":"172.21.15.204:6821","nonce":2677861390}]},"public_addr":"172.21.15.204:6817/2677861390","cluster_addr":"172.21.15.204:6819/2677861390","heartbeat_back_addr":"172.21.15.204:6824/2677861390","heartbeat_front_addr":"172.21.15.204:6821/2677861390","state":["exists","up"]},{"osd":6,"uuid":"85a75312-8faf-4279-9301-134fcb19cbd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3687190393},{"type":"v1","addr":"172.21.15.204:6801","nonce":3687190393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3687190393},{"type":"v1","addr":"172.21.15.204:6803","nonce":3687190393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3687190393},{"type":"v1","addr":"172.21.15.204:6809","nonce":3687190393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3687190393},{"type":"v1","addr":"172.21.15.204:6805","nonce":3687190393}]},"public_addr":"172.21.15.204:6801/3687190393","cluster_addr":"172.21.15.204:6803/3687190393","heartbeat_back_addr":"172.21.15.204:6809/3687190393","heartbeat_front_addr":"172.21.15.204:6805/3687190393","state":["exists","up"]},{"osd":7,"uuid":"f0550e43-5653-4433-8f78-17af5e82316d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":3156311527},{"type":"v1","addr":"172.21.15.204:6825","nonce":3156311527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3156311527},{"type":"v1","addr":"172.21.15.204:6827","nonce":3156311527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3156311527},{"type":"v1","addr":"172.21.15.204:6831","nonce":3156311527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3156311527},{"type":"v1","addr":"172.21.15.204:6829","nonce":3156311527}]},"public_addr":"172.21.15.204:6825/3156311527","cluster_addr":"172.21.15.204:6827/3156311527","heartbeat_back_addr":"172.21.15.204:6831/3156311527","heartbeat_front_addr":"172.21.15.204:6829/3156311527","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:57.123142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:57.210132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:57.107006+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:57.217713+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:57.079193+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:57.281733+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:57.111373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:56.985566+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T15:54:09.593 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-30T15:54:09.593 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-30T15:54:09.594 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-30T15:54:09.594 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-30T15:54:09.594 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-30T15:54:09.595 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-30T15:54:09.595 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-30T15:54:09.595 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-30T15:54:09.850 INFO:teuthology.orchestra.run.smithi006.stdout:55834574852 2024-05-30T15:54:09.851 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-30T15:54:09.855 INFO:teuthology.orchestra.run.smithi006.stdout:55834574852 2024-05-30T15:54:09.855 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-30T15:54:09.857 INFO:teuthology.orchestra.run.smithi006.stdout:55834574853 2024-05-30T15:54:09.858 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-30T15:54:09.862 INFO:teuthology.orchestra.run.smithi006.stdout:55834574852 2024-05-30T15:54:09.863 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-30T15:54:09.868 INFO:teuthology.orchestra.run.smithi006.stdout:55834574852 2024-05-30T15:54:09.868 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-30T15:54:09.870 INFO:teuthology.orchestra.run.smithi006.stdout:55834574852 2024-05-30T15:54:09.870 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-30T15:54:09.872 INFO:teuthology.orchestra.run.smithi006.stdout:55834574853 2024-05-30T15:54:09.872 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-30T15:54:09.907 INFO:teuthology.orchestra.run.smithi006.stdout:55834574853 2024-05-30T15:54:09.907 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-30T15:54:10.368 INFO:teuthology.orchestra.run.smithi006.stdout:55834574852 2024-05-30T15:54:10.389 INFO:teuthology.orchestra.run.smithi006.stdout:55834574851 2024-05-30T15:54:10.391 INFO:teuthology.orchestra.run.smithi006.stdout:55834574851 2024-05-30T15:54:10.392 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.7 2024-05-30T15:54:10.392 INFO:teuthology.orchestra.run.smithi006.stdout:55834574851 2024-05-30T15:54:10.412 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.3 2024-05-30T15:54:10.417 INFO:teuthology.orchestra.run.smithi006.stdout:55834574851 2024-05-30T15:54:10.420 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.6 2024-05-30T15:54:10.427 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.5 2024-05-30T15:54:10.435 INFO:teuthology.orchestra.run.smithi006.stdout:55834574851 2024-05-30T15:54:10.436 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.4 2024-05-30T15:54:10.439 INFO:teuthology.orchestra.run.smithi006.stdout:55834574852 2024-05-30T15:54:10.455 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.0 2024-05-30T15:54:10.458 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.2 2024-05-30T15:54:10.464 INFO:teuthology.orchestra.run.smithi006.stdout:55834574852 2024-05-30T15:54:10.481 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.1 2024-05-30T15:54:11.393 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-30T15:54:11.413 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-30T15:54:11.421 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-30T15:54:11.427 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-30T15:54:11.437 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-30T15:54:11.456 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-30T15:54:11.458 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-30T15:54:11.482 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-30T15:54:11.901 INFO:teuthology.orchestra.run.smithi006.stdout:55834574853 2024-05-30T15:54:11.923 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.7 2024-05-30T15:54:11.924 DEBUG:teuthology.parallel:result is None 2024-05-30T15:54:11.964 INFO:teuthology.orchestra.run.smithi006.stdout:55834574852 2024-05-30T15:54:11.971 INFO:teuthology.orchestra.run.smithi006.stdout:55834574852 2024-05-30T15:54:11.980 INFO:teuthology.orchestra.run.smithi006.stdout:55834574852 2024-05-30T15:54:11.981 INFO:teuthology.orchestra.run.smithi006.stdout:55834574852 2024-05-30T15:54:11.989 INFO:teuthology.orchestra.run.smithi006.stdout:55834574852 2024-05-30T15:54:11.998 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.5 2024-05-30T15:54:11.998 DEBUG:teuthology.parallel:result is None 2024-05-30T15:54:12.006 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.3 2024-05-30T15:54:12.007 DEBUG:teuthology.parallel:result is None 2024-05-30T15:54:12.007 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.0 2024-05-30T15:54:12.007 DEBUG:teuthology.parallel:result is None 2024-05-30T15:54:12.014 INFO:teuthology.orchestra.run.smithi006.stdout:55834574853 2024-05-30T15:54:12.014 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.6 2024-05-30T15:54:12.015 DEBUG:teuthology.parallel:result is None 2024-05-30T15:54:12.015 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.4 2024-05-30T15:54:12.015 DEBUG:teuthology.parallel:result is None 2024-05-30T15:54:12.020 INFO:teuthology.orchestra.run.smithi006.stdout:55834574853 2024-05-30T15:54:12.032 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.1 2024-05-30T15:54:12.032 DEBUG:teuthology.parallel:result is None 2024-05-30T15:54:12.038 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.2 2024-05-30T15:54:12.038 DEBUG:teuthology.parallel:result is None 2024-05-30T15:54:12.038 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-30T15:54:12.039 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T15:54:12.354 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:54:12.354 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-05-30T15:54:12.371 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-05-30T15:54:11.282721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218760,"kb_used_data":2568,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755960,"statfs":{"total":773094113280,"available":772870103040,"internally_reserved":0,"allocated":2629632,"data_stored":1639264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001149"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-30T15:54:09.645125+0000","last_change":"2024-05-30T15:54:08.578053+0000","last_active":"2024-05-30T15:54:09.645125+0000","last_peered":"2024-05-30T15:54:09.645125+0000","last_clean":"2024-05-30T15:54:09.645125+0000","last_became_active":"2024-05-30T15:54:08.577191+0000","last_became_peered":"2024-05-30T15:54:08.577191+0000","last_unstale":"2024-05-30T15:54:09.645125+0000","last_undegraded":"2024-05-30T15:54:09.645125+0000","last_fullsized":"2024-05-30T15:54:09.645125+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:38:36.674998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-30T15:54:09.581089+0000","last_change":"2024-05-30T15:54:08.574181+0000","last_active":"2024-05-30T15:54:09.581089+0000","last_peered":"2024-05-30T15:54:09.581089+0000","last_clean":"2024-05-30T15:54:09.581089+0000","last_became_active":"2024-05-30T15:54:08.573754+0000","last_became_peered":"2024-05-30T15:54:08.573754+0000","last_unstale":"2024-05-30T15:54:09.581089+0000","last_undegraded":"2024-05-30T15:54:09.581089+0000","last_fullsized":"2024-05-30T15:54:09.581089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:35:01.844361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-30T15:54:09.645003+0000","last_change":"2024-05-30T15:54:08.577379+0000","last_active":"2024-05-30T15:54:09.645003+0000","last_peered":"2024-05-30T15:54:09.645003+0000","last_clean":"2024-05-30T15:54:09.645003+0000","last_became_active":"2024-05-30T15:54:08.576779+0000","last_became_peered":"2024-05-30T15:54:08.576779+0000","last_unstale":"2024-05-30T15:54:09.645003+0000","last_undegraded":"2024-05-30T15:54:09.645003+0000","last_fullsized":"2024-05-30T15:54:09.645003+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:09:01.046338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-30T15:54:09.580979+0000","last_change":"2024-05-30T15:54:08.568214+0000","last_active":"2024-05-30T15:54:09.580979+0000","last_peered":"2024-05-30T15:54:09.580979+0000","last_clean":"2024-05-30T15:54:09.580979+0000","last_became_active":"2024-05-30T15:54:08.567900+0000","last_became_peered":"2024-05-30T15:54:08.567900+0000","last_unstale":"2024-05-30T15:54:09.580979+0000","last_undegraded":"2024-05-30T15:54:09.580979+0000","last_fullsized":"2024-05-30T15:54:09.580979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:02:38.154765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-30T15:54:09.645070+0000","last_change":"2024-05-30T15:54:08.575969+0000","last_active":"2024-05-30T15:54:09.645070+0000","last_peered":"2024-05-30T15:54:09.645070+0000","last_clean":"2024-05-30T15:54:09.645070+0000","last_became_active":"2024-05-30T15:54:08.574817+0000","last_became_peered":"2024-05-30T15:54:08.574817+0000","last_unstale":"2024-05-30T15:54:09.645070+0000","last_undegraded":"2024-05-30T15:54:09.645070+0000","last_fullsized":"2024-05-30T15:54:09.645070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:39:30.889837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-30T15:54:09.643925+0000","last_change":"2024-05-30T15:54:08.745221+0000","last_active":"2024-05-30T15:54:09.643925+0000","last_peered":"2024-05-30T15:54:09.643925+0000","last_clean":"2024-05-30T15:54:09.643925+0000","last_became_active":"2024-05-30T15:54:08.744898+0000","last_became_peered":"2024-05-30T15:54:08.744898+0000","last_unstale":"2024-05-30T15:54:09.643925+0000","last_undegraded":"2024-05-30T15:54:09.643925+0000","last_fullsized":"2024-05-30T15:54:09.643925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:38:33.683752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-30T15:54:09.644897+0000","last_change":"2024-05-30T15:54:08.575348+0000","last_active":"2024-05-30T15:54:09.644897+0000","last_peered":"2024-05-30T15:54:09.644897+0000","last_clean":"2024-05-30T15:54:09.644897+0000","last_became_active":"2024-05-30T15:54:08.574645+0000","last_became_peered":"2024-05-30T15:54:08.574645+0000","last_unstale":"2024-05-30T15:54:09.644897+0000","last_undegraded":"2024-05-30T15:54:09.644897+0000","last_fullsized":"2024-05-30T15:54:09.644897+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:41:49.462483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-30T15:54:09.645059+0000","last_change":"2024-05-30T15:54:08.578526+0000","last_active":"2024-05-30T15:54:09.645059+0000","last_peered":"2024-05-30T15:54:09.645059+0000","last_clean":"2024-05-30T15:54:09.645059+0000","last_became_active":"2024-05-30T15:54:08.577556+0000","last_became_peered":"2024-05-30T15:54:08.577556+0000","last_unstale":"2024-05-30T15:54:09.645059+0000","last_undegraded":"2024-05-30T15:54:09.645059+0000","last_fullsized":"2024-05-30T15:54:09.645059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:36:52.339688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":108,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-30T15:54:09.645041+0000","last_change":"2024-05-30T15:54:03.031292+0000","last_active":"2024-05-30T15:54:09.645041+0000","last_peered":"2024-05-30T15:54:09.645041+0000","last_clean":"2024-05-30T15:54:09.645041+0000","last_became_active":"2024-05-30T15:54:03.029214+0000","last_became_peered":"2024-05-30T15:54:03.029214+0000","last_unstale":"2024-05-30T15:54:09.645041+0000","last_undegraded":"2024-05-30T15:54:09.645041+0000","last_fullsized":"2024-05-30T15:54:09.645041+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:01.512251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:01.512251+0000","last_clean_scrub_stamp":"2024-05-30T15:54:01.512251+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:40:07.389706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":647684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":647684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T15:54:12.372 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T15:54:12.665 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:54:12.666 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-05-30T15:54:12.682 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-05-30T15:54:11.282721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218760,"kb_used_data":2568,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755960,"statfs":{"total":773094113280,"available":772870103040,"internally_reserved":0,"allocated":2629632,"data_stored":1639264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001149"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-30T15:54:09.645125+0000","last_change":"2024-05-30T15:54:08.578053+0000","last_active":"2024-05-30T15:54:09.645125+0000","last_peered":"2024-05-30T15:54:09.645125+0000","last_clean":"2024-05-30T15:54:09.645125+0000","last_became_active":"2024-05-30T15:54:08.577191+0000","last_became_peered":"2024-05-30T15:54:08.577191+0000","last_unstale":"2024-05-30T15:54:09.645125+0000","last_undegraded":"2024-05-30T15:54:09.645125+0000","last_fullsized":"2024-05-30T15:54:09.645125+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:38:36.674998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-30T15:54:09.581089+0000","last_change":"2024-05-30T15:54:08.574181+0000","last_active":"2024-05-30T15:54:09.581089+0000","last_peered":"2024-05-30T15:54:09.581089+0000","last_clean":"2024-05-30T15:54:09.581089+0000","last_became_active":"2024-05-30T15:54:08.573754+0000","last_became_peered":"2024-05-30T15:54:08.573754+0000","last_unstale":"2024-05-30T15:54:09.581089+0000","last_undegraded":"2024-05-30T15:54:09.581089+0000","last_fullsized":"2024-05-30T15:54:09.581089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:35:01.844361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-30T15:54:09.645003+0000","last_change":"2024-05-30T15:54:08.577379+0000","last_active":"2024-05-30T15:54:09.645003+0000","last_peered":"2024-05-30T15:54:09.645003+0000","last_clean":"2024-05-30T15:54:09.645003+0000","last_became_active":"2024-05-30T15:54:08.576779+0000","last_became_peered":"2024-05-30T15:54:08.576779+0000","last_unstale":"2024-05-30T15:54:09.645003+0000","last_undegraded":"2024-05-30T15:54:09.645003+0000","last_fullsized":"2024-05-30T15:54:09.645003+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:09:01.046338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-30T15:54:09.580979+0000","last_change":"2024-05-30T15:54:08.568214+0000","last_active":"2024-05-30T15:54:09.580979+0000","last_peered":"2024-05-30T15:54:09.580979+0000","last_clean":"2024-05-30T15:54:09.580979+0000","last_became_active":"2024-05-30T15:54:08.567900+0000","last_became_peered":"2024-05-30T15:54:08.567900+0000","last_unstale":"2024-05-30T15:54:09.580979+0000","last_undegraded":"2024-05-30T15:54:09.580979+0000","last_fullsized":"2024-05-30T15:54:09.580979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:02:38.154765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-30T15:54:09.645070+0000","last_change":"2024-05-30T15:54:08.575969+0000","last_active":"2024-05-30T15:54:09.645070+0000","last_peered":"2024-05-30T15:54:09.645070+0000","last_clean":"2024-05-30T15:54:09.645070+0000","last_became_active":"2024-05-30T15:54:08.574817+0000","last_became_peered":"2024-05-30T15:54:08.574817+0000","last_unstale":"2024-05-30T15:54:09.645070+0000","last_undegraded":"2024-05-30T15:54:09.645070+0000","last_fullsized":"2024-05-30T15:54:09.645070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:39:30.889837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-30T15:54:09.643925+0000","last_change":"2024-05-30T15:54:08.745221+0000","last_active":"2024-05-30T15:54:09.643925+0000","last_peered":"2024-05-30T15:54:09.643925+0000","last_clean":"2024-05-30T15:54:09.643925+0000","last_became_active":"2024-05-30T15:54:08.744898+0000","last_became_peered":"2024-05-30T15:54:08.744898+0000","last_unstale":"2024-05-30T15:54:09.643925+0000","last_undegraded":"2024-05-30T15:54:09.643925+0000","last_fullsized":"2024-05-30T15:54:09.643925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:38:33.683752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-30T15:54:09.644897+0000","last_change":"2024-05-30T15:54:08.575348+0000","last_active":"2024-05-30T15:54:09.644897+0000","last_peered":"2024-05-30T15:54:09.644897+0000","last_clean":"2024-05-30T15:54:09.644897+0000","last_became_active":"2024-05-30T15:54:08.574645+0000","last_became_peered":"2024-05-30T15:54:08.574645+0000","last_unstale":"2024-05-30T15:54:09.644897+0000","last_undegraded":"2024-05-30T15:54:09.644897+0000","last_fullsized":"2024-05-30T15:54:09.644897+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:41:49.462483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-30T15:54:09.645059+0000","last_change":"2024-05-30T15:54:08.578526+0000","last_active":"2024-05-30T15:54:09.645059+0000","last_peered":"2024-05-30T15:54:09.645059+0000","last_clean":"2024-05-30T15:54:09.645059+0000","last_became_active":"2024-05-30T15:54:08.577556+0000","last_became_peered":"2024-05-30T15:54:08.577556+0000","last_unstale":"2024-05-30T15:54:09.645059+0000","last_undegraded":"2024-05-30T15:54:09.645059+0000","last_fullsized":"2024-05-30T15:54:09.645059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:36:52.339688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":108,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-30T15:54:09.645041+0000","last_change":"2024-05-30T15:54:03.031292+0000","last_active":"2024-05-30T15:54:09.645041+0000","last_peered":"2024-05-30T15:54:09.645041+0000","last_clean":"2024-05-30T15:54:09.645041+0000","last_became_active":"2024-05-30T15:54:03.029214+0000","last_became_peered":"2024-05-30T15:54:03.029214+0000","last_unstale":"2024-05-30T15:54:09.645041+0000","last_undegraded":"2024-05-30T15:54:09.645041+0000","last_fullsized":"2024-05-30T15:54:09.645041+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:01.512251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:01.512251+0000","last_clean_scrub_stamp":"2024-05-30T15:54:01.512251+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:40:07.389706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":647684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":647684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T15:54:12.683 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-30T15:54:12.683 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-30T15:54:12.683 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-30T15:54:12.683 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-30T15:54:12.993 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T15:54:12.993 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-30T15:54:13.011 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-30T15:54:13.011 INFO:teuthology.run_tasks:Running task openssl_keys... 2024-05-30T15:54:13.023 INFO:teuthology.run_tasks:Running task rgw... 2024-05-30T15:54:13.086 DEBUG:tasks.rgw:config is {'client.0': {'valgrind': ['--tool=memcheck', '--max-threads=1024']}} 2024-05-30T15:54:13.086 DEBUG:tasks.rgw:client list is dict_keys(['client.0']) 2024-05-30T15:54:13.087 INFO:tasks.rgw:Creating data pools 2024-05-30T15:54:13.087 DEBUG:tasks.rgw:Obtaining remote for client client.0 2024-05-30T15:54:13.087 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph osd erasure-code-profile set client.0 k=2 m=1 crush-failure-domain=osd --cluster ceph 2024-05-30T15:54:13.617 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph osd pool create default.rgw.buckets.data 64 64 erasure client.0 --cluster ceph 2024-05-30T15:54:15.608 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'default.rgw.buckets.data' created 2024-05-30T15:54:15.643 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster ceph 2024-05-30T15:54:16.611 INFO:teuthology.orchestra.run.smithi006.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2024-05-30T15:54:16.679 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster ceph 2024-05-30T15:54:17.614 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'default.rgw.buckets.index' created 2024-05-30T15:54:17.649 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster ceph 2024-05-30T15:54:18.619 INFO:teuthology.orchestra.run.smithi006.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2024-05-30T15:54:18.661 DEBUG:tasks.rgw:Pools created 2024-05-30T15:54:18.661 INFO:tasks.rgw:Configuring compression type = random 2024-05-30T15:54:18.661 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2024-05-30T15:54:18.661 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2024-05-30T15:54:18.662 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2024-05-30T15:54:18.701 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:54:18.701 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:54:20.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.922+0000 7fd78e4c0640 0 -- 172.21.15.6:0/4020014690 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x558c9d082400 msgr2=0x558c9d084870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:54:20.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.930+0000 7fd78ecc1640 0 -- 172.21.15.6:0/1572251780 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x558c9d08a1e0 msgr2=0x558c9d08c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:54:20.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.934+0000 7fd7911b0980 20 rados->read ofs=0 len=0 2024-05-30T15:54:20.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.938+0000 7fd7911b0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:54:20.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.938+0000 7fd7911b0980 20 realm 2024-05-30T15:54:20.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.938+0000 7fd7911b0980 20 rados->read ofs=0 len=0 2024-05-30T15:54:20.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.938+0000 7fd7911b0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:54:20.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.938+0000 7fd7911b0980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T15:54:20.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.938+0000 7fd7911b0980 20 rados->read ofs=0 len=0 2024-05-30T15:54:20.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.938+0000 7fd7911b0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:54:20.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.938+0000 7fd7911b0980 20 rados->read ofs=0 len=0 2024-05-30T15:54:20.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.938+0000 7fd7911b0980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:54:20.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.938+0000 7fd7911b0980 20 rados->read ofs=0 len=0 2024-05-30T15:54:20.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.938+0000 7fd7911b0980 20 rados_obj.operate() r=0 bl.length=993 2024-05-30T15:54:20.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.938+0000 7fd7911b0980 20 searching for the correct realm 2024-05-30T15:54:20.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.966+0000 7fd7911b0980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T15:54:20.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.966+0000 7fd7911b0980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T15:54:20.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.966+0000 7fd7911b0980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T15:54:20.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.966+0000 7fd7911b0980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T15:54:20.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.966+0000 7fd7911b0980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T15:54:20.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.966+0000 7fd7911b0980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T15:54:20.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.966+0000 7fd7911b0980 20 rados->read ofs=0 len=0 2024-05-30T15:54:20.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.966+0000 7fd7911b0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:54:20.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.966+0000 7fd7911b0980 20 rados->read ofs=0 len=0 2024-05-30T15:54:20.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.966+0000 7fd7911b0980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:54:20.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.966+0000 7fd7911b0980 20 rados->read ofs=0 len=0 2024-05-30T15:54:20.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.966+0000 7fd7911b0980 20 rados_obj.operate() r=0 bl.length=436 2024-05-30T15:54:20.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.966+0000 7fd7911b0980 20 zone default found 2024-05-30T15:54:20.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.966+0000 7fd7911b0980 4 Realm: () 2024-05-30T15:54:20.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.966+0000 7fd7911b0980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T15:54:20.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.966+0000 7fd7911b0980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T15:54:20.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.966+0000 7fd7911b0980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T15:54:20.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.966+0000 7fd7911b0980 20 zonegroup default 2024-05-30T15:54:20.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.966+0000 7fd7911b0980 20 rados->read ofs=0 len=0 2024-05-30T15:54:20.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.970+0000 7fd7911b0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:54:20.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:20.970+0000 7fd7911b0980 20 rados->read ofs=0 len=0 2024-05-30T15:54:22.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:22.682+0000 7fd7911b0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:54:22.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:22.682+0000 7fd7911b0980 20 rados->read ofs=0 len=0 2024-05-30T15:54:22.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:22.682+0000 7fd7911b0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:54:22.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:22.682+0000 7fd7911b0980 20 started sync module instance, tier type = 2024-05-30T15:54:22.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:22.682+0000 7fd7911b0980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T15:54:24.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.701+0000 7fd7911b0980 20 add_watcher() i=0 2024-05-30T15:54:24.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.701+0000 7fd7911b0980 20 add_watcher() i=1 2024-05-30T15:54:24.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.701+0000 7fd7911b0980 20 add_watcher() i=2 2024-05-30T15:54:24.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.701+0000 7fd7911b0980 20 add_watcher() i=3 2024-05-30T15:54:24.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.701+0000 7fd7911b0980 20 add_watcher() i=4 2024-05-30T15:54:24.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.701+0000 7fd7911b0980 20 add_watcher() i=5 2024-05-30T15:54:24.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.701+0000 7fd7911b0980 20 add_watcher() i=6 2024-05-30T15:54:24.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.701+0000 7fd7911b0980 20 add_watcher() i=7 2024-05-30T15:54:24.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.701+0000 7fd7911b0980 2 all 8 watchers are set, enabling cache 2024-05-30T15:54:24.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.705+0000 7fd7911b0980 5 tl::expected, obj_version>, boost::system::error_code> logback_generations::read(const DoutPrefixProvider*, optional_yield):410: oid=data_loggenerations_metadata not found 2024-05-30T15:54:24.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.705+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.0 2024-05-30T15:54:24.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.705+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.705+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.0 does not exist 2024-05-30T15:54:24.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.705+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.1 2024-05-30T15:54:24.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.705+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.705+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.1 does not exist 2024-05-30T15:54:24.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.705+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.2 2024-05-30T15:54:24.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.705+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.705+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.2 does not exist 2024-05-30T15:54:24.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.705+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.3 2024-05-30T15:54:24.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.705+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.705+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.3 does not exist 2024-05-30T15:54:24.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.705+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.4 2024-05-30T15:54:24.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.705+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.705+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.4 does not exist 2024-05-30T15:54:24.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.705+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.5 2024-05-30T15:54:24.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.705+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.709+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.5 does not exist 2024-05-30T15:54:24.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.709+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.6 2024-05-30T15:54:24.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.709+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.709+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.6 does not exist 2024-05-30T15:54:24.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.709+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.7 2024-05-30T15:54:24.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.709+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.709+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.7 does not exist 2024-05-30T15:54:24.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.709+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.8 2024-05-30T15:54:24.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.709+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.709+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.8 does not exist 2024-05-30T15:54:24.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.709+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.9 2024-05-30T15:54:24.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.709+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.709+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.9 does not exist 2024-05-30T15:54:24.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.709+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.10 2024-05-30T15:54:24.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.709+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.709+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.10 does not exist 2024-05-30T15:54:24.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.709+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.11 2024-05-30T15:54:24.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.709+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.709+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.11 does not exist 2024-05-30T15:54:24.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.709+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.12 2024-05-30T15:54:24.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.709+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.709+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.12 does not exist 2024-05-30T15:54:24.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.709+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.13 2024-05-30T15:54:24.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.709+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.713+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.13 does not exist 2024-05-30T15:54:24.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.713+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.14 2024-05-30T15:54:24.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.713+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.713+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.14 does not exist 2024-05-30T15:54:24.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.713+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.15 2024-05-30T15:54:24.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.713+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.713+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.15 does not exist 2024-05-30T15:54:24.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.713+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.16 2024-05-30T15:54:24.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.713+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.713+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.16 does not exist 2024-05-30T15:54:24.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.713+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.17 2024-05-30T15:54:24.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.713+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.713+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.17 does not exist 2024-05-30T15:54:24.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.713+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.18 2024-05-30T15:54:24.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.713+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.717+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.18 does not exist 2024-05-30T15:54:24.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.717+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.19 2024-05-30T15:54:24.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.717+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.717+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.19 does not exist 2024-05-30T15:54:24.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.717+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.20 2024-05-30T15:54:24.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.717+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.717+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.20 does not exist 2024-05-30T15:54:24.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.717+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.21 2024-05-30T15:54:24.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.717+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.717+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.21 does not exist 2024-05-30T15:54:24.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.717+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.22 2024-05-30T15:54:24.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.717+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.717+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.22 does not exist 2024-05-30T15:54:24.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.717+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.23 2024-05-30T15:54:24.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.717+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.717+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.23 does not exist 2024-05-30T15:54:24.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.717+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.24 2024-05-30T15:54:24.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.717+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.721+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.24 does not exist 2024-05-30T15:54:24.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.721+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.25 2024-05-30T15:54:24.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.721+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.721+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.25 does not exist 2024-05-30T15:54:24.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.721+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.26 2024-05-30T15:54:24.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.721+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.721+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.26 does not exist 2024-05-30T15:54:24.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.721+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.27 2024-05-30T15:54:24.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.721+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.721+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.27 does not exist 2024-05-30T15:54:24.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.721+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.28 2024-05-30T15:54:24.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.721+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.721+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.28 does not exist 2024-05-30T15:54:24.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.721+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.29 2024-05-30T15:54:24.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.721+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.721+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.29 does not exist 2024-05-30T15:54:24.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.721+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.30 2024-05-30T15:54:24.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.721+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.721+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.30 does not exist 2024-05-30T15:54:24.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.721+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.31 2024-05-30T15:54:24.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.721+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.725+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.31 does not exist 2024-05-30T15:54:24.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.725+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.32 2024-05-30T15:54:24.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.725+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.725+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.32 does not exist 2024-05-30T15:54:24.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.725+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.33 2024-05-30T15:54:24.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.725+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.725+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.33 does not exist 2024-05-30T15:54:24.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.725+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.34 2024-05-30T15:54:24.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.725+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.725+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.34 does not exist 2024-05-30T15:54:24.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.725+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.35 2024-05-30T15:54:24.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.725+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.725+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.35 does not exist 2024-05-30T15:54:24.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.725+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.36 2024-05-30T15:54:24.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.725+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.725+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.36 does not exist 2024-05-30T15:54:24.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.725+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.37 2024-05-30T15:54:24.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.725+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.729+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.37 does not exist 2024-05-30T15:54:24.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.729+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.38 2024-05-30T15:54:24.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.729+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.729+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.38 does not exist 2024-05-30T15:54:24.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.729+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.39 2024-05-30T15:54:24.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.729+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.729+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.39 does not exist 2024-05-30T15:54:24.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.729+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.40 2024-05-30T15:54:24.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.729+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.729+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.40 does not exist 2024-05-30T15:54:24.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.729+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.41 2024-05-30T15:54:24.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.729+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.729+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.41 does not exist 2024-05-30T15:54:24.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.729+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.42 2024-05-30T15:54:24.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.729+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.729+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.42 does not exist 2024-05-30T15:54:24.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.729+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.43 2024-05-30T15:54:24.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.729+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.733+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.43 does not exist 2024-05-30T15:54:24.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.733+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.44 2024-05-30T15:54:24.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.733+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.733+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.44 does not exist 2024-05-30T15:54:24.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.733+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.45 2024-05-30T15:54:24.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.733+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.733+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.45 does not exist 2024-05-30T15:54:24.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.733+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.46 2024-05-30T15:54:24.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.733+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.733+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.46 does not exist 2024-05-30T15:54:24.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.733+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.47 2024-05-30T15:54:24.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.733+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.733+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.47 does not exist 2024-05-30T15:54:24.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.733+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.48 2024-05-30T15:54:24.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.733+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.733+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.48 does not exist 2024-05-30T15:54:24.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.733+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.49 2024-05-30T15:54:24.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.733+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.733+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.49 does not exist 2024-05-30T15:54:24.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.733+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.50 2024-05-30T15:54:24.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.733+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.737+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.50 does not exist 2024-05-30T15:54:24.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.737+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.51 2024-05-30T15:54:24.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.737+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.737+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.51 does not exist 2024-05-30T15:54:24.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.737+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.52 2024-05-30T15:54:24.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.737+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.737+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.52 does not exist 2024-05-30T15:54:24.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.737+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.53 2024-05-30T15:54:24.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.737+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.737+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.53 does not exist 2024-05-30T15:54:24.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.737+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.54 2024-05-30T15:54:24.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.737+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.737+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.54 does not exist 2024-05-30T15:54:24.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.737+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.55 2024-05-30T15:54:24.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.737+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.741+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.55 does not exist 2024-05-30T15:54:24.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.741+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.56 2024-05-30T15:54:24.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.741+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.741+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.56 does not exist 2024-05-30T15:54:24.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.741+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.57 2024-05-30T15:54:24.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.741+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.741+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.57 does not exist 2024-05-30T15:54:24.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.741+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.58 2024-05-30T15:54:24.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.741+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.741+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.58 does not exist 2024-05-30T15:54:24.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.741+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.59 2024-05-30T15:54:24.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.741+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.741+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.59 does not exist 2024-05-30T15:54:24.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.741+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.60 2024-05-30T15:54:24.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.741+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.741+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.60 does not exist 2024-05-30T15:54:24.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.741+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.61 2024-05-30T15:54:24.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.741+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.741+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.61 does not exist 2024-05-30T15:54:24.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.741+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.62 2024-05-30T15:54:24.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.741+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.745+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.62 does not exist 2024-05-30T15:54:24.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.745+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.63 2024-05-30T15:54:24.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.745+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.745+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.63 does not exist 2024-05-30T15:54:24.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.745+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.64 2024-05-30T15:54:24.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.745+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.745+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.64 does not exist 2024-05-30T15:54:24.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.745+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.65 2024-05-30T15:54:24.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.745+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.745+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.65 does not exist 2024-05-30T15:54:24.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.745+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.66 2024-05-30T15:54:24.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.745+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.745+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.66 does not exist 2024-05-30T15:54:24.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.745+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.67 2024-05-30T15:54:24.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.745+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.745+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.67 does not exist 2024-05-30T15:54:24.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.745+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.68 2024-05-30T15:54:24.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.745+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.745+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.68 does not exist 2024-05-30T15:54:24.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.745+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.69 2024-05-30T15:54:24.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.745+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.745+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.69 does not exist 2024-05-30T15:54:24.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.745+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.70 2024-05-30T15:54:24.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.745+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.745+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.70 does not exist 2024-05-30T15:54:24.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.745+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.71 2024-05-30T15:54:24.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.745+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.749+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.71 does not exist 2024-05-30T15:54:24.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.749+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.72 2024-05-30T15:54:24.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.749+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.749+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.72 does not exist 2024-05-30T15:54:24.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.749+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.73 2024-05-30T15:54:24.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.749+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.749+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.73 does not exist 2024-05-30T15:54:24.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.749+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.74 2024-05-30T15:54:24.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.749+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.749+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.74 does not exist 2024-05-30T15:54:24.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.749+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.75 2024-05-30T15:54:24.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.749+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.749+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.75 does not exist 2024-05-30T15:54:24.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.749+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.76 2024-05-30T15:54:24.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.749+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.749+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.76 does not exist 2024-05-30T15:54:24.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.749+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.77 2024-05-30T15:54:24.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.749+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.749+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.77 does not exist 2024-05-30T15:54:24.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.749+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.78 2024-05-30T15:54:24.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.749+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.753+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.78 does not exist 2024-05-30T15:54:24.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.753+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.79 2024-05-30T15:54:24.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.753+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.753+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.79 does not exist 2024-05-30T15:54:24.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.753+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.80 2024-05-30T15:54:24.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.753+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.753+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.80 does not exist 2024-05-30T15:54:24.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.753+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.81 2024-05-30T15:54:24.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.753+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.753+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.81 does not exist 2024-05-30T15:54:24.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.753+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.82 2024-05-30T15:54:24.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.753+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.753+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.82 does not exist 2024-05-30T15:54:24.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.753+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.83 2024-05-30T15:54:24.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.753+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.753+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.83 does not exist 2024-05-30T15:54:24.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.753+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.84 2024-05-30T15:54:24.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.753+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.753+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.84 does not exist 2024-05-30T15:54:24.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.753+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.85 2024-05-30T15:54:24.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.753+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.757+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.85 does not exist 2024-05-30T15:54:24.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.757+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.86 2024-05-30T15:54:24.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.757+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.757+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.86 does not exist 2024-05-30T15:54:24.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.757+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.87 2024-05-30T15:54:24.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.757+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.757+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.87 does not exist 2024-05-30T15:54:24.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.757+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.88 2024-05-30T15:54:24.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.757+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.757+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.88 does not exist 2024-05-30T15:54:24.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.757+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.89 2024-05-30T15:54:24.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.757+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.757+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.89 does not exist 2024-05-30T15:54:24.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.757+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.90 2024-05-30T15:54:24.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.757+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.757+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.90 does not exist 2024-05-30T15:54:24.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.757+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.91 2024-05-30T15:54:24.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.757+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.761+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.91 does not exist 2024-05-30T15:54:24.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.761+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.92 2024-05-30T15:54:24.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.761+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.761+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.92 does not exist 2024-05-30T15:54:24.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.761+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.93 2024-05-30T15:54:24.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.761+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.761+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.93 does not exist 2024-05-30T15:54:24.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.761+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.94 2024-05-30T15:54:24.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.761+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.761+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.94 does not exist 2024-05-30T15:54:24.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.761+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.95 2024-05-30T15:54:24.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.761+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.761+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.95 does not exist 2024-05-30T15:54:24.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.761+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.96 2024-05-30T15:54:24.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.761+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.761+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.96 does not exist 2024-05-30T15:54:24.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.761+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.97 2024-05-30T15:54:24.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.761+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.765+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.97 does not exist 2024-05-30T15:54:24.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.765+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.98 2024-05-30T15:54:24.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.765+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.765+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.98 does not exist 2024-05-30T15:54:24.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.765+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.99 2024-05-30T15:54:24.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.765+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.765+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.99 does not exist 2024-05-30T15:54:24.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.765+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.100 2024-05-30T15:54:24.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.765+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.765+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.100 does not exist 2024-05-30T15:54:24.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.765+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.101 2024-05-30T15:54:24.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.765+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.765+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.101 does not exist 2024-05-30T15:54:24.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.765+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.102 2024-05-30T15:54:24.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.765+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.765+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.102 does not exist 2024-05-30T15:54:24.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.765+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.103 2024-05-30T15:54:24.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.765+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.765+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.103 does not exist 2024-05-30T15:54:24.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.765+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.104 2024-05-30T15:54:24.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.765+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.769+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.104 does not exist 2024-05-30T15:54:24.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.769+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.105 2024-05-30T15:54:24.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.769+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.769+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.105 does not exist 2024-05-30T15:54:24.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.769+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.106 2024-05-30T15:54:24.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.769+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.769+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.106 does not exist 2024-05-30T15:54:24.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.769+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.107 2024-05-30T15:54:24.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.769+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.769+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.107 does not exist 2024-05-30T15:54:24.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.769+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.108 2024-05-30T15:54:24.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.769+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.769+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.108 does not exist 2024-05-30T15:54:24.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.769+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.109 2024-05-30T15:54:24.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.769+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.769+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.109 does not exist 2024-05-30T15:54:24.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.769+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.110 2024-05-30T15:54:24.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.769+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.773+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.110 does not exist 2024-05-30T15:54:24.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.773+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.111 2024-05-30T15:54:24.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.773+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.773+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.111 does not exist 2024-05-30T15:54:24.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.773+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.112 2024-05-30T15:54:24.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.773+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.773+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.112 does not exist 2024-05-30T15:54:24.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.773+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.113 2024-05-30T15:54:24.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.773+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.773+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.113 does not exist 2024-05-30T15:54:24.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.773+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.114 2024-05-30T15:54:24.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.773+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.773+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.114 does not exist 2024-05-30T15:54:24.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.773+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.115 2024-05-30T15:54:24.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.773+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.773+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.115 does not exist 2024-05-30T15:54:24.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.773+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.116 2024-05-30T15:54:24.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.773+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.773+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.116 does not exist 2024-05-30T15:54:24.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.773+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.117 2024-05-30T15:54:24.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.773+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.777+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.117 does not exist 2024-05-30T15:54:24.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.777+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.118 2024-05-30T15:54:24.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.777+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.777+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.118 does not exist 2024-05-30T15:54:24.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.777+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.119 2024-05-30T15:54:24.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.777+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.777+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.119 does not exist 2024-05-30T15:54:24.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.777+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.120 2024-05-30T15:54:24.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.777+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.777+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.120 does not exist 2024-05-30T15:54:24.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.777+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.121 2024-05-30T15:54:24.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.777+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.777+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.121 does not exist 2024-05-30T15:54:24.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.777+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.122 2024-05-30T15:54:24.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.777+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.777+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.122 does not exist 2024-05-30T15:54:24.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.777+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.123 2024-05-30T15:54:24.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.777+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.777+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.123 does not exist 2024-05-30T15:54:24.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.777+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.124 2024-05-30T15:54:24.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.777+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.781+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.124 does not exist 2024-05-30T15:54:24.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.781+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.125 2024-05-30T15:54:24.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.781+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.781+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.125 does not exist 2024-05-30T15:54:24.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.781+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.126 2024-05-30T15:54:24.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.781+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.781+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.126 does not exist 2024-05-30T15:54:24.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.781+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.127 2024-05-30T15:54:24.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.781+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.781+0000 7fd7911b0980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.127 does not exist 2024-05-30T15:54:24.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.781+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::create(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, std::optional >, bool, uint64_t, uint64_t):1211 entering 2024-05-30T15:54:24.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.785+0000 7fd7911b0980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-30T15:54:24.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.793+0000 7fd63d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:54:24.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.793+0000 7fd7911b0980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:54:24.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:24.793+0000 7fd7911b0980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:54:27.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:27.669+0000 7fd7911b0980 10 rgw_init_ioctx warning: failed to set recovery_priority on default.rgw.meta 2024-05-30T15:54:27.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:27.669+0000 7fd7911b0980 5 note: GC not initialized 2024-05-30T15:54:27.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:27.677+0000 7fd5a7fff640 20 reqs_thread_entry: start 2024-05-30T15:54:27.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:27.677+0000 7fd7911b0980 20 init_complete bucket index max shards: 11 2024-05-30T15:54:27.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:27.677+0000 7fd7911b0980 20 Filter name: none 2024-05-30T15:54:27.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:27.677+0000 7fd5a5ffb640 20 reqs_thread_entry: start 2024-05-30T15:54:27.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:27.729+0000 7fd63d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:54:27.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:27.737+0000 7fd7911b0980 20 remove_watcher() i=0 2024-05-30T15:54:27.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:27.737+0000 7fd7911b0980 2 removed watcher, disabling cache 2024-05-30T15:54:27.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:27.737+0000 7fd7911b0980 20 remove_watcher() i=1 2024-05-30T15:54:27.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:27.741+0000 7fd7911b0980 20 remove_watcher() i=2 2024-05-30T15:54:27.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:27.749+0000 7fd7911b0980 20 remove_watcher() i=3 2024-05-30T15:54:27.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:27.749+0000 7fd7911b0980 20 remove_watcher() i=4 2024-05-30T15:54:27.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:27.753+0000 7fd7911b0980 20 remove_watcher() i=5 2024-05-30T15:54:27.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:27.757+0000 7fd7911b0980 20 remove_watcher() i=6 2024-05-30T15:54:27.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:27.757+0000 7fd7911b0980 20 remove_watcher() i=7 2024-05-30T15:54:27.785 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-05-30T15:54:27.785 DEBUG:tasks.util.rgw: json result: [] 2024-05-30T15:54:27.785 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2024-05-30T15:54:27.785 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2024-05-30T15:54:27.786 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement modify --rgw-zone default --placement-id default-placement --compression random 2024-05-30T15:54:27.873 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:54:27.873 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:54:27.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:27.901+0000 7f14ffb0d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:54:27.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:27.901+0000 7f14ffb0d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:54:27.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:27.905+0000 7f133b7fe640 20 reqs_thread_entry: start 2024-05-30T15:54:27.932 INFO:teuthology.orchestra.run.smithi006.stdout:{"id":"601fa7ab-74a8-40e7-9e32-e5827d25af6b","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":""} 2024-05-30T15:54:27.932 DEBUG:tasks.util.rgw: json result: {'id': '601fa7ab-74a8-40e7-9e32-e5827d25af6b', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': ''} 2024-05-30T15:54:27.932 INFO:tasks.rgw:Preparing directory for rgw datacache at /tmp/rgw_datacache 2024-05-30T15:54:27.932 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p /tmp/rgw_datacache 2024-05-30T15:54:27.982 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a+rwx /tmp/rgw_datacache 2024-05-30T15:54:28.042 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2024-05-30T15:54:28.042 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2024-05-30T15:54:28.042 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2024-05-30T15:54:28.121 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:54:28.121 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:54:28.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.165+0000 7f0a8551c980 20 rados->read ofs=0 len=0 2024-05-30T15:54:28.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.165+0000 7f0a8551c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:54:28.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.165+0000 7f0a8551c980 20 realm 2024-05-30T15:54:28.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.165+0000 7f0a8551c980 20 rados->read ofs=0 len=0 2024-05-30T15:54:28.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.165+0000 7f0a8551c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:54:28.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.165+0000 7f0a8551c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T15:54:28.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.165+0000 7f0a8551c980 20 rados->read ofs=0 len=0 2024-05-30T15:54:28.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.165+0000 7f0a8551c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:54:28.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.165+0000 7f0a8551c980 20 rados->read ofs=0 len=0 2024-05-30T15:54:28.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.169+0000 7f0a8551c980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:54:28.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.169+0000 7f0a8551c980 20 rados->read ofs=0 len=0 2024-05-30T15:54:28.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.169+0000 7f0a8551c980 20 rados_obj.operate() r=0 bl.length=1009 2024-05-30T15:54:28.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.169+0000 7f0a8551c980 20 searching for the correct realm 2024-05-30T15:54:28.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.173+0000 7f0a8302d640 0 -- 172.21.15.6:0/692460387 >> [v2:172.21.15.6:6816/2704410641,v1:172.21.15.6:6817/2704410641] conn(0x7f0a7c028070 msgr2=0x7f0a7c0930e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:54:28.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.177+0000 7f0a8302d640 0 -- 172.21.15.6:0/692460387 >> [v2:172.21.15.6:6816/2704410641,v1:172.21.15.6:6817/2704410641] conn(0x7f094c07b5b0 msgr2=0x7f094c07d9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T15:54:28.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.189+0000 7f0a8551c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T15:54:28.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.189+0000 7f0a8551c980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T15:54:28.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.189+0000 7f0a8551c980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T15:54:28.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.189+0000 7f0a8551c980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T15:54:28.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.189+0000 7f0a8551c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T15:54:28.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.189+0000 7f0a8551c980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T15:54:28.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.189+0000 7f0a8551c980 20 rados->read ofs=0 len=0 2024-05-30T15:54:28.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.189+0000 7f0a8551c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:54:28.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.189+0000 7f0a8551c980 20 rados->read ofs=0 len=0 2024-05-30T15:54:28.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.193+0000 7f0a8551c980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:54:28.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.193+0000 7f0a8551c980 20 rados->read ofs=0 len=0 2024-05-30T15:54:28.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.193+0000 7f0a8551c980 20 rados_obj.operate() r=0 bl.length=436 2024-05-30T15:54:28.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.193+0000 7f0a8551c980 20 zone default found 2024-05-30T15:54:28.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.193+0000 7f0a8551c980 4 Realm: () 2024-05-30T15:54:28.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.193+0000 7f0a8551c980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T15:54:28.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.193+0000 7f0a8551c980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T15:54:28.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.193+0000 7f0a8551c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T15:54:28.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.193+0000 7f0a8551c980 20 zonegroup default 2024-05-30T15:54:28.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.193+0000 7f0a8551c980 20 rados->read ofs=0 len=0 2024-05-30T15:54:28.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.193+0000 7f0a8551c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:54:28.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.193+0000 7f0a8551c980 20 rados->read ofs=0 len=0 2024-05-30T15:54:28.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.193+0000 7f0a8551c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:54:28.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.193+0000 7f0a8551c980 20 rados->read ofs=0 len=0 2024-05-30T15:54:28.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.193+0000 7f0a8551c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:54:28.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.193+0000 7f0a8551c980 20 started sync module instance, tier type = 2024-05-30T15:54:28.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.193+0000 7f0a8551c980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T15:54:28.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.221+0000 7f0a8551c980 20 add_watcher() i=0 2024-05-30T15:54:28.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.221+0000 7f0a8551c980 20 add_watcher() i=1 2024-05-30T15:54:28.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.221+0000 7f0a8551c980 20 add_watcher() i=2 2024-05-30T15:54:28.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.221+0000 7f0a8551c980 20 add_watcher() i=3 2024-05-30T15:54:28.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.221+0000 7f0a8551c980 20 add_watcher() i=4 2024-05-30T15:54:28.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.221+0000 7f0a8551c980 20 add_watcher() i=5 2024-05-30T15:54:28.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.221+0000 7f0a8551c980 20 add_watcher() i=6 2024-05-30T15:54:28.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.221+0000 7f0a8551c980 20 add_watcher() i=7 2024-05-30T15:54:28.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.221+0000 7f0a8551c980 2 all 8 watchers are set, enabling cache 2024-05-30T15:54:28.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.225+0000 7f0939fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:54:28.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.225+0000 7f0a8551c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:54:28.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.225+0000 7f0a8551c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:54:28.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.225+0000 7f0a8551c980 5 note: GC not initialized 2024-05-30T15:54:28.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.225+0000 7f0938fe1640 20 reqs_thread_entry: start 2024-05-30T15:54:28.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.225+0000 7f0a8551c980 20 init_complete bucket index max shards: 11 2024-05-30T15:54:28.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.225+0000 7f0a8551c980 20 Filter name: none 2024-05-30T15:54:28.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.225+0000 7f08b27fc640 20 reqs_thread_entry: start 2024-05-30T15:54:28.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.245+0000 7f0939fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:54:28.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.249+0000 7f0a8551c980 20 remove_watcher() i=0 2024-05-30T15:54:28.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.249+0000 7f0a8551c980 2 removed watcher, disabling cache 2024-05-30T15:54:28.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.249+0000 7f0a8551c980 20 remove_watcher() i=1 2024-05-30T15:54:28.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.253+0000 7f0a8551c980 20 remove_watcher() i=2 2024-05-30T15:54:28.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.253+0000 7f0a8551c980 20 remove_watcher() i=3 2024-05-30T15:54:28.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.257+0000 7f0a8551c980 20 remove_watcher() i=4 2024-05-30T15:54:28.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.257+0000 7f0a8551c980 20 remove_watcher() i=5 2024-05-30T15:54:28.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.261+0000 7f0a8551c980 20 remove_watcher() i=6 2024-05-30T15:54:28.263 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.261+0000 7f0a8551c980 20 remove_watcher() i=7 2024-05-30T15:54:28.279 INFO:teuthology.orchestra.run.smithi006.stdout:[] 2024-05-30T15:54:28.279 DEBUG:tasks.util.rgw: json result: [] 2024-05-30T15:54:28.279 INFO:tasks.rgw:Configuring storage class type = LUKEWARM 2024-05-30T15:54:28.280 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2024-05-30T15:54:28.280 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2024-05-30T15:54:28.280 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM 2024-05-30T15:54:28.365 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:54:28.365 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:54:28.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.393+0000 7f72e42f0980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:54:28.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.393+0000 7f72e42f0980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:54:28.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.397+0000 7f71a4fe1640 20 reqs_thread_entry: start 2024-05-30T15:54:28.420 INFO:teuthology.orchestra.run.smithi006.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["LUKEWARM","STANDARD"]}}] 2024-05-30T15:54:28.420 DEBUG:tasks.util.rgw: json result: [{'key': 'default-placement', 'val': {'name': 'default-placement', 'tags': [], 'storage_classes': ['LUKEWARM', 'STANDARD']}}] 2024-05-30T15:54:28.421 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2024-05-30T15:54:28.421 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2024-05-30T15:54:28.421 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM --data-pool default.rgw.buckets.data.lukewarm 2024-05-30T15:54:28.504 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:54:28.505 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:54:28.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.533+0000 7f3062a48980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:54:28.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.533+0000 7f3062a48980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:54:28.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.537+0000 7f2f1e7fc640 20 reqs_thread_entry: start 2024-05-30T15:54:28.559 INFO:teuthology.orchestra.run.smithi006.stdout:{"id":"601fa7ab-74a8-40e7-9e32-e5827d25af6b","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"LUKEWARM":{"data_pool":"default.rgw.buckets.data.lukewarm"},"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":""} 2024-05-30T15:54:28.559 DEBUG:tasks.util.rgw: json result: {'id': '601fa7ab-74a8-40e7-9e32-e5827d25af6b', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'LUKEWARM': {'data_pool': 'default.rgw.buckets.data.lukewarm'}, 'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': ''} 2024-05-30T15:54:28.559 INFO:tasks.rgw:Configuring storage class type = FROZEN 2024-05-30T15:54:28.559 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2024-05-30T15:54:28.559 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2024-05-30T15:54:28.559 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN 2024-05-30T15:54:28.640 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:54:28.640 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:54:28.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.669+0000 7f12f3f28980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:54:28.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.669+0000 7f12f3f28980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:54:28.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.673+0000 7f1127fff640 20 reqs_thread_entry: start 2024-05-30T15:54:28.695 INFO:teuthology.orchestra.run.smithi006.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["FROZEN","LUKEWARM","STANDARD"]}}] 2024-05-30T15:54:28.695 DEBUG:tasks.util.rgw: json result: [{'key': 'default-placement', 'val': {'name': 'default-placement', 'tags': [], 'storage_classes': ['FROZEN', 'LUKEWARM', 'STANDARD']}}] 2024-05-30T15:54:28.695 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2024-05-30T15:54:28.695 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2024-05-30T15:54:28.696 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN --data-pool default.rgw.buckets.data.frozen 2024-05-30T15:54:28.784 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:54:28.784 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:54:28.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.809+0000 7f554c771980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:54:28.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.809+0000 7f554c771980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:54:28.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T15:54:28.813+0000 7f540cfe1640 20 reqs_thread_entry: start 2024-05-30T15:54:28.839 INFO:teuthology.orchestra.run.smithi006.stdout:{"id":"601fa7ab-74a8-40e7-9e32-e5827d25af6b","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"FROZEN":{"data_pool":"default.rgw.buckets.data.frozen"},"LUKEWARM":{"data_pool":"default.rgw.buckets.data.lukewarm"},"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":""} 2024-05-30T15:54:28.839 DEBUG:tasks.util.rgw: json result: {'id': '601fa7ab-74a8-40e7-9e32-e5827d25af6b', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'FROZEN': {'data_pool': 'default.rgw.buckets.data.frozen'}, 'LUKEWARM': {'data_pool': 'default.rgw.buckets.data.lukewarm'}, 'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': ''} 2024-05-30T15:54:28.840 INFO:tasks.rgw:Starting rgw... 2024-05-30T15:54:28.840 INFO:tasks.rgw:rgw client.0 config is {'valgrind': ['--tool=memcheck', '--max-threads=1024']} 2024-05-30T15:54:28.840 INFO:tasks.rgw:Using beast as radosgw frontend 2024-05-30T15:54:28.840 DEBUG:teuthology.orchestra.run.smithi006:> sudo echo -n http://smithi006.front.sepia.ceph.com:80 | sudo tee /home/ubuntu/cephtest/url_file 2024-05-30T15:54:28.893 INFO:teuthology.orchestra.run.smithi006.stdout:http://smithi006.front.sepia.ceph.com:80 2024-05-30T15:54:28.893 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown ceph /home/ubuntu/cephtest/url_file 2024-05-30T15:54:28.948 DEBUG:tasks.ceph_manager:running ceph.client.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/ceph.client.0.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck', '--max-threads=1024'] 2024-05-30T15:54:28.948 INFO:tasks.rgw.client.0:Restarting daemon 2024-05-30T15:54:28.948 DEBUG:teuthology.orchestra.run.smithi006:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/ceph.client.0.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'beast port=80' -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.ceph.client.0.stdout 2>&1 2024-05-30T15:54:28.995 INFO:tasks.rgw.client.0:Started 2024-05-30T15:54:28.995 INFO:tasks.rgw:Polling client.0 until it starts accepting connections on http://smithi006.front.sepia.ceph.com:80/ 2024-05-30T15:54:28.995 DEBUG:teuthology.orchestra.run.smithi006:> curl http://smithi006.front.sepia.ceph.com:80/ 2024-05-30T15:54:29.024 INFO:teuthology.orchestra.run.smithi006.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-05-30T15:54:29.024 INFO:teuthology.orchestra.run.smithi006.stderr: Dload Upload Total Spent Left Speed 2024-05-30T15:54:29.024 INFO:teuthology.orchestra.run.smithi006.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-05-30T15:54:29.024 INFO:teuthology.orchestra.run.smithi006.stderr:curl: (7) Failed to connect to smithi006.front.sepia.ceph.com port 80 after 0 ms: Connection refused 2024-05-30T15:54:29.025 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-05-30T15:54:30.026 DEBUG:teuthology.orchestra.run.smithi006:> curl http://smithi006.front.sepia.ceph.com:80/ 2024-05-30T15:54:30.037 INFO:teuthology.orchestra.run.smithi006.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-05-30T15:54:30.037 INFO:teuthology.orchestra.run.smithi006.stderr: Dload Upload Total Spent Left Speed 2024-05-30T15:54:30.037 INFO:teuthology.orchestra.run.smithi006.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-05-30T15:54:30.037 INFO:teuthology.orchestra.run.smithi006.stderr:curl: (7) Failed to connect to smithi006.front.sepia.ceph.com port 80 after 0 ms: Connection refused 2024-05-30T15:54:30.038 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-05-30T15:54:32.039 DEBUG:teuthology.orchestra.run.smithi006:> curl http://smithi006.front.sepia.ceph.com:80/ 2024-05-30T15:54:32.049 INFO:teuthology.orchestra.run.smithi006.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-05-30T15:54:32.050 INFO:teuthology.orchestra.run.smithi006.stderr: Dload Upload Total Spent Left Speed 2024-05-30T15:54:32.050 INFO:teuthology.orchestra.run.smithi006.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-05-30T15:54:32.050 INFO:teuthology.orchestra.run.smithi006.stderr:curl: (7) Failed to connect to smithi006.front.sepia.ceph.com port 80 after 0 ms: Connection refused 2024-05-30T15:54:32.050 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-05-30T15:54:36.052 DEBUG:teuthology.orchestra.run.smithi006:> curl http://smithi006.front.sepia.ceph.com:80/ 2024-05-30T15:54:36.064 INFO:teuthology.orchestra.run.smithi006.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-05-30T15:54:36.064 INFO:teuthology.orchestra.run.smithi006.stderr: Dload Upload Total Spent Left Speed 2024-05-30T15:54:36.064 INFO:teuthology.orchestra.run.smithi006.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-05-30T15:54:36.064 INFO:teuthology.orchestra.run.smithi006.stderr:curl: (7) Failed to connect to smithi006.front.sepia.ceph.com port 80 after 0 ms: Connection refused 2024-05-30T15:54:36.065 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-05-30T15:54:44.066 DEBUG:teuthology.orchestra.run.smithi006:> curl http://smithi006.front.sepia.ceph.com:80/ 2024-05-30T15:54:44.077 INFO:teuthology.orchestra.run.smithi006.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-05-30T15:54:44.077 INFO:teuthology.orchestra.run.smithi006.stderr: Dload Upload Total Spent Left Speed 2024-05-30T15:54:44.077 INFO:teuthology.orchestra.run.smithi006.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-05-30T15:54:44.077 INFO:teuthology.orchestra.run.smithi006.stderr:curl: (7) Failed to connect to smithi006.front.sepia.ceph.com port 80 after 0 ms: Connection refused 2024-05-30T15:54:44.078 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-05-30T15:55:00.079 DEBUG:teuthology.orchestra.run.smithi006:> curl http://smithi006.front.sepia.ceph.com:80/ 2024-05-30T15:55:00.091 INFO:teuthology.orchestra.run.smithi006.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-05-30T15:55:00.091 INFO:teuthology.orchestra.run.smithi006.stderr: Dload Upload Total Spent Left Speed 2024-05-30T15:55:00.591 INFO:teuthology.orchestra.run.smithi006.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 187 0 187 0 0 374 0 --:--:-- --:--:-- --:--:-- 374 2024-05-30T15:55:00.592 INFO:teuthology.orchestra.run.smithi006.stdout:anonymous 2024-05-30T15:55:00.593 INFO:teuthology.run_tasks:Running task tox... 2024-05-30T15:55:00.605 INFO:tasks.tox:Deploying tox from pip... 2024-05-30T15:55:00.605 DEBUG:teuthology.orchestra.run.smithi006:> python3 -m venv /home/ubuntu/cephtest/tox-venv 2024-05-30T15:55:03.662 DEBUG:teuthology.orchestra.run.smithi006:> source /home/ubuntu/cephtest/tox-venv/bin/activate && pip install tox==3.15.0 2024-05-30T15:55:04.362 INFO:teuthology.orchestra.run.smithi006.stdout:Collecting tox==3.15.0 2024-05-30T15:55:04.488 INFO:teuthology.orchestra.run.smithi006.stdout: Downloading tox-3.15.0-py2.py3-none-any.whl (82 kB) 2024-05-30T15:55:04.548 INFO:teuthology.orchestra.run.smithi006.stdout: ---------------------------------------- 82.1/82.1 KB 1.4 MB/s eta 0:00:00 2024-05-30T15:55:04.670 INFO:teuthology.orchestra.run.smithi006.stdout:Collecting toml>=0.9.4 2024-05-30T15:55:04.689 INFO:teuthology.orchestra.run.smithi006.stdout: Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB) 2024-05-30T15:55:04.986 INFO:teuthology.orchestra.run.smithi006.stdout:Collecting virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0 2024-05-30T15:55:05.006 INFO:teuthology.orchestra.run.smithi006.stdout: Downloading virtualenv-20.26.2-py3-none-any.whl (3.9 MB) 2024-05-30T15:55:05.519 INFO:teuthology.orchestra.run.smithi006.stdout: ---------------------------------------- 3.9/3.9 MB 7.7 MB/s eta 0:00:00 2024-05-30T15:55:05.654 INFO:teuthology.orchestra.run.smithi006.stdout:Collecting py<2,>=1.4.17 2024-05-30T15:55:05.674 INFO:teuthology.orchestra.run.smithi006.stdout: Downloading py-1.11.0-py2.py3-none-any.whl (98 kB) 2024-05-30T15:55:05.701 INFO:teuthology.orchestra.run.smithi006.stdout: ---------------------------------------- 98.7/98.7 KB 3.9 MB/s eta 0:00:00 2024-05-30T15:55:05.827 INFO:teuthology.orchestra.run.smithi006.stdout:Collecting filelock<4,>=3.0.0 2024-05-30T15:55:05.846 INFO:teuthology.orchestra.run.smithi006.stdout: Downloading filelock-3.14.0-py3-none-any.whl (12 kB) 2024-05-30T15:55:05.954 INFO:teuthology.orchestra.run.smithi006.stdout:Collecting six<2,>=1.14.0 2024-05-30T15:55:05.973 INFO:teuthology.orchestra.run.smithi006.stdout: Downloading six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-05-30T15:55:06.255 INFO:teuthology.orchestra.run.smithi006.stdout:Collecting pluggy<1,>=0.12.0 2024-05-30T15:55:06.278 INFO:teuthology.orchestra.run.smithi006.stdout: Downloading pluggy-0.13.1-py2.py3-none-any.whl (18 kB) 2024-05-30T15:55:06.427 INFO:teuthology.orchestra.run.smithi006.stdout:Collecting packaging>=14 2024-05-30T15:55:06.445 INFO:teuthology.orchestra.run.smithi006.stdout: Downloading packaging-24.0-py3-none-any.whl (53 kB) 2024-05-30T15:55:06.480 INFO:teuthology.orchestra.run.smithi006.stdout: ---------------------------------------- 53.5/53.5 KB 1.5 MB/s eta 0:00:00 2024-05-30T15:55:06.636 INFO:teuthology.orchestra.run.smithi006.stdout:Collecting distlib<1,>=0.3.7 2024-05-30T15:55:06.657 INFO:teuthology.orchestra.run.smithi006.stdout: Downloading distlib-0.3.8-py2.py3-none-any.whl (468 kB) 2024-05-30T15:55:06.723 INFO:teuthology.orchestra.run.smithi006.stdout: --------------------------------------- 468.9/468.9 KB 7.6 MB/s eta 0:00:00 2024-05-30T15:55:06.853 INFO:teuthology.orchestra.run.smithi006.stdout:Collecting platformdirs<5,>=3.9.1 2024-05-30T15:55:06.871 INFO:teuthology.orchestra.run.smithi006.stdout: Downloading platformdirs-4.2.2-py3-none-any.whl (18 kB) 2024-05-30T15:55:07.057 INFO:teuthology.orchestra.run.smithi006.stdout:Installing collected packages: distlib, toml, six, py, pluggy, platformdirs, packaging, filelock, virtualenv, tox 2024-05-30T15:55:07.956 INFO:teuthology.orchestra.run.smithi006.stdout:Successfully installed distlib-0.3.8 filelock-3.14.0 packaging-24.0 platformdirs-4.2.2 pluggy-0.13.1 py-1.11.0 six-1.16.0 toml-0.10.2 tox-3.15.0 virtualenv-20.26.2 2024-05-30T15:55:08.044 INFO:teuthology.run_tasks:Running task workunit... 2024-05-30T15:55:08.058 INFO:tasks.workunit:Pulling workunits from ref da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 2024-05-30T15:55:08.058 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-30T15:55:08.058 DEBUG:teuthology.orchestra.run.smithi006:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-30T15:55:08.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-30T15:55:08.064 INFO:teuthology.orchestra.run.smithi006.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-05-30T15:55:08.064 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-05-30T15:55:08.110 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-05-30T15:55:08.110 DEBUG:teuthology.orchestra.run.smithi006:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-05-30T15:55:08.158 INFO:tasks.workunit:timeout=3h 2024-05-30T15:55:08.158 INFO:tasks.workunit:cleanup=True 2024-05-30T15:55:08.159 DEBUG:teuthology.orchestra.run.smithi006:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 2024-05-30T15:55:08.213 INFO:tasks.workunit.client.0.smithi006.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-30T15:56:31.769 INFO:tasks.workunit.client.0.smithi006.stderr:Updating files: 98% (12055/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-30T15:56:32.644 INFO:tasks.workunit.client.0.smithi006.stderr:Note: switching to 'da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2'. 2024-05-30T15:56:32.644 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T15:56:32.644 INFO:tasks.workunit.client.0.smithi006.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-30T15:56:32.645 INFO:tasks.workunit.client.0.smithi006.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-30T15:56:32.645 INFO:tasks.workunit.client.0.smithi006.stderr:state without impacting any branches by switching back to a branch. 2024-05-30T15:56:32.645 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T15:56:32.645 INFO:tasks.workunit.client.0.smithi006.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-30T15:56:32.645 INFO:tasks.workunit.client.0.smithi006.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-30T15:56:32.645 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T15:56:32.645 INFO:tasks.workunit.client.0.smithi006.stderr: git switch -c 2024-05-30T15:56:32.645 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T15:56:32.645 INFO:tasks.workunit.client.0.smithi006.stderr:Or undo this operation with: 2024-05-30T15:56:32.645 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T15:56:32.645 INFO:tasks.workunit.client.0.smithi006.stderr: git switch - 2024-05-30T15:56:32.645 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T15:56:32.646 INFO:tasks.workunit.client.0.smithi006.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-30T15:56:32.646 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T15:56:32.646 INFO:tasks.workunit.client.0.smithi006.stderr:HEAD is now at da98024b78b Merge branches 'pr-57079', 'pr-57536', 'pr-57641' and 'pr-57702' into wip-cbodley-testing 2024-05-30T15:56:32.652 DEBUG:teuthology.orchestra.run.smithi006:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-30T15:56:32.699 INFO:tasks.workunit.client.0.smithi006.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-30T15:56:32.703 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-30T15:56:32.703 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-30T15:56:33.476 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-30T15:56:33.569 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-30T15:56:33.629 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-30T15:56:33.630 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-30T15:56:33.630 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-30T15:56:33.680 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-30T15:56:33.684 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T15:56:33.684 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-30T15:56:33.731 INFO:tasks.workunit:Running workunits matching rgw/run-datacache.sh on client.0... 2024-05-30T15:56:33.732 INFO:tasks.workunit:Running workunit rgw/run-datacache.sh... 2024-05-30T15:56:33.732 DEBUG:teuthology.orchestra.run.smithi006:workunit test rgw/run-datacache.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RGW_DATACACHE_PATH=/tmp/rgw_datacache adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-datacache.sh 2024-05-30T15:56:33.784 INFO:tasks.workunit.client.0.smithi006.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-datacache.sh 2024-05-30T15:56:33.784 INFO:tasks.workunit.client.0.smithi006.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-05-30T15:56:33.784 INFO:tasks.workunit.client.0.smithi006.stderr:+ python3 -m venv /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-05-30T15:56:36.720 INFO:tasks.workunit.client.0.smithi006.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/activate 2024-05-30T15:56:36.720 INFO:tasks.workunit.client.0.smithi006.stderr:++ deactivate nondestructive 2024-05-30T15:56:36.721 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -n '' ']' 2024-05-30T15:56:36.721 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -n '' ']' 2024-05-30T15:56:36.721 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-05-30T15:56:36.721 INFO:tasks.workunit.client.0.smithi006.stderr:++ hash -r 2024-05-30T15:56:36.721 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -n '' ']' 2024-05-30T15:56:36.721 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset VIRTUAL_ENV 2024-05-30T15:56:36.721 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset VIRTUAL_ENV_PROMPT 2024-05-30T15:56:36.721 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '!' nondestructive = nondestructive ']' 2024-05-30T15:56:36.721 INFO:tasks.workunit.client.0.smithi006.stderr:++ VIRTUAL_ENV=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-05-30T15:56:36.721 INFO:tasks.workunit.client.0.smithi006.stderr:++ export VIRTUAL_ENV 2024-05-30T15:56:36.721 INFO:tasks.workunit.client.0.smithi006.stderr:++ _OLD_VIRTUAL_PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-05-30T15:56:36.721 INFO:tasks.workunit.client.0.smithi006.stderr:++ PATH=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-05-30T15:56:36.721 INFO:tasks.workunit.client.0.smithi006.stderr:++ export PATH 2024-05-30T15:56:36.722 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -n '' ']' 2024-05-30T15:56:36.722 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-05-30T15:56:36.722 INFO:tasks.workunit.client.0.smithi006.stderr:++ _OLD_VIRTUAL_PS1= 2024-05-30T15:56:36.722 INFO:tasks.workunit.client.0.smithi006.stderr:++ PS1='(rgw) ' 2024-05-30T15:56:36.722 INFO:tasks.workunit.client.0.smithi006.stderr:++ export PS1 2024-05-30T15:56:36.722 INFO:tasks.workunit.client.0.smithi006.stderr:++ VIRTUAL_ENV_PROMPT='(rgw) ' 2024-05-30T15:56:36.722 INFO:tasks.workunit.client.0.smithi006.stderr:++ export VIRTUAL_ENV_PROMPT 2024-05-30T15:56:36.722 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-05-30T15:56:36.722 INFO:tasks.workunit.client.0.smithi006.stderr:++ hash -r 2024-05-30T15:56:36.722 INFO:tasks.workunit.client.0.smithi006.stderr:+ pip install pip --upgrade 2024-05-30T15:56:37.097 INFO:tasks.workunit.client.0.smithi006.stdout:Requirement already satisfied: pip in /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages (22.0.2) 2024-05-30T15:56:37.436 INFO:tasks.workunit.client.0.smithi006.stdout:Collecting pip 2024-05-30T15:56:37.589 INFO:tasks.workunit.client.0.smithi006.stdout: Downloading pip-24.0-py3-none-any.whl (2.1 MB) 2024-05-30T15:56:37.847 INFO:tasks.workunit.client.0.smithi006.stdout: ---------------------------------------- 2.1/2.1 MB 8.3 MB/s eta 0:00:00 2024-05-30T15:56:37.926 INFO:tasks.workunit.client.0.smithi006.stdout:Installing collected packages: pip 2024-05-30T15:56:37.926 INFO:tasks.workunit.client.0.smithi006.stdout: Attempting uninstall: pip 2024-05-30T15:56:37.927 INFO:tasks.workunit.client.0.smithi006.stdout: Found existing installation: pip 22.0.2 2024-05-30T15:56:38.225 INFO:tasks.workunit.client.0.smithi006.stdout: Uninstalling pip-22.0.2: 2024-05-30T15:56:38.231 INFO:tasks.workunit.client.0.smithi006.stdout: Successfully uninstalled pip-22.0.2 2024-05-30T15:56:39.224 INFO:tasks.workunit.client.0.smithi006.stdout:Successfully installed pip-24.0 2024-05-30T15:56:39.285 INFO:tasks.workunit.client.0.smithi006.stderr:+ pip install configobj 2024-05-30T15:56:39.950 INFO:tasks.workunit.client.0.smithi006.stdout:Collecting configobj 2024-05-30T15:56:40.189 INFO:tasks.workunit.client.0.smithi006.stdout: Downloading configobj-5.0.8-py2.py3-none-any.whl.metadata (3.4 kB) 2024-05-30T15:56:40.383 INFO:tasks.workunit.client.0.smithi006.stdout:Collecting six (from configobj) 2024-05-30T15:56:40.432 INFO:tasks.workunit.client.0.smithi006.stdout: Downloading six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB) 2024-05-30T15:56:40.540 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading configobj-5.0.8-py2.py3-none-any.whl (36 kB) 2024-05-30T15:56:40.682 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-05-30T15:56:40.825 INFO:tasks.workunit.client.0.smithi006.stdout:Installing collected packages: six, configobj 2024-05-30T15:56:40.958 INFO:tasks.workunit.client.0.smithi006.stdout:Successfully installed configobj-5.0.8 six-1.16.0 2024-05-30T15:56:41.536 INFO:tasks.workunit.client.0.smithi006.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/python3 /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_datacache.py 2024-05-30T15:56:41.576 INFO:tasks.workunit.client.0.smithi006.stderr:DEBUG:root:datacache dir from config is: /tmp/rgw_datacache 2024-05-30T15:56:41.576 INFO:tasks.workunit.client.0.smithi006.stderr:DEBUG:root:exec_cmd(pwd) 2024-05-30T15:56:41.577 INFO:tasks.workunit.client.0.smithi006.stderr:INFO:root:command succeeded 2024-05-30T15:56:41.577 INFO:tasks.workunit.client.0.smithi006.stderr:INFO:root:b'/home/ubuntu/cephtest/mnt.0/client.0/tmp\n' 2024-05-30T15:56:41.577 INFO:tasks.workunit.client.0.smithi006.stderr:DEBUG:root:pwd is: /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-30T15:56:41.577 INFO:tasks.workunit.client.0.smithi006.stderr:DEBUG:root:exec_cmd(sudo netstat -nltp | egrep "rados|valgr") 2024-05-30T15:56:41.622 INFO:tasks.workunit.client.0.smithi006.stderr:INFO:root:command succeeded 2024-05-30T15:56:41.623 INFO:tasks.workunit.client.0.smithi006.stderr:INFO:root:b'tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 36196/valgrind.bin \ntcp6 0 0 :::80 :::* LISTEN 36196/valgrind.bin \n' 2024-05-30T15:56:41.623 INFO:tasks.workunit.client.0.smithi006.stderr:INFO:root:radosgw port: 80 2024-05-30T15:56:41.623 INFO:tasks.workunit.client.0.smithi006.stderr:INFO:root:radosgw endpoint is: 127.0.0.1 2024-05-30T15:56:41.623 INFO:tasks.workunit.client.0.smithi006.stderr:DEBUG:root:exec_cmd(dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/client.0/tmp/7M.dat bs=1M count=7) 2024-05-30T15:56:41.648 INFO:tasks.workunit.client.0.smithi006.stderr:INFO:root:command succeeded 2024-05-30T15:56:41.649 INFO:tasks.workunit.client.0.smithi006.stderr:INFO:root:b'' 2024-05-30T15:56:41.649 INFO:tasks.workunit.client.0.smithi006.stderr:DEBUG:root:exec_cmd(radosgw-admin user create --uid rgw_datacache_user --display-name DatacacheUser --access-key NX5QOQKC6BH2IDN8HC7A --secret LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn) 2024-05-30T15:56:42.087 INFO:tasks.workunit.client.0.smithi006.stderr:INFO:root:command succeeded 2024-05-30T15:56:42.088 INFO:tasks.workunit.client.0.smithi006.stderr:INFO:root:b'{\n "user_id": "rgw_datacache_user",\n "display_name": "DatacacheUser",\n "email": "",\n "suspended": 0,\n "max_buckets": 1000,\n "subusers": [],\n "keys": [\n {\n "user": "rgw_datacache_user",\n "access_key": "NX5QOQKC6BH2IDN8HC7A",\n "secret_key": "LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn",\n "active": true,\n "create_date": "2024-05-30T15:56:42.009159Z"\n }\n ],\n "swift_keys": [],\n "caps": [],\n "op_mask": "read, write, delete",\n "default_placement": "",\n "default_storage_class": "",\n "placement_tags": [],\n "bucket_quota": {\n "enabled": false,\n "check_on_raw": false,\n "max_size": -1,\n "max_size_kb": 0,\n "max_objects": -1\n },\n "user_quota": {\n "enabled": false,\n "check_on_raw": false,\n "max_size": -1,\n "max_size_kb": 0,\n "max_objects": -1\n },\n "temp_url_keys": [],\n "type": "rgw",\n "mfa_ids": [],\n "account_id": "",\n "path": "/",\n "create_date": "2024-05-30T15:56:42.009117Z",\n "tags": [],\n "group_ids": []\n}\n\n' 2024-05-30T15:56:42.088 INFO:tasks.workunit.client.0.smithi006.stderr:INFO:root:Creating s3cmd config... 2024-05-30T15:56:42.088 INFO:tasks.workunit.client.0.smithi006.stderr:INFO:root:proto for s3cmd config is http 2024-05-30T15:56:42.088 INFO:tasks.workunit.client.0.smithi006.stderr:INFO:root:s3cmd config written 2024-05-30T15:56:42.088 INFO:tasks.workunit.client.0.smithi006.stderr:DEBUG:root:exec_cmd(s3cmd --access_key=NX5QOQKC6BH2IDN8HC7A --secret_key=LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn --config=/home/ubuntu/cephtest/mnt.0/client.0/tmp/s3cfg --no-check-hostname --host=127.0.0.1 mb s3://datacachebucket) 2024-05-30T15:56:43.109 INFO:tasks.workunit.client.0.smithi006.stderr:INFO:root:command succeeded 2024-05-30T15:56:43.109 INFO:tasks.workunit.client.0.smithi006.stderr:INFO:root:b"Bucket 's3://datacachebucket/' created\n" 2024-05-30T15:56:43.109 INFO:tasks.workunit.client.0.smithi006.stderr:DEBUG:root:exec_cmd(s3cmd --access_key=NX5QOQKC6BH2IDN8HC7A --secret_key=LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn --config=/home/ubuntu/cephtest/mnt.0/client.0/tmp/s3cfg --no-check-hostname --host=127.0.0.1 put /home/ubuntu/cephtest/mnt.0/client.0/tmp/7M.dat s3://datacachebucket) 2024-05-30T15:56:48.561 INFO:tasks.workunit.client.0.smithi006.stderr:INFO:root:command succeeded 2024-05-30T15:56:48.561 INFO:tasks.workunit.client.0.smithi006.stderr:INFO:root:b"upload: '/home/ubuntu/cephtest/mnt.0/client.0/tmp/7M.dat' -> 's3://datacachebucket/7M.dat' (7340032 bytes in 5.1 seconds, 1395.66 KB/s) [1 of 1]\n" 2024-05-30T15:56:48.561 INFO:tasks.workunit.client.0.smithi006.stderr:DEBUG:root:exec_cmd(s3cmd --access_key=NX5QOQKC6BH2IDN8HC7A --secret_key=LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn --config=/home/ubuntu/cephtest/mnt.0/client.0/tmp/s3cfg --no-check-hostname --host=127.0.0.1 get s3://datacachebucket/7M.dat /home/ubuntu/cephtest/mnt.0/client.0/tmp/7M-get.dat --force) 2024-05-30T15:56:49.445 INFO:tasks.workunit.client.0.smithi006.stderr:INFO:root:command succeeded 2024-05-30T15:56:49.445 INFO:tasks.workunit.client.0.smithi006.stderr:INFO:root:b"download: 's3://datacachebucket/7M.dat' -> '/home/ubuntu/cephtest/mnt.0/client.0/tmp/7M-get.dat' (7340032 bytes in 0.7 seconds, 9.85 MB/s)\n" 2024-05-30T15:56:49.445 INFO:tasks.workunit.client.0.smithi006.stderr:DEBUG:root:exec_cmd(radosgw-admin object stat --bucket=datacachebucket --object=7M.dat) 2024-05-30T15:56:49.613 INFO:tasks.workunit.client.0.smithi006.stderr:INFO:root:command succeeded 2024-05-30T15:56:49.613 INFO:tasks.workunit.client.0.smithi006.stderr:INFO:root:b'{\n "name": "7M.dat",\n "size": 7342274,\n "policy": {\n "acl": {\n "acl_user_map": [\n {\n "user": "rgw_datacache_user",\n "acl": 15\n }\n ],\n "acl_group_map": [],\n "grant_map": [\n {\n "id": "rgw_datacache_user",\n "grant": {\n "type": {\n "type": 0\n },\n "id": "rgw_datacache_user",\n "name": "DatacacheUser",\n "permission": {\n "flags": 15\n }\n }\n }\n ]\n },\n "owner": {\n "id": "rgw_datacache_user",\n "display_name": "DatacacheUser"\n }\n },\n "compression": {\n "compression_type": "zlib",\n "orig_size": 7340032,\n "compressor_message": -15,\n "blocks": [\n {\n "old_ofs": 0,\n "new_ofs": 0,\n "len": 4195585\n },\n {\n "old_ofs": 4194304,\n "new_ofs": 4195585,\n "len": 3146689\n }\n ]\n },\n "etag": "36bc2cecf06dc6ef71d7127cc72ef02d",\n "tag": "601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.3354408575614167726",\n "manifest": {\n "objs": [],\n "obj_size": 7342274,\n "explicit_objs": false,\n "head_size": 4194304,\n "max_head_size": 4194304,\n "prefix": ".Xe_4xw1oigM4xNzaJnKl3r0GbZf7yIx_",\n "rules": [\n {\n "key": 0,\n "val": {\n "start_part_num": 0,\n "start_ofs": 4194304,\n "part_size": 0,\n "stripe_max_size": 6291456,\n "override_prefix": ""\n }\n }\n ],\n "tail_instance": "",\n "tail_placement": {\n "bucket": {\n "name": "datacachebucket",\n "marker": "601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.1",\n "bucket_id": "601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.1",\n "tenant": "",\n "explicit_placement": {\n "data_pool": "",\n "data_extra_pool": "",\n "index_pool": ""\n }\n },\n "placement_rule": "default-placement"\n },\n "tier_type": "",\n "begin_iter": {\n "part_ofs": 0,\n "stripe_ofs": 0,\n "ofs": 0,\n "stripe_size": 4194304,\n "cur_part_id": 0,\n "cur_stripe": 0,\n "cur_override_prefix": "",\n "location": {\n "placement_rule": "default-placement",\n "obj": {\n "bucket": {\n "name": "datacachebucket",\n "marker": "601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.1",\n "bucket_id": "601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.1",\n "tenant": "",\n "explicit_placement": {\n "data_pool": "",\n "data_extra_pool": "",\n "index_pool": ""\n }\n },\n "key": {\n "name": "7M.dat",\n "instance": "",\n "ns": ""\n }\n },\n "raw_obj": {\n "pool": "",\n "oid": "",\n "loc": ""\n },\n "is_raw": false\n }\n },\n "end_iter": {\n "part_ofs": 4194304,\n "stripe_ofs": 4194304,\n "ofs": 7342274,\n "stripe_size": 3147970,\n "cur_part_id": 0,\n "cur_stripe": 1,\n "cur_override_prefix": "",\n "location": {\n "placement_rule": "default-placement",\n "obj": {\n "bucket": {\n "name": "datacachebucket",\n "marker": "601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.1",\n "bucket_id": "601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.1",\n "tenant": "",\n "explicit_placement": {\n "data_pool": "",\n "data_extra_pool": "",\n "index_pool": ""\n }\n },\n "key": {\n "name": ".Xe_4xw1oigM4xNzaJnKl3r0GbZf7yIx_1",\n "instance": "",\n "ns": "shadow"\n }\n },\n "raw_obj": {\n "pool": "",\n "oid": "",\n "loc": ""\n },\n "is_raw": false\n }\n }\n },\n "pg_ver": 0,\n "source_zone": 0,\n "torrent": "",\n "mtime": "2024-05-30T15:56:48.226222Z",\n "attrs": {\n "user.rgw.content_type": "application/octet-stream",\n "user.rgw.storage_class": "STANDARD",\n "user.rgw.tail_tag": "601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.3354408575614167726",\n "user.rgw.x-amz-content-sha256": "acbf95c8be87b7458a865524403cea0b48bcc337911ebd52b668d5ec133a41a3",\n "user.rgw.x-amz-date": "20240530T155643Z",\n "user.rgw.x-amz-meta-s3cmd-attrs": "atime:1717084601/ctime:1717084601/gid:1271/gname:ubuntu/md5:36bc2cecf06dc6ef71d7127cc72ef02d/mode:33204/mtime:1717084601/uid:1000/uname:ubuntu"\n }\n}\n' 2024-05-30T15:56:49.613 INFO:tasks.workunit.client.0.smithi006.stderr:DEBUG:root:Cached object name is: .Xe_4xw1oigM4xNzaJnKl3r0GbZf7yIx_ 2024-05-30T15:56:49.613 INFO:tasks.workunit.client.0.smithi006.stderr:DEBUG:root:exec_cmd(find /tmp/rgw_datacache -type f | wc -l) 2024-05-30T15:56:49.614 INFO:tasks.workunit.client.0.smithi006.stderr:INFO:root:command succeeded 2024-05-30T15:56:49.615 INFO:tasks.workunit.client.0.smithi006.stderr:INFO:root:b'0\n' 2024-05-30T15:56:49.615 INFO:tasks.workunit.client.0.smithi006.stderr:DEBUG:root:Check cache dir content: 0 2024-05-30T15:56:49.615 INFO:tasks.workunit.client.0.smithi006.stderr:INFO:root:NOTICE: datacache test object not found, inspect if datacache was bypassed or disabled during this check. 2024-05-30T15:56:49.615 INFO:tasks.workunit.client.0.smithi006.stderr:INFO:root:Completed Datacache tests 2024-05-30T15:56:49.622 INFO:tasks.workunit.client.0.smithi006.stderr:+ deactivate 2024-05-30T15:56:49.622 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -n /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin ']' 2024-05-30T15:56:49.622 INFO:tasks.workunit.client.0.smithi006.stderr:+ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-05-30T15:56:49.622 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-05-30T15:56:49.622 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset _OLD_VIRTUAL_PATH 2024-05-30T15:56:49.622 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -n '' ']' 2024-05-30T15:56:49.622 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -n /usr/bin/bash -o -n '' ']' 2024-05-30T15:56:49.622 INFO:tasks.workunit.client.0.smithi006.stderr:+ hash -r 2024-05-30T15:56:49.622 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -n '' ']' 2024-05-30T15:56:49.622 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset VIRTUAL_ENV 2024-05-30T15:56:49.622 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset VIRTUAL_ENV_PROMPT 2024-05-30T15:56:49.623 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' '!' '' = nondestructive ']' 2024-05-30T15:56:49.623 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset -f deactivate 2024-05-30T15:56:49.623 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo OK. 2024-05-30T15:56:49.624 INFO:tasks.workunit.client.0.smithi006.stdout:OK. 2024-05-30T15:56:49.624 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-30T15:56:49.624 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-30T15:56:49.682 INFO:tasks.workunit:Stopping ['rgw/run-datacache.sh'] on client.0... 2024-05-30T15:56:49.682 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-30T15:56:50.011 DEBUG:teuthology.parallel:result is None 2024-05-30T15:56:50.012 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-30T15:56:50.023 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-30T15:56:50.023 DEBUG:teuthology.orchestra.run.smithi006:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-05-30T15:56:50.075 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-05-30T15:56:50.075 INFO:teuthology.run_tasks:Running task workunit... 2024-05-30T15:56:50.086 INFO:tasks.workunit:Pulling workunits from ref da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 2024-05-30T15:56:50.086 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-30T15:56:50.086 DEBUG:teuthology.orchestra.run.smithi006:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-30T15:56:50.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-30T15:56:50.122 INFO:teuthology.orchestra.run.smithi006.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-05-30T15:56:50.122 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-05-30T15:56:50.170 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-05-30T15:56:50.170 DEBUG:teuthology.orchestra.run.smithi006:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-05-30T15:56:50.218 INFO:tasks.workunit:timeout=3h 2024-05-30T15:56:50.218 INFO:tasks.workunit:cleanup=True 2024-05-30T15:56:50.219 DEBUG:teuthology.orchestra.run.smithi006:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 2024-05-30T15:56:50.268 INFO:tasks.workunit.client.0.smithi006.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-30T15:58:12.149 INFO:tasks.workunit.client.0.smithi006.stderr:Updating files: 96% (11819/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-30T15:58:12.586 INFO:tasks.workunit.client.0.smithi006.stderr:Note: switching to 'da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2'. 2024-05-30T15:58:12.586 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T15:58:12.586 INFO:tasks.workunit.client.0.smithi006.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-30T15:58:12.586 INFO:tasks.workunit.client.0.smithi006.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-30T15:58:12.586 INFO:tasks.workunit.client.0.smithi006.stderr:state without impacting any branches by switching back to a branch. 2024-05-30T15:58:12.586 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T15:58:12.586 INFO:tasks.workunit.client.0.smithi006.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-30T15:58:12.586 INFO:tasks.workunit.client.0.smithi006.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-30T15:58:12.586 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T15:58:12.586 INFO:tasks.workunit.client.0.smithi006.stderr: git switch -c 2024-05-30T15:58:12.587 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T15:58:12.587 INFO:tasks.workunit.client.0.smithi006.stderr:Or undo this operation with: 2024-05-30T15:58:12.587 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T15:58:12.587 INFO:tasks.workunit.client.0.smithi006.stderr: git switch - 2024-05-30T15:58:12.587 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T15:58:12.587 INFO:tasks.workunit.client.0.smithi006.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-30T15:58:12.587 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T15:58:12.587 INFO:tasks.workunit.client.0.smithi006.stderr:HEAD is now at da98024b78b Merge branches 'pr-57079', 'pr-57536', 'pr-57641' and 'pr-57702' into wip-cbodley-testing 2024-05-30T15:58:12.594 DEBUG:teuthology.orchestra.run.smithi006:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-30T15:58:12.644 INFO:tasks.workunit.client.0.smithi006.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-30T15:58:12.646 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-30T15:58:12.646 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-30T15:58:12.703 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-30T15:58:12.765 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-30T15:58:12.810 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-30T15:58:12.813 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-30T15:58:12.813 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-30T15:58:12.853 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-30T15:58:12.857 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T15:58:12.857 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-30T15:58:12.906 INFO:tasks.workunit:Running workunits matching rgw/run-bucket-check.sh on client.0... 2024-05-30T15:58:12.908 INFO:tasks.workunit:Running workunit rgw/run-bucket-check.sh... 2024-05-30T15:58:12.908 DEBUG:teuthology.orchestra.run.smithi006:workunit test rgw/run-bucket-check.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-bucket-check.sh 2024-05-30T15:58:12.958 INFO:tasks.workunit.client.0.smithi006.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-bucket-check.sh 2024-05-30T15:58:12.958 INFO:tasks.workunit.client.0.smithi006.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-05-30T15:58:12.958 INFO:tasks.workunit.client.0.smithi006.stderr:+ python3 -m venv /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-05-30T15:58:15.907 INFO:tasks.workunit.client.0.smithi006.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/activate 2024-05-30T15:58:15.907 INFO:tasks.workunit.client.0.smithi006.stderr:++ deactivate nondestructive 2024-05-30T15:58:15.907 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -n '' ']' 2024-05-30T15:58:15.907 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -n '' ']' 2024-05-30T15:58:15.907 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-05-30T15:58:15.907 INFO:tasks.workunit.client.0.smithi006.stderr:++ hash -r 2024-05-30T15:58:15.908 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -n '' ']' 2024-05-30T15:58:15.908 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset VIRTUAL_ENV 2024-05-30T15:58:15.908 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset VIRTUAL_ENV_PROMPT 2024-05-30T15:58:15.908 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '!' nondestructive = nondestructive ']' 2024-05-30T15:58:15.908 INFO:tasks.workunit.client.0.smithi006.stderr:++ VIRTUAL_ENV=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-05-30T15:58:15.908 INFO:tasks.workunit.client.0.smithi006.stderr:++ export VIRTUAL_ENV 2024-05-30T15:58:15.908 INFO:tasks.workunit.client.0.smithi006.stderr:++ _OLD_VIRTUAL_PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-05-30T15:58:15.908 INFO:tasks.workunit.client.0.smithi006.stderr:++ PATH=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-05-30T15:58:15.909 INFO:tasks.workunit.client.0.smithi006.stderr:++ export PATH 2024-05-30T15:58:15.909 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -n '' ']' 2024-05-30T15:58:15.909 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-05-30T15:58:15.909 INFO:tasks.workunit.client.0.smithi006.stderr:++ _OLD_VIRTUAL_PS1= 2024-05-30T15:58:15.909 INFO:tasks.workunit.client.0.smithi006.stderr:++ PS1='(rgw) ' 2024-05-30T15:58:15.909 INFO:tasks.workunit.client.0.smithi006.stderr:++ export PS1 2024-05-30T15:58:15.909 INFO:tasks.workunit.client.0.smithi006.stderr:++ VIRTUAL_ENV_PROMPT='(rgw) ' 2024-05-30T15:58:15.909 INFO:tasks.workunit.client.0.smithi006.stderr:++ export VIRTUAL_ENV_PROMPT 2024-05-30T15:58:15.910 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-05-30T15:58:15.910 INFO:tasks.workunit.client.0.smithi006.stderr:++ hash -r 2024-05-30T15:58:15.910 INFO:tasks.workunit.client.0.smithi006.stderr:+ pip install pip --upgrade 2024-05-30T15:58:16.301 INFO:tasks.workunit.client.0.smithi006.stdout:Requirement already satisfied: pip in /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages (22.0.2) 2024-05-30T15:58:16.520 INFO:tasks.workunit.client.0.smithi006.stdout:Collecting pip 2024-05-30T15:58:16.527 INFO:tasks.workunit.client.0.smithi006.stdout: Using cached pip-24.0-py3-none-any.whl (2.1 MB) 2024-05-30T15:58:16.566 INFO:tasks.workunit.client.0.smithi006.stdout:Installing collected packages: pip 2024-05-30T15:58:16.566 INFO:tasks.workunit.client.0.smithi006.stdout: Attempting uninstall: pip 2024-05-30T15:58:16.566 INFO:tasks.workunit.client.0.smithi006.stdout: Found existing installation: pip 22.0.2 2024-05-30T15:58:16.841 INFO:tasks.workunit.client.0.smithi006.stdout: Uninstalling pip-22.0.2: 2024-05-30T15:58:16.847 INFO:tasks.workunit.client.0.smithi006.stdout: Successfully uninstalled pip-22.0.2 2024-05-30T15:58:17.817 INFO:tasks.workunit.client.0.smithi006.stdout:Successfully installed pip-24.0 2024-05-30T15:58:17.877 INFO:tasks.workunit.client.0.smithi006.stderr:+ pip install boto3 2024-05-30T15:58:18.972 INFO:tasks.workunit.client.0.smithi006.stdout:Collecting boto3 2024-05-30T15:58:19.089 INFO:tasks.workunit.client.0.smithi006.stdout: Downloading boto3-1.34.115-py3-none-any.whl.metadata (6.6 kB) 2024-05-30T15:58:19.713 INFO:tasks.workunit.client.0.smithi006.stdout:Collecting botocore<1.35.0,>=1.34.115 (from boto3) 2024-05-30T15:58:19.730 INFO:tasks.workunit.client.0.smithi006.stdout: Downloading botocore-1.34.115-py3-none-any.whl.metadata (5.7 kB) 2024-05-30T15:58:19.849 INFO:tasks.workunit.client.0.smithi006.stdout:Collecting jmespath<2.0.0,>=0.7.1 (from boto3) 2024-05-30T15:58:19.870 INFO:tasks.workunit.client.0.smithi006.stdout: Downloading jmespath-1.0.1-py3-none-any.whl.metadata (7.6 kB) 2024-05-30T15:58:20.007 INFO:tasks.workunit.client.0.smithi006.stdout:Collecting s3transfer<0.11.0,>=0.10.0 (from boto3) 2024-05-30T15:58:20.027 INFO:tasks.workunit.client.0.smithi006.stdout: Downloading s3transfer-0.10.1-py3-none-any.whl.metadata (1.7 kB) 2024-05-30T15:58:20.217 INFO:tasks.workunit.client.0.smithi006.stdout:Collecting python-dateutil<3.0.0,>=2.1 (from botocore<1.35.0,>=1.34.115->boto3) 2024-05-30T15:58:20.266 INFO:tasks.workunit.client.0.smithi006.stdout: Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB) 2024-05-30T15:58:20.514 INFO:tasks.workunit.client.0.smithi006.stdout:Collecting urllib3!=2.2.0,<3,>=1.25.4 (from botocore<1.35.0,>=1.34.115->boto3) 2024-05-30T15:58:20.543 INFO:tasks.workunit.client.0.smithi006.stdout: Downloading urllib3-2.2.1-py3-none-any.whl.metadata (6.4 kB) 2024-05-30T15:58:20.909 INFO:tasks.workunit.client.0.smithi006.stdout:Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore<1.35.0,>=1.34.115->boto3) 2024-05-30T15:58:20.910 INFO:tasks.workunit.client.0.smithi006.stdout: Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB) 2024-05-30T15:58:20.944 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading boto3-1.34.115-py3-none-any.whl (139 kB) 2024-05-30T15:58:21.032 INFO:tasks.workunit.client.0.smithi006.stdout: ---------------------------------------- 139.3/139.3 kB 1.6 MB/s eta 0:00:00 2024-05-30T15:58:21.056 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading botocore-1.34.115-py3-none-any.whl (12.3 MB) 2024-05-30T15:58:21.579 INFO:tasks.workunit.client.0.smithi006.stdout: ---------------------------------------- 12.3/12.3 MB 24.5 MB/s eta 0:00:00 2024-05-30T15:58:21.601 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading jmespath-1.0.1-py3-none-any.whl (20 kB) 2024-05-30T15:58:21.674 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading s3transfer-0.10.1-py3-none-any.whl (82 kB) 2024-05-30T15:58:21.733 INFO:tasks.workunit.client.0.smithi006.stdout: ---------------------------------------- 82.2/82.2 kB 1.3 MB/s eta 0:00:00 2024-05-30T15:58:21.754 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) 2024-05-30T15:58:21.807 INFO:tasks.workunit.client.0.smithi006.stdout: ---------------------------------------- 229.9/229.9 kB 4.4 MB/s eta 0:00:00 2024-05-30T15:58:21.844 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading urllib3-2.2.1-py3-none-any.whl (121 kB) 2024-05-30T15:58:21.900 INFO:tasks.workunit.client.0.smithi006.stdout: ---------------------------------------- 121.1/121.1 kB 2.1 MB/s eta 0:00:00 2024-05-30T15:58:21.902 INFO:tasks.workunit.client.0.smithi006.stdout:Using cached six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-05-30T15:58:22.028 INFO:tasks.workunit.client.0.smithi006.stdout:Installing collected packages: urllib3, six, jmespath, python-dateutil, botocore, s3transfer, boto3 2024-05-30T15:58:23.169 INFO:tasks.workunit.client.0.smithi006.stdout:Successfully installed boto3-1.34.115 botocore-1.34.115 jmespath-1.0.1 python-dateutil-2.9.0.post0 s3transfer-0.10.1 six-1.16.0 urllib3-2.2.1 2024-05-30T15:58:23.332 INFO:tasks.workunit.client.0.smithi006.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/python3 /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_bucket_check.py 2024-05-30T15:58:23.488 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin user create --uid check-tester --display-name "Check Testing" --access-key OJODXSLNX4LUNHQG99PA --secret 3l6ffld34qaymfomuh832j94738aie2x4p2o8h6n 2024-05-30T15:58:23.527 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:58:23.527 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:58:23.571 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.571+0000 7f2a8a199980 20 rados->read ofs=0 len=0 2024-05-30T15:58:23.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.571+0000 7f2a8a199980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:23.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.571+0000 7f2a8a199980 20 realm 2024-05-30T15:58:23.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.571+0000 7f2a8a199980 20 rados->read ofs=0 len=0 2024-05-30T15:58:23.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.575+0000 7f2a8a199980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:23.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.575+0000 7f2a8a199980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T15:58:23.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.575+0000 7f2a8a199980 20 rados->read ofs=0 len=0 2024-05-30T15:58:23.574 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.575+0000 7f2a8a199980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:23.574 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.575+0000 7f2a8a199980 20 rados->read ofs=0 len=0 2024-05-30T15:58:23.575 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.575+0000 7f2a8a199980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:23.575 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.575+0000 7f2a8a199980 20 rados->read ofs=0 len=0 2024-05-30T15:58:23.575 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.575+0000 7f2a8a199980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T15:58:23.576 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.575+0000 7f2a8a199980 20 searching for the correct realm 2024-05-30T15:58:23.581 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.583+0000 7f2a874a9640 0 -- 172.21.15.6:0/3666664877 >> [v2:172.21.15.6:6819/1141406589,v1:172.21.15.6:6821/1141406589] conn(0x55711b96d930 msgr2=0x55711b96fdf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:23.588 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.587+0000 7f2a87caa640 0 -- 172.21.15.6:0/3666664877 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f2a7005bde0 msgr2=0x7f2a7007c0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:23.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.595+0000 7f2a8a199980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T15:58:23.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.595+0000 7f2a8a199980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T15:58:23.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.595+0000 7f2a8a199980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T15:58:23.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.595+0000 7f2a8a199980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T15:58:23.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.595+0000 7f2a8a199980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T15:58:23.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.595+0000 7f2a8a199980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T15:58:23.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.595+0000 7f2a8a199980 20 rados->read ofs=0 len=0 2024-05-30T15:58:23.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.595+0000 7f2a8a199980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:23.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.595+0000 7f2a8a199980 20 rados->read ofs=0 len=0 2024-05-30T15:58:23.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.595+0000 7f2a8a199980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:23.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.595+0000 7f2a8a199980 20 rados->read ofs=0 len=0 2024-05-30T15:58:23.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.595+0000 7f2a8a199980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T15:58:23.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.595+0000 7f2a8a199980 20 zone default found 2024-05-30T15:58:23.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.595+0000 7f2a8a199980 4 Realm: () 2024-05-30T15:58:23.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.595+0000 7f2a8a199980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T15:58:23.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.595+0000 7f2a8a199980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T15:58:23.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.595+0000 7f2a8a199980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T15:58:23.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.595+0000 7f2a8a199980 20 zonegroup default 2024-05-30T15:58:23.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.595+0000 7f2a8a199980 20 rados->read ofs=0 len=0 2024-05-30T15:58:23.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.599+0000 7f2a8a199980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:23.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.599+0000 7f2a8a199980 20 rados->read ofs=0 len=0 2024-05-30T15:58:23.599 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.599+0000 7f2a8a199980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:23.599 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.599+0000 7f2a8a199980 20 rados->read ofs=0 len=0 2024-05-30T15:58:23.599 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.599+0000 7f2a8a199980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:23.599 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.599+0000 7f2a8a199980 20 started sync module instance, tier type = 2024-05-30T15:58:23.599 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.599+0000 7f2a8a199980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T15:58:23.619 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.619+0000 7f2a8a199980 20 add_watcher() i=0 2024-05-30T15:58:23.619 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.619+0000 7f2a8a199980 20 add_watcher() i=1 2024-05-30T15:58:23.619 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.619+0000 7f2a8a199980 20 add_watcher() i=2 2024-05-30T15:58:23.620 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.619+0000 7f2a8a199980 20 add_watcher() i=3 2024-05-30T15:58:23.620 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.619+0000 7f2a8a199980 20 add_watcher() i=4 2024-05-30T15:58:23.620 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.619+0000 7f2a8a199980 20 add_watcher() i=5 2024-05-30T15:58:23.620 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.619+0000 7f2a8a199980 20 add_watcher() i=6 2024-05-30T15:58:23.622 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.623+0000 7f2a8a199980 20 add_watcher() i=7 2024-05-30T15:58:23.622 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.623+0000 7f2a8a199980 2 all 8 watchers are set, enabling cache 2024-05-30T15:58:23.625 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.627+0000 7f293dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:23.626 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.627+0000 7f2a8a199980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:58:23.626 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.627+0000 7f2a8a199980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:58:23.626 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.627+0000 7f2a8a199980 5 note: GC not initialized 2024-05-30T15:58:23.627 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.627+0000 7f293cfe1640 20 reqs_thread_entry: start 2024-05-30T15:58:23.627 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.627+0000 7f2a8a199980 20 init_complete bucket index max shards: 11 2024-05-30T15:58:23.627 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.627+0000 7f2a8a199980 20 Filter name: none 2024-05-30T15:58:23.627 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.627+0000 7f28b67fc640 20 reqs_thread_entry: start 2024-05-30T15:58:23.627 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.627+0000 7f2a8a199980 10 cache get: name=default.rgw.meta+users.uid+check-tester : miss 2024-05-30T15:58:23.627 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.627+0000 7f2a8a199980 20 rados->read ofs=0 len=0 2024-05-30T15:58:23.628 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.627+0000 7f2a8a199980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:23.628 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.627+0000 7f2a8a199980 10 cache put: name=default.rgw.meta+users.uid+check-tester info.flags=0x0 2024-05-30T15:58:23.628 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.627+0000 7f2a8a199980 10 adding default.rgw.meta+users.uid+check-tester to cache LRU end 2024-05-30T15:58:23.628 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.627+0000 7f2a8a199980 10 cache get: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA : miss 2024-05-30T15:58:23.628 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.627+0000 7f2a8a199980 20 rados->read ofs=0 len=0 2024-05-30T15:58:23.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.627+0000 7f2a8a199980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:23.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.627+0000 7f2a8a199980 10 cache put: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA info.flags=0x0 2024-05-30T15:58:23.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.627+0000 7f2a8a199980 10 adding default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA to cache LRU end 2024-05-30T15:58:23.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.627+0000 7f2a8a199980 10 cache get: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA : hit (negative entry) 2024-05-30T15:58:23.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.627+0000 7f2a8a199980 10 cache get: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA : hit (negative entry) 2024-05-30T15:58:23.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.635+0000 7f2a8a199980 10 cache put: name=default.rgw.meta+users.uid+check-tester info.flags=0x17 2024-05-30T15:58:23.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.635+0000 7f2a8a199980 10 moving default.rgw.meta+users.uid+check-tester to cache LRU end 2024-05-30T15:58:23.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.635+0000 7f2a8a199980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.uid:check-tester, ofs0, ns] 2024-05-30T15:58:23.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.635+0000 7f2966ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 163208757249 cookie 93944282555024 notifier 4526 bl.length()=593 2024-05-30T15:58:23.635 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.635+0000 7f2966ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+check-tester info.flags=0x17 2024-05-30T15:58:23.635 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.635+0000 7f2966ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+check-tester to cache LRU end 2024-05-30T15:58:23.639 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.639+0000 7f2a8a199980 10 cache put: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA info.flags=0x7 2024-05-30T15:58:23.640 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.639+0000 7f2a8a199980 10 moving default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA to cache LRU end 2024-05-30T15:58:23.640 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.639+0000 7f2a8a199980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.keys:OJODXSLNX4LUNHQG99PA, ofs0, ns] 2024-05-30T15:58:23.640 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.639+0000 7f29667fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 163208757248 cookie 93944282544400 notifier 4526 bl.length()=186 2024-05-30T15:58:23.640 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.639+0000 7f29667fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA info.flags=0x7 2024-05-30T15:58:23.640 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.639+0000 7f29667fc640 10 rgw watcher librados: moving default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA to cache LRU end 2024-05-30T15:58:23.644 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.643+0000 7f293dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:23.648 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.647+0000 7f2a8a199980 20 remove_watcher() i=0 2024-05-30T15:58:23.648 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.647+0000 7f2a8a199980 2 removed watcher, disabling cache 2024-05-30T15:58:23.650 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.651+0000 7f2a8a199980 20 remove_watcher() i=1 2024-05-30T15:58:23.653 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.655+0000 7f2a8a199980 20 remove_watcher() i=2 2024-05-30T15:58:23.655 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.655+0000 7f2a8a199980 20 remove_watcher() i=3 2024-05-30T15:58:23.658 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.659+0000 7f2a8a199980 20 remove_watcher() i=4 2024-05-30T15:58:23.659 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.659+0000 7f2a8a199980 20 remove_watcher() i=5 2024-05-30T15:58:23.660 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.659+0000 7f2a8a199980 20 remove_watcher() i=6 2024-05-30T15:58:23.662 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:23.663+0000 7f2a8a199980 20 remove_watcher() i=7 2024-05-30T15:58:25.856 INFO:tasks.workunit.client.0.smithi006.stderr:TEST: recalculated bucket check stats are correct 2024-05-30T15:58:25.856 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T15:58:25.856 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket check --fix --bucket check-bucket 2024-05-30T15:58:25.891 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:58:25.891 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:58:25.938 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.939+0000 7f80dcb36980 20 rados->read ofs=0 len=0 2024-05-30T15:58:25.940 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.939+0000 7f80dcb36980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:25.940 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.939+0000 7f80dcb36980 20 realm 2024-05-30T15:58:25.940 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.939+0000 7f80dcb36980 20 rados->read ofs=0 len=0 2024-05-30T15:58:25.940 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.939+0000 7f80dcb36980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:25.940 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.939+0000 7f80dcb36980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T15:58:25.941 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.939+0000 7f80dcb36980 20 rados->read ofs=0 len=0 2024-05-30T15:58:25.941 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.939+0000 7f80dcb36980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:25.941 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.939+0000 7f80dcb36980 20 rados->read ofs=0 len=0 2024-05-30T15:58:25.941 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.943+0000 7f80dcb36980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:25.942 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.943+0000 7f80dcb36980 20 rados->read ofs=0 len=0 2024-05-30T15:58:25.942 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.943+0000 7f80dcb36980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T15:58:25.942 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.943+0000 7f80dcb36980 20 searching for the correct realm 2024-05-30T15:58:25.948 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.947+0000 7f80dae48640 0 -- 172.21.15.6:0/3285898638 >> [v2:172.21.15.6:6808/1897362715,v1:172.21.15.6:6809/1897362715] conn(0x55caf509a710 msgr2=0x55caf509cbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T15:58:25.955 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.955+0000 7f80da647640 0 -- 172.21.15.6:0/3285898638 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f80d007cb10 msgr2=0x7f80d0036970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:25.962 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.963+0000 7f80dcb36980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T15:58:25.963 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.963+0000 7f80dcb36980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T15:58:25.963 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.963+0000 7f80dcb36980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T15:58:25.963 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.963+0000 7f80dcb36980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T15:58:25.963 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.963+0000 7f80dcb36980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T15:58:25.963 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.963+0000 7f80dcb36980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T15:58:25.963 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.963+0000 7f80dcb36980 20 rados->read ofs=0 len=0 2024-05-30T15:58:25.963 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.963+0000 7f80dcb36980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:25.963 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.963+0000 7f80dcb36980 20 rados->read ofs=0 len=0 2024-05-30T15:58:25.964 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.963+0000 7f80dcb36980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:25.964 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.963+0000 7f80dcb36980 20 rados->read ofs=0 len=0 2024-05-30T15:58:25.964 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.963+0000 7f80dcb36980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T15:58:25.964 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.963+0000 7f80dcb36980 20 zone default found 2024-05-30T15:58:25.964 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.963+0000 7f80dcb36980 4 Realm: () 2024-05-30T15:58:25.964 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.963+0000 7f80dcb36980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T15:58:25.964 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.963+0000 7f80dcb36980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T15:58:25.965 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.963+0000 7f80dcb36980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T15:58:25.965 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.963+0000 7f80dcb36980 20 zonegroup default 2024-05-30T15:58:25.965 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.963+0000 7f80dcb36980 20 rados->read ofs=0 len=0 2024-05-30T15:58:25.966 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.963+0000 7f80dcb36980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:25.966 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.963+0000 7f80dcb36980 20 rados->read ofs=0 len=0 2024-05-30T15:58:25.966 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.963+0000 7f80dcb36980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:25.966 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.963+0000 7f80dcb36980 20 rados->read ofs=0 len=0 2024-05-30T15:58:25.966 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.967+0000 7f80dcb36980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:25.966 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.967+0000 7f80dcb36980 20 started sync module instance, tier type = 2024-05-30T15:58:25.967 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.967+0000 7f80dcb36980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T15:58:25.983 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.983+0000 7f80dcb36980 20 add_watcher() i=0 2024-05-30T15:58:25.983 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.983+0000 7f80dcb36980 20 add_watcher() i=1 2024-05-30T15:58:25.983 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.983+0000 7f80dcb36980 20 add_watcher() i=2 2024-05-30T15:58:25.983 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.983+0000 7f80dcb36980 20 add_watcher() i=3 2024-05-30T15:58:25.983 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.983+0000 7f80dcb36980 20 add_watcher() i=4 2024-05-30T15:58:25.983 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.983+0000 7f80dcb36980 20 add_watcher() i=5 2024-05-30T15:58:25.984 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.983+0000 7f80dcb36980 20 add_watcher() i=6 2024-05-30T15:58:25.984 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.983+0000 7f80dcb36980 20 add_watcher() i=7 2024-05-30T15:58:25.984 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.983+0000 7f80dcb36980 2 all 8 watchers are set, enabling cache 2024-05-30T15:58:25.987 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.987+0000 7f7f90fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:25.987 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.987+0000 7f80dcb36980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:58:25.988 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.987+0000 7f80dcb36980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:58:25.988 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.987+0000 7f80dcb36980 5 note: GC not initialized 2024-05-30T15:58:25.988 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.987+0000 7f7f0b7fe640 20 reqs_thread_entry: start 2024-05-30T15:58:25.989 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.987+0000 7f80dcb36980 20 init_complete bucket index max shards: 11 2024-05-30T15:58:25.989 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.987+0000 7f80dcb36980 20 Filter name: none 2024-05-30T15:58:25.989 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.987+0000 7f7f097fa640 20 reqs_thread_entry: start 2024-05-30T15:58:25.989 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.987+0000 7f80dcb36980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-05-30T15:58:25.989 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.987+0000 7f80dcb36980 20 rados->read ofs=0 len=0 2024-05-30T15:58:25.990 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.987+0000 7f80dcb36980 20 rados_obj.operate() r=0 bl.length=190 2024-05-30T15:58:25.990 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.987+0000 7f80dcb36980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-05-30T15:58:25.990 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.987+0000 7f80dcb36980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-05-30T15:58:25.990 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.987+0000 7f80dcb36980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 : miss 2024-05-30T15:58:25.991 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.987+0000 7f80dcb36980 20 rados->read ofs=0 len=0 2024-05-30T15:58:25.991 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.991+0000 7f80dcb36980 20 rados_obj.operate() r=0 bl.length=404 2024-05-30T15:58:25.991 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.991+0000 7f80dcb36980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 info.flags=0x17 2024-05-30T15:58:25.991 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.991+0000 7f80dcb36980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 to cache LRU end 2024-05-30T15:58:25.991 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.991+0000 7f80dcb36980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T15:58:25.991 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.991+0000 7f80dcb36980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-05-30T15:58:25.991 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.991+0000 7f80dcb36980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 2024-05-30T15:58:25.991 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.991+0000 7f80dcb36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.0 2024-05-30T15:58:25.992 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.991+0000 7f80dcb36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:25.992 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.991+0000 7f80dcb36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.2 2024-05-30T15:58:25.992 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.991+0000 7f80dcb36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.3 2024-05-30T15:58:25.993 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.991+0000 7f80dcb36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.4 2024-05-30T15:58:25.994 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.995+0000 7f80dcb36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:25.995 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.995+0000 7f80dcb36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.6 2024-05-30T15:58:25.996 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.995+0000 7f80dcb36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.7 2024-05-30T15:58:25.998 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.999+0000 7f80dcb36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:25.998 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.999+0000 7f80dcb36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.9 2024-05-30T15:58:25.999 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:25.999+0000 7f80dcb36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:26.010 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.011+0000 7f7f90fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:26.015 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.015+0000 7f80dcb36980 20 remove_watcher() i=0 2024-05-30T15:58:26.015 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.015+0000 7f80dcb36980 2 removed watcher, disabling cache 2024-05-30T15:58:26.017 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.019+0000 7f80dcb36980 20 remove_watcher() i=1 2024-05-30T15:58:26.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.019+0000 7f80dcb36980 20 remove_watcher() i=2 2024-05-30T15:58:26.022 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.023+0000 7f80dcb36980 20 remove_watcher() i=3 2024-05-30T15:58:26.024 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.023+0000 7f80dcb36980 20 remove_watcher() i=4 2024-05-30T15:58:26.026 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.027+0000 7f80dcb36980 20 remove_watcher() i=5 2024-05-30T15:58:26.027 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.027+0000 7f80dcb36980 20 remove_watcher() i=6 2024-05-30T15:58:26.028 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.027+0000 7f80dcb36980 20 remove_watcher() i=7 2024-05-30T15:58:26.044 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket check-bucket 2024-05-30T15:58:26.071 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:58:26.071 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:58:26.117 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.119+0000 7f90d2c1b980 20 rados->read ofs=0 len=0 2024-05-30T15:58:26.119 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.119+0000 7f90d2c1b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:26.119 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.119+0000 7f90d2c1b980 20 realm 2024-05-30T15:58:26.119 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.119+0000 7f90d2c1b980 20 rados->read ofs=0 len=0 2024-05-30T15:58:26.119 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.119+0000 7f90d2c1b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:26.120 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.119+0000 7f90d2c1b980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T15:58:26.120 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.119+0000 7f90d2c1b980 20 rados->read ofs=0 len=0 2024-05-30T15:58:26.120 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.119+0000 7f90d2c1b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:26.120 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.119+0000 7f90d2c1b980 20 rados->read ofs=0 len=0 2024-05-30T15:58:26.121 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.119+0000 7f90d2c1b980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:26.121 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.119+0000 7f90d2c1b980 20 rados->read ofs=0 len=0 2024-05-30T15:58:26.121 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.123+0000 7f90d2c1b980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T15:58:26.121 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.123+0000 7f90d2c1b980 20 searching for the correct realm 2024-05-30T15:58:26.128 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.127+0000 7f90cff2b640 0 -- 172.21.15.6:0/1506175487 >> [v2:172.21.15.6:6819/1141406589,v1:172.21.15.6:6821/1141406589] conn(0x562c5f4f42f0 msgr2=0x562c5f4f66e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:26.134 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.135+0000 7f90d072c640 0 -- 172.21.15.6:0/1506175487 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f90c4104920 msgr2=0x7f90c40b63f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:26.142 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.143+0000 7f90d2c1b980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T15:58:26.142 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.143+0000 7f90d2c1b980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T15:58:26.142 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.143+0000 7f90d2c1b980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T15:58:26.142 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.143+0000 7f90d2c1b980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T15:58:26.143 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.143+0000 7f90d2c1b980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T15:58:26.143 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.143+0000 7f90d2c1b980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T15:58:26.143 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.143+0000 7f90d2c1b980 20 rados->read ofs=0 len=0 2024-05-30T15:58:26.143 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.143+0000 7f90d2c1b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:26.143 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.143+0000 7f90d2c1b980 20 rados->read ofs=0 len=0 2024-05-30T15:58:26.143 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.143+0000 7f90d2c1b980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:26.143 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.143+0000 7f90d2c1b980 20 rados->read ofs=0 len=0 2024-05-30T15:58:26.144 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.143+0000 7f90d2c1b980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T15:58:26.144 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.143+0000 7f90d2c1b980 20 zone default found 2024-05-30T15:58:26.144 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.143+0000 7f90d2c1b980 4 Realm: () 2024-05-30T15:58:26.144 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.143+0000 7f90d2c1b980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T15:58:26.144 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.143+0000 7f90d2c1b980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T15:58:26.145 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.143+0000 7f90d2c1b980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T15:58:26.145 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.143+0000 7f90d2c1b980 20 zonegroup default 2024-05-30T15:58:26.145 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.143+0000 7f90d2c1b980 20 rados->read ofs=0 len=0 2024-05-30T15:58:26.145 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.143+0000 7f90d2c1b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:26.145 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.143+0000 7f90d2c1b980 20 rados->read ofs=0 len=0 2024-05-30T15:58:26.146 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.143+0000 7f90d2c1b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:26.146 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.143+0000 7f90d2c1b980 20 rados->read ofs=0 len=0 2024-05-30T15:58:26.146 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.147+0000 7f90d2c1b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:26.146 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.147+0000 7f90d2c1b980 20 started sync module instance, tier type = 2024-05-30T15:58:26.146 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.147+0000 7f90d2c1b980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T15:58:26.163 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.163+0000 7f90d2c1b980 20 add_watcher() i=0 2024-05-30T15:58:26.163 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.163+0000 7f90d2c1b980 20 add_watcher() i=1 2024-05-30T15:58:26.164 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.163+0000 7f90d2c1b980 20 add_watcher() i=2 2024-05-30T15:58:26.164 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.163+0000 7f90d2c1b980 20 add_watcher() i=3 2024-05-30T15:58:26.164 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.163+0000 7f90d2c1b980 20 add_watcher() i=4 2024-05-30T15:58:26.164 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.163+0000 7f90d2c1b980 20 add_watcher() i=5 2024-05-30T15:58:26.164 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.163+0000 7f90d2c1b980 20 add_watcher() i=6 2024-05-30T15:58:26.165 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.167+0000 7f90d2c1b980 20 add_watcher() i=7 2024-05-30T15:58:26.166 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.167+0000 7f90d2c1b980 2 all 8 watchers are set, enabling cache 2024-05-30T15:58:26.168 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.167+0000 7f8f86fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:26.169 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.167+0000 7f90d2c1b980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:58:26.170 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.167+0000 7f90d2c1b980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:58:26.170 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.167+0000 7f90d2c1b980 5 note: GC not initialized 2024-05-30T15:58:26.170 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.171+0000 7f8f85fdb640 20 reqs_thread_entry: start 2024-05-30T15:58:26.170 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.171+0000 7f90d2c1b980 20 init_complete bucket index max shards: 11 2024-05-30T15:58:26.170 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.171+0000 7f90d2c1b980 20 Filter name: none 2024-05-30T15:58:26.170 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.171+0000 7f8ef77fe640 20 reqs_thread_entry: start 2024-05-30T15:58:26.170 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.171+0000 7f90d2c1b980 20 rados->read ofs=0 len=0 2024-05-30T15:58:26.171 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.171+0000 7f90d2c1b980 20 rados_obj.operate() r=0 bl.length=190 2024-05-30T15:58:26.171 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.171+0000 7f90d2c1b980 20 rados->read ofs=0 len=0 2024-05-30T15:58:26.171 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.171+0000 7f90d2c1b980 20 rados_obj.operate() r=0 bl.length=404 2024-05-30T15:58:26.171 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.171+0000 7f90d2c1b980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T15:58:26.171 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.171+0000 7f90d2c1b980 20 rados->read ofs=0 len=0 2024-05-30T15:58:26.171 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.171+0000 7f90d2c1b980 20 rados_obj.operate() r=0 bl.length=190 2024-05-30T15:58:26.172 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.171+0000 7f90d2c1b980 20 rados->read ofs=0 len=0 2024-05-30T15:58:26.172 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.171+0000 7f90d2c1b980 20 rados_obj.operate() r=0 bl.length=404 2024-05-30T15:58:26.172 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.171+0000 7f90d2c1b980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T15:58:26.174 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.175+0000 7f8f86fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:26.179 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.179+0000 7f90d2c1b980 20 remove_watcher() i=0 2024-05-30T15:58:26.179 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.179+0000 7f90d2c1b980 2 removed watcher, disabling cache 2024-05-30T15:58:26.181 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.183+0000 7f90d2c1b980 20 remove_watcher() i=1 2024-05-30T15:58:26.184 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.183+0000 7f90d2c1b980 20 remove_watcher() i=2 2024-05-30T15:58:26.186 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.187+0000 7f90d2c1b980 20 remove_watcher() i=3 2024-05-30T15:58:26.188 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.187+0000 7f90d2c1b980 20 remove_watcher() i=4 2024-05-30T15:58:26.189 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.191+0000 7f90d2c1b980 20 remove_watcher() i=5 2024-05-30T15:58:26.190 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.191+0000 7f90d2c1b980 20 remove_watcher() i=6 2024-05-30T15:58:26.192 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.191+0000 7f90d2c1b980 20 remove_watcher() i=7 2024-05-30T15:58:26.207 INFO:tasks.workunit.client.0.smithi006.stderr:{'rgw.main': {'size': 54, 'size_actual': 24576, 'size_utilized': 101, 'size_kb': 1, 'size_kb_actual': 24, 'size_kb_utilized': 1, 'num_objects': 6}} 2024-05-30T15:58:26.207 INFO:tasks.workunit.client.0.smithi006.stderr:TEST: bucket check unlinked does not report normal entries 2024-05-30T15:58:26.207 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T15:58:26.207 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --min-age-hours 0 --dump-keys 2024-05-30T15:58:26.238 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:58:26.238 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:58:26.283 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.283+0000 7f0296888980 20 rados->read ofs=0 len=0 2024-05-30T15:58:26.284 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.283+0000 7f0296888980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:26.284 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.283+0000 7f0296888980 20 realm 2024-05-30T15:58:26.285 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.283+0000 7f0296888980 20 rados->read ofs=0 len=0 2024-05-30T15:58:26.285 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.283+0000 7f0296888980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:26.285 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.283+0000 7f0296888980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T15:58:26.285 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.283+0000 7f0296888980 20 rados->read ofs=0 len=0 2024-05-30T15:58:26.285 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.283+0000 7f0296888980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:26.285 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.283+0000 7f0296888980 20 rados->read ofs=0 len=0 2024-05-30T15:58:26.287 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.287+0000 7f0296888980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:26.287 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.287+0000 7f0296888980 20 rados->read ofs=0 len=0 2024-05-30T15:58:26.287 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.287+0000 7f0296888980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T15:58:26.288 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.287+0000 7f0296888980 20 searching for the correct realm 2024-05-30T15:58:26.295 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.295+0000 7f028bb98640 0 -- 172.21.15.6:0/19083574 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f02840ec940 msgr2=0x7f02840c5410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:26.301 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.303+0000 7f0294b9a640 0 -- 172.21.15.6:0/19083574 >> [v2:172.21.15.204:6800/3687190393,v1:172.21.15.204:6801/3687190393] conn(0x7f027c008030 msgr2=0x7f027c05f210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T15:58:26.309 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.307+0000 7f0296888980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T15:58:26.309 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.307+0000 7f0296888980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T15:58:26.309 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.307+0000 7f0296888980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T15:58:26.309 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.307+0000 7f0296888980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T15:58:26.309 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.307+0000 7f0296888980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T15:58:26.309 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.307+0000 7f0296888980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T15:58:26.309 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.307+0000 7f0296888980 20 rados->read ofs=0 len=0 2024-05-30T15:58:26.309 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.311+0000 7f0296888980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:26.310 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.311+0000 7f0296888980 20 rados->read ofs=0 len=0 2024-05-30T15:58:26.310 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.311+0000 7f0296888980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:26.310 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.311+0000 7f0296888980 20 rados->read ofs=0 len=0 2024-05-30T15:58:26.312 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.311+0000 7f0296888980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T15:58:26.312 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.311+0000 7f0296888980 20 zone default found 2024-05-30T15:58:26.312 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.311+0000 7f0296888980 4 Realm: () 2024-05-30T15:58:26.312 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.311+0000 7f0296888980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T15:58:26.313 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.311+0000 7f0296888980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T15:58:26.313 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.311+0000 7f0296888980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T15:58:26.313 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.311+0000 7f0296888980 20 zonegroup default 2024-05-30T15:58:26.313 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.311+0000 7f0296888980 20 rados->read ofs=0 len=0 2024-05-30T15:58:26.314 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.311+0000 7f0296888980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:26.314 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.311+0000 7f0296888980 20 rados->read ofs=0 len=0 2024-05-30T15:58:26.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.311+0000 7f0296888980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:26.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.311+0000 7f0296888980 20 rados->read ofs=0 len=0 2024-05-30T15:58:26.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.311+0000 7f0296888980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:26.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.311+0000 7f0296888980 20 started sync module instance, tier type = 2024-05-30T15:58:26.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.311+0000 7f0296888980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T15:58:26.333 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.331+0000 7f0296888980 20 add_watcher() i=0 2024-05-30T15:58:26.333 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.331+0000 7f0296888980 20 add_watcher() i=1 2024-05-30T15:58:26.333 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.331+0000 7f0296888980 20 add_watcher() i=2 2024-05-30T15:58:26.333 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.331+0000 7f0296888980 20 add_watcher() i=3 2024-05-30T15:58:26.333 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.331+0000 7f0296888980 20 add_watcher() i=4 2024-05-30T15:58:26.333 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.331+0000 7f0296888980 20 add_watcher() i=5 2024-05-30T15:58:26.333 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.331+0000 7f0296888980 20 add_watcher() i=6 2024-05-30T15:58:26.335 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.335+0000 7f0296888980 20 add_watcher() i=7 2024-05-30T15:58:26.335 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.335+0000 7f0296888980 2 all 8 watchers are set, enabling cache 2024-05-30T15:58:26.339 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.339+0000 7f014b7de640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:26.339 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.339+0000 7f0296888980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:58:26.339 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.339+0000 7f0296888980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:58:26.339 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.339+0000 7f0296888980 5 note: GC not initialized 2024-05-30T15:58:26.339 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.339+0000 7f014a7dc640 20 reqs_thread_entry: start 2024-05-30T15:58:26.339 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.339+0000 7f0296888980 20 init_complete bucket index max shards: 11 2024-05-30T15:58:26.339 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.339+0000 7f0296888980 20 Filter name: none 2024-05-30T15:58:26.339 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.339+0000 7f01437fe640 20 reqs_thread_entry: start 2024-05-30T15:58:26.340 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.339+0000 7f0296888980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-05-30T15:58:26.340 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.339+0000 7f0296888980 20 rados->read ofs=0 len=0 2024-05-30T15:58:26.341 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.339+0000 7f0296888980 20 rados_obj.operate() r=0 bl.length=190 2024-05-30T15:58:26.341 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.339+0000 7f0296888980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-05-30T15:58:26.341 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.339+0000 7f0296888980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-05-30T15:58:26.341 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.339+0000 7f0296888980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 : miss 2024-05-30T15:58:26.341 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.339+0000 7f0296888980 20 rados->read ofs=0 len=0 2024-05-30T15:58:26.341 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.339+0000 7f0296888980 20 rados_obj.operate() r=0 bl.length=404 2024-05-30T15:58:26.341 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.339+0000 7f0296888980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 info.flags=0x17 2024-05-30T15:58:26.341 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.339+0000 7f0296888980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 to cache LRU end 2024-05-30T15:58:26.341 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.339+0000 7f0296888980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T15:58:26.341 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.339+0000 7f0296888980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-05-30T15:58:26.341 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.339+0000 7f0296888980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 2024-05-30T15:58:26.342 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.339+0000 7f0296888980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.0 2024-05-30T15:58:26.342 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.339+0000 7f0296888980 1 NOTICE: finished shard 0 (0 entries found) 2024-05-30T15:58:26.342 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.339+0000 7f0296888980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:26.342 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.343+0000 7f0296888980 1 NOTICE: finished shard 1 (0 entries found) 2024-05-30T15:58:26.342 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.343+0000 7f0296888980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.2 2024-05-30T15:58:26.342 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.343+0000 7f0296888980 1 NOTICE: finished shard 2 (0 entries found) 2024-05-30T15:58:26.342 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.343+0000 7f0296888980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.3 2024-05-30T15:58:26.343 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.343+0000 7f0296888980 1 NOTICE: finished shard 3 (0 entries found) 2024-05-30T15:58:26.343 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.343+0000 7f0296888980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.4 2024-05-30T15:58:26.344 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.343+0000 7f0296888980 1 NOTICE: finished shard 4 (0 entries found) 2024-05-30T15:58:26.344 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.343+0000 7f0296888980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:26.345 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.343+0000 7f0296888980 1 NOTICE: finished shard 5 (0 entries found) 2024-05-30T15:58:26.345 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.343+0000 7f0296888980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.6 2024-05-30T15:58:26.345 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.347+0000 7f0296888980 1 NOTICE: finished shard 6 (0 entries found) 2024-05-30T15:58:26.346 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.347+0000 7f0296888980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.7 2024-05-30T15:58:26.346 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.347+0000 7f0296888980 1 NOTICE: finished shard 7 (0 entries found) 2024-05-30T15:58:26.347 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.347+0000 7f0296888980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:26.347 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.347+0000 7f0296888980 1 NOTICE: finished shard 8 (0 entries found) 2024-05-30T15:58:26.347 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.347+0000 7f0296888980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.9 2024-05-30T15:58:26.348 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.347+0000 7f0296888980 1 NOTICE: finished shard 9 (0 entries found) 2024-05-30T15:58:26.348 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.347+0000 7f0296888980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:26.349 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.347+0000 7f0296888980 1 NOTICE: finished shard 10 (0 entries found) 2024-05-30T15:58:26.349 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.347+0000 7f0296888980 1 NOTICE: finished all shards (0 entries found) 2024-05-30T15:58:26.350 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.351+0000 7f014b7de640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:26.355 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.355+0000 7f0296888980 20 remove_watcher() i=0 2024-05-30T15:58:26.355 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.355+0000 7f0296888980 2 removed watcher, disabling cache 2024-05-30T15:58:26.357 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.359+0000 7f0296888980 20 remove_watcher() i=1 2024-05-30T15:58:26.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.359+0000 7f0296888980 20 remove_watcher() i=2 2024-05-30T15:58:26.362 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.363+0000 7f0296888980 20 remove_watcher() i=3 2024-05-30T15:58:26.364 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.363+0000 7f0296888980 20 remove_watcher() i=4 2024-05-30T15:58:26.365 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.367+0000 7f0296888980 20 remove_watcher() i=5 2024-05-30T15:58:26.366 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.367+0000 7f0296888980 20 remove_watcher() i=6 2024-05-30T15:58:26.368 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:26.367+0000 7f0296888980 20 remove_watcher() i=7 2024-05-30T15:58:26.382 INFO:tasks.workunit.client.0.smithi006.stderr:ceph config set client rgw_debug_inject_set_olh_err 2 2024-05-30T15:58:26.802 INFO:tasks.workunit.client.0.smithi006.stderr:ceph config set client rgw_debug_inject_olh_cancel_modification_err true 2024-05-30T15:58:28.390 INFO:tasks.workunit.client.0.smithi006.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-05-30T15:58:28.391 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bi list --bucket check-bucket --object c 2024-05-30T15:58:28.427 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:58:28.427 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:58:28.473 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.474+0000 7fbc4b77a980 20 rados->read ofs=0 len=0 2024-05-30T15:58:28.475 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.474+0000 7fbc4b77a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:28.475 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.474+0000 7fbc4b77a980 20 realm 2024-05-30T15:58:28.475 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.474+0000 7fbc4b77a980 20 rados->read ofs=0 len=0 2024-05-30T15:58:28.476 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.474+0000 7fbc4b77a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:28.476 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.474+0000 7fbc4b77a980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T15:58:28.476 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.474+0000 7fbc4b77a980 20 rados->read ofs=0 len=0 2024-05-30T15:58:28.476 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.474+0000 7fbc4b77a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:28.476 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.474+0000 7fbc4b77a980 20 rados->read ofs=0 len=0 2024-05-30T15:58:28.477 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.474+0000 7fbc4b77a980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:28.477 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.474+0000 7fbc4b77a980 20 rados->read ofs=0 len=0 2024-05-30T15:58:28.478 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.478+0000 7fbc4b77a980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T15:58:28.478 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.478+0000 7fbc4b77a980 20 searching for the correct realm 2024-05-30T15:58:28.486 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.482+0000 7fbc48a8a640 0 -- 172.21.15.6:0/793910064 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7fbc4406f2b0 msgr2=0x7fbc44079250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:28.490 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.490+0000 7fbc48a8a640 0 -- 172.21.15.6:0/793910064 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7fbb1407ba80 msgr2=0x7fbb1407deb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:28.497 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.498+0000 7fbc4b77a980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T15:58:28.497 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.498+0000 7fbc4b77a980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T15:58:28.498 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.498+0000 7fbc4b77a980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T15:58:28.498 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.498+0000 7fbc4b77a980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T15:58:28.498 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.498+0000 7fbc4b77a980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T15:58:28.498 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.498+0000 7fbc4b77a980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T15:58:28.498 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.498+0000 7fbc4b77a980 20 rados->read ofs=0 len=0 2024-05-30T15:58:28.498 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.498+0000 7fbc4b77a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:28.498 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.498+0000 7fbc4b77a980 20 rados->read ofs=0 len=0 2024-05-30T15:58:28.499 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.498+0000 7fbc4b77a980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:28.499 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.498+0000 7fbc4b77a980 20 rados->read ofs=0 len=0 2024-05-30T15:58:28.499 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.498+0000 7fbc4b77a980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T15:58:28.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.498+0000 7fbc4b77a980 20 zone default found 2024-05-30T15:58:28.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.498+0000 7fbc4b77a980 4 Realm: () 2024-05-30T15:58:28.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.498+0000 7fbc4b77a980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T15:58:28.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.498+0000 7fbc4b77a980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T15:58:28.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.498+0000 7fbc4b77a980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T15:58:28.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.498+0000 7fbc4b77a980 20 zonegroup default 2024-05-30T15:58:28.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.498+0000 7fbc4b77a980 20 rados->read ofs=0 len=0 2024-05-30T15:58:28.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.498+0000 7fbc4b77a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:28.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.498+0000 7fbc4b77a980 20 rados->read ofs=0 len=0 2024-05-30T15:58:28.501 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.498+0000 7fbc4b77a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:28.501 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.498+0000 7fbc4b77a980 20 rados->read ofs=0 len=0 2024-05-30T15:58:28.501 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.498+0000 7fbc4b77a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:28.501 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.498+0000 7fbc4b77a980 20 started sync module instance, tier type = 2024-05-30T15:58:28.501 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.498+0000 7fbc4b77a980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T15:58:28.519 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.518+0000 7fbc4b77a980 20 add_watcher() i=0 2024-05-30T15:58:28.519 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.518+0000 7fbc4b77a980 20 add_watcher() i=1 2024-05-30T15:58:28.519 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.518+0000 7fbc4b77a980 20 add_watcher() i=2 2024-05-30T15:58:28.520 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.518+0000 7fbc4b77a980 20 add_watcher() i=3 2024-05-30T15:58:28.520 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.518+0000 7fbc4b77a980 20 add_watcher() i=4 2024-05-30T15:58:28.520 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.518+0000 7fbc4b77a980 20 add_watcher() i=5 2024-05-30T15:58:28.520 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.518+0000 7fbc4b77a980 20 add_watcher() i=6 2024-05-30T15:58:28.520 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.518+0000 7fbc4b77a980 20 add_watcher() i=7 2024-05-30T15:58:28.520 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.518+0000 7fbc4b77a980 2 all 8 watchers are set, enabling cache 2024-05-30T15:58:28.525 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.522+0000 7fbaff7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:28.526 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.522+0000 7fbc4b77a980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:58:28.526 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.522+0000 7fbc4b77a980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:58:28.526 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.522+0000 7fbc4b77a980 5 note: GC not initialized 2024-05-30T15:58:28.526 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.522+0000 7fbafe7e4640 20 reqs_thread_entry: start 2024-05-30T15:58:28.526 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.522+0000 7fbc4b77a980 20 init_complete bucket index max shards: 11 2024-05-30T15:58:28.526 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.522+0000 7fbc4b77a980 20 Filter name: none 2024-05-30T15:58:28.526 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.522+0000 7fba6ffff640 20 reqs_thread_entry: start 2024-05-30T15:58:28.526 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.522+0000 7fbc4b77a980 20 rados->read ofs=0 len=0 2024-05-30T15:58:28.526 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.522+0000 7fbc4b77a980 20 rados_obj.operate() r=0 bl.length=190 2024-05-30T15:58:28.526 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.522+0000 7fbc4b77a980 20 rados->read ofs=0 len=0 2024-05-30T15:58:28.526 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.522+0000 7fbc4b77a980 20 rados_obj.operate() r=0 bl.length=404 2024-05-30T15:58:28.526 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.522+0000 7fbc4b77a980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T15:58:28.527 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.522+0000 7fbc4b77a980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-05-30T15:58:28.527 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.522+0000 7fbc4b77a980 20 INFO: main: starting shard=0 2024-05-30T15:58:28.527 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.522+0000 7fbc4b77a980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.0 2024-05-30T15:58:28.527 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.526+0000 7fbc4b77a980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:28.527 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.526+0000 7fbc4b77a980 20 INFO: main: starting shard=1 2024-05-30T15:58:28.527 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.526+0000 7fbc4b77a980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:28.527 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.526+0000 7fbc4b77a980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:28.527 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.526+0000 7fbc4b77a980 20 INFO: main: starting shard=2 2024-05-30T15:58:28.527 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.526+0000 7fbc4b77a980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.2 2024-05-30T15:58:28.528 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.526+0000 7fbc4b77a980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:28.528 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.526+0000 7fbc4b77a980 20 INFO: main: starting shard=3 2024-05-30T15:58:28.528 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.526+0000 7fbc4b77a980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.3 2024-05-30T15:58:28.528 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.526+0000 7fbc4b77a980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:28.528 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.526+0000 7fbc4b77a980 20 INFO: main: starting shard=4 2024-05-30T15:58:28.528 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.526+0000 7fbc4b77a980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.4 2024-05-30T15:58:28.528 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.526+0000 7fbc4b77a980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:28.528 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.526+0000 7fbc4b77a980 20 INFO: main: starting shard=5 2024-05-30T15:58:28.529 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.526+0000 7fbc4b77a980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:28.530 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.530+0000 7fbc4b77a980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:28.530 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.530+0000 7fbc4b77a980 20 INFO: main: starting shard=6 2024-05-30T15:58:28.530 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.530+0000 7fbc4b77a980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.6 2024-05-30T15:58:28.530 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.530+0000 7fbc4b77a980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:28.530 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.530+0000 7fbc4b77a980 20 INFO: main: starting shard=7 2024-05-30T15:58:28.530 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.530+0000 7fbc4b77a980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.7 2024-05-30T15:58:28.531 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.530+0000 7fbc4b77a980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:28.531 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.530+0000 7fbc4b77a980 20 INFO: main: starting shard=8 2024-05-30T15:58:28.531 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.530+0000 7fbc4b77a980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:28.532 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.530+0000 7fbc4b77a980 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, marker= 2024-05-30T15:58:28.532 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.530+0000 7fbc4b77a980 20 INFO: main: starting shard=9 2024-05-30T15:58:28.532 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.530+0000 7fbc4b77a980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.9 2024-05-30T15:58:28.533 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.530+0000 7fbc4b77a980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:28.533 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.530+0000 7fbc4b77a980 20 INFO: main: starting shard=10 2024-05-30T15:58:28.533 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.530+0000 7fbc4b77a980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:28.534 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.534+0000 7fbc4b77a980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:28.534 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.534+0000 7fbc4b77a980 20 INFO: main: done 2024-05-30T15:58:28.534 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.534+0000 7fbaff7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:28.539 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.538+0000 7fbc4b77a980 20 remove_watcher() i=0 2024-05-30T15:58:28.539 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.538+0000 7fbc4b77a980 2 removed watcher, disabling cache 2024-05-30T15:58:28.541 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.542+0000 7fbc4b77a980 20 remove_watcher() i=1 2024-05-30T15:58:28.543 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.542+0000 7fbc4b77a980 20 remove_watcher() i=2 2024-05-30T15:58:28.545 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.546+0000 7fbc4b77a980 20 remove_watcher() i=3 2024-05-30T15:58:28.547 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.546+0000 7fbc4b77a980 20 remove_watcher() i=4 2024-05-30T15:58:28.549 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.546+0000 7fbc4b77a980 20 remove_watcher() i=5 2024-05-30T15:58:28.550 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.550+0000 7fbc4b77a980 20 remove_watcher() i=6 2024-05-30T15:58:28.551 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.550+0000 7fbc4b77a980 20 remove_watcher() i=7 2024-05-30T15:58:28.742 INFO:tasks.workunit.client.0.smithi006.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-05-30T15:58:28.743 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bi list --bucket check-bucket --object d 2024-05-30T15:58:28.779 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:58:28.779 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:58:28.827 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.826+0000 7f7ba96c8980 20 rados->read ofs=0 len=0 2024-05-30T15:58:28.829 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.826+0000 7f7ba96c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:28.829 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.826+0000 7f7ba96c8980 20 realm 2024-05-30T15:58:28.829 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.826+0000 7f7ba96c8980 20 rados->read ofs=0 len=0 2024-05-30T15:58:28.829 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.826+0000 7f7ba96c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:28.829 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.826+0000 7f7ba96c8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T15:58:28.829 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.826+0000 7f7ba96c8980 20 rados->read ofs=0 len=0 2024-05-30T15:58:28.829 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.826+0000 7f7ba96c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:28.829 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.826+0000 7f7ba96c8980 20 rados->read ofs=0 len=0 2024-05-30T15:58:28.831 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.830+0000 7f7ba96c8980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:28.831 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.830+0000 7f7ba96c8980 20 rados->read ofs=0 len=0 2024-05-30T15:58:28.831 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.830+0000 7f7ba96c8980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T15:58:28.831 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.830+0000 7f7ba96c8980 20 searching for the correct realm 2024-05-30T15:58:28.838 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.838+0000 7f7ba69d8640 0 -- 172.21.15.6:0/474583319 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f7ba0017020 msgr2=0x7f7ba0072040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T15:58:28.846 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.846+0000 7f7ba69d8640 0 -- 172.21.15.6:0/474583319 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f7a7007bab0 msgr2=0x7f7a7007dee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:28.853 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.850+0000 7f7ba96c8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T15:58:28.853 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.850+0000 7f7ba96c8980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T15:58:28.853 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.850+0000 7f7ba96c8980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T15:58:28.853 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.850+0000 7f7ba96c8980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T15:58:28.853 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.850+0000 7f7ba96c8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T15:58:28.853 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.850+0000 7f7ba96c8980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T15:58:28.854 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.850+0000 7f7ba96c8980 20 rados->read ofs=0 len=0 2024-05-30T15:58:28.854 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.850+0000 7f7ba96c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:28.854 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.850+0000 7f7ba96c8980 20 rados->read ofs=0 len=0 2024-05-30T15:58:28.854 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.854+0000 7f7ba96c8980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:28.854 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.854+0000 7f7ba96c8980 20 rados->read ofs=0 len=0 2024-05-30T15:58:28.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.854+0000 7f7ba96c8980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T15:58:28.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.854+0000 7f7ba96c8980 20 zone default found 2024-05-30T15:58:28.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.854+0000 7f7ba96c8980 4 Realm: () 2024-05-30T15:58:28.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.854+0000 7f7ba96c8980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T15:58:28.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.854+0000 7f7ba96c8980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T15:58:28.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.854+0000 7f7ba96c8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T15:58:28.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.854+0000 7f7ba96c8980 20 zonegroup default 2024-05-30T15:58:28.856 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.854+0000 7f7ba96c8980 20 rados->read ofs=0 len=0 2024-05-30T15:58:28.856 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.854+0000 7f7ba96c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:28.856 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.854+0000 7f7ba96c8980 20 rados->read ofs=0 len=0 2024-05-30T15:58:28.856 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.854+0000 7f7ba96c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:28.856 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.854+0000 7f7ba96c8980 20 rados->read ofs=0 len=0 2024-05-30T15:58:28.856 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.854+0000 7f7ba96c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:28.856 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.854+0000 7f7ba96c8980 20 started sync module instance, tier type = 2024-05-30T15:58:28.857 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.854+0000 7f7ba96c8980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T15:58:28.873 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.870+0000 7f7ba96c8980 20 add_watcher() i=0 2024-05-30T15:58:28.873 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.870+0000 7f7ba96c8980 20 add_watcher() i=1 2024-05-30T15:58:28.873 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.870+0000 7f7ba96c8980 20 add_watcher() i=2 2024-05-30T15:58:28.873 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.870+0000 7f7ba96c8980 20 add_watcher() i=3 2024-05-30T15:58:28.873 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.870+0000 7f7ba96c8980 20 add_watcher() i=4 2024-05-30T15:58:28.873 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.870+0000 7f7ba96c8980 20 add_watcher() i=5 2024-05-30T15:58:28.873 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.870+0000 7f7ba96c8980 20 add_watcher() i=6 2024-05-30T15:58:28.874 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.874+0000 7f7ba96c8980 20 add_watcher() i=7 2024-05-30T15:58:28.874 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.874+0000 7f7ba96c8980 2 all 8 watchers are set, enabling cache 2024-05-30T15:58:28.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.874+0000 7f7a5dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:28.878 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.878+0000 7f7ba96c8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:58:28.878 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.878+0000 7f7ba96c8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:58:28.878 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.878+0000 7f7ba96c8980 5 note: GC not initialized 2024-05-30T15:58:28.878 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.878+0000 7f7a5cfe1640 20 reqs_thread_entry: start 2024-05-30T15:58:28.878 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.878+0000 7f7ba96c8980 20 init_complete bucket index max shards: 11 2024-05-30T15:58:28.878 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.878+0000 7f7ba96c8980 20 Filter name: none 2024-05-30T15:58:28.878 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.878+0000 7f79d67fc640 20 reqs_thread_entry: start 2024-05-30T15:58:28.878 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.878+0000 7f7ba96c8980 20 rados->read ofs=0 len=0 2024-05-30T15:58:28.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.878+0000 7f7ba96c8980 20 rados_obj.operate() r=0 bl.length=190 2024-05-30T15:58:28.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.878+0000 7f7ba96c8980 20 rados->read ofs=0 len=0 2024-05-30T15:58:28.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.878+0000 7f7ba96c8980 20 rados_obj.operate() r=0 bl.length=404 2024-05-30T15:58:28.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.878+0000 7f7ba96c8980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T15:58:28.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.878+0000 7f7ba96c8980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-05-30T15:58:28.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.878+0000 7f7ba96c8980 20 INFO: main: starting shard=0 2024-05-30T15:58:28.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.878+0000 7f7ba96c8980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.0 2024-05-30T15:58:28.880 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.878+0000 7f7ba96c8980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:28.880 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.878+0000 7f7ba96c8980 20 INFO: main: starting shard=1 2024-05-30T15:58:28.880 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.878+0000 7f7ba96c8980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:28.880 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.878+0000 7f7ba96c8980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:28.880 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.878+0000 7f7ba96c8980 20 INFO: main: starting shard=2 2024-05-30T15:58:28.880 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.878+0000 7f7ba96c8980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.2 2024-05-30T15:58:28.881 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.878+0000 7f7ba96c8980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:28.881 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.878+0000 7f7ba96c8980 20 INFO: main: starting shard=3 2024-05-30T15:58:28.881 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.878+0000 7f7ba96c8980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.3 2024-05-30T15:58:28.881 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.882+0000 7f7ba96c8980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:28.881 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.882+0000 7f7ba96c8980 20 INFO: main: starting shard=4 2024-05-30T15:58:28.882 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.882+0000 7f7ba96c8980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.4 2024-05-30T15:58:28.883 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.882+0000 7f7ba96c8980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:28.883 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.882+0000 7f7ba96c8980 20 INFO: main: starting shard=5 2024-05-30T15:58:28.883 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.882+0000 7f7ba96c8980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:28.885 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.882+0000 7f7ba96c8980 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, marker= 2024-05-30T15:58:28.885 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.882+0000 7f7ba96c8980 20 INFO: main: starting shard=6 2024-05-30T15:58:28.885 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.882+0000 7f7ba96c8980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.6 2024-05-30T15:58:28.885 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.886+0000 7f7ba96c8980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:28.885 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.886+0000 7f7ba96c8980 20 INFO: main: starting shard=7 2024-05-30T15:58:28.885 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.886+0000 7f7ba96c8980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.7 2024-05-30T15:58:28.887 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.886+0000 7f7ba96c8980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:28.887 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.886+0000 7f7ba96c8980 20 INFO: main: starting shard=8 2024-05-30T15:58:28.887 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.886+0000 7f7ba96c8980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:28.887 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.886+0000 7f7ba96c8980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:28.887 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.886+0000 7f7ba96c8980 20 INFO: main: starting shard=9 2024-05-30T15:58:28.887 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.886+0000 7f7ba96c8980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.9 2024-05-30T15:58:28.888 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.886+0000 7f7ba96c8980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:28.888 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.886+0000 7f7ba96c8980 20 INFO: main: starting shard=10 2024-05-30T15:58:28.888 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.886+0000 7f7ba96c8980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:28.889 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.886+0000 7f7ba96c8980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:28.889 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.886+0000 7f7ba96c8980 20 INFO: main: done 2024-05-30T15:58:28.890 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.890+0000 7f7a5dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:28.895 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.894+0000 7f7ba96c8980 20 remove_watcher() i=0 2024-05-30T15:58:28.895 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.894+0000 7f7ba96c8980 2 removed watcher, disabling cache 2024-05-30T15:58:28.897 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.898+0000 7f7ba96c8980 20 remove_watcher() i=1 2024-05-30T15:58:28.900 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.898+0000 7f7ba96c8980 20 remove_watcher() i=2 2024-05-30T15:58:28.902 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.902+0000 7f7ba96c8980 20 remove_watcher() i=3 2024-05-30T15:58:28.904 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.902+0000 7f7ba96c8980 20 remove_watcher() i=4 2024-05-30T15:58:28.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.906+0000 7f7ba96c8980 20 remove_watcher() i=5 2024-05-30T15:58:28.907 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.906+0000 7f7ba96c8980 20 remove_watcher() i=6 2024-05-30T15:58:28.908 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:28.906+0000 7f7ba96c8980 20 remove_watcher() i=7 2024-05-30T15:58:29.142 INFO:tasks.workunit.client.0.smithi006.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-05-30T15:58:29.143 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bi list --bucket check-bucket --object e 2024-05-30T15:58:29.179 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:58:29.179 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:58:29.225 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.222+0000 7f79c4b33980 20 rados->read ofs=0 len=0 2024-05-30T15:58:29.227 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.226+0000 7f79c4b33980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:29.227 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.226+0000 7f79c4b33980 20 realm 2024-05-30T15:58:29.227 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.226+0000 7f79c4b33980 20 rados->read ofs=0 len=0 2024-05-30T15:58:29.227 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.226+0000 7f79c4b33980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:29.227 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.226+0000 7f79c4b33980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T15:58:29.227 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.226+0000 7f79c4b33980 20 rados->read ofs=0 len=0 2024-05-30T15:58:29.227 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.226+0000 7f79c4b33980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:29.227 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.226+0000 7f79c4b33980 20 rados->read ofs=0 len=0 2024-05-30T15:58:29.229 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.226+0000 7f79c4b33980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:29.229 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.226+0000 7f79c4b33980 20 rados->read ofs=0 len=0 2024-05-30T15:58:29.230 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.230+0000 7f79c4b33980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T15:58:29.231 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.230+0000 7f79c4b33980 20 searching for the correct realm 2024-05-30T15:58:29.236 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.234+0000 7f79c2e45640 0 -- 172.21.15.6:0/3265873969 >> [v2:172.21.15.6:6808/1897362715,v1:172.21.15.6:6809/1897362715] conn(0x55f8987c79c0 msgr2=0x55f8987c9e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:29.243 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.242+0000 7f79c1e43640 0 -- 172.21.15.6:0/3265873969 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f79b4034060 msgr2=0x7f79b4067490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:29.250 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.250+0000 7f79c4b33980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T15:58:29.250 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.250+0000 7f79c4b33980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T15:58:29.250 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.250+0000 7f79c4b33980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T15:58:29.250 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.250+0000 7f79c4b33980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T15:58:29.250 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.250+0000 7f79c4b33980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T15:58:29.250 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.250+0000 7f79c4b33980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T15:58:29.250 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.250+0000 7f79c4b33980 20 rados->read ofs=0 len=0 2024-05-30T15:58:29.251 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.250+0000 7f79c4b33980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:29.251 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.250+0000 7f79c4b33980 20 rados->read ofs=0 len=0 2024-05-30T15:58:29.251 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.250+0000 7f79c4b33980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:29.251 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.250+0000 7f79c4b33980 20 rados->read ofs=0 len=0 2024-05-30T15:58:29.252 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.250+0000 7f79c4b33980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T15:58:29.252 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.250+0000 7f79c4b33980 20 zone default found 2024-05-30T15:58:29.252 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.250+0000 7f79c4b33980 4 Realm: () 2024-05-30T15:58:29.252 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.250+0000 7f79c4b33980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T15:58:29.252 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.250+0000 7f79c4b33980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T15:58:29.252 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.250+0000 7f79c4b33980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T15:58:29.252 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.250+0000 7f79c4b33980 20 zonegroup default 2024-05-30T15:58:29.252 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.250+0000 7f79c4b33980 20 rados->read ofs=0 len=0 2024-05-30T15:58:29.252 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.250+0000 7f79c4b33980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:29.253 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.250+0000 7f79c4b33980 20 rados->read ofs=0 len=0 2024-05-30T15:58:29.253 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.250+0000 7f79c4b33980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:29.253 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.250+0000 7f79c4b33980 20 rados->read ofs=0 len=0 2024-05-30T15:58:29.253 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.250+0000 7f79c4b33980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:29.253 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.250+0000 7f79c4b33980 20 started sync module instance, tier type = 2024-05-30T15:58:29.253 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.250+0000 7f79c4b33980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T15:58:29.270 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.270+0000 7f79c4b33980 20 add_watcher() i=0 2024-05-30T15:58:29.270 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.270+0000 7f79c4b33980 20 add_watcher() i=1 2024-05-30T15:58:29.270 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.270+0000 7f79c4b33980 20 add_watcher() i=2 2024-05-30T15:58:29.270 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.270+0000 7f79c4b33980 20 add_watcher() i=3 2024-05-30T15:58:29.270 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.270+0000 7f79c4b33980 20 add_watcher() i=4 2024-05-30T15:58:29.271 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.270+0000 7f79c4b33980 20 add_watcher() i=5 2024-05-30T15:58:29.271 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.270+0000 7f79c4b33980 20 add_watcher() i=6 2024-05-30T15:58:29.271 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.270+0000 7f79c4b33980 20 add_watcher() i=7 2024-05-30T15:58:29.271 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.270+0000 7f79c4b33980 2 all 8 watchers are set, enabling cache 2024-05-30T15:58:29.274 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.274+0000 7f7878fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:29.275 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.274+0000 7f79c4b33980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:58:29.275 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.274+0000 7f79c4b33980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:58:29.276 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.274+0000 7f79c4b33980 5 note: GC not initialized 2024-05-30T15:58:29.276 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.274+0000 7f77f37fe640 20 reqs_thread_entry: start 2024-05-30T15:58:29.276 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.274+0000 7f79c4b33980 20 init_complete bucket index max shards: 11 2024-05-30T15:58:29.276 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.274+0000 7f79c4b33980 20 Filter name: none 2024-05-30T15:58:29.276 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.274+0000 7f77f17fa640 20 reqs_thread_entry: start 2024-05-30T15:58:29.276 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.274+0000 7f79c4b33980 20 rados->read ofs=0 len=0 2024-05-30T15:58:29.276 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.274+0000 7f79c4b33980 20 rados_obj.operate() r=0 bl.length=190 2024-05-30T15:58:29.276 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.274+0000 7f79c4b33980 20 rados->read ofs=0 len=0 2024-05-30T15:58:29.278 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.274+0000 7f79c4b33980 20 rados_obj.operate() r=0 bl.length=404 2024-05-30T15:58:29.278 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.274+0000 7f79c4b33980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T15:58:29.278 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.274+0000 7f79c4b33980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-05-30T15:58:29.278 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.274+0000 7f79c4b33980 20 INFO: main: starting shard=0 2024-05-30T15:58:29.278 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.274+0000 7f79c4b33980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.0 2024-05-30T15:58:29.278 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.274+0000 7f79c4b33980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:29.279 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.274+0000 7f79c4b33980 20 INFO: main: starting shard=1 2024-05-30T15:58:29.279 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.274+0000 7f79c4b33980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:29.279 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.278+0000 7f79c4b33980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-05-30T15:58:29.279 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.278+0000 7f79c4b33980 20 INFO: main: starting shard=2 2024-05-30T15:58:29.279 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.278+0000 7f79c4b33980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.2 2024-05-30T15:58:29.279 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.278+0000 7f79c4b33980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:29.279 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.278+0000 7f79c4b33980 20 INFO: main: starting shard=3 2024-05-30T15:58:29.280 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.278+0000 7f79c4b33980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.3 2024-05-30T15:58:29.280 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.278+0000 7f79c4b33980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:29.280 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.278+0000 7f79c4b33980 20 INFO: main: starting shard=4 2024-05-30T15:58:29.280 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.278+0000 7f79c4b33980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.4 2024-05-30T15:58:29.280 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.278+0000 7f79c4b33980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:29.280 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.278+0000 7f79c4b33980 20 INFO: main: starting shard=5 2024-05-30T15:58:29.280 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.278+0000 7f79c4b33980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:29.281 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.282+0000 7f79c4b33980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:29.281 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.282+0000 7f79c4b33980 20 INFO: main: starting shard=6 2024-05-30T15:58:29.281 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.282+0000 7f79c4b33980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.6 2024-05-30T15:58:29.282 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.282+0000 7f79c4b33980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:29.282 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.282+0000 7f79c4b33980 20 INFO: main: starting shard=7 2024-05-30T15:58:29.282 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.282+0000 7f79c4b33980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.7 2024-05-30T15:58:29.283 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.282+0000 7f79c4b33980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:29.283 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.282+0000 7f79c4b33980 20 INFO: main: starting shard=8 2024-05-30T15:58:29.283 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.282+0000 7f79c4b33980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:29.284 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.282+0000 7f79c4b33980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:29.284 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.282+0000 7f79c4b33980 20 INFO: main: starting shard=9 2024-05-30T15:58:29.284 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.282+0000 7f79c4b33980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.9 2024-05-30T15:58:29.284 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.282+0000 7f79c4b33980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:29.284 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.282+0000 7f79c4b33980 20 INFO: main: starting shard=10 2024-05-30T15:58:29.284 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.282+0000 7f79c4b33980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:29.285 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.286+0000 7f79c4b33980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:29.285 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.286+0000 7f79c4b33980 20 INFO: main: done 2024-05-30T15:58:29.286 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.286+0000 7f7878fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:29.291 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.290+0000 7f79c4b33980 20 remove_watcher() i=0 2024-05-30T15:58:29.291 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.290+0000 7f79c4b33980 2 removed watcher, disabling cache 2024-05-30T15:58:29.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.294+0000 7f79c4b33980 20 remove_watcher() i=1 2024-05-30T15:58:29.295 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.294+0000 7f79c4b33980 20 remove_watcher() i=2 2024-05-30T15:58:29.298 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.298+0000 7f79c4b33980 20 remove_watcher() i=3 2024-05-30T15:58:29.303 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.298+0000 7f79c4b33980 20 remove_watcher() i=4 2024-05-30T15:58:29.303 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.302+0000 7f79c4b33980 20 remove_watcher() i=5 2024-05-30T15:58:29.303 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.302+0000 7f79c4b33980 20 remove_watcher() i=6 2024-05-30T15:58:29.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.302+0000 7f79c4b33980 20 remove_watcher() i=7 2024-05-30T15:58:29.454 INFO:tasks.workunit.client.0.smithi006.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-05-30T15:58:29.455 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bi list --bucket check-bucket --object f 2024-05-30T15:58:29.491 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:58:29.491 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:58:29.539 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.538+0000 7f5d4960d980 20 rados->read ofs=0 len=0 2024-05-30T15:58:29.541 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.538+0000 7f5d4960d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:29.541 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.538+0000 7f5d4960d980 20 realm 2024-05-30T15:58:29.541 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.538+0000 7f5d4960d980 20 rados->read ofs=0 len=0 2024-05-30T15:58:29.541 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.538+0000 7f5d4960d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:29.541 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.538+0000 7f5d4960d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T15:58:29.541 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.538+0000 7f5d4960d980 20 rados->read ofs=0 len=0 2024-05-30T15:58:29.541 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.538+0000 7f5d4960d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:29.542 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.538+0000 7f5d4960d980 20 rados->read ofs=0 len=0 2024-05-30T15:58:29.543 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.542+0000 7f5d4960d980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:29.543 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.542+0000 7f5d4960d980 20 rados->read ofs=0 len=0 2024-05-30T15:58:29.543 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.542+0000 7f5d4960d980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T15:58:29.543 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.542+0000 7f5d4960d980 20 searching for the correct realm 2024-05-30T15:58:29.549 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.550+0000 7f5d4791f640 0 -- 172.21.15.6:0/2706202770 >> [v2:172.21.15.6:6808/1897362715,v1:172.21.15.6:6809/1897362715] conn(0x55a1481bb2e0 msgr2=0x55a1481bd780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:29.556 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.554+0000 7f5d4691d640 0 -- 172.21.15.6:0/2706202770 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f5d380d0290 msgr2=0x7f5d38116a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T15:58:29.564 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.562+0000 7f5d4960d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T15:58:29.565 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.562+0000 7f5d4960d980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T15:58:29.565 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.562+0000 7f5d4960d980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T15:58:29.565 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.562+0000 7f5d4960d980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T15:58:29.565 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.562+0000 7f5d4960d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T15:58:29.565 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.562+0000 7f5d4960d980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T15:58:29.565 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.562+0000 7f5d4960d980 20 rados->read ofs=0 len=0 2024-05-30T15:58:29.565 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.562+0000 7f5d4960d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:29.565 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.562+0000 7f5d4960d980 20 rados->read ofs=0 len=0 2024-05-30T15:58:29.566 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.566+0000 7f5d4960d980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:29.566 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.566+0000 7f5d4960d980 20 rados->read ofs=0 len=0 2024-05-30T15:58:29.566 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.566+0000 7f5d4960d980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T15:58:29.567 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.566+0000 7f5d4960d980 20 zone default found 2024-05-30T15:58:29.567 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.566+0000 7f5d4960d980 4 Realm: () 2024-05-30T15:58:29.567 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.566+0000 7f5d4960d980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T15:58:29.567 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.566+0000 7f5d4960d980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T15:58:29.567 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.566+0000 7f5d4960d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T15:58:29.567 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.566+0000 7f5d4960d980 20 zonegroup default 2024-05-30T15:58:29.567 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.566+0000 7f5d4960d980 20 rados->read ofs=0 len=0 2024-05-30T15:58:29.567 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.566+0000 7f5d4960d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:29.567 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.566+0000 7f5d4960d980 20 rados->read ofs=0 len=0 2024-05-30T15:58:29.568 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.566+0000 7f5d4960d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:29.568 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.566+0000 7f5d4960d980 20 rados->read ofs=0 len=0 2024-05-30T15:58:29.568 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.566+0000 7f5d4960d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:29.568 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.566+0000 7f5d4960d980 20 started sync module instance, tier type = 2024-05-30T15:58:29.568 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.566+0000 7f5d4960d980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T15:58:29.585 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.586+0000 7f5d4960d980 20 add_watcher() i=0 2024-05-30T15:58:29.585 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.586+0000 7f5d4960d980 20 add_watcher() i=1 2024-05-30T15:58:29.586 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.586+0000 7f5d4960d980 20 add_watcher() i=2 2024-05-30T15:58:29.586 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.586+0000 7f5d4960d980 20 add_watcher() i=3 2024-05-30T15:58:29.586 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.586+0000 7f5d4960d980 20 add_watcher() i=4 2024-05-30T15:58:29.586 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.586+0000 7f5d4960d980 20 add_watcher() i=5 2024-05-30T15:58:29.586 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.586+0000 7f5d4960d980 20 add_watcher() i=6 2024-05-30T15:58:29.588 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.586+0000 7f5d4960d980 20 add_watcher() i=7 2024-05-30T15:58:29.588 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.586+0000 7f5d4960d980 2 all 8 watchers are set, enabling cache 2024-05-30T15:58:29.591 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.590+0000 7f5bfdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:29.592 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.590+0000 7f5d4960d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:58:29.593 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.590+0000 7f5d4960d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:58:29.593 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.590+0000 7f5d4960d980 5 note: GC not initialized 2024-05-30T15:58:29.593 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.590+0000 7f5bfcfe1640 20 reqs_thread_entry: start 2024-05-30T15:58:29.593 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.590+0000 7f5d4960d980 20 init_complete bucket index max shards: 11 2024-05-30T15:58:29.593 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.590+0000 7f5d4960d980 20 Filter name: none 2024-05-30T15:58:29.593 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.590+0000 7f5b767fc640 20 reqs_thread_entry: start 2024-05-30T15:58:29.593 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.590+0000 7f5d4960d980 20 rados->read ofs=0 len=0 2024-05-30T15:58:29.593 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.590+0000 7f5d4960d980 20 rados_obj.operate() r=0 bl.length=190 2024-05-30T15:58:29.594 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.590+0000 7f5d4960d980 20 rados->read ofs=0 len=0 2024-05-30T15:58:29.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.590+0000 7f5d4960d980 20 rados_obj.operate() r=0 bl.length=404 2024-05-30T15:58:29.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.590+0000 7f5d4960d980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T15:58:29.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.590+0000 7f5d4960d980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-05-30T15:58:29.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.590+0000 7f5d4960d980 20 INFO: main: starting shard=0 2024-05-30T15:58:29.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.590+0000 7f5d4960d980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.0 2024-05-30T15:58:29.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.594+0000 7f5d4960d980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:29.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.594+0000 7f5d4960d980 20 INFO: main: starting shard=1 2024-05-30T15:58:29.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.594+0000 7f5d4960d980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:29.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.594+0000 7f5d4960d980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:29.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.594+0000 7f5d4960d980 20 INFO: main: starting shard=2 2024-05-30T15:58:29.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.594+0000 7f5d4960d980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.2 2024-05-30T15:58:29.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.594+0000 7f5d4960d980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:29.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.594+0000 7f5d4960d980 20 INFO: main: starting shard=3 2024-05-30T15:58:29.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.594+0000 7f5d4960d980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.3 2024-05-30T15:58:29.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.594+0000 7f5d4960d980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:29.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.594+0000 7f5d4960d980 20 INFO: main: starting shard=4 2024-05-30T15:58:29.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.594+0000 7f5d4960d980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.4 2024-05-30T15:58:29.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.594+0000 7f5d4960d980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:29.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.594+0000 7f5d4960d980 20 INFO: main: starting shard=5 2024-05-30T15:58:29.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.594+0000 7f5d4960d980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:29.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.598+0000 7f5d4960d980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:29.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.598+0000 7f5d4960d980 20 INFO: main: starting shard=6 2024-05-30T15:58:29.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.598+0000 7f5d4960d980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.6 2024-05-30T15:58:29.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.598+0000 7f5d4960d980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:29.599 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.598+0000 7f5d4960d980 20 INFO: main: starting shard=7 2024-05-30T15:58:29.599 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.598+0000 7f5d4960d980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.7 2024-05-30T15:58:29.600 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.598+0000 7f5d4960d980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:29.600 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.598+0000 7f5d4960d980 20 INFO: main: starting shard=8 2024-05-30T15:58:29.600 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.598+0000 7f5d4960d980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:29.600 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.598+0000 7f5d4960d980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:29.600 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.598+0000 7f5d4960d980 20 INFO: main: starting shard=9 2024-05-30T15:58:29.600 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.598+0000 7f5d4960d980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.9 2024-05-30T15:58:29.601 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.598+0000 7f5d4960d980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:29.601 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.598+0000 7f5d4960d980 20 INFO: main: starting shard=10 2024-05-30T15:58:29.601 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.598+0000 7f5d4960d980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:29.602 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.602+0000 7f5d4960d980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-05-30T15:58:29.602 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.602+0000 7f5d4960d980 20 INFO: main: done 2024-05-30T15:58:29.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.602+0000 7f5bfdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:29.608 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.606+0000 7f5d4960d980 20 remove_watcher() i=0 2024-05-30T15:58:29.608 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.606+0000 7f5d4960d980 2 removed watcher, disabling cache 2024-05-30T15:58:29.610 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.610+0000 7f5d4960d980 20 remove_watcher() i=1 2024-05-30T15:58:29.613 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.610+0000 7f5d4960d980 20 remove_watcher() i=2 2024-05-30T15:58:29.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.614+0000 7f5d4960d980 20 remove_watcher() i=3 2024-05-30T15:58:29.617 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.618+0000 7f5d4960d980 20 remove_watcher() i=4 2024-05-30T15:58:29.619 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.618+0000 7f5d4960d980 20 remove_watcher() i=5 2024-05-30T15:58:29.620 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.618+0000 7f5d4960d980 20 remove_watcher() i=6 2024-05-30T15:58:29.621 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:29.622+0000 7f5d4960d980 20 remove_watcher() i=7 2024-05-30T15:58:29.663 INFO:tasks.workunit.client.0.smithi006.stderr:ceph config rm client rgw_debug_inject_set_olh_err 2024-05-30T15:58:30.086 INFO:tasks.workunit.client.0.smithi006.stderr:ceph config rm client rgw_debug_inject_olh_cancel_modification_err 2024-05-30T15:58:30.525 INFO:tasks.workunit.client.0.smithi006.stderr:TEST: bucket check unlinked finds unlistable entries 2024-05-30T15:58:30.525 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T15:58:30.525 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --min-age-hours 0 --dump-keys 2024-05-30T15:58:30.562 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:58:30.562 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:58:30.606 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.606+0000 7f6ccd827980 20 rados->read ofs=0 len=0 2024-05-30T15:58:30.608 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.606+0000 7f6ccd827980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:30.608 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.606+0000 7f6ccd827980 20 realm 2024-05-30T15:58:30.608 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.606+0000 7f6ccd827980 20 rados->read ofs=0 len=0 2024-05-30T15:58:30.609 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.606+0000 7f6ccd827980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:30.609 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.606+0000 7f6ccd827980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T15:58:30.609 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.606+0000 7f6ccd827980 20 rados->read ofs=0 len=0 2024-05-30T15:58:30.609 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.606+0000 7f6ccd827980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:30.609 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.606+0000 7f6ccd827980 20 rados->read ofs=0 len=0 2024-05-30T15:58:30.610 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.610+0000 7f6ccd827980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:30.610 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.610+0000 7f6ccd827980 20 rados->read ofs=0 len=0 2024-05-30T15:58:30.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.610+0000 7f6ccd827980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T15:58:30.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.610+0000 7f6ccd827980 20 searching for the correct realm 2024-05-30T15:58:30.617 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.614+0000 7f6ccab37640 0 -- 172.21.15.6:0/3683640636 >> [v2:172.21.15.6:6808/1897362715,v1:172.21.15.6:6809/1897362715] conn(0x5633ed837d70 msgr2=0x5633ed83a1e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:30.623 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.622+0000 7f6ccb338640 0 -- 172.21.15.6:0/3683640636 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f6cc00073c0 msgr2=0x7f6cc0007840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:30.631 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.630+0000 7f6ccd827980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T15:58:30.631 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.630+0000 7f6ccd827980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T15:58:30.631 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.630+0000 7f6ccd827980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T15:58:30.631 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.630+0000 7f6ccd827980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T15:58:30.631 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.630+0000 7f6ccd827980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T15:58:30.631 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.630+0000 7f6ccd827980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T15:58:30.631 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.630+0000 7f6ccd827980 20 rados->read ofs=0 len=0 2024-05-30T15:58:30.631 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.630+0000 7f6ccd827980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:30.631 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.630+0000 7f6ccd827980 20 rados->read ofs=0 len=0 2024-05-30T15:58:30.631 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.630+0000 7f6ccd827980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:30.632 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.630+0000 7f6ccd827980 20 rados->read ofs=0 len=0 2024-05-30T15:58:30.632 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.630+0000 7f6ccd827980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T15:58:30.632 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.630+0000 7f6ccd827980 20 zone default found 2024-05-30T15:58:30.633 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.630+0000 7f6ccd827980 4 Realm: () 2024-05-30T15:58:30.633 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.630+0000 7f6ccd827980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T15:58:30.633 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.630+0000 7f6ccd827980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T15:58:30.633 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.630+0000 7f6ccd827980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T15:58:30.633 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.630+0000 7f6ccd827980 20 zonegroup default 2024-05-30T15:58:30.633 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.630+0000 7f6ccd827980 20 rados->read ofs=0 len=0 2024-05-30T15:58:30.633 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.630+0000 7f6ccd827980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:30.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.630+0000 7f6ccd827980 20 rados->read ofs=0 len=0 2024-05-30T15:58:30.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.634+0000 7f6ccd827980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:30.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.634+0000 7f6ccd827980 20 rados->read ofs=0 len=0 2024-05-30T15:58:30.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.634+0000 7f6ccd827980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:30.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.634+0000 7f6ccd827980 20 started sync module instance, tier type = 2024-05-30T15:58:30.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.634+0000 7f6ccd827980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T15:58:30.654 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.654+0000 7f6ccd827980 20 add_watcher() i=0 2024-05-30T15:58:30.654 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.654+0000 7f6ccd827980 20 add_watcher() i=1 2024-05-30T15:58:30.654 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.654+0000 7f6ccd827980 20 add_watcher() i=2 2024-05-30T15:58:30.654 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.654+0000 7f6ccd827980 20 add_watcher() i=3 2024-05-30T15:58:30.654 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.654+0000 7f6ccd827980 20 add_watcher() i=4 2024-05-30T15:58:30.655 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.654+0000 7f6ccd827980 20 add_watcher() i=5 2024-05-30T15:58:30.655 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.654+0000 7f6ccd827980 20 add_watcher() i=6 2024-05-30T15:58:30.656 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.654+0000 7f6ccd827980 20 add_watcher() i=7 2024-05-30T15:58:30.657 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.654+0000 7f6ccd827980 2 all 8 watchers are set, enabling cache 2024-05-30T15:58:30.659 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.658+0000 7f6b81fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:30.660 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.658+0000 7f6ccd827980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:58:30.660 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.658+0000 7f6ccd827980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:58:30.661 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.658+0000 7f6ccd827980 5 note: GC not initialized 2024-05-30T15:58:30.661 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.658+0000 7f6b80fe1640 20 reqs_thread_entry: start 2024-05-30T15:58:30.661 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.658+0000 7f6ccd827980 20 init_complete bucket index max shards: 11 2024-05-30T15:58:30.661 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.658+0000 7f6ccd827980 20 Filter name: none 2024-05-30T15:58:30.661 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.658+0000 7f6afa7fc640 20 reqs_thread_entry: start 2024-05-30T15:58:30.661 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.658+0000 7f6ccd827980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-05-30T15:58:30.661 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.658+0000 7f6ccd827980 20 rados->read ofs=0 len=0 2024-05-30T15:58:30.662 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.658+0000 7f6ccd827980 20 rados_obj.operate() r=0 bl.length=190 2024-05-30T15:58:30.662 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.662+0000 7f6ccd827980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-05-30T15:58:30.662 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.662+0000 7f6ccd827980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-05-30T15:58:30.662 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.662+0000 7f6ccd827980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 : miss 2024-05-30T15:58:30.662 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.662+0000 7f6ccd827980 20 rados->read ofs=0 len=0 2024-05-30T15:58:30.662 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.662+0000 7f6ccd827980 20 rados_obj.operate() r=0 bl.length=404 2024-05-30T15:58:30.663 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.662+0000 7f6ccd827980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 info.flags=0x17 2024-05-30T15:58:30.663 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.662+0000 7f6ccd827980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 to cache LRU end 2024-05-30T15:58:30.663 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.662+0000 7f6ccd827980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T15:58:30.663 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.662+0000 7f6ccd827980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-05-30T15:58:30.663 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.662+0000 7f6ccd827980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 2024-05-30T15:58:30.663 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.662+0000 7f6ccd827980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.0 2024-05-30T15:58:30.663 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.662+0000 7f6ccd827980 1 NOTICE: finished shard 0 (0 entries found) 2024-05-30T15:58:30.663 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.662+0000 7f6ccd827980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:30.663 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.662+0000 7f6ccd827980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.2 2024-05-30T15:58:30.664 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.662+0000 7f6ccd827980 1 NOTICE: finished shard 2 (0 entries found) 2024-05-30T15:58:30.664 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.662+0000 7f6ccd827980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.3 2024-05-30T15:58:30.664 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.662+0000 7f6ccd827980 1 NOTICE: finished shard 3 (0 entries found) 2024-05-30T15:58:30.665 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.662+0000 7f6ccd827980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.4 2024-05-30T15:58:30.666 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.666+0000 7f6ccd827980 1 NOTICE: finished shard 4 (0 entries found) 2024-05-30T15:58:30.666 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.666+0000 7f6ccd827980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:30.667 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.666+0000 7f6ccd827980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.6 2024-05-30T15:58:30.668 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.666+0000 7f6ccd827980 1 NOTICE: finished shard 6 (0 entries found) 2024-05-30T15:58:30.668 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.666+0000 7f6ccd827980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.7 2024-05-30T15:58:30.668 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.666+0000 7f6ccd827980 1 NOTICE: finished shard 7 (0 entries found) 2024-05-30T15:58:30.669 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.666+0000 7f6ccd827980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:30.669 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.670+0000 7f6ccd827980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.9 2024-05-30T15:58:30.670 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.670+0000 7f6ccd827980 1 NOTICE: finished shard 9 (0 entries found) 2024-05-30T15:58:30.670 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.670+0000 7f6ccd827980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:30.672 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.670+0000 7f6ccd827980 1 NOTICE: finished shard 1 (1 entries found) 2024-05-30T15:58:30.672 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.670+0000 7f6ccd827980 1 NOTICE: finished shard 5 (1 entries found) 2024-05-30T15:58:30.672 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.670+0000 7f6ccd827980 1 NOTICE: finished shard 8 (1 entries found) 2024-05-30T15:58:30.673 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.670+0000 7f6ccd827980 1 NOTICE: finished shard 10 (1 entries found) 2024-05-30T15:58:30.673 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.670+0000 7f6ccd827980 1 NOTICE: finished all shards (4 entries found) 2024-05-30T15:58:30.674 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.674+0000 7f6b81fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:30.679 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.678+0000 7f6ccd827980 20 remove_watcher() i=0 2024-05-30T15:58:30.679 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.678+0000 7f6ccd827980 2 removed watcher, disabling cache 2024-05-30T15:58:30.681 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.678+0000 7f6ccd827980 20 remove_watcher() i=1 2024-05-30T15:58:30.683 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.682+0000 7f6ccd827980 20 remove_watcher() i=2 2024-05-30T15:58:30.685 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.686+0000 7f6ccd827980 20 remove_watcher() i=3 2024-05-30T15:58:30.687 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.686+0000 7f6ccd827980 20 remove_watcher() i=4 2024-05-30T15:58:30.689 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.690+0000 7f6ccd827980 20 remove_watcher() i=5 2024-05-30T15:58:30.690 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.690+0000 7f6ccd827980 20 remove_watcher() i=6 2024-05-30T15:58:30.692 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:30.690+0000 7f6ccd827980 20 remove_watcher() i=7 2024-05-30T15:58:30.706 INFO:tasks.workunit.client.0.smithi006.stderr:TEST: unlinked entries are not listable 2024-05-30T15:58:30.706 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T15:58:30.859 INFO:tasks.workunit.client.0.smithi006.stderr:TEST: GET returns 404 for unlinked entry keys that have no other versions 2024-05-30T15:58:30.859 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T15:58:31.078 INFO:tasks.workunit.client.0.smithi006.stderr:TEST: bucket check unlinked fixes unlistable entries 2024-05-30T15:58:31.078 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T15:58:31.079 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --fix --min-age-hours 0 --rgw-olh-pending-timeout-sec 0 --dump-keys 2024-05-30T15:58:31.121 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:58:31.121 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:58:31.167 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.166+0000 7fd033cd0980 20 rados->read ofs=0 len=0 2024-05-30T15:58:31.169 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.166+0000 7fd033cd0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:31.169 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.166+0000 7fd033cd0980 20 realm 2024-05-30T15:58:31.169 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.166+0000 7fd033cd0980 20 rados->read ofs=0 len=0 2024-05-30T15:58:31.169 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.170+0000 7fd033cd0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:31.170 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.170+0000 7fd033cd0980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T15:58:31.170 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.170+0000 7fd033cd0980 20 rados->read ofs=0 len=0 2024-05-30T15:58:31.170 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.170+0000 7fd033cd0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:31.170 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.170+0000 7fd033cd0980 20 rados->read ofs=0 len=0 2024-05-30T15:58:31.171 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.170+0000 7fd033cd0980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:31.171 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.170+0000 7fd033cd0980 20 rados->read ofs=0 len=0 2024-05-30T15:58:31.171 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.170+0000 7fd033cd0980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T15:58:31.172 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.170+0000 7fd033cd0980 20 searching for the correct realm 2024-05-30T15:58:31.178 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.178+0000 7fd030fe0640 0 -- 172.21.15.6:0/4121678357 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7fd02401a020 msgr2=0x7fd024061950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T15:58:31.185 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.186+0000 7fd030fe0640 0 -- 172.21.15.6:0/4121678357 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7fcefc07ba50 msgr2=0x7fcefc07de80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:31.192 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.190+0000 7fd033cd0980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T15:58:31.192 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.190+0000 7fd033cd0980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T15:58:31.192 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.190+0000 7fd033cd0980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T15:58:31.193 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.190+0000 7fd033cd0980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T15:58:31.193 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.190+0000 7fd033cd0980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T15:58:31.193 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.190+0000 7fd033cd0980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T15:58:31.193 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.190+0000 7fd033cd0980 20 rados->read ofs=0 len=0 2024-05-30T15:58:31.193 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.190+0000 7fd033cd0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:31.193 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.190+0000 7fd033cd0980 20 rados->read ofs=0 len=0 2024-05-30T15:58:31.193 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.194+0000 7fd033cd0980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:31.193 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.194+0000 7fd033cd0980 20 rados->read ofs=0 len=0 2024-05-30T15:58:31.194 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.194+0000 7fd033cd0980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T15:58:31.194 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.194+0000 7fd033cd0980 20 zone default found 2024-05-30T15:58:31.194 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.194+0000 7fd033cd0980 4 Realm: () 2024-05-30T15:58:31.194 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.194+0000 7fd033cd0980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T15:58:31.194 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.194+0000 7fd033cd0980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T15:58:31.194 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.194+0000 7fd033cd0980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T15:58:31.195 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.194+0000 7fd033cd0980 20 zonegroup default 2024-05-30T15:58:31.195 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.194+0000 7fd033cd0980 20 rados->read ofs=0 len=0 2024-05-30T15:58:31.195 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.194+0000 7fd033cd0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:31.195 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.194+0000 7fd033cd0980 20 rados->read ofs=0 len=0 2024-05-30T15:58:31.195 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.194+0000 7fd033cd0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:31.195 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.194+0000 7fd033cd0980 20 rados->read ofs=0 len=0 2024-05-30T15:58:31.196 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.194+0000 7fd033cd0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:31.196 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.194+0000 7fd033cd0980 20 started sync module instance, tier type = 2024-05-30T15:58:31.196 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.194+0000 7fd033cd0980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T15:58:31.211 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.210+0000 7fd033cd0980 20 add_watcher() i=0 2024-05-30T15:58:31.211 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.210+0000 7fd033cd0980 20 add_watcher() i=1 2024-05-30T15:58:31.211 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.210+0000 7fd033cd0980 20 add_watcher() i=2 2024-05-30T15:58:31.212 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.210+0000 7fd033cd0980 20 add_watcher() i=3 2024-05-30T15:58:31.212 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.210+0000 7fd033cd0980 20 add_watcher() i=4 2024-05-30T15:58:31.212 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.210+0000 7fd033cd0980 20 add_watcher() i=5 2024-05-30T15:58:31.212 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.210+0000 7fd033cd0980 20 add_watcher() i=6 2024-05-30T15:58:31.212 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.210+0000 7fd033cd0980 20 add_watcher() i=7 2024-05-30T15:58:31.212 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.210+0000 7fd033cd0980 2 all 8 watchers are set, enabling cache 2024-05-30T15:58:31.215 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.214+0000 7fcee7fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:31.216 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.214+0000 7fd033cd0980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:58:31.216 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.214+0000 7fd033cd0980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:58:31.216 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.214+0000 7fd033cd0980 5 note: GC not initialized 2024-05-30T15:58:31.217 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.214+0000 7fcee6fe5640 20 reqs_thread_entry: start 2024-05-30T15:58:31.217 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.214+0000 7fd033cd0980 20 init_complete bucket index max shards: 11 2024-05-30T15:58:31.217 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.214+0000 7fd033cd0980 20 Filter name: none 2024-05-30T15:58:31.217 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.214+0000 7fcee4fe1640 20 reqs_thread_entry: start 2024-05-30T15:58:31.217 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.214+0000 7fd033cd0980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-05-30T15:58:31.217 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.214+0000 7fd033cd0980 20 rados->read ofs=0 len=0 2024-05-30T15:58:31.217 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.214+0000 7fd033cd0980 20 rados_obj.operate() r=0 bl.length=190 2024-05-30T15:58:31.217 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.214+0000 7fd033cd0980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-05-30T15:58:31.218 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.214+0000 7fd033cd0980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-05-30T15:58:31.218 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.214+0000 7fd033cd0980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 : miss 2024-05-30T15:58:31.218 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.214+0000 7fd033cd0980 20 rados->read ofs=0 len=0 2024-05-30T15:58:31.219 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.218+0000 7fd033cd0980 20 rados_obj.operate() r=0 bl.length=404 2024-05-30T15:58:31.219 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.218+0000 7fd033cd0980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 info.flags=0x17 2024-05-30T15:58:31.219 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.218+0000 7fd033cd0980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 to cache LRU end 2024-05-30T15:58:31.219 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.218+0000 7fd033cd0980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T15:58:31.219 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.218+0000 7fd033cd0980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-05-30T15:58:31.219 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.218+0000 7fd033cd0980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 2024-05-30T15:58:31.219 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.218+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.0 2024-05-30T15:58:31.219 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.218+0000 7fd033cd0980 1 NOTICE: finished shard 0 (0 entries removed) 2024-05-30T15:58:31.219 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.218+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:31.219 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.218+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.2 2024-05-30T15:58:31.220 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.218+0000 7fd033cd0980 1 NOTICE: finished shard 2 (0 entries removed) 2024-05-30T15:58:31.220 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.218+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.3 2024-05-30T15:58:31.220 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.218+0000 7fd033cd0980 1 NOTICE: finished shard 3 (0 entries removed) 2024-05-30T15:58:31.220 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.218+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.4 2024-05-30T15:58:31.221 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.222+0000 7fd033cd0980 1 NOTICE: finished shard 4 (0 entries removed) 2024-05-30T15:58:31.221 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.222+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:31.222 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.222+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.6 2024-05-30T15:58:31.223 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.222+0000 7fd033cd0980 1 NOTICE: finished shard 6 (0 entries removed) 2024-05-30T15:58:31.223 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.222+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.7 2024-05-30T15:58:31.224 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.222+0000 7fd033cd0980 1 NOTICE: finished shard 7 (0 entries removed) 2024-05-30T15:58:31.224 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.222+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:31.225 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.222+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.9 2024-05-30T15:58:31.226 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.226+0000 7fd033cd0980 1 NOTICE: finished shard 9 (0 entries removed) 2024-05-30T15:58:31.226 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.226+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:31.227 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.226+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:31.227 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.226+0000 7fd033cd0980 20 get_obj_state: octx=0x564a21708a30 obj=check-bucket:e state=0x564a219b7418 s->prefetch_data=0 2024-05-30T15:58:31.228 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.226+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:31.228 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.226+0000 7fd033cd0980 20 get_obj_state: octx=0x564a2171a4f0 obj=check-bucket:d state=0x564a219b5db8 s->prefetch_data=0 2024-05-30T15:58:31.229 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.226+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:31.229 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.226+0000 7fd033cd0980 20 get_obj_state: octx=0x564a219b9060 obj=check-bucket:c state=0x564a219b0348 s->prefetch_data=0 2024-05-30T15:58:31.229 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.226+0000 7fd033cd0980 20 get_obj_state: setting s->obj_tag to ca8c5uvq3xg946fg800bp3ukh3fhcgc9 2024-05-30T15:58:31.229 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.226+0000 7fd033cd0980 20 get_obj_state_impl: setting s->olh_tag to 9mkh5j6rbza5jz5b9dq88ixaby8svm3d 2024-05-30T15:58:31.229 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.226+0000 7fd033cd0980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=9mkh5j6rbza5jz5b9dq88ixaby8svm3d 2024-05-30T15:58:31.230 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.226+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:31.231 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.230+0000 7fd033cd0980 20 get_obj_state: octx=0x564a21a43930 obj=check-bucket:f state=0x564a21a450b8 s->prefetch_data=0 2024-05-30T15:58:31.231 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.230+0000 7fd033cd0980 20 get_obj_state: setting s->obj_tag to tn0v5nl9htr8labacw03in173zpj8oi7 2024-05-30T15:58:31.231 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.230+0000 7fd033cd0980 20 get_obj_state_impl: setting s->olh_tag to 3lzteeo6dqdh9sks1jix2732dhjccbdz 2024-05-30T15:58:31.231 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.230+0000 7fd033cd0980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=3lzteeo6dqdh9sks1jix2732dhjccbdz 2024-05-30T15:58:31.231 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.230+0000 7fd033cd0980 20 get_obj_state: setting s->obj_tag to yy20qu0ussswpr322gx3xyyvhuhksvvt 2024-05-30T15:58:31.231 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.230+0000 7fd033cd0980 20 get_obj_state_impl: setting s->olh_tag to abhp9uhv1kaqvgjrirnkp690j0aga0uy 2024-05-30T15:58:31.231 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.230+0000 7fd033cd0980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=abhp9uhv1kaqvgjrirnkp690j0aga0uy 2024-05-30T15:58:31.231 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.230+0000 7fd033cd0980 20 get_obj_state: setting s->obj_tag to oy41x171pppm6noc2n7iq50wjiuewo3e 2024-05-30T15:58:31.231 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.230+0000 7fd033cd0980 20 get_obj_state_impl: setting s->olh_tag to xo3yi31pj6yz3tsyoe551w4k4erhudj9 2024-05-30T15:58:31.232 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.230+0000 7fd033cd0980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=xo3yi31pj6yz3tsyoe551w4k4erhudj9 2024-05-30T15:58:31.232 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.230+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:31.233 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.234+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:31.234 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.234+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:31.235 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.234+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:31.235 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.234+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:31.235 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.234+0000 7fd033cd0980 20 olh_log_entry: epoch=2 op=2 key=e[] 2024-05-30T15:58:31.235 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.234+0000 7fd033cd0980 20 olh_log_entry: epoch=2 op=3 key=e[4U6pGDIFJcrwuZmtpSzbGHHAX9RMKyq] 2024-05-30T15:58:31.235 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.234+0000 7fd033cd0980 20 get_obj_state: octx=0x564a21708a30 obj=check-bucket:_:4U6pGDIFJcrwuZmtpSzbGHHAX9RMKyq_e state=0x564a21a5d708 s->prefetch_data=0 2024-05-30T15:58:31.236 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.234+0000 7fd033cd0980 10 manifest: total_size = 22 2024-05-30T15:58:31.236 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.234+0000 7fd033cd0980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11936358017361252058 2024-05-30T15:58:31.236 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.234+0000 7fd033cd0980 20 get_obj_state: octx=0x564a21708a30 obj=check-bucket:_:4U6pGDIFJcrwuZmtpSzbGHHAX9RMKyq_e state=0x564a21a5d708 s->prefetch_data=0 2024-05-30T15:58:31.236 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.234+0000 7fd033cd0980 20 prepare_atomic_modification: state is not atomic. state=0x564a21a5d708 2024-05-30T15:58:31.236 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.234+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:31.237 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.238+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:31.237 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.238+0000 7fd033cd0980 20 olh_log_entry: epoch=3 op=3 key=c[XJr7Vc2lyrW2dx7I03mUK.xDmRGjIBx] 2024-05-30T15:58:31.237 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.238+0000 7fd033cd0980 20 get_obj_state: octx=0x564a219b9060 obj=check-bucket:_:XJr7Vc2lyrW2dx7I03mUK.xDmRGjIBx_c state=0x564a21a65578 s->prefetch_data=0 2024-05-30T15:58:31.238 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.238+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:31.238 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.238+0000 7fd033cd0980 10 manifest: total_size = 11 2024-05-30T15:58:31.239 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.238+0000 7fd033cd0980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.13828964673071623055 2024-05-30T15:58:31.239 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.238+0000 7fd033cd0980 20 get_obj_state: octx=0x564a219b9060 obj=check-bucket:_:XJr7Vc2lyrW2dx7I03mUK.xDmRGjIBx_c state=0x564a21a65578 s->prefetch_data=0 2024-05-30T15:58:31.239 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.238+0000 7fd033cd0980 20 prepare_atomic_modification: state is not atomic. state=0x564a21a65578 2024-05-30T15:58:31.239 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.238+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:31.239 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.238+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:31.239 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.238+0000 7fd033cd0980 20 olh_log_entry: epoch=2 op=2 key=f[] 2024-05-30T15:58:31.240 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.238+0000 7fd033cd0980 20 olh_log_entry: epoch=2 op=3 key=f[XtsP7x3L2XjJv9hozGab7M6Se7FpxZL] 2024-05-30T15:58:31.240 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.238+0000 7fd033cd0980 20 get_obj_state: octx=0x564a21a43930 obj=check-bucket:_:XtsP7x3L2XjJv9hozGab7M6Se7FpxZL_f state=0x564a21a6c138 s->prefetch_data=0 2024-05-30T15:58:31.240 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.238+0000 7fd033cd0980 10 manifest: total_size = 9 2024-05-30T15:58:31.240 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.238+0000 7fd033cd0980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.14173208196838462876 2024-05-30T15:58:31.240 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.238+0000 7fd033cd0980 20 get_obj_state: octx=0x564a21a43930 obj=check-bucket:_:XtsP7x3L2XjJv9hozGab7M6Se7FpxZL_f state=0x564a21a6c138 s->prefetch_data=0 2024-05-30T15:58:31.241 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.238+0000 7fd033cd0980 20 prepare_atomic_modification: state is not atomic. state=0x564a21a6c138 2024-05-30T15:58:31.241 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.238+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:31.241 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.238+0000 7fd033cd0980 20 olh_log_entry: epoch=3 op=3 key=d[5bUxCUG9NsBjYrBYHksJbwFbpf.JhT-] 2024-05-30T15:58:31.241 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.238+0000 7fd033cd0980 20 get_obj_state: octx=0x564a2171a4f0 obj=check-bucket:_:5bUxCUG9NsBjYrBYHksJbwFbpf.JhT-_d state=0x564a21a76058 s->prefetch_data=0 2024-05-30T15:58:31.241 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.242+0000 7fd033cd0980 10 manifest: total_size = 12 2024-05-30T15:58:31.242 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.242+0000 7fd033cd0980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12218985211164265627 2024-05-30T15:58:31.242 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.242+0000 7fd033cd0980 20 get_obj_state: octx=0x564a2171a4f0 obj=check-bucket:_:5bUxCUG9NsBjYrBYHksJbwFbpf.JhT-_d state=0x564a21a76058 s->prefetch_data=0 2024-05-30T15:58:31.242 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.242+0000 7fd033cd0980 20 prepare_atomic_modification: state is not atomic. state=0x564a21a76058 2024-05-30T15:58:31.242 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.242+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:31.243 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.242+0000 7fcf03fff640 20 handle_completion(): completion ok for obj=e[4U6pGDIFJcrwuZmtpSzbGHHAX9RMKyq] 2024-05-30T15:58:31.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.242+0000 7fd033cd0980 20 get_obj_state: octx=0x564a21708a30 obj=check-bucket:e state=0x564a219b7418 s->prefetch_data=0 2024-05-30T15:58:31.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.242+0000 7fd033cd0980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=9mkh5j6rbza5jz5b9dq88ixaby8svm3d 2024-05-30T15:58:31.249 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.250+0000 7fd033cd0980 20 get_obj_state: octx=0x564a21a43930 obj=check-bucket:f state=0x564a21a450b8 s->prefetch_data=0 2024-05-30T15:58:31.250 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.250+0000 7fd033cd0980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=xo3yi31pj6yz3tsyoe551w4k4erhudj9 2024-05-30T15:58:31.250 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.250+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:31.251 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.250+0000 7fcf03fff640 20 handle_completion(): completion ok for obj=f[XtsP7x3L2XjJv9hozGab7M6Se7FpxZL] 2024-05-30T15:58:31.251 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.250+0000 7fcf03fff640 20 handle_completion(): completion ok for obj=c[XJr7Vc2lyrW2dx7I03mUK.xDmRGjIBx] 2024-05-30T15:58:31.252 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.250+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:31.252 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.250+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:31.253 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.254+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:31.254 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.254+0000 7fd033cd0980 1 NOTICE: finished shard 1 (1 entries removed) 2024-05-30T15:58:31.255 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.254+0000 7fcf14ff9640 20 handle_completion(): completion ok for obj=d[5bUxCUG9NsBjYrBYHksJbwFbpf.JhT-] 2024-05-30T15:58:31.256 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.254+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:31.257 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.254+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:31.257 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.254+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:31.257 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.254+0000 7fd033cd0980 1 NOTICE: finished shard 8 (1 entries removed) 2024-05-30T15:58:31.259 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.258+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:31.260 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.258+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:31.260 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.258+0000 7fd033cd0980 1 NOTICE: finished shard 5 (1 entries removed) 2024-05-30T15:58:31.262 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.262+0000 7fd033cd0980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:31.262 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.262+0000 7fd033cd0980 1 NOTICE: finished shard 10 (1 entries removed) 2024-05-30T15:58:31.262 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.262+0000 7fd033cd0980 1 NOTICE: finished all shards (4 entries removed) 2024-05-30T15:58:31.263 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.262+0000 7fcee7fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:31.269 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.266+0000 7fd033cd0980 20 remove_watcher() i=0 2024-05-30T15:58:31.269 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.266+0000 7fd033cd0980 2 removed watcher, disabling cache 2024-05-30T15:58:31.271 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.270+0000 7fd033cd0980 20 remove_watcher() i=1 2024-05-30T15:58:31.273 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.274+0000 7fd033cd0980 20 remove_watcher() i=2 2024-05-30T15:58:31.275 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.274+0000 7fd033cd0980 20 remove_watcher() i=3 2024-05-30T15:58:31.278 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.278+0000 7fd033cd0980 20 remove_watcher() i=4 2024-05-30T15:58:31.279 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.278+0000 7fd033cd0980 20 remove_watcher() i=5 2024-05-30T15:58:31.281 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.282+0000 7fd033cd0980 20 remove_watcher() i=6 2024-05-30T15:58:31.283 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.282+0000 7fd033cd0980 20 remove_watcher() i=7 2024-05-30T15:58:31.421 INFO:tasks.workunit.client.0.smithi006.stderr:TEST: bucket check unlinked does not affect normal entries 2024-05-30T15:58:31.421 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T15:58:31.829 INFO:tasks.workunit.client.0.smithi006.stderr:TEST: bucket check unlinked does not find new unlistable entries after fix 2024-05-30T15:58:31.829 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T15:58:31.830 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --min-age-hours 0 --dump-keys 2024-05-30T15:58:31.870 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:58:31.870 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:58:31.918 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.918+0000 7f214591f980 20 rados->read ofs=0 len=0 2024-05-30T15:58:31.920 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.918+0000 7f214591f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:31.920 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.918+0000 7f214591f980 20 realm 2024-05-30T15:58:31.920 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.918+0000 7f214591f980 20 rados->read ofs=0 len=0 2024-05-30T15:58:31.920 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.918+0000 7f214591f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:31.920 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.918+0000 7f214591f980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T15:58:31.920 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.918+0000 7f214591f980 20 rados->read ofs=0 len=0 2024-05-30T15:58:31.921 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.918+0000 7f214591f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:31.921 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.918+0000 7f214591f980 20 rados->read ofs=0 len=0 2024-05-30T15:58:31.922 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.922+0000 7f214591f980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:31.922 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.922+0000 7f214591f980 20 rados->read ofs=0 len=0 2024-05-30T15:58:31.923 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.922+0000 7f214591f980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T15:58:31.923 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.922+0000 7f214591f980 20 searching for the correct realm 2024-05-30T15:58:31.929 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.926+0000 7f2143c31640 0 -- 172.21.15.6:0/105865314 >> [v2:172.21.15.6:6808/1897362715,v1:172.21.15.6:6809/1897362715] conn(0x55f22ccd6880 msgr2=0x55f22ccd8d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:31.936 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.934+0000 7f2142c2f640 0 -- 172.21.15.6:0/105865314 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f21340dc060 msgr2=0x7f21340aca20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:31.944 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.942+0000 7f214591f980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T15:58:31.944 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.942+0000 7f214591f980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T15:58:31.944 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.942+0000 7f214591f980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T15:58:31.944 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.942+0000 7f214591f980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T15:58:31.944 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.942+0000 7f214591f980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T15:58:31.944 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.942+0000 7f214591f980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T15:58:31.944 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.942+0000 7f214591f980 20 rados->read ofs=0 len=0 2024-05-30T15:58:31.944 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.942+0000 7f214591f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:31.944 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.942+0000 7f214591f980 20 rados->read ofs=0 len=0 2024-05-30T15:58:31.945 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.942+0000 7f214591f980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:31.945 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.942+0000 7f214591f980 20 rados->read ofs=0 len=0 2024-05-30T15:58:31.946 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.946+0000 7f214591f980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T15:58:31.946 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.946+0000 7f214591f980 20 zone default found 2024-05-30T15:58:31.946 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.946+0000 7f214591f980 4 Realm: () 2024-05-30T15:58:31.946 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.946+0000 7f214591f980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T15:58:31.947 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.946+0000 7f214591f980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T15:58:31.947 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.946+0000 7f214591f980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T15:58:31.947 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.946+0000 7f214591f980 20 zonegroup default 2024-05-30T15:58:31.947 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.946+0000 7f214591f980 20 rados->read ofs=0 len=0 2024-05-30T15:58:31.947 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.946+0000 7f214591f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:31.947 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.946+0000 7f214591f980 20 rados->read ofs=0 len=0 2024-05-30T15:58:31.948 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.946+0000 7f214591f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:31.948 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.946+0000 7f214591f980 20 rados->read ofs=0 len=0 2024-05-30T15:58:31.948 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.946+0000 7f214591f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:31.948 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.946+0000 7f214591f980 20 started sync module instance, tier type = 2024-05-30T15:58:31.948 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.946+0000 7f214591f980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T15:58:31.969 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.966+0000 7f214591f980 20 add_watcher() i=0 2024-05-30T15:58:31.969 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.966+0000 7f214591f980 20 add_watcher() i=1 2024-05-30T15:58:31.969 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.966+0000 7f214591f980 20 add_watcher() i=2 2024-05-30T15:58:31.969 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.966+0000 7f214591f980 20 add_watcher() i=3 2024-05-30T15:58:31.969 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.966+0000 7f214591f980 20 add_watcher() i=4 2024-05-30T15:58:31.969 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.966+0000 7f214591f980 20 add_watcher() i=5 2024-05-30T15:58:31.969 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.966+0000 7f214591f980 20 add_watcher() i=6 2024-05-30T15:58:31.970 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.970+0000 7f214591f980 20 add_watcher() i=7 2024-05-30T15:58:31.970 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.970+0000 7f214591f980 2 all 8 watchers are set, enabling cache 2024-05-30T15:58:31.973 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.970+0000 7f1ff9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:31.974 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.974+0000 7f214591f980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:58:31.974 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.974+0000 7f214591f980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:58:31.974 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.974+0000 7f214591f980 5 note: GC not initialized 2024-05-30T15:58:31.974 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.974+0000 7f1ff8fe1640 20 reqs_thread_entry: start 2024-05-30T15:58:31.974 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.974+0000 7f214591f980 20 init_complete bucket index max shards: 11 2024-05-30T15:58:31.974 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.974+0000 7f214591f980 20 Filter name: none 2024-05-30T15:58:31.974 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.974+0000 7f1f727fc640 20 reqs_thread_entry: start 2024-05-30T15:58:31.974 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.974+0000 7f214591f980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-05-30T15:58:31.974 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.974+0000 7f214591f980 20 rados->read ofs=0 len=0 2024-05-30T15:58:31.975 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.974+0000 7f214591f980 20 rados_obj.operate() r=0 bl.length=190 2024-05-30T15:58:31.975 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.974+0000 7f214591f980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-05-30T15:58:31.975 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.974+0000 7f214591f980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-05-30T15:58:31.976 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.974+0000 7f214591f980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 : miss 2024-05-30T15:58:31.976 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.974+0000 7f214591f980 20 rados->read ofs=0 len=0 2024-05-30T15:58:31.976 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.974+0000 7f214591f980 20 rados_obj.operate() r=0 bl.length=404 2024-05-30T15:58:31.976 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.974+0000 7f214591f980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 info.flags=0x17 2024-05-30T15:58:31.976 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.974+0000 7f214591f980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 to cache LRU end 2024-05-30T15:58:31.976 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.974+0000 7f214591f980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T15:58:31.976 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.974+0000 7f214591f980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-05-30T15:58:31.976 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.974+0000 7f214591f980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 2024-05-30T15:58:31.976 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.974+0000 7f214591f980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.0 2024-05-30T15:58:31.977 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.974+0000 7f214591f980 1 NOTICE: finished shard 0 (0 entries found) 2024-05-30T15:58:31.977 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.974+0000 7f214591f980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:31.977 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.974+0000 7f214591f980 1 NOTICE: finished shard 1 (0 entries found) 2024-05-30T15:58:31.977 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.974+0000 7f214591f980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.2 2024-05-30T15:58:31.977 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.978+0000 7f214591f980 1 NOTICE: finished shard 2 (0 entries found) 2024-05-30T15:58:31.977 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.978+0000 7f214591f980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.3 2024-05-30T15:58:31.978 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.978+0000 7f214591f980 1 NOTICE: finished shard 3 (0 entries found) 2024-05-30T15:58:31.978 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.978+0000 7f214591f980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.4 2024-05-30T15:58:31.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.978+0000 7f214591f980 1 NOTICE: finished shard 4 (0 entries found) 2024-05-30T15:58:31.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.978+0000 7f214591f980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:31.980 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.978+0000 7f214591f980 1 NOTICE: finished shard 5 (0 entries found) 2024-05-30T15:58:31.980 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.978+0000 7f214591f980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.6 2024-05-30T15:58:31.981 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.978+0000 7f214591f980 1 NOTICE: finished shard 6 (0 entries found) 2024-05-30T15:58:31.981 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.978+0000 7f214591f980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.7 2024-05-30T15:58:31.982 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.982+0000 7f214591f980 1 NOTICE: finished shard 7 (0 entries found) 2024-05-30T15:58:31.982 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.982+0000 7f214591f980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:31.982 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.982+0000 7f214591f980 1 NOTICE: finished shard 8 (0 entries found) 2024-05-30T15:58:31.983 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.982+0000 7f214591f980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.9 2024-05-30T15:58:31.983 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.982+0000 7f214591f980 1 NOTICE: finished shard 9 (0 entries found) 2024-05-30T15:58:31.983 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.982+0000 7f214591f980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:31.984 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.982+0000 7f214591f980 1 NOTICE: finished shard 10 (0 entries found) 2024-05-30T15:58:31.984 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.982+0000 7f214591f980 1 NOTICE: finished all shards (0 entries found) 2024-05-30T15:58:31.985 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.986+0000 7f1ff9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:31.990 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.990+0000 7f214591f980 20 remove_watcher() i=0 2024-05-30T15:58:31.990 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.990+0000 7f214591f980 2 removed watcher, disabling cache 2024-05-30T15:58:31.993 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.990+0000 7f214591f980 20 remove_watcher() i=1 2024-05-30T15:58:31.995 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.994+0000 7f214591f980 20 remove_watcher() i=2 2024-05-30T15:58:31.998 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.998+0000 7f214591f980 20 remove_watcher() i=3 2024-05-30T15:58:32.000 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:31.998+0000 7f214591f980 20 remove_watcher() i=4 2024-05-30T15:58:32.001 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.002+0000 7f214591f980 20 remove_watcher() i=5 2024-05-30T15:58:32.003 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.002+0000 7f214591f980 20 remove_watcher() i=6 2024-05-30T15:58:32.004 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.002+0000 7f214591f980 20 remove_watcher() i=7 2024-05-30T15:58:32.415 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bi list --bucket check-bucket --object d 2024-05-30T15:58:32.451 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:58:32.451 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:58:32.498 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.498+0000 7f4afd85b980 20 rados->read ofs=0 len=0 2024-05-30T15:58:32.499 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.498+0000 7f4afd85b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:32.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.498+0000 7f4afd85b980 20 realm 2024-05-30T15:58:32.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.498+0000 7f4afd85b980 20 rados->read ofs=0 len=0 2024-05-30T15:58:32.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.498+0000 7f4afd85b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:32.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.498+0000 7f4afd85b980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T15:58:32.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.498+0000 7f4afd85b980 20 rados->read ofs=0 len=0 2024-05-30T15:58:32.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.498+0000 7f4afd85b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:32.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.498+0000 7f4afd85b980 20 rados->read ofs=0 len=0 2024-05-30T15:58:32.501 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.502+0000 7f4afd85b980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:32.501 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.502+0000 7f4afd85b980 20 rados->read ofs=0 len=0 2024-05-30T15:58:32.502 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.502+0000 7f4afd85b980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T15:58:32.502 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.502+0000 7f4afd85b980 20 searching for the correct realm 2024-05-30T15:58:32.508 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.506+0000 7f4afbb6d640 0 -- 172.21.15.6:0/2110143476 >> [v2:172.21.15.6:6808/1897362715,v1:172.21.15.6:6809/1897362715] conn(0x55e625781b30 msgr2=0x55e625783ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T15:58:32.515 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.514+0000 7f4afb36c640 0 -- 172.21.15.6:0/2110143476 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f4aec008040 msgr2=0x7f4aec016930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:32.523 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.522+0000 7f4afd85b980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T15:58:32.523 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.522+0000 7f4afd85b980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T15:58:32.523 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.522+0000 7f4afd85b980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T15:58:32.523 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.522+0000 7f4afd85b980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T15:58:32.523 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.522+0000 7f4afd85b980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T15:58:32.524 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.522+0000 7f4afd85b980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T15:58:32.524 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.522+0000 7f4afd85b980 20 rados->read ofs=0 len=0 2024-05-30T15:58:32.524 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.522+0000 7f4afd85b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:32.524 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.522+0000 7f4afd85b980 20 rados->read ofs=0 len=0 2024-05-30T15:58:32.524 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.522+0000 7f4afd85b980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:32.524 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.522+0000 7f4afd85b980 20 rados->read ofs=0 len=0 2024-05-30T15:58:32.525 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.522+0000 7f4afd85b980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T15:58:32.525 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.522+0000 7f4afd85b980 20 zone default found 2024-05-30T15:58:32.525 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.522+0000 7f4afd85b980 4 Realm: () 2024-05-30T15:58:32.525 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.522+0000 7f4afd85b980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T15:58:32.525 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.522+0000 7f4afd85b980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T15:58:32.526 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.522+0000 7f4afd85b980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T15:58:32.526 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.522+0000 7f4afd85b980 20 zonegroup default 2024-05-30T15:58:32.526 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.522+0000 7f4afd85b980 20 rados->read ofs=0 len=0 2024-05-30T15:58:32.526 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.522+0000 7f4afd85b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:32.526 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.522+0000 7f4afd85b980 20 rados->read ofs=0 len=0 2024-05-30T15:58:32.527 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.526+0000 7f4afd85b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:32.527 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.526+0000 7f4afd85b980 20 rados->read ofs=0 len=0 2024-05-30T15:58:32.527 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.526+0000 7f4afd85b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:32.527 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.526+0000 7f4afd85b980 20 started sync module instance, tier type = 2024-05-30T15:58:32.527 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.526+0000 7f4afd85b980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T15:58:32.543 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.542+0000 7f4afd85b980 20 add_watcher() i=0 2024-05-30T15:58:32.543 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.542+0000 7f4afd85b980 20 add_watcher() i=1 2024-05-30T15:58:32.543 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.542+0000 7f4afd85b980 20 add_watcher() i=2 2024-05-30T15:58:32.543 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.542+0000 7f4afd85b980 20 add_watcher() i=3 2024-05-30T15:58:32.543 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.542+0000 7f4afd85b980 20 add_watcher() i=4 2024-05-30T15:58:32.543 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.542+0000 7f4afd85b980 20 add_watcher() i=5 2024-05-30T15:58:32.544 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.542+0000 7f4afd85b980 20 add_watcher() i=6 2024-05-30T15:58:32.544 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.542+0000 7f4afd85b980 20 add_watcher() i=7 2024-05-30T15:58:32.544 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.542+0000 7f4afd85b980 2 all 8 watchers are set, enabling cache 2024-05-30T15:58:32.547 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.546+0000 7f49b1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:32.548 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.546+0000 7f4afd85b980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:58:32.548 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.546+0000 7f4afd85b980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:58:32.548 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.546+0000 7f4afd85b980 5 note: GC not initialized 2024-05-30T15:58:32.548 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.546+0000 7f49b0fe1640 20 reqs_thread_entry: start 2024-05-30T15:58:32.548 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.546+0000 7f4afd85b980 20 init_complete bucket index max shards: 11 2024-05-30T15:58:32.548 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.546+0000 7f4afd85b980 20 Filter name: none 2024-05-30T15:58:32.548 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.546+0000 7f492a7fc640 20 reqs_thread_entry: start 2024-05-30T15:58:32.548 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.546+0000 7f4afd85b980 20 rados->read ofs=0 len=0 2024-05-30T15:58:32.549 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.546+0000 7f4afd85b980 20 rados_obj.operate() r=0 bl.length=190 2024-05-30T15:58:32.549 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.546+0000 7f4afd85b980 20 rados->read ofs=0 len=0 2024-05-30T15:58:32.549 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.546+0000 7f4afd85b980 20 rados_obj.operate() r=0 bl.length=404 2024-05-30T15:58:32.550 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.546+0000 7f4afd85b980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T15:58:32.550 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.546+0000 7f4afd85b980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-05-30T15:58:32.550 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.546+0000 7f4afd85b980 20 INFO: main: starting shard=0 2024-05-30T15:58:32.550 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.546+0000 7f4afd85b980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.0 2024-05-30T15:58:32.550 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.550+0000 7f4afd85b980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:32.550 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.550+0000 7f4afd85b980 20 INFO: main: starting shard=1 2024-05-30T15:58:32.550 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.550+0000 7f4afd85b980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:32.550 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.550+0000 7f4afd85b980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:32.551 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.550+0000 7f4afd85b980 20 INFO: main: starting shard=2 2024-05-30T15:58:32.551 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.550+0000 7f4afd85b980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.2 2024-05-30T15:58:32.551 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.550+0000 7f4afd85b980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:32.551 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.550+0000 7f4afd85b980 20 INFO: main: starting shard=3 2024-05-30T15:58:32.551 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.550+0000 7f4afd85b980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.3 2024-05-30T15:58:32.552 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.550+0000 7f4afd85b980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:32.552 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.550+0000 7f4afd85b980 20 INFO: main: starting shard=4 2024-05-30T15:58:32.552 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.550+0000 7f4afd85b980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.4 2024-05-30T15:58:32.553 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.554+0000 7f4afd85b980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:32.553 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.554+0000 7f4afd85b980 20 INFO: main: starting shard=5 2024-05-30T15:58:32.553 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.554+0000 7f4afd85b980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:32.555 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.554+0000 7f4afd85b980 20 INFO: main: bi_list() returned without error; entries.size()=2, is_truncated=0, marker= 2024-05-30T15:58:32.555 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.554+0000 7f4afd85b980 20 INFO: main: starting shard=6 2024-05-30T15:58:32.555 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.554+0000 7f4afd85b980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.6 2024-05-30T15:58:32.555 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.554+0000 7f4afd85b980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:32.555 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.554+0000 7f4afd85b980 20 INFO: main: starting shard=7 2024-05-30T15:58:32.555 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.554+0000 7f4afd85b980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.7 2024-05-30T15:58:32.557 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.554+0000 7f4afd85b980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:32.557 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.554+0000 7f4afd85b980 20 INFO: main: starting shard=8 2024-05-30T15:58:32.557 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.554+0000 7f4afd85b980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:32.557 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.558+0000 7f4afd85b980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:32.558 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.558+0000 7f4afd85b980 20 INFO: main: starting shard=9 2024-05-30T15:58:32.558 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.558+0000 7f4afd85b980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.9 2024-05-30T15:58:32.558 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.558+0000 7f4afd85b980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:32.558 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.558+0000 7f4afd85b980 20 INFO: main: starting shard=10 2024-05-30T15:58:32.558 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.558+0000 7f4afd85b980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:32.560 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.558+0000 7f4afd85b980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:32.560 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.558+0000 7f4afd85b980 20 INFO: main: done 2024-05-30T15:58:32.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.562+0000 7f49b1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:32.566 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.566+0000 7f4afd85b980 20 remove_watcher() i=0 2024-05-30T15:58:32.566 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.566+0000 7f4afd85b980 2 removed watcher, disabling cache 2024-05-30T15:58:32.569 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.566+0000 7f4afd85b980 20 remove_watcher() i=1 2024-05-30T15:58:32.571 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.570+0000 7f4afd85b980 20 remove_watcher() i=2 2024-05-30T15:58:32.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.574+0000 7f4afd85b980 20 remove_watcher() i=3 2024-05-30T15:58:32.576 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.574+0000 7f4afd85b980 20 remove_watcher() i=4 2024-05-30T15:58:32.578 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.578+0000 7f4afd85b980 20 remove_watcher() i=5 2024-05-30T15:58:32.579 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.578+0000 7f4afd85b980 20 remove_watcher() i=6 2024-05-30T15:58:32.580 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.578+0000 7f4afd85b980 20 remove_watcher() i=7 2024-05-30T15:58:32.596 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bi list --bucket check-bucket --object c 2024-05-30T15:58:32.626 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:58:32.626 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:58:32.674 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.674+0000 7f1ffd8e4980 20 rados->read ofs=0 len=0 2024-05-30T15:58:32.676 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.674+0000 7f1ffd8e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:32.676 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.674+0000 7f1ffd8e4980 20 realm 2024-05-30T15:58:32.676 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.674+0000 7f1ffd8e4980 20 rados->read ofs=0 len=0 2024-05-30T15:58:32.677 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.674+0000 7f1ffd8e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:32.677 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.674+0000 7f1ffd8e4980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T15:58:32.677 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.674+0000 7f1ffd8e4980 20 rados->read ofs=0 len=0 2024-05-30T15:58:32.677 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.674+0000 7f1ffd8e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:32.677 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.674+0000 7f1ffd8e4980 20 rados->read ofs=0 len=0 2024-05-30T15:58:32.679 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.678+0000 7f1ffd8e4980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:32.679 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.678+0000 7f1ffd8e4980 20 rados->read ofs=0 len=0 2024-05-30T15:58:32.680 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.678+0000 7f1ffd8e4980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T15:58:32.680 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.678+0000 7f1ffd8e4980 20 searching for the correct realm 2024-05-30T15:58:32.687 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.686+0000 7f1ffbbf6640 0 -- 172.21.15.6:0/3173216098 >> [v2:172.21.15.204:6800/3687190393,v1:172.21.15.204:6801/3687190393] conn(0x7f1ff0058040 msgr2=0x7f1ff0058930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:32.903 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.902+0000 7f1ffb3f5640 0 -- 172.21.15.6:0/3173216098 >> [v2:172.21.15.204:6816/2677861390,v1:172.21.15.204:6817/2677861390] conn(0x560d1b66ef20 msgr2=0x560d1b671310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:32.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.910+0000 7f1ffd8e4980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T15:58:32.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.910+0000 7f1ffd8e4980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T15:58:32.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.910+0000 7f1ffd8e4980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T15:58:32.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.910+0000 7f1ffd8e4980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T15:58:32.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.910+0000 7f1ffd8e4980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T15:58:32.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.910+0000 7f1ffd8e4980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T15:58:32.912 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.910+0000 7f1ffd8e4980 20 rados->read ofs=0 len=0 2024-05-30T15:58:32.912 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.910+0000 7f1ffd8e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:32.912 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.910+0000 7f1ffd8e4980 20 rados->read ofs=0 len=0 2024-05-30T15:58:32.913 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.910+0000 7f1ffd8e4980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:32.913 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.910+0000 7f1ffd8e4980 20 rados->read ofs=0 len=0 2024-05-30T15:58:32.914 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.914+0000 7f1ffd8e4980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T15:58:32.914 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.914+0000 7f1ffd8e4980 20 zone default found 2024-05-30T15:58:32.914 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.914+0000 7f1ffd8e4980 4 Realm: () 2024-05-30T15:58:32.914 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.914+0000 7f1ffd8e4980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T15:58:32.915 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.914+0000 7f1ffd8e4980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T15:58:32.915 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.914+0000 7f1ffd8e4980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T15:58:32.915 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.914+0000 7f1ffd8e4980 20 zonegroup default 2024-05-30T15:58:32.915 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.914+0000 7f1ffd8e4980 20 rados->read ofs=0 len=0 2024-05-30T15:58:32.915 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.914+0000 7f1ffd8e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:32.915 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.914+0000 7f1ffd8e4980 20 rados->read ofs=0 len=0 2024-05-30T15:58:32.916 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.914+0000 7f1ffd8e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:32.916 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.914+0000 7f1ffd8e4980 20 rados->read ofs=0 len=0 2024-05-30T15:58:32.916 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.914+0000 7f1ffd8e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:32.917 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.914+0000 7f1ffd8e4980 20 started sync module instance, tier type = 2024-05-30T15:58:32.917 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.914+0000 7f1ffd8e4980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T15:58:32.941 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.942+0000 7f1ffd8e4980 20 add_watcher() i=0 2024-05-30T15:58:32.941 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.942+0000 7f1ffd8e4980 20 add_watcher() i=1 2024-05-30T15:58:32.942 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.942+0000 7f1ffd8e4980 20 add_watcher() i=2 2024-05-30T15:58:32.942 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.942+0000 7f1ffd8e4980 20 add_watcher() i=3 2024-05-30T15:58:32.942 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.942+0000 7f1ffd8e4980 20 add_watcher() i=4 2024-05-30T15:58:32.942 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.942+0000 7f1ffd8e4980 20 add_watcher() i=5 2024-05-30T15:58:32.942 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.942+0000 7f1ffd8e4980 20 add_watcher() i=6 2024-05-30T15:58:32.944 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.942+0000 7f1ffd8e4980 20 add_watcher() i=7 2024-05-30T15:58:32.944 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.942+0000 7f1ffd8e4980 2 all 8 watchers are set, enabling cache 2024-05-30T15:58:32.948 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.946+0000 7f1eb1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:32.949 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.950+0000 7f1ffd8e4980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:58:32.949 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.950+0000 7f1ffd8e4980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:58:32.949 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.950+0000 7f1ffd8e4980 5 note: GC not initialized 2024-05-30T15:58:32.950 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.950+0000 7f1eb0fe1640 20 reqs_thread_entry: start 2024-05-30T15:58:32.951 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.950+0000 7f1ffd8e4980 20 init_complete bucket index max shards: 11 2024-05-30T15:58:32.951 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.950+0000 7f1ffd8e4980 20 Filter name: none 2024-05-30T15:58:32.951 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.950+0000 7f1e2a7fc640 20 reqs_thread_entry: start 2024-05-30T15:58:32.951 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.950+0000 7f1ffd8e4980 20 rados->read ofs=0 len=0 2024-05-30T15:58:32.952 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.950+0000 7f1ffd8e4980 20 rados_obj.operate() r=0 bl.length=190 2024-05-30T15:58:32.952 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.950+0000 7f1ffd8e4980 20 rados->read ofs=0 len=0 2024-05-30T15:58:32.952 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.950+0000 7f1ffd8e4980 20 rados_obj.operate() r=0 bl.length=404 2024-05-30T15:58:32.952 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.950+0000 7f1ffd8e4980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T15:58:32.952 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.950+0000 7f1ffd8e4980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-05-30T15:58:32.952 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.950+0000 7f1ffd8e4980 20 INFO: main: starting shard=0 2024-05-30T15:58:32.952 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.950+0000 7f1ffd8e4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.0 2024-05-30T15:58:32.953 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.950+0000 7f1ffd8e4980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:32.953 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.950+0000 7f1ffd8e4980 20 INFO: main: starting shard=1 2024-05-30T15:58:32.953 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.950+0000 7f1ffd8e4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:32.953 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.954+0000 7f1ffd8e4980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:32.954 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.954+0000 7f1ffd8e4980 20 INFO: main: starting shard=2 2024-05-30T15:58:32.954 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.954+0000 7f1ffd8e4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.2 2024-05-30T15:58:32.954 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.954+0000 7f1ffd8e4980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:32.955 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.954+0000 7f1ffd8e4980 20 INFO: main: starting shard=3 2024-05-30T15:58:32.955 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.954+0000 7f1ffd8e4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.3 2024-05-30T15:58:32.955 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.954+0000 7f1ffd8e4980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:32.955 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.954+0000 7f1ffd8e4980 20 INFO: main: starting shard=4 2024-05-30T15:58:32.956 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.954+0000 7f1ffd8e4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.4 2024-05-30T15:58:32.957 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.954+0000 7f1ffd8e4980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:32.957 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.954+0000 7f1ffd8e4980 20 INFO: main: starting shard=5 2024-05-30T15:58:32.957 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.954+0000 7f1ffd8e4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:32.959 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.958+0000 7f1ffd8e4980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:32.959 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.958+0000 7f1ffd8e4980 20 INFO: main: starting shard=6 2024-05-30T15:58:32.959 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.958+0000 7f1ffd8e4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.6 2024-05-30T15:58:32.959 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.958+0000 7f1ffd8e4980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:32.959 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.958+0000 7f1ffd8e4980 20 INFO: main: starting shard=7 2024-05-30T15:58:32.959 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.958+0000 7f1ffd8e4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.7 2024-05-30T15:58:32.961 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.958+0000 7f1ffd8e4980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:32.961 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.958+0000 7f1ffd8e4980 20 INFO: main: starting shard=8 2024-05-30T15:58:32.961 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.958+0000 7f1ffd8e4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:32.962 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.962+0000 7f1ffd8e4980 20 INFO: main: bi_list() returned without error; entries.size()=2, is_truncated=0, marker= 2024-05-30T15:58:32.962 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.962+0000 7f1ffd8e4980 20 INFO: main: starting shard=9 2024-05-30T15:58:32.962 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.962+0000 7f1ffd8e4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.9 2024-05-30T15:58:32.963 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.962+0000 7f1ffd8e4980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:32.963 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.962+0000 7f1ffd8e4980 20 INFO: main: starting shard=10 2024-05-30T15:58:32.963 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.962+0000 7f1ffd8e4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:32.965 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.962+0000 7f1ffd8e4980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:32.965 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.962+0000 7f1ffd8e4980 20 INFO: main: done 2024-05-30T15:58:32.967 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.966+0000 7f1eb1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:32.972 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.970+0000 7f1ffd8e4980 20 remove_watcher() i=0 2024-05-30T15:58:32.972 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.970+0000 7f1ffd8e4980 2 removed watcher, disabling cache 2024-05-30T15:58:32.975 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.974+0000 7f1ffd8e4980 20 remove_watcher() i=1 2024-05-30T15:58:32.977 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.978+0000 7f1ffd8e4980 20 remove_watcher() i=2 2024-05-30T15:58:32.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.978+0000 7f1ffd8e4980 20 remove_watcher() i=3 2024-05-30T15:58:32.982 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.982+0000 7f1ffd8e4980 20 remove_watcher() i=4 2024-05-30T15:58:32.984 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.982+0000 7f1ffd8e4980 20 remove_watcher() i=5 2024-05-30T15:58:32.986 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.986+0000 7f1ffd8e4980 20 remove_watcher() i=6 2024-05-30T15:58:32.988 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:32.986+0000 7f1ffd8e4980 20 remove_watcher() i=7 2024-05-30T15:58:33.006 INFO:tasks.workunit.client.0.smithi006.stderr:TEST: bucket check olh finds leftover OLH entries 2024-05-30T15:58:33.006 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T15:58:33.007 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket check olh --bucket check-bucket --dump-keys 2024-05-30T15:58:33.037 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:58:33.037 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:58:33.085 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.082+0000 7ff653c0e980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.086 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.086+0000 7ff653c0e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:33.086 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.086+0000 7ff653c0e980 20 realm 2024-05-30T15:58:33.086 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.086+0000 7ff653c0e980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.087 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.086+0000 7ff653c0e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:33.087 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.086+0000 7ff653c0e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T15:58:33.087 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.086+0000 7ff653c0e980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.088 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.086+0000 7ff653c0e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:33.088 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.086+0000 7ff653c0e980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.089 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.086+0000 7ff653c0e980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:33.089 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.086+0000 7ff653c0e980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.089 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.090+0000 7ff653c0e980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T15:58:33.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.090+0000 7ff653c0e980 20 searching for the correct realm 2024-05-30T15:58:33.095 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.094+0000 7ff651f20640 0 -- 172.21.15.6:0/288386881 >> [v2:172.21.15.6:6808/1897362715,v1:172.21.15.6:6809/1897362715] conn(0x55b6191b6020 msgr2=0x55b6191b84e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:33.101 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.102+0000 7ff650f1e640 0 -- 172.21.15.6:0/288386881 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7ff644026020 msgr2=0x7ff644086200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T15:58:33.110 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.110+0000 7ff653c0e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T15:58:33.110 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.110+0000 7ff653c0e980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T15:58:33.110 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.110+0000 7ff653c0e980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T15:58:33.110 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.110+0000 7ff653c0e980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T15:58:33.110 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.110+0000 7ff653c0e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T15:58:33.110 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.110+0000 7ff653c0e980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T15:58:33.110 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.110+0000 7ff653c0e980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.111 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.110+0000 7ff653c0e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:33.111 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.110+0000 7ff653c0e980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.111 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.110+0000 7ff653c0e980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:33.111 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.110+0000 7ff653c0e980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.112 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.110+0000 7ff653c0e980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T15:58:33.112 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.110+0000 7ff653c0e980 20 zone default found 2024-05-30T15:58:33.112 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.110+0000 7ff653c0e980 4 Realm: () 2024-05-30T15:58:33.112 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.110+0000 7ff653c0e980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T15:58:33.112 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.110+0000 7ff653c0e980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T15:58:33.112 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.110+0000 7ff653c0e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T15:58:33.112 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.110+0000 7ff653c0e980 20 zonegroup default 2024-05-30T15:58:33.112 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.110+0000 7ff653c0e980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.112 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.110+0000 7ff653c0e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:33.112 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.110+0000 7ff653c0e980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.113 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.110+0000 7ff653c0e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:33.113 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.110+0000 7ff653c0e980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.113 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.110+0000 7ff653c0e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:33.113 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.110+0000 7ff653c0e980 20 started sync module instance, tier type = 2024-05-30T15:58:33.113 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.110+0000 7ff653c0e980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T15:58:33.131 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.130+0000 7ff653c0e980 20 add_watcher() i=0 2024-05-30T15:58:33.131 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.130+0000 7ff653c0e980 20 add_watcher() i=1 2024-05-30T15:58:33.131 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.130+0000 7ff653c0e980 20 add_watcher() i=2 2024-05-30T15:58:33.131 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.130+0000 7ff653c0e980 20 add_watcher() i=3 2024-05-30T15:58:33.131 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.130+0000 7ff653c0e980 20 add_watcher() i=4 2024-05-30T15:58:33.131 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.130+0000 7ff653c0e980 20 add_watcher() i=5 2024-05-30T15:58:33.131 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.130+0000 7ff653c0e980 20 add_watcher() i=6 2024-05-30T15:58:33.132 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.130+0000 7ff653c0e980 20 add_watcher() i=7 2024-05-30T15:58:33.132 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.130+0000 7ff653c0e980 2 all 8 watchers are set, enabling cache 2024-05-30T15:58:33.135 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.134+0000 7ff5077e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:33.135 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.134+0000 7ff653c0e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:58:33.135 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.134+0000 7ff653c0e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:58:33.135 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.134+0000 7ff653c0e980 5 note: GC not initialized 2024-05-30T15:58:33.136 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.134+0000 7ff5067e4640 20 reqs_thread_entry: start 2024-05-30T15:58:33.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.134+0000 7ff653c0e980 20 init_complete bucket index max shards: 11 2024-05-30T15:58:33.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.134+0000 7ff653c0e980 20 Filter name: none 2024-05-30T15:58:33.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.134+0000 7ff477fff640 20 reqs_thread_entry: start 2024-05-30T15:58:33.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.134+0000 7ff653c0e980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-05-30T15:58:33.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.134+0000 7ff653c0e980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.134+0000 7ff653c0e980 20 rados_obj.operate() r=0 bl.length=190 2024-05-30T15:58:33.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.134+0000 7ff653c0e980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-05-30T15:58:33.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.134+0000 7ff653c0e980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-05-30T15:58:33.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.134+0000 7ff653c0e980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 : miss 2024-05-30T15:58:33.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.134+0000 7ff653c0e980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.138 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.134+0000 7ff653c0e980 20 rados_obj.operate() r=0 bl.length=404 2024-05-30T15:58:33.138 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.134+0000 7ff653c0e980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 info.flags=0x17 2024-05-30T15:58:33.138 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.134+0000 7ff653c0e980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 to cache LRU end 2024-05-30T15:58:33.138 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.134+0000 7ff653c0e980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T15:58:33.138 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.134+0000 7ff653c0e980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-05-30T15:58:33.138 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.134+0000 7ff653c0e980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 2024-05-30T15:58:33.139 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.134+0000 7ff653c0e980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.0 2024-05-30T15:58:33.139 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.138+0000 7ff653c0e980 1 NOTICE: finished shard 0 (0 entries found) 2024-05-30T15:58:33.139 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.138+0000 7ff653c0e980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:33.139 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.138+0000 7ff653c0e980 1 NOTICE: finished shard 1 (0 entries found) 2024-05-30T15:58:33.139 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.138+0000 7ff653c0e980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.2 2024-05-30T15:58:33.139 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.138+0000 7ff653c0e980 1 NOTICE: finished shard 2 (0 entries found) 2024-05-30T15:58:33.139 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.138+0000 7ff653c0e980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.3 2024-05-30T15:58:33.140 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.138+0000 7ff653c0e980 1 NOTICE: finished shard 3 (0 entries found) 2024-05-30T15:58:33.140 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.138+0000 7ff653c0e980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.4 2024-05-30T15:58:33.141 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.138+0000 7ff653c0e980 1 NOTICE: finished shard 4 (0 entries found) 2024-05-30T15:58:33.141 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.138+0000 7ff653c0e980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:33.142 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.142+0000 7ff653c0e980 1 NOTICE: finished shard 5 (1 entries found) 2024-05-30T15:58:33.142 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.142+0000 7ff653c0e980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.6 2024-05-30T15:58:33.143 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.142+0000 7ff653c0e980 1 NOTICE: finished shard 6 (0 entries found) 2024-05-30T15:58:33.143 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.142+0000 7ff653c0e980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.7 2024-05-30T15:58:33.144 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.142+0000 7ff653c0e980 1 NOTICE: finished shard 7 (0 entries found) 2024-05-30T15:58:33.144 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.142+0000 7ff653c0e980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:33.145 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.142+0000 7ff653c0e980 1 NOTICE: finished shard 8 (1 entries found) 2024-05-30T15:58:33.145 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.142+0000 7ff653c0e980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.9 2024-05-30T15:58:33.145 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.146+0000 7ff653c0e980 1 NOTICE: finished shard 9 (0 entries found) 2024-05-30T15:58:33.145 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.146+0000 7ff653c0e980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:33.147 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.146+0000 7ff653c0e980 1 NOTICE: finished shard 10 (0 entries found) 2024-05-30T15:58:33.147 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.146+0000 7ff653c0e980 1 NOTICE: finished all shards (2 entries found) 2024-05-30T15:58:33.148 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.146+0000 7ff5077e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:33.154 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.154+0000 7ff653c0e980 20 remove_watcher() i=0 2024-05-30T15:58:33.154 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.154+0000 7ff653c0e980 2 removed watcher, disabling cache 2024-05-30T15:58:33.156 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.154+0000 7ff653c0e980 20 remove_watcher() i=1 2024-05-30T15:58:33.158 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.158+0000 7ff653c0e980 20 remove_watcher() i=2 2024-05-30T15:58:33.160 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.158+0000 7ff653c0e980 20 remove_watcher() i=3 2024-05-30T15:58:33.162 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.162+0000 7ff653c0e980 20 remove_watcher() i=4 2024-05-30T15:58:33.164 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.162+0000 7ff653c0e980 20 remove_watcher() i=5 2024-05-30T15:58:33.166 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.166+0000 7ff653c0e980 20 remove_watcher() i=6 2024-05-30T15:58:33.167 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.166+0000 7ff653c0e980 20 remove_watcher() i=7 2024-05-30T15:58:33.183 INFO:tasks.workunit.client.0.smithi006.stderr:TEST: bucket check olh fixes leftover OLH entries 2024-05-30T15:58:33.183 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T15:58:33.183 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket check olh --bucket check-bucket --fix --rgw-olh-pending-timeout-sec 0 --dump-keys 2024-05-30T15:58:33.214 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:58:33.214 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:58:33.260 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.258+0000 7f2b21ae1980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.261 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.262+0000 7f2b21ae1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:33.261 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.262+0000 7f2b21ae1980 20 realm 2024-05-30T15:58:33.262 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.262+0000 7f2b21ae1980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.262 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.262+0000 7f2b21ae1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:33.262 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.262+0000 7f2b21ae1980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T15:58:33.262 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.262+0000 7f2b21ae1980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.262 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.262+0000 7f2b21ae1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:33.262 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.262+0000 7f2b21ae1980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.264 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.262+0000 7f2b21ae1980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:33.264 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.262+0000 7f2b21ae1980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.264 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.262+0000 7f2b21ae1980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T15:58:33.265 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.262+0000 7f2b21ae1980 20 searching for the correct realm 2024-05-30T15:58:33.271 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.270+0000 7f2b1edf1640 0 -- 172.21.15.6:0/2851609920 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f2b10043950 msgr2=0x7f2b1000f3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:33.277 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.278+0000 7f2b1edf1640 0 -- 172.21.15.6:0/2851609920 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f29e807ba50 msgr2=0x7f29e807de80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:33.284 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.282+0000 7f2b21ae1980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T15:58:33.285 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.282+0000 7f2b21ae1980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T15:58:33.285 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.282+0000 7f2b21ae1980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T15:58:33.285 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.282+0000 7f2b21ae1980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T15:58:33.285 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.282+0000 7f2b21ae1980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T15:58:33.285 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.282+0000 7f2b21ae1980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T15:58:33.285 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.282+0000 7f2b21ae1980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.285 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.282+0000 7f2b21ae1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:33.285 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.282+0000 7f2b21ae1980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.285 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.286+0000 7f2b21ae1980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:33.286 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.286+0000 7f2b21ae1980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.287 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.286+0000 7f2b21ae1980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T15:58:33.287 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.286+0000 7f2b21ae1980 20 zone default found 2024-05-30T15:58:33.287 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.286+0000 7f2b21ae1980 4 Realm: () 2024-05-30T15:58:33.287 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.286+0000 7f2b21ae1980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T15:58:33.287 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.286+0000 7f2b21ae1980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T15:58:33.287 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.286+0000 7f2b21ae1980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T15:58:33.287 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.286+0000 7f2b21ae1980 20 zonegroup default 2024-05-30T15:58:33.287 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.286+0000 7f2b21ae1980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.287 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.286+0000 7f2b21ae1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:33.287 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.286+0000 7f2b21ae1980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.288 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.286+0000 7f2b21ae1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:33.288 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.286+0000 7f2b21ae1980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.288 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.286+0000 7f2b21ae1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:33.288 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.286+0000 7f2b21ae1980 20 started sync module instance, tier type = 2024-05-30T15:58:33.288 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.286+0000 7f2b21ae1980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T15:58:33.307 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.306+0000 7f2b21ae1980 20 add_watcher() i=0 2024-05-30T15:58:33.307 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.306+0000 7f2b21ae1980 20 add_watcher() i=1 2024-05-30T15:58:33.307 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.306+0000 7f2b21ae1980 20 add_watcher() i=2 2024-05-30T15:58:33.307 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.306+0000 7f2b21ae1980 20 add_watcher() i=3 2024-05-30T15:58:33.307 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.306+0000 7f2b21ae1980 20 add_watcher() i=4 2024-05-30T15:58:33.307 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.306+0000 7f2b21ae1980 20 add_watcher() i=5 2024-05-30T15:58:33.307 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.306+0000 7f2b21ae1980 20 add_watcher() i=6 2024-05-30T15:58:33.309 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.310+0000 7f2b21ae1980 20 add_watcher() i=7 2024-05-30T15:58:33.309 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.310+0000 7f2b21ae1980 2 all 8 watchers are set, enabling cache 2024-05-30T15:58:33.312 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.310+0000 7f29d5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:33.313 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.310+0000 7f2b21ae1980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:58:33.313 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.310+0000 7f2b21ae1980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:58:33.313 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.310+0000 7f2b21ae1980 5 note: GC not initialized 2024-05-30T15:58:33.313 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.310+0000 7f29d4fe1640 20 reqs_thread_entry: start 2024-05-30T15:58:33.314 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.310+0000 7f2b21ae1980 20 init_complete bucket index max shards: 11 2024-05-30T15:58:33.314 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.310+0000 7f2b21ae1980 20 Filter name: none 2024-05-30T15:58:33.314 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.310+0000 7f294e7fc640 20 reqs_thread_entry: start 2024-05-30T15:58:33.314 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.314+0000 7f2b21ae1980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-05-30T15:58:33.314 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.314+0000 7f2b21ae1980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.314+0000 7f2b21ae1980 20 rados_obj.operate() r=0 bl.length=190 2024-05-30T15:58:33.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.314+0000 7f2b21ae1980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-05-30T15:58:33.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.314+0000 7f2b21ae1980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-05-30T15:58:33.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.314+0000 7f2b21ae1980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 : miss 2024-05-30T15:58:33.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.314+0000 7f2b21ae1980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.314+0000 7f2b21ae1980 20 rados_obj.operate() r=0 bl.length=404 2024-05-30T15:58:33.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.314+0000 7f2b21ae1980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 info.flags=0x17 2024-05-30T15:58:33.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.314+0000 7f2b21ae1980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 to cache LRU end 2024-05-30T15:58:33.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.314+0000 7f2b21ae1980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T15:58:33.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.314+0000 7f2b21ae1980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-05-30T15:58:33.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.314+0000 7f2b21ae1980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 2024-05-30T15:58:33.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.314+0000 7f2b21ae1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.0 2024-05-30T15:58:33.316 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.314+0000 7f2b21ae1980 1 NOTICE: finished shard 0 (0 entries removed) 2024-05-30T15:58:33.316 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.314+0000 7f2b21ae1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:33.316 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.314+0000 7f2b21ae1980 1 NOTICE: finished shard 1 (0 entries removed) 2024-05-30T15:58:33.316 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.314+0000 7f2b21ae1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.2 2024-05-30T15:58:33.316 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.314+0000 7f2b21ae1980 1 NOTICE: finished shard 2 (0 entries removed) 2024-05-30T15:58:33.316 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.314+0000 7f2b21ae1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.3 2024-05-30T15:58:33.317 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.314+0000 7f2b21ae1980 1 NOTICE: finished shard 3 (0 entries removed) 2024-05-30T15:58:33.317 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.314+0000 7f2b21ae1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.4 2024-05-30T15:58:33.318 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.318+0000 7f2b21ae1980 1 NOTICE: finished shard 4 (0 entries removed) 2024-05-30T15:58:33.318 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.318+0000 7f2b21ae1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:33.319 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.318+0000 7f2b21ae1980 20 get_obj_state: octx=0x55acc26f2a20 obj=check-bucket:d state=0x55acc29a4218 s->prefetch_data=0 2024-05-30T15:58:33.319 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.318+0000 7f2b21ae1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.6 2024-05-30T15:58:33.320 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.318+0000 7f2b21ae1980 1 NOTICE: finished shard 6 (0 entries removed) 2024-05-30T15:58:33.320 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.318+0000 7f2b21ae1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.7 2024-05-30T15:58:33.321 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.318+0000 7f2b21ae1980 1 NOTICE: finished shard 7 (0 entries removed) 2024-05-30T15:58:33.321 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.318+0000 7f2b21ae1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:33.321 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.322+0000 7f2b21ae1980 20 get_obj_state: octx=0x55acc27044e0 obj=check-bucket:c state=0x55acc29a02d8 s->prefetch_data=0 2024-05-30T15:58:33.321 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.322+0000 7f2b21ae1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.9 2024-05-30T15:58:33.322 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.322+0000 7f2b21ae1980 1 NOTICE: finished shard 9 (0 entries removed) 2024-05-30T15:58:33.322 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.322+0000 7f2b21ae1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:33.323 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.322+0000 7f2b21ae1980 1 NOTICE: finished shard 10 (0 entries removed) 2024-05-30T15:58:33.323 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.322+0000 7f2b21ae1980 20 get_obj_state: setting s->obj_tag to tn0v5nl9htr8labacw03in173zpj8oi7 2024-05-30T15:58:33.323 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.322+0000 7f2b21ae1980 20 get_obj_state_impl: setting s->olh_tag to 3lzteeo6dqdh9sks1jix2732dhjccbdz 2024-05-30T15:58:33.323 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.322+0000 7f2b21ae1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:33.324 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.322+0000 7f2b21ae1980 20 get_obj_state: setting s->obj_tag to yy20qu0ussswpr322gx3xyyvhuhksvvt 2024-05-30T15:58:33.324 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.322+0000 7f2b21ae1980 20 get_obj_state_impl: setting s->olh_tag to abhp9uhv1kaqvgjrirnkp690j0aga0uy 2024-05-30T15:58:33.324 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.322+0000 7f2b21ae1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:33.324 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.322+0000 7f2b21ae1980 20 olh_log_entry: epoch=4 op=2 key=c[] 2024-05-30T15:58:33.324 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.322+0000 7f2b21ae1980 20 olh_log_entry: epoch=4 op=3 key=c[Bq8wWRmQzjfBdGNoMWam1NMJilknXQk] 2024-05-30T15:58:33.325 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.322+0000 7f2b21ae1980 20 get_obj_state: octx=0x55acc29ec5a0 obj=check-bucket:_:Bq8wWRmQzjfBdGNoMWam1NMJilknXQk_c state=0x55acc29f0028 s->prefetch_data=0 2024-05-30T15:58:33.325 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.322+0000 7f2b21ae1980 20 olh_log_entry: epoch=4 op=2 key=d[] 2024-05-30T15:58:33.325 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.322+0000 7f2b21ae1980 20 olh_log_entry: epoch=4 op=3 key=d[17pTua34WYFZ2QH9MuG-Gd2vDKFc92H] 2024-05-30T15:58:33.325 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.322+0000 7f2b21ae1980 20 get_obj_state: octx=0x55acc29cc5a0 obj=check-bucket:_:17pTua34WYFZ2QH9MuG-Gd2vDKFc92H_d state=0x55acc29f58c8 s->prefetch_data=0 2024-05-30T15:58:33.326 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.326+0000 7f2b21ae1980 20 get_obj_state: octx=0x55acc29ec5a0 obj=check-bucket:c state=0x55acc299cbf8 s->prefetch_data=0 2024-05-30T15:58:33.327 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.326+0000 7f2b21ae1980 20 get_obj_state: setting s->obj_tag to yy20qu0ussswpr322gx3xyyvhuhksvvt 2024-05-30T15:58:33.327 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.326+0000 7f2b21ae1980 20 get_obj_state_impl: setting s->olh_tag to abhp9uhv1kaqvgjrirnkp690j0aga0uy 2024-05-30T15:58:33.327 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.326+0000 7f2b21ae1980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=abhp9uhv1kaqvgjrirnkp690j0aga0uy 2024-05-30T15:58:33.328 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.326+0000 7f2b21ae1980 20 get_obj_state: octx=0x55acc29cc5a0 obj=check-bucket:d state=0x55acc29f0878 s->prefetch_data=0 2024-05-30T15:58:33.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.330+0000 7f2b21ae1980 20 get_obj_state: setting s->obj_tag to tn0v5nl9htr8labacw03in173zpj8oi7 2024-05-30T15:58:33.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.330+0000 7f2b21ae1980 20 get_obj_state_impl: setting s->olh_tag to 3lzteeo6dqdh9sks1jix2732dhjccbdz 2024-05-30T15:58:33.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.330+0000 7f2b21ae1980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=3lzteeo6dqdh9sks1jix2732dhjccbdz 2024-05-30T15:58:33.333 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.330+0000 7f2b21ae1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:33.335 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.334+0000 7f2b21ae1980 1 NOTICE: finished shard 8 (1 entries removed) 2024-05-30T15:58:33.338 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.338+0000 7f2b21ae1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:33.343 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.342+0000 7f2b21ae1980 1 NOTICE: finished shard 5 (1 entries removed) 2024-05-30T15:58:33.343 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.342+0000 7f2b21ae1980 1 NOTICE: finished all shards (2 entries removed) 2024-05-30T15:58:33.344 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.342+0000 7f29d5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:33.349 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.346+0000 7f2b21ae1980 20 remove_watcher() i=0 2024-05-30T15:58:33.349 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.346+0000 7f2b21ae1980 2 removed watcher, disabling cache 2024-05-30T15:58:33.351 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.350+0000 7f2b21ae1980 20 remove_watcher() i=1 2024-05-30T15:58:33.353 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.354+0000 7f2b21ae1980 20 remove_watcher() i=2 2024-05-30T15:58:33.355 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.354+0000 7f2b21ae1980 20 remove_watcher() i=3 2024-05-30T15:58:33.357 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.358+0000 7f2b21ae1980 20 remove_watcher() i=4 2024-05-30T15:58:33.359 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.358+0000 7f2b21ae1980 20 remove_watcher() i=5 2024-05-30T15:58:33.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.358+0000 7f2b21ae1980 20 remove_watcher() i=6 2024-05-30T15:58:33.362 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.362+0000 7f2b21ae1980 20 remove_watcher() i=7 2024-05-30T15:58:33.376 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bi list --bucket check-bucket --object d 2024-05-30T15:58:33.405 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:58:33.405 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:58:33.449 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.446+0000 7fe305dc6980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.451 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.450+0000 7fe305dc6980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:33.451 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.450+0000 7fe305dc6980 20 realm 2024-05-30T15:58:33.451 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.450+0000 7fe305dc6980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.451 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.450+0000 7fe305dc6980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:33.451 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.450+0000 7fe305dc6980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T15:58:33.451 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.450+0000 7fe305dc6980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.451 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.450+0000 7fe305dc6980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:33.452 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.450+0000 7fe305dc6980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.453 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.450+0000 7fe305dc6980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:33.453 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.450+0000 7fe305dc6980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.453 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.454+0000 7fe305dc6980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T15:58:33.453 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.454+0000 7fe305dc6980 20 searching for the correct realm 2024-05-30T15:58:33.461 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.458+0000 7fe3040d8640 0 -- 172.21.15.6:0/3347523923 >> [v2:172.21.15.204:6800/3687190393,v1:172.21.15.204:6801/3687190393] conn(0x7fe2f80b63f0 msgr2=0x7fe2f80b6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:33.676 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.674+0000 7fe3038d7640 0 -- 172.21.15.6:0/3347523923 >> [v2:172.21.15.204:6816/2677861390,v1:172.21.15.204:6817/2677861390] conn(0x5625222b0220 msgr2=0x5625222b2630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:33.684 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.682+0000 7fe305dc6980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T15:58:33.684 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.682+0000 7fe305dc6980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T15:58:33.684 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.682+0000 7fe305dc6980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T15:58:33.684 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.682+0000 7fe305dc6980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T15:58:33.684 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.682+0000 7fe305dc6980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T15:58:33.684 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.682+0000 7fe305dc6980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T15:58:33.684 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.682+0000 7fe305dc6980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.685 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.682+0000 7fe305dc6980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:33.685 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.682+0000 7fe305dc6980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.685 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.686+0000 7fe305dc6980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:33.686 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.686+0000 7fe305dc6980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.687 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.686+0000 7fe305dc6980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T15:58:33.687 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.686+0000 7fe305dc6980 20 zone default found 2024-05-30T15:58:33.687 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.686+0000 7fe305dc6980 4 Realm: () 2024-05-30T15:58:33.687 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.686+0000 7fe305dc6980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T15:58:33.687 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.686+0000 7fe305dc6980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T15:58:33.687 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.686+0000 7fe305dc6980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T15:58:33.687 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.686+0000 7fe305dc6980 20 zonegroup default 2024-05-30T15:58:33.687 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.686+0000 7fe305dc6980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.688 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.686+0000 7fe305dc6980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:33.688 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.686+0000 7fe305dc6980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.688 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.686+0000 7fe305dc6980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:33.688 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.686+0000 7fe305dc6980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.689 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.690+0000 7fe305dc6980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:33.689 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.690+0000 7fe305dc6980 20 started sync module instance, tier type = 2024-05-30T15:58:33.689 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.690+0000 7fe305dc6980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T15:58:33.714 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.714+0000 7fe305dc6980 20 add_watcher() i=0 2024-05-30T15:58:33.714 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.714+0000 7fe305dc6980 20 add_watcher() i=1 2024-05-30T15:58:33.714 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.714+0000 7fe305dc6980 20 add_watcher() i=2 2024-05-30T15:58:33.714 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.714+0000 7fe305dc6980 20 add_watcher() i=3 2024-05-30T15:58:33.714 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.714+0000 7fe305dc6980 20 add_watcher() i=4 2024-05-30T15:58:33.714 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.714+0000 7fe305dc6980 20 add_watcher() i=5 2024-05-30T15:58:33.715 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.714+0000 7fe305dc6980 20 add_watcher() i=6 2024-05-30T15:58:33.716 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.714+0000 7fe305dc6980 20 add_watcher() i=7 2024-05-30T15:58:33.716 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.714+0000 7fe305dc6980 2 all 8 watchers are set, enabling cache 2024-05-30T15:58:33.721 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.718+0000 7fe1b9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:33.722 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.722+0000 7fe305dc6980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:58:33.722 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.722+0000 7fe305dc6980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:58:33.722 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.722+0000 7fe305dc6980 5 note: GC not initialized 2024-05-30T15:58:33.722 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.722+0000 7fe1b8fe1640 20 reqs_thread_entry: start 2024-05-30T15:58:33.723 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.722+0000 7fe305dc6980 20 init_complete bucket index max shards: 11 2024-05-30T15:58:33.723 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.722+0000 7fe305dc6980 20 Filter name: none 2024-05-30T15:58:33.723 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.722+0000 7fe1327fc640 20 reqs_thread_entry: start 2024-05-30T15:58:33.723 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.722+0000 7fe305dc6980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.724 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.722+0000 7fe305dc6980 20 rados_obj.operate() r=0 bl.length=190 2024-05-30T15:58:33.724 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.722+0000 7fe305dc6980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.725 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.722+0000 7fe305dc6980 20 rados_obj.operate() r=0 bl.length=404 2024-05-30T15:58:33.725 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.722+0000 7fe305dc6980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T15:58:33.725 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.722+0000 7fe305dc6980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-05-30T15:58:33.725 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.722+0000 7fe305dc6980 20 INFO: main: starting shard=0 2024-05-30T15:58:33.725 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.722+0000 7fe305dc6980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.0 2024-05-30T15:58:33.726 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.726+0000 7fe305dc6980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:33.726 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.726+0000 7fe305dc6980 20 INFO: main: starting shard=1 2024-05-30T15:58:33.726 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.726+0000 7fe305dc6980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:33.727 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.726+0000 7fe305dc6980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:33.727 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.726+0000 7fe305dc6980 20 INFO: main: starting shard=2 2024-05-30T15:58:33.727 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.726+0000 7fe305dc6980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.2 2024-05-30T15:58:33.727 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.726+0000 7fe305dc6980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:33.727 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.726+0000 7fe305dc6980 20 INFO: main: starting shard=3 2024-05-30T15:58:33.727 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.726+0000 7fe305dc6980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.3 2024-05-30T15:58:33.728 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.726+0000 7fe305dc6980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:33.728 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.726+0000 7fe305dc6980 20 INFO: main: starting shard=4 2024-05-30T15:58:33.728 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.726+0000 7fe305dc6980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.4 2024-05-30T15:58:33.729 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.730+0000 7fe305dc6980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:33.729 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.730+0000 7fe305dc6980 20 INFO: main: starting shard=5 2024-05-30T15:58:33.730 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.730+0000 7fe305dc6980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:33.730 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.730+0000 7fe305dc6980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:33.731 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.730+0000 7fe305dc6980 20 INFO: main: starting shard=6 2024-05-30T15:58:33.731 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.730+0000 7fe305dc6980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.6 2024-05-30T15:58:33.731 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.730+0000 7fe305dc6980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:33.731 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.730+0000 7fe305dc6980 20 INFO: main: starting shard=7 2024-05-30T15:58:33.731 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.730+0000 7fe305dc6980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.7 2024-05-30T15:58:33.733 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.730+0000 7fe305dc6980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:33.733 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.730+0000 7fe305dc6980 20 INFO: main: starting shard=8 2024-05-30T15:58:33.733 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.730+0000 7fe305dc6980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:33.734 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.734+0000 7fe305dc6980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:33.734 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.734+0000 7fe305dc6980 20 INFO: main: starting shard=9 2024-05-30T15:58:33.734 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.734+0000 7fe305dc6980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.9 2024-05-30T15:58:33.734 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.734+0000 7fe305dc6980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:33.734 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.734+0000 7fe305dc6980 20 INFO: main: starting shard=10 2024-05-30T15:58:33.734 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.734+0000 7fe305dc6980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:33.736 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.734+0000 7fe305dc6980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:33.736 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.734+0000 7fe305dc6980 20 INFO: main: done 2024-05-30T15:58:33.738 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.738+0000 7fe1b9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:33.743 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.742+0000 7fe305dc6980 20 remove_watcher() i=0 2024-05-30T15:58:33.743 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.742+0000 7fe305dc6980 2 removed watcher, disabling cache 2024-05-30T15:58:33.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.746+0000 7fe305dc6980 20 remove_watcher() i=1 2024-05-30T15:58:33.748 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.746+0000 7fe305dc6980 20 remove_watcher() i=2 2024-05-30T15:58:33.750 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.750+0000 7fe305dc6980 20 remove_watcher() i=3 2024-05-30T15:58:33.753 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.750+0000 7fe305dc6980 20 remove_watcher() i=4 2024-05-30T15:58:33.754 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.754+0000 7fe305dc6980 20 remove_watcher() i=5 2024-05-30T15:58:33.756 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.754+0000 7fe305dc6980 20 remove_watcher() i=6 2024-05-30T15:58:33.758 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.758+0000 7fe305dc6980 20 remove_watcher() i=7 2024-05-30T15:58:33.775 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bi list --bucket check-bucket --object c 2024-05-30T15:58:33.805 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:58:33.805 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:58:33.851 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.850+0000 7f9e07344980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.853 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.850+0000 7f9e07344980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:33.853 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.850+0000 7f9e07344980 20 realm 2024-05-30T15:58:33.853 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.850+0000 7f9e07344980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.854 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.850+0000 7f9e07344980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:33.854 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.850+0000 7f9e07344980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T15:58:33.854 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.850+0000 7f9e07344980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.854 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.850+0000 7f9e07344980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:33.854 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.850+0000 7f9e07344980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.854+0000 7f9e07344980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:33.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.854+0000 7f9e07344980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.856 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.854+0000 7f9e07344980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T15:58:33.856 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.854+0000 7f9e07344980 20 searching for the correct realm 2024-05-30T15:58:33.862 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.862+0000 7f9e05656640 0 -- 172.21.15.6:0/1197084618 >> [v2:172.21.15.6:6808/1897362715,v1:172.21.15.6:6809/1897362715] conn(0x5633a37509c0 msgr2=0x5633a3752e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:33.868 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.866+0000 7f9e04654640 0 -- 172.21.15.6:0/1197084618 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f9df40c44d0 msgr2=0x7f9df40b9040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T15:58:33.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.874+0000 7f9e07344980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T15:58:33.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.874+0000 7f9e07344980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T15:58:33.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.874+0000 7f9e07344980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T15:58:33.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.874+0000 7f9e07344980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T15:58:33.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.874+0000 7f9e07344980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T15:58:33.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.874+0000 7f9e07344980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T15:58:33.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.874+0000 7f9e07344980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.874+0000 7f9e07344980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:33.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.874+0000 7f9e07344980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.878 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.878+0000 7f9e07344980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:33.878 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.878+0000 7f9e07344980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.878+0000 7f9e07344980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T15:58:33.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.878+0000 7f9e07344980 20 zone default found 2024-05-30T15:58:33.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.878+0000 7f9e07344980 4 Realm: () 2024-05-30T15:58:33.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.878+0000 7f9e07344980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T15:58:33.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.878+0000 7f9e07344980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T15:58:33.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.878+0000 7f9e07344980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T15:58:33.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.878+0000 7f9e07344980 20 zonegroup default 2024-05-30T15:58:33.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.878+0000 7f9e07344980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.878+0000 7f9e07344980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:33.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.878+0000 7f9e07344980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.880 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.878+0000 7f9e07344980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:33.880 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.878+0000 7f9e07344980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.880 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.878+0000 7f9e07344980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:33.880 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.878+0000 7f9e07344980 20 started sync module instance, tier type = 2024-05-30T15:58:33.880 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.878+0000 7f9e07344980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T15:58:33.897 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.894+0000 7f9e07344980 20 add_watcher() i=0 2024-05-30T15:58:33.897 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.894+0000 7f9e07344980 20 add_watcher() i=1 2024-05-30T15:58:33.897 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.894+0000 7f9e07344980 20 add_watcher() i=2 2024-05-30T15:58:33.897 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.894+0000 7f9e07344980 20 add_watcher() i=3 2024-05-30T15:58:33.897 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.894+0000 7f9e07344980 20 add_watcher() i=4 2024-05-30T15:58:33.897 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.894+0000 7f9e07344980 20 add_watcher() i=5 2024-05-30T15:58:33.897 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.894+0000 7f9e07344980 20 add_watcher() i=6 2024-05-30T15:58:33.898 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.898+0000 7f9e07344980 20 add_watcher() i=7 2024-05-30T15:58:33.898 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.898+0000 7f9e07344980 2 all 8 watchers are set, enabling cache 2024-05-30T15:58:33.901 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.898+0000 7f9cbafe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:33.902 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.902+0000 7f9e07344980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:58:33.902 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.902+0000 7f9e07344980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:58:33.902 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.902+0000 7f9e07344980 5 note: GC not initialized 2024-05-30T15:58:33.902 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.902+0000 7f9cb9fe3640 20 reqs_thread_entry: start 2024-05-30T15:58:33.902 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.902+0000 7f9e07344980 20 init_complete bucket index max shards: 11 2024-05-30T15:58:33.902 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.902+0000 7f9e07344980 20 Filter name: none 2024-05-30T15:58:33.902 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.902+0000 7f9c2b7fe640 20 reqs_thread_entry: start 2024-05-30T15:58:33.902 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.902+0000 7f9e07344980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.903 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.902+0000 7f9e07344980 20 rados_obj.operate() r=0 bl.length=190 2024-05-30T15:58:33.903 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.902+0000 7f9e07344980 20 rados->read ofs=0 len=0 2024-05-30T15:58:33.903 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.902+0000 7f9e07344980 20 rados_obj.operate() r=0 bl.length=404 2024-05-30T15:58:33.903 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.902+0000 7f9e07344980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T15:58:33.903 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.902+0000 7f9e07344980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-05-30T15:58:33.903 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.902+0000 7f9e07344980 20 INFO: main: starting shard=0 2024-05-30T15:58:33.904 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.902+0000 7f9e07344980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.0 2024-05-30T15:58:33.904 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.902+0000 7f9e07344980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:33.904 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.902+0000 7f9e07344980 20 INFO: main: starting shard=1 2024-05-30T15:58:33.904 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.902+0000 7f9e07344980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:33.904 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.902+0000 7f9e07344980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:33.904 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.902+0000 7f9e07344980 20 INFO: main: starting shard=2 2024-05-30T15:58:33.904 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.902+0000 7f9e07344980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.2 2024-05-30T15:58:33.905 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.902+0000 7f9e07344980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:33.905 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.902+0000 7f9e07344980 20 INFO: main: starting shard=3 2024-05-30T15:58:33.905 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.902+0000 7f9e07344980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.3 2024-05-30T15:58:33.905 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.906+0000 7f9e07344980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:33.905 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.906+0000 7f9e07344980 20 INFO: main: starting shard=4 2024-05-30T15:58:33.905 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.906+0000 7f9e07344980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.4 2024-05-30T15:58:33.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.906+0000 7f9e07344980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:33.907 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.906+0000 7f9e07344980 20 INFO: main: starting shard=5 2024-05-30T15:58:33.907 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.906+0000 7f9e07344980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:33.908 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.906+0000 7f9e07344980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:33.908 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.906+0000 7f9e07344980 20 INFO: main: starting shard=6 2024-05-30T15:58:33.908 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.906+0000 7f9e07344980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.6 2024-05-30T15:58:33.909 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.906+0000 7f9e07344980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:33.909 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.906+0000 7f9e07344980 20 INFO: main: starting shard=7 2024-05-30T15:58:33.909 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.906+0000 7f9e07344980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.7 2024-05-30T15:58:33.910 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.910+0000 7f9e07344980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:33.910 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.910+0000 7f9e07344980 20 INFO: main: starting shard=8 2024-05-30T15:58:33.910 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.910+0000 7f9e07344980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:33.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.910+0000 7f9e07344980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:33.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.910+0000 7f9e07344980 20 INFO: main: starting shard=9 2024-05-30T15:58:33.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.910+0000 7f9e07344980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.9 2024-05-30T15:58:33.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.910+0000 7f9e07344980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:33.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.910+0000 7f9e07344980 20 INFO: main: starting shard=10 2024-05-30T15:58:33.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.910+0000 7f9e07344980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:33.912 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.910+0000 7f9e07344980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:33.912 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.910+0000 7f9e07344980 20 INFO: main: done 2024-05-30T15:58:33.913 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.914+0000 7f9cbafe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:33.919 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.918+0000 7f9e07344980 20 remove_watcher() i=0 2024-05-30T15:58:33.919 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.918+0000 7f9e07344980 2 removed watcher, disabling cache 2024-05-30T15:58:33.921 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.922+0000 7f9e07344980 20 remove_watcher() i=1 2024-05-30T15:58:33.924 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.922+0000 7f9e07344980 20 remove_watcher() i=2 2024-05-30T15:58:33.926 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.926+0000 7f9e07344980 20 remove_watcher() i=3 2024-05-30T15:58:33.928 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.926+0000 7f9e07344980 20 remove_watcher() i=4 2024-05-30T15:58:33.929 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.930+0000 7f9e07344980 20 remove_watcher() i=5 2024-05-30T15:58:33.931 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.930+0000 7f9e07344980 20 remove_watcher() i=6 2024-05-30T15:58:33.932 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:33.930+0000 7f9e07344980 20 remove_watcher() i=7 2024-05-30T15:58:33.947 INFO:tasks.workunit.client.0.smithi006.stderr:TEST: bucket check olh does not find new leftover OLH entries after fix 2024-05-30T15:58:33.947 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T15:58:33.948 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket check olh --bucket check-bucket --dump-keys 2024-05-30T15:58:33.979 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:58:33.979 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:58:34.024 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.022+0000 7f427a2fe980 20 rados->read ofs=0 len=0 2024-05-30T15:58:34.025 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.022+0000 7f427a2fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:34.025 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.022+0000 7f427a2fe980 20 realm 2024-05-30T15:58:34.025 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.022+0000 7f427a2fe980 20 rados->read ofs=0 len=0 2024-05-30T15:58:34.025 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.026+0000 7f427a2fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:34.025 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.026+0000 7f427a2fe980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T15:58:34.026 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.026+0000 7f427a2fe980 20 rados->read ofs=0 len=0 2024-05-30T15:58:34.026 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.026+0000 7f427a2fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:34.026 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.026+0000 7f427a2fe980 20 rados->read ofs=0 len=0 2024-05-30T15:58:34.027 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.026+0000 7f427a2fe980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:34.027 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.026+0000 7f427a2fe980 20 rados->read ofs=0 len=0 2024-05-30T15:58:34.028 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.026+0000 7f427a2fe980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T15:58:34.028 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.026+0000 7f427a2fe980 20 searching for the correct realm 2024-05-30T15:58:34.032 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.030+0000 7f4278610640 0 -- 172.21.15.6:0/3176934654 >> [v2:172.21.15.204:6823/3156311527,v1:172.21.15.204:6825/3156311527] conn(0x7f426c027040 msgr2=0x7f426c03c960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T15:58:34.040 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.038+0000 7f4277e0f640 0 -- 172.21.15.6:0/3176934654 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f426c046540 msgr2=0x7f426c09b050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T15:58:34.256 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.254+0000 7f427a2fe980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T15:58:34.256 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.254+0000 7f427a2fe980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T15:58:34.256 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.254+0000 7f427a2fe980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T15:58:34.256 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.254+0000 7f427a2fe980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T15:58:34.256 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.254+0000 7f427a2fe980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T15:58:34.256 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.254+0000 7f427a2fe980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T15:58:34.256 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.254+0000 7f427a2fe980 20 rados->read ofs=0 len=0 2024-05-30T15:58:34.256 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.254+0000 7f427a2fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:34.257 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.254+0000 7f427a2fe980 20 rados->read ofs=0 len=0 2024-05-30T15:58:34.257 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.254+0000 7f427a2fe980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:34.257 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.254+0000 7f427a2fe980 20 rados->read ofs=0 len=0 2024-05-30T15:58:34.258 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.258+0000 7f427a2fe980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T15:58:34.258 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.258+0000 7f427a2fe980 20 zone default found 2024-05-30T15:58:34.258 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.258+0000 7f427a2fe980 4 Realm: () 2024-05-30T15:58:34.258 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.258+0000 7f427a2fe980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T15:58:34.258 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.258+0000 7f427a2fe980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T15:58:34.259 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.258+0000 7f427a2fe980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T15:58:34.259 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.258+0000 7f427a2fe980 20 zonegroup default 2024-05-30T15:58:34.259 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.258+0000 7f427a2fe980 20 rados->read ofs=0 len=0 2024-05-30T15:58:34.259 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.258+0000 7f427a2fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:34.259 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.258+0000 7f427a2fe980 20 rados->read ofs=0 len=0 2024-05-30T15:58:34.260 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.258+0000 7f427a2fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:34.260 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.258+0000 7f427a2fe980 20 rados->read ofs=0 len=0 2024-05-30T15:58:34.260 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.258+0000 7f427a2fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:34.261 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.258+0000 7f427a2fe980 20 started sync module instance, tier type = 2024-05-30T15:58:34.261 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.258+0000 7f427a2fe980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T15:58:34.285 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.286+0000 7f427a2fe980 20 add_watcher() i=0 2024-05-30T15:58:34.285 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.286+0000 7f427a2fe980 20 add_watcher() i=1 2024-05-30T15:58:34.286 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.286+0000 7f427a2fe980 20 add_watcher() i=2 2024-05-30T15:58:34.286 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.286+0000 7f427a2fe980 20 add_watcher() i=3 2024-05-30T15:58:34.286 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.286+0000 7f427a2fe980 20 add_watcher() i=4 2024-05-30T15:58:34.286 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.286+0000 7f427a2fe980 20 add_watcher() i=5 2024-05-30T15:58:34.286 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.286+0000 7f427a2fe980 20 add_watcher() i=6 2024-05-30T15:58:34.288 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.286+0000 7f427a2fe980 20 add_watcher() i=7 2024-05-30T15:58:34.288 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.286+0000 7f427a2fe980 2 all 8 watchers are set, enabling cache 2024-05-30T15:58:34.292 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.290+0000 7f412dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:34.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.294+0000 7f427a2fe980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:58:34.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.294+0000 7f427a2fe980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:58:34.294 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.294+0000 7f427a2fe980 5 note: GC not initialized 2024-05-30T15:58:34.294 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.294+0000 7f412cfe1640 20 reqs_thread_entry: start 2024-05-30T15:58:34.295 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.294+0000 7f427a2fe980 20 init_complete bucket index max shards: 11 2024-05-30T15:58:34.295 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.294+0000 7f427a2fe980 20 Filter name: none 2024-05-30T15:58:34.295 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.294+0000 7f40a67fc640 20 reqs_thread_entry: start 2024-05-30T15:58:34.295 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.294+0000 7f427a2fe980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-05-30T15:58:34.295 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.294+0000 7f427a2fe980 20 rados->read ofs=0 len=0 2024-05-30T15:58:34.296 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.294+0000 7f427a2fe980 20 rados_obj.operate() r=0 bl.length=190 2024-05-30T15:58:34.296 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.294+0000 7f427a2fe980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-05-30T15:58:34.296 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.294+0000 7f427a2fe980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-05-30T15:58:34.296 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.294+0000 7f427a2fe980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 : miss 2024-05-30T15:58:34.296 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.294+0000 7f427a2fe980 20 rados->read ofs=0 len=0 2024-05-30T15:58:34.297 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.294+0000 7f427a2fe980 20 rados_obj.operate() r=0 bl.length=404 2024-05-30T15:58:34.297 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.294+0000 7f427a2fe980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 info.flags=0x17 2024-05-30T15:58:34.297 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.294+0000 7f427a2fe980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 to cache LRU end 2024-05-30T15:58:34.297 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.294+0000 7f427a2fe980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T15:58:34.297 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.294+0000 7f427a2fe980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-05-30T15:58:34.297 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.294+0000 7f427a2fe980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 2024-05-30T15:58:34.297 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.294+0000 7f427a2fe980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.0 2024-05-30T15:58:34.298 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.294+0000 7f427a2fe980 1 NOTICE: finished shard 0 (0 entries found) 2024-05-30T15:58:34.298 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.294+0000 7f427a2fe980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:34.298 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.298+0000 7f427a2fe980 1 NOTICE: finished shard 1 (0 entries found) 2024-05-30T15:58:34.298 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.298+0000 7f427a2fe980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.2 2024-05-30T15:58:34.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.298+0000 7f427a2fe980 1 NOTICE: finished shard 2 (0 entries found) 2024-05-30T15:58:34.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.298+0000 7f427a2fe980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.3 2024-05-30T15:58:34.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.298+0000 7f427a2fe980 1 NOTICE: finished shard 3 (0 entries found) 2024-05-30T15:58:34.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.298+0000 7f427a2fe980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.4 2024-05-30T15:58:34.301 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.298+0000 7f427a2fe980 1 NOTICE: finished shard 4 (0 entries found) 2024-05-30T15:58:34.301 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.298+0000 7f427a2fe980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:34.302 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.302+0000 7f427a2fe980 1 NOTICE: finished shard 5 (0 entries found) 2024-05-30T15:58:34.302 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.302+0000 7f427a2fe980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.6 2024-05-30T15:58:34.302 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.302+0000 7f427a2fe980 1 NOTICE: finished shard 6 (0 entries found) 2024-05-30T15:58:34.302 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.302+0000 7f427a2fe980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.7 2024-05-30T15:58:34.303 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.302+0000 7f427a2fe980 1 NOTICE: finished shard 7 (0 entries found) 2024-05-30T15:58:34.304 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.302+0000 7f427a2fe980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:34.304 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.302+0000 7f427a2fe980 1 NOTICE: finished shard 8 (0 entries found) 2024-05-30T15:58:34.304 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.302+0000 7f427a2fe980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.9 2024-05-30T15:58:34.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.302+0000 7f427a2fe980 1 NOTICE: finished shard 9 (0 entries found) 2024-05-30T15:58:34.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.302+0000 7f427a2fe980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:34.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.306+0000 7f427a2fe980 1 NOTICE: finished shard 10 (0 entries found) 2024-05-30T15:58:34.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.306+0000 7f427a2fe980 1 NOTICE: finished all shards (0 entries found) 2024-05-30T15:58:34.307 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.306+0000 7f412dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:34.313 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.310+0000 7f427a2fe980 20 remove_watcher() i=0 2024-05-30T15:58:34.313 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.310+0000 7f427a2fe980 2 removed watcher, disabling cache 2024-05-30T15:58:34.316 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.314+0000 7f427a2fe980 20 remove_watcher() i=1 2024-05-30T15:58:34.318 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.318+0000 7f427a2fe980 20 remove_watcher() i=2 2024-05-30T15:58:34.320 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.318+0000 7f427a2fe980 20 remove_watcher() i=3 2024-05-30T15:58:34.323 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.322+0000 7f427a2fe980 20 remove_watcher() i=4 2024-05-30T15:58:34.324 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.322+0000 7f427a2fe980 20 remove_watcher() i=5 2024-05-30T15:58:34.326 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.326+0000 7f427a2fe980 20 remove_watcher() i=6 2024-05-30T15:58:34.328 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:34.326+0000 7f427a2fe980 20 remove_watcher() i=7 2024-05-30T15:58:34.352 INFO:tasks.workunit.client.0.smithi006.stderr:TEST: verify that bucket check fixes do not affect null version objects 2024-05-30T15:58:34.352 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T15:58:34.573 INFO:tasks.workunit.client.0.smithi006.stderr:TEST: bucket check stats are correct in the presence of unlinked entries 2024-05-30T15:58:34.573 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T15:58:36.439 INFO:tasks.workunit.client.0.smithi006.stderr:ceph config set client rgw_debug_inject_set_olh_err 2 2024-05-30T15:58:36.875 INFO:tasks.workunit.client.0.smithi006.stderr:ceph config set client rgw_debug_inject_olh_cancel_modification_err true 2024-05-30T15:58:38.454 INFO:tasks.workunit.client.0.smithi006.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-05-30T15:58:38.455 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bi list --bucket check-bucket --object c 2024-05-30T15:58:38.493 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:58:38.493 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:58:38.538 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.538+0000 7f7c729bd980 20 rados->read ofs=0 len=0 2024-05-30T15:58:38.540 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.538+0000 7f7c729bd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:38.540 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.538+0000 7f7c729bd980 20 realm 2024-05-30T15:58:38.540 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.538+0000 7f7c729bd980 20 rados->read ofs=0 len=0 2024-05-30T15:58:38.540 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.538+0000 7f7c729bd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:38.540 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.538+0000 7f7c729bd980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T15:58:38.541 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.538+0000 7f7c729bd980 20 rados->read ofs=0 len=0 2024-05-30T15:58:38.541 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.538+0000 7f7c729bd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:38.541 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.538+0000 7f7c729bd980 20 rados->read ofs=0 len=0 2024-05-30T15:58:38.542 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.542+0000 7f7c729bd980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:38.542 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.542+0000 7f7c729bd980 20 rados->read ofs=0 len=0 2024-05-30T15:58:38.543 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.542+0000 7f7c729bd980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T15:58:38.543 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.542+0000 7f7c729bd980 20 searching for the correct realm 2024-05-30T15:58:38.548 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.546+0000 7f7c6fccd640 0 -- 172.21.15.6:0/3604560774 >> [v2:172.21.15.6:6819/1141406589,v1:172.21.15.6:6821/1141406589] conn(0x564e4ce12b00 msgr2=0x564e4ce14ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T15:58:38.555 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.554+0000 7f7c704ce640 0 -- 172.21.15.6:0/3604560774 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f7c64008040 msgr2=0x7f7c64022450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:38.772 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.770+0000 7f7c729bd980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T15:58:38.772 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.770+0000 7f7c729bd980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T15:58:38.772 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.770+0000 7f7c729bd980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T15:58:38.772 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.770+0000 7f7c729bd980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T15:58:38.772 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.770+0000 7f7c729bd980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T15:58:38.773 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.770+0000 7f7c729bd980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T15:58:38.773 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.770+0000 7f7c729bd980 20 rados->read ofs=0 len=0 2024-05-30T15:58:38.773 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.770+0000 7f7c729bd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:38.773 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.770+0000 7f7c729bd980 20 rados->read ofs=0 len=0 2024-05-30T15:58:38.773 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.774+0000 7f7c729bd980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:38.773 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.774+0000 7f7c729bd980 20 rados->read ofs=0 len=0 2024-05-30T15:58:38.775 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.774+0000 7f7c729bd980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T15:58:38.775 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.774+0000 7f7c729bd980 20 zone default found 2024-05-30T15:58:38.775 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.774+0000 7f7c729bd980 4 Realm: () 2024-05-30T15:58:38.775 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.774+0000 7f7c729bd980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T15:58:38.775 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.774+0000 7f7c729bd980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T15:58:38.775 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.774+0000 7f7c729bd980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T15:58:38.775 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.774+0000 7f7c729bd980 20 zonegroup default 2024-05-30T15:58:38.775 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.774+0000 7f7c729bd980 20 rados->read ofs=0 len=0 2024-05-30T15:58:38.775 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.774+0000 7f7c729bd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:38.776 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.774+0000 7f7c729bd980 20 rados->read ofs=0 len=0 2024-05-30T15:58:38.776 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.774+0000 7f7c729bd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:38.777 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.774+0000 7f7c729bd980 20 rados->read ofs=0 len=0 2024-05-30T15:58:38.777 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.778+0000 7f7c729bd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:38.777 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.778+0000 7f7c729bd980 20 started sync module instance, tier type = 2024-05-30T15:58:38.777 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.778+0000 7f7c729bd980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T15:58:38.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.802+0000 7f7c729bd980 20 add_watcher() i=0 2024-05-30T15:58:38.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.802+0000 7f7c729bd980 20 add_watcher() i=1 2024-05-30T15:58:38.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.802+0000 7f7c729bd980 20 add_watcher() i=2 2024-05-30T15:58:38.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.802+0000 7f7c729bd980 20 add_watcher() i=3 2024-05-30T15:58:38.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.802+0000 7f7c729bd980 20 add_watcher() i=4 2024-05-30T15:58:38.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.802+0000 7f7c729bd980 20 add_watcher() i=5 2024-05-30T15:58:38.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.802+0000 7f7c729bd980 20 add_watcher() i=6 2024-05-30T15:58:38.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.806+0000 7f7c729bd980 20 add_watcher() i=7 2024-05-30T15:58:38.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.806+0000 7f7c729bd980 2 all 8 watchers are set, enabling cache 2024-05-30T15:58:38.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.810+0000 7f7b26fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:38.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.810+0000 7f7c729bd980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:58:38.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.810+0000 7f7c729bd980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:58:38.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.810+0000 7f7c729bd980 5 note: GC not initialized 2024-05-30T15:58:38.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.810+0000 7f7b25fdb640 20 reqs_thread_entry: start 2024-05-30T15:58:38.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.810+0000 7f7c729bd980 20 init_complete bucket index max shards: 11 2024-05-30T15:58:38.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.810+0000 7f7c729bd980 20 Filter name: none 2024-05-30T15:58:38.812 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.810+0000 7f7a977fe640 20 reqs_thread_entry: start 2024-05-30T15:58:38.812 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.810+0000 7f7c729bd980 20 rados->read ofs=0 len=0 2024-05-30T15:58:38.812 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.810+0000 7f7c729bd980 20 rados_obj.operate() r=0 bl.length=190 2024-05-30T15:58:38.812 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.810+0000 7f7c729bd980 20 rados->read ofs=0 len=0 2024-05-30T15:58:38.814 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.814+0000 7f7c729bd980 20 rados_obj.operate() r=0 bl.length=404 2024-05-30T15:58:38.814 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.814+0000 7f7c729bd980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T15:58:38.814 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.814+0000 7f7c729bd980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-05-30T15:58:38.814 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.814+0000 7f7c729bd980 20 INFO: main: starting shard=0 2024-05-30T15:58:38.814 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.814+0000 7f7c729bd980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.0 2024-05-30T15:58:38.815 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.814+0000 7f7c729bd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:38.815 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.814+0000 7f7c729bd980 20 INFO: main: starting shard=1 2024-05-30T15:58:38.815 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.814+0000 7f7c729bd980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:38.816 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.814+0000 7f7c729bd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:38.816 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.814+0000 7f7c729bd980 20 INFO: main: starting shard=2 2024-05-30T15:58:38.816 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.814+0000 7f7c729bd980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.2 2024-05-30T15:58:38.818 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.818+0000 7f7c729bd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:38.818 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.818+0000 7f7c729bd980 20 INFO: main: starting shard=3 2024-05-30T15:58:38.818 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.818+0000 7f7c729bd980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.3 2024-05-30T15:58:38.819 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.818+0000 7f7c729bd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:38.819 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.818+0000 7f7c729bd980 20 INFO: main: starting shard=4 2024-05-30T15:58:38.819 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.818+0000 7f7c729bd980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.4 2024-05-30T15:58:38.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.818+0000 7f7c729bd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:38.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.818+0000 7f7c729bd980 20 INFO: main: starting shard=5 2024-05-30T15:58:38.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.818+0000 7f7c729bd980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:38.821 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.822+0000 7f7c729bd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:38.821 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.822+0000 7f7c729bd980 20 INFO: main: starting shard=6 2024-05-30T15:58:38.821 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.822+0000 7f7c729bd980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.6 2024-05-30T15:58:38.822 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.822+0000 7f7c729bd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:38.822 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.822+0000 7f7c729bd980 20 INFO: main: starting shard=7 2024-05-30T15:58:38.822 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.822+0000 7f7c729bd980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.7 2024-05-30T15:58:38.823 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.822+0000 7f7c729bd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:38.823 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.822+0000 7f7c729bd980 20 INFO: main: starting shard=8 2024-05-30T15:58:38.823 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.822+0000 7f7c729bd980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:38.824 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.822+0000 7f7c729bd980 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, marker= 2024-05-30T15:58:38.824 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.822+0000 7f7c729bd980 20 INFO: main: starting shard=9 2024-05-30T15:58:38.824 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.822+0000 7f7c729bd980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.9 2024-05-30T15:58:38.825 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.822+0000 7f7c729bd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:38.825 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.822+0000 7f7c729bd980 20 INFO: main: starting shard=10 2024-05-30T15:58:38.825 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.822+0000 7f7c729bd980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:38.826 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.826+0000 7f7c729bd980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:38.826 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.826+0000 7f7c729bd980 20 INFO: main: done 2024-05-30T15:58:38.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.826+0000 7f7b26fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:38.833 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.830+0000 7f7c729bd980 20 remove_watcher() i=0 2024-05-30T15:58:38.833 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.830+0000 7f7c729bd980 2 removed watcher, disabling cache 2024-05-30T15:58:38.835 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.834+0000 7f7c729bd980 20 remove_watcher() i=1 2024-05-30T15:58:38.837 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.838+0000 7f7c729bd980 20 remove_watcher() i=2 2024-05-30T15:58:38.840 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.838+0000 7f7c729bd980 20 remove_watcher() i=3 2024-05-30T15:58:38.842 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.842+0000 7f7c729bd980 20 remove_watcher() i=4 2024-05-30T15:58:38.844 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.842+0000 7f7c729bd980 20 remove_watcher() i=5 2024-05-30T15:58:38.845 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.846+0000 7f7c729bd980 20 remove_watcher() i=6 2024-05-30T15:58:38.847 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:38.846+0000 7f7c729bd980 20 remove_watcher() i=7 2024-05-30T15:58:39.010 INFO:tasks.workunit.client.0.smithi006.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-05-30T15:58:39.011 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bi list --bucket check-bucket --object d 2024-05-30T15:58:39.048 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:58:39.048 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:58:39.093 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.094+0000 7f12a9838980 20 rados->read ofs=0 len=0 2024-05-30T15:58:39.095 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.094+0000 7f12a9838980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:39.096 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.094+0000 7f12a9838980 20 realm 2024-05-30T15:58:39.096 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.094+0000 7f12a9838980 20 rados->read ofs=0 len=0 2024-05-30T15:58:39.096 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.094+0000 7f12a9838980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:39.096 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.094+0000 7f12a9838980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T15:58:39.096 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.094+0000 7f12a9838980 20 rados->read ofs=0 len=0 2024-05-30T15:58:39.096 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.094+0000 7f12a9838980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:39.096 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.094+0000 7f12a9838980 20 rados->read ofs=0 len=0 2024-05-30T15:58:39.098 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.098+0000 7f12a9838980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:39.098 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.098+0000 7f12a9838980 20 rados->read ofs=0 len=0 2024-05-30T15:58:39.098 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.098+0000 7f12a9838980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T15:58:39.098 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.098+0000 7f12a9838980 20 searching for the correct realm 2024-05-30T15:58:39.103 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.102+0000 7f12a6b48640 0 -- 172.21.15.6:0/1132008863 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f129805ca00 msgr2=0x7f12980609b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:39.110 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.110+0000 7f12a7349640 0 -- 172.21.15.6:0/1132008863 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f129c117ba0 msgr2=0x7f129c0f9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:39.329 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.326+0000 7f12a9838980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T15:58:39.329 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.326+0000 7f12a9838980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T15:58:39.329 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.326+0000 7f12a9838980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T15:58:39.329 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.326+0000 7f12a9838980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T15:58:39.329 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.326+0000 7f12a9838980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T15:58:39.329 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.326+0000 7f12a9838980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T15:58:39.329 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.326+0000 7f12a9838980 20 rados->read ofs=0 len=0 2024-05-30T15:58:39.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.326+0000 7f12a9838980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:39.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.326+0000 7f12a9838980 20 rados->read ofs=0 len=0 2024-05-30T15:58:39.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.330+0000 7f12a9838980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:39.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.330+0000 7f12a9838980 20 rados->read ofs=0 len=0 2024-05-30T15:58:39.331 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.330+0000 7f12a9838980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T15:58:39.331 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.330+0000 7f12a9838980 20 zone default found 2024-05-30T15:58:39.331 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.330+0000 7f12a9838980 4 Realm: () 2024-05-30T15:58:39.331 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.330+0000 7f12a9838980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T15:58:39.332 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.330+0000 7f12a9838980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T15:58:39.332 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.330+0000 7f12a9838980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T15:58:39.332 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.330+0000 7f12a9838980 20 zonegroup default 2024-05-30T15:58:39.332 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.330+0000 7f12a9838980 20 rados->read ofs=0 len=0 2024-05-30T15:58:39.332 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.330+0000 7f12a9838980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:39.332 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.330+0000 7f12a9838980 20 rados->read ofs=0 len=0 2024-05-30T15:58:39.333 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.330+0000 7f12a9838980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:39.333 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.330+0000 7f12a9838980 20 rados->read ofs=0 len=0 2024-05-30T15:58:39.333 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.334+0000 7f12a9838980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:39.334 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.334+0000 7f12a9838980 20 started sync module instance, tier type = 2024-05-30T15:58:39.334 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.334+0000 7f12a9838980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T15:58:39.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.358+0000 7f12a9838980 20 add_watcher() i=0 2024-05-30T15:58:39.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.358+0000 7f12a9838980 20 add_watcher() i=1 2024-05-30T15:58:39.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.358+0000 7f12a9838980 20 add_watcher() i=2 2024-05-30T15:58:39.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.358+0000 7f12a9838980 20 add_watcher() i=3 2024-05-30T15:58:39.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.358+0000 7f12a9838980 20 add_watcher() i=4 2024-05-30T15:58:39.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.358+0000 7f12a9838980 20 add_watcher() i=5 2024-05-30T15:58:39.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.358+0000 7f12a9838980 20 add_watcher() i=6 2024-05-30T15:58:39.362 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.362+0000 7f12a9838980 20 add_watcher() i=7 2024-05-30T15:58:39.362 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.362+0000 7f12a9838980 2 all 8 watchers are set, enabling cache 2024-05-30T15:58:39.366 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.366+0000 7f115dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:39.367 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.366+0000 7f12a9838980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:58:39.367 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.366+0000 7f12a9838980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:58:39.368 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.366+0000 7f12a9838980 5 note: GC not initialized 2024-05-30T15:58:39.368 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.366+0000 7f115cfe1640 20 reqs_thread_entry: start 2024-05-30T15:58:39.368 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.366+0000 7f12a9838980 20 init_complete bucket index max shards: 11 2024-05-30T15:58:39.368 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.366+0000 7f12a9838980 20 Filter name: none 2024-05-30T15:58:39.369 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.366+0000 7f10d67fc640 20 reqs_thread_entry: start 2024-05-30T15:58:39.369 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.366+0000 7f12a9838980 20 rados->read ofs=0 len=0 2024-05-30T15:58:39.369 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.370+0000 7f12a9838980 20 rados_obj.operate() r=0 bl.length=190 2024-05-30T15:58:39.370 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.370+0000 7f12a9838980 20 rados->read ofs=0 len=0 2024-05-30T15:58:39.370 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.370+0000 7f12a9838980 20 rados_obj.operate() r=0 bl.length=404 2024-05-30T15:58:39.370 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.370+0000 7f12a9838980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T15:58:39.371 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.370+0000 7f12a9838980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-05-30T15:58:39.371 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.370+0000 7f12a9838980 20 INFO: main: starting shard=0 2024-05-30T15:58:39.371 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.370+0000 7f12a9838980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.0 2024-05-30T15:58:39.371 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.370+0000 7f12a9838980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:39.372 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.370+0000 7f12a9838980 20 INFO: main: starting shard=1 2024-05-30T15:58:39.372 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.370+0000 7f12a9838980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:39.372 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.370+0000 7f12a9838980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:39.372 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.370+0000 7f12a9838980 20 INFO: main: starting shard=2 2024-05-30T15:58:39.372 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.370+0000 7f12a9838980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.2 2024-05-30T15:58:39.373 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.370+0000 7f12a9838980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:39.373 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.370+0000 7f12a9838980 20 INFO: main: starting shard=3 2024-05-30T15:58:39.373 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.370+0000 7f12a9838980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.3 2024-05-30T15:58:39.374 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.374+0000 7f12a9838980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:39.374 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.374+0000 7f12a9838980 20 INFO: main: starting shard=4 2024-05-30T15:58:39.374 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.374+0000 7f12a9838980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.4 2024-05-30T15:58:39.375 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.374+0000 7f12a9838980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:39.375 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.374+0000 7f12a9838980 20 INFO: main: starting shard=5 2024-05-30T15:58:39.375 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.374+0000 7f12a9838980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:39.377 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.374+0000 7f12a9838980 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, marker= 2024-05-30T15:58:39.377 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.374+0000 7f12a9838980 20 INFO: main: starting shard=6 2024-05-30T15:58:39.377 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.374+0000 7f12a9838980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.6 2024-05-30T15:58:39.378 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.378+0000 7f12a9838980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:39.378 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.378+0000 7f12a9838980 20 INFO: main: starting shard=7 2024-05-30T15:58:39.378 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.378+0000 7f12a9838980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.7 2024-05-30T15:58:39.379 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.378+0000 7f12a9838980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:39.379 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.378+0000 7f12a9838980 20 INFO: main: starting shard=8 2024-05-30T15:58:39.379 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.378+0000 7f12a9838980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:39.380 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.378+0000 7f12a9838980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:39.380 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.378+0000 7f12a9838980 20 INFO: main: starting shard=9 2024-05-30T15:58:39.380 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.378+0000 7f12a9838980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.9 2024-05-30T15:58:39.380 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.378+0000 7f12a9838980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:39.381 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.378+0000 7f12a9838980 20 INFO: main: starting shard=10 2024-05-30T15:58:39.381 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.378+0000 7f12a9838980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:39.382 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.382+0000 7f12a9838980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:39.382 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.382+0000 7f12a9838980 20 INFO: main: done 2024-05-30T15:58:39.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.382+0000 7f115dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:39.389 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.390+0000 7f12a9838980 20 remove_watcher() i=0 2024-05-30T15:58:39.389 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.390+0000 7f12a9838980 2 removed watcher, disabling cache 2024-05-30T15:58:39.392 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.390+0000 7f12a9838980 20 remove_watcher() i=1 2024-05-30T15:58:39.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.394+0000 7f12a9838980 20 remove_watcher() i=2 2024-05-30T15:58:39.397 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.394+0000 7f12a9838980 20 remove_watcher() i=3 2024-05-30T15:58:39.400 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.398+0000 7f12a9838980 20 remove_watcher() i=4 2024-05-30T15:58:39.402 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.402+0000 7f12a9838980 20 remove_watcher() i=5 2024-05-30T15:58:39.405 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.402+0000 7f12a9838980 20 remove_watcher() i=6 2024-05-30T15:58:39.406 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.406+0000 7f12a9838980 20 remove_watcher() i=7 2024-05-30T15:58:39.630 INFO:tasks.workunit.client.0.smithi006.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-05-30T15:58:39.631 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bi list --bucket check-bucket --object e 2024-05-30T15:58:39.668 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:58:39.669 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:58:39.714 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.714+0000 7fb53c042980 20 rados->read ofs=0 len=0 2024-05-30T15:58:39.717 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.714+0000 7fb53c042980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:39.717 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.714+0000 7fb53c042980 20 realm 2024-05-30T15:58:39.717 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.714+0000 7fb53c042980 20 rados->read ofs=0 len=0 2024-05-30T15:58:39.717 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.714+0000 7fb53c042980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:39.717 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.714+0000 7fb53c042980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T15:58:39.717 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.714+0000 7fb53c042980 20 rados->read ofs=0 len=0 2024-05-30T15:58:39.717 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.714+0000 7fb53c042980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:39.717 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.714+0000 7fb53c042980 20 rados->read ofs=0 len=0 2024-05-30T15:58:39.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.718+0000 7fb53c042980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:39.719 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.718+0000 7fb53c042980 20 rados->read ofs=0 len=0 2024-05-30T15:58:39.719 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.718+0000 7fb53c042980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T15:58:39.719 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.718+0000 7fb53c042980 20 searching for the correct realm 2024-05-30T15:58:39.726 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.726+0000 7fb53a354640 0 -- 172.21.15.6:0/773225204 >> [v2:172.21.15.6:6808/1897362715,v1:172.21.15.6:6809/1897362715] conn(0x55ce05a2ef50 msgr2=0x55ce05a31340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:39.732 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.730+0000 7fb539352640 0 -- 172.21.15.6:0/773225204 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7fb52c02c070 msgr2=0x7fb52c03a940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:39.744 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.738+0000 7fb53c042980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T15:58:39.744 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.738+0000 7fb53c042980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T15:58:39.744 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.738+0000 7fb53c042980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T15:58:39.744 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.738+0000 7fb53c042980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T15:58:39.744 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.738+0000 7fb53c042980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T15:58:39.744 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.738+0000 7fb53c042980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T15:58:39.744 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.738+0000 7fb53c042980 20 rados->read ofs=0 len=0 2024-05-30T15:58:39.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.738+0000 7fb53c042980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:39.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.738+0000 7fb53c042980 20 rados->read ofs=0 len=0 2024-05-30T15:58:39.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.742+0000 7fb53c042980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:39.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.742+0000 7fb53c042980 20 rados->read ofs=0 len=0 2024-05-30T15:58:39.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.742+0000 7fb53c042980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T15:58:39.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.742+0000 7fb53c042980 20 zone default found 2024-05-30T15:58:39.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.742+0000 7fb53c042980 4 Realm: () 2024-05-30T15:58:39.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.742+0000 7fb53c042980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T15:58:39.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.742+0000 7fb53c042980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T15:58:39.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.742+0000 7fb53c042980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T15:58:39.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.742+0000 7fb53c042980 20 zonegroup default 2024-05-30T15:58:39.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.742+0000 7fb53c042980 20 rados->read ofs=0 len=0 2024-05-30T15:58:39.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.742+0000 7fb53c042980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:39.746 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.742+0000 7fb53c042980 20 rados->read ofs=0 len=0 2024-05-30T15:58:39.746 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.742+0000 7fb53c042980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:39.746 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.742+0000 7fb53c042980 20 rados->read ofs=0 len=0 2024-05-30T15:58:39.746 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.742+0000 7fb53c042980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:39.746 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.742+0000 7fb53c042980 20 started sync module instance, tier type = 2024-05-30T15:58:39.746 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.742+0000 7fb53c042980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T15:58:39.764 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.762+0000 7fb53c042980 20 add_watcher() i=0 2024-05-30T15:58:39.764 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.762+0000 7fb53c042980 20 add_watcher() i=1 2024-05-30T15:58:39.764 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.762+0000 7fb53c042980 20 add_watcher() i=2 2024-05-30T15:58:39.764 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.762+0000 7fb53c042980 20 add_watcher() i=3 2024-05-30T15:58:39.765 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.762+0000 7fb53c042980 20 add_watcher() i=4 2024-05-30T15:58:39.765 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.762+0000 7fb53c042980 20 add_watcher() i=5 2024-05-30T15:58:39.765 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.762+0000 7fb53c042980 20 add_watcher() i=6 2024-05-30T15:58:39.766 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.766+0000 7fb53c042980 20 add_watcher() i=7 2024-05-30T15:58:39.766 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.766+0000 7fb53c042980 2 all 8 watchers are set, enabling cache 2024-05-30T15:58:39.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.770+0000 7fb3eb7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:39.772 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.770+0000 7fb53c042980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:58:39.772 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.770+0000 7fb53c042980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:58:39.772 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.770+0000 7fb53c042980 5 note: GC not initialized 2024-05-30T15:58:39.773 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.770+0000 7fb3ea7fc640 20 reqs_thread_entry: start 2024-05-30T15:58:39.773 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.770+0000 7fb53c042980 20 init_complete bucket index max shards: 11 2024-05-30T15:58:39.773 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.770+0000 7fb53c042980 20 Filter name: none 2024-05-30T15:58:39.773 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.770+0000 7fb35bfff640 20 reqs_thread_entry: start 2024-05-30T15:58:39.773 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.770+0000 7fb53c042980 20 rados->read ofs=0 len=0 2024-05-30T15:58:39.773 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.770+0000 7fb53c042980 20 rados_obj.operate() r=0 bl.length=190 2024-05-30T15:58:39.773 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.770+0000 7fb53c042980 20 rados->read ofs=0 len=0 2024-05-30T15:58:39.773 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.770+0000 7fb53c042980 20 rados_obj.operate() r=0 bl.length=404 2024-05-30T15:58:39.773 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.770+0000 7fb53c042980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T15:58:39.773 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.770+0000 7fb53c042980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-05-30T15:58:39.774 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.770+0000 7fb53c042980 20 INFO: main: starting shard=0 2024-05-30T15:58:39.774 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.770+0000 7fb53c042980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.0 2024-05-30T15:58:39.774 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.770+0000 7fb53c042980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:39.774 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.770+0000 7fb53c042980 20 INFO: main: starting shard=1 2024-05-30T15:58:39.774 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.770+0000 7fb53c042980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:39.775 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.770+0000 7fb53c042980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-05-30T15:58:39.775 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.774+0000 7fb53c042980 20 INFO: main: starting shard=2 2024-05-30T15:58:39.776 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.774+0000 7fb53c042980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.2 2024-05-30T15:58:39.776 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.774+0000 7fb53c042980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:39.776 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.774+0000 7fb53c042980 20 INFO: main: starting shard=3 2024-05-30T15:58:39.776 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.774+0000 7fb53c042980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.3 2024-05-30T15:58:39.776 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.774+0000 7fb53c042980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:39.776 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.774+0000 7fb53c042980 20 INFO: main: starting shard=4 2024-05-30T15:58:39.776 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.774+0000 7fb53c042980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.4 2024-05-30T15:58:39.777 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.774+0000 7fb53c042980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:39.777 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.774+0000 7fb53c042980 20 INFO: main: starting shard=5 2024-05-30T15:58:39.777 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.774+0000 7fb53c042980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:39.777 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.778+0000 7fb53c042980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:39.777 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.778+0000 7fb53c042980 20 INFO: main: starting shard=6 2024-05-30T15:58:39.777 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.778+0000 7fb53c042980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.6 2024-05-30T15:58:39.778 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.778+0000 7fb53c042980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:39.778 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.778+0000 7fb53c042980 20 INFO: main: starting shard=7 2024-05-30T15:58:39.778 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.778+0000 7fb53c042980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.7 2024-05-30T15:58:39.779 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.778+0000 7fb53c042980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:39.779 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.778+0000 7fb53c042980 20 INFO: main: starting shard=8 2024-05-30T15:58:39.779 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.778+0000 7fb53c042980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:39.779 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.778+0000 7fb53c042980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:39.780 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.778+0000 7fb53c042980 20 INFO: main: starting shard=9 2024-05-30T15:58:39.780 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.778+0000 7fb53c042980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.9 2024-05-30T15:58:39.780 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.778+0000 7fb53c042980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:39.780 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.778+0000 7fb53c042980 20 INFO: main: starting shard=10 2024-05-30T15:58:39.780 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.778+0000 7fb53c042980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:39.781 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.782+0000 7fb53c042980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:39.781 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.782+0000 7fb53c042980 20 INFO: main: done 2024-05-30T15:58:39.782 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.782+0000 7fb3eb7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:39.788 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.786+0000 7fb53c042980 20 remove_watcher() i=0 2024-05-30T15:58:39.788 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.786+0000 7fb53c042980 2 removed watcher, disabling cache 2024-05-30T15:58:39.790 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.790+0000 7fb53c042980 20 remove_watcher() i=1 2024-05-30T15:58:39.792 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.790+0000 7fb53c042980 20 remove_watcher() i=2 2024-05-30T15:58:39.795 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.794+0000 7fb53c042980 20 remove_watcher() i=3 2024-05-30T15:58:39.798 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.798+0000 7fb53c042980 20 remove_watcher() i=4 2024-05-30T15:58:39.800 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.798+0000 7fb53c042980 20 remove_watcher() i=5 2024-05-30T15:58:39.801 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.802+0000 7fb53c042980 20 remove_watcher() i=6 2024-05-30T15:58:39.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:39.802+0000 7fb53c042980 20 remove_watcher() i=7 2024-05-30T15:58:39.962 INFO:tasks.workunit.client.0.smithi006.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-05-30T15:58:39.963 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bi list --bucket check-bucket --object f 2024-05-30T15:58:40.001 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:58:40.001 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:58:40.048 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.046+0000 7f12bb41c980 20 rados->read ofs=0 len=0 2024-05-30T15:58:40.050 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.050+0000 7f12bb41c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:40.050 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.050+0000 7f12bb41c980 20 realm 2024-05-30T15:58:40.050 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.050+0000 7f12bb41c980 20 rados->read ofs=0 len=0 2024-05-30T15:58:40.051 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.050+0000 7f12bb41c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:40.051 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.050+0000 7f12bb41c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T15:58:40.051 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.050+0000 7f12bb41c980 20 rados->read ofs=0 len=0 2024-05-30T15:58:40.051 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.050+0000 7f12bb41c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:40.051 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.050+0000 7f12bb41c980 20 rados->read ofs=0 len=0 2024-05-30T15:58:40.052 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.050+0000 7f12bb41c980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:40.052 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.050+0000 7f12bb41c980 20 rados->read ofs=0 len=0 2024-05-30T15:58:40.053 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.054+0000 7f12bb41c980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T15:58:40.053 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.054+0000 7f12bb41c980 20 searching for the correct realm 2024-05-30T15:58:40.059 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.058+0000 7f12b872c640 0 -- 172.21.15.6:0/3173450697 >> [v2:172.21.15.6:6819/1141406589,v1:172.21.15.6:6821/1141406589] conn(0x56526d59f970 msgr2=0x56526d5a1d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:40.065 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.066+0000 7f12b8f2d640 0 -- 172.21.15.6:0/3173450697 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f12a004c920 msgr2=0x7f12a003b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:40.073 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.070+0000 7f12bb41c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T15:58:40.073 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.070+0000 7f12bb41c980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T15:58:40.073 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.070+0000 7f12bb41c980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T15:58:40.073 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.070+0000 7f12bb41c980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T15:58:40.073 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.070+0000 7f12bb41c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T15:58:40.073 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.070+0000 7f12bb41c980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T15:58:40.073 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.070+0000 7f12bb41c980 20 rados->read ofs=0 len=0 2024-05-30T15:58:40.074 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.074+0000 7f12bb41c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:40.074 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.074+0000 7f12bb41c980 20 rados->read ofs=0 len=0 2024-05-30T15:58:40.074 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.074+0000 7f12bb41c980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:40.074 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.074+0000 7f12bb41c980 20 rados->read ofs=0 len=0 2024-05-30T15:58:40.075 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.074+0000 7f12bb41c980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T15:58:40.075 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.074+0000 7f12bb41c980 20 zone default found 2024-05-30T15:58:40.075 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.074+0000 7f12bb41c980 4 Realm: () 2024-05-30T15:58:40.075 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.074+0000 7f12bb41c980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T15:58:40.075 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.074+0000 7f12bb41c980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T15:58:40.075 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.074+0000 7f12bb41c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T15:58:40.075 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.074+0000 7f12bb41c980 20 zonegroup default 2024-05-30T15:58:40.075 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.074+0000 7f12bb41c980 20 rados->read ofs=0 len=0 2024-05-30T15:58:40.076 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.074+0000 7f12bb41c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:40.076 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.074+0000 7f12bb41c980 20 rados->read ofs=0 len=0 2024-05-30T15:58:40.076 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.074+0000 7f12bb41c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:40.076 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.074+0000 7f12bb41c980 20 rados->read ofs=0 len=0 2024-05-30T15:58:40.076 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.074+0000 7f12bb41c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:40.076 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.074+0000 7f12bb41c980 20 started sync module instance, tier type = 2024-05-30T15:58:40.076 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.074+0000 7f12bb41c980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T15:58:40.096 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.094+0000 7f12bb41c980 20 add_watcher() i=0 2024-05-30T15:58:40.096 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.094+0000 7f12bb41c980 20 add_watcher() i=1 2024-05-30T15:58:40.096 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.094+0000 7f12bb41c980 20 add_watcher() i=2 2024-05-30T15:58:40.096 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.094+0000 7f12bb41c980 20 add_watcher() i=3 2024-05-30T15:58:40.096 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.094+0000 7f12bb41c980 20 add_watcher() i=4 2024-05-30T15:58:40.096 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.094+0000 7f12bb41c980 20 add_watcher() i=5 2024-05-30T15:58:40.096 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.094+0000 7f12bb41c980 20 add_watcher() i=6 2024-05-30T15:58:40.098 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.098+0000 7f12bb41c980 20 add_watcher() i=7 2024-05-30T15:58:40.098 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.098+0000 7f12bb41c980 2 all 8 watchers are set, enabling cache 2024-05-30T15:58:40.101 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.102+0000 7f116efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:40.102 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.102+0000 7f12bb41c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:58:40.102 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.102+0000 7f12bb41c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:58:40.102 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.102+0000 7f12bb41c980 5 note: GC not initialized 2024-05-30T15:58:40.102 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.102+0000 7f116dfe3640 20 reqs_thread_entry: start 2024-05-30T15:58:40.102 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.102+0000 7f12bb41c980 20 init_complete bucket index max shards: 11 2024-05-30T15:58:40.103 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.102+0000 7f12bb41c980 20 Filter name: none 2024-05-30T15:58:40.103 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.102+0000 7f10df7fe640 20 reqs_thread_entry: start 2024-05-30T15:58:40.103 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.102+0000 7f12bb41c980 20 rados->read ofs=0 len=0 2024-05-30T15:58:40.103 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.102+0000 7f12bb41c980 20 rados_obj.operate() r=0 bl.length=190 2024-05-30T15:58:40.104 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.102+0000 7f12bb41c980 20 rados->read ofs=0 len=0 2024-05-30T15:58:40.104 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.102+0000 7f12bb41c980 20 rados_obj.operate() r=0 bl.length=404 2024-05-30T15:58:40.104 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.102+0000 7f12bb41c980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T15:58:40.104 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.102+0000 7f12bb41c980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-05-30T15:58:40.104 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.102+0000 7f12bb41c980 20 INFO: main: starting shard=0 2024-05-30T15:58:40.104 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.102+0000 7f12bb41c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.0 2024-05-30T15:58:40.104 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.102+0000 7f12bb41c980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:40.104 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.102+0000 7f12bb41c980 20 INFO: main: starting shard=1 2024-05-30T15:58:40.104 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.102+0000 7f12bb41c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:40.105 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.102+0000 7f12bb41c980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:40.105 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.102+0000 7f12bb41c980 20 INFO: main: starting shard=2 2024-05-30T15:58:40.105 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.102+0000 7f12bb41c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.2 2024-05-30T15:58:40.106 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.106+0000 7f12bb41c980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:40.106 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.106+0000 7f12bb41c980 20 INFO: main: starting shard=3 2024-05-30T15:58:40.106 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.106+0000 7f12bb41c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.3 2024-05-30T15:58:40.106 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.106+0000 7f12bb41c980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:40.106 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.106+0000 7f12bb41c980 20 INFO: main: starting shard=4 2024-05-30T15:58:40.107 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.106+0000 7f12bb41c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.4 2024-05-30T15:58:40.107 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.106+0000 7f12bb41c980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:40.108 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.106+0000 7f12bb41c980 20 INFO: main: starting shard=5 2024-05-30T15:58:40.108 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.106+0000 7f12bb41c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:40.109 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.106+0000 7f12bb41c980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:40.109 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.106+0000 7f12bb41c980 20 INFO: main: starting shard=6 2024-05-30T15:58:40.109 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.106+0000 7f12bb41c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.6 2024-05-30T15:58:40.109 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.110+0000 7f12bb41c980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:40.109 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.110+0000 7f12bb41c980 20 INFO: main: starting shard=7 2024-05-30T15:58:40.109 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.110+0000 7f12bb41c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.7 2024-05-30T15:58:40.110 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.110+0000 7f12bb41c980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:40.110 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.110+0000 7f12bb41c980 20 INFO: main: starting shard=8 2024-05-30T15:58:40.110 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.110+0000 7f12bb41c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:40.111 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.110+0000 7f12bb41c980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:40.111 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.110+0000 7f12bb41c980 20 INFO: main: starting shard=9 2024-05-30T15:58:40.111 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.110+0000 7f12bb41c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.9 2024-05-30T15:58:40.111 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.110+0000 7f12bb41c980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T15:58:40.111 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.110+0000 7f12bb41c980 20 INFO: main: starting shard=10 2024-05-30T15:58:40.112 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.110+0000 7f12bb41c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:40.113 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.110+0000 7f12bb41c980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-05-30T15:58:40.113 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.110+0000 7f12bb41c980 20 INFO: main: done 2024-05-30T15:58:40.114 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.114+0000 7f116efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:40.119 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.118+0000 7f12bb41c980 20 remove_watcher() i=0 2024-05-30T15:58:40.119 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.118+0000 7f12bb41c980 2 removed watcher, disabling cache 2024-05-30T15:58:40.121 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.122+0000 7f12bb41c980 20 remove_watcher() i=1 2024-05-30T15:58:40.124 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.122+0000 7f12bb41c980 20 remove_watcher() i=2 2024-05-30T15:58:40.126 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.126+0000 7f12bb41c980 20 remove_watcher() i=3 2024-05-30T15:58:40.128 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.126+0000 7f12bb41c980 20 remove_watcher() i=4 2024-05-30T15:58:40.130 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.130+0000 7f12bb41c980 20 remove_watcher() i=5 2024-05-30T15:58:40.131 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.130+0000 7f12bb41c980 20 remove_watcher() i=6 2024-05-30T15:58:40.133 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:40.130+0000 7f12bb41c980 20 remove_watcher() i=7 2024-05-30T15:58:40.177 INFO:tasks.workunit.client.0.smithi006.stderr:ceph config rm client rgw_debug_inject_set_olh_err 2024-05-30T15:58:40.624 INFO:tasks.workunit.client.0.smithi006.stderr:ceph config rm client rgw_debug_inject_olh_cancel_modification_err 2024-05-30T15:58:41.060 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket check --fix --bucket check-bucket 2024-05-30T15:58:41.091 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:58:41.091 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:58:41.131 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.130+0000 7f4f61d9f980 20 rados->read ofs=0 len=0 2024-05-30T15:58:41.133 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.130+0000 7f4f61d9f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:41.133 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.130+0000 7f4f61d9f980 20 realm 2024-05-30T15:58:41.133 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.130+0000 7f4f61d9f980 20 rados->read ofs=0 len=0 2024-05-30T15:58:41.133 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.130+0000 7f4f61d9f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:41.133 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.130+0000 7f4f61d9f980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T15:58:41.134 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.130+0000 7f4f61d9f980 20 rados->read ofs=0 len=0 2024-05-30T15:58:41.134 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.130+0000 7f4f61d9f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:41.134 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.130+0000 7f4f61d9f980 20 rados->read ofs=0 len=0 2024-05-30T15:58:41.136 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.134+0000 7f4f61d9f980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:41.136 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.134+0000 7f4f61d9f980 20 rados->read ofs=0 len=0 2024-05-30T15:58:41.136 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.134+0000 7f4f61d9f980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T15:58:41.136 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.134+0000 7f4f61d9f980 20 searching for the correct realm 2024-05-30T15:58:41.144 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.142+0000 7f4f600b1640 0 -- 172.21.15.6:0/2234174055 >> [v2:172.21.15.204:6800/3687190393,v1:172.21.15.204:6801/3687190393] conn(0x7f4f5406a300 msgr2=0x7f4f540938e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:41.359 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.358+0000 7f4f5f8b0640 0 -- 172.21.15.6:0/2234174055 >> [v2:172.21.15.204:6816/2677861390,v1:172.21.15.204:6817/2677861390] conn(0x55f56461bc30 msgr2=0x55f56461e080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:41.366 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.366+0000 7f4f61d9f980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T15:58:41.366 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.366+0000 7f4f61d9f980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T15:58:41.366 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.366+0000 7f4f61d9f980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T15:58:41.366 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.366+0000 7f4f61d9f980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T15:58:41.367 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.366+0000 7f4f61d9f980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T15:58:41.367 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.366+0000 7f4f61d9f980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T15:58:41.367 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.366+0000 7f4f61d9f980 20 rados->read ofs=0 len=0 2024-05-30T15:58:41.368 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.366+0000 7f4f61d9f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:41.368 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.366+0000 7f4f61d9f980 20 rados->read ofs=0 len=0 2024-05-30T15:58:41.368 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.366+0000 7f4f61d9f980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:41.368 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.366+0000 7f4f61d9f980 20 rados->read ofs=0 len=0 2024-05-30T15:58:41.368 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.366+0000 7f4f61d9f980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T15:58:41.369 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.366+0000 7f4f61d9f980 20 zone default found 2024-05-30T15:58:41.369 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.366+0000 7f4f61d9f980 4 Realm: () 2024-05-30T15:58:41.369 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.366+0000 7f4f61d9f980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T15:58:41.369 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.366+0000 7f4f61d9f980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T15:58:41.369 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.366+0000 7f4f61d9f980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T15:58:41.369 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.366+0000 7f4f61d9f980 20 zonegroup default 2024-05-30T15:58:41.369 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.366+0000 7f4f61d9f980 20 rados->read ofs=0 len=0 2024-05-30T15:58:41.369 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.366+0000 7f4f61d9f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:41.369 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.366+0000 7f4f61d9f980 20 rados->read ofs=0 len=0 2024-05-30T15:58:41.370 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.370+0000 7f4f61d9f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:41.370 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.370+0000 7f4f61d9f980 20 rados->read ofs=0 len=0 2024-05-30T15:58:41.370 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.370+0000 7f4f61d9f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:41.371 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.370+0000 7f4f61d9f980 20 started sync module instance, tier type = 2024-05-30T15:58:41.371 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.370+0000 7f4f61d9f980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T15:58:41.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.394+0000 7f4f61d9f980 20 add_watcher() i=0 2024-05-30T15:58:41.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.394+0000 7f4f61d9f980 20 add_watcher() i=1 2024-05-30T15:58:41.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.394+0000 7f4f61d9f980 20 add_watcher() i=2 2024-05-30T15:58:41.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.394+0000 7f4f61d9f980 20 add_watcher() i=3 2024-05-30T15:58:41.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.394+0000 7f4f61d9f980 20 add_watcher() i=4 2024-05-30T15:58:41.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.394+0000 7f4f61d9f980 20 add_watcher() i=5 2024-05-30T15:58:41.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.394+0000 7f4f61d9f980 20 add_watcher() i=6 2024-05-30T15:58:41.397 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.394+0000 7f4f61d9f980 20 add_watcher() i=7 2024-05-30T15:58:41.397 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.394+0000 7f4f61d9f980 2 all 8 watchers are set, enabling cache 2024-05-30T15:58:41.401 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.398+0000 7f4e167e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:41.402 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.402+0000 7f4f61d9f980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:58:41.402 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.402+0000 7f4f61d9f980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:58:41.402 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.402+0000 7f4f61d9f980 5 note: GC not initialized 2024-05-30T15:58:41.403 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.402+0000 7f4e157e2640 20 reqs_thread_entry: start 2024-05-30T15:58:41.403 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.402+0000 7f4f61d9f980 20 init_complete bucket index max shards: 11 2024-05-30T15:58:41.403 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.402+0000 7f4f61d9f980 20 Filter name: none 2024-05-30T15:58:41.404 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.402+0000 7f4e0e7fc640 20 reqs_thread_entry: start 2024-05-30T15:58:41.404 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.402+0000 7f4f61d9f980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-05-30T15:58:41.404 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.402+0000 7f4f61d9f980 20 rados->read ofs=0 len=0 2024-05-30T15:58:41.404 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.402+0000 7f4f61d9f980 20 rados_obj.operate() r=0 bl.length=190 2024-05-30T15:58:41.404 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.402+0000 7f4f61d9f980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-05-30T15:58:41.404 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.402+0000 7f4f61d9f980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-05-30T15:58:41.404 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.402+0000 7f4f61d9f980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 : miss 2024-05-30T15:58:41.405 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.402+0000 7f4f61d9f980 20 rados->read ofs=0 len=0 2024-05-30T15:58:41.405 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.402+0000 7f4f61d9f980 20 rados_obj.operate() r=0 bl.length=404 2024-05-30T15:58:41.406 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.402+0000 7f4f61d9f980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 info.flags=0x17 2024-05-30T15:58:41.406 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.402+0000 7f4f61d9f980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 to cache LRU end 2024-05-30T15:58:41.406 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.402+0000 7f4f61d9f980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T15:58:41.406 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.402+0000 7f4f61d9f980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-05-30T15:58:41.406 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.402+0000 7f4f61d9f980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 2024-05-30T15:58:41.406 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.402+0000 7f4f61d9f980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.0 2024-05-30T15:58:41.406 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.406+0000 7f4f61d9f980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:41.406 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.406+0000 7f4f61d9f980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.2 2024-05-30T15:58:41.408 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.406+0000 7f4f61d9f980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.3 2024-05-30T15:58:41.409 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.406+0000 7f4f61d9f980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.4 2024-05-30T15:58:41.410 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.410+0000 7f4f61d9f980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:41.412 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.410+0000 7f4f61d9f980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.6 2024-05-30T15:58:41.413 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.410+0000 7f4f61d9f980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.7 2024-05-30T15:58:41.415 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.414+0000 7f4f61d9f980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:41.415 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.414+0000 7f4f61d9f980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.9 2024-05-30T15:58:41.416 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.414+0000 7f4f61d9f980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:41.427 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.426+0000 7f4e167e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:41.434 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.434+0000 7f4f61d9f980 20 remove_watcher() i=0 2024-05-30T15:58:41.434 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.434+0000 7f4f61d9f980 2 removed watcher, disabling cache 2024-05-30T15:58:41.436 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.434+0000 7f4f61d9f980 20 remove_watcher() i=1 2024-05-30T15:58:41.439 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.438+0000 7f4f61d9f980 20 remove_watcher() i=2 2024-05-30T15:58:41.441 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.442+0000 7f4f61d9f980 20 remove_watcher() i=3 2024-05-30T15:58:41.443 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.442+0000 7f4f61d9f980 20 remove_watcher() i=4 2024-05-30T15:58:41.445 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.446+0000 7f4f61d9f980 20 remove_watcher() i=5 2024-05-30T15:58:41.446 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.446+0000 7f4f61d9f980 20 remove_watcher() i=6 2024-05-30T15:58:41.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.446+0000 7f4f61d9f980 20 remove_watcher() i=7 2024-05-30T15:58:41.465 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --fix --min-age-hours 0 --rgw-olh-pending-timeout-sec 0 --dump-keys 2024-05-30T15:58:41.504 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:58:41.504 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:58:41.547 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.546+0000 7fbcca841980 20 rados->read ofs=0 len=0 2024-05-30T15:58:41.549 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.546+0000 7fbcca841980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:41.549 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.546+0000 7fbcca841980 20 realm 2024-05-30T15:58:41.549 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.546+0000 7fbcca841980 20 rados->read ofs=0 len=0 2024-05-30T15:58:41.549 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.546+0000 7fbcca841980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:41.549 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.546+0000 7fbcca841980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T15:58:41.549 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.546+0000 7fbcca841980 20 rados->read ofs=0 len=0 2024-05-30T15:58:41.549 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.546+0000 7fbcca841980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:41.549 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.546+0000 7fbcca841980 20 rados->read ofs=0 len=0 2024-05-30T15:58:41.550 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.550+0000 7fbcca841980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:41.551 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.550+0000 7fbcca841980 20 rados->read ofs=0 len=0 2024-05-30T15:58:41.551 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.550+0000 7fbcca841980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T15:58:41.551 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.550+0000 7fbcca841980 20 searching for the correct realm 2024-05-30T15:58:41.558 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.558+0000 7fbcc7b51640 0 -- 172.21.15.6:0/1681962674 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7fbcb80fca80 msgr2=0x7fbcb80f9120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:41.564 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.562+0000 7fbcc7b51640 0 -- 172.21.15.6:0/1681962674 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7fbb9007b9d0 msgr2=0x7fbb9007de00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:41.572 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.570+0000 7fbcca841980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T15:58:41.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.570+0000 7fbcca841980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T15:58:41.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.570+0000 7fbcca841980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T15:58:41.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.570+0000 7fbcca841980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T15:58:41.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.570+0000 7fbcca841980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T15:58:41.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.570+0000 7fbcca841980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T15:58:41.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.570+0000 7fbcca841980 20 rados->read ofs=0 len=0 2024-05-30T15:58:41.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.570+0000 7fbcca841980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:41.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.570+0000 7fbcca841980 20 rados->read ofs=0 len=0 2024-05-30T15:58:41.575 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.570+0000 7fbcca841980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:41.575 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.570+0000 7fbcca841980 20 rados->read ofs=0 len=0 2024-05-30T15:58:41.575 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.574+0000 7fbcca841980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T15:58:41.575 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.574+0000 7fbcca841980 20 zone default found 2024-05-30T15:58:41.575 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.574+0000 7fbcca841980 4 Realm: () 2024-05-30T15:58:41.575 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.574+0000 7fbcca841980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T15:58:41.575 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.574+0000 7fbcca841980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T15:58:41.576 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.574+0000 7fbcca841980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T15:58:41.576 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.574+0000 7fbcca841980 20 zonegroup default 2024-05-30T15:58:41.576 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.574+0000 7fbcca841980 20 rados->read ofs=0 len=0 2024-05-30T15:58:41.576 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.574+0000 7fbcca841980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:41.576 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.574+0000 7fbcca841980 20 rados->read ofs=0 len=0 2024-05-30T15:58:41.577 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.574+0000 7fbcca841980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:41.577 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.574+0000 7fbcca841980 20 rados->read ofs=0 len=0 2024-05-30T15:58:41.577 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.574+0000 7fbcca841980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:41.577 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.574+0000 7fbcca841980 20 started sync module instance, tier type = 2024-05-30T15:58:41.577 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.574+0000 7fbcca841980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T15:58:41.592 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.590+0000 7fbcca841980 20 add_watcher() i=0 2024-05-30T15:58:41.592 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.590+0000 7fbcca841980 20 add_watcher() i=1 2024-05-30T15:58:41.592 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.590+0000 7fbcca841980 20 add_watcher() i=2 2024-05-30T15:58:41.592 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.590+0000 7fbcca841980 20 add_watcher() i=3 2024-05-30T15:58:41.592 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.590+0000 7fbcca841980 20 add_watcher() i=4 2024-05-30T15:58:41.592 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.590+0000 7fbcca841980 20 add_watcher() i=5 2024-05-30T15:58:41.592 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.590+0000 7fbcca841980 20 add_watcher() i=6 2024-05-30T15:58:41.593 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.590+0000 7fbcca841980 20 add_watcher() i=7 2024-05-30T15:58:41.593 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.590+0000 7fbcca841980 2 all 8 watchers are set, enabling cache 2024-05-30T15:58:41.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.594+0000 7fbb7efdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:41.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.594+0000 7fbcca841980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:58:41.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.594+0000 7fbcca841980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:58:41.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.594+0000 7fbcca841980 5 note: GC not initialized 2024-05-30T15:58:41.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.594+0000 7fbb7dfdb640 20 reqs_thread_entry: start 2024-05-30T15:58:41.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.594+0000 7fbcca841980 20 init_complete bucket index max shards: 11 2024-05-30T15:58:41.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.594+0000 7fbcca841980 20 Filter name: none 2024-05-30T15:58:41.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.594+0000 7fbaef7fe640 20 reqs_thread_entry: start 2024-05-30T15:58:41.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.598+0000 7fbcca841980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-05-30T15:58:41.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.598+0000 7fbcca841980 20 rados->read ofs=0 len=0 2024-05-30T15:58:41.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.598+0000 7fbcca841980 20 rados_obj.operate() r=0 bl.length=190 2024-05-30T15:58:41.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.598+0000 7fbcca841980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-05-30T15:58:41.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.598+0000 7fbcca841980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-05-30T15:58:41.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.598+0000 7fbcca841980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 : miss 2024-05-30T15:58:41.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.598+0000 7fbcca841980 20 rados->read ofs=0 len=0 2024-05-30T15:58:41.599 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.598+0000 7fbcca841980 20 rados_obj.operate() r=0 bl.length=404 2024-05-30T15:58:41.599 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.598+0000 7fbcca841980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 info.flags=0x17 2024-05-30T15:58:41.599 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.598+0000 7fbcca841980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 to cache LRU end 2024-05-30T15:58:41.599 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.598+0000 7fbcca841980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T15:58:41.599 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.598+0000 7fbcca841980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-05-30T15:58:41.599 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.598+0000 7fbcca841980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2 2024-05-30T15:58:41.600 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.598+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.0 2024-05-30T15:58:41.600 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.598+0000 7fbcca841980 1 NOTICE: finished shard 0 (0 entries removed) 2024-05-30T15:58:41.600 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.598+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:41.600 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.598+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.2 2024-05-30T15:58:41.600 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.598+0000 7fbcca841980 1 NOTICE: finished shard 2 (0 entries removed) 2024-05-30T15:58:41.600 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.598+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.3 2024-05-30T15:58:41.601 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.598+0000 7fbcca841980 1 NOTICE: finished shard 3 (0 entries removed) 2024-05-30T15:58:41.601 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.598+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.4 2024-05-30T15:58:41.602 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.602+0000 7fbcca841980 1 NOTICE: finished shard 4 (0 entries removed) 2024-05-30T15:58:41.602 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.602+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:41.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.602+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.6 2024-05-30T15:58:41.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.602+0000 7fbcca841980 1 NOTICE: finished shard 6 (0 entries removed) 2024-05-30T15:58:41.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.602+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.7 2024-05-30T15:58:41.604 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.602+0000 7fbcca841980 1 NOTICE: finished shard 7 (0 entries removed) 2024-05-30T15:58:41.604 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.602+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:41.605 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.602+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.9 2024-05-30T15:58:41.606 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.606+0000 7fbcca841980 1 NOTICE: finished shard 9 (0 entries removed) 2024-05-30T15:58:41.606 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.606+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:41.607 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.606+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:41.607 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.606+0000 7fbcca841980 20 get_obj_state: octx=0x55da98d32a30 obj=check-bucket:e state=0x55da98fd7728 s->prefetch_data=0 2024-05-30T15:58:41.607 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.606+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:41.609 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.606+0000 7fbcca841980 20 get_obj_state: octx=0x55da98d444f0 obj=check-bucket:d state=0x55da98fdd5d8 s->prefetch_data=0 2024-05-30T15:58:41.609 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.606+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:41.609 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.606+0000 7fbcca841980 20 get_obj_state: octx=0x55da98fdf9d0 obj=check-bucket:c state=0x55da98fe2128 s->prefetch_data=0 2024-05-30T15:58:41.609 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.606+0000 7fbcca841980 20 get_obj_state: setting s->obj_tag to ykysyilucdjjg006m6ziyql7jxh6ms44 2024-05-30T15:58:41.609 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.606+0000 7fbcca841980 20 get_obj_state_impl: setting s->olh_tag to yx0t88eg7t14ijmdp45d4mysm5mmgv8t 2024-05-30T15:58:41.609 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.606+0000 7fbcca841980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=yx0t88eg7t14ijmdp45d4mysm5mmgv8t 2024-05-30T15:58:41.610 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.606+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:41.610 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.610+0000 7fbcca841980 20 get_obj_state: octx=0x55da98fd8bc0 obj=check-bucket:f state=0x55da9906be38 s->prefetch_data=0 2024-05-30T15:58:41.610 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.610+0000 7fbcca841980 20 get_obj_state: setting s->obj_tag to 5dfq0lajv7s3tfq6aipq1fl13sq10v0z 2024-05-30T15:58:41.610 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.610+0000 7fbcca841980 20 get_obj_state_impl: setting s->olh_tag to 7rgpu72fs2af2tfcn1dy5gkwq4cilae8 2024-05-30T15:58:41.610 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.610+0000 7fbcca841980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=7rgpu72fs2af2tfcn1dy5gkwq4cilae8 2024-05-30T15:58:41.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.610+0000 7fbcca841980 20 get_obj_state: setting s->obj_tag to akynx521uc07vzkokk6qv2fe9r00nkgh 2024-05-30T15:58:41.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.610+0000 7fbcca841980 20 get_obj_state_impl: setting s->olh_tag to bhajvkynb3kj2vc7aa9bv5yazn720utk 2024-05-30T15:58:41.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.610+0000 7fbcca841980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=bhajvkynb3kj2vc7aa9bv5yazn720utk 2024-05-30T15:58:41.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.610+0000 7fbcca841980 20 get_obj_state: setting s->obj_tag to 6ss8cq3xxm4gk9cb8pllrplueygzvpgi 2024-05-30T15:58:41.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.610+0000 7fbcca841980 20 get_obj_state_impl: setting s->olh_tag to 7xim3wqc59rwkdson503c49bqp9ih65y 2024-05-30T15:58:41.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.610+0000 7fbcca841980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=7xim3wqc59rwkdson503c49bqp9ih65y 2024-05-30T15:58:41.612 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.610+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:41.613 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.610+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:41.614 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.614+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:41.614 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.614+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:41.614 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.614+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:41.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.614+0000 7fbcca841980 20 olh_log_entry: epoch=2 op=2 key=e[] 2024-05-30T15:58:41.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.614+0000 7fbcca841980 20 olh_log_entry: epoch=2 op=3 key=e[LDzRIYUOgLAfmIqCdRb8v5LRlzI77WQ] 2024-05-30T15:58:41.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.614+0000 7fbcca841980 20 get_obj_state: octx=0x55da98d32a30 obj=check-bucket:_:LDzRIYUOgLAfmIqCdRb8v5LRlzI77WQ_e state=0x55da99084b58 s->prefetch_data=0 2024-05-30T15:58:41.616 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.614+0000 7fbcca841980 10 manifest: total_size = 12 2024-05-30T15:58:41.616 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.614+0000 7fbcca841980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12679374379220798050 2024-05-30T15:58:41.616 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.614+0000 7fbcca841980 20 get_obj_state: octx=0x55da98d32a30 obj=check-bucket:_:LDzRIYUOgLAfmIqCdRb8v5LRlzI77WQ_e state=0x55da99084b58 s->prefetch_data=0 2024-05-30T15:58:41.616 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.614+0000 7fbcca841980 20 prepare_atomic_modification: state is not atomic. state=0x55da99084b58 2024-05-30T15:58:41.616 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.614+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:41.617 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.618+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:41.617 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.618+0000 7fbcca841980 20 olh_log_entry: epoch=3 op=3 key=c[-yiRcUi4sH7zS73588U5ncOVmOJ-gmK] 2024-05-30T15:58:41.617 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.618+0000 7fbcca841980 20 get_obj_state: octx=0x55da98fdf9d0 obj=check-bucket:_:-yiRcUi4sH7zS73588U5ncOVmOJ-gmK_c state=0x55da9908d918 s->prefetch_data=0 2024-05-30T15:58:41.618 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.618+0000 7fbcca841980 10 manifest: total_size = 12 2024-05-30T15:58:41.618 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.618+0000 7fbcca841980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.14704259280302444674 2024-05-30T15:58:41.618 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.618+0000 7fbcca841980 20 get_obj_state: octx=0x55da98fdf9d0 obj=check-bucket:_:-yiRcUi4sH7zS73588U5ncOVmOJ-gmK_c state=0x55da9908d918 s->prefetch_data=0 2024-05-30T15:58:41.618 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.618+0000 7fbcca841980 20 prepare_atomic_modification: state is not atomic. state=0x55da9908d918 2024-05-30T15:58:41.618 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.618+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:41.620 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.618+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:41.620 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.618+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:41.621 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.618+0000 7fbcca841980 20 olh_log_entry: epoch=3 op=3 key=d[fLSdYb2sKdvzHpXMKpQ7P6GRmVWkqrP] 2024-05-30T15:58:41.621 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.618+0000 7fbcca841980 20 get_obj_state: octx=0x55da98d444f0 obj=check-bucket:_:fLSdYb2sKdvzHpXMKpQ7P6GRmVWkqrP_d state=0x55da99076758 s->prefetch_data=0 2024-05-30T15:58:41.621 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.622+0000 7fbcca841980 20 olh_log_entry: epoch=2 op=2 key=f[] 2024-05-30T15:58:41.621 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.622+0000 7fbcca841980 20 olh_log_entry: epoch=2 op=3 key=f[eN131lfXPaa.N5nKhFMOC0TRmR1iVLq] 2024-05-30T15:58:41.621 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.622+0000 7fbcca841980 20 get_obj_state: octx=0x55da98fd8bc0 obj=check-bucket:_:eN131lfXPaa.N5nKhFMOC0TRmR1iVLq_f state=0x55da99080f88 s->prefetch_data=0 2024-05-30T15:58:41.622 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.622+0000 7fbcca841980 10 manifest: total_size = 12 2024-05-30T15:58:41.622 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.622+0000 7fbcca841980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.14781975545718396141 2024-05-30T15:58:41.622 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.622+0000 7fbcca841980 20 get_obj_state: octx=0x55da98d444f0 obj=check-bucket:_:fLSdYb2sKdvzHpXMKpQ7P6GRmVWkqrP_d state=0x55da99076758 s->prefetch_data=0 2024-05-30T15:58:41.622 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.622+0000 7fbcca841980 20 prepare_atomic_modification: state is not atomic. state=0x55da99076758 2024-05-30T15:58:41.622 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.622+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:41.623 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.622+0000 7fbcca841980 10 manifest: total_size = 12 2024-05-30T15:58:41.623 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.622+0000 7fbcca841980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2980320780632592175 2024-05-30T15:58:41.623 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.622+0000 7fbcca841980 20 get_obj_state: octx=0x55da98fd8bc0 obj=check-bucket:_:eN131lfXPaa.N5nKhFMOC0TRmR1iVLq_f state=0x55da99080f88 s->prefetch_data=0 2024-05-30T15:58:41.623 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.622+0000 7fbcca841980 20 prepare_atomic_modification: state is not atomic. state=0x55da99080f88 2024-05-30T15:58:41.623 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.622+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:41.627 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.626+0000 7fbb9effd640 20 handle_completion(): completion ok for obj=e[LDzRIYUOgLAfmIqCdRb8v5LRlzI77WQ] 2024-05-30T15:58:41.627 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.626+0000 7fbcca841980 20 get_obj_state: octx=0x55da98d32a30 obj=check-bucket:e state=0x55da98fd7728 s->prefetch_data=0 2024-05-30T15:58:41.628 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.626+0000 7fbcca841980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=yx0t88eg7t14ijmdp45d4mysm5mmgv8t 2024-05-30T15:58:41.633 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.630+0000 7fbb9f7fe640 20 handle_completion(): completion ok for obj=c[-yiRcUi4sH7zS73588U5ncOVmOJ-gmK] 2024-05-30T15:58:41.633 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.630+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:41.633 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.630+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:41.636 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.634+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.8 2024-05-30T15:58:41.636 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.634+0000 7fbcca841980 1 NOTICE: finished shard 8 (1 entries removed) 2024-05-30T15:58:41.636 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.634+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:41.638 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.638+0000 7fbcca841980 20 get_obj_state: octx=0x55da98fd8bc0 obj=check-bucket:f state=0x55da9906be38 s->prefetch_data=0 2024-05-30T15:58:41.638 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.638+0000 7fbcca841980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=7xim3wqc59rwkdson503c49bqp9ih65y 2024-05-30T15:58:41.639 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.638+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:41.639 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.638+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:41.639 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.638+0000 7fbb9effd640 20 handle_completion(): completion ok for obj=f[eN131lfXPaa.N5nKhFMOC0TRmR1iVLq] 2024-05-30T15:58:41.639 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.638+0000 7fbb9effd640 20 handle_completion(): completion ok for obj=d[fLSdYb2sKdvzHpXMKpQ7P6GRmVWkqrP] 2024-05-30T15:58:41.639 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.638+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.1 2024-05-30T15:58:41.640 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.638+0000 7fbcca841980 1 NOTICE: finished shard 1 (1 entries removed) 2024-05-30T15:58:41.643 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.642+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.5 2024-05-30T15:58:41.643 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.642+0000 7fbcca841980 1 NOTICE: finished shard 5 (1 entries removed) 2024-05-30T15:58:41.643 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.642+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:41.646 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.646+0000 7fbcca841980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2.10 2024-05-30T15:58:41.646 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.646+0000 7fbcca841980 1 NOTICE: finished shard 10 (1 entries removed) 2024-05-30T15:58:41.646 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.646+0000 7fbcca841980 1 NOTICE: finished all shards (4 entries removed) 2024-05-30T15:58:41.647 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.646+0000 7fbb7efdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:41.651 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.650+0000 7fbcca841980 20 remove_watcher() i=0 2024-05-30T15:58:41.652 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.650+0000 7fbcca841980 2 removed watcher, disabling cache 2024-05-30T15:58:41.654 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.654+0000 7fbcca841980 20 remove_watcher() i=1 2024-05-30T15:58:41.656 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.654+0000 7fbcca841980 20 remove_watcher() i=2 2024-05-30T15:58:41.658 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.658+0000 7fbcca841980 20 remove_watcher() i=3 2024-05-30T15:58:41.660 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.658+0000 7fbcca841980 20 remove_watcher() i=4 2024-05-30T15:58:41.662 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.662+0000 7fbcca841980 20 remove_watcher() i=5 2024-05-30T15:58:41.663 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.662+0000 7fbcca841980 20 remove_watcher() i=6 2024-05-30T15:58:41.664 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:41.662+0000 7fbcca841980 20 remove_watcher() i=7 2024-05-30T15:58:42.144 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket check-bucket 2024-05-30T15:58:42.176 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T15:58:42.176 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T15:58:42.225 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.226+0000 7fe7dc079980 20 rados->read ofs=0 len=0 2024-05-30T15:58:42.227 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.226+0000 7fe7dc079980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:42.227 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.226+0000 7fe7dc079980 20 realm 2024-05-30T15:58:42.227 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.226+0000 7fe7dc079980 20 rados->read ofs=0 len=0 2024-05-30T15:58:42.228 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.226+0000 7fe7dc079980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:42.228 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.226+0000 7fe7dc079980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T15:58:42.228 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.226+0000 7fe7dc079980 20 rados->read ofs=0 len=0 2024-05-30T15:58:42.228 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.226+0000 7fe7dc079980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:42.228 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.226+0000 7fe7dc079980 20 rados->read ofs=0 len=0 2024-05-30T15:58:42.230 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.230+0000 7fe7dc079980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:42.230 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.230+0000 7fe7dc079980 20 rados->read ofs=0 len=0 2024-05-30T15:58:42.231 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.230+0000 7fe7dc079980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T15:58:42.231 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.230+0000 7fe7dc079980 20 searching for the correct realm 2024-05-30T15:58:42.238 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.238+0000 7fe7d9389640 0 -- 172.21.15.6:0/4250104246 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7fe7cc036070 msgr2=0x7fe7cc036450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T15:58:42.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.242+0000 7fe7da38b640 0 -- 172.21.15.6:0/4250104246 >> [v2:172.21.15.204:6800/3687190393,v1:172.21.15.204:6801/3687190393] conn(0x7fe7d0008040 msgr2=0x7fe7d0063a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T15:58:42.253 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.250+0000 7fe7dc079980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T15:58:42.253 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.250+0000 7fe7dc079980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T15:58:42.253 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.250+0000 7fe7dc079980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T15:58:42.253 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.250+0000 7fe7dc079980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T15:58:42.253 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.250+0000 7fe7dc079980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T15:58:42.253 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.250+0000 7fe7dc079980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T15:58:42.253 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.250+0000 7fe7dc079980 20 rados->read ofs=0 len=0 2024-05-30T15:58:42.254 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.254+0000 7fe7dc079980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:42.254 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.254+0000 7fe7dc079980 20 rados->read ofs=0 len=0 2024-05-30T15:58:42.254 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.254+0000 7fe7dc079980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T15:58:42.254 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.254+0000 7fe7dc079980 20 rados->read ofs=0 len=0 2024-05-30T15:58:42.255 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.254+0000 7fe7dc079980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T15:58:42.255 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.254+0000 7fe7dc079980 20 zone default found 2024-05-30T15:58:42.255 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.254+0000 7fe7dc079980 4 Realm: () 2024-05-30T15:58:42.255 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.254+0000 7fe7dc079980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T15:58:42.255 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.254+0000 7fe7dc079980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T15:58:42.255 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.254+0000 7fe7dc079980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T15:58:42.255 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.254+0000 7fe7dc079980 20 zonegroup default 2024-05-30T15:58:42.255 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.254+0000 7fe7dc079980 20 rados->read ofs=0 len=0 2024-05-30T15:58:42.256 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.254+0000 7fe7dc079980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:42.256 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.254+0000 7fe7dc079980 20 rados->read ofs=0 len=0 2024-05-30T15:58:42.256 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.254+0000 7fe7dc079980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:42.256 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.254+0000 7fe7dc079980 20 rados->read ofs=0 len=0 2024-05-30T15:58:42.256 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.254+0000 7fe7dc079980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T15:58:42.256 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.254+0000 7fe7dc079980 20 started sync module instance, tier type = 2024-05-30T15:58:42.256 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.254+0000 7fe7dc079980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T15:58:42.273 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.274+0000 7fe7dc079980 20 add_watcher() i=0 2024-05-30T15:58:42.273 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.274+0000 7fe7dc079980 20 add_watcher() i=1 2024-05-30T15:58:42.273 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.274+0000 7fe7dc079980 20 add_watcher() i=2 2024-05-30T15:58:42.273 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.274+0000 7fe7dc079980 20 add_watcher() i=3 2024-05-30T15:58:42.274 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.274+0000 7fe7dc079980 20 add_watcher() i=4 2024-05-30T15:58:42.274 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.274+0000 7fe7dc079980 20 add_watcher() i=5 2024-05-30T15:58:42.274 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.274+0000 7fe7dc079980 20 add_watcher() i=6 2024-05-30T15:58:42.274 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.274+0000 7fe7dc079980 20 add_watcher() i=7 2024-05-30T15:58:42.275 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.274+0000 7fe7dc079980 2 all 8 watchers are set, enabling cache 2024-05-30T15:58:42.277 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.278+0000 7fe68b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:42.278 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.278+0000 7fe7dc079980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T15:58:42.278 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.278+0000 7fe7dc079980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T15:58:42.278 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.278+0000 7fe7dc079980 5 note: GC not initialized 2024-05-30T15:58:42.279 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.278+0000 7fe68a7fc640 20 reqs_thread_entry: start 2024-05-30T15:58:42.279 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.278+0000 7fe7dc079980 20 init_complete bucket index max shards: 11 2024-05-30T15:58:42.279 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.278+0000 7fe7dc079980 20 Filter name: none 2024-05-30T15:58:42.279 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.278+0000 7fe5fbfff640 20 reqs_thread_entry: start 2024-05-30T15:58:42.279 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.278+0000 7fe7dc079980 20 rados->read ofs=0 len=0 2024-05-30T15:58:42.279 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.278+0000 7fe7dc079980 20 rados_obj.operate() r=0 bl.length=190 2024-05-30T15:58:42.279 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.278+0000 7fe7dc079980 20 rados->read ofs=0 len=0 2024-05-30T15:58:42.280 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.278+0000 7fe7dc079980 20 rados_obj.operate() r=0 bl.length=404 2024-05-30T15:58:42.280 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.278+0000 7fe7dc079980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T15:58:42.280 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.278+0000 7fe7dc079980 20 rados->read ofs=0 len=0 2024-05-30T15:58:42.280 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.278+0000 7fe7dc079980 20 rados_obj.operate() r=0 bl.length=190 2024-05-30T15:58:42.280 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.278+0000 7fe7dc079980 20 rados->read ofs=0 len=0 2024-05-30T15:58:42.281 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.278+0000 7fe7dc079980 20 rados_obj.operate() r=0 bl.length=404 2024-05-30T15:58:42.281 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.278+0000 7fe7dc079980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T15:58:42.283 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.282+0000 7fe68b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T15:58:42.289 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.286+0000 7fe7dc079980 20 remove_watcher() i=0 2024-05-30T15:58:42.289 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.286+0000 7fe7dc079980 2 removed watcher, disabling cache 2024-05-30T15:58:42.291 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.290+0000 7fe7dc079980 20 remove_watcher() i=1 2024-05-30T15:58:42.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.290+0000 7fe7dc079980 20 remove_watcher() i=2 2024-05-30T15:58:42.295 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.294+0000 7fe7dc079980 20 remove_watcher() i=3 2024-05-30T15:58:42.297 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.298+0000 7fe7dc079980 20 remove_watcher() i=4 2024-05-30T15:58:42.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.298+0000 7fe7dc079980 20 remove_watcher() i=5 2024-05-30T15:58:42.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.298+0000 7fe7dc079980 20 remove_watcher() i=6 2024-05-30T15:58:42.301 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T15:58:42.302+0000 7fe7dc079980 20 remove_watcher() i=7 2024-05-30T15:58:42.315 INFO:tasks.workunit.client.0.smithi006.stderr:{'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}} 2024-05-30T15:58:42.316 INFO:tasks.workunit.client.0.smithi006.stderr:Deleting bucket check-bucket 2024-05-30T15:58:42.874 INFO:tasks.workunit.client.0.smithi006.stderr:Completed bucket check tests 2024-05-30T15:58:42.874 INFO:tasks.workunit.client.0.smithi006.stdout:connected to http://localhost:80 2024-05-30T15:58:42.976 INFO:tasks.workunit.client.0.smithi006.stderr:+ deactivate 2024-05-30T15:58:42.976 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -n /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin ']' 2024-05-30T15:58:42.976 INFO:tasks.workunit.client.0.smithi006.stderr:+ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-05-30T15:58:42.976 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-05-30T15:58:42.976 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset _OLD_VIRTUAL_PATH 2024-05-30T15:58:42.977 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -n '' ']' 2024-05-30T15:58:42.977 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -n /usr/bin/bash -o -n '' ']' 2024-05-30T15:58:42.977 INFO:tasks.workunit.client.0.smithi006.stderr:+ hash -r 2024-05-30T15:58:42.977 INFO:tasks.workunit.client.0.smithi006.stdout:OK. 2024-05-30T15:58:42.978 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -n '' ']' 2024-05-30T15:58:42.978 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset VIRTUAL_ENV 2024-05-30T15:58:42.978 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset VIRTUAL_ENV_PROMPT 2024-05-30T15:58:42.978 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' '!' '' = nondestructive ']' 2024-05-30T15:58:42.978 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset -f deactivate 2024-05-30T15:58:42.978 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo OK. 2024-05-30T15:58:42.979 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-30T15:58:42.979 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-30T15:58:43.034 INFO:tasks.workunit:Stopping ['rgw/run-bucket-check.sh'] on client.0... 2024-05-30T15:58:43.034 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-30T15:58:43.421 DEBUG:teuthology.parallel:result is None 2024-05-30T15:58:43.422 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-30T15:58:43.433 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-30T15:58:43.433 DEBUG:teuthology.orchestra.run.smithi006:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-05-30T15:58:43.481 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-05-30T15:58:43.481 INFO:teuthology.run_tasks:Running task workunit... 2024-05-30T15:58:43.492 INFO:tasks.workunit:Pulling workunits from ref da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 2024-05-30T15:58:43.492 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-30T15:58:43.492 DEBUG:teuthology.orchestra.run.smithi006:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-30T15:58:43.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-30T15:58:43.535 INFO:teuthology.orchestra.run.smithi006.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-05-30T15:58:43.535 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-05-30T15:58:43.582 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-05-30T15:58:43.583 DEBUG:teuthology.orchestra.run.smithi006:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-05-30T15:58:43.630 INFO:tasks.workunit:timeout=3h 2024-05-30T15:58:43.630 INFO:tasks.workunit:cleanup=True 2024-05-30T15:58:43.631 DEBUG:teuthology.orchestra.run.smithi006:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 2024-05-30T15:58:43.680 INFO:tasks.workunit.client.0.smithi006.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-30T16:00:08.313 INFO:tasks.workunit.client.0.smithi006.stderr:Updating files: 99% (12254/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-30T16:00:08.929 INFO:tasks.workunit.client.0.smithi006.stderr:Note: switching to 'da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2'. 2024-05-30T16:00:08.929 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:00:08.929 INFO:tasks.workunit.client.0.smithi006.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-30T16:00:08.929 INFO:tasks.workunit.client.0.smithi006.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-30T16:00:08.929 INFO:tasks.workunit.client.0.smithi006.stderr:state without impacting any branches by switching back to a branch. 2024-05-30T16:00:08.929 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:00:08.929 INFO:tasks.workunit.client.0.smithi006.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-30T16:00:08.929 INFO:tasks.workunit.client.0.smithi006.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-30T16:00:08.929 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:00:08.929 INFO:tasks.workunit.client.0.smithi006.stderr: git switch -c 2024-05-30T16:00:08.930 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:00:08.930 INFO:tasks.workunit.client.0.smithi006.stderr:Or undo this operation with: 2024-05-30T16:00:08.930 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:00:08.930 INFO:tasks.workunit.client.0.smithi006.stderr: git switch - 2024-05-30T16:00:08.930 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:00:08.930 INFO:tasks.workunit.client.0.smithi006.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-30T16:00:08.930 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:00:08.930 INFO:tasks.workunit.client.0.smithi006.stderr:HEAD is now at da98024b78b Merge branches 'pr-57079', 'pr-57536', 'pr-57641' and 'pr-57702' into wip-cbodley-testing 2024-05-30T16:00:08.937 DEBUG:teuthology.orchestra.run.smithi006:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-30T16:00:08.987 INFO:tasks.workunit.client.0.smithi006.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-30T16:00:08.990 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-30T16:00:08.990 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-30T16:00:09.054 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-30T16:00:09.111 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-30T16:00:09.159 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-30T16:00:09.160 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-30T16:00:09.161 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-30T16:00:09.207 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-30T16:00:09.211 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T16:00:09.211 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-30T16:00:09.258 INFO:tasks.workunit:Running workunits matching cls/test_cls_lock.sh on client.0... 2024-05-30T16:00:09.259 INFO:tasks.workunit:Running workunit cls/test_cls_lock.sh... 2024-05-30T16:00:09.260 DEBUG:teuthology.orchestra.run.smithi006:workunit test cls/test_cls_lock.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_lock.sh 2024-05-30T16:00:09.324 INFO:tasks.workunit.client.0.smithi006.stdout:Running main() from gmock_main.cc 2024-05-30T16:00:09.324 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] Running 11 tests from 1 test suite. 2024-05-30T16:00:09.324 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment set-up. 2024-05-30T16:00:09.324 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 11 tests from ClsLock 2024-05-30T16:00:09.324 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] ClsLock.TestMultiLocking 2024-05-30T16:00:10.640 INFO:tasks.workunit.client.0.smithi006.stdout:lock: mylock 2024-05-30T16:00:10.640 INFO:tasks.workunit.client.0.smithi006.stdout: lock_type: exclusive 2024-05-30T16:00:10.640 INFO:tasks.workunit.client.0.smithi006.stdout: tag: 2024-05-30T16:00:10.641 INFO:tasks.workunit.client.0.smithi006.stdout: lockers: 2024-05-30T16:00:10.641 INFO:tasks.workunit.client.0.smithi006.stdout: client.5118 expiration=2024-05-30T16:02:10.635472+0000 addr=v1:172.21.15.6:0/2394013085 cookie= 2024-05-30T16:00:10.648 INFO:tasks.workunit.client.0.smithi006.stdout:lock: mylock 2024-05-30T16:00:10.648 INFO:tasks.workunit.client.0.smithi006.stdout: lock_type: shared 2024-05-30T16:00:10.648 INFO:tasks.workunit.client.0.smithi006.stdout: tag: 2024-05-30T16:00:10.648 INFO:tasks.workunit.client.0.smithi006.stdout: lockers: 2024-05-30T16:00:10.648 INFO:tasks.workunit.client.0.smithi006.stdout: client.5118 expiration=2024-05-30T16:02:10.648896+0000 addr=v1:172.21.15.6:0/2394013085 cookie= 2024-05-30T16:00:10.648 INFO:tasks.workunit.client.0.smithi006.stdout: client.5124 expiration=0.000000 addr=v1:172.21.15.6:0/3677529295 cookie= 2024-05-30T16:00:10.651 INFO:tasks.workunit.client.0.smithi006.stdout:lock: mylock 2024-05-30T16:00:10.651 INFO:tasks.workunit.client.0.smithi006.stdout: lock_type: shared 2024-05-30T16:00:10.651 INFO:tasks.workunit.client.0.smithi006.stdout: tag: 2024-05-30T16:00:10.651 INFO:tasks.workunit.client.0.smithi006.stdout: lockers: 2024-05-30T16:00:10.651 INFO:tasks.workunit.client.0.smithi006.stdout: client.5124 expiration=0.000000 addr=v1:172.21.15.6:0/3677529295 cookie= 2024-05-30T16:00:11.596 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] ClsLock.TestMultiLocking (2271 ms) 2024-05-30T16:00:11.596 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] ClsLock.TestMeta 2024-05-30T16:00:13.685 INFO:tasks.workunit.client.0.smithi006.stdout:lock: mylock 2024-05-30T16:00:13.685 INFO:tasks.workunit.client.0.smithi006.stdout: lock_type: shared 2024-05-30T16:00:13.685 INFO:tasks.workunit.client.0.smithi006.stdout: tag: 2024-05-30T16:00:13.685 INFO:tasks.workunit.client.0.smithi006.stdout: lockers: 2024-05-30T16:00:13.686 INFO:tasks.workunit.client.0.smithi006.stdout: client.5136 expiration=0.000000 addr=v1:172.21.15.6:0/1084172495 cookie= 2024-05-30T16:00:13.689 INFO:tasks.workunit.client.0.smithi006.stdout:lock: mylock 2024-05-30T16:00:13.689 INFO:tasks.workunit.client.0.smithi006.stdout: lock_type: exclusive 2024-05-30T16:00:13.689 INFO:tasks.workunit.client.0.smithi006.stdout: tag: new_tag 2024-05-30T16:00:13.689 INFO:tasks.workunit.client.0.smithi006.stdout: lockers: 2024-05-30T16:00:13.690 INFO:tasks.workunit.client.0.smithi006.stdout: client.5130 expiration=0.000000 addr=v1:172.21.15.6:0/2695261623 cookie= 2024-05-30T16:00:14.608 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] ClsLock.TestMeta (3013 ms) 2024-05-30T16:00:14.609 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] ClsLock.TestCookie 2024-05-30T16:00:16.707 INFO:tasks.workunit.client.0.smithi006.stdout:lock: mylock 2024-05-30T16:00:16.707 INFO:tasks.workunit.client.0.smithi006.stdout: lock_type: exclusive 2024-05-30T16:00:16.707 INFO:tasks.workunit.client.0.smithi006.stdout: tag: 2024-05-30T16:00:16.707 INFO:tasks.workunit.client.0.smithi006.stdout: lockers: 2024-05-30T16:00:16.714 INFO:tasks.workunit.client.0.smithi006.stdout:lock: mylock 2024-05-30T16:00:16.714 INFO:tasks.workunit.client.0.smithi006.stdout: lock_type: shared 2024-05-30T16:00:16.714 INFO:tasks.workunit.client.0.smithi006.stdout: tag: 2024-05-30T16:00:16.714 INFO:tasks.workunit.client.0.smithi006.stdout: lockers: 2024-05-30T16:00:16.714 INFO:tasks.workunit.client.0.smithi006.stdout: client.5142 expiration=0.000000 addr=v1:172.21.15.6:0/599291818 cookie= 2024-05-30T16:00:16.714 INFO:tasks.workunit.client.0.smithi006.stdout: client.5142 expiration=0.000000 addr=v1:172.21.15.6:0/599291818 cookie=new cookie 2024-05-30T16:00:17.680 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] ClsLock.TestCookie (3072 ms) 2024-05-30T16:00:17.680 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] ClsLock.TestMultipleLocks 2024-05-30T16:00:20.688 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] ClsLock.TestMultipleLocks (3008 ms) 2024-05-30T16:00:20.688 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] ClsLock.TestLockDuration 2024-05-30T16:00:28.568 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] ClsLock.TestLockDuration (7879 ms) 2024-05-30T16:00:28.568 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] ClsLock.TestAssertLocked 2024-05-30T16:00:31.615 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] ClsLock.TestAssertLocked (3047 ms) 2024-05-30T16:00:31.615 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] ClsLock.TestSetCookie 2024-05-30T16:00:34.618 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] ClsLock.TestSetCookie (3003 ms) 2024-05-30T16:00:34.619 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] ClsLock.TestRenew 2024-05-30T16:00:55.594 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] ClsLock.TestRenew (20973 ms) 2024-05-30T16:00:55.594 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralBasic 2024-05-30T16:01:02.656 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] ClsLock.TestExclusiveEphemeralBasic (7065 ms) 2024-05-30T16:01:02.657 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealEphemeral 2024-05-30T16:01:10.582 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealEphemeral (7924 ms) 2024-05-30T16:01:10.582 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealExclusive 2024-05-30T16:01:18.657 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealExclusive (8075 ms) 2024-05-30T16:01:18.658 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 11 tests from ClsLock (69331 ms total) 2024-05-30T16:01:18.658 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-05-30T16:01:18.658 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment tear-down 2024-05-30T16:01:18.658 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] 11 tests from 1 test suite ran. (69331 ms total) 2024-05-30T16:01:18.658 INFO:tasks.workunit.client.0.smithi006.stdout:[ PASSED ] 11 tests. 2024-05-30T16:01:18.662 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-30T16:01:18.663 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-30T16:01:18.719 INFO:tasks.workunit:Running workunits matching cls/test_cls_log.sh on client.0... 2024-05-30T16:01:18.720 INFO:tasks.workunit:Running workunit cls/test_cls_log.sh... 2024-05-30T16:01:18.721 DEBUG:teuthology.orchestra.run.smithi006:workunit test cls/test_cls_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_log.sh 2024-05-30T16:01:18.785 INFO:tasks.workunit.client.0.smithi006.stdout:Running main() from gmock_main.cc 2024-05-30T16:01:18.786 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] Running 4 tests from 1 test suite. 2024-05-30T16:01:18.786 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment set-up. 2024-05-30T16:01:18.786 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 4 tests from cls_log 2024-05-30T16:01:18.786 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_log.test_log_add_same_time 2024-05-30T16:01:21.666 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_log.test_log_add_same_time (2880 ms) 2024-05-30T16:01:21.666 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_log.test_log_add_different_time 2024-05-30T16:01:24.730 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_log.test_log_add_different_time (3065 ms) 2024-05-30T16:01:24.730 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_log.trim_by_time 2024-05-30T16:01:27.755 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_log.trim_by_time (3025 ms) 2024-05-30T16:01:27.756 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_log.trim_by_marker 2024-05-30T16:01:30.811 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_log.trim_by_marker (3052 ms) 2024-05-30T16:01:30.812 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 4 tests from cls_log (12022 ms total) 2024-05-30T16:01:30.812 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-05-30T16:01:30.812 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment tear-down 2024-05-30T16:01:30.812 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] 4 tests from 1 test suite ran. (12022 ms total) 2024-05-30T16:01:30.812 INFO:tasks.workunit.client.0.smithi006.stdout:[ PASSED ] 4 tests. 2024-05-30T16:01:30.814 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-30T16:01:30.814 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-30T16:01:30.874 INFO:tasks.workunit:Running workunits matching cls/test_cls_refcount.sh on client.0... 2024-05-30T16:01:30.875 INFO:tasks.workunit:Running workunit cls/test_cls_refcount.sh... 2024-05-30T16:01:30.875 DEBUG:teuthology.orchestra.run.smithi006:workunit test cls/test_cls_refcount.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_refcount.sh 2024-05-30T16:01:30.943 INFO:tasks.workunit.client.0.smithi006.stdout:Running main() from gmock_main.cc 2024-05-30T16:01:30.943 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] Running 10 tests from 1 test suite. 2024-05-30T16:01:30.943 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment set-up. 2024-05-30T16:01:30.943 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 10 tests from cls_refcount 2024-05-30T16:01:30.943 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_refcount.test_implicit 2024-05-30T16:01:33.839 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_refcount.test_implicit (2894 ms) 2024-05-30T16:01:33.840 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_refcount.test_implicit_idempotent 2024-05-30T16:01:36.886 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_refcount.test_implicit_idempotent (3040 ms) 2024-05-30T16:01:36.886 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_refcount.test_put_snap 2024-05-30T16:01:45.609 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_refcount.test_put_snap (8732 ms) 2024-05-30T16:01:45.610 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_refcount.test_explicit 2024-05-30T16:01:48.659 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_refcount.test_explicit (3050 ms) 2024-05-30T16:01:48.659 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_refcount.set 2024-05-30T16:01:51.698 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_refcount.set (3038 ms) 2024-05-30T16:01:51.698 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_refcount.test_implicit_ec 2024-05-30T16:01:58.770 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_refcount.test_implicit_ec (7073 ms) 2024-05-30T16:01:58.771 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_refcount.test_implicit_idempotent_ec 2024-05-30T16:02:05.812 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_refcount.test_implicit_idempotent_ec (7041 ms) 2024-05-30T16:02:05.812 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_refcount.test_put_snap_ec 2024-05-30T16:02:17.952 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_refcount.test_put_snap_ec (12140 ms) 2024-05-30T16:02:17.952 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_refcount.test_explicit_ec 2024-05-30T16:02:25.011 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_refcount.test_explicit_ec (7059 ms) 2024-05-30T16:02:25.011 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_refcount.set_ec 2024-05-30T16:02:32.105 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_refcount.set_ec (7094 ms) 2024-05-30T16:02:32.105 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 10 tests from cls_refcount (61161 ms total) 2024-05-30T16:02:32.105 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-05-30T16:02:32.105 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment tear-down 2024-05-30T16:02:32.105 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] 10 tests from 1 test suite ran. (61161 ms total) 2024-05-30T16:02:32.105 INFO:tasks.workunit.client.0.smithi006.stdout:[ PASSED ] 10 tests. 2024-05-30T16:02:32.112 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-30T16:02:32.112 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-30T16:02:32.167 INFO:tasks.workunit:Running workunits matching cls/test_cls_rgw.sh on client.0... 2024-05-30T16:02:32.169 INFO:tasks.workunit:Running workunit cls/test_cls_rgw.sh... 2024-05-30T16:02:32.169 DEBUG:teuthology.orchestra.run.smithi006:workunit test cls/test_cls_rgw.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh 2024-05-30T16:02:32.234 INFO:tasks.workunit.client.0.smithi006.stdout:Running main() from gmock_main.cc 2024-05-30T16:02:32.234 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] Running 16 tests from 1 test suite. 2024-05-30T16:02:32.234 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment set-up. 2024-05-30T16:02:32.234 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 16 tests from cls_rgw 2024-05-30T16:02:34.114 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_rgw.index_basic 2024-05-30T16:02:34.192 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_rgw.index_basic (78 ms) 2024-05-30T16:02:34.193 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_rgw.index_multiple_obj_writers 2024-05-30T16:02:34.276 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_rgw.index_multiple_obj_writers (84 ms) 2024-05-30T16:02:34.276 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_rgw.index_remove_object 2024-05-30T16:02:34.391 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_rgw.index_remove_object (115 ms) 2024-05-30T16:02:34.391 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_rgw.index_suggest 2024-05-30T16:02:36.951 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_rgw.index_suggest (2559 ms) 2024-05-30T16:02:36.951 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_rgw.index_suggest_complete 2024-05-30T16:02:36.965 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_rgw.index_suggest_complete (15 ms) 2024-05-30T16:02:36.965 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_rgw.index_list 2024-05-30T16:02:37.037 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_rgw.index_list (72 ms) 2024-05-30T16:02:37.037 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_rgw.index_list_delimited 2024-05-30T16:03:30.607 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_rgw.index_list_delimited (53569 ms) 2024-05-30T16:03:30.608 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_rgw.bi_list 2024-05-30T16:03:30.805 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_rgw.bi_list (198 ms) 2024-05-30T16:03:30.805 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_rgw.gc_set 2024-05-30T16:03:30.830 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_rgw.gc_set (25 ms) 2024-05-30T16:03:30.830 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_rgw.gc_list 2024-05-30T16:03:30.858 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_rgw.gc_list (29 ms) 2024-05-30T16:03:30.859 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_rgw.gc_defer 2024-05-30T16:03:38.695 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_rgw.gc_defer (7836 ms) 2024-05-30T16:03:38.695 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_rgw.usage_basic 2024-05-30T16:03:38.901 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_rgw.usage_basic (206 ms) 2024-05-30T16:03:38.901 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_rgw.usage_clear_no_obj 2024-05-30T16:03:38.903 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_rgw.usage_clear_no_obj (4 ms) 2024-05-30T16:03:38.904 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_rgw.usage_clear 2024-05-30T16:03:39.119 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_rgw.usage_clear (215 ms) 2024-05-30T16:03:39.119 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_rgw.bi_log_trim 2024-05-30T16:03:39.242 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_rgw.bi_log_trim (123 ms) 2024-05-30T16:03:39.242 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_rgw.index_racing_removes 2024-05-30T16:03:39.282 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_rgw.index_racing_removes (40 ms) 2024-05-30T16:03:39.714 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 16 tests from cls_rgw (65168 ms total) 2024-05-30T16:03:39.714 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-05-30T16:03:39.714 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment tear-down 2024-05-30T16:03:39.714 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] 16 tests from 1 test suite ran. (67473 ms total) 2024-05-30T16:03:39.714 INFO:tasks.workunit.client.0.smithi006.stdout:[ PASSED ] 16 tests. 2024-05-30T16:03:39.715 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-30T16:03:39.715 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-30T16:03:39.775 INFO:tasks.workunit:Running workunits matching cls/test_cls_rgw_gc.sh on client.0... 2024-05-30T16:03:39.776 INFO:tasks.workunit:Running workunit cls/test_cls_rgw_gc.sh... 2024-05-30T16:03:39.776 DEBUG:teuthology.orchestra.run.smithi006:workunit test cls/test_cls_rgw_gc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw_gc.sh 2024-05-30T16:03:39.841 INFO:tasks.workunit.client.0.smithi006.stdout:Running main() from gmock_main.cc 2024-05-30T16:03:39.841 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] Running 9 tests from 1 test suite. 2024-05-30T16:03:39.841 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment set-up. 2024-05-30T16:03:39.841 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 9 tests from cls_rgw_gc 2024-05-30T16:03:39.841 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_rgw_gc.init 2024-05-30T16:03:41.716 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_rgw_gc.init (1875 ms) 2024-05-30T16:03:41.717 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops1 2024-05-30T16:03:41.748 INFO:tasks.workunit.client.0.smithi006.stderr:[ ] list info tag = chain-0 2024-05-30T16:03:41.749 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_rgw_gc.gc_queue_ops1 (32 ms) 2024-05-30T16:03:41.749 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops2 2024-05-30T16:03:41.762 INFO:tasks.workunit.client.0.smithi006.stderr:[ ] next_marker is: = 0/4263 2024-05-30T16:03:41.763 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_rgw_gc.gc_queue_ops2 (14 ms) 2024-05-30T16:03:41.763 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops5 2024-05-30T16:03:41.779 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_rgw_gc.gc_queue_ops5 (16 ms) 2024-05-30T16:03:41.779 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops6 2024-05-30T16:03:41.841 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_rgw_gc.gc_queue_ops6 (63 ms) 2024-05-30T16:03:41.841 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops7 2024-05-30T16:03:41.861 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_rgw_gc.gc_queue_ops7 (20 ms) 2024-05-30T16:03:41.861 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops8 2024-05-30T16:03:41.882 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_rgw_gc.gc_queue_ops8 (20 ms) 2024-05-30T16:03:41.882 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops9 2024-05-30T16:03:41.903 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_rgw_gc.gc_queue_ops9 (22 ms) 2024-05-30T16:03:41.903 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_rgw_gc.finalize 2024-05-30T16:03:42.710 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_rgw_gc.finalize (807 ms) 2024-05-30T16:03:42.710 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 9 tests from cls_rgw_gc (2869 ms total) 2024-05-30T16:03:42.710 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-05-30T16:03:42.711 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment tear-down 2024-05-30T16:03:42.711 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] 9 tests from 1 test suite ran. (2869 ms total) 2024-05-30T16:03:42.711 INFO:tasks.workunit.client.0.smithi006.stdout:[ PASSED ] 9 tests. 2024-05-30T16:03:42.716 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-30T16:03:42.716 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-30T16:03:42.771 INFO:tasks.workunit:Running workunits matching cls/test_cls_rgw_stats.sh on client.0... 2024-05-30T16:03:42.772 INFO:tasks.workunit:Running workunit cls/test_cls_rgw_stats.sh... 2024-05-30T16:03:42.772 DEBUG:teuthology.orchestra.run.smithi006:workunit test cls/test_cls_rgw_stats.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw_stats.sh 2024-05-30T16:03:42.848 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:03:42.848 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:03:42.848 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] Running 1 test from 1 test suite. 2024-05-30T16:03:42.848 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment set-up. 2024-05-30T16:03:44.760 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 1 test from cls_rgw_stats 2024-05-30T16:03:44.760 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] cls_rgw_stats.simulate 2024-05-30T16:03:44.786 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.787+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~BE4B2TDAQS1A.0 size=1048576 2024-05-30T16:03:44.788 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.787+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~BE4B2TDAQS1A.1 size=1001567 2024-05-30T16:03:44.789 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.787+0000 7f08851138c0 1 > prepared multipart upload key=TPCETCO3Y1M6 upload=BE4B2TDAQS1A tag=967N3I00W2PX type=write size=3098719 2024-05-30T16:03:44.791 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.791+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=XAVHGUXLLHYZ type=del size=3305343 2024-05-30T16:03:44.795 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.795+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~8AO30B7H2A0Z.0 size=1048576 2024-05-30T16:03:44.798 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.799+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~8AO30B7H2A0Z.1 size=162418 2024-05-30T16:03:44.798 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.799+0000 7f08851138c0 1 > prepared multipart upload key=ZALVY70ISLM5 upload=8AO30B7H2A0Z tag=ZAGYHBRM0Y1V type=write size=2259570 2024-05-30T16:03:44.800 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.799+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=GLZQK9H3EVGL type=write size=422135 2024-05-30T16:03:44.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.803+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~8088NU0E2EXI.0 size=1048576 2024-05-30T16:03:44.807 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.807+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~8088NU0E2EXI.1 size=1048576 2024-05-30T16:03:44.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.807+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~8088NU0E2EXI.2 size=902100 2024-05-30T16:03:44.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.807+0000 7f08851138c0 1 > prepared multipart upload key=603HBR8PRH6L upload=8088NU0E2EXI tag=XO41CXYK2JF5 type=write size=4047828 2024-05-30T16:03:44.813 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.815+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~MJOMVA53UBLS.0 size=1048576 2024-05-30T16:03:44.815 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.815+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~MJOMVA53UBLS.1 size=1030539 2024-05-30T16:03:44.815 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.815+0000 7f08851138c0 1 > prepared multipart upload key=Y3CYZJ83WYNI upload=MJOMVA53UBLS tag=TBLL1UUB3PO7 type=write size=3127691 2024-05-30T16:03:44.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.819+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~3LQ3CZMB30MY.0 size=359289 2024-05-30T16:03:44.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.819+0000 7f08851138c0 1 > prepared multipart upload key=PLWFSKPKVO19 upload=3LQ3CZMB30MY tag=QQZBOSR6XZ2J type=write size=1407865 2024-05-30T16:03:44.822 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.823+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=CKQNA181DSBF type=write size=5284 2024-05-30T16:03:44.825 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.823+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=CJSSBAKO7B5I type=del size=2242547 2024-05-30T16:03:44.827 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.827+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=TLNURF9H0E72 type=del size=3520176 2024-05-30T16:03:44.830 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.831+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=IMCCHOQ3JVWB type=del size=230618 2024-05-30T16:03:44.833 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.831+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=W6Q60QAFM49D type=write size=881478 2024-05-30T16:03:44.835 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.835+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=H4NAFU9JL61Z type=del size=1197622 2024-05-30T16:03:44.840 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.839+0000 7f08851138c0 1 > prepared part key=_multipart_MMLF65KLC42Q.2~GGV4NFAXSP39.0 size=1048576 2024-05-30T16:03:44.842 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.843+0000 7f08851138c0 1 > prepared part key=_multipart_MMLF65KLC42Q.2~GGV4NFAXSP39.1 size=1048576 2024-05-30T16:03:44.845 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.843+0000 7f08851138c0 1 > prepared part key=_multipart_MMLF65KLC42Q.2~GGV4NFAXSP39.2 size=738164 2024-05-30T16:03:44.845 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.843+0000 7f08851138c0 1 > prepared multipart upload key=MMLF65KLC42Q upload=GGV4NFAXSP39 tag=RGOXCBFZ3CRM type=write size=3883892 2024-05-30T16:03:44.850 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.851+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~E1T6P5F1M3CR.0 size=1048576 2024-05-30T16:03:44.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.851+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~E1T6P5F1M3CR.1 size=1048576 2024-05-30T16:03:44.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.855+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~E1T6P5F1M3CR.2 size=983263 2024-05-30T16:03:44.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.855+0000 7f08851138c0 1 > prepared multipart upload key=WT9RC0UYVG4F upload=E1T6P5F1M3CR tag=ODEUCBGKZ5F2 type=write size=4128991 2024-05-30T16:03:44.858 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.859+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=QU8YSP8V0X5A type=write size=933955 2024-05-30T16:03:44.861 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.859+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~BE4B2TDAQS1A.0 size=1048576 2024-05-30T16:03:44.864 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.863+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~BE4B2TDAQS1A.1 size=1001567 2024-05-30T16:03:44.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.867+0000 7f08851138c0 1 < completed multipart upload key=TPCETCO3Y1M6 upload=BE4B2TDAQS1A tag=967N3I00W2PX type=write size=3098719 2024-05-30T16:03:44.874 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.875+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~S41ACW2ELGV6.0 size=1048576 2024-05-30T16:03:44.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.875+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~S41ACW2ELGV6.1 size=154049 2024-05-30T16:03:44.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.875+0000 7f08851138c0 1 > prepared multipart upload key=TPCETCO3Y1M6 upload=S41ACW2ELGV6 tag=FPXF6X6RZPHL type=write size=2251201 2024-05-30T16:03:44.883 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.883+0000 7f08851138c0 1 < suggested update operation key=V4OQFOOQUXFQ tag=XAVHGUXLLHYZ type=del size=3305343 2024-05-30T16:03:44.889 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.891+0000 7f08851138c0 1 > prepared part key=_multipart_MMLF65KLC42Q.2~MAHO9JXJ6F48.0 size=1048576 2024-05-30T16:03:44.891 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.891+0000 7f08851138c0 1 > prepared part key=_multipart_MMLF65KLC42Q.2~MAHO9JXJ6F48.1 size=1048576 2024-05-30T16:03:44.894 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.895+0000 7f08851138c0 1 > prepared part key=_multipart_MMLF65KLC42Q.2~MAHO9JXJ6F48.2 size=252827 2024-05-30T16:03:44.894 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.895+0000 7f08851138c0 1 > prepared multipart upload key=MMLF65KLC42Q upload=MAHO9JXJ6F48 tag=01KMIAD2CQRH type=write size=3398555 2024-05-30T16:03:44.898 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.899+0000 7f08851138c0 1 < completed part key=_multipart_ZALVY70ISLM5.2~8AO30B7H2A0Z.0 size=1048576 2024-05-30T16:03:44.900 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.899+0000 7f08851138c0 1 < completed part key=_multipart_ZALVY70ISLM5.2~8AO30B7H2A0Z.1 size=162418 2024-05-30T16:03:44.905 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.907+0000 7f08851138c0 1 < completed multipart upload key=ZALVY70ISLM5 upload=8AO30B7H2A0Z tag=ZAGYHBRM0Y1V type=write size=2259570 2024-05-30T16:03:44.908 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.907+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=1PFH185WRE4M type=write size=1041809 2024-05-30T16:03:44.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.911+0000 7f08851138c0 -1 < failed to complete operation key=V4OQFOOQUXFQ tag=GLZQK9H3EVGL type=write size=422135: (22) Invalid argument 2024-05-30T16:03:44.916 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.915+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~RZDCGV1NWZ33.0 size=1048576 2024-05-30T16:03:44.919 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.919+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~RZDCGV1NWZ33.1 size=891033 2024-05-30T16:03:44.919 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.919+0000 7f08851138c0 1 > prepared multipart upload key=RCTV94WRMO53 upload=RZDCGV1NWZ33 tag=DI3AY35XG5HG type=write size=2988185 2024-05-30T16:03:44.922 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.923+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~8088NU0E2EXI.0 size=1048576 2024-05-30T16:03:44.924 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.923+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~8088NU0E2EXI.1 size=1048576 2024-05-30T16:03:44.927 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.927+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~8088NU0E2EXI.2 size=902100 2024-05-30T16:03:44.931 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.931+0000 7f08851138c0 1 < completed multipart upload key=603HBR8PRH6L upload=8088NU0E2EXI tag=XO41CXYK2JF5 type=write size=4047828 2024-05-30T16:03:44.935 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.935+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=MJR3NSFHDL18 type=del size=3677351 2024-05-30T16:03:44.938 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.939+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~MJOMVA53UBLS.0 size=1048576 2024-05-30T16:03:44.940 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.939+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~MJOMVA53UBLS.1 size=1030539 2024-05-30T16:03:44.945 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.947+0000 7f08851138c0 1 < completed multipart upload key=Y3CYZJ83WYNI upload=MJOMVA53UBLS tag=TBLL1UUB3PO7 type=write size=3127691 2024-05-30T16:03:44.948 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.947+0000 7f08851138c0 1 > prepared operation key=ZALVY70ISLM5 tag=FE8XAU1GXR32 type=del size=2860937 2024-05-30T16:03:44.952 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.951+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~3LQ3CZMB30MY.0 size=359289 2024-05-30T16:03:44.956 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.955+0000 7f08851138c0 1 < completed multipart upload key=PLWFSKPKVO19 upload=3LQ3CZMB30MY tag=QQZBOSR6XZ2J type=write size=1407865 2024-05-30T16:03:44.961 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.963+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~1T3SKT5QM203.0 size=849 2024-05-30T16:03:44.961 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.963+0000 7f08851138c0 1 > prepared multipart upload key=WT9RC0UYVG4F upload=1T3SKT5QM203 tag=CJUKVI0SDYHM type=write size=1049425 2024-05-30T16:03:44.965 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.967+0000 7f08851138c0 1 < completed write operation key=GWXNCSJCLXKW tag=CKQNA181DSBF type=write size=5284 2024-05-30T16:03:44.968 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.967+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=2SH9BL10QDCT type=del size=1397263 2024-05-30T16:03:44.971 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.971+0000 7f08851138c0 1 < canceled operation key=1N6R2HBTWMP2 tag=CJSSBAKO7B5I type=del size=2242547 2024-05-30T16:03:44.974 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.975+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=5EPKMEZNHN66 type=del size=2028682 2024-05-30T16:03:44.980 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.979+0000 7f08851138c0 1 < suggested remove operation key=3IJWPWRB6FHE tag=TLNURF9H0E72 type=del 2024-05-30T16:03:44.984 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.983+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~1LZXHXBTJ0YM.0 size=34029 2024-05-30T16:03:44.985 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.983+0000 7f08851138c0 1 > prepared multipart upload key=JNZI7JX95LX2 upload=1LZXHXBTJ0YM tag=ZQORQ39HMZDR type=write size=1082605 2024-05-30T16:03:44.988 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.987+0000 7f08851138c0 1 < completed delete operation key=1N6R2HBTWMP2 tag=IMCCHOQ3JVWB type=del 2024-05-30T16:03:44.994 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.995+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~FU6X0B7037MV.0 size=828386 2024-05-30T16:03:44.995 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.995+0000 7f08851138c0 1 > prepared multipart upload key=QQILXRJD0C1Z upload=FU6X0B7037MV tag=6HLR5LBL3AC4 type=write size=1876962 2024-05-30T16:03:44.997 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:44.995+0000 7f08851138c0 1 < completed write operation key=Y8SBI0QZ0ZV8 tag=W6Q60QAFM49D type=write size=881478 2024-05-30T16:03:45.001 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.003+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~QG23FA2HVH92.0 size=1048576 2024-05-30T16:03:45.003 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.003+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~QG23FA2HVH92.1 size=331935 2024-05-30T16:03:45.004 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.003+0000 7f08851138c0 1 > prepared multipart upload key=0RZRBW38WKNZ upload=QG23FA2HVH92 tag=Z08FL623UMZU type=write size=2429087 2024-05-30T16:03:45.006 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.007+0000 7f08851138c0 1 < completed delete operation key=OKBB0VU765TW tag=H4NAFU9JL61Z type=del 2024-05-30T16:03:45.012 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.011+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~S41DV1JG9KQY.0 size=793767 2024-05-30T16:03:45.012 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.011+0000 7f08851138c0 1 > prepared multipart upload key=ZALVY70ISLM5 upload=S41DV1JG9KQY tag=VKNDK0PZGC5X type=write size=1842343 2024-05-30T16:03:45.012 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.011+0000 7f08851138c0 1 < canceled part key=_multipart_MMLF65KLC42Q.2~GGV4NFAXSP39.0 size=1048576 2024-05-30T16:03:45.015 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.015+0000 7f08851138c0 1 < completed part key=_multipart_MMLF65KLC42Q.2~GGV4NFAXSP39.1 size=1048576 2024-05-30T16:03:45.017 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.019+0000 7f08851138c0 1 < completed part key=_multipart_MMLF65KLC42Q.2~GGV4NFAXSP39.2 size=738164 2024-05-30T16:03:45.022 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.023+0000 7f08851138c0 1 < canceled multipart upload key=MMLF65KLC42Q upload=GGV4NFAXSP39 tag=RGOXCBFZ3CRM type=write size=3883892 2024-05-30T16:03:45.028 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.027+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~817UTHBLEBRW.0 size=1048576 2024-05-30T16:03:45.030 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.031+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~817UTHBLEBRW.1 size=1048576 2024-05-30T16:03:45.032 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.031+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~817UTHBLEBRW.2 size=372186 2024-05-30T16:03:45.032 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.031+0000 7f08851138c0 1 > prepared multipart upload key=3IJWPWRB6FHE upload=817UTHBLEBRW tag=ZMFONNGRJSBZ type=write size=3517914 2024-05-30T16:03:45.035 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.035+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~E1T6P5F1M3CR.0 size=1048576 2024-05-30T16:03:45.037 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.039+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~E1T6P5F1M3CR.1 size=1048576 2024-05-30T16:03:45.039 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.039+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~E1T6P5F1M3CR.2 size=983263 2024-05-30T16:03:45.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.043+0000 7f08851138c0 1 < completed multipart upload key=WT9RC0UYVG4F upload=E1T6P5F1M3CR tag=ODEUCBGKZ5F2 type=write size=4128991 2024-05-30T16:03:45.048 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.047+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~6Q1DB0H3NR3H.0 size=1048576 2024-05-30T16:03:45.051 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.051+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~6Q1DB0H3NR3H.1 size=1048576 2024-05-30T16:03:45.053 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.051+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~6Q1DB0H3NR3H.2 size=352342 2024-05-30T16:03:45.053 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.051+0000 7f08851138c0 1 > prepared multipart upload key=TPCETCO3Y1M6 upload=6Q1DB0H3NR3H tag=BU5OB7SXWAXW type=write size=3498070 2024-05-30T16:03:45.056 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.055+0000 7f08851138c0 1 < completed write operation key=YRP4CSG92QRO tag=QU8YSP8V0X5A type=write size=933955 2024-05-30T16:03:45.059 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.059+0000 7f08851138c0 1 > prepared operation key=VRXNO8JLAZYH tag=4HWHIWIV7PWR type=del size=1939997 2024-05-30T16:03:45.062 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.063+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~S41ACW2ELGV6.0 size=1048576 2024-05-30T16:03:45.065 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.063+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~S41ACW2ELGV6.1 size=154049 2024-05-30T16:03:45.072 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.071+0000 7f08851138c0 1 < completed multipart upload key=TPCETCO3Y1M6 upload=S41ACW2ELGV6 tag=FPXF6X6RZPHL type=write size=2251201 2024-05-30T16:03:45.075 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.075+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=ZRUA5BZP37XK type=del size=1993392 2024-05-30T16:03:45.078 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.079+0000 7f08851138c0 1 < completed part key=_multipart_MMLF65KLC42Q.2~MAHO9JXJ6F48.0 size=1048576 2024-05-30T16:03:45.080 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.079+0000 7f08851138c0 1 < completed part key=_multipart_MMLF65KLC42Q.2~MAHO9JXJ6F48.1 size=1048576 2024-05-30T16:03:45.083 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.083+0000 7f08851138c0 1 < completed part key=_multipart_MMLF65KLC42Q.2~MAHO9JXJ6F48.2 size=252827 2024-05-30T16:03:45.087 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.087+0000 7f08851138c0 1 < completed multipart upload key=MMLF65KLC42Q upload=MAHO9JXJ6F48 tag=01KMIAD2CQRH type=write size=3398555 2024-05-30T16:03:45.093 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.091+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~9XUUA5R4S4MR.0 size=1048576 2024-05-30T16:03:45.095 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.095+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~9XUUA5R4S4MR.1 size=1048576 2024-05-30T16:03:45.098 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.099+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~9XUUA5R4S4MR.2 size=143199 2024-05-30T16:03:45.098 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.099+0000 7f08851138c0 1 > prepared multipart upload key=NIV7AQ4EV5YP upload=9XUUA5R4S4MR tag=YHJZD2ODBWD2 type=write size=3288927 2024-05-30T16:03:45.101 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.099+0000 7f08851138c0 1 < completed write operation key=73FJ3HGWYQQK tag=1PFH185WRE4M type=write size=1041809 2024-05-30T16:03:45.104 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.103+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=3VBHBAQHEWN1 type=del size=1641659 2024-05-30T16:03:45.107 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.107+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~RZDCGV1NWZ33.0 size=1048576 2024-05-30T16:03:45.110 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.111+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~RZDCGV1NWZ33.1 size=891033 2024-05-30T16:03:45.115 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.115+0000 7f08851138c0 1 < completed multipart upload key=RCTV94WRMO53 upload=RZDCGV1NWZ33 tag=DI3AY35XG5HG type=write size=2988185 2024-05-30T16:03:45.118 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.119+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=HNCY3GI65QMJ type=del size=406295 2024-05-30T16:03:45.120 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.119+0000 7f08851138c0 1 < completed delete operation key=A5BUBY9Z7JQG tag=MJR3NSFHDL18 type=del 2024-05-30T16:03:45.123 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.123+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=201B3AIQ1NFX type=del size=666944 2024-05-30T16:03:45.129 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.127+0000 7f08851138c0 1 < suggested remove operation key=ZALVY70ISLM5 tag=FE8XAU1GXR32 type=del 2024-05-30T16:03:45.132 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.131+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=RSBYTDUINMWN type=del size=1593545 2024-05-30T16:03:45.133 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.131+0000 7f08851138c0 1 < canceled part key=_multipart_WT9RC0UYVG4F.2~1T3SKT5QM203.0 size=849 2024-05-30T16:03:45.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.135+0000 7f08851138c0 1 < canceled multipart upload key=WT9RC0UYVG4F upload=1T3SKT5QM203 tag=CJUKVI0SDYHM type=write size=1049425 2024-05-30T16:03:45.142 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.143+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~S1BR9TZDGT3K.0 size=1048576 2024-05-30T16:03:45.144 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.143+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~S1BR9TZDGT3K.1 size=1048576 2024-05-30T16:03:45.146 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.147+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~S1BR9TZDGT3K.2 size=49323 2024-05-30T16:03:45.146 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.147+0000 7f08851138c0 1 > prepared multipart upload key=Y8SBI0QZ0ZV8 upload=S1BR9TZDGT3K tag=R3UMT0Q520JF type=write size=3195051 2024-05-30T16:03:45.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.151+0000 7f08851138c0 1 < completed delete operation key=O0DJ8SCY766V tag=2SH9BL10QDCT type=del 2024-05-30T16:03:45.152 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.151+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=YGN0VMVZGABN type=del size=487578 2024-05-30T16:03:45.155 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.155+0000 7f08851138c0 1 < completed delete operation key=NIV7AQ4EV5YP tag=5EPKMEZNHN66 type=del 2024-05-30T16:03:45.158 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.159+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=FYF6V47E8L1A type=del size=1517115 2024-05-30T16:03:45.160 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.159+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~1LZXHXBTJ0YM.0 size=34029 2024-05-30T16:03:45.165 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.163+0000 7f08851138c0 1 < completed multipart upload key=JNZI7JX95LX2 upload=1LZXHXBTJ0YM tag=ZQORQ39HMZDR type=write size=1082605 2024-05-30T16:03:45.167 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.167+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=IX7QKXT4LBZ1 type=del size=1035301 2024-05-30T16:03:45.170 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.171+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~FU6X0B7037MV.0 size=828386 2024-05-30T16:03:45.174 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.175+0000 7f08851138c0 1 < completed multipart upload key=QQILXRJD0C1Z upload=FU6X0B7037MV tag=6HLR5LBL3AC4 type=write size=1876962 2024-05-30T16:03:45.180 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.179+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~G6LQ88OY5KTG.0 size=997143 2024-05-30T16:03:45.180 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.179+0000 7f08851138c0 1 > prepared multipart upload key=TPCETCO3Y1M6 upload=G6LQ88OY5KTG tag=1U54IPCKDN5W type=write size=2045719 2024-05-30T16:03:45.183 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.183+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~QG23FA2HVH92.0 size=1048576 2024-05-30T16:03:45.185 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.187+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~QG23FA2HVH92.1 size=331935 2024-05-30T16:03:45.190 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.191+0000 7f08851138c0 1 < completed multipart upload key=0RZRBW38WKNZ upload=QG23FA2HVH92 tag=Z08FL623UMZU type=write size=2429087 2024-05-30T16:03:45.192 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.191+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=8K71I57KMVXF type=del size=141053 2024-05-30T16:03:45.195 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.195+0000 7f08851138c0 1 < completed part key=_multipart_ZALVY70ISLM5.2~S41DV1JG9KQY.0 size=793767 2024-05-30T16:03:45.200 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.199+0000 7f08851138c0 1 < completed multipart upload key=ZALVY70ISLM5 upload=S41DV1JG9KQY tag=VKNDK0PZGC5X type=write size=1842343 2024-05-30T16:03:45.203 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.203+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=VWKGGEGNHXT0 type=del size=3811635 2024-05-30T16:03:45.207 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.207+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~817UTHBLEBRW.0 size=1048576 2024-05-30T16:03:45.209 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.211+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~817UTHBLEBRW.1 size=1048576 2024-05-30T16:03:45.212 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.211+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~817UTHBLEBRW.2 size=372186 2024-05-30T16:03:45.217 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.215+0000 7f08851138c0 1 < completed multipart upload key=3IJWPWRB6FHE upload=817UTHBLEBRW tag=ZMFONNGRJSBZ type=write size=3517914 2024-05-30T16:03:45.220 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.219+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=4XYA70I68EAI type=del size=3442777 2024-05-30T16:03:45.223 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.223+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~6Q1DB0H3NR3H.0 size=1048576 2024-05-30T16:03:45.223 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.223+0000 7f08851138c0 1 < canceled part key=_multipart_TPCETCO3Y1M6.2~6Q1DB0H3NR3H.1 size=1048576 2024-05-30T16:03:45.223 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.223+0000 7f08851138c0 1 < canceled part key=_multipart_TPCETCO3Y1M6.2~6Q1DB0H3NR3H.2 size=352342 2024-05-30T16:03:45.228 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.227+0000 7f08851138c0 1 < canceled multipart upload key=TPCETCO3Y1M6 upload=6Q1DB0H3NR3H tag=BU5OB7SXWAXW type=write size=3498070 2024-05-30T16:03:45.230 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.231+0000 7f08851138c0 1 > prepared operation key=VRXNO8JLAZYH tag=DJ81Y64R6QL1 type=del size=1150038 2024-05-30T16:03:45.233 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.231+0000 7f08851138c0 1 < completed delete operation key=VRXNO8JLAZYH tag=4HWHIWIV7PWR type=del 2024-05-30T16:03:45.236 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.235+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=X6HGW3NDX5EN type=write size=325287 2024-05-30T16:03:45.238 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.239+0000 7f08851138c0 1 < completed delete operation key=QCLY10D2A3VF tag=ZRUA5BZP37XK type=del 2024-05-30T16:03:45.241 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.239+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=8188IIT824LK type=del size=791023 2024-05-30T16:03:45.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.243+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~9XUUA5R4S4MR.0 size=1048576 2024-05-30T16:03:45.247 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.247+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~9XUUA5R4S4MR.1 size=1048576 2024-05-30T16:03:45.249 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.251+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~9XUUA5R4S4MR.2 size=143199 2024-05-30T16:03:45.255 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.255+0000 7f08851138c0 1 < completed multipart upload key=NIV7AQ4EV5YP upload=9XUUA5R4S4MR tag=YHJZD2ODBWD2 type=write size=3288927 2024-05-30T16:03:45.259 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.259+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=EFAG7X9KUZB4 type=del size=3425691 2024-05-30T16:03:45.266 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.267+0000 7f08851138c0 1 < suggested remove operation key=OKBB0VU765TW tag=3VBHBAQHEWN1 type=del 2024-05-30T16:03:45.270 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.271+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=29FNT5BFNMSG type=write size=911899 2024-05-30T16:03:45.273 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.275+0000 7f08851138c0 -1 < failed to complete operation key=OKBB0VU765TW tag=HNCY3GI65QMJ type=del size=406295: (22) Invalid argument 2024-05-30T16:03:45.279 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.279+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~ZNNT6X4NX567.0 size=1048576 2024-05-30T16:03:45.282 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.283+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~ZNNT6X4NX567.1 size=1048576 2024-05-30T16:03:45.285 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.283+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~ZNNT6X4NX567.2 size=490762 2024-05-30T16:03:45.285 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.283+0000 7f08851138c0 1 > prepared multipart upload key=4N58ZY4E1H6E upload=ZNNT6X4NX567 tag=5P6JEQYKBX59 type=write size=3636490 2024-05-30T16:03:45.288 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.287+0000 7f08851138c0 1 < completed delete operation key=V4OQFOOQUXFQ tag=201B3AIQ1NFX type=del 2024-05-30T16:03:45.292 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.291+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=9W3S0U0AVADO type=del size=2846812 2024-05-30T16:03:45.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.299+0000 7f08851138c0 1 < suggested update operation key=XHGMUYBA2VNK tag=RSBYTDUINMWN type=del size=1593545 2024-05-30T16:03:45.302 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.303+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=LWM9CZX34HBG type=del size=2642699 2024-05-30T16:03:45.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.307+0000 7f08851138c0 1 < completed part key=_multipart_Y8SBI0QZ0ZV8.2~S1BR9TZDGT3K.0 size=1048576 2024-05-30T16:03:45.309 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.307+0000 7f08851138c0 1 < completed part key=_multipart_Y8SBI0QZ0ZV8.2~S1BR9TZDGT3K.1 size=1048576 2024-05-30T16:03:45.312 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.311+0000 7f08851138c0 1 < completed part key=_multipart_Y8SBI0QZ0ZV8.2~S1BR9TZDGT3K.2 size=49323 2024-05-30T16:03:45.318 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.319+0000 7f08851138c0 1 < completed multipart upload key=Y8SBI0QZ0ZV8 upload=S1BR9TZDGT3K tag=R3UMT0Q520JF type=write size=3195051 2024-05-30T16:03:45.322 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.323+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=CZ1XXAK4CT8Z type=write size=954545 2024-05-30T16:03:45.325 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.327+0000 7f08851138c0 1 < completed delete operation key=WT9RC0UYVG4F tag=YGN0VMVZGABN type=del 2024-05-30T16:03:45.332 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.331+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~NJJC1EUC6B8U.0 size=1048576 2024-05-30T16:03:45.334 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.335+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~NJJC1EUC6B8U.1 size=1048576 2024-05-30T16:03:45.337 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.335+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~NJJC1EUC6B8U.2 size=518539 2024-05-30T16:03:45.337 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.335+0000 7f08851138c0 1 > prepared multipart upload key=1N6R2HBTWMP2 upload=NJJC1EUC6B8U tag=7CKLVJSTNM5W type=write size=3664267 2024-05-30T16:03:45.340 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.339+0000 7f08851138c0 1 < completed delete operation key=73FJ3HGWYQQK tag=FYF6V47E8L1A type=del 2024-05-30T16:03:45.347 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.347+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~0YNWKZ8LT0XC.0 size=1048576 2024-05-30T16:03:45.349 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.351+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~0YNWKZ8LT0XC.1 size=989587 2024-05-30T16:03:45.350 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.351+0000 7f08851138c0 1 > prepared multipart upload key=RCTV94WRMO53 upload=0YNWKZ8LT0XC tag=QU08PZE0AZX4 type=write size=3086739 2024-05-30T16:03:45.355 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.355+0000 7f08851138c0 1 < completed delete operation key=E2FXSXOEUGIG tag=IX7QKXT4LBZ1 type=del 2024-05-30T16:03:45.359 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.359+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=K82BY3464WZZ type=del size=917690 2024-05-30T16:03:45.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.359+0000 7f08851138c0 1 < canceled part key=_multipart_TPCETCO3Y1M6.2~G6LQ88OY5KTG.0 size=997143 2024-05-30T16:03:45.365 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.363+0000 7f08851138c0 1 < canceled multipart upload key=TPCETCO3Y1M6 upload=G6LQ88OY5KTG tag=1U54IPCKDN5W type=write size=2045719 2024-05-30T16:03:45.372 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.371+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~B4D25FHKKP7L.0 size=1048576 2024-05-30T16:03:45.374 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.375+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~B4D25FHKKP7L.1 size=1048576 2024-05-30T16:03:45.377 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.375+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~B4D25FHKKP7L.2 size=346410 2024-05-30T16:03:45.377 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.375+0000 7f08851138c0 1 > prepared multipart upload key=WT9RC0UYVG4F upload=B4D25FHKKP7L tag=YMKK8YRK1PJN type=write size=3492138 2024-05-30T16:03:45.380 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.379+0000 7f08851138c0 1 < canceled operation key=O0DJ8SCY766V tag=8K71I57KMVXF type=del size=141053 2024-05-30T16:03:45.385 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.387+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~A9P2O2SO179J.0 size=729812 2024-05-30T16:03:45.385 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.387+0000 7f08851138c0 1 > prepared multipart upload key=XHGMUYBA2VNK upload=A9P2O2SO179J tag=Q3BEW3O21W7O type=write size=1778388 2024-05-30T16:03:45.391 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.391+0000 7f08851138c0 1 < suggested remove operation key=QCLY10D2A3VF tag=VWKGGEGNHXT0 type=del 2024-05-30T16:03:45.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.395+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=HL2RAZGJWU7U type=write size=669503 2024-05-30T16:03:45.398 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.399+0000 7f08851138c0 1 < completed delete operation key=364U3OV10VPD tag=4XYA70I68EAI type=del 2024-05-30T16:03:45.401 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.399+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=22P1BBJY2BCF type=write size=464875 2024-05-30T16:03:45.408 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.407+0000 7f08851138c0 1 < suggested update operation key=VRXNO8JLAZYH tag=DJ81Y64R6QL1 type=del size=1150038 2024-05-30T16:03:45.411 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.411+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=TTV1XX5HHPZH type=write size=441238 2024-05-30T16:03:45.415 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.415+0000 7f08851138c0 1 < completed write operation key=MMLF65KLC42Q tag=X6HGW3NDX5EN type=write size=325287 2024-05-30T16:03:45.419 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.419+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=VQPU7TCTXZYQ type=write size=115312 2024-05-30T16:03:45.425 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.423+0000 7f08851138c0 1 < suggested update operation key=YRP4CSG92QRO tag=8188IIT824LK type=del size=791023 2024-05-30T16:03:45.429 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.427+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=XMEPCDHPAR18 type=del size=2424315 2024-05-30T16:03:45.432 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.431+0000 7f08851138c0 1 < completed delete operation key=0RZRBW38WKNZ tag=EFAG7X9KUZB4 type=del 2024-05-30T16:03:45.439 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.439+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~LZFTSQJ6DPQ0.0 size=317476 2024-05-30T16:03:45.439 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.439+0000 7f08851138c0 1 > prepared multipart upload key=PLWFSKPKVO19 upload=LZFTSQJ6DPQ0 tag=45RXZU1AG0KF type=write size=1366052 2024-05-30T16:03:45.446 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.447+0000 7f08851138c0 1 < suggested remove operation key=QQILXRJD0C1Z tag=29FNT5BFNMSG type=write 2024-05-30T16:03:45.450 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.451+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=JSYTG7MY90PU type=del size=2353539 2024-05-30T16:03:45.453 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.455+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~ZNNT6X4NX567.0 size=1048576 2024-05-30T16:03:45.453 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.455+0000 7f08851138c0 1 < canceled part key=_multipart_4N58ZY4E1H6E.2~ZNNT6X4NX567.1 size=1048576 2024-05-30T16:03:45.456 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.455+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~ZNNT6X4NX567.2 size=490762 2024-05-30T16:03:45.461 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.463+0000 7f08851138c0 1 < canceled multipart upload key=4N58ZY4E1H6E upload=ZNNT6X4NX567 tag=5P6JEQYKBX59 type=write size=3636490 2024-05-30T16:03:45.467 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.467+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~Q8TZQYD2UYYW.0 size=1048576 2024-05-30T16:03:45.470 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.471+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~Q8TZQYD2UYYW.1 size=1048576 2024-05-30T16:03:45.473 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.471+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~Q8TZQYD2UYYW.2 size=288966 2024-05-30T16:03:45.473 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.471+0000 7f08851138c0 1 > prepared multipart upload key=V4OQFOOQUXFQ upload=Q8TZQYD2UYYW tag=K1UKI4J4AI2G type=write size=3434694 2024-05-30T16:03:45.480 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.479+0000 7f08851138c0 1 < suggested remove operation key=0T2E9F3K1USF tag=9W3S0U0AVADO type=del 2024-05-30T16:03:45.483 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.483+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=9IR6KMLAVPIY type=del size=4071245 2024-05-30T16:03:45.486 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.487+0000 7f08851138c0 1 < canceled operation key=V4OQFOOQUXFQ tag=LWM9CZX34HBG type=del size=2642699 2024-05-30T16:03:45.492 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.491+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~CLXBCTHQUD12.0 size=692213 2024-05-30T16:03:45.492 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.491+0000 7f08851138c0 1 > prepared multipart upload key=YRP4CSG92QRO upload=CLXBCTHQUD12 tag=P983W0EPVKXJ type=write size=1740789 2024-05-30T16:03:45.496 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.495+0000 7f08851138c0 1 < completed write operation key=3IJWPWRB6FHE tag=CZ1XXAK4CT8Z type=write size=954545 2024-05-30T16:03:45.502 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.503+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~GTCCRCJW8LGP.0 size=1048576 2024-05-30T16:03:45.505 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.503+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~GTCCRCJW8LGP.1 size=1048576 2024-05-30T16:03:45.508 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.507+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~GTCCRCJW8LGP.2 size=742281 2024-05-30T16:03:45.508 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.507+0000 7f08851138c0 1 > prepared multipart upload key=364U3OV10VPD upload=GTCCRCJW8LGP tag=NTKR7SQZDW9D type=write size=3888009 2024-05-30T16:03:45.511 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.511+0000 7f08851138c0 1 < completed part key=_multipart_1N6R2HBTWMP2.2~NJJC1EUC6B8U.0 size=1048576 2024-05-30T16:03:45.514 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.515+0000 7f08851138c0 1 < completed part key=_multipart_1N6R2HBTWMP2.2~NJJC1EUC6B8U.1 size=1048576 2024-05-30T16:03:45.517 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.515+0000 7f08851138c0 1 < completed part key=_multipart_1N6R2HBTWMP2.2~NJJC1EUC6B8U.2 size=518539 2024-05-30T16:03:45.523 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.523+0000 7f08851138c0 1 < completed multipart upload key=1N6R2HBTWMP2 upload=NJJC1EUC6B8U tag=7CKLVJSTNM5W type=write size=3664267 2024-05-30T16:03:45.530 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.531+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~2SN80W632BOA.0 size=1048576 2024-05-30T16:03:45.533 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.531+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~2SN80W632BOA.1 size=490158 2024-05-30T16:03:45.533 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.531+0000 7f08851138c0 1 > prepared multipart upload key=73FJ3HGWYQQK upload=2SN80W632BOA tag=I1WPQZSUFI0D type=write size=2587310 2024-05-30T16:03:45.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.535+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~0YNWKZ8LT0XC.0 size=1048576 2024-05-30T16:03:45.539 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.539+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~0YNWKZ8LT0XC.1 size=989587 2024-05-30T16:03:45.544 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.543+0000 7f08851138c0 1 < completed multipart upload key=RCTV94WRMO53 upload=0YNWKZ8LT0XC tag=QU08PZE0AZX4 type=write size=3086739 2024-05-30T16:03:45.550 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.551+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~HUZJKF4CWZZZ.0 size=1048576 2024-05-30T16:03:45.554 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.555+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~HUZJKF4CWZZZ.1 size=1048576 2024-05-30T16:03:45.557 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.555+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~HUZJKF4CWZZZ.2 size=853508 2024-05-30T16:03:45.557 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.555+0000 7f08851138c0 1 > prepared multipart upload key=JNZI7JX95LX2 upload=HUZJKF4CWZZZ tag=5PWUQXJ0H1H2 type=write size=3999236 2024-05-30T16:03:45.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.559+0000 7f08851138c0 1 < canceled operation key=Y3CYZJ83WYNI tag=K82BY3464WZZ type=del size=917690 2024-05-30T16:03:45.564 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.563+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=NLP27DHFD4UB type=del size=314907 2024-05-30T16:03:45.567 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.567+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~B4D25FHKKP7L.0 size=1048576 2024-05-30T16:03:45.570 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.571+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~B4D25FHKKP7L.1 size=1048576 2024-05-30T16:03:45.572 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.571+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~B4D25FHKKP7L.2 size=346410 2024-05-30T16:03:45.579 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.579+0000 7f08851138c0 1 < completed multipart upload key=WT9RC0UYVG4F upload=B4D25FHKKP7L tag=YMKK8YRK1PJN type=write size=3492138 2024-05-30T16:03:45.582 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.583+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=W45P1BV2BJY8 type=write size=861391 2024-05-30T16:03:45.587 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.587+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~A9P2O2SO179J.0 size=729812 2024-05-30T16:03:45.593 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.591+0000 7f08851138c0 1 < completed multipart upload key=XHGMUYBA2VNK upload=A9P2O2SO179J tag=Q3BEW3O21W7O type=write size=1778388 2024-05-30T16:03:45.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.595+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=8CPHYBLUTAK4 type=del size=1325389 2024-05-30T16:03:45.600 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.599+0000 7f08851138c0 1 < completed write operation key=RCTV94WRMO53 tag=HL2RAZGJWU7U type=write size=669503 2024-05-30T16:03:45.606 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.607+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~QJ6FOVSH0BNY.0 size=1048576 2024-05-30T16:03:45.609 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.611+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~QJ6FOVSH0BNY.1 size=631959 2024-05-30T16:03:45.610 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.611+0000 7f08851138c0 1 > prepared multipart upload key=E2FXSXOEUGIG upload=QJ6FOVSH0BNY tag=WNM9UDZWVERB type=write size=2729111 2024-05-30T16:03:45.613 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.611+0000 7f08851138c0 1 < completed write operation key=MMLF65KLC42Q tag=22P1BBJY2BCF type=write size=464875 2024-05-30T16:03:45.617 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.615+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=8R0MNZQY9P3B type=del size=2334682 2024-05-30T16:03:45.622 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.623+0000 7f08851138c0 1 < canceled operation key=3IJWPWRB6FHE tag=TTV1XX5HHPZH type=write size=441238 2024-05-30T16:03:45.628 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.627+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~CDGKYJRM4LON.0 size=1048576 2024-05-30T16:03:45.631 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.631+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~CDGKYJRM4LON.1 size=1048576 2024-05-30T16:03:45.633 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.635+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~CDGKYJRM4LON.2 size=285050 2024-05-30T16:03:45.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.635+0000 7f08851138c0 1 > prepared multipart upload key=OKBB0VU765TW upload=CDGKYJRM4LON tag=X4UI5E2F7Q2R type=write size=3430778 2024-05-30T16:03:45.637 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.635+0000 7f08851138c0 1 < completed write operation key=WT9RC0UYVG4F tag=VQPU7TCTXZYQ type=write size=115312 2024-05-30T16:03:45.643 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.643+0000 7f08851138c0 1 > prepared part key=_multipart_MMLF65KLC42Q.2~NCOCAK4S8OEF.0 size=1048576 2024-05-30T16:03:45.646 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.647+0000 7f08851138c0 1 > prepared part key=_multipart_MMLF65KLC42Q.2~NCOCAK4S8OEF.1 size=1048576 2024-05-30T16:03:45.648 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.647+0000 7f08851138c0 1 > prepared part key=_multipart_MMLF65KLC42Q.2~NCOCAK4S8OEF.2 size=785608 2024-05-30T16:03:45.648 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.647+0000 7f08851138c0 1 > prepared multipart upload key=MMLF65KLC42Q upload=NCOCAK4S8OEF tag=M1QQ1N5NTXOP type=write size=3931336 2024-05-30T16:03:45.652 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.651+0000 7f08851138c0 1 < completed delete operation key=QCLY10D2A3VF tag=XMEPCDHPAR18 type=del 2024-05-30T16:03:45.656 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.655+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=UHQX0LXT0LIX type=del size=3684585 2024-05-30T16:03:45.659 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.659+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~LZFTSQJ6DPQ0.0 size=317476 2024-05-30T16:03:45.665 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.667+0000 7f08851138c0 1 < completed multipart upload key=PLWFSKPKVO19 upload=LZFTSQJ6DPQ0 tag=45RXZU1AG0KF type=write size=1366052 2024-05-30T16:03:45.672 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.671+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~ACEFQGO2BXUE.0 size=1048576 2024-05-30T16:03:45.677 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.679+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~ACEFQGO2BXUE.1 size=931412 2024-05-30T16:03:45.677 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.679+0000 7f08851138c0 1 > prepared multipart upload key=YRP4CSG92QRO upload=ACEFQGO2BXUE tag=OZ1GM3FGMBE5 type=write size=3028564 2024-05-30T16:03:45.681 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.679+0000 7f08851138c0 1 < canceled operation key=MMLF65KLC42Q tag=JSYTG7MY90PU type=del size=2353539 2024-05-30T16:03:45.685 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.683+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=527ADCL1CGXX type=del size=1136714 2024-05-30T16:03:45.685 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.687+0000 7f08851138c0 1 < canceled part key=_multipart_V4OQFOOQUXFQ.2~Q8TZQYD2UYYW.0 size=1048576 2024-05-30T16:03:45.688 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.687+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~Q8TZQYD2UYYW.1 size=1048576 2024-05-30T16:03:45.691 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.691+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~Q8TZQYD2UYYW.2 size=288966 2024-05-30T16:03:45.697 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.695+0000 7f08851138c0 1 < canceled multipart upload key=V4OQFOOQUXFQ upload=Q8TZQYD2UYYW tag=K1UKI4J4AI2G type=write size=3434694 2024-05-30T16:03:45.700 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.699+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=7QKVV1IX27Z7 type=del size=805663 2024-05-30T16:03:45.703 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.703+0000 7f08851138c0 1 < canceled operation key=WT9RC0UYVG4F tag=9IR6KMLAVPIY type=del size=4071245 2024-05-30T16:03:45.707 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.707+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=Y1K38OQQY57Z type=write size=717207 2024-05-30T16:03:45.710 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.711+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~CLXBCTHQUD12.0 size=692213 2024-05-30T16:03:45.715 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.715+0000 7f08851138c0 1 < completed multipart upload key=YRP4CSG92QRO upload=CLXBCTHQUD12 tag=P983W0EPVKXJ type=write size=1740789 2024-05-30T16:03:45.722 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.723+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~U9P347EKHVIK.0 size=1048576 2024-05-30T16:03:45.724 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.723+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~U9P347EKHVIK.1 size=148303 2024-05-30T16:03:45.724 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.723+0000 7f08851138c0 1 > prepared multipart upload key=O0DJ8SCY766V upload=U9P347EKHVIK tag=EBO3VRLKE63Z type=write size=2245455 2024-05-30T16:03:45.727 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.727+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~GTCCRCJW8LGP.0 size=1048576 2024-05-30T16:03:45.730 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.731+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~GTCCRCJW8LGP.1 size=1048576 2024-05-30T16:03:45.733 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.731+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~GTCCRCJW8LGP.2 size=742281 2024-05-30T16:03:45.741 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.739+0000 7f08851138c0 1 < completed multipart upload key=364U3OV10VPD upload=GTCCRCJW8LGP tag=NTKR7SQZDW9D type=write size=3888009 2024-05-30T16:03:45.762 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.763+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~B619BOBE6U9Q.0 size=264603 2024-05-30T16:03:45.763 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.763+0000 7f08851138c0 1 > prepared multipart upload key=Y8SBI0QZ0ZV8 upload=B619BOBE6U9Q tag=GO2LG5URX0IK type=write size=1313179 2024-05-30T16:03:45.768 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.767+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~2SN80W632BOA.0 size=1048576 2024-05-30T16:03:45.771 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.771+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~2SN80W632BOA.1 size=490158 2024-05-30T16:03:45.776 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.775+0000 7f08851138c0 1 < completed multipart upload key=73FJ3HGWYQQK upload=2SN80W632BOA tag=I1WPQZSUFI0D type=write size=2587310 2024-05-30T16:03:45.782 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.783+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~3IT8NQBYKOC6.0 size=782846 2024-05-30T16:03:45.782 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.783+0000 7f08851138c0 1 > prepared multipart upload key=1N6R2HBTWMP2 upload=3IT8NQBYKOC6 tag=9K1KQ8Y2W3DH type=write size=1831422 2024-05-30T16:03:45.785 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.783+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~HUZJKF4CWZZZ.0 size=1048576 2024-05-30T16:03:45.787 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.787+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~HUZJKF4CWZZZ.1 size=1048576 2024-05-30T16:03:45.789 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.791+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~HUZJKF4CWZZZ.2 size=853508 2024-05-30T16:03:45.794 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.795+0000 7f08851138c0 1 < completed multipart upload key=JNZI7JX95LX2 upload=HUZJKF4CWZZZ tag=5PWUQXJ0H1H2 type=write size=3999236 2024-05-30T16:03:45.797 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.795+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=UJJ33YPTC88R type=del size=528047 2024-05-30T16:03:45.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.799+0000 7f08851138c0 1 < completed delete operation key=Y8SBI0QZ0ZV8 tag=NLP27DHFD4UB type=del 2024-05-30T16:03:45.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.803+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=2A6R4SLU14C1 type=del size=183061 2024-05-30T16:03:45.808 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.807+0000 7f08851138c0 1 < suggested update operation key=E2FXSXOEUGIG tag=W45P1BV2BJY8 type=write size=861391 2024-05-30T16:03:45.812 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.811+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=DVUGWT0KHHCM type=del size=3507713 2024-05-30T16:03:45.815 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.815+0000 7f08851138c0 1 < completed delete operation key=Y8SBI0QZ0ZV8 tag=8CPHYBLUTAK4 type=del 2024-05-30T16:03:45.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.819+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~8VV66MD3Y3NV.0 size=65899 2024-05-30T16:03:45.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.819+0000 7f08851138c0 1 > prepared multipart upload key=KPHZGFJLC2E0 upload=8VV66MD3Y3NV tag=5KN2P1YU90A6 type=write size=1114475 2024-05-30T16:03:45.821 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.819+0000 7f08851138c0 1 < canceled part key=_multipart_E2FXSXOEUGIG.2~QJ6FOVSH0BNY.0 size=1048576 2024-05-30T16:03:45.823 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.823+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~QJ6FOVSH0BNY.1 size=631959 2024-05-30T16:03:45.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.827+0000 7f08851138c0 1 < canceled multipart upload key=E2FXSXOEUGIG upload=QJ6FOVSH0BNY tag=WNM9UDZWVERB type=write size=2729111 2024-05-30T16:03:45.831 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.831+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=8IPU3E0PZXYO type=del size=2734869 2024-05-30T16:03:45.834 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.835+0000 7f08851138c0 1 < completed delete operation key=WT9RC0UYVG4F tag=8R0MNZQY9P3B type=del 2024-05-30T16:03:45.840 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.839+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~0P20ZG9WYZZQ.0 size=1048576 2024-05-30T16:03:45.842 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.843+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~0P20ZG9WYZZQ.1 size=1016055 2024-05-30T16:03:45.842 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.843+0000 7f08851138c0 1 > prepared multipart upload key=A5BUBY9Z7JQG upload=0P20ZG9WYZZQ tag=SMAAW27DYC44 type=write size=3113207 2024-05-30T16:03:45.845 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.847+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~CDGKYJRM4LON.0 size=1048576 2024-05-30T16:03:45.848 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.847+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~CDGKYJRM4LON.1 size=1048576 2024-05-30T16:03:45.851 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.851+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~CDGKYJRM4LON.2 size=285050 2024-05-30T16:03:45.856 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.855+0000 7f08851138c0 1 < completed multipart upload key=OKBB0VU765TW upload=CDGKYJRM4LON tag=X4UI5E2F7Q2R type=write size=3430778 2024-05-30T16:03:45.862 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.863+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~WRDBT0WI11CG.0 size=1048576 2024-05-30T16:03:45.864 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.863+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~WRDBT0WI11CG.1 size=76139 2024-05-30T16:03:45.864 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.863+0000 7f08851138c0 1 > prepared multipart upload key=603HBR8PRH6L upload=WRDBT0WI11CG tag=D376WV1ID4QU type=write size=2173291 2024-05-30T16:03:45.867 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.867+0000 7f08851138c0 1 < completed part key=_multipart_MMLF65KLC42Q.2~NCOCAK4S8OEF.0 size=1048576 2024-05-30T16:03:45.870 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.871+0000 7f08851138c0 1 < completed part key=_multipart_MMLF65KLC42Q.2~NCOCAK4S8OEF.1 size=1048576 2024-05-30T16:03:45.873 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.871+0000 7f08851138c0 1 < completed part key=_multipart_MMLF65KLC42Q.2~NCOCAK4S8OEF.2 size=785608 2024-05-30T16:03:45.878 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.879+0000 7f08851138c0 1 < completed multipart upload key=MMLF65KLC42Q upload=NCOCAK4S8OEF tag=M1QQ1N5NTXOP type=write size=3931336 2024-05-30T16:03:45.881 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.879+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=CONOPN5U1XOF type=del size=1636453 2024-05-30T16:03:45.884 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.883+0000 7f08851138c0 1 < completed delete operation key=1N6R2HBTWMP2 tag=UHQX0LXT0LIX type=del 2024-05-30T16:03:45.887 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.887+0000 7f08851138c0 1 > prepared operation key=ZALVY70ISLM5 tag=MWJ79TXNDUKE type=del size=2359137 2024-05-30T16:03:45.890 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.891+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~ACEFQGO2BXUE.0 size=1048576 2024-05-30T16:03:45.893 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.891+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~ACEFQGO2BXUE.1 size=931412 2024-05-30T16:03:45.899 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.899+0000 7f08851138c0 1 < completed multipart upload key=YRP4CSG92QRO upload=ACEFQGO2BXUE tag=OZ1GM3FGMBE5 type=write size=3028564 2024-05-30T16:03:45.902 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.903+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=5SMALETOYAX2 type=del size=3977725 2024-05-30T16:03:45.904 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.903+0000 7f08851138c0 1 < canceled operation key=V4OQFOOQUXFQ tag=527ADCL1CGXX type=del size=1136714 2024-05-30T16:03:45.910 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.911+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~UQIL6YT9L9RB.0 size=1048576 2024-05-30T16:03:45.912 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.911+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~UQIL6YT9L9RB.1 size=70949 2024-05-30T16:03:45.912 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.911+0000 7f08851138c0 1 > prepared multipart upload key=QQILXRJD0C1Z upload=UQIL6YT9L9RB tag=50E0JGO6UXWT type=write size=2168101 2024-05-30T16:03:45.916 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.915+0000 7f08851138c0 1 < completed delete operation key=KPHZGFJLC2E0 tag=7QKVV1IX27Z7 type=del 2024-05-30T16:03:45.921 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.923+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~2ITAEUEI35NM.0 size=965752 2024-05-30T16:03:45.921 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.923+0000 7f08851138c0 1 > prepared multipart upload key=Y8SBI0QZ0ZV8 upload=2ITAEUEI35NM tag=21CEP959WIB7 type=write size=2014328 2024-05-30T16:03:45.924 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.923+0000 7f08851138c0 1 < completed write operation key=3TBOZWT2P8U8 tag=Y1K38OQQY57Z type=write size=717207 2024-05-30T16:03:45.927 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.927+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=UNRVX9S3RHOG type=del size=3944942 2024-05-30T16:03:45.931 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.931+0000 7f08851138c0 1 < completed part key=_multipart_O0DJ8SCY766V.2~U9P347EKHVIK.0 size=1048576 2024-05-30T16:03:45.933 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.935+0000 7f08851138c0 1 < completed part key=_multipart_O0DJ8SCY766V.2~U9P347EKHVIK.1 size=148303 2024-05-30T16:03:45.939 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.939+0000 7f08851138c0 1 < completed multipart upload key=O0DJ8SCY766V upload=U9P347EKHVIK tag=EBO3VRLKE63Z type=write size=2245455 2024-05-30T16:03:45.942 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.943+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=OJ4Y3L47FML3 type=del size=1233163 2024-05-30T16:03:45.945 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.947+0000 7f08851138c0 1 < completed part key=_multipart_Y8SBI0QZ0ZV8.2~B619BOBE6U9Q.0 size=264603 2024-05-30T16:03:45.951 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.951+0000 7f08851138c0 1 < completed multipart upload key=Y8SBI0QZ0ZV8 upload=B619BOBE6U9Q tag=GO2LG5URX0IK type=write size=1313179 2024-05-30T16:03:45.954 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.955+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=KX7D9AB83XPT type=write size=382110 2024-05-30T16:03:45.958 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.959+0000 7f08851138c0 1 < completed part key=_multipart_1N6R2HBTWMP2.2~3IT8NQBYKOC6.0 size=782846 2024-05-30T16:03:45.964 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.963+0000 7f08851138c0 1 < completed multipart upload key=1N6R2HBTWMP2 upload=3IT8NQBYKOC6 tag=9K1KQ8Y2W3DH type=write size=1831422 2024-05-30T16:03:45.970 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.971+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~J7ULLRVW8P5B.0 size=1048576 2024-05-30T16:03:45.973 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.971+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~J7ULLRVW8P5B.1 size=591257 2024-05-30T16:03:45.973 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.971+0000 7f08851138c0 1 > prepared multipart upload key=603HBR8PRH6L upload=J7ULLRVW8P5B tag=UPPKHU8EFUNI type=write size=2688409 2024-05-30T16:03:45.976 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.975+0000 7f08851138c0 1 < completed delete operation key=RCTV94WRMO53 tag=UJJ33YPTC88R type=del 2024-05-30T16:03:45.980 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.979+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=4OCQRMIWI2MU type=del size=3178452 2024-05-30T16:03:45.984 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.983+0000 7f08851138c0 1 < completed delete operation key=0RZRBW38WKNZ tag=2A6R4SLU14C1 type=del 2024-05-30T16:03:45.987 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.987+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=AVYQ2P05UWL3 type=del size=3494663 2024-05-30T16:03:45.995 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:45.995+0000 7f08851138c0 1 < suggested update operation key=1N6R2HBTWMP2 tag=DVUGWT0KHHCM type=del size=3507713 2024-05-30T16:03:46.003 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.003+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~94Y2R17Q5QEQ.0 size=1048576 2024-05-30T16:03:46.006 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.003+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~94Y2R17Q5QEQ.1 size=664091 2024-05-30T16:03:46.006 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.003+0000 7f08851138c0 1 > prepared multipart upload key=KPHZGFJLC2E0 upload=94Y2R17Q5QEQ tag=ITAY230TOI7P type=write size=2761243 2024-05-30T16:03:46.009 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.007+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~8VV66MD3Y3NV.0 size=65899 2024-05-30T16:03:46.014 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.015+0000 7f08851138c0 1 < completed multipart upload key=KPHZGFJLC2E0 upload=8VV66MD3Y3NV tag=5KN2P1YU90A6 type=write size=1114475 2024-05-30T16:03:46.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.019+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~WW6OSBFTUXIV.0 size=1048576 2024-05-30T16:03:46.023 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.023+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~WW6OSBFTUXIV.1 size=232964 2024-05-30T16:03:46.023 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.023+0000 7f08851138c0 1 > prepared multipart upload key=364U3OV10VPD upload=WW6OSBFTUXIV tag=WK4PW14240UD type=write size=2330116 2024-05-30T16:03:46.027 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.027+0000 7f08851138c0 1 < completed delete operation key=PLWFSKPKVO19 tag=8IPU3E0PZXYO type=del 2024-05-30T16:03:46.030 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.031+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=61Y46ADUS6LI type=write size=112505 2024-05-30T16:03:46.034 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.035+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~0P20ZG9WYZZQ.0 size=1048576 2024-05-30T16:03:46.036 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.035+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~0P20ZG9WYZZQ.1 size=1016055 2024-05-30T16:03:46.043 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.043+0000 7f08851138c0 1 < completed multipart upload key=A5BUBY9Z7JQG upload=0P20ZG9WYZZQ tag=SMAAW27DYC44 type=write size=3113207 2024-05-30T16:03:46.047 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.047+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=W332WD3U5ROY type=del size=3042538 2024-05-30T16:03:46.051 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.051+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~WRDBT0WI11CG.0 size=1048576 2024-05-30T16:03:46.054 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.055+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~WRDBT0WI11CG.1 size=76139 2024-05-30T16:03:46.061 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.059+0000 7f08851138c0 1 < completed multipart upload key=603HBR8PRH6L upload=WRDBT0WI11CG tag=D376WV1ID4QU type=write size=2173291 2024-05-30T16:03:46.065 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.063+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=Q482FSFHWCEY type=write size=445333 2024-05-30T16:03:46.068 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.067+0000 7f08851138c0 1 < canceled operation key=XHGMUYBA2VNK tag=CONOPN5U1XOF type=del size=1636453 2024-05-30T16:03:46.075 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.075+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~T9T657AWJNHM.0 size=1048576 2024-05-30T16:03:46.078 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.079+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~T9T657AWJNHM.1 size=1048576 2024-05-30T16:03:46.081 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.079+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~T9T657AWJNHM.2 size=838102 2024-05-30T16:03:46.081 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.079+0000 7f08851138c0 1 > prepared multipart upload key=YRP4CSG92QRO upload=T9T657AWJNHM tag=TAE4GYZGPSCN type=write size=3983830 2024-05-30T16:03:46.084 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.083+0000 7f08851138c0 1 < completed delete operation key=ZALVY70ISLM5 tag=MWJ79TXNDUKE type=del 2024-05-30T16:03:46.088 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.087+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=EOYIY75V9QX0 type=del size=74847 2024-05-30T16:03:46.092 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.091+0000 7f08851138c0 1 < completed delete operation key=O8CWPJ9GW2GO tag=5SMALETOYAX2 type=del 2024-05-30T16:03:46.096 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.095+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=NZ99FIBMJ04G type=del size=1390897 2024-05-30T16:03:46.100 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.099+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~UQIL6YT9L9RB.0 size=1048576 2024-05-30T16:03:46.103 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.103+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~UQIL6YT9L9RB.1 size=70949 2024-05-30T16:03:46.109 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.107+0000 7f08851138c0 1 < completed multipart upload key=QQILXRJD0C1Z upload=UQIL6YT9L9RB tag=50E0JGO6UXWT type=write size=2168101 2024-05-30T16:03:46.112 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.111+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=XENYT1I0BZSA type=write size=270297 2024-05-30T16:03:46.116 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.115+0000 7f08851138c0 1 < completed part key=_multipart_Y8SBI0QZ0ZV8.2~2ITAEUEI35NM.0 size=965752 2024-05-30T16:03:46.122 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.123+0000 7f08851138c0 1 < completed multipart upload key=Y8SBI0QZ0ZV8 upload=2ITAEUEI35NM tag=21CEP959WIB7 type=write size=2014328 2024-05-30T16:03:46.126 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.127+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=VCTCAWBX0CIS type=del size=575921 2024-05-30T16:03:46.133 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.131+0000 7f08851138c0 1 < suggested remove operation key=0RZRBW38WKNZ tag=UNRVX9S3RHOG type=del 2024-05-30T16:03:46.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.135+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=CFG77Z9TI3Y7 type=del size=2168177 2024-05-30T16:03:46.144 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.143+0000 7f08851138c0 1 < suggested update operation key=Y3CYZJ83WYNI tag=OJ4Y3L47FML3 type=del size=1233163 2024-05-30T16:03:46.148 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.147+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=RX5BOPSUDJZU type=del size=4008243 2024-05-30T16:03:46.152 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.151+0000 7f08851138c0 1 < canceled operation key=A5BUBY9Z7JQG tag=KX7D9AB83XPT type=write size=382110 2024-05-30T16:03:46.158 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.159+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~W3QEXUAB2E5T.0 size=1048576 2024-05-30T16:03:46.161 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.163+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~W3QEXUAB2E5T.1 size=1048576 2024-05-30T16:03:46.164 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.163+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~W3QEXUAB2E5T.2 size=643605 2024-05-30T16:03:46.164 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.163+0000 7f08851138c0 1 > prepared multipart upload key=0T2E9F3K1USF upload=W3QEXUAB2E5T tag=62F2BMUWQB4G type=write size=3789333 2024-05-30T16:03:46.168 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.167+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~J7ULLRVW8P5B.0 size=1048576 2024-05-30T16:03:46.171 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.171+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~J7ULLRVW8P5B.1 size=591257 2024-05-30T16:03:46.177 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.179+0000 7f08851138c0 1 < completed multipart upload key=603HBR8PRH6L upload=J7ULLRVW8P5B tag=UPPKHU8EFUNI type=write size=2688409 2024-05-30T16:03:46.181 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.183+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=6243D95PX89E type=del size=1851924 2024-05-30T16:03:46.185 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.183+0000 7f08851138c0 1 < completed delete operation key=QQILXRJD0C1Z tag=4OCQRMIWI2MU type=del 2024-05-30T16:03:46.189 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.187+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=W624NFQVQ9SH type=del size=1361304 2024-05-30T16:03:46.196 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.195+0000 7f08851138c0 1 < suggested remove operation key=MMLF65KLC42Q tag=AVYQ2P05UWL3 type=del 2024-05-30T16:03:46.203 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.203+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~87LD8N716PJN.0 size=124596 2024-05-30T16:03:46.203 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.203+0000 7f08851138c0 1 > prepared multipart upload key=E2FXSXOEUGIG upload=87LD8N716PJN tag=TJZ5U5ZSODNN type=write size=1173172 2024-05-30T16:03:46.206 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.207+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~94Y2R17Q5QEQ.0 size=1048576 2024-05-30T16:03:46.206 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.207+0000 7f08851138c0 1 < canceled part key=_multipart_KPHZGFJLC2E0.2~94Y2R17Q5QEQ.1 size=664091 2024-05-30T16:03:46.212 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.211+0000 7f08851138c0 1 < canceled multipart upload key=KPHZGFJLC2E0 upload=94Y2R17Q5QEQ tag=ITAY230TOI7P type=write size=2761243 2024-05-30T16:03:46.219 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.219+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~A2ZXFLFMQNLN.0 size=1048576 2024-05-30T16:03:46.222 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.223+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~A2ZXFLFMQNLN.1 size=656445 2024-05-30T16:03:46.222 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.223+0000 7f08851138c0 1 > prepared multipart upload key=A5BUBY9Z7JQG upload=A2ZXFLFMQNLN tag=BWQXRCL80G6I type=write size=2753597 2024-05-30T16:03:46.226 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.227+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~WW6OSBFTUXIV.0 size=1048576 2024-05-30T16:03:46.230 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.231+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~WW6OSBFTUXIV.1 size=232964 2024-05-30T16:03:46.236 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.235+0000 7f08851138c0 1 < completed multipart upload key=364U3OV10VPD upload=WW6OSBFTUXIV tag=WK4PW14240UD type=write size=2330116 2024-05-30T16:03:46.240 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.239+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=2ZBFJJ4UAPJE type=del size=1642257 2024-05-30T16:03:46.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.243+0000 7f08851138c0 1 < completed write operation key=PLWFSKPKVO19 tag=61Y46ADUS6LI type=write size=112505 2024-05-30T16:03:46.248 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.247+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=5DIKH3EICR7G type=del size=573331 2024-05-30T16:03:46.251 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.251+0000 7f08851138c0 1 < completed delete operation key=KPHZGFJLC2E0 tag=W332WD3U5ROY type=del 2024-05-30T16:03:46.257 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.259+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~8BORX3H0N6CF.0 size=903977 2024-05-30T16:03:46.257 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.259+0000 7f08851138c0 1 > prepared multipart upload key=KPHZGFJLC2E0 upload=8BORX3H0N6CF tag=EHOTK530MHNA type=write size=1952553 2024-05-30T16:03:46.261 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.259+0000 7f08851138c0 -1 < failed to complete operation key=Y3CYZJ83WYNI tag=Q482FSFHWCEY type=write size=445333: (22) Invalid argument 2024-05-30T16:03:46.265 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.263+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=4GXR1QSPGXR1 type=write size=533065 2024-05-30T16:03:46.270 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.271+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~T9T657AWJNHM.0 size=1048576 2024-05-30T16:03:46.273 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.271+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~T9T657AWJNHM.1 size=1048576 2024-05-30T16:03:46.273 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.271+0000 7f08851138c0 1 < canceled part key=_multipart_YRP4CSG92QRO.2~T9T657AWJNHM.2 size=838102 2024-05-30T16:03:46.279 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.279+0000 7f08851138c0 1 < canceled multipart upload key=YRP4CSG92QRO upload=T9T657AWJNHM tag=TAE4GYZGPSCN type=write size=3983830 2024-05-30T16:03:46.282 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.283+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=WCULF4QYNUGD type=del size=1793725 2024-05-30T16:03:46.286 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.287+0000 7f08851138c0 1 < completed delete operation key=603HBR8PRH6L tag=EOYIY75V9QX0 type=del 2024-05-30T16:03:46.292 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.291+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~RD0N0RW5CP3V.0 size=448807 2024-05-30T16:03:46.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.291+0000 7f08851138c0 1 > prepared multipart upload key=RCTV94WRMO53 upload=RD0N0RW5CP3V tag=T1DJ0SQQDX50 type=write size=1497383 2024-05-30T16:03:46.296 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.295+0000 7f08851138c0 1 < completed delete operation key=WT9RC0UYVG4F tag=NZ99FIBMJ04G type=del 2024-05-30T16:03:46.303 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.303+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~B1MDBIFKJZX3.0 size=1048576 2024-05-30T16:03:46.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.307+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~B1MDBIFKJZX3.1 size=886627 2024-05-30T16:03:46.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.307+0000 7f08851138c0 1 > prepared multipart upload key=A5BUBY9Z7JQG upload=B1MDBIFKJZX3 tag=JE52KRGANLFZ type=write size=2983779 2024-05-30T16:03:46.310 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.311+0000 7f08851138c0 1 < completed write operation key=E2FXSXOEUGIG tag=XENYT1I0BZSA type=write size=270297 2024-05-30T16:03:46.313 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.311+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=4I915RW2T300 type=write size=159819 2024-05-30T16:03:46.320 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.319+0000 7f08851138c0 1 < suggested update operation key=O8CWPJ9GW2GO tag=VCTCAWBX0CIS type=del size=575921 2024-05-30T16:03:46.324 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.323+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=X1UURSG3ELM5 type=write size=985172 2024-05-30T16:03:46.328 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.327+0000 7f08851138c0 1 < completed delete operation key=O0DJ8SCY766V tag=CFG77Z9TI3Y7 type=del 2024-05-30T16:03:46.337 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.335+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~KBFMHP97GQVD.0 size=949618 2024-05-30T16:03:46.337 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.335+0000 7f08851138c0 1 > prepared multipart upload key=Y8SBI0QZ0ZV8 upload=KBFMHP97GQVD tag=UQVLZ6A9S0YD type=write size=1998194 2024-05-30T16:03:46.340 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.339+0000 7f08851138c0 1 < completed delete operation key=Y8SBI0QZ0ZV8 tag=RX5BOPSUDJZU type=del 2024-05-30T16:03:46.344 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.343+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=MNLHMUX8IRIV type=del size=2501623 2024-05-30T16:03:46.349 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.347+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~W3QEXUAB2E5T.0 size=1048576 2024-05-30T16:03:46.351 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.351+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~W3QEXUAB2E5T.1 size=1048576 2024-05-30T16:03:46.354 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.355+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~W3QEXUAB2E5T.2 size=643605 2024-05-30T16:03:46.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.359+0000 7f08851138c0 1 < completed multipart upload key=0T2E9F3K1USF upload=W3QEXUAB2E5T tag=62F2BMUWQB4G type=write size=3789333 2024-05-30T16:03:46.364 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.363+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=5TF3OZOP7V8P type=del size=2738623 2024-05-30T16:03:46.367 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.367+0000 7f08851138c0 1 < completed delete operation key=TPCETCO3Y1M6 tag=6243D95PX89E type=del 2024-05-30T16:03:46.370 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.371+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=P1CCTCVQODVP type=del size=2827814 2024-05-30T16:03:46.377 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.375+0000 7f08851138c0 1 < suggested remove operation key=Y8SBI0QZ0ZV8 tag=W624NFQVQ9SH type=del 2024-05-30T16:03:46.381 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.379+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=PR7U5HUL7J14 type=del size=209082 2024-05-30T16:03:46.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.383+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~87LD8N716PJN.0 size=124596 2024-05-30T16:03:46.390 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.391+0000 7f08851138c0 1 < completed multipart upload key=E2FXSXOEUGIG upload=87LD8N716PJN tag=TJZ5U5ZSODNN type=write size=1173172 2024-05-30T16:03:46.393 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.395+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=WS4RRPPAUXRQ type=del size=1542376 2024-05-30T16:03:46.397 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.395+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~A2ZXFLFMQNLN.0 size=1048576 2024-05-30T16:03:46.399 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.399+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~A2ZXFLFMQNLN.1 size=656445 2024-05-30T16:03:46.405 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.403+0000 7f08851138c0 1 < completed multipart upload key=A5BUBY9Z7JQG upload=A2ZXFLFMQNLN tag=BWQXRCL80G6I type=write size=2753597 2024-05-30T16:03:46.414 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.411+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~2M58EFHRFWOM.0 size=1048576 2024-05-30T16:03:46.414 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.415+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~2M58EFHRFWOM.1 size=1048576 2024-05-30T16:03:46.417 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.415+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~2M58EFHRFWOM.2 size=520596 2024-05-30T16:03:46.417 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.415+0000 7f08851138c0 1 > prepared multipart upload key=KPHZGFJLC2E0 upload=2M58EFHRFWOM tag=KQOD30J08A78 type=write size=3666324 2024-05-30T16:03:46.420 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.419+0000 7f08851138c0 1 < completed delete operation key=0T2E9F3K1USF tag=2ZBFJJ4UAPJE type=del 2024-05-30T16:03:46.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.423+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=JKS64GAXB60Q type=del size=4108087 2024-05-30T16:03:46.427 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.427+0000 7f08851138c0 1 < completed delete operation key=O0DJ8SCY766V tag=5DIKH3EICR7G type=del 2024-05-30T16:03:46.430 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.431+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=UQ3T3WGTLEHC type=del size=1173662 2024-05-30T16:03:46.433 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.435+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~8BORX3H0N6CF.0 size=903977 2024-05-30T16:03:46.439 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.439+0000 7f08851138c0 1 < completed multipart upload key=KPHZGFJLC2E0 upload=8BORX3H0N6CF tag=EHOTK530MHNA type=write size=1952553 2024-05-30T16:03:46.442 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.443+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=T4KVFX5GTKOI type=del size=2698224 2024-05-30T16:03:46.445 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.447+0000 7f08851138c0 1 < completed write operation key=1N6R2HBTWMP2 tag=4GXR1QSPGXR1 type=write size=533065 2024-05-30T16:03:46.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.447+0000 7f08851138c0 1 > prepared operation key=VRXNO8JLAZYH tag=SS7760TTNE85 type=del size=208193 2024-05-30T16:03:46.452 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.451+0000 7f08851138c0 1 < completed delete operation key=NIV7AQ4EV5YP tag=WCULF4QYNUGD type=del 2024-05-30T16:03:46.458 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.459+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~I1XZRPQHUIUO.0 size=1048576 2024-05-30T16:03:46.460 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.459+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~I1XZRPQHUIUO.1 size=952442 2024-05-30T16:03:46.461 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.459+0000 7f08851138c0 1 > prepared multipart upload key=E2FXSXOEUGIG upload=I1XZRPQHUIUO tag=1SQCTNYF72GR type=write size=3049594 2024-05-30T16:03:46.464 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.463+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~RD0N0RW5CP3V.0 size=448807 2024-05-30T16:03:46.470 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.471+0000 7f08851138c0 1 < completed multipart upload key=RCTV94WRMO53 upload=RD0N0RW5CP3V tag=T1DJ0SQQDX50 type=write size=1497383 2024-05-30T16:03:46.473 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.475+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=O71YHRJ0DYTW type=del size=3713853 2024-05-30T16:03:46.477 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.475+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~B1MDBIFKJZX3.0 size=1048576 2024-05-30T16:03:46.480 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.479+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~B1MDBIFKJZX3.1 size=886627 2024-05-30T16:03:46.485 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.483+0000 7f08851138c0 1 < completed multipart upload key=A5BUBY9Z7JQG upload=B1MDBIFKJZX3 tag=JE52KRGANLFZ type=write size=2983779 2024-05-30T16:03:46.489 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.487+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=M89IKXC6A1YX type=del size=209814 2024-05-30T16:03:46.496 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.495+0000 7f08851138c0 1 < suggested update operation key=V4OQFOOQUXFQ tag=4I915RW2T300 type=write size=159819 2024-05-30T16:03:46.503 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.503+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~OIB3HTQYTQ2G.0 size=905593 2024-05-30T16:03:46.503 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.503+0000 7f08851138c0 1 > prepared multipart upload key=QQILXRJD0C1Z upload=OIB3HTQYTQ2G tag=25MEN10SQZPF type=write size=1954169 2024-05-30T16:03:46.510 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.511+0000 7f08851138c0 1 < suggested remove operation key=V4OQFOOQUXFQ tag=X1UURSG3ELM5 type=write 2024-05-30T16:03:46.517 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.515+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~ZBEK2RMG231N.0 size=1048576 2024-05-30T16:03:46.520 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.519+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~ZBEK2RMG231N.1 size=143503 2024-05-30T16:03:46.520 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.519+0000 7f08851138c0 1 > prepared multipart upload key=4N58ZY4E1H6E upload=ZBEK2RMG231N tag=0Z2IG1KXS9ON type=write size=2240655 2024-05-30T16:03:46.524 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.523+0000 7f08851138c0 1 < completed part key=_multipart_Y8SBI0QZ0ZV8.2~KBFMHP97GQVD.0 size=949618 2024-05-30T16:03:46.530 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.531+0000 7f08851138c0 1 < completed multipart upload key=Y8SBI0QZ0ZV8 upload=KBFMHP97GQVD tag=UQVLZ6A9S0YD type=write size=1998194 2024-05-30T16:03:46.534 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.535+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=EZIY7VM95OAC type=del size=2147762 2024-05-30T16:03:46.538 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.539+0000 7f08851138c0 1 < completed delete operation key=364U3OV10VPD tag=MNLHMUX8IRIV type=del 2024-05-30T16:03:46.545 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.547+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~JVXVL6F8MAQO.0 size=1048576 2024-05-30T16:03:46.548 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.547+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~JVXVL6F8MAQO.1 size=633205 2024-05-30T16:03:46.548 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.547+0000 7f08851138c0 1 > prepared multipart upload key=0T2E9F3K1USF upload=JVXVL6F8MAQO tag=AVUI8ZP5SVQ9 type=write size=2730357 2024-05-30T16:03:46.552 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.551+0000 7f08851138c0 1 < completed delete operation key=0T2E9F3K1USF tag=5TF3OZOP7V8P type=del 2024-05-30T16:03:46.555 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.555+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=H2XA1ZJCV1L2 type=write size=275391 2024-05-30T16:03:46.559 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.559+0000 7f08851138c0 1 < completed delete operation key=A5BUBY9Z7JQG tag=P1CCTCVQODVP type=del 2024-05-30T16:03:46.565 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.563+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~69OUBDXL28N0.0 size=1048576 2024-05-30T16:03:46.568 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.567+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~69OUBDXL28N0.1 size=750483 2024-05-30T16:03:46.568 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.567+0000 7f08851138c0 1 > prepared multipart upload key=0T2E9F3K1USF upload=69OUBDXL28N0 tag=V5RQENNQRX5E type=write size=2847635 2024-05-30T16:03:46.571 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.571+0000 7f08851138c0 1 < completed delete operation key=603HBR8PRH6L tag=PR7U5HUL7J14 type=del 2024-05-30T16:03:46.577 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.575+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~ZXOWZZJGYTJ1.0 size=1048576 2024-05-30T16:03:46.580 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.579+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~ZXOWZZJGYTJ1.1 size=940126 2024-05-30T16:03:46.580 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.579+0000 7f08851138c0 1 > prepared multipart upload key=364U3OV10VPD upload=ZXOWZZJGYTJ1 tag=YKXGQ2AU3JBC type=write size=3037278 2024-05-30T16:03:46.583 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.583+0000 7f08851138c0 1 < completed delete operation key=QQILXRJD0C1Z tag=WS4RRPPAUXRQ type=del 2024-05-30T16:03:46.589 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.591+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~DYJQUIIZDPXC.0 size=1006506 2024-05-30T16:03:46.590 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.591+0000 7f08851138c0 1 > prepared multipart upload key=KPHZGFJLC2E0 upload=DYJQUIIZDPXC tag=QM9HQSXYAABZ type=write size=2055082 2024-05-30T16:03:46.593 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.591+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~2M58EFHRFWOM.0 size=1048576 2024-05-30T16:03:46.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.595+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~2M58EFHRFWOM.1 size=1048576 2024-05-30T16:03:46.600 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.599+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~2M58EFHRFWOM.2 size=520596 2024-05-30T16:03:46.606 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.607+0000 7f08851138c0 1 < completed multipart upload key=KPHZGFJLC2E0 upload=2M58EFHRFWOM tag=KQOD30J08A78 type=write size=3666324 2024-05-30T16:03:46.610 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.611+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=1SG2UUXBRTA9 type=del size=3803646 2024-05-30T16:03:46.614 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.615+0000 7f08851138c0 1 < completed delete operation key=Y8SBI0QZ0ZV8 tag=JKS64GAXB60Q type=del 2024-05-30T16:03:46.618 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.619+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=28W8O1MOP0WP type=del size=3867775 2024-05-30T16:03:46.626 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.627+0000 7f08851138c0 1 < suggested remove operation key=YRP4CSG92QRO tag=UQ3T3WGTLEHC type=del 2024-05-30T16:03:46.630 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.631+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=1WU7X6KWBAIW type=del size=2147713 2024-05-30T16:03:46.637 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.635+0000 7f08851138c0 1 < suggested remove operation key=GWXNCSJCLXKW tag=T4KVFX5GTKOI type=del 2024-05-30T16:03:46.640 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.639+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=DSX7E22P64PL type=write size=968811 2024-05-30T16:03:46.644 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.643+0000 7f08851138c0 1 < completed delete operation key=VRXNO8JLAZYH tag=SS7760TTNE85 type=del 2024-05-30T16:03:46.650 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.651+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~0DK8S2ME64AA.0 size=1048576 2024-05-30T16:03:46.652 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.651+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~0DK8S2ME64AA.1 size=1048576 2024-05-30T16:03:46.655 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.655+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~0DK8S2ME64AA.2 size=527519 2024-05-30T16:03:46.655 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.655+0000 7f08851138c0 1 > prepared multipart upload key=PLWFSKPKVO19 upload=0DK8S2ME64AA tag=013KZ1WIPAQK type=write size=3673247 2024-05-30T16:03:46.659 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.659+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~I1XZRPQHUIUO.0 size=1048576 2024-05-30T16:03:46.659 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.659+0000 7f08851138c0 1 < canceled part key=_multipart_E2FXSXOEUGIG.2~I1XZRPQHUIUO.1 size=952442 2024-05-30T16:03:46.666 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.667+0000 7f08851138c0 1 < canceled multipart upload key=E2FXSXOEUGIG upload=I1XZRPQHUIUO tag=1SQCTNYF72GR type=write size=3049594 2024-05-30T16:03:46.669 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.667+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=0K18Z1GRY896 type=write size=601615 2024-05-30T16:03:46.672 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.671+0000 7f08851138c0 1 < canceled operation key=O8CWPJ9GW2GO tag=O71YHRJ0DYTW type=del size=3713853 2024-05-30T16:03:46.675 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.675+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=EURW3FZCRWRE type=del size=2504763 2024-05-30T16:03:46.681 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.683+0000 7f08851138c0 1 < suggested remove operation key=Y3CYZJ83WYNI tag=M89IKXC6A1YX type=del 2024-05-30T16:03:46.685 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.683+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=RDZETXHM6HU7 type=del size=3493086 2024-05-30T16:03:46.688 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.687+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~OIB3HTQYTQ2G.0 size=905593 2024-05-30T16:03:46.693 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.691+0000 7f08851138c0 1 < completed multipart upload key=QQILXRJD0C1Z upload=OIB3HTQYTQ2G tag=25MEN10SQZPF type=write size=1954169 2024-05-30T16:03:46.698 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.699+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~IGMXJA75T5K1.0 size=1048576 2024-05-30T16:03:46.701 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.699+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~IGMXJA75T5K1.1 size=201943 2024-05-30T16:03:46.701 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.699+0000 7f08851138c0 1 > prepared multipart upload key=ZALVY70ISLM5 upload=IGMXJA75T5K1 tag=TC480VOGB26B type=write size=2299095 2024-05-30T16:03:46.704 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.703+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~ZBEK2RMG231N.0 size=1048576 2024-05-30T16:03:46.707 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.707+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~ZBEK2RMG231N.1 size=143503 2024-05-30T16:03:46.712 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.711+0000 7f08851138c0 1 < completed multipart upload key=4N58ZY4E1H6E upload=ZBEK2RMG231N tag=0Z2IG1KXS9ON type=write size=2240655 2024-05-30T16:03:46.717 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.715+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~YHIWSCALPQ1A.0 size=1048576 2024-05-30T16:03:46.719 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.719+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~YHIWSCALPQ1A.1 size=852594 2024-05-30T16:03:46.719 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.719+0000 7f08851138c0 1 > prepared multipart upload key=WT9RC0UYVG4F upload=YHIWSCALPQ1A tag=O341C7Z1HUF0 type=write size=2949746 2024-05-30T16:03:46.722 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.723+0000 7f08851138c0 1 < completed delete operation key=RCTV94WRMO53 tag=EZIY7VM95OAC type=del 2024-05-30T16:03:46.727 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.727+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~4QGF4V6JRTG2.0 size=1048576 2024-05-30T16:03:46.729 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.727+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~4QGF4V6JRTG2.1 size=1048576 2024-05-30T16:03:46.731 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.731+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~4QGF4V6JRTG2.2 size=112436 2024-05-30T16:03:46.731 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.731+0000 7f08851138c0 1 > prepared multipart upload key=4N58ZY4E1H6E upload=4QGF4V6JRTG2 tag=ILP7JBX63LC9 type=write size=3258164 2024-05-30T16:03:46.734 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.735+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~JVXVL6F8MAQO.0 size=1048576 2024-05-30T16:03:46.736 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.735+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~JVXVL6F8MAQO.1 size=633205 2024-05-30T16:03:46.741 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.743+0000 7f08851138c0 1 < completed multipart upload key=0T2E9F3K1USF upload=JVXVL6F8MAQO tag=AVUI8ZP5SVQ9 type=write size=2730357 2024-05-30T16:03:46.746 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.747+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~HV2P9K60QQPO.0 size=1048576 2024-05-30T16:03:46.748 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.747+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~HV2P9K60QQPO.1 size=1048576 2024-05-30T16:03:46.751 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.751+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~HV2P9K60QQPO.2 size=52077 2024-05-30T16:03:46.751 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.751+0000 7f08851138c0 1 > prepared multipart upload key=QCLY10D2A3VF upload=HV2P9K60QQPO tag=ZV1JFX4MSTS9 type=write size=3197805 2024-05-30T16:03:46.754 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.755+0000 7f08851138c0 1 < completed write operation key=E2FXSXOEUGIG tag=H2XA1ZJCV1L2 type=write size=275391 2024-05-30T16:03:46.760 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.759+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~G1N6EOKJARZU.0 size=1048576 2024-05-30T16:03:46.762 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.763+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~G1N6EOKJARZU.1 size=377282 2024-05-30T16:03:46.762 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.763+0000 7f08851138c0 1 > prepared multipart upload key=GWXNCSJCLXKW upload=G1N6EOKJARZU tag=W2H3TLR2VJ7A type=write size=2474434 2024-05-30T16:03:46.765 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.763+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~69OUBDXL28N0.0 size=1048576 2024-05-30T16:03:46.765 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.763+0000 7f08851138c0 1 < canceled part key=_multipart_0T2E9F3K1USF.2~69OUBDXL28N0.1 size=750483 2024-05-30T16:03:46.770 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.771+0000 7f08851138c0 1 < canceled multipart upload key=0T2E9F3K1USF upload=69OUBDXL28N0 tag=V5RQENNQRX5E type=write size=2847635 2024-05-30T16:03:46.773 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.771+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=4NJD5PZI8AF0 type=del size=908083 2024-05-30T16:03:46.776 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.775+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~ZXOWZZJGYTJ1.0 size=1048576 2024-05-30T16:03:46.779 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.779+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~ZXOWZZJGYTJ1.1 size=940126 2024-05-30T16:03:46.784 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.783+0000 7f08851138c0 1 < completed multipart upload key=364U3OV10VPD upload=ZXOWZZJGYTJ1 tag=YKXGQ2AU3JBC type=write size=3037278 2024-05-30T16:03:46.790 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.791+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~YFNICHWRKX4B.0 size=172065 2024-05-30T16:03:46.790 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.791+0000 7f08851138c0 1 > prepared multipart upload key=Y3CYZJ83WYNI upload=YFNICHWRKX4B tag=1H16V1OGP010 type=write size=1220641 2024-05-30T16:03:46.793 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.795+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~DYJQUIIZDPXC.0 size=1006506 2024-05-30T16:03:46.798 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.799+0000 7f08851138c0 1 < completed multipart upload key=KPHZGFJLC2E0 upload=DYJQUIIZDPXC tag=QM9HQSXYAABZ type=write size=2055082 2024-05-30T16:03:46.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.803+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=C1GLIHN3UN8E type=del size=1245797 2024-05-30T16:03:46.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.803+0000 7f08851138c0 -1 < no bi entry to suggest for operation key=GWXNCSJCLXKW tag=1SG2UUXBRTA9 type=del size=3803646: (2) No such file or directory 2024-05-30T16:03:46.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.807+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=4C2KZIRMN1ZR type=del size=3401273 2024-05-30T16:03:46.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.807+0000 7f08851138c0 1 < completed delete operation key=3TBOZWT2P8U8 tag=28W8O1MOP0WP type=del 2024-05-30T16:03:46.812 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.811+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=07OXSBW4HP5R type=del size=2153543 2024-05-30T16:03:46.816 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.815+0000 7f08851138c0 1 < completed delete operation key=RCTV94WRMO53 tag=1WU7X6KWBAIW type=del 2024-05-30T16:03:46.819 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.819+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=6BRE5AFHLRWP type=del size=3875470 2024-05-30T16:03:46.822 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.823+0000 7f08851138c0 1 < canceled operation key=73FJ3HGWYQQK tag=DSX7E22P64PL type=write size=968811 2024-05-30T16:03:46.829 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.827+0000 7f08851138c0 1 > prepared part key=_multipart_MMLF65KLC42Q.2~61A9BMDTNB6S.0 size=1048576 2024-05-30T16:03:46.831 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.831+0000 7f08851138c0 1 > prepared part key=_multipart_MMLF65KLC42Q.2~61A9BMDTNB6S.1 size=1048576 2024-05-30T16:03:46.833 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.835+0000 7f08851138c0 1 > prepared part key=_multipart_MMLF65KLC42Q.2~61A9BMDTNB6S.2 size=940658 2024-05-30T16:03:46.834 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.835+0000 7f08851138c0 1 > prepared multipart upload key=MMLF65KLC42Q upload=61A9BMDTNB6S tag=HE7OMRU7OFZ2 type=write size=4086386 2024-05-30T16:03:46.837 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.839+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~0DK8S2ME64AA.0 size=1048576 2024-05-30T16:03:46.840 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.839+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~0DK8S2ME64AA.1 size=1048576 2024-05-30T16:03:46.843 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.843+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~0DK8S2ME64AA.2 size=527519 2024-05-30T16:03:46.848 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.847+0000 7f08851138c0 1 < completed multipart upload key=PLWFSKPKVO19 upload=0DK8S2ME64AA tag=013KZ1WIPAQK type=write size=3673247 2024-05-30T16:03:46.853 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.851+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~3F3SNBYZ0IGR.0 size=1048576 2024-05-30T16:03:46.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.855+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~3F3SNBYZ0IGR.1 size=1048576 2024-05-30T16:03:46.857 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.859+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~3F3SNBYZ0IGR.2 size=733293 2024-05-30T16:03:46.858 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.859+0000 7f08851138c0 1 > prepared multipart upload key=E2FXSXOEUGIG upload=3F3SNBYZ0IGR tag=IUTH1JI0M769 type=write size=3879021 2024-05-30T16:03:46.864 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.863+0000 7f08851138c0 1 < suggested remove operation key=RCTV94WRMO53 tag=0K18Z1GRY896 type=write 2024-05-30T16:03:46.867 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.867+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=NRSS6CUU2KRW type=del size=3912238 2024-05-30T16:03:46.874 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.875+0000 7f08851138c0 1 < suggested update operation key=MMLF65KLC42Q tag=EURW3FZCRWRE type=del size=2504763 2024-05-30T16:03:46.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.875+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=DV2NQ5EFFHC9 type=del size=2961237 2024-05-30T16:03:46.880 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.879+0000 7f08851138c0 1 < completed delete operation key=A5BUBY9Z7JQG tag=RDZETXHM6HU7 type=del 2024-05-30T16:03:46.883 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.883+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=YSJQA4VQ6KCU type=del size=1438692 2024-05-30T16:03:46.886 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.887+0000 7f08851138c0 1 < completed part key=_multipart_ZALVY70ISLM5.2~IGMXJA75T5K1.0 size=1048576 2024-05-30T16:03:46.889 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.887+0000 7f08851138c0 1 < completed part key=_multipart_ZALVY70ISLM5.2~IGMXJA75T5K1.1 size=201943 2024-05-30T16:03:46.894 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.895+0000 7f08851138c0 1 < completed multipart upload key=ZALVY70ISLM5 upload=IGMXJA75T5K1 tag=TC480VOGB26B type=write size=2299095 2024-05-30T16:03:46.897 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.899+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=QW9YT1UFREVC type=del size=4004514 2024-05-30T16:03:46.900 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.899+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~YHIWSCALPQ1A.0 size=1048576 2024-05-30T16:03:46.903 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.903+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~YHIWSCALPQ1A.1 size=852594 2024-05-30T16:03:46.907 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.907+0000 7f08851138c0 1 < completed multipart upload key=WT9RC0UYVG4F upload=YHIWSCALPQ1A tag=O341C7Z1HUF0 type=write size=2949746 2024-05-30T16:03:46.913 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.915+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~CNQ6QXF67G5I.0 size=1048576 2024-05-30T16:03:46.916 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.915+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~CNQ6QXF67G5I.1 size=190376 2024-05-30T16:03:46.916 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.915+0000 7f08851138c0 1 > prepared multipart upload key=O8CWPJ9GW2GO upload=CNQ6QXF67G5I tag=EA7ZI1DKDK1J type=write size=2287528 2024-05-30T16:03:46.919 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.919+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~4QGF4V6JRTG2.0 size=1048576 2024-05-30T16:03:46.921 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.923+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~4QGF4V6JRTG2.1 size=1048576 2024-05-30T16:03:46.924 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.923+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~4QGF4V6JRTG2.2 size=112436 2024-05-30T16:03:46.931 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.931+0000 7f08851138c0 1 < completed multipart upload key=4N58ZY4E1H6E upload=4QGF4V6JRTG2 tag=ILP7JBX63LC9 type=write size=3258164 2024-05-30T16:03:46.938 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.939+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~G8MIPVHIXLRT.0 size=291410 2024-05-30T16:03:46.938 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.939+0000 7f08851138c0 1 > prepared multipart upload key=WT9RC0UYVG4F upload=G8MIPVHIXLRT tag=4561N53KOR0T type=write size=1339986 2024-05-30T16:03:46.942 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.943+0000 7f08851138c0 1 < completed part key=_multipart_QCLY10D2A3VF.2~HV2P9K60QQPO.0 size=1048576 2024-05-30T16:03:46.945 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.943+0000 7f08851138c0 1 < completed part key=_multipart_QCLY10D2A3VF.2~HV2P9K60QQPO.1 size=1048576 2024-05-30T16:03:46.945 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.943+0000 7f08851138c0 1 < canceled part key=_multipart_QCLY10D2A3VF.2~HV2P9K60QQPO.2 size=52077 2024-05-30T16:03:46.953 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.951+0000 7f08851138c0 1 < canceled multipart upload key=QCLY10D2A3VF upload=HV2P9K60QQPO tag=ZV1JFX4MSTS9 type=write size=3197805 2024-05-30T16:03:46.956 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.955+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=09HGG05VSH36 type=del size=3301081 2024-05-30T16:03:46.959 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.959+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~G1N6EOKJARZU.0 size=1048576 2024-05-30T16:03:46.962 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.963+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~G1N6EOKJARZU.1 size=377282 2024-05-30T16:03:46.968 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.967+0000 7f08851138c0 1 < completed multipart upload key=GWXNCSJCLXKW upload=G1N6EOKJARZU tag=W2H3TLR2VJ7A type=write size=2474434 2024-05-30T16:03:46.974 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.975+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~GHZ371VRXUOQ.0 size=1048576 2024-05-30T16:03:46.977 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.975+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~GHZ371VRXUOQ.1 size=1048576 2024-05-30T16:03:46.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.979+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~GHZ371VRXUOQ.2 size=545247 2024-05-30T16:03:46.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.979+0000 7f08851138c0 1 > prepared multipart upload key=Y8SBI0QZ0ZV8 upload=GHZ371VRXUOQ tag=JB8L3TX2D9S2 type=write size=3690975 2024-05-30T16:03:46.986 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.987+0000 7f08851138c0 1 < suggested remove operation key=4N58ZY4E1H6E tag=4NJD5PZI8AF0 type=del 2024-05-30T16:03:46.993 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.991+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~Z374E5VSZMN2.0 size=1048576 2024-05-30T16:03:46.996 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.995+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~Z374E5VSZMN2.1 size=730138 2024-05-30T16:03:46.996 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.995+0000 7f08851138c0 1 > prepared multipart upload key=RCTV94WRMO53 upload=Z374E5VSZMN2 tag=A1X6TQ2ANNFF type=write size=2827290 2024-05-30T16:03:47.000 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:46.999+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~YFNICHWRKX4B.0 size=172065 2024-05-30T16:03:47.007 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.007+0000 7f08851138c0 1 < completed multipart upload key=Y3CYZJ83WYNI upload=YFNICHWRKX4B tag=1H16V1OGP010 type=write size=1220641 2024-05-30T16:03:47.011 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.011+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=KU1OH9028IQD type=del size=2795251 2024-05-30T16:03:47.015 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.015+0000 7f08851138c0 1 < completed delete operation key=JNZI7JX95LX2 tag=C1GLIHN3UN8E type=del 2024-05-30T16:03:47.019 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.019+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=M1R3VG2AN5V3 type=del size=2109123 2024-05-30T16:03:47.024 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.023+0000 7f08851138c0 1 < completed delete operation key=0RZRBW38WKNZ tag=4C2KZIRMN1ZR type=del 2024-05-30T16:03:47.030 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.031+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~OXFMZOVM50WP.0 size=1048576 2024-05-30T16:03:47.033 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.031+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~OXFMZOVM50WP.1 size=462164 2024-05-30T16:03:47.033 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.031+0000 7f08851138c0 1 > prepared multipart upload key=O8CWPJ9GW2GO upload=OXFMZOVM50WP tag=GS1U5Y2EE8MY type=write size=2559316 2024-05-30T16:03:47.036 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.035+0000 7f08851138c0 1 < completed delete operation key=A5BUBY9Z7JQG tag=07OXSBW4HP5R type=del 2024-05-30T16:03:47.040 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.039+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=5891ULOAWLJC type=del size=2156734 2024-05-30T16:03:47.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.043+0000 7f08851138c0 1 < completed delete operation key=QQILXRJD0C1Z tag=6BRE5AFHLRWP type=del 2024-05-30T16:03:47.051 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.051+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~1D7Y63GSXE5C.0 size=1048576 2024-05-30T16:03:47.054 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.055+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~1D7Y63GSXE5C.1 size=15431 2024-05-30T16:03:47.054 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.055+0000 7f08851138c0 1 > prepared multipart upload key=E2FXSXOEUGIG upload=1D7Y63GSXE5C tag=RXGDKRPB560H type=write size=2112583 2024-05-30T16:03:47.058 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.059+0000 7f08851138c0 1 < completed part key=_multipart_MMLF65KLC42Q.2~61A9BMDTNB6S.0 size=1048576 2024-05-30T16:03:47.061 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.059+0000 7f08851138c0 1 < completed part key=_multipart_MMLF65KLC42Q.2~61A9BMDTNB6S.1 size=1048576 2024-05-30T16:03:47.064 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.063+0000 7f08851138c0 1 < completed part key=_multipart_MMLF65KLC42Q.2~61A9BMDTNB6S.2 size=940658 2024-05-30T16:03:47.070 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.071+0000 7f08851138c0 1 < completed multipart upload key=MMLF65KLC42Q upload=61A9BMDTNB6S tag=HE7OMRU7OFZ2 type=write size=4086386 2024-05-30T16:03:47.074 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.075+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=ELRSWJIDQ6WX type=del size=2474712 2024-05-30T16:03:47.074 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.075+0000 7f08851138c0 1 < canceled part key=_multipart_E2FXSXOEUGIG.2~3F3SNBYZ0IGR.0 size=1048576 2024-05-30T16:03:47.077 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.075+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~3F3SNBYZ0IGR.1 size=1048576 2024-05-30T16:03:47.080 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.079+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~3F3SNBYZ0IGR.2 size=733293 2024-05-30T16:03:47.085 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.087+0000 7f08851138c0 1 < canceled multipart upload key=E2FXSXOEUGIG upload=3F3SNBYZ0IGR tag=IUTH1JI0M769 type=write size=3879021 2024-05-30T16:03:47.092 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.091+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~9MAIQ09UNRJC.0 size=248606 2024-05-30T16:03:47.092 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.091+0000 7f08851138c0 1 > prepared multipart upload key=73FJ3HGWYQQK upload=9MAIQ09UNRJC tag=IGIWOIN2U5R8 type=write size=1297182 2024-05-30T16:03:47.095 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.095+0000 7f08851138c0 1 < completed delete operation key=O0DJ8SCY766V tag=NRSS6CUU2KRW type=del 2024-05-30T16:03:47.099 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.099+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=UH9SH2RHIGYV type=del size=2470930 2024-05-30T16:03:47.102 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.103+0000 7f08851138c0 1 < completed delete operation key=TPCETCO3Y1M6 tag=DV2NQ5EFFHC9 type=del 2024-05-30T16:03:47.106 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.107+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=AA7ASR0MD8UN type=del size=3204808 2024-05-30T16:03:47.110 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.111+0000 7f08851138c0 1 < completed delete operation key=PLWFSKPKVO19 tag=YSJQA4VQ6KCU type=del 2024-05-30T16:03:47.114 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.115+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=KM3JSLSRP6XJ type=del size=2799662 2024-05-30T16:03:47.117 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.115+0000 7f08851138c0 -1 < failed to complete operation key=4N58ZY4E1H6E tag=QW9YT1UFREVC type=del size=4004514: (22) Invalid argument 2024-05-30T16:03:47.124 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.123+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~76FIYJC4GGIU.0 size=1048576 2024-05-30T16:03:47.127 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.127+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~76FIYJC4GGIU.1 size=199748 2024-05-30T16:03:47.127 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.127+0000 7f08851138c0 1 > prepared multipart upload key=WT9RC0UYVG4F upload=76FIYJC4GGIU tag=BS6Q0IXESL5Z type=write size=2296900 2024-05-30T16:03:47.131 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.131+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~CNQ6QXF67G5I.0 size=1048576 2024-05-30T16:03:47.131 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.131+0000 7f08851138c0 1 < canceled part key=_multipart_O8CWPJ9GW2GO.2~CNQ6QXF67G5I.1 size=190376 2024-05-30T16:03:47.138 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.139+0000 7f08851138c0 1 < canceled multipart upload key=O8CWPJ9GW2GO upload=CNQ6QXF67G5I tag=EA7ZI1DKDK1J type=write size=2287528 2024-05-30T16:03:47.141 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.143+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=GYHPWWZBIRFQ type=del size=3039526 2024-05-30T16:03:47.145 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.147+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~G8MIPVHIXLRT.0 size=291410 2024-05-30T16:03:47.151 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.151+0000 7f08851138c0 1 < completed multipart upload key=WT9RC0UYVG4F upload=G8MIPVHIXLRT tag=4561N53KOR0T type=write size=1339986 2024-05-30T16:03:47.155 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.155+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=71761QQXJ8QL type=del size=475892 2024-05-30T16:03:47.162 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.163+0000 7f08851138c0 1 < suggested remove operation key=QQILXRJD0C1Z tag=09HGG05VSH36 type=del 2024-05-30T16:03:47.165 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.167+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=1UXHI32Z6711 type=del size=346905 2024-05-30T16:03:47.169 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.167+0000 7f08851138c0 1 < completed part key=_multipart_Y8SBI0QZ0ZV8.2~GHZ371VRXUOQ.0 size=1048576 2024-05-30T16:03:47.172 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.171+0000 7f08851138c0 1 < completed part key=_multipart_Y8SBI0QZ0ZV8.2~GHZ371VRXUOQ.1 size=1048576 2024-05-30T16:03:47.175 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.175+0000 7f08851138c0 1 < completed part key=_multipart_Y8SBI0QZ0ZV8.2~GHZ371VRXUOQ.2 size=545247 2024-05-30T16:03:47.181 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.183+0000 7f08851138c0 1 < completed multipart upload key=Y8SBI0QZ0ZV8 upload=GHZ371VRXUOQ tag=JB8L3TX2D9S2 type=write size=3690975 2024-05-30T16:03:47.188 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.187+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~MTA5S2N216KT.0 size=537267 2024-05-30T16:03:47.188 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.187+0000 7f08851138c0 1 > prepared multipart upload key=E2FXSXOEUGIG upload=MTA5S2N216KT tag=SETP6O8XQRZH type=write size=1585843 2024-05-30T16:03:47.192 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.191+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~Z374E5VSZMN2.0 size=1048576 2024-05-30T16:03:47.195 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.195+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~Z374E5VSZMN2.1 size=730138 2024-05-30T16:03:47.201 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.199+0000 7f08851138c0 1 < completed multipart upload key=RCTV94WRMO53 upload=Z374E5VSZMN2 tag=A1X6TQ2ANNFF type=write size=2827290 2024-05-30T16:03:47.205 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.203+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=4ASECP7ISP1I type=del size=3115407 2024-05-30T16:03:47.208 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.207+0000 7f08851138c0 1 < completed delete operation key=YRP4CSG92QRO tag=KU1OH9028IQD type=del 2024-05-30T16:03:47.212 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.211+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=IT0UA43HLTGB type=del size=3922709 2024-05-30T16:03:47.216 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.215+0000 7f08851138c0 1 < completed delete operation key=RCTV94WRMO53 tag=M1R3VG2AN5V3 type=del 2024-05-30T16:03:47.219 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.219+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=FUS5BLAT4FWQ type=write size=48767 2024-05-30T16:03:47.220 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.219+0000 7f08851138c0 1 < canceled part key=_multipart_O8CWPJ9GW2GO.2~OXFMZOVM50WP.0 size=1048576 2024-05-30T16:03:47.220 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.219+0000 7f08851138c0 1 < canceled part key=_multipart_O8CWPJ9GW2GO.2~OXFMZOVM50WP.1 size=462164 2024-05-30T16:03:47.226 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.227+0000 7f08851138c0 1 < canceled multipart upload key=O8CWPJ9GW2GO upload=OXFMZOVM50WP tag=GS1U5Y2EE8MY type=write size=2559316 2024-05-30T16:03:47.230 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.231+0000 7f08851138c0 1 > prepared operation key=VRXNO8JLAZYH tag=M4GXYKCZ937T type=del size=695522 2024-05-30T16:03:47.234 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.235+0000 7f08851138c0 1 < completed delete operation key=Y8SBI0QZ0ZV8 tag=5891ULOAWLJC type=del 2024-05-30T16:03:47.237 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.239+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=W3A483YQ8QV3 type=del size=1713491 2024-05-30T16:03:47.241 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.239+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~1D7Y63GSXE5C.0 size=1048576 2024-05-30T16:03:47.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.243+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~1D7Y63GSXE5C.1 size=15431 2024-05-30T16:03:47.250 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.251+0000 7f08851138c0 1 < completed multipart upload key=E2FXSXOEUGIG upload=1D7Y63GSXE5C tag=RXGDKRPB560H type=write size=2112583 2024-05-30T16:03:47.258 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.259+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~KU0WID3OZ92S.0 size=471395 2024-05-30T16:03:47.258 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.259+0000 7f08851138c0 1 > prepared multipart upload key=VRXNO8JLAZYH upload=KU0WID3OZ92S tag=66L0IYT83JWV type=write size=1519971 2024-05-30T16:03:47.263 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.263+0000 7f08851138c0 1 < completed delete operation key=V4OQFOOQUXFQ tag=ELRSWJIDQ6WX type=del 2024-05-30T16:03:47.267 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.267+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=9Z6ZTDJVB1HP type=del size=3230996 2024-05-30T16:03:47.270 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.271+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~9MAIQ09UNRJC.0 size=248606 2024-05-30T16:03:47.276 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.275+0000 7f08851138c0 1 < completed multipart upload key=73FJ3HGWYQQK upload=9MAIQ09UNRJC tag=IGIWOIN2U5R8 type=write size=1297182 2024-05-30T16:03:47.280 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.279+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=46B501LXETGU type=del size=4053240 2024-05-30T16:03:47.284 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.283+0000 7f08851138c0 1 < canceled operation key=WT9RC0UYVG4F tag=UH9SH2RHIGYV type=del size=2470930 2024-05-30T16:03:47.288 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.287+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=FZF50SUBS6UI type=del size=2223092 2024-05-30T16:03:47.291 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.291+0000 7f08851138c0 1 < completed delete operation key=603HBR8PRH6L tag=AA7ASR0MD8UN type=del 2024-05-30T16:03:47.302 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.303+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~QLA3D6UF24HM.0 size=1048576 2024-05-30T16:03:47.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.307+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~QLA3D6UF24HM.1 size=651375 2024-05-30T16:03:47.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.307+0000 7f08851138c0 1 > prepared multipart upload key=0T2E9F3K1USF upload=QLA3D6UF24HM tag=UYJXYJ3FVAWW type=write size=2748527 2024-05-30T16:03:47.312 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.311+0000 7f08851138c0 1 < suggested update operation key=RCTV94WRMO53 tag=KM3JSLSRP6XJ type=del size=2799662 2024-05-30T16:03:47.319 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.319+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~P4V2EJMQT994.0 size=301902 2024-05-30T16:03:47.319 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.319+0000 7f08851138c0 1 > prepared multipart upload key=RCTV94WRMO53 upload=P4V2EJMQT994 tag=NV1OYWXCO3DH type=write size=1350478 2024-05-30T16:03:47.322 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.323+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~76FIYJC4GGIU.0 size=1048576 2024-05-30T16:03:47.326 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.327+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~76FIYJC4GGIU.1 size=199748 2024-05-30T16:03:47.332 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.331+0000 7f08851138c0 1 < completed multipart upload key=WT9RC0UYVG4F upload=76FIYJC4GGIU tag=BS6Q0IXESL5Z type=write size=2296900 2024-05-30T16:03:47.335 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.335+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=MLSU2ESOIVFJ type=del size=3565827 2024-05-30T16:03:47.339 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.339+0000 7f08851138c0 1 < completed delete operation key=1N6R2HBTWMP2 tag=GYHPWWZBIRFQ type=del 2024-05-30T16:03:47.346 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.347+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~SFAA5WWE7JYW.0 size=425806 2024-05-30T16:03:47.346 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.347+0000 7f08851138c0 1 > prepared multipart upload key=0T2E9F3K1USF upload=SFAA5WWE7JYW tag=DT0KPAI3WPJS type=write size=1474382 2024-05-30T16:03:47.349 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.351+0000 7f08851138c0 1 < completed delete operation key=4N58ZY4E1H6E tag=71761QQXJ8QL type=del 2024-05-30T16:03:47.356 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.355+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~SEBVJCTQTB82.0 size=1048576 2024-05-30T16:03:47.359 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.359+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~SEBVJCTQTB82.1 size=155530 2024-05-30T16:03:47.359 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.359+0000 7f08851138c0 1 > prepared multipart upload key=O0DJ8SCY766V upload=SEBVJCTQTB82 tag=Z9HM1KCVA8D7 type=write size=2252682 2024-05-30T16:03:47.362 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.363+0000 7f08851138c0 1 < canceled operation key=364U3OV10VPD tag=1UXHI32Z6711 type=del size=346905 2024-05-30T16:03:47.365 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.367+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=Y49NXQGTRL5A type=del size=648760 2024-05-30T16:03:47.369 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.367+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~MTA5S2N216KT.0 size=537267 2024-05-30T16:03:47.375 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.375+0000 7f08851138c0 1 < completed multipart upload key=E2FXSXOEUGIG upload=MTA5S2N216KT tag=SETP6O8XQRZH type=write size=1585843 2024-05-30T16:03:47.378 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.379+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=OP2ZLVADNSV2 type=del size=600140 2024-05-30T16:03:47.385 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.383+0000 7f08851138c0 1 < suggested update operation key=GWXNCSJCLXKW tag=4ASECP7ISP1I type=del size=3115407 2024-05-30T16:03:47.389 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.387+0000 7f08851138c0 1 > prepared operation key=ZALVY70ISLM5 tag=DAJ054PJSJ1R type=write size=248277 2024-05-30T16:03:47.392 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.391+0000 7f08851138c0 -1 < failed to complete operation key=RCTV94WRMO53 tag=IT0UA43HLTGB type=del size=3922709: (22) Invalid argument 2024-05-30T16:03:47.396 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.395+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=1RJMJ78DC74G type=del size=2314956 2024-05-30T16:03:47.403 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.403+0000 7f08851138c0 1 < suggested update operation key=JNZI7JX95LX2 tag=FUS5BLAT4FWQ type=write size=48767 2024-05-30T16:03:47.406 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.407+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=8MTY27381322 type=del size=1918994 2024-05-30T16:03:47.410 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.411+0000 7f08851138c0 1 < completed delete operation key=VRXNO8JLAZYH tag=M4GXYKCZ937T type=del 2024-05-30T16:03:47.417 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.415+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~724H5OFLNSCU.0 size=1048576 2024-05-30T16:03:47.420 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.419+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~724H5OFLNSCU.1 size=875851 2024-05-30T16:03:47.420 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.419+0000 7f08851138c0 1 > prepared multipart upload key=3TBOZWT2P8U8 upload=724H5OFLNSCU tag=L5VYYDT336PJ type=write size=2973003 2024-05-30T16:03:47.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.423+0000 7f08851138c0 1 < completed delete operation key=3IJWPWRB6FHE tag=W3A483YQ8QV3 type=del 2024-05-30T16:03:47.427 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.427+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=SOCQ7KB94H6G type=write size=271980 2024-05-30T16:03:47.431 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.431+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~KU0WID3OZ92S.0 size=471395 2024-05-30T16:03:47.438 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.439+0000 7f08851138c0 1 < completed multipart upload key=VRXNO8JLAZYH upload=KU0WID3OZ92S tag=66L0IYT83JWV type=write size=1519971 2024-05-30T16:03:47.444 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.443+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~2L6UBKFJEBJ9.0 size=509481 2024-05-30T16:03:47.444 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.443+0000 7f08851138c0 1 > prepared multipart upload key=Y3CYZJ83WYNI upload=2L6UBKFJEBJ9 tag=529P50MT1R0C type=write size=1558057 2024-05-30T16:03:47.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.447+0000 7f08851138c0 1 < canceled operation key=O8CWPJ9GW2GO tag=9Z6ZTDJVB1HP type=del size=3230996 2024-05-30T16:03:47.454 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.455+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~KLT13G4K60IB.0 size=1048576 2024-05-30T16:03:47.457 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.455+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~KLT13G4K60IB.1 size=1048576 2024-05-30T16:03:47.460 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.459+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~KLT13G4K60IB.2 size=1012490 2024-05-30T16:03:47.460 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.459+0000 7f08851138c0 1 > prepared multipart upload key=NIV7AQ4EV5YP upload=KLT13G4K60IB tag=D5SOX0Q3AZS4 type=write size=4158218 2024-05-30T16:03:47.463 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.463+0000 7f08851138c0 1 < completed delete operation key=MMLF65KLC42Q tag=46B501LXETGU type=del 2024-05-30T16:03:47.467 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.467+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=6VE7KMIP1EDR type=del size=1019134 2024-05-30T16:03:47.471 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.471+0000 7f08851138c0 1 < completed delete operation key=WT9RC0UYVG4F tag=FZF50SUBS6UI type=del 2024-05-30T16:03:47.474 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.475+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=39CDH6RWQTV0 type=write size=514590 2024-05-30T16:03:47.478 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.479+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~QLA3D6UF24HM.0 size=1048576 2024-05-30T16:03:47.481 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.479+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~QLA3D6UF24HM.1 size=651375 2024-05-30T16:03:47.486 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.487+0000 7f08851138c0 1 < completed multipart upload key=0T2E9F3K1USF upload=QLA3D6UF24HM tag=UYJXYJ3FVAWW type=write size=2748527 2024-05-30T16:03:47.490 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.491+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=KDV8YJMO6068 type=write size=489235 2024-05-30T16:03:47.494 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.495+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~P4V2EJMQT994.0 size=301902 2024-05-30T16:03:47.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.499+0000 7f08851138c0 1 < completed multipart upload key=RCTV94WRMO53 upload=P4V2EJMQT994 tag=NV1OYWXCO3DH type=write size=1350478 2024-05-30T16:03:47.503 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.503+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=VCWDIEHJKP6O type=write size=660888 2024-05-30T16:03:47.507 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.507+0000 7f08851138c0 1 < canceled operation key=A5BUBY9Z7JQG tag=MLSU2ESOIVFJ type=del size=3565827 2024-05-30T16:03:47.511 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.511+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=UYDB4AV32M8C type=del size=2042678 2024-05-30T16:03:47.515 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.515+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~SFAA5WWE7JYW.0 size=425806 2024-05-30T16:03:47.522 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.523+0000 7f08851138c0 1 < completed multipart upload key=0T2E9F3K1USF upload=SFAA5WWE7JYW tag=DT0KPAI3WPJS type=write size=1474382 2024-05-30T16:03:47.525 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.523+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=PUKED9O9RWGE type=write size=545956 2024-05-30T16:03:47.529 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.527+0000 7f08851138c0 1 < completed part key=_multipart_O0DJ8SCY766V.2~SEBVJCTQTB82.0 size=1048576 2024-05-30T16:03:47.532 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.531+0000 7f08851138c0 1 < completed part key=_multipart_O0DJ8SCY766V.2~SEBVJCTQTB82.1 size=155530 2024-05-30T16:03:47.538 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.539+0000 7f08851138c0 1 < completed multipart upload key=O0DJ8SCY766V upload=SEBVJCTQTB82 tag=Z9HM1KCVA8D7 type=write size=2252682 2024-05-30T16:03:47.542 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.543+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=N8TB5BJTUYAE type=del size=773560 2024-05-30T16:03:47.546 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.547+0000 7f08851138c0 1 < completed delete operation key=O8CWPJ9GW2GO tag=Y49NXQGTRL5A type=del 2024-05-30T16:03:47.553 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.551+0000 7f08851138c0 1 > prepared part key=_multipart_MMLF65KLC42Q.2~QQQER3KB5U29.0 size=15178 2024-05-30T16:03:47.553 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.551+0000 7f08851138c0 1 > prepared multipart upload key=MMLF65KLC42Q upload=QQQER3KB5U29 tag=HGC9CEA7UOSP type=write size=1063754 2024-05-30T16:03:47.560 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.559+0000 7f08851138c0 1 < suggested remove operation key=RCTV94WRMO53 tag=OP2ZLVADNSV2 type=del 2024-05-30T16:03:47.563 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.563+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=RIQQ5IWTJO3J type=del size=4188722 2024-05-30T16:03:47.567 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.567+0000 7f08851138c0 1 < completed write operation key=ZALVY70ISLM5 tag=DAJ054PJSJ1R type=write size=248277 2024-05-30T16:03:47.571 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.571+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=E7TBSLXIWFE2 type=del size=2376466 2024-05-30T16:03:47.578 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.579+0000 7f08851138c0 1 < suggested update operation key=O8CWPJ9GW2GO tag=1RJMJ78DC74G type=del size=2314956 2024-05-30T16:03:47.586 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.587+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~82EQFXA0VDOZ.0 size=1048576 2024-05-30T16:03:47.589 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.587+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~82EQFXA0VDOZ.1 size=1048576 2024-05-30T16:03:47.592 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.591+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~82EQFXA0VDOZ.2 size=248813 2024-05-30T16:03:47.592 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.591+0000 7f08851138c0 1 > prepared multipart upload key=RCTV94WRMO53 upload=82EQFXA0VDOZ tag=C0FFXU0Z3JPT type=write size=3394541 2024-05-30T16:03:47.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.595+0000 7f08851138c0 1 < completed delete operation key=603HBR8PRH6L tag=8MTY27381322 type=del 2024-05-30T16:03:47.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.603+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~EFKMXMWSQFA3.0 size=1048576 2024-05-30T16:03:47.606 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.607+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~EFKMXMWSQFA3.1 size=170596 2024-05-30T16:03:47.606 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.607+0000 7f08851138c0 1 > prepared multipart upload key=QQILXRJD0C1Z upload=EFKMXMWSQFA3 tag=FANA7AZFBXGY type=write size=2267748 2024-05-30T16:03:47.610 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.611+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~724H5OFLNSCU.0 size=1048576 2024-05-30T16:03:47.613 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.611+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~724H5OFLNSCU.1 size=875851 2024-05-30T16:03:47.619 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.619+0000 7f08851138c0 1 < completed multipart upload key=3TBOZWT2P8U8 upload=724H5OFLNSCU tag=L5VYYDT336PJ type=write size=2973003 2024-05-30T16:03:47.622 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.623+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=YBEJHBBY15XS type=del size=2828505 2024-05-30T16:03:47.626 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.627+0000 7f08851138c0 1 < completed write operation key=A5BUBY9Z7JQG tag=SOCQ7KB94H6G type=write size=271980 2024-05-30T16:03:47.633 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.635+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~QOAX8P1IGOF3.0 size=1048576 2024-05-30T16:03:47.636 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.635+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~QOAX8P1IGOF3.1 size=1048576 2024-05-30T16:03:47.643 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.643+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~QOAX8P1IGOF3.2 size=764044 2024-05-30T16:03:47.643 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.643+0000 7f08851138c0 1 > prepared multipart upload key=1N6R2HBTWMP2 upload=QOAX8P1IGOF3 tag=JH37CCZQFYY4 type=write size=3909772 2024-05-30T16:03:47.647 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.647+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~2L6UBKFJEBJ9.0 size=509481 2024-05-30T16:03:47.653 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.651+0000 7f08851138c0 1 < completed multipart upload key=Y3CYZJ83WYNI upload=2L6UBKFJEBJ9 tag=529P50MT1R0C type=write size=1558057 2024-05-30T16:03:47.656 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.655+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=BBU1BL8T8GMW type=write size=968351 2024-05-30T16:03:47.659 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.659+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~KLT13G4K60IB.0 size=1048576 2024-05-30T16:03:47.662 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.663+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~KLT13G4K60IB.1 size=1048576 2024-05-30T16:03:47.665 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.663+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~KLT13G4K60IB.2 size=1012490 2024-05-30T16:03:47.670 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.671+0000 7f08851138c0 1 < completed multipart upload key=NIV7AQ4EV5YP upload=KLT13G4K60IB tag=D5SOX0Q3AZS4 type=write size=4158218 2024-05-30T16:03:47.674 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.675+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=YID8WFS2MFGS type=del size=508792 2024-05-30T16:03:47.677 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.679+0000 7f08851138c0 1 < completed delete operation key=TPCETCO3Y1M6 tag=6VE7KMIP1EDR type=del 2024-05-30T16:03:47.681 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.679+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=MHQ54QWDCEXI type=write size=236298 2024-05-30T16:03:47.684 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.683+0000 7f08851138c0 1 < canceled operation key=GWXNCSJCLXKW tag=39CDH6RWQTV0 type=write size=514590 2024-05-30T16:03:47.687 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.687+0000 7f08851138c0 1 > prepared operation key=ZALVY70ISLM5 tag=GLJNSQDBM0PG type=del size=1872442 2024-05-30T16:03:47.690 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.691+0000 7f08851138c0 1 < completed write operation key=QCLY10D2A3VF tag=KDV8YJMO6068 type=write size=489235 2024-05-30T16:03:47.693 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.695+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=F8JO3B6H16H0 type=del size=441260 2024-05-30T16:03:47.697 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.695+0000 7f08851138c0 1 < completed write operation key=Y3CYZJ83WYNI tag=VCWDIEHJKP6O type=write size=660888 2024-05-30T16:03:47.700 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.699+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=0A6VUYWCBTVE type=write size=376218 2024-05-30T16:03:47.703 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.703+0000 7f08851138c0 1 < completed delete operation key=3TBOZWT2P8U8 tag=UYDB4AV32M8C type=del 2024-05-30T16:03:47.707 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.707+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=38Y0RQZ813TO type=del size=3075185 2024-05-30T16:03:47.711 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.711+0000 7f08851138c0 1 < completed write operation key=Y8SBI0QZ0ZV8 tag=PUKED9O9RWGE type=write size=545956 2024-05-30T16:03:47.714 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.715+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=EE8IN6O3KVHU type=del size=2701138 2024-05-30T16:03:47.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.719+0000 7f08851138c0 1 < completed delete operation key=3TBOZWT2P8U8 tag=N8TB5BJTUYAE type=del 2024-05-30T16:03:47.725 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.723+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~AB5FBJMDZ2NT.0 size=1048576 2024-05-30T16:03:47.728 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.727+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~AB5FBJMDZ2NT.1 size=41949 2024-05-30T16:03:47.728 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.727+0000 7f08851138c0 1 > prepared multipart upload key=Y3CYZJ83WYNI upload=AB5FBJMDZ2NT tag=SMZUS8IMXLHR type=write size=2139101 2024-05-30T16:03:47.732 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.731+0000 7f08851138c0 1 < completed part key=_multipart_MMLF65KLC42Q.2~QQQER3KB5U29.0 size=15178 2024-05-30T16:03:47.737 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.735+0000 7f08851138c0 1 < completed multipart upload key=MMLF65KLC42Q upload=QQQER3KB5U29 tag=HGC9CEA7UOSP type=write size=1063754 2024-05-30T16:03:47.740 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.739+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=5EPC8R8TO5ZZ type=del size=888505 2024-05-30T16:03:47.743 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.743+0000 7f08851138c0 1 < completed delete operation key=MMLF65KLC42Q tag=RIQQ5IWTJO3J type=del 2024-05-30T16:03:47.746 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.747+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=94WCGC2XDSTV type=del size=2459268 2024-05-30T16:03:47.752 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.751+0000 7f08851138c0 1 < suggested remove operation key=V4OQFOOQUXFQ tag=E7TBSLXIWFE2 type=del 2024-05-30T16:03:47.758 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.759+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~XFHRCCFARLZJ.0 size=1048576 2024-05-30T16:03:47.761 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.759+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~XFHRCCFARLZJ.1 size=469691 2024-05-30T16:03:47.761 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.759+0000 7f08851138c0 1 > prepared multipart upload key=WT9RC0UYVG4F upload=XFHRCCFARLZJ tag=UJHH48YAQK8N type=write size=2566843 2024-05-30T16:03:47.764 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.763+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~82EQFXA0VDOZ.0 size=1048576 2024-05-30T16:03:47.766 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.767+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~82EQFXA0VDOZ.1 size=1048576 2024-05-30T16:03:47.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.767+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~82EQFXA0VDOZ.2 size=248813 2024-05-30T16:03:47.773 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.771+0000 7f08851138c0 1 < completed multipart upload key=RCTV94WRMO53 upload=82EQFXA0VDOZ tag=C0FFXU0Z3JPT type=write size=3394541 2024-05-30T16:03:47.778 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.779+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~JL6U8HFVGNT0.0 size=1048576 2024-05-30T16:03:47.780 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.779+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~JL6U8HFVGNT0.1 size=492247 2024-05-30T16:03:47.780 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.779+0000 7f08851138c0 1 > prepared multipart upload key=YRP4CSG92QRO upload=JL6U8HFVGNT0 tag=J6IVK7ML9EZ1 type=write size=2589399 2024-05-30T16:03:47.783 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.783+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~EFKMXMWSQFA3.0 size=1048576 2024-05-30T16:03:47.785 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.787+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~EFKMXMWSQFA3.1 size=170596 2024-05-30T16:03:47.792 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.791+0000 7f08851138c0 1 < completed multipart upload key=QQILXRJD0C1Z upload=EFKMXMWSQFA3 tag=FANA7AZFBXGY type=write size=2267748 2024-05-30T16:03:47.795 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.795+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=ENOH0STA40QP type=del size=2425561 2024-05-30T16:03:47.798 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.799+0000 7f08851138c0 1 < completed delete operation key=QQILXRJD0C1Z tag=YBEJHBBY15XS type=del 2024-05-30T16:03:47.804 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.803+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~3J8K28WXCHG5.0 size=880761 2024-05-30T16:03:47.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.803+0000 7f08851138c0 1 > prepared multipart upload key=XHGMUYBA2VNK upload=3J8K28WXCHG5 tag=VZTOP8FC7QMW type=write size=1929337 2024-05-30T16:03:47.808 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.807+0000 7f08851138c0 1 < completed part key=_multipart_1N6R2HBTWMP2.2~QOAX8P1IGOF3.0 size=1048576 2024-05-30T16:03:47.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.811+0000 7f08851138c0 1 < completed part key=_multipart_1N6R2HBTWMP2.2~QOAX8P1IGOF3.1 size=1048576 2024-05-30T16:03:47.814 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.815+0000 7f08851138c0 1 < completed part key=_multipart_1N6R2HBTWMP2.2~QOAX8P1IGOF3.2 size=764044 2024-05-30T16:03:47.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.819+0000 7f08851138c0 1 < completed multipart upload key=1N6R2HBTWMP2 upload=QOAX8P1IGOF3 tag=JH37CCZQFYY4 type=write size=3909772 2024-05-30T16:03:47.826 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.827+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~T1D42M8UW70F.0 size=740851 2024-05-30T16:03:47.826 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.827+0000 7f08851138c0 1 > prepared multipart upload key=VRXNO8JLAZYH upload=T1D42M8UW70F tag=CQ754EMXJ7Q3 type=write size=1789427 2024-05-30T16:03:47.830 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.831+0000 7f08851138c0 1 < completed write operation key=XHGMUYBA2VNK tag=BBU1BL8T8GMW type=write size=968351 2024-05-30T16:03:47.836 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.835+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~7VAY91GGEN39.0 size=9202 2024-05-30T16:03:47.836 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.835+0000 7f08851138c0 1 > prepared multipart upload key=JNZI7JX95LX2 upload=7VAY91GGEN39 tag=TBKMR59U3ABQ type=write size=1057778 2024-05-30T16:03:47.840 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.839+0000 7f08851138c0 1 < completed delete operation key=3TBOZWT2P8U8 tag=YID8WFS2MFGS type=del 2024-05-30T16:03:47.847 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.847+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~G5Q1WBKL1OPO.0 size=1048576 2024-05-30T16:03:47.850 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.851+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~G5Q1WBKL1OPO.1 size=900841 2024-05-30T16:03:47.850 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.851+0000 7f08851138c0 1 > prepared multipart upload key=WT9RC0UYVG4F upload=G5Q1WBKL1OPO tag=46ENWIRJF0NC type=write size=2997993 2024-05-30T16:03:47.854 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.855+0000 7f08851138c0 -1 < failed to complete operation key=V4OQFOOQUXFQ tag=MHQ54QWDCEXI type=write size=236298: (22) Invalid argument 2024-05-30T16:03:47.857 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.855+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=2GM9A0WVCI9B type=del size=2655985 2024-05-30T16:03:47.861 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.859+0000 7f08851138c0 1 < canceled operation key=ZALVY70ISLM5 tag=GLJNSQDBM0PG type=del size=1872442 2024-05-30T16:03:47.864 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.863+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=CXB0XB05F69J type=del size=2338637 2024-05-30T16:03:47.868 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.867+0000 7f08851138c0 1 < completed delete operation key=E2FXSXOEUGIG tag=F8JO3B6H16H0 type=del 2024-05-30T16:03:47.875 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.875+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~LHBRTRHR0TMO.0 size=1048576 2024-05-30T16:03:47.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.879+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~LHBRTRHR0TMO.1 size=1048576 2024-05-30T16:03:47.880 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.879+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~LHBRTRHR0TMO.2 size=721654 2024-05-30T16:03:47.880 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.879+0000 7f08851138c0 1 > prepared multipart upload key=XHGMUYBA2VNK upload=LHBRTRHR0TMO tag=ZMFGANHKYEEL type=write size=3867382 2024-05-30T16:03:47.883 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.883+0000 7f08851138c0 1 < completed write operation key=WT9RC0UYVG4F tag=0A6VUYWCBTVE type=write size=376218 2024-05-30T16:03:47.887 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.887+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=VRQNGB8UGBTI type=del size=475093 2024-05-30T16:03:47.890 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.891+0000 7f08851138c0 1 < canceled operation key=JNZI7JX95LX2 tag=38Y0RQZ813TO type=del size=3075185 2024-05-30T16:03:47.894 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.895+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=CES50VYP68SZ type=del size=2186422 2024-05-30T16:03:47.898 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.899+0000 7f08851138c0 1 < canceled operation key=TPCETCO3Y1M6 tag=EE8IN6O3KVHU type=del size=2701138 2024-05-30T16:03:47.901 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.899+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=6A32RQ8ON04B type=del size=3911478 2024-05-30T16:03:47.902 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.903+0000 7f08851138c0 1 < canceled part key=_multipart_Y3CYZJ83WYNI.2~AB5FBJMDZ2NT.0 size=1048576 2024-05-30T16:03:47.905 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.903+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~AB5FBJMDZ2NT.1 size=41949 2024-05-30T16:03:47.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.911+0000 7f08851138c0 1 < canceled multipart upload key=Y3CYZJ83WYNI upload=AB5FBJMDZ2NT tag=SMZUS8IMXLHR type=write size=2139101 2024-05-30T16:03:47.917 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.915+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~UR50UCXZ49EJ.0 size=1048576 2024-05-30T16:03:47.920 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.919+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~UR50UCXZ49EJ.1 size=89840 2024-05-30T16:03:47.920 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.919+0000 7f08851138c0 1 > prepared multipart upload key=0RZRBW38WKNZ upload=UR50UCXZ49EJ tag=X34D8TIER093 type=write size=2186992 2024-05-30T16:03:47.923 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.923+0000 7f08851138c0 1 < completed delete operation key=MMLF65KLC42Q tag=5EPC8R8TO5ZZ type=del 2024-05-30T16:03:47.927 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.927+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=A1ZH6BJN9RL6 type=del size=2054555 2024-05-30T16:03:47.931 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.931+0000 7f08851138c0 1 < completed delete operation key=QCLY10D2A3VF tag=94WCGC2XDSTV type=del 2024-05-30T16:03:47.934 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.935+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=00YJ6IJC1YD5 type=del size=1854224 2024-05-30T16:03:47.938 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.939+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~XFHRCCFARLZJ.0 size=1048576 2024-05-30T16:03:47.941 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.939+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~XFHRCCFARLZJ.1 size=469691 2024-05-30T16:03:47.947 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.947+0000 7f08851138c0 1 < completed multipart upload key=WT9RC0UYVG4F upload=XFHRCCFARLZJ tag=UJHH48YAQK8N type=write size=2566843 2024-05-30T16:03:47.953 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.955+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~FQ501MYGGWKQ.0 size=1048576 2024-05-30T16:03:47.956 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.955+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~FQ501MYGGWKQ.1 size=44056 2024-05-30T16:03:47.956 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.955+0000 7f08851138c0 1 > prepared multipart upload key=V4OQFOOQUXFQ upload=FQ501MYGGWKQ tag=BMGPXC4IMGRF type=write size=2141208 2024-05-30T16:03:47.962 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.963+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~JL6U8HFVGNT0.0 size=1048576 2024-05-30T16:03:47.965 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.963+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~JL6U8HFVGNT0.1 size=492247 2024-05-30T16:03:47.971 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.971+0000 7f08851138c0 1 < completed multipart upload key=YRP4CSG92QRO upload=JL6U8HFVGNT0 tag=J6IVK7ML9EZ1 type=write size=2589399 2024-05-30T16:03:47.974 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.975+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=QEEKQUUYIZQC type=del size=2650284 2024-05-30T16:03:47.981 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.979+0000 7f08851138c0 1 < suggested remove operation key=0RZRBW38WKNZ tag=ENOH0STA40QP type=del 2024-05-30T16:03:47.987 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.987+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~POY0A3ABK3AK.0 size=1048576 2024-05-30T16:03:47.990 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.991+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~POY0A3ABK3AK.1 size=1048576 2024-05-30T16:03:47.992 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.991+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~POY0A3ABK3AK.2 size=926040 2024-05-30T16:03:47.992 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.991+0000 7f08851138c0 1 > prepared multipart upload key=ZALVY70ISLM5 upload=POY0A3ABK3AK tag=9MYKAXLYESAU type=write size=4071768 2024-05-30T16:03:47.996 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:47.995+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~3J8K28WXCHG5.0 size=880761 2024-05-30T16:03:48.002 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.003+0000 7f08851138c0 1 < completed multipart upload key=XHGMUYBA2VNK upload=3J8K28WXCHG5 tag=VZTOP8FC7QMW type=write size=1929337 2024-05-30T16:03:48.005 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.003+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=H42CAHTQY9RH type=write size=251340 2024-05-30T16:03:48.008 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.007+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~T1D42M8UW70F.0 size=740851 2024-05-30T16:03:48.014 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.015+0000 7f08851138c0 1 < completed multipart upload key=VRXNO8JLAZYH upload=T1D42M8UW70F tag=CQ754EMXJ7Q3 type=write size=1789427 2024-05-30T16:03:48.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.019+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~UI0J1OC7TM5G.0 size=1048576 2024-05-30T16:03:48.022 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.023+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~UI0J1OC7TM5G.1 size=1048576 2024-05-30T16:03:48.025 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.023+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~UI0J1OC7TM5G.2 size=458284 2024-05-30T16:03:48.025 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.023+0000 7f08851138c0 1 > prepared multipart upload key=364U3OV10VPD upload=UI0J1OC7TM5G tag=M3BPUDG316YV type=write size=3604012 2024-05-30T16:03:48.028 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.027+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~7VAY91GGEN39.0 size=9202 2024-05-30T16:03:48.033 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.035+0000 7f08851138c0 1 < completed multipart upload key=JNZI7JX95LX2 upload=7VAY91GGEN39 tag=TBKMR59U3ABQ type=write size=1057778 2024-05-30T16:03:48.039 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.039+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~8GPSICI2HCE5.0 size=331957 2024-05-30T16:03:48.039 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.039+0000 7f08851138c0 1 > prepared multipart upload key=JNZI7JX95LX2 upload=8GPSICI2HCE5 tag=OXW5SS6B1KMD type=write size=1380533 2024-05-30T16:03:48.042 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.043+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~G5Q1WBKL1OPO.0 size=1048576 2024-05-30T16:03:48.045 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.047+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~G5Q1WBKL1OPO.1 size=900841 2024-05-30T16:03:48.051 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.051+0000 7f08851138c0 1 < completed multipart upload key=WT9RC0UYVG4F upload=G5Q1WBKL1OPO tag=46ENWIRJF0NC type=write size=2997993 2024-05-30T16:03:48.057 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.055+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~6GN3VCZX2KGP.0 size=417711 2024-05-30T16:03:48.057 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.055+0000 7f08851138c0 1 > prepared multipart upload key=WT9RC0UYVG4F upload=6GN3VCZX2KGP tag=1I26KKMNT0LP type=write size=1466287 2024-05-30T16:03:48.060 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.059+0000 7f08851138c0 1 < canceled operation key=QQILXRJD0C1Z tag=2GM9A0WVCI9B type=del size=2655985 2024-05-30T16:03:48.063 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.063+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=94QXON7NO4H9 type=del size=2818873 2024-05-30T16:03:48.067 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.067+0000 7f08851138c0 1 < completed delete operation key=O8CWPJ9GW2GO tag=CXB0XB05F69J type=del 2024-05-30T16:03:48.070 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.071+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=55IN8JNDLZ59 type=del size=3709309 2024-05-30T16:03:48.074 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.075+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~LHBRTRHR0TMO.0 size=1048576 2024-05-30T16:03:48.077 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.075+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~LHBRTRHR0TMO.1 size=1048576 2024-05-30T16:03:48.080 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.079+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~LHBRTRHR0TMO.2 size=721654 2024-05-30T16:03:48.085 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.083+0000 7f08851138c0 1 < completed multipart upload key=XHGMUYBA2VNK upload=LHBRTRHR0TMO tag=ZMFGANHKYEEL type=write size=3867382 2024-05-30T16:03:48.091 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.091+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~PQDOHS3TFI8C.0 size=1048576 2024-05-30T16:03:48.094 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.095+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~PQDOHS3TFI8C.1 size=771467 2024-05-30T16:03:48.094 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.095+0000 7f08851138c0 1 > prepared multipart upload key=O0DJ8SCY766V upload=PQDOHS3TFI8C tag=BZY0YNBPBYPB type=write size=2868619 2024-05-30T16:03:48.097 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.099+0000 7f08851138c0 1 < completed delete operation key=WT9RC0UYVG4F tag=VRQNGB8UGBTI type=del 2024-05-30T16:03:48.100 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.099+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=4WW3YFGNYTXR type=write size=19519 2024-05-30T16:03:48.104 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.103+0000 7f08851138c0 1 < canceled operation key=GWXNCSJCLXKW tag=CES50VYP68SZ type=del size=2186422 2024-05-30T16:03:48.107 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.107+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=EHCMA8GZIEZ1 type=del size=864546 2024-05-30T16:03:48.111 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.111+0000 7f08851138c0 1 < completed delete operation key=A5BUBY9Z7JQG tag=6A32RQ8ON04B type=del 2024-05-30T16:03:48.114 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.115+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=MIMX55416B1M type=del size=937156 2024-05-30T16:03:48.118 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.119+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~UR50UCXZ49EJ.0 size=1048576 2024-05-30T16:03:48.121 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.119+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~UR50UCXZ49EJ.1 size=89840 2024-05-30T16:03:48.126 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.127+0000 7f08851138c0 1 < completed multipart upload key=0RZRBW38WKNZ upload=UR50UCXZ49EJ tag=X34D8TIER093 type=write size=2186992 2024-05-30T16:03:48.130 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.131+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=A9D5HDE8ON6O type=write size=267946 2024-05-30T16:03:48.134 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.135+0000 7f08851138c0 1 < completed delete operation key=3IJWPWRB6FHE tag=A1ZH6BJN9RL6 type=del 2024-05-30T16:03:48.140 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.139+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~OCD1A991ESDQ.0 size=1048576 2024-05-30T16:03:48.143 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.143+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~OCD1A991ESDQ.1 size=94916 2024-05-30T16:03:48.143 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.143+0000 7f08851138c0 1 > prepared multipart upload key=XHGMUYBA2VNK upload=OCD1A991ESDQ tag=K73SKT20B0P6 type=write size=2192068 2024-05-30T16:03:48.146 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.147+0000 7f08851138c0 1 < completed delete operation key=RCTV94WRMO53 tag=00YJ6IJC1YD5 type=del 2024-05-30T16:03:48.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.151+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=W2N5FX2BEPWB type=del size=2994185 2024-05-30T16:03:48.153 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.151+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~FQ501MYGGWKQ.0 size=1048576 2024-05-30T16:03:48.156 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.155+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~FQ501MYGGWKQ.1 size=44056 2024-05-30T16:03:48.161 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.159+0000 7f08851138c0 1 < completed multipart upload key=V4OQFOOQUXFQ upload=FQ501MYGGWKQ tag=BMGPXC4IMGRF type=write size=2141208 2024-05-30T16:03:48.164 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.163+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=ZX5BSNDPRBM9 type=write size=86868 2024-05-30T16:03:48.168 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.167+0000 7f08851138c0 1 < completed delete operation key=364U3OV10VPD tag=QEEKQUUYIZQC type=del 2024-05-30T16:03:48.174 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.175+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~O9C1PP91YA34.0 size=376099 2024-05-30T16:03:48.174 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.175+0000 7f08851138c0 1 > prepared multipart upload key=3TBOZWT2P8U8 upload=O9C1PP91YA34 tag=1PNMXZIQS47D type=write size=1424675 2024-05-30T16:03:48.177 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.175+0000 7f08851138c0 1 < completed part key=_multipart_ZALVY70ISLM5.2~POY0A3ABK3AK.0 size=1048576 2024-05-30T16:03:48.180 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.179+0000 7f08851138c0 1 < completed part key=_multipart_ZALVY70ISLM5.2~POY0A3ABK3AK.1 size=1048576 2024-05-30T16:03:48.182 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.183+0000 7f08851138c0 1 < completed part key=_multipart_ZALVY70ISLM5.2~POY0A3ABK3AK.2 size=926040 2024-05-30T16:03:48.188 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.187+0000 7f08851138c0 1 < completed multipart upload key=ZALVY70ISLM5 upload=POY0A3ABK3AK tag=9MYKAXLYESAU type=write size=4071768 2024-05-30T16:03:48.193 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.191+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~C1MLIIXZM7FA.0 size=1048576 2024-05-30T16:03:48.196 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.195+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~C1MLIIXZM7FA.1 size=125699 2024-05-30T16:03:48.196 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.195+0000 7f08851138c0 1 > prepared multipart upload key=0RZRBW38WKNZ upload=C1MLIIXZM7FA tag=T13U2OAGE06U type=write size=2222851 2024-05-30T16:03:48.199 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.199+0000 7f08851138c0 1 < completed write operation key=Y8SBI0QZ0ZV8 tag=H42CAHTQY9RH type=write size=251340 2024-05-30T16:03:48.202 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.203+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=BMYQGD4L6858 type=del size=3927206 2024-05-30T16:03:48.205 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.207+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~UI0J1OC7TM5G.0 size=1048576 2024-05-30T16:03:48.208 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.207+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~UI0J1OC7TM5G.1 size=1048576 2024-05-30T16:03:48.211 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.211+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~UI0J1OC7TM5G.2 size=458284 2024-05-30T16:03:48.216 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.215+0000 7f08851138c0 1 < completed multipart upload key=364U3OV10VPD upload=UI0J1OC7TM5G tag=M3BPUDG316YV type=write size=3604012 2024-05-30T16:03:48.220 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.219+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=BWGT2KY8LX9F type=write size=651185 2024-05-30T16:03:48.223 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.223+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~8GPSICI2HCE5.0 size=331957 2024-05-30T16:03:48.228 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.227+0000 7f08851138c0 1 < completed multipart upload key=JNZI7JX95LX2 upload=8GPSICI2HCE5 tag=OXW5SS6B1KMD type=write size=1380533 2024-05-30T16:03:48.232 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.231+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=OK7MBYLGRU8P type=del size=1461639 2024-05-30T16:03:48.235 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.235+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~6GN3VCZX2KGP.0 size=417711 2024-05-30T16:03:48.241 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.239+0000 7f08851138c0 1 < completed multipart upload key=WT9RC0UYVG4F upload=6GN3VCZX2KGP tag=1I26KKMNT0LP type=write size=1466287 2024-05-30T16:03:48.247 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.247+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~L52U0OZBKCMO.0 size=899070 2024-05-30T16:03:48.247 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.247+0000 7f08851138c0 1 > prepared multipart upload key=O8CWPJ9GW2GO upload=L52U0OZBKCMO tag=4NHWQWMMBV3T type=write size=1947646 2024-05-30T16:03:48.253 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.255+0000 7f08851138c0 1 < suggested update operation key=3IJWPWRB6FHE tag=94QXON7NO4H9 type=del size=2818873 2024-05-30T16:03:48.256 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.255+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=SZINI2SP9WQA type=del size=3810733 2024-05-30T16:03:48.263 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.263+0000 7f08851138c0 1 < suggested update operation key=O0DJ8SCY766V tag=55IN8JNDLZ59 type=del size=3709309 2024-05-30T16:03:48.269 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.267+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=N7NKELJPGG1S type=del size=3874978 2024-05-30T16:03:48.269 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.271+0000 7f08851138c0 1 < canceled part key=_multipart_O0DJ8SCY766V.2~PQDOHS3TFI8C.0 size=1048576 2024-05-30T16:03:48.272 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.271+0000 7f08851138c0 1 < completed part key=_multipart_O0DJ8SCY766V.2~PQDOHS3TFI8C.1 size=771467 2024-05-30T16:03:48.278 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.279+0000 7f08851138c0 1 < canceled multipart upload key=O0DJ8SCY766V upload=PQDOHS3TFI8C tag=BZY0YNBPBYPB type=write size=2868619 2024-05-30T16:03:48.282 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.283+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=HR2IYB4S7ODO type=del size=3568979 2024-05-30T16:03:48.285 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.287+0000 7f08851138c0 1 < completed write operation key=4N58ZY4E1H6E tag=4WW3YFGNYTXR type=write size=19519 2024-05-30T16:03:48.288 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.287+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=NLJBCBJZ7KDT type=del size=2742294 2024-05-30T16:03:48.295 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.295+0000 7f08851138c0 1 < suggested update operation key=PLWFSKPKVO19 tag=EHCMA8GZIEZ1 type=del size=864546 2024-05-30T16:03:48.301 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.299+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~NCALIAYD2XUS.0 size=14162 2024-05-30T16:03:48.301 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.299+0000 7f08851138c0 1 > prepared multipart upload key=364U3OV10VPD upload=NCALIAYD2XUS tag=AWQ42B78S4VB type=write size=1062738 2024-05-30T16:03:48.308 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.307+0000 7f08851138c0 1 < suggested update operation key=73FJ3HGWYQQK tag=MIMX55416B1M type=del size=937156 2024-05-30T16:03:48.312 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.311+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=VC0U1TR9EV6D type=del size=848203 2024-05-30T16:03:48.319 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.319+0000 7f08851138c0 1 < suggested update operation key=0RZRBW38WKNZ tag=A9D5HDE8ON6O type=write size=267946 2024-05-30T16:03:48.326 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.327+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~HCCTKS7784VV.0 size=850249 2024-05-30T16:03:48.326 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.327+0000 7f08851138c0 1 > prepared multipart upload key=0RZRBW38WKNZ upload=HCCTKS7784VV tag=8HIRJ64NUS9C type=write size=1898825 2024-05-30T16:03:48.329 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.331+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~OCD1A991ESDQ.0 size=1048576 2024-05-30T16:03:48.332 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.331+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~OCD1A991ESDQ.1 size=94916 2024-05-30T16:03:48.338 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.339+0000 7f08851138c0 1 < completed multipart upload key=XHGMUYBA2VNK upload=OCD1A991ESDQ tag=K73SKT20B0P6 type=write size=2192068 2024-05-30T16:03:48.344 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.343+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~N74SVCSSZALN.0 size=1048576 2024-05-30T16:03:48.346 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.347+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~N74SVCSSZALN.1 size=737067 2024-05-30T16:03:48.347 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.347+0000 7f08851138c0 1 > prepared multipart upload key=Y3CYZJ83WYNI upload=N74SVCSSZALN tag=0U80YT9QLWYZ type=write size=2834219 2024-05-30T16:03:48.350 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.351+0000 7f08851138c0 1 < completed delete operation key=RCTV94WRMO53 tag=W2N5FX2BEPWB type=del 2024-05-30T16:03:48.356 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.355+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~S80I2G3SE9FC.0 size=1048576 2024-05-30T16:03:48.359 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.359+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~S80I2G3SE9FC.1 size=171838 2024-05-30T16:03:48.359 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.359+0000 7f08851138c0 1 > prepared multipart upload key=603HBR8PRH6L upload=S80I2G3SE9FC tag=69MN3TKV24QG type=write size=2268990 2024-05-30T16:03:48.363 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.363+0000 7f08851138c0 1 < completed write operation key=O8CWPJ9GW2GO tag=ZX5BSNDPRBM9 type=write size=86868 2024-05-30T16:03:48.369 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.371+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~ZYUD8WQMHC1F.0 size=1048576 2024-05-30T16:03:48.372 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.371+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~ZYUD8WQMHC1F.1 size=736244 2024-05-30T16:03:48.372 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.371+0000 7f08851138c0 1 > prepared multipart upload key=O0DJ8SCY766V upload=ZYUD8WQMHC1F tag=66XCPL8JHTZT type=write size=2833396 2024-05-30T16:03:48.376 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.375+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~O9C1PP91YA34.0 size=376099 2024-05-30T16:03:48.382 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.383+0000 7f08851138c0 1 < completed multipart upload key=3TBOZWT2P8U8 upload=O9C1PP91YA34 tag=1PNMXZIQS47D type=write size=1424675 2024-05-30T16:03:48.385 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.383+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=JSFQ2FMZN7LQ type=write size=989568 2024-05-30T16:03:48.388 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.387+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~C1MLIIXZM7FA.0 size=1048576 2024-05-30T16:03:48.391 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.391+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~C1MLIIXZM7FA.1 size=125699 2024-05-30T16:03:48.396 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.395+0000 7f08851138c0 1 < completed multipart upload key=0RZRBW38WKNZ upload=C1MLIIXZM7FA tag=T13U2OAGE06U type=write size=2222851 2024-05-30T16:03:48.400 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.399+0000 7f08851138c0 1 > prepared operation key=VRXNO8JLAZYH tag=T9ZFNIYNWM9V type=del size=1765599 2024-05-30T16:03:48.407 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.407+0000 7f08851138c0 1 < suggested update operation key=PLWFSKPKVO19 tag=BMYQGD4L6858 type=del size=3927206 2024-05-30T16:03:48.411 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.411+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=JDY8V20JOEMI type=write size=236422 2024-05-30T16:03:48.415 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.415+0000 7f08851138c0 1 < completed write operation key=RCTV94WRMO53 tag=BWGT2KY8LX9F type=write size=651185 2024-05-30T16:03:48.418 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.419+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=YQGBXY9RTPRA type=write size=756356 2024-05-30T16:03:48.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.423+0000 7f08851138c0 1 < completed delete operation key=Y3CYZJ83WYNI tag=OK7MBYLGRU8P type=del 2024-05-30T16:03:48.428 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.427+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~6P86GGS7OY1B.0 size=635084 2024-05-30T16:03:48.428 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.427+0000 7f08851138c0 1 > prepared multipart upload key=O0DJ8SCY766V upload=6P86GGS7OY1B tag=Q1H66WBGABTT type=write size=1683660 2024-05-30T16:03:48.432 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.431+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~L52U0OZBKCMO.0 size=899070 2024-05-30T16:03:48.438 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.439+0000 7f08851138c0 1 < completed multipart upload key=O8CWPJ9GW2GO upload=L52U0OZBKCMO tag=4NHWQWMMBV3T type=write size=1947646 2024-05-30T16:03:48.441 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.439+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=10C2I3CBZH62 type=del size=347009 2024-05-30T16:03:48.444 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.443+0000 7f08851138c0 1 < completed delete operation key=364U3OV10VPD tag=SZINI2SP9WQA type=del 2024-05-30T16:03:48.451 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.451+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~N9AH4Z8XA469.0 size=1048576 2024-05-30T16:03:48.454 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.455+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~N9AH4Z8XA469.1 size=50002 2024-05-30T16:03:48.454 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.455+0000 7f08851138c0 1 > prepared multipart upload key=VRXNO8JLAZYH upload=N9AH4Z8XA469 tag=5OZQ2RI3YU78 type=write size=2147154 2024-05-30T16:03:48.457 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.455+0000 7f08851138c0 1 < canceled operation key=603HBR8PRH6L tag=N7NKELJPGG1S type=del size=3874978 2024-05-30T16:03:48.461 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.459+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=SQ3TCN56G4Z2 type=del size=2468785 2024-05-30T16:03:48.464 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.463+0000 7f08851138c0 1 < completed delete operation key=4N58ZY4E1H6E tag=HR2IYB4S7ODO type=del 2024-05-30T16:03:48.471 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.471+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~31UI9R18QVN0.0 size=1048576 2024-05-30T16:03:48.474 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.475+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~31UI9R18QVN0.1 size=1048576 2024-05-30T16:03:48.477 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.475+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~31UI9R18QVN0.2 size=327149 2024-05-30T16:03:48.477 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.475+0000 7f08851138c0 1 > prepared multipart upload key=V4OQFOOQUXFQ upload=31UI9R18QVN0 tag=F0MCVSOH74QN type=write size=3472877 2024-05-30T16:03:48.480 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.479+0000 7f08851138c0 -1 < failed to complete operation key=0RZRBW38WKNZ tag=NLJBCBJZ7KDT type=del size=2742294: (22) Invalid argument 2024-05-30T16:03:48.484 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.483+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=3X2XMKC88TUS type=write size=106711 2024-05-30T16:03:48.488 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.487+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~NCALIAYD2XUS.0 size=14162 2024-05-30T16:03:48.494 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.495+0000 7f08851138c0 1 < completed multipart upload key=364U3OV10VPD upload=NCALIAYD2XUS tag=AWQ42B78S4VB type=write size=1062738 2024-05-30T16:03:48.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.499+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~8G2X03E3IMUI.0 size=1048576 2024-05-30T16:03:48.503 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.503+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~8G2X03E3IMUI.1 size=551083 2024-05-30T16:03:48.503 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.503+0000 7f08851138c0 1 > prepared multipart upload key=Y3CYZJ83WYNI upload=8G2X03E3IMUI tag=4JYFP8ZHZPLI type=write size=2648235 2024-05-30T16:03:48.507 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.507+0000 7f08851138c0 1 < canceled operation key=MMLF65KLC42Q tag=VC0U1TR9EV6D type=del size=848203 2024-05-30T16:03:48.513 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.511+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~OE2ZSEHVFY2K.0 size=1048576 2024-05-30T16:03:48.515 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.515+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~OE2ZSEHVFY2K.1 size=1048576 2024-05-30T16:03:48.518 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.519+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~OE2ZSEHVFY2K.2 size=782612 2024-05-30T16:03:48.518 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.519+0000 7f08851138c0 1 > prepared multipart upload key=KPHZGFJLC2E0 upload=OE2ZSEHVFY2K tag=0KCX0H0041EF type=write size=3928340 2024-05-30T16:03:48.521 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.523+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~HCCTKS7784VV.0 size=850249 2024-05-30T16:03:48.526 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.527+0000 7f08851138c0 1 < completed multipart upload key=0RZRBW38WKNZ upload=HCCTKS7784VV tag=8HIRJ64NUS9C type=write size=1898825 2024-05-30T16:03:48.530 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.531+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=6ANBTZPW1OWC type=del size=2297334 2024-05-30T16:03:48.533 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.535+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~N74SVCSSZALN.0 size=1048576 2024-05-30T16:03:48.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.535+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~N74SVCSSZALN.1 size=737067 2024-05-30T16:03:48.542 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.543+0000 7f08851138c0 1 < completed multipart upload key=Y3CYZJ83WYNI upload=N74SVCSSZALN tag=0U80YT9QLWYZ type=write size=2834219 2024-05-30T16:03:48.548 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.547+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~LTRGZBDU3WVM.0 size=951155 2024-05-30T16:03:48.548 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.547+0000 7f08851138c0 1 > prepared multipart upload key=0RZRBW38WKNZ upload=LTRGZBDU3WVM tag=2REXI1MH9LL0 type=write size=1999731 2024-05-30T16:03:48.551 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.551+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~S80I2G3SE9FC.0 size=1048576 2024-05-30T16:03:48.554 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.555+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~S80I2G3SE9FC.1 size=171838 2024-05-30T16:03:48.560 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.559+0000 7f08851138c0 1 < completed multipart upload key=603HBR8PRH6L upload=S80I2G3SE9FC tag=69MN3TKV24QG type=write size=2268990 2024-05-30T16:03:48.564 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.563+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=GGKYLWGA5E1M type=del size=420389 2024-05-30T16:03:48.567 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.567+0000 7f08851138c0 1 < completed part key=_multipart_O0DJ8SCY766V.2~ZYUD8WQMHC1F.0 size=1048576 2024-05-30T16:03:48.571 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.571+0000 7f08851138c0 1 < completed part key=_multipart_O0DJ8SCY766V.2~ZYUD8WQMHC1F.1 size=736244 2024-05-30T16:03:48.578 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.579+0000 7f08851138c0 1 < completed multipart upload key=O0DJ8SCY766V upload=ZYUD8WQMHC1F tag=66XCPL8JHTZT type=write size=2833396 2024-05-30T16:03:48.581 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.579+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=AWGVR1ROGJO9 type=del size=1881750 2024-05-30T16:03:48.585 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.583+0000 7f08851138c0 1 < completed write operation key=1N6R2HBTWMP2 tag=JSFQ2FMZN7LQ type=write size=989568 2024-05-30T16:03:48.588 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.587+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=0KX7RD0V6UKB type=del size=2811392 2024-05-30T16:03:48.594 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.595+0000 7f08851138c0 1 < completed delete operation key=VRXNO8JLAZYH tag=T9ZFNIYNWM9V type=del 2024-05-30T16:03:48.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.599+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=5U4V8JFNILSN type=del size=1871690 2024-05-30T16:03:48.601 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.603+0000 7f08851138c0 1 < completed write operation key=O8CWPJ9GW2GO tag=JDY8V20JOEMI type=write size=236422 2024-05-30T16:03:48.605 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.603+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=D3HL2E78B9KM type=del size=2127144 2024-05-30T16:03:48.608 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.607+0000 7f08851138c0 1 < canceled operation key=MMLF65KLC42Q tag=YQGBXY9RTPRA type=write size=756356 2024-05-30T16:03:48.612 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.611+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=JWPEJXJMU5QR type=write size=926633 2024-05-30T16:03:48.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.615+0000 7f08851138c0 1 < completed part key=_multipart_O0DJ8SCY766V.2~6P86GGS7OY1B.0 size=635084 2024-05-30T16:03:48.621 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.623+0000 7f08851138c0 1 < completed multipart upload key=O0DJ8SCY766V upload=6P86GGS7OY1B tag=Q1H66WBGABTT type=write size=1683660 2024-05-30T16:03:48.628 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.627+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~1Y4QCOL7VC5N.0 size=1048576 2024-05-30T16:03:48.630 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.631+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~1Y4QCOL7VC5N.1 size=1048576 2024-05-30T16:03:48.633 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.631+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~1Y4QCOL7VC5N.2 size=467143 2024-05-30T16:03:48.633 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.631+0000 7f08851138c0 1 > prepared multipart upload key=QCLY10D2A3VF upload=1Y4QCOL7VC5N tag=BT3LT82F1LYR type=write size=3612871 2024-05-30T16:03:48.637 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.635+0000 7f08851138c0 1 < completed delete operation key=OKBB0VU765TW tag=10C2I3CBZH62 type=del 2024-05-30T16:03:48.640 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.639+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=H8COM1APM0NL type=del size=1713609 2024-05-30T16:03:48.645 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.643+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~N9AH4Z8XA469.0 size=1048576 2024-05-30T16:03:48.648 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.647+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~N9AH4Z8XA469.1 size=50002 2024-05-30T16:03:48.653 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.651+0000 7f08851138c0 1 < completed multipart upload key=VRXNO8JLAZYH upload=N9AH4Z8XA469 tag=5OZQ2RI3YU78 type=write size=2147154 2024-05-30T16:03:48.658 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.655+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=OFGO96F4BBIP type=del size=3347418 2024-05-30T16:03:48.665 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.663+0000 7f08851138c0 1 < suggested remove operation key=O8CWPJ9GW2GO tag=SQ3TCN56G4Z2 type=del 2024-05-30T16:03:48.669 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.671+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=CTSW07CON612 type=del size=3456507 2024-05-30T16:03:48.674 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.675+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~31UI9R18QVN0.0 size=1048576 2024-05-30T16:03:48.676 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.675+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~31UI9R18QVN0.1 size=1048576 2024-05-30T16:03:48.679 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.679+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~31UI9R18QVN0.2 size=327149 2024-05-30T16:03:48.685 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.683+0000 7f08851138c0 1 < completed multipart upload key=V4OQFOOQUXFQ upload=31UI9R18QVN0 tag=F0MCVSOH74QN type=write size=3472877 2024-05-30T16:03:48.688 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.687+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=0P6G55G6NQ4T type=del size=2501328 2024-05-30T16:03:48.692 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.691+0000 7f08851138c0 1 < completed write operation key=QQILXRJD0C1Z tag=3X2XMKC88TUS type=write size=106711 2024-05-30T16:03:48.696 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.695+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=RTIQI954Z3O0 type=write size=251091 2024-05-30T16:03:48.700 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.699+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~8G2X03E3IMUI.0 size=1048576 2024-05-30T16:03:48.701 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.699+0000 7f08851138c0 1 < canceled part key=_multipart_Y3CYZJ83WYNI.2~8G2X03E3IMUI.1 size=551083 2024-05-30T16:03:48.706 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.707+0000 7f08851138c0 1 < canceled multipart upload key=Y3CYZJ83WYNI upload=8G2X03E3IMUI tag=4JYFP8ZHZPLI type=write size=2648235 2024-05-30T16:03:48.712 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.711+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~W7D59CJ8YZ9W.0 size=1048576 2024-05-30T16:03:48.715 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.715+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~W7D59CJ8YZ9W.1 size=1048576 2024-05-30T16:03:48.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.719+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~W7D59CJ8YZ9W.2 size=755238 2024-05-30T16:03:48.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.719+0000 7f08851138c0 1 > prepared multipart upload key=OKBB0VU765TW upload=W7D59CJ8YZ9W tag=Y1X3NQ01SQMA type=write size=3900966 2024-05-30T16:03:48.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.719+0000 7f08851138c0 1 < canceled part key=_multipart_KPHZGFJLC2E0.2~OE2ZSEHVFY2K.0 size=1048576 2024-05-30T16:03:48.722 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.723+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~OE2ZSEHVFY2K.1 size=1048576 2024-05-30T16:03:48.722 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.723+0000 7f08851138c0 1 < canceled part key=_multipart_KPHZGFJLC2E0.2~OE2ZSEHVFY2K.2 size=782612 2024-05-30T16:03:48.727 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.727+0000 7f08851138c0 1 < canceled multipart upload key=KPHZGFJLC2E0 upload=OE2ZSEHVFY2K tag=0KCX0H0041EF type=write size=3928340 2024-05-30T16:03:48.734 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.735+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~8HKHW0H9VUQ9.0 size=1048576 2024-05-30T16:03:48.737 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.735+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~8HKHW0H9VUQ9.1 size=1048576 2024-05-30T16:03:48.739 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.739+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~8HKHW0H9VUQ9.2 size=400120 2024-05-30T16:03:48.740 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.739+0000 7f08851138c0 1 > prepared multipart upload key=O8CWPJ9GW2GO upload=8HKHW0H9VUQ9 tag=S98FU0AN9GCX type=write size=3545848 2024-05-30T16:03:48.744 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.743+0000 7f08851138c0 1 < completed delete operation key=OKBB0VU765TW tag=6ANBTZPW1OWC type=del 2024-05-30T16:03:48.750 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.751+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~52RXNTOEB1KC.0 size=1048576 2024-05-30T16:03:48.753 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.751+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~52RXNTOEB1KC.1 size=130085 2024-05-30T16:03:48.753 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.751+0000 7f08851138c0 1 > prepared multipart upload key=0RZRBW38WKNZ upload=52RXNTOEB1KC tag=BDM8QVU7ZD21 type=write size=2227237 2024-05-30T16:03:48.756 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.755+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~LTRGZBDU3WVM.0 size=951155 2024-05-30T16:03:48.763 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.763+0000 7f08851138c0 1 < completed multipart upload key=0RZRBW38WKNZ upload=LTRGZBDU3WVM tag=2REXI1MH9LL0 type=write size=1999731 2024-05-30T16:03:48.770 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.767+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~VDDRNJ2IJCRO.0 size=1048576 2024-05-30T16:03:48.772 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.771+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~VDDRNJ2IJCRO.1 size=752646 2024-05-30T16:03:48.772 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.771+0000 7f08851138c0 1 > prepared multipart upload key=QCLY10D2A3VF upload=VDDRNJ2IJCRO tag=JXKHBAHR9UX7 type=write size=2849798 2024-05-30T16:03:48.779 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.779+0000 7f08851138c0 1 < suggested update operation key=0RZRBW38WKNZ tag=GGKYLWGA5E1M type=del size=420389 2024-05-30T16:03:48.785 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.787+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~62K12IS32P2P.0 size=1048576 2024-05-30T16:03:48.788 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.787+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~62K12IS32P2P.1 size=1048576 2024-05-30T16:03:48.790 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.791+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~62K12IS32P2P.2 size=931472 2024-05-30T16:03:48.791 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.791+0000 7f08851138c0 1 > prepared multipart upload key=JNZI7JX95LX2 upload=62K12IS32P2P tag=R7IHCC201R9O type=write size=4077200 2024-05-30T16:03:48.794 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.795+0000 7f08851138c0 -1 < failed to complete operation key=O8CWPJ9GW2GO tag=AWGVR1ROGJO9 type=del size=1881750: (22) Invalid argument 2024-05-30T16:03:48.798 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.799+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=O85UZ0TGT2J4 type=del size=1925368 2024-05-30T16:03:48.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.803+0000 7f08851138c0 1 < canceled operation key=XHGMUYBA2VNK tag=0KX7RD0V6UKB type=del size=2811392 2024-05-30T16:03:48.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.807+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~4UA6S1S1LM36.0 size=1048576 2024-05-30T16:03:48.812 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.811+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~4UA6S1S1LM36.1 size=1048576 2024-05-30T16:03:48.815 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.815+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~4UA6S1S1LM36.2 size=454892 2024-05-30T16:03:48.815 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.815+0000 7f08851138c0 1 > prepared multipart upload key=603HBR8PRH6L upload=4UA6S1S1LM36 tag=1BB37CPHQIMH type=write size=3600620 2024-05-30T16:03:48.819 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.819+0000 7f08851138c0 1 < completed delete operation key=0T2E9F3K1USF tag=5U4V8JFNILSN type=del 2024-05-30T16:03:48.823 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.823+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=01GJ32AYO4TK type=write size=703852 2024-05-30T16:03:48.826 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.827+0000 7f08851138c0 1 < completed delete operation key=364U3OV10VPD tag=D3HL2E78B9KM type=del 2024-05-30T16:03:48.830 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.831+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=A49JDRR4P36C type=del size=3077408 2024-05-30T16:03:48.834 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.835+0000 7f08851138c0 1 < completed write operation key=JNZI7JX95LX2 tag=JWPEJXJMU5QR type=write size=926633 2024-05-30T16:03:48.837 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.839+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=M9O7TUK0DAE6 type=write size=869415 2024-05-30T16:03:48.838 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.839+0000 7f08851138c0 1 < canceled part key=_multipart_QCLY10D2A3VF.2~1Y4QCOL7VC5N.0 size=1048576 2024-05-30T16:03:48.841 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.843+0000 7f08851138c0 1 < completed part key=_multipart_QCLY10D2A3VF.2~1Y4QCOL7VC5N.1 size=1048576 2024-05-30T16:03:48.844 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.843+0000 7f08851138c0 1 < completed part key=_multipart_QCLY10D2A3VF.2~1Y4QCOL7VC5N.2 size=467143 2024-05-30T16:03:48.850 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.851+0000 7f08851138c0 1 < canceled multipart upload key=QCLY10D2A3VF upload=1Y4QCOL7VC5N tag=BT3LT82F1LYR type=write size=3612871 2024-05-30T16:03:48.857 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.855+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~58106JXWAQDU.0 size=1048576 2024-05-30T16:03:48.860 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.859+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~58106JXWAQDU.1 size=1048576 2024-05-30T16:03:48.862 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.863+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~58106JXWAQDU.2 size=956883 2024-05-30T16:03:48.863 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.863+0000 7f08851138c0 1 > prepared multipart upload key=O8CWPJ9GW2GO upload=58106JXWAQDU tag=MRTR4Y013C7M type=write size=4102611 2024-05-30T16:03:48.866 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.867+0000 7f08851138c0 1 < completed delete operation key=73FJ3HGWYQQK tag=H8COM1APM0NL type=del 2024-05-30T16:03:48.870 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.871+0000 7f08851138c0 1 > prepared operation key=ZALVY70ISLM5 tag=FGUZ1IY0VZA1 type=del size=1136831 2024-05-30T16:03:48.873 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.875+0000 7f08851138c0 1 < completed delete operation key=JNZI7JX95LX2 tag=OFGO96F4BBIP type=del 2024-05-30T16:03:48.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.875+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=P9QKGE6KRUGM type=del size=913084 2024-05-30T16:03:48.881 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.879+0000 7f08851138c0 1 < completed delete operation key=V4OQFOOQUXFQ tag=CTSW07CON612 type=del 2024-05-30T16:03:48.884 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.883+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=D23V2R4UIVEB type=del size=2883175 2024-05-30T16:03:48.888 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.887+0000 7f08851138c0 1 < completed delete operation key=QQILXRJD0C1Z tag=0P6G55G6NQ4T type=del 2024-05-30T16:03:48.892 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.891+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=PT9OD0TXXU7I type=del size=3517585 2024-05-30T16:03:48.895 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.895+0000 7f08851138c0 1 < completed write operation key=0T2E9F3K1USF tag=RTIQI954Z3O0 type=write size=251091 2024-05-30T16:03:48.899 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.899+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=C3MYWIZCKB1S type=del size=1037336 2024-05-30T16:03:48.903 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.903+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~W7D59CJ8YZ9W.0 size=1048576 2024-05-30T16:03:48.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.907+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~W7D59CJ8YZ9W.1 size=1048576 2024-05-30T16:03:48.909 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.907+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~W7D59CJ8YZ9W.2 size=755238 2024-05-30T16:03:48.915 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.915+0000 7f08851138c0 1 < completed multipart upload key=OKBB0VU765TW upload=W7D59CJ8YZ9W tag=Y1X3NQ01SQMA type=write size=3900966 2024-05-30T16:03:48.921 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.923+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~3C2OK8ZQP3SY.0 size=934815 2024-05-30T16:03:48.922 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.923+0000 7f08851138c0 1 > prepared multipart upload key=Y3CYZJ83WYNI upload=3C2OK8ZQP3SY tag=0HNMIK11HE6R type=write size=1983391 2024-05-30T16:03:48.926 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.927+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~8HKHW0H9VUQ9.0 size=1048576 2024-05-30T16:03:48.931 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.931+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~8HKHW0H9VUQ9.1 size=1048576 2024-05-30T16:03:48.934 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.935+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~8HKHW0H9VUQ9.2 size=400120 2024-05-30T16:03:48.941 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.939+0000 7f08851138c0 1 < completed multipart upload key=O8CWPJ9GW2GO upload=8HKHW0H9VUQ9 tag=S98FU0AN9GCX type=write size=3545848 2024-05-30T16:03:48.944 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.943+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=M3YEWPYFP1PB type=del size=1418143 2024-05-30T16:03:48.948 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.947+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~52RXNTOEB1KC.0 size=1048576 2024-05-30T16:03:48.948 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.947+0000 7f08851138c0 1 < canceled part key=_multipart_0RZRBW38WKNZ.2~52RXNTOEB1KC.1 size=130085 2024-05-30T16:03:48.954 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.955+0000 7f08851138c0 1 < canceled multipart upload key=0RZRBW38WKNZ upload=52RXNTOEB1KC tag=BDM8QVU7ZD21 type=write size=2227237 2024-05-30T16:03:48.959 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.959+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=OP6W64B03ISD type=del size=520239 2024-05-30T16:03:48.962 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.963+0000 7f08851138c0 1 < completed part key=_multipart_QCLY10D2A3VF.2~VDDRNJ2IJCRO.0 size=1048576 2024-05-30T16:03:48.965 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.967+0000 7f08851138c0 1 < completed part key=_multipart_QCLY10D2A3VF.2~VDDRNJ2IJCRO.1 size=752646 2024-05-30T16:03:48.972 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.971+0000 7f08851138c0 1 < completed multipart upload key=QCLY10D2A3VF upload=VDDRNJ2IJCRO tag=JXKHBAHR9UX7 type=write size=2849798 2024-05-30T16:03:48.978 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.979+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~TUS0J83UA0VT.0 size=1048576 2024-05-30T16:03:48.981 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.983+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~TUS0J83UA0VT.1 size=915783 2024-05-30T16:03:48.982 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.983+0000 7f08851138c0 1 > prepared multipart upload key=RCTV94WRMO53 upload=TUS0J83UA0VT tag=B2JRHYZCIJDA type=write size=3012935 2024-05-30T16:03:48.985 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.987+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~62K12IS32P2P.0 size=1048576 2024-05-30T16:03:48.988 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.987+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~62K12IS32P2P.1 size=1048576 2024-05-30T16:03:48.991 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.991+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~62K12IS32P2P.2 size=931472 2024-05-30T16:03:48.997 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:48.995+0000 7f08851138c0 1 < completed multipart upload key=JNZI7JX95LX2 upload=62K12IS32P2P tag=R7IHCC201R9O type=write size=4077200 2024-05-30T16:03:49.003 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.003+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~EG6ZT9UYYY5M.0 size=1048576 2024-05-30T16:03:49.006 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.007+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~EG6ZT9UYYY5M.1 size=848203 2024-05-30T16:03:49.006 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.007+0000 7f08851138c0 1 > prepared multipart upload key=3IJWPWRB6FHE upload=EG6ZT9UYYY5M tag=G6FMA4YL0B41 type=write size=2945355 2024-05-30T16:03:49.013 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.011+0000 7f08851138c0 1 < suggested remove operation key=QQILXRJD0C1Z tag=O85UZ0TGT2J4 type=del 2024-05-30T16:03:49.017 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.015+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=L1NB38PLHG0D type=del size=3127789 2024-05-30T16:03:49.021 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.019+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~4UA6S1S1LM36.0 size=1048576 2024-05-30T16:03:49.024 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.023+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~4UA6S1S1LM36.1 size=1048576 2024-05-30T16:03:49.027 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.027+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~4UA6S1S1LM36.2 size=454892 2024-05-30T16:03:49.033 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.031+0000 7f08851138c0 1 < completed multipart upload key=603HBR8PRH6L upload=4UA6S1S1LM36 tag=1BB37CPHQIMH type=write size=3600620 2024-05-30T16:03:49.036 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.035+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=RGS7ND2D6PH0 type=del size=2135662 2024-05-30T16:03:49.040 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.039+0000 7f08851138c0 1 < canceled operation key=OKBB0VU765TW tag=01GJ32AYO4TK type=write size=703852 2024-05-30T16:03:49.047 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.047+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~GDX59UZ4G1US.0 size=1048576 2024-05-30T16:03:49.050 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.051+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~GDX59UZ4G1US.1 size=349655 2024-05-30T16:03:49.051 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.051+0000 7f08851138c0 1 > prepared multipart upload key=ZALVY70ISLM5 upload=GDX59UZ4G1US tag=XKCJN7EB8GQ8 type=write size=2446807 2024-05-30T16:03:49.055 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.055+0000 7f08851138c0 1 < completed delete operation key=0T2E9F3K1USF tag=A49JDRR4P36C type=del 2024-05-30T16:03:49.059 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.059+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=UD896OFW4FQQ type=del size=1076019 2024-05-30T16:03:49.068 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.067+0000 7f08851138c0 1 < suggested remove operation key=GWXNCSJCLXKW tag=M9O7TUK0DAE6 type=write 2024-05-30T16:03:49.072 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.071+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=0WBPLACY86M6 type=del size=89417 2024-05-30T16:03:49.076 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.075+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~58106JXWAQDU.0 size=1048576 2024-05-30T16:03:49.080 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.079+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~58106JXWAQDU.1 size=1048576 2024-05-30T16:03:49.083 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.083+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~58106JXWAQDU.2 size=956883 2024-05-30T16:03:49.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.091+0000 7f08851138c0 1 < completed multipart upload key=O8CWPJ9GW2GO upload=58106JXWAQDU tag=MRTR4Y013C7M type=write size=4102611 2024-05-30T16:03:49.096 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.095+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~XS7ENQZ0D15A.0 size=1048576 2024-05-30T16:03:49.099 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.099+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~XS7ENQZ0D15A.1 size=102772 2024-05-30T16:03:49.099 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.099+0000 7f08851138c0 1 > prepared multipart upload key=TPCETCO3Y1M6 upload=XS7ENQZ0D15A tag=L0XKAR41VZUN type=write size=2199924 2024-05-30T16:03:49.104 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.103+0000 7f08851138c0 1 < canceled operation key=ZALVY70ISLM5 tag=FGUZ1IY0VZA1 type=del size=1136831 2024-05-30T16:03:49.108 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.107+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=OLIIDO4PN8ZA type=del size=1858601 2024-05-30T16:03:49.112 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.111+0000 7f08851138c0 1 < canceled operation key=73FJ3HGWYQQK tag=P9QKGE6KRUGM type=del size=913084 2024-05-30T16:03:49.119 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.119+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~0XDFLBU9D0CG.0 size=1048576 2024-05-30T16:03:49.122 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.123+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~0XDFLBU9D0CG.1 size=1048576 2024-05-30T16:03:49.125 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.127+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~0XDFLBU9D0CG.2 size=339348 2024-05-30T16:03:49.125 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.127+0000 7f08851138c0 1 > prepared multipart upload key=3TBOZWT2P8U8 upload=0XDFLBU9D0CG tag=NS5U8NH7EBS3 type=write size=3485076 2024-05-30T16:03:49.129 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.131+0000 7f08851138c0 1 < completed delete operation key=O8CWPJ9GW2GO tag=D23V2R4UIVEB type=del 2024-05-30T16:03:49.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.135+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~8XF0E42ZC921.0 size=1048576 2024-05-30T16:03:49.139 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.139+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~8XF0E42ZC921.1 size=523837 2024-05-30T16:03:49.140 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.139+0000 7f08851138c0 1 > prepared multipart upload key=OKBB0VU765TW upload=8XF0E42ZC921 tag=6AG7XW50SVF6 type=write size=2620989 2024-05-30T16:03:49.144 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.143+0000 7f08851138c0 1 < canceled operation key=YRP4CSG92QRO tag=PT9OD0TXXU7I type=del size=3517585 2024-05-30T16:03:49.150 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.151+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~O8RJPJ01GQJ0.0 size=719709 2024-05-30T16:03:49.151 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.151+0000 7f08851138c0 1 > prepared multipart upload key=E2FXSXOEUGIG upload=O8RJPJ01GQJ0 tag=SB5IXCSJ41Y0 type=write size=1768285 2024-05-30T16:03:49.155 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.155+0000 7f08851138c0 1 < completed delete operation key=WT9RC0UYVG4F tag=C3MYWIZCKB1S type=del 2024-05-30T16:03:49.159 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.159+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=EOUTUB9KFT4X type=del size=230299 2024-05-30T16:03:49.160 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.159+0000 7f08851138c0 1 < canceled part key=_multipart_Y3CYZJ83WYNI.2~3C2OK8ZQP3SY.0 size=934815 2024-05-30T16:03:49.166 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.167+0000 7f08851138c0 1 < canceled multipart upload key=Y3CYZJ83WYNI upload=3C2OK8ZQP3SY tag=0HNMIK11HE6R type=write size=1983391 2024-05-30T16:03:49.170 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.171+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=L1N0NETEK3H6 type=del size=2530179 2024-05-30T16:03:49.173 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.175+0000 7f08851138c0 1 < completed delete operation key=A5BUBY9Z7JQG tag=M3YEWPYFP1PB type=del 2024-05-30T16:03:49.177 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.179+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=S10V3LJH2Q8C type=write size=29343 2024-05-30T16:03:49.181 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.179+0000 7f08851138c0 1 < completed delete operation key=A5BUBY9Z7JQG tag=OP6W64B03ISD type=del 2024-05-30T16:03:49.188 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.187+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~XNWG2MFIW89A.0 size=733565 2024-05-30T16:03:49.188 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.187+0000 7f08851138c0 1 > prepared multipart upload key=JNZI7JX95LX2 upload=XNWG2MFIW89A tag=XUIL9YY3Q0LE type=write size=1782141 2024-05-30T16:03:49.192 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.191+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~TUS0J83UA0VT.0 size=1048576 2024-05-30T16:03:49.195 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.195+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~TUS0J83UA0VT.1 size=915783 2024-05-30T16:03:49.201 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.199+0000 7f08851138c0 1 < completed multipart upload key=RCTV94WRMO53 upload=TUS0J83UA0VT tag=B2JRHYZCIJDA type=write size=3012935 2024-05-30T16:03:49.205 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.203+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=84L39MA6MQLB type=del size=2021284 2024-05-30T16:03:49.209 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.207+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~EG6ZT9UYYY5M.0 size=1048576 2024-05-30T16:03:49.212 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.211+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~EG6ZT9UYYY5M.1 size=848203 2024-05-30T16:03:49.218 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.219+0000 7f08851138c0 1 < completed multipart upload key=3IJWPWRB6FHE upload=EG6ZT9UYYY5M tag=G6FMA4YL0B41 type=write size=2945355 2024-05-30T16:03:49.223 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.223+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=GZVXB1AHGAI5 type=del size=2555687 2024-05-30T16:03:49.226 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.227+0000 7f08851138c0 1 < completed delete operation key=3TBOZWT2P8U8 tag=L1NB38PLHG0D type=del 2024-05-30T16:03:49.230 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.231+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=282FWM1Q38C9 type=del size=3832596 2024-05-30T16:03:49.233 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.235+0000 7f08851138c0 1 < completed delete operation key=JNZI7JX95LX2 tag=RGS7ND2D6PH0 type=del 2024-05-30T16:03:49.240 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.239+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~9276ODTSHDLS.0 size=1048576 2024-05-30T16:03:49.243 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.243+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~9276ODTSHDLS.1 size=88330 2024-05-30T16:03:49.243 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.243+0000 7f08851138c0 1 > prepared multipart upload key=TPCETCO3Y1M6 upload=9276ODTSHDLS tag=9MOOB25E27MD type=write size=2185482 2024-05-30T16:03:49.246 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.247+0000 7f08851138c0 1 < completed part key=_multipart_ZALVY70ISLM5.2~GDX59UZ4G1US.0 size=1048576 2024-05-30T16:03:49.246 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.247+0000 7f08851138c0 1 < canceled part key=_multipart_ZALVY70ISLM5.2~GDX59UZ4G1US.1 size=349655 2024-05-30T16:03:49.253 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.251+0000 7f08851138c0 1 < canceled multipart upload key=ZALVY70ISLM5 upload=GDX59UZ4G1US tag=XKCJN7EB8GQ8 type=write size=2446807 2024-05-30T16:03:49.259 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.259+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~A3J59B4A3CB9.0 size=4635 2024-05-30T16:03:49.259 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.259+0000 7f08851138c0 1 > prepared multipart upload key=NIV7AQ4EV5YP upload=A3J59B4A3CB9 tag=D9H0CRG9S3QL type=write size=1053211 2024-05-30T16:03:49.263 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.263+0000 7f08851138c0 1 < canceled operation key=PLWFSKPKVO19 tag=UD896OFW4FQQ type=del size=1076019 2024-05-30T16:03:49.269 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.267+0000 7f08851138c0 1 > prepared operation key=ZALVY70ISLM5 tag=57RWCAZ9U6H2 type=del size=3972195 2024-05-30T16:03:49.270 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.271+0000 7f08851138c0 1 < completed delete operation key=RCTV94WRMO53 tag=0WBPLACY86M6 type=del 2024-05-30T16:03:49.274 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.275+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=M7TRFEJ9H4F4 type=del size=927698 2024-05-30T16:03:49.277 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.279+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~XS7ENQZ0D15A.0 size=1048576 2024-05-30T16:03:49.284 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.283+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~XS7ENQZ0D15A.1 size=102772 2024-05-30T16:03:49.290 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.291+0000 7f08851138c0 1 < completed multipart upload key=TPCETCO3Y1M6 upload=XS7ENQZ0D15A tag=L0XKAR41VZUN type=write size=2199924 2024-05-30T16:03:49.296 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.295+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~JPNMW6J6KZP3.0 size=1048576 2024-05-30T16:03:49.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.299+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~JPNMW6J6KZP3.1 size=602571 2024-05-30T16:03:49.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.299+0000 7f08851138c0 1 > prepared multipart upload key=XHGMUYBA2VNK upload=JPNMW6J6KZP3 tag=KKS6Q6F0WMVG type=write size=2699723 2024-05-30T16:03:49.303 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.303+0000 7f08851138c0 1 < completed delete operation key=NIV7AQ4EV5YP tag=OLIIDO4PN8ZA type=del 2024-05-30T16:03:49.307 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.307+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=97LUGHWXB1XP type=del size=2202042 2024-05-30T16:03:49.310 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.311+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~0XDFLBU9D0CG.0 size=1048576 2024-05-30T16:03:49.310 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.311+0000 7f08851138c0 1 < canceled part key=_multipart_3TBOZWT2P8U8.2~0XDFLBU9D0CG.1 size=1048576 2024-05-30T16:03:49.313 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.311+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~0XDFLBU9D0CG.2 size=339348 2024-05-30T16:03:49.319 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.319+0000 7f08851138c0 1 < canceled multipart upload key=3TBOZWT2P8U8 upload=0XDFLBU9D0CG tag=NS5U8NH7EBS3 type=write size=3485076 2024-05-30T16:03:49.322 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.323+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=6QZW7DKEVEA7 type=write size=305896 2024-05-30T16:03:49.326 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.327+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~8XF0E42ZC921.0 size=1048576 2024-05-30T16:03:49.329 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.331+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~8XF0E42ZC921.1 size=523837 2024-05-30T16:03:49.335 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.335+0000 7f08851138c0 1 < completed multipart upload key=OKBB0VU765TW upload=8XF0E42ZC921 tag=6AG7XW50SVF6 type=write size=2620989 2024-05-30T16:03:49.342 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.343+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~VL6IJ3TK1R7G.0 size=927939 2024-05-30T16:03:49.342 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.343+0000 7f08851138c0 1 > prepared multipart upload key=Y3CYZJ83WYNI upload=VL6IJ3TK1R7G tag=52R5ND1KD9M3 type=write size=1976515 2024-05-30T16:03:49.346 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.347+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~O8RJPJ01GQJ0.0 size=719709 2024-05-30T16:03:49.352 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.351+0000 7f08851138c0 1 < completed multipart upload key=E2FXSXOEUGIG upload=O8RJPJ01GQJ0 tag=SB5IXCSJ41Y0 type=write size=1768285 2024-05-30T16:03:49.361 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.359+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~N2G2ZD9KZZR9.0 size=1048576 2024-05-30T16:03:49.364 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.363+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~N2G2ZD9KZZR9.1 size=610827 2024-05-30T16:03:49.364 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.363+0000 7f08851138c0 1 > prepared multipart upload key=3TBOZWT2P8U8 upload=N2G2ZD9KZZR9 tag=T869KH39SER8 type=write size=2707979 2024-05-30T16:03:49.369 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.367+0000 7f08851138c0 1 < completed delete operation key=QQILXRJD0C1Z tag=EOUTUB9KFT4X type=del 2024-05-30T16:03:49.377 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.375+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~KWQ7K92OCWKM.0 size=441365 2024-05-30T16:03:49.377 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.375+0000 7f08851138c0 1 > prepared multipart upload key=V4OQFOOQUXFQ upload=KWQ7K92OCWKM tag=4GPH7ZEP2UQ6 type=write size=1489941 2024-05-30T16:03:49.382 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.383+0000 7f08851138c0 1 < completed delete operation key=E2FXSXOEUGIG tag=L1N0NETEK3H6 type=del 2024-05-30T16:03:49.390 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.391+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~9CZI2MHTXH9E.0 size=578133 2024-05-30T16:03:49.390 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.391+0000 7f08851138c0 1 > prepared multipart upload key=Y8SBI0QZ0ZV8 upload=9CZI2MHTXH9E tag=FI7ITZEA9A8V type=write size=1626709 2024-05-30T16:03:49.393 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.395+0000 7f08851138c0 1 < completed write operation key=V4OQFOOQUXFQ tag=S10V3LJH2Q8C type=write size=29343 2024-05-30T16:03:49.397 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.395+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=DJLH7J9W2I0M type=del size=872342 2024-05-30T16:03:49.401 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.399+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~XNWG2MFIW89A.0 size=733565 2024-05-30T16:03:49.407 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.407+0000 7f08851138c0 1 < completed multipart upload key=JNZI7JX95LX2 upload=XNWG2MFIW89A tag=XUIL9YY3Q0LE type=write size=1782141 2024-05-30T16:03:49.413 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.415+0000 7f08851138c0 1 > prepared part key=_multipart_MMLF65KLC42Q.2~96MI12B4JEL8.0 size=1048576 2024-05-30T16:03:49.416 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.415+0000 7f08851138c0 1 > prepared part key=_multipart_MMLF65KLC42Q.2~96MI12B4JEL8.1 size=470383 2024-05-30T16:03:49.416 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.415+0000 7f08851138c0 1 > prepared multipart upload key=MMLF65KLC42Q upload=96MI12B4JEL8 tag=GC216KHHPB4U type=write size=2567535 2024-05-30T16:03:49.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.423+0000 7f08851138c0 1 < suggested remove operation key=JNZI7JX95LX2 tag=84L39MA6MQLB type=del 2024-05-30T16:03:49.430 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.431+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~CEFHBRA1XIOH.0 size=244855 2024-05-30T16:03:49.430 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.431+0000 7f08851138c0 1 > prepared multipart upload key=XHGMUYBA2VNK upload=CEFHBRA1XIOH tag=ODV2105DE3W0 type=write size=1293431 2024-05-30T16:03:49.434 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.435+0000 7f08851138c0 -1 < failed to complete operation key=JNZI7JX95LX2 tag=GZVXB1AHGAI5 type=del size=2555687: (22) Invalid argument 2024-05-30T16:03:49.440 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.439+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~T4RJQB26YC9B.0 size=1048576 2024-05-30T16:03:49.443 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.443+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~T4RJQB26YC9B.1 size=1048576 2024-05-30T16:03:49.446 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.447+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~T4RJQB26YC9B.2 size=836576 2024-05-30T16:03:49.446 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.447+0000 7f08851138c0 1 > prepared multipart upload key=E2FXSXOEUGIG upload=T4RJQB26YC9B tag=BD1C0DWQE8WC type=write size=3982304 2024-05-30T16:03:49.450 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.451+0000 7f08851138c0 1 < completed delete operation key=QCLY10D2A3VF tag=282FWM1Q38C9 type=del 2024-05-30T16:03:49.455 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.455+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~84LPIVIWUJN1.0 size=1048576 2024-05-30T16:03:49.458 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.459+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~84LPIVIWUJN1.1 size=1048576 2024-05-30T16:03:49.460 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.459+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~84LPIVIWUJN1.2 size=285775 2024-05-30T16:03:49.460 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.459+0000 7f08851138c0 1 > prepared multipart upload key=603HBR8PRH6L upload=84LPIVIWUJN1 tag=SKT666AGY0LH type=write size=3431503 2024-05-30T16:03:49.464 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.463+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~9276ODTSHDLS.0 size=1048576 2024-05-30T16:03:49.467 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.467+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~9276ODTSHDLS.1 size=88330 2024-05-30T16:03:49.472 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.471+0000 7f08851138c0 1 < completed multipart upload key=TPCETCO3Y1M6 upload=9276ODTSHDLS tag=9MOOB25E27MD type=write size=2185482 2024-05-30T16:03:49.478 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.479+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~MWG0TCDCO5NH.0 size=1048576 2024-05-30T16:03:49.481 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.479+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~MWG0TCDCO5NH.1 size=1048576 2024-05-30T16:03:49.483 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.483+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~MWG0TCDCO5NH.2 size=167291 2024-05-30T16:03:49.484 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.483+0000 7f08851138c0 1 > prepared multipart upload key=A5BUBY9Z7JQG upload=MWG0TCDCO5NH tag=P0FPQ7443722 type=write size=3313019 2024-05-30T16:03:49.488 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.487+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~A3J59B4A3CB9.0 size=4635 2024-05-30T16:03:49.494 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.495+0000 7f08851138c0 1 < completed multipart upload key=NIV7AQ4EV5YP upload=A3J59B4A3CB9 tag=D9H0CRG9S3QL type=write size=1053211 2024-05-30T16:03:49.497 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.499+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=QJFDQ37BEILL type=del size=1477408 2024-05-30T16:03:49.501 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.503+0000 7f08851138c0 1 < completed delete operation key=ZALVY70ISLM5 tag=57RWCAZ9U6H2 type=del 2024-05-30T16:03:49.505 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.503+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=RQQJILDDCQ24 type=del size=2024443 2024-05-30T16:03:49.509 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.507+0000 7f08851138c0 1 < completed delete operation key=KPHZGFJLC2E0 tag=M7TRFEJ9H4F4 type=del 2024-05-30T16:03:49.512 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.511+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=EJEG82VSV476 type=del size=3050425 2024-05-30T16:03:49.517 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.519+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~JPNMW6J6KZP3.0 size=1048576 2024-05-30T16:03:49.517 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.519+0000 7f08851138c0 1 < canceled part key=_multipart_XHGMUYBA2VNK.2~JPNMW6J6KZP3.1 size=602571 2024-05-30T16:03:49.524 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.523+0000 7f08851138c0 1 < canceled multipart upload key=XHGMUYBA2VNK upload=JPNMW6J6KZP3 tag=KKS6Q6F0WMVG type=write size=2699723 2024-05-30T16:03:49.532 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.531+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~79764KZBEKM9.0 size=195861 2024-05-30T16:03:49.532 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.531+0000 7f08851138c0 1 > prepared multipart upload key=O0DJ8SCY766V upload=79764KZBEKM9 tag=OB9E1Y465C3K type=write size=1244437 2024-05-30T16:03:49.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.535+0000 7f08851138c0 1 < completed delete operation key=Y8SBI0QZ0ZV8 tag=97LUGHWXB1XP type=del 2024-05-30T16:03:49.539 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.539+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=UBBRXWCXDDW3 type=del size=1697665 2024-05-30T16:03:49.543 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.543+0000 7f08851138c0 1 < completed write operation key=XHGMUYBA2VNK tag=6QZW7DKEVEA7 type=write size=305896 2024-05-30T16:03:49.547 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.547+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=1GR5M63TCHNC type=del size=2621657 2024-05-30T16:03:49.551 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.551+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~VL6IJ3TK1R7G.0 size=927939 2024-05-30T16:03:49.557 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.555+0000 7f08851138c0 1 < completed multipart upload key=Y3CYZJ83WYNI upload=VL6IJ3TK1R7G tag=52R5ND1KD9M3 type=write size=1976515 2024-05-30T16:03:49.560 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.559+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=C5G6746RLO5O type=del size=3140988 2024-05-30T16:03:49.564 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.563+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~N2G2ZD9KZZR9.0 size=1048576 2024-05-30T16:03:49.567 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.567+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~N2G2ZD9KZZR9.1 size=610827 2024-05-30T16:03:49.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.571+0000 7f08851138c0 1 < completed multipart upload key=3TBOZWT2P8U8 upload=N2G2ZD9KZZR9 tag=T869KH39SER8 type=write size=2707979 2024-05-30T16:03:49.580 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.579+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~81W6M312FLWI.0 size=498861 2024-05-30T16:03:49.580 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.579+0000 7f08851138c0 1 > prepared multipart upload key=WT9RC0UYVG4F upload=81W6M312FLWI tag=KT7G2WC4DYWP type=write size=1547437 2024-05-30T16:03:49.583 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.583+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~KWQ7K92OCWKM.0 size=441365 2024-05-30T16:03:49.589 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.591+0000 7f08851138c0 1 < completed multipart upload key=V4OQFOOQUXFQ upload=KWQ7K92OCWKM tag=4GPH7ZEP2UQ6 type=write size=1489941 2024-05-30T16:03:49.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.595+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~0KU8PWJ0GBH5.0 size=1048576 2024-05-30T16:03:49.599 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.599+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~0KU8PWJ0GBH5.1 size=56606 2024-05-30T16:03:49.599 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.599+0000 7f08851138c0 1 > prepared multipart upload key=NIV7AQ4EV5YP upload=0KU8PWJ0GBH5 tag=TL34LOKD49FE type=write size=2153758 2024-05-30T16:03:49.602 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.603+0000 7f08851138c0 1 < completed part key=_multipart_Y8SBI0QZ0ZV8.2~9CZI2MHTXH9E.0 size=578133 2024-05-30T16:03:49.608 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.607+0000 7f08851138c0 1 < completed multipart upload key=Y8SBI0QZ0ZV8 upload=9CZI2MHTXH9E tag=FI7ITZEA9A8V type=write size=1626709 2024-05-30T16:03:49.612 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.611+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=1HFXKICZDHEH type=del size=1609567 2024-05-30T16:03:49.619 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.619+0000 7f08851138c0 1 < suggested update operation key=XHGMUYBA2VNK tag=DJLH7J9W2I0M type=del size=872342 2024-05-30T16:03:49.627 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.627+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~VU4N16AIKACY.0 size=1048576 2024-05-30T16:03:49.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.631+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~VU4N16AIKACY.1 size=1048576 2024-05-30T16:03:49.632 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.631+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~VU4N16AIKACY.2 size=18525 2024-05-30T16:03:49.632 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.631+0000 7f08851138c0 1 > prepared multipart upload key=NIV7AQ4EV5YP upload=VU4N16AIKACY tag=9HJQ9QXZX7U3 type=write size=3164253 2024-05-30T16:03:49.635 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.635+0000 7f08851138c0 1 < completed part key=_multipart_MMLF65KLC42Q.2~96MI12B4JEL8.0 size=1048576 2024-05-30T16:03:49.638 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.639+0000 7f08851138c0 1 < completed part key=_multipart_MMLF65KLC42Q.2~96MI12B4JEL8.1 size=470383 2024-05-30T16:03:49.643 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.643+0000 7f08851138c0 1 < completed multipart upload key=MMLF65KLC42Q upload=96MI12B4JEL8 tag=GC216KHHPB4U type=write size=2567535 2024-05-30T16:03:49.649 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.647+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~3BAHF6767ETX.0 size=834628 2024-05-30T16:03:49.650 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.647+0000 7f08851138c0 1 > prepared multipart upload key=3TBOZWT2P8U8 upload=3BAHF6767ETX tag=4BIEA4182B9U type=write size=1883204 2024-05-30T16:03:49.653 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.651+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~CEFHBRA1XIOH.0 size=244855 2024-05-30T16:03:49.658 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.659+0000 7f08851138c0 1 < completed multipart upload key=XHGMUYBA2VNK upload=CEFHBRA1XIOH tag=ODV2105DE3W0 type=write size=1293431 2024-05-30T16:03:49.665 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.667+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~U1HDPXB1WLX9.0 size=320776 2024-05-30T16:03:49.665 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.667+0000 7f08851138c0 1 > prepared multipart upload key=YRP4CSG92QRO upload=U1HDPXB1WLX9 tag=0YQJK76HQG77 type=write size=1369352 2024-05-30T16:03:49.669 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.667+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~T4RJQB26YC9B.0 size=1048576 2024-05-30T16:03:49.672 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.671+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~T4RJQB26YC9B.1 size=1048576 2024-05-30T16:03:49.676 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.675+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~T4RJQB26YC9B.2 size=836576 2024-05-30T16:03:49.681 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.683+0000 7f08851138c0 1 < completed multipart upload key=E2FXSXOEUGIG upload=T4RJQB26YC9B tag=BD1C0DWQE8WC type=write size=3982304 2024-05-30T16:03:49.684 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.683+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=U9T2W9DHTBUC type=del size=3980082 2024-05-30T16:03:49.688 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.687+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~84LPIVIWUJN1.0 size=1048576 2024-05-30T16:03:49.691 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.691+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~84LPIVIWUJN1.1 size=1048576 2024-05-30T16:03:49.693 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.695+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~84LPIVIWUJN1.2 size=285775 2024-05-30T16:03:49.699 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.699+0000 7f08851138c0 1 < completed multipart upload key=603HBR8PRH6L upload=84LPIVIWUJN1 tag=SKT666AGY0LH type=write size=3431503 2024-05-30T16:03:49.702 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.703+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=TQMLX6DYZ3MF type=del size=1888749 2024-05-30T16:03:49.706 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.707+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~MWG0TCDCO5NH.0 size=1048576 2024-05-30T16:03:49.708 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.707+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~MWG0TCDCO5NH.1 size=1048576 2024-05-30T16:03:49.711 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.711+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~MWG0TCDCO5NH.2 size=167291 2024-05-30T16:03:49.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.719+0000 7f08851138c0 1 < completed multipart upload key=A5BUBY9Z7JQG upload=MWG0TCDCO5NH tag=P0FPQ7443722 type=write size=3313019 2024-05-30T16:03:49.721 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.719+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=C6YPE8OTDOJJ type=del size=2541089 2024-05-30T16:03:49.724 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.723+0000 7f08851138c0 1 < completed delete operation key=GWXNCSJCLXKW tag=QJFDQ37BEILL type=del 2024-05-30T16:03:49.730 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.731+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~YNG3I51GK4NI.0 size=1048576 2024-05-30T16:03:49.733 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.731+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~YNG3I51GK4NI.1 size=761384 2024-05-30T16:03:49.733 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.731+0000 7f08851138c0 1 > prepared multipart upload key=OKBB0VU765TW upload=YNG3I51GK4NI tag=WRYNZXIL82ZL type=write size=2858536 2024-05-30T16:03:49.736 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.735+0000 7f08851138c0 1 < canceled operation key=NIV7AQ4EV5YP tag=RQQJILDDCQ24 type=del size=2024443 2024-05-30T16:03:49.739 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.739+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=UB1107BE40IB type=write size=396717 2024-05-30T16:03:49.742 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.743+0000 7f08851138c0 1 < completed delete operation key=4N58ZY4E1H6E tag=EJEG82VSV476 type=del 2024-05-30T16:03:49.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.743+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=F8ADER8HZAOX type=write size=972995 2024-05-30T16:03:49.748 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.747+0000 7f08851138c0 1 < completed part key=_multipart_O0DJ8SCY766V.2~79764KZBEKM9.0 size=195861 2024-05-30T16:03:49.754 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.755+0000 7f08851138c0 1 < completed multipart upload key=O0DJ8SCY766V upload=79764KZBEKM9 tag=OB9E1Y465C3K type=write size=1244437 2024-05-30T16:03:49.757 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.755+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=KD1YOIWX0DD5 type=del size=28463 2024-05-30T16:03:49.760 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.759+0000 7f08851138c0 1 < completed delete operation key=QQILXRJD0C1Z tag=UBBRXWCXDDW3 type=del 2024-05-30T16:03:49.764 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.763+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=1EHRWKK7PDXG type=write size=68483 2024-05-30T16:03:49.767 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.767+0000 7f08851138c0 1 < completed delete operation key=OKBB0VU765TW tag=1GR5M63TCHNC type=del 2024-05-30T16:03:49.771 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.771+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=LCL6VEES4RFY type=write size=684078 2024-05-30T16:03:49.774 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.775+0000 7f08851138c0 1 < completed delete operation key=JNZI7JX95LX2 tag=C5G6746RLO5O type=del 2024-05-30T16:03:49.777 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.779+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=S2TKI2O4DMTI type=del size=3065701 2024-05-30T16:03:49.781 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.779+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~81W6M312FLWI.0 size=498861 2024-05-30T16:03:49.786 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.787+0000 7f08851138c0 1 < completed multipart upload key=WT9RC0UYVG4F upload=81W6M312FLWI tag=KT7G2WC4DYWP type=write size=1547437 2024-05-30T16:03:49.789 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.791+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=TGCZ83LNMLIB type=del size=2546056 2024-05-30T16:03:49.793 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.791+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~0KU8PWJ0GBH5.0 size=1048576 2024-05-30T16:03:49.796 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.795+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~0KU8PWJ0GBH5.1 size=56606 2024-05-30T16:03:49.801 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.803+0000 7f08851138c0 1 < completed multipart upload key=NIV7AQ4EV5YP upload=0KU8PWJ0GBH5 tag=TL34LOKD49FE type=write size=2153758 2024-05-30T16:03:49.807 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.807+0000 7f08851138c0 1 > prepared part key=_multipart_MMLF65KLC42Q.2~CZH4O7YJIZ1S.0 size=1048576 2024-05-30T16:03:49.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.811+0000 7f08851138c0 1 > prepared part key=_multipart_MMLF65KLC42Q.2~CZH4O7YJIZ1S.1 size=338568 2024-05-30T16:03:49.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.811+0000 7f08851138c0 1 > prepared multipart upload key=MMLF65KLC42Q upload=CZH4O7YJIZ1S tag=7GGX0R26J5AA type=write size=2435720 2024-05-30T16:03:49.817 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.815+0000 7f08851138c0 1 < suggested update operation key=RCTV94WRMO53 tag=1HFXKICZDHEH type=del size=1609567 2024-05-30T16:03:49.823 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.823+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~V304TSFLNZO6.0 size=1048576 2024-05-30T16:03:49.825 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.827+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~V304TSFLNZO6.1 size=937664 2024-05-30T16:03:49.826 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.827+0000 7f08851138c0 1 > prepared multipart upload key=603HBR8PRH6L upload=V304TSFLNZO6 tag=43ADN64FTOXJ type=write size=3034816 2024-05-30T16:03:49.826 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.827+0000 7f08851138c0 1 < canceled part key=_multipart_NIV7AQ4EV5YP.2~VU4N16AIKACY.0 size=1048576 2024-05-30T16:03:49.829 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.827+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~VU4N16AIKACY.1 size=1048576 2024-05-30T16:03:49.832 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.831+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~VU4N16AIKACY.2 size=18525 2024-05-30T16:03:49.838 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.839+0000 7f08851138c0 1 < canceled multipart upload key=NIV7AQ4EV5YP upload=VU4N16AIKACY tag=9HJQ9QXZX7U3 type=write size=3164253 2024-05-30T16:03:49.844 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.843+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~1S92EGC5HEPY.0 size=854335 2024-05-30T16:03:49.844 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.843+0000 7f08851138c0 1 > prepared multipart upload key=V4OQFOOQUXFQ upload=1S92EGC5HEPY tag=39YUO7V8S2ON type=write size=1902911 2024-05-30T16:03:49.848 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.847+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~3BAHF6767ETX.0 size=834628 2024-05-30T16:03:49.854 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.855+0000 7f08851138c0 1 < completed multipart upload key=3TBOZWT2P8U8 upload=3BAHF6767ETX tag=4BIEA4182B9U type=write size=1883204 2024-05-30T16:03:49.860 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.859+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~W8GPNT3H0D8E.0 size=1048576 2024-05-30T16:03:49.864 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.863+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~W8GPNT3H0D8E.1 size=1048576 2024-05-30T16:03:49.867 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.867+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~W8GPNT3H0D8E.2 size=187280 2024-05-30T16:03:49.867 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.867+0000 7f08851138c0 1 > prepared multipart upload key=0T2E9F3K1USF upload=W8GPNT3H0D8E tag=UDC8XB2VY6ZA type=write size=3333008 2024-05-30T16:03:49.871 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.871+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~U1HDPXB1WLX9.0 size=320776 2024-05-30T16:03:49.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.875+0000 7f08851138c0 1 < completed multipart upload key=YRP4CSG92QRO upload=U1HDPXB1WLX9 tag=0YQJK76HQG77 type=write size=1369352 2024-05-30T16:03:49.880 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.879+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=PTXC76EE1LRG type=del size=1120707 2024-05-30T16:03:49.886 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.883+0000 7f08851138c0 1 < canceled operation key=Y8SBI0QZ0ZV8 tag=U9T2W9DHTBUC type=del size=3980082 2024-05-30T16:03:49.888 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.887+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=8XEFSP96EAKO type=del size=2262864 2024-05-30T16:03:49.891 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.891+0000 7f08851138c0 1 < completed delete operation key=TPCETCO3Y1M6 tag=TQMLX6DYZ3MF type=del 2024-05-30T16:03:49.895 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.895+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=EZN3BHL0P12V type=del size=3289312 2024-05-30T16:03:49.900 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.899+0000 7f08851138c0 1 < completed delete operation key=603HBR8PRH6L tag=C6YPE8OTDOJJ type=del 2024-05-30T16:03:49.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.907+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~XHZA8OE3RU5D.0 size=1048576 2024-05-30T16:03:49.909 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.907+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~XHZA8OE3RU5D.1 size=772787 2024-05-30T16:03:49.909 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.907+0000 7f08851138c0 1 > prepared multipart upload key=Y8SBI0QZ0ZV8 upload=XHZA8OE3RU5D tag=BG50U96KBNR8 type=write size=2869939 2024-05-30T16:03:49.913 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.911+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~YNG3I51GK4NI.0 size=1048576 2024-05-30T16:03:49.916 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.915+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~YNG3I51GK4NI.1 size=761384 2024-05-30T16:03:49.922 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.923+0000 7f08851138c0 1 < completed multipart upload key=OKBB0VU765TW upload=YNG3I51GK4NI tag=WRYNZXIL82ZL type=write size=2858536 2024-05-30T16:03:49.925 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.927+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=AU014DX2OINA type=del size=677148 2024-05-30T16:03:49.932 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.931+0000 7f08851138c0 1 < suggested update operation key=XHGMUYBA2VNK tag=UB1107BE40IB type=write size=396717 2024-05-30T16:03:49.936 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.935+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=1NX0LOXZR32N type=write size=76683 2024-05-30T16:03:49.942 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.943+0000 7f08851138c0 1 < completed write operation key=YRP4CSG92QRO tag=F8ADER8HZAOX type=write size=972995 2024-05-30T16:03:49.949 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.947+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~XD03XFUM4ER7.0 size=1048576 2024-05-30T16:03:49.951 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.951+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~XD03XFUM4ER7.1 size=264704 2024-05-30T16:03:49.952 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.951+0000 7f08851138c0 1 > prepared multipart upload key=603HBR8PRH6L upload=XD03XFUM4ER7 tag=AYY6M6ZN7UQH type=write size=2361856 2024-05-30T16:03:49.955 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.955+0000 7f08851138c0 1 < completed delete operation key=YRP4CSG92QRO tag=KD1YOIWX0DD5 type=del 2024-05-30T16:03:49.959 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.959+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=O78TR3TOQOIT type=del size=1415266 2024-05-30T16:03:49.963 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.963+0000 7f08851138c0 1 < canceled operation key=PLWFSKPKVO19 tag=1EHRWKK7PDXG type=write size=68483 2024-05-30T16:03:49.967 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.967+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=7MO2442RIOJ7 type=del size=268283 2024-05-30T16:03:49.970 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.971+0000 7f08851138c0 1 < completed write operation key=1N6R2HBTWMP2 tag=LCL6VEES4RFY type=write size=684078 2024-05-30T16:03:49.977 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.979+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~636JGB9QU27K.0 size=1048576 2024-05-30T16:03:49.980 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.979+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~636JGB9QU27K.1 size=1048576 2024-05-30T16:03:49.983 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.983+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~636JGB9QU27K.2 size=294548 2024-05-30T16:03:49.983 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.983+0000 7f08851138c0 1 > prepared multipart upload key=73FJ3HGWYQQK upload=636JGB9QU27K tag=03R9D8D92TD7 type=write size=3440276 2024-05-30T16:03:49.987 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.987+0000 7f08851138c0 1 < completed delete operation key=NIV7AQ4EV5YP tag=S2TKI2O4DMTI type=del 2024-05-30T16:03:49.993 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.991+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~E7DTXHF1LTC3.0 size=1048576 2024-05-30T16:03:49.996 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.995+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~E7DTXHF1LTC3.1 size=1048576 2024-05-30T16:03:49.999 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.999+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~E7DTXHF1LTC3.2 size=1044827 2024-05-30T16:03:49.999 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:49.999+0000 7f08851138c0 1 > prepared multipart upload key=O8CWPJ9GW2GO upload=E7DTXHF1LTC3 tag=EXMESCD99720 type=write size=4190555 2024-05-30T16:03:50.006 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.007+0000 7f08851138c0 1 < suggested update operation key=1N6R2HBTWMP2 tag=TGCZ83LNMLIB type=del size=2546056 2024-05-30T16:03:50.010 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.011+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=Q2SCOI1XVQTU type=del size=1398328 2024-05-30T16:03:50.013 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.015+0000 7f08851138c0 1 < completed part key=_multipart_MMLF65KLC42Q.2~CZH4O7YJIZ1S.0 size=1048576 2024-05-30T16:03:50.016 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.015+0000 7f08851138c0 1 < completed part key=_multipart_MMLF65KLC42Q.2~CZH4O7YJIZ1S.1 size=338568 2024-05-30T16:03:50.023 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.023+0000 7f08851138c0 1 < completed multipart upload key=MMLF65KLC42Q upload=CZH4O7YJIZ1S tag=7GGX0R26J5AA type=write size=2435720 2024-05-30T16:03:50.027 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.027+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=T2Z7XMS5ZXM2 type=del size=765928 2024-05-30T16:03:50.031 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.031+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~V304TSFLNZO6.0 size=1048576 2024-05-30T16:03:50.035 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.035+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~V304TSFLNZO6.1 size=937664 2024-05-30T16:03:50.040 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.039+0000 7f08851138c0 1 < completed multipart upload key=603HBR8PRH6L upload=V304TSFLNZO6 tag=43ADN64FTOXJ type=write size=3034816 2024-05-30T16:03:50.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.043+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=9QXFMC01E2XP type=del size=1522703 2024-05-30T16:03:50.048 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.047+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~1S92EGC5HEPY.0 size=854335 2024-05-30T16:03:50.054 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.055+0000 7f08851138c0 1 < completed multipart upload key=V4OQFOOQUXFQ upload=1S92EGC5HEPY tag=39YUO7V8S2ON type=write size=1902911 2024-05-30T16:03:50.060 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.059+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~NYXYTFCYLIYA.0 size=652641 2024-05-30T16:03:50.060 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.059+0000 7f08851138c0 1 > prepared multipart upload key=NIV7AQ4EV5YP upload=NYXYTFCYLIYA tag=KZV3BL5MDSGY type=write size=1701217 2024-05-30T16:03:50.066 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.067+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~W8GPNT3H0D8E.0 size=1048576 2024-05-30T16:03:50.066 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.067+0000 7f08851138c0 1 < canceled part key=_multipart_0T2E9F3K1USF.2~W8GPNT3H0D8E.1 size=1048576 2024-05-30T16:03:50.069 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.067+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~W8GPNT3H0D8E.2 size=187280 2024-05-30T16:03:50.074 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.075+0000 7f08851138c0 1 < canceled multipart upload key=0T2E9F3K1USF upload=W8GPNT3H0D8E tag=UDC8XB2VY6ZA type=write size=3333008 2024-05-30T16:03:50.078 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.079+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=PYIDV92TZZM8 type=del size=583315 2024-05-30T16:03:50.082 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.083+0000 7f08851138c0 1 < completed delete operation key=TPCETCO3Y1M6 tag=PTXC76EE1LRG type=del 2024-05-30T16:03:50.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.087+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~ER0QYGX7CEUS.0 size=1048576 2024-05-30T16:03:50.092 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.091+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~ER0QYGX7CEUS.1 size=405383 2024-05-30T16:03:50.092 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.091+0000 7f08851138c0 1 > prepared multipart upload key=364U3OV10VPD upload=ER0QYGX7CEUS tag=HEISBW5R9TAG type=write size=2502535 2024-05-30T16:03:50.096 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.095+0000 7f08851138c0 -1 < failed to complete operation key=1N6R2HBTWMP2 tag=8XEFSP96EAKO type=del size=2262864: (22) Invalid argument 2024-05-30T16:03:50.099 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.099+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=18F7ZPKDH6LS type=del size=3799468 2024-05-30T16:03:50.108 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.107+0000 7f08851138c0 1 < suggested update operation key=Y8SBI0QZ0ZV8 tag=EZN3BHL0P12V type=del size=3289312 2024-05-30T16:03:50.112 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.111+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=5X2N9SZ8CHLN type=del size=4176177 2024-05-30T16:03:50.116 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.115+0000 7f08851138c0 1 < completed part key=_multipart_Y8SBI0QZ0ZV8.2~XHZA8OE3RU5D.0 size=1048576 2024-05-30T16:03:50.119 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.119+0000 7f08851138c0 1 < completed part key=_multipart_Y8SBI0QZ0ZV8.2~XHZA8OE3RU5D.1 size=772787 2024-05-30T16:03:50.125 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.127+0000 7f08851138c0 1 < completed multipart upload key=Y8SBI0QZ0ZV8 upload=XHZA8OE3RU5D tag=BG50U96KBNR8 type=write size=2869939 2024-05-30T16:03:50.130 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.131+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=GZVWRSABF0QN type=del size=2813832 2024-05-30T16:03:50.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.139+0000 7f08851138c0 1 < suggested remove operation key=PLWFSKPKVO19 tag=AU014DX2OINA type=del 2024-05-30T16:03:50.141 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.139+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=W72TXZ6CRJWJ type=del size=733848 2024-05-30T16:03:50.145 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.147+0000 7f08851138c0 1 < completed write operation key=GWXNCSJCLXKW tag=1NX0LOXZR32N type=write size=76683 2024-05-30T16:03:50.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.151+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=OLDYGKQF8VWD type=del size=439557 2024-05-30T16:03:50.154 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.155+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~XD03XFUM4ER7.0 size=1048576 2024-05-30T16:03:50.157 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.159+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~XD03XFUM4ER7.1 size=264704 2024-05-30T16:03:50.164 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.163+0000 7f08851138c0 1 < completed multipart upload key=603HBR8PRH6L upload=XD03XFUM4ER7 tag=AYY6M6ZN7UQH type=write size=2361856 2024-05-30T16:03:50.167 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.167+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=27CURQ9Q8BFT type=write size=58601 2024-05-30T16:03:50.171 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.171+0000 7f08851138c0 1 < completed delete operation key=XHGMUYBA2VNK tag=O78TR3TOQOIT type=del 2024-05-30T16:03:50.174 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.175+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=TSCNSJCH9D8A type=del size=400945 2024-05-30T16:03:50.178 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.179+0000 7f08851138c0 1 < completed delete operation key=XHGMUYBA2VNK tag=7MO2442RIOJ7 type=del 2024-05-30T16:03:50.182 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.183+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=KZE40J5DVA1A type=del size=3005213 2024-05-30T16:03:50.186 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.187+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~636JGB9QU27K.0 size=1048576 2024-05-30T16:03:50.189 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.187+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~636JGB9QU27K.1 size=1048576 2024-05-30T16:03:50.192 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.191+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~636JGB9QU27K.2 size=294548 2024-05-30T16:03:50.198 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.199+0000 7f08851138c0 1 < completed multipart upload key=73FJ3HGWYQQK upload=636JGB9QU27K tag=03R9D8D92TD7 type=write size=3440276 2024-05-30T16:03:50.201 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.203+0000 7f08851138c0 1 > prepared operation key=ZALVY70ISLM5 tag=83R9Y28ODKOL type=del size=116779 2024-05-30T16:03:50.202 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.203+0000 7f08851138c0 1 < canceled part key=_multipart_O8CWPJ9GW2GO.2~E7DTXHF1LTC3.0 size=1048576 2024-05-30T16:03:50.205 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.203+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~E7DTXHF1LTC3.1 size=1048576 2024-05-30T16:03:50.208 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.207+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~E7DTXHF1LTC3.2 size=1044827 2024-05-30T16:03:50.214 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.215+0000 7f08851138c0 1 < canceled multipart upload key=O8CWPJ9GW2GO upload=E7DTXHF1LTC3 tag=EXMESCD99720 type=write size=4190555 2024-05-30T16:03:50.217 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.215+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=J72OLX2EGYBH type=del size=56963 2024-05-30T16:03:50.220 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.219+0000 7f08851138c0 1 < completed delete operation key=3IJWPWRB6FHE tag=Q2SCOI1XVQTU type=del 2024-05-30T16:03:50.224 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.223+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=14LG7IHENY7O type=del size=1900056 2024-05-30T16:03:50.227 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.227+0000 7f08851138c0 1 < completed delete operation key=OKBB0VU765TW tag=T2Z7XMS5ZXM2 type=del 2024-05-30T16:03:50.233 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.235+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~T8YRDCVTI81F.0 size=1048576 2024-05-30T16:03:50.236 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.235+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~T8YRDCVTI81F.1 size=100484 2024-05-30T16:03:50.236 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.235+0000 7f08851138c0 1 > prepared multipart upload key=QQILXRJD0C1Z upload=T8YRDCVTI81F tag=CZL20S2UO01Q type=write size=2197636 2024-05-30T16:03:50.240 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.239+0000 7f08851138c0 1 < completed delete operation key=OKBB0VU765TW tag=9QXFMC01E2XP type=del 2024-05-30T16:03:50.243 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.243+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=82HJN2BQZDTR type=del size=1331599 2024-05-30T16:03:50.246 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.247+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~NYXYTFCYLIYA.0 size=652641 2024-05-30T16:03:50.252 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.251+0000 7f08851138c0 1 < completed multipart upload key=NIV7AQ4EV5YP upload=NYXYTFCYLIYA tag=KZV3BL5MDSGY type=write size=1701217 2024-05-30T16:03:50.258 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.259+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~74OBSFJ9FIQ8.0 size=1048576 2024-05-30T16:03:50.261 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.259+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~74OBSFJ9FIQ8.1 size=341234 2024-05-30T16:03:50.261 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.259+0000 7f08851138c0 1 > prepared multipart upload key=JNZI7JX95LX2 upload=74OBSFJ9FIQ8 tag=IFNVRHVAE8ZE type=write size=2438386 2024-05-30T16:03:50.264 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.263+0000 7f08851138c0 -1 < failed to complete operation key=Y8SBI0QZ0ZV8 tag=PYIDV92TZZM8 type=del size=583315: (22) Invalid argument 2024-05-30T16:03:50.268 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.267+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=EMUO5H7GJUWH type=del size=2148388 2024-05-30T16:03:50.271 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.271+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~ER0QYGX7CEUS.0 size=1048576 2024-05-30T16:03:50.274 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.275+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~ER0QYGX7CEUS.1 size=405383 2024-05-30T16:03:50.280 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.279+0000 7f08851138c0 1 < completed multipart upload key=364U3OV10VPD upload=ER0QYGX7CEUS tag=HEISBW5R9TAG type=write size=2502535 2024-05-30T16:03:50.285 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.283+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=131S92ZLF01N type=del size=3221921 2024-05-30T16:03:50.289 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.287+0000 7f08851138c0 1 < completed delete operation key=WT9RC0UYVG4F tag=18F7ZPKDH6LS type=del 2024-05-30T16:03:50.292 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.291+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=AHFMT5K5SLJM type=del size=569363 2024-05-30T16:03:50.295 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.295+0000 7f08851138c0 1 < canceled operation key=V4OQFOOQUXFQ tag=5X2N9SZ8CHLN type=del size=4176177 2024-05-30T16:03:50.298 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.299+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=F5NCHMDLAY20 type=del size=1665465 2024-05-30T16:03:50.302 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.303+0000 7f08851138c0 1 < completed delete operation key=Y3CYZJ83WYNI tag=GZVWRSABF0QN type=del 2024-05-30T16:03:50.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.303+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=GXT0C1WFYCIU type=del size=2878286 2024-05-30T16:03:50.309 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.307+0000 7f08851138c0 1 < canceled operation key=Y3CYZJ83WYNI tag=W72TXZ6CRJWJ type=del size=733848 2024-05-30T16:03:50.312 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.311+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=UNZFOAJC7SLC type=del size=3543360 2024-05-30T16:03:50.316 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.315+0000 7f08851138c0 1 < completed delete operation key=Y8SBI0QZ0ZV8 tag=OLDYGKQF8VWD type=del 2024-05-30T16:03:50.319 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.319+0000 7f08851138c0 1 > prepared operation key=VRXNO8JLAZYH tag=BP3VWDE710HE type=del size=852393 2024-05-30T16:03:50.323 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.323+0000 7f08851138c0 1 < completed write operation key=QCLY10D2A3VF tag=27CURQ9Q8BFT type=write size=58601 2024-05-30T16:03:50.326 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.327+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=ZQ8DM34COB6O type=write size=732059 2024-05-30T16:03:50.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.331+0000 7f08851138c0 1 < completed delete operation key=Y3CYZJ83WYNI tag=TSCNSJCH9D8A type=del 2024-05-30T16:03:50.333 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.331+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=I2DCOLII2J6P type=del size=4058704 2024-05-30T16:03:50.337 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.335+0000 7f08851138c0 1 < completed delete operation key=V4OQFOOQUXFQ tag=KZE40J5DVA1A type=del 2024-05-30T16:03:50.341 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.339+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=AMN0IDF77THI type=write size=199926 2024-05-30T16:03:50.344 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.343+0000 7f08851138c0 1 < completed delete operation key=ZALVY70ISLM5 tag=83R9Y28ODKOL type=del 2024-05-30T16:03:50.348 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.347+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=FIGGYQWTX1KY type=write size=295633 2024-05-30T16:03:50.352 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.351+0000 7f08851138c0 1 < completed delete operation key=KPHZGFJLC2E0 tag=J72OLX2EGYBH type=del 2024-05-30T16:03:50.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.359+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~18JBONG3JHVL.0 size=254277 2024-05-30T16:03:50.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.359+0000 7f08851138c0 1 > prepared multipart upload key=Y3CYZJ83WYNI upload=18JBONG3JHVL tag=EV8IY3RAZWXA type=write size=1302853 2024-05-30T16:03:50.364 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.363+0000 7f08851138c0 1 < completed delete operation key=1N6R2HBTWMP2 tag=14LG7IHENY7O type=del 2024-05-30T16:03:50.371 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.371+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~Q4A9E8ZKU6W5.0 size=1048576 2024-05-30T16:03:50.374 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.375+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~Q4A9E8ZKU6W5.1 size=1048576 2024-05-30T16:03:50.377 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.379+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~Q4A9E8ZKU6W5.2 size=711803 2024-05-30T16:03:50.377 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.379+0000 7f08851138c0 1 > prepared multipart upload key=ZALVY70ISLM5 upload=Q4A9E8ZKU6W5 tag=5HO7UYJHJ2ZR type=write size=3857531 2024-05-30T16:03:50.381 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.379+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~T8YRDCVTI81F.0 size=1048576 2024-05-30T16:03:50.385 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.383+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~T8YRDCVTI81F.1 size=100484 2024-05-30T16:03:50.392 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.391+0000 7f08851138c0 1 < completed multipart upload key=QQILXRJD0C1Z upload=T8YRDCVTI81F tag=CZL20S2UO01Q type=write size=2197636 2024-05-30T16:03:50.400 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.399+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~PT06M0D5LMKM.0 size=922310 2024-05-30T16:03:50.400 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.399+0000 7f08851138c0 1 > prepared multipart upload key=E2FXSXOEUGIG upload=PT06M0D5LMKM tag=TB9DXU7JL1QU type=write size=1970886 2024-05-30T16:03:50.409 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.407+0000 7f08851138c0 1 < suggested update operation key=73FJ3HGWYQQK tag=82HJN2BQZDTR type=del size=1331599 2024-05-30T16:03:50.417 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.415+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~D3EI2O7MARE5.0 size=1048576 2024-05-30T16:03:50.420 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.419+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~D3EI2O7MARE5.1 size=778853 2024-05-30T16:03:50.420 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.419+0000 7f08851138c0 1 > prepared multipart upload key=V4OQFOOQUXFQ upload=D3EI2O7MARE5 tag=HJ8UU0DYXM33 type=write size=2876005 2024-05-30T16:03:50.425 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.423+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~74OBSFJ9FIQ8.0 size=1048576 2024-05-30T16:03:50.428 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.427+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~74OBSFJ9FIQ8.1 size=341234 2024-05-30T16:03:50.434 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.435+0000 7f08851138c0 1 < completed multipart upload key=JNZI7JX95LX2 upload=74OBSFJ9FIQ8 tag=IFNVRHVAE8ZE type=write size=2438386 2024-05-30T16:03:50.438 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.439+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=5CUN3WW128SU type=del size=1382101 2024-05-30T16:03:50.441 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.443+0000 7f08851138c0 1 < completed delete operation key=V4OQFOOQUXFQ tag=EMUO5H7GJUWH type=del 2024-05-30T16:03:50.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.447+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~M8SZOZC1TUM2.0 size=1048576 2024-05-30T16:03:50.451 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.451+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~M8SZOZC1TUM2.1 size=1048576 2024-05-30T16:03:50.454 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.455+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~M8SZOZC1TUM2.2 size=36640 2024-05-30T16:03:50.454 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.455+0000 7f08851138c0 1 > prepared multipart upload key=A5BUBY9Z7JQG upload=M8SZOZC1TUM2 tag=NW9KFUBOB9FE type=write size=3182368 2024-05-30T16:03:50.461 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.459+0000 7f08851138c0 1 < suggested update operation key=603HBR8PRH6L tag=131S92ZLF01N type=del size=3221921 2024-05-30T16:03:50.468 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.467+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~3GAM63LATVP5.0 size=1048576 2024-05-30T16:03:50.470 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.471+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~3GAM63LATVP5.1 size=314854 2024-05-30T16:03:50.470 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.471+0000 7f08851138c0 1 > prepared multipart upload key=O0DJ8SCY766V upload=3GAM63LATVP5 tag=P457YF0HCHHW type=write size=2412006 2024-05-30T16:03:50.477 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.479+0000 7f08851138c0 1 < suggested remove operation key=GWXNCSJCLXKW tag=AHFMT5K5SLJM type=del 2024-05-30T16:03:50.481 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.479+0000 7f08851138c0 1 > prepared operation key=ZALVY70ISLM5 tag=6H11IK5WKWQ6 type=write size=1030569 2024-05-30T16:03:50.484 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.483+0000 7f08851138c0 -1 < failed to complete operation key=GWXNCSJCLXKW tag=F5NCHMDLAY20 type=del size=1665465: (22) Invalid argument 2024-05-30T16:03:50.490 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.491+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~N44CTY33OBRO.0 size=1048576 2024-05-30T16:03:50.492 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.491+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~N44CTY33OBRO.1 size=314127 2024-05-30T16:03:50.492 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.491+0000 7f08851138c0 1 > prepared multipart upload key=73FJ3HGWYQQK upload=N44CTY33OBRO tag=N38QOJ7H8UGH type=write size=2411279 2024-05-30T16:03:50.498 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.495+0000 7f08851138c0 1 < completed delete operation key=QQILXRJD0C1Z tag=GXT0C1WFYCIU type=del 2024-05-30T16:03:50.502 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.503+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~UD4JUPB6O84T.0 size=1048576 2024-05-30T16:03:50.505 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.507+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~UD4JUPB6O84T.1 size=1048576 2024-05-30T16:03:50.508 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.507+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~UD4JUPB6O84T.2 size=510491 2024-05-30T16:03:50.508 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.507+0000 7f08851138c0 1 > prepared multipart upload key=VRXNO8JLAZYH upload=UD4JUPB6O84T tag=WNLVH1NTKQ07 type=write size=3656219 2024-05-30T16:03:50.515 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.515+0000 7f08851138c0 1 < suggested remove operation key=QCLY10D2A3VF tag=UNZFOAJC7SLC type=del 2024-05-30T16:03:50.518 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.519+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=T8O371KZ3AJ8 type=del size=686972 2024-05-30T16:03:50.522 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.523+0000 7f08851138c0 1 < completed delete operation key=VRXNO8JLAZYH tag=BP3VWDE710HE type=del 2024-05-30T16:03:50.528 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.527+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~7SLIG3BQM6ID.0 size=762710 2024-05-30T16:03:50.528 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.527+0000 7f08851138c0 1 > prepared multipart upload key=GWXNCSJCLXKW upload=7SLIG3BQM6ID tag=NYYE6ER7ALGQ type=write size=1811286 2024-05-30T16:03:50.532 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.531+0000 7f08851138c0 1 < completed write operation key=0RZRBW38WKNZ tag=ZQ8DM34COB6O type=write size=732059 2024-05-30T16:03:50.535 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.535+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=W7W6KF6P7KH2 type=del size=2718419 2024-05-30T16:03:50.539 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.539+0000 7f08851138c0 1 < completed delete operation key=PLWFSKPKVO19 tag=I2DCOLII2J6P type=del 2024-05-30T16:03:50.546 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.547+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~83VYX89BVZ5H.0 size=606851 2024-05-30T16:03:50.546 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.547+0000 7f08851138c0 1 > prepared multipart upload key=XHGMUYBA2VNK upload=83VYX89BVZ5H tag=5WZG8DLKENBI type=write size=1655427 2024-05-30T16:03:50.549 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.547+0000 7f08851138c0 1 < completed write operation key=XHGMUYBA2VNK tag=AMN0IDF77THI type=write size=199926 2024-05-30T16:03:50.555 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.555+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~K1KUVF8H7ETO.0 size=1048576 2024-05-30T16:03:50.559 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.559+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~K1KUVF8H7ETO.1 size=1048576 2024-05-30T16:03:50.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.563+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~K1KUVF8H7ETO.2 size=477700 2024-05-30T16:03:50.562 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.563+0000 7f08851138c0 1 > prepared multipart upload key=PLWFSKPKVO19 upload=K1KUVF8H7ETO tag=Z1TXJR6ZDY8R type=write size=3623428 2024-05-30T16:03:50.565 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.563+0000 7f08851138c0 1 < canceled operation key=KPHZGFJLC2E0 tag=FIGGYQWTX1KY type=write size=295633 2024-05-30T16:03:50.569 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.567+0000 7f08851138c0 1 > prepared operation key=VRXNO8JLAZYH tag=MNTEQOZ70GGC type=del size=2445726 2024-05-30T16:03:50.572 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.571+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~18JBONG3JHVL.0 size=254277 2024-05-30T16:03:50.578 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.579+0000 7f08851138c0 1 < completed multipart upload key=Y3CYZJ83WYNI upload=18JBONG3JHVL tag=EV8IY3RAZWXA type=write size=1302853 2024-05-30T16:03:50.584 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.583+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~W8KNOU16OMRI.0 size=1048576 2024-05-30T16:03:50.586 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.587+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~W8KNOU16OMRI.1 size=168616 2024-05-30T16:03:50.587 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.587+0000 7f08851138c0 1 > prepared multipart upload key=YRP4CSG92QRO upload=W8KNOU16OMRI tag=WUR6NR4RTB67 type=write size=2265768 2024-05-30T16:03:50.590 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.591+0000 7f08851138c0 1 < completed part key=_multipart_ZALVY70ISLM5.2~Q4A9E8ZKU6W5.0 size=1048576 2024-05-30T16:03:50.593 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.591+0000 7f08851138c0 1 < completed part key=_multipart_ZALVY70ISLM5.2~Q4A9E8ZKU6W5.1 size=1048576 2024-05-30T16:03:50.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.595+0000 7f08851138c0 1 < completed part key=_multipart_ZALVY70ISLM5.2~Q4A9E8ZKU6W5.2 size=711803 2024-05-30T16:03:50.601 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.603+0000 7f08851138c0 1 < completed multipart upload key=ZALVY70ISLM5 upload=Q4A9E8ZKU6W5 tag=5HO7UYJHJ2ZR type=write size=3857531 2024-05-30T16:03:50.605 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.603+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=VLQLCT6PUKV1 type=del size=1342011 2024-05-30T16:03:50.605 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.607+0000 7f08851138c0 1 < canceled part key=_multipart_E2FXSXOEUGIG.2~PT06M0D5LMKM.0 size=922310 2024-05-30T16:03:50.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.611+0000 7f08851138c0 1 < canceled multipart upload key=E2FXSXOEUGIG upload=PT06M0D5LMKM tag=TB9DXU7JL1QU type=write size=1970886 2024-05-30T16:03:50.618 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.619+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~DWO8PDJE1K1V.0 size=456282 2024-05-30T16:03:50.618 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.619+0000 7f08851138c0 1 > prepared multipart upload key=QQILXRJD0C1Z upload=DWO8PDJE1K1V tag=3PNMIVP85X6G type=write size=1504858 2024-05-30T16:03:50.621 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.619+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~D3EI2O7MARE5.0 size=1048576 2024-05-30T16:03:50.621 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.619+0000 7f08851138c0 1 < canceled part key=_multipart_V4OQFOOQUXFQ.2~D3EI2O7MARE5.1 size=778853 2024-05-30T16:03:50.630 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.631+0000 7f08851138c0 1 < canceled multipart upload key=V4OQFOOQUXFQ upload=D3EI2O7MARE5 tag=HJ8UU0DYXM33 type=write size=2876005 2024-05-30T16:03:50.633 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.635+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=0XUISC075BGT type=del size=3794627 2024-05-30T16:03:50.637 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.635+0000 7f08851138c0 1 < completed delete operation key=XHGMUYBA2VNK tag=5CUN3WW128SU type=del 2024-05-30T16:03:50.640 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.639+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=0N3693VJCKVQ type=del size=3528708 2024-05-30T16:03:50.644 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.643+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~M8SZOZC1TUM2.0 size=1048576 2024-05-30T16:03:50.647 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.647+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~M8SZOZC1TUM2.1 size=1048576 2024-05-30T16:03:50.651 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.651+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~M8SZOZC1TUM2.2 size=36640 2024-05-30T16:03:50.657 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.655+0000 7f08851138c0 1 < completed multipart upload key=A5BUBY9Z7JQG upload=M8SZOZC1TUM2 tag=NW9KFUBOB9FE type=write size=3182368 2024-05-30T16:03:50.663 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.663+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~2RNYT0DP7JXS.0 size=1048576 2024-05-30T16:03:50.666 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.667+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~2RNYT0DP7JXS.1 size=821264 2024-05-30T16:03:50.666 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.667+0000 7f08851138c0 1 > prepared multipart upload key=0T2E9F3K1USF upload=2RNYT0DP7JXS tag=J3486G36L95U type=write size=2918416 2024-05-30T16:03:50.669 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.671+0000 7f08851138c0 1 < completed part key=_multipart_O0DJ8SCY766V.2~3GAM63LATVP5.0 size=1048576 2024-05-30T16:03:50.672 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.671+0000 7f08851138c0 1 < completed part key=_multipart_O0DJ8SCY766V.2~3GAM63LATVP5.1 size=314854 2024-05-30T16:03:50.678 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.679+0000 7f08851138c0 1 < completed multipart upload key=O0DJ8SCY766V upload=3GAM63LATVP5 tag=P457YF0HCHHW type=write size=2412006 2024-05-30T16:03:50.681 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.679+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=Y5O2UJ3QWQLP type=del size=265204 2024-05-30T16:03:50.684 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.683+0000 7f08851138c0 1 < completed write operation key=ZALVY70ISLM5 tag=6H11IK5WKWQ6 type=write size=1030569 2024-05-30T16:03:50.690 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.691+0000 7f08851138c0 1 > prepared part key=_multipart_MMLF65KLC42Q.2~PVGYQ8HI2XFF.0 size=733063 2024-05-30T16:03:50.690 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.691+0000 7f08851138c0 1 > prepared multipart upload key=MMLF65KLC42Q upload=PVGYQ8HI2XFF tag=ALLCTEFGCVN5 type=write size=1781639 2024-05-30T16:03:50.694 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.695+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~N44CTY33OBRO.0 size=1048576 2024-05-30T16:03:50.697 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.695+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~N44CTY33OBRO.1 size=314127 2024-05-30T16:03:50.702 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.703+0000 7f08851138c0 1 < completed multipart upload key=73FJ3HGWYQQK upload=N44CTY33OBRO tag=N38QOJ7H8UGH type=write size=2411279 2024-05-30T16:03:50.706 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.707+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=NJ9ZHFHWRM12 type=write size=495229 2024-05-30T16:03:50.710 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.711+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~UD4JUPB6O84T.0 size=1048576 2024-05-30T16:03:50.712 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.711+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~UD4JUPB6O84T.1 size=1048576 2024-05-30T16:03:50.715 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.715+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~UD4JUPB6O84T.2 size=510491 2024-05-30T16:03:50.720 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.719+0000 7f08851138c0 1 < completed multipart upload key=VRXNO8JLAZYH upload=UD4JUPB6O84T tag=WNLVH1NTKQ07 type=write size=3656219 2024-05-30T16:03:50.723 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.723+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=92NUGVAHXJKS type=write size=1000490 2024-05-30T16:03:50.729 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.727+0000 7f08851138c0 1 < suggested remove operation key=GWXNCSJCLXKW tag=T8O371KZ3AJ8 type=del 2024-05-30T16:03:50.735 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.735+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~VB91LZ8Z8T41.0 size=1048576 2024-05-30T16:03:50.738 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.739+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~VB91LZ8Z8T41.1 size=1048576 2024-05-30T16:03:50.740 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.739+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~VB91LZ8Z8T41.2 size=421753 2024-05-30T16:03:50.740 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.739+0000 7f08851138c0 1 > prepared multipart upload key=1N6R2HBTWMP2 upload=VB91LZ8Z8T41 tag=QPPUMPBDN7ZZ type=write size=3567481 2024-05-30T16:03:50.744 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.743+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~7SLIG3BQM6ID.0 size=762710 2024-05-30T16:03:50.749 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.747+0000 7f08851138c0 1 < completed multipart upload key=GWXNCSJCLXKW upload=7SLIG3BQM6ID tag=NYYE6ER7ALGQ type=write size=1811286 2024-05-30T16:03:50.755 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.755+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~VCJQGLK246MZ.0 size=1048576 2024-05-30T16:03:50.758 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.759+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~VCJQGLK246MZ.1 size=1048576 2024-05-30T16:03:50.760 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.759+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~VCJQGLK246MZ.2 size=422902 2024-05-30T16:03:50.760 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.759+0000 7f08851138c0 1 > prepared multipart upload key=TPCETCO3Y1M6 upload=VCJQGLK246MZ tag=4QO5GVHTCMKO type=write size=3568630 2024-05-30T16:03:50.763 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.763+0000 7f08851138c0 1 < completed delete operation key=Y8SBI0QZ0ZV8 tag=W7W6KF6P7KH2 type=del 2024-05-30T16:03:50.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.771+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~UVRMEZ17NXOK.0 size=1016302 2024-05-30T16:03:50.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.771+0000 7f08851138c0 1 > prepared multipart upload key=KPHZGFJLC2E0 upload=UVRMEZ17NXOK tag=AQZO33C7RNH9 type=write size=2064878 2024-05-30T16:03:50.770 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.771+0000 7f08851138c0 1 < canceled part key=_multipart_XHGMUYBA2VNK.2~83VYX89BVZ5H.0 size=606851 2024-05-30T16:03:50.775 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.775+0000 7f08851138c0 1 < canceled multipart upload key=XHGMUYBA2VNK upload=83VYX89BVZ5H tag=5WZG8DLKENBI type=write size=1655427 2024-05-30T16:03:50.778 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.779+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=C73SRGU76J56 type=del size=1582853 2024-05-30T16:03:50.782 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.783+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~K1KUVF8H7ETO.0 size=1048576 2024-05-30T16:03:50.784 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.783+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~K1KUVF8H7ETO.1 size=1048576 2024-05-30T16:03:50.787 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.787+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~K1KUVF8H7ETO.2 size=477700 2024-05-30T16:03:50.793 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.791+0000 7f08851138c0 1 < completed multipart upload key=PLWFSKPKVO19 upload=K1KUVF8H7ETO tag=Z1TXJR6ZDY8R type=write size=3623428 2024-05-30T16:03:50.799 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.799+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~0FL9XLG7P72C.0 size=146200 2024-05-30T16:03:50.799 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.799+0000 7f08851138c0 1 > prepared multipart upload key=V4OQFOOQUXFQ upload=0FL9XLG7P72C tag=MIC337KKA339 type=write size=1194776 2024-05-30T16:03:50.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.807+0000 7f08851138c0 1 < suggested remove operation key=VRXNO8JLAZYH tag=MNTEQOZ70GGC type=del 2024-05-30T16:03:50.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.807+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=OEN4RFGZ7DV5 type=del size=1955263 2024-05-30T16:03:50.812 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.811+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~W8KNOU16OMRI.0 size=1048576 2024-05-30T16:03:50.815 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.815+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~W8KNOU16OMRI.1 size=168616 2024-05-30T16:03:50.821 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.819+0000 7f08851138c0 1 < completed multipart upload key=YRP4CSG92QRO upload=W8KNOU16OMRI tag=WUR6NR4RTB67 type=write size=2265768 2024-05-30T16:03:50.824 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.823+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=YGS9HCC8W2AQ type=del size=3607523 2024-05-30T16:03:50.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.827+0000 7f08851138c0 1 < completed delete operation key=3IJWPWRB6FHE tag=VLQLCT6PUKV1 type=del 2024-05-30T16:03:50.834 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.835+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~C2HIMJ90V2YQ.0 size=768084 2024-05-30T16:03:50.834 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.835+0000 7f08851138c0 1 > prepared multipart upload key=WT9RC0UYVG4F upload=C2HIMJ90V2YQ tag=UU8RTR1M260C type=write size=1816660 2024-05-30T16:03:50.838 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.839+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~DWO8PDJE1K1V.0 size=456282 2024-05-30T16:03:50.844 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.843+0000 7f08851138c0 1 < completed multipart upload key=QQILXRJD0C1Z upload=DWO8PDJE1K1V tag=3PNMIVP85X6G type=write size=1504858 2024-05-30T16:03:50.850 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.851+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~1DFUPAO7NAIK.0 size=558912 2024-05-30T16:03:50.850 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.851+0000 7f08851138c0 1 > prepared multipart upload key=RCTV94WRMO53 upload=1DFUPAO7NAIK tag=H8P6EB4STPNO type=write size=1607488 2024-05-30T16:03:50.854 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.855+0000 7f08851138c0 1 < completed delete operation key=MMLF65KLC42Q tag=0XUISC075BGT type=del 2024-05-30T16:03:50.858 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.859+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=3TQLZA1B9Y49 type=del size=3705766 2024-05-30T16:03:50.862 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.863+0000 7f08851138c0 1 < completed delete operation key=A5BUBY9Z7JQG tag=0N3693VJCKVQ type=del 2024-05-30T16:03:50.865 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.867+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=SK8YXSYLVBH5 type=del size=2474979 2024-05-30T16:03:50.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.871+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~2RNYT0DP7JXS.0 size=1048576 2024-05-30T16:03:50.872 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.871+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~2RNYT0DP7JXS.1 size=821264 2024-05-30T16:03:50.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.879+0000 7f08851138c0 1 < completed multipart upload key=0T2E9F3K1USF upload=2RNYT0DP7JXS tag=J3486G36L95U type=write size=2918416 2024-05-30T16:03:50.882 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.883+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=GS237NC5RU0S type=del size=4032864 2024-05-30T16:03:50.886 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.887+0000 7f08851138c0 1 < completed delete operation key=Y8SBI0QZ0ZV8 tag=Y5O2UJ3QWQLP type=del 2024-05-30T16:03:50.889 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.891+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=454Z0551KRHK type=del size=3546918 2024-05-30T16:03:50.890 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.891+0000 7f08851138c0 1 < canceled part key=_multipart_MMLF65KLC42Q.2~PVGYQ8HI2XFF.0 size=733063 2024-05-30T16:03:50.896 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.895+0000 7f08851138c0 1 < canceled multipart upload key=MMLF65KLC42Q upload=PVGYQ8HI2XFF tag=ALLCTEFGCVN5 type=write size=1781639 2024-05-30T16:03:50.900 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.899+0000 7f08851138c0 1 > prepared operation key=ZALVY70ISLM5 tag=H7S3CLQ11CV7 type=del size=3289967 2024-05-30T16:03:50.904 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.903+0000 7f08851138c0 1 < completed write operation key=KPHZGFJLC2E0 tag=NJ9ZHFHWRM12 type=write size=495229 2024-05-30T16:03:50.908 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.907+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=YFZB8V2J7W30 type=del size=1497730 2024-05-30T16:03:50.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.911+0000 7f08851138c0 1 < completed write operation key=NIV7AQ4EV5YP tag=92NUGVAHXJKS type=write size=1000490 2024-05-30T16:03:50.915 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.915+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=MC6QMDU1GTSY type=del size=1969683 2024-05-30T16:03:50.918 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.919+0000 7f08851138c0 1 < completed part key=_multipart_1N6R2HBTWMP2.2~VB91LZ8Z8T41.0 size=1048576 2024-05-30T16:03:50.921 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.919+0000 7f08851138c0 1 < completed part key=_multipart_1N6R2HBTWMP2.2~VB91LZ8Z8T41.1 size=1048576 2024-05-30T16:03:50.924 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.923+0000 7f08851138c0 1 < completed part key=_multipart_1N6R2HBTWMP2.2~VB91LZ8Z8T41.2 size=421753 2024-05-30T16:03:50.930 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.931+0000 7f08851138c0 1 < completed multipart upload key=1N6R2HBTWMP2 upload=VB91LZ8Z8T41 tag=QPPUMPBDN7ZZ type=write size=3567481 2024-05-30T16:03:50.936 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.935+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~CEMFVQ66753J.0 size=15510 2024-05-30T16:03:50.936 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.935+0000 7f08851138c0 1 > prepared multipart upload key=KPHZGFJLC2E0 upload=CEMFVQ66753J tag=CEMYHOD5W3BR type=write size=1064086 2024-05-30T16:03:50.940 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.939+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~VCJQGLK246MZ.0 size=1048576 2024-05-30T16:03:50.945 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.947+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~VCJQGLK246MZ.1 size=1048576 2024-05-30T16:03:50.948 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.947+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~VCJQGLK246MZ.2 size=422902 2024-05-30T16:03:50.954 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.955+0000 7f08851138c0 1 < completed multipart upload key=TPCETCO3Y1M6 upload=VCJQGLK246MZ tag=4QO5GVHTCMKO type=write size=3568630 2024-05-30T16:03:50.961 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.959+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~5758Q2UHDZUV.0 size=1048576 2024-05-30T16:03:50.964 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.963+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~5758Q2UHDZUV.1 size=1048576 2024-05-30T16:03:50.966 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.967+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~5758Q2UHDZUV.2 size=943983 2024-05-30T16:03:50.966 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.967+0000 7f08851138c0 1 > prepared multipart upload key=E2FXSXOEUGIG upload=5758Q2UHDZUV tag=Q615KJI8C0B5 type=write size=4089711 2024-05-30T16:03:50.970 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.971+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~UVRMEZ17NXOK.0 size=1016302 2024-05-30T16:03:50.976 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.975+0000 7f08851138c0 1 < completed multipart upload key=KPHZGFJLC2E0 upload=UVRMEZ17NXOK tag=AQZO33C7RNH9 type=write size=2064878 2024-05-30T16:03:50.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.979+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=UJHC4CUFYZCA type=del size=817430 2024-05-30T16:03:50.988 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.987+0000 7f08851138c0 1 < suggested update operation key=3TBOZWT2P8U8 tag=C73SRGU76J56 type=del size=1582853 2024-05-30T16:03:50.992 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.991+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=PBOEIB847LMG type=del size=3345486 2024-05-30T16:03:50.995 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:50.995+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~0FL9XLG7P72C.0 size=146200 2024-05-30T16:03:51.001 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.003+0000 7f08851138c0 1 < completed multipart upload key=V4OQFOOQUXFQ upload=0FL9XLG7P72C tag=MIC337KKA339 type=write size=1194776 2024-05-30T16:03:51.005 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.003+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=TNJ8Y5IIDI18 type=del size=2265857 2024-05-30T16:03:51.009 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.007+0000 7f08851138c0 1 < completed delete operation key=V4OQFOOQUXFQ tag=OEN4RFGZ7DV5 type=del 2024-05-30T16:03:51.016 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.015+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~OXZXXYLIROIG.0 size=1048576 2024-05-30T16:03:51.019 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.019+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~OXZXXYLIROIG.1 size=938049 2024-05-30T16:03:51.019 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.019+0000 7f08851138c0 1 > prepared multipart upload key=NIV7AQ4EV5YP upload=OXZXXYLIROIG tag=2WSFB8R2WRS8 type=write size=3035201 2024-05-30T16:03:51.022 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.023+0000 7f08851138c0 1 < completed delete operation key=E2FXSXOEUGIG tag=YGS9HCC8W2AQ type=del 2024-05-30T16:03:51.026 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.027+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=RYVH114OK76L type=del size=2520386 2024-05-30T16:03:51.029 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.031+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~C2HIMJ90V2YQ.0 size=768084 2024-05-30T16:03:51.037 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.035+0000 7f08851138c0 1 < completed multipart upload key=WT9RC0UYVG4F upload=C2HIMJ90V2YQ tag=UU8RTR1M260C type=write size=1816660 2024-05-30T16:03:51.042 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.043+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~OSUDOBEICTCJ.0 size=1048576 2024-05-30T16:03:51.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.043+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~OSUDOBEICTCJ.1 size=1048576 2024-05-30T16:03:51.047 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.047+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~OSUDOBEICTCJ.2 size=498209 2024-05-30T16:03:51.047 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.047+0000 7f08851138c0 1 > prepared multipart upload key=O0DJ8SCY766V upload=OSUDOBEICTCJ tag=GB6V9W7S507Q type=write size=3643937 2024-05-30T16:03:51.051 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.051+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~1DFUPAO7NAIK.0 size=558912 2024-05-30T16:03:51.056 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.055+0000 7f08851138c0 1 < completed multipart upload key=RCTV94WRMO53 upload=1DFUPAO7NAIK tag=H8P6EB4STPNO type=write size=1607488 2024-05-30T16:03:51.060 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.059+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=LP7AWTKY9HE3 type=write size=831617 2024-05-30T16:03:51.063 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.063+0000 7f08851138c0 1 < completed delete operation key=MMLF65KLC42Q tag=3TQLZA1B9Y49 type=del 2024-05-30T16:03:51.067 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.067+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=H09N8NRQ14YF type=del size=1690413 2024-05-30T16:03:51.070 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.071+0000 7f08851138c0 1 < completed delete operation key=KPHZGFJLC2E0 tag=SK8YXSYLVBH5 type=del 2024-05-30T16:03:51.074 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.075+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=I6EB776BWLOW type=del size=615625 2024-05-30T16:03:51.078 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.079+0000 7f08851138c0 1 < canceled operation key=1N6R2HBTWMP2 tag=GS237NC5RU0S type=del size=4032864 2024-05-30T16:03:51.084 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.083+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~1Z4U85OKRH6H.0 size=328582 2024-05-30T16:03:51.084 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.083+0000 7f08851138c0 1 > prepared multipart upload key=WT9RC0UYVG4F upload=1Z4U85OKRH6H tag=1Q9TEPC9DUJS type=write size=1377158 2024-05-30T16:03:51.088 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.087+0000 7f08851138c0 1 < completed delete operation key=O0DJ8SCY766V tag=454Z0551KRHK type=del 2024-05-30T16:03:51.092 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.091+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=ET7GV3KI4O1F type=del size=28903 2024-05-30T16:03:51.096 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.095+0000 7f08851138c0 1 < completed delete operation key=ZALVY70ISLM5 tag=H7S3CLQ11CV7 type=del 2024-05-30T16:03:51.099 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.099+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=SRH9DLDGJPP8 type=del size=3039597 2024-05-30T16:03:51.106 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.107+0000 7f08851138c0 1 < suggested update operation key=0T2E9F3K1USF tag=YFZB8V2J7W30 type=del size=1497730 2024-05-30T16:03:51.112 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.111+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~UNO1H270E78V.0 size=1048576 2024-05-30T16:03:51.115 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.115+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~UNO1H270E78V.1 size=650917 2024-05-30T16:03:51.116 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.115+0000 7f08851138c0 1 > prepared multipart upload key=0T2E9F3K1USF upload=UNO1H270E78V tag=LHACCF2HA0B4 type=write size=2748069 2024-05-30T16:03:51.122 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.123+0000 7f08851138c0 1 < suggested remove operation key=JNZI7JX95LX2 tag=MC6QMDU1GTSY type=del 2024-05-30T16:03:51.128 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.127+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~82LD2LQFOYDX.0 size=1048576 2024-05-30T16:03:51.131 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.131+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~82LD2LQFOYDX.1 size=1048576 2024-05-30T16:03:51.134 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.135+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~82LD2LQFOYDX.2 size=652912 2024-05-30T16:03:51.134 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.135+0000 7f08851138c0 1 > prepared multipart upload key=KPHZGFJLC2E0 upload=82LD2LQFOYDX tag=4KT496IEMLY0 type=write size=3798640 2024-05-30T16:03:51.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.139+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~CEMFVQ66753J.0 size=15510 2024-05-30T16:03:51.143 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.143+0000 7f08851138c0 1 < completed multipart upload key=KPHZGFJLC2E0 upload=CEMFVQ66753J tag=CEMYHOD5W3BR type=write size=1064086 2024-05-30T16:03:51.150 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.151+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~BSSC63AE19QN.0 size=1048576 2024-05-30T16:03:51.152 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.151+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~BSSC63AE19QN.1 size=1048576 2024-05-30T16:03:51.155 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.155+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~BSSC63AE19QN.2 size=752405 2024-05-30T16:03:51.155 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.155+0000 7f08851138c0 1 > prepared multipart upload key=VRXNO8JLAZYH upload=BSSC63AE19QN tag=CYZSPSL4FGUB type=write size=3898133 2024-05-30T16:03:51.159 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.159+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~5758Q2UHDZUV.0 size=1048576 2024-05-30T16:03:51.161 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.163+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~5758Q2UHDZUV.1 size=1048576 2024-05-30T16:03:51.164 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.163+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~5758Q2UHDZUV.2 size=943983 2024-05-30T16:03:51.170 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.171+0000 7f08851138c0 1 < completed multipart upload key=E2FXSXOEUGIG upload=5758Q2UHDZUV tag=Q615KJI8C0B5 type=write size=4089711 2024-05-30T16:03:51.173 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.171+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=4VETJUC277ND type=write size=188873 2024-05-30T16:03:51.176 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.175+0000 7f08851138c0 -1 < failed to complete operation key=JNZI7JX95LX2 tag=UJHC4CUFYZCA type=del size=817430: (22) Invalid argument 2024-05-30T16:03:51.180 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.179+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=BRSJ9LCXAJW6 type=del size=3430588 2024-05-30T16:03:51.183 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.183+0000 7f08851138c0 1 < completed delete operation key=QCLY10D2A3VF tag=PBOEIB847LMG type=del 2024-05-30T16:03:51.188 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.187+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=VYNACFK9XDGK type=write size=505596 2024-05-30T16:03:51.192 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.191+0000 7f08851138c0 1 < canceled operation key=E2FXSXOEUGIG tag=TNJ8Y5IIDI18 type=del size=2265857 2024-05-30T16:03:51.200 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.199+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~PFBWHDHWTWY8.0 size=1048576 2024-05-30T16:03:51.202 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.203+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~PFBWHDHWTWY8.1 size=579957 2024-05-30T16:03:51.202 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.203+0000 7f08851138c0 1 > prepared multipart upload key=KPHZGFJLC2E0 upload=PFBWHDHWTWY8 tag=FNHWURNG3E26 type=write size=2677109 2024-05-30T16:03:51.207 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.207+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~OXZXXYLIROIG.0 size=1048576 2024-05-30T16:03:51.210 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.211+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~OXZXXYLIROIG.1 size=938049 2024-05-30T16:03:51.216 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.215+0000 7f08851138c0 1 < completed multipart upload key=NIV7AQ4EV5YP upload=OXZXXYLIROIG tag=2WSFB8R2WRS8 type=write size=3035201 2024-05-30T16:03:51.224 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.223+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~H8X2C424YP5C.0 size=1048576 2024-05-30T16:03:51.226 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.227+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~H8X2C424YP5C.1 size=1048576 2024-05-30T16:03:51.229 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.231+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~H8X2C424YP5C.2 size=695051 2024-05-30T16:03:51.229 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.231+0000 7f08851138c0 1 > prepared multipart upload key=QQILXRJD0C1Z upload=H8X2C424YP5C tag=BVIB67D0XO32 type=write size=3840779 2024-05-30T16:03:51.233 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.231+0000 7f08851138c0 1 < completed delete operation key=3IJWPWRB6FHE tag=RYVH114OK76L type=del 2024-05-30T16:03:51.236 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.235+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=BL297F2BBBHB type=del size=1043168 2024-05-30T16:03:51.240 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.239+0000 7f08851138c0 1 < completed part key=_multipart_O0DJ8SCY766V.2~OSUDOBEICTCJ.0 size=1048576 2024-05-30T16:03:51.243 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.243+0000 7f08851138c0 1 < completed part key=_multipart_O0DJ8SCY766V.2~OSUDOBEICTCJ.1 size=1048576 2024-05-30T16:03:51.246 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.247+0000 7f08851138c0 1 < completed part key=_multipart_O0DJ8SCY766V.2~OSUDOBEICTCJ.2 size=498209 2024-05-30T16:03:51.252 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.251+0000 7f08851138c0 1 < completed multipart upload key=O0DJ8SCY766V upload=OSUDOBEICTCJ tag=GB6V9W7S507Q type=write size=3643937 2024-05-30T16:03:51.255 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.255+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=36QQ5D0ISZUF type=del size=2477799 2024-05-30T16:03:51.259 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.259+0000 7f08851138c0 1 < completed write operation key=364U3OV10VPD tag=LP7AWTKY9HE3 type=write size=831617 2024-05-30T16:03:51.262 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.263+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=5L5GCQZMN7XR type=del size=3333173 2024-05-30T16:03:51.270 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.271+0000 7f08851138c0 1 < suggested remove operation key=73FJ3HGWYQQK tag=H09N8NRQ14YF type=del 2024-05-30T16:03:51.278 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.279+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~2KLFGR0YXD66.0 size=1048576 2024-05-30T16:03:51.281 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.279+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~2KLFGR0YXD66.1 size=1048576 2024-05-30T16:03:51.283 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.283+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~2KLFGR0YXD66.2 size=548951 2024-05-30T16:03:51.283 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.283+0000 7f08851138c0 1 > prepared multipart upload key=O8CWPJ9GW2GO upload=2KLFGR0YXD66 tag=EOOAS4PAELED type=write size=3694679 2024-05-30T16:03:51.287 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.287+0000 7f08851138c0 1 < canceled operation key=Y8SBI0QZ0ZV8 tag=I6EB776BWLOW type=del size=615625 2024-05-30T16:03:51.290 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.291+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=O7IUVJ826WPC type=write size=175220 2024-05-30T16:03:51.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.295+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~1Z4U85OKRH6H.0 size=328582 2024-05-30T16:03:51.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.299+0000 7f08851138c0 1 < completed multipart upload key=WT9RC0UYVG4F upload=1Z4U85OKRH6H tag=1Q9TEPC9DUJS type=write size=1377158 2024-05-30T16:03:51.304 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.303+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=1LYJJOJG44TJ type=del size=21380 2024-05-30T16:03:51.308 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.307+0000 7f08851138c0 1 < completed delete operation key=YRP4CSG92QRO tag=ET7GV3KI4O1F type=del 2024-05-30T16:03:51.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.315+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~0X0JRPXANI5X.0 size=1048576 2024-05-30T16:03:51.317 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.319+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~0X0JRPXANI5X.1 size=1048576 2024-05-30T16:03:51.320 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.319+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~0X0JRPXANI5X.2 size=424422 2024-05-30T16:03:51.320 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.319+0000 7f08851138c0 1 > prepared multipart upload key=1N6R2HBTWMP2 upload=0X0JRPXANI5X tag=CBL49R3X9YKF type=write size=3570150 2024-05-30T16:03:51.324 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.323+0000 7f08851138c0 1 < canceled operation key=RCTV94WRMO53 tag=SRH9DLDGJPP8 type=del size=3039597 2024-05-30T16:03:51.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.331+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~DTILZS6BM2US.0 size=265053 2024-05-30T16:03:51.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.331+0000 7f08851138c0 1 > prepared multipart upload key=3TBOZWT2P8U8 upload=DTILZS6BM2US tag=7AFOO8RN4CGM type=write size=1313629 2024-05-30T16:03:51.334 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.335+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~UNO1H270E78V.0 size=1048576 2024-05-30T16:03:51.334 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.335+0000 7f08851138c0 1 < canceled part key=_multipart_0T2E9F3K1USF.2~UNO1H270E78V.1 size=650917 2024-05-30T16:03:51.340 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.339+0000 7f08851138c0 1 < canceled multipart upload key=0T2E9F3K1USF upload=UNO1H270E78V tag=LHACCF2HA0B4 type=write size=2748069 2024-05-30T16:03:51.347 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.347+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~JBVEMSPPSDZK.0 size=1048576 2024-05-30T16:03:51.350 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.351+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~JBVEMSPPSDZK.1 size=666584 2024-05-30T16:03:51.350 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.351+0000 7f08851138c0 1 > prepared multipart upload key=QQILXRJD0C1Z upload=JBVEMSPPSDZK tag=SGM30ANQZ2W8 type=write size=2763736 2024-05-30T16:03:51.354 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.355+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~82LD2LQFOYDX.0 size=1048576 2024-05-30T16:03:51.357 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.359+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~82LD2LQFOYDX.1 size=1048576 2024-05-30T16:03:51.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.359+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~82LD2LQFOYDX.2 size=652912 2024-05-30T16:03:51.368 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.367+0000 7f08851138c0 1 < completed multipart upload key=KPHZGFJLC2E0 upload=82LD2LQFOYDX tag=4KT496IEMLY0 type=write size=3798640 2024-05-30T16:03:51.372 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.371+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=I3VQ85HZ2UHV type=write size=1021802 2024-05-30T16:03:51.376 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.375+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~BSSC63AE19QN.0 size=1048576 2024-05-30T16:03:51.379 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.379+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~BSSC63AE19QN.1 size=1048576 2024-05-30T16:03:51.382 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.383+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~BSSC63AE19QN.2 size=752405 2024-05-30T16:03:51.388 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.387+0000 7f08851138c0 1 < completed multipart upload key=VRXNO8JLAZYH upload=BSSC63AE19QN tag=CYZSPSL4FGUB type=write size=3898133 2024-05-30T16:03:51.393 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.391+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=VZFCQSFMDOUB type=write size=530902 2024-05-30T16:03:51.401 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.399+0000 7f08851138c0 1 < suggested remove operation key=4N58ZY4E1H6E tag=4VETJUC277ND type=write 2024-05-30T16:03:51.407 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.407+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~ZP4Q1BK477KT.0 size=1048576 2024-05-30T16:03:51.410 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.411+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~ZP4Q1BK477KT.1 size=649314 2024-05-30T16:03:51.410 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.411+0000 7f08851138c0 1 > prepared multipart upload key=XHGMUYBA2VNK upload=ZP4Q1BK477KT tag=347YIMZR48FX type=write size=2746466 2024-05-30T16:03:51.413 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.415+0000 7f08851138c0 1 < completed delete operation key=364U3OV10VPD tag=BRSJ9LCXAJW6 type=del 2024-05-30T16:03:51.417 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.419+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=JXXLMYGRW5SO type=del size=2157341 2024-05-30T16:03:51.421 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.423+0000 7f08851138c0 1 < completed write operation key=1N6R2HBTWMP2 tag=VYNACFK9XDGK type=write size=505596 2024-05-30T16:03:51.429 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.427+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~QB0A7IX5SQDN.0 size=1048576 2024-05-30T16:03:51.432 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.431+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~QB0A7IX5SQDN.1 size=1048576 2024-05-30T16:03:51.434 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.435+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~QB0A7IX5SQDN.2 size=137363 2024-05-30T16:03:51.435 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.435+0000 7f08851138c0 1 > prepared multipart upload key=QCLY10D2A3VF upload=QB0A7IX5SQDN tag=W8APKYOXDQKZ type=write size=3283091 2024-05-30T16:03:51.439 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.439+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~PFBWHDHWTWY8.0 size=1048576 2024-05-30T16:03:51.442 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.443+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~PFBWHDHWTWY8.1 size=579957 2024-05-30T16:03:51.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.447+0000 7f08851138c0 1 < completed multipart upload key=KPHZGFJLC2E0 upload=PFBWHDHWTWY8 tag=FNHWURNG3E26 type=write size=2677109 2024-05-30T16:03:51.451 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.451+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=MPXDTSAGL6OF type=del size=187724 2024-05-30T16:03:51.455 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.455+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~H8X2C424YP5C.0 size=1048576 2024-05-30T16:03:51.458 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.459+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~H8X2C424YP5C.1 size=1048576 2024-05-30T16:03:51.461 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.463+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~H8X2C424YP5C.2 size=695051 2024-05-30T16:03:51.468 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.467+0000 7f08851138c0 1 < completed multipart upload key=QQILXRJD0C1Z upload=H8X2C424YP5C tag=BVIB67D0XO32 type=write size=3840779 2024-05-30T16:03:51.471 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.471+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=85AV2I5Y045J type=del size=1151215 2024-05-30T16:03:51.475 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.475+0000 7f08851138c0 -1 < failed to complete operation key=4N58ZY4E1H6E tag=BL297F2BBBHB type=del size=1043168: (22) Invalid argument 2024-05-30T16:03:51.479 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.479+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=PWDSBT81U3R6 type=del size=2918693 2024-05-30T16:03:51.482 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.483+0000 7f08851138c0 1 < completed delete operation key=O0DJ8SCY766V tag=36QQ5D0ISZUF type=del 2024-05-30T16:03:51.486 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.487+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=NUPREHZUXFXH type=del size=2722368 2024-05-30T16:03:51.489 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.491+0000 7f08851138c0 1 < completed delete operation key=Y8SBI0QZ0ZV8 tag=5L5GCQZMN7XR type=del 2024-05-30T16:03:51.493 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.491+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=4DS9SL9LY81X type=del size=2679992 2024-05-30T16:03:51.494 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.495+0000 7f08851138c0 1 < canceled part key=_multipart_O8CWPJ9GW2GO.2~2KLFGR0YXD66.0 size=1048576 2024-05-30T16:03:51.496 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.495+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~2KLFGR0YXD66.1 size=1048576 2024-05-30T16:03:51.499 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.499+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~2KLFGR0YXD66.2 size=548951 2024-05-30T16:03:51.505 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.503+0000 7f08851138c0 1 < canceled multipart upload key=O8CWPJ9GW2GO upload=2KLFGR0YXD66 tag=EOOAS4PAELED type=write size=3694679 2024-05-30T16:03:51.508 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.507+0000 7f08851138c0 1 > prepared operation key=VRXNO8JLAZYH tag=CI3LCSTWI1CF type=del size=634537 2024-05-30T16:03:51.512 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.511+0000 7f08851138c0 1 < completed write operation key=XHGMUYBA2VNK tag=O7IUVJ826WPC type=write size=175220 2024-05-30T16:03:51.518 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.515+0000 7f08851138c0 1 > prepared operation key=VRXNO8JLAZYH tag=5FXHS2S6GYA5 type=del size=1640444 2024-05-30T16:03:51.519 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.519+0000 7f08851138c0 1 < completed delete operation key=GWXNCSJCLXKW tag=1LYJJOJG44TJ type=del 2024-05-30T16:03:51.523 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.523+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=JH617H5OW2FL type=del size=2120110 2024-05-30T16:03:51.527 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.527+0000 7f08851138c0 1 < completed part key=_multipart_1N6R2HBTWMP2.2~0X0JRPXANI5X.0 size=1048576 2024-05-30T16:03:51.530 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.531+0000 7f08851138c0 1 < completed part key=_multipart_1N6R2HBTWMP2.2~0X0JRPXANI5X.1 size=1048576 2024-05-30T16:03:51.533 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.531+0000 7f08851138c0 1 < completed part key=_multipart_1N6R2HBTWMP2.2~0X0JRPXANI5X.2 size=424422 2024-05-30T16:03:51.539 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.539+0000 7f08851138c0 1 < completed multipart upload key=1N6R2HBTWMP2 upload=0X0JRPXANI5X tag=CBL49R3X9YKF type=write size=3570150 2024-05-30T16:03:51.546 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.547+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~TXYQ5FOJLHUC.0 size=1048576 2024-05-30T16:03:51.548 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.547+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~TXYQ5FOJLHUC.1 size=192245 2024-05-30T16:03:51.548 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.547+0000 7f08851138c0 1 > prepared multipart upload key=WT9RC0UYVG4F upload=TXYQ5FOJLHUC tag=Y7NYJHS5PRT9 type=write size=2289397 2024-05-30T16:03:51.552 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.551+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~DTILZS6BM2US.0 size=265053 2024-05-30T16:03:51.558 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.559+0000 7f08851138c0 1 < completed multipart upload key=3TBOZWT2P8U8 upload=DTILZS6BM2US tag=7AFOO8RN4CGM type=write size=1313629 2024-05-30T16:03:51.565 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.563+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~B8FGT9ELECYB.0 size=856540 2024-05-30T16:03:51.565 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.563+0000 7f08851138c0 1 > prepared multipart upload key=QCLY10D2A3VF upload=B8FGT9ELECYB tag=SWO1N4IE20QH type=write size=1905116 2024-05-30T16:03:51.566 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.567+0000 7f08851138c0 1 < canceled part key=_multipart_QQILXRJD0C1Z.2~JBVEMSPPSDZK.0 size=1048576 2024-05-30T16:03:51.569 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.567+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~JBVEMSPPSDZK.1 size=666584 2024-05-30T16:03:51.574 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.575+0000 7f08851138c0 1 < canceled multipart upload key=QQILXRJD0C1Z upload=JBVEMSPPSDZK tag=SGM30ANQZ2W8 type=write size=2763736 2024-05-30T16:03:51.581 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.583+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~E3ISOBK2MG04.0 size=90565 2024-05-30T16:03:51.582 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.583+0000 7f08851138c0 1 > prepared multipart upload key=3TBOZWT2P8U8 upload=E3ISOBK2MG04 tag=1JT8NU1QCVKP type=write size=1139141 2024-05-30T16:03:51.585 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.583+0000 7f08851138c0 1 < completed write operation key=PLWFSKPKVO19 tag=I3VQ85HZ2UHV type=write size=1021802 2024-05-30T16:03:51.592 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.591+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~DWG6XWKBML3E.0 size=806284 2024-05-30T16:03:51.592 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.591+0000 7f08851138c0 1 > prepared multipart upload key=3IJWPWRB6FHE upload=DWG6XWKBML3E tag=KBP4HADNTBVB type=write size=1854860 2024-05-30T16:03:51.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.595+0000 7f08851138c0 1 < canceled operation key=KPHZGFJLC2E0 tag=VZFCQSFMDOUB type=write size=530902 2024-05-30T16:03:51.604 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.603+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~SFADBCMP5O67.0 size=1048576 2024-05-30T16:03:51.606 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.607+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~SFADBCMP5O67.1 size=788895 2024-05-30T16:03:51.607 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.607+0000 7f08851138c0 1 > prepared multipart upload key=YRP4CSG92QRO upload=SFADBCMP5O67 tag=EQ0RKIZ9AFXY type=write size=2886047 2024-05-30T16:03:51.608 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.607+0000 7f08851138c0 1 < canceled part key=_multipart_XHGMUYBA2VNK.2~ZP4Q1BK477KT.0 size=1048576 2024-05-30T16:03:51.610 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.611+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~ZP4Q1BK477KT.1 size=649314 2024-05-30T16:03:51.618 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.619+0000 7f08851138c0 1 < canceled multipart upload key=XHGMUYBA2VNK upload=ZP4Q1BK477KT tag=347YIMZR48FX type=write size=2746466 2024-05-30T16:03:51.624 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.623+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~EQ4LGVPVBH6C.0 size=557545 2024-05-30T16:03:51.624 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.623+0000 7f08851138c0 1 > prepared multipart upload key=364U3OV10VPD upload=EQ4LGVPVBH6C tag=VHG6XK3PCZY3 type=write size=1606121 2024-05-30T16:03:51.628 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.627+0000 7f08851138c0 1 < completed delete operation key=KPHZGFJLC2E0 tag=JXXLMYGRW5SO type=del 2024-05-30T16:03:51.632 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.631+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=LN70M51K0FVQ type=del size=3497090 2024-05-30T16:03:51.636 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.635+0000 7f08851138c0 1 < completed part key=_multipart_QCLY10D2A3VF.2~QB0A7IX5SQDN.0 size=1048576 2024-05-30T16:03:51.639 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.639+0000 7f08851138c0 1 < completed part key=_multipart_QCLY10D2A3VF.2~QB0A7IX5SQDN.1 size=1048576 2024-05-30T16:03:51.641 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.643+0000 7f08851138c0 1 < completed part key=_multipart_QCLY10D2A3VF.2~QB0A7IX5SQDN.2 size=137363 2024-05-30T16:03:51.647 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.647+0000 7f08851138c0 1 < completed multipart upload key=QCLY10D2A3VF upload=QB0A7IX5SQDN tag=W8APKYOXDQKZ type=write size=3283091 2024-05-30T16:03:51.651 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.651+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=56LQ0U7QFEMO type=del size=3286040 2024-05-30T16:03:51.654 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.655+0000 7f08851138c0 1 < completed delete operation key=A5BUBY9Z7JQG tag=MPXDTSAGL6OF type=del 2024-05-30T16:03:51.657 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.659+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=UKVO1K4V9AXE type=write size=760128 2024-05-30T16:03:51.661 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.659+0000 7f08851138c0 1 < completed delete operation key=QCLY10D2A3VF tag=85AV2I5Y045J type=del 2024-05-30T16:03:51.667 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.667+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~CP3K4KRQZ2CU.0 size=1048576 2024-05-30T16:03:51.670 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.671+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~CP3K4KRQZ2CU.1 size=133749 2024-05-30T16:03:51.670 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.671+0000 7f08851138c0 1 > prepared multipart upload key=V4OQFOOQUXFQ upload=CP3K4KRQZ2CU tag=K5VRE2ITMM94 type=write size=2230901 2024-05-30T16:03:51.673 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.671+0000 7f08851138c0 1 < completed delete operation key=GWXNCSJCLXKW tag=PWDSBT81U3R6 type=del 2024-05-30T16:03:51.676 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.675+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=TNA6UB4FP9F3 type=del size=1690882 2024-05-30T16:03:51.683 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.683+0000 7f08851138c0 1 < suggested remove operation key=3TBOZWT2P8U8 tag=NUPREHZUXFXH type=del 2024-05-30T16:03:51.689 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.691+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~WA0ZIVPG41D8.0 size=1048576 2024-05-30T16:03:51.692 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.691+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~WA0ZIVPG41D8.1 size=982606 2024-05-30T16:03:51.692 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.691+0000 7f08851138c0 1 > prepared multipart upload key=O0DJ8SCY766V upload=WA0ZIVPG41D8 tag=JEVRB7NTA5HV type=write size=3079758 2024-05-30T16:03:51.695 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.695+0000 7f08851138c0 1 < completed delete operation key=TPCETCO3Y1M6 tag=4DS9SL9LY81X type=del 2024-05-30T16:03:51.702 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.703+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~QVWFRFW22F70.0 size=1048576 2024-05-30T16:03:51.704 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.703+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~QVWFRFW22F70.1 size=467891 2024-05-30T16:03:51.704 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.703+0000 7f08851138c0 1 > prepared multipart upload key=WT9RC0UYVG4F upload=QVWFRFW22F70 tag=9WWWS6NNKC3Z type=write size=2565043 2024-05-30T16:03:51.708 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.707+0000 7f08851138c0 1 < completed delete operation key=VRXNO8JLAZYH tag=CI3LCSTWI1CF type=del 2024-05-30T16:03:51.711 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.711+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=GAQDG1EYNW77 type=write size=80828 2024-05-30T16:03:51.719 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.719+0000 7f08851138c0 1 < suggested remove operation key=VRXNO8JLAZYH tag=5FXHS2S6GYA5 type=del 2024-05-30T16:03:51.722 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.723+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=WHJZITK1C7GL type=del size=3635541 2024-05-30T16:03:51.726 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.727+0000 7f08851138c0 1 < completed delete operation key=Y8SBI0QZ0ZV8 tag=JH617H5OW2FL type=del 2024-05-30T16:03:51.730 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.731+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=7Z838NAK7SZG type=del size=1565142 2024-05-30T16:03:51.734 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.735+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~TXYQ5FOJLHUC.0 size=1048576 2024-05-30T16:03:51.734 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.735+0000 7f08851138c0 1 < canceled part key=_multipart_WT9RC0UYVG4F.2~TXYQ5FOJLHUC.1 size=192245 2024-05-30T16:03:51.740 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.739+0000 7f08851138c0 1 < canceled multipart upload key=WT9RC0UYVG4F upload=TXYQ5FOJLHUC tag=Y7NYJHS5PRT9 type=write size=2289397 2024-05-30T16:03:51.746 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.747+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~08101FLAGVIV.0 size=903965 2024-05-30T16:03:51.746 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.747+0000 7f08851138c0 1 > prepared multipart upload key=QCLY10D2A3VF upload=08101FLAGVIV tag=JIAFW2VVCY3Q type=write size=1952541 2024-05-30T16:03:51.750 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.751+0000 7f08851138c0 1 < completed part key=_multipart_QCLY10D2A3VF.2~B8FGT9ELECYB.0 size=856540 2024-05-30T16:03:51.756 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.755+0000 7f08851138c0 1 < completed multipart upload key=QCLY10D2A3VF upload=B8FGT9ELECYB tag=SWO1N4IE20QH type=write size=1905116 2024-05-30T16:03:51.761 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.763+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~2K98UNPSREL1.0 size=1048576 2024-05-30T16:03:51.764 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.763+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~2K98UNPSREL1.1 size=1048576 2024-05-30T16:03:51.767 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.767+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~2K98UNPSREL1.2 size=171218 2024-05-30T16:03:51.767 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.767+0000 7f08851138c0 1 > prepared multipart upload key=603HBR8PRH6L upload=2K98UNPSREL1 tag=P4Z5TAVM0R5R type=write size=3316946 2024-05-30T16:03:51.770 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.771+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~E3ISOBK2MG04.0 size=90565 2024-05-30T16:03:51.776 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.775+0000 7f08851138c0 1 < completed multipart upload key=3TBOZWT2P8U8 upload=E3ISOBK2MG04 tag=1JT8NU1QCVKP type=write size=1139141 2024-05-30T16:03:51.779 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.779+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=B0A133KR32EH type=del size=2690365 2024-05-30T16:03:51.783 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.783+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~DWG6XWKBML3E.0 size=806284 2024-05-30T16:03:51.789 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.787+0000 7f08851138c0 1 < completed multipart upload key=3IJWPWRB6FHE upload=DWG6XWKBML3E tag=KBP4HADNTBVB type=write size=1854860 2024-05-30T16:03:51.792 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.791+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=3PG968ADS0WR type=del size=978546 2024-05-30T16:03:51.796 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.795+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~SFADBCMP5O67.0 size=1048576 2024-05-30T16:03:51.799 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.799+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~SFADBCMP5O67.1 size=788895 2024-05-30T16:03:51.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.807+0000 7f08851138c0 1 < completed multipart upload key=YRP4CSG92QRO upload=SFADBCMP5O67 tag=EQ0RKIZ9AFXY type=write size=2886047 2024-05-30T16:03:51.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.811+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=DFA5JER0Z6LK type=del size=2740502 2024-05-30T16:03:51.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.811+0000 7f08851138c0 1 < canceled part key=_multipart_364U3OV10VPD.2~EQ4LGVPVBH6C.0 size=557545 2024-05-30T16:03:51.816 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.815+0000 7f08851138c0 1 < canceled multipart upload key=364U3OV10VPD upload=EQ4LGVPVBH6C tag=VHG6XK3PCZY3 type=write size=1606121 2024-05-30T16:03:51.819 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.819+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=B9K4O0GMBW1B type=del size=4186880 2024-05-30T16:03:51.823 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.823+0000 7f08851138c0 1 < completed delete operation key=QCLY10D2A3VF tag=LN70M51K0FVQ type=del 2024-05-30T16:03:51.827 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.827+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=2V6XVWBTLL7I type=del size=888889 2024-05-30T16:03:51.830 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.831+0000 7f08851138c0 1 < completed delete operation key=V4OQFOOQUXFQ tag=56LQ0U7QFEMO type=del 2024-05-30T16:03:51.834 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.835+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=DVHMKQAF0O9V type=write size=148177 2024-05-30T16:03:51.838 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.839+0000 7f08851138c0 1 < completed write operation key=73FJ3HGWYQQK tag=UKVO1K4V9AXE type=write size=760128 2024-05-30T16:03:51.842 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.843+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=71H30AUUIV5R type=del size=3532529 2024-05-30T16:03:51.845 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.847+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~CP3K4KRQZ2CU.0 size=1048576 2024-05-30T16:03:51.848 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.847+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~CP3K4KRQZ2CU.1 size=133749 2024-05-30T16:03:51.854 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.855+0000 7f08851138c0 1 < completed multipart upload key=V4OQFOOQUXFQ upload=CP3K4KRQZ2CU tag=K5VRE2ITMM94 type=write size=2230901 2024-05-30T16:03:51.861 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.859+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~NDLG2DAHZZ30.0 size=1048576 2024-05-30T16:03:51.864 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.863+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~NDLG2DAHZZ30.1 size=1048576 2024-05-30T16:03:51.866 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.867+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~NDLG2DAHZZ30.2 size=943272 2024-05-30T16:03:51.866 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.867+0000 7f08851138c0 1 > prepared multipart upload key=O0DJ8SCY766V upload=NDLG2DAHZZ30 tag=N6M0X77EPLIG type=write size=4089000 2024-05-30T16:03:51.870 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.871+0000 7f08851138c0 1 < canceled operation key=Y3CYZJ83WYNI tag=TNA6UB4FP9F3 type=del size=1690882 2024-05-30T16:03:51.876 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.875+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~EVQRN2Y917OZ.0 size=724511 2024-05-30T16:03:51.876 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.875+0000 7f08851138c0 1 > prepared multipart upload key=VRXNO8JLAZYH upload=EVQRN2Y917OZ tag=VE7DN1E54ICI type=write size=1773087 2024-05-30T16:03:51.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.875+0000 7f08851138c0 1 < canceled part key=_multipart_O0DJ8SCY766V.2~WA0ZIVPG41D8.0 size=1048576 2024-05-30T16:03:51.880 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.879+0000 7f08851138c0 1 < completed part key=_multipart_O0DJ8SCY766V.2~WA0ZIVPG41D8.1 size=982606 2024-05-30T16:03:51.886 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.887+0000 7f08851138c0 1 < canceled multipart upload key=O0DJ8SCY766V upload=WA0ZIVPG41D8 tag=JEVRB7NTA5HV type=write size=3079758 2024-05-30T16:03:51.889 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.887+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=PF9DY1YA3MN2 type=del size=3928535 2024-05-30T16:03:51.893 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.891+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~QVWFRFW22F70.0 size=1048576 2024-05-30T16:03:51.896 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.895+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~QVWFRFW22F70.1 size=467891 2024-05-30T16:03:51.902 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.903+0000 7f08851138c0 1 < completed multipart upload key=WT9RC0UYVG4F upload=QVWFRFW22F70 tag=9WWWS6NNKC3Z type=write size=2565043 2024-05-30T16:03:51.905 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.907+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=V5YEHWWHWW5K type=del size=3440717 2024-05-30T16:03:51.909 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.907+0000 7f08851138c0 1 < completed write operation key=0T2E9F3K1USF tag=GAQDG1EYNW77 type=write size=80828 2024-05-30T16:03:51.916 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.915+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~85LJ38NKUUHM.0 size=1048576 2024-05-30T16:03:51.918 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.919+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~85LJ38NKUUHM.1 size=46962 2024-05-30T16:03:51.918 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.919+0000 7f08851138c0 1 > prepared multipart upload key=603HBR8PRH6L upload=85LJ38NKUUHM tag=K7Q533KBLO1S type=write size=2144114 2024-05-30T16:03:51.922 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.923+0000 7f08851138c0 1 < completed delete operation key=3IJWPWRB6FHE tag=WHJZITK1C7GL type=del 2024-05-30T16:03:51.926 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.927+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=YRW6M42U2LOO type=del size=2203563 2024-05-30T16:03:51.929 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.931+0000 7f08851138c0 1 < completed delete operation key=4N58ZY4E1H6E tag=7Z838NAK7SZG type=del 2024-05-30T16:03:51.933 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.931+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=EURYCYEA8CVI type=del size=2342683 2024-05-30T16:03:51.934 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.935+0000 7f08851138c0 1 < canceled part key=_multipart_QCLY10D2A3VF.2~08101FLAGVIV.0 size=903965 2024-05-30T16:03:51.939 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.939+0000 7f08851138c0 1 < canceled multipart upload key=QCLY10D2A3VF upload=08101FLAGVIV tag=JIAFW2VVCY3Q type=write size=1952541 2024-05-30T16:03:51.943 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.943+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=OAGBNFK1J9BL type=del size=2722555 2024-05-30T16:03:51.950 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.951+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~2K98UNPSREL1.0 size=1048576 2024-05-30T16:03:51.950 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.951+0000 7f08851138c0 1 < canceled part key=_multipart_603HBR8PRH6L.2~2K98UNPSREL1.1 size=1048576 2024-05-30T16:03:51.952 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.951+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~2K98UNPSREL1.2 size=171218 2024-05-30T16:03:51.958 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.959+0000 7f08851138c0 1 < canceled multipart upload key=603HBR8PRH6L upload=2K98UNPSREL1 tag=P4Z5TAVM0R5R type=write size=3316946 2024-05-30T16:03:51.962 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.963+0000 7f08851138c0 1 > prepared operation key=ZALVY70ISLM5 tag=F807VQNRNT8W type=del size=2185874 2024-05-30T16:03:51.966 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.967+0000 7f08851138c0 1 < completed delete operation key=NIV7AQ4EV5YP tag=B0A133KR32EH type=del 2024-05-30T16:03:51.972 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.971+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~A3GF9B0XVWPG.0 size=1048576 2024-05-30T16:03:51.975 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.975+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~A3GF9B0XVWPG.1 size=1048576 2024-05-30T16:03:51.977 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.979+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~A3GF9B0XVWPG.2 size=610516 2024-05-30T16:03:51.978 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.979+0000 7f08851138c0 1 > prepared multipart upload key=WT9RC0UYVG4F upload=A3GF9B0XVWPG tag=K2EWS5YI7LNG type=write size=3756244 2024-05-30T16:03:51.985 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.983+0000 7f08851138c0 1 < suggested remove operation key=RCTV94WRMO53 tag=3PG968ADS0WR type=del 2024-05-30T16:03:51.988 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.987+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=VTTOAYD8CFLW type=del size=323691 2024-05-30T16:03:51.992 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.991+0000 7f08851138c0 1 < completed delete operation key=603HBR8PRH6L tag=DFA5JER0Z6LK type=del 2024-05-30T16:03:51.995 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:51.995+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=280EM89EGNM7 type=del size=119570 2024-05-30T16:03:52.003 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.003+0000 7f08851138c0 1 < suggested update operation key=E2FXSXOEUGIG tag=B9K4O0GMBW1B type=del size=4186880 2024-05-30T16:03:52.006 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.007+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=T8RFFS7WUKXL type=del size=3194483 2024-05-30T16:03:52.010 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.011+0000 7f08851138c0 1 < completed delete operation key=QQILXRJD0C1Z tag=2V6XVWBTLL7I type=del 2024-05-30T16:03:52.014 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.015+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=OJUWIQYEYUN2 type=del size=3372873 2024-05-30T16:03:52.017 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.019+0000 7f08851138c0 1 < canceled operation key=OKBB0VU765TW tag=DVHMKQAF0O9V type=write size=148177 2024-05-30T16:03:52.022 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.023+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=S41KR6B25SDE type=del size=1482036 2024-05-30T16:03:52.026 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.027+0000 7f08851138c0 1 < canceled operation key=QCLY10D2A3VF tag=71H30AUUIV5R type=del size=3532529 2024-05-30T16:03:52.033 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.035+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~JL2S3YBYUZMJ.0 size=1048576 2024-05-30T16:03:52.036 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.035+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~JL2S3YBYUZMJ.1 size=1048576 2024-05-30T16:03:52.039 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.039+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~JL2S3YBYUZMJ.2 size=643251 2024-05-30T16:03:52.039 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.039+0000 7f08851138c0 1 > prepared multipart upload key=3IJWPWRB6FHE upload=JL2S3YBYUZMJ tag=0SOA9QGOBUYK type=write size=3788979 2024-05-30T16:03:52.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.043+0000 7f08851138c0 1 < completed part key=_multipart_O0DJ8SCY766V.2~NDLG2DAHZZ30.0 size=1048576 2024-05-30T16:03:52.047 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.047+0000 7f08851138c0 1 < completed part key=_multipart_O0DJ8SCY766V.2~NDLG2DAHZZ30.1 size=1048576 2024-05-30T16:03:52.050 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.051+0000 7f08851138c0 1 < completed part key=_multipart_O0DJ8SCY766V.2~NDLG2DAHZZ30.2 size=943272 2024-05-30T16:03:52.057 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.055+0000 7f08851138c0 1 < completed multipart upload key=O0DJ8SCY766V upload=NDLG2DAHZZ30 tag=N6M0X77EPLIG type=write size=4089000 2024-05-30T16:03:52.061 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.063+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=JEVNAE5G80BB type=write size=628648 2024-05-30T16:03:52.065 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.067+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~EVQRN2Y917OZ.0 size=724511 2024-05-30T16:03:52.071 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.071+0000 7f08851138c0 1 < completed multipart upload key=VRXNO8JLAZYH upload=EVQRN2Y917OZ tag=VE7DN1E54ICI type=write size=1773087 2024-05-30T16:03:52.075 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.075+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=FM7WTYUHD7KE type=write size=466933 2024-05-30T16:03:52.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.079+0000 7f08851138c0 1 < completed delete operation key=0RZRBW38WKNZ tag=PF9DY1YA3MN2 type=del 2024-05-30T16:03:52.083 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.083+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=540LT1HGV8FP type=del size=1027041 2024-05-30T16:03:52.086 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.087+0000 7f08851138c0 1 < completed delete operation key=QCLY10D2A3VF tag=V5YEHWWHWW5K type=del 2024-05-30T16:03:52.092 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.091+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~169AQYA89RRM.0 size=1048576 2024-05-30T16:03:52.095 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.095+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~169AQYA89RRM.1 size=944825 2024-05-30T16:03:52.095 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.095+0000 7f08851138c0 1 > prepared multipart upload key=PLWFSKPKVO19 upload=169AQYA89RRM tag=150U7IB9LT8S type=write size=3041977 2024-05-30T16:03:52.099 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.099+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~85LJ38NKUUHM.0 size=1048576 2024-05-30T16:03:52.102 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.103+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~85LJ38NKUUHM.1 size=46962 2024-05-30T16:03:52.107 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.107+0000 7f08851138c0 1 < completed multipart upload key=603HBR8PRH6L upload=85LJ38NKUUHM tag=K7Q533KBLO1S type=write size=2144114 2024-05-30T16:03:52.114 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.115+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~AXLY6F8SJDOD.0 size=313116 2024-05-30T16:03:52.114 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.115+0000 7f08851138c0 1 > prepared multipart upload key=Y8SBI0QZ0ZV8 upload=AXLY6F8SJDOD tag=ZYG5U4UK13AV type=write size=1361692 2024-05-30T16:03:52.117 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.119+0000 7f08851138c0 1 < completed delete operation key=V4OQFOOQUXFQ tag=YRW6M42U2LOO type=del 2024-05-30T16:03:52.121 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.119+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=K9W7BLX298AA type=del size=2706708 2024-05-30T16:03:52.124 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.123+0000 7f08851138c0 1 < completed delete operation key=JNZI7JX95LX2 tag=EURYCYEA8CVI type=del 2024-05-30T16:03:52.131 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.131+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~ROVLGI934JU0.0 size=1048576 2024-05-30T16:03:52.133 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.135+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~ROVLGI934JU0.1 size=1048576 2024-05-30T16:03:52.136 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.135+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~ROVLGI934JU0.2 size=913900 2024-05-30T16:03:52.136 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.135+0000 7f08851138c0 1 > prepared multipart upload key=KPHZGFJLC2E0 upload=ROVLGI934JU0 tag=WOG6VY68M52E type=write size=4059628 2024-05-30T16:03:52.140 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.139+0000 7f08851138c0 1 < completed delete operation key=MMLF65KLC42Q tag=OAGBNFK1J9BL type=del 2024-05-30T16:03:52.147 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.147+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~I180QICQHOSR.0 size=1048576 2024-05-30T16:03:52.150 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.151+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~I180QICQHOSR.1 size=607970 2024-05-30T16:03:52.150 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.151+0000 7f08851138c0 1 > prepared multipart upload key=A5BUBY9Z7JQG upload=I180QICQHOSR tag=PN6O4DVVX1ST type=write size=2705122 2024-05-30T16:03:52.153 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.155+0000 7f08851138c0 1 < completed delete operation key=ZALVY70ISLM5 tag=F807VQNRNT8W type=del 2024-05-30T16:03:52.158 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.159+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=XYR6GXW97Q5G type=del size=2518023 2024-05-30T16:03:52.162 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.163+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~A3GF9B0XVWPG.0 size=1048576 2024-05-30T16:03:52.165 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.163+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~A3GF9B0XVWPG.1 size=1048576 2024-05-30T16:03:52.169 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.167+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~A3GF9B0XVWPG.2 size=610516 2024-05-30T16:03:52.176 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.175+0000 7f08851138c0 1 < completed multipart upload key=WT9RC0UYVG4F upload=A3GF9B0XVWPG tag=K2EWS5YI7LNG type=write size=3756244 2024-05-30T16:03:52.180 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.179+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=KHZSQMJ6EP09 type=del size=107855 2024-05-30T16:03:52.184 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.183+0000 7f08851138c0 1 < completed delete operation key=O0DJ8SCY766V tag=VTTOAYD8CFLW type=del 2024-05-30T16:03:52.188 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.187+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=UCC6ES4ZO8B8 type=del size=2752956 2024-05-30T16:03:52.192 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.191+0000 7f08851138c0 1 < completed delete operation key=Y3CYZJ83WYNI tag=280EM89EGNM7 type=del 2024-05-30T16:03:52.196 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.195+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=I3X6VZVJ4NR8 type=write size=1008725 2024-05-30T16:03:52.203 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.203+0000 7f08851138c0 1 < suggested update operation key=3IJWPWRB6FHE tag=T8RFFS7WUKXL type=del size=3194483 2024-05-30T16:03:52.210 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.211+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~1KS4GF3EE8ZO.0 size=1048576 2024-05-30T16:03:52.213 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.211+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~1KS4GF3EE8ZO.1 size=1048576 2024-05-30T16:03:52.215 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.215+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~1KS4GF3EE8ZO.2 size=275578 2024-05-30T16:03:52.216 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.215+0000 7f08851138c0 1 > prepared multipart upload key=O0DJ8SCY766V upload=1KS4GF3EE8ZO tag=8NZ071NZX0UF type=write size=3421306 2024-05-30T16:03:52.219 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.219+0000 7f08851138c0 1 < completed delete operation key=0RZRBW38WKNZ tag=OJUWIQYEYUN2 type=del 2024-05-30T16:03:52.222 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.223+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=2967QX4BJHJX type=write size=171435 2024-05-30T16:03:52.226 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.227+0000 7f08851138c0 1 < completed delete operation key=1N6R2HBTWMP2 tag=S41KR6B25SDE type=del 2024-05-30T16:03:52.230 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.231+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=7A4H2OXLIP1N type=del size=582228 2024-05-30T16:03:52.234 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.235+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~JL2S3YBYUZMJ.0 size=1048576 2024-05-30T16:03:52.237 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.235+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~JL2S3YBYUZMJ.1 size=1048576 2024-05-30T16:03:52.240 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.239+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~JL2S3YBYUZMJ.2 size=643251 2024-05-30T16:03:52.246 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.247+0000 7f08851138c0 1 < completed multipart upload key=3IJWPWRB6FHE upload=JL2S3YBYUZMJ tag=0SOA9QGOBUYK type=write size=3788979 2024-05-30T16:03:52.252 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.251+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~QM3S7FNOWE18.0 size=597360 2024-05-30T16:03:52.252 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.251+0000 7f08851138c0 1 > prepared multipart upload key=0RZRBW38WKNZ upload=QM3S7FNOWE18 tag=UCJ6D1Q1UIBD type=write size=1645936 2024-05-30T16:03:52.256 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.255+0000 7f08851138c0 1 < completed write operation key=KPHZGFJLC2E0 tag=JEVNAE5G80BB type=write size=628648 2024-05-30T16:03:52.260 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.259+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=GU62S6ZO4FOQ type=del size=1363932 2024-05-30T16:03:52.263 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.263+0000 7f08851138c0 1 < completed write operation key=O0DJ8SCY766V tag=FM7WTYUHD7KE type=write size=466933 2024-05-30T16:03:52.267 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.267+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=H8PQ1KZDQYIR type=del size=1547857 2024-05-30T16:03:52.271 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.271+0000 7f08851138c0 1 < completed delete operation key=MMLF65KLC42Q tag=540LT1HGV8FP type=del 2024-05-30T16:03:52.275 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.275+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=2VJDQAK8IA66 type=del size=1442065 2024-05-30T16:03:52.278 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.279+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~169AQYA89RRM.0 size=1048576 2024-05-30T16:03:52.281 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.283+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~169AQYA89RRM.1 size=944825 2024-05-30T16:03:52.287 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.287+0000 7f08851138c0 1 < completed multipart upload key=PLWFSKPKVO19 upload=169AQYA89RRM tag=150U7IB9LT8S type=write size=3041977 2024-05-30T16:03:52.291 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.291+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=Z14X6BKTXYKZ type=del size=1077092 2024-05-30T16:03:52.297 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.295+0000 7f08851138c0 1 < completed part key=_multipart_Y8SBI0QZ0ZV8.2~AXLY6F8SJDOD.0 size=313116 2024-05-30T16:03:52.303 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.303+0000 7f08851138c0 1 < completed multipart upload key=Y8SBI0QZ0ZV8 upload=AXLY6F8SJDOD tag=ZYG5U4UK13AV type=write size=1361692 2024-05-30T16:03:52.309 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.307+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~4UGV98NMUE2N.0 size=1048576 2024-05-30T16:03:52.312 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.311+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~4UGV98NMUE2N.1 size=1048576 2024-05-30T16:03:52.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.315+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~4UGV98NMUE2N.2 size=342874 2024-05-30T16:03:52.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.315+0000 7f08851138c0 1 > prepared multipart upload key=VRXNO8JLAZYH upload=4UGV98NMUE2N tag=YXLMNN9ZPBV2 type=write size=3488602 2024-05-30T16:03:52.318 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.319+0000 7f08851138c0 1 < completed delete operation key=MMLF65KLC42Q tag=K9W7BLX298AA type=del 2024-05-30T16:03:52.321 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.319+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=AM8JT53LSEAD type=del size=3332806 2024-05-30T16:03:52.324 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.323+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~ROVLGI934JU0.0 size=1048576 2024-05-30T16:03:52.327 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.327+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~ROVLGI934JU0.1 size=1048576 2024-05-30T16:03:52.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.331+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~ROVLGI934JU0.2 size=913900 2024-05-30T16:03:52.335 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.335+0000 7f08851138c0 1 < completed multipart upload key=KPHZGFJLC2E0 upload=ROVLGI934JU0 tag=WOG6VY68M52E type=write size=4059628 2024-05-30T16:03:52.341 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.343+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~6E0XFRB35H92.0 size=1048576 2024-05-30T16:03:52.344 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.343+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~6E0XFRB35H92.1 size=773458 2024-05-30T16:03:52.344 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.343+0000 7f08851138c0 1 > prepared multipart upload key=QCLY10D2A3VF upload=6E0XFRB35H92 tag=0XAPUHKAISRB type=write size=2870610 2024-05-30T16:03:52.348 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.347+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~I180QICQHOSR.0 size=1048576 2024-05-30T16:03:52.351 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.351+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~I180QICQHOSR.1 size=607970 2024-05-30T16:03:52.357 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.355+0000 7f08851138c0 1 < completed multipart upload key=A5BUBY9Z7JQG upload=I180QICQHOSR tag=PN6O4DVVX1ST type=write size=2705122 2024-05-30T16:03:52.363 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.363+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~Y4H9HPBS1I17.0 size=1048576 2024-05-30T16:03:52.366 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.367+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~Y4H9HPBS1I17.1 size=1048576 2024-05-30T16:03:52.369 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.367+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~Y4H9HPBS1I17.2 size=764100 2024-05-30T16:03:52.369 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.367+0000 7f08851138c0 1 > prepared multipart upload key=0RZRBW38WKNZ upload=Y4H9HPBS1I17 tag=05LZEVSJJVW9 type=write size=3909828 2024-05-30T16:03:52.372 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.371+0000 7f08851138c0 1 < completed delete operation key=V4OQFOOQUXFQ tag=XYR6GXW97Q5G type=del 2024-05-30T16:03:52.376 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.375+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=IGH8F41IGG58 type=del size=2243688 2024-05-30T16:03:52.383 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.383+0000 7f08851138c0 1 < suggested update operation key=O0DJ8SCY766V tag=KHZSQMJ6EP09 type=del size=107855 2024-05-30T16:03:52.387 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.387+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=21XFRWJINPM5 type=del size=2959169 2024-05-30T16:03:52.391 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.391+0000 7f08851138c0 1 < completed delete operation key=KPHZGFJLC2E0 tag=UCC6ES4ZO8B8 type=del 2024-05-30T16:03:52.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.395+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=RQX9PH8D6OMX type=del size=2192503 2024-05-30T16:03:52.398 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.399+0000 7f08851138c0 1 < completed write operation key=XHGMUYBA2VNK tag=I3X6VZVJ4NR8 type=write size=1008725 2024-05-30T16:03:52.402 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.403+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=3XXFJFAAT8IR type=del size=3123500 2024-05-30T16:03:52.406 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.407+0000 7f08851138c0 1 < completed part key=_multipart_O0DJ8SCY766V.2~1KS4GF3EE8ZO.0 size=1048576 2024-05-30T16:03:52.409 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.407+0000 7f08851138c0 1 < completed part key=_multipart_O0DJ8SCY766V.2~1KS4GF3EE8ZO.1 size=1048576 2024-05-30T16:03:52.412 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.411+0000 7f08851138c0 1 < completed part key=_multipart_O0DJ8SCY766V.2~1KS4GF3EE8ZO.2 size=275578 2024-05-30T16:03:52.418 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.419+0000 7f08851138c0 1 < completed multipart upload key=O0DJ8SCY766V upload=1KS4GF3EE8ZO tag=8NZ071NZX0UF type=write size=3421306 2024-05-30T16:03:52.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.423+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=AFHWFSXRJ1CJ type=del size=2923154 2024-05-30T16:03:52.426 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.427+0000 7f08851138c0 1 < completed write operation key=JNZI7JX95LX2 tag=2967QX4BJHJX type=write size=171435 2024-05-30T16:03:52.432 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.431+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~E1F2CERTGRNN.0 size=1048576 2024-05-30T16:03:52.434 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.435+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~E1F2CERTGRNN.1 size=677044 2024-05-30T16:03:52.434 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.435+0000 7f08851138c0 1 > prepared multipart upload key=364U3OV10VPD upload=E1F2CERTGRNN tag=X274F355RAP5 type=write size=2774196 2024-05-30T16:03:52.439 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.439+0000 7f08851138c0 1 < completed delete operation key=MMLF65KLC42Q tag=7A4H2OXLIP1N type=del 2024-05-30T16:03:52.446 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.447+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~MZTUC3W7X7C9.0 size=449860 2024-05-30T16:03:52.446 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.447+0000 7f08851138c0 1 > prepared multipart upload key=3IJWPWRB6FHE upload=MZTUC3W7X7C9 tag=A6YW1CGDV75Z type=write size=1498436 2024-05-30T16:03:52.449 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.451+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~QM3S7FNOWE18.0 size=597360 2024-05-30T16:03:52.455 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.455+0000 7f08851138c0 1 < completed multipart upload key=0RZRBW38WKNZ upload=QM3S7FNOWE18 tag=UCJ6D1Q1UIBD type=write size=1645936 2024-05-30T16:03:52.459 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.459+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=EHHHSCJ27GKQ type=del size=351399 2024-05-30T16:03:52.463 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.463+0000 7f08851138c0 1 < completed delete operation key=Y3CYZJ83WYNI tag=GU62S6ZO4FOQ type=del 2024-05-30T16:03:52.467 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.467+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=L1E0BUQN81M1 type=del size=873588 2024-05-30T16:03:52.472 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.471+0000 7f08851138c0 1 < completed delete operation key=TPCETCO3Y1M6 tag=H8PQ1KZDQYIR type=del 2024-05-30T16:03:52.476 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.475+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=RZL9TR313TKT type=del size=2242048 2024-05-30T16:03:52.481 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.479+0000 7f08851138c0 1 < completed delete operation key=0T2E9F3K1USF tag=2VJDQAK8IA66 type=del 2024-05-30T16:03:52.485 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.483+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=3OJSHAD9UKZT type=del size=3371898 2024-05-30T16:03:52.494 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.495+0000 7f08851138c0 1 < suggested update operation key=QCLY10D2A3VF tag=Z14X6BKTXYKZ type=del size=1077092 2024-05-30T16:03:52.497 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.499+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=1R1X25MMAEVF type=del size=1579886 2024-05-30T16:03:52.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.499+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~4UGV98NMUE2N.0 size=1048576 2024-05-30T16:03:52.503 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.503+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~4UGV98NMUE2N.1 size=1048576 2024-05-30T16:03:52.506 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.507+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~4UGV98NMUE2N.2 size=342874 2024-05-30T16:03:52.512 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.511+0000 7f08851138c0 1 < completed multipart upload key=VRXNO8JLAZYH upload=4UGV98NMUE2N tag=YXLMNN9ZPBV2 type=write size=3488602 2024-05-30T16:03:52.516 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.515+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=0B7C83MKMUC9 type=write size=247648 2024-05-30T16:03:52.519 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.519+0000 7f08851138c0 1 < completed delete operation key=3TBOZWT2P8U8 tag=AM8JT53LSEAD type=del 2024-05-30T16:03:52.525 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.523+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~6Q26UD5EKGEU.0 size=625916 2024-05-30T16:03:52.525 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.523+0000 7f08851138c0 1 > prepared multipart upload key=O0DJ8SCY766V upload=6Q26UD5EKGEU tag=2RMWI03MG15U type=write size=1674492 2024-05-30T16:03:52.528 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.527+0000 7f08851138c0 1 < completed part key=_multipart_QCLY10D2A3VF.2~6E0XFRB35H92.0 size=1048576 2024-05-30T16:03:52.531 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.531+0000 7f08851138c0 1 < completed part key=_multipart_QCLY10D2A3VF.2~6E0XFRB35H92.1 size=773458 2024-05-30T16:03:52.538 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.535+0000 7f08851138c0 1 < completed multipart upload key=QCLY10D2A3VF upload=6E0XFRB35H92 tag=0XAPUHKAISRB type=write size=2870610 2024-05-30T16:03:52.543 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.543+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~D8MTIE1Q0X30.0 size=1048576 2024-05-30T16:03:52.546 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.547+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~D8MTIE1Q0X30.1 size=917598 2024-05-30T16:03:52.546 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.547+0000 7f08851138c0 1 > prepared multipart upload key=KPHZGFJLC2E0 upload=D8MTIE1Q0X30 tag=EESANQWI88UR type=write size=3014750 2024-05-30T16:03:52.550 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.551+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~Y4H9HPBS1I17.0 size=1048576 2024-05-30T16:03:52.552 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.551+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~Y4H9HPBS1I17.1 size=1048576 2024-05-30T16:03:52.556 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.555+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~Y4H9HPBS1I17.2 size=764100 2024-05-30T16:03:52.562 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.563+0000 7f08851138c0 1 < completed multipart upload key=0RZRBW38WKNZ upload=Y4H9HPBS1I17 tag=05LZEVSJJVW9 type=write size=3909828 2024-05-30T16:03:52.567 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.567+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=AT1YCI0F7OGU type=del size=2660839 2024-05-30T16:03:52.571 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.571+0000 7f08851138c0 1 < completed delete operation key=TPCETCO3Y1M6 tag=IGH8F41IGG58 type=del 2024-05-30T16:03:52.578 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.579+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~V64LWIC0N4Z4.0 size=517777 2024-05-30T16:03:52.579 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.579+0000 7f08851138c0 1 > prepared multipart upload key=Y8SBI0QZ0ZV8 upload=V64LWIC0N4Z4 tag=0NK8OGVX3DTM type=write size=1566353 2024-05-30T16:03:52.586 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.587+0000 7f08851138c0 1 < suggested remove operation key=O8CWPJ9GW2GO tag=21XFRWJINPM5 type=del 2024-05-30T16:03:52.594 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.595+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~VSHMZLJUX2EK.0 size=77554 2024-05-30T16:03:52.594 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.595+0000 7f08851138c0 1 > prepared multipart upload key=3TBOZWT2P8U8 upload=VSHMZLJUX2EK tag=XWA53NDBW1GZ type=write size=1126130 2024-05-30T16:03:52.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.599+0000 7f08851138c0 1 < completed delete operation key=WT9RC0UYVG4F tag=RQX9PH8D6OMX type=del 2024-05-30T16:03:52.601 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.599+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=LFK4B1HLC8N3 type=write size=416748 2024-05-30T16:03:52.605 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.603+0000 7f08851138c0 1 < completed delete operation key=0RZRBW38WKNZ tag=3XXFJFAAT8IR type=del 2024-05-30T16:03:52.609 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.607+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=AKA12KSMNPHW type=del size=1599709 2024-05-30T16:03:52.612 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.611+0000 7f08851138c0 1 < completed delete operation key=0T2E9F3K1USF tag=AFHWFSXRJ1CJ type=del 2024-05-30T16:03:52.616 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.615+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=Z54F2NEYKIN1 type=del size=687072 2024-05-30T16:03:52.620 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.619+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~E1F2CERTGRNN.0 size=1048576 2024-05-30T16:03:52.620 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.619+0000 7f08851138c0 1 < canceled part key=_multipart_364U3OV10VPD.2~E1F2CERTGRNN.1 size=677044 2024-05-30T16:03:52.626 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.627+0000 7f08851138c0 1 < canceled multipart upload key=364U3OV10VPD upload=E1F2CERTGRNN tag=X274F355RAP5 type=write size=2774196 2024-05-30T16:03:52.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.631+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=8SW0WQJI2V5M type=del size=2046837 2024-05-30T16:03:52.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.635+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~MZTUC3W7X7C9.0 size=449860 2024-05-30T16:03:52.642 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.643+0000 7f08851138c0 1 < completed multipart upload key=3IJWPWRB6FHE upload=MZTUC3W7X7C9 tag=A6YW1CGDV75Z type=write size=1498436 2024-05-30T16:03:52.646 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.647+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=ZJ50T7L56O8D type=write size=1033042 2024-05-30T16:03:52.650 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.651+0000 7f08851138c0 1 < completed delete operation key=OKBB0VU765TW tag=EHHHSCJ27GKQ type=del 2024-05-30T16:03:52.654 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.655+0000 7f08851138c0 1 > prepared operation key=ZALVY70ISLM5 tag=1V3ZY8BK3LLR type=del size=1359284 2024-05-30T16:03:52.662 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.663+0000 7f08851138c0 1 < suggested update operation key=V4OQFOOQUXFQ tag=L1E0BUQN81M1 type=del size=873588 2024-05-30T16:03:52.666 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.667+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=0VZFC3WMG4IN type=write size=548822 2024-05-30T16:03:52.669 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.671+0000 7f08851138c0 1 < completed delete operation key=OKBB0VU765TW tag=RZL9TR313TKT type=del 2024-05-30T16:03:52.673 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.675+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=G5SGUKWWG4X4 type=del size=824806 2024-05-30T16:03:52.676 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.675+0000 7f08851138c0 1 < completed delete operation key=Y8SBI0QZ0ZV8 tag=3OJSHAD9UKZT type=del 2024-05-30T16:03:52.680 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.679+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=6KQKGDV0BZ3P type=del size=1047537 2024-05-30T16:03:52.683 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.683+0000 7f08851138c0 1 < completed delete operation key=0T2E9F3K1USF tag=1R1X25MMAEVF type=del 2024-05-30T16:03:52.687 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.687+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=79QIVR4MR7PV type=del size=1960814 2024-05-30T16:03:52.690 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.691+0000 7f08851138c0 1 < completed write operation key=603HBR8PRH6L tag=0B7C83MKMUC9 type=write size=247648 2024-05-30T16:03:52.696 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.695+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~DK0EF2HTXO0I.0 size=926952 2024-05-30T16:03:52.696 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.695+0000 7f08851138c0 1 > prepared multipart upload key=QQILXRJD0C1Z upload=DK0EF2HTXO0I tag=QSJLHSSVOP45 type=write size=1975528 2024-05-30T16:03:52.699 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.699+0000 7f08851138c0 1 < completed part key=_multipart_O0DJ8SCY766V.2~6Q26UD5EKGEU.0 size=625916 2024-05-30T16:03:52.704 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.703+0000 7f08851138c0 1 < completed multipart upload key=O0DJ8SCY766V upload=6Q26UD5EKGEU tag=2RMWI03MG15U type=write size=1674492 2024-05-30T16:03:52.707 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.707+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=TUX5FCUTC7CG type=write size=662118 2024-05-30T16:03:52.710 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.711+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~D8MTIE1Q0X30.0 size=1048576 2024-05-30T16:03:52.712 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.711+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~D8MTIE1Q0X30.1 size=917598 2024-05-30T16:03:52.717 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.715+0000 7f08851138c0 1 < completed multipart upload key=KPHZGFJLC2E0 upload=D8MTIE1Q0X30 tag=EESANQWI88UR type=write size=3014750 2024-05-30T16:03:52.720 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.719+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=QL0ES3EIAR1W type=del size=2054016 2024-05-30T16:03:52.723 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.723+0000 7f08851138c0 1 < completed delete operation key=O0DJ8SCY766V tag=AT1YCI0F7OGU type=del 2024-05-30T16:03:52.728 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.727+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~7IG3T0DS89OJ.0 size=1048576 2024-05-30T16:03:52.730 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.731+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~7IG3T0DS89OJ.1 size=619355 2024-05-30T16:03:52.730 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.731+0000 7f08851138c0 1 > prepared multipart upload key=ZALVY70ISLM5 upload=7IG3T0DS89OJ tag=VWE94JA7P1PU type=write size=2716507 2024-05-30T16:03:52.730 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.731+0000 7f08851138c0 1 < canceled part key=_multipart_Y8SBI0QZ0ZV8.2~V64LWIC0N4Z4.0 size=517777 2024-05-30T16:03:52.735 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.735+0000 7f08851138c0 1 < canceled multipart upload key=Y8SBI0QZ0ZV8 upload=V64LWIC0N4Z4 tag=0NK8OGVX3DTM type=write size=1566353 2024-05-30T16:03:52.738 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.739+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=MACTF755EFKS type=del size=1884343 2024-05-30T16:03:52.738 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.739+0000 7f08851138c0 1 < canceled part key=_multipart_3TBOZWT2P8U8.2~VSHMZLJUX2EK.0 size=77554 2024-05-30T16:03:52.743 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.743+0000 7f08851138c0 1 < canceled multipart upload key=3TBOZWT2P8U8 upload=VSHMZLJUX2EK tag=XWA53NDBW1GZ type=write size=1126130 2024-05-30T16:03:52.748 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.747+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~HY5ORA3978BN.0 size=1048576 2024-05-30T16:03:52.750 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.751+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~HY5ORA3978BN.1 size=693894 2024-05-30T16:03:52.750 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.751+0000 7f08851138c0 1 > prepared multipart upload key=KPHZGFJLC2E0 upload=HY5ORA3978BN tag=Q11M4164DVKU type=write size=2791046 2024-05-30T16:03:52.756 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.755+0000 7f08851138c0 1 < suggested update operation key=O8CWPJ9GW2GO tag=LFK4B1HLC8N3 type=write size=416748 2024-05-30T16:03:52.759 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.759+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=YSX7KFDORF3P type=del size=3447533 2024-05-30T16:03:52.762 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.763+0000 7f08851138c0 1 < completed delete operation key=MMLF65KLC42Q tag=AKA12KSMNPHW type=del 2024-05-30T16:03:52.765 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.763+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=4XYE15DBAX7N type=del size=1246351 2024-05-30T16:03:52.768 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.767+0000 7f08851138c0 1 < completed delete operation key=O0DJ8SCY766V tag=Z54F2NEYKIN1 type=del 2024-05-30T16:03:52.770 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.771+0000 7f08851138c0 1 > prepared operation key=VRXNO8JLAZYH tag=WZ07GTZK181J type=del size=3983164 2024-05-30T16:03:52.773 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.775+0000 7f08851138c0 1 < completed delete operation key=OKBB0VU765TW tag=8SW0WQJI2V5M type=del 2024-05-30T16:03:52.778 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.779+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~A5CR97RBL0L9.0 size=1048576 2024-05-30T16:03:52.780 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.779+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~A5CR97RBL0L9.1 size=22448 2024-05-30T16:03:52.780 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.779+0000 7f08851138c0 1 > prepared multipart upload key=V4OQFOOQUXFQ upload=A5CR97RBL0L9 tag=2WIG22TKKJ6Q type=write size=2119600 2024-05-30T16:03:52.783 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.783+0000 7f08851138c0 1 < completed write operation key=GWXNCSJCLXKW tag=ZJ50T7L56O8D type=write size=1033042 2024-05-30T16:03:52.786 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.787+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=CN6PXZGLW499 type=del size=3826716 2024-05-30T16:03:52.788 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.787+0000 7f08851138c0 1 < canceled operation key=ZALVY70ISLM5 tag=1V3ZY8BK3LLR type=del size=1359284 2024-05-30T16:03:52.793 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.795+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~HQSVP5YUSLNC.0 size=980913 2024-05-30T16:03:52.793 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.795+0000 7f08851138c0 1 > prepared multipart upload key=0T2E9F3K1USF upload=HQSVP5YUSLNC tag=11HT73INNFE1 type=write size=2029489 2024-05-30T16:03:52.796 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.795+0000 7f08851138c0 1 < completed write operation key=364U3OV10VPD tag=0VZFC3WMG4IN type=write size=548822 2024-05-30T16:03:52.799 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.799+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=VVK0KKHPL1R8 type=del size=1517089 2024-05-30T16:03:52.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.807+0000 7f08851138c0 1 < suggested remove operation key=1N6R2HBTWMP2 tag=G5SGUKWWG4X4 type=del 2024-05-30T16:03:52.808 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.807+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=JHPVGH8JKE89 type=del size=902719 2024-05-30T16:03:52.814 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.815+0000 7f08851138c0 1 < suggested remove operation key=73FJ3HGWYQQK tag=6KQKGDV0BZ3P type=del 2024-05-30T16:03:52.817 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.815+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=2D2BRP9U0XWA type=del size=720970 2024-05-30T16:03:52.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.819+0000 7f08851138c0 1 < completed delete operation key=A5BUBY9Z7JQG tag=79QIVR4MR7PV type=del 2024-05-30T16:03:52.825 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.823+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~EL7J43WF8PZ2.0 size=1048576 2024-05-30T16:03:52.827 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.827+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~EL7J43WF8PZ2.1 size=592856 2024-05-30T16:03:52.827 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.827+0000 7f08851138c0 1 > prepared multipart upload key=E2FXSXOEUGIG upload=EL7J43WF8PZ2 tag=D34R50OMP08R type=write size=2690008 2024-05-30T16:03:52.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.827+0000 7f08851138c0 1 < canceled part key=_multipart_QQILXRJD0C1Z.2~DK0EF2HTXO0I.0 size=926952 2024-05-30T16:03:52.833 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.831+0000 7f08851138c0 1 < canceled multipart upload key=QQILXRJD0C1Z upload=DK0EF2HTXO0I tag=QSJLHSSVOP45 type=write size=1975528 2024-05-30T16:03:52.838 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.839+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~DXG64DLFISPD.0 size=1048576 2024-05-30T16:03:52.840 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.839+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~DXG64DLFISPD.1 size=1048576 2024-05-30T16:03:52.843 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.843+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~DXG64DLFISPD.2 size=731684 2024-05-30T16:03:52.843 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.843+0000 7f08851138c0 1 > prepared multipart upload key=YRP4CSG92QRO upload=DXG64DLFISPD tag=ZXRXJZWVDRVK type=write size=3877412 2024-05-30T16:03:52.846 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.847+0000 7f08851138c0 1 < completed write operation key=JNZI7JX95LX2 tag=TUX5FCUTC7CG type=write size=662118 2024-05-30T16:03:52.850 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.851+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=IEVH8Z1RJE7V type=del size=739703 2024-05-30T16:03:52.857 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.855+0000 7f08851138c0 1 < suggested remove operation key=3IJWPWRB6FHE tag=QL0ES3EIAR1W type=del 2024-05-30T16:03:52.860 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.859+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=YTPYRM2ZFO4E type=del size=1949013 2024-05-30T16:03:52.866 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.863+0000 7f08851138c0 1 < completed part key=_multipart_ZALVY70ISLM5.2~7IG3T0DS89OJ.0 size=1048576 2024-05-30T16:03:52.866 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.867+0000 7f08851138c0 1 < completed part key=_multipart_ZALVY70ISLM5.2~7IG3T0DS89OJ.1 size=619355 2024-05-30T16:03:52.872 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.871+0000 7f08851138c0 1 < completed multipart upload key=ZALVY70ISLM5 upload=7IG3T0DS89OJ tag=VWE94JA7P1PU type=write size=2716507 2024-05-30T16:03:52.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.879+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~2FXOPSU4NO09.0 size=843302 2024-05-30T16:03:52.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.879+0000 7f08851138c0 1 > prepared multipart upload key=KPHZGFJLC2E0 upload=2FXOPSU4NO09 tag=TWICSD0RBJ86 type=write size=1891878 2024-05-30T16:03:52.885 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.887+0000 7f08851138c0 1 < suggested update operation key=0RZRBW38WKNZ tag=MACTF755EFKS type=del size=1884343 2024-05-30T16:03:52.889 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.891+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=99843DM56L6K type=del size=211516 2024-05-30T16:03:52.893 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.891+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~HY5ORA3978BN.0 size=1048576 2024-05-30T16:03:52.896 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.895+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~HY5ORA3978BN.1 size=693894 2024-05-30T16:03:52.901 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.899+0000 7f08851138c0 1 < completed multipart upload key=KPHZGFJLC2E0 upload=HY5ORA3978BN tag=Q11M4164DVKU type=write size=2791046 2024-05-30T16:03:52.908 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.907+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~7QORTKADNE59.0 size=1048576 2024-05-30T16:03:52.910 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.911+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~7QORTKADNE59.1 size=269217 2024-05-30T16:03:52.910 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.911+0000 7f08851138c0 1 > prepared multipart upload key=YRP4CSG92QRO upload=7QORTKADNE59 tag=VA11O3V3ZWH8 type=write size=2366369 2024-05-30T16:03:52.913 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.911+0000 7f08851138c0 1 < completed delete operation key=OKBB0VU765TW tag=YSX7KFDORF3P type=del 2024-05-30T16:03:52.917 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.915+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=I3UCJEGFLWXP type=write size=701408 2024-05-30T16:03:52.924 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.923+0000 7f08851138c0 1 < suggested update operation key=4N58ZY4E1H6E tag=4XYE15DBAX7N type=del size=1246351 2024-05-30T16:03:52.927 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.927+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=9NB6AFK7V707 type=del size=926117 2024-05-30T16:03:52.932 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.931+0000 7f08851138c0 1 < completed delete operation key=VRXNO8JLAZYH tag=WZ07GTZK181J type=del 2024-05-30T16:03:52.936 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.935+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=QA80SWEB0W5Z type=del size=3126136 2024-05-30T16:03:52.941 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.939+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~A5CR97RBL0L9.0 size=1048576 2024-05-30T16:03:52.944 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.943+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~A5CR97RBL0L9.1 size=22448 2024-05-30T16:03:52.949 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.951+0000 7f08851138c0 1 < completed multipart upload key=V4OQFOOQUXFQ upload=A5CR97RBL0L9 tag=2WIG22TKKJ6Q type=write size=2119600 2024-05-30T16:03:52.953 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.951+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=AFFGTJT39UM9 type=del size=2932940 2024-05-30T16:03:52.956 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.955+0000 7f08851138c0 1 < completed delete operation key=O0DJ8SCY766V tag=CN6PXZGLW499 type=del 2024-05-30T16:03:52.960 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.959+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=IJPUUJ76M79P type=del size=3374933 2024-05-30T16:03:52.963 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.963+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~HQSVP5YUSLNC.0 size=980913 2024-05-30T16:03:52.969 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.967+0000 7f08851138c0 1 < completed multipart upload key=0T2E9F3K1USF upload=HQSVP5YUSLNC tag=11HT73INNFE1 type=write size=2029489 2024-05-30T16:03:52.972 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.971+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=JGZJ2GZWZ090 type=del size=2748664 2024-05-30T16:03:52.976 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.975+0000 7f08851138c0 1 < completed delete operation key=MMLF65KLC42Q tag=VVK0KKHPL1R8 type=del 2024-05-30T16:03:52.980 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.979+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=1OS6OZFO2QLR type=write size=802097 2024-05-30T16:03:52.983 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.983+0000 7f08851138c0 1 < completed delete operation key=O0DJ8SCY766V tag=JHPVGH8JKE89 type=del 2024-05-30T16:03:52.990 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.991+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~YF6FWAV76X59.0 size=1048576 2024-05-30T16:03:52.992 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.991+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~YF6FWAV76X59.1 size=605759 2024-05-30T16:03:52.992 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.991+0000 7f08851138c0 1 > prepared multipart upload key=3TBOZWT2P8U8 upload=YF6FWAV76X59 tag=CVNLC2SO0HJ2 type=write size=2702911 2024-05-30T16:03:52.995 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.995+0000 7f08851138c0 -1 < failed to complete operation key=3IJWPWRB6FHE tag=2D2BRP9U0XWA type=del size=720970: (22) Invalid argument 2024-05-30T16:03:52.999 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.999+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=SETO1X7AX6FA type=del size=675126 2024-05-30T16:03:53.000 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:52.999+0000 7f08851138c0 1 < canceled part key=_multipart_E2FXSXOEUGIG.2~EL7J43WF8PZ2.0 size=1048576 2024-05-30T16:03:53.003 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.003+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~EL7J43WF8PZ2.1 size=592856 2024-05-30T16:03:53.010 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.011+0000 7f08851138c0 1 < canceled multipart upload key=E2FXSXOEUGIG upload=EL7J43WF8PZ2 tag=D34R50OMP08R type=write size=2690008 2024-05-30T16:03:53.016 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.015+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~0Z2MSW8KB91Q.0 size=1048576 2024-05-30T16:03:53.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.019+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~0Z2MSW8KB91Q.1 size=123150 2024-05-30T16:03:53.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.019+0000 7f08851138c0 1 > prepared multipart upload key=VRXNO8JLAZYH upload=0Z2MSW8KB91Q tag=19B5R3IFHLJW type=write size=2220302 2024-05-30T16:03:53.023 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.023+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~DXG64DLFISPD.0 size=1048576 2024-05-30T16:03:53.026 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.027+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~DXG64DLFISPD.1 size=1048576 2024-05-30T16:03:53.029 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.027+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~DXG64DLFISPD.2 size=731684 2024-05-30T16:03:53.035 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.035+0000 7f08851138c0 1 < completed multipart upload key=YRP4CSG92QRO upload=DXG64DLFISPD tag=ZXRXJZWVDRVK type=write size=3877412 2024-05-30T16:03:53.041 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.039+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~G79NPRJO3LIR.0 size=1048576 2024-05-30T16:03:53.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.043+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~G79NPRJO3LIR.1 size=225410 2024-05-30T16:03:53.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.043+0000 7f08851138c0 1 > prepared multipart upload key=JNZI7JX95LX2 upload=G79NPRJO3LIR tag=FTN4Z6NAQLXJ type=write size=2322562 2024-05-30T16:03:53.047 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.047+0000 7f08851138c0 1 < completed delete operation key=WT9RC0UYVG4F tag=IEVH8Z1RJE7V type=del 2024-05-30T16:03:53.050 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.051+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=HIW8Z966SY04 type=write size=688952 2024-05-30T16:03:53.054 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.055+0000 7f08851138c0 1 < completed delete operation key=O8CWPJ9GW2GO tag=YTPYRM2ZFO4E type=del 2024-05-30T16:03:53.060 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.059+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~RSMV76JDYLM8.0 size=37902 2024-05-30T16:03:53.060 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.059+0000 7f08851138c0 1 > prepared multipart upload key=YRP4CSG92QRO upload=RSMV76JDYLM8 tag=PA1LNB7HR6AU type=write size=1086478 2024-05-30T16:03:53.064 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.063+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~2FXOPSU4NO09.0 size=843302 2024-05-30T16:03:53.069 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.071+0000 7f08851138c0 1 < completed multipart upload key=KPHZGFJLC2E0 upload=2FXOPSU4NO09 tag=TWICSD0RBJ86 type=write size=1891878 2024-05-30T16:03:53.075 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.075+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~KV2EAVJZ5K3T.0 size=1048576 2024-05-30T16:03:53.078 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.079+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~KV2EAVJZ5K3T.1 size=1048576 2024-05-30T16:03:53.081 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.079+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~KV2EAVJZ5K3T.2 size=765924 2024-05-30T16:03:53.081 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.079+0000 7f08851138c0 1 > prepared multipart upload key=3IJWPWRB6FHE upload=KV2EAVJZ5K3T tag=CISJ86CCL8OW type=write size=3911652 2024-05-30T16:03:53.084 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.083+0000 7f08851138c0 1 < completed delete operation key=RCTV94WRMO53 tag=99843DM56L6K type=del 2024-05-30T16:03:53.087 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.087+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=NTSHUQPMH5LX type=del size=1348682 2024-05-30T16:03:53.091 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.091+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~7QORTKADNE59.0 size=1048576 2024-05-30T16:03:53.094 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.095+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~7QORTKADNE59.1 size=269217 2024-05-30T16:03:53.100 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.099+0000 7f08851138c0 1 < completed multipart upload key=YRP4CSG92QRO upload=7QORTKADNE59 tag=VA11O3V3ZWH8 type=write size=2366369 2024-05-30T16:03:53.103 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.103+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=6M72QOU2HF2M type=del size=57860 2024-05-30T16:03:53.107 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.107+0000 7f08851138c0 1 < completed write operation key=364U3OV10VPD tag=I3UCJEGFLWXP type=write size=701408 2024-05-30T16:03:53.110 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.111+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=A1T8V2WJRS8K type=write size=773272 2024-05-30T16:03:53.114 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.115+0000 7f08851138c0 1 < completed delete operation key=364U3OV10VPD tag=9NB6AFK7V707 type=del 2024-05-30T16:03:53.120 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.119+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~LAFMLH3AF0VW.0 size=1048576 2024-05-30T16:03:53.123 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.123+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~LAFMLH3AF0VW.1 size=206985 2024-05-30T16:03:53.124 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.123+0000 7f08851138c0 1 > prepared multipart upload key=NIV7AQ4EV5YP upload=LAFMLH3AF0VW tag=WLA2A229DJZ2 type=write size=2304137 2024-05-30T16:03:53.128 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.127+0000 7f08851138c0 1 < completed delete operation key=364U3OV10VPD tag=QA80SWEB0W5Z type=del 2024-05-30T16:03:53.130 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.131+0000 7f08851138c0 1 > prepared operation key=VRXNO8JLAZYH tag=0JL4B03BQBC3 type=del size=2020582 2024-05-30T16:03:53.134 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.135+0000 7f08851138c0 1 < completed delete operation key=YRP4CSG92QRO tag=AFFGTJT39UM9 type=del 2024-05-30T16:03:53.138 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.139+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=ISYTE0FPFEZ4 type=write size=916369 2024-05-30T16:03:53.141 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.139+0000 7f08851138c0 1 < completed delete operation key=Y8SBI0QZ0ZV8 tag=IJPUUJ76M79P type=del 2024-05-30T16:03:53.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.147+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~BSUS532OYXRI.0 size=629127 2024-05-30T16:03:53.150 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.147+0000 7f08851138c0 1 > prepared multipart upload key=364U3OV10VPD upload=BSUS532OYXRI tag=T7BIIPQ0ZG69 type=write size=1677703 2024-05-30T16:03:53.150 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.151+0000 7f08851138c0 1 < completed delete operation key=YRP4CSG92QRO tag=JGZJ2GZWZ090 type=del 2024-05-30T16:03:53.157 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.155+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~U8I2TS4SOSF9.0 size=1048576 2024-05-30T16:03:53.159 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.159+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~U8I2TS4SOSF9.1 size=108781 2024-05-30T16:03:53.159 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.159+0000 7f08851138c0 1 > prepared multipart upload key=3IJWPWRB6FHE upload=U8I2TS4SOSF9 tag=Y7XHVNYK67LG type=write size=2205933 2024-05-30T16:03:53.166 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.167+0000 7f08851138c0 1 < suggested update operation key=603HBR8PRH6L tag=1OS6OZFO2QLR type=write size=802097 2024-05-30T16:03:53.173 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.171+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~U1KQSBFZAGU1.0 size=1048576 2024-05-30T16:03:53.175 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.175+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~U1KQSBFZAGU1.1 size=824182 2024-05-30T16:03:53.175 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.175+0000 7f08851138c0 1 > prepared multipart upload key=QQILXRJD0C1Z upload=U1KQSBFZAGU1 tag=B430LIXVE78U type=write size=2921334 2024-05-30T16:03:53.178 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.179+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~YF6FWAV76X59.0 size=1048576 2024-05-30T16:03:53.178 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.179+0000 7f08851138c0 1 < canceled part key=_multipart_3TBOZWT2P8U8.2~YF6FWAV76X59.1 size=605759 2024-05-30T16:03:53.184 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.183+0000 7f08851138c0 1 < canceled multipart upload key=3TBOZWT2P8U8 upload=YF6FWAV76X59 tag=CVNLC2SO0HJ2 type=write size=2702911 2024-05-30T16:03:53.187 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.187+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=ST0JO49M7339 type=del size=1247750 2024-05-30T16:03:53.191 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.191+0000 7f08851138c0 1 < canceled operation key=JNZI7JX95LX2 tag=SETO1X7AX6FA type=del size=675126 2024-05-30T16:03:53.194 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.195+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=2RTSONKD26GE type=del size=4115960 2024-05-30T16:03:53.198 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.199+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~0Z2MSW8KB91Q.0 size=1048576 2024-05-30T16:03:53.198 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.199+0000 7f08851138c0 1 < canceled part key=_multipart_VRXNO8JLAZYH.2~0Z2MSW8KB91Q.1 size=123150 2024-05-30T16:03:53.205 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.203+0000 7f08851138c0 1 < canceled multipart upload key=VRXNO8JLAZYH upload=0Z2MSW8KB91Q tag=19B5R3IFHLJW type=write size=2220302 2024-05-30T16:03:53.212 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.211+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~12FL5BSB4SX0.0 size=174627 2024-05-30T16:03:53.212 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.211+0000 7f08851138c0 1 > prepared multipart upload key=RCTV94WRMO53 upload=12FL5BSB4SX0 tag=D8ZSBL6ZN7ID type=write size=1223203 2024-05-30T16:03:53.216 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.215+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~G79NPRJO3LIR.0 size=1048576 2024-05-30T16:03:53.219 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.219+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~G79NPRJO3LIR.1 size=225410 2024-05-30T16:03:53.225 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.223+0000 7f08851138c0 1 < completed multipart upload key=JNZI7JX95LX2 upload=G79NPRJO3LIR tag=FTN4Z6NAQLXJ type=write size=2322562 2024-05-30T16:03:53.229 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.227+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=DVY23OHV9SWR type=del size=544312 2024-05-30T16:03:53.232 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.231+0000 7f08851138c0 1 < completed write operation key=NIV7AQ4EV5YP tag=HIW8Z966SY04 type=write size=688952 2024-05-30T16:03:53.239 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.239+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~5VHF6TGHJGN9.0 size=203967 2024-05-30T16:03:53.239 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.239+0000 7f08851138c0 1 > prepared multipart upload key=603HBR8PRH6L upload=5VHF6TGHJGN9 tag=2T9H8S2X1824 type=write size=1252543 2024-05-30T16:03:53.242 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.243+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~RSMV76JDYLM8.0 size=37902 2024-05-30T16:03:53.248 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.247+0000 7f08851138c0 1 < completed multipart upload key=YRP4CSG92QRO upload=RSMV76JDYLM8 tag=PA1LNB7HR6AU type=write size=1086478 2024-05-30T16:03:53.252 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.251+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=9UPWCWW7EL6K type=write size=329156 2024-05-30T16:03:53.256 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.255+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~KV2EAVJZ5K3T.0 size=1048576 2024-05-30T16:03:53.256 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.255+0000 7f08851138c0 1 < canceled part key=_multipart_3IJWPWRB6FHE.2~KV2EAVJZ5K3T.1 size=1048576 2024-05-30T16:03:53.256 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.255+0000 7f08851138c0 1 < canceled part key=_multipart_3IJWPWRB6FHE.2~KV2EAVJZ5K3T.2 size=765924 2024-05-30T16:03:53.262 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.263+0000 7f08851138c0 1 < canceled multipart upload key=3IJWPWRB6FHE upload=KV2EAVJZ5K3T tag=CISJ86CCL8OW type=write size=3911652 2024-05-30T16:03:53.269 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.267+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=V9HE18PH8CYC type=del size=3167207 2024-05-30T16:03:53.272 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.271+0000 7f08851138c0 1 < completed delete operation key=3IJWPWRB6FHE tag=NTSHUQPMH5LX type=del 2024-05-30T16:03:53.276 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.275+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=MPA7U25YHJ1P type=del size=1358498 2024-05-30T16:03:53.279 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.279+0000 7f08851138c0 1 < canceled operation key=PLWFSKPKVO19 tag=6M72QOU2HF2M type=del size=57860 2024-05-30T16:03:53.286 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.287+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~9XXDR490WAJU.0 size=649020 2024-05-30T16:03:53.286 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.287+0000 7f08851138c0 1 > prepared multipart upload key=KPHZGFJLC2E0 upload=9XXDR490WAJU tag=0RZ1SRAU2EKN type=write size=1697596 2024-05-30T16:03:53.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.291+0000 7f08851138c0 1 < suggested update operation key=364U3OV10VPD tag=A1T8V2WJRS8K type=write size=773272 2024-05-30T16:03:53.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.299+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~BPQM3OOHZI9T.0 size=1048576 2024-05-30T16:03:53.303 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.303+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~BPQM3OOHZI9T.1 size=1048576 2024-05-30T16:03:53.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.307+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~BPQM3OOHZI9T.2 size=1008566 2024-05-30T16:03:53.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.307+0000 7f08851138c0 1 > prepared multipart upload key=PLWFSKPKVO19 upload=BPQM3OOHZI9T tag=7PEVRE8U63Z4 type=write size=4154294 2024-05-30T16:03:53.310 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.311+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~LAFMLH3AF0VW.0 size=1048576 2024-05-30T16:03:53.313 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.311+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~LAFMLH3AF0VW.1 size=206985 2024-05-30T16:03:53.319 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.319+0000 7f08851138c0 1 < completed multipart upload key=NIV7AQ4EV5YP upload=LAFMLH3AF0VW tag=WLA2A229DJZ2 type=write size=2304137 2024-05-30T16:03:53.323 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.323+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=BUCG9U3YPE7J type=del size=1742876 2024-05-30T16:03:53.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.331+0000 7f08851138c0 1 < suggested remove operation key=VRXNO8JLAZYH tag=0JL4B03BQBC3 type=del 2024-05-30T16:03:53.334 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.335+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=N02GIRD6KMR7 type=del size=335532 2024-05-30T16:03:53.338 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.339+0000 7f08851138c0 1 < completed write operation key=YRP4CSG92QRO tag=ISYTE0FPFEZ4 type=write size=916369 2024-05-30T16:03:53.346 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.347+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~8VSXAIRN1O2S.0 size=1048576 2024-05-30T16:03:53.349 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.351+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~8VSXAIRN1O2S.1 size=807092 2024-05-30T16:03:53.349 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.351+0000 7f08851138c0 1 > prepared multipart upload key=RCTV94WRMO53 upload=8VSXAIRN1O2S tag=VWILE4H1JEIR type=write size=2904244 2024-05-30T16:03:53.354 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.355+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~BSUS532OYXRI.0 size=629127 2024-05-30T16:03:53.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.359+0000 7f08851138c0 1 < completed multipart upload key=364U3OV10VPD upload=BSUS532OYXRI tag=T7BIIPQ0ZG69 type=write size=1677703 2024-05-30T16:03:53.365 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.363+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=KENR5K60RIPY type=del size=253564 2024-05-30T16:03:53.365 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.367+0000 7f08851138c0 1 < canceled part key=_multipart_3IJWPWRB6FHE.2~U8I2TS4SOSF9.0 size=1048576 2024-05-30T16:03:53.368 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.367+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~U8I2TS4SOSF9.1 size=108781 2024-05-30T16:03:53.374 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.375+0000 7f08851138c0 1 < canceled multipart upload key=3IJWPWRB6FHE upload=U8I2TS4SOSF9 tag=Y7XHVNYK67LG type=write size=2205933 2024-05-30T16:03:53.378 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.379+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=JA6H6Q6Z0SDU type=write size=593020 2024-05-30T16:03:53.381 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.383+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~U1KQSBFZAGU1.0 size=1048576 2024-05-30T16:03:53.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.383+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~U1KQSBFZAGU1.1 size=824182 2024-05-30T16:03:53.390 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.391+0000 7f08851138c0 1 < completed multipart upload key=QQILXRJD0C1Z upload=U1KQSBFZAGU1 tag=B430LIXVE78U type=write size=2921334 2024-05-30T16:03:53.393 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.395+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=R89KBT3BH3TC type=del size=1764033 2024-05-30T16:03:53.397 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.399+0000 7f08851138c0 1 < completed delete operation key=OKBB0VU765TW tag=ST0JO49M7339 type=del 2024-05-30T16:03:53.401 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.399+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=W4AE8R0XRK89 type=del size=2980789 2024-05-30T16:03:53.405 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.403+0000 7f08851138c0 1 < completed delete operation key=JNZI7JX95LX2 tag=2RTSONKD26GE type=del 2024-05-30T16:03:53.409 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.407+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=IDFHU4R2R5K0 type=write size=817007 2024-05-30T16:03:53.412 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.411+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~12FL5BSB4SX0.0 size=174627 2024-05-30T16:03:53.418 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.419+0000 7f08851138c0 1 < completed multipart upload key=RCTV94WRMO53 upload=12FL5BSB4SX0 tag=D8ZSBL6ZN7ID type=write size=1223203 2024-05-30T16:03:53.425 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.423+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~VJDIK62GJYOY.0 size=1048576 2024-05-30T16:03:53.428 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.427+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~VJDIK62GJYOY.1 size=931655 2024-05-30T16:03:53.428 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.427+0000 7f08851138c0 1 > prepared multipart upload key=KPHZGFJLC2E0 upload=VJDIK62GJYOY tag=YYS7C8I2OKK2 type=write size=3028807 2024-05-30T16:03:53.431 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.431+0000 7f08851138c0 1 < canceled operation key=WT9RC0UYVG4F tag=DVY23OHV9SWR type=del size=544312 2024-05-30T16:03:53.437 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.439+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~GI27F87ND1UL.0 size=1048576 2024-05-30T16:03:53.440 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.439+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~GI27F87ND1UL.1 size=1048576 2024-05-30T16:03:53.442 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.443+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~GI27F87ND1UL.2 size=714944 2024-05-30T16:03:53.442 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.443+0000 7f08851138c0 1 > prepared multipart upload key=GWXNCSJCLXKW upload=GI27F87ND1UL tag=0BKX5Z3XH7F9 type=write size=3860672 2024-05-30T16:03:53.446 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.447+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~5VHF6TGHJGN9.0 size=203967 2024-05-30T16:03:53.451 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.451+0000 7f08851138c0 1 < completed multipart upload key=603HBR8PRH6L upload=5VHF6TGHJGN9 tag=2T9H8S2X1824 type=write size=1252543 2024-05-30T16:03:53.458 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.459+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~1I6TM17R6TRA.0 size=1048576 2024-05-30T16:03:53.460 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.459+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~1I6TM17R6TRA.1 size=1048576 2024-05-30T16:03:53.463 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.463+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~1I6TM17R6TRA.2 size=830431 2024-05-30T16:03:53.463 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.463+0000 7f08851138c0 1 > prepared multipart upload key=0RZRBW38WKNZ upload=1I6TM17R6TRA tag=ZGE51CPQ896X type=write size=3976159 2024-05-30T16:03:53.467 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.467+0000 7f08851138c0 1 < completed write operation key=A5BUBY9Z7JQG tag=9UPWCWW7EL6K type=write size=329156 2024-05-30T16:03:53.473 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.475+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~9NHFT41QLT7F.0 size=1048576 2024-05-30T16:03:53.476 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.475+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~9NHFT41QLT7F.1 size=1048576 2024-05-30T16:03:53.480 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.479+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~9NHFT41QLT7F.2 size=252674 2024-05-30T16:03:53.480 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.479+0000 7f08851138c0 1 > prepared multipart upload key=O0DJ8SCY766V upload=9NHFT41QLT7F tag=53FC20HF5SJ8 type=write size=3398402 2024-05-30T16:03:53.485 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.487+0000 7f08851138c0 1 < suggested update operation key=Y8SBI0QZ0ZV8 tag=V9HE18PH8CYC type=del size=3167207 2024-05-30T16:03:53.489 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.487+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=SD2W02FLB3BI type=del size=1393834 2024-05-30T16:03:53.493 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.491+0000 7f08851138c0 1 < completed delete operation key=1N6R2HBTWMP2 tag=MPA7U25YHJ1P type=del 2024-05-30T16:03:53.497 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.495+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=53QVMG22EWJ1 type=del size=3004403 2024-05-30T16:03:53.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.499+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~9XXDR490WAJU.0 size=649020 2024-05-30T16:03:53.506 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.507+0000 7f08851138c0 1 < completed multipart upload key=KPHZGFJLC2E0 upload=9XXDR490WAJU tag=0RZ1SRAU2EKN type=write size=1697596 2024-05-30T16:03:53.510 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.511+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=SBLP0G9QL2D4 type=del size=3303064 2024-05-30T16:03:53.514 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.515+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~BPQM3OOHZI9T.0 size=1048576 2024-05-30T16:03:53.516 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.515+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~BPQM3OOHZI9T.1 size=1048576 2024-05-30T16:03:53.520 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.519+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~BPQM3OOHZI9T.2 size=1008566 2024-05-30T16:03:53.525 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.527+0000 7f08851138c0 1 < completed multipart upload key=PLWFSKPKVO19 upload=BPQM3OOHZI9T tag=7PEVRE8U63Z4 type=write size=4154294 2024-05-30T16:03:53.529 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.527+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=24HT59ERZ7KX type=write size=667559 2024-05-30T16:03:53.533 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.535+0000 7f08851138c0 1 < completed delete operation key=364U3OV10VPD tag=BUCG9U3YPE7J type=del 2024-05-30T16:03:53.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.535+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=NEHSKDAJKZCR type=write size=422125 2024-05-30T16:03:53.543 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.543+0000 7f08851138c0 1 < suggested update operation key=GWXNCSJCLXKW tag=N02GIRD6KMR7 type=del size=335532 2024-05-30T16:03:53.548 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.547+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~7X2WMKQ4DJ43.0 size=444480 2024-05-30T16:03:53.548 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.547+0000 7f08851138c0 1 > prepared multipart upload key=V4OQFOOQUXFQ upload=7X2WMKQ4DJ43 tag=KG96OIWK4ST4 type=write size=1493056 2024-05-30T16:03:53.552 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.551+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~8VSXAIRN1O2S.0 size=1048576 2024-05-30T16:03:53.554 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.555+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~8VSXAIRN1O2S.1 size=807092 2024-05-30T16:03:53.562 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.559+0000 7f08851138c0 1 < completed multipart upload key=RCTV94WRMO53 upload=8VSXAIRN1O2S tag=VWILE4H1JEIR type=write size=2904244 2024-05-30T16:03:53.563 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.563+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=YUR52DL4O1UM type=del size=2009332 2024-05-30T16:03:53.567 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.567+0000 7f08851138c0 1 < completed delete operation key=4N58ZY4E1H6E tag=KENR5K60RIPY type=del 2024-05-30T16:03:53.570 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.571+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=TBYB3LJMWQNQ type=del size=1653020 2024-05-30T16:03:53.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.571+0000 7f08851138c0 -1 < failed to complete operation key=Y8SBI0QZ0ZV8 tag=JA6H6Q6Z0SDU type=write size=593020: (22) Invalid argument 2024-05-30T16:03:53.576 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.575+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=ONDOW1M1ARVN type=del size=1063914 2024-05-30T16:03:53.580 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.579+0000 7f08851138c0 -1 < failed to complete operation key=Y8SBI0QZ0ZV8 tag=R89KBT3BH3TC type=del size=1764033: (22) Invalid argument 2024-05-30T16:03:53.583 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.583+0000 7f08851138c0 1 > prepared operation key=VRXNO8JLAZYH tag=TS2NQGR2AKLX type=write size=123941 2024-05-30T16:03:53.587 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.587+0000 7f08851138c0 1 < completed delete operation key=3IJWPWRB6FHE tag=W4AE8R0XRK89 type=del 2024-05-30T16:03:53.593 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.595+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~F6DB79S64WGK.0 size=1048576 2024-05-30T16:03:53.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.595+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~F6DB79S64WGK.1 size=691017 2024-05-30T16:03:53.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.595+0000 7f08851138c0 1 > prepared multipart upload key=4N58ZY4E1H6E upload=F6DB79S64WGK tag=PHLUV0TM77QY type=write size=2788169 2024-05-30T16:03:53.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.603+0000 7f08851138c0 1 < completed write operation key=O0DJ8SCY766V tag=IDFHU4R2R5K0 type=write size=817007 2024-05-30T16:03:53.606 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.607+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=MWWNMDJPH5V9 type=del size=1198365 2024-05-30T16:03:53.610 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.611+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~VJDIK62GJYOY.0 size=1048576 2024-05-30T16:03:53.613 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.611+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~VJDIK62GJYOY.1 size=931655 2024-05-30T16:03:53.619 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.619+0000 7f08851138c0 1 < completed multipart upload key=KPHZGFJLC2E0 upload=VJDIK62GJYOY tag=YYS7C8I2OKK2 type=write size=3028807 2024-05-30T16:03:53.622 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.623+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=U8RS2DRF62L8 type=del size=1914173 2024-05-30T16:03:53.626 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.627+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~GI27F87ND1UL.0 size=1048576 2024-05-30T16:03:53.626 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.627+0000 7f08851138c0 1 < canceled part key=_multipart_GWXNCSJCLXKW.2~GI27F87ND1UL.1 size=1048576 2024-05-30T16:03:53.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.627+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~GI27F87ND1UL.2 size=714944 2024-05-30T16:03:53.635 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.635+0000 7f08851138c0 1 < canceled multipart upload key=GWXNCSJCLXKW upload=GI27F87ND1UL tag=0BKX5Z3XH7F9 type=write size=3860672 2024-05-30T16:03:53.639 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.639+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=WTR5MKLJXRHW type=del size=2258482 2024-05-30T16:03:53.642 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.643+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~1I6TM17R6TRA.0 size=1048576 2024-05-30T16:03:53.642 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.643+0000 7f08851138c0 1 < canceled part key=_multipart_0RZRBW38WKNZ.2~1I6TM17R6TRA.1 size=1048576 2024-05-30T16:03:53.642 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.643+0000 7f08851138c0 1 < canceled part key=_multipart_0RZRBW38WKNZ.2~1I6TM17R6TRA.2 size=830431 2024-05-30T16:03:53.648 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.647+0000 7f08851138c0 1 < canceled multipart upload key=0RZRBW38WKNZ upload=1I6TM17R6TRA tag=ZGE51CPQ896X type=write size=3976159 2024-05-30T16:03:53.652 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.651+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=2NNU7W7ZB08P type=del size=275921 2024-05-30T16:03:53.656 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.655+0000 7f08851138c0 1 < completed part key=_multipart_O0DJ8SCY766V.2~9NHFT41QLT7F.0 size=1048576 2024-05-30T16:03:53.659 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.659+0000 7f08851138c0 1 < completed part key=_multipart_O0DJ8SCY766V.2~9NHFT41QLT7F.1 size=1048576 2024-05-30T16:03:53.662 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.663+0000 7f08851138c0 1 < completed part key=_multipart_O0DJ8SCY766V.2~9NHFT41QLT7F.2 size=252674 2024-05-30T16:03:53.668 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.667+0000 7f08851138c0 1 < completed multipart upload key=O0DJ8SCY766V upload=9NHFT41QLT7F tag=53FC20HF5SJ8 type=write size=3398402 2024-05-30T16:03:53.672 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.671+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=ZHQMLOF94OZK type=del size=2047151 2024-05-30T16:03:53.679 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.679+0000 7f08851138c0 1 < suggested update operation key=603HBR8PRH6L tag=SD2W02FLB3BI type=del size=1393834 2024-05-30T16:03:53.686 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.687+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~CY9SGN54R7ME.0 size=1048576 2024-05-30T16:03:53.689 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.687+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~CY9SGN54R7ME.1 size=997785 2024-05-30T16:03:53.689 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.687+0000 7f08851138c0 1 > prepared multipart upload key=NIV7AQ4EV5YP upload=CY9SGN54R7ME tag=4DOJ0QUOQQ23 type=write size=3094937 2024-05-30T16:03:53.693 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.691+0000 7f08851138c0 1 < completed delete operation key=73FJ3HGWYQQK tag=53QVMG22EWJ1 type=del 2024-05-30T16:03:53.696 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.695+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=VKFPH117H9H6 type=del size=2827508 2024-05-30T16:03:53.700 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.699+0000 7f08851138c0 1 < completed delete operation key=3IJWPWRB6FHE tag=SBLP0G9QL2D4 type=del 2024-05-30T16:03:53.706 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.707+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~I334OV7WOF0H.0 size=263349 2024-05-30T16:03:53.706 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.707+0000 7f08851138c0 1 > prepared multipart upload key=603HBR8PRH6L upload=I334OV7WOF0H tag=FZQTVXSKW84K type=write size=1311925 2024-05-30T16:03:53.710 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.711+0000 7f08851138c0 1 < completed write operation key=WT9RC0UYVG4F tag=24HT59ERZ7KX type=write size=667559 2024-05-30T16:03:53.713 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.711+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=2IE43TGN3R3N type=del size=2608009 2024-05-30T16:03:53.719 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.719+0000 7f08851138c0 1 < suggested update operation key=XHGMUYBA2VNK tag=NEHSKDAJKZCR type=write size=422125 2024-05-30T16:03:53.723 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.723+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=I5RRJXT6MRYM type=del size=3817757 2024-05-30T16:03:53.726 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.727+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~7X2WMKQ4DJ43.0 size=444480 2024-05-30T16:03:53.734 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.735+0000 7f08851138c0 1 < completed multipart upload key=V4OQFOOQUXFQ upload=7X2WMKQ4DJ43 tag=KG96OIWK4ST4 type=write size=1493056 2024-05-30T16:03:53.737 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.735+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=3D0PQ3A3HW44 type=del size=3374730 2024-05-30T16:03:53.740 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.739+0000 7f08851138c0 1 < completed delete operation key=NIV7AQ4EV5YP tag=YUR52DL4O1UM type=del 2024-05-30T16:03:53.743 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.743+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=0Q314V8J05ZY type=del size=2734064 2024-05-30T16:03:53.750 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.751+0000 7f08851138c0 1 < suggested remove operation key=3TBOZWT2P8U8 tag=TBYB3LJMWQNQ type=del 2024-05-30T16:03:53.756 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.755+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~KE1VPTBQABMM.0 size=1048576 2024-05-30T16:03:53.759 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.759+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~KE1VPTBQABMM.1 size=26076 2024-05-30T16:03:53.759 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.759+0000 7f08851138c0 1 > prepared multipart upload key=E2FXSXOEUGIG upload=KE1VPTBQABMM tag=339XZ0KZMWN1 type=write size=2123228 2024-05-30T16:03:53.766 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.763+0000 7f08851138c0 -1 < failed to complete operation key=3TBOZWT2P8U8 tag=ONDOW1M1ARVN type=del size=1063914: (22) Invalid argument 2024-05-30T16:03:53.766 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.767+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=SXWZZ09Q3YRB type=write size=409190 2024-05-30T16:03:53.770 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.771+0000 7f08851138c0 1 < completed write operation key=VRXNO8JLAZYH tag=TS2NQGR2AKLX type=write size=123941 2024-05-30T16:03:53.776 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.775+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~RUNR0AS92CKQ.0 size=1048576 2024-05-30T16:03:53.778 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.779+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~RUNR0AS92CKQ.1 size=744623 2024-05-30T16:03:53.778 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.779+0000 7f08851138c0 1 > prepared multipart upload key=VRXNO8JLAZYH upload=RUNR0AS92CKQ tag=5CJJ0AQR8SF4 type=write size=2841775 2024-05-30T16:03:53.779 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.779+0000 7f08851138c0 1 < canceled part key=_multipart_4N58ZY4E1H6E.2~F6DB79S64WGK.0 size=1048576 2024-05-30T16:03:53.782 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.783+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~F6DB79S64WGK.1 size=691017 2024-05-30T16:03:53.787 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.787+0000 7f08851138c0 1 < canceled multipart upload key=4N58ZY4E1H6E upload=F6DB79S64WGK tag=PHLUV0TM77QY type=write size=2788169 2024-05-30T16:03:53.791 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.791+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=1GYOB8R2EN7J type=del size=1143516 2024-05-30T16:03:53.797 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.799+0000 7f08851138c0 1 < suggested remove operation key=0RZRBW38WKNZ tag=MWWNMDJPH5V9 type=del 2024-05-30T16:03:53.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.803+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~5TF9UZZWJ0JR.0 size=1048576 2024-05-30T16:03:53.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.807+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~5TF9UZZWJ0JR.1 size=1048576 2024-05-30T16:03:53.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.807+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~5TF9UZZWJ0JR.2 size=928809 2024-05-30T16:03:53.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.807+0000 7f08851138c0 1 > prepared multipart upload key=XHGMUYBA2VNK upload=5TF9UZZWJ0JR tag=FW7OKM5AIPRX type=write size=4074537 2024-05-30T16:03:53.816 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.815+0000 7f08851138c0 1 < suggested update operation key=603HBR8PRH6L tag=U8RS2DRF62L8 type=del size=1914173 2024-05-30T16:03:53.819 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.819+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=TJR87MULS60O type=del size=176917 2024-05-30T16:03:53.823 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.823+0000 7f08851138c0 1 < completed delete operation key=PLWFSKPKVO19 tag=WTR5MKLJXRHW type=del 2024-05-30T16:03:53.827 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.827+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=YDCIB8659EOC type=del size=792060 2024-05-30T16:03:53.831 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.831+0000 7f08851138c0 1 < completed delete operation key=1N6R2HBTWMP2 tag=2NNU7W7ZB08P type=del 2024-05-30T16:03:53.834 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.835+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=25NISNSVQSLJ type=del size=1222660 2024-05-30T16:03:53.838 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.839+0000 7f08851138c0 1 < completed delete operation key=3IJWPWRB6FHE tag=ZHQMLOF94OZK type=del 2024-05-30T16:03:53.842 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.843+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=KHC6KIN67309 type=del size=1984204 2024-05-30T16:03:53.846 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.847+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~CY9SGN54R7ME.0 size=1048576 2024-05-30T16:03:53.848 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.847+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~CY9SGN54R7ME.1 size=997785 2024-05-30T16:03:53.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.855+0000 7f08851138c0 1 < completed multipart upload key=NIV7AQ4EV5YP upload=CY9SGN54R7ME tag=4DOJ0QUOQQ23 type=write size=3094937 2024-05-30T16:03:53.862 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.863+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~LLOOBABSA6AL.0 size=811444 2024-05-30T16:03:53.862 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.863+0000 7f08851138c0 1 > prepared multipart upload key=0T2E9F3K1USF upload=LLOOBABSA6AL tag=BLMA3P1UE8RI type=write size=1860020 2024-05-30T16:03:53.865 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.863+0000 7f08851138c0 1 < completed delete operation key=364U3OV10VPD tag=VKFPH117H9H6 type=del 2024-05-30T16:03:53.872 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.871+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~EIZKA9GJWM3Y.0 size=1048576 2024-05-30T16:03:53.874 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.875+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~EIZKA9GJWM3Y.1 size=1048576 2024-05-30T16:03:53.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.875+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~EIZKA9GJWM3Y.2 size=555604 2024-05-30T16:03:53.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.875+0000 7f08851138c0 1 > prepared multipart upload key=Y3CYZJ83WYNI upload=EIZKA9GJWM3Y tag=K9BQ41UZ2AM1 type=write size=3701332 2024-05-30T16:03:53.880 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.879+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~I334OV7WOF0H.0 size=263349 2024-05-30T16:03:53.886 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.887+0000 7f08851138c0 1 < completed multipart upload key=603HBR8PRH6L upload=I334OV7WOF0H tag=FZQTVXSKW84K type=write size=1311925 2024-05-30T16:03:53.890 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.891+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=9CNGA6T5RVEC type=del size=2399901 2024-05-30T16:03:53.894 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.895+0000 7f08851138c0 1 < completed delete operation key=E2FXSXOEUGIG tag=2IE43TGN3R3N type=del 2024-05-30T16:03:53.901 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.903+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~PFXNWMITZSXG.0 size=1048576 2024-05-30T16:03:53.904 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.903+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~PFXNWMITZSXG.1 size=124636 2024-05-30T16:03:53.904 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.903+0000 7f08851138c0 1 > prepared multipart upload key=4N58ZY4E1H6E upload=PFXNWMITZSXG tag=RO6VUGKHIE1R type=write size=2221788 2024-05-30T16:03:53.908 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.907+0000 7f08851138c0 1 < completed delete operation key=O0DJ8SCY766V tag=I5RRJXT6MRYM type=del 2024-05-30T16:03:53.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.911+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=PDTURPLXQJ4M type=del size=614475 2024-05-30T16:03:53.914 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.915+0000 7f08851138c0 1 < completed delete operation key=73FJ3HGWYQQK tag=3D0PQ3A3HW44 type=del 2024-05-30T16:03:53.919 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.919+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=ONF6ZWTS7X5M type=write size=462352 2024-05-30T16:03:53.922 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.923+0000 7f08851138c0 1 < canceled operation key=364U3OV10VPD tag=0Q314V8J05ZY type=del size=2734064 2024-05-30T16:03:53.926 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.927+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=C8X0JN39B8RM type=del size=1333751 2024-05-30T16:03:53.926 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.927+0000 7f08851138c0 1 < canceled part key=_multipart_E2FXSXOEUGIG.2~KE1VPTBQABMM.0 size=1048576 2024-05-30T16:03:53.929 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.927+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~KE1VPTBQABMM.1 size=26076 2024-05-30T16:03:53.938 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.939+0000 7f08851138c0 1 < canceled multipart upload key=E2FXSXOEUGIG upload=KE1VPTBQABMM tag=339XZ0KZMWN1 type=write size=2123228 2024-05-30T16:03:53.942 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.943+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=N4XZRQ28G38F type=write size=278790 2024-05-30T16:03:53.946 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.947+0000 7f08851138c0 1 < completed write operation key=Y8SBI0QZ0ZV8 tag=SXWZZ09Q3YRB type=write size=409190 2024-05-30T16:03:53.951 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.951+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=3RI2VPJQOUD2 type=del size=3263737 2024-05-30T16:03:53.954 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.955+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~RUNR0AS92CKQ.0 size=1048576 2024-05-30T16:03:53.957 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.955+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~RUNR0AS92CKQ.1 size=744623 2024-05-30T16:03:53.963 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.963+0000 7f08851138c0 1 < completed multipart upload key=VRXNO8JLAZYH upload=RUNR0AS92CKQ tag=5CJJ0AQR8SF4 type=write size=2841775 2024-05-30T16:03:53.969 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.967+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=PGK3ID1G96HG type=write size=938028 2024-05-30T16:03:53.972 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.971+0000 7f08851138c0 1 < completed delete operation key=O0DJ8SCY766V tag=1GYOB8R2EN7J type=del 2024-05-30T16:03:53.977 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.975+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=P44V5M480CCO type=del size=2514595 2024-05-30T16:03:53.981 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.979+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~5TF9UZZWJ0JR.0 size=1048576 2024-05-30T16:03:53.981 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.979+0000 7f08851138c0 1 < canceled part key=_multipart_XHGMUYBA2VNK.2~5TF9UZZWJ0JR.1 size=1048576 2024-05-30T16:03:53.984 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.983+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~5TF9UZZWJ0JR.2 size=928809 2024-05-30T16:03:53.991 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.991+0000 7f08851138c0 1 < canceled multipart upload key=XHGMUYBA2VNK upload=5TF9UZZWJ0JR tag=FW7OKM5AIPRX type=write size=4074537 2024-05-30T16:03:53.997 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.999+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~GIZBN1E3Q78A.0 size=1048576 2024-05-30T16:03:54.000 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:53.999+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~GIZBN1E3Q78A.1 size=1048576 2024-05-30T16:03:54.003 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.003+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~GIZBN1E3Q78A.2 size=770989 2024-05-30T16:03:54.003 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.003+0000 7f08851138c0 1 > prepared multipart upload key=XHGMUYBA2VNK upload=GIZBN1E3Q78A tag=IF8JT5B1T9GF type=write size=3916717 2024-05-30T16:03:54.006 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.007+0000 7f08851138c0 1 < canceled operation key=NIV7AQ4EV5YP tag=TJR87MULS60O type=del size=176917 2024-05-30T16:03:54.009 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.011+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=N5BCP8MAUF4C type=write size=595865 2024-05-30T16:03:54.013 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.015+0000 7f08851138c0 1 < canceled operation key=O0DJ8SCY766V tag=YDCIB8659EOC type=del size=792060 2024-05-30T16:03:54.016 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.015+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=B8XEED8NN7GO type=del size=2066861 2024-05-30T16:03:54.019 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.019+0000 7f08851138c0 1 < completed delete operation key=QCLY10D2A3VF tag=25NISNSVQSLJ type=del 2024-05-30T16:03:54.023 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.023+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=H8VL5DWLYNAS type=del size=3532222 2024-05-30T16:03:54.026 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.027+0000 7f08851138c0 1 < completed delete operation key=0T2E9F3K1USF tag=KHC6KIN67309 type=del 2024-05-30T16:03:54.029 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.031+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=SR17X2Y1BW4L type=del size=280822 2024-05-30T16:03:54.030 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.031+0000 7f08851138c0 1 < canceled part key=_multipart_0T2E9F3K1USF.2~LLOOBABSA6AL.0 size=811444 2024-05-30T16:03:54.036 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.035+0000 7f08851138c0 1 < canceled multipart upload key=0T2E9F3K1USF upload=LLOOBABSA6AL tag=BLMA3P1UE8RI type=write size=1860020 2024-05-30T16:03:54.039 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.039+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=976XR1HLBO1T type=del size=1194169 2024-05-30T16:03:54.043 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.043+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~EIZKA9GJWM3Y.0 size=1048576 2024-05-30T16:03:54.045 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.047+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~EIZKA9GJWM3Y.1 size=1048576 2024-05-30T16:03:54.048 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.047+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~EIZKA9GJWM3Y.2 size=555604 2024-05-30T16:03:54.055 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.055+0000 7f08851138c0 1 < completed multipart upload key=Y3CYZJ83WYNI upload=EIZKA9GJWM3Y tag=K9BQ41UZ2AM1 type=write size=3701332 2024-05-30T16:03:54.058 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.059+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=Q766YXQEHQVW type=write size=309721 2024-05-30T16:03:54.063 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.063+0000 7f08851138c0 1 < completed delete operation key=A5BUBY9Z7JQG tag=9CNGA6T5RVEC type=del 2024-05-30T16:03:54.067 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.067+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=ST9K59OZJXI7 type=del size=1997098 2024-05-30T16:03:54.070 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.071+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~PFXNWMITZSXG.0 size=1048576 2024-05-30T16:03:54.073 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.075+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~PFXNWMITZSXG.1 size=124636 2024-05-30T16:03:54.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.079+0000 7f08851138c0 1 < completed multipart upload key=4N58ZY4E1H6E upload=PFXNWMITZSXG tag=RO6VUGKHIE1R type=write size=2221788 2024-05-30T16:03:54.085 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.087+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~UL1AEBY7Z2ZS.0 size=748618 2024-05-30T16:03:54.085 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.087+0000 7f08851138c0 1 > prepared multipart upload key=Y8SBI0QZ0ZV8 upload=UL1AEBY7Z2ZS tag=85H62CG4RHXZ type=write size=1797194 2024-05-30T16:03:54.089 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.087+0000 7f08851138c0 1 < completed delete operation key=JNZI7JX95LX2 tag=PDTURPLXQJ4M type=del 2024-05-30T16:03:54.092 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.091+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=O7YOT276YVBB type=del size=2565914 2024-05-30T16:03:54.096 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.095+0000 7f08851138c0 1 < completed write operation key=O8CWPJ9GW2GO tag=ONF6ZWTS7X5M type=write size=462352 2024-05-30T16:03:54.102 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.103+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~A4TA963M4KUM.0 size=960781 2024-05-30T16:03:54.102 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.103+0000 7f08851138c0 1 > prepared multipart upload key=VRXNO8JLAZYH upload=A4TA963M4KUM tag=G8KV4O1VFSH9 type=write size=2009357 2024-05-30T16:03:54.105 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.107+0000 7f08851138c0 1 < completed delete operation key=WT9RC0UYVG4F tag=C8X0JN39B8RM type=del 2024-05-30T16:03:54.109 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.107+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=4AI403Q1WU9R type=del size=1526053 2024-05-30T16:03:54.116 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.115+0000 7f08851138c0 1 < suggested update operation key=0RZRBW38WKNZ tag=N4XZRQ28G38F type=write size=278790 2024-05-30T16:03:54.123 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.123+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~ERSO83GB8UDS.0 size=1048576 2024-05-30T16:03:54.126 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.127+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~ERSO83GB8UDS.1 size=627543 2024-05-30T16:03:54.126 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.127+0000 7f08851138c0 1 > prepared multipart upload key=PLWFSKPKVO19 upload=ERSO83GB8UDS tag=RAXF3NRWLL1X type=write size=2724695 2024-05-30T16:03:54.129 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.131+0000 7f08851138c0 1 < completed delete operation key=WT9RC0UYVG4F tag=3RI2VPJQOUD2 type=del 2024-05-30T16:03:54.133 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.131+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=1B9MDJLTSZZ1 type=del size=1859848 2024-05-30T16:03:54.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.135+0000 7f08851138c0 1 < canceled operation key=4N58ZY4E1H6E tag=PGK3ID1G96HG type=write size=938028 2024-05-30T16:03:54.144 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.143+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~3S1ICRSLKE0R.0 size=1048576 2024-05-30T16:03:54.146 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.147+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~3S1ICRSLKE0R.1 size=624927 2024-05-30T16:03:54.146 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.147+0000 7f08851138c0 1 > prepared multipart upload key=0RZRBW38WKNZ upload=3S1ICRSLKE0R tag=OGM1IUX97YIZ type=write size=2722079 2024-05-30T16:03:54.150 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.151+0000 7f08851138c0 1 < completed delete operation key=GWXNCSJCLXKW tag=P44V5M480CCO type=del 2024-05-30T16:03:54.156 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.155+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~G8US8MT5ZTS5.0 size=904505 2024-05-30T16:03:54.156 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.155+0000 7f08851138c0 1 > prepared multipart upload key=73FJ3HGWYQQK upload=G8US8MT5ZTS5 tag=S0RS3NS6ICQW type=write size=1953081 2024-05-30T16:03:54.160 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.159+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~GIZBN1E3Q78A.0 size=1048576 2024-05-30T16:03:54.162 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.163+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~GIZBN1E3Q78A.1 size=1048576 2024-05-30T16:03:54.165 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.163+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~GIZBN1E3Q78A.2 size=770989 2024-05-30T16:03:54.171 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.171+0000 7f08851138c0 1 < completed multipart upload key=XHGMUYBA2VNK upload=GIZBN1E3Q78A tag=IF8JT5B1T9GF type=write size=3916717 2024-05-30T16:03:54.178 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.179+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~0V4SRPL3O0GU.0 size=1048576 2024-05-30T16:03:54.180 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.179+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~0V4SRPL3O0GU.1 size=1048576 2024-05-30T16:03:54.183 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.183+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~0V4SRPL3O0GU.2 size=327556 2024-05-30T16:03:54.183 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.183+0000 7f08851138c0 1 > prepared multipart upload key=4N58ZY4E1H6E upload=0V4SRPL3O0GU tag=KZMPYM97GM4Q type=write size=3473284 2024-05-30T16:03:54.186 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.187+0000 7f08851138c0 1 < completed write operation key=TPCETCO3Y1M6 tag=N5BCP8MAUF4C type=write size=595865 2024-05-30T16:03:54.189 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.191+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=0CZ610EPLJK9 type=del size=3750976 2024-05-30T16:03:54.196 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.195+0000 7f08851138c0 1 < suggested update operation key=WT9RC0UYVG4F tag=B8XEED8NN7GO type=del size=2066861 2024-05-30T16:03:54.200 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.199+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=RPA57OJ6OHAX type=del size=1383397 2024-05-30T16:03:54.204 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.203+0000 7f08851138c0 1 < completed delete operation key=Y3CYZJ83WYNI tag=H8VL5DWLYNAS type=del 2024-05-30T16:03:54.209 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.211+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~3XQFOOFZV0P9.0 size=1048576 2024-05-30T16:03:54.212 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.211+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~3XQFOOFZV0P9.1 size=1048576 2024-05-30T16:03:54.214 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.215+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~3XQFOOFZV0P9.2 size=1041084 2024-05-30T16:03:54.215 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.215+0000 7f08851138c0 1 > prepared multipart upload key=TPCETCO3Y1M6 upload=3XQFOOFZV0P9 tag=WCRD6QQ17323 type=write size=4186812 2024-05-30T16:03:54.220 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.219+0000 7f08851138c0 1 < suggested remove operation key=QQILXRJD0C1Z tag=SR17X2Y1BW4L type=del 2024-05-30T16:03:54.226 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.227+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~WVWORZUHYA84.0 size=941336 2024-05-30T16:03:54.227 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.227+0000 7f08851138c0 1 > prepared multipart upload key=GWXNCSJCLXKW upload=WVWORZUHYA84 tag=XFU02R0S7TA5 type=write size=1989912 2024-05-30T16:03:54.233 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.231+0000 7f08851138c0 1 < suggested remove operation key=73FJ3HGWYQQK tag=976XR1HLBO1T type=del 2024-05-30T16:03:54.236 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.235+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=KTD8F2ELN7BX type=del size=935888 2024-05-30T16:03:54.240 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.239+0000 7f08851138c0 1 < completed write operation key=RCTV94WRMO53 tag=Q766YXQEHQVW type=write size=309721 2024-05-30T16:03:54.243 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.243+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=THTNC9QM0DI9 type=del size=1877143 2024-05-30T16:03:54.251 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.251+0000 7f08851138c0 1 < suggested remove operation key=3IJWPWRB6FHE tag=ST9K59OZJXI7 type=del 2024-05-30T16:03:54.258 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.259+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~S33391EZX0ZB.0 size=914468 2024-05-30T16:03:54.258 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.259+0000 7f08851138c0 1 > prepared multipart upload key=XHGMUYBA2VNK upload=S33391EZX0ZB tag=L6LH1J0AQ40I type=write size=1963044 2024-05-30T16:03:54.261 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.263+0000 7f08851138c0 1 < completed part key=_multipart_Y8SBI0QZ0ZV8.2~UL1AEBY7Z2ZS.0 size=748618 2024-05-30T16:03:54.267 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.267+0000 7f08851138c0 1 < completed multipart upload key=Y8SBI0QZ0ZV8 upload=UL1AEBY7Z2ZS tag=85H62CG4RHXZ type=write size=1797194 2024-05-30T16:03:54.276 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.275+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~JNIVGCN2BOWD.0 size=1043811 2024-05-30T16:03:54.276 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.275+0000 7f08851138c0 1 > prepared multipart upload key=0RZRBW38WKNZ upload=JNIVGCN2BOWD tag=BI5OPPHC53T1 type=write size=2092387 2024-05-30T16:03:54.279 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.279+0000 7f08851138c0 -1 < failed to complete operation key=73FJ3HGWYQQK tag=O7YOT276YVBB type=del size=2565914: (22) Invalid argument 2024-05-30T16:03:54.282 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.283+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=AOWG37QIYJHO type=del size=2827100 2024-05-30T16:03:54.286 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.287+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~A4TA963M4KUM.0 size=960781 2024-05-30T16:03:54.291 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.291+0000 7f08851138c0 1 < completed multipart upload key=VRXNO8JLAZYH upload=A4TA963M4KUM tag=G8KV4O1VFSH9 type=write size=2009357 2024-05-30T16:03:54.294 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.295+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=HL95WLWSZVNC type=del size=3667880 2024-05-30T16:03:54.298 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.299+0000 7f08851138c0 1 < completed delete operation key=RCTV94WRMO53 tag=4AI403Q1WU9R type=del 2024-05-30T16:03:54.301 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.299+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=7VRW681C21TS type=del size=590032 2024-05-30T16:03:54.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.303+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~ERSO83GB8UDS.0 size=1048576 2024-05-30T16:03:54.307 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.307+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~ERSO83GB8UDS.1 size=627543 2024-05-30T16:03:54.312 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.311+0000 7f08851138c0 1 < completed multipart upload key=PLWFSKPKVO19 upload=ERSO83GB8UDS tag=RAXF3NRWLL1X type=write size=2724695 2024-05-30T16:03:54.316 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.315+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=RQ8JRHNG0F8E type=write size=392445 2024-05-30T16:03:54.319 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.319+0000 7f08851138c0 1 < completed delete operation key=O8CWPJ9GW2GO tag=1B9MDJLTSZZ1 type=del 2024-05-30T16:03:54.325 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.327+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~16J9CFWRPSD2.0 size=1048576 2024-05-30T16:03:54.328 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.327+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~16J9CFWRPSD2.1 size=1048576 2024-05-30T16:03:54.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.331+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~16J9CFWRPSD2.2 size=682081 2024-05-30T16:03:54.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.331+0000 7f08851138c0 1 > prepared multipart upload key=NIV7AQ4EV5YP upload=16J9CFWRPSD2 tag=A51QAGDGGXTT type=write size=3827809 2024-05-30T16:03:54.333 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.335+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~3S1ICRSLKE0R.0 size=1048576 2024-05-30T16:03:54.336 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.335+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~3S1ICRSLKE0R.1 size=624927 2024-05-30T16:03:54.341 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.339+0000 7f08851138c0 1 < completed multipart upload key=0RZRBW38WKNZ upload=3S1ICRSLKE0R tag=OGM1IUX97YIZ type=write size=2722079 2024-05-30T16:03:54.344 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.343+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=127GG3VKGHBA type=del size=564796 2024-05-30T16:03:54.347 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.347+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~G8US8MT5ZTS5.0 size=904505 2024-05-30T16:03:54.353 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.351+0000 7f08851138c0 1 < completed multipart upload key=73FJ3HGWYQQK upload=G8US8MT5ZTS5 tag=S0RS3NS6ICQW type=write size=1953081 2024-05-30T16:03:54.356 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.355+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=UZIG28LQX5EX type=write size=811971 2024-05-30T16:03:54.359 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.359+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~0V4SRPL3O0GU.0 size=1048576 2024-05-30T16:03:54.361 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.363+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~0V4SRPL3O0GU.1 size=1048576 2024-05-30T16:03:54.364 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.363+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~0V4SRPL3O0GU.2 size=327556 2024-05-30T16:03:54.370 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.371+0000 7f08851138c0 1 < completed multipart upload key=4N58ZY4E1H6E upload=0V4SRPL3O0GU tag=KZMPYM97GM4Q type=write size=3473284 2024-05-30T16:03:54.373 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.371+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=MLDG66WU8EVU type=write size=132589 2024-05-30T16:03:54.382 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.379+0000 7f08851138c0 1 < suggested remove operation key=4N58ZY4E1H6E tag=0CZ610EPLJK9 type=del 2024-05-30T16:03:54.383 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.383+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=N45B7KE2TIK1 type=del size=3682818 2024-05-30T16:03:54.386 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.387+0000 7f08851138c0 1 < completed delete operation key=JNZI7JX95LX2 tag=RPA57OJ6OHAX type=del 2024-05-30T16:03:54.389 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.387+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=VQRM7GK52WBG type=del size=1823227 2024-05-30T16:03:54.392 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.391+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~3XQFOOFZV0P9.0 size=1048576 2024-05-30T16:03:54.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.395+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~3XQFOOFZV0P9.1 size=1048576 2024-05-30T16:03:54.397 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.399+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~3XQFOOFZV0P9.2 size=1041084 2024-05-30T16:03:54.403 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.403+0000 7f08851138c0 1 < completed multipart upload key=TPCETCO3Y1M6 upload=3XQFOOFZV0P9 tag=WCRD6QQ17323 type=write size=4186812 2024-05-30T16:03:54.406 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.407+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=JJVHD7ETWR8T type=del size=1520807 2024-05-30T16:03:54.409 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.407+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~WVWORZUHYA84.0 size=941336 2024-05-30T16:03:54.414 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.415+0000 7f08851138c0 1 < completed multipart upload key=GWXNCSJCLXKW upload=WVWORZUHYA84 tag=XFU02R0S7TA5 type=write size=1989912 2024-05-30T16:03:54.417 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.419+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=MI0KECHKCDM1 type=del size=3821641 2024-05-30T16:03:54.420 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.419+0000 7f08851138c0 -1 < failed to complete operation key=3IJWPWRB6FHE tag=KTD8F2ELN7BX type=del size=935888: (22) Invalid argument 2024-05-30T16:03:54.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.423+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=9YWMHIASNHNH type=del size=2168380 2024-05-30T16:03:54.427 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.427+0000 7f08851138c0 1 < completed delete operation key=QQILXRJD0C1Z tag=THTNC9QM0DI9 type=del 2024-05-30T16:03:54.432 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.431+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~N9VIBJG835WO.0 size=703167 2024-05-30T16:03:54.432 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.431+0000 7f08851138c0 1 > prepared multipart upload key=O0DJ8SCY766V upload=N9VIBJG835WO tag=03WRQWGD74JG type=write size=1751743 2024-05-30T16:03:54.435 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.435+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~S33391EZX0ZB.0 size=914468 2024-05-30T16:03:54.440 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.439+0000 7f08851138c0 1 < completed multipart upload key=XHGMUYBA2VNK upload=S33391EZX0ZB tag=L6LH1J0AQ40I type=write size=1963044 2024-05-30T16:03:54.445 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.447+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~DGCSMXSUW6CD.0 size=1048576 2024-05-30T16:03:54.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.447+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~DGCSMXSUW6CD.1 size=1048576 2024-05-30T16:03:54.450 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.451+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~DGCSMXSUW6CD.2 size=402012 2024-05-30T16:03:54.450 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.451+0000 7f08851138c0 1 > prepared multipart upload key=YRP4CSG92QRO upload=DGCSMXSUW6CD tag=ALTUEIVVDU3V type=write size=3547740 2024-05-30T16:03:54.454 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.455+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~JNIVGCN2BOWD.0 size=1043811 2024-05-30T16:03:54.459 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.459+0000 7f08851138c0 1 < completed multipart upload key=0RZRBW38WKNZ upload=JNIVGCN2BOWD tag=BI5OPPHC53T1 type=write size=2092387 2024-05-30T16:03:54.463 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.463+0000 7f08851138c0 1 > prepared operation key=ZALVY70ISLM5 tag=GFFTVOY08LEF type=write size=738517 2024-05-30T16:03:54.467 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.467+0000 7f08851138c0 1 < completed delete operation key=RCTV94WRMO53 tag=AOWG37QIYJHO type=del 2024-05-30T16:03:54.472 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.471+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=JEWONNMEFLQN type=del size=3451501 2024-05-30T16:03:54.476 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.475+0000 7f08851138c0 1 < completed delete operation key=YRP4CSG92QRO tag=HL95WLWSZVNC type=del 2024-05-30T16:03:54.479 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.479+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=X8K1CW0JGZCV type=del size=2062876 2024-05-30T16:03:54.483 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.483+0000 7f08851138c0 1 < canceled operation key=Y8SBI0QZ0ZV8 tag=7VRW681C21TS type=del size=590032 2024-05-30T16:03:54.486 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.487+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=CEOJ8M2M4LPR type=del size=1443590 2024-05-30T16:03:54.490 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.491+0000 7f08851138c0 1 < completed write operation key=KPHZGFJLC2E0 tag=RQ8JRHNG0F8E type=write size=392445 2024-05-30T16:03:54.493 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.495+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=3PNLC3YN18VP type=write size=897450 2024-05-30T16:03:54.497 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.499+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~16J9CFWRPSD2.0 size=1048576 2024-05-30T16:03:54.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.499+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~16J9CFWRPSD2.1 size=1048576 2024-05-30T16:03:54.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.499+0000 7f08851138c0 1 < canceled part key=_multipart_NIV7AQ4EV5YP.2~16J9CFWRPSD2.2 size=682081 2024-05-30T16:03:54.506 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.507+0000 7f08851138c0 1 < canceled multipart upload key=NIV7AQ4EV5YP upload=16J9CFWRPSD2 tag=A51QAGDGGXTT type=write size=3827809 2024-05-30T16:03:54.510 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.511+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=XFMFVA7ZHTBW type=write size=882668 2024-05-30T16:03:54.513 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.511+0000 7f08851138c0 1 < canceled operation key=A5BUBY9Z7JQG tag=127GG3VKGHBA type=del size=564796 2024-05-30T16:03:54.517 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.515+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=TQE7L9GDT5GX type=del size=2727265 2024-05-30T16:03:54.521 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.519+0000 7f08851138c0 1 < canceled operation key=YRP4CSG92QRO tag=UZIG28LQX5EX type=write size=811971 2024-05-30T16:03:54.528 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.527+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~W1BNC6MX51OC.0 size=1048576 2024-05-30T16:03:54.531 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.531+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~W1BNC6MX51OC.1 size=1048576 2024-05-30T16:03:54.534 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.535+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~W1BNC6MX51OC.2 size=829812 2024-05-30T16:03:54.534 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.535+0000 7f08851138c0 1 > prepared multipart upload key=GWXNCSJCLXKW upload=W1BNC6MX51OC tag=UJNF5XL88KXS type=write size=3975540 2024-05-30T16:03:54.542 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.543+0000 7f08851138c0 1 < suggested remove operation key=0RZRBW38WKNZ tag=MLDG66WU8EVU type=write 2024-05-30T16:03:54.546 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.547+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=TJJKQSSUAL7B type=del size=2071640 2024-05-30T16:03:54.549 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.551+0000 7f08851138c0 1 < canceled operation key=73FJ3HGWYQQK tag=N45B7KE2TIK1 type=del size=3682818 2024-05-30T16:03:54.553 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.551+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=L70Y7S21Q7MX type=write size=146624 2024-05-30T16:03:54.557 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.555+0000 7f08851138c0 1 < completed delete operation key=4N58ZY4E1H6E tag=VQRM7GK52WBG type=del 2024-05-30T16:03:54.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.559+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=EIMRH0MVRGIZ type=del size=2044818 2024-05-30T16:03:54.564 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.563+0000 7f08851138c0 1 < completed delete operation key=KPHZGFJLC2E0 tag=JJVHD7ETWR8T type=del 2024-05-30T16:03:54.571 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.571+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~2JQI6Z8VFFPJ.0 size=1017710 2024-05-30T16:03:54.571 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.571+0000 7f08851138c0 1 > prepared multipart upload key=KPHZGFJLC2E0 upload=2JQI6Z8VFFPJ tag=GWOT9GYRAVN4 type=write size=2066286 2024-05-30T16:03:54.576 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.575+0000 7f08851138c0 1 < completed delete operation key=KPHZGFJLC2E0 tag=MI0KECHKCDM1 type=del 2024-05-30T16:03:54.579 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.579+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=SJLR0HONZU2I type=del size=2827059 2024-05-30T16:03:54.586 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.587+0000 7f08851138c0 1 < suggested update operation key=O8CWPJ9GW2GO tag=9YWMHIASNHNH type=del size=2168380 2024-05-30T16:03:54.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.595+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~IXKA4OTEVB4H.0 size=1048576 2024-05-30T16:03:54.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.599+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~IXKA4OTEVB4H.1 size=155699 2024-05-30T16:03:54.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.599+0000 7f08851138c0 1 > prepared multipart upload key=364U3OV10VPD upload=IXKA4OTEVB4H tag=KDCVBPTCMQG5 type=write size=2252851 2024-05-30T16:03:54.601 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.599+0000 7f08851138c0 1 < completed part key=_multipart_O0DJ8SCY766V.2~N9VIBJG835WO.0 size=703167 2024-05-30T16:03:54.606 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.607+0000 7f08851138c0 1 < completed multipart upload key=O0DJ8SCY766V upload=N9VIBJG835WO tag=03WRQWGD74JG type=write size=1751743 2024-05-30T16:03:54.613 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.611+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~X6LUGAZR96WF.0 size=18563 2024-05-30T16:03:54.613 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.611+0000 7f08851138c0 1 > prepared multipart upload key=RCTV94WRMO53 upload=X6LUGAZR96WF tag=LV0KX12VQ7BI type=write size=1067139 2024-05-30T16:03:54.616 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.615+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~DGCSMXSUW6CD.0 size=1048576 2024-05-30T16:03:54.619 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.619+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~DGCSMXSUW6CD.1 size=1048576 2024-05-30T16:03:54.622 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.623+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~DGCSMXSUW6CD.2 size=402012 2024-05-30T16:03:54.628 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.627+0000 7f08851138c0 1 < completed multipart upload key=YRP4CSG92QRO upload=DGCSMXSUW6CD tag=ALTUEIVVDU3V type=write size=3547740 2024-05-30T16:03:54.632 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.631+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=SW9FWW33TBWK type=del size=586265 2024-05-30T16:03:54.636 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.635+0000 7f08851138c0 1 < completed write operation key=ZALVY70ISLM5 tag=GFFTVOY08LEF type=write size=738517 2024-05-30T16:03:54.642 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.643+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~SV4V2UYTIG6H.0 size=590038 2024-05-30T16:03:54.642 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.643+0000 7f08851138c0 1 > prepared multipart upload key=TPCETCO3Y1M6 upload=SV4V2UYTIG6H tag=8UK19CVRJZ82 type=write size=1638614 2024-05-30T16:03:54.644 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.643+0000 7f08851138c0 -1 < no bi entry to suggest for operation key=0RZRBW38WKNZ tag=JEWONNMEFLQN type=del size=3451501: (2) No such file or directory 2024-05-30T16:03:54.648 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.647+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=G492JVXPE8KJ type=del size=3315505 2024-05-30T16:03:54.652 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.651+0000 7f08851138c0 1 < completed delete operation key=NIV7AQ4EV5YP tag=X8K1CW0JGZCV type=del 2024-05-30T16:03:54.658 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.659+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=DMT3EYSZZ06P type=del size=1102729 2024-05-30T16:03:54.669 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.667+0000 7f08851138c0 1 < suggested update operation key=O0DJ8SCY766V tag=CEOJ8M2M4LPR type=del size=1443590 2024-05-30T16:03:54.673 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.671+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=UIOFWYR0N31A type=del size=169140 2024-05-30T16:03:54.676 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.675+0000 7f08851138c0 -1 < failed to complete operation key=O0DJ8SCY766V tag=3PNLC3YN18VP type=write size=897450: (22) Invalid argument 2024-05-30T16:03:54.679 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.679+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=VJD0K5WYRU4W type=del size=1807742 2024-05-30T16:03:54.682 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.683+0000 7f08851138c0 1 < completed write operation key=E2FXSXOEUGIG tag=XFMFVA7ZHTBW type=write size=882668 2024-05-30T16:03:54.685 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.683+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=47AH5YUEF0ML type=del size=1557397 2024-05-30T16:03:54.688 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.687+0000 7f08851138c0 1 < completed delete operation key=KPHZGFJLC2E0 tag=TQE7L9GDT5GX type=del 2024-05-30T16:03:54.691 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.691+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=2LLD3AIAYSVE type=del size=3849254 2024-05-30T16:03:54.694 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.695+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~W1BNC6MX51OC.0 size=1048576 2024-05-30T16:03:54.697 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.695+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~W1BNC6MX51OC.1 size=1048576 2024-05-30T16:03:54.699 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.699+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~W1BNC6MX51OC.2 size=829812 2024-05-30T16:03:54.704 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.703+0000 7f08851138c0 1 < completed multipart upload key=GWXNCSJCLXKW upload=W1BNC6MX51OC tag=UJNF5XL88KXS type=write size=3975540 2024-05-30T16:03:54.710 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.711+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~NWA4U340MBHI.0 size=1048576 2024-05-30T16:03:54.712 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.711+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~NWA4U340MBHI.1 size=1048576 2024-05-30T16:03:54.715 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.715+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~NWA4U340MBHI.2 size=856969 2024-05-30T16:03:54.715 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.715+0000 7f08851138c0 1 > prepared multipart upload key=3IJWPWRB6FHE upload=NWA4U340MBHI tag=XW1DM8PUS63B type=write size=4002697 2024-05-30T16:03:54.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.719+0000 7f08851138c0 1 < completed delete operation key=JNZI7JX95LX2 tag=TJJKQSSUAL7B type=del 2024-05-30T16:03:54.721 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.723+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=L3APVWE2J7ZN type=write size=1010863 2024-05-30T16:03:54.725 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.723+0000 7f08851138c0 1 < completed write operation key=V4OQFOOQUXFQ tag=L70Y7S21Q7MX type=write size=146624 2024-05-30T16:03:54.730 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.731+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~GLR0DMMN34F9.0 size=1048576 2024-05-30T16:03:54.733 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.731+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~GLR0DMMN34F9.1 size=1048576 2024-05-30T16:03:54.735 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.735+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~GLR0DMMN34F9.2 size=20328 2024-05-30T16:03:54.735 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.735+0000 7f08851138c0 1 > prepared multipart upload key=V4OQFOOQUXFQ upload=GLR0DMMN34F9 tag=AVTTTEA7UKGY type=write size=3166056 2024-05-30T16:03:54.739 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.739+0000 7f08851138c0 1 < completed delete operation key=E2FXSXOEUGIG tag=EIMRH0MVRGIZ type=del 2024-05-30T16:03:54.744 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.743+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~V9DM0FS7FC5X.0 size=1048576 2024-05-30T16:03:54.747 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.747+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~V9DM0FS7FC5X.1 size=1048576 2024-05-30T16:03:54.749 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.751+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~V9DM0FS7FC5X.2 size=18362 2024-05-30T16:03:54.749 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.751+0000 7f08851138c0 1 > prepared multipart upload key=WT9RC0UYVG4F upload=V9DM0FS7FC5X tag=8P4BR91UXLU0 type=write size=3164090 2024-05-30T16:03:54.750 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.751+0000 7f08851138c0 1 < canceled part key=_multipart_KPHZGFJLC2E0.2~2JQI6Z8VFFPJ.0 size=1017710 2024-05-30T16:03:54.754 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.755+0000 7f08851138c0 1 < canceled multipart upload key=KPHZGFJLC2E0 upload=2JQI6Z8VFFPJ tag=GWOT9GYRAVN4 type=write size=2066286 2024-05-30T16:03:54.760 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.759+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~TIPW4T5HXX9J.0 size=1048576 2024-05-30T16:03:54.763 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.763+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~TIPW4T5HXX9J.1 size=1048576 2024-05-30T16:03:54.766 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.767+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~TIPW4T5HXX9J.2 size=405870 2024-05-30T16:03:54.766 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.767+0000 7f08851138c0 1 > prepared multipart upload key=E2FXSXOEUGIG upload=TIPW4T5HXX9J tag=34KLHY32X8K5 type=write size=3551598 2024-05-30T16:03:54.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.767+0000 7f08851138c0 1 < completed delete operation key=3TBOZWT2P8U8 tag=SJLR0HONZU2I type=del 2024-05-30T16:03:54.772 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.771+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=5FXYI6HNTW8E type=del size=2719219 2024-05-30T16:03:54.776 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.775+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~IXKA4OTEVB4H.0 size=1048576 2024-05-30T16:03:54.778 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.779+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~IXKA4OTEVB4H.1 size=155699 2024-05-30T16:03:54.783 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.783+0000 7f08851138c0 1 < completed multipart upload key=364U3OV10VPD upload=IXKA4OTEVB4H tag=KDCVBPTCMQG5 type=write size=2252851 2024-05-30T16:03:54.787 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.787+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=NNZE7LVUDT04 type=del size=2034041 2024-05-30T16:03:54.790 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.791+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~X6LUGAZR96WF.0 size=18563 2024-05-30T16:03:54.795 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.795+0000 7f08851138c0 1 < completed multipart upload key=RCTV94WRMO53 upload=X6LUGAZR96WF tag=LV0KX12VQ7BI type=write size=1067139 2024-05-30T16:03:54.798 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.799+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=PY0I0TVNEXJR type=del size=3215193 2024-05-30T16:03:54.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.803+0000 7f08851138c0 1 < completed delete operation key=KPHZGFJLC2E0 tag=SW9FWW33TBWK type=del 2024-05-30T16:03:54.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.803+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=MPJYJLASM36H type=del size=3651211 2024-05-30T16:03:54.808 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.807+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~SV4V2UYTIG6H.0 size=590038 2024-05-30T16:03:54.813 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.811+0000 7f08851138c0 1 < completed multipart upload key=TPCETCO3Y1M6 upload=SV4V2UYTIG6H tag=8UK19CVRJZ82 type=write size=1638614 2024-05-30T16:03:54.816 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.815+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=C4548VN85HY5 type=del size=2884712 2024-05-30T16:03:54.819 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.819+0000 7f08851138c0 1 < completed delete operation key=364U3OV10VPD tag=G492JVXPE8KJ type=del 2024-05-30T16:03:54.824 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.823+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~C7L8AD58VOYW.0 size=517229 2024-05-30T16:03:54.824 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.823+0000 7f08851138c0 1 > prepared multipart upload key=VRXNO8JLAZYH upload=C7L8AD58VOYW tag=RL9Q5X8J904U type=write size=1565805 2024-05-30T16:03:54.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.827+0000 7f08851138c0 1 < canceled operation key=GWXNCSJCLXKW tag=DMT3EYSZZ06P type=del size=1102729 2024-05-30T16:03:54.834 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.835+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~FUKPQO6TSBP2.0 size=1048576 2024-05-30T16:03:54.837 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.835+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~FUKPQO6TSBP2.1 size=1048576 2024-05-30T16:03:54.839 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.839+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~FUKPQO6TSBP2.2 size=160171 2024-05-30T16:03:54.839 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.839+0000 7f08851138c0 1 > prepared multipart upload key=E2FXSXOEUGIG upload=FUKPQO6TSBP2 tag=YTTUYTEG5YER type=write size=3305899 2024-05-30T16:03:54.844 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.843+0000 7f08851138c0 1 < suggested update operation key=73FJ3HGWYQQK tag=UIOFWYR0N31A type=del size=169140 2024-05-30T16:03:54.850 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.851+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~53AQV52UU1V8.0 size=241891 2024-05-30T16:03:54.850 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.851+0000 7f08851138c0 1 > prepared multipart upload key=Y8SBI0QZ0ZV8 upload=53AQV52UU1V8 tag=XFA01Z03J4HK type=write size=1290467 2024-05-30T16:03:54.853 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.851+0000 7f08851138c0 1 < completed delete operation key=3IJWPWRB6FHE tag=VJD0K5WYRU4W type=del 2024-05-30T16:03:54.856 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.855+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=HMSAVLPXUQWY type=write size=186133 2024-05-30T16:03:54.860 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.859+0000 7f08851138c0 1 < completed delete operation key=603HBR8PRH6L tag=47AH5YUEF0ML type=del 2024-05-30T16:03:54.863 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.863+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=QYHT0NKJ363G type=del size=441325 2024-05-30T16:03:54.866 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.867+0000 7f08851138c0 1 < completed delete operation key=QQILXRJD0C1Z tag=2LLD3AIAYSVE type=del 2024-05-30T16:03:54.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.871+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=5SKO6S0FBT0U type=del size=2067872 2024-05-30T16:03:54.873 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.871+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~NWA4U340MBHI.0 size=1048576 2024-05-30T16:03:54.873 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.871+0000 7f08851138c0 1 < canceled part key=_multipart_3IJWPWRB6FHE.2~NWA4U340MBHI.1 size=1048576 2024-05-30T16:03:54.875 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.875+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~NWA4U340MBHI.2 size=856969 2024-05-30T16:03:54.880 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.879+0000 7f08851138c0 1 < canceled multipart upload key=3IJWPWRB6FHE upload=NWA4U340MBHI tag=XW1DM8PUS63B type=write size=4002697 2024-05-30T16:03:54.886 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.887+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~A456V6RNNW13.0 size=1048576 2024-05-30T16:03:54.888 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.887+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~A456V6RNNW13.1 size=1048576 2024-05-30T16:03:54.891 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.891+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~A456V6RNNW13.2 size=892924 2024-05-30T16:03:54.891 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.891+0000 7f08851138c0 1 > prepared multipart upload key=XHGMUYBA2VNK upload=A456V6RNNW13 tag=L2UL33IPVT4B type=write size=4038652 2024-05-30T16:03:54.894 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.895+0000 7f08851138c0 1 < completed write operation key=JNZI7JX95LX2 tag=L3APVWE2J7ZN type=write size=1010863 2024-05-30T16:03:54.902 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.903+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=E2PT45PKCM50 type=write size=922069 2024-05-30T16:03:54.905 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.903+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~GLR0DMMN34F9.0 size=1048576 2024-05-30T16:03:54.907 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.907+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~GLR0DMMN34F9.1 size=1048576 2024-05-30T16:03:54.910 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.911+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~GLR0DMMN34F9.2 size=20328 2024-05-30T16:03:54.915 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.915+0000 7f08851138c0 1 < completed multipart upload key=V4OQFOOQUXFQ upload=GLR0DMMN34F9 tag=AVTTTEA7UKGY type=write size=3166056 2024-05-30T16:03:54.917 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.919+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=JUJVMAOR9UZ1 type=del size=1873553 2024-05-30T16:03:54.920 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.919+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~V9DM0FS7FC5X.0 size=1048576 2024-05-30T16:03:54.920 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.919+0000 7f08851138c0 1 < canceled part key=_multipart_WT9RC0UYVG4F.2~V9DM0FS7FC5X.1 size=1048576 2024-05-30T16:03:54.921 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.919+0000 7f08851138c0 1 < canceled part key=_multipart_WT9RC0UYVG4F.2~V9DM0FS7FC5X.2 size=18362 2024-05-30T16:03:54.926 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.927+0000 7f08851138c0 1 < canceled multipart upload key=WT9RC0UYVG4F upload=V9DM0FS7FC5X tag=8P4BR91UXLU0 type=write size=3164090 2024-05-30T16:03:54.929 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.931+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=B7EBDIV0MNY0 type=write size=530138 2024-05-30T16:03:54.933 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.935+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~TIPW4T5HXX9J.0 size=1048576 2024-05-30T16:03:54.936 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.935+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~TIPW4T5HXX9J.1 size=1048576 2024-05-30T16:03:54.939 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.939+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~TIPW4T5HXX9J.2 size=405870 2024-05-30T16:03:54.945 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.943+0000 7f08851138c0 1 < completed multipart upload key=E2FXSXOEUGIG upload=TIPW4T5HXX9J tag=34KLHY32X8K5 type=write size=3551598 2024-05-30T16:03:54.950 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.951+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~5NCJ8RTDACPZ.0 size=1048576 2024-05-30T16:03:54.953 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.951+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~5NCJ8RTDACPZ.1 size=201254 2024-05-30T16:03:54.953 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.951+0000 7f08851138c0 1 > prepared multipart upload key=JNZI7JX95LX2 upload=5NCJ8RTDACPZ tag=VC0ZZUV1BYP2 type=write size=2298406 2024-05-30T16:03:54.956 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.955+0000 7f08851138c0 1 < completed delete operation key=E2FXSXOEUGIG tag=5FXYI6HNTW8E type=del 2024-05-30T16:03:54.962 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.963+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~S4K8TNWKHA42.0 size=1048576 2024-05-30T16:03:54.964 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.963+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~S4K8TNWKHA42.1 size=1048576 2024-05-30T16:03:54.967 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.967+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~S4K8TNWKHA42.2 size=731536 2024-05-30T16:03:54.967 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.967+0000 7f08851138c0 1 > prepared multipart upload key=0T2E9F3K1USF upload=S4K8TNWKHA42 tag=YY9L21C6CCLM type=write size=3877264 2024-05-30T16:03:54.973 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.975+0000 7f08851138c0 1 < suggested remove operation key=NIV7AQ4EV5YP tag=NNZE7LVUDT04 type=del 2024-05-30T16:03:54.977 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.975+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=8SUSKFCPCVDY type=del size=222895 2024-05-30T16:03:54.980 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.979+0000 7f08851138c0 1 < completed delete operation key=3TBOZWT2P8U8 tag=PY0I0TVNEXJR type=del 2024-05-30T16:03:54.986 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.987+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~ZC1YLM5GN2EQ.0 size=1048576 2024-05-30T16:03:54.989 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.987+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~ZC1YLM5GN2EQ.1 size=902953 2024-05-30T16:03:54.989 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.987+0000 7f08851138c0 1 > prepared multipart upload key=1N6R2HBTWMP2 upload=ZC1YLM5GN2EQ tag=II4H2PUN16OW type=write size=3000105 2024-05-30T16:03:54.995 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:54.995+0000 7f08851138c0 1 < suggested remove operation key=1N6R2HBTWMP2 tag=MPJYJLASM36H type=del 2024-05-30T16:03:55.001 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.003+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~9DWCQAYJR32D.0 size=1048576 2024-05-30T16:03:55.004 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.003+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~9DWCQAYJR32D.1 size=1048576 2024-05-30T16:03:55.006 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.007+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~9DWCQAYJR32D.2 size=886216 2024-05-30T16:03:55.007 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.007+0000 7f08851138c0 1 > prepared multipart upload key=4N58ZY4E1H6E upload=9DWCQAYJR32D tag=9Z3V7P1TV78T type=write size=4031944 2024-05-30T16:03:55.010 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.011+0000 7f08851138c0 -1 < failed to complete operation key=1N6R2HBTWMP2 tag=C4548VN85HY5 type=del size=2884712: (22) Invalid argument 2024-05-30T16:03:55.017 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.019+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~5J3413ES8XDZ.0 size=1048576 2024-05-30T16:03:55.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.019+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~5J3413ES8XDZ.1 size=384232 2024-05-30T16:03:55.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.019+0000 7f08851138c0 1 > prepared multipart upload key=364U3OV10VPD upload=5J3413ES8XDZ tag=W00Y63FG183O type=write size=2481384 2024-05-30T16:03:55.023 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.023+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~C7L8AD58VOYW.0 size=517229 2024-05-30T16:03:55.028 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.027+0000 7f08851138c0 1 < completed multipart upload key=VRXNO8JLAZYH upload=C7L8AD58VOYW tag=RL9Q5X8J904U type=write size=1565805 2024-05-30T16:03:55.031 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.031+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=4XW04VRZO6K0 type=del size=1453057 2024-05-30T16:03:55.035 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.035+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~FUKPQO6TSBP2.0 size=1048576 2024-05-30T16:03:55.037 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.039+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~FUKPQO6TSBP2.1 size=1048576 2024-05-30T16:03:55.040 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.039+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~FUKPQO6TSBP2.2 size=160171 2024-05-30T16:03:55.045 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.047+0000 7f08851138c0 1 < completed multipart upload key=E2FXSXOEUGIG upload=FUKPQO6TSBP2 tag=YTTUYTEG5YER type=write size=3305899 2024-05-30T16:03:55.051 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.051+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~ATZM3GXWG7SY.0 size=1048576 2024-05-30T16:03:55.054 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.055+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~ATZM3GXWG7SY.1 size=1048576 2024-05-30T16:03:55.056 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.055+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~ATZM3GXWG7SY.2 size=269825 2024-05-30T16:03:55.056 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.055+0000 7f08851138c0 1 > prepared multipart upload key=NIV7AQ4EV5YP upload=ATZM3GXWG7SY tag=7EXX0ACMSWW5 type=write size=3415553 2024-05-30T16:03:55.059 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.059+0000 7f08851138c0 1 < completed part key=_multipart_Y8SBI0QZ0ZV8.2~53AQV52UU1V8.0 size=241891 2024-05-30T16:03:55.065 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.063+0000 7f08851138c0 1 < completed multipart upload key=Y8SBI0QZ0ZV8 upload=53AQV52UU1V8 tag=XFA01Z03J4HK type=write size=1290467 2024-05-30T16:03:55.068 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.067+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=5UTITDXQRPDQ type=del size=788477 2024-05-30T16:03:55.071 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.071+0000 7f08851138c0 1 < completed write operation key=QCLY10D2A3VF tag=HMSAVLPXUQWY type=write size=186133 2024-05-30T16:03:55.077 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.075+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~BW6AL73O31Q4.0 size=957479 2024-05-30T16:03:55.077 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.075+0000 7f08851138c0 1 > prepared multipart upload key=VRXNO8JLAZYH upload=BW6AL73O31Q4 tag=ZAH36LJHC0M1 type=write size=2006055 2024-05-30T16:03:55.080 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.079+0000 7f08851138c0 1 < completed delete operation key=O8CWPJ9GW2GO tag=QYHT0NKJ363G type=del 2024-05-30T16:03:55.084 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.083+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=YL6Y8G9GGE0F type=del size=392684 2024-05-30T16:03:55.087 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.087+0000 7f08851138c0 1 < completed delete operation key=QCLY10D2A3VF tag=5SKO6S0FBT0U type=del 2024-05-30T16:03:55.093 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.095+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~XXKT5S0UQKY6.0 size=797815 2024-05-30T16:03:55.093 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.095+0000 7f08851138c0 1 > prepared multipart upload key=E2FXSXOEUGIG upload=XXKT5S0UQKY6 tag=N4R9DRTWBCXP type=write size=1846391 2024-05-30T16:03:55.096 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.095+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~A456V6RNNW13.0 size=1048576 2024-05-30T16:03:55.099 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.099+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~A456V6RNNW13.1 size=1048576 2024-05-30T16:03:55.102 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.103+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~A456V6RNNW13.2 size=892924 2024-05-30T16:03:55.108 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.107+0000 7f08851138c0 1 < completed multipart upload key=XHGMUYBA2VNK upload=A456V6RNNW13 tag=L2UL33IPVT4B type=write size=4038652 2024-05-30T16:03:55.112 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.111+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=930U329BSU3S type=write size=446311 2024-05-30T16:03:55.115 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.115+0000 7f08851138c0 1 < completed write operation key=Y3CYZJ83WYNI tag=E2PT45PKCM50 type=write size=922069 2024-05-30T16:03:55.119 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.119+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=IONX7JAW02AV type=del size=3054704 2024-05-30T16:03:55.122 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.123+0000 7f08851138c0 1 < completed delete operation key=GWXNCSJCLXKW tag=JUJVMAOR9UZ1 type=del 2024-05-30T16:03:55.129 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.127+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~DCIY3RC0NITL.0 size=664586 2024-05-30T16:03:55.129 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.127+0000 7f08851138c0 1 > prepared multipart upload key=V4OQFOOQUXFQ upload=DCIY3RC0NITL tag=PZJXV1JCWGY3 type=write size=1713162 2024-05-30T16:03:55.133 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.131+0000 7f08851138c0 1 < completed write operation key=0T2E9F3K1USF tag=B7EBDIV0MNY0 type=write size=530138 2024-05-30T16:03:55.136 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.135+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=7ONHQ0AHQER9 type=del size=583228 2024-05-30T16:03:55.139 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.139+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~5NCJ8RTDACPZ.0 size=1048576 2024-05-30T16:03:55.142 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.143+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~5NCJ8RTDACPZ.1 size=201254 2024-05-30T16:03:55.148 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.147+0000 7f08851138c0 1 < completed multipart upload key=JNZI7JX95LX2 upload=5NCJ8RTDACPZ tag=VC0ZZUV1BYP2 type=write size=2298406 2024-05-30T16:03:55.154 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.155+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~MFU9UCXKL3IJ.0 size=1048576 2024-05-30T16:03:55.157 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.155+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~MFU9UCXKL3IJ.1 size=926937 2024-05-30T16:03:55.157 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.155+0000 7f08851138c0 1 > prepared multipart upload key=NIV7AQ4EV5YP upload=MFU9UCXKL3IJ tag=M6QKHIRQPTSY type=write size=3024089 2024-05-30T16:03:55.161 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.159+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~S4K8TNWKHA42.0 size=1048576 2024-05-30T16:03:55.164 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.163+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~S4K8TNWKHA42.1 size=1048576 2024-05-30T16:03:55.167 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.167+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~S4K8TNWKHA42.2 size=731536 2024-05-30T16:03:55.172 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.171+0000 7f08851138c0 1 < completed multipart upload key=0T2E9F3K1USF upload=S4K8TNWKHA42 tag=YY9L21C6CCLM type=write size=3877264 2024-05-30T16:03:55.176 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.175+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=RX4JDYK2KFDG type=del size=375163 2024-05-30T16:03:55.180 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.179+0000 7f08851138c0 1 < completed delete operation key=JNZI7JX95LX2 tag=8SUSKFCPCVDY type=del 2024-05-30T16:03:55.186 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.187+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~L2HRYXNYD896.0 size=1048576 2024-05-30T16:03:55.189 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.191+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~L2HRYXNYD896.1 size=1048576 2024-05-30T16:03:55.192 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.191+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~L2HRYXNYD896.2 size=245031 2024-05-30T16:03:55.192 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.191+0000 7f08851138c0 1 > prepared multipart upload key=GWXNCSJCLXKW upload=L2HRYXNYD896 tag=560RU7EFPLIY type=write size=3390759 2024-05-30T16:03:55.196 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.195+0000 7f08851138c0 1 < completed part key=_multipart_1N6R2HBTWMP2.2~ZC1YLM5GN2EQ.0 size=1048576 2024-05-30T16:03:55.199 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.199+0000 7f08851138c0 1 < completed part key=_multipart_1N6R2HBTWMP2.2~ZC1YLM5GN2EQ.1 size=902953 2024-05-30T16:03:55.210 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.211+0000 7f08851138c0 1 < completed multipart upload key=1N6R2HBTWMP2 upload=ZC1YLM5GN2EQ tag=II4H2PUN16OW type=write size=3000105 2024-05-30T16:03:55.216 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.215+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~GGCJ3S704G8P.0 size=1048576 2024-05-30T16:03:55.219 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.219+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~GGCJ3S704G8P.1 size=743530 2024-05-30T16:03:55.219 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.219+0000 7f08851138c0 1 > prepared multipart upload key=0T2E9F3K1USF upload=GGCJ3S704G8P tag=4HA27MZJH39B type=write size=2840682 2024-05-30T16:03:55.223 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.223+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~9DWCQAYJR32D.0 size=1048576 2024-05-30T16:03:55.226 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.227+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~9DWCQAYJR32D.1 size=1048576 2024-05-30T16:03:55.229 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.227+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~9DWCQAYJR32D.2 size=886216 2024-05-30T16:03:55.234 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.235+0000 7f08851138c0 1 < completed multipart upload key=4N58ZY4E1H6E upload=9DWCQAYJR32D tag=9Z3V7P1TV78T type=write size=4031944 2024-05-30T16:03:55.238 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.235+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=HPPC58ZB008X type=del size=1040802 2024-05-30T16:03:55.241 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.239+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~5J3413ES8XDZ.0 size=1048576 2024-05-30T16:03:55.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.243+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~5J3413ES8XDZ.1 size=384232 2024-05-30T16:03:55.250 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.251+0000 7f08851138c0 1 < completed multipart upload key=364U3OV10VPD upload=5J3413ES8XDZ tag=W00Y63FG183O type=write size=2481384 2024-05-30T16:03:55.256 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.255+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~9IKZ96LXVUCY.0 size=1048576 2024-05-30T16:03:55.260 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.259+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~9IKZ96LXVUCY.1 size=1048576 2024-05-30T16:03:55.262 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.263+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~9IKZ96LXVUCY.2 size=287980 2024-05-30T16:03:55.263 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.263+0000 7f08851138c0 1 > prepared multipart upload key=PLWFSKPKVO19 upload=9IKZ96LXVUCY tag=A4FBOIGQU8B8 type=write size=3433708 2024-05-30T16:03:55.266 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.267+0000 7f08851138c0 1 < completed delete operation key=WT9RC0UYVG4F tag=4XW04VRZO6K0 type=del 2024-05-30T16:03:55.269 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.271+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=U2UZVGGDWZDS type=del size=734984 2024-05-30T16:03:55.273 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.271+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~ATZM3GXWG7SY.0 size=1048576 2024-05-30T16:03:55.276 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.275+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~ATZM3GXWG7SY.1 size=1048576 2024-05-30T16:03:55.278 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.279+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~ATZM3GXWG7SY.2 size=269825 2024-05-30T16:03:55.284 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.283+0000 7f08851138c0 1 < completed multipart upload key=NIV7AQ4EV5YP upload=ATZM3GXWG7SY tag=7EXX0ACMSWW5 type=write size=3415553 2024-05-30T16:03:55.287 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.287+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=CZL5ZBH6NNT0 type=del size=1981326 2024-05-30T16:03:55.291 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.291+0000 7f08851138c0 1 < completed delete operation key=QQILXRJD0C1Z tag=5UTITDXQRPDQ type=del 2024-05-30T16:03:55.294 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.295+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=JB56VHL5R1Y3 type=del size=2725897 2024-05-30T16:03:55.298 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.299+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~BW6AL73O31Q4.0 size=957479 2024-05-30T16:03:55.304 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.303+0000 7f08851138c0 1 < completed multipart upload key=VRXNO8JLAZYH upload=BW6AL73O31Q4 tag=ZAH36LJHC0M1 type=write size=2006055 2024-05-30T16:03:55.310 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.311+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~JGK84KBQANJ5.0 size=1048576 2024-05-30T16:03:55.313 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.311+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~JGK84KBQANJ5.1 size=778974 2024-05-30T16:03:55.313 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.311+0000 7f08851138c0 1 > prepared multipart upload key=Y8SBI0QZ0ZV8 upload=JGK84KBQANJ5 tag=V17BVZ6AKOKT type=write size=2876126 2024-05-30T16:03:55.316 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.315+0000 7f08851138c0 1 < completed delete operation key=Y3CYZJ83WYNI tag=YL6Y8G9GGE0F type=del 2024-05-30T16:03:55.320 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.319+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=X8Y8MYJWZ0O4 type=del size=3891145 2024-05-30T16:03:55.324 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.323+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~XXKT5S0UQKY6.0 size=797815 2024-05-30T16:03:55.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.331+0000 7f08851138c0 1 < completed multipart upload key=E2FXSXOEUGIG upload=XXKT5S0UQKY6 tag=N4R9DRTWBCXP type=write size=1846391 2024-05-30T16:03:55.337 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.335+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~P33DRRCSBJTC.0 size=1048576 2024-05-30T16:03:55.340 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.339+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~P33DRRCSBJTC.1 size=1048576 2024-05-30T16:03:55.343 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.343+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~P33DRRCSBJTC.2 size=405960 2024-05-30T16:03:55.343 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.343+0000 7f08851138c0 1 > prepared multipart upload key=GWXNCSJCLXKW upload=P33DRRCSBJTC tag=DQDIX3U322MV type=write size=3551688 2024-05-30T16:03:55.351 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.351+0000 7f08851138c0 1 < suggested update operation key=XHGMUYBA2VNK tag=930U329BSU3S type=write size=446311 2024-05-30T16:03:55.358 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.359+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~ZNBAFQEU5AO0.0 size=456922 2024-05-30T16:03:55.358 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.359+0000 7f08851138c0 1 > prepared multipart upload key=A5BUBY9Z7JQG upload=ZNBAFQEU5AO0 tag=PZ109BDQBT3N type=write size=1505498 2024-05-30T16:03:55.362 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.363+0000 7f08851138c0 1 < completed delete operation key=QCLY10D2A3VF tag=IONX7JAW02AV type=del 2024-05-30T16:03:55.370 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.371+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~L1WWSME0MY7C.0 size=1048576 2024-05-30T16:03:55.372 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.371+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~L1WWSME0MY7C.1 size=881218 2024-05-30T16:03:55.372 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.371+0000 7f08851138c0 1 > prepared multipart upload key=ZALVY70ISLM5 upload=L1WWSME0MY7C tag=AH47GA5J5W3Y type=write size=2978370 2024-05-30T16:03:55.376 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.375+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~DCIY3RC0NITL.0 size=664586 2024-05-30T16:03:55.381 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.383+0000 7f08851138c0 1 < completed multipart upload key=V4OQFOOQUXFQ upload=DCIY3RC0NITL tag=PZJXV1JCWGY3 type=write size=1713162 2024-05-30T16:03:55.385 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.383+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=YHCVV06J15YO type=del size=2110073 2024-05-30T16:03:55.388 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.387+0000 7f08851138c0 1 < completed delete operation key=0RZRBW38WKNZ tag=7ONHQ0AHQER9 type=del 2024-05-30T16:03:55.392 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.391+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=CSU6HIHEQGSO type=del size=4008607 2024-05-30T16:03:55.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.395+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~MFU9UCXKL3IJ.0 size=1048576 2024-05-30T16:03:55.398 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.399+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~MFU9UCXKL3IJ.1 size=926937 2024-05-30T16:03:55.404 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.403+0000 7f08851138c0 1 < completed multipart upload key=NIV7AQ4EV5YP upload=MFU9UCXKL3IJ tag=M6QKHIRQPTSY type=write size=3024089 2024-05-30T16:03:55.410 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.411+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~K3SLCHJ2F8AJ.0 size=1048576 2024-05-30T16:03:55.413 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.415+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~K3SLCHJ2F8AJ.1 size=776569 2024-05-30T16:03:55.413 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.415+0000 7f08851138c0 1 > prepared multipart upload key=ZALVY70ISLM5 upload=K3SLCHJ2F8AJ tag=ZQYW2ETH03EI type=write size=2873721 2024-05-30T16:03:55.417 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.415+0000 7f08851138c0 1 < completed delete operation key=PLWFSKPKVO19 tag=RX4JDYK2KFDG type=del 2024-05-30T16:03:55.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.423+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~EMTNW7MQTBQT.0 size=139013 2024-05-30T16:03:55.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.423+0000 7f08851138c0 1 > prepared multipart upload key=364U3OV10VPD upload=EMTNW7MQTBQT tag=CG6P6XMU0AAQ type=write size=1187589 2024-05-30T16:03:55.427 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.427+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~L2HRYXNYD896.0 size=1048576 2024-05-30T16:03:55.430 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.431+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~L2HRYXNYD896.1 size=1048576 2024-05-30T16:03:55.433 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.431+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~L2HRYXNYD896.2 size=245031 2024-05-30T16:03:55.440 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.439+0000 7f08851138c0 1 < completed multipart upload key=GWXNCSJCLXKW upload=L2HRYXNYD896 tag=560RU7EFPLIY type=write size=3390759 2024-05-30T16:03:55.445 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.443+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=KMP6KBW8IO2D type=del size=1865390 2024-05-30T16:03:55.449 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.447+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~GGCJ3S704G8P.0 size=1048576 2024-05-30T16:03:55.453 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.451+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~GGCJ3S704G8P.1 size=743530 2024-05-30T16:03:55.460 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.459+0000 7f08851138c0 1 < completed multipart upload key=0T2E9F3K1USF upload=GGCJ3S704G8P tag=4HA27MZJH39B type=write size=2840682 2024-05-30T16:03:55.464 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.463+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=PNQ5JBET9HVG type=del size=330185 2024-05-30T16:03:55.468 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.467+0000 7f08851138c0 1 < completed delete operation key=3IJWPWRB6FHE tag=HPPC58ZB008X type=del 2024-05-30T16:03:55.472 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.471+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=86ZT2S9FP3RC type=del size=134995 2024-05-30T16:03:55.475 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.475+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~9IKZ96LXVUCY.0 size=1048576 2024-05-30T16:03:55.478 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.479+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~9IKZ96LXVUCY.1 size=1048576 2024-05-30T16:03:55.479 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.479+0000 7f08851138c0 1 < canceled part key=_multipart_PLWFSKPKVO19.2~9IKZ96LXVUCY.2 size=287980 2024-05-30T16:03:55.485 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.483+0000 7f08851138c0 1 < canceled multipart upload key=PLWFSKPKVO19 upload=9IKZ96LXVUCY tag=A4FBOIGQU8B8 type=write size=3433708 2024-05-30T16:03:55.488 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.487+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=MQ76RXSR4SSS type=del size=767939 2024-05-30T16:03:55.496 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.495+0000 7f08851138c0 1 < suggested update operation key=RCTV94WRMO53 tag=U2UZVGGDWZDS type=del size=734984 2024-05-30T16:03:55.499 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.499+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=69Q2V5GEDMLN type=write size=278074 2024-05-30T16:03:55.503 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.503+0000 7f08851138c0 1 < canceled operation key=WT9RC0UYVG4F tag=CZL5ZBH6NNT0 type=del size=1981326 2024-05-30T16:03:55.510 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.511+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~IHWH2B9IVM3A.0 size=1048576 2024-05-30T16:03:55.512 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.511+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~IHWH2B9IVM3A.1 size=1048576 2024-05-30T16:03:55.515 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.515+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~IHWH2B9IVM3A.2 size=838725 2024-05-30T16:03:55.516 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.515+0000 7f08851138c0 1 > prepared multipart upload key=QQILXRJD0C1Z upload=IHWH2B9IVM3A tag=PLQ0R08JM0F0 type=write size=3984453 2024-05-30T16:03:55.519 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.519+0000 7f08851138c0 1 < completed delete operation key=PLWFSKPKVO19 tag=JB56VHL5R1Y3 type=del 2024-05-30T16:03:55.523 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.523+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=778PH4NLI31S type=write size=825951 2024-05-30T16:03:55.527 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.527+0000 7f08851138c0 1 < completed part key=_multipart_Y8SBI0QZ0ZV8.2~JGK84KBQANJ5.0 size=1048576 2024-05-30T16:03:55.530 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.531+0000 7f08851138c0 1 < completed part key=_multipart_Y8SBI0QZ0ZV8.2~JGK84KBQANJ5.1 size=778974 2024-05-30T16:03:55.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.535+0000 7f08851138c0 1 < completed multipart upload key=Y8SBI0QZ0ZV8 upload=JGK84KBQANJ5 tag=V17BVZ6AKOKT type=write size=2876126 2024-05-30T16:03:55.540 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.539+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=I4OWUSLKWWGE type=del size=1255739 2024-05-30T16:03:55.550 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.551+0000 7f08851138c0 -1 < failed to complete operation key=RCTV94WRMO53 tag=X8Y8MYJWZ0O4 type=del size=3891145: (22) Invalid argument 2024-05-30T16:03:55.554 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.555+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=00H6TFK1CQU2 type=write size=347230 2024-05-30T16:03:55.558 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.559+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~P33DRRCSBJTC.0 size=1048576 2024-05-30T16:03:55.558 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.559+0000 7f08851138c0 1 < canceled part key=_multipart_GWXNCSJCLXKW.2~P33DRRCSBJTC.1 size=1048576 2024-05-30T16:03:55.562 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.563+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~P33DRRCSBJTC.2 size=405960 2024-05-30T16:03:55.568 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.567+0000 7f08851138c0 1 < canceled multipart upload key=GWXNCSJCLXKW upload=P33DRRCSBJTC tag=DQDIX3U322MV type=write size=3551688 2024-05-30T16:03:55.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.571+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=F3YZ7YV8P18P type=del size=2193539 2024-05-30T16:03:55.577 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.579+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~ZNBAFQEU5AO0.0 size=456922 2024-05-30T16:03:55.583 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.583+0000 7f08851138c0 1 < completed multipart upload key=A5BUBY9Z7JQG upload=ZNBAFQEU5AO0 tag=PZ109BDQBT3N type=write size=1505498 2024-05-30T16:03:55.587 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.587+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=719QZGSSOM8G type=del size=3072013 2024-05-30T16:03:55.590 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.591+0000 7f08851138c0 1 < completed part key=_multipart_ZALVY70ISLM5.2~L1WWSME0MY7C.0 size=1048576 2024-05-30T16:03:55.591 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.591+0000 7f08851138c0 1 < canceled part key=_multipart_ZALVY70ISLM5.2~L1WWSME0MY7C.1 size=881218 2024-05-30T16:03:55.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.595+0000 7f08851138c0 1 < canceled multipart upload key=ZALVY70ISLM5 upload=L1WWSME0MY7C tag=AH47GA5J5W3Y type=write size=2978370 2024-05-30T16:03:55.599 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.599+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=1T2JV6YJ4757 type=del size=149482 2024-05-30T16:03:55.602 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.603+0000 7f08851138c0 1 < completed delete operation key=NIV7AQ4EV5YP tag=YHCVV06J15YO type=del 2024-05-30T16:03:55.608 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.607+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~VTM5UB0TZLU4.0 size=1048576 2024-05-30T16:03:55.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.611+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~VTM5UB0TZLU4.1 size=1048576 2024-05-30T16:03:55.613 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.615+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~VTM5UB0TZLU4.2 size=123909 2024-05-30T16:03:55.613 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.615+0000 7f08851138c0 1 > prepared multipart upload key=0T2E9F3K1USF upload=VTM5UB0TZLU4 tag=ZESSK3NL7BNG type=write size=3269637 2024-05-30T16:03:55.620 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.619+0000 7f08851138c0 1 < suggested update operation key=WT9RC0UYVG4F tag=CSU6HIHEQGSO type=del size=4008607 2024-05-30T16:03:55.624 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.623+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=63TMLF4ZE3HJ type=del size=3295182 2024-05-30T16:03:55.628 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.627+0000 7f08851138c0 1 < completed part key=_multipart_ZALVY70ISLM5.2~K3SLCHJ2F8AJ.0 size=1048576 2024-05-30T16:03:55.630 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.631+0000 7f08851138c0 1 < completed part key=_multipart_ZALVY70ISLM5.2~K3SLCHJ2F8AJ.1 size=776569 2024-05-30T16:03:55.637 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.635+0000 7f08851138c0 1 < completed multipart upload key=ZALVY70ISLM5 upload=K3SLCHJ2F8AJ tag=ZQYW2ETH03EI type=write size=2873721 2024-05-30T16:03:55.640 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.639+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=DOBDWNYOJL90 type=del size=3574616 2024-05-30T16:03:55.643 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.643+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~EMTNW7MQTBQT.0 size=139013 2024-05-30T16:03:55.649 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.647+0000 7f08851138c0 1 < completed multipart upload key=364U3OV10VPD upload=EMTNW7MQTBQT tag=CG6P6XMU0AAQ type=write size=1187589 2024-05-30T16:03:55.657 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.655+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~IM67JAKU1NTP.0 size=70572 2024-05-30T16:03:55.657 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.655+0000 7f08851138c0 1 > prepared multipart upload key=WT9RC0UYVG4F upload=IM67JAKU1NTP tag=QND3WI5D1JR6 type=write size=1119148 2024-05-30T16:03:55.661 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.663+0000 7f08851138c0 1 < completed delete operation key=1N6R2HBTWMP2 tag=KMP6KBW8IO2D type=del 2024-05-30T16:03:55.665 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.663+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=L8VSC01BZV57 type=del size=2339884 2024-05-30T16:03:55.668 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.667+0000 7f08851138c0 1 < completed delete operation key=3IJWPWRB6FHE tag=PNQ5JBET9HVG type=del 2024-05-30T16:03:55.674 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.675+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~TQD99KEUHWM8.0 size=925172 2024-05-30T16:03:55.674 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.675+0000 7f08851138c0 1 > prepared multipart upload key=A5BUBY9Z7JQG upload=TQD99KEUHWM8 tag=RRRIWCHXA7MG type=write size=1973748 2024-05-30T16:03:55.677 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.675+0000 7f08851138c0 1 < completed delete operation key=QCLY10D2A3VF tag=86ZT2S9FP3RC type=del 2024-05-30T16:03:55.680 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.679+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=HJWG87UIPBET type=del size=1282090 2024-05-30T16:03:55.683 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.683+0000 7f08851138c0 1 < completed delete operation key=Y3CYZJ83WYNI tag=MQ76RXSR4SSS type=del 2024-05-30T16:03:55.687 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.687+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=MWOIYWXJBB9O type=del size=3717622 2024-05-30T16:03:55.690 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.691+0000 7f08851138c0 1 < completed write operation key=GWXNCSJCLXKW tag=69Q2V5GEDMLN type=write size=278074 2024-05-30T16:03:55.693 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.695+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=ZBOM2XKP8O4Z type=del size=1922242 2024-05-30T16:03:55.697 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.695+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~IHWH2B9IVM3A.0 size=1048576 2024-05-30T16:03:55.700 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.699+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~IHWH2B9IVM3A.1 size=1048576 2024-05-30T16:03:55.702 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.703+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~IHWH2B9IVM3A.2 size=838725 2024-05-30T16:03:55.708 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.707+0000 7f08851138c0 1 < completed multipart upload key=QQILXRJD0C1Z upload=IHWH2B9IVM3A tag=PLQ0R08JM0F0 type=write size=3984453 2024-05-30T16:03:55.712 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.711+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=X1TG2KJM37S3 type=write size=386853 2024-05-30T16:03:55.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.719+0000 7f08851138c0 1 < suggested update operation key=0T2E9F3K1USF tag=778PH4NLI31S type=write size=825951 2024-05-30T16:03:55.723 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.723+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~9SINJ2SKOFS7.0 size=693657 2024-05-30T16:03:55.724 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.723+0000 7f08851138c0 1 > prepared multipart upload key=Y3CYZJ83WYNI upload=9SINJ2SKOFS7 tag=Z0IOOTSWF3Z8 type=write size=1742233 2024-05-30T16:03:55.727 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.727+0000 7f08851138c0 1 < completed delete operation key=QCLY10D2A3VF tag=I4OWUSLKWWGE type=del 2024-05-30T16:03:55.733 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.731+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~TEX8SQTQHFNA.0 size=1048576 2024-05-30T16:03:55.735 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.735+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~TEX8SQTQHFNA.1 size=1048576 2024-05-30T16:03:55.738 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.739+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~TEX8SQTQHFNA.2 size=967239 2024-05-30T16:03:55.738 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.739+0000 7f08851138c0 1 > prepared multipart upload key=3IJWPWRB6FHE upload=TEX8SQTQHFNA tag=LOJAW6PY0STV type=write size=4112967 2024-05-30T16:03:55.741 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.739+0000 7f08851138c0 1 < completed write operation key=1N6R2HBTWMP2 tag=00H6TFK1CQU2 type=write size=347230 2024-05-30T16:03:55.744 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.743+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=H7492AMX6OZO type=del size=761557 2024-05-30T16:03:55.747 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.747+0000 7f08851138c0 1 < completed delete operation key=73FJ3HGWYQQK tag=F3YZ7YV8P18P type=del 2024-05-30T16:03:55.751 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.751+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=HDG2O0QPFN6U type=del size=3917903 2024-05-30T16:03:55.754 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.755+0000 7f08851138c0 1 < completed delete operation key=QQILXRJD0C1Z tag=719QZGSSOM8G type=del 2024-05-30T16:03:55.757 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.755+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=ECRFGB0HC3DO type=del size=3228054 2024-05-30T16:03:55.760 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.759+0000 7f08851138c0 1 < completed delete operation key=TPCETCO3Y1M6 tag=1T2JV6YJ4757 type=del 2024-05-30T16:03:55.763 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.763+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=8UC35C0WGUGS type=del size=264321 2024-05-30T16:03:55.764 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.763+0000 7f08851138c0 1 < canceled part key=_multipart_0T2E9F3K1USF.2~VTM5UB0TZLU4.0 size=1048576 2024-05-30T16:03:55.767 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.767+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~VTM5UB0TZLU4.1 size=1048576 2024-05-30T16:03:55.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.771+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~VTM5UB0TZLU4.2 size=123909 2024-05-30T16:03:55.775 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.775+0000 7f08851138c0 1 < canceled multipart upload key=0T2E9F3K1USF upload=VTM5UB0TZLU4 tag=ZESSK3NL7BNG type=write size=3269637 2024-05-30T16:03:55.781 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.779+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~K4U0XGNHJ0PP.0 size=1048576 2024-05-30T16:03:55.783 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.783+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~K4U0XGNHJ0PP.1 size=1048576 2024-05-30T16:03:55.786 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.787+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~K4U0XGNHJ0PP.2 size=813562 2024-05-30T16:03:55.786 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.787+0000 7f08851138c0 1 > prepared multipart upload key=0T2E9F3K1USF upload=K4U0XGNHJ0PP tag=O2PMJ1WM59BG type=write size=3959290 2024-05-30T16:03:55.792 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.791+0000 7f08851138c0 1 < suggested remove operation key=GWXNCSJCLXKW tag=63TMLF4ZE3HJ type=del 2024-05-30T16:03:55.795 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.795+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=TMW3BWAT3Q7N type=del size=548836 2024-05-30T16:03:55.799 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.799+0000 7f08851138c0 1 < completed delete operation key=Y3CYZJ83WYNI tag=DOBDWNYOJL90 type=del 2024-05-30T16:03:55.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.803+0000 7f08851138c0 1 > prepared operation key=VRXNO8JLAZYH tag=PE3NYX3BVX6F type=del size=1849669 2024-05-30T16:03:55.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.807+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~IM67JAKU1NTP.0 size=70572 2024-05-30T16:03:55.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.811+0000 7f08851138c0 1 < completed multipart upload key=WT9RC0UYVG4F upload=IM67JAKU1NTP tag=QND3WI5D1JR6 type=write size=1119148 2024-05-30T16:03:55.814 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.815+0000 7f08851138c0 1 > prepared operation key=ZALVY70ISLM5 tag=OA5S432AJUQL type=write size=796269 2024-05-30T16:03:55.817 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.819+0000 7f08851138c0 1 < completed delete operation key=KPHZGFJLC2E0 tag=L8VSC01BZV57 type=del 2024-05-30T16:03:55.822 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.819+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=A28QBGJSAJPM type=del size=417102 2024-05-30T16:03:55.824 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.823+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~TQD99KEUHWM8.0 size=925172 2024-05-30T16:03:55.831 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.831+0000 7f08851138c0 1 < completed multipart upload key=A5BUBY9Z7JQG upload=TQD99KEUHWM8 tag=RRRIWCHXA7MG type=write size=1973748 2024-05-30T16:03:55.838 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.839+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~U00C5HO39GQF.0 size=1048576 2024-05-30T16:03:55.840 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.839+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~U00C5HO39GQF.1 size=624827 2024-05-30T16:03:55.841 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.839+0000 7f08851138c0 1 > prepared multipart upload key=GWXNCSJCLXKW upload=U00C5HO39GQF tag=ZD854XHT4L95 type=write size=2721979 2024-05-30T16:03:55.844 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.843+0000 7f08851138c0 1 < completed delete operation key=XHGMUYBA2VNK tag=HJWG87UIPBET type=del 2024-05-30T16:03:55.849 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.847+0000 7f08851138c0 1 > prepared operation key=VRXNO8JLAZYH tag=1U63OIIPO4O4 type=write size=759098 2024-05-30T16:03:55.851 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.851+0000 7f08851138c0 1 < completed delete operation key=E2FXSXOEUGIG tag=MWOIYWXJBB9O type=del 2024-05-30T16:03:55.858 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.859+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~0Q5TO7CFPNRV.0 size=1048576 2024-05-30T16:03:55.860 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.859+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~0Q5TO7CFPNRV.1 size=1048576 2024-05-30T16:03:55.863 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.863+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~0Q5TO7CFPNRV.2 size=282502 2024-05-30T16:03:55.863 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.863+0000 7f08851138c0 1 > prepared multipart upload key=V4OQFOOQUXFQ upload=0Q5TO7CFPNRV tag=ZM0ZFKBFQTTW type=write size=3428230 2024-05-30T16:03:55.867 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.867+0000 7f08851138c0 1 < completed delete operation key=O8CWPJ9GW2GO tag=ZBOM2XKP8O4Z type=del 2024-05-30T16:03:55.876 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.875+0000 7f08851138c0 1 > prepared operation key=VRXNO8JLAZYH tag=DG37EF8F65PR type=del size=3840341 2024-05-30T16:03:55.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.879+0000 7f08851138c0 1 < completed write operation key=MMLF65KLC42Q tag=X1TG2KJM37S3 type=write size=386853 2024-05-30T16:03:55.886 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.887+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~DK105A2003Q5.0 size=1048576 2024-05-30T16:03:55.889 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.887+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~DK105A2003Q5.1 size=1048576 2024-05-30T16:03:55.891 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.891+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~DK105A2003Q5.2 size=315007 2024-05-30T16:03:55.892 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.891+0000 7f08851138c0 1 > prepared multipart upload key=RCTV94WRMO53 upload=DK105A2003Q5 tag=PNXR7BURV0YI type=write size=3460735 2024-05-30T16:03:55.895 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.895+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~9SINJ2SKOFS7.0 size=693657 2024-05-30T16:03:55.901 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.899+0000 7f08851138c0 1 < completed multipart upload key=Y3CYZJ83WYNI upload=9SINJ2SKOFS7 tag=Z0IOOTSWF3Z8 type=write size=1742233 2024-05-30T16:03:55.905 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.903+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=5N49G39E2FZX type=del size=2071383 2024-05-30T16:03:55.905 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.907+0000 7f08851138c0 1 < canceled part key=_multipart_3IJWPWRB6FHE.2~TEX8SQTQHFNA.0 size=1048576 2024-05-30T16:03:55.908 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.907+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~TEX8SQTQHFNA.1 size=1048576 2024-05-30T16:03:55.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.911+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~TEX8SQTQHFNA.2 size=967239 2024-05-30T16:03:55.917 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.915+0000 7f08851138c0 1 < canceled multipart upload key=3IJWPWRB6FHE upload=TEX8SQTQHFNA tag=LOJAW6PY0STV type=write size=4112967 2024-05-30T16:03:55.921 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.919+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=CFLY4M05WK5P type=del size=1870189 2024-05-30T16:03:55.925 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.923+0000 7f08851138c0 1 < completed delete operation key=V4OQFOOQUXFQ tag=H7492AMX6OZO type=del 2024-05-30T16:03:55.931 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.931+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~0G9M2OVZU5F1.0 size=842615 2024-05-30T16:03:55.932 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.931+0000 7f08851138c0 1 > prepared multipart upload key=WT9RC0UYVG4F upload=0G9M2OVZU5F1 tag=NHHJTVEKPT1N type=write size=1891191 2024-05-30T16:03:55.935 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.935+0000 7f08851138c0 1 < completed delete operation key=3TBOZWT2P8U8 tag=HDG2O0QPFN6U type=del 2024-05-30T16:03:55.942 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.943+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~0CZGX4G2F419.0 size=1048576 2024-05-30T16:03:55.944 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.943+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~0CZGX4G2F419.1 size=601164 2024-05-30T16:03:55.944 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.943+0000 7f08851138c0 1 > prepared multipart upload key=JNZI7JX95LX2 upload=0CZGX4G2F419 tag=HVY83JEEMOI6 type=write size=2698316 2024-05-30T16:03:55.948 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.947+0000 7f08851138c0 1 < completed delete operation key=V4OQFOOQUXFQ tag=ECRFGB0HC3DO type=del 2024-05-30T16:03:55.955 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.955+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~GE38YJZ6IW7T.0 size=326606 2024-05-30T16:03:55.955 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.955+0000 7f08851138c0 1 > prepared multipart upload key=O8CWPJ9GW2GO upload=GE38YJZ6IW7T tag=MOWBWR9UD1YU type=write size=1375182 2024-05-30T16:03:55.959 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.959+0000 7f08851138c0 1 < completed delete operation key=0RZRBW38WKNZ tag=8UC35C0WGUGS type=del 2024-05-30T16:03:55.962 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.963+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=0BWWKUMLLFHU type=del size=1132696 2024-05-30T16:03:55.966 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.967+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~K4U0XGNHJ0PP.0 size=1048576 2024-05-30T16:03:55.969 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.967+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~K4U0XGNHJ0PP.1 size=1048576 2024-05-30T16:03:55.969 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.967+0000 7f08851138c0 1 < canceled part key=_multipart_0T2E9F3K1USF.2~K4U0XGNHJ0PP.2 size=813562 2024-05-30T16:03:55.975 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.975+0000 7f08851138c0 1 < canceled multipart upload key=0T2E9F3K1USF upload=K4U0XGNHJ0PP tag=O2PMJ1WM59BG type=write size=3959290 2024-05-30T16:03:55.982 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.983+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~6IE0OJ8V1SS9.0 size=1048576 2024-05-30T16:03:55.985 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.983+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~6IE0OJ8V1SS9.1 size=444868 2024-05-30T16:03:55.985 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.983+0000 7f08851138c0 1 > prepared multipart upload key=E2FXSXOEUGIG upload=6IE0OJ8V1SS9 tag=U22V1V8ATJPH type=write size=2542020 2024-05-30T16:03:55.992 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.991+0000 7f08851138c0 1 < suggested remove operation key=PLWFSKPKVO19 tag=TMW3BWAT3Q7N type=del 2024-05-30T16:03:55.995 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.995+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=8ZBV7KH63XVV type=del size=944545 2024-05-30T16:03:55.999 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:55.999+0000 7f08851138c0 1 < completed delete operation key=VRXNO8JLAZYH tag=PE3NYX3BVX6F type=del 2024-05-30T16:03:56.003 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.003+0000 7f08851138c0 1 > prepared operation key=ZALVY70ISLM5 tag=TFIPBWYZMSKI type=del size=3210236 2024-05-30T16:03:56.007 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.007+0000 7f08851138c0 1 < completed write operation key=ZALVY70ISLM5 tag=OA5S432AJUQL type=write size=796269 2024-05-30T16:03:56.010 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.011+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=W7XFYE0RPFK1 type=write size=928939 2024-05-30T16:03:56.014 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.015+0000 7f08851138c0 1 < canceled operation key=O0DJ8SCY766V tag=A28QBGJSAJPM type=del size=417102 2024-05-30T16:03:56.022 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.019+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~2IWCE7W0BSXG.0 size=376700 2024-05-30T16:03:56.022 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.019+0000 7f08851138c0 1 > prepared multipart upload key=ZALVY70ISLM5 upload=2IWCE7W0BSXG tag=MQM92FICZIC0 type=write size=1425276 2024-05-30T16:03:56.022 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.023+0000 7f08851138c0 1 < canceled part key=_multipart_GWXNCSJCLXKW.2~U00C5HO39GQF.0 size=1048576 2024-05-30T16:03:56.026 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.023+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~U00C5HO39GQF.1 size=624827 2024-05-30T16:03:56.031 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.031+0000 7f08851138c0 1 < canceled multipart upload key=GWXNCSJCLXKW upload=U00C5HO39GQF tag=ZD854XHT4L95 type=write size=2721979 2024-05-30T16:03:56.037 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.035+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~NW2M7JCBXV87.0 size=293893 2024-05-30T16:03:56.037 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.035+0000 7f08851138c0 1 > prepared multipart upload key=A5BUBY9Z7JQG upload=NW2M7JCBXV87 tag=HA5WYIAGXUPS type=write size=1342469 2024-05-30T16:03:56.041 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.039+0000 7f08851138c0 1 < completed write operation key=VRXNO8JLAZYH tag=1U63OIIPO4O4 type=write size=759098 2024-05-30T16:03:56.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.043+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=AIL3XRI2ZNMT type=del size=2022889 2024-05-30T16:03:56.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.043+0000 7f08851138c0 1 < canceled part key=_multipart_V4OQFOOQUXFQ.2~0Q5TO7CFPNRV.0 size=1048576 2024-05-30T16:03:56.047 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.047+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~0Q5TO7CFPNRV.1 size=1048576 2024-05-30T16:03:56.051 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.051+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~0Q5TO7CFPNRV.2 size=282502 2024-05-30T16:03:56.056 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.055+0000 7f08851138c0 1 < canceled multipart upload key=V4OQFOOQUXFQ upload=0Q5TO7CFPNRV tag=ZM0ZFKBFQTTW type=write size=3428230 2024-05-30T16:03:56.060 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.059+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=R7FCLLN5615N type=del size=1929251 2024-05-30T16:03:56.063 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.063+0000 7f08851138c0 1 < completed delete operation key=VRXNO8JLAZYH tag=DG37EF8F65PR type=del 2024-05-30T16:03:56.067 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.067+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=N63DX8U5MIO5 type=del size=1317713 2024-05-30T16:03:56.071 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.071+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~DK105A2003Q5.0 size=1048576 2024-05-30T16:03:56.073 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.071+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~DK105A2003Q5.1 size=1048576 2024-05-30T16:03:56.076 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.075+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~DK105A2003Q5.2 size=315007 2024-05-30T16:03:56.083 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.083+0000 7f08851138c0 1 < completed multipart upload key=RCTV94WRMO53 upload=DK105A2003Q5 tag=PNXR7BURV0YI type=write size=3460735 2024-05-30T16:03:56.086 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.087+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=GIFIWVA0932W type=del size=106189 2024-05-30T16:03:56.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.091+0000 7f08851138c0 1 < completed delete operation key=KPHZGFJLC2E0 tag=5N49G39E2FZX type=del 2024-05-30T16:03:56.093 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.091+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=1CZ3XGWWZFTG type=del size=3666257 2024-05-30T16:03:56.097 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.095+0000 7f08851138c0 1 < completed delete operation key=3IJWPWRB6FHE tag=CFLY4M05WK5P type=del 2024-05-30T16:03:56.101 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.099+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=RL18V441OD9S type=del size=3223617 2024-05-30T16:03:56.104 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.103+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~0G9M2OVZU5F1.0 size=842615 2024-05-30T16:03:56.110 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.111+0000 7f08851138c0 1 < completed multipart upload key=WT9RC0UYVG4F upload=0G9M2OVZU5F1 tag=NHHJTVEKPT1N type=write size=1891191 2024-05-30T16:03:56.113 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.115+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=1A9A7YPKZBOP type=del size=291508 2024-05-30T16:03:56.117 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.115+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~0CZGX4G2F419.0 size=1048576 2024-05-30T16:03:56.120 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.119+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~0CZGX4G2F419.1 size=601164 2024-05-30T16:03:56.126 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.127+0000 7f08851138c0 1 < completed multipart upload key=JNZI7JX95LX2 upload=0CZGX4G2F419 tag=HVY83JEEMOI6 type=write size=2698316 2024-05-30T16:03:56.132 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.131+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~DW4F2JKLP93D.0 size=759234 2024-05-30T16:03:56.133 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.131+0000 7f08851138c0 1 > prepared multipart upload key=O8CWPJ9GW2GO upload=DW4F2JKLP93D tag=OXZEWKKEO9NO type=write size=1807810 2024-05-30T16:03:56.136 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.135+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~GE38YJZ6IW7T.0 size=326606 2024-05-30T16:03:56.142 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.143+0000 7f08851138c0 1 < completed multipart upload key=O8CWPJ9GW2GO upload=GE38YJZ6IW7T tag=MOWBWR9UD1YU type=write size=1375182 2024-05-30T16:03:56.146 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.143+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=PBY7LNQ48HCV type=del size=1247324 2024-05-30T16:03:56.150 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.147+0000 7f08851138c0 1 < completed delete operation key=XHGMUYBA2VNK tag=0BWWKUMLLFHU type=del 2024-05-30T16:03:56.153 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.151+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=C5MP2WW7XC5D type=del size=3745715 2024-05-30T16:03:56.158 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.159+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~6IE0OJ8V1SS9.0 size=1048576 2024-05-30T16:03:56.162 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.159+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~6IE0OJ8V1SS9.1 size=444868 2024-05-30T16:03:56.168 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.167+0000 7f08851138c0 1 < completed multipart upload key=E2FXSXOEUGIG upload=6IE0OJ8V1SS9 tag=U22V1V8ATJPH type=write size=2542020 2024-05-30T16:03:56.174 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.175+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~9N8R9Y3PZYEW.0 size=137358 2024-05-30T16:03:56.174 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.175+0000 7f08851138c0 1 > prepared multipart upload key=TPCETCO3Y1M6 upload=9N8R9Y3PZYEW tag=3O44VA7W1DVJ type=write size=1185934 2024-05-30T16:03:56.179 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.179+0000 7f08851138c0 1 < completed delete operation key=RCTV94WRMO53 tag=8ZBV7KH63XVV type=del 2024-05-30T16:03:56.183 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.183+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=26UVL6ULZGD5 type=del size=1180654 2024-05-30T16:03:56.187 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.187+0000 7f08851138c0 1 < completed delete operation key=ZALVY70ISLM5 tag=TFIPBWYZMSKI type=del 2024-05-30T16:03:56.190 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.191+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=NG2L88J0KG9X type=del size=3476259 2024-05-30T16:03:56.194 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.195+0000 7f08851138c0 1 < completed write operation key=OKBB0VU765TW tag=W7XFYE0RPFK1 type=write size=928939 2024-05-30T16:03:56.202 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.203+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~9YXVW01DHCBB.0 size=986432 2024-05-30T16:03:56.202 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.203+0000 7f08851138c0 1 > prepared multipart upload key=QCLY10D2A3VF upload=9YXVW01DHCBB tag=ITENCZA8GZQH type=write size=2035008 2024-05-30T16:03:56.206 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.207+0000 7f08851138c0 1 < completed part key=_multipart_ZALVY70ISLM5.2~2IWCE7W0BSXG.0 size=376700 2024-05-30T16:03:56.212 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.211+0000 7f08851138c0 1 < completed multipart upload key=ZALVY70ISLM5 upload=2IWCE7W0BSXG tag=MQM92FICZIC0 type=write size=1425276 2024-05-30T16:03:56.224 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.223+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~MFFTQF8EFH5F.0 size=300110 2024-05-30T16:03:56.224 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.223+0000 7f08851138c0 1 > prepared multipart upload key=ZALVY70ISLM5 upload=MFFTQF8EFH5F tag=OC69LWYYMM73 type=write size=1348686 2024-05-30T16:03:56.227 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.227+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~NW2M7JCBXV87.0 size=293893 2024-05-30T16:03:56.236 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.235+0000 7f08851138c0 1 < completed multipart upload key=A5BUBY9Z7JQG upload=NW2M7JCBXV87 tag=HA5WYIAGXUPS type=write size=1342469 2024-05-30T16:03:56.238 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.239+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=XUIX3D1LFE7L type=del size=1563513 2024-05-30T16:03:56.242 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.243+0000 7f08851138c0 1 < completed delete operation key=1N6R2HBTWMP2 tag=AIL3XRI2ZNMT type=del 2024-05-30T16:03:56.245 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.247+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=L9WA8GM8EJYJ type=del size=3286721 2024-05-30T16:03:56.249 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.247+0000 7f08851138c0 1 < completed delete operation key=364U3OV10VPD tag=R7FCLLN5615N type=del 2024-05-30T16:03:56.256 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.255+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~AK80ODCX3QT1.0 size=1048576 2024-05-30T16:03:56.259 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.259+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~AK80ODCX3QT1.1 size=1048576 2024-05-30T16:03:56.261 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.263+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~AK80ODCX3QT1.2 size=353618 2024-05-30T16:03:56.262 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.263+0000 7f08851138c0 1 > prepared multipart upload key=NIV7AQ4EV5YP upload=AK80ODCX3QT1 tag=UP5SAOK3SJA0 type=write size=3499346 2024-05-30T16:03:56.266 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.267+0000 7f08851138c0 1 < completed delete operation key=KPHZGFJLC2E0 tag=N63DX8U5MIO5 type=del 2024-05-30T16:03:56.273 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.271+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~UDFGKWN0KEZ7.0 size=510216 2024-05-30T16:03:56.273 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.271+0000 7f08851138c0 1 > prepared multipart upload key=ZALVY70ISLM5 upload=UDFGKWN0KEZ7 tag=PJNUIJEZG809 type=write size=1558792 2024-05-30T16:03:56.277 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.275+0000 7f08851138c0 1 < canceled operation key=GWXNCSJCLXKW tag=GIFIWVA0932W type=del size=106189 2024-05-30T16:03:56.282 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.279+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=RZ01MPCB334S type=del size=2688235 2024-05-30T16:03:56.286 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.287+0000 7f08851138c0 1 < completed delete operation key=V4OQFOOQUXFQ tag=1CZ3XGWWZFTG type=del 2024-05-30T16:03:56.292 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.291+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~A579ZGJU360H.0 size=591713 2024-05-30T16:03:56.292 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.291+0000 7f08851138c0 1 > prepared multipart upload key=73FJ3HGWYQQK upload=A579ZGJU360H tag=X1DPIISRTGV0 type=write size=1640289 2024-05-30T16:03:56.296 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.295+0000 7f08851138c0 1 < completed delete operation key=Y8SBI0QZ0ZV8 tag=RL18V441OD9S type=del 2024-05-30T16:03:56.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.299+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=E6D7N2BEBEJ9 type=del size=1833892 2024-05-30T16:03:56.308 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.307+0000 7f08851138c0 1 < suggested remove operation key=QCLY10D2A3VF tag=1A9A7YPKZBOP type=del 2024-05-30T16:03:56.312 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.311+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=440VZJUCAXFB type=del size=1195858 2024-05-30T16:03:56.316 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.315+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~DW4F2JKLP93D.0 size=759234 2024-05-30T16:03:56.321 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.319+0000 7f08851138c0 1 < completed multipart upload key=O8CWPJ9GW2GO upload=DW4F2JKLP93D tag=OXZEWKKEO9NO type=write size=1807810 2024-05-30T16:03:56.328 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.327+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~I0Y1BOPCW6BX.0 size=761542 2024-05-30T16:03:56.328 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.327+0000 7f08851138c0 1 > prepared multipart upload key=JNZI7JX95LX2 upload=I0Y1BOPCW6BX tag=7LWFIFBZPEHI type=write size=1810118 2024-05-30T16:03:56.335 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.335+0000 7f08851138c0 1 < suggested update operation key=NIV7AQ4EV5YP tag=PBY7LNQ48HCV type=del size=1247324 2024-05-30T16:03:56.338 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.339+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=623JYB2D99JN type=del size=1578074 2024-05-30T16:03:56.345 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.343+0000 7f08851138c0 1 < suggested update operation key=0RZRBW38WKNZ tag=C5MP2WW7XC5D type=del size=3745715 2024-05-30T16:03:56.352 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.351+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~B9W3VE5BKXHN.0 size=1048576 2024-05-30T16:03:56.354 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.355+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~B9W3VE5BKXHN.1 size=1048576 2024-05-30T16:03:56.357 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.355+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~B9W3VE5BKXHN.2 size=13139 2024-05-30T16:03:56.357 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.355+0000 7f08851138c0 1 > prepared multipart upload key=XHGMUYBA2VNK upload=B9W3VE5BKXHN tag=740AXCDVS7HX type=write size=3158867 2024-05-30T16:03:56.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.359+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~9N8R9Y3PZYEW.0 size=137358 2024-05-30T16:03:56.365 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.367+0000 7f08851138c0 1 < completed multipart upload key=TPCETCO3Y1M6 upload=9N8R9Y3PZYEW tag=3O44VA7W1DVJ type=write size=1185934 2024-05-30T16:03:56.370 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.367+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=JH1ROSB21A9F type=del size=3209563 2024-05-30T16:03:56.373 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.371+0000 7f08851138c0 1 < completed delete operation key=WT9RC0UYVG4F tag=26UVL6ULZGD5 type=del 2024-05-30T16:03:56.379 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.379+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~8L5JUL7QYF0F.0 size=1048576 2024-05-30T16:03:56.382 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.383+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~8L5JUL7QYF0F.1 size=1048576 2024-05-30T16:03:56.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.383+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~8L5JUL7QYF0F.2 size=316138 2024-05-30T16:03:56.385 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.383+0000 7f08851138c0 1 > prepared multipart upload key=PLWFSKPKVO19 upload=8L5JUL7QYF0F tag=XF46BLLC1TEQ type=write size=3461866 2024-05-30T16:03:56.388 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.387+0000 7f08851138c0 1 < completed delete operation key=YRP4CSG92QRO tag=NG2L88J0KG9X type=del 2024-05-30T16:03:56.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.395+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~4HG1YZQ8HRFH.0 size=1048576 2024-05-30T16:03:56.397 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.399+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~4HG1YZQ8HRFH.1 size=1048576 2024-05-30T16:03:56.400 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.399+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~4HG1YZQ8HRFH.2 size=230392 2024-05-30T16:03:56.400 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.399+0000 7f08851138c0 1 > prepared multipart upload key=4N58ZY4E1H6E upload=4HG1YZQ8HRFH tag=U6DWH0PW1MYN type=write size=3376120 2024-05-30T16:03:56.401 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.399+0000 7f08851138c0 1 < canceled part key=_multipart_QCLY10D2A3VF.2~9YXVW01DHCBB.0 size=986432 2024-05-30T16:03:56.407 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.407+0000 7f08851138c0 1 < canceled multipart upload key=QCLY10D2A3VF upload=9YXVW01DHCBB tag=ITENCZA8GZQH type=write size=2035008 2024-05-30T16:03:56.413 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.411+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~HMIEJFRJWVC1.0 size=1048576 2024-05-30T16:03:56.416 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.415+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~HMIEJFRJWVC1.1 size=672914 2024-05-30T16:03:56.416 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.415+0000 7f08851138c0 1 > prepared multipart upload key=364U3OV10VPD upload=HMIEJFRJWVC1 tag=JNF53MDA4NDM type=write size=2770066 2024-05-30T16:03:56.419 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.419+0000 7f08851138c0 1 < completed part key=_multipart_ZALVY70ISLM5.2~MFFTQF8EFH5F.0 size=300110 2024-05-30T16:03:56.425 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.423+0000 7f08851138c0 1 < completed multipart upload key=ZALVY70ISLM5 upload=MFFTQF8EFH5F tag=OC69LWYYMM73 type=write size=1348686 2024-05-30T16:03:56.431 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.431+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~NLBK8O1QVRZ5.0 size=1048576 2024-05-30T16:03:56.439 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.435+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~NLBK8O1QVRZ5.1 size=141021 2024-05-30T16:03:56.439 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.435+0000 7f08851138c0 1 > prepared multipart upload key=QCLY10D2A3VF upload=NLBK8O1QVRZ5 tag=YLCW8VSWM9V7 type=write size=2238173 2024-05-30T16:03:56.439 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.435+0000 7f08851138c0 1 < canceled operation key=RCTV94WRMO53 tag=XUIX3D1LFE7L type=del size=1563513 2024-05-30T16:03:56.441 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.439+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=QRCZEGS8NM3D type=del size=3254867 2024-05-30T16:03:56.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.447+0000 7f08851138c0 1 < suggested update operation key=364U3OV10VPD tag=L9WA8GM8EJYJ type=del size=3286721 2024-05-30T16:03:56.451 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.451+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=2Q247HB08B5M type=del size=3642955 2024-05-30T16:03:56.455 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.455+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~AK80ODCX3QT1.0 size=1048576 2024-05-30T16:03:56.458 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.459+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~AK80ODCX3QT1.1 size=1048576 2024-05-30T16:03:56.461 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.459+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~AK80ODCX3QT1.2 size=353618 2024-05-30T16:03:56.466 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.467+0000 7f08851138c0 1 < completed multipart upload key=NIV7AQ4EV5YP upload=AK80ODCX3QT1 tag=UP5SAOK3SJA0 type=write size=3499346 2024-05-30T16:03:56.470 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.471+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=9C4RWFRU8YL2 type=del size=47408 2024-05-30T16:03:56.473 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.475+0000 7f08851138c0 1 < completed part key=_multipart_ZALVY70ISLM5.2~UDFGKWN0KEZ7.0 size=510216 2024-05-30T16:03:56.479 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.479+0000 7f08851138c0 1 < completed multipart upload key=ZALVY70ISLM5 upload=UDFGKWN0KEZ7 tag=PJNUIJEZG809 type=write size=1558792 2024-05-30T16:03:56.483 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.483+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=LCN0IT8IIXBY type=del size=2512720 2024-05-30T16:03:56.486 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.487+0000 7f08851138c0 1 < canceled operation key=E2FXSXOEUGIG tag=RZ01MPCB334S type=del size=2688235 2024-05-30T16:03:56.492 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.491+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~L0KAYBRVSTK9.0 size=1048576 2024-05-30T16:03:56.496 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.495+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~L0KAYBRVSTK9.1 size=513445 2024-05-30T16:03:56.496 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.495+0000 7f08851138c0 1 > prepared multipart upload key=VRXNO8JLAZYH upload=L0KAYBRVSTK9 tag=OWXG88QEN30U type=write size=2610597 2024-05-30T16:03:56.499 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.499+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~A579ZGJU360H.0 size=591713 2024-05-30T16:03:56.506 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.507+0000 7f08851138c0 1 < completed multipart upload key=73FJ3HGWYQQK upload=A579ZGJU360H tag=X1DPIISRTGV0 type=write size=1640289 2024-05-30T16:03:56.508 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.507+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=SVRH9JHBL4AB type=del size=3921805 2024-05-30T16:03:56.513 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.511+0000 7f08851138c0 1 < completed delete operation key=PLWFSKPKVO19 tag=E6D7N2BEBEJ9 type=del 2024-05-30T16:03:56.517 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.515+0000 7f08851138c0 1 > prepared operation key=ZALVY70ISLM5 tag=AYEKGYDJEU9B type=del size=1847581 2024-05-30T16:03:56.521 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.519+0000 7f08851138c0 1 < completed delete operation key=PLWFSKPKVO19 tag=440VZJUCAXFB type=del 2024-05-30T16:03:56.527 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.527+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~P5OZ6QTGKMQ2.0 size=162425 2024-05-30T16:03:56.527 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.527+0000 7f08851138c0 1 > prepared multipart upload key=OKBB0VU765TW upload=P5OZ6QTGKMQ2 tag=J5HL9WBFVMEB type=write size=1211001 2024-05-30T16:03:56.530 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.531+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~I0Y1BOPCW6BX.0 size=761542 2024-05-30T16:03:56.538 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.539+0000 7f08851138c0 1 < completed multipart upload key=JNZI7JX95LX2 upload=I0Y1BOPCW6BX tag=7LWFIFBZPEHI type=write size=1810118 2024-05-30T16:03:56.541 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.539+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=QJET9SXLA000 type=del size=2917800 2024-05-30T16:03:56.544 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.543+0000 7f08851138c0 1 < canceled operation key=603HBR8PRH6L tag=623JYB2D99JN type=del size=1578074 2024-05-30T16:03:56.548 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.547+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=JIAK5Q61UV2N type=del size=3121591 2024-05-30T16:03:56.548 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.547+0000 7f08851138c0 1 < canceled part key=_multipart_XHGMUYBA2VNK.2~B9W3VE5BKXHN.0 size=1048576 2024-05-30T16:03:56.551 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.551+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~B9W3VE5BKXHN.1 size=1048576 2024-05-30T16:03:56.552 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.551+0000 7f08851138c0 1 < canceled part key=_multipart_XHGMUYBA2VNK.2~B9W3VE5BKXHN.2 size=13139 2024-05-30T16:03:56.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.563+0000 7f08851138c0 1 < canceled multipart upload key=XHGMUYBA2VNK upload=B9W3VE5BKXHN tag=740AXCDVS7HX type=write size=3158867 2024-05-30T16:03:56.565 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.563+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=RVQR3PQTBMT9 type=del size=2637222 2024-05-30T16:03:56.568 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.567+0000 7f08851138c0 1 < completed delete operation key=QCLY10D2A3VF tag=JH1ROSB21A9F type=del 2024-05-30T16:03:56.572 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.571+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=I1YM4FQ27OKU type=del size=416917 2024-05-30T16:03:56.575 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.575+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~8L5JUL7QYF0F.0 size=1048576 2024-05-30T16:03:56.578 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.579+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~8L5JUL7QYF0F.1 size=1048576 2024-05-30T16:03:56.581 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.583+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~8L5JUL7QYF0F.2 size=316138 2024-05-30T16:03:56.587 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.587+0000 7f08851138c0 1 < completed multipart upload key=PLWFSKPKVO19 upload=8L5JUL7QYF0F tag=XF46BLLC1TEQ type=write size=3461866 2024-05-30T16:03:56.594 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.595+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~6ER6WO5Q2OFO.0 size=1048576 2024-05-30T16:03:56.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.595+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~6ER6WO5Q2OFO.1 size=900443 2024-05-30T16:03:56.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.595+0000 7f08851138c0 1 > prepared multipart upload key=Y8SBI0QZ0ZV8 upload=6ER6WO5Q2OFO tag=4JH2WS201G3P type=write size=2997595 2024-05-30T16:03:56.599 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.599+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~4HG1YZQ8HRFH.0 size=1048576 2024-05-30T16:03:56.602 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.603+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~4HG1YZQ8HRFH.1 size=1048576 2024-05-30T16:03:56.606 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.607+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~4HG1YZQ8HRFH.2 size=230392 2024-05-30T16:03:56.612 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.611+0000 7f08851138c0 1 < completed multipart upload key=4N58ZY4E1H6E upload=4HG1YZQ8HRFH tag=U6DWH0PW1MYN type=write size=3376120 2024-05-30T16:03:56.616 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.615+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=62N4IVSPLQDB type=del size=3275322 2024-05-30T16:03:56.619 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.619+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~HMIEJFRJWVC1.0 size=1048576 2024-05-30T16:03:56.622 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.623+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~HMIEJFRJWVC1.1 size=672914 2024-05-30T16:03:56.627 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.627+0000 7f08851138c0 1 < completed multipart upload key=364U3OV10VPD upload=HMIEJFRJWVC1 tag=JNF53MDA4NDM type=write size=2770066 2024-05-30T16:03:56.633 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.631+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~ZEEXDAQTE4XQ.0 size=1048576 2024-05-30T16:03:56.635 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.635+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~ZEEXDAQTE4XQ.1 size=301997 2024-05-30T16:03:56.635 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.635+0000 7f08851138c0 1 > prepared multipart upload key=364U3OV10VPD upload=ZEEXDAQTE4XQ tag=11M18F0742II type=write size=2399149 2024-05-30T16:03:56.638 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.639+0000 7f08851138c0 1 < completed part key=_multipart_QCLY10D2A3VF.2~NLBK8O1QVRZ5.0 size=1048576 2024-05-30T16:03:56.641 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.639+0000 7f08851138c0 1 < completed part key=_multipart_QCLY10D2A3VF.2~NLBK8O1QVRZ5.1 size=141021 2024-05-30T16:03:56.647 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.647+0000 7f08851138c0 1 < completed multipart upload key=QCLY10D2A3VF upload=NLBK8O1QVRZ5 tag=YLCW8VSWM9V7 type=write size=2238173 2024-05-30T16:03:56.650 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.651+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=UM2NHR666HT3 type=del size=3806290 2024-05-30T16:03:56.653 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.651+0000 7f08851138c0 1 < completed delete operation key=JNZI7JX95LX2 tag=QRCZEGS8NM3D type=del 2024-05-30T16:03:56.658 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.655+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=9E05UUJ2VKN7 type=del size=569292 2024-05-30T16:03:56.661 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.659+0000 7f08851138c0 1 < completed delete operation key=RCTV94WRMO53 tag=2Q247HB08B5M type=del 2024-05-30T16:03:56.668 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.667+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~HO0BNU5V6TXU.0 size=1048576 2024-05-30T16:03:56.670 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.671+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~HO0BNU5V6TXU.1 size=1048576 2024-05-30T16:03:56.672 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.671+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~HO0BNU5V6TXU.2 size=763702 2024-05-30T16:03:56.672 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.671+0000 7f08851138c0 1 > prepared multipart upload key=Y8SBI0QZ0ZV8 upload=HO0BNU5V6TXU tag=GYXRLNW0QHUH type=write size=3909430 2024-05-30T16:03:56.675 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.675+0000 7f08851138c0 1 < completed delete operation key=JNZI7JX95LX2 tag=9C4RWFRU8YL2 type=del 2024-05-30T16:03:56.678 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.679+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=JE6WZNZ0FJE8 type=del size=3883357 2024-05-30T16:03:56.681 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.679+0000 7f08851138c0 1 < completed delete operation key=0RZRBW38WKNZ tag=LCN0IT8IIXBY type=del 2024-05-30T16:03:56.683 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.683+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=AU3UT9TYOFIV type=del size=182547 2024-05-30T16:03:56.684 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.683+0000 7f08851138c0 1 < canceled part key=_multipart_VRXNO8JLAZYH.2~L0KAYBRVSTK9.0 size=1048576 2024-05-30T16:03:56.686 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.687+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~L0KAYBRVSTK9.1 size=513445 2024-05-30T16:03:56.691 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.691+0000 7f08851138c0 1 < canceled multipart upload key=VRXNO8JLAZYH upload=L0KAYBRVSTK9 tag=OWXG88QEN30U type=write size=2610597 2024-05-30T16:03:56.694 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.695+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=ULKQ46NBPCHU type=del size=1895194 2024-05-30T16:03:56.696 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.695+0000 7f08851138c0 1 < completed delete operation key=0RZRBW38WKNZ tag=SVRH9JHBL4AB type=del 2024-05-30T16:03:56.701 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.699+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~9LDBJAKQST3C.0 size=1048576 2024-05-30T16:03:56.703 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.703+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~9LDBJAKQST3C.1 size=110869 2024-05-30T16:03:56.703 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.703+0000 7f08851138c0 1 > prepared multipart upload key=KPHZGFJLC2E0 upload=9LDBJAKQST3C tag=WHKUIVHN03QY type=write size=2208021 2024-05-30T16:03:56.706 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.707+0000 7f08851138c0 1 < completed delete operation key=ZALVY70ISLM5 tag=AYEKGYDJEU9B type=del 2024-05-30T16:03:56.708 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.707+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=NHVYN3SR8KWS type=del size=3607252 2024-05-30T16:03:56.711 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.711+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~P5OZ6QTGKMQ2.0 size=162425 2024-05-30T16:03:56.716 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.715+0000 7f08851138c0 1 < completed multipart upload key=OKBB0VU765TW upload=P5OZ6QTGKMQ2 tag=J5HL9WBFVMEB type=write size=1211001 2024-05-30T16:03:56.722 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.723+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~1SY8B3X98U5R.0 size=1048576 2024-05-30T16:03:56.724 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.723+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~1SY8B3X98U5R.1 size=1048576 2024-05-30T16:03:56.726 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.727+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~1SY8B3X98U5R.2 size=514388 2024-05-30T16:03:56.726 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.727+0000 7f08851138c0 1 > prepared multipart upload key=JNZI7JX95LX2 upload=1SY8B3X98U5R tag=RJ6JT7Q9Z9VE type=write size=3660116 2024-05-30T16:03:56.729 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.731+0000 7f08851138c0 1 < completed delete operation key=MMLF65KLC42Q tag=QJET9SXLA000 type=del 2024-05-30T16:03:56.735 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.735+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~XSQGATFUW7Y0.0 size=907071 2024-05-30T16:03:56.736 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.735+0000 7f08851138c0 1 > prepared multipart upload key=OKBB0VU765TW upload=XSQGATFUW7Y0 tag=RWZ4K1Z961P0 type=write size=1955647 2024-05-30T16:03:56.741 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.739+0000 7f08851138c0 1 < suggested update operation key=O0DJ8SCY766V tag=JIAK5Q61UV2N type=del size=3121591 2024-05-30T16:03:56.744 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.743+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=MOD8PSAB7S19 type=del size=3000758 2024-05-30T16:03:56.746 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.747+0000 7f08851138c0 1 < completed delete operation key=Y3CYZJ83WYNI tag=RVQR3PQTBMT9 type=del 2024-05-30T16:03:56.749 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.751+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=JAZXIBM1DP15 type=write size=968223 2024-05-30T16:03:56.756 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.755+0000 7f08851138c0 1 < suggested update operation key=KPHZGFJLC2E0 tag=I1YM4FQ27OKU type=del size=416917 2024-05-30T16:03:56.761 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.763+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~AMJ8825V8KCN.0 size=1048576 2024-05-30T16:03:56.764 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.763+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~AMJ8825V8KCN.1 size=1048576 2024-05-30T16:03:56.766 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.767+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~AMJ8825V8KCN.2 size=794776 2024-05-30T16:03:56.766 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.767+0000 7f08851138c0 1 > prepared multipart upload key=QCLY10D2A3VF upload=AMJ8825V8KCN tag=22CVQH69MJ0W type=write size=3940504 2024-05-30T16:03:56.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.771+0000 7f08851138c0 1 < completed part key=_multipart_Y8SBI0QZ0ZV8.2~6ER6WO5Q2OFO.0 size=1048576 2024-05-30T16:03:56.772 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.771+0000 7f08851138c0 1 < completed part key=_multipart_Y8SBI0QZ0ZV8.2~6ER6WO5Q2OFO.1 size=900443 2024-05-30T16:03:56.776 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.775+0000 7f08851138c0 1 < completed multipart upload key=Y8SBI0QZ0ZV8 upload=6ER6WO5Q2OFO tag=4JH2WS201G3P type=write size=2997595 2024-05-30T16:03:56.779 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.779+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=WMN9U5EYK9FQ type=del size=168284 2024-05-30T16:03:56.782 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.783+0000 7f08851138c0 1 < canceled operation key=JNZI7JX95LX2 tag=62N4IVSPLQDB type=del size=3275322 2024-05-30T16:03:56.785 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.787+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=GD2ZR64COMVK type=del size=2386190 2024-05-30T16:03:56.786 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.787+0000 7f08851138c0 1 < canceled part key=_multipart_364U3OV10VPD.2~ZEEXDAQTE4XQ.0 size=1048576 2024-05-30T16:03:56.786 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.787+0000 7f08851138c0 1 < canceled part key=_multipart_364U3OV10VPD.2~ZEEXDAQTE4XQ.1 size=301997 2024-05-30T16:03:56.791 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.791+0000 7f08851138c0 1 < canceled multipart upload key=364U3OV10VPD upload=ZEEXDAQTE4XQ tag=11M18F0742II type=write size=2399149 2024-05-30T16:03:56.793 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.795+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=L9Y544RHLLOJ type=del size=3248381 2024-05-30T16:03:56.799 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.799+0000 7f08851138c0 1 < suggested update operation key=GWXNCSJCLXKW tag=UM2NHR666HT3 type=del size=3806290 2024-05-30T16:03:56.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.803+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~QGCIYHF19E9I.0 size=79230 2024-05-30T16:03:56.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.803+0000 7f08851138c0 1 > prepared multipart upload key=0RZRBW38WKNZ upload=QGCIYHF19E9I tag=LESER9DPZQLY type=write size=1127806 2024-05-30T16:03:56.808 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.807+0000 7f08851138c0 1 < completed delete operation key=Y8SBI0QZ0ZV8 tag=9E05UUJ2VKN7 type=del 2024-05-30T16:03:56.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.811+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=OPBHVYF2OPPY type=del size=2294581 2024-05-30T16:03:56.814 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.815+0000 7f08851138c0 1 < completed part key=_multipart_Y8SBI0QZ0ZV8.2~HO0BNU5V6TXU.0 size=1048576 2024-05-30T16:03:56.817 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.815+0000 7f08851138c0 1 < completed part key=_multipart_Y8SBI0QZ0ZV8.2~HO0BNU5V6TXU.1 size=1048576 2024-05-30T16:03:56.817 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.815+0000 7f08851138c0 1 < canceled part key=_multipart_Y8SBI0QZ0ZV8.2~HO0BNU5V6TXU.2 size=763702 2024-05-30T16:03:56.821 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.823+0000 7f08851138c0 1 < canceled multipart upload key=Y8SBI0QZ0ZV8 upload=HO0BNU5V6TXU tag=GYXRLNW0QHUH type=write size=3909430 2024-05-30T16:03:56.827 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.827+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~CD89DQGO3C2K.0 size=580675 2024-05-30T16:03:56.827 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.827+0000 7f08851138c0 1 > prepared multipart upload key=VRXNO8JLAZYH upload=CD89DQGO3C2K tag=9XFLBP1ZTSQZ type=write size=1629251 2024-05-30T16:03:56.830 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.831+0000 7f08851138c0 1 < completed delete operation key=4N58ZY4E1H6E tag=JE6WZNZ0FJE8 type=del 2024-05-30T16:03:56.835 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.835+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~QT9IJ8GK16E4.0 size=339030 2024-05-30T16:03:56.835 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.835+0000 7f08851138c0 1 > prepared multipart upload key=364U3OV10VPD upload=QT9IJ8GK16E4 tag=A6EAE1CQ27Q3 type=write size=1387606 2024-05-30T16:03:56.838 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.839+0000 7f08851138c0 1 < completed delete operation key=TPCETCO3Y1M6 tag=AU3UT9TYOFIV type=del 2024-05-30T16:03:56.841 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.839+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=PNLUQ0NTPAEQ type=del size=4048325 2024-05-30T16:03:56.847 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.847+0000 7f08851138c0 1 < completed delete operation key=73FJ3HGWYQQK tag=ULKQ46NBPCHU type=del 2024-05-30T16:03:56.850 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.851+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=72EK4NNF0T6T type=del size=919778 2024-05-30T16:03:56.854 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.855+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~9LDBJAKQST3C.0 size=1048576 2024-05-30T16:03:56.856 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.855+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~9LDBJAKQST3C.1 size=110869 2024-05-30T16:03:56.861 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.859+0000 7f08851138c0 1 < completed multipart upload key=KPHZGFJLC2E0 upload=9LDBJAKQST3C tag=WHKUIVHN03QY type=write size=2208021 2024-05-30T16:03:56.864 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.863+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=B1V4NFGI1L28 type=del size=159677 2024-05-30T16:03:56.868 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.867+0000 7f08851138c0 1 < completed delete operation key=603HBR8PRH6L tag=NHVYN3SR8KWS type=del 2024-05-30T16:03:56.871 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.871+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=SPFTM09CQRQ3 type=del size=2061540 2024-05-30T16:03:56.874 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.875+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~1SY8B3X98U5R.0 size=1048576 2024-05-30T16:03:56.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.875+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~1SY8B3X98U5R.1 size=1048576 2024-05-30T16:03:56.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.879+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~1SY8B3X98U5R.2 size=514388 2024-05-30T16:03:56.885 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.887+0000 7f08851138c0 1 < completed multipart upload key=JNZI7JX95LX2 upload=1SY8B3X98U5R tag=RJ6JT7Q9Z9VE type=write size=3660116 2024-05-30T16:03:56.889 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.887+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=1410TS63N9I0 type=del size=3858297 2024-05-30T16:03:56.892 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.891+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~XSQGATFUW7Y0.0 size=907071 2024-05-30T16:03:56.898 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.899+0000 7f08851138c0 1 < completed multipart upload key=OKBB0VU765TW upload=XSQGATFUW7Y0 tag=RWZ4K1Z961P0 type=write size=1955647 2024-05-30T16:03:56.902 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.903+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=WCUT9V8XZ9YR type=del size=2828774 2024-05-30T16:03:56.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.907+0000 7f08851138c0 1 < canceled operation key=NIV7AQ4EV5YP tag=MOD8PSAB7S19 type=del size=3000758 2024-05-30T16:03:56.912 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.911+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~UTKHLUGUKSMI.0 size=1048576 2024-05-30T16:03:56.916 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.915+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~UTKHLUGUKSMI.1 size=1048576 2024-05-30T16:03:56.918 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.919+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~UTKHLUGUKSMI.2 size=644567 2024-05-30T16:03:56.918 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.919+0000 7f08851138c0 1 > prepared multipart upload key=603HBR8PRH6L upload=UTKHLUGUKSMI tag=390BLJOKWOJ5 type=write size=3790295 2024-05-30T16:03:56.922 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.923+0000 7f08851138c0 1 < completed write operation key=3IJWPWRB6FHE tag=JAZXIBM1DP15 type=write size=968223 2024-05-30T16:03:56.926 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.927+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=5EP1J5GSJINB type=del size=1411531 2024-05-30T16:03:56.927 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.927+0000 7f08851138c0 1 < canceled part key=_multipart_QCLY10D2A3VF.2~AMJ8825V8KCN.0 size=1048576 2024-05-30T16:03:56.930 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.931+0000 7f08851138c0 1 < completed part key=_multipart_QCLY10D2A3VF.2~AMJ8825V8KCN.1 size=1048576 2024-05-30T16:03:56.933 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.931+0000 7f08851138c0 1 < completed part key=_multipart_QCLY10D2A3VF.2~AMJ8825V8KCN.2 size=794776 2024-05-30T16:03:56.939 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.939+0000 7f08851138c0 1 < canceled multipart upload key=QCLY10D2A3VF upload=AMJ8825V8KCN tag=22CVQH69MJ0W type=write size=3940504 2024-05-30T16:03:56.946 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.947+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~78LG2IXVFX9K.0 size=1048576 2024-05-30T16:03:56.949 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.947+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~78LG2IXVFX9K.1 size=862114 2024-05-30T16:03:56.949 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.947+0000 7f08851138c0 1 > prepared multipart upload key=73FJ3HGWYQQK upload=78LG2IXVFX9K tag=XOYQ5R5UWT6I type=write size=2959266 2024-05-30T16:03:56.952 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.951+0000 7f08851138c0 1 < completed delete operation key=V4OQFOOQUXFQ tag=WMN9U5EYK9FQ type=del 2024-05-30T16:03:56.958 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.959+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~D5SEN8BE3730.0 size=738326 2024-05-30T16:03:56.958 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.959+0000 7f08851138c0 1 > prepared multipart upload key=3IJWPWRB6FHE upload=D5SEN8BE3730 tag=XKL726F2K2E7 type=write size=1786902 2024-05-30T16:03:56.962 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.963+0000 7f08851138c0 1 < completed delete operation key=364U3OV10VPD tag=GD2ZR64COMVK type=del 2024-05-30T16:03:56.966 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.967+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=9GBSMXB7A1KE type=del size=3101092 2024-05-30T16:03:56.973 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.971+0000 7f08851138c0 1 < suggested update operation key=O0DJ8SCY766V tag=L9Y544RHLLOJ type=del size=3248381 2024-05-30T16:03:56.976 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.975+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=A5UUESQLRGSU type=del size=1485113 2024-05-30T16:03:56.980 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.979+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~QGCIYHF19E9I.0 size=79230 2024-05-30T16:03:56.985 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.987+0000 7f08851138c0 1 < completed multipart upload key=0RZRBW38WKNZ upload=QGCIYHF19E9I tag=LESER9DPZQLY type=write size=1127806 2024-05-30T16:03:56.991 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.991+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~UTYRRX7GU9OF.0 size=1048576 2024-05-30T16:03:56.994 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.995+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~UTYRRX7GU9OF.1 size=933872 2024-05-30T16:03:56.994 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.995+0000 7f08851138c0 1 > prepared multipart upload key=73FJ3HGWYQQK upload=UTYRRX7GU9OF tag=9XB2VD3FPJ3F type=write size=3031024 2024-05-30T16:03:56.998 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.999+0000 7f08851138c0 1 < completed delete operation key=603HBR8PRH6L tag=OPBHVYF2OPPY type=del 2024-05-30T16:03:57.001 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:56.999+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=XMXR5AIWGWCQ type=write size=525736 2024-05-30T16:03:57.005 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.003+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~CD89DQGO3C2K.0 size=580675 2024-05-30T16:03:57.010 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.011+0000 7f08851138c0 1 < completed multipart upload key=VRXNO8JLAZYH upload=CD89DQGO3C2K tag=9XFLBP1ZTSQZ type=write size=1629251 2024-05-30T16:03:57.017 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.015+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~5DAB2LVGEW2K.0 size=1048576 2024-05-30T16:03:57.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.019+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~5DAB2LVGEW2K.1 size=1048576 2024-05-30T16:03:57.022 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.023+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~5DAB2LVGEW2K.2 size=603839 2024-05-30T16:03:57.023 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.023+0000 7f08851138c0 1 > prepared multipart upload key=1N6R2HBTWMP2 upload=5DAB2LVGEW2K tag=YU8N8275EYDY type=write size=3749567 2024-05-30T16:03:57.026 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.027+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~QT9IJ8GK16E4.0 size=339030 2024-05-30T16:03:57.032 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.031+0000 7f08851138c0 1 < completed multipart upload key=364U3OV10VPD upload=QT9IJ8GK16E4 tag=A6EAE1CQ27Q3 type=write size=1387606 2024-05-30T16:03:57.035 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.035+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=8OESOCS261KB type=del size=3103419 2024-05-30T16:03:57.039 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.039+0000 7f08851138c0 1 < completed delete operation key=TPCETCO3Y1M6 tag=PNLUQ0NTPAEQ type=del 2024-05-30T16:03:57.042 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.043+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=CWTFK00FEA3A type=del size=3511211 2024-05-30T16:03:57.049 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.051+0000 7f08851138c0 1 < suggested remove operation key=0RZRBW38WKNZ tag=72EK4NNF0T6T type=del 2024-05-30T16:03:57.053 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.051+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=N91H6N7PK710 type=write size=4163 2024-05-30T16:03:57.058 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.055+0000 7f08851138c0 1 < completed delete operation key=3TBOZWT2P8U8 tag=B1V4NFGI1L28 type=del 2024-05-30T16:03:57.064 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.063+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~5M66OX262ZNG.0 size=1048576 2024-05-30T16:03:57.066 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.067+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~5M66OX262ZNG.1 size=1048576 2024-05-30T16:03:57.069 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.071+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~5M66OX262ZNG.2 size=594609 2024-05-30T16:03:57.069 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.071+0000 7f08851138c0 1 > prepared multipart upload key=V4OQFOOQUXFQ upload=5M66OX262ZNG tag=R8HZ1GKAI5RH type=write size=3740337 2024-05-30T16:03:57.073 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.071+0000 7f08851138c0 1 < completed delete operation key=Y3CYZJ83WYNI tag=SPFTM09CQRQ3 type=del 2024-05-30T16:03:57.080 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.079+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~VSWDC5IN3S3Y.0 size=1048576 2024-05-30T16:03:57.083 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.083+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~VSWDC5IN3S3Y.1 size=961363 2024-05-30T16:03:57.083 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.083+0000 7f08851138c0 1 > prepared multipart upload key=JNZI7JX95LX2 upload=VSWDC5IN3S3Y tag=MTVNDICBJ6F7 type=write size=3058515 2024-05-30T16:03:57.087 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.087+0000 7f08851138c0 1 < completed delete operation key=3TBOZWT2P8U8 tag=1410TS63N9I0 type=del 2024-05-30T16:03:57.091 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.091+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=I3LPIDDQF6WH type=del size=437900 2024-05-30T16:03:57.099 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.099+0000 7f08851138c0 1 < suggested remove operation key=A5BUBY9Z7JQG tag=WCUT9V8XZ9YR type=del 2024-05-30T16:03:57.103 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.103+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=8WUULF1CZV1Q type=del size=2787459 2024-05-30T16:03:57.107 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.107+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~UTKHLUGUKSMI.0 size=1048576 2024-05-30T16:03:57.110 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.111+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~UTKHLUGUKSMI.1 size=1048576 2024-05-30T16:03:57.114 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.115+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~UTKHLUGUKSMI.2 size=644567 2024-05-30T16:03:57.121 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.119+0000 7f08851138c0 1 < completed multipart upload key=603HBR8PRH6L upload=UTKHLUGUKSMI tag=390BLJOKWOJ5 type=write size=3790295 2024-05-30T16:03:57.128 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.127+0000 7f08851138c0 1 > prepared part key=_multipart_MMLF65KLC42Q.2~NBZXNNJH1ADK.0 size=620828 2024-05-30T16:03:57.128 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.127+0000 7f08851138c0 1 > prepared multipart upload key=MMLF65KLC42Q upload=NBZXNNJH1ADK tag=JO3WL8QXS45F type=write size=1669404 2024-05-30T16:03:57.132 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.131+0000 7f08851138c0 1 < completed delete operation key=WT9RC0UYVG4F tag=5EP1J5GSJINB type=del 2024-05-30T16:03:57.136 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.135+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=HLXWWUF5RQK8 type=del size=1830904 2024-05-30T16:03:57.140 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.139+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~78LG2IXVFX9K.0 size=1048576 2024-05-30T16:03:57.143 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.143+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~78LG2IXVFX9K.1 size=862114 2024-05-30T16:03:57.150 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.151+0000 7f08851138c0 1 < completed multipart upload key=73FJ3HGWYQQK upload=78LG2IXVFX9K tag=XOYQ5R5UWT6I type=write size=2959266 2024-05-30T16:03:57.158 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.159+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~3WJRZ1WMKB9H.0 size=1048576 2024-05-30T16:03:57.161 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.159+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~3WJRZ1WMKB9H.1 size=1048576 2024-05-30T16:03:57.164 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.163+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~3WJRZ1WMKB9H.2 size=930876 2024-05-30T16:03:57.164 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.163+0000 7f08851138c0 1 > prepared multipart upload key=73FJ3HGWYQQK upload=3WJRZ1WMKB9H tag=CI47UM9M12PQ type=write size=4076604 2024-05-30T16:03:57.168 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.167+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~D5SEN8BE3730.0 size=738326 2024-05-30T16:03:57.174 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.175+0000 7f08851138c0 1 < completed multipart upload key=3IJWPWRB6FHE upload=D5SEN8BE3730 tag=XKL726F2K2E7 type=write size=1786902 2024-05-30T16:03:57.184 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.183+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~KGV3TI742V4H.0 size=1048576 2024-05-30T16:03:57.187 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.187+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~KGV3TI742V4H.1 size=1048576 2024-05-30T16:03:57.190 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.191+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~KGV3TI742V4H.2 size=214445 2024-05-30T16:03:57.190 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.191+0000 7f08851138c0 1 > prepared multipart upload key=NIV7AQ4EV5YP upload=KGV3TI742V4H tag=VMYM7QH346JY type=write size=3360173 2024-05-30T16:03:57.196 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.195+0000 7f08851138c0 1 < suggested remove operation key=YRP4CSG92QRO tag=9GBSMXB7A1KE type=del 2024-05-30T16:03:57.200 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.199+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=UEQYKNJ5FOQ7 type=del size=1572968 2024-05-30T16:03:57.204 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.203+0000 7f08851138c0 1 < completed delete operation key=0T2E9F3K1USF tag=A5UUESQLRGSU type=del 2024-05-30T16:03:57.208 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.207+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=NT1GZQUGNJH3 type=write size=343542 2024-05-30T16:03:57.212 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.211+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~UTYRRX7GU9OF.0 size=1048576 2024-05-30T16:03:57.215 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.215+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~UTYRRX7GU9OF.1 size=933872 2024-05-30T16:03:57.221 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.223+0000 7f08851138c0 1 < completed multipart upload key=73FJ3HGWYQQK upload=UTYRRX7GU9OF tag=9XB2VD3FPJ3F type=write size=3031024 2024-05-30T16:03:57.229 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.227+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~WXR3C5YFSZWU.0 size=1048576 2024-05-30T16:03:57.232 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.231+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~WXR3C5YFSZWU.1 size=1048576 2024-05-30T16:03:57.235 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.235+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~WXR3C5YFSZWU.2 size=754642 2024-05-30T16:03:57.235 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.235+0000 7f08851138c0 1 > prepared multipart upload key=O8CWPJ9GW2GO upload=WXR3C5YFSZWU tag=378P1BSOYYQS type=write size=3900370 2024-05-30T16:03:57.238 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.239+0000 7f08851138c0 -1 < failed to complete operation key=0RZRBW38WKNZ tag=XMXR5AIWGWCQ type=write size=525736: (22) Invalid argument 2024-05-30T16:03:57.241 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.239+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=ZR17JQZ5H3RH type=del size=2410938 2024-05-30T16:03:57.242 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.243+0000 7f08851138c0 1 < canceled part key=_multipart_1N6R2HBTWMP2.2~5DAB2LVGEW2K.0 size=1048576 2024-05-30T16:03:57.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.243+0000 7f08851138c0 1 < completed part key=_multipart_1N6R2HBTWMP2.2~5DAB2LVGEW2K.1 size=1048576 2024-05-30T16:03:57.247 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.247+0000 7f08851138c0 1 < completed part key=_multipart_1N6R2HBTWMP2.2~5DAB2LVGEW2K.2 size=603839 2024-05-30T16:03:57.253 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.251+0000 7f08851138c0 1 < canceled multipart upload key=1N6R2HBTWMP2 upload=5DAB2LVGEW2K tag=YU8N8275EYDY type=write size=3749567 2024-05-30T16:03:57.256 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.255+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=2U8XNLTJBZPG type=del size=448188 2024-05-30T16:03:57.262 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.259+0000 7f08851138c0 1 < completed delete operation key=E2FXSXOEUGIG tag=8OESOCS261KB type=del 2024-05-30T16:03:57.263 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.263+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=LQZYSDI9WKRM type=del size=3197614 2024-05-30T16:03:57.270 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.271+0000 7f08851138c0 1 < suggested update operation key=Y8SBI0QZ0ZV8 tag=CWTFK00FEA3A type=del size=3511211 2024-05-30T16:03:57.274 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.275+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=1A26Q92C60UH type=del size=654688 2024-05-30T16:03:57.280 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.279+0000 7f08851138c0 1 < suggested remove operation key=XHGMUYBA2VNK tag=N91H6N7PK710 type=write 2024-05-30T16:03:57.283 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.283+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=7B4HIAMVXFFU type=write size=71099 2024-05-30T16:03:57.287 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.287+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~5M66OX262ZNG.0 size=1048576 2024-05-30T16:03:57.291 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.291+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~5M66OX262ZNG.1 size=1048576 2024-05-30T16:03:57.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.295+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~5M66OX262ZNG.2 size=594609 2024-05-30T16:03:57.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.299+0000 7f08851138c0 1 < completed multipart upload key=V4OQFOOQUXFQ upload=5M66OX262ZNG tag=R8HZ1GKAI5RH type=write size=3740337 2024-05-30T16:03:57.303 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.303+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=OJRK6BJV25LJ type=write size=543744 2024-05-30T16:03:57.307 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.307+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~VSWDC5IN3S3Y.0 size=1048576 2024-05-30T16:03:57.309 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.311+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~VSWDC5IN3S3Y.1 size=961363 2024-05-30T16:03:57.316 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.315+0000 7f08851138c0 1 < completed multipart upload key=JNZI7JX95LX2 upload=VSWDC5IN3S3Y tag=MTVNDICBJ6F7 type=write size=3058515 2024-05-30T16:03:57.320 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.319+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=8RYX8TZ61X7Y type=del size=165831 2024-05-30T16:03:57.327 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.327+0000 7f08851138c0 1 < suggested remove operation key=Y8SBI0QZ0ZV8 tag=I3LPIDDQF6WH type=del 2024-05-30T16:03:57.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.331+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=CA236M176QKU type=del size=3875087 2024-05-30T16:03:57.334 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.335+0000 7f08851138c0 1 < completed delete operation key=4N58ZY4E1H6E tag=8WUULF1CZV1Q type=del 2024-05-30T16:03:57.337 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.335+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=GGIALAEF5WCO type=write size=87557 2024-05-30T16:03:57.340 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.339+0000 7f08851138c0 1 < completed part key=_multipart_MMLF65KLC42Q.2~NBZXNNJH1ADK.0 size=620828 2024-05-30T16:03:57.346 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.347+0000 7f08851138c0 1 < completed multipart upload key=MMLF65KLC42Q upload=NBZXNNJH1ADK tag=JO3WL8QXS45F type=write size=1669404 2024-05-30T16:03:57.353 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.351+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~06P3PA5OXCGF.0 size=1048576 2024-05-30T16:03:57.356 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.355+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~06P3PA5OXCGF.1 size=1048576 2024-05-30T16:03:57.358 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.359+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~06P3PA5OXCGF.2 size=100340 2024-05-30T16:03:57.359 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.359+0000 7f08851138c0 1 > prepared multipart upload key=603HBR8PRH6L upload=06P3PA5OXCGF tag=05GDYXLYPSMJ type=write size=3246068 2024-05-30T16:03:57.362 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.363+0000 7f08851138c0 1 < completed delete operation key=O8CWPJ9GW2GO tag=HLXWWUF5RQK8 type=del 2024-05-30T16:03:57.366 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.367+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=OFMKOUYH7NH0 type=del size=2093865 2024-05-30T16:03:57.370 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.371+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~3WJRZ1WMKB9H.0 size=1048576 2024-05-30T16:03:57.370 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.371+0000 7f08851138c0 1 < canceled part key=_multipart_73FJ3HGWYQQK.2~3WJRZ1WMKB9H.1 size=1048576 2024-05-30T16:03:57.373 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.371+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~3WJRZ1WMKB9H.2 size=930876 2024-05-30T16:03:57.379 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.379+0000 7f08851138c0 1 < canceled multipart upload key=73FJ3HGWYQQK upload=3WJRZ1WMKB9H tag=CI47UM9M12PQ type=write size=4076604 2024-05-30T16:03:57.386 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.387+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~4MCOEGEFNECM.0 size=49152 2024-05-30T16:03:57.386 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.387+0000 7f08851138c0 1 > prepared multipart upload key=A5BUBY9Z7JQG upload=4MCOEGEFNECM tag=YDV3CJZHS6UH type=write size=1097728 2024-05-30T16:03:57.389 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.391+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~KGV3TI742V4H.0 size=1048576 2024-05-30T16:03:57.392 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.391+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~KGV3TI742V4H.1 size=1048576 2024-05-30T16:03:57.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.395+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~KGV3TI742V4H.2 size=214445 2024-05-30T16:03:57.401 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.403+0000 7f08851138c0 1 < completed multipart upload key=NIV7AQ4EV5YP upload=KGV3TI742V4H tag=VMYM7QH346JY type=write size=3360173 2024-05-30T16:03:57.405 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.403+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=W3S7EIRDJ5XN type=write size=760645 2024-05-30T16:03:57.409 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.407+0000 7f08851138c0 1 < canceled operation key=OKBB0VU765TW tag=UEQYKNJ5FOQ7 type=del size=1572968 2024-05-30T16:03:57.413 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.411+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=Q9AJ50JP6H46 type=del size=194846 2024-05-30T16:03:57.420 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.419+0000 7f08851138c0 1 < suggested remove operation key=O0DJ8SCY766V tag=NT1GZQUGNJH3 type=write 2024-05-30T16:03:57.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.423+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=I3MRT5HHWTUM type=del size=1885200 2024-05-30T16:03:57.427 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.427+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~WXR3C5YFSZWU.0 size=1048576 2024-05-30T16:03:57.430 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.431+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~WXR3C5YFSZWU.1 size=1048576 2024-05-30T16:03:57.433 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.431+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~WXR3C5YFSZWU.2 size=754642 2024-05-30T16:03:57.439 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.439+0000 7f08851138c0 1 < completed multipart upload key=O8CWPJ9GW2GO upload=WXR3C5YFSZWU tag=378P1BSOYYQS type=write size=3900370 2024-05-30T16:03:57.446 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.447+0000 7f08851138c0 1 > prepared part key=_multipart_MMLF65KLC42Q.2~12FKZY2O77WI.0 size=1048576 2024-05-30T16:03:57.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.447+0000 7f08851138c0 1 > prepared part key=_multipart_MMLF65KLC42Q.2~12FKZY2O77WI.1 size=131506 2024-05-30T16:03:57.449 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.447+0000 7f08851138c0 1 > prepared multipart upload key=MMLF65KLC42Q upload=12FKZY2O77WI tag=2IBB38IGNHXG type=write size=2228658 2024-05-30T16:03:57.452 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.451+0000 7f08851138c0 1 < canceled operation key=4N58ZY4E1H6E tag=ZR17JQZ5H3RH type=del size=2410938 2024-05-30T16:03:57.459 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.459+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~RESK5VIUZCNJ.0 size=678893 2024-05-30T16:03:57.459 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.459+0000 7f08851138c0 1 > prepared multipart upload key=Y8SBI0QZ0ZV8 upload=RESK5VIUZCNJ tag=WEUSA3Z0NZVV type=write size=1727469 2024-05-30T16:03:57.462 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.463+0000 7f08851138c0 1 < completed delete operation key=Y3CYZJ83WYNI tag=2U8XNLTJBZPG type=del 2024-05-30T16:03:57.466 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.467+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=FYBMVOX5PJQK type=del size=2232868 2024-05-30T16:03:57.469 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.471+0000 7f08851138c0 1 < completed delete operation key=364U3OV10VPD tag=LQZYSDI9WKRM type=del 2024-05-30T16:03:57.473 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.471+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=8PCTM11Y988J type=write size=46892 2024-05-30T16:03:57.476 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.475+0000 7f08851138c0 1 < completed delete operation key=RCTV94WRMO53 tag=1A26Q92C60UH type=del 2024-05-30T16:03:57.483 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.483+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~P57NHQJC11AK.0 size=631828 2024-05-30T16:03:57.483 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.483+0000 7f08851138c0 1 > prepared multipart upload key=4N58ZY4E1H6E upload=P57NHQJC11AK tag=34LPUHC1PYA9 type=write size=1680404 2024-05-30T16:03:57.486 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.487+0000 7f08851138c0 1 < completed write operation key=XHGMUYBA2VNK tag=7B4HIAMVXFFU type=write size=71099 2024-05-30T16:03:57.490 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.491+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=85WGVIS91S9K type=del size=3002971 2024-05-30T16:03:57.495 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.495+0000 7f08851138c0 1 < completed write operation key=3IJWPWRB6FHE tag=OJRK6BJV25LJ type=write size=543744 2024-05-30T16:03:57.501 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.503+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~W5QO5NGNSSOV.0 size=200291 2024-05-30T16:03:57.501 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.503+0000 7f08851138c0 1 > prepared multipart upload key=VRXNO8JLAZYH upload=W5QO5NGNSSOV tag=MPOBYKXCIY7E type=write size=1248867 2024-05-30T16:03:57.505 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.503+0000 7f08851138c0 1 < completed delete operation key=NIV7AQ4EV5YP tag=8RYX8TZ61X7Y type=del 2024-05-30T16:03:57.512 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.511+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~0MNJE7APOMMK.0 size=1048576 2024-05-30T16:03:57.515 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.515+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~0MNJE7APOMMK.1 size=1048576 2024-05-30T16:03:57.523 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.523+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~0MNJE7APOMMK.2 size=553048 2024-05-30T16:03:57.523 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.523+0000 7f08851138c0 1 > prepared multipart upload key=KPHZGFJLC2E0 upload=0MNJE7APOMMK tag=9IC2UQ1W3D3I type=write size=3698776 2024-05-30T16:03:57.527 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.527+0000 7f08851138c0 1 < completed delete operation key=A5BUBY9Z7JQG tag=CA236M176QKU type=del 2024-05-30T16:03:57.535 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.535+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~HX2728Z7I9YG.0 size=1048576 2024-05-30T16:03:57.538 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.539+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~HX2728Z7I9YG.1 size=1048576 2024-05-30T16:03:57.540 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.539+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~HX2728Z7I9YG.2 size=937822 2024-05-30T16:03:57.541 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.539+0000 7f08851138c0 1 > prepared multipart upload key=GWXNCSJCLXKW upload=HX2728Z7I9YG tag=G5J6KLRIXAN6 type=write size=4083550 2024-05-30T16:03:57.547 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.547+0000 7f08851138c0 1 < suggested remove operation key=1N6R2HBTWMP2 tag=GGIALAEF5WCO type=write 2024-05-30T16:03:57.551 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.551+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=D593VYSTOPAX type=del size=3760439 2024-05-30T16:03:57.552 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.551+0000 7f08851138c0 1 < canceled part key=_multipart_603HBR8PRH6L.2~06P3PA5OXCGF.0 size=1048576 2024-05-30T16:03:57.554 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.555+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~06P3PA5OXCGF.1 size=1048576 2024-05-30T16:03:57.557 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.559+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~06P3PA5OXCGF.2 size=100340 2024-05-30T16:03:57.564 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.563+0000 7f08851138c0 1 < canceled multipart upload key=603HBR8PRH6L upload=06P3PA5OXCGF tag=05GDYXLYPSMJ type=write size=3246068 2024-05-30T16:03:57.568 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.567+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=XSPXPMSCM5TG type=write size=576326 2024-05-30T16:03:57.572 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.571+0000 7f08851138c0 1 < completed delete operation key=603HBR8PRH6L tag=OFMKOUYH7NH0 type=del 2024-05-30T16:03:57.579 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.579+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~Q2UI1H2QOYML.0 size=8339 2024-05-30T16:03:57.579 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.579+0000 7f08851138c0 1 > prepared multipart upload key=E2FXSXOEUGIG upload=Q2UI1H2QOYML tag=FEBCF3S521OI type=write size=1056915 2024-05-30T16:03:57.584 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.583+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~4MCOEGEFNECM.0 size=49152 2024-05-30T16:03:57.590 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.591+0000 7f08851138c0 1 < completed multipart upload key=A5BUBY9Z7JQG upload=4MCOEGEFNECM tag=YDV3CJZHS6UH type=write size=1097728 2024-05-30T16:03:57.594 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.595+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=XZ9EI3AW3QYR type=del size=970613 2024-05-30T16:03:57.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.599+0000 7f08851138c0 1 < completed write operation key=QCLY10D2A3VF tag=W3S7EIRDJ5XN type=write size=760645 2024-05-30T16:03:57.605 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.603+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~T2UEMJSS1ANU.0 size=1048576 2024-05-30T16:03:57.608 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.607+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~T2UEMJSS1ANU.1 size=547993 2024-05-30T16:03:57.608 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.607+0000 7f08851138c0 1 > prepared multipart upload key=PLWFSKPKVO19 upload=T2UEMJSS1ANU tag=CU4GJ0V47YPU type=write size=2645145 2024-05-30T16:03:57.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.611+0000 7f08851138c0 -1 < failed to complete operation key=O0DJ8SCY766V tag=Q9AJ50JP6H46 type=del size=194846: (22) Invalid argument 2024-05-30T16:03:57.618 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.619+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~L6VYDHX2ORT1.0 size=1048576 2024-05-30T16:03:57.621 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.619+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~L6VYDHX2ORT1.1 size=22191 2024-05-30T16:03:57.621 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.619+0000 7f08851138c0 1 > prepared multipart upload key=KPHZGFJLC2E0 upload=L6VYDHX2ORT1 tag=53707MCR7ZWJ type=write size=2119343 2024-05-30T16:03:57.624 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.623+0000 7f08851138c0 1 < completed delete operation key=3TBOZWT2P8U8 tag=I3MRT5HHWTUM type=del 2024-05-30T16:03:57.628 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.627+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=KAFRISGH5F1H type=del size=1128394 2024-05-30T16:03:57.632 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.631+0000 7f08851138c0 1 < completed part key=_multipart_MMLF65KLC42Q.2~12FKZY2O77WI.0 size=1048576 2024-05-30T16:03:57.635 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.635+0000 7f08851138c0 1 < completed part key=_multipart_MMLF65KLC42Q.2~12FKZY2O77WI.1 size=131506 2024-05-30T16:03:57.641 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.639+0000 7f08851138c0 1 < completed multipart upload key=MMLF65KLC42Q upload=12FKZY2O77WI tag=2IBB38IGNHXG type=write size=2228658 2024-05-30T16:03:57.648 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.647+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~ZWDDRN6908R1.0 size=1048576 2024-05-30T16:03:57.651 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.651+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~ZWDDRN6908R1.1 size=731217 2024-05-30T16:03:57.651 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.651+0000 7f08851138c0 1 > prepared multipart upload key=E2FXSXOEUGIG upload=ZWDDRN6908R1 tag=5V101U7K2NO5 type=write size=2828369 2024-05-30T16:03:57.655 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.655+0000 7f08851138c0 1 < completed part key=_multipart_Y8SBI0QZ0ZV8.2~RESK5VIUZCNJ.0 size=678893 2024-05-30T16:03:57.661 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.659+0000 7f08851138c0 1 < completed multipart upload key=Y8SBI0QZ0ZV8 upload=RESK5VIUZCNJ tag=WEUSA3Z0NZVV type=write size=1727469 2024-05-30T16:03:57.666 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.667+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=KK884A82TT0U type=write size=163714 2024-05-30T16:03:57.670 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.671+0000 7f08851138c0 -1 < failed to complete operation key=1N6R2HBTWMP2 tag=FYBMVOX5PJQK type=del size=2232868: (22) Invalid argument 2024-05-30T16:03:57.674 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.675+0000 7f08851138c0 1 > prepared operation key=VRXNO8JLAZYH tag=D4JEZF9J0913 type=del size=999844 2024-05-30T16:03:57.677 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.679+0000 7f08851138c0 1 < completed write operation key=3IJWPWRB6FHE tag=8PCTM11Y988J type=write size=46892 2024-05-30T16:03:57.681 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.679+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=AY1LELJUI9O0 type=del size=777926 2024-05-30T16:03:57.685 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.683+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~P57NHQJC11AK.0 size=631828 2024-05-30T16:03:57.691 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.691+0000 7f08851138c0 1 < completed multipart upload key=4N58ZY4E1H6E upload=P57NHQJC11AK tag=34LPUHC1PYA9 type=write size=1680404 2024-05-30T16:03:57.698 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.699+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~VFIRUO758H9O.0 size=1048576 2024-05-30T16:03:57.701 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.699+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~VFIRUO758H9O.1 size=741939 2024-05-30T16:03:57.701 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.699+0000 7f08851138c0 1 > prepared multipart upload key=0T2E9F3K1USF upload=VFIRUO758H9O tag=KLBNCSUOSGVQ type=write size=2839091 2024-05-30T16:03:57.704 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.703+0000 7f08851138c0 1 < completed delete operation key=JNZI7JX95LX2 tag=85WGVIS91S9K type=del 2024-05-30T16:03:57.708 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.707+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=XLIM34IH236Z type=del size=3697399 2024-05-30T16:03:57.712 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.711+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~W5QO5NGNSSOV.0 size=200291 2024-05-30T16:03:57.719 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.719+0000 7f08851138c0 1 < completed multipart upload key=VRXNO8JLAZYH upload=W5QO5NGNSSOV tag=MPOBYKXCIY7E type=write size=1248867 2024-05-30T16:03:57.726 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.727+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~TN9KAIKXMNVX.0 size=37130 2024-05-30T16:03:57.726 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.727+0000 7f08851138c0 1 > prepared multipart upload key=O8CWPJ9GW2GO upload=TN9KAIKXMNVX tag=JFCVW9AD74CU type=write size=1085706 2024-05-30T16:03:57.727 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.727+0000 7f08851138c0 1 < canceled part key=_multipart_KPHZGFJLC2E0.2~0MNJE7APOMMK.0 size=1048576 2024-05-30T16:03:57.729 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.731+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~0MNJE7APOMMK.1 size=1048576 2024-05-30T16:03:57.733 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.731+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~0MNJE7APOMMK.2 size=553048 2024-05-30T16:03:57.739 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.739+0000 7f08851138c0 1 < canceled multipart upload key=KPHZGFJLC2E0 upload=0MNJE7APOMMK tag=9IC2UQ1W3D3I type=write size=3698776 2024-05-30T16:03:57.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.743+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~EBTASH9Y39IY.0 size=1048576 2024-05-30T16:03:57.748 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.747+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~EBTASH9Y39IY.1 size=385743 2024-05-30T16:03:57.748 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.747+0000 7f08851138c0 1 > prepared multipart upload key=E2FXSXOEUGIG upload=EBTASH9Y39IY tag=UPQSR2TTTE8E type=write size=2482895 2024-05-30T16:03:57.752 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.751+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~HX2728Z7I9YG.0 size=1048576 2024-05-30T16:03:57.755 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.755+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~HX2728Z7I9YG.1 size=1048576 2024-05-30T16:03:57.758 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.759+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~HX2728Z7I9YG.2 size=937822 2024-05-30T16:03:57.765 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.763+0000 7f08851138c0 1 < completed multipart upload key=GWXNCSJCLXKW upload=HX2728Z7I9YG tag=G5J6KLRIXAN6 type=write size=4083550 2024-05-30T16:03:57.768 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.767+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=51WAX7KOCE5L type=del size=3734718 2024-05-30T16:03:57.775 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.775+0000 7f08851138c0 1 < suggested update operation key=3IJWPWRB6FHE tag=D593VYSTOPAX type=del size=3760439 2024-05-30T16:03:57.779 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.779+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=5RX0KK3WJTB5 type=del size=2322471 2024-05-30T16:03:57.787 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.787+0000 7f08851138c0 1 < suggested update operation key=V4OQFOOQUXFQ tag=XSPXPMSCM5TG type=write size=576326 2024-05-30T16:03:57.794 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.795+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~INC5YHURDMDA.0 size=1048576 2024-05-30T16:03:57.796 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.795+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~INC5YHURDMDA.1 size=1048576 2024-05-30T16:03:57.799 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.799+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~INC5YHURDMDA.2 size=844190 2024-05-30T16:03:57.799 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.799+0000 7f08851138c0 1 > prepared multipart upload key=PLWFSKPKVO19 upload=INC5YHURDMDA tag=7U99KLWB3FSW type=write size=3989918 2024-05-30T16:03:57.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.803+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~Q2UI1H2QOYML.0 size=8339 2024-05-30T16:03:57.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.807+0000 7f08851138c0 1 < completed multipart upload key=E2FXSXOEUGIG upload=Q2UI1H2QOYML tag=FEBCF3S521OI type=write size=1056915 2024-05-30T16:03:57.813 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.811+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=UZ9UDQ76F0TR type=del size=2108504 2024-05-30T16:03:57.816 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.815+0000 7f08851138c0 1 < completed delete operation key=JNZI7JX95LX2 tag=XZ9EI3AW3QYR type=del 2024-05-30T16:03:57.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.819+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=2SPMOOY2XVCI type=del size=1044770 2024-05-30T16:03:57.824 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.823+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~T2UEMJSS1ANU.0 size=1048576 2024-05-30T16:03:57.827 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.827+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~T2UEMJSS1ANU.1 size=547993 2024-05-30T16:03:57.832 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.831+0000 7f08851138c0 1 < completed multipart upload key=PLWFSKPKVO19 upload=T2UEMJSS1ANU tag=CU4GJ0V47YPU type=write size=2645145 2024-05-30T16:03:57.839 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.839+0000 7f08851138c0 1 > prepared part key=_multipart_MMLF65KLC42Q.2~8LLECF99TVPY.0 size=237391 2024-05-30T16:03:57.839 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.839+0000 7f08851138c0 1 > prepared multipart upload key=MMLF65KLC42Q upload=8LLECF99TVPY tag=6OZD72N0EKV6 type=write size=1285967 2024-05-30T16:03:57.842 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.843+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~L6VYDHX2ORT1.0 size=1048576 2024-05-30T16:03:57.843 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.843+0000 7f08851138c0 1 < canceled part key=_multipart_KPHZGFJLC2E0.2~L6VYDHX2ORT1.1 size=22191 2024-05-30T16:03:57.848 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.847+0000 7f08851138c0 1 < canceled multipart upload key=KPHZGFJLC2E0 upload=L6VYDHX2ORT1 tag=53707MCR7ZWJ type=write size=2119343 2024-05-30T16:03:57.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.851+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=Z4EAHJFM51LN type=del size=1895008 2024-05-30T16:03:57.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.855+0000 7f08851138c0 1 < completed delete operation key=KPHZGFJLC2E0 tag=KAFRISGH5F1H type=del 2024-05-30T16:03:57.858 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.859+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=FFYPX5009P36 type=del size=3677406 2024-05-30T16:03:57.861 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.863+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~ZWDDRN6908R1.0 size=1048576 2024-05-30T16:03:57.867 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.867+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~ZWDDRN6908R1.1 size=731217 2024-05-30T16:03:57.874 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.871+0000 7f08851138c0 1 < completed multipart upload key=E2FXSXOEUGIG upload=ZWDDRN6908R1 tag=5V101U7K2NO5 type=write size=2828369 2024-05-30T16:03:57.876 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.875+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=A1RRX2THS9Z0 type=del size=1651118 2024-05-30T16:03:57.883 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.883+0000 7f08851138c0 1 < suggested remove operation key=GWXNCSJCLXKW tag=KK884A82TT0U type=write 2024-05-30T16:03:57.887 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.887+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=L9I13UFAZCGQ type=del size=732377 2024-05-30T16:03:57.890 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.891+0000 7f08851138c0 1 < completed delete operation key=VRXNO8JLAZYH tag=D4JEZF9J0913 type=del 2024-05-30T16:03:57.893 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.895+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=E96XIYO40VWP type=del size=2255295 2024-05-30T16:03:57.897 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.895+0000 7f08851138c0 -1 < failed to complete operation key=V4OQFOOQUXFQ tag=AY1LELJUI9O0 type=del size=777926: (22) Invalid argument 2024-05-30T16:03:57.903 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.903+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~BHMGGRA06EW8.0 size=1048576 2024-05-30T16:03:57.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.907+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~BHMGGRA06EW8.1 size=1048576 2024-05-30T16:03:57.910 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.911+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~BHMGGRA06EW8.2 size=540198 2024-05-30T16:03:57.910 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.911+0000 7f08851138c0 1 > prepared multipart upload key=73FJ3HGWYQQK upload=BHMGGRA06EW8 tag=CTSOPACY7057 type=write size=3685926 2024-05-30T16:03:57.910 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.911+0000 7f08851138c0 1 < canceled part key=_multipart_0T2E9F3K1USF.2~VFIRUO758H9O.0 size=1048576 2024-05-30T16:03:57.914 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.915+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~VFIRUO758H9O.1 size=741939 2024-05-30T16:03:57.920 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.919+0000 7f08851138c0 1 < canceled multipart upload key=0T2E9F3K1USF upload=VFIRUO758H9O tag=KLBNCSUOSGVQ type=write size=2839091 2024-05-30T16:03:57.923 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.923+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=9A22LR0L8GSX type=del size=108416 2024-05-30T16:03:57.926 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.927+0000 7f08851138c0 1 < completed delete operation key=0RZRBW38WKNZ tag=XLIM34IH236Z type=del 2024-05-30T16:03:57.929 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.931+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=RZV77FAF1MK6 type=del size=1142856 2024-05-30T16:03:57.933 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.931+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~TN9KAIKXMNVX.0 size=37130 2024-05-30T16:03:57.939 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.939+0000 7f08851138c0 1 < completed multipart upload key=O8CWPJ9GW2GO upload=TN9KAIKXMNVX tag=JFCVW9AD74CU type=write size=1085706 2024-05-30T16:03:57.942 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.943+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=4BOGQR05KY5N type=del size=3016399 2024-05-30T16:03:57.945 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.943+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~EBTASH9Y39IY.0 size=1048576 2024-05-30T16:03:57.948 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.947+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~EBTASH9Y39IY.1 size=385743 2024-05-30T16:03:57.953 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.955+0000 7f08851138c0 1 < completed multipart upload key=E2FXSXOEUGIG upload=EBTASH9Y39IY tag=UPQSR2TTTE8E type=write size=2482895 2024-05-30T16:03:57.957 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.955+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=LTHCTLJ2PVQE type=del size=58653 2024-05-30T16:03:57.961 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.959+0000 7f08851138c0 1 < completed delete operation key=JNZI7JX95LX2 tag=51WAX7KOCE5L type=del 2024-05-30T16:03:57.964 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.963+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=NQ5MFLSMWF1V type=write size=540264 2024-05-30T16:03:57.967 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.967+0000 7f08851138c0 1 < completed delete operation key=Y8SBI0QZ0ZV8 tag=5RX0KK3WJTB5 type=del 2024-05-30T16:03:57.971 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.971+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=PWE5H7DME30K type=del size=1948072 2024-05-30T16:03:57.974 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.975+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~INC5YHURDMDA.0 size=1048576 2024-05-30T16:03:57.976 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.975+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~INC5YHURDMDA.1 size=1048576 2024-05-30T16:03:57.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.979+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~INC5YHURDMDA.2 size=844190 2024-05-30T16:03:57.983 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.983+0000 7f08851138c0 1 < completed multipart upload key=PLWFSKPKVO19 upload=INC5YHURDMDA tag=7U99KLWB3FSW type=write size=3989918 2024-05-30T16:03:57.989 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.991+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~Q1IJIZ0ZS5IQ.0 size=1048576 2024-05-30T16:03:57.992 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.991+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~Q1IJIZ0ZS5IQ.1 size=450299 2024-05-30T16:03:57.992 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.991+0000 7f08851138c0 1 > prepared multipart upload key=O8CWPJ9GW2GO upload=Q1IJIZ0ZS5IQ tag=WMW4LAISSRTF type=write size=2547451 2024-05-30T16:03:57.995 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.995+0000 7f08851138c0 1 < completed delete operation key=603HBR8PRH6L tag=UZ9UDQ76F0TR type=del 2024-05-30T16:03:57.998 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:57.999+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=AWUSB65L1UTY type=del size=4154767 2024-05-30T16:03:58.002 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.003+0000 7f08851138c0 1 < canceled operation key=1N6R2HBTWMP2 tag=2SPMOOY2XVCI type=del size=1044770 2024-05-30T16:03:58.008 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.007+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~51LI9OEW4S4B.0 size=1048576 2024-05-30T16:03:58.010 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.011+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~51LI9OEW4S4B.1 size=1048576 2024-05-30T16:03:58.013 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.015+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~51LI9OEW4S4B.2 size=102288 2024-05-30T16:03:58.013 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.015+0000 7f08851138c0 1 > prepared multipart upload key=XHGMUYBA2VNK upload=51LI9OEW4S4B tag=XB431EE12YQ7 type=write size=3248016 2024-05-30T16:03:58.017 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.015+0000 7f08851138c0 1 < completed part key=_multipart_MMLF65KLC42Q.2~8LLECF99TVPY.0 size=237391 2024-05-30T16:03:58.022 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.023+0000 7f08851138c0 1 < completed multipart upload key=MMLF65KLC42Q upload=8LLECF99TVPY tag=6OZD72N0EKV6 type=write size=1285967 2024-05-30T16:03:58.026 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.027+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=TVPLW77AOHKY type=del size=2765324 2024-05-30T16:03:58.032 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.031+0000 7f08851138c0 1 < suggested remove operation key=V4OQFOOQUXFQ tag=Z4EAHJFM51LN type=del 2024-05-30T16:03:58.036 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.035+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=W9U0JHU8RDNF type=del size=3467990 2024-05-30T16:03:58.040 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.039+0000 7f08851138c0 1 < completed delete operation key=YRP4CSG92QRO tag=FFYPX5009P36 type=del 2024-05-30T16:03:58.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.047+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~FXN7KFKS91G8.0 size=1048576 2024-05-30T16:03:58.049 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.047+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~FXN7KFKS91G8.1 size=461752 2024-05-30T16:03:58.049 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.047+0000 7f08851138c0 1 > prepared multipart upload key=O8CWPJ9GW2GO upload=FXN7KFKS91G8 tag=KKYFA1PY5D4U type=write size=2558904 2024-05-30T16:03:58.053 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.051+0000 7f08851138c0 1 < completed delete operation key=73FJ3HGWYQQK tag=A1RRX2THS9Z0 type=del 2024-05-30T16:03:58.056 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.055+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=9SPG3WV97OSF type=del size=2296184 2024-05-30T16:03:58.059 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.059+0000 7f08851138c0 1 < canceled operation key=4N58ZY4E1H6E tag=L9I13UFAZCGQ type=del size=732377 2024-05-30T16:03:58.063 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.063+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=9H61CQFRCFET type=del size=371908 2024-05-30T16:03:58.066 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.067+0000 7f08851138c0 1 < completed delete operation key=A5BUBY9Z7JQG tag=E96XIYO40VWP type=del 2024-05-30T16:03:58.070 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.071+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=GPNA21WKQEVQ type=del size=2101793 2024-05-30T16:03:58.074 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.075+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~BHMGGRA06EW8.0 size=1048576 2024-05-30T16:03:58.077 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.075+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~BHMGGRA06EW8.1 size=1048576 2024-05-30T16:03:58.082 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.079+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~BHMGGRA06EW8.2 size=540198 2024-05-30T16:03:58.086 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.087+0000 7f08851138c0 1 < completed multipart upload key=73FJ3HGWYQQK upload=BHMGGRA06EW8 tag=CTSOPACY7057 type=write size=3685926 2024-05-30T16:03:58.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.091+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=1Q2ZVQ2HT3DC type=del size=1172255 2024-05-30T16:03:58.097 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.099+0000 7f08851138c0 1 < suggested update operation key=4N58ZY4E1H6E tag=9A22LR0L8GSX type=del size=108416 2024-05-30T16:03:58.101 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.099+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=PZATTM5C6WEE type=del size=3673002 2024-05-30T16:03:58.104 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.103+0000 7f08851138c0 1 < completed delete operation key=TPCETCO3Y1M6 tag=RZV77FAF1MK6 type=del 2024-05-30T16:03:58.108 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.107+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=7AB8VK5E2NHB type=write size=175593 2024-05-30T16:03:58.115 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.115+0000 7f08851138c0 1 < suggested remove operation key=RCTV94WRMO53 tag=4BOGQR05KY5N type=del 2024-05-30T16:03:58.121 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.123+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~J1QQ34ALU3E1.0 size=1048576 2024-05-30T16:03:58.124 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.123+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~J1QQ34ALU3E1.1 size=1048576 2024-05-30T16:03:58.127 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.127+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~J1QQ34ALU3E1.2 size=953684 2024-05-30T16:03:58.127 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.127+0000 7f08851138c0 1 > prepared multipart upload key=GWXNCSJCLXKW upload=J1QQ34ALU3E1 tag=USYO736BE44T type=write size=4099412 2024-05-30T16:03:58.131 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.131+0000 7f08851138c0 1 < completed delete operation key=XHGMUYBA2VNK tag=LTHCTLJ2PVQE type=del 2024-05-30T16:03:58.134 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.135+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=L53HEY9RT09P type=del size=1660498 2024-05-30T16:03:58.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.139+0000 7f08851138c0 1 < completed write operation key=PLWFSKPKVO19 tag=NQ5MFLSMWF1V type=write size=540264 2024-05-30T16:03:58.141 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.139+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=5KIGNC38AX5N type=del size=1429758 2024-05-30T16:03:58.144 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.143+0000 7f08851138c0 -1 < failed to complete operation key=RCTV94WRMO53 tag=PWE5H7DME30K type=del size=1948072: (22) Invalid argument 2024-05-30T16:03:58.150 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.151+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~779ATCN05HCW.0 size=400824 2024-05-30T16:03:58.150 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.151+0000 7f08851138c0 1 > prepared multipart upload key=PLWFSKPKVO19 upload=779ATCN05HCW tag=04UEGYAXI4LZ type=write size=1449400 2024-05-30T16:03:58.153 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.155+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~Q1IJIZ0ZS5IQ.0 size=1048576 2024-05-30T16:03:58.156 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.155+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~Q1IJIZ0ZS5IQ.1 size=450299 2024-05-30T16:03:58.162 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.163+0000 7f08851138c0 1 < completed multipart upload key=O8CWPJ9GW2GO upload=Q1IJIZ0ZS5IQ tag=WMW4LAISSRTF type=write size=2547451 2024-05-30T16:03:58.166 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.167+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=ZW1JGYP5XRAP type=del size=1292583 2024-05-30T16:03:58.169 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.171+0000 7f08851138c0 1 < completed delete operation key=GWXNCSJCLXKW tag=AWUSB65L1UTY type=del 2024-05-30T16:03:58.173 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.171+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=5B4KQK39NQAN type=del size=870414 2024-05-30T16:03:58.173 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.175+0000 7f08851138c0 1 < canceled part key=_multipart_XHGMUYBA2VNK.2~51LI9OEW4S4B.0 size=1048576 2024-05-30T16:03:58.176 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.175+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~51LI9OEW4S4B.1 size=1048576 2024-05-30T16:03:58.179 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.179+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~51LI9OEW4S4B.2 size=102288 2024-05-30T16:03:58.185 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.187+0000 7f08851138c0 1 < canceled multipart upload key=XHGMUYBA2VNK upload=51LI9OEW4S4B tag=XB431EE12YQ7 type=write size=3248016 2024-05-30T16:03:58.194 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.195+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=B5CAN6BQPLNR type=del size=2316575 2024-05-30T16:03:58.198 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.199+0000 7f08851138c0 1 < completed delete operation key=TPCETCO3Y1M6 tag=TVPLW77AOHKY type=del 2024-05-30T16:03:58.201 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.203+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=ZL8BVZ51FJHI type=write size=901742 2024-05-30T16:03:58.205 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.207+0000 7f08851138c0 1 < completed delete operation key=TPCETCO3Y1M6 tag=W9U0JHU8RDNF type=del 2024-05-30T16:03:58.208 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.207+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=H64VWAR3KIFE type=del size=2664714 2024-05-30T16:03:58.212 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.211+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~FXN7KFKS91G8.0 size=1048576 2024-05-30T16:03:58.215 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.215+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~FXN7KFKS91G8.1 size=461752 2024-05-30T16:03:58.221 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.223+0000 7f08851138c0 1 < completed multipart upload key=O8CWPJ9GW2GO upload=FXN7KFKS91G8 tag=KKYFA1PY5D4U type=write size=2558904 2024-05-30T16:03:58.228 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.227+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~G20B9XO3HH4H.0 size=314992 2024-05-30T16:03:58.228 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.227+0000 7f08851138c0 1 > prepared multipart upload key=XHGMUYBA2VNK upload=G20B9XO3HH4H tag=YBKD5H03RYJE type=write size=1363568 2024-05-30T16:03:58.236 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.235+0000 7f08851138c0 1 < suggested remove operation key=603HBR8PRH6L tag=9SPG3WV97OSF type=del 2024-05-30T16:03:58.239 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.239+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=NZ3WQF0010M3 type=del size=155278 2024-05-30T16:03:58.243 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.243+0000 7f08851138c0 1 < completed delete operation key=YRP4CSG92QRO tag=9H61CQFRCFET type=del 2024-05-30T16:03:58.247 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.247+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=NHGILBJJQ9MD type=write size=38629 2024-05-30T16:03:58.251 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.251+0000 7f08851138c0 1 < completed delete operation key=WT9RC0UYVG4F tag=GPNA21WKQEVQ type=del 2024-05-30T16:03:58.255 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.255+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=LMZ5X1XGZUX4 type=del size=3440104 2024-05-30T16:03:58.259 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.259+0000 7f08851138c0 1 < completed delete operation key=O8CWPJ9GW2GO tag=1Q2ZVQ2HT3DC type=del 2024-05-30T16:03:58.263 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.263+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=XY45889KAPFG type=write size=1024271 2024-05-30T16:03:58.267 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.267+0000 7f08851138c0 1 < completed delete operation key=YRP4CSG92QRO tag=PZATTM5C6WEE type=del 2024-05-30T16:03:58.270 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.271+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=71XG8IYY2B73 type=del size=2637115 2024-05-30T16:03:58.274 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.275+0000 7f08851138c0 1 < completed write operation key=4N58ZY4E1H6E tag=7AB8VK5E2NHB type=write size=175593 2024-05-30T16:03:58.281 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.279+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~CIR03L85JE4G.0 size=1048576 2024-05-30T16:03:58.286 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.283+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~CIR03L85JE4G.1 size=576219 2024-05-30T16:03:58.286 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.283+0000 7f08851138c0 1 > prepared multipart upload key=4N58ZY4E1H6E upload=CIR03L85JE4G tag=LAEVHY1S010A type=write size=2673371 2024-05-30T16:03:58.287 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.287+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~J1QQ34ALU3E1.0 size=1048576 2024-05-30T16:03:58.289 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.291+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~J1QQ34ALU3E1.1 size=1048576 2024-05-30T16:03:58.292 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.291+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~J1QQ34ALU3E1.2 size=953684 2024-05-30T16:03:58.298 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.299+0000 7f08851138c0 1 < completed multipart upload key=GWXNCSJCLXKW upload=J1QQ34ALU3E1 tag=USYO736BE44T type=write size=4099412 2024-05-30T16:03:58.301 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.303+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=B5E7RDQQQZ1B type=del size=3806570 2024-05-30T16:03:58.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.303+0000 7f08851138c0 1 < completed delete operation key=O0DJ8SCY766V tag=L53HEY9RT09P type=del 2024-05-30T16:03:58.313 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.311+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~9HSILQ7ATAX9.0 size=1048576 2024-05-30T16:03:58.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.315+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~9HSILQ7ATAX9.1 size=1048576 2024-05-30T16:03:58.318 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.319+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~9HSILQ7ATAX9.2 size=15082 2024-05-30T16:03:58.318 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.319+0000 7f08851138c0 1 > prepared multipart upload key=OKBB0VU765TW upload=9HSILQ7ATAX9 tag=MNV6V70QT9XV type=write size=3160810 2024-05-30T16:03:58.322 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.323+0000 7f08851138c0 1 < completed delete operation key=XHGMUYBA2VNK tag=5KIGNC38AX5N type=del 2024-05-30T16:03:58.325 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.327+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=WGVK7VIR7F69 type=del size=18712 2024-05-30T16:03:58.329 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.327+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~779ATCN05HCW.0 size=400824 2024-05-30T16:03:58.334 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.335+0000 7f08851138c0 1 < completed multipart upload key=PLWFSKPKVO19 upload=779ATCN05HCW tag=04UEGYAXI4LZ type=write size=1449400 2024-05-30T16:03:58.338 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.339+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=MDAVVFJZTDJW type=del size=1974116 2024-05-30T16:03:58.341 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.339+0000 7f08851138c0 1 < canceled operation key=0T2E9F3K1USF tag=ZW1JGYP5XRAP type=del size=1292583 2024-05-30T16:03:58.344 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.343+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=HH6DP02TI745 type=del size=2608286 2024-05-30T16:03:58.351 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.351+0000 7f08851138c0 1 < suggested remove operation key=QQILXRJD0C1Z tag=5B4KQK39NQAN type=del 2024-05-30T16:03:58.357 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.355+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~V7R0QGUR59XG.0 size=1048576 2024-05-30T16:03:58.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.359+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~V7R0QGUR59XG.1 size=741314 2024-05-30T16:03:58.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.359+0000 7f08851138c0 1 > prepared multipart upload key=YRP4CSG92QRO upload=V7R0QGUR59XG tag=HGX8LNUZDT3S type=write size=2838466 2024-05-30T16:03:58.363 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.363+0000 7f08851138c0 1 < completed delete operation key=WT9RC0UYVG4F tag=B5CAN6BQPLNR type=del 2024-05-30T16:03:58.366 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.367+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=KK6MP5PJ37I0 type=del size=1029151 2024-05-30T16:03:58.370 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.371+0000 7f08851138c0 1 < completed write operation key=RCTV94WRMO53 tag=ZL8BVZ51FJHI type=write size=901742 2024-05-30T16:03:58.375 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.375+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~VX9ZUWDOKH6X.0 size=357461 2024-05-30T16:03:58.376 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.375+0000 7f08851138c0 1 > prepared multipart upload key=O8CWPJ9GW2GO upload=VX9ZUWDOKH6X tag=3C3B3B6F8H1C type=write size=1406037 2024-05-30T16:03:58.382 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.383+0000 7f08851138c0 1 < suggested update operation key=KPHZGFJLC2E0 tag=H64VWAR3KIFE type=del size=2664714 2024-05-30T16:03:58.385 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.387+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=2BFYLV8N3TJ6 type=del size=205180 2024-05-30T16:03:58.388 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.387+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~G20B9XO3HH4H.0 size=314992 2024-05-30T16:03:58.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.395+0000 7f08851138c0 1 < completed multipart upload key=XHGMUYBA2VNK upload=G20B9XO3HH4H tag=YBKD5H03RYJE type=write size=1363568 2024-05-30T16:03:58.397 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.395+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=Y6O2QAUKDRE9 type=del size=3436052 2024-05-30T16:03:58.400 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.399+0000 7f08851138c0 1 < completed delete operation key=MMLF65KLC42Q tag=NZ3WQF0010M3 type=del 2024-05-30T16:03:58.403 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.403+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=S7PRN8B1KHMR type=del size=3236346 2024-05-30T16:03:58.406 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.407+0000 7f08851138c0 1 < canceled operation key=4N58ZY4E1H6E tag=NHGILBJJQ9MD type=write size=38629 2024-05-30T16:03:58.412 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.411+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~1H3E7DC8CGHG.0 size=983954 2024-05-30T16:03:58.412 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.411+0000 7f08851138c0 1 > prepared multipart upload key=E2FXSXOEUGIG upload=1H3E7DC8CGHG tag=NZQAJ0HSUJMI type=write size=2032530 2024-05-30T16:03:58.415 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.415+0000 7f08851138c0 1 < completed delete operation key=0RZRBW38WKNZ tag=LMZ5X1XGZUX4 type=del 2024-05-30T16:03:58.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.423+0000 7f08851138c0 1 > prepared part key=_multipart_MMLF65KLC42Q.2~IGP282AFJHJS.0 size=1048576 2024-05-30T16:03:58.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.423+0000 7f08851138c0 1 > prepared part key=_multipart_MMLF65KLC42Q.2~IGP282AFJHJS.1 size=264232 2024-05-30T16:03:58.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.423+0000 7f08851138c0 1 > prepared multipart upload key=MMLF65KLC42Q upload=IGP282AFJHJS tag=OT54YUQW11WI type=write size=2361384 2024-05-30T16:03:58.428 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.427+0000 7f08851138c0 1 < completed write operation key=1N6R2HBTWMP2 tag=XY45889KAPFG type=write size=1024271 2024-05-30T16:03:58.431 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.431+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=H29296SREV7T type=del size=1392334 2024-05-30T16:03:58.435 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.435+0000 7f08851138c0 1 < completed delete operation key=603HBR8PRH6L tag=71XG8IYY2B73 type=del 2024-05-30T16:03:58.438 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.439+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=WVUHYOXP0AK7 type=write size=427551 2024-05-30T16:03:58.442 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.443+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~CIR03L85JE4G.0 size=1048576 2024-05-30T16:03:58.444 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.443+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~CIR03L85JE4G.1 size=576219 2024-05-30T16:03:58.450 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.451+0000 7f08851138c0 1 < completed multipart upload key=4N58ZY4E1H6E upload=CIR03L85JE4G tag=LAEVHY1S010A type=write size=2673371 2024-05-30T16:03:58.453 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.451+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=L31D9PTRRVCW type=write size=133643 2024-05-30T16:03:58.456 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.455+0000 7f08851138c0 1 < completed delete operation key=4N58ZY4E1H6E tag=B5E7RDQQQZ1B type=del 2024-05-30T16:03:58.460 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.459+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=JBWRWM0NMZFO type=del size=1649674 2024-05-30T16:03:58.464 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.463+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~9HSILQ7ATAX9.0 size=1048576 2024-05-30T16:03:58.464 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.463+0000 7f08851138c0 1 < canceled part key=_multipart_OKBB0VU765TW.2~9HSILQ7ATAX9.1 size=1048576 2024-05-30T16:03:58.466 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.467+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~9HSILQ7ATAX9.2 size=15082 2024-05-30T16:03:58.472 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.471+0000 7f08851138c0 1 < canceled multipart upload key=OKBB0VU765TW upload=9HSILQ7ATAX9 tag=MNV6V70QT9XV type=write size=3160810 2024-05-30T16:03:58.475 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.475+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=12N0MPGI4DWL type=del size=1670221 2024-05-30T16:03:58.478 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.479+0000 7f08851138c0 1 < completed delete operation key=O8CWPJ9GW2GO tag=WGVK7VIR7F69 type=del 2024-05-30T16:03:58.484 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.483+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~O7MKXY9PPWQ2.0 size=1048576 2024-05-30T16:03:58.487 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.487+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~O7MKXY9PPWQ2.1 size=975841 2024-05-30T16:03:58.487 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.487+0000 7f08851138c0 1 > prepared multipart upload key=PLWFSKPKVO19 upload=O7MKXY9PPWQ2 tag=OPSRCHNV53AR type=write size=3072993 2024-05-30T16:03:58.490 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.491+0000 7f08851138c0 1 < completed delete operation key=YRP4CSG92QRO tag=MDAVVFJZTDJW type=del 2024-05-30T16:03:58.494 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.495+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=3BNZYDC3RVHF type=del size=1860462 2024-05-30T16:03:58.497 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.495+0000 7f08851138c0 1 < completed delete operation key=3IJWPWRB6FHE tag=HH6DP02TI745 type=del 2024-05-30T16:03:58.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.499+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=LV3FWPQIOKRO type=del size=2331785 2024-05-30T16:03:58.504 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.503+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~V7R0QGUR59XG.0 size=1048576 2024-05-30T16:03:58.507 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.507+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~V7R0QGUR59XG.1 size=741314 2024-05-30T16:03:58.513 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.511+0000 7f08851138c0 1 < completed multipart upload key=YRP4CSG92QRO upload=V7R0QGUR59XG tag=HGX8LNUZDT3S type=write size=2838466 2024-05-30T16:03:58.516 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.515+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=1NE2KZ3UHNGA type=del size=2605351 2024-05-30T16:03:58.522 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.523+0000 7f08851138c0 1 < completed delete operation key=YRP4CSG92QRO tag=KK6MP5PJ37I0 type=del 2024-05-30T16:03:58.528 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.527+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~K3POGZ8D9MAP.0 size=1048576 2024-05-30T16:03:58.531 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.531+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~K3POGZ8D9MAP.1 size=1048576 2024-05-30T16:03:58.534 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.535+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~K3POGZ8D9MAP.2 size=7480 2024-05-30T16:03:58.534 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.535+0000 7f08851138c0 1 > prepared multipart upload key=YRP4CSG92QRO upload=K3POGZ8D9MAP tag=9XA5QCR5X3OH type=write size=3153208 2024-05-30T16:03:58.538 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.539+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~VX9ZUWDOKH6X.0 size=357461 2024-05-30T16:03:58.544 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.543+0000 7f08851138c0 1 < completed multipart upload key=O8CWPJ9GW2GO upload=VX9ZUWDOKH6X tag=3C3B3B6F8H1C type=write size=1406037 2024-05-30T16:03:58.547 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.547+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=MJ6UUDVK4T9J type=del size=549217 2024-05-30T16:03:58.550 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.551+0000 7f08851138c0 1 < completed delete operation key=3IJWPWRB6FHE tag=2BFYLV8N3TJ6 type=del 2024-05-30T16:03:58.557 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.555+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~5Y1Z5DX8BH00.0 size=579726 2024-05-30T16:03:58.557 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.555+0000 7f08851138c0 1 > prepared multipart upload key=GWXNCSJCLXKW upload=5Y1Z5DX8BH00 tag=FMOVUDNGZ640 type=write size=1628302 2024-05-30T16:03:58.560 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.559+0000 7f08851138c0 1 < canceled operation key=1N6R2HBTWMP2 tag=Y6O2QAUKDRE9 type=del size=3436052 2024-05-30T16:03:58.567 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.567+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~66TFFY87DBFY.0 size=1048576 2024-05-30T16:03:58.570 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.571+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~66TFFY87DBFY.1 size=86205 2024-05-30T16:03:58.570 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.571+0000 7f08851138c0 1 > prepared multipart upload key=0RZRBW38WKNZ upload=66TFFY87DBFY tag=6XU7EZ0XI6PT type=write size=2183357 2024-05-30T16:03:58.574 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.575+0000 7f08851138c0 1 < completed delete operation key=OKBB0VU765TW tag=S7PRN8B1KHMR type=del 2024-05-30T16:03:58.580 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.579+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~ONQR7EFQ19CT.0 size=771107 2024-05-30T16:03:58.580 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.579+0000 7f08851138c0 1 > prepared multipart upload key=QQILXRJD0C1Z upload=ONQR7EFQ19CT tag=I7252T0O9YX1 type=write size=1819683 2024-05-30T16:03:58.584 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.583+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~1H3E7DC8CGHG.0 size=983954 2024-05-30T16:03:58.590 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.591+0000 7f08851138c0 1 < completed multipart upload key=E2FXSXOEUGIG upload=1H3E7DC8CGHG tag=NZQAJ0HSUJMI type=write size=2032530 2024-05-30T16:03:58.594 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.595+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=4F4J9OKWH2EW type=del size=1803004 2024-05-30T16:03:58.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.599+0000 7f08851138c0 1 < completed part key=_multipart_MMLF65KLC42Q.2~IGP282AFJHJS.0 size=1048576 2024-05-30T16:03:58.601 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.599+0000 7f08851138c0 1 < completed part key=_multipart_MMLF65KLC42Q.2~IGP282AFJHJS.1 size=264232 2024-05-30T16:03:58.607 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.607+0000 7f08851138c0 1 < completed multipart upload key=MMLF65KLC42Q upload=IGP282AFJHJS tag=OT54YUQW11WI type=write size=2361384 2024-05-30T16:03:58.613 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.615+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~ZM7AEKQ3V6U9.0 size=538472 2024-05-30T16:03:58.613 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.615+0000 7f08851138c0 1 > prepared multipart upload key=4N58ZY4E1H6E upload=ZM7AEKQ3V6U9 tag=BDC339L0L7MA type=write size=1587048 2024-05-30T16:03:58.617 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.615+0000 7f08851138c0 1 < canceled operation key=TPCETCO3Y1M6 tag=H29296SREV7T type=del size=1392334 2024-05-30T16:03:58.624 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.623+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~G7DHEWLLGUF5.0 size=116085 2024-05-30T16:03:58.624 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.623+0000 7f08851138c0 1 > prepared multipart upload key=VRXNO8JLAZYH upload=G7DHEWLLGUF5 tag=IPGD9ACVA8B7 type=write size=1164661 2024-05-30T16:03:58.627 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.627+0000 7f08851138c0 1 < completed write operation key=364U3OV10VPD tag=WVUHYOXP0AK7 type=write size=427551 2024-05-30T16:03:58.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.635+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~D9ESZB1XYAIW.0 size=1048576 2024-05-30T16:03:58.637 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.639+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~D9ESZB1XYAIW.1 size=1048576 2024-05-30T16:03:58.640 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.639+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~D9ESZB1XYAIW.2 size=167977 2024-05-30T16:03:58.640 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.639+0000 7f08851138c0 1 > prepared multipart upload key=JNZI7JX95LX2 upload=D9ESZB1XYAIW tag=BETGGTAJRDX2 type=write size=3313705 2024-05-30T16:03:58.644 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.643+0000 7f08851138c0 1 < completed write operation key=Y3CYZJ83WYNI tag=L31D9PTRRVCW type=write size=133643 2024-05-30T16:03:58.647 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.647+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=A81U691O2FG3 type=del size=1227790 2024-05-30T16:03:58.659 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.659+0000 7f08851138c0 1 < suggested remove operation key=PLWFSKPKVO19 tag=JBWRWM0NMZFO type=del 2024-05-30T16:03:58.663 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.663+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=51TG8D366IRA type=del size=3268621 2024-05-30T16:03:58.672 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.671+0000 7f08851138c0 1 < suggested update operation key=RCTV94WRMO53 tag=12N0MPGI4DWL type=del size=1670221 2024-05-30T16:03:58.680 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.679+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~QW2YXUNC1V1F.0 size=1048576 2024-05-30T16:03:58.682 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.683+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~QW2YXUNC1V1F.1 size=100588 2024-05-30T16:03:58.683 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.683+0000 7f08851138c0 1 > prepared multipart upload key=3TBOZWT2P8U8 upload=QW2YXUNC1V1F tag=JHZ163COJBWQ type=write size=2197740 2024-05-30T16:03:58.685 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.687+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~O7MKXY9PPWQ2.0 size=1048576 2024-05-30T16:03:58.688 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.687+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~O7MKXY9PPWQ2.1 size=975841 2024-05-30T16:03:58.693 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.691+0000 7f08851138c0 1 < completed multipart upload key=PLWFSKPKVO19 upload=O7MKXY9PPWQ2 tag=OPSRCHNV53AR type=write size=3072993 2024-05-30T16:03:58.698 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.695+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=528RMGE7EFDS type=del size=2728437 2024-05-30T16:03:58.699 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.699+0000 7f08851138c0 1 < completed delete operation key=4N58ZY4E1H6E tag=3BNZYDC3RVHF type=del 2024-05-30T16:03:58.705 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.703+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~4E0O961DE3T4.0 size=910851 2024-05-30T16:03:58.705 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.703+0000 7f08851138c0 1 > prepared multipart upload key=QQILXRJD0C1Z upload=4E0O961DE3T4 tag=XEQE3AQRUF9C type=write size=1959427 2024-05-30T16:03:58.712 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.711+0000 7f08851138c0 1 < suggested remove operation key=73FJ3HGWYQQK tag=LV3FWPQIOKRO type=del 2024-05-30T16:03:58.717 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.719+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~TFBTY65GZ57G.0 size=1048576 2024-05-30T16:03:58.720 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.719+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~TFBTY65GZ57G.1 size=1048576 2024-05-30T16:03:58.723 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.723+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~TFBTY65GZ57G.2 size=936573 2024-05-30T16:03:58.723 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.723+0000 7f08851138c0 1 > prepared multipart upload key=QQILXRJD0C1Z upload=TFBTY65GZ57G tag=E3286MCD94J6 type=write size=4082301 2024-05-30T16:03:58.729 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.731+0000 7f08851138c0 1 < suggested remove operation key=YRP4CSG92QRO tag=1NE2KZ3UHNGA type=del 2024-05-30T16:03:58.732 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.731+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=G8WK2BM9B1F8 type=write size=141513 2024-05-30T16:03:58.735 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.735+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~K3POGZ8D9MAP.0 size=1048576 2024-05-30T16:03:58.737 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.739+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~K3POGZ8D9MAP.1 size=1048576 2024-05-30T16:03:58.740 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.739+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~K3POGZ8D9MAP.2 size=7480 2024-05-30T16:03:58.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.747+0000 7f08851138c0 1 < completed multipart upload key=YRP4CSG92QRO upload=K3POGZ8D9MAP tag=9XA5QCR5X3OH type=write size=3153208 2024-05-30T16:03:58.748 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.747+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=PPL3B7MNB0LX type=del size=482965 2024-05-30T16:03:58.751 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.751+0000 7f08851138c0 1 < completed delete operation key=3TBOZWT2P8U8 tag=MJ6UUDVK4T9J type=del 2024-05-30T16:03:58.754 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.755+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=M2FS31KA81GX type=del size=3158152 2024-05-30T16:03:58.757 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.759+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~5Y1Z5DX8BH00.0 size=579726 2024-05-30T16:03:58.763 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.763+0000 7f08851138c0 1 < completed multipart upload key=GWXNCSJCLXKW upload=5Y1Z5DX8BH00 tag=FMOVUDNGZ640 type=write size=1628302 2024-05-30T16:03:58.766 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.767+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=R1SXLAOQ44KO type=del size=3752319 2024-05-30T16:03:58.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.771+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~66TFFY87DBFY.0 size=1048576 2024-05-30T16:03:58.772 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.771+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~66TFFY87DBFY.1 size=86205 2024-05-30T16:03:58.777 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.779+0000 7f08851138c0 1 < completed multipart upload key=0RZRBW38WKNZ upload=66TFFY87DBFY tag=6XU7EZ0XI6PT type=write size=2183357 2024-05-30T16:03:58.783 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.783+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~G3W3G3RKPIUD.0 size=1048576 2024-05-30T16:03:58.786 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.787+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~G3W3G3RKPIUD.1 size=165607 2024-05-30T16:03:58.786 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.787+0000 7f08851138c0 1 > prepared multipart upload key=NIV7AQ4EV5YP upload=G3W3G3RKPIUD tag=XHDT5JIM2OMV type=write size=2262759 2024-05-30T16:03:58.789 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.787+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~ONQR7EFQ19CT.0 size=771107 2024-05-30T16:03:58.794 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.795+0000 7f08851138c0 1 < completed multipart upload key=QQILXRJD0C1Z upload=ONQR7EFQ19CT tag=I7252T0O9YX1 type=write size=1819683 2024-05-30T16:03:58.800 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.799+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~92IS5HFYNJC0.0 size=1048576 2024-05-30T16:03:58.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.803+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~92IS5HFYNJC0.1 size=841757 2024-05-30T16:03:58.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.803+0000 7f08851138c0 1 > prepared multipart upload key=E2FXSXOEUGIG upload=92IS5HFYNJC0 tag=Y4923AVV3UNH type=write size=2938909 2024-05-30T16:03:58.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.807+0000 7f08851138c0 1 < completed delete operation key=QCLY10D2A3VF tag=4F4J9OKWH2EW type=del 2024-05-30T16:03:58.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.807+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=KNEQ2I1OOONG type=del size=1867906 2024-05-30T16:03:58.812 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.811+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~ZM7AEKQ3V6U9.0 size=538472 2024-05-30T16:03:58.817 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.815+0000 7f08851138c0 1 < completed multipart upload key=4N58ZY4E1H6E upload=ZM7AEKQ3V6U9 tag=BDC339L0L7MA type=write size=1587048 2024-05-30T16:03:58.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.819+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=7K5Q9T15974J type=del size=4089967 2024-05-30T16:03:58.823 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.823+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~G7DHEWLLGUF5.0 size=116085 2024-05-30T16:03:58.829 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.827+0000 7f08851138c0 1 < completed multipart upload key=VRXNO8JLAZYH upload=G7DHEWLLGUF5 tag=IPGD9ACVA8B7 type=write size=1164661 2024-05-30T16:03:58.832 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.831+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=GO2543OMQ7A6 type=del size=3034501 2024-05-30T16:03:58.837 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.835+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~D9ESZB1XYAIW.0 size=1048576 2024-05-30T16:03:58.843 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.843+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~D9ESZB1XYAIW.1 size=1048576 2024-05-30T16:03:58.846 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.847+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~D9ESZB1XYAIW.2 size=167977 2024-05-30T16:03:58.851 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.851+0000 7f08851138c0 1 < completed multipart upload key=JNZI7JX95LX2 upload=D9ESZB1XYAIW tag=BETGGTAJRDX2 type=write size=3313705 2024-05-30T16:03:58.854 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.855+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=YLOXNRS6R61O type=write size=463937 2024-05-30T16:03:58.857 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.859+0000 7f08851138c0 1 < completed delete operation key=O8CWPJ9GW2GO tag=A81U691O2FG3 type=del 2024-05-30T16:03:58.863 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.863+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~EKEO4SKAOSL0.0 size=1048576 2024-05-30T16:03:58.865 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.863+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~EKEO4SKAOSL0.1 size=1048576 2024-05-30T16:03:58.868 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.867+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~EKEO4SKAOSL0.2 size=990040 2024-05-30T16:03:58.868 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.867+0000 7f08851138c0 1 > prepared multipart upload key=V4OQFOOQUXFQ upload=EKEO4SKAOSL0 tag=Z9PDXAF19U6N type=write size=4135768 2024-05-30T16:03:58.871 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.871+0000 7f08851138c0 1 < canceled operation key=0T2E9F3K1USF tag=51TG8D366IRA type=del size=3268621 2024-05-30T16:03:58.874 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.875+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=RCVWZ5Y583UA type=del size=2707963 2024-05-30T16:03:58.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.879+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~QW2YXUNC1V1F.0 size=1048576 2024-05-30T16:03:58.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.879+0000 7f08851138c0 1 < canceled part key=_multipart_3TBOZWT2P8U8.2~QW2YXUNC1V1F.1 size=100588 2024-05-30T16:03:58.882 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.883+0000 7f08851138c0 1 < canceled multipart upload key=3TBOZWT2P8U8 upload=QW2YXUNC1V1F tag=JHZ163COJBWQ type=write size=2197740 2024-05-30T16:03:58.885 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.887+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=HVWB8KOKF54Z type=del size=905772 2024-05-30T16:03:58.889 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.887+0000 7f08851138c0 1 < completed delete operation key=V4OQFOOQUXFQ tag=528RMGE7EFDS type=del 2024-05-30T16:03:58.892 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.891+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=MGH2SFAM1R3X type=del size=2063171 2024-05-30T16:03:58.896 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.895+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~4E0O961DE3T4.0 size=910851 2024-05-30T16:03:58.902 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.899+0000 7f08851138c0 1 < completed multipart upload key=QQILXRJD0C1Z upload=4E0O961DE3T4 tag=XEQE3AQRUF9C type=write size=1959427 2024-05-30T16:03:58.904 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.903+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=4ZWXTXVX0YU7 type=del size=2944703 2024-05-30T16:03:58.907 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.907+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~TFBTY65GZ57G.0 size=1048576 2024-05-30T16:03:58.907 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.907+0000 7f08851138c0 1 < canceled part key=_multipart_QQILXRJD0C1Z.2~TFBTY65GZ57G.1 size=1048576 2024-05-30T16:03:58.910 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.911+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~TFBTY65GZ57G.2 size=936573 2024-05-30T16:03:58.915 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.915+0000 7f08851138c0 1 < canceled multipart upload key=QQILXRJD0C1Z upload=TFBTY65GZ57G tag=E3286MCD94J6 type=write size=4082301 2024-05-30T16:03:58.918 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.919+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=K2L4QS8RVN9Q type=write size=402549 2024-05-30T16:03:58.921 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.923+0000 7f08851138c0 1 < completed write operation key=JNZI7JX95LX2 tag=G8WK2BM9B1F8 type=write size=141513 2024-05-30T16:03:58.924 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.923+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=M445J1A9QDO8 type=del size=3166362 2024-05-30T16:03:58.931 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.931+0000 7f08851138c0 1 < suggested update operation key=QQILXRJD0C1Z tag=PPL3B7MNB0LX type=del size=482965 2024-05-30T16:03:58.934 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.935+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=22MTXHDIQXQ8 type=del size=2314902 2024-05-30T16:03:58.938 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.939+0000 7f08851138c0 1 < completed delete operation key=603HBR8PRH6L tag=M2FS31KA81GX type=del 2024-05-30T16:03:58.941 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.939+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=K78C35AMCEKM type=del size=2708774 2024-05-30T16:03:58.948 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.947+0000 7f08851138c0 1 < suggested update operation key=3TBOZWT2P8U8 tag=R1SXLAOQ44KO type=del size=3752319 2024-05-30T16:03:58.954 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.955+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~V25KIALZROIE.0 size=659620 2024-05-30T16:03:58.954 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.955+0000 7f08851138c0 1 > prepared multipart upload key=3IJWPWRB6FHE upload=V25KIALZROIE tag=LT74K0A9FUDT type=write size=1708196 2024-05-30T16:03:58.957 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.959+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~G3W3G3RKPIUD.0 size=1048576 2024-05-30T16:03:58.960 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.959+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~G3W3G3RKPIUD.1 size=165607 2024-05-30T16:03:58.966 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.967+0000 7f08851138c0 1 < completed multipart upload key=NIV7AQ4EV5YP upload=G3W3G3RKPIUD tag=XHDT5JIM2OMV type=write size=2262759 2024-05-30T16:03:58.969 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.971+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=3RL3JG7XT4XL type=del size=2479375 2024-05-30T16:03:58.973 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.975+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~92IS5HFYNJC0.0 size=1048576 2024-05-30T16:03:58.973 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.975+0000 7f08851138c0 1 < canceled part key=_multipart_E2FXSXOEUGIG.2~92IS5HFYNJC0.1 size=841757 2024-05-30T16:03:58.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.979+0000 7f08851138c0 1 < canceled multipart upload key=E2FXSXOEUGIG upload=92IS5HFYNJC0 tag=Y4923AVV3UNH type=write size=2938909 2024-05-30T16:03:58.983 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.983+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=8IFWCBE5UI95 type=write size=260866 2024-05-30T16:03:58.989 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.987+0000 7f08851138c0 1 < suggested remove operation key=73FJ3HGWYQQK tag=KNEQ2I1OOONG type=del 2024-05-30T16:03:58.995 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.995+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~4NGND8GUQIJ1.0 size=1048576 2024-05-30T16:03:58.997 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.999+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~4NGND8GUQIJ1.1 size=1048576 2024-05-30T16:03:59.000 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.999+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~4NGND8GUQIJ1.2 size=672125 2024-05-30T16:03:59.000 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:58.999+0000 7f08851138c0 1 > prepared multipart upload key=E2FXSXOEUGIG upload=4NGND8GUQIJ1 tag=01HCI8WQE7IK type=write size=3817853 2024-05-30T16:03:59.004 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.003+0000 7f08851138c0 1 < completed delete operation key=QCLY10D2A3VF tag=7K5Q9T15974J type=del 2024-05-30T16:03:59.007 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.007+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=2PJZDT4V34IG type=del size=1539130 2024-05-30T16:03:59.011 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.011+0000 7f08851138c0 1 < completed delete operation key=QCLY10D2A3VF tag=GO2543OMQ7A6 type=del 2024-05-30T16:03:59.016 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.015+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=SCWHJXFNE333 type=del size=2636578 2024-05-30T16:03:59.018 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.019+0000 7f08851138c0 1 < completed write operation key=603HBR8PRH6L tag=YLOXNRS6R61O type=write size=463937 2024-05-30T16:03:59.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.019+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=2829QIMDIJZ6 type=del size=1304367 2024-05-30T16:03:59.024 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.023+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~EKEO4SKAOSL0.0 size=1048576 2024-05-30T16:03:59.027 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.027+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~EKEO4SKAOSL0.1 size=1048576 2024-05-30T16:03:59.029 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.031+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~EKEO4SKAOSL0.2 size=990040 2024-05-30T16:03:59.035 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.035+0000 7f08851138c0 1 < completed multipart upload key=V4OQFOOQUXFQ upload=EKEO4SKAOSL0 tag=Z9PDXAF19U6N type=write size=4135768 2024-05-30T16:03:59.041 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.039+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~JT0KAZPUOATX.0 size=1048576 2024-05-30T16:03:59.043 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.043+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~JT0KAZPUOATX.1 size=217231 2024-05-30T16:03:59.043 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.043+0000 7f08851138c0 1 > prepared multipart upload key=WT9RC0UYVG4F upload=JT0KAZPUOATX tag=VD7O8HEEU02Z type=write size=2314383 2024-05-30T16:03:59.047 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.047+0000 7f08851138c0 1 < completed delete operation key=TPCETCO3Y1M6 tag=RCVWZ5Y583UA type=del 2024-05-30T16:03:59.050 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.051+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=WHFSV0VR1O5T type=del size=3566520 2024-05-30T16:03:59.053 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.051+0000 7f08851138c0 1 < completed delete operation key=XHGMUYBA2VNK tag=HVWB8KOKF54Z type=del 2024-05-30T16:03:59.056 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.055+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=KVERLZTV75VY type=write size=447054 2024-05-30T16:03:59.060 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.059+0000 7f08851138c0 1 < completed delete operation key=Y3CYZJ83WYNI tag=MGH2SFAM1R3X type=del 2024-05-30T16:03:59.063 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.063+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=SERI4S6NIOI0 type=del size=99613 2024-05-30T16:03:59.067 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.067+0000 7f08851138c0 1 < completed delete operation key=4N58ZY4E1H6E tag=4ZWXTXVX0YU7 type=del 2024-05-30T16:03:59.070 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.071+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=R48433IHX62I type=del size=251526 2024-05-30T16:03:59.074 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.075+0000 7f08851138c0 1 < completed write operation key=Y3CYZJ83WYNI tag=K2L4QS8RVN9Q type=write size=402549 2024-05-30T16:03:59.080 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.079+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~W1Y8878X74A5.0 size=1048576 2024-05-30T16:03:59.083 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.083+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~W1Y8878X74A5.1 size=1048576 2024-05-30T16:03:59.085 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.087+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~W1Y8878X74A5.2 size=1042226 2024-05-30T16:03:59.085 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.087+0000 7f08851138c0 1 > prepared multipart upload key=364U3OV10VPD upload=W1Y8878X74A5 tag=MAQDV0G89N1U type=write size=4187954 2024-05-30T16:03:59.089 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.087+0000 7f08851138c0 1 < completed delete operation key=4N58ZY4E1H6E tag=M445J1A9QDO8 type=del 2024-05-30T16:03:59.092 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.091+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=W22T6YF14JEV type=del size=3371279 2024-05-30T16:03:59.096 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.095+0000 7f08851138c0 -1 < failed to complete operation key=73FJ3HGWYQQK tag=22MTXHDIQXQ8 type=del size=2314902: (22) Invalid argument 2024-05-30T16:03:59.099 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.099+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=ZGOHJMJ6M3SY type=del size=2595988 2024-05-30T16:03:59.106 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.103+0000 7f08851138c0 1 < canceled operation key=Y3CYZJ83WYNI tag=K78C35AMCEKM type=del size=2708774 2024-05-30T16:03:59.106 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.107+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=F1FA6PPM701O type=del size=1762605 2024-05-30T16:03:59.110 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.111+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~V25KIALZROIE.0 size=659620 2024-05-30T16:03:59.115 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.115+0000 7f08851138c0 1 < completed multipart upload key=3IJWPWRB6FHE upload=V25KIALZROIE tag=LT74K0A9FUDT type=write size=1708196 2024-05-30T16:03:59.122 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.123+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~JMT63KO77J6O.0 size=1048576 2024-05-30T16:03:59.124 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.123+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~JMT63KO77J6O.1 size=1048576 2024-05-30T16:03:59.127 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.127+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~JMT63KO77J6O.2 size=810447 2024-05-30T16:03:59.127 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.127+0000 7f08851138c0 1 > prepared multipart upload key=O8CWPJ9GW2GO upload=JMT63KO77J6O tag=RY3H7BPC453A type=write size=3956175 2024-05-30T16:03:59.130 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.131+0000 7f08851138c0 1 < completed delete operation key=E2FXSXOEUGIG tag=3RL3JG7XT4XL type=del 2024-05-30T16:03:59.133 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.135+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=HJF3WDVKQGIQ type=del size=1012062 2024-05-30T16:03:59.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.135+0000 7f08851138c0 1 < completed write operation key=603HBR8PRH6L tag=8IFWCBE5UI95 type=write size=260866 2024-05-30T16:03:59.140 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.139+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=1S5WQT4UYOXU type=del size=2638795 2024-05-30T16:03:59.144 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.143+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~4NGND8GUQIJ1.0 size=1048576 2024-05-30T16:03:59.146 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.147+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~4NGND8GUQIJ1.1 size=1048576 2024-05-30T16:03:59.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.151+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~4NGND8GUQIJ1.2 size=672125 2024-05-30T16:03:59.160 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.159+0000 7f08851138c0 1 < completed multipart upload key=E2FXSXOEUGIG upload=4NGND8GUQIJ1 tag=01HCI8WQE7IK type=write size=3817853 2024-05-30T16:03:59.165 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.167+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~QDTBL3JGB1C3.0 size=102156 2024-05-30T16:03:59.165 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.167+0000 7f08851138c0 1 > prepared multipart upload key=3IJWPWRB6FHE upload=QDTBL3JGB1C3 tag=5KLQCAWZCFAW type=write size=1150732 2024-05-30T16:03:59.169 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.167+0000 7f08851138c0 1 < canceled operation key=603HBR8PRH6L tag=2PJZDT4V34IG type=del size=1539130 2024-05-30T16:03:59.175 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.175+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~SDFLNLTPDGEY.0 size=1048576 2024-05-30T16:03:59.176 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.175+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~SDFLNLTPDGEY.1 size=1048576 2024-05-30T16:03:59.179 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.179+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~SDFLNLTPDGEY.2 size=206769 2024-05-30T16:03:59.179 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.179+0000 7f08851138c0 1 > prepared multipart upload key=QQILXRJD0C1Z upload=SDFLNLTPDGEY tag=NXGED651PKDO type=write size=3352497 2024-05-30T16:03:59.183 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.183+0000 7f08851138c0 1 < completed delete operation key=E2FXSXOEUGIG tag=SCWHJXFNE333 type=del 2024-05-30T16:03:59.189 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.187+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~51TVKHCDIUBE.0 size=1048576 2024-05-30T16:03:59.192 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.191+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~51TVKHCDIUBE.1 size=1048576 2024-05-30T16:03:59.195 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.195+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~51TVKHCDIUBE.2 size=830584 2024-05-30T16:03:59.195 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.195+0000 7f08851138c0 1 > prepared multipart upload key=364U3OV10VPD upload=51TVKHCDIUBE tag=ZWW1IWN38HAL type=write size=3976312 2024-05-30T16:03:59.198 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.199+0000 7f08851138c0 1 < completed delete operation key=YRP4CSG92QRO tag=2829QIMDIJZ6 type=del 2024-05-30T16:03:59.202 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.203+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=WNZO2U0YIJ0V type=write size=539016 2024-05-30T16:03:59.206 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.207+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~JT0KAZPUOATX.0 size=1048576 2024-05-30T16:03:59.206 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.207+0000 7f08851138c0 1 < canceled part key=_multipart_WT9RC0UYVG4F.2~JT0KAZPUOATX.1 size=217231 2024-05-30T16:03:59.212 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.211+0000 7f08851138c0 1 < canceled multipart upload key=WT9RC0UYVG4F upload=JT0KAZPUOATX tag=VD7O8HEEU02Z type=write size=2314383 2024-05-30T16:03:59.215 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.215+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=47TWHKNRL548 type=write size=614912 2024-05-30T16:03:59.222 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.223+0000 7f08851138c0 1 < suggested remove operation key=MMLF65KLC42Q tag=WHFSV0VR1O5T type=del 2024-05-30T16:03:59.225 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.227+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=63465J01BIY3 type=del size=3641414 2024-05-30T16:03:59.229 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.227+0000 7f08851138c0 -1 < failed to complete operation key=MMLF65KLC42Q tag=KVERLZTV75VY type=write size=447054: (22) Invalid argument 2024-05-30T16:03:59.232 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.231+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=HQB4ARQVF4G0 type=del size=2293379 2024-05-30T16:03:59.236 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.235+0000 7f08851138c0 1 < completed delete operation key=YRP4CSG92QRO tag=SERI4S6NIOI0 type=del 2024-05-30T16:03:59.240 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.239+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=R62M21LD40AF type=del size=996118 2024-05-30T16:03:59.243 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.243+0000 7f08851138c0 1 < completed delete operation key=QQILXRJD0C1Z tag=R48433IHX62I type=del 2024-05-30T16:03:59.249 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.247+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~Q7VQ4O2IL4OQ.0 size=854208 2024-05-30T16:03:59.249 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.247+0000 7f08851138c0 1 > prepared multipart upload key=603HBR8PRH6L upload=Q7VQ4O2IL4OQ tag=ZHKEBU9QDPOF type=write size=1902784 2024-05-30T16:03:59.253 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.255+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~W1Y8878X74A5.0 size=1048576 2024-05-30T16:03:59.256 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.255+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~W1Y8878X74A5.1 size=1048576 2024-05-30T16:03:59.259 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.259+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~W1Y8878X74A5.2 size=1042226 2024-05-30T16:03:59.265 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.263+0000 7f08851138c0 1 < completed multipart upload key=364U3OV10VPD upload=W1Y8878X74A5 tag=MAQDV0G89N1U type=write size=4187954 2024-05-30T16:03:59.268 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.267+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=Q9YQIHLC1YM4 type=del size=3404825 2024-05-30T16:03:59.272 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.271+0000 7f08851138c0 1 < completed delete operation key=WT9RC0UYVG4F tag=W22T6YF14JEV type=del 2024-05-30T16:03:59.276 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.275+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=74DW8370AXKL type=del size=940685 2024-05-30T16:03:59.280 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.279+0000 7f08851138c0 1 < canceled operation key=Y3CYZJ83WYNI tag=ZGOHJMJ6M3SY type=del size=2595988 2024-05-30T16:03:59.283 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.283+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=T7USZXW9OUT4 type=del size=2414252 2024-05-30T16:03:59.290 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.291+0000 7f08851138c0 1 < suggested remove operation key=1N6R2HBTWMP2 tag=F1FA6PPM701O type=del 2024-05-30T16:03:59.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.291+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=GRS0OKWTMY5Z type=write size=94871 2024-05-30T16:03:59.297 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.295+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~JMT63KO77J6O.0 size=1048576 2024-05-30T16:03:59.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.299+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~JMT63KO77J6O.1 size=1048576 2024-05-30T16:03:59.303 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.303+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~JMT63KO77J6O.2 size=810447 2024-05-30T16:03:59.309 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.307+0000 7f08851138c0 1 < completed multipart upload key=O8CWPJ9GW2GO upload=JMT63KO77J6O tag=RY3H7BPC453A type=write size=3956175 2024-05-30T16:03:59.316 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.315+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~Y3U6SZN6MNWF.0 size=893067 2024-05-30T16:03:59.316 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.315+0000 7f08851138c0 1 > prepared multipart upload key=A5BUBY9Z7JQG upload=Y3U6SZN6MNWF tag=3GEDZXBO8M1O type=write size=1941643 2024-05-30T16:03:59.324 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.323+0000 7f08851138c0 1 < suggested update operation key=Y3CYZJ83WYNI tag=HJF3WDVKQGIQ type=del size=1012062 2024-05-30T16:03:59.328 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.327+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=DPSFBANUAMWJ type=del size=3929724 2024-05-30T16:03:59.332 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.331+0000 7f08851138c0 1 < canceled operation key=XHGMUYBA2VNK tag=1S5WQT4UYOXU type=del size=2638795 2024-05-30T16:03:59.336 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.335+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=TUFSMGDNP1WY type=del size=394579 2024-05-30T16:03:59.336 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.335+0000 7f08851138c0 1 < canceled part key=_multipart_3IJWPWRB6FHE.2~QDTBL3JGB1C3.0 size=102156 2024-05-30T16:03:59.343 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.343+0000 7f08851138c0 1 < canceled multipart upload key=3IJWPWRB6FHE upload=QDTBL3JGB1C3 tag=5KLQCAWZCFAW type=write size=1150732 2024-05-30T16:03:59.349 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.347+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~974VPDKAA476.0 size=726790 2024-05-30T16:03:59.349 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.351+0000 7f08851138c0 1 > prepared multipart upload key=73FJ3HGWYQQK upload=974VPDKAA476 tag=AOJ9K5FBYZEY type=write size=1775366 2024-05-30T16:03:59.353 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.355+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~SDFLNLTPDGEY.0 size=1048576 2024-05-30T16:03:59.356 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.355+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~SDFLNLTPDGEY.1 size=1048576 2024-05-30T16:03:59.359 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.359+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~SDFLNLTPDGEY.2 size=206769 2024-05-30T16:03:59.366 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.367+0000 7f08851138c0 1 < completed multipart upload key=QQILXRJD0C1Z upload=SDFLNLTPDGEY tag=NXGED651PKDO type=write size=3352497 2024-05-30T16:03:59.372 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.371+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~54XWUEEDA6E9.0 size=566119 2024-05-30T16:03:59.372 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.371+0000 7f08851138c0 1 > prepared multipart upload key=JNZI7JX95LX2 upload=54XWUEEDA6E9 tag=YWIH96IP2IN0 type=write size=1614695 2024-05-30T16:03:59.376 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.375+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~51TVKHCDIUBE.0 size=1048576 2024-05-30T16:03:59.379 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.379+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~51TVKHCDIUBE.1 size=1048576 2024-05-30T16:03:59.382 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.383+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~51TVKHCDIUBE.2 size=830584 2024-05-30T16:03:59.388 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.387+0000 7f08851138c0 1 < completed multipart upload key=364U3OV10VPD upload=51TVKHCDIUBE tag=ZWW1IWN38HAL type=write size=3976312 2024-05-30T16:03:59.392 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.391+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=3ZRPNY3GE9JB type=del size=2444337 2024-05-30T16:03:59.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.395+0000 7f08851138c0 1 < completed write operation key=A5BUBY9Z7JQG tag=WNZO2U0YIJ0V type=write size=539016 2024-05-30T16:03:59.402 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.403+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~VZYHO6SSKVC4.0 size=304206 2024-05-30T16:03:59.402 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.403+0000 7f08851138c0 1 > prepared multipart upload key=RCTV94WRMO53 upload=VZYHO6SSKVC4 tag=6QJY40NDFRHN type=write size=1352782 2024-05-30T16:03:59.409 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.411+0000 7f08851138c0 1 < suggested update operation key=NIV7AQ4EV5YP tag=47TWHKNRL548 type=write size=614912 2024-05-30T16:03:59.416 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.415+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~FCWPCALZOZOQ.0 size=1048576 2024-05-30T16:03:59.419 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.419+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~FCWPCALZOZOQ.1 size=1048576 2024-05-30T16:03:59.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.423+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~FCWPCALZOZOQ.2 size=678345 2024-05-30T16:03:59.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.423+0000 7f08851138c0 1 > prepared multipart upload key=KPHZGFJLC2E0 upload=FCWPCALZOZOQ tag=OK16P438MBL5 type=write size=3824073 2024-05-30T16:03:59.425 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.427+0000 7f08851138c0 1 < completed delete operation key=PLWFSKPKVO19 tag=63465J01BIY3 type=del 2024-05-30T16:03:59.429 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.427+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=ULYGFPPQPCP2 type=del size=1982280 2024-05-30T16:03:59.432 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.431+0000 7f08851138c0 1 < completed delete operation key=A5BUBY9Z7JQG tag=HQB4ARQVF4G0 type=del 2024-05-30T16:03:59.436 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.435+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=5O7T8ZHX5BNY type=del size=4114699 2024-05-30T16:03:59.439 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.439+0000 7f08851138c0 1 < completed delete operation key=GWXNCSJCLXKW tag=R62M21LD40AF type=del 2024-05-30T16:03:59.443 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.443+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=GKIO9MRX3C4R type=del size=3597167 2024-05-30T16:03:59.444 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.443+0000 7f08851138c0 1 < canceled part key=_multipart_603HBR8PRH6L.2~Q7VQ4O2IL4OQ.0 size=854208 2024-05-30T16:03:59.451 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.451+0000 7f08851138c0 1 < canceled multipart upload key=603HBR8PRH6L upload=Q7VQ4O2IL4OQ tag=ZHKEBU9QDPOF type=write size=1902784 2024-05-30T16:03:59.455 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.455+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=UH3P29X11SW0 type=del size=3771870 2024-05-30T16:03:59.458 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.459+0000 7f08851138c0 -1 < failed to complete operation key=NIV7AQ4EV5YP tag=Q9YQIHLC1YM4 type=del size=3404825: (22) Invalid argument 2024-05-30T16:03:59.464 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.463+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~JWIIC3DGF8SV.0 size=1048576 2024-05-30T16:03:59.467 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.467+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~JWIIC3DGF8SV.1 size=861817 2024-05-30T16:03:59.467 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.467+0000 7f08851138c0 1 > prepared multipart upload key=XHGMUYBA2VNK upload=JWIIC3DGF8SV tag=I8VLDQUW551R type=write size=2958969 2024-05-30T16:03:59.470 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.471+0000 7f08851138c0 1 < completed delete operation key=XHGMUYBA2VNK tag=74DW8370AXKL type=del 2024-05-30T16:03:59.474 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.475+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=A05WMR8CWPR3 type=del size=1504424 2024-05-30T16:03:59.477 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.479+0000 7f08851138c0 1 < completed delete operation key=364U3OV10VPD tag=T7USZXW9OUT4 type=del 2024-05-30T16:03:59.481 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.479+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=LTSLZ0DHBJS7 type=del size=4001434 2024-05-30T16:03:59.485 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.483+0000 7f08851138c0 1 < completed write operation key=GWXNCSJCLXKW tag=GRS0OKWTMY5Z type=write size=94871 2024-05-30T16:03:59.494 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.495+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~4VXN18VX0BNS.0 size=405854 2024-05-30T16:03:59.494 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.495+0000 7f08851138c0 1 > prepared multipart upload key=JNZI7JX95LX2 upload=4VXN18VX0BNS tag=KI2K5GBAZE9Z type=write size=1454430 2024-05-30T16:03:59.498 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.499+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~Y3U6SZN6MNWF.0 size=893067 2024-05-30T16:03:59.503 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.503+0000 7f08851138c0 1 < completed multipart upload key=A5BUBY9Z7JQG upload=Y3U6SZN6MNWF tag=3GEDZXBO8M1O type=write size=1941643 2024-05-30T16:03:59.507 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.507+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=4HQC167CG63X type=del size=1556218 2024-05-30T16:03:59.510 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.511+0000 7f08851138c0 1 < completed delete operation key=3TBOZWT2P8U8 tag=DPSFBANUAMWJ type=del 2024-05-30T16:03:59.516 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.515+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~R6K2BQGD7ADL.0 size=153705 2024-05-30T16:03:59.516 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.515+0000 7f08851138c0 1 > prepared multipart upload key=RCTV94WRMO53 upload=R6K2BQGD7ADL tag=91GTOJK01BQ0 type=write size=1202281 2024-05-30T16:03:59.524 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.523+0000 7f08851138c0 1 < suggested remove operation key=GWXNCSJCLXKW tag=TUFSMGDNP1WY type=del 2024-05-30T16:03:59.530 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.531+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~EE06YNZT2KNF.0 size=1048576 2024-05-30T16:03:59.533 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.531+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~EE06YNZT2KNF.1 size=524158 2024-05-30T16:03:59.533 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.531+0000 7f08851138c0 1 > prepared multipart upload key=PLWFSKPKVO19 upload=EE06YNZT2KNF tag=VTWQ0JGZZSBS type=write size=2621310 2024-05-30T16:03:59.537 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.535+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~974VPDKAA476.0 size=726790 2024-05-30T16:03:59.544 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.543+0000 7f08851138c0 1 < completed multipart upload key=73FJ3HGWYQQK upload=974VPDKAA476 tag=AOJ9K5FBYZEY type=write size=1775366 2024-05-30T16:03:59.548 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.547+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=OYD6HDOLHHPA type=del size=3875372 2024-05-30T16:03:59.552 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.551+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~54XWUEEDA6E9.0 size=566119 2024-05-30T16:03:59.558 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.559+0000 7f08851138c0 1 < completed multipart upload key=JNZI7JX95LX2 upload=54XWUEEDA6E9 tag=YWIH96IP2IN0 type=write size=1614695 2024-05-30T16:03:59.562 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.563+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=XN5SFOPFTF4X type=del size=215747 2024-05-30T16:03:59.565 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.567+0000 7f08851138c0 1 < completed delete operation key=YRP4CSG92QRO tag=3ZRPNY3GE9JB type=del 2024-05-30T16:03:59.572 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.571+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~IM2HLDUQOCZ1.0 size=1048576 2024-05-30T16:03:59.575 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.575+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~IM2HLDUQOCZ1.1 size=753531 2024-05-30T16:03:59.575 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.575+0000 7f08851138c0 1 > prepared multipart upload key=Y3CYZJ83WYNI upload=IM2HLDUQOCZ1 tag=3VI3M1OW6J9X type=write size=2850683 2024-05-30T16:03:59.578 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.579+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~VZYHO6SSKVC4.0 size=304206 2024-05-30T16:03:59.585 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.583+0000 7f08851138c0 1 < completed multipart upload key=RCTV94WRMO53 upload=VZYHO6SSKVC4 tag=6QJY40NDFRHN type=write size=1352782 2024-05-30T16:03:59.588 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.587+0000 7f08851138c0 1 > prepared operation key=ZALVY70ISLM5 tag=GZLFF6YE77A3 type=del size=3295577 2024-05-30T16:03:59.592 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.591+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~FCWPCALZOZOQ.0 size=1048576 2024-05-30T16:03:59.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.595+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~FCWPCALZOZOQ.1 size=1048576 2024-05-30T16:03:59.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.599+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~FCWPCALZOZOQ.2 size=678345 2024-05-30T16:03:59.604 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.603+0000 7f08851138c0 1 < completed multipart upload key=KPHZGFJLC2E0 upload=FCWPCALZOZOQ tag=OK16P438MBL5 type=write size=3824073 2024-05-30T16:03:59.610 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.611+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~0LDDP9949RUW.0 size=1048576 2024-05-30T16:03:59.613 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.615+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~0LDDP9949RUW.1 size=1048576 2024-05-30T16:03:59.616 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.615+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~0LDDP9949RUW.2 size=319616 2024-05-30T16:03:59.616 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.615+0000 7f08851138c0 1 > prepared multipart upload key=3IJWPWRB6FHE upload=0LDDP9949RUW tag=0OI5OEX1W0ZC type=write size=3465344 2024-05-30T16:03:59.621 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.619+0000 7f08851138c0 1 < completed delete operation key=PLWFSKPKVO19 tag=ULYGFPPQPCP2 type=del 2024-05-30T16:03:59.625 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.623+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=K4YZ6UD5EPM8 type=del size=3752459 2024-05-30T16:03:59.633 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.635+0000 7f08851138c0 1 < suggested update operation key=JNZI7JX95LX2 tag=5O7T8ZHX5BNY type=del size=4114699 2024-05-30T16:03:59.637 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.635+0000 7f08851138c0 1 > prepared operation key=VRXNO8JLAZYH tag=JJI7H41BGIS6 type=del size=1611584 2024-05-30T16:03:59.644 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.643+0000 7f08851138c0 1 < suggested update operation key=XHGMUYBA2VNK tag=GKIO9MRX3C4R type=del size=3597167 2024-05-30T16:03:59.648 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.647+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=VP1K0D59YVNP type=del size=3379632 2024-05-30T16:03:59.651 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.651+0000 7f08851138c0 1 < canceled operation key=PLWFSKPKVO19 tag=UH3P29X11SW0 type=del size=3771870 2024-05-30T16:03:59.655 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.655+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=WXEGR3PODL09 type=del size=781227 2024-05-30T16:03:59.658 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.659+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~JWIIC3DGF8SV.0 size=1048576 2024-05-30T16:03:59.658 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.659+0000 7f08851138c0 1 < canceled part key=_multipart_XHGMUYBA2VNK.2~JWIIC3DGF8SV.1 size=861817 2024-05-30T16:03:59.664 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.663+0000 7f08851138c0 1 < canceled multipart upload key=XHGMUYBA2VNK upload=JWIIC3DGF8SV tag=I8VLDQUW551R type=write size=2958969 2024-05-30T16:03:59.671 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.671+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~84NRY20ER5Y6.0 size=616660 2024-05-30T16:03:59.671 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.671+0000 7f08851138c0 1 > prepared multipart upload key=Y8SBI0QZ0ZV8 upload=84NRY20ER5Y6 tag=00UTYRWBC3R6 type=write size=1665236 2024-05-30T16:03:59.675 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.675+0000 7f08851138c0 1 < completed delete operation key=O0DJ8SCY766V tag=A05WMR8CWPR3 type=del 2024-05-30T16:03:59.681 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.679+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~OUC9TNEQN1D3.0 size=1048576 2024-05-30T16:03:59.684 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.683+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~OUC9TNEQN1D3.1 size=60773 2024-05-30T16:03:59.684 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.683+0000 7f08851138c0 1 > prepared multipart upload key=O8CWPJ9GW2GO upload=OUC9TNEQN1D3 tag=QVBM79QNR8FA type=write size=2157925 2024-05-30T16:03:59.691 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.691+0000 7f08851138c0 1 < suggested update operation key=E2FXSXOEUGIG tag=LTSLZ0DHBJS7 type=del size=4001434 2024-05-30T16:03:59.694 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.695+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=205Z215OYIPF type=del size=3621480 2024-05-30T16:03:59.697 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.699+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~4VXN18VX0BNS.0 size=405854 2024-05-30T16:03:59.703 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.703+0000 7f08851138c0 1 < completed multipart upload key=JNZI7JX95LX2 upload=4VXN18VX0BNS tag=KI2K5GBAZE9Z type=write size=1454430 2024-05-30T16:03:59.706 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.707+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=GH8G9UFYL2CO type=del size=192366 2024-05-30T16:03:59.710 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.711+0000 7f08851138c0 1 < completed delete operation key=YRP4CSG92QRO tag=4HQC167CG63X type=del 2024-05-30T16:03:59.717 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.715+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~WP1EBT4JIIPU.0 size=1048576 2024-05-30T16:03:59.719 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.719+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~WP1EBT4JIIPU.1 size=938895 2024-05-30T16:03:59.720 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.719+0000 7f08851138c0 1 > prepared multipart upload key=XHGMUYBA2VNK upload=WP1EBT4JIIPU tag=N06A5HSF45AC type=write size=3036047 2024-05-30T16:03:59.723 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.723+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~R6K2BQGD7ADL.0 size=153705 2024-05-30T16:03:59.729 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.731+0000 7f08851138c0 1 < completed multipart upload key=RCTV94WRMO53 upload=R6K2BQGD7ADL tag=91GTOJK01BQ0 type=write size=1202281 2024-05-30T16:03:59.736 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.735+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~18SUQZVBASIP.0 size=1048576 2024-05-30T16:03:59.739 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.739+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~18SUQZVBASIP.1 size=1048576 2024-05-30T16:03:59.741 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.743+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~18SUQZVBASIP.2 size=447490 2024-05-30T16:03:59.741 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.743+0000 7f08851138c0 1 > prepared multipart upload key=603HBR8PRH6L upload=18SUQZVBASIP tag=5XL8IC04ERW5 type=write size=3593218 2024-05-30T16:03:59.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.743+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~EE06YNZT2KNF.0 size=1048576 2024-05-30T16:03:59.748 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.747+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~EE06YNZT2KNF.1 size=524158 2024-05-30T16:03:59.754 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.755+0000 7f08851138c0 1 < completed multipart upload key=PLWFSKPKVO19 upload=EE06YNZT2KNF tag=VTWQ0JGZZSBS type=write size=2621310 2024-05-30T16:03:59.760 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.759+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~U97ED99OCV5G.0 size=1048576 2024-05-30T16:03:59.763 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.763+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~U97ED99OCV5G.1 size=1048576 2024-05-30T16:03:59.765 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.767+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~U97ED99OCV5G.2 size=4074 2024-05-30T16:03:59.765 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.767+0000 7f08851138c0 1 > prepared multipart upload key=KPHZGFJLC2E0 upload=U97ED99OCV5G tag=ZGEDN68QSVDQ type=write size=3149802 2024-05-30T16:03:59.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.767+0000 7f08851138c0 -1 < failed to complete operation key=JNZI7JX95LX2 tag=OYD6HDOLHHPA type=del size=3875372: (22) Invalid argument 2024-05-30T16:03:59.772 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.771+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=OLUWP2OCPNJN type=del size=2932200 2024-05-30T16:03:59.776 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.775+0000 7f08851138c0 1 < completed delete operation key=0RZRBW38WKNZ tag=XN5SFOPFTF4X type=del 2024-05-30T16:03:59.780 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.779+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=2V4H9SNGDEPO type=write size=133645 2024-05-30T16:03:59.784 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.783+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~IM2HLDUQOCZ1.0 size=1048576 2024-05-30T16:03:59.787 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.787+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~IM2HLDUQOCZ1.1 size=753531 2024-05-30T16:03:59.794 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.795+0000 7f08851138c0 1 < completed multipart upload key=Y3CYZJ83WYNI upload=IM2HLDUQOCZ1 tag=3VI3M1OW6J9X type=write size=2850683 2024-05-30T16:03:59.798 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.799+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=9WFXE38156X3 type=del size=3142564 2024-05-30T16:03:59.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.803+0000 7f08851138c0 1 < completed delete operation key=ZALVY70ISLM5 tag=GZLFF6YE77A3 type=del 2024-05-30T16:03:59.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.807+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=SM1NFDL9SDGH type=del size=2638266 2024-05-30T16:03:59.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.811+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~0LDDP9949RUW.0 size=1048576 2024-05-30T16:03:59.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.811+0000 7f08851138c0 1 < canceled part key=_multipart_3IJWPWRB6FHE.2~0LDDP9949RUW.1 size=1048576 2024-05-30T16:03:59.813 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.815+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~0LDDP9949RUW.2 size=319616 2024-05-30T16:03:59.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.819+0000 7f08851138c0 1 < canceled multipart upload key=3IJWPWRB6FHE upload=0LDDP9949RUW tag=0OI5OEX1W0ZC type=write size=3465344 2024-05-30T16:03:59.823 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.823+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=H7V8WOQKTJSO type=write size=289716 2024-05-30T16:03:59.827 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.827+0000 7f08851138c0 1 < completed delete operation key=OKBB0VU765TW tag=K4YZ6UD5EPM8 type=del 2024-05-30T16:03:59.836 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.835+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=OYG0LF3FTEMH type=del size=3370474 2024-05-30T16:03:59.840 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.839+0000 7f08851138c0 1 < completed delete operation key=VRXNO8JLAZYH tag=JJI7H41BGIS6 type=del 2024-05-30T16:03:59.844 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.843+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=ZFCM3XXSDZWG type=del size=84084 2024-05-30T16:03:59.851 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.851+0000 7f08851138c0 1 < suggested remove operation key=364U3OV10VPD tag=VP1K0D59YVNP type=del 2024-05-30T16:03:59.854 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.855+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=ICZ5P73RJI38 type=del size=4007712 2024-05-30T16:03:59.858 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.859+0000 7f08851138c0 1 < completed delete operation key=73FJ3HGWYQQK tag=WXEGR3PODL09 type=del 2024-05-30T16:03:59.864 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.863+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~LXQY3W7G8TOF.0 size=1048576 2024-05-30T16:03:59.867 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.867+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~LXQY3W7G8TOF.1 size=1048576 2024-05-30T16:03:59.870 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.871+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~LXQY3W7G8TOF.2 size=679991 2024-05-30T16:03:59.870 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.871+0000 7f08851138c0 1 > prepared multipart upload key=364U3OV10VPD upload=LXQY3W7G8TOF tag=V2L0YHVUBP23 type=write size=3825719 2024-05-30T16:03:59.871 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.871+0000 7f08851138c0 1 < canceled part key=_multipart_Y8SBI0QZ0ZV8.2~84NRY20ER5Y6.0 size=616660 2024-05-30T16:03:59.878 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.879+0000 7f08851138c0 1 < canceled multipart upload key=Y8SBI0QZ0ZV8 upload=84NRY20ER5Y6 tag=00UTYRWBC3R6 type=write size=1665236 2024-05-30T16:03:59.882 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.883+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=A528BN00MSUU type=del size=823793 2024-05-30T16:03:59.886 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.887+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~OUC9TNEQN1D3.0 size=1048576 2024-05-30T16:03:59.889 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.887+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~OUC9TNEQN1D3.1 size=60773 2024-05-30T16:03:59.895 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.895+0000 7f08851138c0 1 < completed multipart upload key=O8CWPJ9GW2GO upload=OUC9TNEQN1D3 tag=QVBM79QNR8FA type=write size=2157925 2024-05-30T16:03:59.898 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.899+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=TTWVXUNVBYOJ type=del size=2558357 2024-05-30T16:03:59.901 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.903+0000 7f08851138c0 1 < completed delete operation key=NIV7AQ4EV5YP tag=205Z215OYIPF type=del 2024-05-30T16:03:59.905 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.903+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=MZYFPS9NYML7 type=del size=3165391 2024-05-30T16:03:59.912 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.911+0000 7f08851138c0 1 < suggested remove operation key=O0DJ8SCY766V tag=GH8G9UFYL2CO type=del 2024-05-30T16:03:59.917 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.915+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=012NY72NGRXI type=del size=2626079 2024-05-30T16:03:59.920 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.919+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~WP1EBT4JIIPU.0 size=1048576 2024-05-30T16:03:59.923 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.923+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~WP1EBT4JIIPU.1 size=938895 2024-05-30T16:03:59.930 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.931+0000 7f08851138c0 1 < completed multipart upload key=XHGMUYBA2VNK upload=WP1EBT4JIIPU tag=N06A5HSF45AC type=write size=3036047 2024-05-30T16:03:59.932 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.931+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=274J8BOOO0H3 type=del size=195204 2024-05-30T16:03:59.936 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.935+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~18SUQZVBASIP.0 size=1048576 2024-05-30T16:03:59.936 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.935+0000 7f08851138c0 1 < canceled part key=_multipart_603HBR8PRH6L.2~18SUQZVBASIP.1 size=1048576 2024-05-30T16:03:59.937 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.935+0000 7f08851138c0 1 < canceled part key=_multipart_603HBR8PRH6L.2~18SUQZVBASIP.2 size=447490 2024-05-30T16:03:59.942 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.943+0000 7f08851138c0 1 < canceled multipart upload key=603HBR8PRH6L upload=18SUQZVBASIP tag=5XL8IC04ERW5 type=write size=3593218 2024-05-30T16:03:59.946 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.947+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=00386UTY6J3P type=del size=1020524 2024-05-30T16:03:59.950 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.951+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~U97ED99OCV5G.0 size=1048576 2024-05-30T16:03:59.952 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.951+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~U97ED99OCV5G.1 size=1048576 2024-05-30T16:03:59.955 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.955+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~U97ED99OCV5G.2 size=4074 2024-05-30T16:03:59.961 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.959+0000 7f08851138c0 1 < completed multipart upload key=KPHZGFJLC2E0 upload=U97ED99OCV5G tag=ZGEDN68QSVDQ type=write size=3149802 2024-05-30T16:03:59.965 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.963+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=UQ0Y8L7ZIQ4W type=del size=3543080 2024-05-30T16:03:59.969 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.967+0000 7f08851138c0 1 < completed delete operation key=3TBOZWT2P8U8 tag=OLUWP2OCPNJN type=del 2024-05-30T16:03:59.975 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.975+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~JIY8RTPAGNJT.0 size=120482 2024-05-30T16:03:59.975 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.975+0000 7f08851138c0 1 > prepared multipart upload key=0T2E9F3K1USF upload=JIY8RTPAGNJT tag=DA3T0PHGA51L type=write size=1169058 2024-05-30T16:03:59.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.979+0000 7f08851138c0 1 < completed write operation key=0RZRBW38WKNZ tag=2V4H9SNGDEPO type=write size=133645 2024-05-30T16:03:59.983 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.983+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=VQ723697TXHG type=del size=1719587 2024-05-30T16:03:59.986 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.987+0000 7f08851138c0 1 < canceled operation key=3TBOZWT2P8U8 tag=9WFXE38156X3 type=del size=3142564 2024-05-30T16:03:59.993 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.991+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~7QAQX6O60B6I.0 size=1048576 2024-05-30T16:03:59.996 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.995+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~7QAQX6O60B6I.1 size=1048576 2024-05-30T16:03:59.999 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.999+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~7QAQX6O60B6I.2 size=944997 2024-05-30T16:03:59.999 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:03:59.999+0000 7f08851138c0 1 > prepared multipart upload key=XHGMUYBA2VNK upload=7QAQX6O60B6I tag=201QSPR5DRR0 type=write size=4090725 2024-05-30T16:04:00.002 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.003+0000 7f08851138c0 1 < completed delete operation key=0T2E9F3K1USF tag=SM1NFDL9SDGH type=del 2024-05-30T16:04:00.006 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.007+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=WSN2AZ9A7X7S type=del size=3353244 2024-05-30T16:04:00.010 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.011+0000 7f08851138c0 1 < completed write operation key=KPHZGFJLC2E0 tag=H7V8WOQKTJSO type=write size=289716 2024-05-30T16:04:00.018 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.019+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~GYKVNTLBHRP5.0 size=1048576 2024-05-30T16:04:00.021 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.019+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~GYKVNTLBHRP5.1 size=944929 2024-05-30T16:04:00.021 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.019+0000 7f08851138c0 1 > prepared multipart upload key=WT9RC0UYVG4F upload=GYKVNTLBHRP5 tag=E0C38ZU1M6M6 type=write size=3042081 2024-05-30T16:04:00.028 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.027+0000 7f08851138c0 1 < suggested update operation key=O8CWPJ9GW2GO tag=OYG0LF3FTEMH type=del size=3370474 2024-05-30T16:04:00.032 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.031+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=YABUYKJX3912 type=del size=1596708 2024-05-30T16:04:00.037 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.035+0000 7f08851138c0 1 < completed delete operation key=1N6R2HBTWMP2 tag=ZFCM3XXSDZWG type=del 2024-05-30T16:04:00.041 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.039+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=0N0B55P7VFU6 type=del size=4165988 2024-05-30T16:04:00.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.047+0000 7f08851138c0 1 < completed delete operation key=1N6R2HBTWMP2 tag=ICZ5P73RJI38 type=del 2024-05-30T16:04:00.049 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.051+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=LVF6WHPDG690 type=del size=151940 2024-05-30T16:04:00.054 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.055+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~LXQY3W7G8TOF.0 size=1048576 2024-05-30T16:04:00.057 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.055+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~LXQY3W7G8TOF.1 size=1048576 2024-05-30T16:04:00.060 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.059+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~LXQY3W7G8TOF.2 size=679991 2024-05-30T16:04:00.067 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.067+0000 7f08851138c0 1 < completed multipart upload key=364U3OV10VPD upload=LXQY3W7G8TOF tag=V2L0YHVUBP23 type=write size=3825719 2024-05-30T16:04:00.070 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.071+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=AUTPDCCXKH1Z type=write size=421045 2024-05-30T16:04:00.073 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.075+0000 7f08851138c0 1 < completed delete operation key=QQILXRJD0C1Z tag=A528BN00MSUU type=del 2024-05-30T16:04:00.077 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.075+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=J55M47C92GEE type=del size=1334396 2024-05-30T16:04:00.081 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.079+0000 7f08851138c0 1 < completed delete operation key=PLWFSKPKVO19 tag=TTWVXUNVBYOJ type=del 2024-05-30T16:04:00.087 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.087+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~R2N2KQ31XQLS.0 size=760734 2024-05-30T16:04:00.087 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.087+0000 7f08851138c0 1 > prepared multipart upload key=GWXNCSJCLXKW upload=R2N2KQ31XQLS tag=FC7A5NG5LCQU type=write size=1809310 2024-05-30T16:04:00.091 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.091+0000 7f08851138c0 1 < completed delete operation key=TPCETCO3Y1M6 tag=MZYFPS9NYML7 type=del 2024-05-30T16:04:00.094 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.095+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=8GX5A452KF6P type=del size=121114 2024-05-30T16:04:00.097 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.095+0000 7f08851138c0 1 < completed delete operation key=KPHZGFJLC2E0 tag=012NY72NGRXI type=del 2024-05-30T16:04:00.100 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.099+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=OMCDX0KCVLJU type=del size=2586155 2024-05-30T16:04:00.103 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.103+0000 7f08851138c0 1 < completed delete operation key=KPHZGFJLC2E0 tag=274J8BOOO0H3 type=del 2024-05-30T16:04:00.108 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.107+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~166V3P9XQRWZ.0 size=37363 2024-05-30T16:04:00.108 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.107+0000 7f08851138c0 1 > prepared multipart upload key=3IJWPWRB6FHE upload=166V3P9XQRWZ tag=CV7QHK87OYAV type=write size=1085939 2024-05-30T16:04:00.111 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.111+0000 7f08851138c0 -1 < failed to complete operation key=O8CWPJ9GW2GO tag=00386UTY6J3P type=del size=1020524: (22) Invalid argument 2024-05-30T16:04:00.116 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.115+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~AMNXR6WCQ9HW.0 size=1048576 2024-05-30T16:04:00.118 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.119+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~AMNXR6WCQ9HW.1 size=1048576 2024-05-30T16:04:00.120 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.119+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~AMNXR6WCQ9HW.2 size=268008 2024-05-30T16:04:00.120 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.119+0000 7f08851138c0 1 > prepared multipart upload key=603HBR8PRH6L upload=AMNXR6WCQ9HW tag=MBC2S8RQXNE4 type=write size=3413736 2024-05-30T16:04:00.123 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.123+0000 7f08851138c0 1 < completed delete operation key=3IJWPWRB6FHE tag=UQ0Y8L7ZIQ4W type=del 2024-05-30T16:04:00.126 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.127+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=1TX40I3LNJF6 type=write size=368475 2024-05-30T16:04:00.129 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.127+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~JIY8RTPAGNJT.0 size=120482 2024-05-30T16:04:00.134 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.135+0000 7f08851138c0 1 < completed multipart upload key=0T2E9F3K1USF upload=JIY8RTPAGNJT tag=DA3T0PHGA51L type=write size=1169058 2024-05-30T16:04:00.139 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.139+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~9IIMCO3U6983.0 size=357589 2024-05-30T16:04:00.139 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.139+0000 7f08851138c0 1 > prepared multipart upload key=QQILXRJD0C1Z upload=9IIMCO3U6983 tag=BI26QNHVQJRN type=write size=1406165 2024-05-30T16:04:00.141 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.143+0000 7f08851138c0 1 < completed delete operation key=0RZRBW38WKNZ tag=VQ723697TXHG type=del 2024-05-30T16:04:00.144 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.143+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=MGPRD2ED31UZ type=del size=2905698 2024-05-30T16:04:00.147 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.147+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~7QAQX6O60B6I.0 size=1048576 2024-05-30T16:04:00.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.147+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~7QAQX6O60B6I.1 size=1048576 2024-05-30T16:04:00.151 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.151+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~7QAQX6O60B6I.2 size=944997 2024-05-30T16:04:00.155 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.155+0000 7f08851138c0 1 < completed multipart upload key=XHGMUYBA2VNK upload=7QAQX6O60B6I tag=201QSPR5DRR0 type=write size=4090725 2024-05-30T16:04:00.158 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.159+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=QEX6LNGPBEX9 type=del size=3580283 2024-05-30T16:04:00.163 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.163+0000 7f08851138c0 1 < completed delete operation key=QQILXRJD0C1Z tag=WSN2AZ9A7X7S type=del 2024-05-30T16:04:00.166 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.167+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=C4NSV1900B3W type=del size=1749230 2024-05-30T16:04:00.168 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.167+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~GYKVNTLBHRP5.0 size=1048576 2024-05-30T16:04:00.169 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.171+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~GYKVNTLBHRP5.1 size=944929 2024-05-30T16:04:00.172 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.171+0000 7f08851138c0 1 < completed multipart upload key=WT9RC0UYVG4F upload=GYKVNTLBHRP5 tag=E0C38ZU1M6M6 type=write size=3042081 2024-05-30T16:04:00.175 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.175+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~P5KOIYU4KF8H.0 size=1048576 2024-05-30T16:04:00.176 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.175+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~P5KOIYU4KF8H.1 size=498070 2024-05-30T16:04:00.176 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.175+0000 7f08851138c0 1 > prepared multipart upload key=4N58ZY4E1H6E upload=P5KOIYU4KF8H tag=0T6IAS5UUONA type=write size=2595222 2024-05-30T16:04:00.178 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.179+0000 7f08851138c0 1 < canceled operation key=E2FXSXOEUGIG tag=YABUYKJX3912 type=del size=1596708 2024-05-30T16:04:00.180 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.179+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=1XT6PGF19TBY type=write size=967569 2024-05-30T16:04:00.181 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.183+0000 7f08851138c0 1 < completed delete operation key=OKBB0VU765TW tag=0N0B55P7VFU6 type=del 2024-05-30T16:04:00.186 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.187+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~M3WV8GO4AEU4.0 size=1048576 2024-05-30T16:04:00.187 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.187+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~M3WV8GO4AEU4.1 size=796489 2024-05-30T16:04:00.187 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.187+0000 7f08851138c0 1 > prepared multipart upload key=3IJWPWRB6FHE upload=M3WV8GO4AEU4 tag=F174AYJL9MWS type=write size=2893641 2024-05-30T16:04:00.189 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.191+0000 7f08851138c0 1 < completed delete operation key=OKBB0VU765TW tag=LVF6WHPDG690 type=del 2024-05-30T16:04:00.192 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.191+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=SPFOTVWWJYBQ type=del size=3685878 2024-05-30T16:04:00.194 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.195+0000 7f08851138c0 1 < completed write operation key=QQILXRJD0C1Z tag=AUTPDCCXKH1Z type=write size=421045 2024-05-30T16:04:00.200 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.199+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~CEJ374CIM7YM.0 size=1048576 2024-05-30T16:04:00.202 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.203+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~CEJ374CIM7YM.1 size=328412 2024-05-30T16:04:00.202 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.203+0000 7f08851138c0 1 > prepared multipart upload key=PLWFSKPKVO19 upload=CEJ374CIM7YM tag=DQBBKA5CUJLR type=write size=2425564 2024-05-30T16:04:00.206 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.207+0000 7f08851138c0 1 < completed delete operation key=364U3OV10VPD tag=J55M47C92GEE type=del 2024-05-30T16:04:00.210 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.211+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=G6T0H3JF6C59 type=write size=367085 2024-05-30T16:04:00.211 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.211+0000 7f08851138c0 1 < canceled part key=_multipart_GWXNCSJCLXKW.2~R2N2KQ31XQLS.0 size=760734 2024-05-30T16:04:00.216 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.215+0000 7f08851138c0 1 < canceled multipart upload key=GWXNCSJCLXKW upload=R2N2KQ31XQLS tag=FC7A5NG5LCQU type=write size=1809310 2024-05-30T16:04:00.223 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.223+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~FG7DOG143QLE.0 size=1048576 2024-05-30T16:04:00.225 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.223+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~FG7DOG143QLE.1 size=1048576 2024-05-30T16:04:00.226 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.227+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~FG7DOG143QLE.2 size=843080 2024-05-30T16:04:00.226 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.227+0000 7f08851138c0 1 > prepared multipart upload key=XHGMUYBA2VNK upload=FG7DOG143QLE tag=JN7MV5OQAPAE type=write size=3988808 2024-05-30T16:04:00.229 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.227+0000 7f08851138c0 1 < completed delete operation key=QQILXRJD0C1Z tag=8GX5A452KF6P type=del 2024-05-30T16:04:00.232 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.231+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=ZE7H1LHGKMO7 type=write size=207853 2024-05-30T16:04:00.236 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.235+0000 7f08851138c0 1 < suggested remove operation key=364U3OV10VPD tag=OMCDX0KCVLJU type=del 2024-05-30T16:04:00.239 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.239+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=1TMJHC3Y0LVM type=write size=1018810 2024-05-30T16:04:00.242 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.243+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~166V3P9XQRWZ.0 size=37363 2024-05-30T16:04:00.247 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.247+0000 7f08851138c0 1 < completed multipart upload key=3IJWPWRB6FHE upload=166V3P9XQRWZ tag=CV7QHK87OYAV type=write size=1085939 2024-05-30T16:04:00.250 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.251+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=08Q3OY2G900I type=del size=1096463 2024-05-30T16:04:00.250 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.251+0000 7f08851138c0 1 < canceled part key=_multipart_603HBR8PRH6L.2~AMNXR6WCQ9HW.0 size=1048576 2024-05-30T16:04:00.253 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.251+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~AMNXR6WCQ9HW.1 size=1048576 2024-05-30T16:04:00.255 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.255+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~AMNXR6WCQ9HW.2 size=268008 2024-05-30T16:04:00.260 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.259+0000 7f08851138c0 1 < canceled multipart upload key=603HBR8PRH6L upload=AMNXR6WCQ9HW tag=MBC2S8RQXNE4 type=write size=3413736 2024-05-30T16:04:00.263 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.263+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=SOKK07U27R90 type=del size=2981455 2024-05-30T16:04:00.267 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.267+0000 7f08851138c0 1 < completed write operation key=TPCETCO3Y1M6 tag=1TX40I3LNJF6 type=write size=368475 2024-05-30T16:04:00.271 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.271+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=UBBXLK106ZSO type=del size=596020 2024-05-30T16:04:00.276 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.275+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~9IIMCO3U6983.0 size=357589 2024-05-30T16:04:00.282 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.283+0000 7f08851138c0 1 < completed multipart upload key=QQILXRJD0C1Z upload=9IIMCO3U6983 tag=BI26QNHVQJRN type=write size=1406165 2024-05-30T16:04:00.289 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.287+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~NNI9GOJ7G1YF.0 size=1048576 2024-05-30T16:04:00.292 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.291+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~NNI9GOJ7G1YF.1 size=1048576 2024-05-30T16:04:00.295 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.295+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~NNI9GOJ7G1YF.2 size=156098 2024-05-30T16:04:00.295 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.295+0000 7f08851138c0 1 > prepared multipart upload key=O8CWPJ9GW2GO upload=NNI9GOJ7G1YF tag=172QWZANZ8VI type=write size=3301826 2024-05-30T16:04:00.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.299+0000 7f08851138c0 1 < completed delete operation key=3IJWPWRB6FHE tag=MGPRD2ED31UZ type=del 2024-05-30T16:04:00.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.307+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~2PEEC9RHY460.0 size=1048576 2024-05-30T16:04:00.309 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.307+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~2PEEC9RHY460.1 size=736962 2024-05-30T16:04:00.309 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.307+0000 7f08851138c0 1 > prepared multipart upload key=A5BUBY9Z7JQG upload=2PEEC9RHY460 tag=42TUFL2U3GT1 type=write size=2834114 2024-05-30T16:04:00.313 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.315+0000 7f08851138c0 1 < completed delete operation key=V4OQFOOQUXFQ tag=QEX6LNGPBEX9 type=del 2024-05-30T16:04:00.317 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.315+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=OA1OK28WT4SZ type=del size=402556 2024-05-30T16:04:00.321 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.319+0000 7f08851138c0 1 < completed delete operation key=YRP4CSG92QRO tag=C4NSV1900B3W type=del 2024-05-30T16:04:00.324 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.323+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=LJD7HP5HVYQU type=del size=4138174 2024-05-30T16:04:00.327 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.327+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~P5KOIYU4KF8H.0 size=1048576 2024-05-30T16:04:00.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.331+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~P5KOIYU4KF8H.1 size=498070 2024-05-30T16:04:00.335 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.335+0000 7f08851138c0 1 < completed multipart upload key=4N58ZY4E1H6E upload=P5KOIYU4KF8H tag=0T6IAS5UUONA type=write size=2595222 2024-05-30T16:04:00.338 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.339+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=QLVBLOJLC6RP type=del size=2873056 2024-05-30T16:04:00.344 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.343+0000 7f08851138c0 1 < suggested update operation key=NIV7AQ4EV5YP tag=1XT6PGF19TBY type=write size=967569 2024-05-30T16:04:00.350 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.351+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~RN0Q3O3HSTI9.0 size=647572 2024-05-30T16:04:00.350 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.351+0000 7f08851138c0 1 > prepared multipart upload key=0T2E9F3K1USF upload=RN0Q3O3HSTI9 tag=LR5WX2I5S1CM type=write size=1696148 2024-05-30T16:04:00.353 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.351+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~M3WV8GO4AEU4.0 size=1048576 2024-05-30T16:04:00.355 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.355+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~M3WV8GO4AEU4.1 size=796489 2024-05-30T16:04:00.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.359+0000 7f08851138c0 1 < completed multipart upload key=3IJWPWRB6FHE upload=M3WV8GO4AEU4 tag=F174AYJL9MWS type=write size=2893641 2024-05-30T16:04:00.363 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.363+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=OQOYOGC94SN2 type=del size=3459987 2024-05-30T16:04:00.366 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.367+0000 7f08851138c0 1 < completed delete operation key=GWXNCSJCLXKW tag=SPFOTVWWJYBQ type=del 2024-05-30T16:04:00.373 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.371+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~OY4R2D2GUKS5.0 size=1048576 2024-05-30T16:04:00.375 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.375+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~OY4R2D2GUKS5.1 size=137674 2024-05-30T16:04:00.375 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.375+0000 7f08851138c0 1 > prepared multipart upload key=ZALVY70ISLM5 upload=OY4R2D2GUKS5 tag=MKNQN2HQEWKV type=write size=2234826 2024-05-30T16:04:00.379 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.379+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~CEJ374CIM7YM.0 size=1048576 2024-05-30T16:04:00.379 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.379+0000 7f08851138c0 1 < canceled part key=_multipart_PLWFSKPKVO19.2~CEJ374CIM7YM.1 size=328412 2024-05-30T16:04:00.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.383+0000 7f08851138c0 1 < canceled multipart upload key=PLWFSKPKVO19 upload=CEJ374CIM7YM tag=DQBBKA5CUJLR type=write size=2425564 2024-05-30T16:04:00.387 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.387+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=QQCQQG8S73CT type=del size=2519000 2024-05-30T16:04:00.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.395+0000 7f08851138c0 1 < suggested update operation key=3TBOZWT2P8U8 tag=G6T0H3JF6C59 type=write size=367085 2024-05-30T16:04:00.399 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.399+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=JDN7JE1P3OMC type=del size=3763252 2024-05-30T16:04:00.402 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.403+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~FG7DOG143QLE.0 size=1048576 2024-05-30T16:04:00.406 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.407+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~FG7DOG143QLE.1 size=1048576 2024-05-30T16:04:00.409 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.407+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~FG7DOG143QLE.2 size=843080 2024-05-30T16:04:00.415 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.415+0000 7f08851138c0 1 < completed multipart upload key=XHGMUYBA2VNK upload=FG7DOG143QLE tag=JN7MV5OQAPAE type=write size=3988808 2024-05-30T16:04:00.418 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.419+0000 7f08851138c0 1 > prepared operation key=VRXNO8JLAZYH tag=QQMOIGZZBR21 type=write size=435067 2024-05-30T16:04:00.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.423+0000 7f08851138c0 1 < completed write operation key=QQILXRJD0C1Z tag=ZE7H1LHGKMO7 type=write size=207853 2024-05-30T16:04:00.426 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.427+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=DXG0FY4STZQJ type=del size=1491718 2024-05-30T16:04:00.431 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.431+0000 7f08851138c0 1 < completed write operation key=73FJ3HGWYQQK tag=1TMJHC3Y0LVM type=write size=1018810 2024-05-30T16:04:00.435 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.435+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=CARBJMPCIAWA type=write size=567777 2024-05-30T16:04:00.439 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.439+0000 7f08851138c0 1 < completed delete operation key=QCLY10D2A3VF tag=08Q3OY2G900I type=del 2024-05-30T16:04:00.443 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.443+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=J5QCU4B6EDD4 type=del size=3691254 2024-05-30T16:04:00.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.447+0000 7f08851138c0 1 < completed delete operation key=WT9RC0UYVG4F tag=SOKK07U27R90 type=del 2024-05-30T16:04:00.452 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.451+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=X1P2L33RB26U type=del size=1396221 2024-05-30T16:04:00.456 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.455+0000 7f08851138c0 1 < completed delete operation key=JNZI7JX95LX2 tag=UBBXLK106ZSO type=del 2024-05-30T16:04:00.468 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.467+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~DGV4XTA4SEIT.0 size=1048576 2024-05-30T16:04:00.471 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.471+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~DGV4XTA4SEIT.1 size=1048576 2024-05-30T16:04:00.473 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.475+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~DGV4XTA4SEIT.2 size=55769 2024-05-30T16:04:00.473 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.475+0000 7f08851138c0 1 > prepared multipart upload key=OKBB0VU765TW upload=DGV4XTA4SEIT tag=A9LG3FWJZXRC type=write size=3201497 2024-05-30T16:04:00.478 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.479+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~NNI9GOJ7G1YF.0 size=1048576 2024-05-30T16:04:00.481 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.479+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~NNI9GOJ7G1YF.1 size=1048576 2024-05-30T16:04:00.484 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.483+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~NNI9GOJ7G1YF.2 size=156098 2024-05-30T16:04:00.490 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.491+0000 7f08851138c0 1 < completed multipart upload key=O8CWPJ9GW2GO upload=NNI9GOJ7G1YF tag=172QWZANZ8VI type=write size=3301826 2024-05-30T16:04:00.493 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.495+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=HIQ5QGFVQPQE type=del size=2533467 2024-05-30T16:04:00.494 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.495+0000 7f08851138c0 1 < canceled part key=_multipart_A5BUBY9Z7JQG.2~2PEEC9RHY460.0 size=1048576 2024-05-30T16:04:00.497 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.495+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~2PEEC9RHY460.1 size=736962 2024-05-30T16:04:00.503 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.503+0000 7f08851138c0 1 < canceled multipart upload key=A5BUBY9Z7JQG upload=2PEEC9RHY460 tag=42TUFL2U3GT1 type=write size=2834114 2024-05-30T16:04:00.507 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.507+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=V5U8BRQMN38I type=del size=2935690 2024-05-30T16:04:00.510 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.511+0000 7f08851138c0 1 < completed delete operation key=3IJWPWRB6FHE tag=OA1OK28WT4SZ type=del 2024-05-30T16:04:00.516 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.515+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~SHU0GTUB2NW8.0 size=27148 2024-05-30T16:04:00.517 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.515+0000 7f08851138c0 1 > prepared multipart upload key=KPHZGFJLC2E0 upload=SHU0GTUB2NW8 tag=FXY7ZCJI4L9C type=write size=1075724 2024-05-30T16:04:00.523 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.523+0000 7f08851138c0 1 < suggested update operation key=V4OQFOOQUXFQ tag=LJD7HP5HVYQU type=del size=4138174 2024-05-30T16:04:00.526 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.527+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=X2GD6NIFTI0R type=del size=2571079 2024-05-30T16:04:00.530 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.531+0000 7f08851138c0 1 < completed delete operation key=TPCETCO3Y1M6 tag=QLVBLOJLC6RP type=del 2024-05-30T16:04:00.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.535+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~PCKXFRJO3QAG.0 size=1048576 2024-05-30T16:04:00.539 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.539+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~PCKXFRJO3QAG.1 size=161656 2024-05-30T16:04:00.539 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.539+0000 7f08851138c0 1 > prepared multipart upload key=NIV7AQ4EV5YP upload=PCKXFRJO3QAG tag=5OPT68XG5LQR type=write size=2258808 2024-05-30T16:04:00.542 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.543+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~RN0Q3O3HSTI9.0 size=647572 2024-05-30T16:04:00.547 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.547+0000 7f08851138c0 1 < completed multipart upload key=0T2E9F3K1USF upload=RN0Q3O3HSTI9 tag=LR5WX2I5S1CM type=write size=1696148 2024-05-30T16:04:00.551 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.551+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=7SWY6X9BV2EE type=del size=382594 2024-05-30T16:04:00.554 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.555+0000 7f08851138c0 1 < canceled operation key=GWXNCSJCLXKW tag=OQOYOGC94SN2 type=del size=3459987 2024-05-30T16:04:00.557 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.559+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=SQ0O95N4AO0J type=write size=684493 2024-05-30T16:04:00.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.559+0000 7f08851138c0 1 < completed part key=_multipart_ZALVY70ISLM5.2~OY4R2D2GUKS5.0 size=1048576 2024-05-30T16:04:00.564 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.563+0000 7f08851138c0 1 < completed part key=_multipart_ZALVY70ISLM5.2~OY4R2D2GUKS5.1 size=137674 2024-05-30T16:04:00.568 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.567+0000 7f08851138c0 1 < completed multipart upload key=ZALVY70ISLM5 upload=OY4R2D2GUKS5 tag=MKNQN2HQEWKV type=write size=2234826 2024-05-30T16:04:00.572 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.571+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=4LKQCIQF6VYX type=del size=1372056 2024-05-30T16:04:00.578 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.579+0000 7f08851138c0 1 < suggested remove operation key=WT9RC0UYVG4F tag=QQCQQG8S73CT type=del 2024-05-30T16:04:00.584 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.583+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~EM7R0J477WQT.0 size=1048576 2024-05-30T16:04:00.587 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.587+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~EM7R0J477WQT.1 size=273065 2024-05-30T16:04:00.587 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.587+0000 7f08851138c0 1 > prepared multipart upload key=0T2E9F3K1USF upload=EM7R0J477WQT tag=0B30QT779N9U type=write size=2370217 2024-05-30T16:04:00.593 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.591+0000 7f08851138c0 1 < suggested update operation key=1N6R2HBTWMP2 tag=JDN7JE1P3OMC type=del size=3763252 2024-05-30T16:04:00.599 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.599+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~W5EO2ZIWXWI1.0 size=1048576 2024-05-30T16:04:00.601 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.603+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~W5EO2ZIWXWI1.1 size=1048576 2024-05-30T16:04:00.604 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.603+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~W5EO2ZIWXWI1.2 size=184833 2024-05-30T16:04:00.604 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.603+0000 7f08851138c0 1 > prepared multipart upload key=VRXNO8JLAZYH upload=W5EO2ZIWXWI1 tag=Y0R4IM3I4ZTL type=write size=3330561 2024-05-30T16:04:00.608 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.607+0000 7f08851138c0 1 < completed write operation key=VRXNO8JLAZYH tag=QQMOIGZZBR21 type=write size=435067 2024-05-30T16:04:00.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.611+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=BLQ6NJ6M70I7 type=write size=947050 2024-05-30T16:04:00.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.615+0000 7f08851138c0 1 < completed delete operation key=4N58ZY4E1H6E tag=DXG0FY4STZQJ type=del 2024-05-30T16:04:00.621 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.623+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~H4RNP7WXEIQL.0 size=282646 2024-05-30T16:04:00.621 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.623+0000 7f08851138c0 1 > prepared multipart upload key=VRXNO8JLAZYH upload=H4RNP7WXEIQL tag=60RQQM5OU2FO type=write size=1331222 2024-05-30T16:04:00.625 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.623+0000 7f08851138c0 1 < completed write operation key=3TBOZWT2P8U8 tag=CARBJMPCIAWA type=write size=567777 2024-05-30T16:04:00.628 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.627+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=VCN021GJJP1H type=del size=2845058 2024-05-30T16:04:00.635 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.635+0000 7f08851138c0 1 < suggested update operation key=NIV7AQ4EV5YP tag=J5QCU4B6EDD4 type=del size=3691254 2024-05-30T16:04:00.641 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.639+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~30BY4RKBT6YD.0 size=640897 2024-05-30T16:04:00.641 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.639+0000 7f08851138c0 1 > prepared multipart upload key=QQILXRJD0C1Z upload=30BY4RKBT6YD tag=TINFYOADVG2A type=write size=1689473 2024-05-30T16:04:00.644 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.643+0000 7f08851138c0 1 < completed delete operation key=YRP4CSG92QRO tag=X1P2L33RB26U type=del 2024-05-30T16:04:00.647 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.647+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=810I2K6KOPNY type=del size=29985 2024-05-30T16:04:00.650 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.651+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~DGV4XTA4SEIT.0 size=1048576 2024-05-30T16:04:00.653 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.651+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~DGV4XTA4SEIT.1 size=1048576 2024-05-30T16:04:00.655 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.655+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~DGV4XTA4SEIT.2 size=55769 2024-05-30T16:04:00.660 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.659+0000 7f08851138c0 1 < completed multipart upload key=OKBB0VU765TW upload=DGV4XTA4SEIT tag=A9LG3FWJZXRC type=write size=3201497 2024-05-30T16:04:00.663 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.663+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=R3IUOHMH618V type=write size=173935 2024-05-30T16:04:00.667 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.667+0000 7f08851138c0 1 < completed delete operation key=Y3CYZJ83WYNI tag=HIQ5QGFVQPQE type=del 2024-05-30T16:04:00.671 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.671+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=WHEWSPPSK4N0 type=del size=431637 2024-05-30T16:04:00.678 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.679+0000 7f08851138c0 1 < suggested remove operation key=KPHZGFJLC2E0 tag=V5U8BRQMN38I type=del 2024-05-30T16:04:00.685 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.683+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~XXMLZXJ8PAM2.0 size=1048576 2024-05-30T16:04:00.688 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.687+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~XXMLZXJ8PAM2.1 size=1048576 2024-05-30T16:04:00.691 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.691+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~XXMLZXJ8PAM2.2 size=799895 2024-05-30T16:04:00.691 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.691+0000 7f08851138c0 1 > prepared multipart upload key=73FJ3HGWYQQK upload=XXMLZXJ8PAM2 tag=Z7C0FJV35O7Y type=write size=3945623 2024-05-30T16:04:00.694 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.695+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~SHU0GTUB2NW8.0 size=27148 2024-05-30T16:04:00.699 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.699+0000 7f08851138c0 1 < completed multipart upload key=KPHZGFJLC2E0 upload=SHU0GTUB2NW8 tag=FXY7ZCJI4L9C type=write size=1075724 2024-05-30T16:04:00.702 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.703+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=X5KW4FHL0RS1 type=del size=4012481 2024-05-30T16:04:00.705 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.703+0000 7f08851138c0 1 < canceled operation key=4N58ZY4E1H6E tag=X2GD6NIFTI0R type=del size=2571079 2024-05-30T16:04:00.708 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.707+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=SU8AJ3A1Y4X3 type=del size=517359 2024-05-30T16:04:00.708 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.707+0000 7f08851138c0 1 < canceled part key=_multipart_NIV7AQ4EV5YP.2~PCKXFRJO3QAG.0 size=1048576 2024-05-30T16:04:00.710 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.711+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~PCKXFRJO3QAG.1 size=161656 2024-05-30T16:04:00.715 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.715+0000 7f08851138c0 1 < canceled multipart upload key=NIV7AQ4EV5YP upload=PCKXFRJO3QAG tag=5OPT68XG5LQR type=write size=2258808 2024-05-30T16:04:00.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.719+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=3T8W3VOCTFFQ type=del size=1005122 2024-05-30T16:04:00.721 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.719+0000 7f08851138c0 1 < completed delete operation key=O0DJ8SCY766V tag=7SWY6X9BV2EE type=del 2024-05-30T16:04:00.724 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.723+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=ACGDBFXCY134 type=del size=1628440 2024-05-30T16:04:00.727 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.727+0000 7f08851138c0 1 < completed write operation key=Y8SBI0QZ0ZV8 tag=SQ0O95N4AO0J type=write size=684493 2024-05-30T16:04:00.730 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.731+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=J6ZX5B13IDES type=write size=288531 2024-05-30T16:04:00.733 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.731+0000 7f08851138c0 1 < completed delete operation key=73FJ3HGWYQQK tag=4LKQCIQF6VYX type=del 2024-05-30T16:04:00.738 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.739+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~LBPWN0Q5OSC8.0 size=1048576 2024-05-30T16:04:00.740 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.739+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~LBPWN0Q5OSC8.1 size=352808 2024-05-30T16:04:00.740 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.739+0000 7f08851138c0 1 > prepared multipart upload key=Y3CYZJ83WYNI upload=LBPWN0Q5OSC8 tag=TJ7P7D0M9MYZ type=write size=2449960 2024-05-30T16:04:00.740 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.739+0000 7f08851138c0 1 < canceled part key=_multipart_0T2E9F3K1USF.2~EM7R0J477WQT.0 size=1048576 2024-05-30T16:04:00.743 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.743+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~EM7R0J477WQT.1 size=273065 2024-05-30T16:04:00.750 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.747+0000 7f08851138c0 1 < canceled multipart upload key=0T2E9F3K1USF upload=EM7R0J477WQT tag=0B30QT779N9U type=write size=2370217 2024-05-30T16:04:00.750 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.751+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=27BNJU7YXWD7 type=write size=287358 2024-05-30T16:04:00.753 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.755+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~W5EO2ZIWXWI1.0 size=1048576 2024-05-30T16:04:00.755 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.755+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~W5EO2ZIWXWI1.1 size=1048576 2024-05-30T16:04:00.758 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.759+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~W5EO2ZIWXWI1.2 size=184833 2024-05-30T16:04:00.763 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.763+0000 7f08851138c0 1 < completed multipart upload key=VRXNO8JLAZYH upload=W5EO2ZIWXWI1 tag=Y0R4IM3I4ZTL type=write size=3330561 2024-05-30T16:04:00.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.767+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=H8C14FRJ2JX3 type=del size=1943312 2024-05-30T16:04:00.773 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.771+0000 7f08851138c0 1 < completed write operation key=0RZRBW38WKNZ tag=BLQ6NJ6M70I7 type=write size=947050 2024-05-30T16:04:00.775 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.775+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=YC8N7ODYFHTL type=del size=1008802 2024-05-30T16:04:00.778 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.779+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~H4RNP7WXEIQL.0 size=282646 2024-05-30T16:04:00.783 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.783+0000 7f08851138c0 1 < completed multipart upload key=VRXNO8JLAZYH upload=H4RNP7WXEIQL tag=60RQQM5OU2FO type=write size=1331222 2024-05-30T16:04:00.785 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.787+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=2B2RN0QGJ0U7 type=del size=1352308 2024-05-30T16:04:00.790 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.791+0000 7f08851138c0 1 < suggested remove operation key=0RZRBW38WKNZ tag=VCN021GJJP1H type=del 2024-05-30T16:04:00.793 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.791+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=6B7M3SBFFIDJ type=del size=3819339 2024-05-30T16:04:00.795 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.795+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~30BY4RKBT6YD.0 size=640897 2024-05-30T16:04:00.800 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.799+0000 7f08851138c0 1 < completed multipart upload key=QQILXRJD0C1Z upload=30BY4RKBT6YD tag=TINFYOADVG2A type=write size=1689473 2024-05-30T16:04:00.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.803+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=UGWYHDGA1PZ8 type=del size=3274974 2024-05-30T16:04:00.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.807+0000 7f08851138c0 1 < completed delete operation key=MMLF65KLC42Q tag=810I2K6KOPNY type=del 2024-05-30T16:04:00.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.807+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=VZIJKJ8J2UU6 type=write size=259434 2024-05-30T16:04:00.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.811+0000 7f08851138c0 1 < completed write operation key=A5BUBY9Z7JQG tag=R3IUOHMH618V type=write size=173935 2024-05-30T16:04:00.815 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.815+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=OC1NSDE6GA1A type=del size=721016 2024-05-30T16:04:00.818 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.819+0000 7f08851138c0 1 < completed delete operation key=PLWFSKPKVO19 tag=WHEWSPPSK4N0 type=del 2024-05-30T16:04:00.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.819+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=4X6FUVRGYVJH type=del size=2274862 2024-05-30T16:04:00.823 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.823+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~XXMLZXJ8PAM2.0 size=1048576 2024-05-30T16:04:00.825 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.827+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~XXMLZXJ8PAM2.1 size=1048576 2024-05-30T16:04:00.827 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.827+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~XXMLZXJ8PAM2.2 size=799895 2024-05-30T16:04:00.832 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.831+0000 7f08851138c0 1 < completed multipart upload key=73FJ3HGWYQQK upload=XXMLZXJ8PAM2 tag=Z7C0FJV35O7Y type=write size=3945623 2024-05-30T16:04:00.835 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.835+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=KY4127JJQF70 type=del size=2547400 2024-05-30T16:04:00.837 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.839+0000 7f08851138c0 1 < completed delete operation key=A5BUBY9Z7JQG tag=X5KW4FHL0RS1 type=del 2024-05-30T16:04:00.843 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.843+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~L9Y5RCXEX4Q2.0 size=1048576 2024-05-30T16:04:00.845 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.843+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~L9Y5RCXEX4Q2.1 size=1048576 2024-05-30T16:04:00.847 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.847+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~L9Y5RCXEX4Q2.2 size=973436 2024-05-30T16:04:00.847 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.847+0000 7f08851138c0 1 > prepared multipart upload key=4N58ZY4E1H6E upload=L9Y5RCXEX4Q2 tag=0FDFNA2M4JWA type=write size=4119164 2024-05-30T16:04:00.850 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.851+0000 7f08851138c0 1 < completed delete operation key=XHGMUYBA2VNK tag=SU8AJ3A1Y4X3 type=del 2024-05-30T16:04:00.853 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.851+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=SM87H938EN6P type=write size=246744 2024-05-30T16:04:00.856 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.855+0000 7f08851138c0 1 < completed delete operation key=TPCETCO3Y1M6 tag=3T8W3VOCTFFQ type=del 2024-05-30T16:04:00.861 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.859+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~F4G6IYUF1B16.0 size=1048576 2024-05-30T16:04:00.862 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.863+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~F4G6IYUF1B16.1 size=237657 2024-05-30T16:04:00.863 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.863+0000 7f08851138c0 1 > prepared multipart upload key=0RZRBW38WKNZ upload=F4G6IYUF1B16 tag=E6BXMPE1S3U9 type=write size=2334809 2024-05-30T16:04:00.865 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.867+0000 7f08851138c0 1 < completed delete operation key=364U3OV10VPD tag=ACGDBFXCY134 type=del 2024-05-30T16:04:00.870 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.871+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~3Z0K2LWASFWY.0 size=1048576 2024-05-30T16:04:00.872 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.871+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~3Z0K2LWASFWY.1 size=1048576 2024-05-30T16:04:00.874 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.875+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~3Z0K2LWASFWY.2 size=1020239 2024-05-30T16:04:00.874 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.875+0000 7f08851138c0 1 > prepared multipart upload key=PLWFSKPKVO19 upload=3Z0K2LWASFWY tag=1X25QW121I8J type=write size=4165967 2024-05-30T16:04:00.878 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.879+0000 7f08851138c0 1 < completed write operation key=QQILXRJD0C1Z tag=J6ZX5B13IDES type=write size=288531 2024-05-30T16:04:00.881 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.879+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=NZN9TS7PYNL2 type=del size=3873809 2024-05-30T16:04:00.883 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.883+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~LBPWN0Q5OSC8.0 size=1048576 2024-05-30T16:04:00.885 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.887+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~LBPWN0Q5OSC8.1 size=352808 2024-05-30T16:04:00.889 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.891+0000 7f08851138c0 1 < completed multipart upload key=Y3CYZJ83WYNI upload=LBPWN0Q5OSC8 tag=TJ7P7D0M9MYZ type=write size=2449960 2024-05-30T16:04:00.892 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.891+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=BNOCWW92DWZO type=del size=158783 2024-05-30T16:04:00.895 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.895+0000 7f08851138c0 1 < canceled operation key=YRP4CSG92QRO tag=27BNJU7YXWD7 type=write size=287358 2024-05-30T16:04:00.899 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.899+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~UK9PRUN63P7J.0 size=1048576 2024-05-30T16:04:00.902 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.903+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~UK9PRUN63P7J.1 size=353787 2024-05-30T16:04:00.902 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.903+0000 7f08851138c0 1 > prepared multipart upload key=Y3CYZJ83WYNI upload=UK9PRUN63P7J tag=LL6XXDVH2IWU type=write size=2450939 2024-05-30T16:04:00.904 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.903+0000 7f08851138c0 1 < completed delete operation key=GWXNCSJCLXKW tag=H8C14FRJ2JX3 type=del 2024-05-30T16:04:00.909 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.907+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~1HMSHJ6RR4DS.0 size=633636 2024-05-30T16:04:00.909 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.907+0000 7f08851138c0 1 > prepared multipart upload key=4N58ZY4E1H6E upload=1HMSHJ6RR4DS tag=7I7D3L3YQN7S type=write size=1682212 2024-05-30T16:04:00.912 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.911+0000 7f08851138c0 1 < completed delete operation key=PLWFSKPKVO19 tag=YC8N7ODYFHTL type=del 2024-05-30T16:04:00.915 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.915+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=FI81VPBCBWYY type=write size=857471 2024-05-30T16:04:00.921 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.919+0000 7f08851138c0 1 < suggested remove operation key=TPCETCO3Y1M6 tag=2B2RN0QGJ0U7 type=del 2024-05-30T16:04:00.923 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.923+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=1VOW7MJOO56F type=write size=1022303 2024-05-30T16:04:00.926 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.927+0000 7f08851138c0 1 < completed delete operation key=PLWFSKPKVO19 tag=6B7M3SBFFIDJ type=del 2024-05-30T16:04:00.930 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.927+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=X9GJ9X2UBP3W type=del size=3104760 2024-05-30T16:04:00.931 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.931+0000 7f08851138c0 1 < completed delete operation key=MMLF65KLC42Q tag=UGWYHDGA1PZ8 type=del 2024-05-30T16:04:00.934 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.935+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=HLFVO4KDMHMG type=del size=17947 2024-05-30T16:04:00.937 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.939+0000 7f08851138c0 1 < completed write operation key=E2FXSXOEUGIG tag=VZIJKJ8J2UU6 type=write size=259434 2024-05-30T16:04:00.940 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.939+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=N1LHCFWR2VPV type=del size=2613008 2024-05-30T16:04:00.946 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.947+0000 7f08851138c0 1 < suggested update operation key=A5BUBY9Z7JQG tag=OC1NSDE6GA1A type=del size=721016 2024-05-30T16:04:00.952 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.951+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~BOKF2XC4K1B6.0 size=1048576 2024-05-30T16:04:00.954 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.955+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~BOKF2XC4K1B6.1 size=1048576 2024-05-30T16:04:00.957 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.955+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~BOKF2XC4K1B6.2 size=904214 2024-05-30T16:04:00.957 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.955+0000 7f08851138c0 1 > prepared multipart upload key=73FJ3HGWYQQK upload=BOKF2XC4K1B6 tag=QVW2F1W7Q34C type=write size=4049942 2024-05-30T16:04:00.960 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.959+0000 7f08851138c0 1 < completed delete operation key=QQILXRJD0C1Z tag=4X6FUVRGYVJH type=del 2024-05-30T16:04:00.963 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.963+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=928NK5818MDM type=del size=489973 2024-05-30T16:04:00.967 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.967+0000 7f08851138c0 1 < completed delete operation key=V4OQFOOQUXFQ tag=KY4127JJQF70 type=del 2024-05-30T16:04:00.970 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.971+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=0Q1AMVPF1ZDX type=del size=1555383 2024-05-30T16:04:00.973 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.975+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~L9Y5RCXEX4Q2.0 size=1048576 2024-05-30T16:04:00.976 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.975+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~L9Y5RCXEX4Q2.1 size=1048576 2024-05-30T16:04:00.976 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.975+0000 7f08851138c0 1 < canceled part key=_multipart_4N58ZY4E1H6E.2~L9Y5RCXEX4Q2.2 size=973436 2024-05-30T16:04:00.981 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.979+0000 7f08851138c0 1 < canceled multipart upload key=4N58ZY4E1H6E upload=L9Y5RCXEX4Q2 tag=0FDFNA2M4JWA type=write size=4119164 2024-05-30T16:04:00.984 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.983+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=ZX48XEUYTHP1 type=del size=1543162 2024-05-30T16:04:00.987 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.987+0000 7f08851138c0 1 < completed write operation key=0T2E9F3K1USF tag=SM87H938EN6P type=write size=246744 2024-05-30T16:04:00.993 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.991+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~1OVT2EKFUJ98.0 size=1048576 2024-05-30T16:04:00.995 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.995+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~1OVT2EKFUJ98.1 size=868432 2024-05-30T16:04:00.995 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.995+0000 7f08851138c0 1 > prepared multipart upload key=0T2E9F3K1USF upload=1OVT2EKFUJ98 tag=ZKVS6DODA98O type=write size=2965584 2024-05-30T16:04:00.998 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.999+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~F4G6IYUF1B16.0 size=1048576 2024-05-30T16:04:01.001 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:00.999+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~F4G6IYUF1B16.1 size=237657 2024-05-30T16:04:01.006 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.007+0000 7f08851138c0 1 < completed multipart upload key=0RZRBW38WKNZ upload=F4G6IYUF1B16 tag=E6BXMPE1S3U9 type=write size=2334809 2024-05-30T16:04:01.010 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.011+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=16QCQB12Z0K6 type=del size=1478798 2024-05-30T16:04:01.014 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.015+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~3Z0K2LWASFWY.0 size=1048576 2024-05-30T16:04:01.017 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.015+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~3Z0K2LWASFWY.1 size=1048576 2024-05-30T16:04:01.019 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.019+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~3Z0K2LWASFWY.2 size=1020239 2024-05-30T16:04:01.025 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.027+0000 7f08851138c0 1 < completed multipart upload key=PLWFSKPKVO19 upload=3Z0K2LWASFWY tag=1X25QW121I8J type=write size=4165967 2024-05-30T16:04:01.029 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.027+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=9FB3411LKPUX type=del size=1908479 2024-05-30T16:04:01.032 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.031+0000 7f08851138c0 1 < completed delete operation key=1N6R2HBTWMP2 tag=NZN9TS7PYNL2 type=del 2024-05-30T16:04:01.036 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.035+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=74Z4I9WW19YK type=del size=1794302 2024-05-30T16:04:01.040 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.039+0000 7f08851138c0 1 < completed delete operation key=Y3CYZJ83WYNI tag=BNOCWW92DWZO type=del 2024-05-30T16:04:01.050 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.051+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~TP5X0A952LLA.0 size=31191 2024-05-30T16:04:01.050 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.051+0000 7f08851138c0 1 > prepared multipart upload key=3IJWPWRB6FHE upload=TP5X0A952LLA tag=361DYSH288YG type=write size=1079767 2024-05-30T16:04:01.053 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.055+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~UK9PRUN63P7J.0 size=1048576 2024-05-30T16:04:01.056 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.055+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~UK9PRUN63P7J.1 size=353787 2024-05-30T16:04:01.062 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.063+0000 7f08851138c0 1 < completed multipart upload key=Y3CYZJ83WYNI upload=UK9PRUN63P7J tag=LL6XXDVH2IWU type=write size=2450939 2024-05-30T16:04:01.065 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.067+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=ZZJCXAN192WP type=del size=850397 2024-05-30T16:04:01.069 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.067+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~1HMSHJ6RR4DS.0 size=633636 2024-05-30T16:04:01.074 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.075+0000 7f08851138c0 1 < completed multipart upload key=4N58ZY4E1H6E upload=1HMSHJ6RR4DS tag=7I7D3L3YQN7S type=write size=1682212 2024-05-30T16:04:01.080 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.079+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~5ZA2BJAEA9W7.0 size=1013515 2024-05-30T16:04:01.080 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.079+0000 7f08851138c0 1 > prepared multipart upload key=QQILXRJD0C1Z upload=5ZA2BJAEA9W7 tag=SZKS7G9V6CSV type=write size=2062091 2024-05-30T16:04:01.084 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.083+0000 7f08851138c0 1 < completed write operation key=Y3CYZJ83WYNI tag=FI81VPBCBWYY type=write size=857471 2024-05-30T16:04:01.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.091+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~EGCABC14RF3Q.0 size=383162 2024-05-30T16:04:01.091 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.091+0000 7f08851138c0 1 > prepared multipart upload key=XHGMUYBA2VNK upload=EGCABC14RF3Q tag=K054R13HX4H4 type=write size=1431738 2024-05-30T16:04:01.094 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.095+0000 7f08851138c0 1 < completed write operation key=V4OQFOOQUXFQ tag=1VOW7MJOO56F type=write size=1022303 2024-05-30T16:04:01.097 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.099+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=72H9BGMUBZQT type=del size=1483590 2024-05-30T16:04:01.104 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.103+0000 7f08851138c0 1 < suggested remove operation key=E2FXSXOEUGIG tag=X9GJ9X2UBP3W type=del 2024-05-30T16:04:01.108 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.107+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=332SDIWLZ4MQ type=del size=3480947 2024-05-30T16:04:01.115 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.115+0000 7f08851138c0 1 < suggested remove operation key=73FJ3HGWYQQK tag=HLFVO4KDMHMG type=del 2024-05-30T16:04:01.118 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.119+0000 7f08851138c0 1 > prepared operation key=ZALVY70ISLM5 tag=01S91A7SC5UF type=del size=1850590 2024-05-30T16:04:01.121 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.123+0000 7f08851138c0 1 < canceled operation key=3TBOZWT2P8U8 tag=N1LHCFWR2VPV type=del size=2613008 2024-05-30T16:04:01.127 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.127+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~ZYMPDC7Q5EI1.0 size=1048576 2024-05-30T16:04:01.130 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.131+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~ZYMPDC7Q5EI1.1 size=120057 2024-05-30T16:04:01.130 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.131+0000 7f08851138c0 1 > prepared multipart upload key=73FJ3HGWYQQK upload=ZYMPDC7Q5EI1 tag=D6THXMYP37I6 type=write size=2217209 2024-05-30T16:04:01.133 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.135+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~BOKF2XC4K1B6.0 size=1048576 2024-05-30T16:04:01.136 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.135+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~BOKF2XC4K1B6.1 size=1048576 2024-05-30T16:04:01.139 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.139+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~BOKF2XC4K1B6.2 size=904214 2024-05-30T16:04:01.145 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.143+0000 7f08851138c0 1 < completed multipart upload key=73FJ3HGWYQQK upload=BOKF2XC4K1B6 tag=QVW2F1W7Q34C type=write size=4049942 2024-05-30T16:04:01.151 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.151+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~6AGS99NUDE7Y.0 size=1048576 2024-05-30T16:04:01.153 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.155+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~6AGS99NUDE7Y.1 size=1048576 2024-05-30T16:04:01.158 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.155+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~6AGS99NUDE7Y.2 size=784718 2024-05-30T16:04:01.158 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.155+0000 7f08851138c0 1 > prepared multipart upload key=A5BUBY9Z7JQG upload=6AGS99NUDE7Y tag=X21MD67TOJY8 type=write size=3930446 2024-05-30T16:04:01.159 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.159+0000 7f08851138c0 1 < completed delete operation key=JNZI7JX95LX2 tag=928NK5818MDM type=del 2024-05-30T16:04:01.165 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.167+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~VSBP1SGHFFRB.0 size=1048576 2024-05-30T16:04:01.168 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.167+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~VSBP1SGHFFRB.1 size=1048576 2024-05-30T16:04:01.171 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.171+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~VSBP1SGHFFRB.2 size=168744 2024-05-30T16:04:01.171 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.171+0000 7f08851138c0 1 > prepared multipart upload key=3TBOZWT2P8U8 upload=VSBP1SGHFFRB tag=YGFOJP474VWB type=write size=3314472 2024-05-30T16:04:01.174 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.175+0000 7f08851138c0 1 < completed delete operation key=PLWFSKPKVO19 tag=0Q1AMVPF1ZDX type=del 2024-05-30T16:04:01.180 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.179+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~IGK828N9XC6I.0 size=908380 2024-05-30T16:04:01.180 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.179+0000 7f08851138c0 1 > prepared multipart upload key=YRP4CSG92QRO upload=IGK828N9XC6I tag=ZNCZJ1C04X80 type=write size=1956956 2024-05-30T16:04:01.183 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.183+0000 7f08851138c0 1 < completed delete operation key=Y8SBI0QZ0ZV8 tag=ZX48XEUYTHP1 type=del 2024-05-30T16:04:01.224 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.187+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=TNBYHDY3FU1O type=del size=656001 2024-05-30T16:04:01.224 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.191+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~1OVT2EKFUJ98.0 size=1048576 2024-05-30T16:04:01.224 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.195+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~1OVT2EKFUJ98.1 size=868432 2024-05-30T16:04:01.225 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.199+0000 7f08851138c0 1 < completed multipart upload key=0T2E9F3K1USF upload=1OVT2EKFUJ98 tag=ZKVS6DODA98O type=write size=2965584 2024-05-30T16:04:01.225 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.203+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=AG14IQDBY5GE type=del size=2663897 2024-05-30T16:04:01.225 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.203+0000 7f08851138c0 1 < completed delete operation key=O0DJ8SCY766V tag=16QCQB12Z0K6 type=del 2024-05-30T16:04:01.225 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.207+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=1PKTJZ2ORFQT type=del size=3960504 2024-05-30T16:04:01.225 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.215+0000 7f08851138c0 1 < suggested update operation key=TPCETCO3Y1M6 tag=9FB3411LKPUX type=del size=1908479 2024-05-30T16:04:01.225 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.219+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=B7TBTRBMUBCP type=del size=133572 2024-05-30T16:04:01.225 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.223+0000 7f08851138c0 1 < suggested update operation key=0RZRBW38WKNZ tag=74Z4I9WW19YK type=del size=1794302 2024-05-30T16:04:01.230 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.231+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~Z37RSTSCZRSQ.0 size=1048576 2024-05-30T16:04:01.233 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.231+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~Z37RSTSCZRSQ.1 size=821681 2024-05-30T16:04:01.233 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.231+0000 7f08851138c0 1 > prepared multipart upload key=QQILXRJD0C1Z upload=Z37RSTSCZRSQ tag=J30YMBS2RN0H type=write size=2918833 2024-05-30T16:04:01.236 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.235+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~TP5X0A952LLA.0 size=31191 2024-05-30T16:04:01.241 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.243+0000 7f08851138c0 1 < completed multipart upload key=3IJWPWRB6FHE upload=TP5X0A952LLA tag=361DYSH288YG type=write size=1079767 2024-05-30T16:04:01.246 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.247+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=4KFRT95ZFEKN type=del size=2041733 2024-05-30T16:04:01.250 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.251+0000 7f08851138c0 1 < completed delete operation key=603HBR8PRH6L tag=ZZJCXAN192WP type=del 2024-05-30T16:04:01.253 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.251+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=E62Y7QX4YU4I type=del size=3833993 2024-05-30T16:04:01.256 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.255+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~5ZA2BJAEA9W7.0 size=1013515 2024-05-30T16:04:01.261 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.259+0000 7f08851138c0 1 < completed multipart upload key=QQILXRJD0C1Z upload=5ZA2BJAEA9W7 tag=SZKS7G9V6CSV type=write size=2062091 2024-05-30T16:04:01.264 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.263+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=QE2221JASHSO type=del size=3756650 2024-05-30T16:04:01.268 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.267+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~EGCABC14RF3Q.0 size=383162 2024-05-30T16:04:01.274 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.275+0000 7f08851138c0 1 < completed multipart upload key=XHGMUYBA2VNK upload=EGCABC14RF3Q tag=K054R13HX4H4 type=write size=1431738 2024-05-30T16:04:01.277 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.275+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=I7JAR4Y56JZZ type=del size=3648432 2024-05-30T16:04:01.280 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.279+0000 7f08851138c0 1 < canceled operation key=O8CWPJ9GW2GO tag=72H9BGMUBZQT type=del size=1483590 2024-05-30T16:04:01.285 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.283+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~O7GKYH7SETXK.0 size=1048576 2024-05-30T16:04:01.288 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.287+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~O7GKYH7SETXK.1 size=1048576 2024-05-30T16:04:01.290 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.291+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~O7GKYH7SETXK.2 size=216064 2024-05-30T16:04:01.290 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.291+0000 7f08851138c0 1 > prepared multipart upload key=Y8SBI0QZ0ZV8 upload=O7GKYH7SETXK tag=41EPUULCO02G type=write size=3361792 2024-05-30T16:04:01.294 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.295+0000 7f08851138c0 1 < completed delete operation key=NIV7AQ4EV5YP tag=332SDIWLZ4MQ type=del 2024-05-30T16:04:01.297 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.299+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=6PBF772XMRH2 type=write size=679595 2024-05-30T16:04:01.301 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.299+0000 7f08851138c0 1 < completed delete operation key=ZALVY70ISLM5 tag=01S91A7SC5UF type=del 2024-05-30T16:04:01.304 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.303+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=G36J000NUNEE type=del size=2527335 2024-05-30T16:04:01.308 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.307+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~ZYMPDC7Q5EI1.0 size=1048576 2024-05-30T16:04:01.311 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.311+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~ZYMPDC7Q5EI1.1 size=120057 2024-05-30T16:04:01.316 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.315+0000 7f08851138c0 1 < completed multipart upload key=73FJ3HGWYQQK upload=ZYMPDC7Q5EI1 tag=D6THXMYP37I6 type=write size=2217209 2024-05-30T16:04:01.323 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.323+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~Y1TZ53501CJQ.0 size=71051 2024-05-30T16:04:01.323 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.323+0000 7f08851138c0 1 > prepared multipart upload key=73FJ3HGWYQQK upload=Y1TZ53501CJQ tag=UV65WASNPARR type=write size=1119627 2024-05-30T16:04:01.327 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.327+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~6AGS99NUDE7Y.0 size=1048576 2024-05-30T16:04:01.327 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.327+0000 7f08851138c0 1 < canceled part key=_multipart_A5BUBY9Z7JQG.2~6AGS99NUDE7Y.1 size=1048576 2024-05-30T16:04:01.327 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.327+0000 7f08851138c0 1 < canceled part key=_multipart_A5BUBY9Z7JQG.2~6AGS99NUDE7Y.2 size=784718 2024-05-30T16:04:01.334 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.335+0000 7f08851138c0 1 < canceled multipart upload key=A5BUBY9Z7JQG upload=6AGS99NUDE7Y tag=X21MD67TOJY8 type=write size=3930446 2024-05-30T16:04:01.337 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.339+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=V0KJZBQVOQ2T type=write size=553917 2024-05-30T16:04:01.341 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.343+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~VSBP1SGHFFRB.0 size=1048576 2024-05-30T16:04:01.344 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.343+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~VSBP1SGHFFRB.1 size=1048576 2024-05-30T16:04:01.347 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.347+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~VSBP1SGHFFRB.2 size=168744 2024-05-30T16:04:01.354 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.355+0000 7f08851138c0 1 < completed multipart upload key=3TBOZWT2P8U8 upload=VSBP1SGHFFRB tag=YGFOJP474VWB type=write size=3314472 2024-05-30T16:04:01.358 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.359+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=FXERJJQHKQKC type=del size=4042930 2024-05-30T16:04:01.366 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.367+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~IGK828N9XC6I.0 size=908380 2024-05-30T16:04:01.373 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.371+0000 7f08851138c0 1 < completed multipart upload key=YRP4CSG92QRO upload=IGK828N9XC6I tag=ZNCZJ1C04X80 type=write size=1956956 2024-05-30T16:04:01.377 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.375+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=OLZ8KFOQJO4I type=del size=2378864 2024-05-30T16:04:01.380 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.379+0000 7f08851138c0 1 < completed delete operation key=3IJWPWRB6FHE tag=TNBYHDY3FU1O type=del 2024-05-30T16:04:01.387 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.387+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~W1HC09A952RK.0 size=1048576 2024-05-30T16:04:01.390 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.391+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~W1HC09A952RK.1 size=1048576 2024-05-30T16:04:01.392 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.391+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~W1HC09A952RK.2 size=971835 2024-05-30T16:04:01.392 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.391+0000 7f08851138c0 1 > prepared multipart upload key=4N58ZY4E1H6E upload=W1HC09A952RK tag=ER1ZRXS16JVJ type=write size=4117563 2024-05-30T16:04:01.399 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.399+0000 7f08851138c0 1 < suggested remove operation key=V4OQFOOQUXFQ tag=AG14IQDBY5GE type=del 2024-05-30T16:04:01.402 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.403+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=9BZKQF3O2YLP type=del size=3201210 2024-05-30T16:04:01.405 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.403+0000 7f08851138c0 1 < completed delete operation key=3IJWPWRB6FHE tag=1PKTJZ2ORFQT type=del 2024-05-30T16:04:01.411 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.411+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~3GG9YQLV23IR.0 size=495001 2024-05-30T16:04:01.411 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.411+0000 7f08851138c0 1 > prepared multipart upload key=TPCETCO3Y1M6 upload=3GG9YQLV23IR tag=QI62FPV245X5 type=write size=1543577 2024-05-30T16:04:01.418 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.419+0000 7f08851138c0 1 < suggested update operation key=3TBOZWT2P8U8 tag=B7TBTRBMUBCP type=del size=133572 2024-05-30T16:04:01.420 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.419+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=NO2RQ34QLO4M type=write size=65409 2024-05-30T16:04:01.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.423+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~Z37RSTSCZRSQ.0 size=1048576 2024-05-30T16:04:01.427 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.427+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~Z37RSTSCZRSQ.1 size=821681 2024-05-30T16:04:01.433 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.431+0000 7f08851138c0 1 < completed multipart upload key=QQILXRJD0C1Z upload=Z37RSTSCZRSQ tag=J30YMBS2RN0H type=write size=2918833 2024-05-30T16:04:01.436 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.435+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=3L34HIWY3KBK type=del size=525235 2024-05-30T16:04:01.440 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.439+0000 7f08851138c0 1 < completed delete operation key=QQILXRJD0C1Z tag=4KFRT95ZFEKN type=del 2024-05-30T16:04:01.444 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.443+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=O6KHNB5M0J9S type=write size=61091 2024-05-30T16:04:01.447 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.447+0000 7f08851138c0 1 < completed delete operation key=WT9RC0UYVG4F tag=E62Y7QX4YU4I type=del 2024-05-30T16:04:01.453 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.455+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~JU9IE24G0FMZ.0 size=1048576 2024-05-30T16:04:01.456 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.455+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~JU9IE24G0FMZ.1 size=1048576 2024-05-30T16:04:01.458 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.459+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~JU9IE24G0FMZ.2 size=1008807 2024-05-30T16:04:01.458 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.459+0000 7f08851138c0 1 > prepared multipart upload key=A5BUBY9Z7JQG upload=JU9IE24G0FMZ tag=AQP8V4W6RCVL type=write size=4154535 2024-05-30T16:04:01.466 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.467+0000 7f08851138c0 1 < suggested update operation key=YRP4CSG92QRO tag=QE2221JASHSO type=del size=3756650 2024-05-30T16:04:01.472 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.471+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~0MTXX6Z1AE9R.0 size=1048576 2024-05-30T16:04:01.475 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.475+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~0MTXX6Z1AE9R.1 size=316962 2024-05-30T16:04:01.475 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.475+0000 7f08851138c0 1 > prepared multipart upload key=73FJ3HGWYQQK upload=0MTXX6Z1AE9R tag=NGUSD0ATPPBB type=write size=2414114 2024-05-30T16:04:01.478 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.479+0000 7f08851138c0 1 < completed delete operation key=OKBB0VU765TW tag=I7JAR4Y56JZZ type=del 2024-05-30T16:04:01.482 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.483+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=STUEUPW4ZNPO type=write size=829072 2024-05-30T16:04:01.486 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.487+0000 7f08851138c0 1 < completed part key=_multipart_Y8SBI0QZ0ZV8.2~O7GKYH7SETXK.0 size=1048576 2024-05-30T16:04:01.489 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.491+0000 7f08851138c0 1 < completed part key=_multipart_Y8SBI0QZ0ZV8.2~O7GKYH7SETXK.1 size=1048576 2024-05-30T16:04:01.492 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.491+0000 7f08851138c0 1 < completed part key=_multipart_Y8SBI0QZ0ZV8.2~O7GKYH7SETXK.2 size=216064 2024-05-30T16:04:01.498 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.499+0000 7f08851138c0 1 < completed multipart upload key=Y8SBI0QZ0ZV8 upload=O7GKYH7SETXK tag=41EPUULCO02G type=write size=3361792 2024-05-30T16:04:01.505 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.503+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~F6O9UFYWKK5W.0 size=1048576 2024-05-30T16:04:01.508 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.507+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~F6O9UFYWKK5W.1 size=1048576 2024-05-30T16:04:01.510 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.511+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~F6O9UFYWKK5W.2 size=1006147 2024-05-30T16:04:01.510 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.511+0000 7f08851138c0 1 > prepared multipart upload key=73FJ3HGWYQQK upload=F6O9UFYWKK5W tag=MVULU307SV7X type=write size=4151875 2024-05-30T16:04:01.514 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.515+0000 7f08851138c0 1 < completed write operation key=E2FXSXOEUGIG tag=6PBF772XMRH2 type=write size=679595 2024-05-30T16:04:01.518 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.519+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=1C2ZS2HJ6ZEK type=write size=946825 2024-05-30T16:04:01.522 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.523+0000 7f08851138c0 1 < completed delete operation key=0T2E9F3K1USF tag=G36J000NUNEE type=del 2024-05-30T16:04:01.526 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.527+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=4793WWWJ1KJ6 type=del size=2639582 2024-05-30T16:04:01.531 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.531+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~Y1TZ53501CJQ.0 size=71051 2024-05-30T16:04:01.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.535+0000 7f08851138c0 1 < completed multipart upload key=73FJ3HGWYQQK upload=Y1TZ53501CJQ tag=UV65WASNPARR type=write size=1119627 2024-05-30T16:04:01.543 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.543+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~ZYALP0D7443A.0 size=847132 2024-05-30T16:04:01.543 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.543+0000 7f08851138c0 1 > prepared multipart upload key=GWXNCSJCLXKW upload=ZYALP0D7443A tag=KNYZOAU5536C type=write size=1895708 2024-05-30T16:04:01.546 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.547+0000 7f08851138c0 1 < completed write operation key=GWXNCSJCLXKW tag=V0KJZBQVOQ2T type=write size=553917 2024-05-30T16:04:01.554 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.555+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~BNNBBP436SIC.0 size=50524 2024-05-30T16:04:01.554 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.555+0000 7f08851138c0 1 > prepared multipart upload key=GWXNCSJCLXKW upload=BNNBBP436SIC tag=XLP67AZRVN2J type=write size=1099100 2024-05-30T16:04:01.557 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.559+0000 7f08851138c0 1 < completed delete operation key=GWXNCSJCLXKW tag=FXERJJQHKQKC type=del 2024-05-30T16:04:01.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.559+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=URR97IDBQ1Y1 type=del size=265067 2024-05-30T16:04:01.566 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.563+0000 7f08851138c0 1 < canceled operation key=364U3OV10VPD tag=OLZ8KFOQJO4I type=del size=2378864 2024-05-30T16:04:01.568 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.567+0000 7f08851138c0 1 > prepared operation key=VRXNO8JLAZYH tag=F85WMG6ZT0K6 type=del size=1726854 2024-05-30T16:04:01.572 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.571+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~W1HC09A952RK.0 size=1048576 2024-05-30T16:04:01.575 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.575+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~W1HC09A952RK.1 size=1048576 2024-05-30T16:04:01.578 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.579+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~W1HC09A952RK.2 size=971835 2024-05-30T16:04:01.585 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.587+0000 7f08851138c0 1 < completed multipart upload key=4N58ZY4E1H6E upload=W1HC09A952RK tag=ER1ZRXS16JVJ type=write size=4117563 2024-05-30T16:04:01.593 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.595+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~DP71S9EE30CK.0 size=1048576 2024-05-30T16:04:01.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.595+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~DP71S9EE30CK.1 size=1048576 2024-05-30T16:04:01.600 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.599+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~DP71S9EE30CK.2 size=374804 2024-05-30T16:04:01.600 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.599+0000 7f08851138c0 1 > prepared multipart upload key=Y3CYZJ83WYNI upload=DP71S9EE30CK tag=ZRJZ8OMV2IH4 type=write size=3520532 2024-05-30T16:04:01.606 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.607+0000 7f08851138c0 1 < completed delete operation key=OKBB0VU765TW tag=9BZKQF3O2YLP type=del 2024-05-30T16:04:01.610 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.611+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=RNDTMZRWAPA8 type=del size=2372751 2024-05-30T16:04:01.613 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.615+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~3GG9YQLV23IR.0 size=495001 2024-05-30T16:04:01.620 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.619+0000 7f08851138c0 1 < completed multipart upload key=TPCETCO3Y1M6 upload=3GG9YQLV23IR tag=QI62FPV245X5 type=write size=1543577 2024-05-30T16:04:01.623 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.623+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=P4NMDQEH03OD type=del size=1008471 2024-05-30T16:04:01.630 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.631+0000 7f08851138c0 1 < suggested update operation key=0RZRBW38WKNZ tag=NO2RQ34QLO4M type=write size=65409 2024-05-30T16:04:01.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.635+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=R3LQQFERIK9E type=write size=245149 2024-05-30T16:04:01.637 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.639+0000 7f08851138c0 1 < completed delete operation key=4N58ZY4E1H6E tag=3L34HIWY3KBK type=del 2024-05-30T16:04:01.641 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.639+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=9H1HOG88492E type=del size=628576 2024-05-30T16:04:01.644 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.643+0000 7f08851138c0 1 < completed write operation key=1N6R2HBTWMP2 tag=O6KHNB5M0J9S type=write size=61091 2024-05-30T16:04:01.651 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.651+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~2CYD0YZQAXNT.0 size=1048576 2024-05-30T16:04:01.653 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.655+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~2CYD0YZQAXNT.1 size=1048576 2024-05-30T16:04:01.656 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.655+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~2CYD0YZQAXNT.2 size=729724 2024-05-30T16:04:01.656 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.655+0000 7f08851138c0 1 > prepared multipart upload key=O8CWPJ9GW2GO upload=2CYD0YZQAXNT tag=USM6JRRX3F2N type=write size=3875452 2024-05-30T16:04:01.660 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.659+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~JU9IE24G0FMZ.0 size=1048576 2024-05-30T16:04:01.662 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.663+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~JU9IE24G0FMZ.1 size=1048576 2024-05-30T16:04:01.665 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.667+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~JU9IE24G0FMZ.2 size=1008807 2024-05-30T16:04:01.672 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.671+0000 7f08851138c0 1 < completed multipart upload key=A5BUBY9Z7JQG upload=JU9IE24G0FMZ tag=AQP8V4W6RCVL type=write size=4154535 2024-05-30T16:04:01.679 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.679+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~0O39HY17KM4X.0 size=1048576 2024-05-30T16:04:01.682 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.683+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~0O39HY17KM4X.1 size=1048576 2024-05-30T16:04:01.685 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.683+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~0O39HY17KM4X.2 size=862748 2024-05-30T16:04:01.685 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.683+0000 7f08851138c0 1 > prepared multipart upload key=E2FXSXOEUGIG upload=0O39HY17KM4X tag=6S49XAK00YO2 type=write size=4008476 2024-05-30T16:04:01.688 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.687+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~0MTXX6Z1AE9R.0 size=1048576 2024-05-30T16:04:01.691 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.691+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~0MTXX6Z1AE9R.1 size=316962 2024-05-30T16:04:01.696 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.695+0000 7f08851138c0 1 < completed multipart upload key=73FJ3HGWYQQK upload=0MTXX6Z1AE9R tag=NGUSD0ATPPBB type=write size=2414114 2024-05-30T16:04:01.703 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.703+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=1C3TJY33H73J type=del size=3272924 2024-05-30T16:04:01.707 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.707+0000 7f08851138c0 1 < completed write operation key=V4OQFOOQUXFQ tag=STUEUPW4ZNPO type=write size=829072 2024-05-30T16:04:01.711 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.711+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=005Z04TWMK8Z type=del size=2325166 2024-05-30T16:04:01.715 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.715+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~F6O9UFYWKK5W.0 size=1048576 2024-05-30T16:04:01.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.719+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~F6O9UFYWKK5W.1 size=1048576 2024-05-30T16:04:01.721 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.719+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~F6O9UFYWKK5W.2 size=1006147 2024-05-30T16:04:01.726 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.727+0000 7f08851138c0 1 < completed multipart upload key=73FJ3HGWYQQK upload=F6O9UFYWKK5W tag=MVULU307SV7X type=write size=4151875 2024-05-30T16:04:01.733 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.731+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~C1VNNL2DRDP9.0 size=1002811 2024-05-30T16:04:01.733 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.731+0000 7f08851138c0 1 > prepared multipart upload key=3TBOZWT2P8U8 upload=C1VNNL2DRDP9 tag=YQ2HOW2U7HVD type=write size=2051387 2024-05-30T16:04:01.737 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.735+0000 7f08851138c0 1 < canceled operation key=YRP4CSG92QRO tag=1C2ZS2HJ6ZEK type=write size=946825 2024-05-30T16:04:01.740 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.739+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=J44G89GH93NM type=del size=3879981 2024-05-30T16:04:01.747 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.747+0000 7f08851138c0 1 < suggested update operation key=MMLF65KLC42Q tag=4793WWWJ1KJ6 type=del size=2639582 2024-05-30T16:04:01.753 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.755+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~Q0R5C3XAERTE.0 size=1048576 2024-05-30T16:04:01.756 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.755+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~Q0R5C3XAERTE.1 size=1048576 2024-05-30T16:04:01.758 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.759+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~Q0R5C3XAERTE.2 size=620567 2024-05-30T16:04:01.759 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.759+0000 7f08851138c0 1 > prepared multipart upload key=0T2E9F3K1USF upload=Q0R5C3XAERTE tag=JRGCRRTFLKQ8 type=write size=3766295 2024-05-30T16:04:01.762 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.763+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~ZYALP0D7443A.0 size=847132 2024-05-30T16:04:01.770 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.767+0000 7f08851138c0 1 < completed multipart upload key=GWXNCSJCLXKW upload=ZYALP0D7443A tag=KNYZOAU5536C type=write size=1895708 2024-05-30T16:04:01.771 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.771+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=C5EGJ0IERP2Z type=del size=2045714 2024-05-30T16:04:01.775 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.775+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~BNNBBP436SIC.0 size=50524 2024-05-30T16:04:01.782 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.783+0000 7f08851138c0 1 < completed multipart upload key=GWXNCSJCLXKW upload=BNNBBP436SIC tag=XLP67AZRVN2J type=write size=1099100 2024-05-30T16:04:01.785 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.787+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=LRC217AUECME type=del size=2733280 2024-05-30T16:04:01.793 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.795+0000 7f08851138c0 1 < suggested update operation key=E2FXSXOEUGIG tag=URR97IDBQ1Y1 type=del size=265067 2024-05-30T16:04:01.801 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.799+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~7N9DNIE762R8.0 size=375567 2024-05-30T16:04:01.801 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.799+0000 7f08851138c0 1 > prepared multipart upload key=E2FXSXOEUGIG upload=7N9DNIE762R8 tag=TKUQJP35HRTI type=write size=1424143 2024-05-30T16:04:01.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.803+0000 7f08851138c0 1 < completed delete operation key=VRXNO8JLAZYH tag=F85WMG6ZT0K6 type=del 2024-05-30T16:04:01.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.807+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=HYM2LJQ2DQU2 type=del size=455258 2024-05-30T16:04:01.812 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.811+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~DP71S9EE30CK.0 size=1048576 2024-05-30T16:04:01.812 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.811+0000 7f08851138c0 1 < canceled part key=_multipart_Y3CYZJ83WYNI.2~DP71S9EE30CK.1 size=1048576 2024-05-30T16:04:01.815 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.815+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~DP71S9EE30CK.2 size=374804 2024-05-30T16:04:01.821 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.823+0000 7f08851138c0 1 < canceled multipart upload key=Y3CYZJ83WYNI upload=DP71S9EE30CK tag=ZRJZ8OMV2IH4 type=write size=3520532 2024-05-30T16:04:01.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.827+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~D21P4LED7B3Y.0 size=1048576 2024-05-30T16:04:01.831 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.831+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~D21P4LED7B3Y.1 size=960350 2024-05-30T16:04:01.831 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.831+0000 7f08851138c0 1 > prepared multipart upload key=603HBR8PRH6L upload=D21P4LED7B3Y tag=SAOCYHP3TIEP type=write size=3057502 2024-05-30T16:04:01.835 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.835+0000 7f08851138c0 1 < completed delete operation key=4N58ZY4E1H6E tag=RNDTMZRWAPA8 type=del 2024-05-30T16:04:01.838 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.839+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=EUI06HPQIB0H type=write size=878103 2024-05-30T16:04:01.842 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.843+0000 7f08851138c0 1 < completed delete operation key=JNZI7JX95LX2 tag=P4NMDQEH03OD type=del 2024-05-30T16:04:01.846 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.847+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=1L07QZKPUQE4 type=del size=874872 2024-05-30T16:04:01.850 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.851+0000 7f08851138c0 1 < completed write operation key=O0DJ8SCY766V tag=R3LQQFERIK9E type=write size=245149 2024-05-30T16:04:01.857 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.855+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~AO4JHI78DUUD.0 size=683428 2024-05-30T16:04:01.857 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.855+0000 7f08851138c0 1 > prepared multipart upload key=4N58ZY4E1H6E upload=AO4JHI78DUUD tag=VNOXCBRJ1J77 type=write size=1732004 2024-05-30T16:04:01.860 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.859+0000 7f08851138c0 1 < completed delete operation key=NIV7AQ4EV5YP tag=9H1HOG88492E type=del 2024-05-30T16:04:01.867 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.867+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~REJX0VOV8D42.0 size=30351 2024-05-30T16:04:01.867 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.867+0000 7f08851138c0 1 > prepared multipart upload key=0T2E9F3K1USF upload=REJX0VOV8D42 tag=6YDTKCL46IPE type=write size=1078927 2024-05-30T16:04:01.871 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.871+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~2CYD0YZQAXNT.0 size=1048576 2024-05-30T16:04:01.873 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.871+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~2CYD0YZQAXNT.1 size=1048576 2024-05-30T16:04:01.875 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.875+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~2CYD0YZQAXNT.2 size=729724 2024-05-30T16:04:01.881 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.879+0000 7f08851138c0 1 < completed multipart upload key=O8CWPJ9GW2GO upload=2CYD0YZQAXNT tag=USM6JRRX3F2N type=write size=3875452 2024-05-30T16:04:01.886 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.887+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~IM2LXT73LOQ6.0 size=983402 2024-05-30T16:04:01.886 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.887+0000 7f08851138c0 1 > prepared multipart upload key=TPCETCO3Y1M6 upload=IM2LXT73LOQ6 tag=07KKU5YVES9Y type=write size=2031978 2024-05-30T16:04:01.890 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.891+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~0O39HY17KM4X.0 size=1048576 2024-05-30T16:04:01.892 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.891+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~0O39HY17KM4X.1 size=1048576 2024-05-30T16:04:01.894 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.895+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~0O39HY17KM4X.2 size=862748 2024-05-30T16:04:01.900 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.899+0000 7f08851138c0 1 < completed multipart upload key=E2FXSXOEUGIG upload=0O39HY17KM4X tag=6S49XAK00YO2 type=write size=4008476 2024-05-30T16:04:01.904 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.903+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=MFKI9PZS0Z05 type=del size=1215352 2024-05-30T16:04:01.907 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.907+0000 7f08851138c0 1 < completed delete operation key=3TBOZWT2P8U8 tag=1C3TJY33H73J type=del 2024-05-30T16:04:01.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.911+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=Y2VAXP6Q8FHZ type=write size=117115 2024-05-30T16:04:01.914 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.915+0000 7f08851138c0 1 < completed delete operation key=GWXNCSJCLXKW tag=005Z04TWMK8Z type=del 2024-05-30T16:04:01.918 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.919+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=FRXB7AI502RG type=del size=3964708 2024-05-30T16:04:01.921 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.919+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~C1VNNL2DRDP9.0 size=1002811 2024-05-30T16:04:01.926 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.927+0000 7f08851138c0 1 < completed multipart upload key=3TBOZWT2P8U8 upload=C1VNNL2DRDP9 tag=YQ2HOW2U7HVD type=write size=2051387 2024-05-30T16:04:01.930 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.931+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=BOKGAFP8R5W0 type=del size=1633545 2024-05-30T16:04:01.933 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.935+0000 7f08851138c0 1 < completed delete operation key=Y8SBI0QZ0ZV8 tag=J44G89GH93NM type=del 2024-05-30T16:04:01.939 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.939+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~KC8YNSG4U9XR.0 size=1048576 2024-05-30T16:04:01.942 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.943+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~KC8YNSG4U9XR.1 size=1048576 2024-05-30T16:04:01.944 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.943+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~KC8YNSG4U9XR.2 size=483636 2024-05-30T16:04:01.944 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.943+0000 7f08851138c0 1 > prepared multipart upload key=NIV7AQ4EV5YP upload=KC8YNSG4U9XR tag=OAZZGIE2UYMT type=write size=3629364 2024-05-30T16:04:01.948 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.947+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~Q0R5C3XAERTE.0 size=1048576 2024-05-30T16:04:01.951 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.951+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~Q0R5C3XAERTE.1 size=1048576 2024-05-30T16:04:01.954 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.955+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~Q0R5C3XAERTE.2 size=620567 2024-05-30T16:04:01.960 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.959+0000 7f08851138c0 1 < completed multipart upload key=0T2E9F3K1USF upload=Q0R5C3XAERTE tag=JRGCRRTFLKQ8 type=write size=3766295 2024-05-30T16:04:01.963 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.963+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=OS34EQ0GZKKD type=write size=656650 2024-05-30T16:04:01.970 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.971+0000 7f08851138c0 1 < suggested update operation key=XHGMUYBA2VNK tag=C5EGJ0IERP2Z type=del size=2045714 2024-05-30T16:04:01.974 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.975+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=CZ8DNJPXIG4F type=del size=2810062 2024-05-30T16:04:01.980 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.979+0000 7f08851138c0 1 < suggested remove operation key=Y3CYZJ83WYNI tag=LRC217AUECME type=del 2024-05-30T16:04:01.983 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.983+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=EDQIRE9E8JT9 type=del size=3769720 2024-05-30T16:04:01.986 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.987+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~7N9DNIE762R8.0 size=375567 2024-05-30T16:04:01.992 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.991+0000 7f08851138c0 1 < completed multipart upload key=E2FXSXOEUGIG upload=7N9DNIE762R8 tag=TKUQJP35HRTI type=write size=1424143 2024-05-30T16:04:01.996 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.995+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=BW271NKINKFU type=del size=3445324 2024-05-30T16:04:02.000 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:01.999+0000 7f08851138c0 1 < completed delete operation key=0T2E9F3K1USF tag=HYM2LJQ2DQU2 type=del 2024-05-30T16:04:02.003 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.003+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=ZCO5FY2M5D6O type=del size=2387332 2024-05-30T16:04:02.007 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.007+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~D21P4LED7B3Y.0 size=1048576 2024-05-30T16:04:02.010 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.011+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~D21P4LED7B3Y.1 size=960350 2024-05-30T16:04:02.015 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.015+0000 7f08851138c0 1 < completed multipart upload key=603HBR8PRH6L upload=D21P4LED7B3Y tag=SAOCYHP3TIEP type=write size=3057502 2024-05-30T16:04:02.021 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.023+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~MLE63BM6KEIX.0 size=1048576 2024-05-30T16:04:02.024 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.023+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~MLE63BM6KEIX.1 size=461495 2024-05-30T16:04:02.024 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.023+0000 7f08851138c0 1 > prepared multipart upload key=OKBB0VU765TW upload=MLE63BM6KEIX tag=20OG7DD7A12V type=write size=2558647 2024-05-30T16:04:02.031 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.031+0000 7f08851138c0 1 < completed write operation key=3IJWPWRB6FHE tag=EUI06HPQIB0H type=write size=878103 2024-05-30T16:04:02.034 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.035+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=QZRUDMW6GGMI type=del size=1307573 2024-05-30T16:04:02.037 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.035+0000 7f08851138c0 1 < completed delete operation key=1N6R2HBTWMP2 tag=1L07QZKPUQE4 type=del 2024-05-30T16:04:02.040 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.039+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=6HTJD46XBMVO type=del size=1953392 2024-05-30T16:04:02.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.043+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~AO4JHI78DUUD.0 size=683428 2024-05-30T16:04:02.050 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.051+0000 7f08851138c0 1 < completed multipart upload key=4N58ZY4E1H6E upload=AO4JHI78DUUD tag=VNOXCBRJ1J77 type=write size=1732004 2024-05-30T16:04:02.056 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.055+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~H9VMBLAXJMQG.0 size=1048576 2024-05-30T16:04:02.059 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.059+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~H9VMBLAXJMQG.1 size=1048576 2024-05-30T16:04:02.062 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.063+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~H9VMBLAXJMQG.2 size=400469 2024-05-30T16:04:02.062 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.063+0000 7f08851138c0 1 > prepared multipart upload key=GWXNCSJCLXKW upload=H9VMBLAXJMQG tag=O6FRSZY3JEWW type=write size=3546197 2024-05-30T16:04:02.066 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.067+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~REJX0VOV8D42.0 size=30351 2024-05-30T16:04:02.072 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.071+0000 7f08851138c0 1 < completed multipart upload key=0T2E9F3K1USF upload=REJX0VOV8D42 tag=6YDTKCL46IPE type=write size=1078927 2024-05-30T16:04:02.076 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.075+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=ZKD8NNVJKOHD type=write size=591777 2024-05-30T16:04:02.080 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.079+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~IM2LXT73LOQ6.0 size=983402 2024-05-30T16:04:02.086 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.087+0000 7f08851138c0 1 < completed multipart upload key=TPCETCO3Y1M6 upload=IM2LXT73LOQ6 tag=07KKU5YVES9Y type=write size=2031978 2024-05-30T16:04:02.089 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.091+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=82RARWV9SEPV type=del size=308651 2024-05-30T16:04:02.093 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.091+0000 7f08851138c0 1 < completed delete operation key=73FJ3HGWYQQK tag=MFKI9PZS0Z05 type=del 2024-05-30T16:04:02.097 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.095+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=IS4HT8HR0HXT type=del size=1435781 2024-05-30T16:04:02.101 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.099+0000 7f08851138c0 1 < canceled operation key=RCTV94WRMO53 tag=Y2VAXP6Q8FHZ type=write size=117115 2024-05-30T16:04:02.107 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.107+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~CRKMC4NB1STO.0 size=1048576 2024-05-30T16:04:02.110 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.111+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~CRKMC4NB1STO.1 size=1048576 2024-05-30T16:04:02.112 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.111+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~CRKMC4NB1STO.2 size=9804 2024-05-30T16:04:02.112 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.111+0000 7f08851138c0 1 > prepared multipart upload key=V4OQFOOQUXFQ upload=CRKMC4NB1STO tag=NJG66C1V5YE6 type=write size=3155532 2024-05-30T16:04:02.116 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.115+0000 7f08851138c0 1 < completed delete operation key=V4OQFOOQUXFQ tag=FRXB7AI502RG type=del 2024-05-30T16:04:02.123 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.123+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~050TT8X3V0E7.0 size=556378 2024-05-30T16:04:02.123 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.123+0000 7f08851138c0 1 > prepared multipart upload key=364U3OV10VPD upload=050TT8X3V0E7 tag=I0ITFN57STG0 type=write size=1604954 2024-05-30T16:04:02.126 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.127+0000 7f08851138c0 1 < completed delete operation key=O0DJ8SCY766V tag=BOKGAFP8R5W0 type=del 2024-05-30T16:04:02.130 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.131+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=EMMQK2R10BSA type=del size=632766 2024-05-30T16:04:02.134 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.135+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~KC8YNSG4U9XR.0 size=1048576 2024-05-30T16:04:02.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.135+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~KC8YNSG4U9XR.1 size=1048576 2024-05-30T16:04:02.140 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.139+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~KC8YNSG4U9XR.2 size=483636 2024-05-30T16:04:02.146 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.147+0000 7f08851138c0 1 < completed multipart upload key=NIV7AQ4EV5YP upload=KC8YNSG4U9XR tag=OAZZGIE2UYMT type=write size=3629364 2024-05-30T16:04:02.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.151+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=2AP2NKNJ3GOF type=del size=2965802 2024-05-30T16:04:02.153 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.151+0000 7f08851138c0 1 < canceled operation key=GWXNCSJCLXKW tag=OS34EQ0GZKKD type=write size=656650 2024-05-30T16:04:02.157 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.155+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=N5774RFH60JM type=del size=3323037 2024-05-30T16:04:02.160 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.159+0000 7f08851138c0 1 < completed delete operation key=73FJ3HGWYQQK tag=CZ8DNJPXIG4F type=del 2024-05-30T16:04:02.164 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.163+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=0S3N2E5EDBS4 type=del size=2280857 2024-05-30T16:04:02.168 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.167+0000 7f08851138c0 1 < completed delete operation key=XHGMUYBA2VNK tag=EDQIRE9E8JT9 type=del 2024-05-30T16:04:02.178 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.175+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~AJK3LRZR8N8T.0 size=1048576 2024-05-30T16:04:02.178 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.179+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~AJK3LRZR8N8T.1 size=176069 2024-05-30T16:04:02.178 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.179+0000 7f08851138c0 1 > prepared multipart upload key=73FJ3HGWYQQK upload=AJK3LRZR8N8T tag=QYXN0O1Q79UE type=write size=2273221 2024-05-30T16:04:02.184 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.183+0000 7f08851138c0 1 < suggested remove operation key=A5BUBY9Z7JQG tag=BW271NKINKFU type=del 2024-05-30T16:04:02.188 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.187+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=0XQRXR1VGTAS type=write size=584928 2024-05-30T16:04:02.195 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.195+0000 7f08851138c0 1 < suggested remove operation key=WT9RC0UYVG4F tag=ZCO5FY2M5D6O type=del 2024-05-30T16:04:02.199 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.199+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=HR2SL6FY9CT2 type=del size=3934857 2024-05-30T16:04:02.203 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.203+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~MLE63BM6KEIX.0 size=1048576 2024-05-30T16:04:02.206 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.207+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~MLE63BM6KEIX.1 size=461495 2024-05-30T16:04:02.211 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.211+0000 7f08851138c0 1 < completed multipart upload key=OKBB0VU765TW upload=MLE63BM6KEIX tag=20OG7DD7A12V type=write size=2558647 2024-05-30T16:04:02.215 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.215+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=Y6GIMS96SYI9 type=del size=4046618 2024-05-30T16:04:02.219 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.219+0000 7f08851138c0 1 < completed delete operation key=QCLY10D2A3VF tag=QZRUDMW6GGMI type=del 2024-05-30T16:04:02.223 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.223+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=0R6GXBB48A02 type=write size=386098 2024-05-30T16:04:02.230 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.231+0000 7f08851138c0 1 < suggested update operation key=1N6R2HBTWMP2 tag=6HTJD46XBMVO type=del size=1953392 2024-05-30T16:04:02.234 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.235+0000 7f08851138c0 1 > prepared operation key=ZALVY70ISLM5 tag=VGC4NC9HWWXG type=write size=500286 2024-05-30T16:04:02.237 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.239+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~H9VMBLAXJMQG.0 size=1048576 2024-05-30T16:04:02.240 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.239+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~H9VMBLAXJMQG.1 size=1048576 2024-05-30T16:04:02.243 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.243+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~H9VMBLAXJMQG.2 size=400469 2024-05-30T16:04:02.250 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.251+0000 7f08851138c0 1 < completed multipart upload key=GWXNCSJCLXKW upload=H9VMBLAXJMQG tag=O6FRSZY3JEWW type=write size=3546197 2024-05-30T16:04:02.257 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.255+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~AC72E9X3W6XH.0 size=1048576 2024-05-30T16:04:02.259 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.259+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~AC72E9X3W6XH.1 size=1032303 2024-05-30T16:04:02.260 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.259+0000 7f08851138c0 1 > prepared multipart upload key=0T2E9F3K1USF upload=AC72E9X3W6XH tag=FZSRODR1WWKP type=write size=3129455 2024-05-30T16:04:02.263 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.263+0000 7f08851138c0 1 < completed write operation key=QCLY10D2A3VF tag=ZKD8NNVJKOHD type=write size=591777 2024-05-30T16:04:02.270 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.271+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~RQM2H0QUJ946.0 size=1048576 2024-05-30T16:04:02.274 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.275+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~RQM2H0QUJ946.1 size=795074 2024-05-30T16:04:02.274 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.275+0000 7f08851138c0 1 > prepared multipart upload key=0T2E9F3K1USF upload=RQM2H0QUJ946 tag=M8G9NV36RYUJ type=write size=2892226 2024-05-30T16:04:02.278 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.279+0000 7f08851138c0 1 < completed delete operation key=TPCETCO3Y1M6 tag=82RARWV9SEPV type=del 2024-05-30T16:04:02.285 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.283+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~ATCFHJ67FHDV.0 size=1048576 2024-05-30T16:04:02.288 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.287+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~ATCFHJ67FHDV.1 size=1048576 2024-05-30T16:04:02.291 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.291+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~ATCFHJ67FHDV.2 size=698949 2024-05-30T16:04:02.291 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.291+0000 7f08851138c0 1 > prepared multipart upload key=O8CWPJ9GW2GO upload=ATCFHJ67FHDV tag=7VND4SL65WVV type=write size=3844677 2024-05-30T16:04:02.294 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.295+0000 7f08851138c0 1 < completed delete operation key=V4OQFOOQUXFQ tag=IS4HT8HR0HXT type=del 2024-05-30T16:04:02.301 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.299+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~NJHKCEYU2OMP.0 size=288462 2024-05-30T16:04:02.301 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.299+0000 7f08851138c0 1 > prepared multipart upload key=KPHZGFJLC2E0 upload=NJHKCEYU2OMP tag=E8T4SL9HXDRN type=write size=1337038 2024-05-30T16:04:02.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.303+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~CRKMC4NB1STO.0 size=1048576 2024-05-30T16:04:02.308 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.307+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~CRKMC4NB1STO.1 size=1048576 2024-05-30T16:04:02.311 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.311+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~CRKMC4NB1STO.2 size=9804 2024-05-30T16:04:02.317 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.315+0000 7f08851138c0 1 < completed multipart upload key=V4OQFOOQUXFQ upload=CRKMC4NB1STO tag=NJG66C1V5YE6 type=write size=3155532 2024-05-30T16:04:02.320 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.319+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=15ASN4VA0BUY type=del size=4020913 2024-05-30T16:04:02.323 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.323+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~050TT8X3V0E7.0 size=556378 2024-05-30T16:04:02.329 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.327+0000 7f08851138c0 1 < completed multipart upload key=364U3OV10VPD upload=050TT8X3V0E7 tag=I0ITFN57STG0 type=write size=1604954 2024-05-30T16:04:02.336 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.335+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~0I7G6NLSZ31N.0 size=1048576 2024-05-30T16:04:02.338 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.339+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~0I7G6NLSZ31N.1 size=714114 2024-05-30T16:04:02.338 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.339+0000 7f08851138c0 1 > prepared multipart upload key=3TBOZWT2P8U8 upload=0I7G6NLSZ31N tag=TRHKO5KQ3RQN type=write size=2811266 2024-05-30T16:04:02.342 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.343+0000 7f08851138c0 1 < canceled operation key=MMLF65KLC42Q tag=EMMQK2R10BSA type=del size=632766 2024-05-30T16:04:02.345 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.347+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=B4E7WNB2F0R4 type=del size=3475435 2024-05-30T16:04:02.352 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.351+0000 7f08851138c0 1 < suggested remove operation key=364U3OV10VPD tag=2AP2NKNJ3GOF type=del 2024-05-30T16:04:02.356 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.355+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=QZIV177FWHSW type=del size=1792332 2024-05-30T16:04:02.359 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.359+0000 7f08851138c0 1 < canceled operation key=TPCETCO3Y1M6 tag=N5774RFH60JM type=del size=3323037 2024-05-30T16:04:02.371 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.371+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~7U7RIEZZKHGB.0 size=1048576 2024-05-30T16:04:02.374 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.375+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~7U7RIEZZKHGB.1 size=822763 2024-05-30T16:04:02.374 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.375+0000 7f08851138c0 1 > prepared multipart upload key=73FJ3HGWYQQK upload=7U7RIEZZKHGB tag=R8WTRKNLPOT2 type=write size=2919915 2024-05-30T16:04:02.377 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.375+0000 7f08851138c0 -1 < failed to complete operation key=A5BUBY9Z7JQG tag=0S3N2E5EDBS4 type=del size=2280857: (22) Invalid argument 2024-05-30T16:04:02.380 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.379+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=IY4ONPJ93LKP type=del size=352534 2024-05-30T16:04:02.386 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.383+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~AJK3LRZR8N8T.0 size=1048576 2024-05-30T16:04:02.387 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.387+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~AJK3LRZR8N8T.1 size=176069 2024-05-30T16:04:02.392 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.391+0000 7f08851138c0 1 < completed multipart upload key=73FJ3HGWYQQK upload=AJK3LRZR8N8T tag=QYXN0O1Q79UE type=write size=2273221 2024-05-30T16:04:02.398 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.399+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~3W95N0QCS196.0 size=804969 2024-05-30T16:04:02.398 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.399+0000 7f08851138c0 1 > prepared multipart upload key=1N6R2HBTWMP2 upload=3W95N0QCS196 tag=R80W5F4P8J2Z type=write size=1853545 2024-05-30T16:04:02.403 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.403+0000 7f08851138c0 1 < completed write operation key=603HBR8PRH6L tag=0XQRXR1VGTAS type=write size=584928 2024-05-30T16:04:02.407 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.407+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=SXP5EQ5TH4NN type=del size=447769 2024-05-30T16:04:02.410 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.411+0000 7f08851138c0 1 < completed delete operation key=NIV7AQ4EV5YP tag=HR2SL6FY9CT2 type=del 2024-05-30T16:04:02.414 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.415+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=WHB853HYIOA1 type=write size=1008983 2024-05-30T16:04:02.421 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.419+0000 7f08851138c0 1 < suggested remove operation key=V4OQFOOQUXFQ tag=Y6GIMS96SYI9 type=del 2024-05-30T16:04:02.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.423+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=ZR4YG8YXMRR1 type=del size=2594011 2024-05-30T16:04:02.431 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.431+0000 7f08851138c0 1 < suggested update operation key=KPHZGFJLC2E0 tag=0R6GXBB48A02 type=write size=386098 2024-05-30T16:04:02.434 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.435+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=WNVFQ4I63LIU type=write size=739341 2024-05-30T16:04:02.437 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.439+0000 7f08851138c0 1 < completed write operation key=ZALVY70ISLM5 tag=VGC4NC9HWWXG type=write size=500286 2024-05-30T16:04:02.441 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.439+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=M0XMTT98G75X type=del size=1397020 2024-05-30T16:04:02.442 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.443+0000 7f08851138c0 1 < canceled part key=_multipart_0T2E9F3K1USF.2~AC72E9X3W6XH.0 size=1048576 2024-05-30T16:04:02.445 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.443+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~AC72E9X3W6XH.1 size=1032303 2024-05-30T16:04:02.450 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.451+0000 7f08851138c0 1 < canceled multipart upload key=0T2E9F3K1USF upload=AC72E9X3W6XH tag=FZSRODR1WWKP type=write size=3129455 2024-05-30T16:04:02.453 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.451+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=6YNQ75N1YM0A type=del size=1407813 2024-05-30T16:04:02.456 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.455+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~RQM2H0QUJ946.0 size=1048576 2024-05-30T16:04:02.459 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.459+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~RQM2H0QUJ946.1 size=795074 2024-05-30T16:04:02.465 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.463+0000 7f08851138c0 1 < completed multipart upload key=0T2E9F3K1USF upload=RQM2H0QUJ946 tag=M8G9NV36RYUJ type=write size=2892226 2024-05-30T16:04:02.468 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.467+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=NPXMAD45F1OT type=del size=164043 2024-05-30T16:04:02.469 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.467+0000 7f08851138c0 1 < canceled part key=_multipart_O8CWPJ9GW2GO.2~ATCFHJ67FHDV.0 size=1048576 2024-05-30T16:04:02.471 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.471+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~ATCFHJ67FHDV.1 size=1048576 2024-05-30T16:04:02.471 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.471+0000 7f08851138c0 1 < canceled part key=_multipart_O8CWPJ9GW2GO.2~ATCFHJ67FHDV.2 size=698949 2024-05-30T16:04:02.477 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.475+0000 7f08851138c0 1 < canceled multipart upload key=O8CWPJ9GW2GO upload=ATCFHJ67FHDV tag=7VND4SL65WVV type=write size=3844677 2024-05-30T16:04:02.480 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.479+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=TZ0UCYHSLPCL type=del size=3338403 2024-05-30T16:04:02.484 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.483+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~NJHKCEYU2OMP.0 size=288462 2024-05-30T16:04:02.490 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.491+0000 7f08851138c0 1 < completed multipart upload key=KPHZGFJLC2E0 upload=NJHKCEYU2OMP tag=E8T4SL9HXDRN type=write size=1337038 2024-05-30T16:04:02.493 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.491+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=2NWO9OCYZCCU type=write size=366539 2024-05-30T16:04:02.496 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.495+0000 7f08851138c0 1 < completed delete operation key=3TBOZWT2P8U8 tag=15ASN4VA0BUY type=del 2024-05-30T16:04:02.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.499+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=XFFZUD9N1LPY type=del size=249946 2024-05-30T16:04:02.504 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.503+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~0I7G6NLSZ31N.0 size=1048576 2024-05-30T16:04:02.507 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.507+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~0I7G6NLSZ31N.1 size=714114 2024-05-30T16:04:02.513 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.511+0000 7f08851138c0 1 < completed multipart upload key=3TBOZWT2P8U8 upload=0I7G6NLSZ31N tag=TRHKO5KQ3RQN type=write size=2811266 2024-05-30T16:04:02.517 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.515+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=DHGR5EBFAN3Z type=write size=432235 2024-05-30T16:04:02.521 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.523+0000 7f08851138c0 1 < completed delete operation key=JNZI7JX95LX2 tag=B4E7WNB2F0R4 type=del 2024-05-30T16:04:02.525 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.527+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=96HX1REGZROT type=write size=970693 2024-05-30T16:04:02.530 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.531+0000 7f08851138c0 1 < canceled operation key=3IJWPWRB6FHE tag=QZIV177FWHSW type=del size=1792332 2024-05-30T16:04:02.537 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.535+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~V3VYUDC9IIPV.0 size=1048576 2024-05-30T16:04:02.539 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.539+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~V3VYUDC9IIPV.1 size=1048576 2024-05-30T16:04:02.542 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.543+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~V3VYUDC9IIPV.2 size=424120 2024-05-30T16:04:02.542 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.543+0000 7f08851138c0 1 > prepared multipart upload key=VRXNO8JLAZYH upload=V3VYUDC9IIPV tag=8VX8YAH5AIQW type=write size=3569848 2024-05-30T16:04:02.546 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.547+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~7U7RIEZZKHGB.0 size=1048576 2024-05-30T16:04:02.548 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.547+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~7U7RIEZZKHGB.1 size=822763 2024-05-30T16:04:02.554 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.555+0000 7f08851138c0 1 < completed multipart upload key=73FJ3HGWYQQK upload=7U7RIEZZKHGB tag=R8WTRKNLPOT2 type=write size=2919915 2024-05-30T16:04:02.560 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.559+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~NUXAZBKPTP22.0 size=1048576 2024-05-30T16:04:02.563 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.563+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~NUXAZBKPTP22.1 size=268720 2024-05-30T16:04:02.563 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.563+0000 7f08851138c0 1 > prepared multipart upload key=WT9RC0UYVG4F upload=NUXAZBKPTP22 tag=8HBT8Z2V7KCK type=write size=2365872 2024-05-30T16:04:02.566 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.567+0000 7f08851138c0 1 < completed delete operation key=QQILXRJD0C1Z tag=IY4ONPJ93LKP type=del 2024-05-30T16:04:02.574 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.575+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~DCW55QJ27IP3.0 size=1048576 2024-05-30T16:04:02.577 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.575+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~DCW55QJ27IP3.1 size=1048576 2024-05-30T16:04:02.580 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.579+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~DCW55QJ27IP3.2 size=438122 2024-05-30T16:04:02.580 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.579+0000 7f08851138c0 1 > prepared multipart upload key=GWXNCSJCLXKW upload=DCW55QJ27IP3 tag=UYT0VO2FQX23 type=write size=3583850 2024-05-30T16:04:02.584 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.583+0000 7f08851138c0 1 < completed part key=_multipart_1N6R2HBTWMP2.2~3W95N0QCS196.0 size=804969 2024-05-30T16:04:02.590 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.591+0000 7f08851138c0 1 < completed multipart upload key=1N6R2HBTWMP2 upload=3W95N0QCS196 tag=R80W5F4P8J2Z type=write size=1853545 2024-05-30T16:04:02.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.595+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~7BXFAWZ1EYAG.0 size=1048576 2024-05-30T16:04:02.599 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.599+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~7BXFAWZ1EYAG.1 size=896046 2024-05-30T16:04:02.599 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.599+0000 7f08851138c0 1 > prepared multipart upload key=3IJWPWRB6FHE upload=7BXFAWZ1EYAG tag=SESKSPTYGR36 type=write size=2993198 2024-05-30T16:04:02.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.603+0000 7f08851138c0 1 < completed delete operation key=3IJWPWRB6FHE tag=SXP5EQ5TH4NN type=del 2024-05-30T16:04:02.606 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.607+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=J292E09SEOMZ type=write size=550729 2024-05-30T16:04:02.610 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.611+0000 7f08851138c0 1 < completed write operation key=73FJ3HGWYQQK tag=WHB853HYIOA1 type=write size=1008983 2024-05-30T16:04:02.616 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.615+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~LIKU4I6OF0RG.0 size=1048576 2024-05-30T16:04:02.618 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.619+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~LIKU4I6OF0RG.1 size=838187 2024-05-30T16:04:02.619 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.619+0000 7f08851138c0 1 > prepared multipart upload key=O0DJ8SCY766V upload=LIKU4I6OF0RG tag=72AXY9SX7T7X type=write size=2935339 2024-05-30T16:04:02.625 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.627+0000 7f08851138c0 1 < suggested remove operation key=603HBR8PRH6L tag=ZR4YG8YXMRR1 type=del 2024-05-30T16:04:02.630 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.631+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=P14GU0HCJGBP type=del size=95017 2024-05-30T16:04:02.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.635+0000 7f08851138c0 1 < completed write operation key=NIV7AQ4EV5YP tag=WNVFQ4I63LIU type=write size=739341 2024-05-30T16:04:02.638 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.639+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=X0Q0P5O2EXN9 type=del size=3314610 2024-05-30T16:04:02.641 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.643+0000 7f08851138c0 1 < canceled operation key=O8CWPJ9GW2GO tag=M0XMTT98G75X type=del size=1397020 2024-05-30T16:04:02.645 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.647+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=1SDP3H816K4Y type=write size=501604 2024-05-30T16:04:02.649 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.651+0000 7f08851138c0 1 < canceled operation key=XHGMUYBA2VNK tag=6YNQ75N1YM0A type=del size=1407813 2024-05-30T16:04:02.654 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.655+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=P5VFIBCC4JIH type=del size=1044699 2024-05-30T16:04:02.657 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.659+0000 7f08851138c0 1 < canceled operation key=3TBOZWT2P8U8 tag=NPXMAD45F1OT type=del size=164043 2024-05-30T16:04:02.662 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.663+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=JZSLFBYCMFVY type=del size=3665697 2024-05-30T16:04:02.669 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.667+0000 7f08851138c0 1 < suggested update operation key=O8CWPJ9GW2GO tag=TZ0UCYHSLPCL type=del size=3338403 2024-05-30T16:04:02.677 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.675+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~TSM17J6UW5C8.0 size=129749 2024-05-30T16:04:02.677 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.675+0000 7f08851138c0 1 > prepared multipart upload key=QQILXRJD0C1Z upload=TSM17J6UW5C8 tag=K3UGLYIX2E2K type=write size=1178325 2024-05-30T16:04:02.681 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.679+0000 7f08851138c0 1 < completed write operation key=Y3CYZJ83WYNI tag=2NWO9OCYZCCU type=write size=366539 2024-05-30T16:04:02.684 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.683+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=JHH5TV7LOFT6 type=write size=615468 2024-05-30T16:04:02.687 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.687+0000 7f08851138c0 1 < completed delete operation key=GWXNCSJCLXKW tag=XFFZUD9N1LPY type=del 2024-05-30T16:04:02.691 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.691+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=YASX56B8ICLF type=write size=62569 2024-05-30T16:04:02.698 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.699+0000 7f08851138c0 1 < suggested remove operation key=364U3OV10VPD tag=DHGR5EBFAN3Z type=write 2024-05-30T16:04:02.702 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.703+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=TM9HP30Q9RAI type=del size=1157495 2024-05-30T16:04:02.714 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.715+0000 7f08851138c0 1 < suggested update operation key=73FJ3HGWYQQK tag=96HX1REGZROT type=write size=970693 2024-05-30T16:04:02.717 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.715+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=IW6GUNCGC4RQ type=write size=434154 2024-05-30T16:04:02.720 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.719+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~V3VYUDC9IIPV.0 size=1048576 2024-05-30T16:04:02.722 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.723+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~V3VYUDC9IIPV.1 size=1048576 2024-05-30T16:04:02.725 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.723+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~V3VYUDC9IIPV.2 size=424120 2024-05-30T16:04:02.730 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.731+0000 7f08851138c0 1 < completed multipart upload key=VRXNO8JLAZYH upload=V3VYUDC9IIPV tag=8VX8YAH5AIQW type=write size=3569848 2024-05-30T16:04:02.735 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.735+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~8QZY3F39YLY5.0 size=1048576 2024-05-30T16:04:02.737 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.735+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~8QZY3F39YLY5.1 size=140454 2024-05-30T16:04:02.737 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.735+0000 7f08851138c0 1 > prepared multipart upload key=0T2E9F3K1USF upload=8QZY3F39YLY5 tag=J3AHR023S936 type=write size=2237606 2024-05-30T16:04:02.740 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.739+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~NUXAZBKPTP22.0 size=1048576 2024-05-30T16:04:02.742 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.743+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~NUXAZBKPTP22.1 size=268720 2024-05-30T16:04:02.747 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.747+0000 7f08851138c0 1 < completed multipart upload key=WT9RC0UYVG4F upload=NUXAZBKPTP22 tag=8HBT8Z2V7KCK type=write size=2365872 2024-05-30T16:04:02.750 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.751+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=MUKQ9HH1FJG4 type=del size=271245 2024-05-30T16:04:02.753 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.751+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~DCW55QJ27IP3.0 size=1048576 2024-05-30T16:04:02.755 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.755+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~DCW55QJ27IP3.1 size=1048576 2024-05-30T16:04:02.757 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.759+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~DCW55QJ27IP3.2 size=438122 2024-05-30T16:04:02.762 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.763+0000 7f08851138c0 1 < completed multipart upload key=GWXNCSJCLXKW upload=DCW55QJ27IP3 tag=UYT0VO2FQX23 type=write size=3583850 2024-05-30T16:04:02.765 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.763+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=WJFQABUIJB29 type=write size=904086 2024-05-30T16:04:02.768 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.767+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~7BXFAWZ1EYAG.0 size=1048576 2024-05-30T16:04:02.768 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.767+0000 7f08851138c0 1 < canceled part key=_multipart_3IJWPWRB6FHE.2~7BXFAWZ1EYAG.1 size=896046 2024-05-30T16:04:02.773 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.775+0000 7f08851138c0 1 < canceled multipart upload key=3IJWPWRB6FHE upload=7BXFAWZ1EYAG tag=SESKSPTYGR36 type=write size=2993198 2024-05-30T16:04:02.779 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.779+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~CGON4KD03LZM.0 size=382001 2024-05-30T16:04:02.779 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.779+0000 7f08851138c0 1 > prepared multipart upload key=E2FXSXOEUGIG upload=CGON4KD03LZM tag=B2VZFRS54TB0 type=write size=1430577 2024-05-30T16:04:02.784 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.783+0000 7f08851138c0 1 < suggested remove operation key=KPHZGFJLC2E0 tag=J292E09SEOMZ type=write 2024-05-30T16:04:02.789 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.791+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~4FV9347C93MH.0 size=1048576 2024-05-30T16:04:02.792 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.791+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~4FV9347C93MH.1 size=162723 2024-05-30T16:04:02.792 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.791+0000 7f08851138c0 1 > prepared multipart upload key=0T2E9F3K1USF upload=4FV9347C93MH tag=DSVB3WZEPIEX type=write size=2259875 2024-05-30T16:04:02.795 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.795+0000 7f08851138c0 1 < completed part key=_multipart_O0DJ8SCY766V.2~LIKU4I6OF0RG.0 size=1048576 2024-05-30T16:04:02.798 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.799+0000 7f08851138c0 1 < completed part key=_multipart_O0DJ8SCY766V.2~LIKU4I6OF0RG.1 size=838187 2024-05-30T16:04:02.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.803+0000 7f08851138c0 1 < completed multipart upload key=O0DJ8SCY766V upload=LIKU4I6OF0RG tag=72AXY9SX7T7X type=write size=2935339 2024-05-30T16:04:02.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.807+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~KXJ5Q4WI3OFH.0 size=467929 2024-05-30T16:04:02.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.807+0000 7f08851138c0 1 > prepared multipart upload key=603HBR8PRH6L upload=KXJ5Q4WI3OFH tag=683YSM06WM1I type=write size=1516505 2024-05-30T16:04:02.812 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.811+0000 7f08851138c0 1 < completed delete operation key=YRP4CSG92QRO tag=P14GU0HCJGBP type=del 2024-05-30T16:04:02.817 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.815+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~JC2M9BUA3676.0 size=1048576 2024-05-30T16:04:02.819 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.819+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~JC2M9BUA3676.1 size=1048576 2024-05-30T16:04:02.821 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.823+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~JC2M9BUA3676.2 size=202905 2024-05-30T16:04:02.821 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.823+0000 7f08851138c0 1 > prepared multipart upload key=3TBOZWT2P8U8 upload=JC2M9BUA3676 tag=1TWUPHKM349P type=write size=3348633 2024-05-30T16:04:02.823 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.823+0000 7f08851138c0 -1 < no bi entry to suggest for operation key=364U3OV10VPD tag=X0Q0P5O2EXN9 type=del size=3314610: (2) No such file or directory 2024-05-30T16:04:02.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.827+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~KI0W5XJDXYPF.0 size=1048576 2024-05-30T16:04:02.831 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.831+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~KI0W5XJDXYPF.1 size=450536 2024-05-30T16:04:02.831 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.831+0000 7f08851138c0 1 > prepared multipart upload key=NIV7AQ4EV5YP upload=KI0W5XJDXYPF tag=KUDLVT6L1U9U type=write size=2547688 2024-05-30T16:04:02.834 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.835+0000 7f08851138c0 1 < completed write operation key=O0DJ8SCY766V tag=1SDP3H816K4Y type=write size=501604 2024-05-30T16:04:02.840 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.839+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~TLEH0MTYOYJ5.0 size=1048576 2024-05-30T16:04:02.843 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.843+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~TLEH0MTYOYJ5.1 size=467999 2024-05-30T16:04:02.843 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.843+0000 7f08851138c0 1 > prepared multipart upload key=0RZRBW38WKNZ upload=TLEH0MTYOYJ5 tag=XFTSHIAQLQUM type=write size=2565151 2024-05-30T16:04:02.846 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.847+0000 7f08851138c0 1 < completed delete operation key=3IJWPWRB6FHE tag=P5VFIBCC4JIH type=del 2024-05-30T16:04:02.849 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.847+0000 7f08851138c0 1 > prepared operation key=ZALVY70ISLM5 tag=US3XNMSW1N86 type=del size=2160590 2024-05-30T16:04:02.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.851+0000 7f08851138c0 1 < completed delete operation key=MMLF65KLC42Q tag=JZSLFBYCMFVY type=del 2024-05-30T16:04:02.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.855+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=1QQYYK0VBQDO type=del size=2060122 2024-05-30T16:04:02.858 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.859+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~TSM17J6UW5C8.0 size=129749 2024-05-30T16:04:02.863 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.863+0000 7f08851138c0 1 < completed multipart upload key=QQILXRJD0C1Z upload=TSM17J6UW5C8 tag=K3UGLYIX2E2K type=write size=1178325 2024-05-30T16:04:02.868 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.867+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~YHOWQO1RMDZB.0 size=1048576 2024-05-30T16:04:02.871 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.871+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~YHOWQO1RMDZB.1 size=517488 2024-05-30T16:04:02.871 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.871+0000 7f08851138c0 1 > prepared multipart upload key=603HBR8PRH6L upload=YHOWQO1RMDZB tag=VCID05JM19TL type=write size=2614640 2024-05-30T16:04:02.874 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.875+0000 7f08851138c0 -1 < failed to complete operation key=73FJ3HGWYQQK tag=JHH5TV7LOFT6 type=write size=615468: (22) Invalid argument 2024-05-30T16:04:02.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.879+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~FQ9CH04O5AOK.0 size=1048576 2024-05-30T16:04:02.882 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.883+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~FQ9CH04O5AOK.1 size=274783 2024-05-30T16:04:02.882 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.883+0000 7f08851138c0 1 > prepared multipart upload key=OKBB0VU765TW upload=FQ9CH04O5AOK tag=HX3XBHA5NLQS type=write size=2371935 2024-05-30T16:04:02.885 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.883+0000 7f08851138c0 1 < canceled operation key=QQILXRJD0C1Z tag=YASX56B8ICLF type=write size=62569 2024-05-30T16:04:02.888 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.887+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=M7GV6RDNFLA0 type=del size=2620484 2024-05-30T16:04:02.891 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.891+0000 7f08851138c0 -1 < failed to complete operation key=73FJ3HGWYQQK tag=TM9HP30Q9RAI type=del size=1157495: (22) Invalid argument 2024-05-30T16:04:02.894 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.895+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=LIHY87F6P2QT type=del size=135615 2024-05-30T16:04:02.897 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.895+0000 7f08851138c0 1 < completed write operation key=3IJWPWRB6FHE tag=IW6GUNCGC4RQ type=write size=434154 2024-05-30T16:04:02.900 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.899+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=X1VJUUCLS8C0 type=del size=3460473 2024-05-30T16:04:02.903 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.903+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~8QZY3F39YLY5.0 size=1048576 2024-05-30T16:04:02.903 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.903+0000 7f08851138c0 1 < canceled part key=_multipart_0T2E9F3K1USF.2~8QZY3F39YLY5.1 size=140454 2024-05-30T16:04:02.908 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.907+0000 7f08851138c0 1 < canceled multipart upload key=0T2E9F3K1USF upload=8QZY3F39YLY5 tag=J3AHR023S936 type=write size=2237606 2024-05-30T16:04:02.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.911+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=Y96TBXO0CQVI type=del size=2269870 2024-05-30T16:04:02.915 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.915+0000 7f08851138c0 1 < completed delete operation key=603HBR8PRH6L tag=MUKQ9HH1FJG4 type=del 2024-05-30T16:04:02.921 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.919+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~HMS0UE2IQ5QN.0 size=594225 2024-05-30T16:04:02.921 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.919+0000 7f08851138c0 1 > prepared multipart upload key=VRXNO8JLAZYH upload=HMS0UE2IQ5QN tag=KOOYCV86XHKY type=write size=1642801 2024-05-30T16:04:02.923 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.923+0000 7f08851138c0 1 < canceled operation key=TPCETCO3Y1M6 tag=WJFQABUIJB29 type=write size=904086 2024-05-30T16:04:02.926 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.927+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=V66SBY2FGXZ9 type=del size=255689 2024-05-30T16:04:02.929 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.927+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~CGON4KD03LZM.0 size=382001 2024-05-30T16:04:02.933 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.935+0000 7f08851138c0 1 < completed multipart upload key=E2FXSXOEUGIG upload=CGON4KD03LZM tag=B2VZFRS54TB0 type=write size=1430577 2024-05-30T16:04:02.936 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.935+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=E6ZL1RMGWX7V type=del size=3066567 2024-05-30T16:04:02.937 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.935+0000 7f08851138c0 1 < canceled part key=_multipart_0T2E9F3K1USF.2~4FV9347C93MH.0 size=1048576 2024-05-30T16:04:02.939 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.939+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~4FV9347C93MH.1 size=162723 2024-05-30T16:04:02.947 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.943+0000 7f08851138c0 1 < canceled multipart upload key=0T2E9F3K1USF upload=4FV9347C93MH tag=DSVB3WZEPIEX type=write size=2259875 2024-05-30T16:04:02.947 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.947+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=BCQVKIFMCV1A type=write size=941536 2024-05-30T16:04:02.950 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.951+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~KXJ5Q4WI3OFH.0 size=467929 2024-05-30T16:04:02.954 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.955+0000 7f08851138c0 1 < completed multipart upload key=603HBR8PRH6L upload=KXJ5Q4WI3OFH tag=683YSM06WM1I type=write size=1516505 2024-05-30T16:04:02.960 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.959+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~2NZFEO0IZE12.0 size=1048576 2024-05-30T16:04:02.962 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.963+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~2NZFEO0IZE12.1 size=648102 2024-05-30T16:04:02.962 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.963+0000 7f08851138c0 1 > prepared multipart upload key=0T2E9F3K1USF upload=2NZFEO0IZE12 tag=8303SHU2SDKR type=write size=2745254 2024-05-30T16:04:02.962 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.963+0000 7f08851138c0 1 < canceled part key=_multipart_3TBOZWT2P8U8.2~JC2M9BUA3676.0 size=1048576 2024-05-30T16:04:02.965 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.963+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~JC2M9BUA3676.1 size=1048576 2024-05-30T16:04:02.967 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.967+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~JC2M9BUA3676.2 size=202905 2024-05-30T16:04:02.973 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.971+0000 7f08851138c0 1 < canceled multipart upload key=3TBOZWT2P8U8 upload=JC2M9BUA3676 tag=1TWUPHKM349P type=write size=3348633 2024-05-30T16:04:02.976 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.975+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=NMTUF0JPVU55 type=del size=2798307 2024-05-30T16:04:02.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.979+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~KI0W5XJDXYPF.0 size=1048576 2024-05-30T16:04:02.982 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.983+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~KI0W5XJDXYPF.1 size=450536 2024-05-30T16:04:02.991 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.991+0000 7f08851138c0 1 < completed multipart upload key=NIV7AQ4EV5YP upload=KI0W5XJDXYPF tag=KUDLVT6L1U9U type=write size=2547688 2024-05-30T16:04:02.998 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.999+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~50Z2JH2E5MWU.0 size=1048576 2024-05-30T16:04:03.001 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.999+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~50Z2JH2E5MWU.1 size=479341 2024-05-30T16:04:03.001 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:02.999+0000 7f08851138c0 1 > prepared multipart upload key=KPHZGFJLC2E0 upload=50Z2JH2E5MWU tag=2QRM28FQ3UFV type=write size=2576493 2024-05-30T16:04:03.005 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.003+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~TLEH0MTYOYJ5.0 size=1048576 2024-05-30T16:04:03.008 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.007+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~TLEH0MTYOYJ5.1 size=467999 2024-05-30T16:04:03.014 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.015+0000 7f08851138c0 1 < completed multipart upload key=0RZRBW38WKNZ upload=TLEH0MTYOYJ5 tag=XFTSHIAQLQUM type=write size=2565151 2024-05-30T16:04:03.018 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.019+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=5XC0Q0DRN7FZ type=write size=370639 2024-05-30T16:04:03.027 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.027+0000 7f08851138c0 1 < suggested remove operation key=ZALVY70ISLM5 tag=US3XNMSW1N86 type=del 2024-05-30T16:04:03.034 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.035+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~K4KTX40BPVC9.0 size=1048576 2024-05-30T16:04:03.037 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.035+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~K4KTX40BPVC9.1 size=221125 2024-05-30T16:04:03.037 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.035+0000 7f08851138c0 1 > prepared multipart upload key=QCLY10D2A3VF upload=K4KTX40BPVC9 tag=ZDH9K3PWKMAP type=write size=2318277 2024-05-30T16:04:03.041 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.039+0000 7f08851138c0 1 < completed delete operation key=Y3CYZJ83WYNI tag=1QQYYK0VBQDO type=del 2024-05-30T16:04:03.048 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.047+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~NR3QQCTP28DW.0 size=1008725 2024-05-30T16:04:03.048 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.047+0000 7f08851138c0 1 > prepared multipart upload key=Y8SBI0QZ0ZV8 upload=NR3QQCTP28DW tag=6Z1SY82OCHJP type=write size=2057301 2024-05-30T16:04:03.051 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.051+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~YHOWQO1RMDZB.0 size=1048576 2024-05-30T16:04:03.054 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.055+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~YHOWQO1RMDZB.1 size=517488 2024-05-30T16:04:03.060 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.059+0000 7f08851138c0 1 < completed multipart upload key=603HBR8PRH6L upload=YHOWQO1RMDZB tag=VCID05JM19TL type=write size=2614640 2024-05-30T16:04:03.067 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.067+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~DJ3MYUVFA253.0 size=1048576 2024-05-30T16:04:03.070 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.071+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~DJ3MYUVFA253.1 size=1048576 2024-05-30T16:04:03.072 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.071+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~DJ3MYUVFA253.2 size=581175 2024-05-30T16:04:03.073 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.071+0000 7f08851138c0 1 > prepared multipart upload key=YRP4CSG92QRO upload=DJ3MYUVFA253 tag=Y7G7J20C634N type=write size=3726903 2024-05-30T16:04:03.076 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.075+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~FQ9CH04O5AOK.0 size=1048576 2024-05-30T16:04:03.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.079+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~FQ9CH04O5AOK.1 size=274783 2024-05-30T16:04:03.084 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.083+0000 7f08851138c0 1 < completed multipart upload key=OKBB0VU765TW upload=FQ9CH04O5AOK tag=HX3XBHA5NLQS type=write size=2371935 2024-05-30T16:04:03.091 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.091+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~PK9ZBF6QFDOQ.0 size=1048576 2024-05-30T16:04:03.093 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.095+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~PK9ZBF6QFDOQ.1 size=713121 2024-05-30T16:04:03.094 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.095+0000 7f08851138c0 1 > prepared multipart upload key=1N6R2HBTWMP2 upload=PK9ZBF6QFDOQ tag=QBJ8IOOX3LH3 type=write size=2810273 2024-05-30T16:04:03.097 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.095+0000 7f08851138c0 1 < canceled operation key=JNZI7JX95LX2 tag=M7GV6RDNFLA0 type=del size=2620484 2024-05-30T16:04:03.101 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.099+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=44P583KB1NO5 type=write size=940978 2024-05-30T16:04:03.108 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.107+0000 7f08851138c0 1 < suggested remove operation key=E2FXSXOEUGIG tag=LIHY87F6P2QT type=del 2024-05-30T16:04:03.112 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.111+0000 7f08851138c0 1 > prepared operation key=VRXNO8JLAZYH tag=1590TQBKFQYI type=del size=3082638 2024-05-30T16:04:03.116 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.115+0000 7f08851138c0 1 < completed delete operation key=MMLF65KLC42Q tag=X1VJUUCLS8C0 type=del 2024-05-30T16:04:03.120 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.119+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=XIFL4JWE5U8Z type=del size=1172574 2024-05-30T16:04:03.124 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.123+0000 7f08851138c0 1 < completed delete operation key=4N58ZY4E1H6E tag=Y96TBXO0CQVI type=del 2024-05-30T16:04:03.128 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.127+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=KMWVP1112R5S type=write size=677015 2024-05-30T16:04:03.132 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.131+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~HMS0UE2IQ5QN.0 size=594225 2024-05-30T16:04:03.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.139+0000 7f08851138c0 1 < completed multipart upload key=VRXNO8JLAZYH upload=HMS0UE2IQ5QN tag=KOOYCV86XHKY type=write size=1642801 2024-05-30T16:04:03.141 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.139+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=Z4E1FWSSG1DL type=del size=4094121 2024-05-30T16:04:03.145 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.143+0000 7f08851138c0 1 < completed delete operation key=73FJ3HGWYQQK tag=V66SBY2FGXZ9 type=del 2024-05-30T16:04:03.148 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.147+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=AKLXZ5NNLTE0 type=del size=3180449 2024-05-30T16:04:03.152 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.151+0000 7f08851138c0 1 < completed delete operation key=3TBOZWT2P8U8 tag=E6ZL1RMGWX7V type=del 2024-05-30T16:04:03.159 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.159+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~9BA1OKC6MTR8.0 size=1048576 2024-05-30T16:04:03.162 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.163+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~9BA1OKC6MTR8.1 size=1048576 2024-05-30T16:04:03.164 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.163+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~9BA1OKC6MTR8.2 size=482818 2024-05-30T16:04:03.164 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.163+0000 7f08851138c0 1 > prepared multipart upload key=QCLY10D2A3VF upload=9BA1OKC6MTR8 tag=O44ARYHK99Z7 type=write size=3628546 2024-05-30T16:04:03.168 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.167+0000 7f08851138c0 1 < completed write operation key=YRP4CSG92QRO tag=BCQVKIFMCV1A type=write size=941536 2024-05-30T16:04:03.175 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.175+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~M7F9X4OOR4MA.0 size=1048576 2024-05-30T16:04:03.178 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.179+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~M7F9X4OOR4MA.1 size=1048576 2024-05-30T16:04:03.181 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.179+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~M7F9X4OOR4MA.2 size=559902 2024-05-30T16:04:03.181 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.179+0000 7f08851138c0 1 > prepared multipart upload key=QQILXRJD0C1Z upload=M7F9X4OOR4MA tag=I0HE3QPNL7JV type=write size=3705630 2024-05-30T16:04:03.184 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.183+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~2NZFEO0IZE12.0 size=1048576 2024-05-30T16:04:03.188 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.187+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~2NZFEO0IZE12.1 size=648102 2024-05-30T16:04:03.194 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.195+0000 7f08851138c0 1 < completed multipart upload key=0T2E9F3K1USF upload=2NZFEO0IZE12 tag=8303SHU2SDKR type=write size=2745254 2024-05-30T16:04:03.202 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.199+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~3HUOH8ELEY31.0 size=489006 2024-05-30T16:04:03.202 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.199+0000 7f08851138c0 1 > prepared multipart upload key=A5BUBY9Z7JQG upload=3HUOH8ELEY31 tag=C0XL8K9K7GR9 type=write size=1537582 2024-05-30T16:04:03.208 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.207+0000 7f08851138c0 1 < suggested remove operation key=OKBB0VU765TW tag=NMTUF0JPVU55 type=del 2024-05-30T16:04:03.212 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.211+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=Q8LAVRQ6ADCF type=del size=3810740 2024-05-30T16:04:03.216 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.215+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~50Z2JH2E5MWU.0 size=1048576 2024-05-30T16:04:03.219 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.219+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~50Z2JH2E5MWU.1 size=479341 2024-05-30T16:04:03.225 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.227+0000 7f08851138c0 1 < completed multipart upload key=KPHZGFJLC2E0 upload=50Z2JH2E5MWU tag=2QRM28FQ3UFV type=write size=2576493 2024-05-30T16:04:03.230 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.231+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=800LTTGF3FPT type=write size=334492 2024-05-30T16:04:03.234 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.235+0000 7f08851138c0 1 < completed write operation key=TPCETCO3Y1M6 tag=5XC0Q0DRN7FZ type=write size=370639 2024-05-30T16:04:03.238 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.239+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=CD0UV5EP367A type=write size=265835 2024-05-30T16:04:03.242 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.243+0000 7f08851138c0 1 < completed part key=_multipart_QCLY10D2A3VF.2~K4KTX40BPVC9.0 size=1048576 2024-05-30T16:04:03.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.243+0000 7f08851138c0 1 < completed part key=_multipart_QCLY10D2A3VF.2~K4KTX40BPVC9.1 size=221125 2024-05-30T16:04:03.251 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.251+0000 7f08851138c0 1 < completed multipart upload key=QCLY10D2A3VF upload=K4KTX40BPVC9 tag=ZDH9K3PWKMAP type=write size=2318277 2024-05-30T16:04:03.255 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.255+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=BU1YJR2N688P type=del size=1642829 2024-05-30T16:04:03.258 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.259+0000 7f08851138c0 1 < completed part key=_multipart_Y8SBI0QZ0ZV8.2~NR3QQCTP28DW.0 size=1008725 2024-05-30T16:04:03.265 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.263+0000 7f08851138c0 1 < completed multipart upload key=Y8SBI0QZ0ZV8 upload=NR3QQCTP28DW tag=6Z1SY82OCHJP type=write size=2057301 2024-05-30T16:04:03.272 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.271+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~RYEO1G284NOX.0 size=1048576 2024-05-30T16:04:03.275 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.275+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~RYEO1G284NOX.1 size=935308 2024-05-30T16:04:03.275 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.275+0000 7f08851138c0 1 > prepared multipart upload key=E2FXSXOEUGIG upload=RYEO1G284NOX tag=PNDW1XKF0K0H type=write size=3032460 2024-05-30T16:04:03.278 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.279+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~DJ3MYUVFA253.0 size=1048576 2024-05-30T16:04:03.281 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.279+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~DJ3MYUVFA253.1 size=1048576 2024-05-30T16:04:03.284 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.283+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~DJ3MYUVFA253.2 size=581175 2024-05-30T16:04:03.290 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.291+0000 7f08851138c0 1 < completed multipart upload key=YRP4CSG92QRO upload=DJ3MYUVFA253 tag=Y7G7J20C634N type=write size=3726903 2024-05-30T16:04:03.294 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.295+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=5IR6ITACB6Y7 type=del size=829160 2024-05-30T16:04:03.294 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.295+0000 7f08851138c0 1 < canceled part key=_multipart_1N6R2HBTWMP2.2~PK9ZBF6QFDOQ.0 size=1048576 2024-05-30T16:04:03.297 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.299+0000 7f08851138c0 1 < completed part key=_multipart_1N6R2HBTWMP2.2~PK9ZBF6QFDOQ.1 size=713121 2024-05-30T16:04:03.303 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.303+0000 7f08851138c0 1 < canceled multipart upload key=1N6R2HBTWMP2 upload=PK9ZBF6QFDOQ tag=QBJ8IOOX3LH3 type=write size=2810273 2024-05-30T16:04:03.310 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.311+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~J09ZRFH2X1O3.0 size=782851 2024-05-30T16:04:03.310 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.311+0000 7f08851138c0 1 > prepared multipart upload key=VRXNO8JLAZYH upload=J09ZRFH2X1O3 tag=WP1F4ASCKIE7 type=write size=1831427 2024-05-30T16:04:03.314 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.315+0000 7f08851138c0 1 < completed write operation key=0RZRBW38WKNZ tag=44P583KB1NO5 type=write size=940978 2024-05-30T16:04:03.318 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.319+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=FTS0H1PHIOQQ type=write size=159579 2024-05-30T16:04:03.325 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.327+0000 7f08851138c0 1 < suggested update operation key=VRXNO8JLAZYH tag=1590TQBKFQYI type=del size=3082638 2024-05-30T16:04:03.336 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.335+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~JPVFH7DEVLKH.0 size=1048576 2024-05-30T16:04:03.339 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.339+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~JPVFH7DEVLKH.1 size=803569 2024-05-30T16:04:03.339 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.339+0000 7f08851138c0 1 > prepared multipart upload key=A5BUBY9Z7JQG upload=JPVFH7DEVLKH tag=WDNO3DRQ27NC type=write size=2900721 2024-05-30T16:04:03.343 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.343+0000 7f08851138c0 1 < completed delete operation key=1N6R2HBTWMP2 tag=XIFL4JWE5U8Z type=del 2024-05-30T16:04:03.347 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.347+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=V3RPCRRLUFDX type=del size=829668 2024-05-30T16:04:03.350 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.351+0000 7f08851138c0 1 < canceled operation key=603HBR8PRH6L tag=KMWVP1112R5S type=write size=677015 2024-05-30T16:04:03.357 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.359+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~G62EA5BHOOUA.0 size=1048576 2024-05-30T16:04:03.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.359+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~G62EA5BHOOUA.1 size=1048576 2024-05-30T16:04:03.363 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.363+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~G62EA5BHOOUA.2 size=292298 2024-05-30T16:04:03.363 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.363+0000 7f08851138c0 1 > prepared multipart upload key=Y8SBI0QZ0ZV8 upload=G62EA5BHOOUA tag=1DVB2IJQ8RJ7 type=write size=3438026 2024-05-30T16:04:03.367 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.367+0000 7f08851138c0 1 < completed delete operation key=O8CWPJ9GW2GO tag=Z4E1FWSSG1DL type=del 2024-05-30T16:04:03.370 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.371+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=JRPD0AZ1RN13 type=del size=538863 2024-05-30T16:04:03.374 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.375+0000 7f08851138c0 1 < canceled operation key=RCTV94WRMO53 tag=AKLXZ5NNLTE0 type=del size=3180449 2024-05-30T16:04:03.378 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.379+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=UC8YLO4KQ151 type=del size=1855603 2024-05-30T16:04:03.379 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.379+0000 7f08851138c0 1 < canceled part key=_multipart_QCLY10D2A3VF.2~9BA1OKC6MTR8.0 size=1048576 2024-05-30T16:04:03.382 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.383+0000 7f08851138c0 1 < completed part key=_multipart_QCLY10D2A3VF.2~9BA1OKC6MTR8.1 size=1048576 2024-05-30T16:04:03.385 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.383+0000 7f08851138c0 1 < completed part key=_multipart_QCLY10D2A3VF.2~9BA1OKC6MTR8.2 size=482818 2024-05-30T16:04:03.391 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.391+0000 7f08851138c0 1 < canceled multipart upload key=QCLY10D2A3VF upload=9BA1OKC6MTR8 tag=O44ARYHK99Z7 type=write size=3628546 2024-05-30T16:04:03.398 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.399+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~3ALL0DM003IB.0 size=423002 2024-05-30T16:04:03.399 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.399+0000 7f08851138c0 1 > prepared multipart upload key=XHGMUYBA2VNK upload=3ALL0DM003IB tag=OQ2GQ7PKKDHW type=write size=1471578 2024-05-30T16:04:03.403 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.403+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~M7F9X4OOR4MA.0 size=1048576 2024-05-30T16:04:03.406 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.407+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~M7F9X4OOR4MA.1 size=1048576 2024-05-30T16:04:03.410 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.407+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~M7F9X4OOR4MA.2 size=559902 2024-05-30T16:04:03.416 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.415+0000 7f08851138c0 1 < completed multipart upload key=QQILXRJD0C1Z upload=M7F9X4OOR4MA tag=I0HE3QPNL7JV type=write size=3705630 2024-05-30T16:04:03.420 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.419+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=TS9W8345M9RL type=del size=1428370 2024-05-30T16:04:03.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.423+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~3HUOH8ELEY31.0 size=489006 2024-05-30T16:04:03.430 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.431+0000 7f08851138c0 1 < completed multipart upload key=A5BUBY9Z7JQG upload=3HUOH8ELEY31 tag=C0XL8K9K7GR9 type=write size=1537582 2024-05-30T16:04:03.433 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.435+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=L9X7BYY1HAQR type=del size=3069028 2024-05-30T16:04:03.437 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.435+0000 7f08851138c0 1 < completed delete operation key=0RZRBW38WKNZ tag=Q8LAVRQ6ADCF type=del 2024-05-30T16:04:03.441 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.439+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=U7Q8J1OW3XC0 type=del size=13087 2024-05-30T16:04:03.444 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.443+0000 7f08851138c0 1 < completed write operation key=OKBB0VU765TW tag=800LTTGF3FPT type=write size=334492 2024-05-30T16:04:03.450 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.451+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~YDMXK2UHA6AD.0 size=1048576 2024-05-30T16:04:03.452 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.451+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~YDMXK2UHA6AD.1 size=1048576 2024-05-30T16:04:03.455 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.455+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~YDMXK2UHA6AD.2 size=915523 2024-05-30T16:04:03.455 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.455+0000 7f08851138c0 1 > prepared multipart upload key=VRXNO8JLAZYH upload=YDMXK2UHA6AD tag=E3UWXI5CVSO0 type=write size=4061251 2024-05-30T16:04:03.458 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.459+0000 7f08851138c0 1 < canceled operation key=YRP4CSG92QRO tag=CD0UV5EP367A type=write size=265835 2024-05-30T16:04:03.465 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.463+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~9FDJKA9PFIDT.0 size=1048576 2024-05-30T16:04:03.467 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.467+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~9FDJKA9PFIDT.1 size=756723 2024-05-30T16:04:03.467 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.467+0000 7f08851138c0 1 > prepared multipart upload key=4N58ZY4E1H6E upload=9FDJKA9PFIDT tag=7A1MI7TSGP3Q type=write size=2853875 2024-05-30T16:04:03.471 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.471+0000 7f08851138c0 1 < completed delete operation key=73FJ3HGWYQQK tag=BU1YJR2N688P type=del 2024-05-30T16:04:03.474 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.475+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=P1QF1VKZK42C type=del size=1584291 2024-05-30T16:04:03.475 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.475+0000 7f08851138c0 1 < canceled part key=_multipart_E2FXSXOEUGIG.2~RYEO1G284NOX.0 size=1048576 2024-05-30T16:04:03.478 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.479+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~RYEO1G284NOX.1 size=935308 2024-05-30T16:04:03.484 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.483+0000 7f08851138c0 1 < canceled multipart upload key=E2FXSXOEUGIG upload=RYEO1G284NOX tag=PNDW1XKF0K0H type=write size=3032460 2024-05-30T16:04:03.490 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.491+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~XBT9Y8C38WKM.0 size=697392 2024-05-30T16:04:03.490 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.491+0000 7f08851138c0 1 > prepared multipart upload key=4N58ZY4E1H6E upload=XBT9Y8C38WKM tag=DFPZ5IH0ANPB type=write size=1745968 2024-05-30T16:04:03.493 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.491+0000 7f08851138c0 1 < completed delete operation key=TPCETCO3Y1M6 tag=5IR6ITACB6Y7 type=del 2024-05-30T16:04:03.499 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.499+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~SA5ELGVW1LQU.0 size=1048576 2024-05-30T16:04:03.502 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.503+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~SA5ELGVW1LQU.1 size=1048576 2024-05-30T16:04:03.504 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.503+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~SA5ELGVW1LQU.2 size=214957 2024-05-30T16:04:03.504 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.503+0000 7f08851138c0 1 > prepared multipart upload key=JNZI7JX95LX2 upload=SA5ELGVW1LQU tag=ZF636653SJ1H type=write size=3360685 2024-05-30T16:04:03.508 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.507+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~J09ZRFH2X1O3.0 size=782851 2024-05-30T16:04:03.514 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.515+0000 7f08851138c0 1 < completed multipart upload key=VRXNO8JLAZYH upload=J09ZRFH2X1O3 tag=WP1F4ASCKIE7 type=write size=1831427 2024-05-30T16:04:03.520 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.519+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~DCEX8XC1MW2Y.0 size=560377 2024-05-30T16:04:03.520 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.519+0000 7f08851138c0 1 > prepared multipart upload key=GWXNCSJCLXKW upload=DCEX8XC1MW2Y tag=CDI1C1T35JYT type=write size=1608953 2024-05-30T16:04:03.524 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.523+0000 7f08851138c0 1 < completed write operation key=MMLF65KLC42Q tag=FTS0H1PHIOQQ type=write size=159579 2024-05-30T16:04:03.527 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.527+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=2OKAVRTRUACK type=del size=2635787 2024-05-30T16:04:03.531 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.531+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~JPVFH7DEVLKH.0 size=1048576 2024-05-30T16:04:03.534 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.535+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~JPVFH7DEVLKH.1 size=803569 2024-05-30T16:04:03.540 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.539+0000 7f08851138c0 1 < completed multipart upload key=A5BUBY9Z7JQG upload=JPVFH7DEVLKH tag=WDNO3DRQ27NC type=write size=2900721 2024-05-30T16:04:03.543 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.543+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=10P6VGG4Y0YS type=del size=1384016 2024-05-30T16:04:03.546 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.547+0000 7f08851138c0 1 < canceled operation key=MMLF65KLC42Q tag=V3RPCRRLUFDX type=del size=829668 2024-05-30T16:04:03.552 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.551+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~O95TG3ORCFH1.0 size=1048576 2024-05-30T16:04:03.555 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.555+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~O95TG3ORCFH1.1 size=474322 2024-05-30T16:04:03.555 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.555+0000 7f08851138c0 1 > prepared multipart upload key=4N58ZY4E1H6E upload=O95TG3ORCFH1 tag=TUX1QVLXXG2W type=write size=2571474 2024-05-30T16:04:03.555 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.555+0000 7f08851138c0 1 < canceled part key=_multipart_Y8SBI0QZ0ZV8.2~G62EA5BHOOUA.0 size=1048576 2024-05-30T16:04:03.558 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.559+0000 7f08851138c0 1 < completed part key=_multipart_Y8SBI0QZ0ZV8.2~G62EA5BHOOUA.1 size=1048576 2024-05-30T16:04:03.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.559+0000 7f08851138c0 1 < completed part key=_multipart_Y8SBI0QZ0ZV8.2~G62EA5BHOOUA.2 size=292298 2024-05-30T16:04:03.567 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.567+0000 7f08851138c0 1 < canceled multipart upload key=Y8SBI0QZ0ZV8 upload=G62EA5BHOOUA tag=1DVB2IJQ8RJ7 type=write size=3438026 2024-05-30T16:04:03.574 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.575+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~5J9C82O8S0F2.0 size=1048576 2024-05-30T16:04:03.577 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.575+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~5J9C82O8S0F2.1 size=1048576 2024-05-30T16:04:03.580 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.579+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~5J9C82O8S0F2.2 size=346424 2024-05-30T16:04:03.580 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.579+0000 7f08851138c0 1 > prepared multipart upload key=QCLY10D2A3VF upload=5J9C82O8S0F2 tag=D7SQVKHO2O53 type=write size=3492152 2024-05-30T16:04:03.584 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.583+0000 7f08851138c0 1 < canceled operation key=Y8SBI0QZ0ZV8 tag=JRPD0AZ1RN13 type=del size=538863 2024-05-30T16:04:03.587 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.587+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=J4K1JB97W8BY type=del size=492340 2024-05-30T16:04:03.591 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.591+0000 7f08851138c0 1 < canceled operation key=0T2E9F3K1USF tag=UC8YLO4KQ151 type=del size=1855603 2024-05-30T16:04:03.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.599+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~FGT698H6RLU6.0 size=1048576 2024-05-30T16:04:03.601 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.599+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~FGT698H6RLU6.1 size=1048576 2024-05-30T16:04:03.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.603+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~FGT698H6RLU6.2 size=668544 2024-05-30T16:04:03.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.603+0000 7f08851138c0 1 > prepared multipart upload key=NIV7AQ4EV5YP upload=FGT698H6RLU6 tag=TJ1CB3MI80MF type=write size=3814272 2024-05-30T16:04:03.607 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.607+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~3ALL0DM003IB.0 size=423002 2024-05-30T16:04:03.614 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.615+0000 7f08851138c0 1 < completed multipart upload key=XHGMUYBA2VNK upload=3ALL0DM003IB tag=OQ2GQ7PKKDHW type=write size=1471578 2024-05-30T16:04:03.617 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.619+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=GVB1Z9OVKAGT type=del size=2655803 2024-05-30T16:04:03.625 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.623+0000 7f08851138c0 1 < suggested remove operation key=OKBB0VU765TW tag=TS9W8345M9RL type=del 2024-05-30T16:04:03.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.627+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=9JULEDVVDNSR type=del size=837385 2024-05-30T16:04:03.632 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.631+0000 7f08851138c0 1 < completed delete operation key=TPCETCO3Y1M6 tag=L9X7BYY1HAQR type=del 2024-05-30T16:04:03.640 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.639+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~FFUQRUHHDFRE.0 size=1048576 2024-05-30T16:04:03.643 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.643+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~FFUQRUHHDFRE.1 size=983747 2024-05-30T16:04:03.643 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.643+0000 7f08851138c0 1 > prepared multipart upload key=XHGMUYBA2VNK upload=FFUQRUHHDFRE tag=E2NKKGI2NFC8 type=write size=3080899 2024-05-30T16:04:03.647 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.647+0000 7f08851138c0 1 < completed delete operation key=RCTV94WRMO53 tag=U7Q8J1OW3XC0 type=del 2024-05-30T16:04:03.654 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.655+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~HLA29HOTZ69A.0 size=1048576 2024-05-30T16:04:03.658 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.659+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~HLA29HOTZ69A.1 size=706831 2024-05-30T16:04:03.658 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.659+0000 7f08851138c0 1 > prepared multipart upload key=QQILXRJD0C1Z upload=HLA29HOTZ69A tag=ENWABT0IMU5V type=write size=2803983 2024-05-30T16:04:03.662 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.663+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~YDMXK2UHA6AD.0 size=1048576 2024-05-30T16:04:03.662 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.663+0000 7f08851138c0 1 < canceled part key=_multipart_VRXNO8JLAZYH.2~YDMXK2UHA6AD.1 size=1048576 2024-05-30T16:04:03.665 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.663+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~YDMXK2UHA6AD.2 size=915523 2024-05-30T16:04:03.677 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.675+0000 7f08851138c0 1 < canceled multipart upload key=VRXNO8JLAZYH upload=YDMXK2UHA6AD tag=E3UWXI5CVSO0 type=write size=4061251 2024-05-30T16:04:03.681 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.683+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=UWNW0KCQS8WZ type=del size=3894965 2024-05-30T16:04:03.682 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.683+0000 7f08851138c0 1 < canceled part key=_multipart_4N58ZY4E1H6E.2~9FDJKA9PFIDT.0 size=1048576 2024-05-30T16:04:03.686 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.687+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~9FDJKA9PFIDT.1 size=756723 2024-05-30T16:04:03.692 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.691+0000 7f08851138c0 1 < canceled multipart upload key=4N58ZY4E1H6E upload=9FDJKA9PFIDT tag=7A1MI7TSGP3Q type=write size=2853875 2024-05-30T16:04:03.695 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.695+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=77QWGBL8HONH type=write size=482335 2024-05-30T16:04:03.698 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.699+0000 7f08851138c0 1 < completed delete operation key=A5BUBY9Z7JQG tag=P1QF1VKZK42C type=del 2024-05-30T16:04:03.701 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.703+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=GX0PY0R3GGU8 type=del size=135013 2024-05-30T16:04:03.705 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.703+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~XBT9Y8C38WKM.0 size=697392 2024-05-30T16:04:03.711 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.711+0000 7f08851138c0 1 < completed multipart upload key=4N58ZY4E1H6E upload=XBT9Y8C38WKM tag=DFPZ5IH0ANPB type=write size=1745968 2024-05-30T16:04:03.714 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.715+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=JDAKH13436R0 type=del size=3527735 2024-05-30T16:04:03.717 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.719+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~SA5ELGVW1LQU.0 size=1048576 2024-05-30T16:04:03.720 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.719+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~SA5ELGVW1LQU.1 size=1048576 2024-05-30T16:04:03.723 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.723+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~SA5ELGVW1LQU.2 size=214957 2024-05-30T16:04:03.729 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.731+0000 7f08851138c0 1 < completed multipart upload key=JNZI7JX95LX2 upload=SA5ELGVW1LQU tag=ZF636653SJ1H type=write size=3360685 2024-05-30T16:04:03.735 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.735+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~2OVBU8IIRDWR.0 size=1048576 2024-05-30T16:04:03.738 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.739+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~2OVBU8IIRDWR.1 size=1048576 2024-05-30T16:04:03.740 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.739+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~2OVBU8IIRDWR.2 size=524108 2024-05-30T16:04:03.741 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.739+0000 7f08851138c0 1 > prepared multipart upload key=A5BUBY9Z7JQG upload=2OVBU8IIRDWR tag=UF699GNQ9WE0 type=write size=3669836 2024-05-30T16:04:03.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.743+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~DCEX8XC1MW2Y.0 size=560377 2024-05-30T16:04:03.750 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.751+0000 7f08851138c0 1 < completed multipart upload key=GWXNCSJCLXKW upload=DCEX8XC1MW2Y tag=CDI1C1T35JYT type=write size=1608953 2024-05-30T16:04:03.756 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.755+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~ZZAFKRDR9K01.0 size=1048576 2024-05-30T16:04:03.758 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.759+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~ZZAFKRDR9K01.1 size=497451 2024-05-30T16:04:03.758 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.759+0000 7f08851138c0 1 > prepared multipart upload key=Y3CYZJ83WYNI upload=ZZAFKRDR9K01 tag=QUQ1GAFGQXXL type=write size=2594603 2024-05-30T16:04:03.762 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.763+0000 7f08851138c0 1 < completed delete operation key=TPCETCO3Y1M6 tag=2OKAVRTRUACK type=del 2024-05-30T16:04:03.765 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.767+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=03D0S594CJJ3 type=del size=2998289 2024-05-30T16:04:03.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.767+0000 7f08851138c0 1 < completed delete operation key=MMLF65KLC42Q tag=10P6VGG4Y0YS type=del 2024-05-30T16:04:03.772 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.771+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=2GPF7NDD0VHV type=del size=371933 2024-05-30T16:04:03.776 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.775+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~O95TG3ORCFH1.0 size=1048576 2024-05-30T16:04:03.779 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.779+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~O95TG3ORCFH1.1 size=474322 2024-05-30T16:04:03.784 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.783+0000 7f08851138c0 1 < completed multipart upload key=4N58ZY4E1H6E upload=O95TG3ORCFH1 tag=TUX1QVLXXG2W type=write size=2571474 2024-05-30T16:04:03.788 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.787+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=25Y1XKJ0ERCY type=del size=1646944 2024-05-30T16:04:03.792 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.791+0000 7f08851138c0 1 < completed part key=_multipart_QCLY10D2A3VF.2~5J9C82O8S0F2.0 size=1048576 2024-05-30T16:04:03.794 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.795+0000 7f08851138c0 1 < completed part key=_multipart_QCLY10D2A3VF.2~5J9C82O8S0F2.1 size=1048576 2024-05-30T16:04:03.794 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.795+0000 7f08851138c0 1 < canceled part key=_multipart_QCLY10D2A3VF.2~5J9C82O8S0F2.2 size=346424 2024-05-30T16:04:03.800 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.799+0000 7f08851138c0 1 < canceled multipart upload key=QCLY10D2A3VF upload=5J9C82O8S0F2 tag=D7SQVKHO2O53 type=write size=3492152 2024-05-30T16:04:03.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.803+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=CX3CY105KJ2V type=del size=3484552 2024-05-30T16:04:03.807 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.807+0000 7f08851138c0 1 < completed delete operation key=Y3CYZJ83WYNI tag=J4K1JB97W8BY type=del 2024-05-30T16:04:03.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.811+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=NY17KTR1KC66 type=del size=782945 2024-05-30T16:04:03.814 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.815+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~FGT698H6RLU6.0 size=1048576 2024-05-30T16:04:03.818 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.815+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~FGT698H6RLU6.1 size=1048576 2024-05-30T16:04:03.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.819+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~FGT698H6RLU6.2 size=668544 2024-05-30T16:04:03.825 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.827+0000 7f08851138c0 1 < completed multipart upload key=NIV7AQ4EV5YP upload=FGT698H6RLU6 tag=TJ1CB3MI80MF type=write size=3814272 2024-05-30T16:04:03.829 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.827+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=6IGPICQSEA2R type=del size=1168346 2024-05-30T16:04:03.833 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.831+0000 7f08851138c0 1 < completed delete operation key=4N58ZY4E1H6E tag=GVB1Z9OVKAGT type=del 2024-05-30T16:04:03.836 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.835+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=MLUF8YBC1T98 type=write size=735972 2024-05-30T16:04:03.840 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.839+0000 7f08851138c0 1 < completed delete operation key=3IJWPWRB6FHE tag=9JULEDVVDNSR type=del 2024-05-30T16:04:03.843 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.843+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=U35XVDRPE901 type=del size=3012445 2024-05-30T16:04:03.847 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.847+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~FFUQRUHHDFRE.0 size=1048576 2024-05-30T16:04:03.850 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.851+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~FFUQRUHHDFRE.1 size=983747 2024-05-30T16:04:03.856 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.855+0000 7f08851138c0 1 < completed multipart upload key=XHGMUYBA2VNK upload=FFUQRUHHDFRE tag=E2NKKGI2NFC8 type=write size=3080899 2024-05-30T16:04:03.862 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.863+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~BAMZ9OLP0M2H.0 size=486642 2024-05-30T16:04:03.862 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.863+0000 7f08851138c0 1 > prepared multipart upload key=OKBB0VU765TW upload=BAMZ9OLP0M2H tag=NZVVREZLAHBI type=write size=1535218 2024-05-30T16:04:03.866 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.867+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~HLA29HOTZ69A.0 size=1048576 2024-05-30T16:04:03.866 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.867+0000 7f08851138c0 1 < canceled part key=_multipart_QQILXRJD0C1Z.2~HLA29HOTZ69A.1 size=706831 2024-05-30T16:04:03.872 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.871+0000 7f08851138c0 1 < canceled multipart upload key=QQILXRJD0C1Z upload=HLA29HOTZ69A tag=ENWABT0IMU5V type=write size=2803983 2024-05-30T16:04:03.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.879+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~Q8U1W7R3P31Z.0 size=1037898 2024-05-30T16:04:03.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.879+0000 7f08851138c0 1 > prepared multipart upload key=RCTV94WRMO53 upload=Q8U1W7R3P31Z tag=F2KC59YZTEIV type=write size=2086474 2024-05-30T16:04:03.882 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.883+0000 7f08851138c0 1 < completed delete operation key=RCTV94WRMO53 tag=UWNW0KCQS8WZ type=del 2024-05-30T16:04:03.889 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.891+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~I65TLX3UM17B.0 size=997423 2024-05-30T16:04:03.889 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.891+0000 7f08851138c0 1 > prepared multipart upload key=RCTV94WRMO53 upload=I65TLX3UM17B tag=1NHX7K6ODR3G type=write size=2045999 2024-05-30T16:04:03.893 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.891+0000 7f08851138c0 1 < completed write operation key=O0DJ8SCY766V tag=77QWGBL8HONH type=write size=482335 2024-05-30T16:04:03.896 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.895+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=J4XQFKLDWAZ8 type=write size=19157 2024-05-30T16:04:03.900 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.899+0000 7f08851138c0 1 < completed delete operation key=E2FXSXOEUGIG tag=GX0PY0R3GGU8 type=del 2024-05-30T16:04:03.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.907+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~RFSNA7VGW43C.0 size=1048576 2024-05-30T16:04:03.908 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.907+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~RFSNA7VGW43C.1 size=1048576 2024-05-30T16:04:03.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.911+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~RFSNA7VGW43C.2 size=258863 2024-05-30T16:04:03.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.911+0000 7f08851138c0 1 > prepared multipart upload key=YRP4CSG92QRO upload=RFSNA7VGW43C tag=RX1CV2CCJSKI type=write size=3404591 2024-05-30T16:04:03.915 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.915+0000 7f08851138c0 1 < completed delete operation key=A5BUBY9Z7JQG tag=JDAKH13436R0 type=del 2024-05-30T16:04:03.918 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.919+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=A1I55UCRSD1P type=del size=3594553 2024-05-30T16:04:03.922 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.923+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~2OVBU8IIRDWR.0 size=1048576 2024-05-30T16:04:03.925 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.923+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~2OVBU8IIRDWR.1 size=1048576 2024-05-30T16:04:03.928 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.927+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~2OVBU8IIRDWR.2 size=524108 2024-05-30T16:04:03.934 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.935+0000 7f08851138c0 1 < completed multipart upload key=A5BUBY9Z7JQG upload=2OVBU8IIRDWR tag=UF699GNQ9WE0 type=write size=3669836 2024-05-30T16:04:03.938 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.939+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=P1N849D64Z16 type=del size=287618 2024-05-30T16:04:03.942 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.943+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~ZZAFKRDR9K01.0 size=1048576 2024-05-30T16:04:03.945 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.943+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~ZZAFKRDR9K01.1 size=497451 2024-05-30T16:04:03.951 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.951+0000 7f08851138c0 1 < completed multipart upload key=Y3CYZJ83WYNI upload=ZZAFKRDR9K01 tag=QUQ1GAFGQXXL type=write size=2594603 2024-05-30T16:04:03.954 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.955+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=FK1D1XM2SNUG type=del size=3690997 2024-05-30T16:04:03.958 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.959+0000 7f08851138c0 1 < completed delete operation key=KPHZGFJLC2E0 tag=03D0S594CJJ3 type=del 2024-05-30T16:04:03.964 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.963+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~HTHJKY9YGRNJ.0 size=1048576 2024-05-30T16:04:03.967 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.967+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~HTHJKY9YGRNJ.1 size=991044 2024-05-30T16:04:03.967 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.967+0000 7f08851138c0 1 > prepared multipart upload key=RCTV94WRMO53 upload=HTHJKY9YGRNJ tag=CQTWS1YWNBQH type=write size=3088196 2024-05-30T16:04:03.971 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.971+0000 7f08851138c0 1 < completed delete operation key=1N6R2HBTWMP2 tag=2GPF7NDD0VHV type=del 2024-05-30T16:04:03.975 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.975+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=DA4UIOBFFBVK type=del size=3446121 2024-05-30T16:04:03.978 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.979+0000 7f08851138c0 1 < completed delete operation key=RCTV94WRMO53 tag=25Y1XKJ0ERCY type=del 2024-05-30T16:04:03.985 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.983+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~AGLQA6BZX9K3.0 size=1048576 2024-05-30T16:04:03.988 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.987+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~AGLQA6BZX9K3.1 size=1048576 2024-05-30T16:04:03.991 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.991+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~AGLQA6BZX9K3.2 size=298481 2024-05-30T16:04:03.991 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.991+0000 7f08851138c0 1 > prepared multipart upload key=E2FXSXOEUGIG upload=AGLQA6BZX9K3 tag=OSX2FJBSR0GJ type=write size=3444209 2024-05-30T16:04:03.994 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:03.995+0000 7f08851138c0 1 < completed delete operation key=4N58ZY4E1H6E tag=CX3CY105KJ2V type=del 2024-05-30T16:04:04.005 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.007+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~AZKXTGHTN5OW.0 size=1048576 2024-05-30T16:04:04.008 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.007+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~AZKXTGHTN5OW.1 size=247726 2024-05-30T16:04:04.008 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.007+0000 7f08851138c0 1 > prepared multipart upload key=3IJWPWRB6FHE upload=AZKXTGHTN5OW tag=GK7GWRBPLHD3 type=write size=2344878 2024-05-30T16:04:04.012 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.011+0000 7f08851138c0 1 < completed delete operation key=Y3CYZJ83WYNI tag=NY17KTR1KC66 type=del 2024-05-30T16:04:04.015 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.015+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=M4ZIY9ZMJ0VC type=del size=366962 2024-05-30T16:04:04.018 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.019+0000 7f08851138c0 1 < completed delete operation key=364U3OV10VPD tag=6IGPICQSEA2R type=del 2024-05-30T16:04:04.022 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.023+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=YDMQA7VE25YZ type=del size=1389527 2024-05-30T16:04:04.026 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.027+0000 7f08851138c0 1 < canceled operation key=QQILXRJD0C1Z tag=MLUF8YBC1T98 type=write size=735972 2024-05-30T16:04:04.032 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.031+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~R6REF83ES7BZ.0 size=217027 2024-05-30T16:04:04.032 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.031+0000 7f08851138c0 1 > prepared multipart upload key=QCLY10D2A3VF upload=R6REF83ES7BZ tag=CCJMJM7EB3CF type=write size=1265603 2024-05-30T16:04:04.036 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.035+0000 7f08851138c0 1 < completed delete operation key=JNZI7JX95LX2 tag=U35XVDRPE901 type=del 2024-05-30T16:04:04.040 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.039+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=UONRM9O5NVZ3 type=del size=2390317 2024-05-30T16:04:04.043 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.043+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~BAMZ9OLP0M2H.0 size=486642 2024-05-30T16:04:04.048 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.047+0000 7f08851138c0 1 < completed multipart upload key=OKBB0VU765TW upload=BAMZ9OLP0M2H tag=NZVVREZLAHBI type=write size=1535218 2024-05-30T16:04:04.052 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.051+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=P80B1ZSML9D6 type=del size=4094536 2024-05-30T16:04:04.055 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.055+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~Q8U1W7R3P31Z.0 size=1037898 2024-05-30T16:04:04.061 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.059+0000 7f08851138c0 1 < completed multipart upload key=RCTV94WRMO53 upload=Q8U1W7R3P31Z tag=F2KC59YZTEIV type=write size=2086474 2024-05-30T16:04:04.067 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.067+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~ZJJLMXM8Y8Q6.0 size=867680 2024-05-30T16:04:04.067 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.067+0000 7f08851138c0 1 > prepared multipart upload key=JNZI7JX95LX2 upload=ZJJLMXM8Y8Q6 tag=F1XXSMAO4FMD type=write size=1916256 2024-05-30T16:04:04.070 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.071+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~I65TLX3UM17B.0 size=997423 2024-05-30T16:04:04.076 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.075+0000 7f08851138c0 1 < completed multipart upload key=RCTV94WRMO53 upload=I65TLX3UM17B tag=1NHX7K6ODR3G type=write size=2045999 2024-05-30T16:04:04.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.079+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=081SYD39UKVT type=del size=2082774 2024-05-30T16:04:04.084 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.083+0000 7f08851138c0 1 < completed write operation key=3IJWPWRB6FHE tag=J4XQFKLDWAZ8 type=write size=19157 2024-05-30T16:04:04.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.091+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~8PK3Y0DPGNSL.0 size=267405 2024-05-30T16:04:04.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.091+0000 7f08851138c0 1 > prepared multipart upload key=Y8SBI0QZ0ZV8 upload=8PK3Y0DPGNSL tag=9UC7S27NWCK0 type=write size=1315981 2024-05-30T16:04:04.093 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.095+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~RFSNA7VGW43C.0 size=1048576 2024-05-30T16:04:04.096 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.095+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~RFSNA7VGW43C.1 size=1048576 2024-05-30T16:04:04.099 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.099+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~RFSNA7VGW43C.2 size=258863 2024-05-30T16:04:04.104 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.103+0000 7f08851138c0 1 < completed multipart upload key=YRP4CSG92QRO upload=RFSNA7VGW43C tag=RX1CV2CCJSKI type=write size=3404591 2024-05-30T16:04:04.108 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.107+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=RDNRGJCY5FZE type=write size=453011 2024-05-30T16:04:04.111 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.111+0000 7f08851138c0 1 < completed delete operation key=3TBOZWT2P8U8 tag=A1I55UCRSD1P type=del 2024-05-30T16:04:04.115 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.115+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=16QDLTF9XF38 type=write size=611575 2024-05-30T16:04:04.119 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.119+0000 7f08851138c0 1 < completed delete operation key=YRP4CSG92QRO tag=P1N849D64Z16 type=del 2024-05-30T16:04:04.122 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.123+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=YBE5KD130V2M type=write size=315305 2024-05-30T16:04:04.126 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.127+0000 7f08851138c0 1 < completed delete operation key=GWXNCSJCLXKW tag=FK1D1XM2SNUG type=del 2024-05-30T16:04:04.129 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.131+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=JSO3I1L18BEO type=del size=2697446 2024-05-30T16:04:04.134 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.135+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~HTHJKY9YGRNJ.0 size=1048576 2024-05-30T16:04:04.136 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.135+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~HTHJKY9YGRNJ.1 size=991044 2024-05-30T16:04:04.143 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.143+0000 7f08851138c0 1 < completed multipart upload key=RCTV94WRMO53 upload=HTHJKY9YGRNJ tag=CQTWS1YWNBQH type=write size=3088196 2024-05-30T16:04:04.151 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.151+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~3MA9WK34ZT4G.0 size=329553 2024-05-30T16:04:04.151 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.151+0000 7f08851138c0 1 > prepared multipart upload key=VRXNO8JLAZYH upload=3MA9WK34ZT4G tag=PUMDBAYVEJ8R type=write size=1378129 2024-05-30T16:04:04.155 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.155+0000 7f08851138c0 1 < completed delete operation key=0T2E9F3K1USF tag=DA4UIOBFFBVK type=del 2024-05-30T16:04:04.159 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.159+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=U9CQBHVR3EAC type=write size=470828 2024-05-30T16:04:04.162 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.163+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~AGLQA6BZX9K3.0 size=1048576 2024-05-30T16:04:04.165 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.163+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~AGLQA6BZX9K3.1 size=1048576 2024-05-30T16:04:04.167 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.167+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~AGLQA6BZX9K3.2 size=298481 2024-05-30T16:04:04.173 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.175+0000 7f08851138c0 1 < completed multipart upload key=E2FXSXOEUGIG upload=AGLQA6BZX9K3 tag=OSX2FJBSR0GJ type=write size=3444209 2024-05-30T16:04:04.180 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.179+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~80AKQ0MP1CD2.0 size=1048576 2024-05-30T16:04:04.182 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.183+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~80AKQ0MP1CD2.1 size=1048576 2024-05-30T16:04:04.185 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.183+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~80AKQ0MP1CD2.2 size=920373 2024-05-30T16:04:04.185 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.183+0000 7f08851138c0 1 > prepared multipart upload key=QQILXRJD0C1Z upload=80AKQ0MP1CD2 tag=2H77QO9MSIMH type=write size=4066101 2024-05-30T16:04:04.188 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.187+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~AZKXTGHTN5OW.0 size=1048576 2024-05-30T16:04:04.191 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.191+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~AZKXTGHTN5OW.1 size=247726 2024-05-30T16:04:04.197 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.195+0000 7f08851138c0 1 < completed multipart upload key=3IJWPWRB6FHE upload=AZKXTGHTN5OW tag=GK7GWRBPLHD3 type=write size=2344878 2024-05-30T16:04:04.200 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.199+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=LUJA30IQ9ZCF type=del size=2692360 2024-05-30T16:04:04.204 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.203+0000 7f08851138c0 1 < completed delete operation key=JNZI7JX95LX2 tag=M4ZIY9ZMJ0VC type=del 2024-05-30T16:04:04.210 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.211+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~21YMGMPNWA7W.0 size=1048576 2024-05-30T16:04:04.213 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.215+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~21YMGMPNWA7W.1 size=797267 2024-05-30T16:04:04.214 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.215+0000 7f08851138c0 1 > prepared multipart upload key=Y3CYZJ83WYNI upload=21YMGMPNWA7W tag=7M124RIAWV5S type=write size=2894419 2024-05-30T16:04:04.217 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.219+0000 7f08851138c0 1 < completed delete operation key=WT9RC0UYVG4F tag=YDMQA7VE25YZ type=del 2024-05-30T16:04:04.224 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.223+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~2U4ZK5IR8DDF.0 size=1048576 2024-05-30T16:04:04.226 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.227+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~2U4ZK5IR8DDF.1 size=1048576 2024-05-30T16:04:04.229 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.227+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~2U4ZK5IR8DDF.2 size=920225 2024-05-30T16:04:04.229 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.227+0000 7f08851138c0 1 > prepared multipart upload key=0RZRBW38WKNZ upload=2U4ZK5IR8DDF tag=ZXZ65TOW9QE1 type=write size=4065953 2024-05-30T16:04:04.232 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.231+0000 7f08851138c0 1 < completed part key=_multipart_QCLY10D2A3VF.2~R6REF83ES7BZ.0 size=217027 2024-05-30T16:04:04.238 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.239+0000 7f08851138c0 1 < completed multipart upload key=QCLY10D2A3VF upload=R6REF83ES7BZ tag=CCJMJM7EB3CF type=write size=1265603 2024-05-30T16:04:04.242 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.243+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=H4U6JMWSFL30 type=del size=1291481 2024-05-30T16:04:04.245 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.243+0000 7f08851138c0 1 < completed delete operation key=WT9RC0UYVG4F tag=UONRM9O5NVZ3 type=del 2024-05-30T16:04:04.248 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.247+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=NSBW7SN7K7TV type=write size=133288 2024-05-30T16:04:04.252 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.251+0000 7f08851138c0 1 < canceled operation key=NIV7AQ4EV5YP tag=P80B1ZSML9D6 type=del size=4094536 2024-05-30T16:04:04.255 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.255+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=LOQ0KUEZ4T6U type=del size=2736563 2024-05-30T16:04:04.259 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.259+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~ZJJLMXM8Y8Q6.0 size=867680 2024-05-30T16:04:04.264 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.263+0000 7f08851138c0 1 < completed multipart upload key=JNZI7JX95LX2 upload=ZJJLMXM8Y8Q6 tag=F1XXSMAO4FMD type=write size=1916256 2024-05-30T16:04:04.268 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.267+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=W1C9PZZY33OP type=del size=1846793 2024-05-30T16:04:04.272 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.271+0000 7f08851138c0 1 < completed delete operation key=0RZRBW38WKNZ tag=081SYD39UKVT type=del 2024-05-30T16:04:04.275 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.275+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=2IP9J333V3FH type=del size=446561 2024-05-30T16:04:04.279 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.279+0000 7f08851138c0 1 < completed part key=_multipart_Y8SBI0QZ0ZV8.2~8PK3Y0DPGNSL.0 size=267405 2024-05-30T16:04:04.285 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.283+0000 7f08851138c0 1 < completed multipart upload key=Y8SBI0QZ0ZV8 upload=8PK3Y0DPGNSL tag=9UC7S27NWCK0 type=write size=1315981 2024-05-30T16:04:04.289 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.287+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=IHOEWPS2OJ16 type=del size=2098663 2024-05-30T16:04:04.297 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.295+0000 7f08851138c0 1 < suggested update operation key=TPCETCO3Y1M6 tag=RDNRGJCY5FZE type=write size=453011 2024-05-30T16:04:04.304 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.303+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~Z6MZX05GWAOC.0 size=716639 2024-05-30T16:04:04.304 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.303+0000 7f08851138c0 1 > prepared multipart upload key=QQILXRJD0C1Z upload=Z6MZX05GWAOC tag=7RQKQNITY2VA type=write size=1765215 2024-05-30T16:04:04.308 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.307+0000 7f08851138c0 1 < completed write operation key=RCTV94WRMO53 tag=16QDLTF9XF38 type=write size=611575 2024-05-30T16:04:04.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.315+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~LVYRZ15K1HJI.0 size=1048576 2024-05-30T16:04:04.318 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.319+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~LVYRZ15K1HJI.1 size=769885 2024-05-30T16:04:04.318 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.319+0000 7f08851138c0 1 > prepared multipart upload key=0RZRBW38WKNZ upload=LVYRZ15K1HJI tag=9UZJ2KB59A83 type=write size=2867037 2024-05-30T16:04:04.321 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.323+0000 7f08851138c0 1 < canceled operation key=0T2E9F3K1USF tag=YBE5KD130V2M type=write size=315305 2024-05-30T16:04:04.325 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.323+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=H604S0L0WGOU type=del size=3688634 2024-05-30T16:04:04.329 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.327+0000 7f08851138c0 1 < completed delete operation key=YRP4CSG92QRO tag=JSO3I1L18BEO type=del 2024-05-30T16:04:04.339 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.339+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~30NIA4M30G6Q.0 size=1048576 2024-05-30T16:04:04.342 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.343+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~30NIA4M30G6Q.1 size=1048576 2024-05-30T16:04:04.345 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.343+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~30NIA4M30G6Q.2 size=243701 2024-05-30T16:04:04.345 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.343+0000 7f08851138c0 1 > prepared multipart upload key=O8CWPJ9GW2GO upload=30NIA4M30G6Q tag=590VDQ5F8VPF type=write size=3389429 2024-05-30T16:04:04.349 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.347+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~3MA9WK34ZT4G.0 size=329553 2024-05-30T16:04:04.355 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.355+0000 7f08851138c0 1 < completed multipart upload key=VRXNO8JLAZYH upload=3MA9WK34ZT4G tag=PUMDBAYVEJ8R type=write size=1378129 2024-05-30T16:04:04.361 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.359+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~LRJOGTQ4KUZ4.0 size=985161 2024-05-30T16:04:04.361 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.359+0000 7f08851138c0 1 > prepared multipart upload key=1N6R2HBTWMP2 upload=LRJOGTQ4KUZ4 tag=HQ0ZKQO9U2N7 type=write size=2033737 2024-05-30T16:04:04.364 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.363+0000 7f08851138c0 1 < completed write operation key=3TBOZWT2P8U8 tag=U9CQBHVR3EAC type=write size=470828 2024-05-30T16:04:04.371 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.371+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~TBRT248B13GP.0 size=1048576 2024-05-30T16:04:04.374 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.375+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~TBRT248B13GP.1 size=1048576 2024-05-30T16:04:04.377 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.375+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~TBRT248B13GP.2 size=1022370 2024-05-30T16:04:04.377 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.375+0000 7f08851138c0 1 > prepared multipart upload key=WT9RC0UYVG4F upload=TBRT248B13GP tag=ZOD9GG285OJY type=write size=4168098 2024-05-30T16:04:04.382 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.383+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~80AKQ0MP1CD2.0 size=1048576 2024-05-30T16:04:04.385 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.383+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~80AKQ0MP1CD2.1 size=1048576 2024-05-30T16:04:04.387 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.387+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~80AKQ0MP1CD2.2 size=920373 2024-05-30T16:04:04.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.395+0000 7f08851138c0 1 < completed multipart upload key=QQILXRJD0C1Z upload=80AKQ0MP1CD2 tag=2H77QO9MSIMH type=write size=4066101 2024-05-30T16:04:04.400 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.399+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~IBYKYPLJ2P4X.0 size=1048576 2024-05-30T16:04:04.403 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.403+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~IBYKYPLJ2P4X.1 size=1048576 2024-05-30T16:04:04.405 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.407+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~IBYKYPLJ2P4X.2 size=485714 2024-05-30T16:04:04.405 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.407+0000 7f08851138c0 1 > prepared multipart upload key=3TBOZWT2P8U8 upload=IBYKYPLJ2P4X tag=LTNG6QO4N4X7 type=write size=3631442 2024-05-30T16:04:04.412 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.411+0000 7f08851138c0 1 < suggested update operation key=RCTV94WRMO53 tag=LUJA30IQ9ZCF type=del size=2692360 2024-05-30T16:04:04.416 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.415+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=DY9JVIBFZAI7 type=write size=571404 2024-05-30T16:04:04.419 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.419+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~21YMGMPNWA7W.0 size=1048576 2024-05-30T16:04:04.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.423+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~21YMGMPNWA7W.1 size=797267 2024-05-30T16:04:04.429 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.431+0000 7f08851138c0 1 < completed multipart upload key=Y3CYZJ83WYNI upload=21YMGMPNWA7W tag=7M124RIAWV5S type=write size=2894419 2024-05-30T16:04:04.434 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.435+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=ES26UHWH9IYD type=write size=360291 2024-05-30T16:04:04.438 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.439+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~2U4ZK5IR8DDF.0 size=1048576 2024-05-30T16:04:04.442 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.443+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~2U4ZK5IR8DDF.1 size=1048576 2024-05-30T16:04:04.445 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.443+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~2U4ZK5IR8DDF.2 size=920225 2024-05-30T16:04:04.451 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.451+0000 7f08851138c0 1 < completed multipart upload key=0RZRBW38WKNZ upload=2U4ZK5IR8DDF tag=ZXZ65TOW9QE1 type=write size=4065953 2024-05-30T16:04:04.459 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.459+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~XQQMX58PH63B.0 size=1048576 2024-05-30T16:04:04.462 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.463+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~XQQMX58PH63B.1 size=1048576 2024-05-30T16:04:04.464 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.463+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~XQQMX58PH63B.2 size=297258 2024-05-30T16:04:04.465 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.463+0000 7f08851138c0 1 > prepared multipart upload key=XHGMUYBA2VNK upload=XQQMX58PH63B tag=9SWFDDYD8OUT type=write size=3442986 2024-05-30T16:04:04.468 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.467+0000 7f08851138c0 1 < completed delete operation key=MMLF65KLC42Q tag=H4U6JMWSFL30 type=del 2024-05-30T16:04:04.472 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.471+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=GH32KD0UC52A type=del size=2212086 2024-05-30T16:04:04.476 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.475+0000 7f08851138c0 1 < completed write operation key=3IJWPWRB6FHE tag=NSBW7SN7K7TV type=write size=133288 2024-05-30T16:04:04.482 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.483+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~YZY3LPH1CNHZ.0 size=322930 2024-05-30T16:04:04.482 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.483+0000 7f08851138c0 1 > prepared multipart upload key=4N58ZY4E1H6E upload=YZY3LPH1CNHZ tag=P03VUCW3ODC7 type=write size=1371506 2024-05-30T16:04:04.486 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.487+0000 7f08851138c0 1 < canceled operation key=4N58ZY4E1H6E tag=LOQ0KUEZ4T6U type=del size=2736563 2024-05-30T16:04:04.490 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.491+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=G6GM1Q549JI2 type=write size=677337 2024-05-30T16:04:04.493 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.491+0000 7f08851138c0 1 < completed delete operation key=GWXNCSJCLXKW tag=W1C9PZZY33OP type=del 2024-05-30T16:04:04.496 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.495+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=GUAJCRHO81L7 type=write size=174930 2024-05-30T16:04:04.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.499+0000 7f08851138c0 1 < completed delete operation key=E2FXSXOEUGIG tag=2IP9J333V3FH type=del 2024-05-30T16:04:04.503 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.503+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=66LU3EKMHO9V type=write size=457059 2024-05-30T16:04:04.507 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.507+0000 7f08851138c0 1 < completed delete operation key=0T2E9F3K1USF tag=IHOEWPS2OJ16 type=del 2024-05-30T16:04:04.511 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.511+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=7VXI1N3S797C type=write size=620964 2024-05-30T16:04:04.515 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.515+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~Z6MZX05GWAOC.0 size=716639 2024-05-30T16:04:04.521 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.519+0000 7f08851138c0 1 < completed multipart upload key=QQILXRJD0C1Z upload=Z6MZX05GWAOC tag=7RQKQNITY2VA type=write size=1765215 2024-05-30T16:04:04.524 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.523+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=8LAOUV6WSNX9 type=del size=286577 2024-05-30T16:04:04.525 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.523+0000 7f08851138c0 1 < canceled part key=_multipart_0RZRBW38WKNZ.2~LVYRZ15K1HJI.0 size=1048576 2024-05-30T16:04:04.528 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.527+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~LVYRZ15K1HJI.1 size=769885 2024-05-30T16:04:04.533 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.535+0000 7f08851138c0 1 < canceled multipart upload key=0RZRBW38WKNZ upload=LVYRZ15K1HJI tag=9UZJ2KB59A83 type=write size=2867037 2024-05-30T16:04:04.537 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.535+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=IL1T5ZN6PH22 type=del size=302645 2024-05-30T16:04:04.540 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.539+0000 7f08851138c0 -1 < failed to complete operation key=RCTV94WRMO53 tag=H604S0L0WGOU type=del size=3688634: (22) Invalid argument 2024-05-30T16:04:04.543 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.543+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=SNXX2T43XS59 type=write size=873948 2024-05-30T16:04:04.547 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.547+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~30NIA4M30G6Q.0 size=1048576 2024-05-30T16:04:04.550 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.551+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~30NIA4M30G6Q.1 size=1048576 2024-05-30T16:04:04.550 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.551+0000 7f08851138c0 1 < canceled part key=_multipart_O8CWPJ9GW2GO.2~30NIA4M30G6Q.2 size=243701 2024-05-30T16:04:04.556 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.555+0000 7f08851138c0 1 < canceled multipart upload key=O8CWPJ9GW2GO upload=30NIA4M30G6Q tag=590VDQ5F8VPF type=write size=3389429 2024-05-30T16:04:04.560 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.559+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=VNJ4BO8IQMU3 type=del size=3862318 2024-05-30T16:04:04.564 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.563+0000 7f08851138c0 1 < completed part key=_multipart_1N6R2HBTWMP2.2~LRJOGTQ4KUZ4.0 size=985161 2024-05-30T16:04:04.569 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.571+0000 7f08851138c0 1 < completed multipart upload key=1N6R2HBTWMP2 upload=LRJOGTQ4KUZ4 tag=HQ0ZKQO9U2N7 type=write size=2033737 2024-05-30T16:04:04.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.571+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=4TL2Y9HP0YLV type=del size=1994768 2024-05-30T16:04:04.577 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.575+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~TBRT248B13GP.0 size=1048576 2024-05-30T16:04:04.579 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.579+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~TBRT248B13GP.1 size=1048576 2024-05-30T16:04:04.582 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.583+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~TBRT248B13GP.2 size=1022370 2024-05-30T16:04:04.588 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.587+0000 7f08851138c0 1 < completed multipart upload key=WT9RC0UYVG4F upload=TBRT248B13GP tag=ZOD9GG285OJY type=write size=4168098 2024-05-30T16:04:04.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.595+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~N6AV3ND6LJTX.0 size=1048576 2024-05-30T16:04:04.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.599+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~N6AV3ND6LJTX.1 size=1048576 2024-05-30T16:04:04.600 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.599+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~N6AV3ND6LJTX.2 size=51812 2024-05-30T16:04:04.600 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.599+0000 7f08851138c0 1 > prepared multipart upload key=ZALVY70ISLM5 upload=N6AV3ND6LJTX tag=A51WWGSG98TO type=write size=3197540 2024-05-30T16:04:04.604 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.603+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~IBYKYPLJ2P4X.0 size=1048576 2024-05-30T16:04:04.606 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.607+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~IBYKYPLJ2P4X.1 size=1048576 2024-05-30T16:04:04.609 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.607+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~IBYKYPLJ2P4X.2 size=485714 2024-05-30T16:04:04.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.615+0000 7f08851138c0 1 < completed multipart upload key=3TBOZWT2P8U8 upload=IBYKYPLJ2P4X tag=LTNG6QO4N4X7 type=write size=3631442 2024-05-30T16:04:04.621 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.619+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~C17177JE0Q55.0 size=1048576 2024-05-30T16:04:04.624 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.623+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~C17177JE0Q55.1 size=556471 2024-05-30T16:04:04.624 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.623+0000 7f08851138c0 1 > prepared multipart upload key=3TBOZWT2P8U8 upload=C17177JE0Q55 tag=9LU4RDE7ULKM type=write size=2653623 2024-05-30T16:04:04.627 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.627+0000 7f08851138c0 1 < completed write operation key=V4OQFOOQUXFQ tag=DY9JVIBFZAI7 type=write size=571404 2024-05-30T16:04:04.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.635+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~27M8XI8FCZDJ.0 size=1048576 2024-05-30T16:04:04.636 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.635+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~27M8XI8FCZDJ.1 size=454720 2024-05-30T16:04:04.637 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.635+0000 7f08851138c0 1 > prepared multipart upload key=TPCETCO3Y1M6 upload=27M8XI8FCZDJ tag=IRZ4LYH3UYNR type=write size=2551872 2024-05-30T16:04:04.640 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.639+0000 7f08851138c0 1 < completed write operation key=Y3CYZJ83WYNI tag=ES26UHWH9IYD type=write size=360291 2024-05-30T16:04:04.647 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.647+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~YEW1EQ1W3XYT.0 size=1048576 2024-05-30T16:04:04.649 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.651+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~YEW1EQ1W3XYT.1 size=1048576 2024-05-30T16:04:04.653 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.651+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~YEW1EQ1W3XYT.2 size=476601 2024-05-30T16:04:04.653 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.651+0000 7f08851138c0 1 > prepared multipart upload key=0RZRBW38WKNZ upload=YEW1EQ1W3XYT tag=CHP9TIXZ3RXV type=write size=3622329 2024-05-30T16:04:04.656 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.655+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~XQQMX58PH63B.0 size=1048576 2024-05-30T16:04:04.659 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.659+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~XQQMX58PH63B.1 size=1048576 2024-05-30T16:04:04.666 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.667+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~XQQMX58PH63B.2 size=297258 2024-05-30T16:04:04.673 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.671+0000 7f08851138c0 1 < completed multipart upload key=XHGMUYBA2VNK upload=XQQMX58PH63B tag=9SWFDDYD8OUT type=write size=3442986 2024-05-30T16:04:04.680 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.679+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~2XY26ZSSAAA1.0 size=1048576 2024-05-30T16:04:04.682 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.683+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~2XY26ZSSAAA1.1 size=58799 2024-05-30T16:04:04.683 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.683+0000 7f08851138c0 1 > prepared multipart upload key=NIV7AQ4EV5YP upload=2XY26ZSSAAA1 tag=3BGK7J9I21QL type=write size=2155951 2024-05-30T16:04:04.690 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.691+0000 7f08851138c0 1 < suggested remove operation key=603HBR8PRH6L tag=GH32KD0UC52A type=del 2024-05-30T16:04:04.697 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.699+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~HBNVMHQMM2IP.0 size=1048576 2024-05-30T16:04:04.700 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.699+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~HBNVMHQMM2IP.1 size=1048576 2024-05-30T16:04:04.703 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.703+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~HBNVMHQMM2IP.2 size=582611 2024-05-30T16:04:04.703 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.703+0000 7f08851138c0 1 > prepared multipart upload key=4N58ZY4E1H6E upload=HBNVMHQMM2IP tag=BJDFS56N6VGW type=write size=3728339 2024-05-30T16:04:04.708 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.707+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~YZY3LPH1CNHZ.0 size=322930 2024-05-30T16:04:04.714 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.715+0000 7f08851138c0 1 < completed multipart upload key=4N58ZY4E1H6E upload=YZY3LPH1CNHZ tag=P03VUCW3ODC7 type=write size=1371506 2024-05-30T16:04:04.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.719+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=GL15MLGH8X5B type=del size=1753379 2024-05-30T16:04:04.725 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.723+0000 7f08851138c0 1 < suggested remove operation key=O8CWPJ9GW2GO tag=G6GM1Q549JI2 type=write 2024-05-30T16:04:04.728 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.727+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=WY3YVWIWL1VM type=write size=982411 2024-05-30T16:04:04.732 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.731+0000 7f08851138c0 1 < completed write operation key=A5BUBY9Z7JQG tag=GUAJCRHO81L7 type=write size=174930 2024-05-30T16:04:04.739 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.739+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~1IU6K1K3UC7N.0 size=1048576 2024-05-30T16:04:04.742 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.743+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~1IU6K1K3UC7N.1 size=1048576 2024-05-30T16:04:04.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.743+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~1IU6K1K3UC7N.2 size=943306 2024-05-30T16:04:04.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.743+0000 7f08851138c0 1 > prepared multipart upload key=RCTV94WRMO53 upload=1IU6K1K3UC7N tag=FEOC8RY6KSNX type=write size=4089034 2024-05-30T16:04:04.752 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.751+0000 7f08851138c0 1 < suggested update operation key=V4OQFOOQUXFQ tag=66LU3EKMHO9V type=write size=457059 2024-05-30T16:04:04.758 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.759+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~2EQNQGGGEH0X.0 size=1048576 2024-05-30T16:04:04.761 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.759+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~2EQNQGGGEH0X.1 size=496687 2024-05-30T16:04:04.761 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.759+0000 7f08851138c0 1 > prepared multipart upload key=RCTV94WRMO53 upload=2EQNQGGGEH0X tag=D4KKPN03P58M type=write size=2593839 2024-05-30T16:04:04.764 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.763+0000 7f08851138c0 1 < completed write operation key=TPCETCO3Y1M6 tag=7VXI1N3S797C type=write size=620964 2024-05-30T16:04:04.770 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.771+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~HFUN22F99QX1.0 size=413363 2024-05-30T16:04:04.771 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.771+0000 7f08851138c0 1 > prepared multipart upload key=GWXNCSJCLXKW upload=HFUN22F99QX1 tag=4HP6HHBM9ZPZ type=write size=1461939 2024-05-30T16:04:04.775 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.775+0000 7f08851138c0 -1 < failed to complete operation key=V4OQFOOQUXFQ tag=8LAOUV6WSNX9 type=del size=286577: (22) Invalid argument 2024-05-30T16:04:04.781 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.779+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~79LF86E4WNLZ.0 size=1048576 2024-05-30T16:04:04.784 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.783+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~79LF86E4WNLZ.1 size=284360 2024-05-30T16:04:04.784 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.783+0000 7f08851138c0 1 > prepared multipart upload key=Y3CYZJ83WYNI upload=79LF86E4WNLZ tag=9T77WA5XV1JS type=write size=2381512 2024-05-30T16:04:04.788 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.787+0000 7f08851138c0 1 < completed delete operation key=GWXNCSJCLXKW tag=IL1T5ZN6PH22 type=del 2024-05-30T16:04:04.793 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.795+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~T59ZV9ZJVLXA.0 size=345132 2024-05-30T16:04:04.794 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.795+0000 7f08851138c0 1 > prepared multipart upload key=73FJ3HGWYQQK upload=T59ZV9ZJVLXA tag=5LEKVESOHAQP type=write size=1393708 2024-05-30T16:04:04.797 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.799+0000 7f08851138c0 1 < completed write operation key=NIV7AQ4EV5YP tag=SNXX2T43XS59 type=write size=873948 2024-05-30T16:04:04.801 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.799+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=DHB5495HRK7O type=del size=3285797 2024-05-30T16:04:04.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.807+0000 7f08851138c0 1 < completed delete operation key=A5BUBY9Z7JQG tag=VNJ4BO8IQMU3 type=del 2024-05-30T16:04:04.808 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.807+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=L70HQ2UAXG6F type=del size=1263877 2024-05-30T16:04:04.812 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.811+0000 7f08851138c0 1 < completed delete operation key=0T2E9F3K1USF tag=4TL2Y9HP0YLV type=del 2024-05-30T16:04:04.816 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.815+0000 7f08851138c0 1 > prepared operation key=VRXNO8JLAZYH tag=882HEPBDAL3N type=del size=2120723 2024-05-30T16:04:04.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.819+0000 7f08851138c0 1 < completed part key=_multipart_ZALVY70ISLM5.2~N6AV3ND6LJTX.0 size=1048576 2024-05-30T16:04:04.823 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.823+0000 7f08851138c0 1 < completed part key=_multipart_ZALVY70ISLM5.2~N6AV3ND6LJTX.1 size=1048576 2024-05-30T16:04:04.826 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.827+0000 7f08851138c0 1 < completed part key=_multipart_ZALVY70ISLM5.2~N6AV3ND6LJTX.2 size=51812 2024-05-30T16:04:04.831 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.831+0000 7f08851138c0 1 < completed multipart upload key=ZALVY70ISLM5 upload=N6AV3ND6LJTX tag=A51WWGSG98TO type=write size=3197540 2024-05-30T16:04:04.837 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.839+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~MJTIR5TZE0Q2.0 size=1048576 2024-05-30T16:04:04.840 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.839+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~MJTIR5TZE0Q2.1 size=387768 2024-05-30T16:04:04.840 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.839+0000 7f08851138c0 1 > prepared multipart upload key=JNZI7JX95LX2 upload=MJTIR5TZE0Q2 tag=PIH0LKI2D8JF type=write size=2484920 2024-05-30T16:04:04.844 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.843+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~C17177JE0Q55.0 size=1048576 2024-05-30T16:04:04.847 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.847+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~C17177JE0Q55.1 size=556471 2024-05-30T16:04:04.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.851+0000 7f08851138c0 1 < completed multipart upload key=3TBOZWT2P8U8 upload=C17177JE0Q55 tag=9LU4RDE7ULKM type=write size=2653623 2024-05-30T16:04:04.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.855+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=8K8QAS5AVT5S type=del size=2015235 2024-05-30T16:04:04.859 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.859+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~27M8XI8FCZDJ.0 size=1048576 2024-05-30T16:04:04.861 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.863+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~27M8XI8FCZDJ.1 size=454720 2024-05-30T16:04:04.867 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.867+0000 7f08851138c0 1 < completed multipart upload key=TPCETCO3Y1M6 upload=27M8XI8FCZDJ tag=IRZ4LYH3UYNR type=write size=2551872 2024-05-30T16:04:04.871 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.871+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=RHSMO3AUYJ7P type=del size=283672 2024-05-30T16:04:04.875 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.875+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~YEW1EQ1W3XYT.0 size=1048576 2024-05-30T16:04:04.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.879+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~YEW1EQ1W3XYT.1 size=1048576 2024-05-30T16:04:04.881 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.883+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~YEW1EQ1W3XYT.2 size=476601 2024-05-30T16:04:04.888 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.887+0000 7f08851138c0 1 < completed multipart upload key=0RZRBW38WKNZ upload=YEW1EQ1W3XYT tag=CHP9TIXZ3RXV type=write size=3622329 2024-05-30T16:04:04.891 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.891+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=B4SPRIC35IK4 type=del size=1701277 2024-05-30T16:04:04.895 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.895+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~2XY26ZSSAAA1.0 size=1048576 2024-05-30T16:04:04.898 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.899+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~2XY26ZSSAAA1.1 size=58799 2024-05-30T16:04:04.904 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.903+0000 7f08851138c0 1 < completed multipart upload key=NIV7AQ4EV5YP upload=2XY26ZSSAAA1 tag=3BGK7J9I21QL type=write size=2155951 2024-05-30T16:04:04.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.911+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~VRU7OJNL3P5F.0 size=1048576 2024-05-30T16:04:04.913 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.915+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~VRU7OJNL3P5F.1 size=1048576 2024-05-30T16:04:04.916 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.915+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~VRU7OJNL3P5F.2 size=746207 2024-05-30T16:04:04.917 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.915+0000 7f08851138c0 1 > prepared multipart upload key=A5BUBY9Z7JQG upload=VRU7OJNL3P5F tag=DAJQULCPNTX5 type=write size=3891935 2024-05-30T16:04:04.920 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.919+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~HBNVMHQMM2IP.0 size=1048576 2024-05-30T16:04:04.923 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.923+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~HBNVMHQMM2IP.1 size=1048576 2024-05-30T16:04:04.925 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.923+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~HBNVMHQMM2IP.2 size=582611 2024-05-30T16:04:04.930 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.931+0000 7f08851138c0 1 < completed multipart upload key=4N58ZY4E1H6E upload=HBNVMHQMM2IP tag=BJDFS56N6VGW type=write size=3728339 2024-05-30T16:04:04.937 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.935+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~S9G5DXJ9S5ED.0 size=1048576 2024-05-30T16:04:04.939 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.939+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~S9G5DXJ9S5ED.1 size=994458 2024-05-30T16:04:04.939 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.939+0000 7f08851138c0 1 > prepared multipart upload key=TPCETCO3Y1M6 upload=S9G5DXJ9S5ED tag=JHT0J1ASAN4Y type=write size=3091610 2024-05-30T16:04:04.946 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.947+0000 7f08851138c0 1 < suggested update operation key=603HBR8PRH6L tag=GL15MLGH8X5B type=del size=1753379 2024-05-30T16:04:04.949 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.951+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=73IR2YFBVSWM type=del size=1349680 2024-05-30T16:04:04.953 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.955+0000 7f08851138c0 1 < completed write operation key=RCTV94WRMO53 tag=WY3YVWIWL1VM type=write size=982411 2024-05-30T16:04:04.960 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.959+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~B51JW6FCA3MJ.0 size=938057 2024-05-30T16:04:04.960 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.959+0000 7f08851138c0 1 > prepared multipart upload key=WT9RC0UYVG4F upload=B51JW6FCA3MJ tag=11BJTR0509OI type=write size=1986633 2024-05-30T16:04:04.961 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.959+0000 7f08851138c0 1 < canceled part key=_multipart_RCTV94WRMO53.2~1IU6K1K3UC7N.0 size=1048576 2024-05-30T16:04:04.963 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.963+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~1IU6K1K3UC7N.1 size=1048576 2024-05-30T16:04:04.966 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.967+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~1IU6K1K3UC7N.2 size=943306 2024-05-30T16:04:04.972 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.971+0000 7f08851138c0 1 < canceled multipart upload key=RCTV94WRMO53 upload=1IU6K1K3UC7N tag=FEOC8RY6KSNX type=write size=4089034 2024-05-30T16:04:04.978 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.979+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~K204FWSZ6WWH.0 size=174060 2024-05-30T16:04:04.978 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.979+0000 7f08851138c0 1 > prepared multipart upload key=NIV7AQ4EV5YP upload=K204FWSZ6WWH tag=R2MUJ45FPT2C type=write size=1222636 2024-05-30T16:04:04.981 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.983+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~2EQNQGGGEH0X.0 size=1048576 2024-05-30T16:04:04.984 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.983+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~2EQNQGGGEH0X.1 size=496687 2024-05-30T16:04:04.994 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.995+0000 7f08851138c0 1 < completed multipart upload key=RCTV94WRMO53 upload=2EQNQGGGEH0X tag=D4KKPN03P58M type=write size=2593839 2024-05-30T16:04:04.998 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:04.999+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=RHIZBJRB9HGL type=del size=3665987 2024-05-30T16:04:05.001 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.003+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~HFUN22F99QX1.0 size=413363 2024-05-30T16:04:05.008 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.007+0000 7f08851138c0 1 < completed multipart upload key=GWXNCSJCLXKW upload=HFUN22F99QX1 tag=4HP6HHBM9ZPZ type=write size=1461939 2024-05-30T16:04:05.015 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.015+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~9MQX39FC09QG.0 size=359176 2024-05-30T16:04:05.015 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.015+0000 7f08851138c0 1 > prepared multipart upload key=XHGMUYBA2VNK upload=9MQX39FC09QG tag=06HNKX783HYY type=write size=1407752 2024-05-30T16:04:05.019 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.019+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~79LF86E4WNLZ.0 size=1048576 2024-05-30T16:04:05.022 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.023+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~79LF86E4WNLZ.1 size=284360 2024-05-30T16:04:05.028 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.027+0000 7f08851138c0 1 < completed multipart upload key=Y3CYZJ83WYNI upload=79LF86E4WNLZ tag=9T77WA5XV1JS type=write size=2381512 2024-05-30T16:04:05.033 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.035+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~XOJO9UU2THEQ.0 size=444458 2024-05-30T16:04:05.034 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.035+0000 7f08851138c0 1 > prepared multipart upload key=3TBOZWT2P8U8 upload=XOJO9UU2THEQ tag=UAX2MFNDQCRS type=write size=1493034 2024-05-30T16:04:05.034 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.035+0000 7f08851138c0 1 < canceled part key=_multipart_73FJ3HGWYQQK.2~T59ZV9ZJVLXA.0 size=345132 2024-05-30T16:04:05.039 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.039+0000 7f08851138c0 1 < canceled multipart upload key=73FJ3HGWYQQK upload=T59ZV9ZJVLXA tag=5LEKVESOHAQP type=write size=1393708 2024-05-30T16:04:05.043 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.043+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=IRSEDPPHNN2Z type=del size=3960698 2024-05-30T16:04:05.047 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.047+0000 7f08851138c0 1 < completed delete operation key=O8CWPJ9GW2GO tag=DHB5495HRK7O type=del 2024-05-30T16:04:05.050 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.051+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=GD0F18LVW83B type=write size=810377 2024-05-30T16:04:05.054 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.055+0000 7f08851138c0 1 < completed delete operation key=364U3OV10VPD tag=L70HQ2UAXG6F type=del 2024-05-30T16:04:05.057 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.055+0000 7f08851138c0 1 > prepared operation key=VRXNO8JLAZYH tag=TYZR0QZGE8PH type=del size=1766021 2024-05-30T16:04:05.061 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.059+0000 7f08851138c0 1 < completed delete operation key=VRXNO8JLAZYH tag=882HEPBDAL3N type=del 2024-05-30T16:04:05.065 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.063+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=9HQ51KPD0GCP type=del size=2948080 2024-05-30T16:04:05.068 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.067+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~MJTIR5TZE0Q2.0 size=1048576 2024-05-30T16:04:05.071 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.071+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~MJTIR5TZE0Q2.1 size=387768 2024-05-30T16:04:05.077 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.075+0000 7f08851138c0 1 < completed multipart upload key=JNZI7JX95LX2 upload=MJTIR5TZE0Q2 tag=PIH0LKI2D8JF type=write size=2484920 2024-05-30T16:04:05.083 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.083+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~5Y2867E8MHF2.0 size=1048576 2024-05-30T16:04:05.086 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.087+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~5Y2867E8MHF2.1 size=1048576 2024-05-30T16:04:05.089 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.087+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~5Y2867E8MHF2.2 size=698035 2024-05-30T16:04:05.089 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.087+0000 7f08851138c0 1 > prepared multipart upload key=73FJ3HGWYQQK upload=5Y2867E8MHF2 tag=WV5HB85ZHORP type=write size=3843763 2024-05-30T16:04:05.096 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.095+0000 7f08851138c0 1 < suggested remove operation key=O8CWPJ9GW2GO tag=8K8QAS5AVT5S type=del 2024-05-30T16:04:05.099 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.099+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=IWDJ7TH0HB5E type=write size=568385 2024-05-30T16:04:05.106 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.107+0000 7f08851138c0 1 < suggested update operation key=0T2E9F3K1USF tag=RHSMO3AUYJ7P type=del size=283672 2024-05-30T16:04:05.110 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.111+0000 7f08851138c0 1 > prepared operation key=ZALVY70ISLM5 tag=5QOG3M7TKC9S type=write size=285646 2024-05-30T16:04:05.114 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.115+0000 7f08851138c0 1 < completed delete operation key=PLWFSKPKVO19 tag=B4SPRIC35IK4 type=del 2024-05-30T16:04:05.117 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.115+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=WDZ7EKHLHZVB type=del size=1648053 2024-05-30T16:04:05.118 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.119+0000 7f08851138c0 1 < canceled part key=_multipart_A5BUBY9Z7JQG.2~VRU7OJNL3P5F.0 size=1048576 2024-05-30T16:04:05.121 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.119+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~VRU7OJNL3P5F.1 size=1048576 2024-05-30T16:04:05.123 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.123+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~VRU7OJNL3P5F.2 size=746207 2024-05-30T16:04:05.129 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.131+0000 7f08851138c0 1 < canceled multipart upload key=A5BUBY9Z7JQG upload=VRU7OJNL3P5F tag=DAJQULCPNTX5 type=write size=3891935 2024-05-30T16:04:05.133 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.131+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=DZKR1UJWDQ28 type=write size=279721 2024-05-30T16:04:05.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.135+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~S9G5DXJ9S5ED.0 size=1048576 2024-05-30T16:04:05.140 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.139+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~S9G5DXJ9S5ED.1 size=994458 2024-05-30T16:04:05.147 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.147+0000 7f08851138c0 1 < completed multipart upload key=TPCETCO3Y1M6 upload=S9G5DXJ9S5ED tag=JHT0J1ASAN4Y type=write size=3091610 2024-05-30T16:04:05.150 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.151+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=MEHGRF802KNB type=del size=1315746 2024-05-30T16:04:05.154 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.155+0000 7f08851138c0 1 < canceled operation key=JNZI7JX95LX2 tag=73IR2YFBVSWM type=del size=1349680 2024-05-30T16:04:05.158 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.159+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=74QZ2GS74ZIG type=del size=1818864 2024-05-30T16:04:05.162 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.163+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~B51JW6FCA3MJ.0 size=938057 2024-05-30T16:04:05.168 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.167+0000 7f08851138c0 1 < completed multipart upload key=WT9RC0UYVG4F upload=B51JW6FCA3MJ tag=11BJTR0509OI type=write size=1986633 2024-05-30T16:04:05.172 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.171+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=XGGWNSK98EW0 type=del size=2702239 2024-05-30T16:04:05.176 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.175+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~K204FWSZ6WWH.0 size=174060 2024-05-30T16:04:05.182 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.183+0000 7f08851138c0 1 < completed multipart upload key=NIV7AQ4EV5YP upload=K204FWSZ6WWH tag=R2MUJ45FPT2C type=write size=1222636 2024-05-30T16:04:05.185 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.183+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=YN3CBQH5AI2X type=write size=723112 2024-05-30T16:04:05.188 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.187+0000 7f08851138c0 1 < completed delete operation key=73FJ3HGWYQQK tag=RHIZBJRB9HGL type=del 2024-05-30T16:04:05.192 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.191+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=N167U39DH7W2 type=del size=1289852 2024-05-30T16:04:05.196 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.195+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~9MQX39FC09QG.0 size=359176 2024-05-30T16:04:05.201 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.203+0000 7f08851138c0 1 < completed multipart upload key=XHGMUYBA2VNK upload=9MQX39FC09QG tag=06HNKX783HYY type=write size=1407752 2024-05-30T16:04:05.205 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.203+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=FZCDMOS0AF6K type=del size=1235638 2024-05-30T16:04:05.208 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.207+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~XOJO9UU2THEQ.0 size=444458 2024-05-30T16:04:05.214 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.215+0000 7f08851138c0 1 < completed multipart upload key=3TBOZWT2P8U8 upload=XOJO9UU2THEQ tag=UAX2MFNDQCRS type=write size=1493034 2024-05-30T16:04:05.218 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.219+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=SSPPN6VUNUIE type=write size=865795 2024-05-30T16:04:05.221 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.223+0000 7f08851138c0 1 < completed delete operation key=603HBR8PRH6L tag=IRSEDPPHNN2Z type=del 2024-05-30T16:04:05.225 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.223+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=0Q3OBYLIISXU type=del size=1014746 2024-05-30T16:04:05.232 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.231+0000 7f08851138c0 1 < suggested remove operation key=O0DJ8SCY766V tag=GD0F18LVW83B type=write 2024-05-30T16:04:05.235 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.235+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=I0D7Y0IETB00 type=del size=1005065 2024-05-30T16:04:05.239 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.239+0000 7f08851138c0 1 < completed delete operation key=VRXNO8JLAZYH tag=TYZR0QZGE8PH type=del 2024-05-30T16:04:05.243 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.243+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=HDSXMNJ71W84 type=del size=1322014 2024-05-30T16:04:05.247 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.247+0000 7f08851138c0 -1 < failed to complete operation key=O0DJ8SCY766V tag=9HQ51KPD0GCP type=del size=2948080: (22) Invalid argument 2024-05-30T16:04:05.250 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.251+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=OVBAK4ENYVW5 type=del size=1550923 2024-05-30T16:04:05.254 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.255+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~5Y2867E8MHF2.0 size=1048576 2024-05-30T16:04:05.256 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.255+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~5Y2867E8MHF2.1 size=1048576 2024-05-30T16:04:05.259 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.259+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~5Y2867E8MHF2.2 size=698035 2024-05-30T16:04:05.266 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.267+0000 7f08851138c0 1 < completed multipart upload key=73FJ3HGWYQQK upload=5Y2867E8MHF2 tag=WV5HB85ZHORP type=write size=3843763 2024-05-30T16:04:05.272 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.271+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~Q5K4TMZW99U5.0 size=1048576 2024-05-30T16:04:05.274 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.275+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~Q5K4TMZW99U5.1 size=1048576 2024-05-30T16:04:05.277 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.275+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~Q5K4TMZW99U5.2 size=837170 2024-05-30T16:04:05.277 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.275+0000 7f08851138c0 1 > prepared multipart upload key=603HBR8PRH6L upload=Q5K4TMZW99U5 tag=HVND4SD8K0JG type=write size=3982898 2024-05-30T16:04:05.280 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.279+0000 7f08851138c0 1 < completed write operation key=73FJ3HGWYQQK tag=IWDJ7TH0HB5E type=write size=568385 2024-05-30T16:04:05.287 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.287+0000 7f08851138c0 1 > prepared part key=_multipart_MMLF65KLC42Q.2~84CW4907ZM99.0 size=1048576 2024-05-30T16:04:05.289 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.291+0000 7f08851138c0 1 > prepared part key=_multipart_MMLF65KLC42Q.2~84CW4907ZM99.1 size=800622 2024-05-30T16:04:05.289 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.291+0000 7f08851138c0 1 > prepared multipart upload key=MMLF65KLC42Q upload=84CW4907ZM99 tag=8EOD3IJ7ANSL type=write size=2897774 2024-05-30T16:04:05.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.291+0000 7f08851138c0 1 < completed write operation key=ZALVY70ISLM5 tag=5QOG3M7TKC9S type=write size=285646 2024-05-30T16:04:05.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.299+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~05OEEZ0FL608.0 size=1048576 2024-05-30T16:04:05.302 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.303+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~05OEEZ0FL608.1 size=215321 2024-05-30T16:04:05.302 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.303+0000 7f08851138c0 1 > prepared multipart upload key=TPCETCO3Y1M6 upload=05OEEZ0FL608 tag=SCLIQP8Q2JLX type=write size=2312473 2024-05-30T16:04:05.309 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.307+0000 7f08851138c0 1 < suggested remove operation key=QCLY10D2A3VF tag=WDZ7EKHLHZVB type=del 2024-05-30T16:04:05.312 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.311+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=PVDPG3XXBQU2 type=write size=964912 2024-05-30T16:04:05.316 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.315+0000 7f08851138c0 1 < completed write operation key=RCTV94WRMO53 tag=DZKR1UJWDQ28 type=write size=279721 2024-05-30T16:04:05.319 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.319+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=6331PXL8BSNP type=del size=872822 2024-05-30T16:04:05.323 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.323+0000 7f08851138c0 1 < completed delete operation key=MMLF65KLC42Q tag=MEHGRF802KNB type=del 2024-05-30T16:04:05.329 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.331+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=TI3RLVJ3C0YK type=write size=913646 2024-05-30T16:04:05.333 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.331+0000 7f08851138c0 1 < completed delete operation key=603HBR8PRH6L tag=74QZ2GS74ZIG type=del 2024-05-30T16:04:05.336 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.335+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=1E91VTW5YD90 type=write size=74603 2024-05-30T16:04:05.340 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.339+0000 7f08851138c0 1 < completed delete operation key=YRP4CSG92QRO tag=XGGWNSK98EW0 type=del 2024-05-30T16:04:05.346 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.347+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~HVLXTPXH6J56.0 size=1048576 2024-05-30T16:04:05.349 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.347+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~HVLXTPXH6J56.1 size=13301 2024-05-30T16:04:05.349 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.347+0000 7f08851138c0 1 > prepared multipart upload key=NIV7AQ4EV5YP upload=HVLXTPXH6J56 tag=HYIZUEZMIMD8 type=write size=2110453 2024-05-30T16:04:05.356 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.355+0000 7f08851138c0 1 < suggested remove operation key=3IJWPWRB6FHE tag=YN3CBQH5AI2X type=write 2024-05-30T16:04:05.359 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.359+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=RJCOSDACCPVX type=del size=1318532 2024-05-30T16:04:05.366 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.367+0000 7f08851138c0 1 < suggested update operation key=Y3CYZJ83WYNI tag=N167U39DH7W2 type=del size=1289852 2024-05-30T16:04:05.369 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.371+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=Q6PIOFDS42X4 type=del size=1362884 2024-05-30T16:04:05.373 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.375+0000 7f08851138c0 1 < completed delete operation key=4N58ZY4E1H6E tag=FZCDMOS0AF6K type=del 2024-05-30T16:04:05.377 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.375+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=FKZYHHZBJQV4 type=del size=1531089 2024-05-30T16:04:05.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.383+0000 7f08851138c0 1 < suggested remove operation key=364U3OV10VPD tag=SSPPN6VUNUIE type=write 2024-05-30T16:04:05.388 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.387+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=WFEFN9AVDJ7A type=write size=485506 2024-05-30T16:04:05.391 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.391+0000 7f08851138c0 1 < completed delete operation key=RCTV94WRMO53 tag=0Q3OBYLIISXU type=del 2024-05-30T16:04:05.398 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.399+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~D86FF0LTWKN4.0 size=1048576 2024-05-30T16:04:05.401 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.399+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~D86FF0LTWKN4.1 size=679633 2024-05-30T16:04:05.401 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.399+0000 7f08851138c0 1 > prepared multipart upload key=XHGMUYBA2VNK upload=D86FF0LTWKN4 tag=KOLG0FO1W0ZJ type=write size=2776785 2024-05-30T16:04:05.409 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.411+0000 7f08851138c0 1 < suggested update operation key=NIV7AQ4EV5YP tag=I0D7Y0IETB00 type=del size=1005065 2024-05-30T16:04:05.416 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.415+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~KJ3PC8T1SQHL.0 size=1048576 2024-05-30T16:04:05.419 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.419+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~KJ3PC8T1SQHL.1 size=1048576 2024-05-30T16:04:05.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.423+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~KJ3PC8T1SQHL.2 size=44551 2024-05-30T16:04:05.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.423+0000 7f08851138c0 1 > prepared multipart upload key=VRXNO8JLAZYH upload=KJ3PC8T1SQHL tag=JKT7KZA6R074 type=write size=3190279 2024-05-30T16:04:05.426 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.427+0000 7f08851138c0 1 < completed delete operation key=3TBOZWT2P8U8 tag=HDSXMNJ71W84 type=del 2024-05-30T16:04:05.430 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.431+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=A1VG3OKRAQ37 type=del size=3836264 2024-05-30T16:04:05.437 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.439+0000 7f08851138c0 1 < suggested remove operation key=MMLF65KLC42Q tag=OVBAK4ENYVW5 type=del 2024-05-30T16:04:05.445 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.447+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~KQQPSB1D618L.0 size=1048576 2024-05-30T16:04:05.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.447+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~KQQPSB1D618L.1 size=984192 2024-05-30T16:04:05.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.447+0000 7f08851138c0 1 > prepared multipart upload key=1N6R2HBTWMP2 upload=KQQPSB1D618L tag=L4HZ7CQ5U26Z type=write size=3081344 2024-05-30T16:04:05.454 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.451+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~Q5K4TMZW99U5.0 size=1048576 2024-05-30T16:04:05.455 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.455+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~Q5K4TMZW99U5.1 size=1048576 2024-05-30T16:04:05.455 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.455+0000 7f08851138c0 1 < canceled part key=_multipart_603HBR8PRH6L.2~Q5K4TMZW99U5.2 size=837170 2024-05-30T16:04:05.462 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.463+0000 7f08851138c0 1 < canceled multipart upload key=603HBR8PRH6L upload=Q5K4TMZW99U5 tag=HVND4SD8K0JG type=write size=3982898 2024-05-30T16:04:05.468 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.467+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~BCD9CC1LAU25.0 size=1048576 2024-05-30T16:04:05.471 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.471+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~BCD9CC1LAU25.1 size=685166 2024-05-30T16:04:05.471 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.471+0000 7f08851138c0 1 > prepared multipart upload key=3TBOZWT2P8U8 upload=BCD9CC1LAU25 tag=A93UBQZ7FMSZ type=write size=2782318 2024-05-30T16:04:05.475 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.475+0000 7f08851138c0 1 < completed part key=_multipart_MMLF65KLC42Q.2~84CW4907ZM99.0 size=1048576 2024-05-30T16:04:05.478 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.479+0000 7f08851138c0 1 < completed part key=_multipart_MMLF65KLC42Q.2~84CW4907ZM99.1 size=800622 2024-05-30T16:04:05.484 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.483+0000 7f08851138c0 1 < completed multipart upload key=MMLF65KLC42Q upload=84CW4907ZM99 tag=8EOD3IJ7ANSL type=write size=2897774 2024-05-30T16:04:05.488 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.487+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=SOZH5G7LGYD2 type=del size=1496734 2024-05-30T16:04:05.492 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.491+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~05OEEZ0FL608.0 size=1048576 2024-05-30T16:04:05.496 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.495+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~05OEEZ0FL608.1 size=215321 2024-05-30T16:04:05.502 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.503+0000 7f08851138c0 1 < completed multipart upload key=TPCETCO3Y1M6 upload=05OEEZ0FL608 tag=SCLIQP8Q2JLX type=write size=2312473 2024-05-30T16:04:05.509 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.507+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~TYWMLKYZSTL8.0 size=1048576 2024-05-30T16:04:05.511 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.511+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~TYWMLKYZSTL8.1 size=1048576 2024-05-30T16:04:05.514 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.515+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~TYWMLKYZSTL8.2 size=637137 2024-05-30T16:04:05.514 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.515+0000 7f08851138c0 1 > prepared multipart upload key=A5BUBY9Z7JQG upload=TYWMLKYZSTL8 tag=CJSZXMMKADN8 type=write size=3782865 2024-05-30T16:04:05.518 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.519+0000 7f08851138c0 1 < completed write operation key=O8CWPJ9GW2GO tag=PVDPG3XXBQU2 type=write size=964912 2024-05-30T16:04:05.522 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.523+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=92Z8R2QJAHRJ type=del size=3464480 2024-05-30T16:04:05.526 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.527+0000 7f08851138c0 -1 < failed to complete operation key=MMLF65KLC42Q tag=6331PXL8BSNP type=del size=872822: (22) Invalid argument 2024-05-30T16:04:05.533 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.531+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~CKSQORVBWC65.0 size=1048576 2024-05-30T16:04:05.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.535+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~CKSQORVBWC65.1 size=29554 2024-05-30T16:04:05.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.535+0000 7f08851138c0 1 > prepared multipart upload key=0T2E9F3K1USF upload=CKSQORVBWC65 tag=RF2QQUN3PPRM type=write size=2126706 2024-05-30T16:04:05.543 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.543+0000 7f08851138c0 1 < suggested remove operation key=1N6R2HBTWMP2 tag=TI3RLVJ3C0YK type=write 2024-05-30T16:04:05.547 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.547+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=72YON2ORWG5U type=del size=687707 2024-05-30T16:04:05.550 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.551+0000 7f08851138c0 1 < completed write operation key=Y8SBI0QZ0ZV8 tag=1E91VTW5YD90 type=write size=74603 2024-05-30T16:04:05.554 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.555+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=B3LTOXOYGIEP type=write size=980351 2024-05-30T16:04:05.558 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.559+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~HVLXTPXH6J56.0 size=1048576 2024-05-30T16:04:05.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.559+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~HVLXTPXH6J56.1 size=13301 2024-05-30T16:04:05.567 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.567+0000 7f08851138c0 1 < completed multipart upload key=NIV7AQ4EV5YP upload=HVLXTPXH6J56 tag=HYIZUEZMIMD8 type=write size=2110453 2024-05-30T16:04:05.571 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.571+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=18VC37IS3SWQ type=del size=3917661 2024-05-30T16:04:05.574 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.575+0000 7f08851138c0 1 < completed delete operation key=PLWFSKPKVO19 tag=RJCOSDACCPVX type=del 2024-05-30T16:04:05.581 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.583+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~1UNGJLVORUBX.0 size=1048576 2024-05-30T16:04:05.584 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.583+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~1UNGJLVORUBX.1 size=505978 2024-05-30T16:04:05.584 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.583+0000 7f08851138c0 1 > prepared multipart upload key=4N58ZY4E1H6E upload=1UNGJLVORUBX tag=X41GQR7MRC2B type=write size=2603130 2024-05-30T16:04:05.588 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.587+0000 7f08851138c0 1 < completed delete operation key=OKBB0VU765TW tag=Q6PIOFDS42X4 type=del 2024-05-30T16:04:05.594 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.595+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~844IYRXUKGQO.0 size=1048576 2024-05-30T16:04:05.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.595+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~844IYRXUKGQO.1 size=746453 2024-05-30T16:04:05.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.595+0000 7f08851138c0 1 > prepared multipart upload key=0T2E9F3K1USF upload=844IYRXUKGQO tag=HU130QPL8BOB type=write size=2843605 2024-05-30T16:04:05.600 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.599+0000 7f08851138c0 1 < completed delete operation key=E2FXSXOEUGIG tag=FKZYHHZBJQV4 type=del 2024-05-30T16:04:05.604 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.603+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=VUKENUS524L5 type=del size=2192590 2024-05-30T16:04:05.607 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.607+0000 7f08851138c0 1 < completed write operation key=GWXNCSJCLXKW tag=WFEFN9AVDJ7A type=write size=485506 2024-05-30T16:04:05.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.611+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=EE1J6S4HS8D4 type=del size=1059368 2024-05-30T16:04:05.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.615+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~D86FF0LTWKN4.0 size=1048576 2024-05-30T16:04:05.617 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.619+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~D86FF0LTWKN4.1 size=679633 2024-05-30T16:04:05.623 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.623+0000 7f08851138c0 1 < completed multipart upload key=XHGMUYBA2VNK upload=D86FF0LTWKN4 tag=KOLG0FO1W0ZJ type=write size=2776785 2024-05-30T16:04:05.630 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.631+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~Y99MH18Q8M7F.0 size=390139 2024-05-30T16:04:05.630 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.631+0000 7f08851138c0 1 > prepared multipart upload key=ZALVY70ISLM5 upload=Y99MH18Q8M7F tag=9VOUGLYAWN5W type=write size=1438715 2024-05-30T16:04:05.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.635+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~KJ3PC8T1SQHL.0 size=1048576 2024-05-30T16:04:05.637 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.635+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~KJ3PC8T1SQHL.1 size=1048576 2024-05-30T16:04:05.640 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.639+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~KJ3PC8T1SQHL.2 size=44551 2024-05-30T16:04:05.646 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.647+0000 7f08851138c0 1 < completed multipart upload key=VRXNO8JLAZYH upload=KJ3PC8T1SQHL tag=JKT7KZA6R074 type=write size=3190279 2024-05-30T16:04:05.653 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.651+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~A9UBAVRQJRPU.0 size=1048576 2024-05-30T16:04:05.656 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.655+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~A9UBAVRQJRPU.1 size=1048576 2024-05-30T16:04:05.659 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.659+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~A9UBAVRQJRPU.2 size=396390 2024-05-30T16:04:05.662 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.659+0000 7f08851138c0 1 > prepared multipart upload key=GWXNCSJCLXKW upload=A9UBAVRQJRPU tag=5YMN1LT732FG type=write size=3542118 2024-05-30T16:04:05.663 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.663+0000 7f08851138c0 1 < completed delete operation key=JNZI7JX95LX2 tag=A1VG3OKRAQ37 type=del 2024-05-30T16:04:05.667 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.667+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=0WAMR7U1QYJK type=del size=1243236 2024-05-30T16:04:05.668 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.667+0000 7f08851138c0 1 < canceled part key=_multipart_1N6R2HBTWMP2.2~KQQPSB1D618L.0 size=1048576 2024-05-30T16:04:05.676 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.675+0000 7f08851138c0 1 < completed part key=_multipart_1N6R2HBTWMP2.2~KQQPSB1D618L.1 size=984192 2024-05-30T16:04:05.683 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.683+0000 7f08851138c0 1 < canceled multipart upload key=1N6R2HBTWMP2 upload=KQQPSB1D618L tag=L4HZ7CQ5U26Z type=write size=3081344 2024-05-30T16:04:05.687 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.687+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=IQ8HY2T78WW9 type=del size=2240424 2024-05-30T16:04:05.688 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.687+0000 7f08851138c0 1 < canceled part key=_multipart_3TBOZWT2P8U8.2~BCD9CC1LAU25.0 size=1048576 2024-05-30T16:04:05.691 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.691+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~BCD9CC1LAU25.1 size=685166 2024-05-30T16:04:05.697 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.695+0000 7f08851138c0 1 < canceled multipart upload key=3TBOZWT2P8U8 upload=BCD9CC1LAU25 tag=A93UBQZ7FMSZ type=write size=2782318 2024-05-30T16:04:05.700 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.699+0000 7f08851138c0 1 > prepared operation key=VRXNO8JLAZYH tag=BB6V2BPQ4XL6 type=del size=333356 2024-05-30T16:04:05.702 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.703+0000 7f08851138c0 -1 < no bi entry to suggest for operation key=1N6R2HBTWMP2 tag=SOZH5G7LGYD2 type=del size=1496734: (2) No such file or directory 2024-05-30T16:04:05.706 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.707+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=0WERE32IQ7XF type=del size=2117555 2024-05-30T16:04:05.710 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.711+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~TYWMLKYZSTL8.0 size=1048576 2024-05-30T16:04:05.713 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.711+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~TYWMLKYZSTL8.1 size=1048576 2024-05-30T16:04:05.716 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.715+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~TYWMLKYZSTL8.2 size=637137 2024-05-30T16:04:05.723 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.723+0000 7f08851138c0 1 < completed multipart upload key=A5BUBY9Z7JQG upload=TYWMLKYZSTL8 tag=CJSZXMMKADN8 type=write size=3782865 2024-05-30T16:04:05.727 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.727+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=Y38VQL30U1W2 type=del size=291677 2024-05-30T16:04:05.732 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.731+0000 7f08851138c0 1 < completed delete operation key=XHGMUYBA2VNK tag=92Z8R2QJAHRJ type=del 2024-05-30T16:04:05.736 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.735+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=GG3RS1NSDK3U type=del size=2092743 2024-05-30T16:04:05.740 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.739+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~CKSQORVBWC65.0 size=1048576 2024-05-30T16:04:05.743 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.743+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~CKSQORVBWC65.1 size=29554 2024-05-30T16:04:05.750 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.751+0000 7f08851138c0 1 < completed multipart upload key=0T2E9F3K1USF upload=CKSQORVBWC65 tag=RF2QQUN3PPRM type=write size=2126706 2024-05-30T16:04:05.754 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.755+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=ABZOQKLZUQRQ type=write size=620610 2024-05-30T16:04:05.758 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.759+0000 7f08851138c0 1 < completed delete operation key=QCLY10D2A3VF tag=72YON2ORWG5U type=del 2024-05-30T16:04:05.766 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.767+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~7JO75O1VGD7B.0 size=1048576 2024-05-30T16:04:05.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.771+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~7JO75O1VGD7B.1 size=572608 2024-05-30T16:04:05.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.771+0000 7f08851138c0 1 > prepared multipart upload key=JNZI7JX95LX2 upload=7JO75O1VGD7B tag=5Z5DP66VEJTB type=write size=2669760 2024-05-30T16:04:05.777 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.779+0000 7f08851138c0 1 < suggested update operation key=XHGMUYBA2VNK tag=B3LTOXOYGIEP type=write size=980351 2024-05-30T16:04:05.781 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.783+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=1JIJ0QB58V25 type=write size=982776 2024-05-30T16:04:05.785 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.783+0000 7f08851138c0 1 < completed delete operation key=QCLY10D2A3VF tag=18VC37IS3SWQ type=del 2024-05-30T16:04:05.789 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.787+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=T2JSETF9BEXK type=del size=2425458 2024-05-30T16:04:05.790 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.791+0000 7f08851138c0 1 < canceled part key=_multipart_4N58ZY4E1H6E.2~1UNGJLVORUBX.0 size=1048576 2024-05-30T16:04:05.793 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.791+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~1UNGJLVORUBX.1 size=505978 2024-05-30T16:04:05.799 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.799+0000 7f08851138c0 1 < canceled multipart upload key=4N58ZY4E1H6E upload=1UNGJLVORUBX tag=X41GQR7MRC2B type=write size=2603130 2024-05-30T16:04:05.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.803+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=XX4C951E1VEN type=del size=1763844 2024-05-30T16:04:05.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.803+0000 7f08851138c0 1 < canceled part key=_multipart_0T2E9F3K1USF.2~844IYRXUKGQO.0 size=1048576 2024-05-30T16:04:05.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.807+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~844IYRXUKGQO.1 size=746453 2024-05-30T16:04:05.812 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.811+0000 7f08851138c0 1 < canceled multipart upload key=0T2E9F3K1USF upload=844IYRXUKGQO tag=HU130QPL8BOB type=write size=2843605 2024-05-30T16:04:05.816 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.815+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=Y12F6DM87710 type=del size=1904688 2024-05-30T16:04:05.819 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.819+0000 7f08851138c0 1 < completed delete operation key=O8CWPJ9GW2GO tag=VUKENUS524L5 type=del 2024-05-30T16:04:05.825 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.823+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~JHXANCNLIUB2.0 size=1048576 2024-05-30T16:04:05.827 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.827+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~JHXANCNLIUB2.1 size=1048576 2024-05-30T16:04:05.830 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.831+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~JHXANCNLIUB2.2 size=79060 2024-05-30T16:04:05.830 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.831+0000 7f08851138c0 1 > prepared multipart upload key=QQILXRJD0C1Z upload=JHXANCNLIUB2 tag=ARSIVUYF6Q98 type=write size=3224788 2024-05-30T16:04:05.833 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.835+0000 7f08851138c0 1 < completed delete operation key=KPHZGFJLC2E0 tag=EE1J6S4HS8D4 type=del 2024-05-30T16:04:05.836 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.835+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=E595T5JWFU2X type=del size=2370261 2024-05-30T16:04:05.839 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.839+0000 7f08851138c0 1 < completed part key=_multipart_ZALVY70ISLM5.2~Y99MH18Q8M7F.0 size=390139 2024-05-30T16:04:05.845 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.847+0000 7f08851138c0 1 < completed multipart upload key=ZALVY70ISLM5 upload=Y99MH18Q8M7F tag=9VOUGLYAWN5W type=write size=1438715 2024-05-30T16:04:05.851 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.851+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~FQT4RHE9M3LK.0 size=1048576 2024-05-30T16:04:05.854 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.855+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~FQT4RHE9M3LK.1 size=1048576 2024-05-30T16:04:05.857 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.855+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~FQT4RHE9M3LK.2 size=678623 2024-05-30T16:04:05.857 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.855+0000 7f08851138c0 1 > prepared multipart upload key=603HBR8PRH6L upload=FQT4RHE9M3LK tag=OISDL12VFKQR type=write size=3824351 2024-05-30T16:04:05.860 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.859+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~A9UBAVRQJRPU.0 size=1048576 2024-05-30T16:04:05.866 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.863+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~A9UBAVRQJRPU.1 size=1048576 2024-05-30T16:04:05.866 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.867+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~A9UBAVRQJRPU.2 size=396390 2024-05-30T16:04:05.871 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.871+0000 7f08851138c0 1 < completed multipart upload key=GWXNCSJCLXKW upload=A9UBAVRQJRPU tag=5YMN1LT732FG type=write size=3542118 2024-05-30T16:04:05.875 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.875+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=05UY11BQGS9R type=del size=4052846 2024-05-30T16:04:05.878 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.879+0000 7f08851138c0 1 < completed delete operation key=OKBB0VU765TW tag=0WAMR7U1QYJK type=del 2024-05-30T16:04:05.882 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.883+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=KRFWZXRHKFAH type=del size=3676427 2024-05-30T16:04:05.885 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.883+0000 7f08851138c0 1 < canceled operation key=TPCETCO3Y1M6 tag=IQ8HY2T78WW9 type=del size=2240424 2024-05-30T16:04:05.891 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.891+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~JTQNNG4LWA4U.0 size=780281 2024-05-30T16:04:05.891 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.891+0000 7f08851138c0 1 > prepared multipart upload key=WT9RC0UYVG4F upload=JTQNNG4LWA4U tag=TLREDQNTJFQF type=write size=1828857 2024-05-30T16:04:05.895 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.895+0000 7f08851138c0 1 < canceled operation key=VRXNO8JLAZYH tag=BB6V2BPQ4XL6 type=del size=333356 2024-05-30T16:04:05.902 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.903+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~29JRU9AV5HDN.0 size=1048576 2024-05-30T16:04:05.904 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.903+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~29JRU9AV5HDN.1 size=1048576 2024-05-30T16:04:05.907 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.907+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~29JRU9AV5HDN.2 size=830545 2024-05-30T16:04:05.907 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.907+0000 7f08851138c0 1 > prepared multipart upload key=4N58ZY4E1H6E upload=29JRU9AV5HDN tag=6VA0OH8J1VZ1 type=write size=3976273 2024-05-30T16:04:05.914 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.915+0000 7f08851138c0 1 < suggested remove operation key=YRP4CSG92QRO tag=0WERE32IQ7XF type=del 2024-05-30T16:04:05.920 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.919+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~PWX3VRZC509H.0 size=1048576 2024-05-30T16:04:05.922 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.923+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~PWX3VRZC509H.1 size=1048576 2024-05-30T16:04:05.925 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.923+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~PWX3VRZC509H.2 size=637929 2024-05-30T16:04:05.925 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.923+0000 7f08851138c0 1 > prepared multipart upload key=O8CWPJ9GW2GO upload=PWX3VRZC509H tag=3VWPZF5L4B4U type=write size=3783657 2024-05-30T16:04:05.928 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.927+0000 7f08851138c0 1 < completed delete operation key=JNZI7JX95LX2 tag=Y38VQL30U1W2 type=del 2024-05-30T16:04:05.932 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.931+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=DQACAJTT40RK type=del size=435380 2024-05-30T16:04:05.938 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.939+0000 7f08851138c0 1 < suggested remove operation key=E2FXSXOEUGIG tag=GG3RS1NSDK3U type=del 2024-05-30T16:04:05.945 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.943+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~2XEM5D04Z3ZT.0 size=1048576 2024-05-30T16:04:05.947 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.947+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~2XEM5D04Z3ZT.1 size=1048576 2024-05-30T16:04:05.950 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.951+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~2XEM5D04Z3ZT.2 size=935356 2024-05-30T16:04:05.950 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.951+0000 7f08851138c0 1 > prepared multipart upload key=603HBR8PRH6L upload=2XEM5D04Z3ZT tag=3T65ME74VJEH type=write size=4081084 2024-05-30T16:04:05.954 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.955+0000 7f08851138c0 1 < completed write operation key=WT9RC0UYVG4F tag=ABZOQKLZUQRQ type=write size=620610 2024-05-30T16:04:05.957 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.955+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=S8TM59LDEU39 type=write size=627091 2024-05-30T16:04:05.960 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.959+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~7JO75O1VGD7B.0 size=1048576 2024-05-30T16:04:05.963 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.963+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~7JO75O1VGD7B.1 size=572608 2024-05-30T16:04:05.969 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.967+0000 7f08851138c0 1 < completed multipart upload key=JNZI7JX95LX2 upload=7JO75O1VGD7B tag=5Z5DP66VEJTB type=write size=2669760 2024-05-30T16:04:05.972 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.971+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=88SP99A2XBKL type=write size=342278 2024-05-30T16:04:05.976 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.975+0000 7f08851138c0 1 < completed write operation key=NIV7AQ4EV5YP tag=1JIJ0QB58V25 type=write size=982776 2024-05-30T16:04:05.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.979+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=2V3YB256IZ8V type=write size=487408 2024-05-30T16:04:05.985 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.987+0000 7f08851138c0 1 < suggested remove operation key=603HBR8PRH6L tag=T2JSETF9BEXK type=del 2024-05-30T16:04:05.989 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.987+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=DEJSAYF37VUU type=del size=2796658 2024-05-30T16:04:05.992 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.991+0000 7f08851138c0 1 < completed delete operation key=364U3OV10VPD tag=XX4C951E1VEN type=del 2024-05-30T16:04:05.999 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.999+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~I3724P9KVWAM.0 size=970612 2024-05-30T16:04:05.999 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:05.999+0000 7f08851138c0 1 > prepared multipart upload key=VRXNO8JLAZYH upload=I3724P9KVWAM tag=LFRW6TZLAV24 type=write size=2019188 2024-05-30T16:04:06.006 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.007+0000 7f08851138c0 1 < suggested remove operation key=0T2E9F3K1USF tag=Y12F6DM87710 type=del 2024-05-30T16:04:06.013 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.011+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=IMV1D298L0AS type=del size=1549917 2024-05-30T16:04:06.017 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.015+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~JHXANCNLIUB2.0 size=1048576 2024-05-30T16:04:06.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.019+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~JHXANCNLIUB2.1 size=1048576 2024-05-30T16:04:06.022 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.023+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~JHXANCNLIUB2.2 size=79060 2024-05-30T16:04:06.028 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.027+0000 7f08851138c0 1 < completed multipart upload key=QQILXRJD0C1Z upload=JHXANCNLIUB2 tag=ARSIVUYF6Q98 type=write size=3224788 2024-05-30T16:04:06.031 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.031+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=VQOP4GYVJAMO type=del size=765154 2024-05-30T16:04:06.035 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.035+0000 7f08851138c0 1 < canceled operation key=QQILXRJD0C1Z tag=E595T5JWFU2X type=del size=2370261 2024-05-30T16:04:06.042 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.043+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~DTLIMRGPRUQF.0 size=1048576 2024-05-30T16:04:06.045 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.043+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~DTLIMRGPRUQF.1 size=1048576 2024-05-30T16:04:06.047 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.047+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~DTLIMRGPRUQF.2 size=656271 2024-05-30T16:04:06.047 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.047+0000 7f08851138c0 1 > prepared multipart upload key=E2FXSXOEUGIG upload=DTLIMRGPRUQF tag=T5IOW3MA6X0R type=write size=3801999 2024-05-30T16:04:06.051 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.051+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~FQT4RHE9M3LK.0 size=1048576 2024-05-30T16:04:06.054 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.055+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~FQT4RHE9M3LK.1 size=1048576 2024-05-30T16:04:06.057 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.055+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~FQT4RHE9M3LK.2 size=678623 2024-05-30T16:04:06.063 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.063+0000 7f08851138c0 1 < completed multipart upload key=603HBR8PRH6L upload=FQT4RHE9M3LK tag=OISDL12VFKQR type=write size=3824351 2024-05-30T16:04:06.066 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.067+0000 7f08851138c0 1 > prepared operation key=ZALVY70ISLM5 tag=WA732JNXJ12W type=del size=4172345 2024-05-30T16:04:06.070 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.071+0000 7f08851138c0 1 < completed delete operation key=O8CWPJ9GW2GO tag=05UY11BQGS9R type=del 2024-05-30T16:04:06.073 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.071+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=25I8I34B8VG0 type=del size=2919010 2024-05-30T16:04:06.076 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.075+0000 7f08851138c0 1 < completed delete operation key=V4OQFOOQUXFQ tag=KRFWZXRHKFAH type=del 2024-05-30T16:04:06.080 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.079+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=L7NQP0BVDMK0 type=del size=3881221 2024-05-30T16:04:06.081 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.079+0000 7f08851138c0 1 < canceled part key=_multipart_WT9RC0UYVG4F.2~JTQNNG4LWA4U.0 size=780281 2024-05-30T16:04:06.086 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.087+0000 7f08851138c0 1 < canceled multipart upload key=WT9RC0UYVG4F upload=JTQNNG4LWA4U tag=TLREDQNTJFQF type=write size=1828857 2024-05-30T16:04:06.092 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.091+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~O5Y6GRN0O34V.0 size=1048576 2024-05-30T16:04:06.095 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.095+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~O5Y6GRN0O34V.1 size=1048576 2024-05-30T16:04:06.097 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.095+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~O5Y6GRN0O34V.2 size=911063 2024-05-30T16:04:06.097 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.095+0000 7f08851138c0 1 > prepared multipart upload key=0RZRBW38WKNZ upload=O5Y6GRN0O34V tag=3GMOXELM1FN0 type=write size=4056791 2024-05-30T16:04:06.100 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.099+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~29JRU9AV5HDN.0 size=1048576 2024-05-30T16:04:06.103 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.103+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~29JRU9AV5HDN.1 size=1048576 2024-05-30T16:04:06.103 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.103+0000 7f08851138c0 1 < canceled part key=_multipart_4N58ZY4E1H6E.2~29JRU9AV5HDN.2 size=830545 2024-05-30T16:04:06.109 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.107+0000 7f08851138c0 1 < canceled multipart upload key=4N58ZY4E1H6E upload=29JRU9AV5HDN tag=6VA0OH8J1VZ1 type=write size=3976273 2024-05-30T16:04:06.115 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.115+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~3W68YESIHPRB.0 size=1048576 2024-05-30T16:04:06.118 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.119+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~3W68YESIHPRB.1 size=1048576 2024-05-30T16:04:06.120 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.119+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~3W68YESIHPRB.2 size=445367 2024-05-30T16:04:06.120 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.119+0000 7f08851138c0 1 > prepared multipart upload key=0RZRBW38WKNZ upload=3W68YESIHPRB tag=49988ILM4M9I type=write size=3591095 2024-05-30T16:04:06.124 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.123+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~PWX3VRZC509H.0 size=1048576 2024-05-30T16:04:06.127 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.127+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~PWX3VRZC509H.1 size=1048576 2024-05-30T16:04:06.129 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.131+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~PWX3VRZC509H.2 size=637929 2024-05-30T16:04:06.136 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.135+0000 7f08851138c0 1 < completed multipart upload key=O8CWPJ9GW2GO upload=PWX3VRZC509H tag=3VWPZF5L4B4U type=write size=3783657 2024-05-30T16:04:06.140 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.139+0000 7f08851138c0 1 > prepared operation key=VRXNO8JLAZYH tag=UAJ9D59HG4O2 type=del size=501184 2024-05-30T16:04:06.144 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.143+0000 7f08851138c0 -1 < failed to complete operation key=0T2E9F3K1USF tag=DQACAJTT40RK type=del size=435380: (22) Invalid argument 2024-05-30T16:04:06.147 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.147+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=LW1FTATYOIKV type=write size=561849 2024-05-30T16:04:06.151 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.151+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~2XEM5D04Z3ZT.0 size=1048576 2024-05-30T16:04:06.154 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.155+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~2XEM5D04Z3ZT.1 size=1048576 2024-05-30T16:04:06.157 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.155+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~2XEM5D04Z3ZT.2 size=935356 2024-05-30T16:04:06.163 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.163+0000 7f08851138c0 1 < completed multipart upload key=603HBR8PRH6L upload=2XEM5D04Z3ZT tag=3T65ME74VJEH type=write size=4081084 2024-05-30T16:04:06.166 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.167+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=PA76CYY2U0AJ type=del size=3347905 2024-05-30T16:04:06.173 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.171+0000 7f08851138c0 1 < suggested update operation key=OKBB0VU765TW tag=S8TM59LDEU39 type=write size=627091 2024-05-30T16:04:06.176 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.175+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=WCW4QEJ23GUO type=del size=1508122 2024-05-30T16:04:06.180 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.179+0000 7f08851138c0 1 < completed write operation key=RCTV94WRMO53 tag=88SP99A2XBKL type=write size=342278 2024-05-30T16:04:06.184 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.183+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=TBLF0E8KD83P type=del size=1302206 2024-05-30T16:04:06.188 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.187+0000 7f08851138c0 1 < completed write operation key=V4OQFOOQUXFQ tag=2V3YB256IZ8V type=write size=487408 2024-05-30T16:04:06.194 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.195+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~SJ6PP9WNIUF2.0 size=1048576 2024-05-30T16:04:06.197 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.195+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~SJ6PP9WNIUF2.1 size=1048576 2024-05-30T16:04:06.199 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.199+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~SJ6PP9WNIUF2.2 size=5563 2024-05-30T16:04:06.200 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.199+0000 7f08851138c0 1 > prepared multipart upload key=GWXNCSJCLXKW upload=SJ6PP9WNIUF2 tag=N45KUZQD5ZFE type=write size=3151291 2024-05-30T16:04:06.203 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.203+0000 7f08851138c0 1 < completed delete operation key=JNZI7JX95LX2 tag=DEJSAYF37VUU type=del 2024-05-30T16:04:06.207 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.207+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=JWRR5CI9ZM7A type=del size=3113652 2024-05-30T16:04:06.210 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.211+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~I3724P9KVWAM.0 size=970612 2024-05-30T16:04:06.217 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.215+0000 7f08851138c0 1 < completed multipart upload key=VRXNO8JLAZYH upload=I3724P9KVWAM tag=LFRW6TZLAV24 type=write size=2019188 2024-05-30T16:04:06.221 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.223+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=40TK47CKDHVT type=write size=784416 2024-05-30T16:04:06.226 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.227+0000 7f08851138c0 1 < canceled operation key=RCTV94WRMO53 tag=IMV1D298L0AS type=del size=1549917 2024-05-30T16:04:06.229 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.231+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=6G4NK8AP1NFY type=write size=645871 2024-05-30T16:04:06.233 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.231+0000 7f08851138c0 1 < canceled operation key=QCLY10D2A3VF tag=VQOP4GYVJAMO type=del size=765154 2024-05-30T16:04:06.240 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.239+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~S1KZL7CM0X4X.0 size=1048576 2024-05-30T16:04:06.242 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.243+0000 7f08851138c0 1 > prepared part key=_multipart_0T2E9F3K1USF.2~S1KZL7CM0X4X.1 size=831480 2024-05-30T16:04:06.242 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.243+0000 7f08851138c0 1 > prepared multipart upload key=0T2E9F3K1USF upload=S1KZL7CM0X4X tag=EM4V2P7JIOR8 type=write size=2928632 2024-05-30T16:04:06.246 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.247+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~DTLIMRGPRUQF.0 size=1048576 2024-05-30T16:04:06.249 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.247+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~DTLIMRGPRUQF.1 size=1048576 2024-05-30T16:04:06.251 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.251+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~DTLIMRGPRUQF.2 size=656271 2024-05-30T16:04:06.257 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.255+0000 7f08851138c0 1 < completed multipart upload key=E2FXSXOEUGIG upload=DTLIMRGPRUQF tag=T5IOW3MA6X0R type=write size=3801999 2024-05-30T16:04:06.263 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.263+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~KMR8D9HYMUM6.0 size=1048576 2024-05-30T16:04:06.265 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.267+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~KMR8D9HYMUM6.1 size=326014 2024-05-30T16:04:06.266 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.267+0000 7f08851138c0 1 > prepared multipart upload key=O0DJ8SCY766V upload=KMR8D9HYMUM6 tag=R54KC80GRRE2 type=write size=2423166 2024-05-30T16:04:06.269 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.271+0000 7f08851138c0 1 < completed delete operation key=ZALVY70ISLM5 tag=WA732JNXJ12W type=del 2024-05-30T16:04:06.273 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.271+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=M928T3YIQLID type=del size=816477 2024-05-30T16:04:06.276 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.275+0000 7f08851138c0 1 < completed delete operation key=1N6R2HBTWMP2 tag=25I8I34B8VG0 type=del 2024-05-30T16:04:06.280 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.279+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=IP0YPRAB669F type=write size=478500 2024-05-30T16:04:06.283 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.283+0000 7f08851138c0 -1 < failed to complete operation key=OKBB0VU765TW tag=L7NQP0BVDMK0 type=del size=3881221: (22) Invalid argument 2024-05-30T16:04:06.287 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.287+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=WLYUK8KNBQ12 type=write size=320571 2024-05-30T16:04:06.288 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.287+0000 7f08851138c0 1 < canceled part key=_multipart_0RZRBW38WKNZ.2~O5Y6GRN0O34V.0 size=1048576 2024-05-30T16:04:06.291 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.291+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~O5Y6GRN0O34V.1 size=1048576 2024-05-30T16:04:06.294 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.295+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~O5Y6GRN0O34V.2 size=911063 2024-05-30T16:04:06.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.299+0000 7f08851138c0 1 < canceled multipart upload key=0RZRBW38WKNZ upload=O5Y6GRN0O34V tag=3GMOXELM1FN0 type=write size=4056791 2024-05-30T16:04:06.303 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.303+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=V34HKJZWJMR4 type=del size=272908 2024-05-30T16:04:06.307 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.307+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~3W68YESIHPRB.0 size=1048576 2024-05-30T16:04:06.310 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.311+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~3W68YESIHPRB.1 size=1048576 2024-05-30T16:04:06.313 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.311+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~3W68YESIHPRB.2 size=445367 2024-05-30T16:04:06.319 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.319+0000 7f08851138c0 1 < completed multipart upload key=0RZRBW38WKNZ upload=3W68YESIHPRB tag=49988ILM4M9I type=write size=3591095 2024-05-30T16:04:06.322 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.323+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=WZDVE8962LTN type=del size=628310 2024-05-30T16:04:06.326 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.327+0000 7f08851138c0 1 < completed delete operation key=VRXNO8JLAZYH tag=UAJ9D59HG4O2 type=del 2024-05-30T16:04:06.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.331+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=9ZNG99M2RKRI type=del size=1675644 2024-05-30T16:04:06.333 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.335+0000 7f08851138c0 1 < completed write operation key=364U3OV10VPD tag=LW1FTATYOIKV type=write size=561849 2024-05-30T16:04:06.346 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.347+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~EO4T4HD8OXZK.0 size=1048576 2024-05-30T16:04:06.349 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.347+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~EO4T4HD8OXZK.1 size=392102 2024-05-30T16:04:06.349 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.347+0000 7f08851138c0 1 > prepared multipart upload key=PLWFSKPKVO19 upload=EO4T4HD8OXZK tag=5PXDIEERR5RO type=write size=2489254 2024-05-30T16:04:06.356 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.355+0000 7f08851138c0 1 < suggested remove operation key=OKBB0VU765TW tag=PA76CYY2U0AJ type=del 2024-05-30T16:04:06.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.359+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=DAMGJPZS706D type=del size=1843750 2024-05-30T16:04:06.363 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.363+0000 7f08851138c0 1 < completed delete operation key=WT9RC0UYVG4F tag=WCW4QEJ23GUO type=del 2024-05-30T16:04:06.366 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.367+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=SWRO1HQ74CGU type=write size=67798 2024-05-30T16:04:06.373 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.371+0000 7f08851138c0 1 < suggested update operation key=0T2E9F3K1USF tag=TBLF0E8KD83P type=del size=1302206 2024-05-30T16:04:06.379 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.379+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~22W1FW94FA7A.0 size=1048576 2024-05-30T16:04:06.382 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.383+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~22W1FW94FA7A.1 size=1048576 2024-05-30T16:04:06.385 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.383+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~22W1FW94FA7A.2 size=10881 2024-05-30T16:04:06.385 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.383+0000 7f08851138c0 1 > prepared multipart upload key=OKBB0VU765TW upload=22W1FW94FA7A tag=KTIPFWHFXC0A type=write size=3156609 2024-05-30T16:04:06.388 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.387+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~SJ6PP9WNIUF2.0 size=1048576 2024-05-30T16:04:06.391 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.391+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~SJ6PP9WNIUF2.1 size=1048576 2024-05-30T16:04:06.391 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.391+0000 7f08851138c0 1 < canceled part key=_multipart_GWXNCSJCLXKW.2~SJ6PP9WNIUF2.2 size=5563 2024-05-30T16:04:06.396 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.395+0000 7f08851138c0 1 < canceled multipart upload key=GWXNCSJCLXKW upload=SJ6PP9WNIUF2 tag=N45KUZQD5ZFE type=write size=3151291 2024-05-30T16:04:06.400 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.399+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=VVWQ8I5D731G type=del size=3605311 2024-05-30T16:04:06.403 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.403+0000 7f08851138c0 1 < completed delete operation key=3IJWPWRB6FHE tag=JWRR5CI9ZM7A type=del 2024-05-30T16:04:06.408 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.407+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=4I6EN17KBMT9 type=del size=443225 2024-05-30T16:04:06.411 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.411+0000 7f08851138c0 1 < completed write operation key=YRP4CSG92QRO tag=40TK47CKDHVT type=write size=784416 2024-05-30T16:04:06.414 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.415+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=RENDFFESITR2 type=del size=3995579 2024-05-30T16:04:06.418 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.419+0000 7f08851138c0 1 < completed write operation key=V4OQFOOQUXFQ tag=6G4NK8AP1NFY type=write size=645871 2024-05-30T16:04:06.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.423+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~PE6AOTTLAG5O.0 size=1048576 2024-05-30T16:04:06.427 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.427+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~PE6AOTTLAG5O.1 size=404528 2024-05-30T16:04:06.427 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.427+0000 7f08851138c0 1 > prepared multipart upload key=3TBOZWT2P8U8 upload=PE6AOTTLAG5O tag=10D3MGVYXODE type=write size=2501680 2024-05-30T16:04:06.430 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.431+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~S1KZL7CM0X4X.0 size=1048576 2024-05-30T16:04:06.433 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.431+0000 7f08851138c0 1 < completed part key=_multipart_0T2E9F3K1USF.2~S1KZL7CM0X4X.1 size=831480 2024-05-30T16:04:06.438 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.439+0000 7f08851138c0 1 < completed multipart upload key=0T2E9F3K1USF upload=S1KZL7CM0X4X tag=EM4V2P7JIOR8 type=write size=2928632 2024-05-30T16:04:06.442 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.443+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=TUUKSM2TEIG7 type=del size=1316304 2024-05-30T16:04:06.445 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.447+0000 7f08851138c0 1 < completed part key=_multipart_O0DJ8SCY766V.2~KMR8D9HYMUM6.0 size=1048576 2024-05-30T16:04:06.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.447+0000 7f08851138c0 1 < completed part key=_multipart_O0DJ8SCY766V.2~KMR8D9HYMUM6.1 size=326014 2024-05-30T16:04:06.454 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.455+0000 7f08851138c0 1 < completed multipart upload key=O0DJ8SCY766V upload=KMR8D9HYMUM6 tag=R54KC80GRRE2 type=write size=2423166 2024-05-30T16:04:06.461 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.459+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~2A6W65QP9WNG.0 size=1048576 2024-05-30T16:04:06.464 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.463+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~2A6W65QP9WNG.1 size=867502 2024-05-30T16:04:06.464 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.463+0000 7f08851138c0 1 > prepared multipart upload key=TPCETCO3Y1M6 upload=2A6W65QP9WNG tag=NYRPI59AA296 type=write size=2964654 2024-05-30T16:04:06.467 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.467+0000 7f08851138c0 1 < completed delete operation key=4N58ZY4E1H6E tag=M928T3YIQLID type=del 2024-05-30T16:04:06.474 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.475+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~8MUJ8YYCPCXQ.0 size=1048576 2024-05-30T16:04:06.477 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.475+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~8MUJ8YYCPCXQ.1 size=236835 2024-05-30T16:04:06.477 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.475+0000 7f08851138c0 1 > prepared multipart upload key=NIV7AQ4EV5YP upload=8MUJ8YYCPCXQ tag=3RMRI7E1QP4F type=write size=2333987 2024-05-30T16:04:06.484 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.483+0000 7f08851138c0 1 < suggested remove operation key=XHGMUYBA2VNK tag=IP0YPRAB669F type=write 2024-05-30T16:04:06.487 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.487+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=IRZS6DFS6BLK type=del size=647398 2024-05-30T16:04:06.495 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.495+0000 7f08851138c0 1 < suggested update operation key=GWXNCSJCLXKW tag=WLYUK8KNBQ12 type=write size=320571 2024-05-30T16:04:06.501 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.499+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~D6MYYZHFN522.0 size=1048576 2024-05-30T16:04:06.504 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.503+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~D6MYYZHFN522.1 size=1048576 2024-05-30T16:04:06.507 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.507+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~D6MYYZHFN522.2 size=155369 2024-05-30T16:04:06.507 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.507+0000 7f08851138c0 1 > prepared multipart upload key=XHGMUYBA2VNK upload=D6MYYZHFN522 tag=K0BCMZ7K413C type=write size=3301097 2024-05-30T16:04:06.510 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.511+0000 7f08851138c0 1 < completed delete operation key=O8CWPJ9GW2GO tag=V34HKJZWJMR4 type=del 2024-05-30T16:04:06.514 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.515+0000 7f08851138c0 1 > prepared operation key=VRXNO8JLAZYH tag=L6KHZAPVQOXS type=write size=170870 2024-05-30T16:04:06.517 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.519+0000 7f08851138c0 -1 < failed to complete operation key=0T2E9F3K1USF tag=WZDVE8962LTN type=del size=628310: (22) Invalid argument 2024-05-30T16:04:06.524 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.523+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~K9RZ6G563T2X.0 size=1048576 2024-05-30T16:04:06.526 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.527+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~K9RZ6G563T2X.1 size=978568 2024-05-30T16:04:06.527 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.527+0000 7f08851138c0 1 > prepared multipart upload key=364U3OV10VPD upload=K9RZ6G563T2X tag=3E7PML4PWH4N type=write size=3075720 2024-05-30T16:04:06.530 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.531+0000 7f08851138c0 1 < completed delete operation key=PLWFSKPKVO19 tag=9ZNG99M2RKRI type=del 2024-05-30T16:04:06.537 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.535+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~O3DPWVCSRTDY.0 size=1048576 2024-05-30T16:04:06.540 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.539+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~O3DPWVCSRTDY.1 size=852692 2024-05-30T16:04:06.540 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.539+0000 7f08851138c0 1 > prepared multipart upload key=ZALVY70ISLM5 upload=O3DPWVCSRTDY tag=92Y6C5SIJCN6 type=write size=2949844 2024-05-30T16:04:06.543 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.543+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~EO4T4HD8OXZK.0 size=1048576 2024-05-30T16:04:06.546 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.547+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~EO4T4HD8OXZK.1 size=392102 2024-05-30T16:04:06.552 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.551+0000 7f08851138c0 1 < completed multipart upload key=PLWFSKPKVO19 upload=EO4T4HD8OXZK tag=5PXDIEERR5RO type=write size=2489254 2024-05-30T16:04:06.555 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.555+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=WJFPVENVNYQB type=del size=1433888 2024-05-30T16:04:06.559 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.559+0000 7f08851138c0 1 < completed delete operation key=Y8SBI0QZ0ZV8 tag=DAMGJPZS706D type=del 2024-05-30T16:04:06.565 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.567+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~VFYXKDI6D5R4.0 size=671775 2024-05-30T16:04:06.566 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.567+0000 7f08851138c0 1 > prepared multipart upload key=603HBR8PRH6L upload=VFYXKDI6D5R4 tag=TLCLFIXJWYYZ type=write size=1720351 2024-05-30T16:04:06.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.571+0000 7f08851138c0 1 < suggested update operation key=MMLF65KLC42Q tag=SWRO1HQ74CGU type=write size=67798 2024-05-30T16:04:06.579 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.579+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~RDUWXW6C80AV.0 size=1048576 2024-05-30T16:04:06.582 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.583+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~RDUWXW6C80AV.1 size=497860 2024-05-30T16:04:06.582 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.583+0000 7f08851138c0 1 > prepared multipart upload key=TPCETCO3Y1M6 upload=RDUWXW6C80AV tag=YM4953LNJWEC type=write size=2595012 2024-05-30T16:04:06.586 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.587+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~22W1FW94FA7A.0 size=1048576 2024-05-30T16:04:06.589 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.587+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~22W1FW94FA7A.1 size=1048576 2024-05-30T16:04:06.592 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.591+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~22W1FW94FA7A.2 size=10881 2024-05-30T16:04:06.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.599+0000 7f08851138c0 1 < completed multipart upload key=OKBB0VU765TW upload=22W1FW94FA7A tag=KTIPFWHFXC0A type=write size=3156609 2024-05-30T16:04:06.602 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.603+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=2W0BWC5R581B type=del size=488775 2024-05-30T16:04:06.605 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.607+0000 7f08851138c0 1 < completed delete operation key=QCLY10D2A3VF tag=VVWQ8I5D731G type=del 2024-05-30T16:04:06.613 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.611+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~N0824V6MAM7Q.0 size=1048576 2024-05-30T16:04:06.616 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.615+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~N0824V6MAM7Q.1 size=271099 2024-05-30T16:04:06.616 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.615+0000 7f08851138c0 1 > prepared multipart upload key=YRP4CSG92QRO upload=N0824V6MAM7Q tag=LNTO8CJ7GU42 type=write size=2368251 2024-05-30T16:04:06.624 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.623+0000 7f08851138c0 1 < suggested remove operation key=JNZI7JX95LX2 tag=4I6EN17KBMT9 type=del 2024-05-30T16:04:06.631 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.631+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~UKJOBJYWHD5X.0 size=1048576 2024-05-30T16:04:06.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.635+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~UKJOBJYWHD5X.1 size=1048576 2024-05-30T16:04:06.637 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.635+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~UKJOBJYWHD5X.2 size=478448 2024-05-30T16:04:06.637 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.635+0000 7f08851138c0 1 > prepared multipart upload key=Y3CYZJ83WYNI upload=UKJOBJYWHD5X tag=WD8ZXE9JJFB9 type=write size=3624176 2024-05-30T16:04:06.641 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.639+0000 7f08851138c0 1 < canceled operation key=QCLY10D2A3VF tag=RENDFFESITR2 type=del size=3995579 2024-05-30T16:04:06.648 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.647+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~8FL7I6BV7VY3.0 size=792268 2024-05-30T16:04:06.648 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.647+0000 7f08851138c0 1 > prepared multipart upload key=VRXNO8JLAZYH upload=8FL7I6BV7VY3 tag=EYSWHPW737DS type=write size=1840844 2024-05-30T16:04:06.649 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.647+0000 7f08851138c0 1 < canceled part key=_multipart_3TBOZWT2P8U8.2~PE6AOTTLAG5O.0 size=1048576 2024-05-30T16:04:06.652 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.651+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~PE6AOTTLAG5O.1 size=404528 2024-05-30T16:04:06.658 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.659+0000 7f08851138c0 1 < canceled multipart upload key=3TBOZWT2P8U8 upload=PE6AOTTLAG5O tag=10D3MGVYXODE type=write size=2501680 2024-05-30T16:04:06.662 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.663+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=IB72K1N4MZPQ type=write size=803666 2024-05-30T16:04:06.665 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.663+0000 7f08851138c0 -1 < failed to complete operation key=XHGMUYBA2VNK tag=TUUKSM2TEIG7 type=del size=1316304: (22) Invalid argument 2024-05-30T16:04:06.675 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.675+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~SM3OF63LCQFH.0 size=577388 2024-05-30T16:04:06.675 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.675+0000 7f08851138c0 1 > prepared multipart upload key=RCTV94WRMO53 upload=SM3OF63LCQFH tag=TB0WWCK8IO0J type=write size=1625964 2024-05-30T16:04:06.678 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.679+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~2A6W65QP9WNG.0 size=1048576 2024-05-30T16:04:06.682 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.679+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~2A6W65QP9WNG.1 size=867502 2024-05-30T16:04:06.687 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.687+0000 7f08851138c0 1 < completed multipart upload key=TPCETCO3Y1M6 upload=2A6W65QP9WNG tag=NYRPI59AA296 type=write size=2964654 2024-05-30T16:04:06.693 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.691+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~8TTYTFTHC2SH.0 size=14791 2024-05-30T16:04:06.693 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.691+0000 7f08851138c0 1 > prepared multipart upload key=XHGMUYBA2VNK upload=8TTYTFTHC2SH tag=46UKLB1MRBQ9 type=write size=1063367 2024-05-30T16:04:06.697 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.695+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~8MUJ8YYCPCXQ.0 size=1048576 2024-05-30T16:04:06.699 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.699+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~8MUJ8YYCPCXQ.1 size=236835 2024-05-30T16:04:06.705 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.703+0000 7f08851138c0 1 < completed multipart upload key=NIV7AQ4EV5YP upload=8MUJ8YYCPCXQ tag=3RMRI7E1QP4F type=write size=2333987 2024-05-30T16:04:06.708 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.707+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=YIE5UZ3DEIHP type=del size=1612956 2024-05-30T16:04:06.712 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.711+0000 7f08851138c0 1 < completed delete operation key=E2FXSXOEUGIG tag=IRZS6DFS6BLK type=del 2024-05-30T16:04:06.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.719+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~D5NSDEE6DBJW.0 size=871284 2024-05-30T16:04:06.719 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.719+0000 7f08851138c0 1 > prepared multipart upload key=VRXNO8JLAZYH upload=D5NSDEE6DBJW tag=UCK1C9MHU7IS type=write size=1919860 2024-05-30T16:04:06.719 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.719+0000 7f08851138c0 1 < canceled part key=_multipart_XHGMUYBA2VNK.2~D6MYYZHFN522.0 size=1048576 2024-05-30T16:04:06.722 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.723+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~D6MYYZHFN522.1 size=1048576 2024-05-30T16:04:06.725 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.723+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~D6MYYZHFN522.2 size=155369 2024-05-30T16:04:06.730 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.731+0000 7f08851138c0 1 < canceled multipart upload key=XHGMUYBA2VNK upload=D6MYYZHFN522 tag=K0BCMZ7K413C type=write size=3301097 2024-05-30T16:04:06.734 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.735+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=O9W7O91IUXHT type=del size=2153594 2024-05-30T16:04:06.737 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.739+0000 7f08851138c0 1 < completed write operation key=VRXNO8JLAZYH tag=L6KHZAPVQOXS type=write size=170870 2024-05-30T16:04:06.743 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.743+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~HAVGJSI70BQ5.0 size=1048576 2024-05-30T16:04:06.746 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.747+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~HAVGJSI70BQ5.1 size=515661 2024-05-30T16:04:06.746 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.747+0000 7f08851138c0 1 > prepared multipart upload key=KPHZGFJLC2E0 upload=HAVGJSI70BQ5 tag=GNEZDMUEH6WU type=write size=2612813 2024-05-30T16:04:06.749 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.751+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~K9RZ6G563T2X.0 size=1048576 2024-05-30T16:04:06.749 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.751+0000 7f08851138c0 1 < canceled part key=_multipart_364U3OV10VPD.2~K9RZ6G563T2X.1 size=978568 2024-05-30T16:04:06.755 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.755+0000 7f08851138c0 1 < canceled multipart upload key=364U3OV10VPD upload=K9RZ6G563T2X tag=3E7PML4PWH4N type=write size=3075720 2024-05-30T16:04:06.758 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.759+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=48N50FL3BLE8 type=write size=591023 2024-05-30T16:04:06.762 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.763+0000 7f08851138c0 1 < completed part key=_multipart_ZALVY70ISLM5.2~O3DPWVCSRTDY.0 size=1048576 2024-05-30T16:04:06.765 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.763+0000 7f08851138c0 1 < completed part key=_multipart_ZALVY70ISLM5.2~O3DPWVCSRTDY.1 size=852692 2024-05-30T16:04:06.771 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.771+0000 7f08851138c0 1 < completed multipart upload key=ZALVY70ISLM5 upload=O3DPWVCSRTDY tag=92Y6C5SIJCN6 type=write size=2949844 2024-05-30T16:04:06.773 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.775+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=2N1NNLUYK92T type=del size=796135 2024-05-30T16:04:06.778 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.779+0000 7f08851138c0 1 < completed delete operation key=QQILXRJD0C1Z tag=WJFPVENVNYQB type=del 2024-05-30T16:04:06.781 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.783+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=S1K63SDW1CAL type=del size=701490 2024-05-30T16:04:06.785 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.787+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~VFYXKDI6D5R4.0 size=671775 2024-05-30T16:04:06.790 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.791+0000 7f08851138c0 1 < completed multipart upload key=603HBR8PRH6L upload=VFYXKDI6D5R4 tag=TLCLFIXJWYYZ type=write size=1720351 2024-05-30T16:04:06.796 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.795+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~R3A464YHR9UH.0 size=1048576 2024-05-30T16:04:06.799 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.799+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~R3A464YHR9UH.1 size=1048576 2024-05-30T16:04:06.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.803+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~R3A464YHR9UH.2 size=169008 2024-05-30T16:04:06.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.803+0000 7f08851138c0 1 > prepared multipart upload key=GWXNCSJCLXKW upload=R3A464YHR9UH tag=F3DS9MQ2K2G5 type=write size=3314736 2024-05-30T16:04:06.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.807+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~RDUWXW6C80AV.0 size=1048576 2024-05-30T16:04:06.808 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.807+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~RDUWXW6C80AV.1 size=497860 2024-05-30T16:04:06.814 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.815+0000 7f08851138c0 1 < completed multipart upload key=TPCETCO3Y1M6 upload=RDUWXW6C80AV tag=YM4953LNJWEC type=write size=2595012 2024-05-30T16:04:06.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.819+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~VHU4BRRO2PQZ.0 size=1048576 2024-05-30T16:04:06.823 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.823+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~VHU4BRRO2PQZ.1 size=1048576 2024-05-30T16:04:06.825 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.823+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~VHU4BRRO2PQZ.2 size=676831 2024-05-30T16:04:06.825 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.823+0000 7f08851138c0 1 > prepared multipart upload key=Y3CYZJ83WYNI upload=VHU4BRRO2PQZ tag=BM95SSG2DWBH type=write size=3822559 2024-05-30T16:04:06.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.827+0000 7f08851138c0 1 < completed delete operation key=GWXNCSJCLXKW tag=2W0BWC5R581B type=del 2024-05-30T16:04:06.834 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.835+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~EHSSB16GUW19.0 size=1048576 2024-05-30T16:04:06.836 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.835+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~EHSSB16GUW19.1 size=527702 2024-05-30T16:04:06.836 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.835+0000 7f08851138c0 1 > prepared multipart upload key=1N6R2HBTWMP2 upload=EHSSB16GUW19 tag=V6P9MRFT7NW4 type=write size=2624854 2024-05-30T16:04:06.839 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.839+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~N0824V6MAM7Q.0 size=1048576 2024-05-30T16:04:06.842 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.843+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~N0824V6MAM7Q.1 size=271099 2024-05-30T16:04:06.847 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.847+0000 7f08851138c0 1 < completed multipart upload key=YRP4CSG92QRO upload=N0824V6MAM7Q tag=LNTO8CJ7GU42 type=write size=2368251 2024-05-30T16:04:06.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.851+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~M0F6F1LZKGTS.0 size=1048576 2024-05-30T16:04:06.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.855+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~M0F6F1LZKGTS.1 size=1048576 2024-05-30T16:04:06.857 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.859+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~M0F6F1LZKGTS.2 size=652843 2024-05-30T16:04:06.858 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.859+0000 7f08851138c0 1 > prepared multipart upload key=3TBOZWT2P8U8 upload=M0F6F1LZKGTS tag=H384U4ADUFVT type=write size=3798571 2024-05-30T16:04:06.861 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.859+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~UKJOBJYWHD5X.0 size=1048576 2024-05-30T16:04:06.863 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.863+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~UKJOBJYWHD5X.1 size=1048576 2024-05-30T16:04:06.863 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.863+0000 7f08851138c0 1 < canceled part key=_multipart_Y3CYZJ83WYNI.2~UKJOBJYWHD5X.2 size=478448 2024-05-30T16:04:06.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.867+0000 7f08851138c0 1 < canceled multipart upload key=Y3CYZJ83WYNI upload=UKJOBJYWHD5X tag=WD8ZXE9JJFB9 type=write size=3624176 2024-05-30T16:04:06.872 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.871+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=G4NLHM8P1DM4 type=del size=4081790 2024-05-30T16:04:06.876 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.875+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~8FL7I6BV7VY3.0 size=792268 2024-05-30T16:04:06.881 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.883+0000 7f08851138c0 1 < completed multipart upload key=VRXNO8JLAZYH upload=8FL7I6BV7VY3 tag=EYSWHPW737DS type=write size=1840844 2024-05-30T16:04:06.886 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.883+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=7NCNFF6CDBAP type=write size=744576 2024-05-30T16:04:06.888 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.887+0000 7f08851138c0 1 < completed write operation key=0RZRBW38WKNZ tag=IB72K1N4MZPQ type=write size=803666 2024-05-30T16:04:06.893 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.895+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~8KASQZ8MNJ59.0 size=1048576 2024-05-30T16:04:06.896 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.895+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~8KASQZ8MNJ59.1 size=1048576 2024-05-30T16:04:06.898 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.899+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~8KASQZ8MNJ59.2 size=71932 2024-05-30T16:04:06.898 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.899+0000 7f08851138c0 1 > prepared multipart upload key=QCLY10D2A3VF upload=8KASQZ8MNJ59 tag=L0KA799QYABT type=write size=3217660 2024-05-30T16:04:06.901 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.903+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~SM3OF63LCQFH.0 size=577388 2024-05-30T16:04:06.907 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.907+0000 7f08851138c0 1 < completed multipart upload key=RCTV94WRMO53 upload=SM3OF63LCQFH tag=TB0WWCK8IO0J type=write size=1625964 2024-05-30T16:04:06.910 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.911+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=77INHIB9E4ZS type=del size=1453374 2024-05-30T16:04:06.913 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.911+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~8TTYTFTHC2SH.0 size=14791 2024-05-30T16:04:06.918 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.919+0000 7f08851138c0 1 < completed multipart upload key=XHGMUYBA2VNK upload=8TTYTFTHC2SH tag=46UKLB1MRBQ9 type=write size=1063367 2024-05-30T16:04:06.921 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.923+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=YBAOJ7N1X3KV type=del size=2750740 2024-05-30T16:04:06.925 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.923+0000 7f08851138c0 1 < completed delete operation key=Y3CYZJ83WYNI tag=YIE5UZ3DEIHP type=del 2024-05-30T16:04:06.931 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.931+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~Y1K494D34XE2.0 size=1048576 2024-05-30T16:04:06.934 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.935+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~Y1K494D34XE2.1 size=1048576 2024-05-30T16:04:06.937 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.935+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~Y1K494D34XE2.2 size=814504 2024-05-30T16:04:06.937 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.935+0000 7f08851138c0 1 > prepared multipart upload key=3IJWPWRB6FHE upload=Y1K494D34XE2 tag=FZ6S277TPTKB type=write size=3960232 2024-05-30T16:04:06.938 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.939+0000 7f08851138c0 1 < canceled part key=_multipart_VRXNO8JLAZYH.2~D5NSDEE6DBJW.0 size=871284 2024-05-30T16:04:06.943 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.943+0000 7f08851138c0 1 < canceled multipart upload key=VRXNO8JLAZYH upload=D5NSDEE6DBJW tag=UCK1C9MHU7IS type=write size=1919860 2024-05-30T16:04:06.947 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.947+0000 7f08851138c0 1 > prepared operation key=VRXNO8JLAZYH tag=40UGUV3MF4NQ type=del size=1491848 2024-05-30T16:04:06.951 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.951+0000 7f08851138c0 1 < completed delete operation key=E2FXSXOEUGIG tag=O9W7O91IUXHT type=del 2024-05-30T16:04:06.958 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.959+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~TO4BPNQIT17E.0 size=1048576 2024-05-30T16:04:06.960 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.959+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~TO4BPNQIT17E.1 size=84247 2024-05-30T16:04:06.961 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.959+0000 7f08851138c0 1 > prepared multipart upload key=3IJWPWRB6FHE upload=TO4BPNQIT17E tag=ZWCQ5ZKNYRUH type=write size=2181399 2024-05-30T16:04:06.961 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.963+0000 7f08851138c0 1 < canceled part key=_multipart_KPHZGFJLC2E0.2~HAVGJSI70BQ5.0 size=1048576 2024-05-30T16:04:06.964 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.963+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~HAVGJSI70BQ5.1 size=515661 2024-05-30T16:04:06.970 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.971+0000 7f08851138c0 1 < canceled multipart upload key=KPHZGFJLC2E0 upload=HAVGJSI70BQ5 tag=GNEZDMUEH6WU type=write size=2612813 2024-05-30T16:04:06.975 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.975+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=6D6XPI2PVJ1J type=write size=941527 2024-05-30T16:04:06.982 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.983+0000 7f08851138c0 1 < completed write operation key=4N58ZY4E1H6E tag=48N50FL3BLE8 type=write size=591023 2024-05-30T16:04:06.985 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.987+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=8YNV3KWFEZIR type=del size=3593279 2024-05-30T16:04:06.989 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.991+0000 7f08851138c0 1 < completed delete operation key=NIV7AQ4EV5YP tag=2N1NNLUYK92T type=del 2024-05-30T16:04:06.993 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.991+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=4NNDNL9WP97E type=write size=832267 2024-05-30T16:04:06.996 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.995+0000 7f08851138c0 1 < completed delete operation key=QCLY10D2A3VF tag=S1K63SDW1CAL type=del 2024-05-30T16:04:07.000 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:06.999+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=JVAA5TV2MBJT type=del size=151097 2024-05-30T16:04:07.003 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.003+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~R3A464YHR9UH.0 size=1048576 2024-05-30T16:04:07.007 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.007+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~R3A464YHR9UH.1 size=1048576 2024-05-30T16:04:07.009 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.011+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~R3A464YHR9UH.2 size=169008 2024-05-30T16:04:07.015 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.015+0000 7f08851138c0 1 < completed multipart upload key=GWXNCSJCLXKW upload=R3A464YHR9UH tag=F3DS9MQ2K2G5 type=write size=3314736 2024-05-30T16:04:07.022 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.023+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~I4PCTCRDCIS5.0 size=1048576 2024-05-30T16:04:07.025 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.023+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~I4PCTCRDCIS5.1 size=347101 2024-05-30T16:04:07.025 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.023+0000 7f08851138c0 1 > prepared multipart upload key=WT9RC0UYVG4F upload=I4PCTCRDCIS5 tag=U6ETD9QG6C0T type=write size=2444253 2024-05-30T16:04:07.029 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.027+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~VHU4BRRO2PQZ.0 size=1048576 2024-05-30T16:04:07.029 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.027+0000 7f08851138c0 1 < canceled part key=_multipart_Y3CYZJ83WYNI.2~VHU4BRRO2PQZ.1 size=1048576 2024-05-30T16:04:07.032 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.031+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~VHU4BRRO2PQZ.2 size=676831 2024-05-30T16:04:07.038 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.039+0000 7f08851138c0 1 < canceled multipart upload key=Y3CYZJ83WYNI upload=VHU4BRRO2PQZ tag=BM95SSG2DWBH type=write size=3822559 2024-05-30T16:04:07.042 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.043+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=8DWXO31IB0UV type=del size=896350 2024-05-30T16:04:07.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.047+0000 7f08851138c0 1 < completed part key=_multipart_1N6R2HBTWMP2.2~EHSSB16GUW19.0 size=1048576 2024-05-30T16:04:07.049 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.051+0000 7f08851138c0 1 < completed part key=_multipart_1N6R2HBTWMP2.2~EHSSB16GUW19.1 size=527702 2024-05-30T16:04:07.055 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.055+0000 7f08851138c0 1 < completed multipart upload key=1N6R2HBTWMP2 upload=EHSSB16GUW19 tag=V6P9MRFT7NW4 type=write size=2624854 2024-05-30T16:04:07.059 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.059+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=CZSJY0QB9RIK type=del size=538095 2024-05-30T16:04:07.062 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.063+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~M0F6F1LZKGTS.0 size=1048576 2024-05-30T16:04:07.065 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.067+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~M0F6F1LZKGTS.1 size=1048576 2024-05-30T16:04:07.069 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.067+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~M0F6F1LZKGTS.2 size=652843 2024-05-30T16:04:07.074 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.075+0000 7f08851138c0 1 < completed multipart upload key=3TBOZWT2P8U8 upload=M0F6F1LZKGTS tag=H384U4ADUFVT type=write size=3798571 2024-05-30T16:04:07.078 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.079+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=BTJSDZKLHTW0 type=write size=356413 2024-05-30T16:04:07.081 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.083+0000 7f08851138c0 1 < completed delete operation key=MMLF65KLC42Q tag=G4NLHM8P1DM4 type=del 2024-05-30T16:04:07.088 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.087+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~CBB0DN58VONI.0 size=1027979 2024-05-30T16:04:07.088 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.087+0000 7f08851138c0 1 > prepared multipart upload key=RCTV94WRMO53 upload=CBB0DN58VONI tag=1TAWCR0KCDD7 type=write size=2076555 2024-05-30T16:04:07.094 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.091+0000 7f08851138c0 1 < completed write operation key=JNZI7JX95LX2 tag=7NCNFF6CDBAP type=write size=744576 2024-05-30T16:04:07.095 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.095+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=JTC3LSVJ61W2 type=del size=2718209 2024-05-30T16:04:07.098 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.099+0000 7f08851138c0 1 < completed part key=_multipart_QCLY10D2A3VF.2~8KASQZ8MNJ59.0 size=1048576 2024-05-30T16:04:07.101 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.103+0000 7f08851138c0 1 < completed part key=_multipart_QCLY10D2A3VF.2~8KASQZ8MNJ59.1 size=1048576 2024-05-30T16:04:07.104 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.103+0000 7f08851138c0 1 < completed part key=_multipart_QCLY10D2A3VF.2~8KASQZ8MNJ59.2 size=71932 2024-05-30T16:04:07.110 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.111+0000 7f08851138c0 1 < completed multipart upload key=QCLY10D2A3VF upload=8KASQZ8MNJ59 tag=L0KA799QYABT type=write size=3217660 2024-05-30T16:04:07.116 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.115+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~K3R4K3FX981Y.0 size=1048576 2024-05-30T16:04:07.119 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.119+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~K3R4K3FX981Y.1 size=1048576 2024-05-30T16:04:07.122 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.123+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~K3R4K3FX981Y.2 size=397743 2024-05-30T16:04:07.122 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.123+0000 7f08851138c0 1 > prepared multipart upload key=V4OQFOOQUXFQ upload=K3R4K3FX981Y tag=6DPMMO9EEZY1 type=write size=3543471 2024-05-30T16:04:07.125 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.123+0000 7f08851138c0 1 < completed delete operation key=NIV7AQ4EV5YP tag=77INHIB9E4ZS type=del 2024-05-30T16:04:07.131 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.131+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~20SWJZ5QRVGG.0 size=1048576 2024-05-30T16:04:07.134 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.135+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~20SWJZ5QRVGG.1 size=321285 2024-05-30T16:04:07.134 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.135+0000 7f08851138c0 1 > prepared multipart upload key=TPCETCO3Y1M6 upload=20SWJZ5QRVGG tag=PNAQUQ45YB56 type=write size=2418437 2024-05-30T16:04:07.138 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.139+0000 7f08851138c0 1 < completed delete operation key=0T2E9F3K1USF tag=YBAOJ7N1X3KV type=del 2024-05-30T16:04:07.141 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.139+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=LVSZBF3MO806 type=del size=662423 2024-05-30T16:04:07.145 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.143+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~Y1K494D34XE2.0 size=1048576 2024-05-30T16:04:07.148 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.147+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~Y1K494D34XE2.1 size=1048576 2024-05-30T16:04:07.151 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.151+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~Y1K494D34XE2.2 size=814504 2024-05-30T16:04:07.157 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.159+0000 7f08851138c0 1 < completed multipart upload key=3IJWPWRB6FHE upload=Y1K494D34XE2 tag=FZ6S277TPTKB type=write size=3960232 2024-05-30T16:04:07.161 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.163+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=YVSZUXDWC6FU type=del size=649042 2024-05-30T16:04:07.165 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.163+0000 7f08851138c0 1 < completed delete operation key=VRXNO8JLAZYH tag=40UGUV3MF4NQ type=del 2024-05-30T16:04:07.169 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.167+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=1ZT0MA2Z0ZN3 type=del size=3774660 2024-05-30T16:04:07.172 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.171+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~TO4BPNQIT17E.0 size=1048576 2024-05-30T16:04:07.175 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.175+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~TO4BPNQIT17E.1 size=84247 2024-05-30T16:04:07.182 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.183+0000 7f08851138c0 1 < completed multipart upload key=3IJWPWRB6FHE upload=TO4BPNQIT17E tag=ZWCQ5ZKNYRUH type=write size=2181399 2024-05-30T16:04:07.185 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.187+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=WRIHTSS1J8ZU type=del size=3021320 2024-05-30T16:04:07.189 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.187+0000 7f08851138c0 1 < canceled operation key=V4OQFOOQUXFQ tag=6D6XPI2PVJ1J type=write size=941527 2024-05-30T16:04:07.193 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.191+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=KRE6INQ9CWC7 type=del size=2571202 2024-05-30T16:04:07.197 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.195+0000 7f08851138c0 1 < completed delete operation key=KPHZGFJLC2E0 tag=8YNV3KWFEZIR type=del 2024-05-30T16:04:07.200 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.199+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=Z9K373YGLDB2 type=del size=3224072 2024-05-30T16:04:07.204 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.203+0000 7f08851138c0 1 < completed write operation key=WT9RC0UYVG4F tag=4NNDNL9WP97E type=write size=832267 2024-05-30T16:04:07.208 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.207+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=3W96MPKBMRTA type=del size=2783243 2024-05-30T16:04:07.212 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.211+0000 7f08851138c0 1 < completed delete operation key=V4OQFOOQUXFQ tag=JVAA5TV2MBJT type=del 2024-05-30T16:04:07.218 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.219+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~Y6AYA7LUNDV2.0 size=1048576 2024-05-30T16:04:07.221 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.223+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~Y6AYA7LUNDV2.1 size=1048576 2024-05-30T16:04:07.224 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.223+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~Y6AYA7LUNDV2.2 size=40514 2024-05-30T16:04:07.224 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.223+0000 7f08851138c0 1 > prepared multipart upload key=YRP4CSG92QRO upload=Y6AYA7LUNDV2 tag=PP79BRZK2VI8 type=write size=3186242 2024-05-30T16:04:07.228 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.227+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~I4PCTCRDCIS5.0 size=1048576 2024-05-30T16:04:07.231 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.231+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~I4PCTCRDCIS5.1 size=347101 2024-05-30T16:04:07.236 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.235+0000 7f08851138c0 1 < completed multipart upload key=WT9RC0UYVG4F upload=I4PCTCRDCIS5 tag=U6ETD9QG6C0T type=write size=2444253 2024-05-30T16:04:07.243 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.243+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~AGYPE4HG28WH.0 size=197087 2024-05-30T16:04:07.243 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.243+0000 7f08851138c0 1 > prepared multipart upload key=603HBR8PRH6L upload=AGYPE4HG28WH tag=32PQAHWFHHBP type=write size=1245663 2024-05-30T16:04:07.247 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.247+0000 7f08851138c0 1 < completed delete operation key=OKBB0VU765TW tag=8DWXO31IB0UV type=del 2024-05-30T16:04:07.251 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.251+0000 7f08851138c0 1 > prepared operation key=VRXNO8JLAZYH tag=3BA0QOB752EZ type=del size=1093072 2024-05-30T16:04:07.254 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.255+0000 7f08851138c0 1 < completed delete operation key=YRP4CSG92QRO tag=CZSJY0QB9RIK type=del 2024-05-30T16:04:07.258 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.259+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=KNF6H9ZEZM8O type=del size=2915829 2024-05-30T16:04:07.266 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.267+0000 7f08851138c0 1 < suggested update operation key=Y8SBI0QZ0ZV8 tag=BTJSDZKLHTW0 type=write size=356413 2024-05-30T16:04:07.272 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.271+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~VY8FHENGJWMI.0 size=1048576 2024-05-30T16:04:07.275 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.275+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~VY8FHENGJWMI.1 size=1048576 2024-05-30T16:04:07.278 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.279+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~VY8FHENGJWMI.2 size=925274 2024-05-30T16:04:07.278 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.279+0000 7f08851138c0 1 > prepared multipart upload key=1N6R2HBTWMP2 upload=VY8FHENGJWMI tag=U5GZKUCBD52X type=write size=4071002 2024-05-30T16:04:07.278 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.279+0000 7f08851138c0 1 < canceled part key=_multipart_RCTV94WRMO53.2~CBB0DN58VONI.0 size=1027979 2024-05-30T16:04:07.284 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.283+0000 7f08851138c0 1 < canceled multipart upload key=RCTV94WRMO53 upload=CBB0DN58VONI tag=1TAWCR0KCDD7 type=write size=2076555 2024-05-30T16:04:07.288 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.287+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=1JOK8V6EB1OB type=del size=350972 2024-05-30T16:04:07.291 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.291+0000 7f08851138c0 1 < completed delete operation key=TPCETCO3Y1M6 tag=JTC3LSVJ61W2 type=del 2024-05-30T16:04:07.295 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.295+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=6N2WUKS222JQ type=del size=3369750 2024-05-30T16:04:07.298 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.299+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~K3R4K3FX981Y.0 size=1048576 2024-05-30T16:04:07.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.299+0000 7f08851138c0 1 < canceled part key=_multipart_V4OQFOOQUXFQ.2~K3R4K3FX981Y.1 size=1048576 2024-05-30T16:04:07.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.299+0000 7f08851138c0 1 < canceled part key=_multipart_V4OQFOOQUXFQ.2~K3R4K3FX981Y.2 size=397743 2024-05-30T16:04:07.304 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.303+0000 7f08851138c0 1 < canceled multipart upload key=V4OQFOOQUXFQ upload=K3R4K3FX981Y tag=6DPMMO9EEZY1 type=write size=3543471 2024-05-30T16:04:07.308 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.307+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=AUVM4T9U3FXG type=del size=71789 2024-05-30T16:04:07.312 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.311+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~20SWJZ5QRVGG.0 size=1048576 2024-05-30T16:04:07.319 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.319+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~20SWJZ5QRVGG.1 size=321285 2024-05-30T16:04:07.324 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.323+0000 7f08851138c0 1 < completed multipart upload key=TPCETCO3Y1M6 upload=20SWJZ5QRVGG tag=PNAQUQ45YB56 type=write size=2418437 2024-05-30T16:04:07.328 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.327+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=0FQ1PO3TL20I type=write size=529680 2024-05-30T16:04:07.332 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.331+0000 7f08851138c0 1 < completed delete operation key=A5BUBY9Z7JQG tag=LVSZBF3MO806 type=del 2024-05-30T16:04:07.336 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.335+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=YDIPB61YVLL3 type=del size=3280744 2024-05-30T16:04:07.339 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.339+0000 7f08851138c0 1 < completed delete operation key=73FJ3HGWYQQK tag=YVSZUXDWC6FU type=del 2024-05-30T16:04:07.346 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.347+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~RGO3LZ7RTU6V.0 size=305399 2024-05-30T16:04:07.346 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.347+0000 7f08851138c0 1 > prepared multipart upload key=V4OQFOOQUXFQ upload=RGO3LZ7RTU6V tag=D8TKBPW7C4LW type=write size=1353975 2024-05-30T16:04:07.350 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.351+0000 7f08851138c0 1 < completed delete operation key=364U3OV10VPD tag=1ZT0MA2Z0ZN3 type=del 2024-05-30T16:04:07.353 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.355+0000 7f08851138c0 1 > prepared operation key=VRXNO8JLAZYH tag=9BWZ01BZ1VMZ type=del size=978555 2024-05-30T16:04:07.357 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.359+0000 7f08851138c0 1 < completed delete operation key=JNZI7JX95LX2 tag=WRIHTSS1J8ZU type=del 2024-05-30T16:04:07.364 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.363+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~DC2UDWAXDEKR.0 size=1048576 2024-05-30T16:04:07.367 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.367+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~DC2UDWAXDEKR.1 size=312505 2024-05-30T16:04:07.367 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.367+0000 7f08851138c0 1 > prepared multipart upload key=TPCETCO3Y1M6 upload=DC2UDWAXDEKR tag=P4F98F3WHQ7Z type=write size=2409657 2024-05-30T16:04:07.371 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.371+0000 7f08851138c0 1 < completed delete operation key=QCLY10D2A3VF tag=KRE6INQ9CWC7 type=del 2024-05-30T16:04:07.377 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.375+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~N6HO3VWLWRM3.0 size=157803 2024-05-30T16:04:07.377 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.379+0000 7f08851138c0 1 > prepared multipart upload key=QQILXRJD0C1Z upload=N6HO3VWLWRM3 tag=SNG9K3E3HCT0 type=write size=1206379 2024-05-30T16:04:07.380 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.379+0000 7f08851138c0 1 < completed delete operation key=OKBB0VU765TW tag=Z9K373YGLDB2 type=del 2024-05-30T16:04:07.387 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.387+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~EQ77P2CAP2S5.0 size=1048576 2024-05-30T16:04:07.390 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.391+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~EQ77P2CAP2S5.1 size=587348 2024-05-30T16:04:07.390 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.391+0000 7f08851138c0 1 > prepared multipart upload key=0RZRBW38WKNZ upload=EQ77P2CAP2S5 tag=75UXISR1O0IR type=write size=2684500 2024-05-30T16:04:07.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.395+0000 7f08851138c0 1 < canceled operation key=O0DJ8SCY766V tag=3W96MPKBMRTA type=del size=2783243 2024-05-30T16:04:07.400 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.399+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~5J3W5R44MJFK.0 size=1048576 2024-05-30T16:04:07.403 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.403+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~5J3W5R44MJFK.1 size=1048576 2024-05-30T16:04:07.406 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.407+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~5J3W5R44MJFK.2 size=633100 2024-05-30T16:04:07.406 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.407+0000 7f08851138c0 1 > prepared multipart upload key=1N6R2HBTWMP2 upload=5J3W5R44MJFK tag=ZHRSXODI62VZ type=write size=3778828 2024-05-30T16:04:07.409 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.411+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~Y6AYA7LUNDV2.0 size=1048576 2024-05-30T16:04:07.412 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.411+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~Y6AYA7LUNDV2.1 size=1048576 2024-05-30T16:04:07.415 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.415+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~Y6AYA7LUNDV2.2 size=40514 2024-05-30T16:04:07.421 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.419+0000 7f08851138c0 1 < completed multipart upload key=YRP4CSG92QRO upload=Y6AYA7LUNDV2 tag=PP79BRZK2VI8 type=write size=3186242 2024-05-30T16:04:07.427 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.427+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~1NK9UR2F23O5.0 size=335174 2024-05-30T16:04:07.427 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.427+0000 7f08851138c0 1 > prepared multipart upload key=603HBR8PRH6L upload=1NK9UR2F23O5 tag=H6EON8BISAC8 type=write size=1383750 2024-05-30T16:04:07.431 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.431+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~AGYPE4HG28WH.0 size=197087 2024-05-30T16:04:07.437 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.435+0000 7f08851138c0 1 < completed multipart upload key=603HBR8PRH6L upload=AGYPE4HG28WH tag=32PQAHWFHHBP type=write size=1245663 2024-05-30T16:04:07.443 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.443+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~ZD5ZE4UUN02Y.0 size=301825 2024-05-30T16:04:07.443 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.443+0000 7f08851138c0 1 > prepared multipart upload key=OKBB0VU765TW upload=ZD5ZE4UUN02Y tag=3AX34HGH7UG2 type=write size=1350401 2024-05-30T16:04:07.447 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.447+0000 7f08851138c0 1 < completed delete operation key=VRXNO8JLAZYH tag=3BA0QOB752EZ type=del 2024-05-30T16:04:07.451 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.451+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=W7AMHNA8AS1T type=del size=414476 2024-05-30T16:04:07.455 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.455+0000 7f08851138c0 1 < completed delete operation key=3TBOZWT2P8U8 tag=KNF6H9ZEZM8O type=del 2024-05-30T16:04:07.462 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.463+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~Q3097T5CSG31.0 size=275727 2024-05-30T16:04:07.462 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.463+0000 7f08851138c0 1 > prepared multipart upload key=E2FXSXOEUGIG upload=Q3097T5CSG31 tag=CFGWMPSG2NFD type=write size=1324303 2024-05-30T16:04:07.466 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.467+0000 7f08851138c0 1 < completed part key=_multipart_1N6R2HBTWMP2.2~VY8FHENGJWMI.0 size=1048576 2024-05-30T16:04:07.467 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.467+0000 7f08851138c0 1 < canceled part key=_multipart_1N6R2HBTWMP2.2~VY8FHENGJWMI.1 size=1048576 2024-05-30T16:04:07.469 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.471+0000 7f08851138c0 1 < completed part key=_multipart_1N6R2HBTWMP2.2~VY8FHENGJWMI.2 size=925274 2024-05-30T16:04:07.475 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.475+0000 7f08851138c0 1 < canceled multipart upload key=1N6R2HBTWMP2 upload=VY8FHENGJWMI tag=U5GZKUCBD52X type=write size=4071002 2024-05-30T16:04:07.479 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.479+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=3Y34Y2D42QG5 type=del size=1032162 2024-05-30T16:04:07.482 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.483+0000 7f08851138c0 1 < completed delete operation key=WT9RC0UYVG4F tag=1JOK8V6EB1OB type=del 2024-05-30T16:04:07.489 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.487+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~0A3WKKG1NSK0.0 size=748200 2024-05-30T16:04:07.489 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.487+0000 7f08851138c0 1 > prepared multipart upload key=JNZI7JX95LX2 upload=0A3WKKG1NSK0 tag=WWV0NJMFMUYU type=write size=1796776 2024-05-30T16:04:07.492 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.491+0000 7f08851138c0 1 < completed delete operation key=E2FXSXOEUGIG tag=6N2WUKS222JQ type=del 2024-05-30T16:04:07.498 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.495+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=H3F34693KEI3 type=del size=1269986 2024-05-30T16:04:07.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.499+0000 7f08851138c0 1 < canceled operation key=RCTV94WRMO53 tag=AUVM4T9U3FXG type=del size=71789 2024-05-30T16:04:07.504 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.503+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=M5SHCW44FMTX type=del size=3562395 2024-05-30T16:04:07.507 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.507+0000 7f08851138c0 1 < completed write operation key=O0DJ8SCY766V tag=0FQ1PO3TL20I type=write size=529680 2024-05-30T16:04:07.511 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.511+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=YXH4K9BES55U type=del size=2048991 2024-05-30T16:04:07.515 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.515+0000 7f08851138c0 1 < completed delete operation key=XHGMUYBA2VNK tag=YDIPB61YVLL3 type=del 2024-05-30T16:04:07.521 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.519+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~1XAU0J1FACWV.0 size=1048576 2024-05-30T16:04:07.524 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.523+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~1XAU0J1FACWV.1 size=1048576 2024-05-30T16:04:07.527 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.527+0000 7f08851138c0 1 > prepared part key=_multipart_1N6R2HBTWMP2.2~1XAU0J1FACWV.2 size=847888 2024-05-30T16:04:07.527 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.527+0000 7f08851138c0 1 > prepared multipart upload key=1N6R2HBTWMP2 upload=1XAU0J1FACWV tag=I43URHOCZ5ZB type=write size=3993616 2024-05-30T16:04:07.530 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.531+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~RGO3LZ7RTU6V.0 size=305399 2024-05-30T16:04:07.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.535+0000 7f08851138c0 1 < completed multipart upload key=V4OQFOOQUXFQ upload=RGO3LZ7RTU6V tag=D8TKBPW7C4LW type=write size=1353975 2024-05-30T16:04:07.540 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.539+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=DS0TDRC6Z3QI type=del size=641666 2024-05-30T16:04:07.547 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.547+0000 7f08851138c0 1 < suggested update operation key=VRXNO8JLAZYH tag=9BWZ01BZ1VMZ type=del size=978555 2024-05-30T16:04:07.551 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.551+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=T9DBPJA2FBSU type=del size=196269 2024-05-30T16:04:07.555 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.555+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~DC2UDWAXDEKR.0 size=1048576 2024-05-30T16:04:07.558 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.559+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~DC2UDWAXDEKR.1 size=312505 2024-05-30T16:04:07.564 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.563+0000 7f08851138c0 1 < completed multipart upload key=TPCETCO3Y1M6 upload=DC2UDWAXDEKR tag=P4F98F3WHQ7Z type=write size=2409657 2024-05-30T16:04:07.568 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.567+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=0S90UJWFMY77 type=write size=283060 2024-05-30T16:04:07.568 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.567+0000 7f08851138c0 1 < canceled part key=_multipart_QQILXRJD0C1Z.2~N6HO3VWLWRM3.0 size=157803 2024-05-30T16:04:07.575 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.575+0000 7f08851138c0 1 < canceled multipart upload key=QQILXRJD0C1Z upload=N6HO3VWLWRM3 tag=SNG9K3E3HCT0 type=write size=1206379 2024-05-30T16:04:07.583 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.583+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~0Z124GYNOFMN.0 size=1048576 2024-05-30T16:04:07.586 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.587+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~0Z124GYNOFMN.1 size=980568 2024-05-30T16:04:07.586 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.587+0000 7f08851138c0 1 > prepared multipart upload key=YRP4CSG92QRO upload=0Z124GYNOFMN tag=ZHVG4KX9HIK4 type=write size=3077720 2024-05-30T16:04:07.590 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.591+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~EQ77P2CAP2S5.0 size=1048576 2024-05-30T16:04:07.592 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.591+0000 7f08851138c0 1 < completed part key=_multipart_0RZRBW38WKNZ.2~EQ77P2CAP2S5.1 size=587348 2024-05-30T16:04:07.599 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.599+0000 7f08851138c0 1 < completed multipart upload key=0RZRBW38WKNZ upload=EQ77P2CAP2S5 tag=75UXISR1O0IR type=write size=2684500 2024-05-30T16:04:07.607 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.607+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~5K3XYUJMUUTH.0 size=502552 2024-05-30T16:04:07.607 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.607+0000 7f08851138c0 1 > prepared multipart upload key=O8CWPJ9GW2GO upload=5K3XYUJMUUTH tag=ORMQ3SYXY9JF type=write size=1551128 2024-05-30T16:04:07.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.611+0000 7f08851138c0 1 < completed part key=_multipart_1N6R2HBTWMP2.2~5J3W5R44MJFK.0 size=1048576 2024-05-30T16:04:07.613 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.615+0000 7f08851138c0 1 < completed part key=_multipart_1N6R2HBTWMP2.2~5J3W5R44MJFK.1 size=1048576 2024-05-30T16:04:07.616 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.615+0000 7f08851138c0 1 < completed part key=_multipart_1N6R2HBTWMP2.2~5J3W5R44MJFK.2 size=633100 2024-05-30T16:04:07.623 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.623+0000 7f08851138c0 1 < completed multipart upload key=1N6R2HBTWMP2 upload=5J3W5R44MJFK tag=ZHRSXODI62VZ type=write size=3778828 2024-05-30T16:04:07.627 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.627+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=9D9PKK31NIWT type=del size=246059 2024-05-30T16:04:07.628 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.627+0000 7f08851138c0 1 < canceled part key=_multipart_603HBR8PRH6L.2~1NK9UR2F23O5.0 size=335174 2024-05-30T16:04:07.635 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.635+0000 7f08851138c0 1 < canceled multipart upload key=603HBR8PRH6L upload=1NK9UR2F23O5 tag=H6EON8BISAC8 type=write size=1383750 2024-05-30T16:04:07.638 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.639+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=39EDVGRGO2VV type=del size=3513011 2024-05-30T16:04:07.642 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.643+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~ZD5ZE4UUN02Y.0 size=301825 2024-05-30T16:04:07.647 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.647+0000 7f08851138c0 1 < completed multipart upload key=OKBB0VU765TW upload=ZD5ZE4UUN02Y tag=3AX34HGH7UG2 type=write size=1350401 2024-05-30T16:04:07.651 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.651+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=XE3LK5OHA1HG type=del size=1992404 2024-05-30T16:04:07.663 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.663+0000 7f08851138c0 1 < suggested update operation key=3IJWPWRB6FHE tag=W7AMHNA8AS1T type=del size=414476 2024-05-30T16:04:07.670 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.671+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~8VDZP358GAAX.0 size=1048576 2024-05-30T16:04:07.673 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.671+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~8VDZP358GAAX.1 size=932251 2024-05-30T16:04:07.673 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.671+0000 7f08851138c0 1 > prepared multipart upload key=V4OQFOOQUXFQ upload=8VDZP358GAAX tag=XLMMMEF9A29Q type=write size=3029403 2024-05-30T16:04:07.677 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.679+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~Q3097T5CSG31.0 size=275727 2024-05-30T16:04:07.683 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.683+0000 7f08851138c0 1 < completed multipart upload key=E2FXSXOEUGIG upload=Q3097T5CSG31 tag=CFGWMPSG2NFD type=write size=1324303 2024-05-30T16:04:07.687 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.687+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=DRKIZKCVNLL3 type=del size=2955303 2024-05-30T16:04:07.691 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.691+0000 7f08851138c0 1 < canceled operation key=Y8SBI0QZ0ZV8 tag=3Y34Y2D42QG5 type=del size=1032162 2024-05-30T16:04:07.698 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.699+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~94DLJVGXQ83M.0 size=558370 2024-05-30T16:04:07.698 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.699+0000 7f08851138c0 1 > prepared multipart upload key=PLWFSKPKVO19 upload=94DLJVGXQ83M tag=ZG0A250LWPAQ type=write size=1606946 2024-05-30T16:04:07.702 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.703+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~0A3WKKG1NSK0.0 size=748200 2024-05-30T16:04:07.708 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.707+0000 7f08851138c0 1 < completed multipart upload key=JNZI7JX95LX2 upload=0A3WKKG1NSK0 tag=WWV0NJMFMUYU type=write size=1796776 2024-05-30T16:04:07.711 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.711+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=57P1WAI2BUJ0 type=del size=1007064 2024-05-30T16:04:07.715 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.715+0000 7f08851138c0 1 < completed delete operation key=1N6R2HBTWMP2 tag=H3F34693KEI3 type=del 2024-05-30T16:04:07.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.719+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=NAO7HLEIE3NB type=del size=3083505 2024-05-30T16:04:07.722 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.723+0000 7f08851138c0 1 < completed delete operation key=MMLF65KLC42Q tag=M5SHCW44FMTX type=del 2024-05-30T16:04:07.725 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.723+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=X8EOOHY63KJY type=write size=988315 2024-05-30T16:04:07.728 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.727+0000 7f08851138c0 1 < completed delete operation key=0T2E9F3K1USF tag=YXH4K9BES55U type=del 2024-05-30T16:04:07.735 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.735+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~3P7E1OBC8NAL.0 size=1048576 2024-05-30T16:04:07.737 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.739+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~3P7E1OBC8NAL.1 size=1048576 2024-05-30T16:04:07.740 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.739+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~3P7E1OBC8NAL.2 size=45713 2024-05-30T16:04:07.740 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.739+0000 7f08851138c0 1 > prepared multipart upload key=V4OQFOOQUXFQ upload=3P7E1OBC8NAL tag=5O3SK6ADIICC type=write size=3191441 2024-05-30T16:04:07.744 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.743+0000 7f08851138c0 1 < completed part key=_multipart_1N6R2HBTWMP2.2~1XAU0J1FACWV.0 size=1048576 2024-05-30T16:04:07.746 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.747+0000 7f08851138c0 1 < completed part key=_multipart_1N6R2HBTWMP2.2~1XAU0J1FACWV.1 size=1048576 2024-05-30T16:04:07.750 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.751+0000 7f08851138c0 1 < completed part key=_multipart_1N6R2HBTWMP2.2~1XAU0J1FACWV.2 size=847888 2024-05-30T16:04:07.755 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.755+0000 7f08851138c0 1 < completed multipart upload key=1N6R2HBTWMP2 upload=1XAU0J1FACWV tag=I43URHOCZ5ZB type=write size=3993616 2024-05-30T16:04:07.758 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.759+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=4H5ZW5AOTZU6 type=del size=3540490 2024-05-30T16:04:07.762 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.763+0000 7f08851138c0 1 < completed delete operation key=3TBOZWT2P8U8 tag=DS0TDRC6Z3QI type=del 2024-05-30T16:04:07.767 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.767+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~91GH3M3J7ND2.0 size=1048576 2024-05-30T16:04:07.770 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.771+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~91GH3M3J7ND2.1 size=1048576 2024-05-30T16:04:07.772 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.771+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~91GH3M3J7ND2.2 size=873686 2024-05-30T16:04:07.773 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.771+0000 7f08851138c0 1 > prepared multipart upload key=JNZI7JX95LX2 upload=91GH3M3J7ND2 tag=WD6AVMJIQG1D type=write size=4019414 2024-05-30T16:04:07.776 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.775+0000 7f08851138c0 1 < completed delete operation key=364U3OV10VPD tag=T9DBPJA2FBSU type=del 2024-05-30T16:04:07.781 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.783+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=CQKZA6PKBQMP type=del size=278155 2024-05-30T16:04:07.785 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.783+0000 7f08851138c0 1 < completed write operation key=QCLY10D2A3VF tag=0S90UJWFMY77 type=write size=283060 2024-05-30T16:04:07.788 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.787+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=SYFI3H7L64RW type=del size=335005 2024-05-30T16:04:07.792 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.791+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~0Z124GYNOFMN.0 size=1048576 2024-05-30T16:04:07.795 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.795+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~0Z124GYNOFMN.1 size=980568 2024-05-30T16:04:07.801 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.799+0000 7f08851138c0 1 < completed multipart upload key=YRP4CSG92QRO upload=0Z124GYNOFMN tag=ZHVG4KX9HIK4 type=write size=3077720 2024-05-30T16:04:07.807 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.807+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~1XXZ5TYJCO0S.0 size=519277 2024-05-30T16:04:07.807 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.807+0000 7f08851138c0 1 > prepared multipart upload key=JNZI7JX95LX2 upload=1XXZ5TYJCO0S tag=C53JSMOLSXEI type=write size=1567853 2024-05-30T16:04:07.812 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.811+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~5K3XYUJMUUTH.0 size=502552 2024-05-30T16:04:07.817 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.819+0000 7f08851138c0 1 < completed multipart upload key=O8CWPJ9GW2GO upload=5K3XYUJMUUTH tag=ORMQ3SYXY9JF type=write size=1551128 2024-05-30T16:04:07.821 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.819+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=S9Z53QZTAJ2Q type=del size=786940 2024-05-30T16:04:07.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.827+0000 7f08851138c0 1 < suggested update operation key=RCTV94WRMO53 tag=9D9PKK31NIWT type=del size=246059 2024-05-30T16:04:07.834 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.835+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~B84CD0Q2ZJOH.0 size=666182 2024-05-30T16:04:07.834 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.835+0000 7f08851138c0 1 > prepared multipart upload key=603HBR8PRH6L upload=B84CD0Q2ZJOH tag=MQ0TX8EQ6TI3 type=write size=1714758 2024-05-30T16:04:07.838 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.839+0000 7f08851138c0 1 < completed delete operation key=KPHZGFJLC2E0 tag=39EDVGRGO2VV type=del 2024-05-30T16:04:07.844 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.843+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~0WZ6E5KJ8CX2.0 size=1048576 2024-05-30T16:04:07.847 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.847+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~0WZ6E5KJ8CX2.1 size=110205 2024-05-30T16:04:07.847 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.847+0000 7f08851138c0 1 > prepared multipart upload key=JNZI7JX95LX2 upload=0WZ6E5KJ8CX2 tag=687M1XRLIYAH type=write size=2207357 2024-05-30T16:04:07.851 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.851+0000 7f08851138c0 1 < completed delete operation key=OKBB0VU765TW tag=XE3LK5OHA1HG type=del 2024-05-30T16:04:07.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.855+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=UG42PHK8I9BI type=del size=1980380 2024-05-30T16:04:07.856 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.855+0000 7f08851138c0 1 < canceled part key=_multipart_V4OQFOOQUXFQ.2~8VDZP358GAAX.0 size=1048576 2024-05-30T16:04:07.859 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.859+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~8VDZP358GAAX.1 size=932251 2024-05-30T16:04:07.865 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.863+0000 7f08851138c0 1 < canceled multipart upload key=V4OQFOOQUXFQ upload=8VDZP358GAAX tag=XLMMMEF9A29Q type=write size=3029403 2024-05-30T16:04:07.871 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.871+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~42HDN214O0V9.0 size=1048576 2024-05-30T16:04:07.874 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.875+0000 7f08851138c0 1 > prepared part key=_multipart_VRXNO8JLAZYH.2~42HDN214O0V9.1 size=549281 2024-05-30T16:04:07.874 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.875+0000 7f08851138c0 1 > prepared multipart upload key=VRXNO8JLAZYH upload=42HDN214O0V9 tag=ZF50DYLU5LI2 type=write size=2646433 2024-05-30T16:04:07.878 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.879+0000 7f08851138c0 1 < completed delete operation key=QCLY10D2A3VF tag=DRKIZKCVNLL3 type=del 2024-05-30T16:04:07.881 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.879+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=TKXWB8DHK16R type=del size=3600172 2024-05-30T16:04:07.882 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.883+0000 7f08851138c0 1 < canceled part key=_multipart_PLWFSKPKVO19.2~94DLJVGXQ83M.0 size=558370 2024-05-30T16:04:07.888 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.887+0000 7f08851138c0 1 < canceled multipart upload key=PLWFSKPKVO19 upload=94DLJVGXQ83M tag=ZG0A250LWPAQ type=write size=1606946 2024-05-30T16:04:07.895 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.895+0000 7f08851138c0 1 > prepared part key=_multipart_4N58ZY4E1H6E.2~ZBWTT0J7VQHY.0 size=290433 2024-05-30T16:04:07.895 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.895+0000 7f08851138c0 1 > prepared multipart upload key=4N58ZY4E1H6E upload=ZBWTT0J7VQHY tag=XID8BI22AQYT type=write size=1339009 2024-05-30T16:04:07.899 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.899+0000 7f08851138c0 1 < canceled operation key=QQILXRJD0C1Z tag=57P1WAI2BUJ0 type=del size=1007064 2024-05-30T16:04:07.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.907+0000 7f08851138c0 1 > prepared part key=_multipart_73FJ3HGWYQQK.2~OXXN2OP8ACK0.0 size=889382 2024-05-30T16:04:07.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.907+0000 7f08851138c0 1 > prepared multipart upload key=73FJ3HGWYQQK upload=OXXN2OP8ACK0 tag=4KC2VTJ1434Y type=write size=1937958 2024-05-30T16:04:07.910 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.911+0000 7f08851138c0 1 < completed delete operation key=0T2E9F3K1USF tag=NAO7HLEIE3NB type=del 2024-05-30T16:04:07.917 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.915+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~09BCNAXPPW9G.0 size=1048576 2024-05-30T16:04:07.920 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.919+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~09BCNAXPPW9G.1 size=1048576 2024-05-30T16:04:07.923 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.923+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~09BCNAXPPW9G.2 size=579391 2024-05-30T16:04:07.923 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.923+0000 7f08851138c0 1 > prepared multipart upload key=3TBOZWT2P8U8 upload=09BCNAXPPW9G tag=U9EW9SAQKBTE type=write size=3725119 2024-05-30T16:04:07.927 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.927+0000 7f08851138c0 1 < completed write operation key=OKBB0VU765TW tag=X8EOOHY63KJY type=write size=988315 2024-05-30T16:04:07.930 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.931+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=JEW1J0E6BJE1 type=write size=1017242 2024-05-30T16:04:07.934 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.935+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~3P7E1OBC8NAL.0 size=1048576 2024-05-30T16:04:07.937 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.939+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~3P7E1OBC8NAL.1 size=1048576 2024-05-30T16:04:07.938 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.939+0000 7f08851138c0 1 < canceled part key=_multipart_V4OQFOOQUXFQ.2~3P7E1OBC8NAL.2 size=45713 2024-05-30T16:04:07.944 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.943+0000 7f08851138c0 1 < canceled multipart upload key=V4OQFOOQUXFQ upload=3P7E1OBC8NAL tag=5O3SK6ADIICC type=write size=3191441 2024-05-30T16:04:07.948 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.947+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=9M50H0MFE5VL type=del size=3912974 2024-05-30T16:04:07.952 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.951+0000 7f08851138c0 1 < canceled operation key=A5BUBY9Z7JQG tag=4H5ZW5AOTZU6 type=del size=3540490 2024-05-30T16:04:07.956 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.955+0000 7f08851138c0 1 > prepared operation key=VRXNO8JLAZYH tag=LVYNALXN2XTD type=del size=1157461 2024-05-30T16:04:07.956 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.955+0000 7f08851138c0 1 < canceled part key=_multipart_JNZI7JX95LX2.2~91GH3M3J7ND2.0 size=1048576 2024-05-30T16:04:07.956 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.955+0000 7f08851138c0 1 < canceled part key=_multipart_JNZI7JX95LX2.2~91GH3M3J7ND2.1 size=1048576 2024-05-30T16:04:07.959 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.959+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~91GH3M3J7ND2.2 size=873686 2024-05-30T16:04:07.965 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.967+0000 7f08851138c0 1 < canceled multipart upload key=JNZI7JX95LX2 upload=91GH3M3J7ND2 tag=WD6AVMJIQG1D type=write size=4019414 2024-05-30T16:04:07.969 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.967+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=9DXRC60ONF3S type=del size=2232849 2024-05-30T16:04:07.972 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.971+0000 7f08851138c0 1 < completed delete operation key=73FJ3HGWYQQK tag=CQKZA6PKBQMP type=del 2024-05-30T16:04:07.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.979+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~AU4WT9N40J6H.0 size=1048576 2024-05-30T16:04:07.982 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.983+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~AU4WT9N40J6H.1 size=1048576 2024-05-30T16:04:07.985 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.983+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~AU4WT9N40J6H.2 size=808338 2024-05-30T16:04:07.985 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.983+0000 7f08851138c0 1 > prepared multipart upload key=V4OQFOOQUXFQ upload=AU4WT9N40J6H tag=CRXMEL6ZF78Q type=write size=3954066 2024-05-30T16:04:07.988 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.987+0000 7f08851138c0 1 < completed delete operation key=GWXNCSJCLXKW tag=SYFI3H7L64RW type=del 2024-05-30T16:04:07.992 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.991+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=FZUAVIE6REWV type=del size=4179752 2024-05-30T16:04:07.995 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:07.995+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~1XXZ5TYJCO0S.0 size=519277 2024-05-30T16:04:08.005 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.003+0000 7f08851138c0 1 < completed multipart upload key=JNZI7JX95LX2 upload=1XXZ5TYJCO0S tag=C53JSMOLSXEI type=write size=1567853 2024-05-30T16:04:08.011 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.011+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~S4LS9QGAY3AO.0 size=1048576 2024-05-30T16:04:08.014 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.015+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~S4LS9QGAY3AO.1 size=846937 2024-05-30T16:04:08.014 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.015+0000 7f08851138c0 1 > prepared multipart upload key=RCTV94WRMO53 upload=S4LS9QGAY3AO tag=49ZLM271FIUD type=write size=2944089 2024-05-30T16:04:08.022 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.023+0000 7f08851138c0 1 < suggested remove operation key=Y8SBI0QZ0ZV8 tag=S9Z53QZTAJ2Q type=del 2024-05-30T16:04:08.028 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.027+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~ELPCGUHGYT7E.0 size=1048576 2024-05-30T16:04:08.031 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.031+0000 7f08851138c0 1 > prepared part key=_multipart_ZALVY70ISLM5.2~ELPCGUHGYT7E.1 size=737333 2024-05-30T16:04:08.031 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.031+0000 7f08851138c0 1 > prepared multipart upload key=ZALVY70ISLM5 upload=ELPCGUHGYT7E tag=36DJT5LZTRWL type=write size=2834485 2024-05-30T16:04:08.034 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.035+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~B84CD0Q2ZJOH.0 size=666182 2024-05-30T16:04:08.040 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.039+0000 7f08851138c0 1 < completed multipart upload key=603HBR8PRH6L upload=B84CD0Q2ZJOH tag=MQ0TX8EQ6TI3 type=write size=1714758 2024-05-30T16:04:08.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.043+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=YTR147OT1F0O type=del size=1882104 2024-05-30T16:04:08.045 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.043+0000 7f08851138c0 1 < canceled part key=_multipart_JNZI7JX95LX2.2~0WZ6E5KJ8CX2.0 size=1048576 2024-05-30T16:04:08.048 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.047+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~0WZ6E5KJ8CX2.1 size=110205 2024-05-30T16:04:08.054 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.055+0000 7f08851138c0 1 < canceled multipart upload key=JNZI7JX95LX2 upload=0WZ6E5KJ8CX2 tag=687M1XRLIYAH type=write size=2207357 2024-05-30T16:04:08.058 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.059+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=A0HAV18S3W41 type=del size=1779492 2024-05-30T16:04:08.062 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.063+0000 7f08851138c0 1 < completed delete operation key=XHGMUYBA2VNK tag=UG42PHK8I9BI type=del 2024-05-30T16:04:08.069 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.071+0000 7f08851138c0 1 > prepared part key=_multipart_QCLY10D2A3VF.2~OFGXQSKG5JIA.0 size=969077 2024-05-30T16:04:08.070 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.071+0000 7f08851138c0 1 > prepared multipart upload key=QCLY10D2A3VF upload=OFGXQSKG5JIA tag=QIAIP5OQ1UG6 type=write size=2017653 2024-05-30T16:04:08.073 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.075+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~42HDN214O0V9.0 size=1048576 2024-05-30T16:04:08.076 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.075+0000 7f08851138c0 1 < completed part key=_multipart_VRXNO8JLAZYH.2~42HDN214O0V9.1 size=549281 2024-05-30T16:04:08.083 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.083+0000 7f08851138c0 1 < completed multipart upload key=VRXNO8JLAZYH upload=42HDN214O0V9 tag=ZF50DYLU5LI2 type=write size=2646433 2024-05-30T16:04:08.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.091+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~USJXZFCCZACX.0 size=1048576 2024-05-30T16:04:08.093 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.091+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~USJXZFCCZACX.1 size=1048576 2024-05-30T16:04:08.096 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.095+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~USJXZFCCZACX.2 size=466561 2024-05-30T16:04:08.096 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.095+0000 7f08851138c0 1 > prepared multipart upload key=PLWFSKPKVO19 upload=USJXZFCCZACX tag=287X82Q4AIAN type=write size=3612289 2024-05-30T16:04:08.099 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.099+0000 7f08851138c0 1 < completed delete operation key=Y3CYZJ83WYNI tag=TKXWB8DHK16R type=del 2024-05-30T16:04:08.103 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.103+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=UKB33ZAV27IK type=del size=3795743 2024-05-30T16:04:08.108 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.107+0000 7f08851138c0 1 < completed part key=_multipart_4N58ZY4E1H6E.2~ZBWTT0J7VQHY.0 size=290433 2024-05-30T16:04:08.113 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.115+0000 7f08851138c0 1 < completed multipart upload key=4N58ZY4E1H6E upload=ZBWTT0J7VQHY tag=XID8BI22AQYT type=write size=1339009 2024-05-30T16:04:08.120 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.119+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~TPQEMSVTY34H.0 size=888569 2024-05-30T16:04:08.120 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.119+0000 7f08851138c0 1 > prepared multipart upload key=Y8SBI0QZ0ZV8 upload=TPQEMSVTY34H tag=82JXD72DZ7CM type=write size=1937145 2024-05-30T16:04:08.124 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.123+0000 7f08851138c0 1 < completed part key=_multipart_73FJ3HGWYQQK.2~OXXN2OP8ACK0.0 size=889382 2024-05-30T16:04:08.131 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.131+0000 7f08851138c0 1 < completed multipart upload key=73FJ3HGWYQQK upload=OXXN2OP8ACK0 tag=4KC2VTJ1434Y type=write size=1937958 2024-05-30T16:04:08.139 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.139+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~64C6WT3MTH9L.0 size=1048576 2024-05-30T16:04:08.142 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.143+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~64C6WT3MTH9L.1 size=1048576 2024-05-30T16:04:08.145 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.143+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~64C6WT3MTH9L.2 size=505272 2024-05-30T16:04:08.145 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.143+0000 7f08851138c0 1 > prepared multipart upload key=3IJWPWRB6FHE upload=64C6WT3MTH9L tag=D3D5P0U1I1Q5 type=write size=3651000 2024-05-30T16:04:08.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.147+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~09BCNAXPPW9G.0 size=1048576 2024-05-30T16:04:08.152 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.151+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~09BCNAXPPW9G.1 size=1048576 2024-05-30T16:04:08.155 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.155+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~09BCNAXPPW9G.2 size=579391 2024-05-30T16:04:08.162 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.163+0000 7f08851138c0 1 < completed multipart upload key=3TBOZWT2P8U8 upload=09BCNAXPPW9G tag=U9EW9SAQKBTE type=write size=3725119 2024-05-30T16:04:08.166 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.167+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=LAQYUYLBV30S type=del size=894664 2024-05-30T16:04:08.170 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.171+0000 7f08851138c0 1 < canceled operation key=WT9RC0UYVG4F tag=JEW1J0E6BJE1 type=write size=1017242 2024-05-30T16:04:08.174 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.175+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=3EG0B8FJE5TU type=del size=1146346 2024-05-30T16:04:08.178 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.179+0000 7f08851138c0 -1 < failed to complete operation key=Y8SBI0QZ0ZV8 tag=9M50H0MFE5VL type=del size=3912974: (22) Invalid argument 2024-05-30T16:04:08.181 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.183+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=0BPSIK82O7JC type=del size=1491734 2024-05-30T16:04:08.185 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.183+0000 7f08851138c0 1 < completed delete operation key=VRXNO8JLAZYH tag=LVYNALXN2XTD type=del 2024-05-30T16:04:08.194 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.191+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~CBSEUJIS21QV.0 size=1048576 2024-05-30T16:04:08.196 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.195+0000 7f08851138c0 1 > prepared part key=_multipart_V4OQFOOQUXFQ.2~CBSEUJIS21QV.1 size=777513 2024-05-30T16:04:08.196 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.195+0000 7f08851138c0 1 > prepared multipart upload key=V4OQFOOQUXFQ upload=CBSEUJIS21QV tag=OYP2MMZF9VFF type=write size=2874665 2024-05-30T16:04:08.200 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.199+0000 7f08851138c0 1 < completed delete operation key=JNZI7JX95LX2 tag=9DXRC60ONF3S type=del 2024-05-30T16:04:08.204 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.203+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=YCJM6FT4Y9IN type=del size=448398 2024-05-30T16:04:08.208 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.207+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~AU4WT9N40J6H.0 size=1048576 2024-05-30T16:04:08.211 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.211+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~AU4WT9N40J6H.1 size=1048576 2024-05-30T16:04:08.214 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.215+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~AU4WT9N40J6H.2 size=808338 2024-05-30T16:04:08.220 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.219+0000 7f08851138c0 1 < completed multipart upload key=V4OQFOOQUXFQ upload=AU4WT9N40J6H tag=CRXMEL6ZF78Q type=write size=3954066 2024-05-30T16:04:08.223 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.223+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=SV8PPD2AHZ14 type=del size=2567870 2024-05-30T16:04:08.227 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.227+0000 7f08851138c0 1 < completed delete operation key=XHGMUYBA2VNK tag=FZUAVIE6REWV type=del 2024-05-30T16:04:08.231 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.231+0000 7f08851138c0 1 > prepared operation key=Y3CYZJ83WYNI tag=0DZI9GISVTK8 type=del size=3880225 2024-05-30T16:04:08.231 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.231+0000 7f08851138c0 1 < canceled part key=_multipart_RCTV94WRMO53.2~S4LS9QGAY3AO.0 size=1048576 2024-05-30T16:04:08.234 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.235+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~S4LS9QGAY3AO.1 size=846937 2024-05-30T16:04:08.241 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.239+0000 7f08851138c0 1 < canceled multipart upload key=RCTV94WRMO53 upload=S4LS9QGAY3AO tag=49ZLM271FIUD type=write size=2944089 2024-05-30T16:04:08.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.243+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=92HM8K5N23HD type=del size=3094428 2024-05-30T16:04:08.248 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.247+0000 7f08851138c0 1 < completed part key=_multipart_ZALVY70ISLM5.2~ELPCGUHGYT7E.0 size=1048576 2024-05-30T16:04:08.248 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.247+0000 7f08851138c0 1 < canceled part key=_multipart_ZALVY70ISLM5.2~ELPCGUHGYT7E.1 size=737333 2024-05-30T16:04:08.255 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.255+0000 7f08851138c0 1 < canceled multipart upload key=ZALVY70ISLM5 upload=ELPCGUHGYT7E tag=36DJT5LZTRWL type=write size=2834485 2024-05-30T16:04:08.258 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.259+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=Y198MXL88LWA type=write size=100176 2024-05-30T16:04:08.262 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.263+0000 7f08851138c0 1 < completed delete operation key=3TBOZWT2P8U8 tag=YTR147OT1F0O type=del 2024-05-30T16:04:08.266 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.267+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=YN9A9BH84Z06 type=del size=2831635 2024-05-30T16:04:08.270 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.271+0000 7f08851138c0 1 < completed delete operation key=PLWFSKPKVO19 tag=A0HAV18S3W41 type=del 2024-05-30T16:04:08.273 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.271+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=GZU92VODRCQI type=write size=300850 2024-05-30T16:04:08.277 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.275+0000 7f08851138c0 1 < completed part key=_multipart_QCLY10D2A3VF.2~OFGXQSKG5JIA.0 size=969077 2024-05-30T16:04:08.282 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.283+0000 7f08851138c0 1 < completed multipart upload key=QCLY10D2A3VF upload=OFGXQSKG5JIA tag=QIAIP5OQ1UG6 type=write size=2017653 2024-05-30T16:04:08.286 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.287+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=3S3Q8D2NFXDO type=del size=1158256 2024-05-30T16:04:08.290 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.291+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~USJXZFCCZACX.0 size=1048576 2024-05-30T16:04:08.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.291+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~USJXZFCCZACX.1 size=1048576 2024-05-30T16:04:08.296 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.295+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~USJXZFCCZACX.2 size=466561 2024-05-30T16:04:08.301 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.303+0000 7f08851138c0 1 < completed multipart upload key=PLWFSKPKVO19 upload=USJXZFCCZACX tag=287X82Q4AIAN type=write size=3612289 2024-05-30T16:04:08.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.303+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=KAT5AUIG8X2D type=write size=992064 2024-05-30T16:04:08.309 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.307+0000 7f08851138c0 1 < completed delete operation key=3TBOZWT2P8U8 tag=UKB33ZAV27IK type=del 2024-05-30T16:04:08.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.315+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~JIO69FL9JNIC.0 size=715559 2024-05-30T16:04:08.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.315+0000 7f08851138c0 1 > prepared multipart upload key=TPCETCO3Y1M6 upload=JIO69FL9JNIC tag=0ZUKP3YV2I9D type=write size=1764135 2024-05-30T16:04:08.319 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.319+0000 7f08851138c0 1 < completed part key=_multipart_Y8SBI0QZ0ZV8.2~TPQEMSVTY34H.0 size=888569 2024-05-30T16:04:08.325 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.323+0000 7f08851138c0 1 < completed multipart upload key=Y8SBI0QZ0ZV8 upload=TPQEMSVTY34H tag=82JXD72DZ7CM type=write size=1937145 2024-05-30T16:04:08.331 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.331+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~B3MI4Q455KC0.0 size=716948 2024-05-30T16:04:08.331 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.331+0000 7f08851138c0 1 > prepared multipart upload key=QQILXRJD0C1Z upload=B3MI4Q455KC0 tag=3ZZNN11XNM8W type=write size=1765524 2024-05-30T16:04:08.334 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.335+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~64C6WT3MTH9L.0 size=1048576 2024-05-30T16:04:08.337 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.339+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~64C6WT3MTH9L.1 size=1048576 2024-05-30T16:04:08.340 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.339+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~64C6WT3MTH9L.2 size=505272 2024-05-30T16:04:08.349 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.351+0000 7f08851138c0 1 < completed multipart upload key=3IJWPWRB6FHE upload=64C6WT3MTH9L tag=D3D5P0U1I1Q5 type=write size=3651000 2024-05-30T16:04:08.353 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.355+0000 7f08851138c0 1 > prepared operation key=XHGMUYBA2VNK tag=FMNI1X5CT3RW type=del size=3992103 2024-05-30T16:04:08.357 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.355+0000 7f08851138c0 1 < completed delete operation key=TPCETCO3Y1M6 tag=LAQYUYLBV30S type=del 2024-05-30T16:04:08.364 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.363+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~TMWLXMSHKNVN.0 size=1048576 2024-05-30T16:04:08.366 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.367+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~TMWLXMSHKNVN.1 size=1048576 2024-05-30T16:04:08.369 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.367+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~TMWLXMSHKNVN.2 size=954768 2024-05-30T16:04:08.369 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.371+0000 7f08851138c0 1 > prepared multipart upload key=364U3OV10VPD upload=TMWLXMSHKNVN tag=MF1E7FPPBO4Z type=write size=4100496 2024-05-30T16:04:08.373 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.371+0000 7f08851138c0 1 < canceled operation key=QQILXRJD0C1Z tag=3EG0B8FJE5TU type=del size=1146346 2024-05-30T16:04:08.376 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.375+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=W4I8SD5OAT0K type=del size=332600 2024-05-30T16:04:08.383 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.383+0000 7f08851138c0 1 < suggested update operation key=JNZI7JX95LX2 tag=0BPSIK82O7JC type=del size=1491734 2024-05-30T16:04:08.387 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.387+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=KEACPNITCWM7 type=del size=2283992 2024-05-30T16:04:08.390 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.391+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~CBSEUJIS21QV.0 size=1048576 2024-05-30T16:04:08.393 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.395+0000 7f08851138c0 1 < completed part key=_multipart_V4OQFOOQUXFQ.2~CBSEUJIS21QV.1 size=777513 2024-05-30T16:04:08.399 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.399+0000 7f08851138c0 1 < completed multipart upload key=V4OQFOOQUXFQ upload=CBSEUJIS21QV tag=OYP2MMZF9VFF type=write size=2874665 2024-05-30T16:04:08.403 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.403+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=SU1LDESZD97O type=del size=3135800 2024-05-30T16:04:08.407 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.407+0000 7f08851138c0 1 < completed delete operation key=1N6R2HBTWMP2 tag=YCJM6FT4Y9IN type=del 2024-05-30T16:04:08.413 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.411+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~TJVXXS7KSR6A.0 size=1048576 2024-05-30T16:04:08.415 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.415+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~TJVXXS7KSR6A.1 size=1048576 2024-05-30T16:04:08.418 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.419+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~TJVXXS7KSR6A.2 size=909118 2024-05-30T16:04:08.418 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.419+0000 7f08851138c0 1 > prepared multipart upload key=QQILXRJD0C1Z upload=TJVXXS7KSR6A tag=G3W9O743T520 type=write size=4054846 2024-05-30T16:04:08.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.423+0000 7f08851138c0 1 < completed delete operation key=WT9RC0UYVG4F tag=SV8PPD2AHZ14 type=del 2024-05-30T16:04:08.425 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.423+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=JWAG0GCJIX4E type=del size=3953839 2024-05-30T16:04:08.428 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.427+0000 7f08851138c0 1 < completed delete operation key=Y3CYZJ83WYNI tag=0DZI9GISVTK8 type=del 2024-05-30T16:04:08.431 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.431+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=TNIFRFWKSQDD type=del size=600845 2024-05-30T16:04:08.434 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.435+0000 7f08851138c0 1 < completed delete operation key=MMLF65KLC42Q tag=92HM8K5N23HD type=del 2024-05-30T16:04:08.440 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.439+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~1QPZB2KM3CQB.0 size=228086 2024-05-30T16:04:08.440 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.439+0000 7f08851138c0 1 > prepared multipart upload key=TPCETCO3Y1M6 upload=1QPZB2KM3CQB tag=01Y48GTMM2FW type=write size=1276662 2024-05-30T16:04:08.444 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.443+0000 7f08851138c0 1 < completed write operation key=73FJ3HGWYQQK tag=Y198MXL88LWA type=write size=100176 2024-05-30T16:04:08.447 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.447+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=C7ENQF473OCB type=write size=720727 2024-05-30T16:04:08.450 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.451+0000 7f08851138c0 1 < completed delete operation key=4N58ZY4E1H6E tag=YN9A9BH84Z06 type=del 2024-05-30T16:04:08.454 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.455+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=ORU2QG5QIDTY type=del size=1613821 2024-05-30T16:04:08.461 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.459+0000 7f08851138c0 1 < suggested update operation key=E2FXSXOEUGIG tag=GZU92VODRCQI type=write size=300850 2024-05-30T16:04:08.464 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.463+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=6AGNONH8SUXN type=del size=3613447 2024-05-30T16:04:08.471 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.471+0000 7f08851138c0 1 < suggested update operation key=3TBOZWT2P8U8 tag=3S3Q8D2NFXDO type=del size=1158256 2024-05-30T16:04:08.474 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.475+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=2JLN6E6Z021D type=write size=377234 2024-05-30T16:04:08.477 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.479+0000 7f08851138c0 -1 < failed to complete operation key=JNZI7JX95LX2 tag=KAT5AUIG8X2D type=write size=992064: (22) Invalid argument 2024-05-30T16:04:08.481 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.479+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=3SL7JUS6DDL9 type=del size=1484181 2024-05-30T16:04:08.485 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.483+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~JIO69FL9JNIC.0 size=715559 2024-05-30T16:04:08.490 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.491+0000 7f08851138c0 1 < completed multipart upload key=TPCETCO3Y1M6 upload=JIO69FL9JNIC tag=0ZUKP3YV2I9D type=write size=1764135 2024-05-30T16:04:08.494 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.495+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=WO42YRQNCKN2 type=del size=3740648 2024-05-30T16:04:08.498 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.499+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~B3MI4Q455KC0.0 size=716948 2024-05-30T16:04:08.503 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.503+0000 7f08851138c0 1 < completed multipart upload key=QQILXRJD0C1Z upload=B3MI4Q455KC0 tag=3ZZNN11XNM8W type=write size=1765524 2024-05-30T16:04:08.509 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.511+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~GFU193I39UEF.0 size=1048576 2024-05-30T16:04:08.512 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.511+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~GFU193I39UEF.1 size=1048576 2024-05-30T16:04:08.515 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.515+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~GFU193I39UEF.2 size=115548 2024-05-30T16:04:08.515 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.515+0000 7f08851138c0 1 > prepared multipart upload key=XHGMUYBA2VNK upload=GFU193I39UEF tag=OO4E9JAKLXF4 type=write size=3261276 2024-05-30T16:04:08.518 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.519+0000 7f08851138c0 1 < completed delete operation key=XHGMUYBA2VNK tag=FMNI1X5CT3RW type=del 2024-05-30T16:04:08.525 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.523+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~0J72M9MDHQ94.0 size=1048576 2024-05-30T16:04:08.527 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.527+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~0J72M9MDHQ94.1 size=453184 2024-05-30T16:04:08.527 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.527+0000 7f08851138c0 1 > prepared multipart upload key=YRP4CSG92QRO upload=0J72M9MDHQ94 tag=E0L05QTE0527 type=write size=2550336 2024-05-30T16:04:08.531 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.531+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~TMWLXMSHKNVN.0 size=1048576 2024-05-30T16:04:08.534 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.535+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~TMWLXMSHKNVN.1 size=1048576 2024-05-30T16:04:08.537 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.539+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~TMWLXMSHKNVN.2 size=954768 2024-05-30T16:04:08.544 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.543+0000 7f08851138c0 1 < completed multipart upload key=364U3OV10VPD upload=TMWLXMSHKNVN tag=MF1E7FPPBO4Z type=write size=4100496 2024-05-30T16:04:08.549 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.547+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=YE7U1YV4QSSM type=write size=498077 2024-05-30T16:04:08.552 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.551+0000 7f08851138c0 1 < completed delete operation key=WT9RC0UYVG4F tag=W4I8SD5OAT0K type=del 2024-05-30T16:04:08.556 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.555+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=TBQ11TIWSMIL type=del size=2853706 2024-05-30T16:04:08.560 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.559+0000 7f08851138c0 1 < completed delete operation key=603HBR8PRH6L tag=KEACPNITCWM7 type=del 2024-05-30T16:04:08.564 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.563+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=VJEG1L73GO0X type=del size=2582157 2024-05-30T16:04:08.567 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.567+0000 7f08851138c0 1 < completed delete operation key=0RZRBW38WKNZ tag=SU1LDESZD97O type=del 2024-05-30T16:04:08.571 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.571+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=G078QC2D4XIY type=del size=3642802 2024-05-30T16:04:08.575 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.575+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~TJVXXS7KSR6A.0 size=1048576 2024-05-30T16:04:08.578 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.579+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~TJVXXS7KSR6A.1 size=1048576 2024-05-30T16:04:08.581 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.579+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~TJVXXS7KSR6A.2 size=909118 2024-05-30T16:04:08.588 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.587+0000 7f08851138c0 1 < completed multipart upload key=QQILXRJD0C1Z upload=TJVXXS7KSR6A tag=G3W9O743T520 type=write size=4054846 2024-05-30T16:04:08.594 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.595+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~TN0N5AZQ159A.0 size=1048576 2024-05-30T16:04:08.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.595+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~TN0N5AZQ159A.1 size=27136 2024-05-30T16:04:08.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.595+0000 7f08851138c0 1 > prepared multipart upload key=NIV7AQ4EV5YP upload=TN0N5AZQ159A tag=WZU7R5SYOC03 type=write size=2124288 2024-05-30T16:04:08.601 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.599+0000 7f08851138c0 1 < completed delete operation key=3IJWPWRB6FHE tag=JWAG0GCJIX4E type=del 2024-05-30T16:04:08.604 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.603+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=XCS4JIAX36RT type=del size=538471 2024-05-30T16:04:08.608 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.607+0000 7f08851138c0 1 < completed delete operation key=73FJ3HGWYQQK tag=TNIFRFWKSQDD type=del 2024-05-30T16:04:08.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.611+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=EZU0ID2JC8R8 type=del size=192808 2024-05-30T16:04:08.612 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.611+0000 7f08851138c0 1 < canceled part key=_multipart_TPCETCO3Y1M6.2~1QPZB2KM3CQB.0 size=228086 2024-05-30T16:04:08.617 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.619+0000 7f08851138c0 1 < canceled multipart upload key=TPCETCO3Y1M6 upload=1QPZB2KM3CQB tag=01Y48GTMM2FW type=write size=1276662 2024-05-30T16:04:08.621 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.619+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=EQ2OI2G8ZLQV type=del size=3402268 2024-05-30T16:04:08.625 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.623+0000 7f08851138c0 1 < completed write operation key=V4OQFOOQUXFQ tag=C7ENQF473OCB type=write size=720727 2024-05-30T16:04:08.628 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.627+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=2YG1OV6UJD4Y type=del size=4134471 2024-05-30T16:04:08.632 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.631+0000 7f08851138c0 1 < completed delete operation key=MMLF65KLC42Q tag=ORU2QG5QIDTY type=del 2024-05-30T16:04:08.638 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.639+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~5ZFS5CNBR1D9.0 size=1048576 2024-05-30T16:04:08.642 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.643+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~5ZFS5CNBR1D9.1 size=107781 2024-05-30T16:04:08.642 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.643+0000 7f08851138c0 1 > prepared multipart upload key=Y3CYZJ83WYNI upload=5ZFS5CNBR1D9 tag=9X5CA433SCF5 type=write size=2204933 2024-05-30T16:04:08.649 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.647+0000 7f08851138c0 1 < suggested update operation key=0RZRBW38WKNZ tag=6AGNONH8SUXN type=del size=3613447 2024-05-30T16:04:08.652 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.651+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=EO78PB2WLLUL type=del size=3809292 2024-05-30T16:04:08.656 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.655+0000 7f08851138c0 1 < completed write operation key=QQILXRJD0C1Z tag=2JLN6E6Z021D type=write size=377234 2024-05-30T16:04:08.664 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.663+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~6NIE7LRCTB1H.0 size=1048576 2024-05-30T16:04:08.667 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.667+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~6NIE7LRCTB1H.1 size=1043080 2024-05-30T16:04:08.667 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.667+0000 7f08851138c0 1 > prepared multipart upload key=603HBR8PRH6L upload=6NIE7LRCTB1H tag=1K15KRHRQ6IW type=write size=3140232 2024-05-30T16:04:08.670 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.671+0000 7f08851138c0 1 < completed delete operation key=4N58ZY4E1H6E tag=3SL7JUS6DDL9 type=del 2024-05-30T16:04:08.674 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.675+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=2U0NL656KHUS type=del size=4050301 2024-05-30T16:04:08.683 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.683+0000 7f08851138c0 1 < suggested remove operation key=OKBB0VU765TW tag=WO42YRQNCKN2 type=del 2024-05-30T16:04:08.690 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.691+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~DY8XZEK8VBXQ.0 size=349311 2024-05-30T16:04:08.690 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.691+0000 7f08851138c0 1 > prepared multipart upload key=O8CWPJ9GW2GO upload=DY8XZEK8VBXQ tag=I7VGZS0P2LLK type=write size=1397887 2024-05-30T16:04:08.692 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.691+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~GFU193I39UEF.0 size=1048576 2024-05-30T16:04:08.695 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.695+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~GFU193I39UEF.1 size=1048576 2024-05-30T16:04:08.695 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.695+0000 7f08851138c0 1 < canceled part key=_multipart_XHGMUYBA2VNK.2~GFU193I39UEF.2 size=115548 2024-05-30T16:04:08.699 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.699+0000 7f08851138c0 1 < canceled multipart upload key=XHGMUYBA2VNK upload=GFU193I39UEF tag=OO4E9JAKLXF4 type=write size=3261276 2024-05-30T16:04:08.703 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.703+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=18UYLP0R780R type=write size=80377 2024-05-30T16:04:08.706 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.707+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~0J72M9MDHQ94.0 size=1048576 2024-05-30T16:04:08.708 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.707+0000 7f08851138c0 1 < completed part key=_multipart_YRP4CSG92QRO.2~0J72M9MDHQ94.1 size=453184 2024-05-30T16:04:08.712 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.711+0000 7f08851138c0 1 < completed multipart upload key=YRP4CSG92QRO upload=0J72M9MDHQ94 tag=E0L05QTE0527 type=write size=2550336 2024-05-30T16:04:08.715 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.715+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=OTTCS4MPJ5S2 type=del size=3082690 2024-05-30T16:04:08.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.719+0000 7f08851138c0 1 < completed write operation key=O0DJ8SCY766V tag=YE7U1YV4QSSM type=write size=498077 2024-05-30T16:04:08.726 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.723+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~QQLDW260RI6H.0 size=1048576 2024-05-30T16:04:08.726 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.727+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~QQLDW260RI6H.1 size=954439 2024-05-30T16:04:08.726 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.727+0000 7f08851138c0 1 > prepared multipart upload key=QQILXRJD0C1Z upload=QQLDW260RI6H tag=SV32CUFBDCGB type=write size=3051591 2024-05-30T16:04:08.731 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.731+0000 7f08851138c0 1 < suggested update operation key=A5BUBY9Z7JQG tag=TBQ11TIWSMIL type=del size=2853706 2024-05-30T16:04:08.736 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.735+0000 7f08851138c0 1 > prepared part key=_multipart_O0DJ8SCY766V.2~HTBBPWQ63LHA.0 size=674069 2024-05-30T16:04:08.736 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.735+0000 7f08851138c0 1 > prepared multipart upload key=O0DJ8SCY766V upload=HTBBPWQ63LHA tag=C9SABUR35EX8 type=write size=1722645 2024-05-30T16:04:08.739 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.739+0000 7f08851138c0 1 < completed delete operation key=WT9RC0UYVG4F tag=VJEG1L73GO0X type=del 2024-05-30T16:04:08.741 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.743+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=H3995M0L3Z6U type=del size=2909622 2024-05-30T16:04:08.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.743+0000 7f08851138c0 1 < completed delete operation key=NIV7AQ4EV5YP tag=G078QC2D4XIY type=del 2024-05-30T16:04:08.750 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.751+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~0EH3MXWVYPUR.0 size=1048576 2024-05-30T16:04:08.752 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.751+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~0EH3MXWVYPUR.1 size=772123 2024-05-30T16:04:08.752 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.751+0000 7f08851138c0 1 > prepared multipart upload key=O8CWPJ9GW2GO upload=0EH3MXWVYPUR tag=3WTR06LFXV1C type=write size=2869275 2024-05-30T16:04:08.753 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.751+0000 7f08851138c0 1 < canceled part key=_multipart_NIV7AQ4EV5YP.2~TN0N5AZQ159A.0 size=1048576 2024-05-30T16:04:08.755 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.755+0000 7f08851138c0 1 < completed part key=_multipart_NIV7AQ4EV5YP.2~TN0N5AZQ159A.1 size=27136 2024-05-30T16:04:08.760 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.759+0000 7f08851138c0 1 < canceled multipart upload key=NIV7AQ4EV5YP upload=TN0N5AZQ159A tag=WZU7R5SYOC03 type=write size=2124288 2024-05-30T16:04:08.765 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.767+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~VOF68NWPTP3A.0 size=617366 2024-05-30T16:04:08.766 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.767+0000 7f08851138c0 1 > prepared multipart upload key=GWXNCSJCLXKW upload=VOF68NWPTP3A tag=XEY8TW2V7HHD type=write size=1665942 2024-05-30T16:04:08.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.767+0000 7f08851138c0 1 < completed delete operation key=NIV7AQ4EV5YP tag=XCS4JIAX36RT type=del 2024-05-30T16:04:08.772 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.771+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=WSQ1GB6HC7HW type=write size=94273 2024-05-30T16:04:08.775 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.775+0000 7f08851138c0 1 < canceled operation key=1N6R2HBTWMP2 tag=EZU0ID2JC8R8 type=del size=192808 2024-05-30T16:04:08.781 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.783+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~30FW4YZ88V36.0 size=1048576 2024-05-30T16:04:08.784 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.783+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~30FW4YZ88V36.1 size=309724 2024-05-30T16:04:08.784 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.783+0000 7f08851138c0 1 > prepared multipart upload key=QQILXRJD0C1Z upload=30FW4YZ88V36 tag=THANU2UB1TYN type=write size=2406876 2024-05-30T16:04:08.787 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.787+0000 7f08851138c0 1 < completed delete operation key=QQILXRJD0C1Z tag=EQ2OI2G8ZLQV type=del 2024-05-30T16:04:08.793 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.791+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~1SWP1PEIT711.0 size=1048576 2024-05-30T16:04:08.795 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.795+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~1SWP1PEIT711.1 size=1048576 2024-05-30T16:04:08.797 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.799+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~1SWP1PEIT711.2 size=860946 2024-05-30T16:04:08.798 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.799+0000 7f08851138c0 1 > prepared multipart upload key=O8CWPJ9GW2GO upload=1SWP1PEIT711 tag=EIQUA0NFWB1T type=write size=4006674 2024-05-30T16:04:08.800 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.799+0000 7f08851138c0 1 < completed delete operation key=WT9RC0UYVG4F tag=2YG1OV6UJD4Y type=del 2024-05-30T16:04:08.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.803+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=3CA0KX8UOS21 type=write size=94520 2024-05-30T16:04:08.804 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.803+0000 7f08851138c0 1 < canceled part key=_multipart_Y3CYZJ83WYNI.2~5ZFS5CNBR1D9.0 size=1048576 2024-05-30T16:04:08.807 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.807+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~5ZFS5CNBR1D9.1 size=107781 2024-05-30T16:04:08.812 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.811+0000 7f08851138c0 1 < canceled multipart upload key=Y3CYZJ83WYNI upload=5ZFS5CNBR1D9 tag=9X5CA433SCF5 type=write size=2204933 2024-05-30T16:04:08.818 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.819+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~8DBZK4G9GF9U.0 size=1048576 2024-05-30T16:04:08.821 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.819+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~8DBZK4G9GF9U.1 size=1048576 2024-05-30T16:04:08.823 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.823+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~8DBZK4G9GF9U.2 size=149411 2024-05-30T16:04:08.823 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.823+0000 7f08851138c0 1 > prepared multipart upload key=A5BUBY9Z7JQG upload=8DBZK4G9GF9U tag=UWBC3JI05MSC type=write size=3295139 2024-05-30T16:04:08.830 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.831+0000 7f08851138c0 1 < suggested update operation key=V4OQFOOQUXFQ tag=EO78PB2WLLUL type=del size=3809292 2024-05-30T16:04:08.834 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.835+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=HRELSDUK6N7V type=del size=3855524 2024-05-30T16:04:08.837 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.835+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~6NIE7LRCTB1H.0 size=1048576 2024-05-30T16:04:08.839 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.839+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~6NIE7LRCTB1H.1 size=1043080 2024-05-30T16:04:08.845 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.843+0000 7f08851138c0 1 < completed multipart upload key=603HBR8PRH6L upload=6NIE7LRCTB1H tag=1K15KRHRQ6IW type=write size=3140232 2024-05-30T16:04:08.847 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.847+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=W1WLDUP1HUV9 type=del size=2262381 2024-05-30T16:04:08.850 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.851+0000 7f08851138c0 1 < completed delete operation key=3IJWPWRB6FHE tag=2U0NL656KHUS type=del 2024-05-30T16:04:08.853 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.851+0000 7f08851138c0 1 > prepared operation key=QQILXRJD0C1Z tag=1UEHA4ZV8BCA type=del size=2642629 2024-05-30T16:04:08.856 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.855+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~DY8XZEK8VBXQ.0 size=349311 2024-05-30T16:04:08.861 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.859+0000 7f08851138c0 1 < completed multipart upload key=O8CWPJ9GW2GO upload=DY8XZEK8VBXQ tag=I7VGZS0P2LLK type=write size=1397887 2024-05-30T16:04:08.867 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.867+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~DTAXWBD60LIQ.0 size=1048576 2024-05-30T16:04:08.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.871+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~DTAXWBD60LIQ.1 size=1048576 2024-05-30T16:04:08.871 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.871+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~DTAXWBD60LIQ.2 size=614273 2024-05-30T16:04:08.872 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.871+0000 7f08851138c0 1 > prepared multipart upload key=E2FXSXOEUGIG upload=DTAXWBD60LIQ tag=EWI1NWO4RO1V type=write size=3760001 2024-05-30T16:04:08.878 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.879+0000 7f08851138c0 1 < suggested remove operation key=3TBOZWT2P8U8 tag=18UYLP0R780R type=write 2024-05-30T16:04:08.883 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.883+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~HJVM8FBAA054.0 size=1048576 2024-05-30T16:04:08.886 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.887+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~HJVM8FBAA054.1 size=1048576 2024-05-30T16:04:08.888 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.887+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~HJVM8FBAA054.2 size=167948 2024-05-30T16:04:08.888 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.887+0000 7f08851138c0 1 > prepared multipart upload key=WT9RC0UYVG4F upload=HJVM8FBAA054 tag=C809RFSW2WW2 type=write size=3313676 2024-05-30T16:04:08.890 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.891+0000 7f08851138c0 -1 < no bi entry to suggest for operation key=3TBOZWT2P8U8 tag=OTTCS4MPJ5S2 type=del size=3082690: (2) No such file or directory 2024-05-30T16:04:08.896 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.895+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~8V4M3UWRFE49.0 size=1048576 2024-05-30T16:04:08.899 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.899+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~8V4M3UWRFE49.1 size=738269 2024-05-30T16:04:08.899 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.899+0000 7f08851138c0 1 > prepared multipart upload key=3TBOZWT2P8U8 upload=8V4M3UWRFE49 tag=GGZRQ8II1XQ9 type=write size=2835421 2024-05-30T16:04:08.903 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.903+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~QQLDW260RI6H.0 size=1048576 2024-05-30T16:04:08.905 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.903+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~QQLDW260RI6H.1 size=954439 2024-05-30T16:04:08.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.911+0000 7f08851138c0 1 < completed multipart upload key=QQILXRJD0C1Z upload=QQLDW260RI6H tag=SV32CUFBDCGB type=write size=3051591 2024-05-30T16:04:08.914 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.915+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=1QO2NIKMGTRT type=del size=1642285 2024-05-30T16:04:08.918 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.919+0000 7f08851138c0 1 < completed part key=_multipart_O0DJ8SCY766V.2~HTBBPWQ63LHA.0 size=674069 2024-05-30T16:04:08.924 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.923+0000 7f08851138c0 1 < completed multipart upload key=O0DJ8SCY766V upload=HTBBPWQ63LHA tag=C9SABUR35EX8 type=write size=1722645 2024-05-30T16:04:08.930 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.931+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~R3UTQ0HJM9FL.0 size=82542 2024-05-30T16:04:08.931 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.931+0000 7f08851138c0 1 > prepared multipart upload key=603HBR8PRH6L upload=R3UTQ0HJM9FL tag=2STSJ0SCIGTN type=write size=1131118 2024-05-30T16:04:08.934 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.935+0000 7f08851138c0 1 < completed delete operation key=PLWFSKPKVO19 tag=H3995M0L3Z6U type=del 2024-05-30T16:04:08.944 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.943+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=Z0IE5NLAJD97 type=del size=128899 2024-05-30T16:04:08.947 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.947+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~0EH3MXWVYPUR.0 size=1048576 2024-05-30T16:04:08.951 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.951+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~0EH3MXWVYPUR.1 size=772123 2024-05-30T16:04:08.956 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.955+0000 7f08851138c0 1 < completed multipart upload key=O8CWPJ9GW2GO upload=0EH3MXWVYPUR tag=3WTR06LFXV1C type=write size=2869275 2024-05-30T16:04:08.960 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.959+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=H9V7ZD6V324X type=del size=2842913 2024-05-30T16:04:08.963 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.963+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~VOF68NWPTP3A.0 size=617366 2024-05-30T16:04:08.969 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.971+0000 7f08851138c0 1 < completed multipart upload key=GWXNCSJCLXKW upload=VOF68NWPTP3A tag=XEY8TW2V7HHD type=write size=1665942 2024-05-30T16:04:08.975 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.975+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~0B7YLX348L2D.0 size=1048576 2024-05-30T16:04:08.982 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.983+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~0B7YLX348L2D.1 size=145996 2024-05-30T16:04:08.982 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.983+0000 7f08851138c0 1 > prepared multipart upload key=WT9RC0UYVG4F upload=0B7YLX348L2D tag=KCR5SIETDXYQ type=write size=2243148 2024-05-30T16:04:08.986 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.987+0000 7f08851138c0 1 < completed write operation key=73FJ3HGWYQQK tag=WSQ1GB6HC7HW type=write size=94273 2024-05-30T16:04:08.992 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.991+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~THLEOKZ5NWDL.0 size=1048576 2024-05-30T16:04:08.995 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.995+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~THLEOKZ5NWDL.1 size=895502 2024-05-30T16:04:08.995 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.995+0000 7f08851138c0 1 > prepared multipart upload key=603HBR8PRH6L upload=THLEOKZ5NWDL tag=RS9BKSPO7MUU type=write size=2992654 2024-05-30T16:04:08.999 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:08.999+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~30FW4YZ88V36.0 size=1048576 2024-05-30T16:04:09.001 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.003+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~30FW4YZ88V36.1 size=309724 2024-05-30T16:04:09.008 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.007+0000 7f08851138c0 1 < completed multipart upload key=QQILXRJD0C1Z upload=30FW4YZ88V36 tag=THANU2UB1TYN type=write size=2406876 2024-05-30T16:04:09.012 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.011+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=I5SKQFFYXT9U type=del size=1595169 2024-05-30T16:04:09.015 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.015+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~1SWP1PEIT711.0 size=1048576 2024-05-30T16:04:09.018 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.019+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~1SWP1PEIT711.1 size=1048576 2024-05-30T16:04:09.021 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.019+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~1SWP1PEIT711.2 size=860946 2024-05-30T16:04:09.027 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.027+0000 7f08851138c0 1 < completed multipart upload key=O8CWPJ9GW2GO upload=1SWP1PEIT711 tag=EIQUA0NFWB1T type=write size=4006674 2024-05-30T16:04:09.034 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.035+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~CLHV7XQ1PGIM.0 size=1048576 2024-05-30T16:04:09.037 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.035+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~CLHV7XQ1PGIM.1 size=1048576 2024-05-30T16:04:09.040 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.039+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~CLHV7XQ1PGIM.2 size=869415 2024-05-30T16:04:09.040 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.039+0000 7f08851138c0 1 > prepared multipart upload key=OKBB0VU765TW upload=CLHV7XQ1PGIM tag=MBVLYSRY3RYA type=write size=4015143 2024-05-30T16:04:09.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.043+0000 7f08851138c0 1 < completed write operation key=KPHZGFJLC2E0 tag=3CA0KX8UOS21 type=write size=94520 2024-05-30T16:04:09.047 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.047+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=382SKTCUDY86 type=del size=3112806 2024-05-30T16:04:09.051 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.051+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~8DBZK4G9GF9U.0 size=1048576 2024-05-30T16:04:09.053 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.055+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~8DBZK4G9GF9U.1 size=1048576 2024-05-30T16:04:09.056 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.055+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~8DBZK4G9GF9U.2 size=149411 2024-05-30T16:04:09.062 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.063+0000 7f08851138c0 1 < completed multipart upload key=A5BUBY9Z7JQG upload=8DBZK4G9GF9U tag=UWBC3JI05MSC type=write size=3295139 2024-05-30T16:04:09.066 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.067+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=RYV5ZZSNCV6K type=del size=1274584 2024-05-30T16:04:09.069 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.067+0000 7f08851138c0 1 < completed delete operation key=O8CWPJ9GW2GO tag=HRELSDUK6N7V type=del 2024-05-30T16:04:09.072 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.071+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=NKZ0R128B97P type=del size=770650 2024-05-30T16:04:09.076 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.075+0000 7f08851138c0 1 < canceled operation key=OKBB0VU765TW tag=W1WLDUP1HUV9 type=del size=2262381 2024-05-30T16:04:09.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.079+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=EHPBUVTRJTFC type=del size=79114 2024-05-30T16:04:09.083 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.083+0000 7f08851138c0 1 < completed delete operation key=QQILXRJD0C1Z tag=1UEHA4ZV8BCA type=del 2024-05-30T16:04:09.089 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.087+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~8UP026YS053H.0 size=1048576 2024-05-30T16:04:09.091 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.091+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~8UP026YS053H.1 size=1048576 2024-05-30T16:04:09.094 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.095+0000 7f08851138c0 1 > prepared part key=_multipart_A5BUBY9Z7JQG.2~8UP026YS053H.2 size=598175 2024-05-30T16:04:09.094 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.095+0000 7f08851138c0 1 > prepared multipart upload key=A5BUBY9Z7JQG upload=8UP026YS053H tag=17ST6IQM68CO type=write size=3743903 2024-05-30T16:04:09.097 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.099+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~DTAXWBD60LIQ.0 size=1048576 2024-05-30T16:04:09.100 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.099+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~DTAXWBD60LIQ.1 size=1048576 2024-05-30T16:04:09.103 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.103+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~DTAXWBD60LIQ.2 size=614273 2024-05-30T16:04:09.109 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.111+0000 7f08851138c0 1 < completed multipart upload key=E2FXSXOEUGIG upload=DTAXWBD60LIQ tag=EWI1NWO4RO1V type=write size=3760001 2024-05-30T16:04:09.113 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.115+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=3J0B2MOAEU9O type=write size=575027 2024-05-30T16:04:09.117 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.115+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~HJVM8FBAA054.0 size=1048576 2024-05-30T16:04:09.120 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.119+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~HJVM8FBAA054.1 size=1048576 2024-05-30T16:04:09.123 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.123+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~HJVM8FBAA054.2 size=167948 2024-05-30T16:04:09.130 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.131+0000 7f08851138c0 1 < completed multipart upload key=WT9RC0UYVG4F upload=HJVM8FBAA054 tag=C809RFSW2WW2 type=write size=3313676 2024-05-30T16:04:09.133 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.135+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=OLB9R7MNAB3T type=del size=1801892 2024-05-30T16:04:09.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.139+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~8V4M3UWRFE49.0 size=1048576 2024-05-30T16:04:09.140 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.139+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~8V4M3UWRFE49.1 size=738269 2024-05-30T16:04:09.146 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.147+0000 7f08851138c0 1 < completed multipart upload key=3TBOZWT2P8U8 upload=8V4M3UWRFE49 tag=GGZRQ8II1XQ9 type=write size=2835421 2024-05-30T16:04:09.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.151+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=SQN50KGSOXQG type=del size=2139370 2024-05-30T16:04:09.153 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.155+0000 7f08851138c0 1 < completed delete operation key=1N6R2HBTWMP2 tag=1QO2NIKMGTRT type=del 2024-05-30T16:04:09.157 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.155+0000 7f08851138c0 1 > prepared operation key=QCLY10D2A3VF tag=BMHB3BL363YV type=del size=800980 2024-05-30T16:04:09.161 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.159+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~R3UTQ0HJM9FL.0 size=82542 2024-05-30T16:04:09.166 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.167+0000 7f08851138c0 1 < completed multipart upload key=603HBR8PRH6L upload=R3UTQ0HJM9FL tag=2STSJ0SCIGTN type=write size=1131118 2024-05-30T16:04:09.170 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.171+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=SVESY4HDWNN6 type=del size=4132747 2024-05-30T16:04:09.174 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.175+0000 7f08851138c0 1 < completed delete operation key=OKBB0VU765TW tag=Z0IE5NLAJD97 type=del 2024-05-30T16:04:09.180 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.179+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~19OGLXJZ9ILD.0 size=942294 2024-05-30T16:04:09.180 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.179+0000 7f08851138c0 1 > prepared multipart upload key=0RZRBW38WKNZ upload=19OGLXJZ9ILD tag=DWB9EP5QG5W1 type=write size=1990870 2024-05-30T16:04:09.184 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.183+0000 7f08851138c0 1 < canceled operation key=1N6R2HBTWMP2 tag=H9V7ZD6V324X type=del size=2842913 2024-05-30T16:04:09.187 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.187+0000 7f08851138c0 1 > prepared operation key=RCTV94WRMO53 tag=HGDTFGRCBJUZ type=del size=2384150 2024-05-30T16:04:09.191 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.191+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~0B7YLX348L2D.0 size=1048576 2024-05-30T16:04:09.194 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.195+0000 7f08851138c0 1 < completed part key=_multipart_WT9RC0UYVG4F.2~0B7YLX348L2D.1 size=145996 2024-05-30T16:04:09.200 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.199+0000 7f08851138c0 1 < completed multipart upload key=WT9RC0UYVG4F upload=0B7YLX348L2D tag=KCR5SIETDXYQ type=write size=2243148 2024-05-30T16:04:09.204 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.203+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=OWP3PCE4VQCA type=del size=8344 2024-05-30T16:04:09.208 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.207+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~THLEOKZ5NWDL.0 size=1048576 2024-05-30T16:04:09.211 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.211+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~THLEOKZ5NWDL.1 size=895502 2024-05-30T16:04:09.217 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.215+0000 7f08851138c0 1 < completed multipart upload key=603HBR8PRH6L upload=THLEOKZ5NWDL tag=RS9BKSPO7MUU type=write size=2992654 2024-05-30T16:04:09.221 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.219+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=HXC3ZZB877A1 type=del size=3815522 2024-05-30T16:04:09.224 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.223+0000 7f08851138c0 1 < completed delete operation key=73FJ3HGWYQQK tag=I5SKQFFYXT9U type=del 2024-05-30T16:04:09.231 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.231+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~54JKZFPIXDXA.0 size=1048576 2024-05-30T16:04:09.234 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.235+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~54JKZFPIXDXA.1 size=1048576 2024-05-30T16:04:09.237 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.235+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~54JKZFPIXDXA.2 size=136750 2024-05-30T16:04:09.237 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.235+0000 7f08851138c0 1 > prepared multipart upload key=Y3CYZJ83WYNI upload=54JKZFPIXDXA tag=NRHTL0W5GQJH type=write size=3282478 2024-05-30T16:04:09.240 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.239+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~CLHV7XQ1PGIM.0 size=1048576 2024-05-30T16:04:09.243 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.243+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~CLHV7XQ1PGIM.1 size=1048576 2024-05-30T16:04:09.246 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.247+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~CLHV7XQ1PGIM.2 size=869415 2024-05-30T16:04:09.252 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.251+0000 7f08851138c0 1 < completed multipart upload key=OKBB0VU765TW upload=CLHV7XQ1PGIM tag=MBVLYSRY3RYA type=write size=4015143 2024-05-30T16:04:09.256 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.255+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=LRT297OSYIAQ type=del size=3864011 2024-05-30T16:04:09.264 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.263+0000 7f08851138c0 1 < suggested remove operation key=MMLF65KLC42Q tag=382SKTCUDY86 type=del 2024-05-30T16:04:09.267 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.267+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=R0BD3IIHPCCN type=del size=1158004 2024-05-30T16:04:09.275 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.275+0000 7f08851138c0 1 < suggested update operation key=NIV7AQ4EV5YP tag=RYV5ZZSNCV6K type=del size=1274584 2024-05-30T16:04:09.282 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.283+0000 7f08851138c0 1 > prepared part key=_multipart_MMLF65KLC42Q.2~GTG1Z6E2KMOE.0 size=388510 2024-05-30T16:04:09.282 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.283+0000 7f08851138c0 1 > prepared multipart upload key=MMLF65KLC42Q upload=GTG1Z6E2KMOE tag=5FM43T39B7B5 type=write size=1437086 2024-05-30T16:04:09.289 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.291+0000 7f08851138c0 1 < suggested remove operation key=PLWFSKPKVO19 tag=NKZ0R128B97P type=del 2024-05-30T16:04:09.294 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.295+0000 7f08851138c0 1 > prepared operation key=VRXNO8JLAZYH tag=IA4KVUUDC9JN type=del size=333969 2024-05-30T16:04:09.297 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.299+0000 7f08851138c0 1 < completed delete operation key=1N6R2HBTWMP2 tag=EHPBUVTRJTFC type=del 2024-05-30T16:04:09.302 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.303+0000 7f08851138c0 1 > prepared operation key=4N58ZY4E1H6E tag=VJ2ZR8MRU1LK type=del size=2480365 2024-05-30T16:04:09.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.307+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~8UP026YS053H.0 size=1048576 2024-05-30T16:04:09.309 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.307+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~8UP026YS053H.1 size=1048576 2024-05-30T16:04:09.312 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.311+0000 7f08851138c0 1 < completed part key=_multipart_A5BUBY9Z7JQG.2~8UP026YS053H.2 size=598175 2024-05-30T16:04:09.323 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.323+0000 7f08851138c0 1 < completed multipart upload key=A5BUBY9Z7JQG upload=8UP026YS053H tag=17ST6IQM68CO type=write size=3743903 2024-05-30T16:04:09.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.331+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~H2V3QLGLF7EP.0 size=437868 2024-05-30T16:04:09.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.331+0000 7f08851138c0 1 > prepared multipart upload key=TPCETCO3Y1M6 upload=H2V3QLGLF7EP tag=J0EU0JF30G6G type=write size=1486444 2024-05-30T16:04:09.334 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.335+0000 7f08851138c0 -1 < failed to complete operation key=NIV7AQ4EV5YP tag=3J0B2MOAEU9O type=write size=575027: (22) Invalid argument 2024-05-30T16:04:09.341 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.339+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~9HBWI8UXK5MJ.0 size=1048576 2024-05-30T16:04:09.344 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.343+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~9HBWI8UXK5MJ.1 size=1048576 2024-05-30T16:04:09.347 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.347+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~9HBWI8UXK5MJ.2 size=805485 2024-05-30T16:04:09.347 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.347+0000 7f08851138c0 1 > prepared multipart upload key=JNZI7JX95LX2 upload=9HBWI8UXK5MJ tag=7DUNYSS3G9VX type=write size=3951213 2024-05-30T16:04:09.351 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.351+0000 7f08851138c0 1 < completed delete operation key=3IJWPWRB6FHE tag=OLB9R7MNAB3T type=del 2024-05-30T16:04:09.355 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.355+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=5ENKCOW3NP3U type=del size=459652 2024-05-30T16:04:09.363 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.363+0000 7f08851138c0 1 < suggested remove operation key=E2FXSXOEUGIG tag=SQN50KGSOXQG type=del 2024-05-30T16:04:09.370 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.371+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~S9RWN30XG1AV.0 size=1048576 2024-05-30T16:04:09.373 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.375+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~S9RWN30XG1AV.1 size=1048576 2024-05-30T16:04:09.376 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.375+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~S9RWN30XG1AV.2 size=834499 2024-05-30T16:04:09.376 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.375+0000 7f08851138c0 1 > prepared multipart upload key=KPHZGFJLC2E0 upload=S9RWN30XG1AV tag=Z2VIJHYMLYP1 type=write size=3980227 2024-05-30T16:04:09.380 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.379+0000 7f08851138c0 1 < canceled operation key=QCLY10D2A3VF tag=BMHB3BL363YV type=del size=800980 2024-05-30T16:04:09.387 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.387+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~H6KYEPX93A3S.0 size=1048576 2024-05-30T16:04:09.390 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.391+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~H6KYEPX93A3S.1 size=1048576 2024-05-30T16:04:09.393 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.391+0000 7f08851138c0 1 > prepared part key=_multipart_JNZI7JX95LX2.2~H6KYEPX93A3S.2 size=1004734 2024-05-30T16:04:09.393 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.391+0000 7f08851138c0 1 > prepared multipart upload key=JNZI7JX95LX2 upload=H6KYEPX93A3S tag=8Z6PT4YMVJC2 type=write size=4150462 2024-05-30T16:04:09.396 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.395+0000 7f08851138c0 1 < completed delete operation key=JNZI7JX95LX2 tag=SVESY4HDWNN6 type=del 2024-05-30T16:04:09.400 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.399+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=C8U92EHMGQUL type=del size=3482405 2024-05-30T16:04:09.401 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.399+0000 7f08851138c0 1 < canceled part key=_multipart_0RZRBW38WKNZ.2~19OGLXJZ9ILD.0 size=942294 2024-05-30T16:04:09.407 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.407+0000 7f08851138c0 1 < canceled multipart upload key=0RZRBW38WKNZ upload=19OGLXJZ9ILD tag=DWB9EP5QG5W1 type=write size=1990870 2024-05-30T16:04:09.410 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.411+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=VEE606SZGOSG type=del size=1830174 2024-05-30T16:04:09.414 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.415+0000 7f08851138c0 1 < completed delete operation key=RCTV94WRMO53 tag=HGDTFGRCBJUZ type=del 2024-05-30T16:04:09.418 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.419+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=Z3LT7OS4VLJL type=del size=2451425 2024-05-30T16:04:09.421 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.423+0000 7f08851138c0 -1 < failed to complete operation key=NIV7AQ4EV5YP tag=OWP3PCE4VQCA type=del size=8344: (22) Invalid argument 2024-05-30T16:04:09.425 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.427+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=HA88DWDUDT9J type=del size=1561523 2024-05-30T16:04:09.429 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.427+0000 7f08851138c0 1 < completed delete operation key=0RZRBW38WKNZ tag=HXC3ZZB877A1 type=del 2024-05-30T16:04:09.433 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.431+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=20TR8EHC5MFZ type=del size=1250383 2024-05-30T16:04:09.437 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.435+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~54JKZFPIXDXA.0 size=1048576 2024-05-30T16:04:09.440 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.439+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~54JKZFPIXDXA.1 size=1048576 2024-05-30T16:04:09.443 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.443+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~54JKZFPIXDXA.2 size=136750 2024-05-30T16:04:09.450 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.451+0000 7f08851138c0 1 < completed multipart upload key=Y3CYZJ83WYNI upload=54JKZFPIXDXA tag=NRHTL0W5GQJH type=write size=3282478 2024-05-30T16:04:09.456 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.455+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~P5ADT013TV0N.0 size=1048576 2024-05-30T16:04:09.459 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.459+0000 7f08851138c0 1 > prepared part key=_multipart_RCTV94WRMO53.2~P5ADT013TV0N.1 size=103658 2024-05-30T16:04:09.460 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.459+0000 7f08851138c0 1 > prepared multipart upload key=RCTV94WRMO53 upload=P5ADT013TV0N tag=CI40FV90KYUN type=write size=2200810 2024-05-30T16:04:09.464 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.463+0000 7f08851138c0 1 < completed delete operation key=YRP4CSG92QRO tag=LRT297OSYIAQ type=del 2024-05-30T16:04:09.468 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.467+0000 7f08851138c0 1 > prepared operation key=YRP4CSG92QRO tag=9XNX9RYFH5OU type=del size=1905775 2024-05-30T16:04:09.472 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.471+0000 7f08851138c0 1 < completed delete operation key=4N58ZY4E1H6E tag=R0BD3IIHPCCN type=del 2024-05-30T16:04:09.476 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.475+0000 7f08851138c0 1 > prepared operation key=V4OQFOOQUXFQ tag=HUWEBWAARZ8Y type=del size=1039555 2024-05-30T16:04:09.480 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.479+0000 7f08851138c0 1 < completed part key=_multipart_MMLF65KLC42Q.2~GTG1Z6E2KMOE.0 size=388510 2024-05-30T16:04:09.486 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.487+0000 7f08851138c0 1 < completed multipart upload key=MMLF65KLC42Q upload=GTG1Z6E2KMOE tag=5FM43T39B7B5 type=write size=1437086 2024-05-30T16:04:09.490 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.491+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=N6Z13ETK79XI type=write size=257731 2024-05-30T16:04:09.495 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.495+0000 7f08851138c0 1 < completed delete operation key=VRXNO8JLAZYH tag=IA4KVUUDC9JN type=del 2024-05-30T16:04:09.498 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.499+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=12XBQT77NDQA type=del size=3001111 2024-05-30T16:04:09.505 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.507+0000 7f08851138c0 1 < suggested remove operation key=4N58ZY4E1H6E tag=VJ2ZR8MRU1LK type=del 2024-05-30T16:04:09.513 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.511+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~43TAFC5BEXAD.0 size=1048576 2024-05-30T16:04:09.516 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.515+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~43TAFC5BEXAD.1 size=1048576 2024-05-30T16:04:09.519 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.519+0000 7f08851138c0 1 > prepared part key=_multipart_E2FXSXOEUGIG.2~43TAFC5BEXAD.2 size=806378 2024-05-30T16:04:09.519 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.519+0000 7f08851138c0 1 > prepared multipart upload key=E2FXSXOEUGIG upload=43TAFC5BEXAD tag=SRUJO3UWCVKB type=write size=3952106 2024-05-30T16:04:09.523 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.523+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~H2V3QLGLF7EP.0 size=437868 2024-05-30T16:04:09.529 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.527+0000 7f08851138c0 1 < completed multipart upload key=TPCETCO3Y1M6 upload=H2V3QLGLF7EP tag=J0EU0JF30G6G type=write size=1486444 2024-05-30T16:04:09.533 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.535+0000 7f08851138c0 1 > prepared operation key=Y8SBI0QZ0ZV8 tag=SEXM3JCKSL0B type=del size=3190312 2024-05-30T16:04:09.537 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.539+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~9HBWI8UXK5MJ.0 size=1048576 2024-05-30T16:04:09.540 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.539+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~9HBWI8UXK5MJ.1 size=1048576 2024-05-30T16:04:09.543 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.543+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~9HBWI8UXK5MJ.2 size=805485 2024-05-30T16:04:09.549 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.551+0000 7f08851138c0 1 < completed multipart upload key=JNZI7JX95LX2 upload=9HBWI8UXK5MJ tag=7DUNYSS3G9VX type=write size=3951213 2024-05-30T16:04:09.556 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.555+0000 7f08851138c0 1 > prepared part key=_multipart_Y8SBI0QZ0ZV8.2~DRS0VSWP4FR3.0 size=290998 2024-05-30T16:04:09.556 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.555+0000 7f08851138c0 1 > prepared multipart upload key=Y8SBI0QZ0ZV8 upload=DRS0VSWP4FR3 tag=I6QNYWNBE3J9 type=write size=1339574 2024-05-30T16:04:09.560 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.559+0000 7f08851138c0 1 < completed delete operation key=V4OQFOOQUXFQ tag=5ENKCOW3NP3U type=del 2024-05-30T16:04:09.564 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.563+0000 7f08851138c0 1 > prepared operation key=GWXNCSJCLXKW tag=A2G9I0QQ68EO type=write size=960497 2024-05-30T16:04:09.568 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.567+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~S9RWN30XG1AV.0 size=1048576 2024-05-30T16:04:09.571 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.571+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~S9RWN30XG1AV.1 size=1048576 2024-05-30T16:04:09.574 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.575+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~S9RWN30XG1AV.2 size=834499 2024-05-30T16:04:09.580 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.579+0000 7f08851138c0 1 < completed multipart upload key=KPHZGFJLC2E0 upload=S9RWN30XG1AV tag=Z2VIJHYMLYP1 type=write size=3980227 2024-05-30T16:04:09.587 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.587+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~TFWZ5EMU8B4G.0 size=1048576 2024-05-30T16:04:09.590 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.591+0000 7f08851138c0 1 > prepared part key=_multipart_KPHZGFJLC2E0.2~TFWZ5EMU8B4G.1 size=891375 2024-05-30T16:04:09.590 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.591+0000 7f08851138c0 1 > prepared multipart upload key=KPHZGFJLC2E0 upload=TFWZ5EMU8B4G tag=WNNV0QQ4SYN7 type=write size=2988527 2024-05-30T16:04:09.593 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.595+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~H6KYEPX93A3S.0 size=1048576 2024-05-30T16:04:09.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.595+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~H6KYEPX93A3S.1 size=1048576 2024-05-30T16:04:09.599 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.599+0000 7f08851138c0 1 < completed part key=_multipart_JNZI7JX95LX2.2~H6KYEPX93A3S.2 size=1004734 2024-05-30T16:04:09.605 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.607+0000 7f08851138c0 1 < completed multipart upload key=JNZI7JX95LX2 upload=H6KYEPX93A3S tag=8Z6PT4YMVJC2 type=write size=4150462 2024-05-30T16:04:09.609 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.607+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=5948N3SC953V type=write size=643286 2024-05-30T16:04:09.612 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.611+0000 7f08851138c0 1 < completed delete operation key=NIV7AQ4EV5YP tag=C8U92EHMGQUL type=del 2024-05-30T16:04:09.616 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.615+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=WFGIUHLXG5LS type=del size=3824933 2024-05-30T16:04:09.620 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.619+0000 7f08851138c0 1 < completed delete operation key=E2FXSXOEUGIG tag=VEE606SZGOSG type=del 2024-05-30T16:04:09.623 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.623+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=TT8P2JSOYYH3 type=del size=1902767 2024-05-30T16:04:09.627 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.627+0000 7f08851138c0 1 < completed delete operation key=NIV7AQ4EV5YP tag=Z3LT7OS4VLJL type=del 2024-05-30T16:04:09.630 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.631+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=UQWB5H46M093 type=del size=3988099 2024-05-30T16:04:09.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.635+0000 7f08851138c0 1 < completed delete operation key=3IJWPWRB6FHE tag=HA88DWDUDT9J type=del 2024-05-30T16:04:09.639 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.639+0000 7f08851138c0 1 > prepared operation key=ZALVY70ISLM5 tag=WMGLW3UUFSOQ type=del size=987515 2024-05-30T16:04:09.642 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.643+0000 7f08851138c0 1 < completed delete operation key=3IJWPWRB6FHE tag=20TR8EHC5MFZ type=del 2024-05-30T16:04:09.646 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.647+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=UY53A9V7EQ08 type=del size=4010445 2024-05-30T16:04:09.649 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.651+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~P5ADT013TV0N.0 size=1048576 2024-05-30T16:04:09.653 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.651+0000 7f08851138c0 1 < completed part key=_multipart_RCTV94WRMO53.2~P5ADT013TV0N.1 size=103658 2024-05-30T16:04:09.659 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.659+0000 7f08851138c0 1 < completed multipart upload key=RCTV94WRMO53 upload=P5ADT013TV0N tag=CI40FV90KYUN type=write size=2200810 2024-05-30T16:04:09.666 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.667+0000 7f08851138c0 1 > prepared part key=_multipart_603HBR8PRH6L.2~9SPQSIEH1PJ7.0 size=245177 2024-05-30T16:04:09.667 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.667+0000 7f08851138c0 1 > prepared multipart upload key=603HBR8PRH6L upload=9SPQSIEH1PJ7 tag=5W8SFSOXHM8S type=write size=1293753 2024-05-30T16:04:09.674 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.675+0000 7f08851138c0 1 < canceled operation key=YRP4CSG92QRO tag=9XNX9RYFH5OU type=del size=1905775 2024-05-30T16:04:09.678 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.679+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=6DEUKW0GVXXY type=del size=3668353 2024-05-30T16:04:09.682 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.683+0000 7f08851138c0 1 < completed delete operation key=V4OQFOOQUXFQ tag=HUWEBWAARZ8Y type=del 2024-05-30T16:04:09.686 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.687+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=10TJLDK3WBC4 type=write size=215267 2024-05-30T16:04:09.690 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.691+0000 7f08851138c0 1 < completed write operation key=364U3OV10VPD tag=N6Z13ETK79XI type=write size=257731 2024-05-30T16:04:09.694 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.695+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=BR8EBA7XKDSH type=del size=745464 2024-05-30T16:04:09.698 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.699+0000 7f08851138c0 1 < completed delete operation key=Y8SBI0QZ0ZV8 tag=12XBQT77NDQA type=del 2024-05-30T16:04:09.705 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.703+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~N791NMDDITCQ.0 size=1048576 2024-05-30T16:04:09.708 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.707+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~N791NMDDITCQ.1 size=402274 2024-05-30T16:04:09.708 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.707+0000 7f08851138c0 1 > prepared multipart upload key=XHGMUYBA2VNK upload=N791NMDDITCQ tag=4E579ASZORJ8 type=write size=2499426 2024-05-30T16:04:09.712 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.711+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~43TAFC5BEXAD.0 size=1048576 2024-05-30T16:04:09.715 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.715+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~43TAFC5BEXAD.1 size=1048576 2024-05-30T16:04:09.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.719+0000 7f08851138c0 1 < completed part key=_multipart_E2FXSXOEUGIG.2~43TAFC5BEXAD.2 size=806378 2024-05-30T16:04:09.724 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.723+0000 7f08851138c0 1 < completed multipart upload key=E2FXSXOEUGIG upload=43TAFC5BEXAD tag=SRUJO3UWCVKB type=write size=3952106 2024-05-30T16:04:09.731 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.731+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~SWGYLIZJMSNL.0 size=1048576 2024-05-30T16:04:09.734 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.735+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~SWGYLIZJMSNL.1 size=1048576 2024-05-30T16:04:09.737 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.735+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~SWGYLIZJMSNL.2 size=1005023 2024-05-30T16:04:09.737 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.735+0000 7f08851138c0 1 > prepared multipart upload key=Y3CYZJ83WYNI upload=SWGYLIZJMSNL tag=2S3RDC7TZNPA type=write size=4150751 2024-05-30T16:04:09.740 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.739+0000 7f08851138c0 1 < completed delete operation key=Y8SBI0QZ0ZV8 tag=SEXM3JCKSL0B type=del 2024-05-30T16:04:09.744 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.743+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=P97GT7XM0I46 type=del size=2415538 2024-05-30T16:04:09.747 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.747+0000 7f08851138c0 1 < completed part key=_multipart_Y8SBI0QZ0ZV8.2~DRS0VSWP4FR3.0 size=290998 2024-05-30T16:04:09.753 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.751+0000 7f08851138c0 1 < completed multipart upload key=Y8SBI0QZ0ZV8 upload=DRS0VSWP4FR3 tag=I6QNYWNBE3J9 type=write size=1339574 2024-05-30T16:04:09.759 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.759+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~424EY50BUD5B.0 size=793082 2024-05-30T16:04:09.759 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.759+0000 7f08851138c0 1 > prepared multipart upload key=XHGMUYBA2VNK upload=424EY50BUD5B tag=QNWCEINV8R6M type=write size=1841658 2024-05-30T16:04:09.766 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.767+0000 7f08851138c0 1 < suggested remove operation key=GWXNCSJCLXKW tag=A2G9I0QQ68EO type=write 2024-05-30T16:04:09.773 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.771+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~Y7Z3AUO5RFJ3.0 size=1048576 2024-05-30T16:04:09.776 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.775+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~Y7Z3AUO5RFJ3.1 size=1048576 2024-05-30T16:04:09.779 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.779+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~Y7Z3AUO5RFJ3.2 size=373309 2024-05-30T16:04:09.779 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.779+0000 7f08851138c0 1 > prepared multipart upload key=OKBB0VU765TW upload=Y7Z3AUO5RFJ3 tag=IJW6UGU9YBG9 type=write size=3519037 2024-05-30T16:04:09.782 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.783+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~TFWZ5EMU8B4G.0 size=1048576 2024-05-30T16:04:09.785 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.787+0000 7f08851138c0 1 < completed part key=_multipart_KPHZGFJLC2E0.2~TFWZ5EMU8B4G.1 size=891375 2024-05-30T16:04:09.791 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.791+0000 7f08851138c0 1 < completed multipart upload key=KPHZGFJLC2E0 upload=TFWZ5EMU8B4G tag=WNNV0QQ4SYN7 type=write size=2988527 2024-05-30T16:04:09.798 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.799+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~Q8O792PE3USY.0 size=812142 2024-05-30T16:04:09.798 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.799+0000 7f08851138c0 1 > prepared multipart upload key=3IJWPWRB6FHE upload=Q8O792PE3USY tag=BSK7BMF01JM7 type=write size=1860718 2024-05-30T16:04:09.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.803+0000 7f08851138c0 1 < completed write operation key=1N6R2HBTWMP2 tag=5948N3SC953V type=write size=643286 2024-05-30T16:04:09.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.807+0000 7f08851138c0 1 > prepared operation key=OKBB0VU765TW tag=HGDCF385Z0HG type=del size=2593350 2024-05-30T16:04:09.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.811+0000 7f08851138c0 1 < completed delete operation key=364U3OV10VPD tag=WFGIUHLXG5LS type=del 2024-05-30T16:04:09.813 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.811+0000 7f08851138c0 1 > prepared operation key=MMLF65KLC42Q tag=ZNRWN1Y07Q63 type=del size=3462440 2024-05-30T16:04:09.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.819+0000 7f08851138c0 1 < suggested remove operation key=3IJWPWRB6FHE tag=TT8P2JSOYYH3 type=del 2024-05-30T16:04:09.824 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.823+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=TOXI3GPQI5TO type=del size=2299808 2024-05-30T16:04:09.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.827+0000 7f08851138c0 1 < completed delete operation key=3TBOZWT2P8U8 tag=UQWB5H46M093 type=del 2024-05-30T16:04:09.832 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.831+0000 7f08851138c0 1 > prepared operation key=KPHZGFJLC2E0 tag=RUST4RZ8NAPA type=del size=3999721 2024-05-30T16:04:09.836 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.835+0000 7f08851138c0 1 < completed delete operation key=ZALVY70ISLM5 tag=WMGLW3UUFSOQ type=del 2024-05-30T16:04:09.843 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.843+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~NCEYLZMH5HCC.0 size=1048576 2024-05-30T16:04:09.845 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.847+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~NCEYLZMH5HCC.1 size=1048576 2024-05-30T16:04:09.849 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.847+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~NCEYLZMH5HCC.2 size=202468 2024-05-30T16:04:09.849 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.847+0000 7f08851138c0 1 > prepared multipart upload key=GWXNCSJCLXKW upload=NCEYLZMH5HCC tag=3DJ6DJ0RBXS2 type=write size=3348196 2024-05-30T16:04:09.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.851+0000 7f08851138c0 1 < completed delete operation key=E2FXSXOEUGIG tag=UY53A9V7EQ08 type=del 2024-05-30T16:04:09.856 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.855+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=T23UJMVN0CAC type=write size=65685 2024-05-30T16:04:09.859 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.859+0000 7f08851138c0 1 < completed part key=_multipart_603HBR8PRH6L.2~9SPQSIEH1PJ7.0 size=245177 2024-05-30T16:04:09.865 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.863+0000 7f08851138c0 1 < completed multipart upload key=603HBR8PRH6L upload=9SPQSIEH1PJ7 tag=5W8SFSOXHM8S type=write size=1293753 2024-05-30T16:04:09.868 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.867+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=FBKSHGJ4NOGL type=del size=4015868 2024-05-30T16:04:09.872 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.871+0000 7f08851138c0 1 < completed delete operation key=E2FXSXOEUGIG tag=6DEUKW0GVXXY type=del 2024-05-30T16:04:09.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.879+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~ZFVWLNYALPEV.0 size=1048576 2024-05-30T16:04:09.882 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.883+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~ZFVWLNYALPEV.1 size=1018990 2024-05-30T16:04:09.882 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.883+0000 7f08851138c0 1 > prepared multipart upload key=3IJWPWRB6FHE upload=ZFVWLNYALPEV tag=YWL6SQJGZI31 type=write size=3116142 2024-05-30T16:04:09.885 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.887+0000 7f08851138c0 1 < completed write operation key=MMLF65KLC42Q tag=10TJLDK3WBC4 type=write size=215267 2024-05-30T16:04:09.889 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.891+0000 7f08851138c0 1 > prepared operation key=3IJWPWRB6FHE tag=FB9VX70SC6UC type=del size=467233 2024-05-30T16:04:09.893 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.891+0000 7f08851138c0 1 < completed delete operation key=0RZRBW38WKNZ tag=BR8EBA7XKDSH type=del 2024-05-30T16:04:09.900 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.899+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~FI75SBR2QXTU.0 size=1048576 2024-05-30T16:04:09.903 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.903+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~FI75SBR2QXTU.1 size=1048576 2024-05-30T16:04:09.905 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.907+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~FI75SBR2QXTU.2 size=56842 2024-05-30T16:04:09.905 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.907+0000 7f08851138c0 1 > prepared multipart upload key=364U3OV10VPD upload=FI75SBR2QXTU tag=072C3MLOD6W6 type=write size=3202570 2024-05-30T16:04:09.909 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.911+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~N791NMDDITCQ.0 size=1048576 2024-05-30T16:04:09.912 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.911+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~N791NMDDITCQ.1 size=402274 2024-05-30T16:04:09.918 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.919+0000 7f08851138c0 1 < completed multipart upload key=XHGMUYBA2VNK upload=N791NMDDITCQ tag=4E579ASZORJ8 type=write size=2499426 2024-05-30T16:04:09.922 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.923+0000 7f08851138c0 1 > prepared operation key=O8CWPJ9GW2GO tag=CG6LCLCUSAX2 type=del size=1072250 2024-05-30T16:04:09.925 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.927+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~SWGYLIZJMSNL.0 size=1048576 2024-05-30T16:04:09.928 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.927+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~SWGYLIZJMSNL.1 size=1048576 2024-05-30T16:04:09.931 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.931+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~SWGYLIZJMSNL.2 size=1005023 2024-05-30T16:04:09.937 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.939+0000 7f08851138c0 1 < completed multipart upload key=Y3CYZJ83WYNI upload=SWGYLIZJMSNL tag=2S3RDC7TZNPA type=write size=4150751 2024-05-30T16:04:09.941 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.939+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=9NUMPBFV25QY type=del size=1565254 2024-05-30T16:04:09.945 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.947+0000 7f08851138c0 1 < completed delete operation key=O8CWPJ9GW2GO tag=P97GT7XM0I46 type=del 2024-05-30T16:04:09.949 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.951+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=VVSZ23Q5HCL2 type=del size=1928614 2024-05-30T16:04:09.950 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.951+0000 7f08851138c0 1 < canceled part key=_multipart_XHGMUYBA2VNK.2~424EY50BUD5B.0 size=793082 2024-05-30T16:04:09.956 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.955+0000 7f08851138c0 1 < canceled multipart upload key=XHGMUYBA2VNK upload=424EY50BUD5B tag=QNWCEINV8R6M type=write size=1841658 2024-05-30T16:04:09.965 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.963+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~GPFQRAIQM4N9.0 size=1048576 2024-05-30T16:04:09.968 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.967+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~GPFQRAIQM4N9.1 size=1048576 2024-05-30T16:04:09.971 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.971+0000 7f08851138c0 1 > prepared part key=_multipart_3IJWPWRB6FHE.2~GPFQRAIQM4N9.2 size=660470 2024-05-30T16:04:09.971 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.971+0000 7f08851138c0 1 > prepared multipart upload key=3IJWPWRB6FHE upload=GPFQRAIQM4N9 tag=UD5FTJ9SYIAL type=write size=3806198 2024-05-30T16:04:09.972 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.971+0000 7f08851138c0 1 < canceled part key=_multipart_OKBB0VU765TW.2~Y7Z3AUO5RFJ3.0 size=1048576 2024-05-30T16:04:09.975 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.975+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~Y7Z3AUO5RFJ3.1 size=1048576 2024-05-30T16:04:09.978 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.979+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~Y7Z3AUO5RFJ3.2 size=373309 2024-05-30T16:04:09.984 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.983+0000 7f08851138c0 1 < canceled multipart upload key=OKBB0VU765TW upload=Y7Z3AUO5RFJ3 tag=IJW6UGU9YBG9 type=write size=3519037 2024-05-30T16:04:09.991 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.991+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~FPCKLHWVGXQA.0 size=1048576 2024-05-30T16:04:09.994 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.995+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~FPCKLHWVGXQA.1 size=913251 2024-05-30T16:04:09.994 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.995+0000 7f08851138c0 1 > prepared multipart upload key=GWXNCSJCLXKW upload=FPCKLHWVGXQA tag=I5QSOM8UASB1 type=write size=3010403 2024-05-30T16:04:09.997 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:09.999+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~Q8O792PE3USY.0 size=812142 2024-05-30T16:04:10.003 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.003+0000 7f08851138c0 1 < completed multipart upload key=3IJWPWRB6FHE upload=Q8O792PE3USY tag=BSK7BMF01JM7 type=write size=1860718 2024-05-30T16:04:10.010 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.011+0000 7f08851138c0 1 > prepared part key=_multipart_TPCETCO3Y1M6.2~Z0PG5PL0Z2CA.0 size=188366 2024-05-30T16:04:10.010 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.011+0000 7f08851138c0 1 > prepared multipart upload key=TPCETCO3Y1M6 upload=Z0PG5PL0Z2CA tag=OZW9Y7G7DR32 type=write size=1236942 2024-05-30T16:04:10.017 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.019+0000 7f08851138c0 1 < completed delete operation key=OKBB0VU765TW tag=HGDCF385Z0HG type=del 2024-05-30T16:04:10.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.019+0000 7f08851138c0 1 > prepared operation key=1N6R2HBTWMP2 tag=PRCDGQ4INS6A type=write size=157535 2024-05-30T16:04:10.023 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.023+0000 7f08851138c0 1 < canceled operation key=MMLF65KLC42Q tag=ZNRWN1Y07Q63 type=del size=3462440 2024-05-30T16:04:10.026 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.027+0000 7f08851138c0 1 > prepared operation key=A5BUBY9Z7JQG tag=21AW4P329DU8 type=del size=76192 2024-05-30T16:04:10.029 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.027+0000 7f08851138c0 1 < completed delete operation key=TPCETCO3Y1M6 tag=TOXI3GPQI5TO type=del 2024-05-30T16:04:10.032 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.031+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=992QR424B6NC type=del size=1402572 2024-05-30T16:04:10.037 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.035+0000 7f08851138c0 1 < suggested remove operation key=KPHZGFJLC2E0 tag=RUST4RZ8NAPA type=del 2024-05-30T16:04:10.040 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.039+0000 7f08851138c0 1 > prepared operation key=0T2E9F3K1USF tag=S3ZCVV9NAZ72 type=del size=3468421 2024-05-30T16:04:10.042 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.043+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~NCEYLZMH5HCC.0 size=1048576 2024-05-30T16:04:10.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.043+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~NCEYLZMH5HCC.1 size=1048576 2024-05-30T16:04:10.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.047+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~NCEYLZMH5HCC.2 size=202468 2024-05-30T16:04:10.050 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.051+0000 7f08851138c0 1 < completed multipart upload key=GWXNCSJCLXKW upload=NCEYLZMH5HCC tag=3DJ6DJ0RBXS2 type=write size=3348196 2024-05-30T16:04:10.054 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.055+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~NWBU3DDLG58S.0 size=1048576 2024-05-30T16:04:10.056 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.055+0000 7f08851138c0 1 > prepared part key=_multipart_O8CWPJ9GW2GO.2~NWBU3DDLG58S.1 size=186925 2024-05-30T16:04:10.056 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.055+0000 7f08851138c0 1 > prepared multipart upload key=O8CWPJ9GW2GO upload=NWBU3DDLG58S tag=H6CAI4EKOBBJ type=write size=2284077 2024-05-30T16:04:10.059 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.059+0000 7f08851138c0 1 < completed write operation key=73FJ3HGWYQQK tag=T23UJMVN0CAC type=write size=65685 2024-05-30T16:04:10.061 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.063+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=Y7T1WDL2BBZ6 type=del size=2152538 2024-05-30T16:04:10.063 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.063+0000 7f08851138c0 1 < completed delete operation key=E2FXSXOEUGIG tag=FBKSHGJ4NOGL type=del 2024-05-30T16:04:10.066 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.067+0000 7f08851138c0 1 > prepared operation key=JNZI7JX95LX2 tag=SA07PCHCBNDT type=del size=142119 2024-05-30T16:04:10.068 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.067+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~ZFVWLNYALPEV.0 size=1048576 2024-05-30T16:04:10.071 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.071+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~ZFVWLNYALPEV.1 size=1018990 2024-05-30T16:04:10.076 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.075+0000 7f08851138c0 1 < completed multipart upload key=3IJWPWRB6FHE upload=ZFVWLNYALPEV tag=YWL6SQJGZI31 type=write size=3116142 2024-05-30T16:04:10.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.079+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=KKC4ONW6XL3L type=write size=234540 2024-05-30T16:04:10.083 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.083+0000 7f08851138c0 1 < completed delete operation key=3IJWPWRB6FHE tag=FB9VX70SC6UC type=del 2024-05-30T16:04:10.087 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.087+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=UK0W82K359E8 type=del size=68804 2024-05-30T16:04:10.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.091+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~FI75SBR2QXTU.0 size=1048576 2024-05-30T16:04:10.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.091+0000 7f08851138c0 1 < canceled part key=_multipart_364U3OV10VPD.2~FI75SBR2QXTU.1 size=1048576 2024-05-30T16:04:10.093 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.091+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~FI75SBR2QXTU.2 size=56842 2024-05-30T16:04:10.098 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.099+0000 7f08851138c0 1 < canceled multipart upload key=364U3OV10VPD upload=FI75SBR2QXTU tag=072C3MLOD6W6 type=write size=3202570 2024-05-30T16:04:10.101 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.103+0000 7f08851138c0 1 > prepared operation key=0RZRBW38WKNZ tag=VQY77S70YXIN type=del size=4045247 2024-05-30T16:04:10.105 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.103+0000 7f08851138c0 1 < canceled operation key=O8CWPJ9GW2GO tag=CG6LCLCUSAX2 type=del size=1072250 2024-05-30T16:04:10.111 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.111+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~LIM9YCUY6Y6Z.0 size=966880 2024-05-30T16:04:10.111 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.111+0000 7f08851138c0 1 > prepared multipart upload key=Y3CYZJ83WYNI upload=LIM9YCUY6Y6Z tag=LZS7DKYKKFZQ type=write size=2015456 2024-05-30T16:04:10.115 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.115+0000 7f08851138c0 1 < completed delete operation key=WT9RC0UYVG4F tag=9NUMPBFV25QY type=del 2024-05-30T16:04:10.118 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.119+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=XEBEC4UEGDNY type=del size=2132098 2024-05-30T16:04:10.122 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.123+0000 7f08851138c0 1 < completed delete operation key=NIV7AQ4EV5YP tag=VVSZ23Q5HCL2 type=del 2024-05-30T16:04:10.125 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.127+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=H2EAQZJOJ414 type=write size=505267 2024-05-30T16:04:10.129 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.127+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~GPFQRAIQM4N9.0 size=1048576 2024-05-30T16:04:10.132 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.131+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~GPFQRAIQM4N9.1 size=1048576 2024-05-30T16:04:10.135 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.135+0000 7f08851138c0 1 < completed part key=_multipart_3IJWPWRB6FHE.2~GPFQRAIQM4N9.2 size=660470 2024-05-30T16:04:10.142 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.143+0000 7f08851138c0 1 < completed multipart upload key=3IJWPWRB6FHE upload=GPFQRAIQM4N9 tag=UD5FTJ9SYIAL type=write size=3806198 2024-05-30T16:04:10.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.151+0000 7f08851138c0 1 > prepared part key=_multipart_3TBOZWT2P8U8.2~1QA28H4CGF1C.0 size=835230 2024-05-30T16:04:10.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.151+0000 7f08851138c0 1 > prepared multipart upload key=3TBOZWT2P8U8 upload=1QA28H4CGF1C tag=SJW6VV4JOMQF type=write size=1883806 2024-05-30T16:04:10.153 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.155+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~FPCKLHWVGXQA.0 size=1048576 2024-05-30T16:04:10.156 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.155+0000 7f08851138c0 1 < completed part key=_multipart_GWXNCSJCLXKW.2~FPCKLHWVGXQA.1 size=913251 2024-05-30T16:04:10.163 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.163+0000 7f08851138c0 1 < completed multipart upload key=GWXNCSJCLXKW upload=FPCKLHWVGXQA tag=I5QSOM8UASB1 type=write size=3010403 2024-05-30T16:04:10.169 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.167+0000 7f08851138c0 1 > prepared part key=_multipart_WT9RC0UYVG4F.2~IP0QM34AHKD6.0 size=42570 2024-05-30T16:04:10.169 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.167+0000 7f08851138c0 1 > prepared multipart upload key=WT9RC0UYVG4F upload=IP0QM34AHKD6 tag=B8SCN97RXQOB type=write size=1091146 2024-05-30T16:04:10.173 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.171+0000 7f08851138c0 1 < completed part key=_multipart_TPCETCO3Y1M6.2~Z0PG5PL0Z2CA.0 size=188366 2024-05-30T16:04:10.178 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.179+0000 7f08851138c0 1 < completed multipart upload key=TPCETCO3Y1M6 upload=Z0PG5PL0Z2CA tag=OZW9Y7G7DR32 type=write size=1236942 2024-05-30T16:04:10.184 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.183+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~GM1EXTU9YUOC.0 size=1048576 2024-05-30T16:04:10.187 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.187+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~GM1EXTU9YUOC.1 size=1048576 2024-05-30T16:04:10.189 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.191+0000 7f08851138c0 1 > prepared part key=_multipart_PLWFSKPKVO19.2~GM1EXTU9YUOC.2 size=112651 2024-05-30T16:04:10.190 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.191+0000 7f08851138c0 1 > prepared multipart upload key=PLWFSKPKVO19 upload=GM1EXTU9YUOC tag=P00DYETV3RYM type=write size=3258379 2024-05-30T16:04:10.196 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.195+0000 7f08851138c0 1 < suggested update operation key=1N6R2HBTWMP2 tag=PRCDGQ4INS6A type=write size=157535 2024-05-30T16:04:10.202 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.203+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~7F5AOCIW4TJ7.0 size=1048576 2024-05-30T16:04:10.205 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.207+0000 7f08851138c0 1 > prepared part key=_multipart_OKBB0VU765TW.2~7F5AOCIW4TJ7.1 size=573546 2024-05-30T16:04:10.205 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.207+0000 7f08851138c0 1 > prepared multipart upload key=OKBB0VU765TW upload=7F5AOCIW4TJ7 tag=0G2Y1PK9OVSC type=write size=2670698 2024-05-30T16:04:10.209 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.207+0000 7f08851138c0 1 < completed delete operation key=A5BUBY9Z7JQG tag=21AW4P329DU8 type=del 2024-05-30T16:04:10.215 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.215+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~2WXXOTWNHLZ3.0 size=1048576 2024-05-30T16:04:10.218 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.219+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~2WXXOTWNHLZ3.1 size=1048576 2024-05-30T16:04:10.220 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.219+0000 7f08851138c0 1 > prepared part key=_multipart_XHGMUYBA2VNK.2~2WXXOTWNHLZ3.2 size=933689 2024-05-30T16:04:10.220 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.219+0000 7f08851138c0 1 > prepared multipart upload key=XHGMUYBA2VNK upload=2WXXOTWNHLZ3 tag=SQ4J1NNH9LJG type=write size=4079417 2024-05-30T16:04:10.224 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.223+0000 7f08851138c0 1 < completed delete operation key=364U3OV10VPD tag=992QR424B6NC type=del 2024-05-30T16:04:10.228 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.227+0000 7f08851138c0 1 > prepared operation key=364U3OV10VPD tag=PMMG76BAU2JQ type=del size=1436675 2024-05-30T16:04:10.232 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.231+0000 7f08851138c0 1 < completed delete operation key=0T2E9F3K1USF tag=S3ZCVV9NAZ72 type=del 2024-05-30T16:04:10.238 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.239+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~63SUXUJW7RLZ.0 size=1048576 2024-05-30T16:04:10.241 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.243+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~63SUXUJW7RLZ.1 size=1048576 2024-05-30T16:04:10.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.243+0000 7f08851138c0 1 > prepared part key=_multipart_QQILXRJD0C1Z.2~63SUXUJW7RLZ.2 size=281581 2024-05-30T16:04:10.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.243+0000 7f08851138c0 1 > prepared multipart upload key=QQILXRJD0C1Z upload=63SUXUJW7RLZ tag=RQDQ0OESX8KX type=write size=3427309 2024-05-30T16:04:10.248 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.247+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~NWBU3DDLG58S.0 size=1048576 2024-05-30T16:04:10.251 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.251+0000 7f08851138c0 1 < completed part key=_multipart_O8CWPJ9GW2GO.2~NWBU3DDLG58S.1 size=186925 2024-05-30T16:04:10.257 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.259+0000 7f08851138c0 1 < completed multipart upload key=O8CWPJ9GW2GO upload=NWBU3DDLG58S tag=H6CAI4EKOBBJ type=write size=2284077 2024-05-30T16:04:10.261 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.263+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=P03KOT85O6FI type=del size=1219887 2024-05-30T16:04:10.266 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.267+0000 7f08851138c0 1 < completed delete operation key=603HBR8PRH6L tag=Y7T1WDL2BBZ6 type=del 2024-05-30T16:04:10.269 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.267+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=0S30L0DY9ZFU type=del size=453148 2024-05-30T16:04:10.277 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.275+0000 7f08851138c0 1 < suggested remove operation key=JNZI7JX95LX2 tag=SA07PCHCBNDT type=del 2024-05-30T16:04:10.280 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.279+0000 7f08851138c0 1 > prepared operation key=3TBOZWT2P8U8 tag=L2RMK7K7STM1 type=del size=540516 2024-05-30T16:04:10.284 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.283+0000 7f08851138c0 1 < completed write operation key=3TBOZWT2P8U8 tag=KKC4ONW6XL3L type=write size=234540 2024-05-30T16:04:10.290 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.291+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~70A9FZOI31Y7.0 size=1048576 2024-05-30T16:04:10.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.291+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~70A9FZOI31Y7.1 size=1048576 2024-05-30T16:04:10.296 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.295+0000 7f08851138c0 1 > prepared part key=_multipart_364U3OV10VPD.2~70A9FZOI31Y7.2 size=508691 2024-05-30T16:04:10.296 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.295+0000 7f08851138c0 1 > prepared multipart upload key=364U3OV10VPD upload=70A9FZOI31Y7 tag=X6IYXG6K2EUJ type=write size=3654419 2024-05-30T16:04:10.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.299+0000 7f08851138c0 1 < canceled operation key=WT9RC0UYVG4F tag=UK0W82K359E8 type=del size=68804 2024-05-30T16:04:10.304 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.303+0000 7f08851138c0 1 > prepared operation key=603HBR8PRH6L tag=1X8WZU9W6IMU type=del size=1005712 2024-05-30T16:04:10.312 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.311+0000 7f08851138c0 1 < suggested update operation key=0RZRBW38WKNZ tag=VQY77S70YXIN type=del size=4045247 2024-05-30T16:04:10.316 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.315+0000 7f08851138c0 1 > prepared operation key=PLWFSKPKVO19 tag=WM9SP1C9SWVP type=write size=899581 2024-05-30T16:04:10.320 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.319+0000 7f08851138c0 1 < completed part key=_multipart_Y3CYZJ83WYNI.2~LIM9YCUY6Y6Z.0 size=966880 2024-05-30T16:04:10.326 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.327+0000 7f08851138c0 1 < completed multipart upload key=Y3CYZJ83WYNI upload=LIM9YCUY6Y6Z tag=LZS7DKYKKFZQ type=write size=2015456 2024-05-30T16:04:10.334 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.335+0000 7f08851138c0 1 > prepared operation key=WT9RC0UYVG4F tag=G6W7US24VR9W type=del size=2597118 2024-05-30T16:04:10.338 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.339+0000 7f08851138c0 1 < completed delete operation key=TPCETCO3Y1M6 tag=XEBEC4UEGDNY type=del 2024-05-30T16:04:10.342 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.343+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=GCXTVCW9HGAT type=del size=1255880 2024-05-30T16:04:10.346 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.347+0000 7f08851138c0 1 < completed write operation key=TPCETCO3Y1M6 tag=H2EAQZJOJ414 type=write size=505267 2024-05-30T16:04:10.353 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.351+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~V1OFTYT69FUT.0 size=1048576 2024-05-30T16:04:10.355 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.355+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~V1OFTYT69FUT.1 size=1048576 2024-05-30T16:04:10.358 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.359+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~V1OFTYT69FUT.2 size=313892 2024-05-30T16:04:10.358 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.359+0000 7f08851138c0 1 > prepared multipart upload key=YRP4CSG92QRO upload=V1OFTYT69FUT tag=DSGY9E0LOUQ4 type=write size=3459620 2024-05-30T16:04:10.362 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.363+0000 7f08851138c0 1 < completed part key=_multipart_3TBOZWT2P8U8.2~1QA28H4CGF1C.0 size=835230 2024-05-30T16:04:10.367 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.367+0000 7f08851138c0 1 < completed multipart upload key=3TBOZWT2P8U8 upload=1QA28H4CGF1C tag=SJW6VV4JOMQF type=write size=1883806 2024-05-30T16:04:10.371 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.371+0000 7f08851138c0 1 > prepared operation key=E2FXSXOEUGIG tag=TR086161ZLF2 type=del size=3576913 2024-05-30T16:04:10.372 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.371+0000 7f08851138c0 1 < canceled part key=_multipart_WT9RC0UYVG4F.2~IP0QM34AHKD6.0 size=42570 2024-05-30T16:04:10.377 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.379+0000 7f08851138c0 1 < canceled multipart upload key=WT9RC0UYVG4F upload=IP0QM34AHKD6 tag=B8SCN97RXQOB type=write size=1091146 2024-05-30T16:04:10.381 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.379+0000 7f08851138c0 1 > prepared operation key=NIV7AQ4EV5YP tag=IX3JNPXW7FZB type=del size=706197 2024-05-30T16:04:10.385 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.383+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~GM1EXTU9YUOC.0 size=1048576 2024-05-30T16:04:10.388 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.387+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~GM1EXTU9YUOC.1 size=1048576 2024-05-30T16:04:10.391 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.391+0000 7f08851138c0 1 < completed part key=_multipart_PLWFSKPKVO19.2~GM1EXTU9YUOC.2 size=112651 2024-05-30T16:04:10.397 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.395+0000 7f08851138c0 1 < completed multipart upload key=PLWFSKPKVO19 upload=GM1EXTU9YUOC tag=P00DYETV3RYM type=write size=3258379 2024-05-30T16:04:10.401 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.399+0000 7f08851138c0 1 > prepared operation key=O0DJ8SCY766V tag=YD0GUR6HFYY9 type=del size=1142245 2024-05-30T16:04:10.404 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.403+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~7F5AOCIW4TJ7.0 size=1048576 2024-05-30T16:04:10.408 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.407+0000 7f08851138c0 1 < completed part key=_multipart_OKBB0VU765TW.2~7F5AOCIW4TJ7.1 size=573546 2024-05-30T16:04:10.414 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.415+0000 7f08851138c0 1 < completed multipart upload key=OKBB0VU765TW upload=7F5AOCIW4TJ7 tag=0G2Y1PK9OVSC type=write size=2670698 2024-05-30T16:04:10.420 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.419+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~JOYX3CTK2MTB.0 size=1048576 2024-05-30T16:04:10.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.423+0000 7f08851138c0 1 > prepared part key=_multipart_NIV7AQ4EV5YP.2~JOYX3CTK2MTB.1 size=186733 2024-05-30T16:04:10.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.423+0000 7f08851138c0 1 > prepared multipart upload key=NIV7AQ4EV5YP upload=JOYX3CTK2MTB tag=4DYWBT2DCP70 type=write size=2283885 2024-05-30T16:04:10.427 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.427+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~2WXXOTWNHLZ3.0 size=1048576 2024-05-30T16:04:10.429 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.431+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~2WXXOTWNHLZ3.1 size=1048576 2024-05-30T16:04:10.432 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.431+0000 7f08851138c0 1 < completed part key=_multipart_XHGMUYBA2VNK.2~2WXXOTWNHLZ3.2 size=933689 2024-05-30T16:04:10.438 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.439+0000 7f08851138c0 1 < completed multipart upload key=XHGMUYBA2VNK upload=2WXXOTWNHLZ3 tag=SQ4J1NNH9LJG type=write size=4079417 2024-05-30T16:04:10.441 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.443+0000 7f08851138c0 1 > prepared operation key=TPCETCO3Y1M6 tag=HN2IIIEN5XFI type=write size=539692 2024-05-30T16:04:10.445 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.443+0000 7f08851138c0 1 < completed delete operation key=364U3OV10VPD tag=PMMG76BAU2JQ type=del 2024-05-30T16:04:10.452 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.451+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~4TG4WLXF6W3I.0 size=1048576 2024-05-30T16:04:10.454 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.455+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~4TG4WLXF6W3I.1 size=1048576 2024-05-30T16:04:10.457 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.455+0000 7f08851138c0 1 > prepared part key=_multipart_0RZRBW38WKNZ.2~4TG4WLXF6W3I.2 size=581609 2024-05-30T16:04:10.457 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.455+0000 7f08851138c0 1 > prepared multipart upload key=0RZRBW38WKNZ upload=4TG4WLXF6W3I tag=JJNMCJY021O5 type=write size=3727337 2024-05-30T16:04:10.461 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.459+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~63SUXUJW7RLZ.0 size=1048576 2024-05-30T16:04:10.461 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.459+0000 7f08851138c0 1 < canceled part key=_multipart_QQILXRJD0C1Z.2~63SUXUJW7RLZ.1 size=1048576 2024-05-30T16:04:10.464 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.463+0000 7f08851138c0 1 < completed part key=_multipart_QQILXRJD0C1Z.2~63SUXUJW7RLZ.2 size=281581 2024-05-30T16:04:10.471 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.471+0000 7f08851138c0 1 < canceled multipart upload key=QQILXRJD0C1Z upload=63SUXUJW7RLZ tag=RQDQ0OESX8KX type=write size=3427309 2024-05-30T16:04:10.477 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.479+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~PJXP655W48TH.0 size=1048576 2024-05-30T16:04:10.480 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.479+0000 7f08851138c0 1 > prepared part key=_multipart_YRP4CSG92QRO.2~PJXP655W48TH.1 size=404162 2024-05-30T16:04:10.480 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.479+0000 7f08851138c0 1 > prepared multipart upload key=YRP4CSG92QRO upload=PJXP655W48TH tag=HA6OV6V524Y7 type=write size=2501314 2024-05-30T16:04:10.484 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.483+0000 7f08851138c0 1 < completed delete operation key=E2FXSXOEUGIG tag=P03KOT85O6FI type=del 2024-05-30T16:04:10.491 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.491+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~DY2BOQAH93T7.0 size=1048576 2024-05-30T16:04:10.494 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.495+0000 7f08851138c0 1 > prepared part key=_multipart_GWXNCSJCLXKW.2~DY2BOQAH93T7.1 size=382840 2024-05-30T16:04:10.495 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.495+0000 7f08851138c0 1 > prepared multipart upload key=GWXNCSJCLXKW upload=DY2BOQAH93T7 tag=XQ2PBGXULSMA type=write size=2479992 2024-05-30T16:04:10.502 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.503+0000 7f08851138c0 1 < suggested update operation key=NIV7AQ4EV5YP tag=0S30L0DY9ZFU type=del size=453148 2024-05-30T16:04:10.506 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.507+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=Y5ID4LEIVVA1 type=write size=648232 2024-05-30T16:04:10.510 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.511+0000 7f08851138c0 1 < completed delete operation key=3TBOZWT2P8U8 tag=L2RMK7K7STM1 type=del 2024-05-30T16:04:10.517 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.515+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~V44UTHH4ENLK.0 size=1048576 2024-05-30T16:04:10.520 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.519+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~V44UTHH4ENLK.1 size=1048576 2024-05-30T16:04:10.523 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.523+0000 7f08851138c0 1 > prepared part key=_multipart_Y3CYZJ83WYNI.2~V44UTHH4ENLK.2 size=418630 2024-05-30T16:04:10.523 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.523+0000 7f08851138c0 1 > prepared multipart upload key=Y3CYZJ83WYNI upload=V44UTHH4ENLK tag=6N0T2JWQX9Q4 type=write size=3564358 2024-05-30T16:04:10.526 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.527+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~70A9FZOI31Y7.0 size=1048576 2024-05-30T16:04:10.529 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.531+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~70A9FZOI31Y7.1 size=1048576 2024-05-30T16:04:10.532 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.531+0000 7f08851138c0 1 < completed part key=_multipart_364U3OV10VPD.2~70A9FZOI31Y7.2 size=508691 2024-05-30T16:04:10.538 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.539+0000 7f08851138c0 1 < completed multipart upload key=364U3OV10VPD upload=70A9FZOI31Y7 tag=X6IYXG6K2EUJ type=write size=3654419 2024-05-30T16:04:10.542 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:04:10.543+0000 7f08851138c0 1 > prepared operation key=73FJ3HGWYQQK tag=9GOJ8ZSBU9IT type=del size=121740 2024-05-30T16:04:10.543 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] cls_rgw_stats.simulate (25783 ms) 2024-05-30T16:04:10.543 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 1 test from cls_rgw_stats (25783 ms total) 2024-05-30T16:04:10.543 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-05-30T16:04:10.543 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment tear-down 2024-05-30T16:04:10.677 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] 1 test from 1 test suite ran. (27829 ms total) 2024-05-30T16:04:10.678 INFO:tasks.workunit.client.0.smithi006.stdout:[ PASSED ] 1 test. 2024-05-30T16:04:10.687 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-30T16:04:10.687 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-30T16:04:10.747 INFO:tasks.workunit:Running workunits matching cls/test_cls_cmpomap.sh on client.0... 2024-05-30T16:04:10.748 INFO:tasks.workunit:Running workunit cls/test_cls_cmpomap.sh... 2024-05-30T16:04:10.749 DEBUG:teuthology.orchestra.run.smithi006:workunit test cls/test_cls_cmpomap.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_cmpomap.sh 2024-05-30T16:04:10.812 INFO:tasks.workunit.client.0.smithi006.stdout:Running main() from gmock_main.cc 2024-05-30T16:04:10.813 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] Running 31 tests from 1 test suite. 2024-05-30T16:04:10.813 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment set-up. 2024-05-30T16:04:12.691 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 31 tests from CmpOmap 2024-05-30T16:04:12.691 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_str 2024-05-30T16:04:12.731 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] CmpOmap.cmp_vals_noexist_str (39 ms) 2024-05-30T16:04:12.731 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_str_default 2024-05-30T16:04:12.736 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] CmpOmap.cmp_vals_noexist_str_default (6 ms) 2024-05-30T16:04:12.736 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_u64 2024-05-30T16:04:12.741 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] CmpOmap.cmp_vals_noexist_u64 (5 ms) 2024-05-30T16:04:12.741 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_u64_default 2024-05-30T16:04:12.747 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] CmpOmap.cmp_vals_noexist_u64_default (6 ms) 2024-05-30T16:04:12.747 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] CmpOmap.cmp_vals_str 2024-05-30T16:04:12.760 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] CmpOmap.cmp_vals_str (13 ms) 2024-05-30T16:04:12.760 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] CmpOmap.cmp_vals_u64 2024-05-30T16:04:12.772 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] CmpOmap.cmp_vals_u64 (12 ms) 2024-05-30T16:04:12.773 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_input 2024-05-30T16:04:12.780 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_input (8 ms) 2024-05-30T16:04:12.780 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_default 2024-05-30T16:04:12.788 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_default (8 ms) 2024-05-30T16:04:12.788 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] CmpOmap.cmp_vals_u64_empty_default 2024-05-30T16:04:12.799 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] CmpOmap.cmp_vals_u64_empty_default (11 ms) 2024-05-30T16:04:12.799 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_value 2024-05-30T16:04:12.810 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_value (11 ms) 2024-05-30T16:04:12.810 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] CmpOmap.cmp_vals_at_max_keys 2024-05-30T16:04:12.811 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] CmpOmap.cmp_vals_at_max_keys (1 ms) 2024-05-30T16:04:12.811 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] CmpOmap.cmp_vals_over_max_keys 2024-05-30T16:04:12.813 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] CmpOmap.cmp_vals_over_max_keys (2 ms) 2024-05-30T16:04:12.813 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_str 2024-05-30T16:04:12.828 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_str (15 ms) 2024-05-30T16:04:12.828 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_str_default 2024-05-30T16:04:12.847 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_str_default (18 ms) 2024-05-30T16:04:12.847 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_u64 2024-05-30T16:04:12.864 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_u64 (18 ms) 2024-05-30T16:04:12.864 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_u64_default 2024-05-30T16:04:12.880 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_u64_default (16 ms) 2024-05-30T16:04:12.880 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] CmpOmap.cmp_set_vals_str 2024-05-30T16:04:12.901 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] CmpOmap.cmp_set_vals_str (21 ms) 2024-05-30T16:04:12.901 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64 2024-05-30T16:04:12.923 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] CmpOmap.cmp_set_vals_u64 (22 ms) 2024-05-30T16:04:12.923 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64_einval 2024-05-30T16:04:12.930 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] CmpOmap.cmp_set_vals_u64_einval (7 ms) 2024-05-30T16:04:12.931 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64_eio 2024-05-30T16:04:12.937 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] CmpOmap.cmp_set_vals_u64_eio (7 ms) 2024-05-30T16:04:12.937 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] CmpOmap.cmp_set_vals_at_max_keys 2024-05-30T16:04:12.940 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] CmpOmap.cmp_set_vals_at_max_keys (3 ms) 2024-05-30T16:04:12.940 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] CmpOmap.cmp_set_vals_over_max_keys 2024-05-30T16:04:12.942 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] CmpOmap.cmp_set_vals_over_max_keys (3 ms) 2024-05-30T16:04:12.943 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] CmpOmap.cmp_rm_keys_noexist_str 2024-05-30T16:04:12.960 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] CmpOmap.cmp_rm_keys_noexist_str (17 ms) 2024-05-30T16:04:12.960 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] CmpOmap.cmp_rm_keys_noexist_u64 2024-05-30T16:04:12.977 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] CmpOmap.cmp_rm_keys_noexist_u64 (17 ms) 2024-05-30T16:04:12.977 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] CmpOmap.cmp_rm_keys_str 2024-05-30T16:04:12.998 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] CmpOmap.cmp_rm_keys_str (21 ms) 2024-05-30T16:04:12.998 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64 2024-05-30T16:04:13.020 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64 (22 ms) 2024-05-30T16:04:13.020 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_einval 2024-05-30T16:04:13.026 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_einval (6 ms) 2024-05-30T16:04:13.026 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_eio 2024-05-30T16:04:13.033 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_eio (7 ms) 2024-05-30T16:04:13.033 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] CmpOmap.cmp_rm_keys_at_max_keys 2024-05-30T16:04:13.036 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] CmpOmap.cmp_rm_keys_at_max_keys (3 ms) 2024-05-30T16:04:13.036 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] CmpOmap.cmp_rm_keys_over_max_keys 2024-05-30T16:04:13.038 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] CmpOmap.cmp_rm_keys_over_max_keys (2 ms) 2024-05-30T16:04:13.038 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_empty 2024-05-30T16:04:13.058 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_empty (20 ms) 2024-05-30T16:04:13.058 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 31 tests from CmpOmap (367 ms total) 2024-05-30T16:04:13.059 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-05-30T16:04:13.059 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment tear-down 2024-05-30T16:04:13.701 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] 31 tests from 1 test suite ran. (2888 ms total) 2024-05-30T16:04:13.701 INFO:tasks.workunit.client.0.smithi006.stdout:[ PASSED ] 31 tests. 2024-05-30T16:04:13.706 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-30T16:04:13.706 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-30T16:04:13.763 INFO:tasks.workunit:Running workunits matching cls/test_cls_2pc_queue.sh on client.0... 2024-05-30T16:04:13.764 INFO:tasks.workunit:Running workunit cls/test_cls_2pc_queue.sh... 2024-05-30T16:04:13.765 DEBUG:teuthology.orchestra.run.smithi006:workunit test cls/test_cls_2pc_queue.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_2pc_queue.sh 2024-05-30T16:04:13.831 INFO:tasks.workunit.client.0.smithi006.stdout:Running main() from gmock_main.cc 2024-05-30T16:04:13.831 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] Running 24 tests from 1 test suite. 2024-05-30T16:04:13.831 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment set-up. 2024-05-30T16:04:13.831 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 24 tests from TestCls2PCQueue 2024-05-30T16:04:13.831 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] TestCls2PCQueue.GetCapacity 2024-05-30T16:04:16.740 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] TestCls2PCQueue.GetCapacity (2909 ms) 2024-05-30T16:04:16.740 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] TestCls2PCQueue.AsyncGetCapacity 2024-05-30T16:04:19.761 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] TestCls2PCQueue.AsyncGetCapacity (3021 ms) 2024-05-30T16:04:19.762 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] TestCls2PCQueue.Reserve 2024-05-30T16:04:22.821 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] TestCls2PCQueue.Reserve (3060 ms) 2024-05-30T16:04:22.822 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] TestCls2PCQueue.AsyncReserve 2024-05-30T16:04:25.866 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] TestCls2PCQueue.AsyncReserve (3044 ms) 2024-05-30T16:04:25.866 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] TestCls2PCQueue.Commit 2024-05-30T16:04:29.910 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] TestCls2PCQueue.Commit (4045 ms) 2024-05-30T16:04:29.911 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] TestCls2PCQueue.Stats 2024-05-30T16:04:33.953 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] TestCls2PCQueue.Stats (4043 ms) 2024-05-30T16:04:33.953 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] TestCls2PCQueue.UpgradeFromReef 2024-05-30T16:04:38.008 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] TestCls2PCQueue.UpgradeFromReef (4054 ms) 2024-05-30T16:04:38.008 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] TestCls2PCQueue.Abort 2024-05-30T16:04:41.031 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] TestCls2PCQueue.Abort (3024 ms) 2024-05-30T16:04:41.031 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] TestCls2PCQueue.ReserveError 2024-05-30T16:04:44.039 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] TestCls2PCQueue.ReserveError (3008 ms) 2024-05-30T16:04:44.040 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] TestCls2PCQueue.CommitError 2024-05-30T16:04:47.118 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] TestCls2PCQueue.CommitError (3078 ms) 2024-05-30T16:04:47.118 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] TestCls2PCQueue.AbortError 2024-05-30T16:04:50.150 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] TestCls2PCQueue.AbortError (3033 ms) 2024-05-30T16:04:50.150 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] TestCls2PCQueue.MultiReserve 2024-05-30T16:04:53.158 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] TestCls2PCQueue.MultiReserve (3007 ms) 2024-05-30T16:04:53.158 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] TestCls2PCQueue.MultiCommit 2024-05-30T16:04:56.166 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] TestCls2PCQueue.MultiCommit (3008 ms) 2024-05-30T16:04:56.166 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] TestCls2PCQueue.MultiAbort 2024-05-30T16:04:59.176 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] TestCls2PCQueue.MultiAbort (3011 ms) 2024-05-30T16:04:59.177 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] TestCls2PCQueue.ReserveCommit 2024-05-30T16:05:02.233 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] TestCls2PCQueue.ReserveCommit (3057 ms) 2024-05-30T16:05:02.234 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] TestCls2PCQueue.ReserveAbort 2024-05-30T16:05:05.242 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] TestCls2PCQueue.ReserveAbort (3009 ms) 2024-05-30T16:05:05.242 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] TestCls2PCQueue.ManualCleanup 2024-05-30T16:05:08.325 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] TestCls2PCQueue.ManualCleanup (3083 ms) 2024-05-30T16:05:08.325 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] TestCls2PCQueue.Cleanup 2024-05-30T16:05:11.335 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] TestCls2PCQueue.Cleanup (3010 ms) 2024-05-30T16:05:11.336 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] TestCls2PCQueue.MultiProducer 2024-05-30T16:05:24.753 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] TestCls2PCQueue.MultiProducer (13417 ms) 2024-05-30T16:05:24.753 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] TestCls2PCQueue.AsyncConsumer 2024-05-30T16:05:29.284 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] TestCls2PCQueue.AsyncConsumer (4532 ms) 2024-05-30T16:05:29.284 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] TestCls2PCQueue.MultiProducerConsumer 2024-05-30T16:05:42.758 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] TestCls2PCQueue.MultiProducerConsumer (13474 ms) 2024-05-30T16:05:42.759 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] TestCls2PCQueue.ReserveSpillover 2024-05-30T16:05:48.758 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] TestCls2PCQueue.ReserveSpillover (6000 ms) 2024-05-30T16:05:48.759 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] TestCls2PCQueue.CommitSpillover 2024-05-30T16:05:58.766 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] TestCls2PCQueue.CommitSpillover (10008 ms) 2024-05-30T16:05:58.766 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] TestCls2PCQueue.AbortSpillover 2024-05-30T16:06:08.762 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] TestCls2PCQueue.AbortSpillover (9995 ms) 2024-05-30T16:06:08.762 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 24 tests from TestCls2PCQueue (114930 ms total) 2024-05-30T16:06:08.762 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-05-30T16:06:08.762 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment tear-down 2024-05-30T16:06:08.762 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] 24 tests from 1 test suite ran. (114930 ms total) 2024-05-30T16:06:08.763 INFO:tasks.workunit.client.0.smithi006.stdout:[ PASSED ] 24 tests. 2024-05-30T16:06:08.768 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-30T16:06:08.768 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-30T16:06:08.823 INFO:tasks.workunit:Running workunits matching cls/test_cls_user.sh on client.0... 2024-05-30T16:06:08.824 INFO:tasks.workunit:Running workunit cls/test_cls_user.sh... 2024-05-30T16:06:08.824 DEBUG:teuthology.orchestra.run.smithi006:workunit test cls/test_cls_user.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_user.sh 2024-05-30T16:06:08.902 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:06:08.902 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:06:08.902 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] Running 4 tests from 1 test suite. 2024-05-30T16:06:08.903 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment set-up. 2024-05-30T16:06:10.767 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 4 tests from ClsAccount 2024-05-30T16:06:10.767 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] ClsAccount.add 2024-05-30T16:06:10.811 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] ClsAccount.add (45 ms) 2024-05-30T16:06:10.812 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] ClsAccount.get 2024-05-30T16:06:10.821 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] ClsAccount.get (10 ms) 2024-05-30T16:06:10.821 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] ClsAccount.rm 2024-05-30T16:06:11.044 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] ClsAccount.rm (222 ms) 2024-05-30T16:06:11.044 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] ClsAccount.list 2024-05-30T16:06:11.088 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] ClsAccount.list (45 ms) 2024-05-30T16:06:11.088 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 4 tests from ClsAccount (322 ms total) 2024-05-30T16:06:11.088 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-05-30T16:06:11.088 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment tear-down 2024-05-30T16:06:11.794 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] 4 tests from 1 test suite ran. (2892 ms total) 2024-05-30T16:06:11.794 INFO:tasks.workunit.client.0.smithi006.stdout:[ PASSED ] 4 tests. 2024-05-30T16:06:11.800 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-30T16:06:11.800 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-30T16:06:11.858 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_gc_log.sh on client.0... 2024-05-30T16:06:11.859 INFO:tasks.workunit:Running workunit rgw/test_rgw_gc_log.sh... 2024-05-30T16:06:11.859 DEBUG:teuthology.orchestra.run.smithi006:workunit test rgw/test_rgw_gc_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_gc_log.sh 2024-05-30T16:06:11.935 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:06:11.935 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:06:11.936 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] Running 6 tests from 1 test suite. 2024-05-30T16:06:11.936 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment set-up. 2024-05-30T16:06:13.808 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 6 tests from rgw_gc_log 2024-05-30T16:06:13.808 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] rgw_gc_log.init_existing_queue 2024-05-30T16:06:13.839 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] rgw_gc_log.init_existing_queue (31 ms) 2024-05-30T16:06:13.839 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] rgw_gc_log.init_existing_omap 2024-05-30T16:06:13.844 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] rgw_gc_log.init_existing_omap (5 ms) 2024-05-30T16:06:13.844 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] rgw_gc_log.enqueue1_after_init 2024-05-30T16:06:13.851 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] rgw_gc_log.enqueue1_after_init (7 ms) 2024-05-30T16:06:13.851 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] rgw_gc_log.enqueue2_before_init 2024-05-30T16:06:13.854 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] rgw_gc_log.enqueue2_before_init (3 ms) 2024-05-30T16:06:13.854 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] rgw_gc_log.defer1_after_init 2024-05-30T16:06:13.858 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] rgw_gc_log.defer1_after_init (4 ms) 2024-05-30T16:06:13.858 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] rgw_gc_log.defer2_before_init 2024-05-30T16:06:13.861 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] rgw_gc_log.defer2_before_init (3 ms) 2024-05-30T16:06:13.861 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 6 tests from rgw_gc_log (53 ms total) 2024-05-30T16:06:13.862 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-05-30T16:06:13.862 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment tear-down 2024-05-30T16:06:14.802 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] 6 tests from 1 test suite ran. (2867 ms total) 2024-05-30T16:06:14.803 INFO:tasks.workunit.client.0.smithi006.stdout:[ PASSED ] 6 tests. 2024-05-30T16:06:14.809 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-30T16:06:14.809 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-30T16:06:14.867 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_obj.sh on client.0... 2024-05-30T16:06:14.868 INFO:tasks.workunit:Running workunit rgw/test_rgw_obj.sh... 2024-05-30T16:06:14.869 DEBUG:teuthology.orchestra.run.smithi006:workunit test rgw/test_rgw_obj.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_obj.sh 2024-05-30T16:06:14.940 INFO:tasks.workunit.client.0.smithi006.stdout:Running main() from gmock_main.cc 2024-05-30T16:06:14.940 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] Running 4 tests from 1 test suite. 2024-05-30T16:06:14.941 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment set-up. 2024-05-30T16:06:14.941 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 4 tests from TestRGWObj 2024-05-30T16:06:14.941 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] TestRGWObj.underscore 2024-05-30T16:06:14.941 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: _obj 2024-05-30T16:06:14.941 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: _obj 2024-05-30T16:06:14.941 INFO:tasks.workunit.client.0.smithi006.stdout:stripped: _obj ns= i= 2024-05-30T16:06:14.941 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: _obj 2024-05-30T16:06:14.941 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: _obj 2024-05-30T16:06:14.941 INFO:tasks.workunit.client.0.smithi006.stdout:stripped: _obj ns= i= 2024-05-30T16:06:14.941 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: _obj 2024-05-30T16:06:14.941 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: _obj 2024-05-30T16:06:14.941 INFO:tasks.workunit.client.0.smithi006.stdout:stripped: _obj ns= i= 2024-05-30T16:06:14.941 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:14.942 INFO:tasks.workunit.client.0.smithi006.stdout: "obj1": { 2024-05-30T16:06:14.942 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.942 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.942 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.942 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.942 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.942 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.942 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.942 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.942 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.942 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.942 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.942 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.943 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_obj", 2024-05-30T16:06:14.943 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:14.943 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:14.943 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.943 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.943 INFO:tasks.workunit.client.0.smithi006.stdout: "obj2": { 2024-05-30T16:06:14.943 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.943 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.943 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.943 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.943 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.943 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.943 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.944 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.944 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.944 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.944 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.944 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.944 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_obj", 2024-05-30T16:06:14.944 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:14.944 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:14.944 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.944 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.944 INFO:tasks.workunit.client.0.smithi006.stdout: "obj3": { 2024-05-30T16:06:14.944 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.944 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.944 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.945 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.945 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.945 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.945 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.945 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.945 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.945 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.945 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.945 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.945 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_obj", 2024-05-30T16:06:14.945 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:14.945 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:14.945 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.946 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.946 INFO:tasks.workunit.client.0.smithi006.stdout: "obj3-2": { 2024-05-30T16:06:14.946 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.946 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.946 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.946 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.946 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.946 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.946 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.946 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.946 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.946 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.946 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.947 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.947 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_obj", 2024-05-30T16:06:14.947 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:14.947 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:14.947 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.947 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.947 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:14.947 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: _obj 2024-05-30T16:06:14.947 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: _obj 2024-05-30T16:06:14.947 INFO:tasks.workunit.client.0.smithi006.stdout:stripped: _obj ns=ns i= 2024-05-30T16:06:14.947 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: _obj 2024-05-30T16:06:14.947 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: _obj 2024-05-30T16:06:14.947 INFO:tasks.workunit.client.0.smithi006.stdout:stripped: _obj ns=ns i= 2024-05-30T16:06:14.947 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: _obj 2024-05-30T16:06:14.948 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: _obj 2024-05-30T16:06:14.948 INFO:tasks.workunit.client.0.smithi006.stdout:stripped: _obj ns=ns i= 2024-05-30T16:06:14.948 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:14.948 INFO:tasks.workunit.client.0.smithi006.stdout: "obj1": { 2024-05-30T16:06:14.948 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.948 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.948 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.948 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.948 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.948 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.948 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.948 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.948 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.949 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.949 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.949 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.949 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_obj", 2024-05-30T16:06:14.949 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:14.949 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:14.949 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.949 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.949 INFO:tasks.workunit.client.0.smithi006.stdout: "obj2": { 2024-05-30T16:06:14.949 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.949 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.949 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.949 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.949 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.950 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.950 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.950 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.950 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.950 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.950 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.950 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.950 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_obj", 2024-05-30T16:06:14.950 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:14.950 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:14.950 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.950 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.950 INFO:tasks.workunit.client.0.smithi006.stdout: "obj3": { 2024-05-30T16:06:14.950 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.951 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.951 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.951 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.951 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.951 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.951 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.951 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.951 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.951 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.951 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.951 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.951 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_obj", 2024-05-30T16:06:14.951 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:14.952 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:14.952 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.952 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.952 INFO:tasks.workunit.client.0.smithi006.stdout: "obj3-2": { 2024-05-30T16:06:14.952 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.952 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.952 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.952 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.952 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.952 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.952 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.952 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.952 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.953 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.953 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.953 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.953 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_obj", 2024-05-30T16:06:14.953 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:14.953 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:14.953 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.953 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.953 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:14.953 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: _obj[v1] 2024-05-30T16:06:14.953 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: _obj[v1] 2024-05-30T16:06:14.953 INFO:tasks.workunit.client.0.smithi006.stdout:stripped: _obj ns= i=v1 2024-05-30T16:06:14.953 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: _obj[v1] 2024-05-30T16:06:14.954 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: _obj[v1] 2024-05-30T16:06:14.954 INFO:tasks.workunit.client.0.smithi006.stdout:stripped: _obj ns= i=v1 2024-05-30T16:06:14.954 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: _obj[v1] 2024-05-30T16:06:14.954 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: _obj[v1] 2024-05-30T16:06:14.954 INFO:tasks.workunit.client.0.smithi006.stdout:stripped: _obj ns= i=v1 2024-05-30T16:06:14.954 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:14.954 INFO:tasks.workunit.client.0.smithi006.stdout: "obj1": { 2024-05-30T16:06:14.954 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.954 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.954 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.954 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.954 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.954 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.955 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.955 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.955 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.955 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.955 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.955 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.955 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_obj", 2024-05-30T16:06:14.955 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "v1", 2024-05-30T16:06:14.955 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:14.955 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.955 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.955 INFO:tasks.workunit.client.0.smithi006.stdout: "obj2": { 2024-05-30T16:06:14.955 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.956 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.956 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.956 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.956 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.956 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.956 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.956 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.956 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.956 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.956 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.956 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.956 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_obj", 2024-05-30T16:06:14.956 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "v1", 2024-05-30T16:06:14.956 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:14.957 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.957 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.957 INFO:tasks.workunit.client.0.smithi006.stdout: "obj3": { 2024-05-30T16:06:14.957 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.957 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.957 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.957 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.957 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.957 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.957 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.957 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.957 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.957 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.957 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.958 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.958 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_obj", 2024-05-30T16:06:14.958 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:14.958 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:14.958 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.958 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.958 INFO:tasks.workunit.client.0.smithi006.stdout: "obj3-2": { 2024-05-30T16:06:14.958 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.958 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.958 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.958 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.958 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.958 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.958 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.959 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.959 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.959 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.959 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.959 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.959 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_obj", 2024-05-30T16:06:14.959 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "v1", 2024-05-30T16:06:14.959 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:14.959 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.959 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.959 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:14.959 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: _obj[v1] 2024-05-30T16:06:14.959 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: _obj[v1] 2024-05-30T16:06:14.960 INFO:tasks.workunit.client.0.smithi006.stdout:stripped: _obj ns=ns i=v1 2024-05-30T16:06:14.960 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: _obj[v1] 2024-05-30T16:06:14.960 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: _obj[v1] 2024-05-30T16:06:14.960 INFO:tasks.workunit.client.0.smithi006.stdout:stripped: _obj ns=ns i=v1 2024-05-30T16:06:14.960 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: _obj[v1] 2024-05-30T16:06:14.960 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: _obj[v1] 2024-05-30T16:06:14.960 INFO:tasks.workunit.client.0.smithi006.stdout:stripped: _obj ns=ns i=v1 2024-05-30T16:06:14.960 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:14.960 INFO:tasks.workunit.client.0.smithi006.stdout: "obj1": { 2024-05-30T16:06:14.960 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.960 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.960 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.960 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.961 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.961 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.961 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.961 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.961 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.961 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.961 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.961 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.961 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_obj", 2024-05-30T16:06:14.961 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "v1", 2024-05-30T16:06:14.961 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:14.961 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.961 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.961 INFO:tasks.workunit.client.0.smithi006.stdout: "obj2": { 2024-05-30T16:06:14.962 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.962 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.962 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.962 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.962 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.962 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.962 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.962 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.962 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.962 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.962 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.962 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.962 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_obj", 2024-05-30T16:06:14.962 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "v1", 2024-05-30T16:06:14.963 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:14.963 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.963 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.963 INFO:tasks.workunit.client.0.smithi006.stdout: "obj3": { 2024-05-30T16:06:14.963 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.963 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.963 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.963 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.963 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.963 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.963 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.963 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.963 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.964 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.964 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.964 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.964 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_obj", 2024-05-30T16:06:14.964 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:14.964 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:14.964 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.964 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.964 INFO:tasks.workunit.client.0.smithi006.stdout: "obj3-2": { 2024-05-30T16:06:14.964 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.964 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.964 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.964 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.965 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.965 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.965 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.965 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.965 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.965 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.965 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.965 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.965 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_obj", 2024-05-30T16:06:14.965 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "v1", 2024-05-30T16:06:14.965 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:14.965 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.965 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.965 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:14.966 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] TestRGWObj.underscore (0 ms) 2024-05-30T16:06:14.966 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] TestRGWObj.no_underscore 2024-05-30T16:06:14.966 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: obj 2024-05-30T16:06:14.966 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: obj 2024-05-30T16:06:14.966 INFO:tasks.workunit.client.0.smithi006.stdout:stripped: obj ns= i= 2024-05-30T16:06:14.966 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: obj 2024-05-30T16:06:14.966 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: obj 2024-05-30T16:06:14.966 INFO:tasks.workunit.client.0.smithi006.stdout:stripped: obj ns= i= 2024-05-30T16:06:14.966 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: obj 2024-05-30T16:06:14.966 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: obj 2024-05-30T16:06:14.966 INFO:tasks.workunit.client.0.smithi006.stdout:stripped: obj ns= i= 2024-05-30T16:06:14.966 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:14.966 INFO:tasks.workunit.client.0.smithi006.stdout: "obj1": { 2024-05-30T16:06:14.966 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.967 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.967 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.967 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.967 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.967 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.967 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.967 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.967 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.967 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.967 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.967 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.967 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "obj", 2024-05-30T16:06:14.967 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:14.967 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:14.968 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.968 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.968 INFO:tasks.workunit.client.0.smithi006.stdout: "obj2": { 2024-05-30T16:06:14.968 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.968 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.968 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.968 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.968 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.968 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.968 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.968 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.968 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.968 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.968 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.969 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.969 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "obj", 2024-05-30T16:06:14.969 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:14.969 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:14.969 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.969 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.969 INFO:tasks.workunit.client.0.smithi006.stdout: "obj3": { 2024-05-30T16:06:14.969 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.969 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.969 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.969 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.969 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.969 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.970 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.970 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.970 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.970 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.970 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.970 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.970 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "obj", 2024-05-30T16:06:14.970 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:14.970 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:14.970 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.970 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.970 INFO:tasks.workunit.client.0.smithi006.stdout: "obj3-2": { 2024-05-30T16:06:14.970 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.970 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.971 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.971 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.971 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.971 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.971 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.971 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.971 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.971 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.971 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.971 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.971 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "obj", 2024-05-30T16:06:14.971 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:14.971 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:14.972 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.972 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.972 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:14.972 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: obj 2024-05-30T16:06:14.972 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: obj 2024-05-30T16:06:14.972 INFO:tasks.workunit.client.0.smithi006.stdout:stripped: obj ns=ns i= 2024-05-30T16:06:14.972 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: obj 2024-05-30T16:06:14.972 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: obj 2024-05-30T16:06:14.972 INFO:tasks.workunit.client.0.smithi006.stdout:stripped: obj ns=ns i= 2024-05-30T16:06:14.972 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: obj 2024-05-30T16:06:14.972 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: obj 2024-05-30T16:06:14.972 INFO:tasks.workunit.client.0.smithi006.stdout:stripped: obj ns=ns i= 2024-05-30T16:06:14.972 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:14.973 INFO:tasks.workunit.client.0.smithi006.stdout: "obj1": { 2024-05-30T16:06:14.973 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.973 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.973 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.973 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.973 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.973 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.973 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.973 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.973 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.973 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.973 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.973 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.974 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "obj", 2024-05-30T16:06:14.974 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:14.974 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:14.974 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.974 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.974 INFO:tasks.workunit.client.0.smithi006.stdout: "obj2": { 2024-05-30T16:06:14.974 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.974 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.974 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.974 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.974 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.974 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.974 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.974 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.975 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.975 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.975 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.975 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.975 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "obj", 2024-05-30T16:06:14.975 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:14.975 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:14.975 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.975 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.975 INFO:tasks.workunit.client.0.smithi006.stdout: "obj3": { 2024-05-30T16:06:14.975 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.975 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.975 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.976 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.976 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.976 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.976 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.976 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.976 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.976 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.976 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.976 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.976 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "obj", 2024-05-30T16:06:14.976 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:14.976 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:14.976 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.976 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.977 INFO:tasks.workunit.client.0.smithi006.stdout: "obj3-2": { 2024-05-30T16:06:14.977 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.977 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.977 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.977 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.977 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.977 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.977 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.977 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.977 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.977 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.977 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.977 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.977 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "obj", 2024-05-30T16:06:14.978 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:14.978 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:14.978 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.978 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.978 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:14.978 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: obj[v1] 2024-05-30T16:06:14.978 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: obj[v1] 2024-05-30T16:06:14.978 INFO:tasks.workunit.client.0.smithi006.stdout:stripped: obj ns= i=v1 2024-05-30T16:06:14.978 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: obj[v1] 2024-05-30T16:06:14.978 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: obj[v1] 2024-05-30T16:06:14.978 INFO:tasks.workunit.client.0.smithi006.stdout:stripped: obj ns= i=v1 2024-05-30T16:06:14.978 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: obj[v1] 2024-05-30T16:06:14.978 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: obj[v1] 2024-05-30T16:06:14.979 INFO:tasks.workunit.client.0.smithi006.stdout:stripped: obj ns= i=v1 2024-05-30T16:06:14.979 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:14.979 INFO:tasks.workunit.client.0.smithi006.stdout: "obj1": { 2024-05-30T16:06:14.979 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.979 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.979 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.979 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.979 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.979 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.979 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.979 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.979 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.979 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.979 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.980 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.980 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "obj", 2024-05-30T16:06:14.980 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "v1", 2024-05-30T16:06:14.980 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:14.980 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.980 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.980 INFO:tasks.workunit.client.0.smithi006.stdout: "obj2": { 2024-05-30T16:06:14.980 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.980 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.980 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.980 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.980 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.980 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.981 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.981 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.981 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.981 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.981 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.981 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.981 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "obj", 2024-05-30T16:06:14.981 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "v1", 2024-05-30T16:06:14.981 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:14.981 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.981 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.981 INFO:tasks.workunit.client.0.smithi006.stdout: "obj3": { 2024-05-30T16:06:14.981 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.982 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.982 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.982 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.982 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.982 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.982 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.982 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.982 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.982 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.982 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.982 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.982 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "obj", 2024-05-30T16:06:14.982 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:14.983 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:14.983 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.983 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.983 INFO:tasks.workunit.client.0.smithi006.stdout: "obj3-2": { 2024-05-30T16:06:14.983 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.983 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.983 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.983 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.983 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.983 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.983 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.983 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.983 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.984 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.984 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.984 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.984 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "obj", 2024-05-30T16:06:14.984 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "v1", 2024-05-30T16:06:14.984 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:14.984 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.984 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.984 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:14.984 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: obj[v1] 2024-05-30T16:06:14.984 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: obj[v1] 2024-05-30T16:06:14.984 INFO:tasks.workunit.client.0.smithi006.stdout:stripped: obj ns=ns i=v1 2024-05-30T16:06:14.984 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: obj[v1] 2024-05-30T16:06:14.985 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: obj[v1] 2024-05-30T16:06:14.985 INFO:tasks.workunit.client.0.smithi006.stdout:stripped: obj ns=ns i=v1 2024-05-30T16:06:14.985 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: obj[v1] 2024-05-30T16:06:14.985 INFO:tasks.workunit.client.0.smithi006.stdout:parsed: obj[v1] 2024-05-30T16:06:14.985 INFO:tasks.workunit.client.0.smithi006.stdout:stripped: obj ns=ns i=v1 2024-05-30T16:06:14.985 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:14.985 INFO:tasks.workunit.client.0.smithi006.stdout: "obj1": { 2024-05-30T16:06:14.985 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.985 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.985 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.985 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.985 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.985 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.985 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.986 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.986 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.986 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.986 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.986 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.986 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "obj", 2024-05-30T16:06:14.986 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "v1", 2024-05-30T16:06:14.986 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:14.986 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.986 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.986 INFO:tasks.workunit.client.0.smithi006.stdout: "obj2": { 2024-05-30T16:06:14.986 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.986 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.987 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.987 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.987 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.987 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.987 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.987 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.987 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.987 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.987 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.987 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.987 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "obj", 2024-05-30T16:06:14.987 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "v1", 2024-05-30T16:06:14.987 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:14.988 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.988 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.988 INFO:tasks.workunit.client.0.smithi006.stdout: "obj3": { 2024-05-30T16:06:14.988 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.988 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.988 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.988 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.988 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.988 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.988 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.988 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.988 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.988 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.988 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.989 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.989 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "obj", 2024-05-30T16:06:14.989 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:14.989 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:14.989 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.989 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.989 INFO:tasks.workunit.client.0.smithi006.stdout: "obj3-2": { 2024-05-30T16:06:14.989 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.989 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.989 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.989 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.989 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.989 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.989 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.990 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.990 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.990 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.990 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.990 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.990 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "obj", 2024-05-30T16:06:14.990 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "v1", 2024-05-30T16:06:14.990 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:14.990 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.990 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.990 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:14.990 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] TestRGWObj.no_underscore (0 ms) 2024-05-30T16:06:14.990 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] TestRGWObj.obj_to_raw 2024-05-30T16:06:14.990 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:14.991 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.991 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.991 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.991 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.991 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.991 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.991 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.991 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.991 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.991 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.991 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.991 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:14.991 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:14.991 INFO:tasks.workunit.client.0.smithi006.stdout: "obj": { 2024-05-30T16:06:14.992 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.992 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.992 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.992 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.992 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.992 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.992 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.992 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.992 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.992 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.992 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.992 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.992 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "myobj", 2024-05-30T16:06:14.993 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:14.993 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:14.993 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.993 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.993 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:14.993 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:14.993 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:14.993 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": "default-placement.data", 2024-05-30T16:06:14.993 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker_myobj", 2024-05-30T16:06:14.993 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:14.993 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.993 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:14.993 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:14.994 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:14.994 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.994 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:14.994 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.994 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.994 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.994 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.994 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:14.994 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.994 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:14.994 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.994 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.994 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.995 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "myobj", 2024-05-30T16:06:14.995 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:14.995 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:14.995 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.995 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.995 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:14.995 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:14.995 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.995 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:14.995 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.995 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.995 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.995 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.996 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:14.996 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.996 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:14.996 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.996 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.996 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:14.996 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:14.996 INFO:tasks.workunit.client.0.smithi006.stdout: "obj": { 2024-05-30T16:06:14.996 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.996 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:14.996 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.996 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.996 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.996 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.997 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:14.997 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.997 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:14.997 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.997 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.997 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.997 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "myobj", 2024-05-30T16:06:14.997 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:14.997 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:14.997 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.997 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.997 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:14.997 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:14.998 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:14.998 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:14.998 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker_myobj", 2024-05-30T16:06:14.998 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:14.998 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.998 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:14.998 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:14.998 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:14.998 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:14.998 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:14.998 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:14.998 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:14.998 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:14.998 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:14.999 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:14.999 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:14.999 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:14.999 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.999 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:14.999 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:14.999 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "myobj", 2024-05-30T16:06:14.999 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:14.999 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:14.999 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.999 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:14.999 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:14.999 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:14.999 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.000 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.000 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.000 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.000 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.000 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.000 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.000 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.000 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.000 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.000 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.000 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.000 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.000 INFO:tasks.workunit.client.0.smithi006.stdout: "obj": { 2024-05-30T16:06:15.000 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.001 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.001 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.001 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.001 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.001 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.001 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.001 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.001 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.001 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.001 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.001 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.001 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "myobj", 2024-05-30T16:06:15.001 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.002 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.002 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.002 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.002 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.002 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.002 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.002 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": "default-placement.data", 2024-05-30T16:06:15.002 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__ns_myobj", 2024-05-30T16:06:15.002 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:15.002 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.002 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.002 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.002 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.003 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.003 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.003 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.003 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.003 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.003 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.003 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.003 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.003 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.003 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.003 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.003 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.003 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "myobj", 2024-05-30T16:06:15.003 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.004 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.004 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.004 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.004 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.004 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.004 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.004 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.004 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.004 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.004 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.004 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.004 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.004 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.004 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.005 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.005 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.005 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.005 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.005 INFO:tasks.workunit.client.0.smithi006.stdout: "obj": { 2024-05-30T16:06:15.005 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.005 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.005 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.005 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.005 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.005 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.005 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.005 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.005 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.006 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.006 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.006 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.006 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "myobj", 2024-05-30T16:06:15.006 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.006 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.006 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.006 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.006 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.006 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.006 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.006 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.006 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__ns_myobj", 2024-05-30T16:06:15.006 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:15.007 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.007 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.007 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.007 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.007 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.007 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.007 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.007 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.007 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.007 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.007 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.007 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.007 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.007 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.008 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.008 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.008 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "myobj", 2024-05-30T16:06:15.008 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.008 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.008 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.008 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.008 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.008 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.008 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.008 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.008 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.008 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.009 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.009 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.009 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.009 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.009 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.009 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.009 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.009 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.009 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.009 INFO:tasks.workunit.client.0.smithi006.stdout: "obj": { 2024-05-30T16:06:15.009 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.009 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.009 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.009 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.010 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.010 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.010 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.010 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.010 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.010 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.010 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.010 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.010 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "myobj", 2024-05-30T16:06:15.010 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.010 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.010 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.010 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.010 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.011 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.011 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.011 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": "default-placement.data", 2024-05-30T16:06:15.011 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__:inst_myobj", 2024-05-30T16:06:15.011 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:15.011 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.011 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.011 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.011 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.011 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.011 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.011 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.011 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.011 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.012 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.012 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.012 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.012 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.012 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.012 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.012 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.012 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "myobj", 2024-05-30T16:06:15.012 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.012 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.012 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.012 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.012 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.012 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.013 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.013 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.013 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.013 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.013 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.013 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.013 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.013 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.013 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.013 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.013 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.013 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.013 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.013 INFO:tasks.workunit.client.0.smithi006.stdout: "obj": { 2024-05-30T16:06:15.014 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.014 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.014 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.014 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.014 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.014 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.014 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.014 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.014 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.014 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.014 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.014 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.014 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "myobj", 2024-05-30T16:06:15.014 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.015 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.015 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.015 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.015 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.015 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.015 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.015 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.015 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__:inst_myobj", 2024-05-30T16:06:15.015 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:15.015 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.015 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.015 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.015 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.016 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.016 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.016 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.016 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.016 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.016 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.016 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.016 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.016 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.016 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.016 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.016 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.016 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "myobj", 2024-05-30T16:06:15.016 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.017 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.017 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.017 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.017 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.017 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.017 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.017 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.017 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.017 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.017 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.017 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.017 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.017 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.018 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.018 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.018 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.018 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.018 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.018 INFO:tasks.workunit.client.0.smithi006.stdout: "obj": { 2024-05-30T16:06:15.018 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.018 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.018 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.018 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.018 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.018 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.018 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.018 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.019 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.019 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.019 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.019 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.019 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "myobj", 2024-05-30T16:06:15.019 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.019 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.019 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.019 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.019 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.019 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.019 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.019 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": "default-placement.data", 2024-05-30T16:06:15.019 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__ns:inst_myobj", 2024-05-30T16:06:15.020 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:15.020 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.020 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.020 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.020 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.020 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.020 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.020 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.020 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.020 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.020 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.020 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.020 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.021 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.021 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.021 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.021 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.021 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "myobj", 2024-05-30T16:06:15.021 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.021 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.021 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.021 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.021 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.021 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.021 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.021 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.022 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.022 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.022 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.022 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.022 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.022 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.022 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.022 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.022 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.022 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.022 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.022 INFO:tasks.workunit.client.0.smithi006.stdout: "obj": { 2024-05-30T16:06:15.023 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.023 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.023 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.023 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.023 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.023 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.023 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.023 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.023 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.023 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.023 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.023 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.023 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "myobj", 2024-05-30T16:06:15.024 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.024 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.024 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.024 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.024 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.024 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.024 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.024 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.024 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__ns:inst_myobj", 2024-05-30T16:06:15.024 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:15.024 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.024 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.024 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.024 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.025 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.025 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.025 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.025 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.025 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.025 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.025 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.025 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.025 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.025 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.025 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.025 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.025 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "myobj", 2024-05-30T16:06:15.025 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.026 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.026 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.026 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.026 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.026 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.026 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.026 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.026 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.026 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.026 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.026 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.026 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.026 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.027 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.027 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.027 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.027 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.027 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.027 INFO:tasks.workunit.client.0.smithi006.stdout: "obj": { 2024-05-30T16:06:15.027 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.027 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.027 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.027 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.027 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.027 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.027 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.028 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.028 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.028 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.028 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.028 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.028 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj", 2024-05-30T16:06:15.028 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.028 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.028 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.028 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.028 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.028 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.028 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.029 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": "default-placement.data", 2024-05-30T16:06:15.029 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker___myobj", 2024-05-30T16:06:15.029 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "marker__myobj" 2024-05-30T16:06:15.029 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.029 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.029 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.029 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.029 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.029 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.029 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.029 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.029 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.029 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.030 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.030 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.030 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.030 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.030 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.030 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.030 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj", 2024-05-30T16:06:15.030 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.030 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.030 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.030 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.030 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.030 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.031 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.031 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.031 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.031 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.031 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.031 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.031 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.031 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.031 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.031 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.031 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.031 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.031 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.032 INFO:tasks.workunit.client.0.smithi006.stdout: "obj": { 2024-05-30T16:06:15.032 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.032 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.032 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.032 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.032 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.032 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.032 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.032 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.032 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.032 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.032 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.032 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.032 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj", 2024-05-30T16:06:15.033 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.033 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.033 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.033 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.033 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.033 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.033 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.033 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.033 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker___myobj", 2024-05-30T16:06:15.033 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "marker__myobj" 2024-05-30T16:06:15.033 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.033 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.033 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.033 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.034 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.034 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.034 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.034 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.034 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.034 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.034 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.034 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.034 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.034 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.034 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.034 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.034 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj", 2024-05-30T16:06:15.034 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.035 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.035 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.035 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.035 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.035 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.035 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.035 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.035 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.035 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.035 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.035 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.035 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.035 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.035 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.036 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.036 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.036 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.036 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.036 INFO:tasks.workunit.client.0.smithi006.stdout: "obj": { 2024-05-30T16:06:15.036 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.036 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.036 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.036 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.036 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.036 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.036 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.036 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.036 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.037 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.037 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.037 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.037 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj", 2024-05-30T16:06:15.037 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.037 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.037 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.037 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.037 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.037 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.037 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.037 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": "default-placement.data", 2024-05-30T16:06:15.037 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__ns__myobj", 2024-05-30T16:06:15.037 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:15.038 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.038 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.038 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.038 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.038 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.038 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.038 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.038 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.038 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.038 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.038 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.038 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.038 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.038 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.039 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.039 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.039 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj", 2024-05-30T16:06:15.039 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.039 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.039 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.039 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.039 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.039 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.039 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.039 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.039 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.039 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.039 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.040 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.040 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.040 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.040 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.040 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.040 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.040 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.040 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.040 INFO:tasks.workunit.client.0.smithi006.stdout: "obj": { 2024-05-30T16:06:15.040 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.040 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.040 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.040 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.040 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.041 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.041 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.041 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.041 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.041 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.041 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.041 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.041 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj", 2024-05-30T16:06:15.041 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.041 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.041 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.041 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.041 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.041 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.041 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.042 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.042 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__ns__myobj", 2024-05-30T16:06:15.042 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:15.042 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.042 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.042 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.042 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.042 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.042 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.042 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.042 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.042 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.042 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.043 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.043 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.043 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.043 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.043 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.043 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.043 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj", 2024-05-30T16:06:15.043 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.043 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.043 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.043 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.043 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.043 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.043 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.043 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.044 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.044 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.044 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.044 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.044 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.044 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.044 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.044 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.044 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.044 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.044 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.044 INFO:tasks.workunit.client.0.smithi006.stdout: "obj": { 2024-05-30T16:06:15.044 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.045 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.045 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.045 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.045 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.045 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.045 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.045 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.045 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.045 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.045 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.045 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.045 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj", 2024-05-30T16:06:15.045 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.045 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.046 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.046 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.046 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.046 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.046 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.046 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": "default-placement.data", 2024-05-30T16:06:15.046 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__:inst__myobj", 2024-05-30T16:06:15.046 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "marker__myobj" 2024-05-30T16:06:15.046 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.046 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.046 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.046 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.046 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.046 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.047 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.047 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.047 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.047 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.047 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.047 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.047 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.047 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.047 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.047 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.047 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj", 2024-05-30T16:06:15.047 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.047 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.047 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.048 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.048 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.048 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.048 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.048 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.048 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.048 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.048 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.048 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.048 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.048 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.048 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.049 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.049 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.049 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.049 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.049 INFO:tasks.workunit.client.0.smithi006.stdout: "obj": { 2024-05-30T16:06:15.049 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.049 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.049 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.049 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.049 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.049 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.049 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.049 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.049 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.050 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.050 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.050 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.050 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj", 2024-05-30T16:06:15.050 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.050 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.050 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.050 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.050 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.050 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.050 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.050 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.050 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__:inst__myobj", 2024-05-30T16:06:15.050 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "marker__myobj" 2024-05-30T16:06:15.051 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.051 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.051 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.051 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.051 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.051 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.051 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.051 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.051 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.051 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.051 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.051 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.051 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.052 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.052 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.052 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.052 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj", 2024-05-30T16:06:15.052 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.052 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.052 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.052 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.052 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.052 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.052 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.052 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.052 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.052 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.053 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.053 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.053 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.053 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.053 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.053 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.053 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.053 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.053 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.053 INFO:tasks.workunit.client.0.smithi006.stdout: "obj": { 2024-05-30T16:06:15.053 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.053 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.053 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.053 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.054 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.054 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.054 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.054 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.054 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.054 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.054 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.054 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.054 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj", 2024-05-30T16:06:15.054 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.054 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.054 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.054 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.055 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.055 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.055 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.055 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": "default-placement.data", 2024-05-30T16:06:15.055 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__ns:inst__myobj", 2024-05-30T16:06:15.055 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:15.055 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.055 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.055 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.055 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.055 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.055 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.055 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.055 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.056 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.056 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.056 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.056 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.056 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.056 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.056 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.056 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.056 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj", 2024-05-30T16:06:15.056 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.056 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.056 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.056 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.056 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.056 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.057 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.057 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.057 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.057 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.057 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.057 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.057 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.057 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.057 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.057 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.057 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.057 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.057 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.057 INFO:tasks.workunit.client.0.smithi006.stdout: "obj": { 2024-05-30T16:06:15.058 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.058 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.058 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.058 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.058 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.058 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.058 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.058 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.058 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.058 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.058 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.058 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.058 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj", 2024-05-30T16:06:15.059 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.059 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.059 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.059 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.059 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.059 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.059 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.059 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.059 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__ns:inst__myobj", 2024-05-30T16:06:15.059 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:15.059 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.059 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.059 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.059 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.060 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.060 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.060 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.060 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.060 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.060 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.060 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.060 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.060 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.060 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.060 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.060 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.060 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj", 2024-05-30T16:06:15.061 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.061 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.061 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.061 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.061 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.061 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.061 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.061 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.061 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.061 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.061 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.061 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.061 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.061 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.062 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.062 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.062 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.062 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.062 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.062 INFO:tasks.workunit.client.0.smithi006.stdout: "obj": { 2024-05-30T16:06:15.062 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.062 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.062 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.062 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.062 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.062 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.062 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.062 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.063 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.063 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.063 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.063 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.063 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj_", 2024-05-30T16:06:15.063 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.063 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.063 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.063 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.063 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.063 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.063 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.063 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": "default-placement.data", 2024-05-30T16:06:15.063 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker___myobj_", 2024-05-30T16:06:15.064 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "marker__myobj_" 2024-05-30T16:06:15.064 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.064 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.064 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.064 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.064 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.064 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.064 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.064 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.064 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.064 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.064 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.064 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.064 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.065 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.065 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.065 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.065 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj_", 2024-05-30T16:06:15.065 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.065 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.065 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.065 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.065 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.065 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.065 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.065 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.065 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.065 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.066 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.066 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.066 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.066 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.066 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.066 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.066 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.066 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.066 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.066 INFO:tasks.workunit.client.0.smithi006.stdout: "obj": { 2024-05-30T16:06:15.066 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.066 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.066 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.067 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.067 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.067 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.067 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.067 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.067 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.067 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.067 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.067 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.067 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj_", 2024-05-30T16:06:15.067 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.067 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.067 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.067 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.068 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.068 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.068 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.068 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.068 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker___myobj_", 2024-05-30T16:06:15.068 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "marker__myobj_" 2024-05-30T16:06:15.068 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.068 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.068 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.068 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.068 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.068 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.068 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.068 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.069 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.069 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.069 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.069 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.069 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.069 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.069 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.069 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.069 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj_", 2024-05-30T16:06:15.069 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.069 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.069 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.069 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.069 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.070 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.070 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.070 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.070 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.070 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.070 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.070 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.070 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.070 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.070 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.070 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.070 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.070 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.071 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.071 INFO:tasks.workunit.client.0.smithi006.stdout: "obj": { 2024-05-30T16:06:15.071 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.071 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.071 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.071 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.071 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.071 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.071 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.071 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.071 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.071 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.071 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.071 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.072 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj_", 2024-05-30T16:06:15.072 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.072 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.072 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.072 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.072 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.072 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.072 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.072 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": "default-placement.data", 2024-05-30T16:06:15.072 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__ns__myobj_", 2024-05-30T16:06:15.072 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:15.072 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.072 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.072 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.072 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.073 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.073 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.073 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.073 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.073 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.073 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.073 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.073 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.073 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.073 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.073 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.073 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.073 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj_", 2024-05-30T16:06:15.073 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.074 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.074 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.074 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.074 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.074 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.074 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.074 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.074 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.074 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.074 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.074 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.074 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.074 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.074 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.075 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.075 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.075 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.075 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.075 INFO:tasks.workunit.client.0.smithi006.stdout: "obj": { 2024-05-30T16:06:15.075 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.075 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.075 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.075 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.075 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.075 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.075 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.075 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.076 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.076 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.076 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.076 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.076 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj_", 2024-05-30T16:06:15.076 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.076 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.076 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.076 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.076 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.076 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.076 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.076 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.076 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__ns__myobj_", 2024-05-30T16:06:15.077 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:15.077 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.077 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.077 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.077 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.077 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.077 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.077 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.077 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.077 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.077 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.077 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.077 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.077 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.078 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.078 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.078 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.078 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj_", 2024-05-30T16:06:15.078 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.078 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.078 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.078 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.078 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.078 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.078 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.078 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.078 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.079 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.079 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.079 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.079 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.079 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.079 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.079 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.079 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.079 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.079 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.079 INFO:tasks.workunit.client.0.smithi006.stdout: "obj": { 2024-05-30T16:06:15.079 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.079 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.079 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.080 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.080 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.080 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.080 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.080 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.080 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.080 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.080 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.080 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.080 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj_", 2024-05-30T16:06:15.080 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.080 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.080 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.080 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.081 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.081 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.081 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.081 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": "default-placement.data", 2024-05-30T16:06:15.081 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__:inst__myobj_", 2024-05-30T16:06:15.081 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "marker__myobj_" 2024-05-30T16:06:15.081 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.081 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.081 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.081 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.081 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.081 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.081 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.081 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.082 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.082 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.082 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.082 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.082 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.082 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.082 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.082 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.082 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj_", 2024-05-30T16:06:15.082 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.082 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.082 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.082 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.083 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.083 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.083 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.083 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.083 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.083 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.083 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.083 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.083 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.083 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.083 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.083 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.083 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.083 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.084 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.084 INFO:tasks.workunit.client.0.smithi006.stdout: "obj": { 2024-05-30T16:06:15.084 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.084 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.084 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.084 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.084 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.084 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.084 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.084 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.084 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.084 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.084 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.084 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.085 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj_", 2024-05-30T16:06:15.085 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.085 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.085 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.085 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.085 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.085 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.085 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.085 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.085 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__:inst__myobj_", 2024-05-30T16:06:15.085 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "marker__myobj_" 2024-05-30T16:06:15.085 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.085 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.085 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.086 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.086 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.086 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.086 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.086 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.086 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.086 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.086 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.086 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.086 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.086 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.086 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.086 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.086 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj_", 2024-05-30T16:06:15.087 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.087 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.087 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.087 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.087 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.087 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.087 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.087 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.087 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.087 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.087 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.087 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.087 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.087 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.088 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.088 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.088 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.088 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.088 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.088 INFO:tasks.workunit.client.0.smithi006.stdout: "obj": { 2024-05-30T16:06:15.088 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.088 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.088 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.088 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.088 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.088 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.088 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.088 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.089 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.089 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.089 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.089 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.089 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj_", 2024-05-30T16:06:15.089 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.089 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.089 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.089 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.089 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.089 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.089 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.089 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": "default-placement.data", 2024-05-30T16:06:15.090 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__ns:inst__myobj_", 2024-05-30T16:06:15.090 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:15.090 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.090 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.090 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.090 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.090 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.090 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "test", 2024-05-30T16:06:15.090 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.090 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.090 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.090 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.090 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": "", 2024-05-30T16:06:15.091 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.091 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": "" 2024-05-30T16:06:15.091 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.091 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.091 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.091 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj_", 2024-05-30T16:06:15.091 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.091 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.091 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.091 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.091 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.091 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.091 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.091 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.092 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.092 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.092 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.092 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.092 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.092 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.092 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.092 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.092 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.092 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.092 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.092 INFO:tasks.workunit.client.0.smithi006.stdout: "obj": { 2024-05-30T16:06:15.092 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.093 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.093 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.093 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.093 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.093 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.093 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.093 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.093 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.093 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.093 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.093 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.093 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj_", 2024-05-30T16:06:15.093 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.093 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.094 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.094 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.094 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.094 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.094 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.094 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.094 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__ns:inst__myobj_", 2024-05-30T16:06:15.094 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:15.094 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.094 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.094 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.094 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.094 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.095 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.095 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.095 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.095 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.095 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.095 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.095 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.095 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.095 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.095 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.095 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.095 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj_", 2024-05-30T16:06:15.095 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.095 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.096 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.096 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.096 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.096 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] TestRGWObj.obj_to_raw (1 ms) 2024-05-30T16:06:15.096 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] TestRGWObj.old_to_raw 2024-05-30T16:06:15.096 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.096 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.096 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.096 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker_myobj", 2024-05-30T16:06:15.096 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:15.096 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.096 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.096 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.096 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.097 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.097 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.097 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.097 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.097 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.097 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.097 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.097 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.097 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.097 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.097 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.097 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.097 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "myobj", 2024-05-30T16:06:15.097 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.098 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.098 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.098 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.098 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.098 INFO:tasks.workunit.client.0.smithi006.stdout:raw=.data-pool:marker_myobj 2024-05-30T16:06:15.098 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.098 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj2": { 2024-05-30T16:06:15.098 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.098 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker_myobj", 2024-05-30T16:06:15.098 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:15.098 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.098 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.098 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.099 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj2": { 2024-05-30T16:06:15.099 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.099 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.099 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.099 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.099 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.099 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.099 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.099 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.099 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.099 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.099 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.099 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.099 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "myobj", 2024-05-30T16:06:15.100 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.100 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.100 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.100 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.100 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.100 INFO:tasks.workunit.client.0.smithi006.stdout:raw2=.data-pool:marker_myobj 2024-05-30T16:06:15.100 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.100 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.100 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.100 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__ns_myobj", 2024-05-30T16:06:15.100 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:15.100 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.100 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.101 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.101 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.101 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.101 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.101 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.101 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.101 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.101 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.101 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.101 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.101 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.101 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.101 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.102 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.102 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "myobj", 2024-05-30T16:06:15.102 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.102 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.102 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.102 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.102 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.102 INFO:tasks.workunit.client.0.smithi006.stdout:raw=.data-pool:marker__ns_myobj 2024-05-30T16:06:15.102 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.102 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj2": { 2024-05-30T16:06:15.102 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.102 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__ns_myobj", 2024-05-30T16:06:15.102 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:15.103 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.103 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.103 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.103 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj2": { 2024-05-30T16:06:15.103 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.103 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.103 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.103 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.103 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.103 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.103 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.103 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.103 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.104 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.104 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.104 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.104 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "myobj", 2024-05-30T16:06:15.104 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.104 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.104 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.104 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.104 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.104 INFO:tasks.workunit.client.0.smithi006.stdout:raw2=.data-pool:marker__ns_myobj 2024-05-30T16:06:15.104 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.104 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.104 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.104 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__:inst_myobj", 2024-05-30T16:06:15.105 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:15.105 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.105 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.105 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.105 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.105 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.105 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.105 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.105 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.105 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.105 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.105 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.105 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.106 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.106 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.106 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.106 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.106 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "myobj", 2024-05-30T16:06:15.106 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.106 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.106 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.106 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.106 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.106 INFO:tasks.workunit.client.0.smithi006.stdout:raw=.data-pool:marker__:inst_myobj 2024-05-30T16:06:15.106 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.106 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj2": { 2024-05-30T16:06:15.107 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.107 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__:inst_myobj", 2024-05-30T16:06:15.107 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:15.107 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.107 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.107 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.107 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj2": { 2024-05-30T16:06:15.107 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.107 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.107 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.107 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.107 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.107 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.107 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.108 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.108 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.108 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.108 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.108 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.108 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "myobj", 2024-05-30T16:06:15.108 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.108 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.108 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.108 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.108 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.108 INFO:tasks.workunit.client.0.smithi006.stdout:raw2=.data-pool:marker__:inst_myobj 2024-05-30T16:06:15.108 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.108 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.108 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.109 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__ns:inst_myobj", 2024-05-30T16:06:15.109 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:15.109 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.109 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.109 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.109 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.109 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.109 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.109 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.109 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.109 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.109 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.109 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.109 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.110 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.110 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.110 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.110 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.110 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "myobj", 2024-05-30T16:06:15.110 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.110 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.110 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.110 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.110 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.110 INFO:tasks.workunit.client.0.smithi006.stdout:raw=.data-pool:marker__ns:inst_myobj 2024-05-30T16:06:15.110 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.110 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj2": { 2024-05-30T16:06:15.111 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.111 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__ns:inst_myobj", 2024-05-30T16:06:15.111 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:15.111 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.111 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.111 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.111 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj2": { 2024-05-30T16:06:15.111 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.111 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.111 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.111 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.111 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.111 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.111 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.112 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.112 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.112 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.112 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.112 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.112 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "myobj", 2024-05-30T16:06:15.112 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.112 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.112 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.112 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.112 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.112 INFO:tasks.workunit.client.0.smithi006.stdout:raw2=.data-pool:marker__ns:inst_myobj 2024-05-30T16:06:15.112 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.112 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.113 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.113 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker___myobj", 2024-05-30T16:06:15.113 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "marker__myobj" 2024-05-30T16:06:15.113 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.113 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.113 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.113 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.113 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.113 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.113 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.113 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.113 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.113 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.113 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.113 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.114 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.114 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.114 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.114 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.114 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj", 2024-05-30T16:06:15.114 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.114 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.114 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.114 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.114 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.114 INFO:tasks.workunit.client.0.smithi006.stdout:raw=.data-pool:marker___myobj 2024-05-30T16:06:15.114 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.114 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj2": { 2024-05-30T16:06:15.115 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.115 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker___myobj", 2024-05-30T16:06:15.115 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "marker__myobj" 2024-05-30T16:06:15.115 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.115 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.115 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.115 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj2": { 2024-05-30T16:06:15.115 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.115 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.115 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.115 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.115 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.115 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.115 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.116 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.116 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.116 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.116 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.116 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.116 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj", 2024-05-30T16:06:15.116 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.116 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.116 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.116 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.116 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.116 INFO:tasks.workunit.client.0.smithi006.stdout:raw2=.data-pool:marker___myobj 2024-05-30T16:06:15.116 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.116 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.117 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.117 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__ns__myobj", 2024-05-30T16:06:15.117 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:15.117 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.117 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.117 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.117 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.117 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.117 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.117 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.117 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.117 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.117 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.117 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.118 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.118 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.118 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.118 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.118 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.118 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj", 2024-05-30T16:06:15.118 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.118 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.118 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.118 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.118 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.118 INFO:tasks.workunit.client.0.smithi006.stdout:raw=.data-pool:marker__ns__myobj 2024-05-30T16:06:15.118 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.118 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj2": { 2024-05-30T16:06:15.119 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.119 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__ns__myobj", 2024-05-30T16:06:15.119 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:15.119 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.119 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.119 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.119 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj2": { 2024-05-30T16:06:15.119 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.119 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.119 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.119 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.119 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.120 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.120 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.120 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.120 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.120 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.120 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.120 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.120 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj", 2024-05-30T16:06:15.120 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.120 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.120 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.120 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.120 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.120 INFO:tasks.workunit.client.0.smithi006.stdout:raw2=.data-pool:marker__ns__myobj 2024-05-30T16:06:15.121 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.121 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.121 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.121 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__:inst__myobj", 2024-05-30T16:06:15.121 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "marker__myobj" 2024-05-30T16:06:15.121 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.121 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.121 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.121 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.121 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.121 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.121 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.121 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.121 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.122 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.122 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.122 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.122 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.122 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.122 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.122 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.122 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj", 2024-05-30T16:06:15.122 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.122 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.122 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.122 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.122 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.123 INFO:tasks.workunit.client.0.smithi006.stdout:raw=.data-pool:marker__:inst__myobj 2024-05-30T16:06:15.123 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.123 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj2": { 2024-05-30T16:06:15.123 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.123 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__:inst__myobj", 2024-05-30T16:06:15.123 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "marker__myobj" 2024-05-30T16:06:15.123 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.123 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.123 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.123 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj2": { 2024-05-30T16:06:15.123 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.123 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.123 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.123 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.124 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.124 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.124 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.124 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.124 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.124 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.124 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.124 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.124 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj", 2024-05-30T16:06:15.124 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.124 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.124 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.124 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.124 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.125 INFO:tasks.workunit.client.0.smithi006.stdout:raw2=.data-pool:marker__:inst__myobj 2024-05-30T16:06:15.125 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.125 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.125 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.125 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__ns:inst__myobj", 2024-05-30T16:06:15.125 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:15.125 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.125 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.125 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.125 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.125 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.125 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.125 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.126 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.126 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.126 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.126 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.126 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.126 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.126 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.126 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.126 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.126 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj", 2024-05-30T16:06:15.126 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.126 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.126 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.126 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.127 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.127 INFO:tasks.workunit.client.0.smithi006.stdout:raw=.data-pool:marker__ns:inst__myobj 2024-05-30T16:06:15.127 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.127 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj2": { 2024-05-30T16:06:15.127 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.127 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__ns:inst__myobj", 2024-05-30T16:06:15.127 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:15.127 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.127 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.127 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.127 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj2": { 2024-05-30T16:06:15.127 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.127 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.127 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.128 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.128 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.128 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.128 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.128 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.128 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.128 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.128 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.128 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.128 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj", 2024-05-30T16:06:15.128 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.128 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.128 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.128 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.128 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.129 INFO:tasks.workunit.client.0.smithi006.stdout:raw2=.data-pool:marker__ns:inst__myobj 2024-05-30T16:06:15.129 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.129 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.129 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.129 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker___myobj_", 2024-05-30T16:06:15.129 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "marker__myobj_" 2024-05-30T16:06:15.129 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.129 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.129 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.129 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.129 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.129 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.129 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.129 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.130 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.130 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.130 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.130 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.130 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.130 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.130 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.130 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.130 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj_", 2024-05-30T16:06:15.130 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.130 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.130 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.130 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.131 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.131 INFO:tasks.workunit.client.0.smithi006.stdout:raw=.data-pool:marker___myobj_ 2024-05-30T16:06:15.131 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.131 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj2": { 2024-05-30T16:06:15.131 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.131 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker___myobj_", 2024-05-30T16:06:15.131 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "marker__myobj_" 2024-05-30T16:06:15.131 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.131 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.131 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.131 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj2": { 2024-05-30T16:06:15.131 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.131 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.131 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.132 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.132 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.132 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.132 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.132 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.132 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.132 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.132 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.132 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.132 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj_", 2024-05-30T16:06:15.132 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.132 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.132 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.133 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.133 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.133 INFO:tasks.workunit.client.0.smithi006.stdout:raw2=.data-pool:marker___myobj_ 2024-05-30T16:06:15.133 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.133 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.133 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.133 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__ns__myobj_", 2024-05-30T16:06:15.133 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:15.133 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.133 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.133 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.133 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.133 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.133 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.133 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.134 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.134 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.134 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.134 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.134 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.134 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.134 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.134 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.134 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.134 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj_", 2024-05-30T16:06:15.134 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.134 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.134 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.135 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.135 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.135 INFO:tasks.workunit.client.0.smithi006.stdout:raw=.data-pool:marker__ns__myobj_ 2024-05-30T16:06:15.135 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.135 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj2": { 2024-05-30T16:06:15.135 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.135 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__ns__myobj_", 2024-05-30T16:06:15.135 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:15.135 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.135 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.135 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.135 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj2": { 2024-05-30T16:06:15.135 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.136 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.136 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.136 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.136 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.136 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.136 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.136 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.136 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.136 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.136 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.136 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.136 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj_", 2024-05-30T16:06:15.136 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "", 2024-05-30T16:06:15.136 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.137 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.137 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.137 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.137 INFO:tasks.workunit.client.0.smithi006.stdout:raw2=.data-pool:marker__ns__myobj_ 2024-05-30T16:06:15.137 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.137 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.137 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.137 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__:inst__myobj_", 2024-05-30T16:06:15.137 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "marker__myobj_" 2024-05-30T16:06:15.137 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.137 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.137 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.137 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.137 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.138 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.138 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.138 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.138 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.138 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.138 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.138 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.138 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.138 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.138 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.138 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.138 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj_", 2024-05-30T16:06:15.138 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.139 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.139 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.139 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.139 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.139 INFO:tasks.workunit.client.0.smithi006.stdout:raw=.data-pool:marker__:inst__myobj_ 2024-05-30T16:06:15.139 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.139 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj2": { 2024-05-30T16:06:15.139 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.139 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__:inst__myobj_", 2024-05-30T16:06:15.139 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "marker__myobj_" 2024-05-30T16:06:15.139 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.139 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.139 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.139 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj2": { 2024-05-30T16:06:15.140 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.140 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.140 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.140 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.140 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.140 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.140 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.140 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.140 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.140 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.140 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.140 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.140 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj_", 2024-05-30T16:06:15.141 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.141 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "" 2024-05-30T16:06:15.141 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.141 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.141 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.141 INFO:tasks.workunit.client.0.smithi006.stdout:raw2=.data-pool:marker__:inst__myobj_ 2024-05-30T16:06:15.141 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.141 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj": { 2024-05-30T16:06:15.141 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.141 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__ns:inst__myobj_", 2024-05-30T16:06:15.141 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:15.141 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.141 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.141 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.142 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj": { 2024-05-30T16:06:15.142 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.142 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.142 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.142 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.142 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.142 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.142 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.142 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.142 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.142 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.142 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.142 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.142 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj_", 2024-05-30T16:06:15.143 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.143 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.143 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.143 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.143 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.143 INFO:tasks.workunit.client.0.smithi006.stdout:raw=.data-pool:marker__ns:inst__myobj_ 2024-05-30T16:06:15.143 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.143 INFO:tasks.workunit.client.0.smithi006.stdout: "raw_obj2": { 2024-05-30T16:06:15.143 INFO:tasks.workunit.client.0.smithi006.stdout: "pool": ".data-pool", 2024-05-30T16:06:15.143 INFO:tasks.workunit.client.0.smithi006.stdout: "oid": "marker__ns:inst__myobj_", 2024-05-30T16:06:15.143 INFO:tasks.workunit.client.0.smithi006.stdout: "loc": "" 2024-05-30T16:06:15.143 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.143 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.143 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.144 INFO:tasks.workunit.client.0.smithi006.stdout: "new_obj2": { 2024-05-30T16:06:15.144 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket": { 2024-05-30T16:06:15.144 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "ebtest", 2024-05-30T16:06:15.144 INFO:tasks.workunit.client.0.smithi006.stdout: "marker": "marker", 2024-05-30T16:06:15.144 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_id": "bucket-id", 2024-05-30T16:06:15.144 INFO:tasks.workunit.client.0.smithi006.stdout: "tenant": "", 2024-05-30T16:06:15.144 INFO:tasks.workunit.client.0.smithi006.stdout: "explicit_placement": { 2024-05-30T16:06:15.144 INFO:tasks.workunit.client.0.smithi006.stdout: "data_pool": ".data-pool", 2024-05-30T16:06:15.144 INFO:tasks.workunit.client.0.smithi006.stdout: "data_extra_pool": "", 2024-05-30T16:06:15.144 INFO:tasks.workunit.client.0.smithi006.stdout: "index_pool": ".index-pool" 2024-05-30T16:06:15.144 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.144 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.144 INFO:tasks.workunit.client.0.smithi006.stdout: "key": { 2024-05-30T16:06:15.144 INFO:tasks.workunit.client.0.smithi006.stdout: "name": "_myobj_", 2024-05-30T16:06:15.144 INFO:tasks.workunit.client.0.smithi006.stdout: "instance": "inst", 2024-05-30T16:06:15.145 INFO:tasks.workunit.client.0.smithi006.stdout: "ns": "ns" 2024-05-30T16:06:15.145 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.145 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.145 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.145 INFO:tasks.workunit.client.0.smithi006.stdout:raw2=.data-pool:marker__ns:inst__myobj_ 2024-05-30T16:06:15.145 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] TestRGWObj.old_to_raw (0 ms) 2024-05-30T16:06:15.145 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 4 tests from TestRGWObj (1 ms total) 2024-05-30T16:06:15.145 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-05-30T16:06:15.145 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment tear-down 2024-05-30T16:06:15.145 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] 4 tests from 1 test suite ran. (1 ms total) 2024-05-30T16:06:15.145 INFO:tasks.workunit.client.0.smithi006.stdout:[ PASSED ] 4 tests. 2024-05-30T16:06:15.145 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-30T16:06:15.146 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-30T16:06:15.161 INFO:tasks.workunit:Running workunits matching rgw/test_librgw_file.sh on client.0... 2024-05-30T16:06:15.162 INFO:tasks.workunit:Running workunit rgw/test_librgw_file.sh... 2024-05-30T16:06:15.162 DEBUG:teuthology.orchestra.run.smithi006:workunit test rgw/test_librgw_file.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_librgw_file.sh 2024-05-30T16:06:15.303 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:06:15.303 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:06:15.373 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.369+0000 7f45f8f99980 20 rados->read ofs=0 len=0 2024-05-30T16:06:15.374 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.373+0000 7f45f8f99980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:06:15.374 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.373+0000 7f45f8f99980 20 realm 2024-05-30T16:06:15.375 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.373+0000 7f45f8f99980 20 rados->read ofs=0 len=0 2024-05-30T16:06:15.375 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.373+0000 7f45f8f99980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:06:15.375 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.373+0000 7f45f8f99980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:06:15.376 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.373+0000 7f45f8f99980 20 rados->read ofs=0 len=0 2024-05-30T16:06:15.376 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.373+0000 7f45f8f99980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:06:15.376 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.373+0000 7f45f8f99980 20 rados->read ofs=0 len=0 2024-05-30T16:06:15.377 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.373+0000 7f45f8f99980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:06:15.377 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.373+0000 7f45f8f99980 20 rados->read ofs=0 len=0 2024-05-30T16:06:15.378 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.377+0000 7f45f8f99980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:06:15.378 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.377+0000 7f45f8f99980 20 searching for the correct realm 2024-05-30T16:06:15.385 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.381+0000 7f45f6aaa640 0 -- 172.21.15.6:0/2329854566 >> [v2:172.21.15.6:6816/2704410641,v1:172.21.15.6:6817/2704410641] conn(0x7f45f0026060 msgr2=0x7f45f005c250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:06:15.404 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.401+0000 7f45f72ab640 0 -- 172.21.15.6:0/2329854566 >> [v2:172.21.15.204:6823/3156311527,v1:172.21.15.204:6825/3156311527] conn(0x7f45e0028410 msgr2=0x7f45e0028830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:06:15.421 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.417+0000 7f45f8f99980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:06:15.421 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.417+0000 7f45f8f99980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:06:15.421 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.417+0000 7f45f8f99980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:06:15.421 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.417+0000 7f45f8f99980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:06:15.421 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.417+0000 7f45f8f99980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:06:15.421 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.417+0000 7f45f8f99980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:06:15.421 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.417+0000 7f45f8f99980 20 rados->read ofs=0 len=0 2024-05-30T16:06:15.421 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.417+0000 7f45f8f99980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:06:15.421 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.417+0000 7f45f8f99980 20 rados->read ofs=0 len=0 2024-05-30T16:06:15.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.421+0000 7f45f8f99980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:06:15.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.421+0000 7f45f8f99980 20 rados->read ofs=0 len=0 2024-05-30T16:06:15.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.421+0000 7f45f8f99980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:06:15.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.421+0000 7f45f8f99980 20 zone default found 2024-05-30T16:06:15.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.421+0000 7f45f8f99980 4 Realm: () 2024-05-30T16:06:15.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.421+0000 7f45f8f99980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:06:15.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.421+0000 7f45f8f99980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:06:15.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.421+0000 7f45f8f99980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:06:15.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.421+0000 7f45f8f99980 20 zonegroup default 2024-05-30T16:06:15.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.421+0000 7f45f8f99980 20 rados->read ofs=0 len=0 2024-05-30T16:06:15.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.421+0000 7f45f8f99980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:06:15.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.421+0000 7f45f8f99980 20 rados->read ofs=0 len=0 2024-05-30T16:06:15.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.421+0000 7f45f8f99980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:06:15.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.421+0000 7f45f8f99980 20 rados->read ofs=0 len=0 2024-05-30T16:06:15.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.421+0000 7f45f8f99980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:06:15.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.421+0000 7f45f8f99980 20 started sync module instance, tier type = 2024-05-30T16:06:15.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.421+0000 7f45f8f99980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:06:15.440 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.437+0000 7f45f8f99980 20 add_watcher() i=0 2024-05-30T16:06:15.441 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.437+0000 7f45f8f99980 20 add_watcher() i=1 2024-05-30T16:06:15.441 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.437+0000 7f45f8f99980 20 add_watcher() i=2 2024-05-30T16:06:15.441 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.437+0000 7f45f8f99980 20 add_watcher() i=3 2024-05-30T16:06:15.441 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.437+0000 7f45f8f99980 20 add_watcher() i=4 2024-05-30T16:06:15.441 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.437+0000 7f45f8f99980 20 add_watcher() i=5 2024-05-30T16:06:15.441 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.437+0000 7f45f8f99980 20 add_watcher() i=6 2024-05-30T16:06:15.442 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.441+0000 7f45f8f99980 20 add_watcher() i=7 2024-05-30T16:06:15.442 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.441+0000 7f45f8f99980 2 all 8 watchers are set, enabling cache 2024-05-30T16:06:15.444 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.441+0000 7f44ad7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:06:15.445 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.441+0000 7f45f8f99980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:06:15.445 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.441+0000 7f45f8f99980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:06:15.446 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.441+0000 7f45f8f99980 5 note: GC not initialized 2024-05-30T16:06:15.446 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.441+0000 7f4427fff640 20 reqs_thread_entry: start 2024-05-30T16:06:15.446 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.445+0000 7f45f8f99980 20 init_complete bucket index max shards: 11 2024-05-30T16:06:15.446 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.445+0000 7f45f8f99980 20 Filter name: none 2024-05-30T16:06:15.446 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.445+0000 7f4425ffb640 20 reqs_thread_entry: start 2024-05-30T16:06:15.446 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.445+0000 7f45f8f99980 10 cache get: name=default.rgw.meta+users.uid+ceph-test-librgw-file : miss 2024-05-30T16:06:15.446 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.445+0000 7f45f8f99980 20 rados->read ofs=0 len=0 2024-05-30T16:06:15.447 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.445+0000 7f45f8f99980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:06:15.447 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.445+0000 7f45f8f99980 10 cache put: name=default.rgw.meta+users.uid+ceph-test-librgw-file info.flags=0x0 2024-05-30T16:06:15.447 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.445+0000 7f45f8f99980 10 adding default.rgw.meta+users.uid+ceph-test-librgw-file to cache LRU end 2024-05-30T16:06:15.447 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.445+0000 7f45f8f99980 10 cache get: name=default.rgw.meta+users.email+librgw@example.com : miss 2024-05-30T16:06:15.447 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.445+0000 7f45f8f99980 20 rados->read ofs=0 len=0 2024-05-30T16:06:15.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.445+0000 7f45f8f99980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:06:15.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.445+0000 7f45f8f99980 10 cache put: name=default.rgw.meta+users.email+librgw@example.com info.flags=0x0 2024-05-30T16:06:15.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.445+0000 7f45f8f99980 10 adding default.rgw.meta+users.email+librgw@example.com to cache LRU end 2024-05-30T16:06:15.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.445+0000 7f45f8f99980 10 cache get: name=default.rgw.meta+users.keys+PSQkDFTK7i+AKmWG3+HVHZzq5X4= : miss 2024-05-30T16:06:15.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.445+0000 7f45f8f99980 20 rados->read ofs=0 len=0 2024-05-30T16:06:15.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.445+0000 7f45f8f99980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:06:15.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.445+0000 7f45f8f99980 10 cache put: name=default.rgw.meta+users.keys+PSQkDFTK7i+AKmWG3+HVHZzq5X4= info.flags=0x0 2024-05-30T16:06:15.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.445+0000 7f45f8f99980 10 adding default.rgw.meta+users.keys+PSQkDFTK7i+AKmWG3+HVHZzq5X4= to cache LRU end 2024-05-30T16:06:15.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.445+0000 7f45f8f99980 10 cache get: name=default.rgw.meta+users.keys+PSQkDFTK7i+AKmWG3+HVHZzq5X4= : hit (negative entry) 2024-05-30T16:06:15.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.445+0000 7f45f8f99980 10 cache get: name=default.rgw.meta+users.keys+PSQkDFTK7i+AKmWG3+HVHZzq5X4= : hit (negative entry) 2024-05-30T16:06:15.450 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.449+0000 7f45f8f99980 10 cache put: name=default.rgw.meta+users.uid+ceph-test-librgw-file info.flags=0x17 2024-05-30T16:06:15.450 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.449+0000 7f45f8f99980 10 moving default.rgw.meta+users.uid+ceph-test-librgw-file to cache LRU end 2024-05-30T16:06:15.450 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.449+0000 7f45f8f99980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.uid:ceph-test-librgw-file, ofs0, ns] 2024-05-30T16:06:15.451 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.449+0000 7f44da7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1078036791298 cookie 94621060750976 notifier 5282 bl.length()=697 2024-05-30T16:06:15.451 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.449+0000 7f44da7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ceph-test-librgw-file info.flags=0x17 2024-05-30T16:06:15.451 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.449+0000 7f44da7fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ceph-test-librgw-file to cache LRU end 2024-05-30T16:06:15.455 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.453+0000 7f45f8f99980 10 cache put: name=default.rgw.meta+users.email+librgw@example.com info.flags=0x7 2024-05-30T16:06:15.456 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.453+0000 7f45f8f99980 10 moving default.rgw.meta+users.email+librgw@example.com to cache LRU end 2024-05-30T16:06:15.456 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.453+0000 7f45f8f99980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.email:librgw@example.com, ofs0, ns] 2024-05-30T16:06:15.456 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.453+0000 7f44da7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1078036791296 cookie 94621060759760 notifier 5282 bl.length()=194 2024-05-30T16:06:15.456 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.453+0000 7f44da7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+librgw@example.com info.flags=0x7 2024-05-30T16:06:15.457 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.453+0000 7f44da7fc640 10 rgw watcher librados: moving default.rgw.meta+users.email+librgw@example.com to cache LRU end 2024-05-30T16:06:15.463 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.461+0000 7f45f8f99980 10 cache put: name=default.rgw.meta+users.keys+PSQkDFTK7i+AKmWG3+HVHZzq5X4= info.flags=0x7 2024-05-30T16:06:15.463 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.461+0000 7f45f8f99980 10 moving default.rgw.meta+users.keys+PSQkDFTK7i+AKmWG3+HVHZzq5X4= to cache LRU end 2024-05-30T16:06:15.463 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.461+0000 7f45f8f99980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.keys:PSQkDFTK7i+AKmWG3+HVHZzq5X4=, ofs0, ns] 2024-05-30T16:06:15.463 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.461+0000 7f44d9ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1078036791300 cookie 94621060765248 notifier 5282 bl.length()=203 2024-05-30T16:06:15.464 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.461+0000 7f44d9ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+PSQkDFTK7i+AKmWG3+HVHZzq5X4= info.flags=0x7 2024-05-30T16:06:15.464 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.461+0000 7f44d9ffb640 10 rgw watcher librados: moving default.rgw.meta+users.keys+PSQkDFTK7i+AKmWG3+HVHZzq5X4= to cache LRU end 2024-05-30T16:06:15.466 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:06:15.466 INFO:tasks.workunit.client.0.smithi006.stdout: "user_id": "ceph-test-librgw-file", 2024-05-30T16:06:15.466 INFO:tasks.workunit.client.0.smithi006.stdout: "display_name": "librgw test user", 2024-05-30T16:06:15.466 INFO:tasks.workunit.client.0.smithi006.stdout: "email": "librgw@example.com", 2024-05-30T16:06:15.466 INFO:tasks.workunit.client.0.smithi006.stdout: "suspended": 0, 2024-05-30T16:06:15.466 INFO:tasks.workunit.client.0.smithi006.stdout: "max_buckets": 1000, 2024-05-30T16:06:15.466 INFO:tasks.workunit.client.0.smithi006.stdout: "subusers": [], 2024-05-30T16:06:15.466 INFO:tasks.workunit.client.0.smithi006.stdout: "keys": [ 2024-05-30T16:06:15.466 INFO:tasks.workunit.client.0.smithi006.stdout: { 2024-05-30T16:06:15.466 INFO:tasks.workunit.client.0.smithi006.stdout: "user": "ceph-test-librgw-file", 2024-05-30T16:06:15.467 INFO:tasks.workunit.client.0.smithi006.stdout: "access_key": "PSQkDFTK7i+AKmWG3+HVHZzq5X4=", 2024-05-30T16:06:15.467 INFO:tasks.workunit.client.0.smithi006.stdout: "secret_key": "LJQZywzc2MAwu/pWcRLM4YZ1M6tnR4qoNbCaZlyPxXiELiqEgWkSzw==", 2024-05-30T16:06:15.467 INFO:tasks.workunit.client.0.smithi006.stdout: "active": true, 2024-05-30T16:06:15.467 INFO:tasks.workunit.client.0.smithi006.stdout: "create_date": "2024-05-30T16:06:15.450366Z" 2024-05-30T16:06:15.467 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:06:15.467 INFO:tasks.workunit.client.0.smithi006.stdout: ], 2024-05-30T16:06:15.467 INFO:tasks.workunit.client.0.smithi006.stdout: "swift_keys": [], 2024-05-30T16:06:15.467 INFO:tasks.workunit.client.0.smithi006.stdout: "caps": [], 2024-05-30T16:06:15.467 INFO:tasks.workunit.client.0.smithi006.stdout: "op_mask": "read, write, delete", 2024-05-30T16:06:15.467 INFO:tasks.workunit.client.0.smithi006.stdout: "default_placement": "", 2024-05-30T16:06:15.467 INFO:tasks.workunit.client.0.smithi006.stdout: "default_storage_class": "", 2024-05-30T16:06:15.467 INFO:tasks.workunit.client.0.smithi006.stdout: "placement_tags": [], 2024-05-30T16:06:15.467 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_quota": { 2024-05-30T16:06:15.467 INFO:tasks.workunit.client.0.smithi006.stdout: "enabled": false, 2024-05-30T16:06:15.468 INFO:tasks.workunit.client.0.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:06:15.468 INFO:tasks.workunit.client.0.smithi006.stdout: "max_size": -1, 2024-05-30T16:06:15.468 INFO:tasks.workunit.client.0.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:06:15.468 INFO:tasks.workunit.client.0.smithi006.stdout: "max_objects": -1 2024-05-30T16:06:15.468 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.468 INFO:tasks.workunit.client.0.smithi006.stdout: "user_quota": { 2024-05-30T16:06:15.468 INFO:tasks.workunit.client.0.smithi006.stdout: "enabled": false, 2024-05-30T16:06:15.468 INFO:tasks.workunit.client.0.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:06:15.468 INFO:tasks.workunit.client.0.smithi006.stdout: "max_size": -1, 2024-05-30T16:06:15.468 INFO:tasks.workunit.client.0.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:06:15.468 INFO:tasks.workunit.client.0.smithi006.stdout: "max_objects": -1 2024-05-30T16:06:15.468 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:06:15.469 INFO:tasks.workunit.client.0.smithi006.stdout: "temp_url_keys": [], 2024-05-30T16:06:15.469 INFO:tasks.workunit.client.0.smithi006.stdout: "type": "rgw", 2024-05-30T16:06:15.469 INFO:tasks.workunit.client.0.smithi006.stdout: "mfa_ids": [], 2024-05-30T16:06:15.469 INFO:tasks.workunit.client.0.smithi006.stdout: "account_id": "", 2024-05-30T16:06:15.469 INFO:tasks.workunit.client.0.smithi006.stdout: "path": "/", 2024-05-30T16:06:15.469 INFO:tasks.workunit.client.0.smithi006.stdout: "create_date": "2024-05-30T16:06:15.450348Z", 2024-05-30T16:06:15.469 INFO:tasks.workunit.client.0.smithi006.stdout: "tags": [], 2024-05-30T16:06:15.469 INFO:tasks.workunit.client.0.smithi006.stdout: "group_ids": [] 2024-05-30T16:06:15.469 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:06:15.469 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-05-30T16:06:15.470 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.465+0000 7f44ad7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:06:15.473 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.469+0000 7f45f8f99980 20 remove_watcher() i=0 2024-05-30T16:06:15.473 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.469+0000 7f45f8f99980 2 removed watcher, disabling cache 2024-05-30T16:06:15.475 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.473+0000 7f45f8f99980 20 remove_watcher() i=1 2024-05-30T16:06:15.477 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.473+0000 7f45f8f99980 20 remove_watcher() i=2 2024-05-30T16:06:15.479 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.477+0000 7f45f8f99980 20 remove_watcher() i=3 2024-05-30T16:06:15.481 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.481+0000 7f45f8f99980 20 remove_watcher() i=4 2024-05-30T16:06:15.483 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.481+0000 7f45f8f99980 20 remove_watcher() i=5 2024-05-30T16:06:15.484 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.481+0000 7f45f8f99980 20 remove_watcher() i=6 2024-05-30T16:06:15.486 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:15.485+0000 7f45f8f99980 20 remove_watcher() i=7 2024-05-30T16:06:15.499 INFO:tasks.workunit.client.0.smithi006.stdout:phase 1.1 2024-05-30T16:06:15.516 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] Running 24 tests from 1 test suite. 2024-05-30T16:06:15.516 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment set-up. 2024-05-30T16:06:15.517 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 24 tests from LibRGW 2024-05-30T16:06:15.517 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.INIT 2024-05-30T16:06:15.528 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:06:15.528 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:06:16.121 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.INIT (605 ms) 2024-05-30T16:06:16.121 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MOUNT_NOROOT 2024-05-30T16:06:16.123 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MOUNT_NOROOT (2 ms) 2024-05-30T16:06:16.124 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MOUNT 2024-05-30T16:06:16.124 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2024-05-30T16:06:16.124 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.SETUP_HIER1 2024-05-30T16:06:16.143 INFO:tasks.workunit.client.0.smithi006.stdout:creating: nfsroot:foo/bar/baz/quux 2024-05-30T16:06:16.151 INFO:tasks.workunit.client.0.smithi006.stdout:creating: nfsroot:foo/f1 2024-05-30T16:06:16.156 INFO:tasks.workunit.client.0.smithi006.stdout:creating: nfsroot:foo/f2 2024-05-30T16:06:16.162 INFO:tasks.workunit.client.0.smithi006.stdout:creating: nfsroot:foo/bar/f1 2024-05-30T16:06:16.168 INFO:tasks.workunit.client.0.smithi006.stdout:creating: nfsroot:foo/bar/d1/ 2024-05-30T16:06:16.175 INFO:tasks.workunit.client.0.smithi006.stdout:creating: nfsroot:foo/bar/baz/hungry 2024-05-30T16:06:16.181 INFO:tasks.workunit.client.0.smithi006.stdout:creating: nfsroot:foo/bar/baz/hungry/ 2024-05-30T16:06:16.186 INFO:tasks.workunit.client.0.smithi006.stdout:creating: nfsroot:foo/bar/baz/momma 2024-05-30T16:06:16.192 INFO:tasks.workunit.client.0.smithi006.stdout:creating: nfsroot:foo/bar/baz/bear/ 2024-05-30T16:06:16.199 INFO:tasks.workunit.client.0.smithi006.stdout:creating: nfsroot:foo/bar/baz/sasquatch 2024-05-30T16:06:16.205 INFO:tasks.workunit.client.0.smithi006.stdout:creating: nfsroot:foo/bar/baz/sasquatch/ 2024-05-30T16:06:16.211 INFO:tasks.workunit.client.0.smithi006.stdout:creating: nfsroot:foo/bar/baz/frobozz 2024-05-30T16:06:16.217 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.SETUP_HIER1 (94 ms) 2024-05-30T16:06:16.217 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.SETUP_DIRS1 2024-05-30T16:06:16.466 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.SETUP_DIRS1 (249 ms) 2024-05-30T16:06:16.466 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.SETATTR 2024-05-30T16:06:16.466 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.SETATTR (0 ms) 2024-05-30T16:06:16.466 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.RGW_CREATE_DIRS1 2024-05-30T16:06:16.508 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.RGW_CREATE_DIRS1 (42 ms) 2024-05-30T16:06:16.509 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.RGW_SETUP_RENAME1 2024-05-30T16:06:16.623 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.RGW_SETUP_RENAME1 (115 ms) 2024-05-30T16:06:16.624 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.RGW_INTRABUCKET_RENAME1 2024-05-30T16:06:16.624 INFO:tasks.workunit.client.0.smithi006.stdout:rename file rfile_0 to rfile_r0 (bucket brename0) 2024-05-30T16:06:16.642 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.RGW_INTRABUCKET_RENAME1 (19 ms) 2024-05-30T16:06:16.642 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.RGW_CROSSBUCKET_RENAME1 2024-05-30T16:06:16.642 INFO:tasks.workunit.client.0.smithi006.stdout:rename file rfile_1 (bucket brename0) to rfile_rhilldog (bucket brename1) 2024-05-30T16:06:16.661 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.RGW_CROSSBUCKET_RENAME1 (19 ms) 2024-05-30T16:06:16.661 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.GETATTR_DIRS1 2024-05-30T16:06:16.661 INFO:tasks.workunit.client.0.smithi006.stdout:scanning objects in /bdirs1/dir_0 2024-05-30T16:06:16.661 INFO:tasks.workunit.client.0.smithi006.stdout: /bdirs1/dir_0/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-05-30T16:06:16.661 INFO:tasks.workunit.client.0.smithi006.stdout: /bdirs1/dir_0/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-05-30T16:06:16.662 INFO:tasks.workunit.client.0.smithi006.stdout: /bdirs1/dir_0/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-05-30T16:06:16.662 INFO:tasks.workunit.client.0.smithi006.stdout: /bdirs1/dir_0/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-05-30T16:06:16.662 INFO:tasks.workunit.client.0.smithi006.stdout:scanning objects in /bdirs1/dir_1 2024-05-30T16:06:16.662 INFO:tasks.workunit.client.0.smithi006.stdout: /bdirs1/dir_1/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-05-30T16:06:16.662 INFO:tasks.workunit.client.0.smithi006.stdout: /bdirs1/dir_1/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-05-30T16:06:16.662 INFO:tasks.workunit.client.0.smithi006.stdout: /bdirs1/dir_1/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-05-30T16:06:16.662 INFO:tasks.workunit.client.0.smithi006.stdout: /bdirs1/dir_1/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-05-30T16:06:16.662 INFO:tasks.workunit.client.0.smithi006.stdout:scanning objects in /bdirs1/dir_2 2024-05-30T16:06:16.662 INFO:tasks.workunit.client.0.smithi006.stdout: /bdirs1/dir_2/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-05-30T16:06:16.662 INFO:tasks.workunit.client.0.smithi006.stdout: /bdirs1/dir_2/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-05-30T16:06:16.662 INFO:tasks.workunit.client.0.smithi006.stdout: /bdirs1/dir_2/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-05-30T16:06:16.662 INFO:tasks.workunit.client.0.smithi006.stdout: /bdirs1/dir_2/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-05-30T16:06:16.662 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.GETATTR_DIRS1 (0 ms) 2024-05-30T16:06:16.662 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.READ_DIRS1 2024-05-30T16:06:16.663 INFO:tasks.workunit.client.0.smithi006.stdout:read back objects in /bdirs1/dir_0 2024-05-30T16:06:16.663 INFO:tasks.workunit.client.0.smithi006.stdout:reading 0,256 dir_0/sfile_0 2024-05-30T16:06:16.670 INFO:tasks.workunit.client.0.smithi006.stdout: read back from sfile_0 : "data for sfile_0" 2024-05-30T16:06:16.670 INFO:tasks.workunit.client.0.smithi006.stdout:reading 0,256 dir_0/sfile_1 2024-05-30T16:06:16.674 INFO:tasks.workunit.client.0.smithi006.stdout: read back from sfile_1 : "data for sfile_1" 2024-05-30T16:06:16.674 INFO:tasks.workunit.client.0.smithi006.stdout:read back objects in /bdirs1/dir_1 2024-05-30T16:06:16.675 INFO:tasks.workunit.client.0.smithi006.stdout:reading 0,256 dir_1/sfile_0 2024-05-30T16:06:16.679 INFO:tasks.workunit.client.0.smithi006.stdout: read back from sfile_0 : "data for sfile_0" 2024-05-30T16:06:16.679 INFO:tasks.workunit.client.0.smithi006.stdout:reading 0,256 dir_1/sfile_1 2024-05-30T16:06:16.683 INFO:tasks.workunit.client.0.smithi006.stdout: read back from sfile_1 : "data for sfile_1" 2024-05-30T16:06:16.683 INFO:tasks.workunit.client.0.smithi006.stdout:read back objects in /bdirs1/dir_2 2024-05-30T16:06:16.683 INFO:tasks.workunit.client.0.smithi006.stdout:reading 0,256 dir_2/sfile_0 2024-05-30T16:06:16.687 INFO:tasks.workunit.client.0.smithi006.stdout: read back from sfile_0 : "data for sfile_0" 2024-05-30T16:06:16.687 INFO:tasks.workunit.client.0.smithi006.stdout:reading 0,256 dir_2/sfile_1 2024-05-30T16:06:16.691 INFO:tasks.workunit.client.0.smithi006.stdout: read back from sfile_1 : "data for sfile_1" 2024-05-30T16:06:16.691 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.READ_DIRS1 (30 ms) 2024-05-30T16:06:16.691 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.READF_SETUP1 2024-05-30T16:06:16.939 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.READF_SETUP1 (248 ms) 2024-05-30T16:06:16.939 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.READF_DIRS1 2024-05-30T16:06:16.939 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.READF_DIRS1 (0 ms) 2024-05-30T16:06:16.939 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.WRITEF_DIRS1 2024-05-30T16:06:16.940 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.WRITEF_DIRS1 (0 ms) 2024-05-30T16:06:16.940 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.RELEASE_DIRS1 2024-05-30T16:06:16.940 INFO:tasks.workunit.client.0.smithi006.stdout:release sdir_0 type: DIR refs: 2 2024-05-30T16:06:16.940 INFO:tasks.workunit.client.0.smithi006.stdout:release sfile_0 type: FILE refs: 2 2024-05-30T16:06:16.940 INFO:tasks.workunit.client.0.smithi006.stdout:release sdir_1 type: DIR refs: 2 2024-05-30T16:06:16.940 INFO:tasks.workunit.client.0.smithi006.stdout:release sfile_1 type: FILE refs: 2 2024-05-30T16:06:16.940 INFO:tasks.workunit.client.0.smithi006.stdout:release sdir_0 type: DIR refs: 2 2024-05-30T16:06:16.940 INFO:tasks.workunit.client.0.smithi006.stdout:release sfile_0 type: FILE refs: 2 2024-05-30T16:06:16.940 INFO:tasks.workunit.client.0.smithi006.stdout:release sdir_1 type: DIR refs: 2 2024-05-30T16:06:16.940 INFO:tasks.workunit.client.0.smithi006.stdout:release sfile_1 type: FILE refs: 2 2024-05-30T16:06:16.940 INFO:tasks.workunit.client.0.smithi006.stdout:release sdir_0 type: DIR refs: 2 2024-05-30T16:06:16.940 INFO:tasks.workunit.client.0.smithi006.stdout:release sfile_0 type: FILE refs: 2 2024-05-30T16:06:16.940 INFO:tasks.workunit.client.0.smithi006.stdout:release sdir_1 type: DIR refs: 2 2024-05-30T16:06:16.940 INFO:tasks.workunit.client.0.smithi006.stdout:release sfile_1 type: FILE refs: 2 2024-05-30T16:06:16.941 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.RELEASE_DIRS1 (0 ms) 2024-05-30T16:06:16.941 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.HIER1 2024-05-30T16:06:16.959 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot/foo/f2 (f2): FILE 2024-05-30T16:06:16.959 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot/foo/f1 (f1): FILE 2024-05-30T16:06:16.967 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot/foo/bar/f1 (f1): FILE 2024-05-30T16:06:16.972 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot/foo/bar/d1 (d1): DIR 2024-05-30T16:06:16.980 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2024-05-30T16:06:16.981 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2024-05-30T16:06:16.982 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot/foo/bar/baz/quux (quux): FILE 2024-05-30T16:06:16.984 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot/foo/bar/baz/momma (momma): FILE 2024-05-30T16:06:16.985 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2024-05-30T16:06:16.986 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2024-05-30T16:06:16.987 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot/foo/bar/baz/frobozz (frobozz): FILE 2024-05-30T16:06:16.992 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot/foo/bar/baz/bear (bear): DIR 2024-05-30T16:06:16.992 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot/foo/bar/baz (baz): DIR 2024-05-30T16:06:16.992 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot/foo/bar (bar): DIR 2024-05-30T16:06:16.992 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot/foo (foo): DIR 2024-05-30T16:06:16.992 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot (nfsroot): DIR 2024-05-30T16:06:16.992 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.HIER1 (52 ms) 2024-05-30T16:06:16.992 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-05-30T16:06:16.992 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (0 ms) 2024-05-30T16:06:16.992 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-05-30T16:06:16.993 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2024-05-30T16:06:16.993 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-05-30T16:06:16.993 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2024-05-30T16:06:16.993 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-05-30T16:06:16.993 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-05-30T16:06:16.993 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.CLEANUP 2024-05-30T16:06:16.993 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-05-30T16:06:16.993 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.UMOUNT 2024-05-30T16:06:16.993 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-05-30T16:06:16.993 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-05-30T16:06:16.993 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:16.989+0000 7f4f784f7a80 -1 shutting down 2024-05-30T16:06:16.994 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:16.993+0000 7f4e52249640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 55029, si_uid : 1000, si_addr0x3e80000d6f5, si_status0 2024-05-30T16:06:46.381 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.SHUTDOWN (29390 ms) 2024-05-30T16:06:46.381 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 24 tests from LibRGW (30865 ms total) 2024-05-30T16:06:46.381 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-05-30T16:06:46.382 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment tear-down 2024-05-30T16:06:46.382 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] 24 tests from 1 test suite ran. (30865 ms total) 2024-05-30T16:06:46.382 INFO:tasks.workunit.client.0.smithi006.stdout:[ PASSED ] 24 tests. 2024-05-30T16:06:46.394 INFO:tasks.workunit.client.0.smithi006.stdout:phase 1.2 2024-05-30T16:06:46.418 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] Running 24 tests from 1 test suite. 2024-05-30T16:06:46.418 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment set-up. 2024-05-30T16:06:46.418 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 24 tests from LibRGW 2024-05-30T16:06:46.418 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.INIT 2024-05-30T16:06:46.428 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:06:46.428 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:06:46.722 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.INIT (304 ms) 2024-05-30T16:06:46.722 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MOUNT_NOROOT 2024-05-30T16:06:46.724 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MOUNT_NOROOT (1 ms) 2024-05-30T16:06:46.724 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MOUNT 2024-05-30T16:06:46.724 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MOUNT (1 ms) 2024-05-30T16:06:46.724 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.SETUP_HIER1 2024-05-30T16:06:46.725 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.SETUP_HIER1 (0 ms) 2024-05-30T16:06:46.725 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.SETUP_DIRS1 2024-05-30T16:06:46.725 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.SETUP_DIRS1 (0 ms) 2024-05-30T16:06:46.725 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.SETATTR 2024-05-30T16:06:46.725 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.SETATTR (0 ms) 2024-05-30T16:06:46.725 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.RGW_CREATE_DIRS1 2024-05-30T16:06:46.725 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.RGW_CREATE_DIRS1 (0 ms) 2024-05-30T16:06:46.725 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.RGW_SETUP_RENAME1 2024-05-30T16:06:46.725 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.RGW_SETUP_RENAME1 (0 ms) 2024-05-30T16:06:46.725 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.RGW_INTRABUCKET_RENAME1 2024-05-30T16:06:46.725 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.RGW_INTRABUCKET_RENAME1 (0 ms) 2024-05-30T16:06:46.725 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.RGW_CROSSBUCKET_RENAME1 2024-05-30T16:06:46.725 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.RGW_CROSSBUCKET_RENAME1 (0 ms) 2024-05-30T16:06:46.726 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.GETATTR_DIRS1 2024-05-30T16:06:46.726 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.GETATTR_DIRS1 (0 ms) 2024-05-30T16:06:46.726 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.READ_DIRS1 2024-05-30T16:06:46.726 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.READ_DIRS1 (0 ms) 2024-05-30T16:06:46.726 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.READF_SETUP1 2024-05-30T16:06:46.726 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.READF_SETUP1 (0 ms) 2024-05-30T16:06:46.726 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.READF_DIRS1 2024-05-30T16:06:46.726 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.READF_DIRS1 (0 ms) 2024-05-30T16:06:46.726 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.WRITEF_DIRS1 2024-05-30T16:06:46.726 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.WRITEF_DIRS1 (0 ms) 2024-05-30T16:06:46.726 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.RELEASE_DIRS1 2024-05-30T16:06:46.726 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.RELEASE_DIRS1 (0 ms) 2024-05-30T16:06:46.726 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.HIER1 2024-05-30T16:06:46.727 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.HIER1 (0 ms) 2024-05-30T16:06:46.727 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-05-30T16:06:46.727 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (0 ms) 2024-05-30T16:06:46.727 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-05-30T16:06:46.727 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2024-05-30T16:06:46.727 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-05-30T16:06:46.727 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2024-05-30T16:06:46.727 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-05-30T16:06:46.727 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-05-30T16:06:46.727 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.CLEANUP 2024-05-30T16:06:46.727 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-05-30T16:06:46.727 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.UMOUNT 2024-05-30T16:06:46.727 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-05-30T16:06:46.727 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-05-30T16:06:46.728 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:46.721+0000 7fc00b6aba80 -1 shutting down 2024-05-30T16:06:46.728 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:06:46.725+0000 7fbee53fd640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 55674, si_uid : 1000, si_addr0x3e80000d97a, si_status0 2024-05-30T16:07:17.115 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.SHUTDOWN (30390 ms) 2024-05-30T16:07:17.115 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 24 tests from LibRGW (30696 ms total) 2024-05-30T16:07:17.115 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-05-30T16:07:17.115 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment tear-down 2024-05-30T16:07:17.115 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] 24 tests from 1 test suite ran. (30696 ms total) 2024-05-30T16:07:17.116 INFO:tasks.workunit.client.0.smithi006.stdout:[ PASSED ] 24 tests. 2024-05-30T16:07:17.130 INFO:tasks.workunit.client.0.smithi006.stdout:phase 1.3 2024-05-30T16:07:17.152 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] Running 24 tests from 1 test suite. 2024-05-30T16:07:17.152 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment set-up. 2024-05-30T16:07:17.152 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 24 tests from LibRGW 2024-05-30T16:07:17.152 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.INIT 2024-05-30T16:07:17.165 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:07:17.165 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:07:17.700 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.INIT (548 ms) 2024-05-30T16:07:17.700 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MOUNT_NOROOT 2024-05-30T16:07:17.702 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MOUNT_NOROOT (2 ms) 2024-05-30T16:07:17.702 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MOUNT 2024-05-30T16:07:17.702 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2024-05-30T16:07:17.702 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.SETUP_HIER1 2024-05-30T16:07:17.711 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.SETUP_HIER1 (9 ms) 2024-05-30T16:07:17.711 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.SETUP_DIRS1 2024-05-30T16:07:17.839 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.SETUP_DIRS1 (127 ms) 2024-05-30T16:07:17.839 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.SETATTR 2024-05-30T16:07:17.839 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.SETATTR (0 ms) 2024-05-30T16:07:17.839 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.RGW_CREATE_DIRS1 2024-05-30T16:07:17.839 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.RGW_CREATE_DIRS1 (0 ms) 2024-05-30T16:07:17.839 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.RGW_SETUP_RENAME1 2024-05-30T16:07:17.839 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.RGW_SETUP_RENAME1 (0 ms) 2024-05-30T16:07:17.839 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.RGW_INTRABUCKET_RENAME1 2024-05-30T16:07:17.839 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.RGW_INTRABUCKET_RENAME1 (0 ms) 2024-05-30T16:07:17.839 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.RGW_CROSSBUCKET_RENAME1 2024-05-30T16:07:17.839 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.RGW_CROSSBUCKET_RENAME1 (0 ms) 2024-05-30T16:07:17.840 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.GETATTR_DIRS1 2024-05-30T16:07:17.840 INFO:tasks.workunit.client.0.smithi006.stdout:scanning objects in /bdirs1/dir_0 2024-05-30T16:07:17.840 INFO:tasks.workunit.client.0.smithi006.stdout: /bdirs1/dir_0/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-05-30T16:07:17.840 INFO:tasks.workunit.client.0.smithi006.stdout: /bdirs1/dir_0/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-05-30T16:07:17.840 INFO:tasks.workunit.client.0.smithi006.stdout: /bdirs1/dir_0/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-05-30T16:07:17.840 INFO:tasks.workunit.client.0.smithi006.stdout: /bdirs1/dir_0/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-05-30T16:07:17.840 INFO:tasks.workunit.client.0.smithi006.stdout:scanning objects in /bdirs1/dir_1 2024-05-30T16:07:17.840 INFO:tasks.workunit.client.0.smithi006.stdout: /bdirs1/dir_1/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-05-30T16:07:17.840 INFO:tasks.workunit.client.0.smithi006.stdout: /bdirs1/dir_1/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-05-30T16:07:17.840 INFO:tasks.workunit.client.0.smithi006.stdout: /bdirs1/dir_1/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-05-30T16:07:17.840 INFO:tasks.workunit.client.0.smithi006.stdout: /bdirs1/dir_1/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-05-30T16:07:17.840 INFO:tasks.workunit.client.0.smithi006.stdout:scanning objects in /bdirs1/dir_2 2024-05-30T16:07:17.840 INFO:tasks.workunit.client.0.smithi006.stdout: /bdirs1/dir_2/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-05-30T16:07:17.840 INFO:tasks.workunit.client.0.smithi006.stdout: /bdirs1/dir_2/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-05-30T16:07:17.841 INFO:tasks.workunit.client.0.smithi006.stdout: /bdirs1/dir_2/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-05-30T16:07:17.841 INFO:tasks.workunit.client.0.smithi006.stdout: /bdirs1/dir_2/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-05-30T16:07:17.841 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.GETATTR_DIRS1 (0 ms) 2024-05-30T16:07:17.841 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.READ_DIRS1 2024-05-30T16:07:17.841 INFO:tasks.workunit.client.0.smithi006.stdout:read back objects in /bdirs1/dir_0 2024-05-30T16:07:17.841 INFO:tasks.workunit.client.0.smithi006.stdout:reading 0,256 dir_0/sfile_0 2024-05-30T16:07:17.843 INFO:tasks.workunit.client.0.smithi006.stdout: read back from sfile_0 : "data for sfile_0" 2024-05-30T16:07:17.844 INFO:tasks.workunit.client.0.smithi006.stdout:reading 0,256 dir_0/sfile_1 2024-05-30T16:07:17.846 INFO:tasks.workunit.client.0.smithi006.stdout: read back from sfile_1 : "data for sfile_1" 2024-05-30T16:07:17.846 INFO:tasks.workunit.client.0.smithi006.stdout:read back objects in /bdirs1/dir_1 2024-05-30T16:07:17.846 INFO:tasks.workunit.client.0.smithi006.stdout:reading 0,256 dir_1/sfile_0 2024-05-30T16:07:17.850 INFO:tasks.workunit.client.0.smithi006.stdout: read back from sfile_0 : "data for sfile_0" 2024-05-30T16:07:17.850 INFO:tasks.workunit.client.0.smithi006.stdout:reading 0,256 dir_1/sfile_1 2024-05-30T16:07:17.854 INFO:tasks.workunit.client.0.smithi006.stdout: read back from sfile_1 : "data for sfile_1" 2024-05-30T16:07:17.854 INFO:tasks.workunit.client.0.smithi006.stdout:read back objects in /bdirs1/dir_2 2024-05-30T16:07:17.855 INFO:tasks.workunit.client.0.smithi006.stdout:reading 0,256 dir_2/sfile_0 2024-05-30T16:07:17.859 INFO:tasks.workunit.client.0.smithi006.stdout: read back from sfile_0 : "data for sfile_0" 2024-05-30T16:07:17.859 INFO:tasks.workunit.client.0.smithi006.stdout:reading 0,256 dir_2/sfile_1 2024-05-30T16:07:17.863 INFO:tasks.workunit.client.0.smithi006.stdout: read back from sfile_1 : "data for sfile_1" 2024-05-30T16:07:17.863 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.READ_DIRS1 (24 ms) 2024-05-30T16:07:17.863 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.READF_SETUP1 2024-05-30T16:07:17.863 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.READF_SETUP1 (0 ms) 2024-05-30T16:07:17.863 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.READF_DIRS1 2024-05-30T16:07:17.863 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.READF_DIRS1 (0 ms) 2024-05-30T16:07:17.863 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.WRITEF_DIRS1 2024-05-30T16:07:17.863 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.WRITEF_DIRS1 (0 ms) 2024-05-30T16:07:17.863 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.RELEASE_DIRS1 2024-05-30T16:07:17.864 INFO:tasks.workunit.client.0.smithi006.stdout:release sdir_0 type: DIR refs: 2 2024-05-30T16:07:17.864 INFO:tasks.workunit.client.0.smithi006.stdout:release sfile_0 type: FILE refs: 2 2024-05-30T16:07:17.864 INFO:tasks.workunit.client.0.smithi006.stdout:release sdir_1 type: DIR refs: 2 2024-05-30T16:07:17.864 INFO:tasks.workunit.client.0.smithi006.stdout:release sfile_1 type: FILE refs: 2 2024-05-30T16:07:17.864 INFO:tasks.workunit.client.0.smithi006.stdout:release sdir_0 type: DIR refs: 2 2024-05-30T16:07:17.864 INFO:tasks.workunit.client.0.smithi006.stdout:release sfile_0 type: FILE refs: 2 2024-05-30T16:07:17.864 INFO:tasks.workunit.client.0.smithi006.stdout:release sdir_1 type: DIR refs: 2 2024-05-30T16:07:17.864 INFO:tasks.workunit.client.0.smithi006.stdout:release sfile_1 type: FILE refs: 2 2024-05-30T16:07:17.864 INFO:tasks.workunit.client.0.smithi006.stdout:release sdir_0 type: DIR refs: 2 2024-05-30T16:07:17.864 INFO:tasks.workunit.client.0.smithi006.stdout:release sfile_0 type: FILE refs: 2 2024-05-30T16:07:17.864 INFO:tasks.workunit.client.0.smithi006.stdout:release sdir_1 type: DIR refs: 2 2024-05-30T16:07:17.864 INFO:tasks.workunit.client.0.smithi006.stdout:release sfile_1 type: FILE refs: 2 2024-05-30T16:07:17.864 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.RELEASE_DIRS1 (0 ms) 2024-05-30T16:07:17.864 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.HIER1 2024-05-30T16:07:17.882 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot/foo/f2 (f2): FILE 2024-05-30T16:07:17.882 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot/foo/f1 (f1): FILE 2024-05-30T16:07:17.889 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot/foo/bar/f1 (f1): FILE 2024-05-30T16:07:17.893 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot/foo/bar/d1 (d1): DIR 2024-05-30T16:07:17.900 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2024-05-30T16:07:17.901 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2024-05-30T16:07:17.903 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot/foo/bar/baz/quux (quux): FILE 2024-05-30T16:07:17.904 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot/foo/bar/baz/momma (momma): FILE 2024-05-30T16:07:17.905 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2024-05-30T16:07:17.905 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2024-05-30T16:07:17.906 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot/foo/bar/baz/frobozz (frobozz): FILE 2024-05-30T16:07:17.912 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot/foo/bar/baz/bear (bear): DIR 2024-05-30T16:07:17.912 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot/foo/bar/baz (baz): DIR 2024-05-30T16:07:17.912 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot/foo/bar (bar): DIR 2024-05-30T16:07:17.912 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot/foo (foo): DIR 2024-05-30T16:07:17.912 INFO:tasks.workunit.client.0.smithi006.stdout:/nfsroot (nfsroot): DIR 2024-05-30T16:07:17.912 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.HIER1 (48 ms) 2024-05-30T16:07:17.912 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-05-30T16:07:17.912 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (0 ms) 2024-05-30T16:07:17.912 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-05-30T16:07:17.912 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2024-05-30T16:07:17.912 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-05-30T16:07:17.912 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2024-05-30T16:07:17.913 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-05-30T16:07:17.913 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-05-30T16:07:17.913 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.CLEANUP 2024-05-30T16:07:17.913 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-05-30T16:07:17.913 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.UMOUNT 2024-05-30T16:07:17.913 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-05-30T16:07:17.913 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-05-30T16:07:17.913 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:07:17.909+0000 7ffba9405a80 -1 shutting down 2024-05-30T16:07:17.914 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:07:17.913+0000 7ffa83157640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 56319, si_uid : 1000, si_addr0x3e80000dbff, si_status0 2024-05-30T16:07:47.942 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.SHUTDOWN (30030 ms) 2024-05-30T16:07:47.942 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 24 tests from LibRGW (30789 ms total) 2024-05-30T16:07:47.942 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-05-30T16:07:47.942 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment tear-down 2024-05-30T16:07:47.943 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] 24 tests from 1 test suite ran. (30789 ms total) 2024-05-30T16:07:47.943 INFO:tasks.workunit.client.0.smithi006.stdout:[ PASSED ] 24 tests. 2024-05-30T16:07:47.956 INFO:tasks.workunit.client.0.smithi006.stdout:phase 2.1 2024-05-30T16:07:47.979 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] Running 9 tests from 1 test suite. 2024-05-30T16:07:47.979 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment set-up. 2024-05-30T16:07:47.979 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 9 tests from LibRGW 2024-05-30T16:07:47.980 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.INIT 2024-05-30T16:07:47.991 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:07:47.991 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:07:48.533 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.INIT (554 ms) 2024-05-30T16:07:48.533 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MOUNT 2024-05-30T16:07:48.534 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MOUNT (1 ms) 2024-05-30T16:07:48.534 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-05-30T16:07:48.552 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.CREATE_BUCKET (18 ms) 2024-05-30T16:07:48.553 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-05-30T16:07:48.553 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.DELETE_BUCKET (0 ms) 2024-05-30T16:07:48.553 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.CREATE_BUCKET_MULTI 2024-05-30T16:07:48.572 INFO:tasks.workunit.client.0.smithi006.stdout:created: sorrydave0 2024-05-30T16:07:48.592 INFO:tasks.workunit.client.0.smithi006.stdout:created: sorrydave1 2024-05-30T16:07:48.612 INFO:tasks.workunit.client.0.smithi006.stdout:created: sorrydave2 2024-05-30T16:07:48.632 INFO:tasks.workunit.client.0.smithi006.stdout:created: sorrydave3 2024-05-30T16:07:48.652 INFO:tasks.workunit.client.0.smithi006.stdout:created: sorrydave4 2024-05-30T16:07:48.672 INFO:tasks.workunit.client.0.smithi006.stdout:created: sorrydave5 2024-05-30T16:07:48.695 INFO:tasks.workunit.client.0.smithi006.stdout:created: sorrydave6 2024-05-30T16:07:48.717 INFO:tasks.workunit.client.0.smithi006.stdout:created: sorrydave7 2024-05-30T16:07:48.740 INFO:tasks.workunit.client.0.smithi006.stdout:created: sorrydave8 2024-05-30T16:07:48.764 INFO:tasks.workunit.client.0.smithi006.stdout:created: sorrydave9 2024-05-30T16:07:48.764 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.CREATE_BUCKET_MULTI (212 ms) 2024-05-30T16:07:48.764 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.DELETE_BUCKET_MULTI 2024-05-30T16:07:49.584 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.DELETE_BUCKET_MULTI (820 ms) 2024-05-30T16:07:49.585 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.CLEANUP 2024-05-30T16:07:49.585 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-05-30T16:07:49.585 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.UMOUNT 2024-05-30T16:07:49.585 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-05-30T16:07:49.585 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-05-30T16:07:49.585 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:07:49.580+0000 7f1ed737ca80 -1 shutting down 2024-05-30T16:07:49.587 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:07:49.584+0000 7f1db10ce640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 56965, si_uid : 1000, si_addr0x3e80000de85, si_status0 2024-05-30T16:08:18.645 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.SHUTDOWN (29060 ms) 2024-05-30T16:08:18.645 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 9 tests from LibRGW (30665 ms total) 2024-05-30T16:08:18.645 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-05-30T16:08:18.646 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment tear-down 2024-05-30T16:08:18.646 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] 9 tests from 1 test suite ran. (30665 ms total) 2024-05-30T16:08:18.646 INFO:tasks.workunit.client.0.smithi006.stdout:[ PASSED ] 9 tests. 2024-05-30T16:08:18.659 INFO:tasks.workunit.client.0.smithi006.stdout:phase 2.2 2024-05-30T16:08:18.682 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] Running 9 tests from 1 test suite. 2024-05-30T16:08:18.683 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment set-up. 2024-05-30T16:08:18.683 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 9 tests from LibRGW 2024-05-30T16:08:18.683 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.INIT 2024-05-30T16:08:18.695 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:08:18.695 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:08:19.471 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.INIT (789 ms) 2024-05-30T16:08:19.472 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MOUNT 2024-05-30T16:08:19.473 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MOUNT (2 ms) 2024-05-30T16:08:19.473 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-05-30T16:08:19.473 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.CREATE_BUCKET (0 ms) 2024-05-30T16:08:19.473 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-05-30T16:08:19.540 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.DELETE_BUCKET (67 ms) 2024-05-30T16:08:19.540 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.CREATE_BUCKET_MULTI 2024-05-30T16:08:19.561 INFO:tasks.workunit.client.0.smithi006.stdout:created: sorrydave0 2024-05-30T16:08:19.584 INFO:tasks.workunit.client.0.smithi006.stdout:created: sorrydave1 2024-05-30T16:08:19.606 INFO:tasks.workunit.client.0.smithi006.stdout:created: sorrydave2 2024-05-30T16:08:19.627 INFO:tasks.workunit.client.0.smithi006.stdout:created: sorrydave3 2024-05-30T16:08:19.649 INFO:tasks.workunit.client.0.smithi006.stdout:created: sorrydave4 2024-05-30T16:08:19.671 INFO:tasks.workunit.client.0.smithi006.stdout:created: sorrydave5 2024-05-30T16:08:19.693 INFO:tasks.workunit.client.0.smithi006.stdout:created: sorrydave6 2024-05-30T16:08:19.715 INFO:tasks.workunit.client.0.smithi006.stdout:created: sorrydave7 2024-05-30T16:08:19.735 INFO:tasks.workunit.client.0.smithi006.stdout:created: sorrydave8 2024-05-30T16:08:19.758 INFO:tasks.workunit.client.0.smithi006.stdout:created: sorrydave9 2024-05-30T16:08:19.758 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.CREATE_BUCKET_MULTI (218 ms) 2024-05-30T16:08:19.758 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.DELETE_BUCKET_MULTI 2024-05-30T16:08:20.538 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.DELETE_BUCKET_MULTI (780 ms) 2024-05-30T16:08:20.538 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.CLEANUP 2024-05-30T16:08:20.538 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-05-30T16:08:20.538 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.UMOUNT 2024-05-30T16:08:20.538 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-05-30T16:08:20.538 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-05-30T16:08:20.539 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:08:20.540+0000 7f00d953ca80 -1 shutting down 2024-05-30T16:08:20.541 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:08:20.540+0000 7effb328e640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 57610, si_uid : 1000, si_addr0x3e80000e10a, si_status0 2024-05-30T16:08:49.532 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.SHUTDOWN (28993 ms) 2024-05-30T16:08:49.532 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 9 tests from LibRGW (30849 ms total) 2024-05-30T16:08:49.532 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-05-30T16:08:49.532 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment tear-down 2024-05-30T16:08:49.532 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] 9 tests from 1 test suite ran. (30849 ms total) 2024-05-30T16:08:49.532 INFO:tasks.workunit.client.0.smithi006.stdout:[ PASSED ] 9 tests. 2024-05-30T16:08:49.541 INFO:tasks.workunit.client.0.smithi006.stdout:phase 3.1 2024-05-30T16:08:49.877 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] Running 13 tests from 1 test suite. 2024-05-30T16:08:49.877 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment set-up. 2024-05-30T16:08:49.877 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 13 tests from LibRGW 2024-05-30T16:08:49.877 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.INIT 2024-05-30T16:08:49.887 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:08:49.887 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:08:50.451 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.INIT (574 ms) 2024-05-30T16:08:50.452 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MOUNT 2024-05-30T16:08:50.453 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MOUNT (2 ms) 2024-05-30T16:08:50.453 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-05-30T16:08:50.474 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.CREATE_BUCKET (21 ms) 2024-05-30T16:08:50.474 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2024-05-30T16:08:50.481 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (7 ms) 2024-05-30T16:08:50.481 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2024-05-30T16:08:50.484 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (3 ms) 2024-05-30T16:08:50.484 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.LARGE1 2024-05-30T16:08:51.051 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.LARGE1 (567 ms) 2024-05-30T16:08:51.051 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.LARGE2 2024-05-30T16:08:51.142 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.LARGE2 (91 ms) 2024-05-30T16:08:51.142 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.STAT_OBJECT 2024-05-30T16:08:51.142 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2024-05-30T16:08:51.142 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2024-05-30T16:08:51.142 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.DELETE_OBJECT (0 ms) 2024-05-30T16:08:51.142 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-05-30T16:08:51.142 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.DELETE_BUCKET (0 ms) 2024-05-30T16:08:51.142 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.CLEANUP 2024-05-30T16:08:51.142 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-05-30T16:08:51.142 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.UMOUNT 2024-05-30T16:08:51.143 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-05-30T16:08:51.143 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-05-30T16:08:51.143 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:08:51.144+0000 7f034ebcba80 -1 shutting down 2024-05-30T16:08:51.145 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:08:51.144+0000 7f022891d640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 58255, si_uid : 1000, si_addr0x3e80000e38f, si_status0 2024-05-30T16:09:20.654 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.SHUTDOWN (29512 ms) 2024-05-30T16:09:20.655 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 13 tests from LibRGW (30777 ms total) 2024-05-30T16:09:20.655 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-05-30T16:09:20.655 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment tear-down 2024-05-30T16:09:20.655 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] 13 tests from 1 test suite ran. (30777 ms total) 2024-05-30T16:09:20.655 INFO:tasks.workunit.client.0.smithi006.stdout:[ PASSED ] 13 tests. 2024-05-30T16:09:20.672 INFO:tasks.workunit.client.0.smithi006.stdout:phase 3.2 2024-05-30T16:09:21.013 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] Running 13 tests from 1 test suite. 2024-05-30T16:09:21.013 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment set-up. 2024-05-30T16:09:21.013 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 13 tests from LibRGW 2024-05-30T16:09:21.013 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.INIT 2024-05-30T16:09:21.022 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:09:21.022 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:09:21.325 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.INIT (312 ms) 2024-05-30T16:09:21.325 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MOUNT 2024-05-30T16:09:21.326 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MOUNT (1 ms) 2024-05-30T16:09:21.326 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-05-30T16:09:21.326 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.CREATE_BUCKET (0 ms) 2024-05-30T16:09:21.326 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2024-05-30T16:09:21.337 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (12 ms) 2024-05-30T16:09:21.338 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2024-05-30T16:09:21.343 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (5 ms) 2024-05-30T16:09:21.343 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.LARGE1 2024-05-30T16:09:21.437 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.LARGE1 (94 ms) 2024-05-30T16:09:21.437 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.LARGE2 2024-05-30T16:09:21.437 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.LARGE2 (0 ms) 2024-05-30T16:09:21.437 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.STAT_OBJECT 2024-05-30T16:09:21.437 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2024-05-30T16:09:21.437 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2024-05-30T16:09:21.447 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.DELETE_OBJECT (10 ms) 2024-05-30T16:09:21.470 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-05-30T16:09:21.524 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.DELETE_BUCKET (77 ms) 2024-05-30T16:09:21.524 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.CLEANUP 2024-05-30T16:09:21.524 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-05-30T16:09:21.524 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.UMOUNT 2024-05-30T16:09:21.524 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-05-30T16:09:21.524 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-05-30T16:09:21.525 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:09:21.523+0000 7f1869abaa80 -1 shutting down 2024-05-30T16:09:21.527 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:09:21.527+0000 7f174380c640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 58902, si_uid : 1000, si_addr0x3e80000e616, si_status0 2024-05-30T16:09:51.465 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.SHUTDOWN (29940 ms) 2024-05-30T16:09:51.465 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 13 tests from LibRGW (30451 ms total) 2024-05-30T16:09:51.465 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-05-30T16:09:51.465 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment tear-down 2024-05-30T16:09:51.465 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] 13 tests from 1 test suite ran. (30451 ms total) 2024-05-30T16:09:51.466 INFO:tasks.workunit.client.0.smithi006.stdout:[ PASSED ] 13 tests. 2024-05-30T16:09:51.484 INFO:tasks.workunit.client.0.smithi006.stdout:phase 4.1 2024-05-30T16:09:51.508 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] Running 11 tests from 1 test suite. 2024-05-30T16:09:51.508 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment set-up. 2024-05-30T16:09:51.508 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 11 tests from LibRGW 2024-05-30T16:09:51.508 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.TVAR 2024-05-30T16:09:51.508 INFO:tasks.workunit.client.0.smithi006.stdout:read i1: 64001 2024-05-30T16:09:51.508 INFO:tasks.workunit.client.0.smithi006.stdout:read s1: blunderbuss 2024-05-30T16:09:51.508 INFO:tasks.workunit.client.0.smithi006.stdout:read s3: [ OK ] LibRGW.TVAR (0 ms) 2024-05-30T16:09:51.508 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.INIT 2024-05-30T16:09:51.521 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:09:51.521 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:09:52.048 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.INIT (540 ms) 2024-05-30T16:09:52.048 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MOUNT 2024-05-30T16:09:52.049 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MOUNT (1 ms) 2024-05-30T16:09:52.049 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-05-30T16:09:52.083 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (35 ms) 2024-05-30T16:09:52.083 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-05-30T16:09:53.233 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_0 2024-05-30T16:09:53.233 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_1 2024-05-30T16:09:53.233 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_2 2024-05-30T16:09:53.233 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_3 2024-05-30T16:09:53.233 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_4 2024-05-30T16:09:53.233 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_5 2024-05-30T16:09:53.233 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_6 2024-05-30T16:09:53.233 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_7 2024-05-30T16:09:53.233 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_8 2024-05-30T16:09:53.233 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_9 2024-05-30T16:09:53.233 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_10 2024-05-30T16:09:53.234 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_11 2024-05-30T16:09:53.234 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_12 2024-05-30T16:09:53.234 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_13 2024-05-30T16:09:53.234 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_14 2024-05-30T16:09:53.234 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_15 2024-05-30T16:09:53.234 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_16 2024-05-30T16:09:53.234 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_17 2024-05-30T16:09:53.234 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_18 2024-05-30T16:09:53.234 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_19 2024-05-30T16:09:53.234 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_20 2024-05-30T16:09:53.234 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_21 2024-05-30T16:09:53.234 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_22 2024-05-30T16:09:53.234 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_23 2024-05-30T16:09:53.235 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_24 2024-05-30T16:09:53.235 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_25 2024-05-30T16:09:53.235 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_26 2024-05-30T16:09:53.235 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_27 2024-05-30T16:09:53.235 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_28 2024-05-30T16:09:53.235 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_29 2024-05-30T16:09:53.235 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_30 2024-05-30T16:09:53.235 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_31 2024-05-30T16:09:53.235 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_32 2024-05-30T16:09:53.235 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_33 2024-05-30T16:09:53.235 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_34 2024-05-30T16:09:53.235 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_35 2024-05-30T16:09:53.235 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_36 2024-05-30T16:09:53.235 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_37 2024-05-30T16:09:53.236 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_38 2024-05-30T16:09:53.236 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_39 2024-05-30T16:09:53.236 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_40 2024-05-30T16:09:53.236 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_41 2024-05-30T16:09:53.236 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_42 2024-05-30T16:09:53.236 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_43 2024-05-30T16:09:53.236 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_44 2024-05-30T16:09:53.236 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_45 2024-05-30T16:09:53.236 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_46 2024-05-30T16:09:53.236 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_47 2024-05-30T16:09:53.236 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_48 2024-05-30T16:09:53.236 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_49 2024-05-30T16:09:53.236 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_50 2024-05-30T16:09:53.236 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_51 2024-05-30T16:09:53.237 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_52 2024-05-30T16:09:53.237 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_53 2024-05-30T16:09:53.237 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_54 2024-05-30T16:09:53.237 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_55 2024-05-30T16:09:53.237 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_56 2024-05-30T16:09:53.237 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_57 2024-05-30T16:09:53.237 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_58 2024-05-30T16:09:53.237 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_59 2024-05-30T16:09:53.237 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_60 2024-05-30T16:09:53.237 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_61 2024-05-30T16:09:53.237 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_62 2024-05-30T16:09:53.237 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_63 2024-05-30T16:09:53.237 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_64 2024-05-30T16:09:53.237 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_65 2024-05-30T16:09:53.238 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_66 2024-05-30T16:09:53.238 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_67 2024-05-30T16:09:53.238 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_68 2024-05-30T16:09:53.238 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_69 2024-05-30T16:09:53.238 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_70 2024-05-30T16:09:53.238 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_71 2024-05-30T16:09:53.238 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_72 2024-05-30T16:09:53.238 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_73 2024-05-30T16:09:53.238 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_74 2024-05-30T16:09:53.238 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_75 2024-05-30T16:09:53.238 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_76 2024-05-30T16:09:53.238 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_77 2024-05-30T16:09:53.238 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_78 2024-05-30T16:09:53.239 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_79 2024-05-30T16:09:53.239 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_80 2024-05-30T16:09:53.239 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_81 2024-05-30T16:09:53.239 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_82 2024-05-30T16:09:53.239 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_83 2024-05-30T16:09:53.239 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_84 2024-05-30T16:09:53.239 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_85 2024-05-30T16:09:53.239 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_86 2024-05-30T16:09:53.239 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_87 2024-05-30T16:09:53.239 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_88 2024-05-30T16:09:53.239 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_89 2024-05-30T16:09:53.239 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_90 2024-05-30T16:09:53.239 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_91 2024-05-30T16:09:53.240 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_92 2024-05-30T16:09:53.240 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_93 2024-05-30T16:09:53.240 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_94 2024-05-30T16:09:53.240 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_95 2024-05-30T16:09:53.240 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_96 2024-05-30T16:09:53.240 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_97 2024-05-30T16:09:53.240 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_98 2024-05-30T16:09:53.240 INFO:tasks.workunit.client.0.smithi006.stdout:created: dmarker:f_99 2024-05-30T16:09:53.240 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (1148 ms) 2024-05-30T16:09:53.240 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-05-30T16:09:53.241 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=. flags=4 2024-05-30T16:09:53.241 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=.. flags=4 2024-05-30T16:09:53.241 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_0 flags=8 2024-05-30T16:09:53.241 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_1 flags=8 2024-05-30T16:09:53.241 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_10 flags=8 2024-05-30T16:09:53.241 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_11 flags=8 2024-05-30T16:09:53.241 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_12 flags=8 2024-05-30T16:09:53.241 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_13 flags=8 2024-05-30T16:09:53.242 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_14 flags=8 2024-05-30T16:09:53.242 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_15 flags=8 2024-05-30T16:09:53.242 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_16 flags=8 2024-05-30T16:09:53.242 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_17 flags=8 2024-05-30T16:09:53.242 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_18 flags=8 2024-05-30T16:09:53.242 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_19 flags=8 2024-05-30T16:09:53.242 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_2 flags=8 2024-05-30T16:09:53.242 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_20 flags=8 2024-05-30T16:09:53.242 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_21 flags=8 2024-05-30T16:09:53.242 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_22 flags=8 2024-05-30T16:09:53.242 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_23 flags=8 2024-05-30T16:09:53.242 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_24 flags=8 2024-05-30T16:09:53.242 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_25 flags=8 2024-05-30T16:09:53.243 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_26 flags=8 2024-05-30T16:09:53.243 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_27 flags=8 2024-05-30T16:09:53.243 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_28 flags=8 2024-05-30T16:09:53.243 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_29 flags=8 2024-05-30T16:09:53.243 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_3 flags=8 2024-05-30T16:09:53.243 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_30 flags=8 2024-05-30T16:09:53.243 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_31 flags=8 2024-05-30T16:09:53.243 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_32 flags=8 2024-05-30T16:09:53.243 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_33 flags=8 2024-05-30T16:09:53.243 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_34 flags=8 2024-05-30T16:09:53.243 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_35 flags=8 2024-05-30T16:09:53.243 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_36 flags=8 2024-05-30T16:09:53.244 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_37 flags=8 2024-05-30T16:09:53.244 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_38 flags=8 2024-05-30T16:09:53.244 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_39 flags=8 2024-05-30T16:09:53.244 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_4 flags=8 2024-05-30T16:09:53.244 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_40 flags=8 2024-05-30T16:09:53.244 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_41 flags=8 2024-05-30T16:09:53.244 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_42 flags=8 2024-05-30T16:09:53.244 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_43 flags=8 2024-05-30T16:09:53.244 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_44 flags=8 2024-05-30T16:09:53.244 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_45 flags=8 2024-05-30T16:09:53.244 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_46 flags=8 2024-05-30T16:09:53.244 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_47 flags=8 2024-05-30T16:09:53.244 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_48 flags=8 2024-05-30T16:09:53.245 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_49 flags=8 2024-05-30T16:09:53.245 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_5 flags=8 2024-05-30T16:09:53.245 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_50 flags=8 2024-05-30T16:09:53.245 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_51 flags=8 2024-05-30T16:09:53.245 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_52 flags=8 2024-05-30T16:09:53.245 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_53 flags=8 2024-05-30T16:09:53.245 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_54 flags=8 2024-05-30T16:09:53.245 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_55 flags=8 2024-05-30T16:09:53.245 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_56 flags=8 2024-05-30T16:09:53.245 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_57 flags=8 2024-05-30T16:09:53.245 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_58 flags=8 2024-05-30T16:09:53.245 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_59 flags=8 2024-05-30T16:09:53.245 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_6 flags=8 2024-05-30T16:09:53.246 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_60 flags=8 2024-05-30T16:09:53.246 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_61 flags=8 2024-05-30T16:09:53.246 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_62 flags=8 2024-05-30T16:09:53.246 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_63 flags=8 2024-05-30T16:09:53.246 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_64 flags=8 2024-05-30T16:09:53.246 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_65 flags=8 2024-05-30T16:09:53.246 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_66 flags=8 2024-05-30T16:09:53.246 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_67 flags=8 2024-05-30T16:09:53.246 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_68 flags=8 2024-05-30T16:09:53.246 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_69 flags=8 2024-05-30T16:09:53.246 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_7 flags=8 2024-05-30T16:09:53.246 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_70 flags=8 2024-05-30T16:09:53.246 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_71 flags=8 2024-05-30T16:09:53.247 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_72 flags=8 2024-05-30T16:09:53.247 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_73 flags=8 2024-05-30T16:09:53.247 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_74 flags=8 2024-05-30T16:09:53.247 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_75 flags=8 2024-05-30T16:09:53.247 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_76 flags=8 2024-05-30T16:09:53.247 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_77 flags=8 2024-05-30T16:09:53.247 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_78 flags=8 2024-05-30T16:09:53.247 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_79 flags=8 2024-05-30T16:09:53.247 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_8 flags=8 2024-05-30T16:09:53.247 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_80 flags=8 2024-05-30T16:09:53.247 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_81 flags=8 2024-05-30T16:09:53.247 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_82 flags=8 2024-05-30T16:09:53.247 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_83 flags=8 2024-05-30T16:09:53.247 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_84 flags=8 2024-05-30T16:09:53.247 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_85 flags=8 2024-05-30T16:09:53.248 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_86 flags=8 2024-05-30T16:09:53.248 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_87 flags=8 2024-05-30T16:09:53.248 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_88 flags=8 2024-05-30T16:09:53.248 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_89 flags=8 2024-05-30T16:09:53.248 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_9 flags=8 2024-05-30T16:09:53.248 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_90 flags=8 2024-05-30T16:09:53.248 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_91 flags=8 2024-05-30T16:09:53.248 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_92 flags=8 2024-05-30T16:09:53.248 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_93 flags=8 2024-05-30T16:09:53.248 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_94 flags=8 2024-05-30T16:09:53.248 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_95 flags=8 2024-05-30T16:09:53.248 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_96 flags=8 2024-05-30T16:09:53.248 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_97 flags=8 2024-05-30T16:09:53.249 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_98 flags=8 2024-05-30T16:09:53.249 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_99 flags=8 2024-05-30T16:09:53.249 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MARKER1_READDIR (8 ms) 2024-05-30T16:09:53.249 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MARKER2_READDIR 2024-05-30T16:09:53.250 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_0 flags=8 2024-05-30T16:09:53.250 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_1 flags=8 2024-05-30T16:09:53.250 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_10 flags=8 2024-05-30T16:09:53.250 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_11 flags=8 2024-05-30T16:09:53.250 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_12 flags=8 2024-05-30T16:09:53.250 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_13 flags=8 2024-05-30T16:09:53.250 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_14 flags=8 2024-05-30T16:09:53.250 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_15 flags=8 2024-05-30T16:09:53.250 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_16 flags=8 2024-05-30T16:09:53.251 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_17 flags=8 2024-05-30T16:09:53.251 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_18 flags=8 2024-05-30T16:09:53.251 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_19 flags=8 2024-05-30T16:09:53.251 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_2 flags=8 2024-05-30T16:09:53.251 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_20 flags=8 2024-05-30T16:09:53.251 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_21 flags=8 2024-05-30T16:09:53.251 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_22 flags=8 2024-05-30T16:09:53.251 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_23 flags=8 2024-05-30T16:09:53.251 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_24 flags=8 2024-05-30T16:09:53.251 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_25 flags=8 2024-05-30T16:09:53.251 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_26 flags=8 2024-05-30T16:09:53.251 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_27 flags=8 2024-05-30T16:09:53.251 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_28 flags=8 2024-05-30T16:09:53.252 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_29 flags=8 2024-05-30T16:09:53.252 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_3 flags=8 2024-05-30T16:09:53.252 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_30 flags=8 2024-05-30T16:09:53.252 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_31 flags=8 2024-05-30T16:09:53.252 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_32 flags=8 2024-05-30T16:09:53.252 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_33 flags=8 2024-05-30T16:09:53.252 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_34 flags=8 2024-05-30T16:09:53.252 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_35 flags=8 2024-05-30T16:09:53.252 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_36 flags=8 2024-05-30T16:09:53.252 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_37 flags=8 2024-05-30T16:09:53.252 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_38 flags=8 2024-05-30T16:09:53.252 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_39 flags=8 2024-05-30T16:09:53.252 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_4 flags=8 2024-05-30T16:09:53.253 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_40 flags=8 2024-05-30T16:09:53.253 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_41 flags=8 2024-05-30T16:09:53.253 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_42 flags=8 2024-05-30T16:09:53.253 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_43 flags=8 2024-05-30T16:09:53.253 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_44 flags=8 2024-05-30T16:09:53.253 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_45 flags=8 2024-05-30T16:09:53.253 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_46 flags=8 2024-05-30T16:09:53.253 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_47 flags=8 2024-05-30T16:09:53.253 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_48 flags=8 2024-05-30T16:09:53.253 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_49 flags=8 2024-05-30T16:09:53.253 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_5 flags=8 2024-05-30T16:09:53.253 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_50 flags=8 2024-05-30T16:09:53.254 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_51 flags=8 2024-05-30T16:09:53.254 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_52 flags=8 2024-05-30T16:09:53.254 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_53 flags=8 2024-05-30T16:09:53.254 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_54 flags=8 2024-05-30T16:09:53.254 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_55 flags=8 2024-05-30T16:09:53.254 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_56 flags=8 2024-05-30T16:09:53.254 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_57 flags=8 2024-05-30T16:09:53.254 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_58 flags=8 2024-05-30T16:09:53.254 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_59 flags=8 2024-05-30T16:09:53.254 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_6 flags=8 2024-05-30T16:09:53.254 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_60 flags=8 2024-05-30T16:09:53.254 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_61 flags=8 2024-05-30T16:09:53.255 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_62 flags=8 2024-05-30T16:09:53.255 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_63 flags=8 2024-05-30T16:09:53.255 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_64 flags=8 2024-05-30T16:09:53.255 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_65 flags=8 2024-05-30T16:09:53.255 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_66 flags=8 2024-05-30T16:09:53.255 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_67 flags=8 2024-05-30T16:09:53.255 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_68 flags=8 2024-05-30T16:09:53.255 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_69 flags=8 2024-05-30T16:09:53.255 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_7 flags=8 2024-05-30T16:09:53.255 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_70 flags=8 2024-05-30T16:09:53.255 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_71 flags=8 2024-05-30T16:09:53.255 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_72 flags=8 2024-05-30T16:09:53.255 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_73 flags=8 2024-05-30T16:09:53.257 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_74 flags=8 2024-05-30T16:09:53.257 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_75 flags=8 2024-05-30T16:09:53.257 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_76 flags=8 2024-05-30T16:09:53.257 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_77 flags=8 2024-05-30T16:09:53.257 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_78 flags=8 2024-05-30T16:09:53.257 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_79 flags=8 2024-05-30T16:09:53.257 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_8 flags=8 2024-05-30T16:09:53.257 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_80 flags=8 2024-05-30T16:09:53.258 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_81 flags=8 2024-05-30T16:09:53.258 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_82 flags=8 2024-05-30T16:09:53.258 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_83 flags=8 2024-05-30T16:09:53.258 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_84 flags=8 2024-05-30T16:09:53.258 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_85 flags=8 2024-05-30T16:09:53.258 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_86 flags=8 2024-05-30T16:09:53.258 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_87 flags=8 2024-05-30T16:09:53.258 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_88 flags=8 2024-05-30T16:09:53.258 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_89 flags=8 2024-05-30T16:09:53.258 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_9 flags=8 2024-05-30T16:09:53.258 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_90 flags=8 2024-05-30T16:09:53.258 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_91 flags=8 2024-05-30T16:09:53.258 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_92 flags=8 2024-05-30T16:09:53.258 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_93 flags=8 2024-05-30T16:09:53.259 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_94 flags=8 2024-05-30T16:09:53.259 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_95 flags=8 2024-05-30T16:09:53.259 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_96 flags=8 2024-05-30T16:09:53.259 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_97 flags=8 2024-05-30T16:09:53.259 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_98 flags=8 2024-05-30T16:09:53.259 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_99 flags=8 2024-05-30T16:09:53.259 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MARKER2_READDIR (5 ms) 2024-05-30T16:09:53.259 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-05-30T16:09:53.259 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-05-30T16:09:53.259 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.CLEANUP 2024-05-30T16:09:53.259 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-05-30T16:09:53.259 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.UMOUNT 2024-05-30T16:09:53.259 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-05-30T16:09:53.259 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-05-30T16:09:53.260 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:09:53.247+0000 7f6e3c9ada80 -1 shutting down 2024-05-30T16:09:53.260 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:09:53.247+0000 7f6d166ff640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 59547, si_uid : 1000, si_addr0x3e80000e89b, si_status0 2024-05-30T16:10:22.158 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.SHUTDOWN (28912 ms) 2024-05-30T16:10:22.158 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 11 tests from LibRGW (30649 ms total) 2024-05-30T16:10:22.158 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-05-30T16:10:22.158 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment tear-down 2024-05-30T16:10:22.158 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] 11 tests from 1 test suite ran. (30649 ms total) 2024-05-30T16:10:22.158 INFO:tasks.workunit.client.0.smithi006.stdout:[ PASSED ] 11 tests. 2024-05-30T16:10:22.173 INFO:tasks.workunit.client.0.smithi006.stdout:phase 4.2 2024-05-30T16:10:22.196 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] Running 11 tests from 1 test suite. 2024-05-30T16:10:22.196 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment set-up. 2024-05-30T16:10:22.196 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 11 tests from LibRGW 2024-05-30T16:10:22.196 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.TVAR 2024-05-30T16:10:22.196 INFO:tasks.workunit.client.0.smithi006.stdout:read i1: 64001 2024-05-30T16:10:22.196 INFO:tasks.workunit.client.0.smithi006.stdout:read s1: blunderbuss 2024-05-30T16:10:22.196 INFO:tasks.workunit.client.0.smithi006.stdout:read s3: [ OK ] LibRGW.TVAR (0 ms) 2024-05-30T16:10:22.196 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.INIT 2024-05-30T16:10:22.206 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:10:22.206 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:10:22.525 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.INIT (329 ms) 2024-05-30T16:10:22.525 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MOUNT 2024-05-30T16:10:22.526 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MOUNT (1 ms) 2024-05-30T16:10:22.526 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-05-30T16:10:22.545 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (19 ms) 2024-05-30T16:10:22.545 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-05-30T16:10:22.545 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2024-05-30T16:10:22.545 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-05-30T16:10:22.545 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2024-05-30T16:10:22.545 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MARKER2_READDIR 2024-05-30T16:10:22.550 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_0 flags=8 2024-05-30T16:10:22.550 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_1 flags=8 2024-05-30T16:10:22.551 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_10 flags=8 2024-05-30T16:10:22.551 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_11 flags=8 2024-05-30T16:10:22.551 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_12 flags=8 2024-05-30T16:10:22.551 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_13 flags=8 2024-05-30T16:10:22.551 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_14 flags=8 2024-05-30T16:10:22.551 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_15 flags=8 2024-05-30T16:10:22.551 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_16 flags=8 2024-05-30T16:10:22.551 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_17 flags=8 2024-05-30T16:10:22.551 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_18 flags=8 2024-05-30T16:10:22.551 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_19 flags=8 2024-05-30T16:10:22.551 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_2 flags=8 2024-05-30T16:10:22.552 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_20 flags=8 2024-05-30T16:10:22.552 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_21 flags=8 2024-05-30T16:10:22.552 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_22 flags=8 2024-05-30T16:10:22.552 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_23 flags=8 2024-05-30T16:10:22.552 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_24 flags=8 2024-05-30T16:10:22.552 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_25 flags=8 2024-05-30T16:10:22.552 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_26 flags=8 2024-05-30T16:10:22.552 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_27 flags=8 2024-05-30T16:10:22.552 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_28 flags=8 2024-05-30T16:10:22.552 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_29 flags=8 2024-05-30T16:10:22.553 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_3 flags=8 2024-05-30T16:10:22.553 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_30 flags=8 2024-05-30T16:10:22.553 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_31 flags=8 2024-05-30T16:10:22.553 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_32 flags=8 2024-05-30T16:10:22.553 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_33 flags=8 2024-05-30T16:10:22.553 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_34 flags=8 2024-05-30T16:10:22.553 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_35 flags=8 2024-05-30T16:10:22.553 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_36 flags=8 2024-05-30T16:10:22.553 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_37 flags=8 2024-05-30T16:10:22.553 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_38 flags=8 2024-05-30T16:10:22.553 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_39 flags=8 2024-05-30T16:10:22.554 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_4 flags=8 2024-05-30T16:10:22.554 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_40 flags=8 2024-05-30T16:10:22.554 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_41 flags=8 2024-05-30T16:10:22.554 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_42 flags=8 2024-05-30T16:10:22.554 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_43 flags=8 2024-05-30T16:10:22.554 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_44 flags=8 2024-05-30T16:10:22.554 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_45 flags=8 2024-05-30T16:10:22.554 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_46 flags=8 2024-05-30T16:10:22.554 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_47 flags=8 2024-05-30T16:10:22.554 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_48 flags=8 2024-05-30T16:10:22.555 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_49 flags=8 2024-05-30T16:10:22.555 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_5 flags=8 2024-05-30T16:10:22.555 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_50 flags=8 2024-05-30T16:10:22.555 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_51 flags=8 2024-05-30T16:10:22.555 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_52 flags=8 2024-05-30T16:10:22.555 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_53 flags=8 2024-05-30T16:10:22.555 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_54 flags=8 2024-05-30T16:10:22.555 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_55 flags=8 2024-05-30T16:10:22.555 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_56 flags=8 2024-05-30T16:10:22.555 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_57 flags=8 2024-05-30T16:10:22.555 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_58 flags=8 2024-05-30T16:10:22.556 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_59 flags=8 2024-05-30T16:10:22.556 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_6 flags=8 2024-05-30T16:10:22.556 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_60 flags=8 2024-05-30T16:10:22.556 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_61 flags=8 2024-05-30T16:10:22.556 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_62 flags=8 2024-05-30T16:10:22.556 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_63 flags=8 2024-05-30T16:10:22.556 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_64 flags=8 2024-05-30T16:10:22.556 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_65 flags=8 2024-05-30T16:10:22.556 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_66 flags=8 2024-05-30T16:10:22.556 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_67 flags=8 2024-05-30T16:10:22.556 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_68 flags=8 2024-05-30T16:10:22.556 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_69 flags=8 2024-05-30T16:10:22.557 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_7 flags=8 2024-05-30T16:10:22.557 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_70 flags=8 2024-05-30T16:10:22.557 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_71 flags=8 2024-05-30T16:10:22.557 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_72 flags=8 2024-05-30T16:10:22.557 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_73 flags=8 2024-05-30T16:10:22.557 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_74 flags=8 2024-05-30T16:10:22.557 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_75 flags=8 2024-05-30T16:10:22.557 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_76 flags=8 2024-05-30T16:10:22.557 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_77 flags=8 2024-05-30T16:10:22.557 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_78 flags=8 2024-05-30T16:10:22.557 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_79 flags=8 2024-05-30T16:10:22.558 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_8 flags=8 2024-05-30T16:10:22.558 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_80 flags=8 2024-05-30T16:10:22.558 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_81 flags=8 2024-05-30T16:10:22.558 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_82 flags=8 2024-05-30T16:10:22.558 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_83 flags=8 2024-05-30T16:10:22.558 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_84 flags=8 2024-05-30T16:10:22.558 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_85 flags=8 2024-05-30T16:10:22.558 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_86 flags=8 2024-05-30T16:10:22.558 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_87 flags=8 2024-05-30T16:10:22.558 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_88 flags=8 2024-05-30T16:10:22.559 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_89 flags=8 2024-05-30T16:10:22.559 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_9 flags=8 2024-05-30T16:10:22.559 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_90 flags=8 2024-05-30T16:10:22.559 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_91 flags=8 2024-05-30T16:10:22.559 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_92 flags=8 2024-05-30T16:10:22.559 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_93 flags=8 2024-05-30T16:10:22.559 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_94 flags=8 2024-05-30T16:10:22.559 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_95 flags=8 2024-05-30T16:10:22.559 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_96 flags=8 2024-05-30T16:10:22.559 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_97 flags=8 2024-05-30T16:10:22.559 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_98 flags=8 2024-05-30T16:10:22.560 INFO:tasks.workunit.client.0.smithi006.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_99 flags=8 2024-05-30T16:10:22.560 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MARKER2_READDIR (5 ms) 2024-05-30T16:10:22.560 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-05-30T16:10:22.560 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-05-30T16:10:22.560 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.CLEANUP 2024-05-30T16:10:22.560 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-05-30T16:10:22.560 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.UMOUNT 2024-05-30T16:10:22.560 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-05-30T16:10:22.560 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-05-30T16:10:22.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:10:22.551+0000 7f89d8510a80 -1 shutting down 2024-05-30T16:10:22.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:10:22.551+0000 7f88b2262640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 60192, si_uid : 1000, si_addr0x3e80000eb20, si_status0 2024-05-30T16:10:52.814 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.SHUTDOWN (30263 ms) 2024-05-30T16:10:52.814 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 11 tests from LibRGW (30617 ms total) 2024-05-30T16:10:52.814 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-05-30T16:10:52.814 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment tear-down 2024-05-30T16:10:52.814 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] 11 tests from 1 test suite ran. (30617 ms total) 2024-05-30T16:10:52.814 INFO:tasks.workunit.client.0.smithi006.stdout:[ PASSED ] 11 tests. 2024-05-30T16:10:52.827 INFO:tasks.workunit.client.0.smithi006.stdout:phase 5.1 2024-05-30T16:10:52.862 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] Running 19 tests from 1 test suite. 2024-05-30T16:10:52.862 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment set-up. 2024-05-30T16:10:52.862 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 19 tests from LibRGW 2024-05-30T16:10:52.862 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.INIT 2024-05-30T16:10:52.874 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:10:52.874 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:10:53.163 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.INIT (301 ms) 2024-05-30T16:10:53.163 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MOUNT 2024-05-30T16:10:53.164 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MOUNT (1 ms) 2024-05-30T16:10:53.164 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-05-30T16:10:53.185 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.CREATE_BUCKET (21 ms) 2024-05-30T16:10:53.185 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2024-05-30T16:10:53.192 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (7 ms) 2024-05-30T16:10:53.192 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.LIST_OBJECTS 2024-05-30T16:10:53.192 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.LIST_OBJECTS (0 ms) 2024-05-30T16:10:53.192 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2024-05-30T16:10:53.196 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (5 ms) 2024-05-30T16:10:53.196 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.OBJ_OPEN 2024-05-30T16:10:53.196 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.OBJ_OPEN (0 ms) 2024-05-30T16:10:53.197 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.PUT_OBJECT 2024-05-30T16:10:53.202 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.PUT_OBJECT (5 ms) 2024-05-30T16:10:53.202 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.GET_OBJECT 2024-05-30T16:10:53.205 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.GET_OBJECT (3 ms) 2024-05-30T16:10:53.206 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.STAT_OBJECT 2024-05-30T16:10:53.206 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2024-05-30T16:10:53.206 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.WRITE_READ_VERIFY 2024-05-30T16:10:53.206 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.WRITE_READ_VERIFY (0 ms) 2024-05-30T16:10:53.206 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.WRITEV 2024-05-30T16:10:53.206 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.WRITEV (0 ms) 2024-05-30T16:10:53.206 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.READV 2024-05-30T16:10:53.206 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.READV (0 ms) 2024-05-30T16:10:53.206 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.READV_AFTER_WRITEV 2024-05-30T16:10:53.206 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.READV_AFTER_WRITEV (0 ms) 2024-05-30T16:10:53.206 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2024-05-30T16:10:53.206 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.DELETE_OBJECT (0 ms) 2024-05-30T16:10:53.207 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-05-30T16:10:53.207 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.DELETE_BUCKET (0 ms) 2024-05-30T16:10:53.207 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.CLEANUP 2024-05-30T16:10:53.207 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-05-30T16:10:53.207 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.UMOUNT 2024-05-30T16:10:53.207 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-05-30T16:10:53.207 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-05-30T16:10:53.207 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:10:53.206+0000 7fb660251a80 -1 shutting down 2024-05-30T16:10:53.208 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:10:53.206+0000 7fb539fa3640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 60838, si_uid : 1000, si_addr0x3e80000eda6, si_status0 2024-05-30T16:11:23.322 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.SHUTDOWN (30116 ms) 2024-05-30T16:11:23.322 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 19 tests from LibRGW (30459 ms total) 2024-05-30T16:11:23.323 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-05-30T16:11:23.323 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment tear-down 2024-05-30T16:11:23.323 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] 19 tests from 1 test suite ran. (30459 ms total) 2024-05-30T16:11:23.323 INFO:tasks.workunit.client.0.smithi006.stdout:[ PASSED ] 19 tests. 2024-05-30T16:11:23.336 INFO:tasks.workunit.client.0.smithi006.stdout:phase 5.2 2024-05-30T16:11:23.371 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] Running 19 tests from 1 test suite. 2024-05-30T16:11:23.371 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment set-up. 2024-05-30T16:11:23.371 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 19 tests from LibRGW 2024-05-30T16:11:23.371 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.INIT 2024-05-30T16:11:23.380 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:11:23.380 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:11:23.703 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.INIT (333 ms) 2024-05-30T16:11:23.704 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MOUNT 2024-05-30T16:11:23.705 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MOUNT (1 ms) 2024-05-30T16:11:23.705 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-05-30T16:11:23.705 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.CREATE_BUCKET (0 ms) 2024-05-30T16:11:23.705 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2024-05-30T16:11:23.717 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (12 ms) 2024-05-30T16:11:23.717 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.LIST_OBJECTS 2024-05-30T16:11:23.717 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.LIST_OBJECTS (0 ms) 2024-05-30T16:11:23.717 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2024-05-30T16:11:23.718 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (0 ms) 2024-05-30T16:11:23.718 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.OBJ_OPEN 2024-05-30T16:11:23.718 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.OBJ_OPEN (0 ms) 2024-05-30T16:11:23.718 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.PUT_OBJECT 2024-05-30T16:11:23.718 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.PUT_OBJECT (0 ms) 2024-05-30T16:11:23.718 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.GET_OBJECT 2024-05-30T16:11:23.718 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.GET_OBJECT (0 ms) 2024-05-30T16:11:23.718 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.STAT_OBJECT 2024-05-30T16:11:23.718 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2024-05-30T16:11:23.718 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.WRITE_READ_VERIFY 2024-05-30T16:11:23.718 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.WRITE_READ_VERIFY (0 ms) 2024-05-30T16:11:23.718 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.WRITEV 2024-05-30T16:11:23.719 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.WRITEV (0 ms) 2024-05-30T16:11:23.719 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.READV 2024-05-30T16:11:23.719 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.READV (0 ms) 2024-05-30T16:11:23.719 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.READV_AFTER_WRITEV 2024-05-30T16:11:23.719 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.READV_AFTER_WRITEV (0 ms) 2024-05-30T16:11:23.719 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2024-05-30T16:11:23.723 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.DELETE_OBJECT (6 ms) 2024-05-30T16:11:23.723 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-05-30T16:11:23.787 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.DELETE_BUCKET (64 ms) 2024-05-30T16:11:23.787 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.CLEANUP 2024-05-30T16:11:23.787 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-05-30T16:11:23.787 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.UMOUNT 2024-05-30T16:11:23.787 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-05-30T16:11:23.787 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-05-30T16:11:23.788 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:11:23.786+0000 7fe878c09a80 -1 shutting down 2024-05-30T16:11:23.790 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:11:23.790+0000 7fe75295b640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 61483, si_uid : 1000, si_addr0x3e80000f02b, si_status0 2024-05-30T16:11:54.007 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.SHUTDOWN (30219 ms) 2024-05-30T16:11:54.007 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 19 tests from LibRGW (30635 ms total) 2024-05-30T16:11:54.007 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-05-30T16:11:54.007 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment tear-down 2024-05-30T16:11:54.007 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] 19 tests from 1 test suite ran. (30635 ms total) 2024-05-30T16:11:54.007 INFO:tasks.workunit.client.0.smithi006.stdout:[ PASSED ] 19 tests. 2024-05-30T16:11:54.020 INFO:tasks.workunit.client.0.smithi006.stdout:phase 6.1 2024-05-30T16:11:54.045 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] Running 10 tests from 1 test suite. 2024-05-30T16:11:54.045 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment set-up. 2024-05-30T16:11:54.045 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 10 tests from LibRGW 2024-05-30T16:11:54.045 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.INIT 2024-05-30T16:11:54.054 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:11:54.054 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:11:54.626 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.INIT (581 ms) 2024-05-30T16:11:54.626 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.MOUNT 2024-05-30T16:11:54.627 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.MOUNT (2 ms) 2024-05-30T16:11:54.627 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.CREATE_BUCKETS 2024-05-30T16:11:54.667 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.CREATE_BUCKETS (39 ms) 2024-05-30T16:11:54.667 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.LOOKUP_BUCKETS 2024-05-30T16:11:54.682 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.LOOKUP_BUCKETS (16 ms) 2024-05-30T16:11:54.682 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.TOPDIR_RENAME 2024-05-30T16:11:54.717 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.TOPDIR_RENAME (34 ms) 2024-05-30T16:11:54.717 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.SUBDIR_RENAME 2024-05-30T16:11:54.761 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.SUBDIR_RENAME (44 ms) 2024-05-30T16:11:54.761 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.CROSS_BUCKET_RENAME 2024-05-30T16:11:54.804 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.CROSS_BUCKET_RENAME (43 ms) 2024-05-30T16:11:54.805 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.CLEANUP 2024-05-30T16:11:54.805 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-05-30T16:11:54.805 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.UMOUNT 2024-05-30T16:11:54.805 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-05-30T16:11:54.805 INFO:tasks.workunit.client.0.smithi006.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-05-30T16:11:54.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:11:54.802+0000 7f47c680ca80 -1 shutting down 2024-05-30T16:11:54.808 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:11:54.806+0000 7f46a055e640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 62128, si_uid : 1000, si_addr0x3e80000f2b0, si_status0 2024-05-30T16:12:24.767 INFO:tasks.workunit.client.0.smithi006.stdout:[ OK ] LibRGW.SHUTDOWN (29961 ms) 2024-05-30T16:12:24.767 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] 10 tests from LibRGW (30721 ms total) 2024-05-30T16:12:24.767 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-05-30T16:12:24.768 INFO:tasks.workunit.client.0.smithi006.stdout:[----------] Global test environment tear-down 2024-05-30T16:12:24.768 INFO:tasks.workunit.client.0.smithi006.stdout:[==========] 10 tests from 1 test suite ran. (30721 ms total) 2024-05-30T16:12:24.768 INFO:tasks.workunit.client.0.smithi006.stdout:[ PASSED ] 10 tests. 2024-05-30T16:12:24.780 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-30T16:12:24.780 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-30T16:12:24.797 INFO:tasks.workunit:Running workunits matching rgw/test_awssdkv4_sig.sh on client.0... 2024-05-30T16:12:24.798 INFO:tasks.workunit:Running workunit rgw/test_awssdkv4_sig.sh... 2024-05-30T16:12:24.798 DEBUG:teuthology.orchestra.run.smithi006:workunit test rgw/test_awssdkv4_sig.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_awssdkv4_sig.sh 2024-05-30T16:12:24.849 INFO:tasks.workunit.client.0.smithi006.stderr:+ [ -z ] 2024-05-30T16:12:24.850 INFO:tasks.workunit.client.0.smithi006.stderr:+ export AWS_ACCESS_KEY_ID=lNCnR47C2g+ZidCWBAUuwfSAA7Q= 2024-05-30T16:12:24.850 INFO:tasks.workunit.client.0.smithi006.stderr:+ export AWS_SECRET_ACCESS_KEY=tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g== 2024-05-30T16:12:24.850 INFO:tasks.workunit.client.0.smithi006.stderr:+ radosgw-admin user create --uid ceph-test-maven --access-key lNCnR47C2g+ZidCWBAUuwfSAA7Q= --secret tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g== --display-name maven test user --email sigv4@example.com 2024-05-30T16:12:24.892 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:12:24.892 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:12:24.938 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.937+0000 7f35b2d5c980 20 rados->read ofs=0 len=0 2024-05-30T16:12:24.940 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.937+0000 7f35b2d5c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:12:24.940 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.937+0000 7f35b2d5c980 20 realm 2024-05-30T16:12:24.941 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.937+0000 7f35b2d5c980 20 rados->read ofs=0 len=0 2024-05-30T16:12:24.941 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.937+0000 7f35b2d5c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:12:24.941 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.937+0000 7f35b2d5c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:12:24.941 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.937+0000 7f35b2d5c980 20 rados->read ofs=0 len=0 2024-05-30T16:12:24.941 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.937+0000 7f35b2d5c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:12:24.941 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.937+0000 7f35b2d5c980 20 rados->read ofs=0 len=0 2024-05-30T16:12:24.942 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.941+0000 7f35b2d5c980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:12:24.942 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.941+0000 7f35b2d5c980 20 rados->read ofs=0 len=0 2024-05-30T16:12:24.943 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.941+0000 7f35b2d5c980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:12:24.943 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.941+0000 7f35b2d5c980 20 searching for the correct realm 2024-05-30T16:12:24.949 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.945+0000 7f35b006c640 0 -- 172.21.15.6:0/1841866098 >> [v2:172.21.15.6:6819/1141406589,v1:172.21.15.6:6821/1141406589] conn(0x55f4eb6ca510 msgr2=0x55f4eb6cc9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:12:24.955 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.953+0000 7f35b086d640 0 -- 172.21.15.6:0/1841866098 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f35a4044930 msgr2=0x7f35a402f3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:12:24.964 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.961+0000 7f35b2d5c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:12:24.964 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.961+0000 7f35b2d5c980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:12:24.964 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.961+0000 7f35b2d5c980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:12:24.964 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.961+0000 7f35b2d5c980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:12:24.964 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.961+0000 7f35b2d5c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:12:24.964 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.961+0000 7f35b2d5c980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:12:24.964 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.961+0000 7f35b2d5c980 20 rados->read ofs=0 len=0 2024-05-30T16:12:24.964 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.961+0000 7f35b2d5c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:12:24.964 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.961+0000 7f35b2d5c980 20 rados->read ofs=0 len=0 2024-05-30T16:12:24.965 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.961+0000 7f35b2d5c980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:12:24.965 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.961+0000 7f35b2d5c980 20 rados->read ofs=0 len=0 2024-05-30T16:12:24.965 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.965+0000 7f35b2d5c980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:12:24.966 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.965+0000 7f35b2d5c980 20 zone default found 2024-05-30T16:12:24.966 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.965+0000 7f35b2d5c980 4 Realm: () 2024-05-30T16:12:24.966 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.965+0000 7f35b2d5c980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:12:24.966 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.965+0000 7f35b2d5c980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:12:24.966 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.965+0000 7f35b2d5c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:12:24.966 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.965+0000 7f35b2d5c980 20 zonegroup default 2024-05-30T16:12:24.966 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.965+0000 7f35b2d5c980 20 rados->read ofs=0 len=0 2024-05-30T16:12:24.966 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.965+0000 7f35b2d5c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:12:24.966 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.965+0000 7f35b2d5c980 20 rados->read ofs=0 len=0 2024-05-30T16:12:24.967 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.965+0000 7f35b2d5c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:12:24.967 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.965+0000 7f35b2d5c980 20 rados->read ofs=0 len=0 2024-05-30T16:12:24.967 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.965+0000 7f35b2d5c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:12:24.967 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.965+0000 7f35b2d5c980 20 started sync module instance, tier type = 2024-05-30T16:12:24.967 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.965+0000 7f35b2d5c980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:12:24.983 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.981+0000 7f35b2d5c980 20 add_watcher() i=0 2024-05-30T16:12:24.984 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.981+0000 7f35b2d5c980 20 add_watcher() i=1 2024-05-30T16:12:24.984 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.981+0000 7f35b2d5c980 20 add_watcher() i=2 2024-05-30T16:12:24.984 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.981+0000 7f35b2d5c980 20 add_watcher() i=3 2024-05-30T16:12:24.984 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.981+0000 7f35b2d5c980 20 add_watcher() i=4 2024-05-30T16:12:24.984 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.981+0000 7f35b2d5c980 20 add_watcher() i=5 2024-05-30T16:12:24.984 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.981+0000 7f35b2d5c980 20 add_watcher() i=6 2024-05-30T16:12:24.985 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.981+0000 7f35b2d5c980 20 add_watcher() i=7 2024-05-30T16:12:24.985 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.981+0000 7f35b2d5c980 2 all 8 watchers are set, enabling cache 2024-05-30T16:12:24.987 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.985+0000 7f3466fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:12:24.989 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.985+0000 7f35b2d5c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:12:24.989 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.985+0000 7f35b2d5c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:12:24.989 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.985+0000 7f35b2d5c980 5 note: GC not initialized 2024-05-30T16:12:24.989 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.985+0000 7f3465fe3640 20 reqs_thread_entry: start 2024-05-30T16:12:24.989 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.985+0000 7f35b2d5c980 20 init_complete bucket index max shards: 11 2024-05-30T16:12:24.989 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.985+0000 7f35b2d5c980 20 Filter name: none 2024-05-30T16:12:24.989 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.985+0000 7f33d77fe640 20 reqs_thread_entry: start 2024-05-30T16:12:24.989 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.985+0000 7f35b2d5c980 10 cache get: name=default.rgw.meta+users.uid+ceph-test-maven : miss 2024-05-30T16:12:24.989 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.985+0000 7f35b2d5c980 20 rados->read ofs=0 len=0 2024-05-30T16:12:24.990 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.989+0000 7f35b2d5c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:12:24.990 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.989+0000 7f35b2d5c980 10 cache put: name=default.rgw.meta+users.uid+ceph-test-maven info.flags=0x0 2024-05-30T16:12:24.990 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.989+0000 7f35b2d5c980 10 adding default.rgw.meta+users.uid+ceph-test-maven to cache LRU end 2024-05-30T16:12:24.991 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.989+0000 7f35b2d5c980 10 cache get: name=default.rgw.meta+users.email+sigv4@example.com : miss 2024-05-30T16:12:24.991 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.989+0000 7f35b2d5c980 20 rados->read ofs=0 len=0 2024-05-30T16:12:24.991 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.989+0000 7f35b2d5c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:12:24.991 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.989+0000 7f35b2d5c980 10 cache put: name=default.rgw.meta+users.email+sigv4@example.com info.flags=0x0 2024-05-30T16:12:24.991 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.989+0000 7f35b2d5c980 10 adding default.rgw.meta+users.email+sigv4@example.com to cache LRU end 2024-05-30T16:12:24.991 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.989+0000 7f35b2d5c980 10 cache get: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= : miss 2024-05-30T16:12:24.991 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.989+0000 7f35b2d5c980 20 rados->read ofs=0 len=0 2024-05-30T16:12:24.991 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.989+0000 7f35b2d5c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:12:24.991 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.989+0000 7f35b2d5c980 10 cache put: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= info.flags=0x0 2024-05-30T16:12:24.991 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.989+0000 7f35b2d5c980 10 adding default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= to cache LRU end 2024-05-30T16:12:24.991 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.989+0000 7f35b2d5c980 10 cache get: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= : hit (negative entry) 2024-05-30T16:12:24.991 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.989+0000 7f35b2d5c980 10 cache get: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= : hit (negative entry) 2024-05-30T16:12:24.992 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.989+0000 7f35b2d5c980 10 cache put: name=default.rgw.meta+users.uid+ceph-test-maven info.flags=0x17 2024-05-30T16:12:24.992 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.989+0000 7f35b2d5c980 10 moving default.rgw.meta+users.uid+ceph-test-maven to cache LRU end 2024-05-30T16:12:24.992 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.989+0000 7f35b2d5c980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:ceph-test-maven, ofs0, ns] 2024-05-30T16:12:24.992 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.989+0000 7f3487fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1078036791304 cookie 94510410173424 notifier 5724 bl.length()=677 2024-05-30T16:12:24.992 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.989+0000 7f3487fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ceph-test-maven info.flags=0x17 2024-05-30T16:12:24.992 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.989+0000 7f3487fff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ceph-test-maven to cache LRU end 2024-05-30T16:12:24.996 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.993+0000 7f35b2d5c980 10 cache put: name=default.rgw.meta+users.email+sigv4@example.com info.flags=0x7 2024-05-30T16:12:24.996 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.993+0000 7f35b2d5c980 10 moving default.rgw.meta+users.email+sigv4@example.com to cache LRU end 2024-05-30T16:12:24.996 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.993+0000 7f35b2d5c980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.email:sigv4@example.com, ofs0, ns] 2024-05-30T16:12:24.997 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.993+0000 7f34877fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1078036791305 cookie 94510410152912 notifier 5724 bl.length()=187 2024-05-30T16:12:24.997 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.993+0000 7f34877fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+sigv4@example.com info.flags=0x7 2024-05-30T16:12:24.997 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.993+0000 7f34877fe640 10 rgw watcher librados: moving default.rgw.meta+users.email+sigv4@example.com to cache LRU end 2024-05-30T16:12:25.001 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.997+0000 7f35b2d5c980 10 cache put: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= info.flags=0x7 2024-05-30T16:12:25.001 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.997+0000 7f35b2d5c980 10 moving default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= to cache LRU end 2024-05-30T16:12:25.001 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:24.997+0000 7f35b2d5c980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:lNCnR47C2g+ZidCWBAUuwfSAA7Q=, ofs0, ns] 2024-05-30T16:12:25.001 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:25.001+0000 7f34877fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1078036791306 cookie 94510410152912 notifier 5724 bl.length()=197 2024-05-30T16:12:25.001 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:25.001+0000 7f34877fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= info.flags=0x7 2024-05-30T16:12:25.002 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:25.001+0000 7f34877fe640 10 rgw watcher librados: moving default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= to cache LRU end 2024-05-30T16:12:25.003 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:12:25.003 INFO:tasks.workunit.client.0.smithi006.stdout: "user_id": "ceph-test-maven", 2024-05-30T16:12:25.003 INFO:tasks.workunit.client.0.smithi006.stdout: "display_name": "maven test user", 2024-05-30T16:12:25.004 INFO:tasks.workunit.client.0.smithi006.stdout: "email": "sigv4@example.com", 2024-05-30T16:12:25.004 INFO:tasks.workunit.client.0.smithi006.stdout: "suspended": 0, 2024-05-30T16:12:25.004 INFO:tasks.workunit.client.0.smithi006.stdout: "max_buckets": 1000, 2024-05-30T16:12:25.004 INFO:tasks.workunit.client.0.smithi006.stdout: "subusers": [], 2024-05-30T16:12:25.004 INFO:tasks.workunit.client.0.smithi006.stdout: "keys": [ 2024-05-30T16:12:25.004 INFO:tasks.workunit.client.0.smithi006.stdout: { 2024-05-30T16:12:25.004 INFO:tasks.workunit.client.0.smithi006.stdout: "user": "ceph-test-maven", 2024-05-30T16:12:25.004 INFO:tasks.workunit.client.0.smithi006.stdout: "access_key": "lNCnR47C2g+ZidCWBAUuwfSAA7Q=", 2024-05-30T16:12:25.004 INFO:tasks.workunit.client.0.smithi006.stdout: "secret_key": "tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g==", 2024-05-30T16:12:25.004 INFO:tasks.workunit.client.0.smithi006.stdout: "active": true, 2024-05-30T16:12:25.004 INFO:tasks.workunit.client.0.smithi006.stdout: "create_date": "2024-05-30T16:12:24.993493Z" 2024-05-30T16:12:25.004 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:12:25.004 INFO:tasks.workunit.client.0.smithi006.stdout: ], 2024-05-30T16:12:25.004 INFO:tasks.workunit.client.0.smithi006.stdout: "swift_keys": [], 2024-05-30T16:12:25.005 INFO:tasks.workunit.client.0.smithi006.stdout: "caps": [], 2024-05-30T16:12:25.005 INFO:tasks.workunit.client.0.smithi006.stdout: "op_mask": "read, write, delete", 2024-05-30T16:12:25.005 INFO:tasks.workunit.client.0.smithi006.stdout: "default_placement": "", 2024-05-30T16:12:25.005 INFO:tasks.workunit.client.0.smithi006.stdout: "default_storage_class": "", 2024-05-30T16:12:25.005 INFO:tasks.workunit.client.0.smithi006.stdout: "placement_tags": [], 2024-05-30T16:12:25.005 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_quota": { 2024-05-30T16:12:25.005 INFO:tasks.workunit.client.0.smithi006.stdout: "enabled": false, 2024-05-30T16:12:25.005 INFO:tasks.workunit.client.0.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:12:25.005 INFO:tasks.workunit.client.0.smithi006.stdout: "max_size": -1, 2024-05-30T16:12:25.005 INFO:tasks.workunit.client.0.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:12:25.005 INFO:tasks.workunit.client.0.smithi006.stdout: "max_objects": -1 2024-05-30T16:12:25.005 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:12:25.005 INFO:tasks.workunit.client.0.smithi006.stdout: "user_quota": { 2024-05-30T16:12:25.005 INFO:tasks.workunit.client.0.smithi006.stdout: "enabled": false, 2024-05-30T16:12:25.006 INFO:tasks.workunit.client.0.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:12:25.006 INFO:tasks.workunit.client.0.smithi006.stdout: "max_size": -1, 2024-05-30T16:12:25.006 INFO:tasks.workunit.client.0.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:12:25.006 INFO:tasks.workunit.client.0.smithi006.stdout: "max_objects": -1 2024-05-30T16:12:25.006 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:12:25.006 INFO:tasks.workunit.client.0.smithi006.stdout: "temp_url_keys": [], 2024-05-30T16:12:25.006 INFO:tasks.workunit.client.0.smithi006.stdout: "type": "rgw", 2024-05-30T16:12:25.006 INFO:tasks.workunit.client.0.smithi006.stdout: "mfa_ids": [], 2024-05-30T16:12:25.006 INFO:tasks.workunit.client.0.smithi006.stdout: "account_id": "", 2024-05-30T16:12:25.006 INFO:tasks.workunit.client.0.smithi006.stdout: "path": "/", 2024-05-30T16:12:25.006 INFO:tasks.workunit.client.0.smithi006.stdout: "create_date": "2024-05-30T16:12:24.993470Z", 2024-05-30T16:12:25.006 INFO:tasks.workunit.client.0.smithi006.stdout: "tags": [], 2024-05-30T16:12:25.006 INFO:tasks.workunit.client.0.smithi006.stdout: "group_ids": [] 2024-05-30T16:12:25.006 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:12:25.007 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-05-30T16:12:25.007 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:25.001+0000 7f3466fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:12:25.009 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:25.009+0000 7f35b2d5c980 20 remove_watcher() i=0 2024-05-30T16:12:25.009 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:25.009+0000 7f35b2d5c980 2 removed watcher, disabling cache 2024-05-30T16:12:25.011 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:25.009+0000 7f35b2d5c980 20 remove_watcher() i=1 2024-05-30T16:12:25.013 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:25.013+0000 7f35b2d5c980 20 remove_watcher() i=2 2024-05-30T16:12:25.015 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:25.013+0000 7f35b2d5c980 20 remove_watcher() i=3 2024-05-30T16:12:25.018 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:25.017+0000 7f35b2d5c980 20 remove_watcher() i=4 2024-05-30T16:12:25.019 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:25.017+0000 7f35b2d5c980 20 remove_watcher() i=5 2024-05-30T16:12:25.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:25.017+0000 7f35b2d5c980 20 remove_watcher() i=6 2024-05-30T16:12:25.022 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:12:25.021+0000 7f35b2d5c980 20 remove_watcher() i=7 2024-05-30T16:12:25.037 INFO:tasks.workunit.client.0.smithi006.stderr:+ [ -z ] 2024-05-30T16:12:25.037 INFO:tasks.workunit.client.0.smithi006.stderr:+ [ -z /home/ubuntu/cephtest ] 2024-05-30T16:12:25.037 INFO:tasks.workunit.client.0.smithi006.stderr:+ cat /home/ubuntu/cephtest/url_file 2024-05-30T16:12:25.037 INFO:tasks.workunit.client.0.smithi006.stderr:+ export RGW_HTTP_ENDPOINT_URL=http://smithi006.front.sepia.ceph.com:80 2024-05-30T16:12:25.037 INFO:tasks.workunit.client.0.smithi006.stderr:+ [ -z /home/ubuntu/cephtest/clone.client.0 ] 2024-05-30T16:12:25.038 INFO:tasks.workunit.client.0.smithi006.stderr:+ cd /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum 2024-05-30T16:12:25.038 INFO:tasks.workunit.client.0.smithi006.stderr:+ ./mvnw clean package 2024-05-30T16:12:25.043 INFO:tasks.workunit.client.0.smithi006.stdout:Warning: JAVA_HOME environment variable is not set. 2024-05-30T16:12:29.263 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] Scanning for projects... 2024-05-30T16:12:29.423 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.1/junit-bom-5.10.1.pom 2024-05-30T16:12:29.585 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 1.4/5.6 kB Progress (1): 2.8/5.6 kB Progress (1): 4.1/5.6 kB Progress (1): 5.5/5.6 kB Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.1/junit-bom-5.10.1.pom (5.6 kB at 34 kB/s) 2024-05-30T16:12:29.622 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom/2.20.43/bom-2.20.43.pom 2024-05-30T16:12:29.725 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 1.4/83 kB Progress (1): 2.8/83 kB Progress (1): 4.1/83 kB Progress (1): 5.5/83 kB Progress (1): 6.9/83 kB Progress (1): 8.3/83 kB Progress (1): 9.7/83 kB Progress (1): 11/83 kB Progress (1): 12/83 kB Progress (1): 14/83 kB Progress (1): 15/83 kB Progress (1): 16/83 kB Progress (1): 33/83 kB Progress (1): 49/83 kB Progress (1): 66/83 kB Progress (1): 82/83 kB Progress (1): 83 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom/2.20.43/bom-2.20.43.pom (83 kB at 805 kB/s) 2024-05-30T16:12:29.744 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-sdk-java-pom/2.20.43/aws-sdk-java-pom-2.20.43.pom 2024-05-30T16:12:29.839 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/77 kB Progress (1): 33/77 kB Progress (1): 49/77 kB Progress (1): 64/77 kB Progress (1): 77 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-sdk-java-pom/2.20.43/aws-sdk-java-pom-2.20.43.pom (77 kB at 808 kB/s) 2024-05-30T16:12:29.904 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] 2024-05-30T16:12:29.905 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] ---------------------------< io.ceph:jcksum >--------------------------- 2024-05-30T16:12:29.905 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] Building jcksum 1.0 2024-05-30T16:12:29.905 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] from pom.xml 2024-05-30T16:12:29.905 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] --------------------------------[ jar ]--------------------------------- 2024-05-30T16:12:29.911 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.pom 2024-05-30T16:12:29.964 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 5.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.pom (5.3 kB at 98 kB/s) 2024-05-30T16:12:29.979 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/35/maven-plugins-35.pom 2024-05-30T16:12:30.018 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 9.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/35/maven-plugins-35.pom (9.9 kB at 254 kB/s) 2024-05-30T16:12:30.026 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/35/maven-parent-35.pom 2024-05-30T16:12:30.070 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/45 kB Progress (1): 33/45 kB Progress (1): 45 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/35/maven-parent-35.pom (45 kB at 1.0 MB/s) 2024-05-30T16:12:30.083 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/25/apache-25.pom 2024-05-30T16:12:30.115 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/21 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/25/apache-25.pom (21 kB at 642 kB/s) 2024-05-30T16:12:30.144 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.jar 2024-05-30T16:12:30.178 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/36 kB Progress (1): 32/36 kB Progress (1): 36 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.jar (36 kB at 1.0 MB/s) 2024-05-30T16:12:30.196 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.pom 2024-05-30T16:12:30.225 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 8.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.pom (8.2 kB at 272 kB/s) 2024-05-30T16:12:30.232 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/39/maven-plugins-39.pom 2024-05-30T16:12:30.254 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 8.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/39/maven-plugins-39.pom (8.1 kB at 368 kB/s) 2024-05-30T16:12:30.260 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/39/maven-parent-39.pom 2024-05-30T16:12:30.290 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/48 kB Progress (1): 33/48 kB Progress (1): 48 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/39/maven-parent-39.pom (48 kB at 1.5 MB/s) 2024-05-30T16:12:30.301 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/29/apache-29.pom 2024-05-30T16:12:30.324 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/21 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/29/apache-29.pom (21 kB at 901 kB/s) 2024-05-30T16:12:30.337 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.jar 2024-05-30T16:12:30.363 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/31 kB Progress (1): 31 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.jar (31 kB at 1.1 MB/s) 2024-05-30T16:12:30.378 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.pom 2024-05-30T16:12:30.409 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 9.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.pom (9.8 kB at 317 kB/s) 2024-05-30T16:12:30.425 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.jar 2024-05-30T16:12:30.460 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/66 kB Progress (1): 33/66 kB Progress (1): 49/66 kB Progress (1): 66/66 kB Progress (1): 66 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.jar (66 kB at 1.9 MB/s) 2024-05-30T16:12:30.473 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.pom 2024-05-30T16:12:30.502 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 5.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.pom (5.3 kB at 183 kB/s) 2024-05-30T16:12:30.506 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire/3.2.3/surefire-3.2.3.pom 2024-05-30T16:12:30.534 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/22 kB Progress (1): 19/22 kB Progress (1): 22 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire/3.2.3/surefire-3.2.3.pom (22 kB at 756 kB/s) 2024-05-30T16:12:30.541 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/41/maven-parent-41.pom 2024-05-30T16:12:30.570 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/50 kB Progress (1): 33/50 kB Progress (1): 49/50 kB Progress (1): 50 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/41/maven-parent-41.pom (50 kB at 1.7 MB/s) 2024-05-30T16:12:30.583 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/31/apache-31.pom 2024-05-30T16:12:30.606 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/24 kB Progress (1): 24 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/31/apache-31.pom (24 kB at 980 kB/s) 2024-05-30T16:12:30.621 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.3/junit-bom-5.9.3.pom 2024-05-30T16:12:30.642 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.3/junit-bom-5.9.3.pom (5.6 kB at 268 kB/s) 2024-05-30T16:12:30.649 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.jar 2024-05-30T16:12:30.683 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/45 kB Progress (1): 32/45 kB Progress (1): 45 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.jar (45 kB at 1.3 MB/s) 2024-05-30T16:12:30.700 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.pom 2024-05-30T16:12:30.725 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 6.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.pom (6.8 kB at 260 kB/s) 2024-05-30T16:12:30.732 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/37/maven-plugins-37.pom 2024-05-30T16:12:30.756 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 9.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/37/maven-plugins-37.pom (9.9 kB at 413 kB/s) 2024-05-30T16:12:30.762 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/37/maven-parent-37.pom 2024-05-30T16:12:30.791 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/46 kB Progress (1): 33/46 kB Progress (1): 46 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/37/maven-parent-37.pom (46 kB at 1.6 MB/s) 2024-05-30T16:12:30.803 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/27/apache-27.pom 2024-05-30T16:12:30.832 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/27/apache-27.pom (20 kB at 702 kB/s) 2024-05-30T16:12:30.848 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.jar 2024-05-30T16:12:30.876 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/27 kB Progress (1): 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.jar (27 kB at 976 kB/s) 2024-05-30T16:12:30.915 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.pom 2024-05-30T16:12:30.937 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 6.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.pom (6.1 kB at 277 kB/s) 2024-05-30T16:12:30.942 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/services/2.20.43/services-2.20.43.pom 2024-05-30T16:12:30.967 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/21 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/services/2.20.43/services-2.20.43.pom (21 kB at 853 kB/s) 2024-05-30T16:12:30.977 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom-internal/2.20.43/bom-internal-2.20.43.pom 2024-05-30T16:12:31.003 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom-internal/2.20.43/bom-internal-2.20.43.pom (20 kB at 770 kB/s) 2024-05-30T16:12:31.018 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.pom 2024-05-30T16:12:31.039 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 4.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.pom (4.0 kB at 191 kB/s) 2024-05-30T16:12:31.044 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocols/2.20.43/protocols-2.20.43.pom 2024-05-30T16:12:31.064 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 1.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocols/2.20.43/protocols-2.20.43.pom (1.6 kB at 78 kB/s) 2024-05-30T16:12:31.068 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/core/2.20.43/core-2.20.43.pom 2024-05-30T16:12:31.090 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/core/2.20.43/core-2.20.43.pom (2.2 kB at 101 kB/s) 2024-05-30T16:12:31.099 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.pom 2024-05-30T16:12:31.119 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 3.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.pom (3.6 kB at 182 kB/s) 2024-05-30T16:12:31.127 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.pom 2024-05-30T16:12:31.147 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.pom (3.2 kB at 161 kB/s) 2024-05-30T16:12:31.156 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.pom 2024-05-30T16:12:31.176 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 11 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.pom (11 kB at 522 kB/s) 2024-05-30T16:12:31.185 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.pom 2024-05-30T16:12:31.205 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 1.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.pom (1.7 kB at 84 kB/s) 2024-05-30T16:12:31.212 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.pom 2024-05-30T16:12:31.233 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 3.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.pom (3.8 kB at 182 kB/s) 2024-05-30T16:12:31.240 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.pom 2024-05-30T16:12:31.264 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 4.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.pom (4.9 kB at 214 kB/s) 2024-05-30T16:12:31.272 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.pom 2024-05-30T16:12:31.294 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 1.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.pom (1.2 kB at 53 kB/s) 2024-05-30T16:12:31.298 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.30/slf4j-api-1.7.30.pom 2024-05-30T16:12:31.321 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 3.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.30/slf4j-api-1.7.30.pom (3.8 kB at 167 kB/s) 2024-05-30T16:12:31.324 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.30/slf4j-parent-1.7.30.pom 2024-05-30T16:12:31.345 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.30/slf4j-parent-1.7.30.pom (14 kB at 628 kB/s) 2024-05-30T16:12:31.352 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.pom 2024-05-30T16:12:31.372 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 3.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.pom (3.7 kB at 184 kB/s) 2024-05-30T16:12:31.380 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.pom 2024-05-30T16:12:31.401 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 2.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.pom (2.5 kB at 117 kB/s) 2024-05-30T16:12:31.409 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.pom 2024-05-30T16:12:31.432 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 2.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.pom (2.9 kB at 127 kB/s) 2024-05-30T16:12:31.440 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.pom 2024-05-30T16:12:31.461 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 7.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.pom (7.4 kB at 370 kB/s) 2024-05-30T16:12:31.469 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.pom 2024-05-30T16:12:31.492 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 5.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.pom (5.5 kB at 237 kB/s) 2024-05-30T16:12:31.501 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.pom 2024-05-30T16:12:31.520 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.pom (3.5 kB at 184 kB/s) 2024-05-30T16:12:31.529 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.pom 2024-05-30T16:12:31.549 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 4.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.pom (4.4 kB at 220 kB/s) 2024-05-30T16:12:31.553 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party/2.20.43/third-party-2.20.43.pom 2024-05-30T16:12:31.573 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 2.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party/2.20.43/third-party-2.20.43.pom (2.4 kB at 115 kB/s) 2024-05-30T16:12:31.581 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.pom 2024-05-30T16:12:31.607 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.pom (6.4 kB at 238 kB/s) 2024-05-30T16:12:31.616 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.pom 2024-05-30T16:12:31.641 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 6.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.pom (6.1 kB at 243 kB/s) 2024-05-30T16:12:31.649 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.pom 2024-05-30T16:12:31.674 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.pom (3.0 kB at 119 kB/s) 2024-05-30T16:12:31.682 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.pom 2024-05-30T16:12:31.708 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.pom (3.5 kB at 134 kB/s) 2024-05-30T16:12:31.717 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.pom 2024-05-30T16:12:31.742 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 4.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.pom (4.1 kB at 163 kB/s) 2024-05-30T16:12:31.746 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-clients/2.20.43/http-clients-2.20.43.pom 2024-05-30T16:12:31.771 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 1.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-clients/2.20.43/http-clients-2.20.43.pom (1.8 kB at 71 kB/s) 2024-05-30T16:12:31.779 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.pom 2024-05-30T16:12:31.805 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 6.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.pom (6.6 kB at 264 kB/s) 2024-05-30T16:12:31.809 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.5.13/httpcomponents-client-4.5.13.pom 2024-05-30T16:12:31.836 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.5.13/httpcomponents-client-4.5.13.pom (16 kB at 607 kB/s) 2024-05-30T16:12:31.841 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-parent/11/httpcomponents-parent-11.pom 2024-05-30T16:12:31.869 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/35 kB Progress (1): 32/35 kB Progress (1): 35 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-parent/11/httpcomponents-parent-11.pom (35 kB at 1.2 MB/s) 2024-05-30T16:12:31.876 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/21/apache-21.pom 2024-05-30T16:12:31.899 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/21/apache-21.pom (17 kB at 744 kB/s) 2024-05-30T16:12:31.907 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.pom 2024-05-30T16:12:31.929 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 5.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.pom (5.0 kB at 226 kB/s) 2024-05-30T16:12:31.934 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-core/4.4.13/httpcomponents-core-4.4.13.pom 2024-05-30T16:12:31.956 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-core/4.4.13/httpcomponents-core-4.4.13.pom (13 kB at 598 kB/s) 2024-05-30T16:12:31.964 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.11/commons-codec-1.11.pom 2024-05-30T16:12:31.987 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.11/commons-codec-1.11.pom (14 kB at 607 kB/s) 2024-05-30T16:12:31.991 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/42/commons-parent-42.pom 2024-05-30T16:12:32.025 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/68 kB Progress (1): 32/68 kB Progress (1): 49/68 kB Progress (1): 65/68 kB Progress (1): 68 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/42/commons-parent-42.pom (68 kB at 2.0 MB/s) 2024-05-30T16:12:32.047 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/18/apache-18.pom 2024-05-30T16:12:32.067 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/18/apache-18.pom (16 kB at 783 kB/s) 2024-05-30T16:12:32.077 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.pom 2024-05-30T16:12:32.098 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.pom (15 kB at 735 kB/s) 2024-05-30T16:12:32.103 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/52/commons-parent-52.pom 2024-05-30T16:12:32.137 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/79 kB Progress (1): 33/79 kB Progress (1): 49/79 kB Progress (1): 66/79 kB Progress (1): 79 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/52/commons-parent-52.pom (79 kB at 2.3 MB/s) 2024-05-30T16:12:32.146 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/23/apache-23.pom 2024-05-30T16:12:32.168 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/18 kB Progress (1): 18 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/23/apache-23.pom (18 kB at 877 kB/s) 2024-05-30T16:12:32.176 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.pom 2024-05-30T16:12:32.197 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.pom (20 kB at 908 kB/s) 2024-05-30T16:12:32.203 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/65/commons-parent-65.pom 2024-05-30T16:12:32.237 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/78 kB Progress (1): 32/78 kB Progress (1): 49/78 kB Progress (1): 65/78 kB Progress (1): 78 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/65/commons-parent-65.pom (78 kB at 2.2 MB/s) 2024-05-30T16:12:32.249 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.pom 2024-05-30T16:12:32.269 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 3.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.pom (3.8 kB at 192 kB/s) 2024-05-30T16:12:32.273 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.28/slf4j-parent-1.7.28.pom 2024-05-30T16:12:32.294 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.28/slf4j-parent-1.7.28.pom (14 kB at 690 kB/s) 2024-05-30T16:12:32.300 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.pom 2024-05-30T16:12:32.319 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.pom (3.2 kB at 169 kB/s) 2024-05-30T16:12:32.323 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.pom 2024-05-30T16:12:32.343 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.pom (3.2 kB at 168 kB/s) 2024-05-30T16:12:32.347 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.pom 2024-05-30T16:12:32.366 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 2.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.pom (2.0 kB at 107 kB/s) 2024-05-30T16:12:32.370 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.pom 2024-05-30T16:12:32.390 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 2.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.pom (2.8 kB at 142 kB/s) 2024-05-30T16:12:32.394 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.pom 2024-05-30T16:12:32.414 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 1.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.pom (1.5 kB at 76 kB/s) 2024-05-30T16:12:32.418 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.pom 2024-05-30T16:12:32.438 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.pom (3.0 kB at 158 kB/s) 2024-05-30T16:12:32.443 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.pom 2024-05-30T16:12:32.462 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.pom (3.2 kB at 160 kB/s) 2024-05-30T16:12:32.467 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.pom 2024-05-30T16:12:32.486 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.pom (3.2 kB at 160 kB/s) 2024-05-30T16:12:32.492 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.pom 2024-05-30T16:12:32.511 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 1.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.pom (1.0 kB at 51 kB/s) 2024-05-30T16:12:32.519 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.pom 2024-05-30T16:12:32.538 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 964 B Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.pom (964 B at 51 kB/s) 2024-05-30T16:12:32.544 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.pom 2024-05-30T16:12:32.563 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 3.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.pom (3.1 kB at 161 kB/s) 2024-05-30T16:12:32.568 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.pom 2024-05-30T16:12:32.587 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.pom (3.0 kB at 152 kB/s) 2024-05-30T16:12:32.593 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.pom 2024-05-30T16:12:32.614 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 3.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.pom (3.4 kB at 164 kB/s) 2024-05-30T16:12:32.619 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.pom 2024-05-30T16:12:32.639 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 3.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.pom (3.4 kB at 181 kB/s) 2024-05-30T16:12:32.644 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.pom 2024-05-30T16:12:32.664 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.pom (3.0 kB at 144 kB/s) 2024-05-30T16:12:32.717 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.jar 2024-05-30T16:12:33.329 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 0/3.3 MB Progress (1): 0/3.3 MB Progress (1): 0/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.3/3.3 MB Progress (1): 3.3/3.3 MB Progress (1): 3.3 MB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.jar (3.3 MB at 5.4 MB/s) 2024-05-30T16:12:33.332 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.jar 2024-05-30T16:12:33.333 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.jar 2024-05-30T16:12:33.333 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.jar 2024-05-30T16:12:33.334 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.jar 2024-05-30T16:12:33.338 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.jar 2024-05-30T16:12:33.378 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/71 kB Progress (1): 33/71 kB Progress (1): 49/71 kB Progress (1): 66/71 kB Progress (1): 71 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.jar (71 kB at 1.5 MB/s) 2024-05-30T16:12:33.378 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.jar 2024-05-30T16:12:33.419 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.jar (16 kB at 189 kB/s) 2024-05-30T16:12:33.420 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.jar 2024-05-30T16:12:33.450 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 1.3/35 kB Progress (1): 2.7/35 kB Progress (1): 4.0/35 kB Progress (2): 4.0/35 kB | 1.4/98 kB Progress (2): 5.4/35 kB | 1.4/98 kB Progress (2): 5.4/35 kB | 2.8/98 kB Progress (2): 6.8/35 kB | 2.8/98 kB Progress (2): 6.8/35 kB | 4.1/98 kB Progress (2): 8.2/35 kB | 4.1/98 kB Progress (2): 8.2/35 kB | 5.5/98 kB Progress (2): 9.6/35 kB | 5.5/98 kB Progress (2): 9.6/35 kB | 6.9/98 kB Progress (2): 11/35 kB | 6.9/98 kB Progress (2): 11/35 kB | 8.3/98 kB Progress (2): 12/35 kB | 8.3/98 kB Progress (2): 12/35 kB | 9.7/98 kB Progress (2): 14/35 kB | 9.7/98 kB Progress (2): 14/35 kB | 11/98 kB Progress (2): 15/35 kB | 11/98 kB Progress (2): 15/35 kB | 12/98 kB Progress (2): 16/35 kB | 12/98 kB Progress (2): 16/35 kB | 14/98 kB Progress (2): 18/35 kB | 14/98 kB Progress (2): 18/35 kB | 15/98 kB Progress (2): 19/35 kB | 15/98 kB Progress (2): 19/35 kB | 16/98 kB Progress (2): 20/35 kB | 16/98 kB Progress (2): 20/35 kB | 18/98 kB Progress (2): 22/35 kB | 18/98 kB Progress (2): 22/35 kB | 19/98 kB Progress (2): 23/35 kB | 19/98 kB Progress (2): 23/35 kB | 21/98 kB Progress (2): 25/35 kB | 21/98 kB Progress (2): 25/35 kB | 22/98 kB Progress (2): 26/35 kB | 22/98 kB Progress (2): 26/35 kB | 23/98 kB Progress (2): 27/35 kB | 23/98 kB Progress (2): 27/35 kB | 25/98 kB Progress (2): 29/35 kB | 25/98 kB Progress (2): 29/35 kB | 26/98 kB Progress (3): 29/35 kB | 26/98 kB | 16/782 kB Progress (3): 30/35 kB | 26/98 kB | 16/782 kB Progress (3): 30/35 kB | 27/98 kB | 16/782 kB Progress (3): 31/35 kB | 27/98 kB | 16/782 kB Progress (3): 31/35 kB | 29/98 kB | 16/782 kB Progress (3): 33/35 kB | 29/98 kB | 16/782 kB Progress (3): 33/35 kB | 30/98 kB | 16/782 kB Progress (3): 34/35 kB | 30/98 kB | 16/782 kB Progress (3): 34/35 kB | 32/98 kB | 16/782 kB Progress (3): 35 kB | 32/98 kB | 16/782 kB Progress (3): 35 kB | 32/98 kB | 33/782 kB Progress (3): 35 kB | 33/98 kB | 33/782 kB Progress (3): 35 kB | 34/98 kB | 33/782 kB Progress (3): 35 kB | 36/98 kB | 33/782 kB Progress (3): 35 kB | 37/98 kB | 33/782 kB Progress (3): 35 kB | 37/98 kB | 49/782 kB Progress (3): 35 kB | 38/98 kB | 49/782 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.jar (35 kB at 298 kB/s) 2024-05-30T16:12:33.450 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.jar 2024-05-30T16:12:33.461 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (2): 38/98 kB | 63/782 kB Progress (2): 38/98 kB | 79/782 kB Progress (2): 38/98 kB | 96/782 kB Progress (2): 38/98 kB | 112/782 kB Progress (2): 38/98 kB | 128/782 kB Progress (3): 38/98 kB | 128/782 kB | 1.4/49 kB Progress (4): 38/98 kB | 128/782 kB | 1.4/49 kB | 1.4/16 kB Progress (4): 38/98 kB | 128/782 kB | 2.8/49 kB | 1.4/16 kB Progress (4): 38/98 kB | 128/782 kB | 2.8/49 kB | 2.8/16 kB Progress (4): 38/98 kB | 128/782 kB | 2.8/49 kB | 4.1/16 kB Progress (4): 38/98 kB | 128/782 kB | 4.1/49 kB | 4.1/16 kB Progress (4): 38/98 kB | 128/782 kB | 4.1/49 kB | 5.5/16 kB Progress (4): 38/98 kB | 128/782 kB | 4.1/49 kB | 6.9/16 kB Progress (4): 38/98 kB | 128/782 kB | 5.5/49 kB | 6.9/16 kB Progress (4): 38/98 kB | 128/782 kB | 5.5/49 kB | 8.3/16 kB Progress (4): 38/98 kB | 128/782 kB | 6.9/49 kB | 8.3/16 kB Progress (4): 38/98 kB | 128/782 kB | 6.9/49 kB | 9.7/16 kB Progress (4): 38/98 kB | 128/782 kB | 6.9/49 kB | 11/16 kB Progress (4): 38/98 kB | 128/782 kB | 8.3/49 kB | 11/16 kB Progress (4): 38/98 kB | 128/782 kB | 8.3/49 kB | 12/16 kB Progress (4): 38/98 kB | 145/782 kB | 8.3/49 kB | 12/16 kB Progress (4): 38/98 kB | 145/782 kB | 9.7/49 kB | 12/16 kB Progress (4): 38/98 kB | 145/782 kB | 9.7/49 kB | 14/16 kB Progress (4): 38/98 kB | 145/782 kB | 9.7/49 kB | 15/16 kB Progress (4): 38/98 kB | 145/782 kB | 11/49 kB | 15/16 kB Progress (4): 38/98 kB | 145/782 kB | 11/49 kB | 16 kB Progress (4): 38/98 kB | 145/782 kB | 12/49 kB | 16 kB Progress (4): 38/98 kB | 145/782 kB | 14/49 kB | 16 kB Progress (4): 40/98 kB | 145/782 kB | 14/49 kB | 16 kB Progress (4): 40/98 kB | 145/782 kB | 15/49 kB | 16 kB Progress (4): 41/98 kB | 145/782 kB | 15/49 kB | 16 kB Progress (4): 41/98 kB | 161/782 kB | 15/49 kB | 16 kB Progress (4): 43/98 kB | 161/782 kB | 15/49 kB | 16 kB Progress (4): 43/98 kB | 161/782 kB | 17/49 kB | 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.jar (16 kB at 123 kB/s) 2024-05-30T16:12:33.461 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (3): 44/98 kB | 161/782 kB | 17/49 kB Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.jar 2024-05-30T16:12:33.480 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (3): 44/98 kB | 161/782 kB | 18/49 kB Progress (3): 45/98 kB | 161/782 kB | 18/49 kB Progress (3): 45/98 kB | 178/782 kB | 18/49 kB Progress (3): 47/98 kB | 178/782 kB | 18/49 kB Progress (3): 47/98 kB | 178/782 kB | 19/49 kB Progress (3): 48/98 kB | 178/782 kB | 19/49 kB Progress (3): 48/98 kB | 178/782 kB | 21/49 kB Progress (3): 49/98 kB | 178/782 kB | 21/49 kB Progress (3): 49/98 kB | 178/782 kB | 22/49 kB Progress (3): 49/98 kB | 194/782 kB | 22/49 kB Progress (3): 51/98 kB | 194/782 kB | 22/49 kB Progress (3): 51/98 kB | 194/782 kB | 23/49 kB Progress (3): 52/98 kB | 194/782 kB | 23/49 kB Progress (3): 52/98 kB | 194/782 kB | 25/49 kB Progress (3): 53/98 kB | 194/782 kB | 25/49 kB Progress (3): 53/98 kB | 194/782 kB | 26/49 kB Progress (3): 55/98 kB | 194/782 kB | 26/49 kB Progress (3): 55/98 kB | 194/782 kB | 28/49 kB Progress (3): 56/98 kB | 194/782 kB | 28/49 kB Progress (3): 56/98 kB | 194/782 kB | 29/49 kB Progress (3): 56/98 kB | 210/782 kB | 29/49 kB Progress (3): 57/98 kB | 210/782 kB | 29/49 kB Progress (3): 57/98 kB | 210/782 kB | 30/49 kB Progress (3): 59/98 kB | 210/782 kB | 30/49 kB Progress (3): 59/98 kB | 210/782 kB | 32/49 kB Progress (3): 60/98 kB | 210/782 kB | 32/49 kB Progress (3): 60/98 kB | 210/782 kB | 33/49 kB Progress (3): 62/98 kB | 210/782 kB | 33/49 kB Progress (3): 62/98 kB | 210/782 kB | 34/49 kB Progress (3): 62/98 kB | 210/782 kB | 34/49 kB Progress (3): 62/98 kB | 210/782 kB | 36/49 kB Progress (3): 62/98 kB | 210/782 kB | 37/49 kB Progress (3): 62/98 kB | 210/782 kB | 39/49 kB Progress (3): 62/98 kB | 227/782 kB | 39/49 kB Progress (3): 62/98 kB | 243/782 kB | 39/49 kB Progress (3): 62/98 kB | 260/782 kB | 39/49 kB Progress (3): 62/98 kB | 276/782 kB | 39/49 kB Progress (3): 62/98 kB | 292/782 kB | 39/49 kB Progress (3): 62/98 kB | 309/782 kB | 39/49 kB Progress (4): 62/98 kB | 309/782 kB | 39/49 kB | 1.4/11 kB Progress (4): 62/98 kB | 309/782 kB | 39/49 kB | 2.8/11 kB Progress (4): 62/98 kB | 309/782 kB | 39/49 kB | 4.1/11 kB Progress (4): 62/98 kB | 309/782 kB | 39/49 kB | 5.5/11 kB Progress (4): 62/98 kB | 325/782 kB | 39/49 kB | 5.5/11 kB Progress (4): 62/98 kB | 325/782 kB | 39/49 kB | 6.9/11 kB Progress (4): 62/98 kB | 325/782 kB | 39/49 kB | 8.3/11 kB Progress (4): 62/98 kB | 325/782 kB | 39/49 kB | 9.7/11 kB Progress (4): 62/98 kB | 325/782 kB | 39/49 kB | 11/11 kB Progress (4): 62/98 kB | 325/782 kB | 40/49 kB | 11/11 kB Progress (4): 62/98 kB | 325/782 kB | 40/49 kB | 11 kB Progress (4): 62/98 kB | 341/782 kB | 40/49 kB | 11 kB Progress (4): 62/98 kB | 341/782 kB | 41/49 kB | 11 kB Progress (4): 62/98 kB | 341/782 kB | 43/49 kB | 11 kB Progress (4): 62/98 kB | 341/782 kB | 44/49 kB | 11 kB Progress (4): 62/98 kB | 358/782 kB | 44/49 kB | 11 kB Progress (4): 62/98 kB | 358/782 kB | 46/49 kB | 11 kB Progress (4): 62/98 kB | 358/782 kB | 47/49 kB | 11 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.jar (11 kB at 80 kB/s) 2024-05-30T16:12:33.481 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (3): 62/98 kB | 358/782 kB | 48/49 kB Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.jar 2024-05-30T16:12:33.484 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (3): 62/98 kB | 374/782 kB | 48/49 kB Progress (3): 62/98 kB | 374/782 kB | 49 kB Progress (3): 62/98 kB | 391/782 kB | 49 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.jar (49 kB at 328 kB/s) 2024-05-30T16:12:33.484 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (2): 62/98 kB | 407/782 kB Progress (2): 78/98 kB | 407/782 kB Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.jar 2024-05-30T16:12:33.487 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (2): 78/98 kB | 423/782 kB Progress (2): 95/98 kB | 423/782 kB Progress (2): 98 kB | 423/782 kB Progress (3): 98 kB | 423/782 kB | 1.4/205 kB Progress (3): 98 kB | 440/782 kB | 1.4/205 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.jar (98 kB at 625 kB/s) 2024-05-30T16:12:33.487 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (2): 440/782 kB | 2.8/205 kB Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.jar 2024-05-30T16:12:33.514 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (2): 440/782 kB | 4.1/205 kB Progress (2): 456/782 kB | 4.1/205 kB Progress (2): 456/782 kB | 5.5/205 kB Progress (2): 456/782 kB | 6.9/205 kB Progress (2): 456/782 kB | 8.3/205 kB Progress (2): 473/782 kB | 8.3/205 kB Progress (2): 473/782 kB | 9.7/205 kB Progress (2): 473/782 kB | 11/205 kB Progress (2): 473/782 kB | 12/205 kB Progress (2): 473/782 kB | 14/205 kB Progress (2): 473/782 kB | 15/205 kB Progress (2): 489/782 kB | 15/205 kB Progress (2): 489/782 kB | 16/205 kB Progress (2): 505/782 kB | 16/205 kB Progress (2): 505/782 kB | 33/205 kB Progress (2): 522/782 kB | 33/205 kB Progress (2): 538/782 kB | 33/205 kB Progress (2): 554/782 kB | 33/205 kB Progress (2): 571/782 kB | 33/205 kB Progress (2): 587/782 kB | 33/205 kB Progress (2): 604/782 kB | 33/205 kB Progress (2): 620/782 kB | 33/205 kB Progress (2): 636/782 kB | 33/205 kB Progress (2): 653/782 kB | 33/205 kB Progress (2): 669/782 kB | 33/205 kB Progress (2): 686/782 kB | 33/205 kB Progress (3): 686/782 kB | 33/205 kB | 16/30 kB Progress (3): 686/782 kB | 49/205 kB | 16/30 kB Progress (3): 702/782 kB | 49/205 kB | 16/30 kB Progress (4): 702/782 kB | 49/205 kB | 16/30 kB | 1.4/81 kB Progress (4): 718/782 kB | 49/205 kB | 16/30 kB | 1.4/81 kB Progress (4): 718/782 kB | 49/205 kB | 16/30 kB | 2.8/81 kB Progress (4): 718/782 kB | 49/205 kB | 30 kB | 2.8/81 kB Progress (4): 718/782 kB | 49/205 kB | 30 kB | 4.1/81 kB Progress (4): 718/782 kB | 49/205 kB | 30 kB | 5.5/81 kB Progress (4): 718/782 kB | 65/205 kB | 30 kB | 5.5/81 kB Progress (4): 718/782 kB | 65/205 kB | 30 kB | 6.9/81 kB Progress (4): 735/782 kB | 65/205 kB | 30 kB | 6.9/81 kB Progress (4): 735/782 kB | 65/205 kB | 30 kB | 8.3/81 kB Progress (4): 735/782 kB | 65/205 kB | 30 kB | 9.7/81 kB Progress (5): 735/782 kB | 65/205 kB | 30 kB | 9.7/81 kB | 16/798 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.jar (30 kB at 173 kB/s) 2024-05-30T16:12:33.514 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (4): 751/782 kB | 65/205 kB | 9.7/81 kB | 16/798 kB Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.jar 2024-05-30T16:12:33.519 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (4): 751/782 kB | 65/205 kB | 11/81 kB | 16/798 kB Progress (4): 751/782 kB | 65/205 kB | 12/81 kB | 16/798 kB Progress (4): 751/782 kB | 65/205 kB | 14/81 kB | 16/798 kB Progress (4): 751/782 kB | 65/205 kB | 15/81 kB | 16/798 kB Progress (4): 751/782 kB | 65/205 kB | 16/81 kB | 16/798 kB Progress (4): 751/782 kB | 65/205 kB | 16/81 kB | 33/798 kB Progress (4): 751/782 kB | 81/205 kB | 16/81 kB | 33/798 kB Progress (4): 767/782 kB | 81/205 kB | 16/81 kB | 33/798 kB Progress (4): 782 kB | 81/205 kB | 16/81 kB | 33/798 kB Progress (4): 782 kB | 81/205 kB | 16/81 kB | 49/798 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.jar (782 kB at 4.3 MB/s) 2024-05-30T16:12:33.519 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.jar 2024-05-30T16:12:33.544 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (3): 97/205 kB | 16/81 kB | 49/798 kB Progress (3): 97/205 kB | 16/81 kB | 62/798 kB Progress (3): 97/205 kB | 16/81 kB | 78/798 kB Progress (3): 97/205 kB | 33/81 kB | 78/798 kB Progress (3): 97/205 kB | 33/81 kB | 95/798 kB Progress (3): 97/205 kB | 49/81 kB | 95/798 kB Progress (3): 114/205 kB | 49/81 kB | 95/798 kB Progress (3): 130/205 kB | 49/81 kB | 95/798 kB Progress (3): 130/205 kB | 64/81 kB | 95/798 kB Progress (3): 147/205 kB | 64/81 kB | 95/798 kB Progress (3): 147/205 kB | 64/81 kB | 111/798 kB Progress (3): 163/205 kB | 64/81 kB | 111/798 kB Progress (3): 163/205 kB | 64/81 kB | 128/798 kB Progress (3): 179/205 kB | 64/81 kB | 128/798 kB Progress (3): 179/205 kB | 64/81 kB | 144/798 kB Progress (3): 179/205 kB | 80/81 kB | 144/798 kB Progress (3): 196/205 kB | 80/81 kB | 144/798 kB Progress (3): 196/205 kB | 81 kB | 144/798 kB Progress (4): 196/205 kB | 81 kB | 144/798 kB | 13 kB Progress (4): 196/205 kB | 81 kB | 160/798 kB | 13 kB Progress (4): 205 kB | 81 kB | 160/798 kB | 13 kB Progress (4): 205 kB | 81 kB | 177/798 kB | 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.jar (13 kB at 65 kB/s) 2024-05-30T16:12:33.544 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.jar 2024-05-30T16:12:33.544 INFO:tasks.workunit.client.0.smithi006.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.jar (81 kB at 398 kB/s) 2024-05-30T16:12:33.544 INFO:tasks.workunit.client.0.smithi006.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.jar (205 kB at 1.0 MB/s) 2024-05-30T16:12:33.545 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.jar 2024-05-30T16:12:33.545 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.jar 2024-05-30T16:12:33.558 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 193/798 kB Progress (1): 210/798 kB Progress (2): 210/798 kB | 16/182 kB Progress (2): 226/798 kB | 16/182 kB Progress (2): 226/798 kB | 31/182 kB Progress (2): 242/798 kB | 31/182 kB Progress (2): 242/798 kB | 48/182 kB Progress (2): 259/798 kB | 48/182 kB Progress (2): 259/798 kB | 64/182 kB Progress (2): 275/798 kB | 64/182 kB Progress (2): 275/798 kB | 81/182 kB Progress (2): 291/798 kB | 81/182 kB Progress (2): 291/798 kB | 96/182 kB Progress (2): 291/798 kB | 112/182 kB Progress (2): 291/798 kB | 129/182 kB Progress (2): 291/798 kB | 145/182 kB Progress (2): 291/798 kB | 162/182 kB Progress (2): 291/798 kB | 178/182 kB Progress (2): 291/798 kB | 182 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.jar (182 kB at 831 kB/s) 2024-05-30T16:12:33.559 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.jar 2024-05-30T16:12:33.604 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 308/798 kB Progress (1): 324/798 kB Progress (1): 341/798 kB Progress (1): 357/798 kB Progress (1): 373/798 kB Progress (1): 390/798 kB Progress (1): 406/798 kB Progress (1): 423/798 kB Progress (1): 439/798 kB Progress (2): 439/798 kB | 16/27 kB Progress (3): 439/798 kB | 16/27 kB | 16/385 kB Progress (4): 439/798 kB | 16/27 kB | 16/385 kB | 16/151 kB Progress (4): 439/798 kB | 27 kB | 16/385 kB | 16/151 kB Progress (4): 455/798 kB | 27 kB | 16/385 kB | 16/151 kB Progress (5): 455/798 kB | 27 kB | 16/385 kB | 16/151 kB | 16/30 kB Progress (5): 455/798 kB | 27 kB | 16/385 kB | 33/151 kB | 16/30 kB Progress (5): 455/798 kB | 27 kB | 33/385 kB | 33/151 kB | 16/30 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.jar (27 kB at 104 kB/s) 2024-05-30T16:12:33.604 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (4): 472/798 kB | 33/385 kB | 33/151 kB | 16/30 kB Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.jar 2024-05-30T16:12:33.604 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (4): 472/798 kB | 33/385 kB | 33/151 kB | 30 kB Progress (4): 472/798 kB | 33/385 kB | 49/151 kB | 30 kB Progress (4): 488/798 kB | 33/385 kB | 49/151 kB | 30 kB Progress (4): 488/798 kB | 49/385 kB | 49/151 kB | 30 kB Progress (4): 488/798 kB | 49/385 kB | 61/151 kB | 30 kB Progress (4): 504/798 kB | 49/385 kB | 61/151 kB | 30 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.jar (30 kB at 114 kB/s) 2024-05-30T16:12:33.604 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (3): 504/798 kB | 63/385 kB | 61/151 kB Progress (3): 504/798 kB | 63/385 kB | 78/151 kB Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.jar 2024-05-30T16:12:33.608 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (3): 521/798 kB | 63/385 kB | 78/151 kB Progress (3): 521/798 kB | 63/385 kB | 94/151 kB Progress (3): 521/798 kB | 79/385 kB | 94/151 kB Progress (3): 537/798 kB | 79/385 kB | 94/151 kB Progress (3): 537/798 kB | 79/385 kB | 111/151 kB Progress (3): 537/798 kB | 96/385 kB | 111/151 kB Progress (3): 554/798 kB | 96/385 kB | 111/151 kB Progress (3): 554/798 kB | 96/385 kB | 127/151 kB Progress (3): 554/798 kB | 112/385 kB | 127/151 kB Progress (3): 570/798 kB | 112/385 kB | 127/151 kB Progress (3): 570/798 kB | 112/385 kB | 143/151 kB Progress (3): 570/798 kB | 128/385 kB | 143/151 kB Progress (3): 586/798 kB | 128/385 kB | 143/151 kB Progress (3): 586/798 kB | 128/385 kB | 151 kB Progress (3): 603/798 kB | 128/385 kB | 151 kB Progress (3): 603/798 kB | 145/385 kB | 151 kB Progress (3): 619/798 kB | 145/385 kB | 151 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.jar (151 kB at 562 kB/s) 2024-05-30T16:12:33.608 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (2): 619/798 kB | 161/385 kB Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.jar 2024-05-30T16:12:33.613 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (2): 636/798 kB | 161/385 kB Progress (2): 636/798 kB | 178/385 kB Progress (2): 652/798 kB | 178/385 kB Progress (2): 652/798 kB | 194/385 kB Progress (2): 668/798 kB | 194/385 kB Progress (2): 685/798 kB | 194/385 kB Progress (2): 685/798 kB | 210/385 kB Progress (2): 701/798 kB | 210/385 kB Progress (2): 701/798 kB | 227/385 kB Progress (2): 717/798 kB | 227/385 kB Progress (2): 734/798 kB | 227/385 kB Progress (2): 734/798 kB | 243/385 kB Progress (2): 750/798 kB | 243/385 kB Progress (2): 767/798 kB | 243/385 kB Progress (2): 767/798 kB | 259/385 kB Progress (2): 783/798 kB | 259/385 kB Progress (2): 783/798 kB | 276/385 kB Progress (2): 798 kB | 276/385 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.jar (798 kB at 2.9 MB/s) 2024-05-30T16:12:33.613 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.jar 2024-05-30T16:12:33.631 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 292/385 kB Progress (1): 309/385 kB Progress (1): 325/385 kB Progress (1): 341/385 kB Progress (1): 358/385 kB Progress (1): 374/385 kB Progress (1): 378/385 kB Progress (1): 385 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.jar (385 kB at 1.3 MB/s) 2024-05-30T16:12:33.631 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.jar 2024-05-30T16:12:33.633 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/75 kB Progress (2): 16/75 kB | 13 kB Progress (2): 33/75 kB | 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.jar (13 kB at 45 kB/s) 2024-05-30T16:12:33.633 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.jar 2024-05-30T16:12:33.641 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 49/75 kB Progress (2): 49/75 kB | 16/780 kB Progress (2): 49/75 kB | 32/780 kB Progress (2): 62/75 kB | 32/780 kB Progress (2): 62/75 kB | 49/780 kB Progress (3): 62/75 kB | 49/780 kB | 16/329 kB Progress (3): 75 kB | 49/780 kB | 16/329 kB Progress (3): 75 kB | 49/780 kB | 33/329 kB Progress (3): 75 kB | 49/780 kB | 49/329 kB Progress (3): 75 kB | 49/780 kB | 66/329 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.jar (75 kB at 248 kB/s) 2024-05-30T16:12:33.641 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (2): 65/780 kB | 66/329 kB Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.jar 2024-05-30T16:12:33.649 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (2): 65/780 kB | 82/329 kB Progress (2): 65/780 kB | 98/329 kB Progress (2): 65/780 kB | 115/329 kB Progress (2): 65/780 kB | 131/329 kB Progress (2): 81/780 kB | 131/329 kB Progress (2): 81/780 kB | 147/329 kB Progress (2): 81/780 kB | 164/329 kB Progress (2): 81/780 kB | 180/329 kB Progress (2): 81/780 kB | 197/329 kB Progress (2): 81/780 kB | 213/329 kB Progress (2): 81/780 kB | 229/329 kB Progress (2): 81/780 kB | 246/329 kB Progress (2): 81/780 kB | 256/329 kB Progress (2): 81/780 kB | 272/329 kB Progress (2): 98/780 kB | 272/329 kB Progress (2): 98/780 kB | 289/329 kB Progress (2): 98/780 kB | 305/329 kB Progress (2): 98/780 kB | 321/329 kB Progress (2): 98/780 kB | 329 kB Progress (2): 114/780 kB | 329 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.jar (329 kB at 1.1 MB/s) 2024-05-30T16:12:33.649 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.jar 2024-05-30T16:12:33.678 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 130/780 kB Progress (1): 147/780 kB Progress (1): 163/780 kB Progress (1): 180/780 kB Progress (1): 196/780 kB Progress (1): 212/780 kB Progress (1): 229/780 kB Progress (1): 245/780 kB Progress (2): 245/780 kB | 16/501 kB Progress (2): 262/780 kB | 16/501 kB Progress (2): 262/780 kB | 32/501 kB Progress (2): 278/780 kB | 32/501 kB Progress (2): 278/780 kB | 49/501 kB Progress (2): 294/780 kB | 49/501 kB Progress (2): 294/780 kB | 65/501 kB Progress (2): 294/780 kB | 81/501 kB Progress (2): 294/780 kB | 97/501 kB Progress (2): 311/780 kB | 97/501 kB Progress (3): 311/780 kB | 97/501 kB | 16/354 kB Progress (3): 311/780 kB | 114/501 kB | 16/354 kB Progress (3): 327/780 kB | 114/501 kB | 16/354 kB Progress (3): 327/780 kB | 130/501 kB | 16/354 kB Progress (3): 327/780 kB | 147/501 kB | 16/354 kB Progress (3): 327/780 kB | 163/501 kB | 16/354 kB Progress (3): 343/780 kB | 163/501 kB | 16/354 kB Progress (3): 343/780 kB | 163/501 kB | 33/354 kB Progress (3): 343/780 kB | 179/501 kB | 33/354 kB Progress (3): 343/780 kB | 196/501 kB | 33/354 kB Progress (3): 343/780 kB | 212/501 kB | 33/354 kB Progress (3): 343/780 kB | 228/501 kB | 33/354 kB Progress (4): 343/780 kB | 228/501 kB | 33/354 kB | 16/41 kB Progress (4): 360/780 kB | 228/501 kB | 33/354 kB | 16/41 kB Progress (4): 360/780 kB | 228/501 kB | 33/354 kB | 33/41 kB Progress (4): 360/780 kB | 228/501 kB | 49/354 kB | 33/41 kB Progress (4): 360/780 kB | 245/501 kB | 49/354 kB | 33/41 kB Progress (4): 360/780 kB | 245/501 kB | 49/354 kB | 41 kB Progress (4): 360/780 kB | 261/501 kB | 49/354 kB | 41 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.jar (41 kB at 121 kB/s) 2024-05-30T16:12:33.679 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.jar 2024-05-30T16:12:33.684 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (3): 360/780 kB | 278/501 kB | 49/354 kB Progress (3): 376/780 kB | 278/501 kB | 49/354 kB Progress (3): 376/780 kB | 294/501 kB | 49/354 kB Progress (3): 376/780 kB | 294/501 kB | 66/354 kB Progress (3): 393/780 kB | 294/501 kB | 66/354 kB Progress (3): 402/780 kB | 294/501 kB | 66/354 kB Progress (3): 418/780 kB | 294/501 kB | 66/354 kB Progress (3): 435/780 kB | 294/501 kB | 66/354 kB Progress (4): 435/780 kB | 294/501 kB | 66/354 kB | 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.jar (6.4 kB at 18 kB/s) 2024-05-30T16:12:33.684 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.jar 2024-05-30T16:12:33.711 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (3): 451/780 kB | 294/501 kB | 66/354 kB Progress (3): 468/780 kB | 294/501 kB | 66/354 kB Progress (3): 484/780 kB | 294/501 kB | 66/354 kB Progress (3): 500/780 kB | 294/501 kB | 66/354 kB Progress (3): 517/780 kB | 294/501 kB | 66/354 kB Progress (3): 533/780 kB | 294/501 kB | 66/354 kB Progress (3): 533/780 kB | 310/501 kB | 66/354 kB Progress (3): 533/780 kB | 327/501 kB | 66/354 kB Progress (3): 549/780 kB | 327/501 kB | 66/354 kB Progress (3): 549/780 kB | 343/501 kB | 66/354 kB Progress (3): 566/780 kB | 343/501 kB | 66/354 kB Progress (3): 566/780 kB | 343/501 kB | 82/354 kB Progress (3): 566/780 kB | 360/501 kB | 82/354 kB Progress (3): 582/780 kB | 360/501 kB | 82/354 kB Progress (3): 599/780 kB | 360/501 kB | 82/354 kB Progress (3): 599/780 kB | 376/501 kB | 82/354 kB Progress (3): 599/780 kB | 392/501 kB | 82/354 kB Progress (3): 615/780 kB | 392/501 kB | 82/354 kB Progress (3): 615/780 kB | 409/501 kB | 82/354 kB Progress (3): 615/780 kB | 409/501 kB | 98/354 kB Progress (3): 631/780 kB | 409/501 kB | 98/354 kB Progress (3): 631/780 kB | 425/501 kB | 98/354 kB Progress (3): 648/780 kB | 425/501 kB | 98/354 kB Progress (3): 648/780 kB | 441/501 kB | 98/354 kB Progress (3): 664/780 kB | 441/501 kB | 98/354 kB Progress (3): 664/780 kB | 458/501 kB | 98/354 kB Progress (3): 664/780 kB | 474/501 kB | 98/354 kB Progress (3): 664/780 kB | 474/501 kB | 115/354 kB Progress (3): 681/780 kB | 474/501 kB | 115/354 kB Progress (3): 681/780 kB | 491/501 kB | 115/354 kB Progress (3): 681/780 kB | 501 kB | 115/354 kB Progress (3): 697/780 kB | 501 kB | 115/354 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.jar (501 kB at 1.4 MB/s) 2024-05-30T16:12:33.711 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.jar 2024-05-30T16:12:33.719 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (2): 713/780 kB | 115/354 kB Progress (2): 730/780 kB | 115/354 kB Progress (2): 730/780 kB | 131/354 kB Progress (3): 730/780 kB | 131/354 kB | 16/586 kB Progress (3): 746/780 kB | 131/354 kB | 16/586 kB Progress (3): 762/780 kB | 131/354 kB | 16/586 kB Progress (3): 762/780 kB | 131/354 kB | 33/586 kB Progress (3): 779/780 kB | 131/354 kB | 33/586 kB Progress (4): 779/780 kB | 131/354 kB | 33/586 kB | 16/211 kB Progress (4): 780 kB | 131/354 kB | 33/586 kB | 16/211 kB Progress (4): 780 kB | 131/354 kB | 33/586 kB | 33/211 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.jar (780 kB at 2.1 MB/s) 2024-05-30T16:12:33.719 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (3): 131/354 kB | 33/586 kB | 49/211 kB Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.jar 2024-05-30T16:12:33.725 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (3): 131/354 kB | 33/586 kB | 66/211 kB Progress (3): 131/354 kB | 33/586 kB | 78/211 kB Progress (3): 131/354 kB | 33/586 kB | 95/211 kB Progress (3): 131/354 kB | 49/586 kB | 95/211 kB Progress (3): 147/354 kB | 49/586 kB | 95/211 kB Progress (3): 147/354 kB | 49/586 kB | 111/211 kB Progress (3): 147/354 kB | 49/586 kB | 127/211 kB Progress (3): 147/354 kB | 49/586 kB | 144/211 kB Progress (3): 147/354 kB | 49/586 kB | 160/211 kB Progress (3): 147/354 kB | 49/586 kB | 177/211 kB Progress (3): 147/354 kB | 63/586 kB | 177/211 kB Progress (3): 147/354 kB | 63/586 kB | 193/211 kB Progress (3): 147/354 kB | 63/586 kB | 209/211 kB Progress (3): 147/354 kB | 63/586 kB | 211 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.jar (211 kB at 549 kB/s) 2024-05-30T16:12:33.725 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar 2024-05-30T16:12:33.744 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (2): 147/354 kB | 80/586 kB Progress (2): 164/354 kB | 80/586 kB Progress (2): 164/354 kB | 96/586 kB Progress (2): 164/354 kB | 112/586 kB Progress (2): 164/354 kB | 129/586 kB Progress (2): 164/354 kB | 145/586 kB Progress (2): 180/354 kB | 145/586 kB Progress (2): 197/354 kB | 145/586 kB Progress (2): 197/354 kB | 162/586 kB Progress (3): 197/354 kB | 162/586 kB | 14 kB Progress (3): 213/354 kB | 162/586 kB | 14 kB Progress (3): 213/354 kB | 178/586 kB | 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.jar (14 kB at 35 kB/s) 2024-05-30T16:12:33.744 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.jar 2024-05-30T16:12:33.752 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (2): 213/354 kB | 194/586 kB Progress (3): 213/354 kB | 194/586 kB | 16/106 kB Progress (3): 229/354 kB | 194/586 kB | 16/106 kB Progress (3): 229/354 kB | 211/586 kB | 16/106 kB Progress (3): 229/354 kB | 211/586 kB | 33/106 kB Progress (3): 229/354 kB | 211/586 kB | 49/106 kB Progress (4): 229/354 kB | 211/586 kB | 49/106 kB | 6.8 kB Progress (4): 229/354 kB | 227/586 kB | 49/106 kB | 6.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar (6.8 kB at 17 kB/s) 2024-05-30T16:12:33.752 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.jar 2024-05-30T16:12:33.757 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (3): 229/354 kB | 227/586 kB | 66/106 kB Progress (3): 229/354 kB | 227/586 kB | 82/106 kB Progress (3): 246/354 kB | 227/586 kB | 82/106 kB Progress (3): 246/354 kB | 244/586 kB | 82/106 kB Progress (3): 246/354 kB | 244/586 kB | 96/106 kB Progress (3): 246/354 kB | 244/586 kB | 106 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.jar (106 kB at 255 kB/s) 2024-05-30T16:12:33.757 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.jar 2024-05-30T16:12:33.790 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (2): 246/354 kB | 260/586 kB Progress (2): 262/354 kB | 260/586 kB Progress (2): 262/354 kB | 276/586 kB Progress (2): 262/354 kB | 293/586 kB Progress (2): 279/354 kB | 293/586 kB Progress (2): 279/354 kB | 309/586 kB Progress (2): 279/354 kB | 325/586 kB Progress (2): 295/354 kB | 325/586 kB Progress (2): 295/354 kB | 342/586 kB Progress (3): 295/354 kB | 342/586 kB | 16/245 kB Progress (3): 295/354 kB | 342/586 kB | 33/245 kB Progress (3): 295/354 kB | 358/586 kB | 33/245 kB Progress (3): 311/354 kB | 358/586 kB | 33/245 kB Progress (3): 311/354 kB | 358/586 kB | 49/245 kB Progress (3): 311/354 kB | 375/586 kB | 49/245 kB Progress (3): 311/354 kB | 375/586 kB | 66/245 kB Progress (3): 311/354 kB | 391/586 kB | 66/245 kB Progress (3): 311/354 kB | 391/586 kB | 82/245 kB Progress (3): 328/354 kB | 391/586 kB | 82/245 kB Progress (3): 328/354 kB | 391/586 kB | 98/245 kB Progress (3): 328/354 kB | 407/586 kB | 98/245 kB Progress (4): 328/354 kB | 407/586 kB | 98/245 kB | 16/205 kB Progress (4): 328/354 kB | 407/586 kB | 115/245 kB | 16/205 kB Progress (4): 328/354 kB | 407/586 kB | 115/245 kB | 33/205 kB Progress (4): 328/354 kB | 407/586 kB | 131/245 kB | 33/205 kB Progress (4): 328/354 kB | 407/586 kB | 131/245 kB | 49/205 kB Progress (4): 328/354 kB | 407/586 kB | 131/245 kB | 66/205 kB Progress (4): 328/354 kB | 407/586 kB | 131/245 kB | 82/205 kB Progress (4): 328/354 kB | 407/586 kB | 147/245 kB | 82/205 kB Progress (4): 328/354 kB | 407/586 kB | 147/245 kB | 94/205 kB Progress (4): 328/354 kB | 424/586 kB | 147/245 kB | 94/205 kB Progress (4): 328/354 kB | 424/586 kB | 164/245 kB | 94/205 kB Progress (4): 344/354 kB | 424/586 kB | 164/245 kB | 94/205 kB Progress (4): 344/354 kB | 424/586 kB | 164/245 kB | 111/205 kB Progress (4): 344/354 kB | 424/586 kB | 164/245 kB | 127/205 kB Progress (4): 344/354 kB | 424/586 kB | 164/245 kB | 144/205 kB Progress (4): 344/354 kB | 424/586 kB | 180/245 kB | 144/205 kB Progress (5): 344/354 kB | 424/586 kB | 180/245 kB | 144/205 kB | 15 kB Progress (5): 344/354 kB | 424/586 kB | 180/245 kB | 160/205 kB | 15 kB Progress (5): 344/354 kB | 440/586 kB | 180/245 kB | 160/205 kB | 15 kB Progress (5): 344/354 kB | 440/586 kB | 197/245 kB | 160/205 kB | 15 kB Progress (5): 344/354 kB | 440/586 kB | 197/245 kB | 176/205 kB | 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.jar (15 kB at 34 kB/s) 2024-05-30T16:12:33.790 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (4): 354 kB | 440/586 kB | 197/245 kB | 176/205 kB Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.jar 2024-05-30T16:12:33.791 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (4): 354 kB | 440/586 kB | 197/245 kB | 193/205 kB Progress (4): 354 kB | 440/586 kB | 213/245 kB | 193/205 kB Progress (4): 354 kB | 457/586 kB | 213/245 kB | 193/205 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.jar (354 kB at 784 kB/s) 2024-05-30T16:12:33.791 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.jar 2024-05-30T16:12:33.792 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (3): 457/586 kB | 229/245 kB | 193/205 kB Progress (3): 457/586 kB | 229/245 kB | 205 kB Progress (3): 457/586 kB | 245 kB | 205 kB Progress (3): 473/586 kB | 245 kB | 205 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.jar (205 kB at 453 kB/s) 2024-05-30T16:12:33.793 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.jar 2024-05-30T16:12:33.793 INFO:tasks.workunit.client.0.smithi006.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.jar (245 kB at 541 kB/s) 2024-05-30T16:12:33.793 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.jar 2024-05-30T16:12:33.810 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 489/586 kB Progress (1): 506/586 kB Progress (1): 522/586 kB Progress (1): 538/586 kB Progress (1): 555/586 kB Progress (1): 571/586 kB Progress (1): 586 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.jar (586 kB at 1.2 MB/s) 2024-05-30T16:12:33.810 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.jar 2024-05-30T16:12:33.819 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.jar (6.4 kB at 13 kB/s) 2024-05-30T16:12:33.819 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.jar 2024-05-30T16:12:33.821 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/23 kB Progress (2): 16/23 kB | 16/24 kB Progress (2): 23 kB | 16/24 kB Progress (3): 23 kB | 16/24 kB | 16/16 kB Progress (3): 23 kB | 24 kB | 16/16 kB Progress (3): 23 kB | 24 kB | 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.jar (24 kB at 50 kB/s) 2024-05-30T16:12:33.821 INFO:tasks.workunit.client.0.smithi006.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.jar (23 kB at 47 kB/s) 2024-05-30T16:12:33.823 INFO:tasks.workunit.client.0.smithi006.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.jar (16 kB at 34 kB/s) 2024-05-30T16:12:33.836 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.jar (17 kB at 35 kB/s) 2024-05-30T16:12:33.858 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/184 kB Progress (1): 33/184 kB Progress (1): 49/184 kB Progress (1): 66/184 kB Progress (1): 78/184 kB Progress (1): 95/184 kB Progress (1): 111/184 kB Progress (1): 127/184 kB Progress (1): 144/184 kB Progress (1): 160/184 kB Progress (1): 177/184 kB Progress (1): 184 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.jar (184 kB at 356 kB/s) 2024-05-30T16:12:33.887 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] 2024-05-30T16:12:33.887 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] --- clean:3.2.0:clean (default-clean) @ jcksum --- 2024-05-30T16:12:33.895 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.pom 2024-05-30T16:12:33.922 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 5.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.pom (5.8 kB at 216 kB/s) 2024-05-30T16:12:33.925 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/34/maven-shared-components-34.pom 2024-05-30T16:12:33.950 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 5.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/34/maven-shared-components-34.pom (5.1 kB at 204 kB/s) 2024-05-30T16:12:33.952 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/34/maven-parent-34.pom 2024-05-30T16:12:33.979 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/43 kB Progress (1): 33/43 kB Progress (1): 43 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/34/maven-parent-34.pom (43 kB at 1.5 MB/s) 2024-05-30T16:12:33.984 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.pom 2024-05-30T16:12:34.008 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.pom (14 kB at 594 kB/s) 2024-05-30T16:12:34.015 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.jar 2024-05-30T16:12:34.050 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/153 kB Progress (1): 33/153 kB Progress (1): 49/153 kB Progress (1): 66/153 kB Progress (1): 82/153 kB Progress (1): 98/153 kB Progress (1): 115/153 kB Progress (1): 131/153 kB Progress (1): 147/153 kB Progress (1): 153 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.jar (153 kB at 4.3 MB/s) 2024-05-30T16:12:34.051 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.jar 2024-05-30T16:12:34.090 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/215 kB Progress (1): 33/215 kB Progress (1): 49/215 kB Progress (1): 66/215 kB Progress (1): 79/215 kB Progress (1): 96/215 kB Progress (1): 112/215 kB Progress (1): 128/215 kB Progress (1): 145/215 kB Progress (1): 161/215 kB Progress (1): 178/215 kB Progress (1): 194/215 kB Progress (1): 210/215 kB Progress (1): 215 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.jar (215 kB at 5.5 MB/s) 2024-05-30T16:12:34.185 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] 2024-05-30T16:12:34.186 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] --- resources:3.3.1:resources (default-resources) @ jcksum --- 2024-05-30T16:12:34.191 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.pom 2024-05-30T16:12:34.217 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 2.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.pom (2.7 kB at 102 kB/s) 2024-05-30T16:12:34.220 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/5.1/plexus-5.1.pom 2024-05-30T16:12:34.244 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/23 kB Progress (1): 23 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/5.1/plexus-5.1.pom (23 kB at 900 kB/s) 2024-05-30T16:12:34.250 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.pom 2024-05-30T16:12:34.277 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 8.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.pom (8.8 kB at 325 kB/s) 2024-05-30T16:12:34.281 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/10/plexus-10.pom 2024-05-30T16:12:34.305 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/25 kB Progress (1): 25 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/10/plexus-10.pom (25 kB at 1.1 MB/s) 2024-05-30T16:12:34.310 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.pom 2024-05-30T16:12:34.335 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 6.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.pom (6.0 kB at 241 kB/s) 2024-05-30T16:12:34.338 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/39/maven-shared-components-39.pom 2024-05-30T16:12:34.362 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/39/maven-shared-components-39.pom (3.2 kB at 134 kB/s) 2024-05-30T16:12:34.367 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.pom 2024-05-30T16:12:34.399 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 612 B Downloaded from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.pom (612 B at 19 kB/s) 2024-05-30T16:12:34.403 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.pom 2024-05-30T16:12:34.430 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 2.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.pom (2.7 kB at 102 kB/s) 2024-05-30T16:12:34.433 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.36/slf4j-parent-1.7.36.pom 2024-05-30T16:12:34.459 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.36/slf4j-parent-1.7.36.pom (14 kB at 542 kB/s) 2024-05-30T16:12:34.465 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.pom 2024-05-30T16:12:34.494 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.pom (3.2 kB at 110 kB/s) 2024-05-30T16:12:34.498 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/15/spice-parent-15.pom 2024-05-30T16:12:34.525 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/15/spice-parent-15.pom (8.4 kB at 310 kB/s) 2024-05-30T16:12:34.529 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/5/forge-parent-5.pom 2024-05-30T16:12:34.556 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/5/forge-parent-5.pom (8.4 kB at 310 kB/s) 2024-05-30T16:12:34.562 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.pom 2024-05-30T16:12:34.589 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 8.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.pom (8.0 kB at 297 kB/s) 2024-05-30T16:12:34.596 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.pom 2024-05-30T16:12:34.621 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.pom (20 kB at 789 kB/s) 2024-05-30T16:12:34.632 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.2/junit-bom-5.7.2.pom 2024-05-30T16:12:34.660 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 5.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.2/junit-bom-5.7.2.pom (5.1 kB at 182 kB/s) 2024-05-30T16:12:34.665 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.pom 2024-05-30T16:12:34.691 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/31 kB Progress (1): 31 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.pom (31 kB at 1.2 MB/s) 2024-05-30T16:12:34.704 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.1/junit-bom-5.7.1.pom 2024-05-30T16:12:34.730 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 5.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.1/junit-bom-5.7.1.pom (5.1 kB at 189 kB/s) 2024-05-30T16:12:34.740 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.jar 2024-05-30T16:12:34.770 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/85 kB Progress (1): 33/85 kB Progress (1): 37/85 kB Progress (1): 53/85 kB Progress (1): 69/85 kB Progress (1): 85 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.jar (85 kB at 2.8 MB/s) 2024-05-30T16:12:34.771 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.jar 2024-05-30T16:12:34.771 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.jar 2024-05-30T16:12:34.772 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.jar 2024-05-30T16:12:34.772 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.jar 2024-05-30T16:12:34.773 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.jar 2024-05-30T16:12:34.798 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/55 kB Progress (2): 16/55 kB | 2.5 kB Progress (2): 33/55 kB | 2.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.jar (2.5 kB at 96 kB/s) 2024-05-30T16:12:34.798 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (2): 33/55 kB | 16/269 kB Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.jar 2024-05-30T16:12:34.799 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (3): 33/55 kB | 16/269 kB | 16/41 kB Progress (3): 49/55 kB | 16/269 kB | 16/41 kB Progress (4): 49/55 kB | 16/269 kB | 16/41 kB | 8.5 kB Progress (4): 55 kB | 16/269 kB | 16/41 kB | 8.5 kB Progress (4): 55 kB | 33/269 kB | 16/41 kB | 8.5 kB Progress (4): 55 kB | 33/269 kB | 33/41 kB | 8.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.jar (8.5 kB at 314 kB/s) 2024-05-30T16:12:34.800 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.jar 2024-05-30T16:12:34.800 INFO:tasks.workunit.client.0.smithi006.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.jar (55 kB at 1.9 MB/s) 2024-05-30T16:12:34.800 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (2): 33/269 kB | 41 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.jar (41 kB at 1.4 MB/s) 2024-05-30T16:12:34.830 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 49/269 kB Progress (1): 63/269 kB Progress (1): 79/269 kB Progress (1): 95/269 kB Progress (1): 112/269 kB Progress (1): 128/269 kB Progress (1): 145/269 kB Progress (1): 161/269 kB Progress (1): 177/269 kB Progress (1): 194/269 kB Progress (1): 210/269 kB Progress (1): 226/269 kB Progress (2): 226/269 kB | 16/587 kB Progress (2): 243/269 kB | 16/587 kB Progress (3): 243/269 kB | 16/587 kB | 16/327 kB Progress (3): 243/269 kB | 33/587 kB | 16/327 kB Progress (3): 243/269 kB | 49/587 kB | 16/327 kB Progress (3): 243/269 kB | 64/587 kB | 16/327 kB Progress (3): 259/269 kB | 64/587 kB | 16/327 kB Progress (3): 259/269 kB | 81/587 kB | 16/327 kB Progress (3): 269 kB | 81/587 kB | 16/327 kB Progress (3): 269 kB | 97/587 kB | 16/327 kB Progress (3): 269 kB | 97/587 kB | 32/327 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.jar (269 kB at 4.6 MB/s) 2024-05-30T16:12:34.863 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (2): 113/587 kB | 32/327 kB Progress (2): 130/587 kB | 32/327 kB Progress (2): 146/587 kB | 32/327 kB Progress (2): 162/587 kB | 32/327 kB Progress (2): 179/587 kB | 32/327 kB Progress (2): 179/587 kB | 49/327 kB Progress (2): 195/587 kB | 49/327 kB Progress (2): 212/587 kB | 49/327 kB Progress (2): 228/587 kB | 49/327 kB Progress (2): 244/587 kB | 49/327 kB Progress (2): 244/587 kB | 65/327 kB Progress (2): 261/587 kB | 65/327 kB Progress (2): 277/587 kB | 65/327 kB Progress (2): 294/587 kB | 65/327 kB Progress (2): 310/587 kB | 65/327 kB Progress (2): 310/587 kB | 78/327 kB Progress (2): 326/587 kB | 78/327 kB Progress (2): 326/587 kB | 94/327 kB Progress (2): 343/587 kB | 94/327 kB Progress (2): 359/587 kB | 94/327 kB Progress (2): 375/587 kB | 94/327 kB Progress (2): 392/587 kB | 94/327 kB Progress (2): 392/587 kB | 111/327 kB Progress (2): 408/587 kB | 111/327 kB Progress (2): 425/587 kB | 111/327 kB Progress (2): 441/587 kB | 111/327 kB Progress (2): 457/587 kB | 111/327 kB Progress (2): 457/587 kB | 127/327 kB Progress (2): 474/587 kB | 127/327 kB Progress (2): 490/587 kB | 127/327 kB Progress (2): 507/587 kB | 127/327 kB Progress (2): 507/587 kB | 144/327 kB Progress (2): 523/587 kB | 144/327 kB Progress (2): 539/587 kB | 144/327 kB Progress (2): 556/587 kB | 144/327 kB Progress (2): 572/587 kB | 144/327 kB Progress (2): 587 kB | 144/327 kB Progress (2): 587 kB | 160/327 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.jar (587 kB at 6.5 MB/s) 2024-05-30T16:12:34.900 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 176/327 kB Progress (1): 193/327 kB Progress (1): 209/327 kB Progress (1): 225/327 kB Progress (1): 242/327 kB Progress (1): 258/327 kB Progress (1): 275/327 kB Progress (1): 291/327 kB Progress (1): 307/327 kB Progress (1): 324/327 kB Progress (1): 327 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.jar (327 kB at 2.6 MB/s) 2024-05-30T16:12:34.943 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/main/resources 2024-05-30T16:12:34.944 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] 2024-05-30T16:12:34.944 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] --- compiler:3.11.0:compile (default-compile) @ jcksum --- 2024-05-30T16:12:34.949 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.pom 2024-05-30T16:12:34.977 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 4.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.pom (4.7 kB at 164 kB/s) 2024-05-30T16:12:34.980 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/19/maven-shared-components-19.pom 2024-05-30T16:12:35.005 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/19/maven-shared-components-19.pom (6.4 kB at 254 kB/s) 2024-05-30T16:12:35.007 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/23/maven-parent-23.pom 2024-05-30T16:12:35.038 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/33 kB Progress (1): 32/33 kB Progress (1): 33 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/23/maven-parent-23.pom (33 kB at 1.0 MB/s) 2024-05-30T16:12:35.041 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/13/apache-13.pom 2024-05-30T16:12:35.069 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/13/apache-13.pom (14 kB at 499 kB/s) 2024-05-30T16:12:35.075 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.pom 2024-05-30T16:12:35.102 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 815 B Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.pom (815 B at 30 kB/s) 2024-05-30T16:12:35.106 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/1.5.5/plexus-containers-1.5.5.pom 2024-05-30T16:12:35.134 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 4.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/1.5.5/plexus-containers-1.5.5.pom (4.2 kB at 163 kB/s) 2024-05-30T16:12:35.136 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/2.0.7/plexus-2.0.7.pom 2024-05-30T16:12:35.164 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/2.0.7/plexus-2.0.7.pom (17 kB at 618 kB/s) 2024-05-30T16:12:35.170 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.pom 2024-05-30T16:12:35.199 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 5.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.pom (5.0 kB at 171 kB/s) 2024-05-30T16:12:35.203 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.1.2/plexus-languages-1.1.2.pom 2024-05-30T16:12:35.230 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 4.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.1.2/plexus-languages-1.1.2.pom (4.1 kB at 153 kB/s) 2024-05-30T16:12:35.236 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.pom 2024-05-30T16:12:35.263 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 2.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.pom (2.4 kB at 88 kB/s) 2024-05-30T16:12:35.267 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/ow2/1.5.1/ow2-1.5.1.pom 2024-05-30T16:12:35.296 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 11 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/ow2/1.5.1/ow2-1.5.1.pom (11 kB at 389 kB/s) 2024-05-30T16:12:35.302 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.pom 2024-05-30T16:12:35.335 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.pom (17 kB at 522 kB/s) 2024-05-30T16:12:35.340 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/9/oss-parent-9.pom 2024-05-30T16:12:35.369 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 6.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/9/oss-parent-9.pom (6.6 kB at 219 kB/s) 2024-05-30T16:12:35.375 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.pom 2024-05-30T16:12:35.405 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 1.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.pom (1.1 kB at 35 kB/s) 2024-05-30T16:12:35.408 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler/2.13.0/plexus-compiler-2.13.0.pom 2024-05-30T16:12:35.437 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler/2.13.0/plexus-compiler-2.13.0.pom (8.4 kB at 299 kB/s) 2024-05-30T16:12:35.441 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/10.0/plexus-components-10.0.pom 2024-05-30T16:12:35.467 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 2.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/10.0/plexus-components-10.0.pom (2.7 kB at 103 kB/s) 2024-05-30T16:12:35.473 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.1/junit-bom-5.9.1.pom 2024-05-30T16:12:35.502 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.1/junit-bom-5.9.1.pom (5.6 kB at 194 kB/s) 2024-05-30T16:12:35.508 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.pom 2024-05-30T16:12:35.536 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 1.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.pom (1.1 kB at 41 kB/s) 2024-05-30T16:12:35.543 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.1/plexus-component-annotations-2.1.1.pom 2024-05-30T16:12:35.569 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 770 B Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.1/plexus-component-annotations-2.1.1.pom (770 B at 30 kB/s) 2024-05-30T16:12:35.572 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.1/plexus-containers-2.1.1.pom 2024-05-30T16:12:35.600 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 6.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.1/plexus-containers-2.1.1.pom (6.0 kB at 216 kB/s) 2024-05-30T16:12:35.603 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/6.5/plexus-6.5.pom 2024-05-30T16:12:35.636 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/26 kB Progress (1): 26 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/6.5/plexus-6.5.pom (26 kB at 779 kB/s) 2024-05-30T16:12:35.643 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.pom 2024-05-30T16:12:35.673 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 1.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.pom (1.2 kB at 40 kB/s) 2024-05-30T16:12:35.677 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compilers/2.13.0/plexus-compilers-2.13.0.pom 2024-05-30T16:12:35.706 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 1.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compilers/2.13.0/plexus-compilers-2.13.0.pom (1.3 kB at 46 kB/s) 2024-05-30T16:12:35.720 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.jar 2024-05-30T16:12:35.751 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.jar (14 kB at 437 kB/s) 2024-05-30T16:12:35.751 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.jar 2024-05-30T16:12:35.752 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.jar 2024-05-30T16:12:35.752 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.jar 2024-05-30T16:12:35.752 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.jar 2024-05-30T16:12:35.753 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.jar 2024-05-30T16:12:35.778 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 4.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.jar (4.2 kB at 156 kB/s) 2024-05-30T16:12:35.778 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.jar 2024-05-30T16:12:35.779 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/122 kB Progress (2): 16/122 kB | 16/55 kB Progress (3): 16/122 kB | 16/55 kB | 16/27 kB Progress (3): 16/122 kB | 33/55 kB | 16/27 kB Progress (3): 16/122 kB | 33/55 kB | 27 kB Progress (4): 16/122 kB | 33/55 kB | 27 kB | 16/334 kB Progress (4): 32/122 kB | 33/55 kB | 27 kB | 16/334 kB Progress (4): 32/122 kB | 49/55 kB | 27 kB | 16/334 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.jar (27 kB at 1.0 MB/s) 2024-05-30T16:12:35.780 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.jar 2024-05-30T16:12:35.781 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (3): 32/122 kB | 49/55 kB | 33/334 kB Progress (3): 32/122 kB | 55 kB | 33/334 kB Progress (3): 32/122 kB | 55 kB | 49/334 kB Progress (3): 49/122 kB | 55 kB | 49/334 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.jar (55 kB at 1.8 MB/s) 2024-05-30T16:12:35.781 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.jar 2024-05-30T16:12:35.788 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (2): 49/122 kB | 64/334 kB Progress (2): 63/122 kB | 64/334 kB Progress (2): 63/122 kB | 80/334 kB Progress (2): 63/122 kB | 96/334 kB Progress (2): 80/122 kB | 96/334 kB Progress (2): 80/122 kB | 113/334 kB Progress (2): 96/122 kB | 113/334 kB Progress (2): 96/122 kB | 129/334 kB Progress (2): 96/122 kB | 146/334 kB Progress (2): 112/122 kB | 146/334 kB Progress (2): 122 kB | 146/334 kB Progress (2): 122 kB | 162/334 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.jar (122 kB at 3.4 MB/s) 2024-05-30T16:12:35.802 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 178/334 kB Progress (1): 195/334 kB Progress (1): 211/334 kB Progress (1): 227/334 kB Progress (1): 244/334 kB Progress (1): 260/334 kB Progress (1): 277/334 kB Progress (1): 293/334 kB Progress (1): 309/334 kB Progress (1): 326/334 kB Progress (1): 334 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.jar (334 kB at 6.6 MB/s) 2024-05-30T16:12:35.805 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 4.7 kB Progress (2): 4.7 kB | 16/267 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.jar (4.7 kB at 90 kB/s) 2024-05-30T16:12:35.806 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (2): 16/267 kB | 16/23 kB Progress (2): 16/267 kB | 23 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.jar (23 kB at 430 kB/s) 2024-05-30T16:12:35.858 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 33/267 kB Progress (1): 49/267 kB Progress (1): 65/267 kB Progress (1): 81/267 kB Progress (1): 98/267 kB Progress (1): 114/267 kB Progress (1): 130/267 kB Progress (1): 147/267 kB Progress (1): 163/267 kB Progress (1): 180/267 kB Progress (1): 196/267 kB Progress (1): 212/267 kB Progress (1): 229/267 kB Progress (1): 245/267 kB Progress (1): 262/267 kB Progress (1): 267 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.jar (267 kB at 2.5 MB/s) 2024-05-30T16:12:36.045 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] Changes detected - recompiling the module! :source 2024-05-30T16:12:36.048 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] Compiling 1 source file with javac [debug target 1.9] to target/classes 2024-05-30T16:12:37.330 INFO:tasks.workunit.client.0.smithi006.stdout:[WARNING] bootstrap class path not set in conjunction with -source 9 2024-05-30T16:12:37.330 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] 2024-05-30T16:12:37.330 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] --- resources:3.3.1:testResources (default-testResources) @ jcksum --- 2024-05-30T16:12:37.334 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/test/resources 2024-05-30T16:12:37.334 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] 2024-05-30T16:12:37.334 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] --- compiler:3.11.0:testCompile (default-testCompile) @ jcksum --- 2024-05-30T16:12:37.341 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] Changes detected - recompiling the module! :dependency 2024-05-30T16:12:37.341 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] Compiling 1 source file with javac [debug target 1.9] to target/test-classes 2024-05-30T16:12:37.573 INFO:tasks.workunit.client.0.smithi006.stdout:[WARNING] bootstrap class path not set in conjunction with -source 9 2024-05-30T16:12:37.574 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] 2024-05-30T16:12:37.574 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] --- surefire:3.2.3:test (default-test) @ jcksum --- 2024-05-30T16:12:37.580 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.pom 2024-05-30T16:12:37.601 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 6.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.pom (6.2 kB at 297 kB/s) 2024-05-30T16:12:37.606 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.pom 2024-05-30T16:12:37.627 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.pom (3.5 kB at 160 kB/s) 2024-05-30T16:12:37.631 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.pom 2024-05-30T16:12:37.653 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 3.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.pom (3.3 kB at 148 kB/s) 2024-05-30T16:12:37.658 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.pom 2024-05-30T16:12:37.680 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 4.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.pom (4.1 kB at 184 kB/s) 2024-05-30T16:12:37.684 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.pom 2024-05-30T16:12:37.705 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 3.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.pom (3.3 kB at 157 kB/s) 2024-05-30T16:12:37.710 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.pom 2024-05-30T16:12:37.733 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 4.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.pom (4.5 kB at 194 kB/s) 2024-05-30T16:12:37.737 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.pom 2024-05-30T16:12:37.759 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 1.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.pom (1.8 kB at 80 kB/s) 2024-05-30T16:12:37.763 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.pom 2024-05-30T16:12:37.785 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.pom (2.2 kB at 100 kB/s) 2024-05-30T16:12:37.787 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether/1.0.0.v20140518/aether-1.0.0.v20140518.pom 2024-05-30T16:12:37.813 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/30 kB Progress (1): 30 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether/1.0.0.v20140518/aether-1.0.0.v20140518.pom (30 kB at 1.2 MB/s) 2024-05-30T16:12:37.816 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.pom 2024-05-30T16:12:37.838 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 1.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.pom (1.9 kB at 86 kB/s) 2024-05-30T16:12:37.842 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.pom 2024-05-30T16:12:37.862 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 5.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.pom (5.8 kB at 290 kB/s) 2024-05-30T16:12:37.867 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-artifact/3.2.5/maven-artifact-3.2.5.pom 2024-05-30T16:12:37.891 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 2.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-artifact/3.2.5/maven-artifact-3.2.5.pom (2.3 kB at 98 kB/s) 2024-05-30T16:12:37.894 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven/3.2.5/maven-3.2.5.pom 2024-05-30T16:12:37.918 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/22 kB Progress (1): 22 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven/3.2.5/maven-3.2.5.pom (22 kB at 932 kB/s) 2024-05-30T16:12:37.920 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/25/maven-parent-25.pom 2024-05-30T16:12:37.947 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/37 kB Progress (1): 33/37 kB Progress (1): 37 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/25/maven-parent-25.pom (37 kB at 1.4 MB/s) 2024-05-30T16:12:37.949 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/15/apache-15.pom 2024-05-30T16:12:37.976 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/15/apache-15.pom (15 kB at 564 kB/s) 2024-05-30T16:12:37.980 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/4.0.0/plexus-utils-4.0.0.pom 2024-05-30T16:12:38.000 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 8.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/4.0.0/plexus-utils-4.0.0.pom (8.7 kB at 412 kB/s) 2024-05-30T16:12:38.002 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/13/plexus-13.pom 2024-05-30T16:12:38.026 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/27 kB Progress (1): 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/13/plexus-13.pom (27 kB at 1.2 MB/s) 2024-05-30T16:12:38.030 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-core/3.2.5/maven-core-3.2.5.pom 2024-05-30T16:12:38.051 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 8.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-core/3.2.5/maven-core-3.2.5.pom (8.1 kB at 384 kB/s) 2024-05-30T16:12:38.053 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings/3.2.5/maven-settings-3.2.5.pom 2024-05-30T16:12:38.073 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings/3.2.5/maven-settings-3.2.5.pom (2.2 kB at 109 kB/s) 2024-05-30T16:12:38.076 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings-builder/3.2.5/maven-settings-builder-3.2.5.pom 2024-05-30T16:12:38.096 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 2.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings-builder/3.2.5/maven-settings-builder-3.2.5.pom (2.6 kB at 124 kB/s) 2024-05-30T16:12:38.099 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.21/plexus-interpolation-1.21.pom 2024-05-30T16:12:38.119 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 1.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.21/plexus-interpolation-1.21.pom (1.5 kB at 77 kB/s) 2024-05-30T16:12:38.122 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/1.3.1/plexus-components-1.3.1.pom 2024-05-30T16:12:38.147 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 3.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/1.3.1/plexus-components-1.3.1.pom (3.1 kB at 123 kB/s) 2024-05-30T16:12:38.150 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/3.3.1/plexus-3.3.1.pom 2024-05-30T16:12:38.173 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/3.3.1/plexus-3.3.1.pom (20 kB at 852 kB/s) 2024-05-30T16:12:38.179 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/17/spice-parent-17.pom 2024-05-30T16:12:38.201 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 6.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/17/spice-parent-17.pom (6.8 kB at 307 kB/s) 2024-05-30T16:12:38.206 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/10/forge-parent-10.pom 2024-05-30T16:12:38.228 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/10/forge-parent-10.pom (14 kB at 616 kB/s) 2024-05-30T16:12:38.235 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-sec-dispatcher/1.3/plexus-sec-dispatcher-1.3.pom 2024-05-30T16:12:38.256 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-sec-dispatcher/1.3/plexus-sec-dispatcher-1.3.pom (3.0 kB at 141 kB/s) 2024-05-30T16:12:38.258 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/12/spice-parent-12.pom 2024-05-30T16:12:38.279 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 6.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/12/spice-parent-12.pom (6.8 kB at 340 kB/s) 2024-05-30T16:12:38.281 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/4/forge-parent-4.pom 2024-05-30T16:12:38.303 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/4/forge-parent-4.pom (8.4 kB at 365 kB/s) 2024-05-30T16:12:38.308 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-cipher/1.4/plexus-cipher-1.4.pom 2024-05-30T16:12:38.330 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 2.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-cipher/1.4/plexus-cipher-1.4.pom (2.1 kB at 94 kB/s) 2024-05-30T16:12:38.336 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-repository-metadata/3.2.5/maven-repository-metadata-3.2.5.pom 2024-05-30T16:12:38.360 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-repository-metadata/3.2.5/maven-repository-metadata-3.2.5.pom (2.2 kB at 93 kB/s) 2024-05-30T16:12:38.371 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-plugin-api/3.2.5/maven-plugin-api-3.2.5.pom 2024-05-30T16:12:38.393 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-plugin-api/3.2.5/maven-plugin-api-3.2.5.pom (3.0 kB at 138 kB/s) 2024-05-30T16:12:38.401 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.plexus/0.9.0.M2/org.eclipse.sisu.plexus-0.9.0.M2.pom 2024-05-30T16:12:38.426 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.plexus/0.9.0.M2/org.eclipse.sisu.plexus-0.9.0.M2.pom (15 kB at 599 kB/s) 2024-05-30T16:12:38.428 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-plexus/0.9.0.M2/sisu-plexus-0.9.0.M2.pom 2024-05-30T16:12:38.451 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-plexus/0.9.0.M2/sisu-plexus-0.9.0.M2.pom (15 kB at 631 kB/s) 2024-05-30T16:12:38.457 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/annotation/javax.annotation-api/1.2/javax.annotation-api-1.2.pom 2024-05-30T16:12:38.480 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/javax/annotation/javax.annotation-api/1.2/javax.annotation-api-1.2.pom (13 kB at 584 kB/s) 2024-05-30T16:12:38.484 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/net/java/jvnet-parent/3/jvnet-parent-3.pom 2024-05-30T16:12:38.505 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 4.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/net/java/jvnet-parent/3/jvnet-parent-3.pom (4.8 kB at 218 kB/s) 2024-05-30T16:12:38.510 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/enterprise/cdi-api/1.2/cdi-api-1.2.pom 2024-05-30T16:12:38.532 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 6.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/javax/enterprise/cdi-api/1.2/cdi-api-1.2.pom (6.3 kB at 273 kB/s) 2024-05-30T16:12:38.536 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/jboss/weld/weld-parent/26/weld-parent-26.pom 2024-05-30T16:12:38.562 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/32 kB Progress (1): 32 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/jboss/weld/weld-parent/26/weld-parent-26.pom (32 kB at 1.3 MB/s) 2024-05-30T16:12:38.568 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.inject/0.9.0.M2/org.eclipse.sisu.inject-0.9.0.M2.pom 2024-05-30T16:12:38.592 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.inject/0.9.0.M2/org.eclipse.sisu.inject-0.9.0.M2.pom (17 kB at 723 kB/s) 2024-05-30T16:12:38.596 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-inject/0.9.0.M2/sisu-inject-0.9.0.M2.pom 2024-05-30T16:12:38.622 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-inject/0.9.0.M2/sisu-inject-0.9.0.M2.pom (15 kB at 643 kB/s) 2024-05-30T16:12:38.625 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.0/plexus-component-annotations-2.1.0.pom 2024-05-30T16:12:38.647 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 750 B Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.0/plexus-component-annotations-2.1.0.pom (750 B at 36 kB/s) 2024-05-30T16:12:38.650 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.0/plexus-containers-2.1.0.pom 2024-05-30T16:12:38.670 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 4.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.0/plexus-containers-2.1.0.pom (4.8 kB at 240 kB/s) 2024-05-30T16:12:38.675 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.6.0/plexus-classworlds-2.6.0.pom 2024-05-30T16:12:38.697 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 7.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.6.0/plexus-classworlds-2.6.0.pom (7.9 kB at 360 kB/s) 2024-05-30T16:12:38.703 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-model-builder/3.2.5/maven-model-builder-3.2.5.pom 2024-05-30T16:12:38.724 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-model-builder/3.2.5/maven-model-builder-3.2.5.pom (3.0 kB at 143 kB/s) 2024-05-30T16:12:38.731 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-aether-provider/3.2.5/maven-aether-provider-3.2.5.pom 2024-05-30T16:12:38.749 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 4.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-aether-provider/3.2.5/maven-aether-provider-3.2.5.pom (4.2 kB at 224 kB/s) 2024-05-30T16:12:38.758 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-spi/1.0.0.v20140518/aether-spi-1.0.0.v20140518.pom 2024-05-30T16:12:38.777 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 2.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-spi/1.0.0.v20140518/aether-spi-1.0.0.v20140518.pom (2.1 kB at 108 kB/s) 2024-05-30T16:12:38.783 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-impl/1.0.0.v20140518/aether-impl-1.0.0.v20140518.pom 2024-05-30T16:12:38.802 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-impl/1.0.0.v20140518/aether-impl-1.0.0.v20140518.pom (3.5 kB at 183 kB/s) 2024-05-30T16:12:38.809 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/sisu-guice/3.2.3/sisu-guice-3.2.3.pom 2024-05-30T16:12:38.828 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 11 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/sisu-guice/3.2.3/sisu-guice-3.2.3.pom (11 kB at 576 kB/s) 2024-05-30T16:12:38.832 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/inject/guice-parent/3.2.3/guice-parent-3.2.3.pom 2024-05-30T16:12:38.853 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/inject/guice-parent/3.2.3/guice-parent-3.2.3.pom (13 kB at 642 kB/s) 2024-05-30T16:12:38.857 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/38/forge-parent-38.pom 2024-05-30T16:12:38.877 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/19 kB Progress (1): 19 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/38/forge-parent-38.pom (19 kB at 937 kB/s) 2024-05-30T16:12:38.883 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/aopalliance/aopalliance/1.0/aopalliance-1.0.pom 2024-05-30T16:12:38.901 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 363 B Downloaded from central: https://repo.maven.apache.org/maven2/aopalliance/aopalliance/1.0/aopalliance-1.0.pom (363 B at 20 kB/s) 2024-05-30T16:12:38.905 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/google/guava/guava/16.0.1/guava-16.0.1.pom 2024-05-30T16:12:38.926 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 6.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/google/guava/guava/16.0.1/guava-16.0.1.pom (6.1 kB at 291 kB/s) 2024-05-30T16:12:38.930 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/google/guava/guava-parent/16.0.1/guava-parent-16.0.1.pom 2024-05-30T16:12:38.949 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 7.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/google/guava/guava-parent/16.0.1/guava-parent-16.0.1.pom (7.3 kB at 386 kB/s) 2024-05-30T16:12:38.953 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/7/oss-parent-7.pom 2024-05-30T16:12:38.975 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 4.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/7/oss-parent-7.pom (4.8 kB at 219 kB/s) 2024-05-30T16:12:38.980 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.5.2/plexus-classworlds-2.5.2.pom 2024-05-30T16:12:39.002 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 7.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.5.2/plexus-classworlds-2.5.2.pom (7.3 kB at 332 kB/s) 2024-05-30T16:12:39.008 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.pom 2024-05-30T16:12:39.032 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/22 kB Progress (1): 22 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.pom (22 kB at 902 kB/s) 2024-05-30T16:12:39.037 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/64/commons-parent-64.pom 2024-05-30T16:12:39.069 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/78 kB Progress (1): 33/78 kB Progress (1): 49/78 kB Progress (1): 66/78 kB Progress (1): 78 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/64/commons-parent-64.pom (78 kB at 2.4 MB/s) 2024-05-30T16:12:39.077 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/30/apache-30.pom 2024-05-30T16:12:39.100 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/23 kB Progress (1): 23 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/30/apache-30.pom (23 kB at 968 kB/s) 2024-05-30T16:12:39.107 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.0/junit-bom-5.10.0.pom 2024-05-30T16:12:39.127 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.0/junit-bom-5.10.0.pom (5.6 kB at 269 kB/s) 2024-05-30T16:12:39.132 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.pom 2024-05-30T16:12:39.152 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 4.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.pom (4.3 kB at 214 kB/s) 2024-05-30T16:12:39.156 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.2.0/plexus-languages-1.2.0.pom 2024-05-30T16:12:39.174 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.2.0/plexus-languages-1.2.0.pom (3.2 kB at 177 kB/s) 2024-05-30T16:12:39.178 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/15/plexus-15.pom 2024-05-30T16:12:39.201 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/28 kB Progress (1): 28 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/15/plexus-15.pom (28 kB at 1.2 MB/s) 2024-05-30T16:12:39.207 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.pom 2024-05-30T16:12:39.226 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 2.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.pom (2.4 kB at 132 kB/s) 2024-05-30T16:12:39.239 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.jar 2024-05-30T16:12:39.298 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/308 kB Progress (1): 33/308 kB Progress (1): 49/308 kB Progress (1): 62/308 kB Progress (1): 78/308 kB Progress (1): 95/308 kB Progress (1): 111/308 kB Progress (1): 128/308 kB Progress (1): 144/308 kB Progress (1): 160/308 kB Progress (1): 177/308 kB Progress (1): 193/308 kB Progress (1): 209/308 kB Progress (1): 226/308 kB Progress (1): 242/308 kB Progress (1): 259/308 kB Progress (1): 275/308 kB Progress (1): 291/308 kB Progress (1): 308/308 kB Progress (1): 308 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.jar (308 kB at 5.2 MB/s) 2024-05-30T16:12:39.299 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.jar 2024-05-30T16:12:39.299 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.jar 2024-05-30T16:12:39.300 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.jar 2024-05-30T16:12:39.300 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.jar 2024-05-30T16:12:39.300 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.jar 2024-05-30T16:12:39.322 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/171 kB Progress (2): 16/171 kB | 14 kB Progress (2): 33/171 kB | 14 kB Progress (3): 33/171 kB | 14 kB | 16/118 kB Progress (4): 33/171 kB | 14 kB | 16/118 kB | 16/26 kB Progress (4): 33/171 kB | 14 kB | 33/118 kB | 16/26 kB Progress (4): 33/171 kB | 14 kB | 33/118 kB | 26 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.jar (14 kB at 591 kB/s) 2024-05-30T16:12:39.322 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.jar 2024-05-30T16:12:39.323 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (4): 33/171 kB | 33/118 kB | 26 kB | 8.2 kB Progress (4): 49/171 kB | 33/118 kB | 26 kB | 8.2 kB Progress (4): 49/171 kB | 49/118 kB | 26 kB | 8.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.jar (26 kB at 1.1 MB/s) 2024-05-30T16:12:39.323 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.jar 2024-05-30T16:12:39.324 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (3): 49/171 kB | 66/118 kB | 8.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.jar (8.2 kB at 341 kB/s) 2024-05-30T16:12:39.324 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.jar 2024-05-30T16:12:39.327 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (2): 65/171 kB | 66/118 kB Progress (2): 65/171 kB | 82/118 kB Progress (2): 65/171 kB | 98/118 kB Progress (2): 81/171 kB | 98/118 kB Progress (2): 81/171 kB | 115/118 kB Progress (2): 81/171 kB | 118 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.jar (118 kB at 4.4 MB/s) 2024-05-30T16:12:39.327 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.jar 2024-05-30T16:12:39.340 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 98/171 kB Progress (1): 114/171 kB Progress (1): 130/171 kB Progress (1): 147/171 kB Progress (1): 163/171 kB Progress (1): 171 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.jar (171 kB at 4.1 MB/s) 2024-05-30T16:12:39.340 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.jar 2024-05-30T16:12:39.348 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/146 kB Progress (2): 16/146 kB | 16/136 kB Progress (2): 16/146 kB | 33/136 kB Progress (2): 16/146 kB | 49/136 kB Progress (2): 33/146 kB | 49/136 kB Progress (2): 33/146 kB | 63/136 kB Progress (3): 33/146 kB | 63/136 kB | 16/61 kB Progress (3): 33/146 kB | 79/136 kB | 16/61 kB Progress (3): 33/146 kB | 79/136 kB | 33/61 kB Progress (3): 33/146 kB | 96/136 kB | 33/61 kB Progress (3): 33/146 kB | 112/136 kB | 33/61 kB Progress (3): 33/146 kB | 112/136 kB | 49/61 kB Progress (3): 33/146 kB | 129/136 kB | 49/61 kB Progress (3): 49/146 kB | 129/136 kB | 49/61 kB Progress (3): 49/146 kB | 136 kB | 49/61 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.jar (136 kB at 2.9 MB/s) 2024-05-30T16:12:39.348 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.jar 2024-05-30T16:12:39.350 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (2): 66/146 kB | 49/61 kB Progress (2): 66/146 kB | 61 kB Progress (2): 82/146 kB | 61 kB Progress (3): 82/146 kB | 61 kB | 16/501 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.jar (61 kB at 1.2 MB/s) 2024-05-30T16:12:39.350 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.jar 2024-05-30T16:12:39.356 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (2): 82/146 kB | 32/501 kB Progress (2): 82/146 kB | 49/501 kB Progress (2): 82/146 kB | 65/501 kB Progress (2): 98/146 kB | 65/501 kB Progress (2): 98/146 kB | 81/501 kB Progress (2): 98/146 kB | 98/501 kB Progress (2): 115/146 kB | 98/501 kB Progress (2): 115/146 kB | 114/501 kB Progress (2): 115/146 kB | 130/501 kB Progress (2): 131/146 kB | 130/501 kB Progress (2): 131/146 kB | 147/501 kB Progress (2): 131/146 kB | 163/501 kB Progress (2): 146 kB | 163/501 kB Progress (2): 146 kB | 180/501 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.jar (146 kB at 2.7 MB/s) 2024-05-30T16:12:39.364 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 193/501 kB Progress (1): 209/501 kB Progress (1): 226/501 kB Progress (1): 242/501 kB Progress (1): 259/501 kB Progress (2): 259/501 kB | 16/58 kB Progress (2): 275/501 kB | 16/58 kB Progress (2): 291/501 kB | 16/58 kB Progress (2): 291/501 kB | 33/58 kB Progress (2): 308/501 kB | 33/58 kB Progress (2): 324/501 kB | 33/58 kB Progress (2): 324/501 kB | 49/58 kB Progress (2): 324/501 kB | 58 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.jar (58 kB at 899 kB/s) 2024-05-30T16:12:39.370 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (2): 324/501 kB | 16/124 kB Progress (2): 341/501 kB | 16/124 kB Progress (2): 341/501 kB | 33/124 kB Progress (2): 357/501 kB | 33/124 kB Progress (2): 357/501 kB | 49/124 kB Progress (2): 373/501 kB | 49/124 kB Progress (2): 373/501 kB | 66/124 kB Progress (2): 390/501 kB | 66/124 kB Progress (2): 390/501 kB | 82/124 kB Progress (2): 390/501 kB | 98/124 kB Progress (2): 406/501 kB | 98/124 kB Progress (2): 406/501 kB | 115/124 kB Progress (2): 406/501 kB | 124 kB Progress (2): 422/501 kB | 124 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.jar (124 kB at 1.8 MB/s) 2024-05-30T16:12:39.374 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 439/501 kB Progress (2): 439/501 kB | 0/2.4 MB Progress (2): 455/501 kB | 0/2.4 MB Progress (2): 455/501 kB | 0/2.4 MB Progress (2): 455/501 kB | 0/2.4 MB Progress (2): 472/501 kB | 0/2.4 MB Progress (2): 472/501 kB | 0.1/2.4 MB Progress (2): 488/501 kB | 0.1/2.4 MB Progress (2): 501 kB | 0.1/2.4 MB Progress (2): 501 kB | 0.1/2.4 MB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.jar (501 kB at 6.9 MB/s) 2024-05-30T16:12:39.507 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 0.1/2.4 MB Progress (1): 0.1/2.4 MB Progress (1): 0.1/2.4 MB Progress (1): 0.1/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4 MB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.jar (2.4 MB at 12 MB/s) 2024-05-30T16:12:39.646 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] 2024-05-30T16:12:39.646 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] --- jar:3.3.0:jar (default-jar) @ jcksum --- 2024-05-30T16:12:39.650 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.pom 2024-05-30T16:12:39.671 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 4.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.pom (4.5 kB at 214 kB/s) 2024-05-30T16:12:39.674 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/36/maven-shared-components-36.pom 2024-05-30T16:12:39.695 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 4.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/36/maven-shared-components-36.pom (4.9 kB at 233 kB/s) 2024-05-30T16:12:39.697 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/36/maven-parent-36.pom 2024-05-30T16:12:39.721 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/45 kB Progress (1): 33/45 kB Progress (1): 45 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/36/maven-parent-36.pom (45 kB at 1.9 MB/s) 2024-05-30T16:12:39.725 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/26/apache-26.pom 2024-05-30T16:12:39.748 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/21 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/26/apache-26.pom (21 kB at 893 kB/s) 2024-05-30T16:12:39.752 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.3.0/plexus-utils-3.3.0.pom 2024-05-30T16:12:39.773 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 5.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.3.0/plexus-utils-3.3.0.pom (5.2 kB at 247 kB/s) 2024-05-30T16:12:39.776 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.pom 2024-05-30T16:12:39.797 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 3.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.pom (3.9 kB at 187 kB/s) 2024-05-30T16:12:39.801 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.pom 2024-05-30T16:12:39.823 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 6.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.pom (6.0 kB at 273 kB/s) 2024-05-30T16:12:39.827 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.pom 2024-05-30T16:12:39.849 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 6.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.pom (6.3 kB at 285 kB/s) 2024-05-30T16:12:39.853 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.pom 2024-05-30T16:12:39.874 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.pom (20 kB at 895 kB/s) 2024-05-30T16:12:39.883 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.pom 2024-05-30T16:12:39.905 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.pom (15 kB at 661 kB/s) 2024-05-30T16:12:39.910 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.pom 2024-05-30T16:12:39.931 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 2.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.pom (2.0 kB at 97 kB/s) 2024-05-30T16:12:39.934 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.pom 2024-05-30T16:12:39.956 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 8.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.pom (8.2 kB at 391 kB/s) 2024-05-30T16:12:39.959 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/8/plexus-8.pom 2024-05-30T16:12:39.980 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/25 kB Progress (1): 25 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/8/plexus-8.pom (25 kB at 1.2 MB/s) 2024-05-30T16:12:39.989 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.jar 2024-05-30T16:12:40.012 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/36 kB Progress (1): 33/36 kB Progress (1): 36 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.jar (36 kB at 1.6 MB/s) 2024-05-30T16:12:40.012 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.jar 2024-05-30T16:12:40.012 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.jar 2024-05-30T16:12:40.013 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.jar 2024-05-30T16:12:40.014 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.jar 2024-05-30T16:12:40.014 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.jar 2024-05-30T16:12:40.034 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/26 kB Progress (2): 16/26 kB | 16/79 kB Progress (2): 26 kB | 16/79 kB Progress (3): 26 kB | 16/79 kB | 16/211 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.jar (26 kB at 1.1 MB/s) 2024-05-30T16:12:40.035 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (2): 16/79 kB | 33/211 kB Downloading from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.jar 2024-05-30T16:12:40.038 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (2): 33/79 kB | 33/211 kB Progress (3): 33/79 kB | 33/211 kB | 16/58 kB Progress (3): 33/79 kB | 49/211 kB | 16/58 kB Progress (3): 33/79 kB | 62/211 kB | 16/58 kB Progress (3): 49/79 kB | 62/211 kB | 16/58 kB Progress (3): 49/79 kB | 62/211 kB | 32/58 kB Progress (3): 49/79 kB | 78/211 kB | 32/58 kB Progress (3): 49/79 kB | 95/211 kB | 32/58 kB Progress (3): 49/79 kB | 95/211 kB | 49/58 kB Progress (3): 49/79 kB | 111/211 kB | 49/58 kB Progress (3): 66/79 kB | 111/211 kB | 49/58 kB Progress (3): 79 kB | 111/211 kB | 49/58 kB Progress (4): 79 kB | 111/211 kB | 49/58 kB | 0/1.0 MB Progress (4): 79 kB | 128/211 kB | 49/58 kB | 0/1.0 MB Progress (4): 79 kB | 128/211 kB | 58 kB | 0/1.0 MB Progress (4): 79 kB | 144/211 kB | 58 kB | 0/1.0 MB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.jar (79 kB at 3.1 MB/s) 2024-05-30T16:12:40.038 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.jar 2024-05-30T16:12:40.038 INFO:tasks.workunit.client.0.smithi006.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.jar (58 kB at 2.3 MB/s) 2024-05-30T16:12:40.039 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (2): 160/211 kB | 0/1.0 MB Progress (2): 177/211 kB | 0/1.0 MB Progress (2): 193/211 kB | 0/1.0 MB Progress (2): 193/211 kB | 0/1.0 MB Progress (2): 209/211 kB | 0/1.0 MB Progress (2): 211 kB | 0/1.0 MB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.jar (211 kB at 7.8 MB/s) 2024-05-30T16:12:40.060 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 0/1.0 MB Progress (1): 0.1/1.0 MB Progress (1): 0.1/1.0 MB Progress (1): 0.1/1.0 MB Progress (1): 0.1/1.0 MB Progress (1): 0.1/1.0 MB Progress (1): 0.1/1.0 MB Progress (2): 0.1/1.0 MB | 16/116 kB Progress (2): 0.1/1.0 MB | 33/116 kB Progress (2): 0.2/1.0 MB | 33/116 kB Progress (2): 0.2/1.0 MB | 49/116 kB Progress (2): 0.2/1.0 MB | 66/116 kB Progress (2): 0.2/1.0 MB | 82/116 kB Progress (2): 0.2/1.0 MB | 82/116 kB Progress (2): 0.2/1.0 MB | 98/116 kB Progress (3): 0.2/1.0 MB | 98/116 kB | 16/267 kB Progress (3): 0.2/1.0 MB | 115/116 kB | 16/267 kB Progress (3): 0.2/1.0 MB | 116 kB | 16/267 kB Progress (3): 0.2/1.0 MB | 116 kB | 16/267 kB Progress (3): 0.2/1.0 MB | 116 kB | 33/267 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.jar (116 kB at 2.5 MB/s) 2024-05-30T16:12:40.082 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (2): 0.2/1.0 MB | 49/267 kB Progress (2): 0.2/1.0 MB | 49/267 kB Progress (2): 0.2/1.0 MB | 63/267 kB Progress (2): 0.2/1.0 MB | 63/267 kB Progress (2): 0.2/1.0 MB | 79/267 kB Progress (2): 0.2/1.0 MB | 79/267 kB Progress (2): 0.2/1.0 MB | 96/267 kB Progress (2): 0.3/1.0 MB | 96/267 kB Progress (2): 0.3/1.0 MB | 112/267 kB Progress (2): 0.3/1.0 MB | 129/267 kB Progress (2): 0.3/1.0 MB | 129/267 kB Progress (2): 0.3/1.0 MB | 145/267 kB Progress (2): 0.3/1.0 MB | 145/267 kB Progress (2): 0.3/1.0 MB | 161/267 kB Progress (2): 0.3/1.0 MB | 178/267 kB Progress (2): 0.3/1.0 MB | 178/267 kB Progress (2): 0.3/1.0 MB | 178/267 kB Progress (2): 0.3/1.0 MB | 178/267 kB Progress (2): 0.3/1.0 MB | 194/267 kB Progress (2): 0.3/1.0 MB | 210/267 kB Progress (2): 0.4/1.0 MB | 210/267 kB Progress (2): 0.4/1.0 MB | 227/267 kB Progress (2): 0.4/1.0 MB | 227/267 kB Progress (2): 0.4/1.0 MB | 243/267 kB Progress (2): 0.4/1.0 MB | 260/267 kB Progress (2): 0.4/1.0 MB | 267 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.jar (267 kB at 3.9 MB/s) 2024-05-30T16:12:40.152 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 0.4/1.0 MB Progress (1): 0.4/1.0 MB Progress (1): 0.4/1.0 MB Progress (1): 0.4/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 1.0/1.0 MB Progress (1): 1.0/1.0 MB Progress (1): 1.0/1.0 MB Progress (1): 1.0/1.0 MB Progress (1): 1.0 MB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.jar (1.0 MB at 7.3 MB/s) 2024-05-30T16:12:40.374 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] Building jar: /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/target/jcksum-1.0.jar 2024-05-30T16:12:40.417 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] ------------------------------------------------------------------------ 2024-05-30T16:12:40.418 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] BUILD SUCCESS 2024-05-30T16:12:40.418 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] ------------------------------------------------------------------------ 2024-05-30T16:12:40.418 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] Total time: 11.176 s 2024-05-30T16:12:40.419 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] Finished at: 2024-05-30T16:12:40Z 2024-05-30T16:12:40.419 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] ------------------------------------------------------------------------ 2024-05-30T16:12:40.449 INFO:tasks.workunit.client.0.smithi006.stderr:+ ./mvnw test -Dtest=PutObjects 2024-05-30T16:12:40.454 INFO:tasks.workunit.client.0.smithi006.stdout:Warning: JAVA_HOME environment variable is not set. 2024-05-30T16:12:41.757 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] Scanning for projects... 2024-05-30T16:12:41.838 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] 2024-05-30T16:12:41.839 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] ---------------------------< io.ceph:jcksum >--------------------------- 2024-05-30T16:12:41.839 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] Building jcksum 1.0 2024-05-30T16:12:41.839 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] from pom.xml 2024-05-30T16:12:41.839 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] --------------------------------[ jar ]--------------------------------- 2024-05-30T16:12:42.109 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] 2024-05-30T16:12:42.110 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] --- resources:3.3.1:resources (default-resources) @ jcksum --- 2024-05-30T16:12:42.239 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/main/resources 2024-05-30T16:12:42.239 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] 2024-05-30T16:12:42.239 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] --- compiler:3.11.0:compile (default-compile) @ jcksum --- 2024-05-30T16:12:42.366 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] Nothing to compile - all classes are up to date 2024-05-30T16:12:42.366 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] 2024-05-30T16:12:42.367 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] --- resources:3.3.1:testResources (default-testResources) @ jcksum --- 2024-05-30T16:12:42.369 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/test/resources 2024-05-30T16:12:42.370 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] 2024-05-30T16:12:42.370 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] --- compiler:3.11.0:testCompile (default-testCompile) @ jcksum --- 2024-05-30T16:12:42.378 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] Nothing to compile - all classes are up to date 2024-05-30T16:12:42.378 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] 2024-05-30T16:12:42.378 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] --- surefire:3.2.3:test (default-test) @ jcksum --- 2024-05-30T16:12:42.594 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] Using auto detected provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider 2024-05-30T16:12:42.859 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.pom 2024-05-30T16:12:43.086 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 1.4/4.7 kB Progress (1): 2.8/4.7 kB Progress (1): 4.1/4.7 kB Progress (1): 4.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.pom (4.7 kB at 20 kB/s) 2024-05-30T16:12:43.095 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-providers/3.2.3/surefire-providers-3.2.3.pom 2024-05-30T16:12:43.121 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 1.4/2.6 kB Progress (1): 2.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-providers/3.2.3/surefire-providers-3.2.3.pom (2.6 kB at 97 kB/s) 2024-05-30T16:12:43.127 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.pom 2024-05-30T16:12:43.153 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 1.4/2.8 kB Progress (1): 2.8/2.8 kB Progress (1): 2.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.pom (2.8 kB at 113 kB/s) 2024-05-30T16:12:43.158 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.pom 2024-05-30T16:12:43.183 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 1.4/3.0 kB Progress (1): 2.8/3.0 kB Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.pom (3.0 kB at 126 kB/s) 2024-05-30T16:12:43.187 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.pom 2024-05-30T16:12:43.212 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 1.4/3.2 kB Progress (1): 2.8/3.2 kB Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.pom (3.2 kB at 123 kB/s) 2024-05-30T16:12:43.216 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.pom 2024-05-30T16:12:43.240 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 1.4/1.7 kB Progress (1): 1.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.pom (1.7 kB at 70 kB/s) 2024-05-30T16:12:43.243 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.pom 2024-05-30T16:12:43.267 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 1.4/2.8 kB Progress (1): 2.8/2.8 kB Progress (1): 2.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.pom (2.8 kB at 118 kB/s) 2024-05-30T16:12:43.273 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.jar 2024-05-30T16:12:43.306 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 1.4/27 kB Progress (1): 2.8/27 kB Progress (1): 4.1/27 kB Progress (1): 5.5/27 kB Progress (1): 6.9/27 kB Progress (1): 8.3/27 kB Progress (1): 9.7/27 kB Progress (1): 11/27 kB Progress (1): 12/27 kB Progress (1): 14/27 kB Progress (1): 15/27 kB Progress (1): 17/27 kB Progress (1): 18/27 kB Progress (1): 19/27 kB Progress (1): 21/27 kB Progress (1): 22/27 kB Progress (1): 23/27 kB Progress (1): 25/27 kB Progress (1): 26/27 kB Progress (1): 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.jar (27 kB at 823 kB/s) 2024-05-30T16:12:43.308 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.jar 2024-05-30T16:12:43.309 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.jar 2024-05-30T16:12:43.309 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.jar 2024-05-30T16:12:43.309 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar 2024-05-30T16:12:43.318 INFO:tasks.workunit.client.0.smithi006.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.jar 2024-05-30T16:12:43.346 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 16/18 kB Progress (1): 18 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.jar (18 kB at 450 kB/s) 2024-05-30T16:12:43.390 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 1.4/169 kB Progress (1): 2.8/169 kB Progress (2): 2.8/169 kB | 1.4/7.7 kB Progress (2): 4.1/169 kB | 1.4/7.7 kB Progress (2): 4.1/169 kB | 2.8/7.7 kB Progress (2): 4.1/169 kB | 4.1/7.7 kB Progress (2): 5.5/169 kB | 4.1/7.7 kB Progress (2): 5.5/169 kB | 5.5/7.7 kB Progress (2): 6.9/169 kB | 5.5/7.7 kB Progress (2): 6.9/169 kB | 6.9/7.7 kB Progress (3): 6.9/169 kB | 6.9/7.7 kB | 1.4/103 kB Progress (3): 6.9/169 kB | 7.7 kB | 1.4/103 kB Progress (3): 8.3/169 kB | 7.7 kB | 1.4/103 kB Progress (4): 8.3/169 kB | 7.7 kB | 1.4/103 kB | 1.4/189 kB Progress (4): 9.7/169 kB | 7.7 kB | 1.4/103 kB | 1.4/189 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar (7.7 kB at 93 kB/s) 2024-05-30T16:12:43.440 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (3): 9.7/169 kB | 2.8/103 kB | 1.4/189 kB Progress (3): 11/169 kB | 2.8/103 kB | 1.4/189 kB Progress (3): 11/169 kB | 2.8/103 kB | 2.8/189 kB Progress (3): 12/169 kB | 2.8/103 kB | 2.8/189 kB Progress (3): 12/169 kB | 4.1/103 kB | 2.8/189 kB Progress (3): 14/169 kB | 4.1/103 kB | 2.8/189 kB Progress (3): 14/169 kB | 4.1/103 kB | 4.1/189 kB Progress (3): 15/169 kB | 4.1/103 kB | 4.1/189 kB Progress (3): 15/169 kB | 5.5/103 kB | 4.1/189 kB Progress (3): 16/169 kB | 5.5/103 kB | 4.1/189 kB Progress (3): 16/169 kB | 5.5/103 kB | 5.5/189 kB Progress (3): 18/169 kB | 5.5/103 kB | 5.5/189 kB Progress (3): 18/169 kB | 6.9/103 kB | 5.5/189 kB Progress (3): 19/169 kB | 6.9/103 kB | 5.5/189 kB Progress (3): 19/169 kB | 6.9/103 kB | 6.9/189 kB Progress (3): 21/169 kB | 6.9/103 kB | 6.9/189 kB Progress (3): 21/169 kB | 8.3/103 kB | 6.9/189 kB Progress (3): 22/169 kB | 8.3/103 kB | 6.9/189 kB Progress (3): 22/169 kB | 8.3/103 kB | 8.3/189 kB Progress (3): 23/169 kB | 8.3/103 kB | 8.3/189 kB Progress (3): 23/169 kB | 9.7/103 kB | 8.3/189 kB Progress (3): 25/169 kB | 9.7/103 kB | 8.3/189 kB Progress (3): 25/169 kB | 9.7/103 kB | 9.7/189 kB Progress (3): 25/169 kB | 11/103 kB | 9.7/189 kB Progress (3): 26/169 kB | 11/103 kB | 9.7/189 kB Progress (3): 26/169 kB | 11/103 kB | 11/189 kB Progress (3): 27/169 kB | 11/103 kB | 11/189 kB Progress (3): 27/169 kB | 12/103 kB | 11/189 kB Progress (3): 27/169 kB | 12/103 kB | 12/189 kB Progress (3): 29/169 kB | 12/103 kB | 12/189 kB Progress (3): 29/169 kB | 14/103 kB | 12/189 kB Progress (3): 30/169 kB | 14/103 kB | 12/189 kB Progress (3): 30/169 kB | 14/103 kB | 14/189 kB Progress (3): 32/169 kB | 14/103 kB | 14/189 kB Progress (3): 32/169 kB | 15/103 kB | 14/189 kB Progress (3): 33/169 kB | 15/103 kB | 14/189 kB Progress (3): 33/169 kB | 15/103 kB | 15/189 kB Progress (3): 34/169 kB | 15/103 kB | 15/189 kB Progress (3): 34/169 kB | 16/103 kB | 15/189 kB Progress (3): 34/169 kB | 16/103 kB | 16/189 kB Progress (3): 36/169 kB | 16/103 kB | 16/189 kB Progress (3): 36/169 kB | 18/103 kB | 16/189 kB Progress (3): 36/169 kB | 18/103 kB | 18/189 kB Progress (3): 37/169 kB | 18/103 kB | 18/189 kB Progress (3): 37/169 kB | 19/103 kB | 18/189 kB Progress (3): 37/169 kB | 19/103 kB | 19/189 kB Progress (3): 38/169 kB | 19/103 kB | 19/189 kB Progress (3): 38/169 kB | 21/103 kB | 19/189 kB Progress (3): 40/169 kB | 21/103 kB | 19/189 kB Progress (3): 40/169 kB | 21/103 kB | 21/189 kB Progress (3): 40/169 kB | 22/103 kB | 21/189 kB Progress (3): 41/169 kB | 22/103 kB | 21/189 kB Progress (3): 41/169 kB | 22/103 kB | 22/189 kB Progress (3): 41/169 kB | 23/103 kB | 22/189 kB Progress (3): 43/169 kB | 23/103 kB | 22/189 kB Progress (3): 43/169 kB | 23/103 kB | 23/189 kB Progress (3): 44/169 kB | 23/103 kB | 23/189 kB Progress (3): 44/169 kB | 25/103 kB | 23/189 kB Progress (3): 45/169 kB | 25/103 kB | 23/189 kB Progress (3): 45/169 kB | 25/103 kB | 25/189 kB Progress (3): 47/169 kB | 25/103 kB | 25/189 kB Progress (3): 47/169 kB | 26/103 kB | 25/189 kB Progress (3): 48/169 kB | 26/103 kB | 25/189 kB Progress (3): 48/169 kB | 26/103 kB | 26/189 kB Progress (3): 49/169 kB | 26/103 kB | 26/189 kB Progress (3): 49/169 kB | 27/103 kB | 26/189 kB Progress (3): 49/169 kB | 27/103 kB | 27/189 kB Progress (3): 51/169 kB | 27/103 kB | 27/189 kB Progress (3): 51/169 kB | 29/103 kB | 27/189 kB Progress (3): 51/169 kB | 29/103 kB | 29/189 kB Progress (3): 52/169 kB | 29/103 kB | 29/189 kB Progress (3): 52/169 kB | 30/103 kB | 29/189 kB Progress (3): 54/169 kB | 30/103 kB | 29/189 kB Progress (3): 54/169 kB | 30/103 kB | 30/189 kB Progress (3): 55/169 kB | 30/103 kB | 30/189 kB Progress (3): 55/169 kB | 32/103 kB | 30/189 kB Progress (3): 56/169 kB | 32/103 kB | 30/189 kB Progress (3): 56/169 kB | 33/103 kB | 30/189 kB Progress (3): 56/169 kB | 33/103 kB | 32/189 kB Progress (3): 58/169 kB | 33/103 kB | 32/189 kB Progress (3): 58/169 kB | 33/103 kB | 33/189 kB Progress (3): 58/169 kB | 34/103 kB | 33/189 kB Progress (3): 58/169 kB | 34/103 kB | 34/189 kB Progress (3): 59/169 kB | 34/103 kB | 34/189 kB Progress (3): 59/169 kB | 34/103 kB | 36/189 kB Progress (3): 59/169 kB | 36/103 kB | 36/189 kB Progress (3): 59/169 kB | 36/103 kB | 37/189 kB Progress (3): 61/169 kB | 36/103 kB | 37/189 kB Progress (3): 61/169 kB | 36/103 kB | 38/189 kB Progress (3): 61/169 kB | 37/103 kB | 38/189 kB Progress (3): 61/169 kB | 37/103 kB | 40/189 kB Progress (3): 62/169 kB | 37/103 kB | 40/189 kB Progress (3): 62/169 kB | 37/103 kB | 41/189 kB Progress (3): 62/169 kB | 38/103 kB | 41/189 kB Progress (3): 62/169 kB | 38/103 kB | 43/189 kB Progress (3): 63/169 kB | 38/103 kB | 43/189 kB Progress (3): 63/169 kB | 38/103 kB | 44/189 kB Progress (3): 63/169 kB | 40/103 kB | 44/189 kB Progress (3): 63/169 kB | 40/103 kB | 45/189 kB Progress (3): 64/169 kB | 40/103 kB | 45/189 kB Progress (3): 64/169 kB | 40/103 kB | 47/189 kB Progress (3): 64/169 kB | 41/103 kB | 47/189 kB Progress (3): 64/169 kB | 41/103 kB | 48/189 kB Progress (3): 64/169 kB | 43/103 kB | 48/189 kB Progress (3): 64/169 kB | 43/103 kB | 49/189 kB Progress (3): 80/169 kB | 43/103 kB | 49/189 kB Progress (3): 80/169 kB | 44/103 kB | 49/189 kB Progress (3): 80/169 kB | 44/103 kB | 51/189 kB Progress (3): 80/169 kB | 44/103 kB | 52/189 kB Progress (3): 80/169 kB | 45/103 kB | 52/189 kB Progress (3): 96/169 kB | 45/103 kB | 52/189 kB Progress (3): 96/169 kB | 45/103 kB | 53/189 kB Progress (3): 96/169 kB | 47/103 kB | 53/189 kB Progress (3): 96/169 kB | 47/103 kB | 55/189 kB Progress (3): 96/169 kB | 48/103 kB | 55/189 kB Progress (3): 96/169 kB | 48/103 kB | 56/189 kB Progress (3): 113/169 kB | 48/103 kB | 56/189 kB Progress (3): 113/169 kB | 50/103 kB | 56/189 kB Progress (3): 113/169 kB | 50/103 kB | 57/189 kB Progress (3): 113/169 kB | 51/103 kB | 57/189 kB Progress (3): 113/169 kB | 51/103 kB | 59/189 kB Progress (3): 113/169 kB | 52/103 kB | 59/189 kB Progress (3): 113/169 kB | 52/103 kB | 60/189 kB Progress (3): 129/169 kB | 52/103 kB | 60/189 kB Progress (3): 129/169 kB | 52/103 kB | 62/189 kB Progress (3): 129/169 kB | 54/103 kB | 62/189 kB Progress (3): 129/169 kB | 54/103 kB | 63/189 kB Progress (3): 129/169 kB | 55/103 kB | 63/189 kB Progress (3): 129/169 kB | 55/103 kB | 64/189 kB Progress (3): 146/169 kB | 55/103 kB | 64/189 kB Progress (3): 146/169 kB | 56/103 kB | 64/189 kB Progress (3): 146/169 kB | 58/103 kB | 64/189 kB Progress (3): 146/169 kB | 58/103 kB | 78/189 kB Progress (3): 162/169 kB | 58/103 kB | 78/189 kB Progress (3): 162/169 kB | 59/103 kB | 78/189 kB Progress (3): 162/169 kB | 61/103 kB | 78/189 kB Progress (3): 169 kB | 61/103 kB | 78/189 kB Progress (3): 169 kB | 61/103 kB | 94/189 kB Progress (3): 169 kB | 62/103 kB | 94/189 kB Progress (3): 169 kB | 63/103 kB | 94/189 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.jar (169 kB at 1.3 MB/s) 2024-05-30T16:12:43.446 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (2): 64/103 kB | 94/189 kB Progress (2): 81/103 kB | 94/189 kB Progress (2): 97/103 kB | 94/189 kB Progress (2): 103 kB | 94/189 kB Progress (2): 103 kB | 111/189 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.jar (103 kB at 746 kB/s) 2024-05-30T16:12:43.454 INFO:tasks.workunit.client.0.smithi006.stdout:Progress (1): 127/189 kB Progress (1): 143/189 kB Progress (1): 160/189 kB Progress (1): 176/189 kB Progress (1): 189 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.jar (189 kB at 1.3 MB/s) 2024-05-30T16:12:43.476 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] 2024-05-30T16:12:43.476 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] ------------------------------------------------------- 2024-05-30T16:12:43.476 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] T E S T S 2024-05-30T16:12:43.476 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] ------------------------------------------------------- 2024-05-30T16:12:44.056 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] Running io.ceph.jcksum.PutObjects 2024-05-30T16:12:44.136 INFO:tasks.workunit.client.0.smithi006.stdout:PutObjects.java: starting test run: 2024-05-30T16:12:44.136 INFO:tasks.workunit.client.0.smithi006.stdout: AccessKey=lNCnR47C2g+ZidCWBAUuwfSAA7Q= 2024-05-30T16:12:44.136 INFO:tasks.workunit.client.0.smithi006.stdout: SecretKey=tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g== 2024-05-30T16:12:44.136 INFO:tasks.workunit.client.0.smithi006.stdout: EndpointUrl=http://smithi006.front.sepia.ceph.com:80 2024-05-30T16:12:44.136 INFO:tasks.workunit.client.0.smithi006.stderr:[main] WARN software.amazon.awssdk.http.apache.ApacheHttpClient - SSL Certificate verification is disabled. This is not a safe setting and should only be used for testing. 2024-05-30T16:12:44.332 INFO:tasks.workunit.client.0.smithi006.stdout:DEBUG: Environment Variables Read 2024-05-30T16:12:44.734 INFO:tasks.workunit.client.0.smithi006.stdout:DEBUG: S3 Client Initialized 2024-05-30T16:12:44.734 INFO:tasks.workunit.client.0.smithi006.stdout:DEBUG: Generating File 2024-05-30T16:12:44.734 INFO:tasks.workunit.client.0.smithi006.stdout:DEBUG: File Generated 2024-05-30T16:12:44.734 INFO:tasks.workunit.client.0.smithi006.stdout:DEBUG: Generating File 2024-05-30T16:12:44.734 INFO:tasks.workunit.client.0.smithi006.stdout:DEBUG: File Generated 2024-05-30T16:12:44.734 INFO:tasks.workunit.client.0.smithi006.stdout:DEBUG: Generating File 2024-05-30T16:12:44.834 INFO:tasks.workunit.client.0.smithi006.stdout:DEBUG: File Generated 2024-05-30T16:12:44.834 INFO:tasks.workunit.client.0.smithi006.stdout:DEBUG: Generating File 2024-05-30T16:12:44.834 INFO:tasks.workunit.client.0.smithi006.stdout:DEBUG: File Generated 2024-05-30T16:12:44.834 INFO:tasks.workunit.client.0.smithi006.stdout:DEBUG: Generating File 2024-05-30T16:12:44.834 INFO:tasks.workunit.client.0.smithi006.stdout:DEBUG: File Generated 2024-05-30T16:12:44.834 INFO:tasks.workunit.client.0.smithi006.stdout:DEBUG: Generating File 2024-05-30T16:12:44.834 INFO:tasks.workunit.client.0.smithi006.stdout:DEBUG: File Generated 2024-05-30T16:12:44.835 INFO:tasks.workunit.client.0.smithi006.stdout:DEBUG: Generated Big Files 2024-05-30T16:12:45.134 INFO:tasks.workunit.client.0.smithi006.stdout:HeadBucketResponse() 2024-05-30T16:12:45.135 INFO:tasks.workunit.client.0.smithi006.stdout:sheik is ready 2024-05-30T16:12:45.135 INFO:tasks.workunit.client.0.smithi006.stdout:DEBUG: Test Bucket Created 2024-05-30T16:12:45.234 INFO:tasks.workunit.client.0.smithi006.stdout:putObjectFromFileNoCksum called with file-0b 2024-05-30T16:12:45.636 INFO:tasks.workunit.client.0.smithi006.stdout:putObjectFromFileNoCksum called with file-8b 2024-05-30T16:12:45.936 INFO:tasks.workunit.client.0.smithi006.stdout:putObjectFromFileNoCksum called with file-200b 2024-05-30T16:12:46.337 INFO:tasks.workunit.client.0.smithi006.stdout:putObjectFromFileNoCksum called with file-21983b 2024-05-30T16:12:46.638 INFO:tasks.workunit.client.0.smithi006.stdout:putObjectFromFileNoCksum called with file-5519b 2024-05-30T16:12:46.839 INFO:tasks.workunit.client.0.smithi006.stdout:putObjectFromFileNoCksum called with file-204329b 2024-05-30T16:12:47.240 INFO:tasks.workunit.client.0.smithi006.stdout:putObjectFromFileNoCksum called with file-256k 2024-05-30T16:12:47.640 INFO:tasks.workunit.client.0.smithi006.stdout:putObjectFromFileNoCksum called with file-1m 2024-05-30T16:12:48.543 INFO:tasks.workunit.client.0.smithi006.stdout:putObjectFromFileNoCksum called with file-1038757b 2024-05-30T16:12:49.345 INFO:tasks.workunit.client.0.smithi006.stdout:mpuObjectFromFileNoCksum called with file-5m 2024-05-30T16:12:53.855 INFO:tasks.workunit.client.0.smithi006.stdout:mpuObjectFromFileNoCksum called with file-10m 2024-05-30T16:12:58.064 INFO:tasks.workunit.client.0.smithi006.stdout:mpuObjectFromFileNoCksum called with file-100m 2024-05-30T16:13:58.201 INFO:tasks.workunit.client.0.smithi006.stdout:arg: file-0b 2024-05-30T16:13:58.202 INFO:tasks.workunit.client.0.smithi006.stdout:arg: file-8b 2024-05-30T16:13:58.202 INFO:tasks.workunit.client.0.smithi006.stdout:arg: file-200b 2024-05-30T16:13:58.203 INFO:tasks.workunit.client.0.smithi006.stdout:arg: file-21983b 2024-05-30T16:13:58.204 INFO:tasks.workunit.client.0.smithi006.stdout:arg: file-5519b 2024-05-30T16:13:58.205 INFO:tasks.workunit.client.0.smithi006.stdout:arg: file-204329b 2024-05-30T16:13:58.206 INFO:tasks.workunit.client.0.smithi006.stdout:arg: file-256k 2024-05-30T16:13:58.207 INFO:tasks.workunit.client.0.smithi006.stdout:arg: file-1m 2024-05-30T16:13:58.209 INFO:tasks.workunit.client.0.smithi006.stdout:arg: file-1038757b 2024-05-30T16:13:58.257 INFO:tasks.workunit.client.0.smithi006.stdout:putObjectFromFileCksum called with file-0b 2024-05-30T16:13:58.458 INFO:tasks.workunit.client.0.smithi006.stdout:PutObjectResponse 2024-05-30T16:13:58.558 INFO:tasks.workunit.client.0.smithi006.stdout:putObjectFromFileCksum called with file-8b 2024-05-30T16:13:58.658 INFO:tasks.workunit.client.0.smithi006.stdout:PutObjectResponse 2024-05-30T16:13:58.758 INFO:tasks.workunit.client.0.smithi006.stdout:putObjectFromFileCksum called with file-200b 2024-05-30T16:13:58.858 INFO:tasks.workunit.client.0.smithi006.stdout:PutObjectResponse 2024-05-30T16:13:59.059 INFO:tasks.workunit.client.0.smithi006.stdout:putObjectFromFileCksum called with file-21983b 2024-05-30T16:13:59.158 INFO:tasks.workunit.client.0.smithi006.stdout:PutObjectResponse 2024-05-30T16:13:59.359 INFO:tasks.workunit.client.0.smithi006.stdout:putObjectFromFileCksum called with file-5519b 2024-05-30T16:13:59.459 INFO:tasks.workunit.client.0.smithi006.stdout:PutObjectResponse 2024-05-30T16:13:59.560 INFO:tasks.workunit.client.0.smithi006.stdout:putObjectFromFileCksum called with file-204329b 2024-05-30T16:13:59.760 INFO:tasks.workunit.client.0.smithi006.stdout:PutObjectResponse 2024-05-30T16:13:59.960 INFO:tasks.workunit.client.0.smithi006.stdout:putObjectFromFileCksum called with file-256k 2024-05-30T16:14:00.260 INFO:tasks.workunit.client.0.smithi006.stdout:PutObjectResponse 2024-05-30T16:14:00.361 INFO:tasks.workunit.client.0.smithi006.stdout:putObjectFromFileCksum called with file-1m 2024-05-30T16:14:00.861 INFO:tasks.workunit.client.0.smithi006.stdout:PutObjectResponse 2024-05-30T16:14:01.062 INFO:tasks.workunit.client.0.smithi006.stdout:putObjectFromFileCksum called with file-1038757b 2024-05-30T16:14:01.462 INFO:tasks.workunit.client.0.smithi006.stdout:PutObjectResponse 2024-05-30T16:14:01.662 INFO:tasks.workunit.client.0.smithi006.stdout:mpuObjectFromFileCksum called with file-5m 2024-05-30T16:14:03.966 INFO:tasks.workunit.client.0.smithi006.stdout:mpuObjectFromFileCksum called with file-10m 2024-05-30T16:14:07.971 INFO:tasks.workunit.client.0.smithi006.stdout:mpuObjectFromFileCksum called with file-100m 2024-05-30T16:15:07.905 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] Tests run: 33, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 143.8 s -- in io.ceph.jcksum.PutObjects 2024-05-30T16:15:07.932 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] 2024-05-30T16:15:07.932 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] Results: 2024-05-30T16:15:07.932 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] 2024-05-30T16:15:07.932 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] Tests run: 33, Failures: 0, Errors: 0, Skipped: 0 2024-05-30T16:15:07.932 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] 2024-05-30T16:15:07.934 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] ------------------------------------------------------------------------ 2024-05-30T16:15:07.934 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] BUILD SUCCESS 2024-05-30T16:15:07.934 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] ------------------------------------------------------------------------ 2024-05-30T16:15:07.935 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] Total time: 02:26 min 2024-05-30T16:15:07.935 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] Finished at: 2024-05-30T16:15:07Z 2024-05-30T16:15:07.935 INFO:tasks.workunit.client.0.smithi006.stdout:[INFO] ------------------------------------------------------------------------ 2024-05-30T16:15:07.959 INFO:tasks.workunit.client.0.smithi006.stderr:+ exit 2024-05-30T16:15:07.959 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-30T16:15:07.959 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-30T16:15:08.013 INFO:tasks.workunit:Stopping ['cls/test_cls_lock.sh', 'cls/test_cls_log.sh', 'cls/test_cls_refcount.sh', 'cls/test_cls_rgw.sh', 'cls/test_cls_rgw_gc.sh', 'cls/test_cls_rgw_stats.sh', 'cls/test_cls_cmpomap.sh', 'cls/test_cls_2pc_queue.sh', 'cls/test_cls_user.sh', 'rgw/test_rgw_gc_log.sh', 'rgw/test_rgw_obj.sh', 'rgw/test_librgw_file.sh', 'rgw/test_awssdkv4_sig.sh'] on client.0... 2024-05-30T16:15:08.013 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-30T16:15:08.394 DEBUG:teuthology.parallel:result is None 2024-05-30T16:15:08.394 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-30T16:15:08.406 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-30T16:15:08.406 DEBUG:teuthology.orchestra.run.smithi006:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-05-30T16:15:08.458 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-05-30T16:15:08.458 INFO:teuthology.run_tasks:Running task workunit... 2024-05-30T16:15:08.469 INFO:tasks.workunit:Pulling workunits from ref da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 2024-05-30T16:15:08.469 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-30T16:15:08.470 DEBUG:teuthology.orchestra.run.smithi006:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-30T16:15:08.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-30T16:15:08.509 INFO:teuthology.orchestra.run.smithi006.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-05-30T16:15:08.510 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-05-30T16:15:08.561 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-05-30T16:15:08.562 DEBUG:teuthology.orchestra.run.smithi006:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-05-30T16:15:08.614 INFO:tasks.workunit:timeout=3h 2024-05-30T16:15:08.614 INFO:tasks.workunit:cleanup=True 2024-05-30T16:15:08.614 DEBUG:teuthology.orchestra.run.smithi006:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 2024-05-30T16:15:08.664 INFO:tasks.workunit.client.0.smithi006.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-30T16:16:32.968 INFO:tasks.workunit.client.0.smithi006.stderr:Updating files: 94% (11567/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-30T16:16:33.935 INFO:tasks.workunit.client.0.smithi006.stderr:Note: switching to 'da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2'. 2024-05-30T16:16:33.936 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:16:33.936 INFO:tasks.workunit.client.0.smithi006.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-30T16:16:33.936 INFO:tasks.workunit.client.0.smithi006.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-30T16:16:33.936 INFO:tasks.workunit.client.0.smithi006.stderr:state without impacting any branches by switching back to a branch. 2024-05-30T16:16:33.936 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:16:33.936 INFO:tasks.workunit.client.0.smithi006.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-30T16:16:33.936 INFO:tasks.workunit.client.0.smithi006.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-30T16:16:33.936 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:16:33.936 INFO:tasks.workunit.client.0.smithi006.stderr: git switch -c 2024-05-30T16:16:33.936 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:16:33.936 INFO:tasks.workunit.client.0.smithi006.stderr:Or undo this operation with: 2024-05-30T16:16:33.937 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:16:33.937 INFO:tasks.workunit.client.0.smithi006.stderr: git switch - 2024-05-30T16:16:33.937 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:16:33.937 INFO:tasks.workunit.client.0.smithi006.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-30T16:16:33.937 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:16:33.937 INFO:tasks.workunit.client.0.smithi006.stderr:HEAD is now at da98024b78b Merge branches 'pr-57079', 'pr-57536', 'pr-57641' and 'pr-57702' into wip-cbodley-testing 2024-05-30T16:16:33.945 DEBUG:teuthology.orchestra.run.smithi006:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-30T16:16:33.990 INFO:tasks.workunit.client.0.smithi006.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-30T16:16:33.993 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-30T16:16:33.993 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-30T16:16:34.057 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-30T16:16:34.124 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-30T16:16:34.169 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-30T16:16:34.170 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-30T16:16:34.171 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-30T16:16:34.219 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-30T16:16:34.224 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T16:16:34.224 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-30T16:16:34.270 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_s3_mp_reupload.sh on client.0... 2024-05-30T16:16:34.271 INFO:tasks.workunit:Running workunit rgw/test_rgw_s3_mp_reupload.sh... 2024-05-30T16:16:34.272 DEBUG:teuthology.orchestra.run.smithi006:workunit test rgw/test_rgw_s3_mp_reupload.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_s3_mp_reupload.sh 2024-05-30T16:16:34.326 INFO:tasks.workunit.client.0.smithi006.stdout:INFO: fully qualified domain name: smithi006.front.sepia.ceph.com 2024-05-30T16:16:34.327 INFO:tasks.workunit.client.0.smithi006.stdout:== TESTING *ABORTING* MULTIPART UPLOAD WITH RE-UPLOADS == 2024-05-30T16:16:34.327 INFO:tasks.workunit.client.0.smithi006.stdout:== TESTING VERSIONED BUCKET == 2024-05-30T16:16:37.720 INFO:tasks.workunit.client.0.smithi006.stdout:Requirement already satisfied: pip in /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages (22.0.2) 2024-05-30T16:16:37.934 INFO:tasks.workunit.client.0.smithi006.stdout:Collecting pip 2024-05-30T16:16:37.941 INFO:tasks.workunit.client.0.smithi006.stdout: Using cached pip-24.0-py3-none-any.whl (2.1 MB) 2024-05-30T16:16:37.979 INFO:tasks.workunit.client.0.smithi006.stdout:Installing collected packages: pip 2024-05-30T16:16:37.980 INFO:tasks.workunit.client.0.smithi006.stdout: Attempting uninstall: pip 2024-05-30T16:16:37.980 INFO:tasks.workunit.client.0.smithi006.stdout: Found existing installation: pip 22.0.2 2024-05-30T16:16:38.247 INFO:tasks.workunit.client.0.smithi006.stdout: Uninstalling pip-22.0.2: 2024-05-30T16:16:38.253 INFO:tasks.workunit.client.0.smithi006.stdout: Successfully uninstalled pip-22.0.2 2024-05-30T16:16:39.234 INFO:tasks.workunit.client.0.smithi006.stdout:Successfully installed pip-24.0 2024-05-30T16:16:40.260 INFO:tasks.workunit.client.0.smithi006.stdout:Collecting boto3 2024-05-30T16:16:40.262 INFO:tasks.workunit.client.0.smithi006.stdout: Using cached boto3-1.34.115-py3-none-any.whl.metadata (6.6 kB) 2024-05-30T16:16:40.796 INFO:tasks.workunit.client.0.smithi006.stdout:Collecting botocore<1.35.0,>=1.34.115 (from boto3) 2024-05-30T16:16:40.797 INFO:tasks.workunit.client.0.smithi006.stdout: Using cached botocore-1.34.115-py3-none-any.whl.metadata (5.7 kB) 2024-05-30T16:16:40.854 INFO:tasks.workunit.client.0.smithi006.stdout:Collecting jmespath<2.0.0,>=0.7.1 (from boto3) 2024-05-30T16:16:40.855 INFO:tasks.workunit.client.0.smithi006.stdout: Using cached jmespath-1.0.1-py3-none-any.whl.metadata (7.6 kB) 2024-05-30T16:16:40.909 INFO:tasks.workunit.client.0.smithi006.stdout:Collecting s3transfer<0.11.0,>=0.10.0 (from boto3) 2024-05-30T16:16:40.910 INFO:tasks.workunit.client.0.smithi006.stdout: Using cached s3transfer-0.10.1-py3-none-any.whl.metadata (1.7 kB) 2024-05-30T16:16:40.983 INFO:tasks.workunit.client.0.smithi006.stdout:Collecting python-dateutil<3.0.0,>=2.1 (from botocore<1.35.0,>=1.34.115->boto3) 2024-05-30T16:16:40.985 INFO:tasks.workunit.client.0.smithi006.stdout: Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB) 2024-05-30T16:16:41.092 INFO:tasks.workunit.client.0.smithi006.stdout:Collecting urllib3!=2.2.0,<3,>=1.25.4 (from botocore<1.35.0,>=1.34.115->boto3) 2024-05-30T16:16:41.094 INFO:tasks.workunit.client.0.smithi006.stdout: Using cached urllib3-2.2.1-py3-none-any.whl.metadata (6.4 kB) 2024-05-30T16:16:41.257 INFO:tasks.workunit.client.0.smithi006.stdout:Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore<1.35.0,>=1.34.115->boto3) 2024-05-30T16:16:41.258 INFO:tasks.workunit.client.0.smithi006.stdout: Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB) 2024-05-30T16:16:41.270 INFO:tasks.workunit.client.0.smithi006.stdout:Using cached boto3-1.34.115-py3-none-any.whl (139 kB) 2024-05-30T16:16:41.271 INFO:tasks.workunit.client.0.smithi006.stdout:Using cached botocore-1.34.115-py3-none-any.whl (12.3 MB) 2024-05-30T16:16:41.293 INFO:tasks.workunit.client.0.smithi006.stdout:Using cached jmespath-1.0.1-py3-none-any.whl (20 kB) 2024-05-30T16:16:41.294 INFO:tasks.workunit.client.0.smithi006.stdout:Using cached s3transfer-0.10.1-py3-none-any.whl (82 kB) 2024-05-30T16:16:41.295 INFO:tasks.workunit.client.0.smithi006.stdout:Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) 2024-05-30T16:16:41.297 INFO:tasks.workunit.client.0.smithi006.stdout:Using cached urllib3-2.2.1-py3-none-any.whl (121 kB) 2024-05-30T16:16:41.298 INFO:tasks.workunit.client.0.smithi006.stdout:Using cached six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-05-30T16:16:41.393 INFO:tasks.workunit.client.0.smithi006.stdout:Installing collected packages: urllib3, six, jmespath, python-dateutil, botocore, s3transfer, boto3 2024-05-30T16:16:42.570 INFO:tasks.workunit.client.0.smithi006.stdout:Successfully installed boto3-1.34.115 botocore-1.34.115 jmespath-1.0.1 python-dateutil-2.9.0.post0 s3transfer-0.10.1 six-1.16.0 urllib3-2.2.1 2024-05-30T16:16:42.910 INFO:tasks.workunit.client.0.smithi006.stdout:INFO: creating user 2024-05-30T16:16:43.055 INFO:tasks.workunit.client.0.smithi006.stdout:{ 2024-05-30T16:16:43.055 INFO:tasks.workunit.client.0.smithi006.stdout: "user_id": "testid", 2024-05-30T16:16:43.055 INFO:tasks.workunit.client.0.smithi006.stdout: "display_name": "M. Tester", 2024-05-30T16:16:43.055 INFO:tasks.workunit.client.0.smithi006.stdout: "email": "tester@ceph.com", 2024-05-30T16:16:43.055 INFO:tasks.workunit.client.0.smithi006.stdout: "suspended": 0, 2024-05-30T16:16:43.055 INFO:tasks.workunit.client.0.smithi006.stdout: "max_buckets": 1000, 2024-05-30T16:16:43.055 INFO:tasks.workunit.client.0.smithi006.stdout: "subusers": [], 2024-05-30T16:16:43.056 INFO:tasks.workunit.client.0.smithi006.stdout: "keys": [ 2024-05-30T16:16:43.056 INFO:tasks.workunit.client.0.smithi006.stdout: { 2024-05-30T16:16:43.056 INFO:tasks.workunit.client.0.smithi006.stdout: "user": "testid", 2024-05-30T16:16:43.056 INFO:tasks.workunit.client.0.smithi006.stdout: "access_key": "0555b35654ad1656d804", 2024-05-30T16:16:43.056 INFO:tasks.workunit.client.0.smithi006.stdout: "secret_key": "h7GhxuBLTrlhVUyxSPUKUV8r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q==", 2024-05-30T16:16:43.056 INFO:tasks.workunit.client.0.smithi006.stdout: "active": true, 2024-05-30T16:16:43.056 INFO:tasks.workunit.client.0.smithi006.stdout: "create_date": "2024-05-30T16:16:43.043931Z" 2024-05-30T16:16:43.056 INFO:tasks.workunit.client.0.smithi006.stdout: } 2024-05-30T16:16:43.056 INFO:tasks.workunit.client.0.smithi006.stdout: ], 2024-05-30T16:16:43.056 INFO:tasks.workunit.client.0.smithi006.stdout: "swift_keys": [], 2024-05-30T16:16:43.056 INFO:tasks.workunit.client.0.smithi006.stdout: "caps": [], 2024-05-30T16:16:43.056 INFO:tasks.workunit.client.0.smithi006.stdout: "op_mask": "read, write, delete", 2024-05-30T16:16:43.057 INFO:tasks.workunit.client.0.smithi006.stdout: "default_placement": "", 2024-05-30T16:16:43.057 INFO:tasks.workunit.client.0.smithi006.stdout: "default_storage_class": "", 2024-05-30T16:16:43.057 INFO:tasks.workunit.client.0.smithi006.stdout: "placement_tags": [], 2024-05-30T16:16:43.057 INFO:tasks.workunit.client.0.smithi006.stdout: "bucket_quota": { 2024-05-30T16:16:43.057 INFO:tasks.workunit.client.0.smithi006.stdout: "enabled": false, 2024-05-30T16:16:43.057 INFO:tasks.workunit.client.0.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:16:43.057 INFO:tasks.workunit.client.0.smithi006.stdout: "max_size": -1, 2024-05-30T16:16:43.057 INFO:tasks.workunit.client.0.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:16:43.057 INFO:tasks.workunit.client.0.smithi006.stdout: "max_objects": -1 2024-05-30T16:16:43.057 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:16:43.057 INFO:tasks.workunit.client.0.smithi006.stdout: "user_quota": { 2024-05-30T16:16:43.057 INFO:tasks.workunit.client.0.smithi006.stdout: "enabled": false, 2024-05-30T16:16:43.057 INFO:tasks.workunit.client.0.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:16:43.057 INFO:tasks.workunit.client.0.smithi006.stdout: "max_size": -1, 2024-05-30T16:16:43.058 INFO:tasks.workunit.client.0.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:16:43.058 INFO:tasks.workunit.client.0.smithi006.stdout: "max_objects": -1 2024-05-30T16:16:43.058 INFO:tasks.workunit.client.0.smithi006.stdout: }, 2024-05-30T16:16:43.058 INFO:tasks.workunit.client.0.smithi006.stdout: "temp_url_keys": [], 2024-05-30T16:16:43.058 INFO:tasks.workunit.client.0.smithi006.stdout: "type": "rgw", 2024-05-30T16:16:43.058 INFO:tasks.workunit.client.0.smithi006.stdout: "mfa_ids": [], 2024-05-30T16:16:43.058 INFO:tasks.workunit.client.0.smithi006.stdout: "account_id": "", 2024-05-30T16:16:43.058 INFO:tasks.workunit.client.0.smithi006.stdout: "path": "/", 2024-05-30T16:16:43.058 INFO:tasks.workunit.client.0.smithi006.stdout: "create_date": "2024-05-30T16:16:43.043899Z", 2024-05-30T16:16:43.058 INFO:tasks.workunit.client.0.smithi006.stdout: "tags": [], 2024-05-30T16:16:43.058 INFO:tasks.workunit.client.0.smithi006.stdout: "group_ids": [] 2024-05-30T16:16:43.058 INFO:tasks.workunit.client.0.smithi006.stdout:} 2024-05-30T16:16:43.058 INFO:tasks.workunit.client.0.smithi006.stdout: 2024-05-30T16:16:43.514 INFO:tasks.workunit.client.0.smithi006.stderr:5+0 records in 2024-05-30T16:16:43.514 INFO:tasks.workunit.client.0.smithi006.stderr:5+0 records out 2024-05-30T16:16:43.514 INFO:tasks.workunit.client.0.smithi006.stderr:5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.0226226 s, 232 MB/s 2024-05-30T16:16:57.566 INFO:tasks.workunit.client.0.smithi006.stdout:bucket nams is reupload-bkt-120382 2024-05-30T16:16:57.566 INFO:tasks.workunit.client.0.smithi006.stdout:endpoint is http://smithi006.front.sepia.ceph.com:80 2024-05-30T16:16:57.566 INFO:tasks.workunit.client.0.smithi006.stdout:start UploadId=2~fZvYpZi-DmuNamOsEti7-AAp1UnTPyq 2024-05-30T16:16:57.567 INFO:tasks.workunit.client.0.smithi006.stdout:phase 1 uploaded part {'ETag': '"1a6549712c042f16072e043dc2be9ee5"', 'PartNumber': 1} 2024-05-30T16:16:57.567 INFO:tasks.workunit.client.0.smithi006.stdout:phase 1 uploaded part {'ETag': '"1a6549712c042f16072e043dc2be9ee5"', 'PartNumber': 2} 2024-05-30T16:16:57.567 INFO:tasks.workunit.client.0.smithi006.stdout:phase 2 uploaded part {'ETag': '"1a6549712c042f16072e043dc2be9ee5"', 'PartNumber': 1} 2024-05-30T16:16:57.567 INFO:tasks.workunit.client.0.smithi006.stdout:phase 2 uploaded part {'ETag': '"1a6549712c042f16072e043dc2be9ee5"', 'PartNumber': 1} 2024-05-30T16:16:57.567 INFO:tasks.workunit.client.0.smithi006.stdout:phase 2 uploaded part {'ETag': '"1a6549712c042f16072e043dc2be9ee5"', 'PartNumber': 1} 2024-05-30T16:16:57.567 INFO:tasks.workunit.client.0.smithi006.stdout:phase 2 uploaded part {'ETag': '"1a6549712c042f16072e043dc2be9ee5"', 'PartNumber': 1} 2024-05-30T16:16:57.567 INFO:tasks.workunit.client.0.smithi006.stdout:phase 2 uploaded part {'ETag': '"1a6549712c042f16072e043dc2be9ee5"', 'PartNumber': 1} 2024-05-30T16:16:57.567 INFO:tasks.workunit.client.0.smithi006.stdout:phase 2 uploaded part {'ETag': '"1a6549712c042f16072e043dc2be9ee5"', 'PartNumber': 1} 2024-05-30T16:16:57.567 INFO:tasks.workunit.client.0.smithi006.stdout:phase 2 uploaded part {'ETag': '"1a6549712c042f16072e043dc2be9ee5"', 'PartNumber': 1} 2024-05-30T16:16:57.567 INFO:tasks.workunit.client.0.smithi006.stdout:phase 2 uploaded part {'ETag': '"1a6549712c042f16072e043dc2be9ee5"', 'PartNumber': 1} 2024-05-30T16:16:57.567 INFO:tasks.workunit.client.0.smithi006.stdout:phase 2 uploaded part {'ETag': '"1a6549712c042f16072e043dc2be9ee5"', 'PartNumber': 1} 2024-05-30T16:16:57.567 INFO:tasks.workunit.client.0.smithi006.stdout:phase 2 uploaded part {'ETag': '"1a6549712c042f16072e043dc2be9ee5"', 'PartNumber': 1} 2024-05-30T16:16:57.567 INFO:tasks.workunit.client.0.smithi006.stdout:phase 2 uploaded part {'ETag': '"1a6549712c042f16072e043dc2be9ee5"', 'PartNumber': 1} 2024-05-30T16:16:57.567 INFO:tasks.workunit.client.0.smithi006.stdout:aborting multipart upload, parts=[{'ETag': '"1a6549712c042f16072e043dc2be9ee5"', 'PartNumber': 1}, {'ETag': '"1a6549712c042f16072e043dc2be9ee5"', 'PartNumber': 2}] 2024-05-30T16:16:59.266 INFO:tasks.workunit.client.0.smithi006.stdout:radosgw-admin bucket radoslist: 2024-05-30T16:16:59.351 INFO:tasks.workunit.client.0.smithi006.stdout:rados ls: 2024-05-30T16:16:59.368 INFO:tasks.workunit.client.0.smithi006.stdout:== PASSED == 2024-05-30T16:16:59.368 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-30T16:16:59.368 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-30T16:16:59.419 INFO:tasks.workunit:Stopping ['rgw/test_rgw_s3_mp_reupload.sh'] on client.0... 2024-05-30T16:16:59.420 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-30T16:16:59.807 DEBUG:teuthology.parallel:result is None 2024-05-30T16:16:59.808 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-30T16:16:59.820 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-30T16:16:59.820 DEBUG:teuthology.orchestra.run.smithi006:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-05-30T16:16:59.869 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-05-30T16:16:59.870 INFO:teuthology.run_tasks:Running task ragweed... 2024-05-30T16:16:59.884 DEBUG:tasks.ragweed:ragweed config is {'client.0': {'default-branch': 'ceph-master', 'rgw_server': 'client.0', 'stages': 'prepare,check'}} 2024-05-30T16:16:59.884 INFO:tasks.ragweed:Downloading ragweed... 2024-05-30T16:16:59.884 INFO:tasks.ragweed:Using branch 'wip-cbodley-testing' for ragweed 2024-05-30T16:16:59.884 DEBUG:teuthology.orchestra.run.smithi006:> git clone -b wip-cbodley-testing https://git.ceph.com/ragweed.git /home/ubuntu/cephtest/ragweed.client.0 2024-05-30T16:16:59.917 INFO:teuthology.orchestra.run.smithi006.stderr:Cloning into '/home/ubuntu/cephtest/ragweed.client.0'... 2024-05-30T16:16:59.977 INFO:teuthology.orchestra.run.smithi006.stderr:fatal: Remote branch wip-cbodley-testing not found in upstream origin 2024-05-30T16:16:59.978 DEBUG:teuthology.orchestra.run:got remote process result: 128 2024-05-30T16:16:59.979 INFO:tasks.ragweed:Using branch 'ceph-master' for ragweed 2024-05-30T16:16:59.979 DEBUG:teuthology.orchestra.run.smithi006:> git clone -b ceph-master https://git.ceph.com/ragweed.git /home/ubuntu/cephtest/ragweed.client.0 2024-05-30T16:17:00.025 INFO:teuthology.orchestra.run.smithi006.stderr:Cloning into '/home/ubuntu/cephtest/ragweed.client.0'... 2024-05-30T16:17:00.204 INFO:tasks.ragweed:Creating rgw users... 2024-05-30T16:17:00.204 DEBUG:tasks.ragweed:Creating user ragweed.client.0 on client.0 2024-05-30T16:17:00.204 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid ragweed.client.0 --display-name 'Mr. ragweed.client.0' --access-key SJFTXZIBRWVIGSTRSZSW --secret 3FByAb79iSMDnsXK7w2BgHKk1PZSkxmwaLktGFjeP0MEXLZcrM/Mjw== --email ragweed.client.0+test@test.test --system false 2024-05-30T16:17:00.294 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:17:00.294 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:17:00.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.335+0000 7fde26888980 20 rados->read ofs=0 len=0 2024-05-30T16:17:00.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.339+0000 7fde26888980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:17:00.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.339+0000 7fde26888980 20 realm 2024-05-30T16:17:00.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.339+0000 7fde26888980 20 rados->read ofs=0 len=0 2024-05-30T16:17:00.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.339+0000 7fde26888980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:17:00.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.339+0000 7fde26888980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:17:00.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.339+0000 7fde26888980 20 rados->read ofs=0 len=0 2024-05-30T16:17:00.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.339+0000 7fde26888980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:17:00.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.339+0000 7fde26888980 20 rados->read ofs=0 len=0 2024-05-30T16:17:00.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.339+0000 7fde26888980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:17:00.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.339+0000 7fde26888980 20 rados->read ofs=0 len=0 2024-05-30T16:17:00.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.339+0000 7fde26888980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:17:00.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.339+0000 7fde26888980 20 searching for the correct realm 2024-05-30T16:17:00.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.347+0000 7fde23b98640 0 -- 172.21.15.6:0/1262739067 >> [v2:172.21.15.6:6819/1141406589,v1:172.21.15.6:6821/1141406589] conn(0x557652f3cd80 msgr2=0x557652f3f240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:17:00.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.355+0000 7fde24399640 0 -- 172.21.15.6:0/1262739067 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7fde180aa3d0 msgr2=0x7fde180aa830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:17:00.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.363+0000 7fde26888980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:17:00.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.363+0000 7fde26888980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:17:00.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.363+0000 7fde26888980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:17:00.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.363+0000 7fde26888980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:17:00.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.363+0000 7fde26888980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:17:00.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.363+0000 7fde26888980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:17:00.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.363+0000 7fde26888980 20 rados->read ofs=0 len=0 2024-05-30T16:17:00.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.363+0000 7fde26888980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:17:00.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.363+0000 7fde26888980 20 rados->read ofs=0 len=0 2024-05-30T16:17:00.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.363+0000 7fde26888980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:17:00.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.363+0000 7fde26888980 20 rados->read ofs=0 len=0 2024-05-30T16:17:00.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.363+0000 7fde26888980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:17:00.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.363+0000 7fde26888980 20 zone default found 2024-05-30T16:17:00.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.363+0000 7fde26888980 4 Realm: () 2024-05-30T16:17:00.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.363+0000 7fde26888980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:17:00.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.363+0000 7fde26888980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:17:00.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.363+0000 7fde26888980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:17:00.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.363+0000 7fde26888980 20 zonegroup default 2024-05-30T16:17:00.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.363+0000 7fde26888980 20 rados->read ofs=0 len=0 2024-05-30T16:17:00.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.363+0000 7fde26888980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:17:00.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.363+0000 7fde26888980 20 rados->read ofs=0 len=0 2024-05-30T16:17:00.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.367+0000 7fde26888980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:17:00.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.367+0000 7fde26888980 20 rados->read ofs=0 len=0 2024-05-30T16:17:00.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.367+0000 7fde26888980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:17:00.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.367+0000 7fde26888980 20 started sync module instance, tier type = 2024-05-30T16:17:00.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.367+0000 7fde26888980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:17:00.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.383+0000 7fde26888980 20 add_watcher() i=0 2024-05-30T16:17:00.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.383+0000 7fde26888980 20 add_watcher() i=1 2024-05-30T16:17:00.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.383+0000 7fde26888980 20 add_watcher() i=2 2024-05-30T16:17:00.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.383+0000 7fde26888980 20 add_watcher() i=3 2024-05-30T16:17:00.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.383+0000 7fde26888980 20 add_watcher() i=4 2024-05-30T16:17:00.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.383+0000 7fde26888980 20 add_watcher() i=5 2024-05-30T16:17:00.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.383+0000 7fde26888980 20 add_watcher() i=6 2024-05-30T16:17:00.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.383+0000 7fde26888980 20 add_watcher() i=7 2024-05-30T16:17:00.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.383+0000 7fde26888980 2 all 8 watchers are set, enabling cache 2024-05-30T16:17:00.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.387+0000 7fdcdafdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:17:00.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.387+0000 7fde26888980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:17:00.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.387+0000 7fde26888980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:17:00.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.387+0000 7fde26888980 5 note: GC not initialized 2024-05-30T16:17:00.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.387+0000 7fdcd9fdb640 20 reqs_thread_entry: start 2024-05-30T16:17:00.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.387+0000 7fde26888980 20 init_complete bucket index max shards: 11 2024-05-30T16:17:00.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.387+0000 7fde26888980 20 Filter name: none 2024-05-30T16:17:00.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.387+0000 7fdc4b7fe640 20 reqs_thread_entry: start 2024-05-30T16:17:00.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.387+0000 7fde26888980 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : miss 2024-05-30T16:17:00.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.387+0000 7fde26888980 20 rados->read ofs=0 len=0 2024-05-30T16:17:00.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.387+0000 7fde26888980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:17:00.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.387+0000 7fde26888980 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x0 2024-05-30T16:17:00.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.387+0000 7fde26888980 10 adding default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2024-05-30T16:17:00.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.387+0000 7fde26888980 10 cache get: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test : miss 2024-05-30T16:17:00.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.387+0000 7fde26888980 20 rados->read ofs=0 len=0 2024-05-30T16:17:00.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.391+0000 7fde26888980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:17:00.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.391+0000 7fde26888980 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x0 2024-05-30T16:17:00.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.391+0000 7fde26888980 10 adding default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2024-05-30T16:17:00.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.391+0000 7fde26888980 10 cache get: name=default.rgw.meta+users.keys+SJFTXZIBRWVIGSTRSZSW : miss 2024-05-30T16:17:00.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.391+0000 7fde26888980 20 rados->read ofs=0 len=0 2024-05-30T16:17:00.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.391+0000 7fde26888980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:17:00.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.391+0000 7fde26888980 10 cache put: name=default.rgw.meta+users.keys+SJFTXZIBRWVIGSTRSZSW info.flags=0x0 2024-05-30T16:17:00.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.391+0000 7fde26888980 10 adding default.rgw.meta+users.keys+SJFTXZIBRWVIGSTRSZSW to cache LRU end 2024-05-30T16:17:00.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.391+0000 7fde26888980 10 cache get: name=default.rgw.meta+users.keys+SJFTXZIBRWVIGSTRSZSW : hit (negative entry) 2024-05-30T16:17:00.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.391+0000 7fde26888980 10 cache get: name=default.rgw.meta+users.keys+SJFTXZIBRWVIGSTRSZSW : hit (negative entry) 2024-05-30T16:17:00.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.395+0000 7fde26888980 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x17 2024-05-30T16:17:00.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.395+0000 7fde26888980 10 moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2024-05-30T16:17:00.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.395+0000 7fde26888980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ragweed.client.0, ofs0, ns] 2024-05-30T16:17:00.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.395+0000 7fdcfb7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1090921693205 cookie 93966686229296 notifier 5618 bl.length()=675 2024-05-30T16:17:00.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.395+0000 7fdcfb7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x17 2024-05-30T16:17:00.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.395+0000 7fdcfb7fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2024-05-30T16:17:00.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.399+0000 7fde26888980 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x7 2024-05-30T16:17:00.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.399+0000 7fde26888980 10 moving default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2024-05-30T16:17:00.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.399+0000 7fde26888980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.email:ragweed.client.0+test@test.test, ofs0, ns] 2024-05-30T16:17:00.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.399+0000 7fdcfb7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1090921693195 cookie 93966686217424 notifier 5618 bl.length()=202 2024-05-30T16:17:00.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.399+0000 7fdcfb7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x7 2024-05-30T16:17:00.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.399+0000 7fdcfb7fe640 10 rgw watcher librados: moving default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2024-05-30T16:17:00.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.407+0000 7fde26888980 10 cache put: name=default.rgw.meta+users.keys+SJFTXZIBRWVIGSTRSZSW info.flags=0x7 2024-05-30T16:17:00.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.407+0000 7fde26888980 10 moving default.rgw.meta+users.keys+SJFTXZIBRWVIGSTRSZSW to cache LRU end 2024-05-30T16:17:00.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.407+0000 7fde26888980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.keys:SJFTXZIBRWVIGSTRSZSW, ofs0, ns] 2024-05-30T16:17:00.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.407+0000 7fdcfb7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1090921693231 cookie 93966686225472 notifier 5618 bl.length()=190 2024-05-30T16:17:00.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.407+0000 7fdcfb7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+SJFTXZIBRWVIGSTRSZSW info.flags=0x7 2024-05-30T16:17:00.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.407+0000 7fdcfb7fe640 10 rgw watcher librados: moving default.rgw.meta+users.keys+SJFTXZIBRWVIGSTRSZSW to cache LRU end 2024-05-30T16:17:00.408 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-30T16:17:00.408 INFO:teuthology.orchestra.run.smithi006.stdout: "user_id": "ragweed.client.0", 2024-05-30T16:17:00.408 INFO:teuthology.orchestra.run.smithi006.stdout: "display_name": "Mr. ragweed.client.0", 2024-05-30T16:17:00.408 INFO:teuthology.orchestra.run.smithi006.stdout: "email": "ragweed.client.0+test@test.test", 2024-05-30T16:17:00.408 INFO:teuthology.orchestra.run.smithi006.stdout: "suspended": 0, 2024-05-30T16:17:00.409 INFO:teuthology.orchestra.run.smithi006.stdout: "max_buckets": 1000, 2024-05-30T16:17:00.409 INFO:teuthology.orchestra.run.smithi006.stdout: "subusers": [], 2024-05-30T16:17:00.409 INFO:teuthology.orchestra.run.smithi006.stdout: "keys": [ 2024-05-30T16:17:00.409 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T16:17:00.409 INFO:teuthology.orchestra.run.smithi006.stdout: "user": "ragweed.client.0", 2024-05-30T16:17:00.409 INFO:teuthology.orchestra.run.smithi006.stdout: "access_key": "SJFTXZIBRWVIGSTRSZSW", 2024-05-30T16:17:00.409 INFO:teuthology.orchestra.run.smithi006.stdout: "secret_key": "3FByAb79iSMDnsXK7w2BgHKk1PZSkxmwaLktGFjeP0MEXLZcrM/Mjw==", 2024-05-30T16:17:00.409 INFO:teuthology.orchestra.run.smithi006.stdout: "active": true, 2024-05-30T16:17:00.409 INFO:teuthology.orchestra.run.smithi006.stdout: "create_date": "2024-05-30T16:17:00.393533Z" 2024-05-30T16:17:00.409 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T16:17:00.409 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-30T16:17:00.409 INFO:teuthology.orchestra.run.smithi006.stdout: "swift_keys": [], 2024-05-30T16:17:00.409 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": [], 2024-05-30T16:17:00.409 INFO:teuthology.orchestra.run.smithi006.stdout: "op_mask": "read, write, delete", 2024-05-30T16:17:00.410 INFO:teuthology.orchestra.run.smithi006.stdout: "default_placement": "", 2024-05-30T16:17:00.410 INFO:teuthology.orchestra.run.smithi006.stdout: "default_storage_class": "", 2024-05-30T16:17:00.410 INFO:teuthology.orchestra.run.smithi006.stdout: "placement_tags": [], 2024-05-30T16:17:00.410 INFO:teuthology.orchestra.run.smithi006.stdout: "bucket_quota": { 2024-05-30T16:17:00.410 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": false, 2024-05-30T16:17:00.410 INFO:teuthology.orchestra.run.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:17:00.410 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size": -1, 2024-05-30T16:17:00.410 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:17:00.410 INFO:teuthology.orchestra.run.smithi006.stdout: "max_objects": -1 2024-05-30T16:17:00.410 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T16:17:00.410 INFO:teuthology.orchestra.run.smithi006.stdout: "user_quota": { 2024-05-30T16:17:00.410 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": false, 2024-05-30T16:17:00.410 INFO:teuthology.orchestra.run.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:17:00.411 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size": -1, 2024-05-30T16:17:00.411 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:17:00.411 INFO:teuthology.orchestra.run.smithi006.stdout: "max_objects": -1 2024-05-30T16:17:00.411 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T16:17:00.411 INFO:teuthology.orchestra.run.smithi006.stdout: "temp_url_keys": [], 2024-05-30T16:17:00.411 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "rgw", 2024-05-30T16:17:00.411 INFO:teuthology.orchestra.run.smithi006.stdout: "mfa_ids": [], 2024-05-30T16:17:00.411 INFO:teuthology.orchestra.run.smithi006.stdout: "account_id": "", 2024-05-30T16:17:00.411 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2024-05-30T16:17:00.411 INFO:teuthology.orchestra.run.smithi006.stdout: "create_date": "2024-05-30T16:17:00.393500Z", 2024-05-30T16:17:00.411 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-30T16:17:00.411 INFO:teuthology.orchestra.run.smithi006.stdout: "group_ids": [] 2024-05-30T16:17:00.411 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-30T16:17:00.411 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:17:00.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.411+0000 7fdcdafdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:17:00.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.415+0000 7fde26888980 20 remove_watcher() i=0 2024-05-30T16:17:00.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.415+0000 7fde26888980 2 removed watcher, disabling cache 2024-05-30T16:17:00.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.419+0000 7fde26888980 20 remove_watcher() i=1 2024-05-30T16:17:00.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.419+0000 7fde26888980 20 remove_watcher() i=2 2024-05-30T16:17:00.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.423+0000 7fde26888980 20 remove_watcher() i=3 2024-05-30T16:17:00.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.423+0000 7fde26888980 20 remove_watcher() i=4 2024-05-30T16:17:00.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.423+0000 7fde26888980 20 remove_watcher() i=5 2024-05-30T16:17:00.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.427+0000 7fde26888980 20 remove_watcher() i=6 2024-05-30T16:17:00.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.427+0000 7fde26888980 20 remove_watcher() i=7 2024-05-30T16:17:00.442 DEBUG:tasks.ragweed:Creating user sysuser.client.0 on client.0 2024-05-30T16:17:00.442 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid sysuser.client.0 --display-name 'Mr. sysuser.client.0' --access-key SEQBSIIZTUZARSATLBUJ --secret 2c64qenxGzKyPp+jYLad/eXYbxKSLMZX9RmjXlK9HpfAklbnOr3faQ== --email sysuser.client.0+test@test.test --system true 2024-05-30T16:17:00.483 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:17:00.483 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:17:00.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.523+0000 7fb520c66980 20 rados->read ofs=0 len=0 2024-05-30T16:17:00.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.527+0000 7fb520c66980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:17:00.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.527+0000 7fb520c66980 20 realm 2024-05-30T16:17:00.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.527+0000 7fb520c66980 20 rados->read ofs=0 len=0 2024-05-30T16:17:00.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.527+0000 7fb520c66980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:17:00.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.527+0000 7fb520c66980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:17:00.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.527+0000 7fb520c66980 20 rados->read ofs=0 len=0 2024-05-30T16:17:00.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.527+0000 7fb520c66980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:17:00.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.527+0000 7fb520c66980 20 rados->read ofs=0 len=0 2024-05-30T16:17:00.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.527+0000 7fb520c66980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:17:00.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.527+0000 7fb520c66980 20 rados->read ofs=0 len=0 2024-05-30T16:17:00.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.527+0000 7fb520c66980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:17:00.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.527+0000 7fb520c66980 20 searching for the correct realm 2024-05-30T16:17:00.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.535+0000 7fb51ef78640 0 -- 172.21.15.6:0/3453138414 >> [v2:172.21.15.204:6800/3687190393,v1:172.21.15.204:6801/3687190393] conn(0x7fb51006e6a0 msgr2=0x7fb51006e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:17:00.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.751+0000 7fb51e777640 0 -- 172.21.15.6:0/3453138414 >> [v2:172.21.15.204:6816/2677861390,v1:172.21.15.204:6817/2677861390] conn(0x556bb38a8fd0 msgr2=0x556bb38ab400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:17:00.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.759+0000 7fb520c66980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:17:00.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.759+0000 7fb520c66980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:17:00.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.759+0000 7fb520c66980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:17:00.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.759+0000 7fb520c66980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:17:00.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.759+0000 7fb520c66980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:17:00.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.759+0000 7fb520c66980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:17:00.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.759+0000 7fb520c66980 20 rados->read ofs=0 len=0 2024-05-30T16:17:00.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.759+0000 7fb520c66980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:17:00.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.759+0000 7fb520c66980 20 rados->read ofs=0 len=0 2024-05-30T16:17:00.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.759+0000 7fb520c66980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:17:00.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.759+0000 7fb520c66980 20 rados->read ofs=0 len=0 2024-05-30T16:17:00.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.763+0000 7fb520c66980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:17:00.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.763+0000 7fb520c66980 20 zone default found 2024-05-30T16:17:00.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.763+0000 7fb520c66980 4 Realm: () 2024-05-30T16:17:00.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.763+0000 7fb520c66980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:17:00.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.763+0000 7fb520c66980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:17:00.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.763+0000 7fb520c66980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:17:00.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.763+0000 7fb520c66980 20 zonegroup default 2024-05-30T16:17:00.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.763+0000 7fb520c66980 20 rados->read ofs=0 len=0 2024-05-30T16:17:00.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.763+0000 7fb520c66980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:17:00.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.763+0000 7fb520c66980 20 rados->read ofs=0 len=0 2024-05-30T16:17:00.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.763+0000 7fb520c66980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:17:00.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.763+0000 7fb520c66980 20 rados->read ofs=0 len=0 2024-05-30T16:17:00.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.763+0000 7fb520c66980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:17:00.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.763+0000 7fb520c66980 20 started sync module instance, tier type = 2024-05-30T16:17:00.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.763+0000 7fb520c66980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:17:00.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.787+0000 7fb520c66980 20 add_watcher() i=0 2024-05-30T16:17:00.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.787+0000 7fb520c66980 20 add_watcher() i=1 2024-05-30T16:17:00.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.787+0000 7fb520c66980 20 add_watcher() i=2 2024-05-30T16:17:00.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.787+0000 7fb520c66980 20 add_watcher() i=3 2024-05-30T16:17:00.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.787+0000 7fb520c66980 20 add_watcher() i=4 2024-05-30T16:17:00.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.787+0000 7fb520c66980 20 add_watcher() i=5 2024-05-30T16:17:00.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.791+0000 7fb520c66980 20 add_watcher() i=6 2024-05-30T16:17:00.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.791+0000 7fb520c66980 20 add_watcher() i=7 2024-05-30T16:17:00.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.791+0000 7fb520c66980 2 all 8 watchers are set, enabling cache 2024-05-30T16:17:00.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.795+0000 7fb3d4fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:17:00.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.795+0000 7fb520c66980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:17:00.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.795+0000 7fb520c66980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:17:00.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.795+0000 7fb520c66980 5 note: GC not initialized 2024-05-30T16:17:00.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.795+0000 7fb34f7fe640 20 reqs_thread_entry: start 2024-05-30T16:17:00.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.799+0000 7fb520c66980 20 init_complete bucket index max shards: 11 2024-05-30T16:17:00.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.799+0000 7fb520c66980 20 Filter name: none 2024-05-30T16:17:00.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.799+0000 7fb34d7fa640 20 reqs_thread_entry: start 2024-05-30T16:17:00.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.799+0000 7fb520c66980 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : miss 2024-05-30T16:17:00.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.799+0000 7fb520c66980 20 rados->read ofs=0 len=0 2024-05-30T16:17:00.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.799+0000 7fb520c66980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:17:00.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.799+0000 7fb520c66980 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x0 2024-05-30T16:17:00.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.799+0000 7fb520c66980 10 adding default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2024-05-30T16:17:00.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.799+0000 7fb520c66980 10 cache get: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test : miss 2024-05-30T16:17:00.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.799+0000 7fb520c66980 20 rados->read ofs=0 len=0 2024-05-30T16:17:00.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.799+0000 7fb520c66980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:17:00.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.799+0000 7fb520c66980 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x0 2024-05-30T16:17:00.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.799+0000 7fb520c66980 10 adding default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2024-05-30T16:17:00.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.799+0000 7fb520c66980 10 cache get: name=default.rgw.meta+users.keys+SEQBSIIZTUZARSATLBUJ : miss 2024-05-30T16:17:00.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.799+0000 7fb520c66980 20 rados->read ofs=0 len=0 2024-05-30T16:17:00.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.799+0000 7fb520c66980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:17:00.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.799+0000 7fb520c66980 10 cache put: name=default.rgw.meta+users.keys+SEQBSIIZTUZARSATLBUJ info.flags=0x0 2024-05-30T16:17:00.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.799+0000 7fb520c66980 10 adding default.rgw.meta+users.keys+SEQBSIIZTUZARSATLBUJ to cache LRU end 2024-05-30T16:17:00.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.799+0000 7fb520c66980 10 cache get: name=default.rgw.meta+users.keys+SEQBSIIZTUZARSATLBUJ : hit (negative entry) 2024-05-30T16:17:00.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.799+0000 7fb520c66980 10 cache get: name=default.rgw.meta+users.keys+SEQBSIIZTUZARSATLBUJ : hit (negative entry) 2024-05-30T16:17:00.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.803+0000 7fb520c66980 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x17 2024-05-30T16:17:00.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.803+0000 7fb520c66980 10 moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2024-05-30T16:17:00.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.803+0000 7fb520c66980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.uid:sysuser.client.0, ofs0, ns] 2024-05-30T16:17:00.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.803+0000 7fb401ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1090921693196 cookie 93921062100960 notifier 5633 bl.length()=675 2024-05-30T16:17:00.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.803+0000 7fb401ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x17 2024-05-30T16:17:00.804 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.803+0000 7fb401ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2024-05-30T16:17:00.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.811+0000 7fb520c66980 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x7 2024-05-30T16:17:00.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.811+0000 7fb520c66980 10 moving default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2024-05-30T16:17:00.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.811+0000 7fb520c66980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.email:sysuser.client.0+test@test.test, ofs0, ns] 2024-05-30T16:17:00.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.811+0000 7fb4017fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1090921693206 cookie 93921062107888 notifier 5633 bl.length()=202 2024-05-30T16:17:00.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.811+0000 7fb4017fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x7 2024-05-30T16:17:00.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.811+0000 7fb4017fa640 10 rgw watcher librados: moving default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2024-05-30T16:17:00.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.815+0000 7fb520c66980 10 cache put: name=default.rgw.meta+users.keys+SEQBSIIZTUZARSATLBUJ info.flags=0x7 2024-05-30T16:17:00.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.815+0000 7fb520c66980 10 moving default.rgw.meta+users.keys+SEQBSIIZTUZARSATLBUJ to cache LRU end 2024-05-30T16:17:00.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.815+0000 7fb520c66980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.keys:SEQBSIIZTUZARSATLBUJ, ofs0, ns] 2024-05-30T16:17:00.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.815+0000 7fb401ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1090921693207 cookie 93921062070096 notifier 5633 bl.length()=190 2024-05-30T16:17:00.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.815+0000 7fb401ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+SEQBSIIZTUZARSATLBUJ info.flags=0x7 2024-05-30T16:17:00.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.815+0000 7fb401ffb640 10 rgw watcher librados: moving default.rgw.meta+users.keys+SEQBSIIZTUZARSATLBUJ to cache LRU end 2024-05-30T16:17:00.819 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-30T16:17:00.819 INFO:teuthology.orchestra.run.smithi006.stdout: "user_id": "sysuser.client.0", 2024-05-30T16:17:00.819 INFO:teuthology.orchestra.run.smithi006.stdout: "display_name": "Mr. sysuser.client.0", 2024-05-30T16:17:00.819 INFO:teuthology.orchestra.run.smithi006.stdout: "email": "sysuser.client.0+test@test.test", 2024-05-30T16:17:00.819 INFO:teuthology.orchestra.run.smithi006.stdout: "suspended": 0, 2024-05-30T16:17:00.819 INFO:teuthology.orchestra.run.smithi006.stdout: "max_buckets": 1000, 2024-05-30T16:17:00.820 INFO:teuthology.orchestra.run.smithi006.stdout: "subusers": [], 2024-05-30T16:17:00.820 INFO:teuthology.orchestra.run.smithi006.stdout: "keys": [ 2024-05-30T16:17:00.820 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T16:17:00.820 INFO:teuthology.orchestra.run.smithi006.stdout: "user": "sysuser.client.0", 2024-05-30T16:17:00.820 INFO:teuthology.orchestra.run.smithi006.stdout: "access_key": "SEQBSIIZTUZARSATLBUJ", 2024-05-30T16:17:00.820 INFO:teuthology.orchestra.run.smithi006.stdout: "secret_key": "2c64qenxGzKyPp+jYLad/eXYbxKSLMZX9RmjXlK9HpfAklbnOr3faQ==", 2024-05-30T16:17:00.820 INFO:teuthology.orchestra.run.smithi006.stdout: "active": true, 2024-05-30T16:17:00.820 INFO:teuthology.orchestra.run.smithi006.stdout: "create_date": "2024-05-30T16:17:00.803543Z" 2024-05-30T16:17:00.820 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T16:17:00.820 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-30T16:17:00.820 INFO:teuthology.orchestra.run.smithi006.stdout: "swift_keys": [], 2024-05-30T16:17:00.820 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": [], 2024-05-30T16:17:00.820 INFO:teuthology.orchestra.run.smithi006.stdout: "op_mask": "read, write, delete", 2024-05-30T16:17:00.820 INFO:teuthology.orchestra.run.smithi006.stdout: "system": true, 2024-05-30T16:17:00.821 INFO:teuthology.orchestra.run.smithi006.stdout: "default_placement": "", 2024-05-30T16:17:00.821 INFO:teuthology.orchestra.run.smithi006.stdout: "default_storage_class": "", 2024-05-30T16:17:00.821 INFO:teuthology.orchestra.run.smithi006.stdout: "placement_tags": [], 2024-05-30T16:17:00.821 INFO:teuthology.orchestra.run.smithi006.stdout: "bucket_quota": { 2024-05-30T16:17:00.821 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": false, 2024-05-30T16:17:00.821 INFO:teuthology.orchestra.run.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:17:00.821 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size": -1, 2024-05-30T16:17:00.821 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:17:00.821 INFO:teuthology.orchestra.run.smithi006.stdout: "max_objects": -1 2024-05-30T16:17:00.821 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T16:17:00.821 INFO:teuthology.orchestra.run.smithi006.stdout: "user_quota": { 2024-05-30T16:17:00.821 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": false, 2024-05-30T16:17:00.821 INFO:teuthology.orchestra.run.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:17:00.821 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size": -1, 2024-05-30T16:17:00.822 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:17:00.822 INFO:teuthology.orchestra.run.smithi006.stdout: "max_objects": -1 2024-05-30T16:17:00.822 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T16:17:00.822 INFO:teuthology.orchestra.run.smithi006.stdout: "temp_url_keys": [], 2024-05-30T16:17:00.822 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "rgw", 2024-05-30T16:17:00.822 INFO:teuthology.orchestra.run.smithi006.stdout: "mfa_ids": [], 2024-05-30T16:17:00.822 INFO:teuthology.orchestra.run.smithi006.stdout: "account_id": "", 2024-05-30T16:17:00.822 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2024-05-30T16:17:00.822 INFO:teuthology.orchestra.run.smithi006.stdout: "create_date": "2024-05-30T16:17:00.803486Z", 2024-05-30T16:17:00.822 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-30T16:17:00.822 INFO:teuthology.orchestra.run.smithi006.stdout: "group_ids": [] 2024-05-30T16:17:00.822 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-30T16:17:00.822 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:17:00.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.819+0000 7fb3d4fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:17:00.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.827+0000 7fb520c66980 20 remove_watcher() i=0 2024-05-30T16:17:00.825 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.827+0000 7fb520c66980 2 removed watcher, disabling cache 2024-05-30T16:17:00.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.827+0000 7fb520c66980 20 remove_watcher() i=1 2024-05-30T16:17:00.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.831+0000 7fb520c66980 20 remove_watcher() i=2 2024-05-30T16:17:00.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.831+0000 7fb520c66980 20 remove_watcher() i=3 2024-05-30T16:17:00.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.835+0000 7fb520c66980 20 remove_watcher() i=4 2024-05-30T16:17:00.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.839+0000 7fb520c66980 20 remove_watcher() i=5 2024-05-30T16:17:00.838 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.839+0000 7fb520c66980 20 remove_watcher() i=6 2024-05-30T16:17:00.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:17:00.839+0000 7fb520c66980 20 remove_watcher() i=7 2024-05-30T16:17:00.857 INFO:tasks.ragweed:Configuring ragweed... 2024-05-30T16:17:00.857 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T16:17:00.857 DEBUG:teuthology.orchestra.run.smithi006:> dd of=/home/ubuntu/cephtest/archive/ragweed.client.0.conf 2024-05-30T16:17:00.906 INFO:tasks.ragweed:Configuring boto... 2024-05-30T16:17:00.906 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T16:17:00.906 DEBUG:teuthology.orchestra.run.smithi006:> dd of=/home/ubuntu/cephtest/boto.cfg 2024-05-30T16:17:00.954 DEBUG:teuthology.orchestra.run.smithi006:ragweed tests against rgw> source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/ragweed.client.0 && RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg tox --sitepackages -- -v -m 'not fails_on_rgw' 2024-05-30T16:17:01.134 INFO:teuthology.orchestra.run.smithi006.stdout:GLOB sdist-make: /home/ubuntu/cephtest/ragweed.client.0/setup.py 2024-05-30T16:17:01.317 INFO:teuthology.orchestra.run.smithi006.stdout:py create: /home/ubuntu/cephtest/ragweed.client.0/.tox/py 2024-05-30T16:17:01.868 INFO:teuthology.orchestra.run.smithi006.stdout:py installdeps: -rrequirements.txt 2024-05-30T16:17:05.311 INFO:teuthology.orchestra.run.smithi006.stdout:py inst: /home/ubuntu/cephtest/ragweed.client.0/.tox/.tmp/package/1/ragweed-0.0.1.zip 2024-05-30T16:17:08.330 INFO:teuthology.orchestra.run.smithi006.stdout:py installed: appdirs==1.4.4,asyncssh==2.5.0,attrs==21.2.0,Babel==2.8.0,bcrypt==3.2.0,beautifulsoup4==4.10.0,beniget==0.4.1,blinker==1.4,boto==2.49.0,Brotli==1.0.9,cachetools==5.0.0,ceph==1.0.0,ceph-volume==1.0.0,cephfs==2.0.0,certifi==2020.6.20,chardet==4.0.0,cheroot==8.5.2+ds1,CherryPy==18.6.1,command-not-found==0.3,configobj==5.0.6,cryptography==3.4.8,cycler==0.11.0,dbus-python==1.2.18,decorator==4.4.2,distlib==0.3.4,distro==1.7.0,distro-info===1.1build1,docopt==0.6.2,execnet==1.9.0,filelock==3.6.0,fonttools==4.29.1,fs==2.4.12,gast==0.5.2,gevent==21.8.0,google-auth==1.5.1,greenlet==1.1.2,html5lib==1.1,httplib2==0.20.2,idna==3.3,importlib-metadata==4.6.4,iniconfig==1.1.1,isodate==0.6.1,jaraco.classes==3.2.1,jaraco.collections==3.4.0,jaraco.functools==3.4.0,jaraco.text==3.6.0,jeepney==0.7.1,Jinja2==3.0.3,joblib==0.17.0,keyring==23.5.0,kiwisolver==1.3.2,kubernetes==12.0.1,language-selector==0.1,launchpadlib==1.10.16,lazr.restfulclient==0.14.4,lazr.uri==1.0.6,logutils==0.3.3,lxml==4.8.0,lz4==3.1.3+dfsg,Mako==1.1.3,MarkupSafe==2.0.1,matplotlib==3.5.1,more-itertools==8.10.0,mpmath==0.0.0,munch==4.0.0,natsort==8.0.2,netifaces==0.11.0,nose==1.3.7,numpy==1.21.5,oauthlib==3.2.0,olefile==0.46,packaging==21.3,Paste==3.5.0,PasteDeploy==2.1.1,PasteScript==2.0.2,pecan==1.3.3,Pillow==9.0.1,platformdirs==2.5.1,pluggy==0.13.0,ply==3.11,portend==3.0.0,prettytable==2.5.0,psutil==5.9.0,py==1.10.0,pyasn1==0.4.8,pyasn1-modules==0.2.1,Pygments==2.11.2,PyGObject==3.42.1,pyinotify==0.9.6,PyJWT==2.3.0,pymacaroons==0.13.0,PyNaCl==1.5.0,pyOpenSSL==21.0.0,pyparsing==2.4.7,pytest==6.2.5,python-apt==2.3.0+ubuntu2.1,python-dateutil==2.8.1,python-magic==0.4.24,pythran==0.10.0,pytz==2022.1,PyYAML==5.4.1,rados==2.0.0,ragweed @ file:///home/ubuntu/cephtest/ragweed.client.0/.tox/.tmp/package/1/ragweed-0.0.1.zip#sha256=283e8d90d096b8eab5db97cfa70d39fd8cb6a3a63ffaa4aeb5da5faa61f45ff4,rbd==2.0.0,registries-conf-ctl @ git+https://github.com/sebastian-philipp/registries-conf-ctl@0e9bf2dc202aa9be9b0bf8e8265aac3f2308f9db,remoto==1.2.1,repoze.lru==0.7,requests==2.25.1,requests-oauthlib==1.3.0,rgw==2.0.0,Routes==2.5.1,rsa==4.8,s3cmd==2.2.0,scikit-learn==0.23.2,scipy==1.8.0,SecretStorage==3.3.1,simplegeneric==0.8.1,simplejson==3.17.6,six==1.16.0,soupsieve==2.3.1,ssh-import-id==5.11,sympy==1.9,Tempita==0.5.2,tempora==4.1.2,threadpoolctl==3.1.0,toml==0.10.2,ubuntu-advantage-tools==27.12,ufoLib2==0.13.1,ufw==0.36.1,unattended-upgrades==0.1,unicodedata2==14.0.0,urllib3==1.26.5,virtualenv==20.13.0+ds,wadllib==1.3.6,waitress==1.4.4,wcwidth==0.2.5,webencodings==0.5.1,WebOb==1.8.6,websocket-client==1.2.3,WebTest==2.0.35,Werkzeug==2.0.2,zc.lockfile==2.0,zipp==1.0.0,zope.event==4.4,zope.interface==5.4.0 2024-05-30T16:17:08.330 INFO:teuthology.orchestra.run.smithi006.stdout:py run-test-pre: PYTHONHASHSEED='2511159263' 2024-05-30T16:17:08.330 INFO:teuthology.orchestra.run.smithi006.stdout:py run-test: commands[0] | python -m pytest -v -m 'not fails_on_rgw' 2024-05-30T16:17:08.512 INFO:teuthology.orchestra.run.smithi006.stdout:============================= test session starts ============================== 2024-05-30T16:17:08.512 INFO:teuthology.orchestra.run.smithi006.stdout:platform linux -- Python 3.10.6, pytest-6.2.5, py-1.10.0, pluggy-0.13.0 -- /home/ubuntu/cephtest/ragweed.client.0/.tox/py/bin/python 2024-05-30T16:17:08.513 INFO:teuthology.orchestra.run.smithi006.stdout:cachedir: .tox/py/.pytest_cache 2024-05-30T16:17:08.513 INFO:teuthology.orchestra.run.smithi006.stdout:rootdir: /home/ubuntu/cephtest/ragweed.client.0, configfile: pytest.ini 2024-05-30T16:17:08.676 INFO:teuthology.orchestra.run.smithi006.stdout:collecting ... collected 9 items 2024-05-30T16:17:08.676 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:17:13.722 INFO:teuthology.orchestra.run.smithi006.stdout:ragweed/tests/tests.py::r_test_small_obj_data::test PASSED [ 11%] 2024-05-30T16:17:17.666 INFO:teuthology.orchestra.run.smithi006.stdout:ragweed/tests/tests.py::r_test_multipart_simple::test PASSED [ 22%] 2024-05-30T16:17:21.307 INFO:teuthology.orchestra.run.smithi006.stdout:ragweed/tests/tests.py::r_test_multipart_index_versioning::test PASSED [ 33%] 2024-05-30T16:17:25.439 INFO:teuthology.orchestra.run.smithi006.stdout:ragweed/tests/tests.py::r_test_multipart_defer_complete::test PASSED [ 44%] 2024-05-30T16:17:29.107 INFO:teuthology.orchestra.run.smithi006.stdout:ragweed/tests/tests.py::r_test_multipart_defer_update_complete::test PASSED [ 55%] 2024-05-30T16:17:40.605 INFO:teuthology.orchestra.run.smithi006.stdout:ragweed/tests/tests.py::r_test_obj_storage_class::test PASSED [ 66%] 2024-05-30T16:17:49.612 INFO:teuthology.orchestra.run.smithi006.stdout:ragweed/tests/tests.py::r_test_obj_storage_class_multipart::test PASSED [ 77%] 2024-05-30T16:18:00.722 INFO:teuthology.orchestra.run.smithi006.stdout:ragweed/tests/tests.py::r_test_obj_storage_class_copy::test PASSED [ 88%] 2024-05-30T16:18:17.609 INFO:teuthology.orchestra.run.smithi006.stdout:ragweed/tests/tests.py::r_test_obj_storage_class_multipart_copy::test PASSED [100%] 2024-05-30T16:18:17.609 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:18:17.609 INFO:teuthology.orchestra.run.smithi006.stdout:=============================== warnings summary =============================== 2024-05-30T16:18:17.609 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/boto/plugin.py:40 2024-05-30T16:18:17.609 INFO:teuthology.orchestra.run.smithi006.stdout: /home/ubuntu/cephtest/ragweed.client.0/.tox/py/lib/python3.10/site-packages/boto/plugin.py:40: DeprecationWarning: the imp module is deprecated in favour of importlib and slated for removal in Python 3.12; see the module's documentation for alternative uses 2024-05-30T16:18:17.610 INFO:teuthology.orchestra.run.smithi006.stdout: import imp 2024-05-30T16:18:17.610 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:18:17.610 INFO:teuthology.orchestra.run.smithi006.stdout:-- Docs: https://docs.pytest.org/en/stable/warnings.html 2024-05-30T16:18:17.610 INFO:teuthology.orchestra.run.smithi006.stdout:=================== 9 passed, 1 warning in 69.10s (0:01:09) ==================== 2024-05-30T16:18:17.662 INFO:teuthology.orchestra.run.smithi006.stdout:___________________________________ summary ____________________________________ 2024-05-30T16:18:17.662 INFO:teuthology.orchestra.run.smithi006.stdout: py: commands succeeded 2024-05-30T16:18:17.662 INFO:teuthology.orchestra.run.smithi006.stdout: congratulations :) 2024-05-30T16:18:17.683 INFO:tasks.ragweed:Cleaning up boto... 2024-05-30T16:18:17.684 DEBUG:teuthology.orchestra.run.smithi006:> rm -f /home/ubuntu/cephtest/boto.cfg 2024-05-30T16:18:17.690 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid ragweed.client.0 --purge-data 2024-05-30T16:18:17.770 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:18:17.770 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:18:17.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:17.810+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:17.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:17.814+0000 7fbc1ba93980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:18:17.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:17.814+0000 7fbc1ba93980 20 realm 2024-05-30T16:18:17.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:17.814+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:17.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:17.814+0000 7fbc1ba93980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:18:17.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:17.814+0000 7fbc1ba93980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:18:17.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:17.814+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:17.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:17.814+0000 7fbc1ba93980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:18:17.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:17.814+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:17.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:17.814+0000 7fbc1ba93980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:18:17.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:17.814+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:17.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:17.818+0000 7fbc1ba93980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:18:17.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:17.818+0000 7fbc1ba93980 20 searching for the correct realm 2024-05-30T16:18:17.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:17.822+0000 7fbc19da5640 0 -- 172.21.15.6:0/827276860 >> [v2:172.21.15.204:6800/3687190393,v1:172.21.15.204:6801/3687190393] conn(0x7fbc10070860 msgr2=0x7fbc100880c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:18:18.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.038+0000 7fbc195a4640 0 -- 172.21.15.6:0/827276860 >> [v2:172.21.15.204:6816/2677861390,v1:172.21.15.204:6817/2677861390] conn(0x55968968ae70 msgr2=0x55968968d2c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:18:18.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.046+0000 7fbc1ba93980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:18:18.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.046+0000 7fbc1ba93980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:18:18.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.046+0000 7fbc1ba93980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:18:18.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.046+0000 7fbc1ba93980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:18:18.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.046+0000 7fbc1ba93980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:18:18.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.046+0000 7fbc1ba93980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:18:18.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.046+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.046+0000 7fbc1ba93980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:18:18.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.046+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.046+0000 7fbc1ba93980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:18:18.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.046+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.050+0000 7fbc1ba93980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:18:18.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.050+0000 7fbc1ba93980 20 zone default found 2024-05-30T16:18:18.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.050+0000 7fbc1ba93980 4 Realm: () 2024-05-30T16:18:18.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.050+0000 7fbc1ba93980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:18:18.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.050+0000 7fbc1ba93980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:18:18.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.050+0000 7fbc1ba93980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:18:18.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.050+0000 7fbc1ba93980 20 zonegroup default 2024-05-30T16:18:18.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.050+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.050+0000 7fbc1ba93980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:18:18.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.050+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.050+0000 7fbc1ba93980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:18:18.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.050+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.050+0000 7fbc1ba93980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:18:18.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.050+0000 7fbc1ba93980 20 started sync module instance, tier type = 2024-05-30T16:18:18.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.050+0000 7fbc1ba93980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:18:18.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.074+0000 7fbc1ba93980 20 add_watcher() i=0 2024-05-30T16:18:18.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.074+0000 7fbc1ba93980 20 add_watcher() i=1 2024-05-30T16:18:18.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.074+0000 7fbc1ba93980 20 add_watcher() i=2 2024-05-30T16:18:18.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.074+0000 7fbc1ba93980 20 add_watcher() i=3 2024-05-30T16:18:18.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.074+0000 7fbc1ba93980 20 add_watcher() i=4 2024-05-30T16:18:18.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.074+0000 7fbc1ba93980 20 add_watcher() i=5 2024-05-30T16:18:18.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.078+0000 7fbc1ba93980 20 add_watcher() i=6 2024-05-30T16:18:18.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.078+0000 7fbc1ba93980 20 add_watcher() i=7 2024-05-30T16:18:18.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.078+0000 7fbc1ba93980 2 all 8 watchers are set, enabling cache 2024-05-30T16:18:18.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.082+0000 7fbacf7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:18:18.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.082+0000 7fbc1ba93980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:18:18.084 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.082+0000 7fbc1ba93980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:18:18.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.178+0000 7fbace7e4640 20 reqs_thread_entry: start 2024-05-30T16:18:18.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.182+0000 7fbc1ba93980 20 init_complete bucket index max shards: 11 2024-05-30T16:18:18.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.182+0000 7fbc1ba93980 20 Filter name: none 2024-05-30T16:18:18.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.182+0000 7fba3ffff640 20 reqs_thread_entry: start 2024-05-30T16:18:18.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.182+0000 7fbc1ba93980 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : miss 2024-05-30T16:18:18.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.182+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.182+0000 7fbc1ba93980 20 rados_obj.operate() r=0 bl.length=486 2024-05-30T16:18:18.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.182+0000 7fbc1ba93980 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x13 2024-05-30T16:18:18.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.182+0000 7fbc1ba93980 10 adding default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2024-05-30T16:18:18.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.182+0000 7fbc1ba93980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:18:18.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.182+0000 7fbc1ba93980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.5 : miss 2024-05-30T16:18:18.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.182+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.186+0000 7fbc1ba93980 20 rados_obj.operate() r=0 bl.length=418 2024-05-30T16:18:18.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.186+0000 7fbc1ba93980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.5 info.flags=0x17 2024-05-30T16:18:18.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.186+0000 7fbc1ba93980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-conf:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.5 to cache LRU end 2024-05-30T16:18:18.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.186+0000 7fbc1ba93980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:18:18.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.186+0000 7fbc1ba93980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-05-30T16:18:18.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.186+0000 7fbc1ba93980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-conf:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.5 2024-05-30T16:18:18.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.186+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class_multipart_copy 2024-05-30T16:18:18.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.190+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got tests/r_test_multipart_index_versioning 2024-05-30T16:18:18.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.190+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class_copy 2024-05-30T16:18:18.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.190+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got tests/r_test_multipart_defer_update_complete 2024-05-30T16:18:18.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.190+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got tests/r_test_multipart_defer_complete 2024-05-30T16:18:18.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.190+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class 2024-05-30T16:18:18.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.190+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got tests/r_test_multipart_simple 2024-05-30T16:18:18.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.194+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class_multipart 2024-05-30T16:18:18.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.194+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got tests/r_test_small_obj_data 2024-05-30T16:18:18.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.198+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc14077040 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart_copy state=0x559689740648 s->prefetch_data=0 2024-05-30T16:18:18.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.198+0000 7fbc1ba93980 10 manifest: total_size = 5328 2024-05-30T16:18:18.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.198+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.4297999016613179313 2024-05-30T16:18:18.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.198+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc14077040 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart_copy state=0x559689740648 s->prefetch_data=0 2024-05-30T16:18:18.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.198+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x559689740648 2024-05-30T16:18:18.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.198+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.5.1 2024-05-30T16:18:18.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.206+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc14077040 obj=test-client.0-conf:tests/r_test_multipart_index_versioning state=0x559689740608 s->prefetch_data=0 2024-05-30T16:18:18.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.206+0000 7fbc1ba93980 10 manifest: total_size = 5123 2024-05-30T16:18:18.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.206+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11525266850379308638 2024-05-30T16:18:18.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.206+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc14077040 obj=test-client.0-conf:tests/r_test_multipart_index_versioning state=0x559689740608 s->prefetch_data=0 2024-05-30T16:18:18.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.206+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x559689740608 2024-05-30T16:18:18.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.206+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.5.2 2024-05-30T16:18:18.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.206+0000 7fbafcff9640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class_multipart_copy 2024-05-30T16:18:18.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.214+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc14077040 obj=test-client.0-conf:tests/r_test_obj_storage_class_copy state=0x5596897400f8 s->prefetch_data=0 2024-05-30T16:18:18.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.214+0000 7fbc1ba93980 10 manifest: total_size = 7728 2024-05-30T16:18:18.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.214+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.4977425390364985828 2024-05-30T16:18:18.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.214+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc14077040 obj=test-client.0-conf:tests/r_test_obj_storage_class_copy state=0x5596897400f8 s->prefetch_data=0 2024-05-30T16:18:18.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.214+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596897400f8 2024-05-30T16:18:18.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.214+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.5.2 2024-05-30T16:18:18.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.214+0000 7fbaeffff640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_index_versioning 2024-05-30T16:18:18.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.218+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc14077040 obj=test-client.0-conf:tests/r_test_multipart_defer_update_complete state=0x5596897400f8 s->prefetch_data=0 2024-05-30T16:18:18.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.222+0000 7fbc1ba93980 10 manifest: total_size = 13085 2024-05-30T16:18:18.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.222+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.3570463654805198501 2024-05-30T16:18:18.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.222+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc14077040 obj=test-client.0-conf:tests/r_test_multipart_defer_update_complete state=0x5596897400f8 s->prefetch_data=0 2024-05-30T16:18:18.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.222+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596897400f8 2024-05-30T16:18:18.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.222+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.5.3 2024-05-30T16:18:18.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.222+0000 7fbafcff9640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class_copy 2024-05-30T16:18:18.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.226+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc14077040 obj=test-client.0-conf:tests/r_test_multipart_defer_complete state=0x5596897400f8 s->prefetch_data=0 2024-05-30T16:18:18.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.230+0000 7fbc1ba93980 10 manifest: total_size = 5248 2024-05-30T16:18:18.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.230+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.4240449144114929145 2024-05-30T16:18:18.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.230+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc14077040 obj=test-client.0-conf:tests/r_test_multipart_defer_complete state=0x5596897400f8 s->prefetch_data=0 2024-05-30T16:18:18.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.230+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596897400f8 2024-05-30T16:18:18.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.230+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.5.4 2024-05-30T16:18:18.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.230+0000 7fbaeffff640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_defer_update_complete 2024-05-30T16:18:18.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.238+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc14077040 obj=test-client.0-conf:tests/r_test_obj_storage_class state=0x5596897400f8 s->prefetch_data=0 2024-05-30T16:18:18.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.238+0000 7fbc1ba93980 10 manifest: total_size = 4947 2024-05-30T16:18:18.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.238+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.13730088469358338515 2024-05-30T16:18:18.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.238+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc14077040 obj=test-client.0-conf:tests/r_test_obj_storage_class state=0x5596897400f8 s->prefetch_data=0 2024-05-30T16:18:18.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.238+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596897400f8 2024-05-30T16:18:18.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.238+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.5.4 2024-05-30T16:18:18.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.238+0000 7fbafcff9640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_defer_complete 2024-05-30T16:18:18.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.246+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896cb180 obj=test-client.0-conf:tests/r_test_multipart_simple state=0x5596897400f8 s->prefetch_data=0 2024-05-30T16:18:18.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.246+0000 7fbc1ba93980 10 manifest: total_size = 7430 2024-05-30T16:18:18.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.246+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.711106160848865759 2024-05-30T16:18:18.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.246+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896cb180 obj=test-client.0-conf:tests/r_test_multipart_simple state=0x5596897400f8 s->prefetch_data=0 2024-05-30T16:18:18.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.246+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596897400f8 2024-05-30T16:18:18.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.246+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.5.5 2024-05-30T16:18:18.248 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.246+0000 7fbaeffff640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class 2024-05-30T16:18:18.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.250+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896cb180 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart state=0x5596897400f8 s->prefetch_data=0 2024-05-30T16:18:18.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.254+0000 7fbc1ba93980 10 manifest: total_size = 5140 2024-05-30T16:18:18.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.254+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.6584126948383207797 2024-05-30T16:18:18.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.254+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896cb180 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart state=0x5596897400f8 s->prefetch_data=0 2024-05-30T16:18:18.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.254+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596897400f8 2024-05-30T16:18:18.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.254+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.5.7 2024-05-30T16:18:18.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.254+0000 7fbafcff9640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_simple 2024-05-30T16:18:18.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.258+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896c2000 obj=test-client.0-conf:tests/r_test_small_obj_data state=0x5596897400f8 s->prefetch_data=0 2024-05-30T16:18:18.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.258+0000 7fbc1ba93980 10 manifest: total_size = 5422 2024-05-30T16:18:18.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.258+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.13072889876356702901 2024-05-30T16:18:18.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.258+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896c2000 obj=test-client.0-conf:tests/r_test_small_obj_data state=0x5596897400f8 s->prefetch_data=0 2024-05-30T16:18:18.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.258+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596897400f8 2024-05-30T16:18:18.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.258+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.5.8 2024-05-30T16:18:18.263 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.262+0000 7fbaeffff640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class_multipart 2024-05-30T16:18:18.267 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.266+0000 7fbc1ba93980 20 list_objects_ordered: starting attempt 1 2024-05-30T16:18:18.267 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.266+0000 7fbc1ba93980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-05-30T16:18:18.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.270+0000 7fbafcff9640 20 handle_completion(): completion ok for obj=tests/r_test_small_obj_data 2024-05-30T16:18:18.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.270+0000 7fbc1ba93980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-05-30T16:18:18.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.270+0000 7fbc1ba93980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-05-30T16:18:18.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.270+0000 7fbc1ba93980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-05-30T16:18:18.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.270+0000 7fbc1ba93980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-05-30T16:18:18.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.270+0000 7fbc1ba93980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-conf[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.5])"; uploads.size()=0; is_truncated=0 2024-05-30T16:18:18.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.274+0000 7fbc1ba93980 10 cache get: name=default.rgw.meta+root+test-client.0-conf : miss 2024-05-30T16:18:18.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.274+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.274+0000 7fbc1ba93980 20 rados_obj.operate() r=0 bl.length=204 2024-05-30T16:18:18.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.274+0000 7fbc1ba93980 10 cache put: name=default.rgw.meta+root+test-client.0-conf info.flags=0x11 2024-05-30T16:18:18.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.274+0000 7fbc1ba93980 10 adding default.rgw.meta+root+test-client.0-conf to cache LRU end 2024-05-30T16:18:18.278 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.278+0000 7fbc1ba93980 10 removing default.rgw.meta+root+test-client.0-conf from cache 2024-05-30T16:18:18.278 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.278+0000 7fbc1ba93980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-conf, ofs0, ns] 2024-05-30T16:18:18.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.278+0000 7fbaeffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431584 cookie 94105038800128 notifier 5889 bl.length()=162 2024-05-30T16:18:18.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.286+0000 7fbc1ba93980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-conf:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.5 from cache 2024-05-30T16:18:18.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.286+0000 7fbc1ba93980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-conf:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.5, ofs0, ns] 2024-05-30T16:18:18.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.286+0000 7fbaeffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431615 cookie 94105038840864 notifier 5889 bl.length()=219 2024-05-30T16:18:18.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.294+0000 7fbc1ba93980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-conf/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.5 : miss 2024-05-30T16:18:18.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.294+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.294+0000 7fbc1ba93980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:18:18.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.294+0000 7fbc1ba93980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-conf/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.5 info.flags=0x0 2024-05-30T16:18:18.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.294+0000 7fbc1ba93980 10 adding default.rgw.log++pubsub..bucket.test-client.0-conf/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.5 to cache LRU end 2024-05-30T16:18:18.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.298+0000 7fbc1ba93980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12 : miss 2024-05-30T16:18:18.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.298+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.298+0000 7fbc1ba93980 20 rados_obj.operate() r=0 bl.length=449 2024-05-30T16:18:18.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.298+0000 7fbc1ba93980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12 info.flags=0x17 2024-05-30T16:18:18.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.298+0000 7fbc1ba93980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12 to cache LRU end 2024-05-30T16:18:18.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.298+0000 7fbc1ba93980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:18:18.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.298+0000 7fbc1ba93980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-05-30T16:18:18.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.298+0000 7fbc1ba93980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12 2024-05-30T16:18:18.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.310+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo 2024-05-30T16:18:18.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.310+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896ca050 obj=test-client.0-r-test-multipart-defer-complete.1:foo state=0x5596896c7c48 s->prefetch_data=0 2024-05-30T16:18:18.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.310+0000 7fbc1ba93980 10 manifest: total_size = 74198 2024-05-30T16:18:18.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.310+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12830297622961229936 2024-05-30T16:18:18.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.310+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896ca050 obj=test-client.0-r-test-multipart-defer-complete.1:foo state=0x5596896c7c48 s->prefetch_data=0 2024-05-30T16:18:18.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.310+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896c7c48 2024-05-30T16:18:18.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.310+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12.10 2024-05-30T16:18:18.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.318+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2024-05-30T16:18:18.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.318+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=20613 2024-05-30T16:18:18.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.318+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=20613 stripe_ofs=20613 part_ofs=20613 rule->part_size=20613 2024-05-30T16:18:18.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.318+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2024-05-30T16:18:18.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.318+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6312069 part_ofs=20613 rule->part_size=20613 2024-05-30T16:18:18.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.318+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=41226 stripe_ofs=41226 part_ofs=41226 rule->part_size=20613 2024-05-30T16:18:18.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.318+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2024-05-30T16:18:18.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.318+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6332682 part_ofs=41226 rule->part_size=20613 2024-05-30T16:18:18.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.318+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=61839 stripe_ofs=61839 part_ofs=61839 rule->part_size=12359 2024-05-30T16:18:18.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.318+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=12359 rules.size()=2 2024-05-30T16:18:18.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.318+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6353295 part_ofs=61839 rule->part_size=12359 2024-05-30T16:18:18.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.318+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=74198 stripe_ofs=74198 part_ofs=74198 rule->part_size=12359 2024-05-30T16:18:18.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.318+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12830297622961229936 2024-05-30T16:18:18.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.318+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:18.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.318+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12__multipart_foo.2~oEALAIvQxkhRs-gz1eBFd-L8SHqQ_ZD.1 2024-05-30T16:18:18.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.318+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 347 2024-05-30T16:18:18.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.318+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12__multipart_foo.2~oEALAIvQxkhRs-gz1eBFd-L8SHqQ_ZD.2 2024-05-30T16:18:18.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.318+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 593 2024-05-30T16:18:18.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.318+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12__multipart_foo.2~oEALAIvQxkhRs-gz1eBFd-L8SHqQ_ZD.3 2024-05-30T16:18:18.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.318+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 839 2024-05-30T16:18:18.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.318+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12__multipart_foo.2~oEALAIvQxkhRs-gz1eBFd-L8SHqQ_ZD.4 2024-05-30T16:18:18.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.318+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1085 2024-05-30T16:18:18.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.318+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:18.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.318+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.26tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12830297622961229936 2024-05-30T16:18:18.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.322+0000 7fbc1ba93980 20 list_objects_ordered: starting attempt 1 2024-05-30T16:18:18.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.322+0000 7fbc1ba93980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-05-30T16:18:18.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.322+0000 7fbafcff9640 20 handle_completion(): completion ok for obj=foo 2024-05-30T16:18:18.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.322+0000 7fbc1ba93980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-05-30T16:18:18.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.322+0000 7fbc1ba93980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-05-30T16:18:18.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.322+0000 7fbc1ba93980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-05-30T16:18:18.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.322+0000 7fbc1ba93980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-05-30T16:18:18.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.322+0000 7fbc1ba93980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-defer-complete.1[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12])"; uploads.size()=0; is_truncated=0 2024-05-30T16:18:18.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.326+0000 7fbc1ba93980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : miss 2024-05-30T16:18:18.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.326+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.326+0000 7fbc1ba93980 20 rados_obj.operate() r=0 bl.length=235 2024-05-30T16:18:18.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.326+0000 7fbc1ba93980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 info.flags=0x11 2024-05-30T16:18:18.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.326+0000 7fbc1ba93980 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 to cache LRU end 2024-05-30T16:18:18.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.330+0000 7fbc1ba93980 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 from cache 2024-05-30T16:18:18.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.330+0000 7fbc1ba93980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-defer-complete.1, ofs0, ns] 2024-05-30T16:18:18.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.330+0000 7fbafcff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431605 cookie 94105038829744 notifier 5889 bl.length()=191 2024-05-30T16:18:18.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.338+0000 7fbc1ba93980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12 from cache 2024-05-30T16:18:18.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.338+0000 7fbc1ba93980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12, ofs0, ns] 2024-05-30T16:18:18.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.338+0000 7fbafcff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431606 cookie 94105038829744 notifier 5889 bl.length()=249 2024-05-30T16:18:18.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.346+0000 7fbc1ba93980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-complete.1/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12 : miss 2024-05-30T16:18:18.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.346+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.346+0000 7fbc1ba93980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:18:18.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.346+0000 7fbc1ba93980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-complete.1/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12 info.flags=0x0 2024-05-30T16:18:18.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.346+0000 7fbc1ba93980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-complete.1/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12 to cache LRU end 2024-05-30T16:18:18.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.354+0000 7fbc1ba93980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.13 : miss 2024-05-30T16:18:18.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.354+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.354+0000 7fbc1ba93980 20 rados_obj.operate() r=0 bl.length=456 2024-05-30T16:18:18.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.354+0000 7fbc1ba93980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.13 info.flags=0x17 2024-05-30T16:18:18.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.354+0000 7fbc1ba93980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.13 to cache LRU end 2024-05-30T16:18:18.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.354+0000 7fbc1ba93980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:18:18.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.354+0000 7fbc1ba93980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-05-30T16:18:18.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.354+0000 7fbc1ba93980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.13 2024-05-30T16:18:18.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.366+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo 2024-05-30T16:18:18.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.366+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896c78a0 obj=test-client.0-r-test-multipart-defer-update-complete.1:foo state=0x5596896c6ce8 s->prefetch_data=0 2024-05-30T16:18:18.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.366+0000 7fbc1ba93980 10 manifest: total_size = 1667 2024-05-30T16:18:18.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.366+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11952383582725206237 2024-05-30T16:18:18.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.366+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896c78a0 obj=test-client.0-r-test-multipart-defer-update-complete.1:foo state=0x5596896c6ce8 s->prefetch_data=0 2024-05-30T16:18:18.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.366+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896c6ce8 2024-05-30T16:18:18.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.366+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.13.10 2024-05-30T16:18:18.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.374+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-05-30T16:18:18.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.374+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=516 2024-05-30T16:18:18.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.374+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=516 stripe_ofs=516 part_ofs=516 rule->part_size=516 2024-05-30T16:18:18.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.374+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-05-30T16:18:18.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.374+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6291972 part_ofs=516 rule->part_size=516 2024-05-30T16:18:18.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.374+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=1032 stripe_ofs=1032 part_ofs=1032 rule->part_size=516 2024-05-30T16:18:18.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.374+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-05-30T16:18:18.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.374+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6292488 part_ofs=1032 rule->part_size=516 2024-05-30T16:18:18.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.374+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=1548 stripe_ofs=1548 part_ofs=1548 rule->part_size=119 2024-05-30T16:18:18.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.374+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=119 rules.size()=2 2024-05-30T16:18:18.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.374+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6293004 part_ofs=1548 rule->part_size=119 2024-05-30T16:18:18.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.374+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=1667 stripe_ofs=1667 part_ofs=1667 rule->part_size=119 2024-05-30T16:18:18.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.374+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11952383582725206237 2024-05-30T16:18:18.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.374+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:18.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.374+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.13__multipart_foo.2~5teFu6HIxTDVSEf9D0cZ9ZwIH5aOMFH.1 2024-05-30T16:18:18.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.374+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 347 2024-05-30T16:18:18.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.374+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.13__multipart_foo.2~5teFu6HIxTDVSEf9D0cZ9ZwIH5aOMFH.2 2024-05-30T16:18:18.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.374+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 593 2024-05-30T16:18:18.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.374+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.13__multipart_foo.2~5teFu6HIxTDVSEf9D0cZ9ZwIH5aOMFH.3 2024-05-30T16:18:18.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.374+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 839 2024-05-30T16:18:18.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.374+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.13__multipart_foo.2~5teFu6HIxTDVSEf9D0cZ9ZwIH5aOMFH.4 2024-05-30T16:18:18.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.374+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1085 2024-05-30T16:18:18.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.374+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:18.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.374+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.16tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11952383582725206237 2024-05-30T16:18:18.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.374+0000 7fbafcff9640 20 handle_completion(): completion ok for obj=foo 2024-05-30T16:18:18.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.378+0000 7fbc1ba93980 20 list_objects_ordered: starting attempt 1 2024-05-30T16:18:18.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.378+0000 7fbc1ba93980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-05-30T16:18:18.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.378+0000 7fbc1ba93980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-05-30T16:18:18.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.378+0000 7fbc1ba93980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-05-30T16:18:18.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.378+0000 7fbc1ba93980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-05-30T16:18:18.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.378+0000 7fbc1ba93980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-05-30T16:18:18.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.378+0000 7fbc1ba93980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-defer-update-complete.1[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.13])"; uploads.size()=0; is_truncated=0 2024-05-30T16:18:18.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.382+0000 7fbc1ba93980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : miss 2024-05-30T16:18:18.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.382+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.382+0000 7fbc1ba93980 20 rados_obj.operate() r=0 bl.length=242 2024-05-30T16:18:18.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.382+0000 7fbc1ba93980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 info.flags=0x11 2024-05-30T16:18:18.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.382+0000 7fbc1ba93980 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 to cache LRU end 2024-05-30T16:18:18.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.386+0000 7fbc1ba93980 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 from cache 2024-05-30T16:18:18.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.386+0000 7fbc1ba93980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-defer-update-complete.1, ofs0, ns] 2024-05-30T16:18:18.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.386+0000 7fbafcff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431570 cookie 94105038832944 notifier 5889 bl.length()=198 2024-05-30T16:18:18.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.390+0000 7fbc1ba93980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.13 from cache 2024-05-30T16:18:18.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.390+0000 7fbc1ba93980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.13, ofs0, ns] 2024-05-30T16:18:18.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.394+0000 7fbafcff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431616 cookie 94105038845488 notifier 5889 bl.length()=256 2024-05-30T16:18:18.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.402+0000 7fbc1ba93980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-update-complete.1/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.13 : miss 2024-05-30T16:18:18.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.402+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.402+0000 7fbc1ba93980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:18:18.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.402+0000 7fbc1ba93980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-update-complete.1/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.13 info.flags=0x0 2024-05-30T16:18:18.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.402+0000 7fbc1ba93980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-update-complete.1/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.13 to cache LRU end 2024-05-30T16:18:18.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.406+0000 7fbc1ba93980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11 : miss 2024-05-30T16:18:18.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.406+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.406+0000 7fbc1ba93980 20 rados_obj.operate() r=0 bl.length=451 2024-05-30T16:18:18.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.406+0000 7fbc1ba93980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11 info.flags=0x17 2024-05-30T16:18:18.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.406+0000 7fbc1ba93980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11 to cache LRU end 2024-05-30T16:18:18.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.406+0000 7fbc1ba93980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:18:18.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.406+0000 7fbc1ba93980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-05-30T16:18:18.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.406+0000 7fbc1ba93980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11 2024-05-30T16:18:18.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.418+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo[zJHnZ7Thp8aoNgNXEcRkx5cA47QgZXh] 2024-05-30T16:18:18.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.418+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11.10 2024-05-30T16:18:18.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.418+0000 7fbc1ba93980 20 get_obj_state: octx=0x559689698fe0 obj=test-client.0-r-test-multipart-index-versioning.1:foo state=0x5596896c7b38 s->prefetch_data=0 2024-05-30T16:18:18.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.418+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to ybo1epwfb6bqkqyjanmc7mf41l3enjkg 2024-05-30T16:18:18.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.418+0000 7fbc1ba93980 20 get_obj_state_impl: setting s->olh_tag to jq9dfsnx7emiwxskjo8xv4puydoqblr2 2024-05-30T16:18:18.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.418+0000 7fbc1ba93980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=jq9dfsnx7emiwxskjo8xv4puydoqblr2 2024-05-30T16:18:18.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.422+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11.10 2024-05-30T16:18:18.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.426+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11.10 2024-05-30T16:18:18.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.426+0000 7fbc1ba93980 20 olh_log_entry: epoch=3 op=2 key=foo[] 2024-05-30T16:18:18.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.426+0000 7fbc1ba93980 20 olh_log_entry: epoch=3 op=3 key=foo[zJHnZ7Thp8aoNgNXEcRkx5cA47QgZXh] 2024-05-30T16:18:18.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.426+0000 7fbc1ba93980 20 get_obj_state: octx=0x559689698fe0 obj=test-client.0-r-test-multipart-index-versioning.1:_:zJHnZ7Thp8aoNgNXEcRkx5cA47QgZXh_foo state=0x5596896d7508 s->prefetch_data=0 2024-05-30T16:18:18.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.426+0000 7fbc1ba93980 10 manifest: total_size = 1667 2024-05-30T16:18:18.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.426+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.3911489254370521540 2024-05-30T16:18:18.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.426+0000 7fbc1ba93980 20 get_obj_state: octx=0x559689698fe0 obj=test-client.0-r-test-multipart-index-versioning.1:_:zJHnZ7Thp8aoNgNXEcRkx5cA47QgZXh_foo state=0x5596896d7508 s->prefetch_data=0 2024-05-30T16:18:18.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.426+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896d7508 2024-05-30T16:18:18.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.426+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11.10 2024-05-30T16:18:18.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.434+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-05-30T16:18:18.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.434+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=516 2024-05-30T16:18:18.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.434+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=516 stripe_ofs=516 part_ofs=516 rule->part_size=516 2024-05-30T16:18:18.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.434+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-05-30T16:18:18.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.434+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6291972 part_ofs=516 rule->part_size=516 2024-05-30T16:18:18.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.434+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=1032 stripe_ofs=1032 part_ofs=1032 rule->part_size=516 2024-05-30T16:18:18.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.434+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-05-30T16:18:18.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.434+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6292488 part_ofs=1032 rule->part_size=516 2024-05-30T16:18:18.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.434+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=1548 stripe_ofs=1548 part_ofs=1548 rule->part_size=119 2024-05-30T16:18:18.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.434+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=119 rules.size()=2 2024-05-30T16:18:18.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.434+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6293004 part_ofs=1548 rule->part_size=119 2024-05-30T16:18:18.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.434+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=1667 stripe_ofs=1667 part_ofs=1667 rule->part_size=119 2024-05-30T16:18:18.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.434+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.3911489254370521540 2024-05-30T16:18:18.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.434+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:18.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.434+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11__multipart_foo.2~rsfKWR_VmhHpQAGvhcCSpjk4c7U0f4N.1 2024-05-30T16:18:18.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.434+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 346 2024-05-30T16:18:18.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.434+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11__multipart_foo.2~rsfKWR_VmhHpQAGvhcCSpjk4c7U0f4N.2 2024-05-30T16:18:18.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.434+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 592 2024-05-30T16:18:18.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.434+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11__multipart_foo.2~rsfKWR_VmhHpQAGvhcCSpjk4c7U0f4N.3 2024-05-30T16:18:18.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.434+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 838 2024-05-30T16:18:18.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.434+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11__multipart_foo.2~rsfKWR_VmhHpQAGvhcCSpjk4c7U0f4N.4 2024-05-30T16:18:18.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.434+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1084 2024-05-30T16:18:18.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.434+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:18.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.434+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.19tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.3911489254370521540 2024-05-30T16:18:18.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.434+0000 7fbafcff9640 20 handle_completion(): completion ok for obj=foo[zJHnZ7Thp8aoNgNXEcRkx5cA47QgZXh] 2024-05-30T16:18:18.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.438+0000 7fbc1ba93980 20 get_obj_state: octx=0x559689698fe0 obj=test-client.0-r-test-multipart-index-versioning.1:foo state=0x5596896c7b38 s->prefetch_data=0 2024-05-30T16:18:18.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.442+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11.10 2024-05-30T16:18:18.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.446+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11.10 2024-05-30T16:18:18.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.446+0000 7fbc1ba93980 20 list_objects_ordered: starting attempt 1 2024-05-30T16:18:18.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.446+0000 7fbc1ba93980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-05-30T16:18:18.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.446+0000 7fbc1ba93980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-05-30T16:18:18.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.446+0000 7fbc1ba93980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-05-30T16:18:18.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.446+0000 7fbc1ba93980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-05-30T16:18:18.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.446+0000 7fbc1ba93980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-05-30T16:18:18.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.446+0000 7fbc1ba93980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-index-versioning.1[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11])"; uploads.size()=0; is_truncated=0 2024-05-30T16:18:18.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.454+0000 7fbc1ba93980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : miss 2024-05-30T16:18:18.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.454+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.454+0000 7fbc1ba93980 20 rados_obj.operate() r=0 bl.length=237 2024-05-30T16:18:18.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.454+0000 7fbc1ba93980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 info.flags=0x11 2024-05-30T16:18:18.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.454+0000 7fbc1ba93980 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 to cache LRU end 2024-05-30T16:18:18.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.454+0000 7fbc1ba93980 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 from cache 2024-05-30T16:18:18.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.454+0000 7fbc1ba93980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-index-versioning.1, ofs0, ns] 2024-05-30T16:18:18.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.458+0000 7fbafcff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431571 cookie 94105038832944 notifier 5889 bl.length()=193 2024-05-30T16:18:18.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.462+0000 7fbc1ba93980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11 from cache 2024-05-30T16:18:18.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.462+0000 7fbc1ba93980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11, ofs0, ns] 2024-05-30T16:18:18.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.462+0000 7fbaeffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431585 cookie 94105038836800 notifier 5889 bl.length()=251 2024-05-30T16:18:18.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.474+0000 7fbc1ba93980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-index-versioning.1/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11 : miss 2024-05-30T16:18:18.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.474+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.474+0000 7fbc1ba93980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:18:18.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.474+0000 7fbc1ba93980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-index-versioning.1/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11 info.flags=0x0 2024-05-30T16:18:18.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.474+0000 7fbc1ba93980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-index-versioning.1/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11 to cache LRU end 2024-05-30T16:18:18.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.478+0000 7fbc1ba93980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.10 : miss 2024-05-30T16:18:18.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.478+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.478+0000 7fbc1ba93980 20 rados_obj.operate() r=0 bl.length=441 2024-05-30T16:18:18.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.478+0000 7fbc1ba93980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.10 info.flags=0x17 2024-05-30T16:18:18.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.478+0000 7fbc1ba93980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.10 to cache LRU end 2024-05-30T16:18:18.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.478+0000 7fbc1ba93980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:18:18.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.478+0000 7fbc1ba93980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-05-30T16:18:18.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.478+0000 7fbc1ba93980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.10 2024-05-30T16:18:18.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.490+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo 2024-05-30T16:18:18.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.490+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896ce960 obj=test-client.0-r-test-multipart-simple.1:foo state=0x5596896da3b8 s->prefetch_data=0 2024-05-30T16:18:18.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.490+0000 7fbc1ba93980 10 manifest: total_size = 1667 2024-05-30T16:18:18.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.490+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12523056908549656105 2024-05-30T16:18:18.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.490+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896ce960 obj=test-client.0-r-test-multipart-simple.1:foo state=0x5596896da3b8 s->prefetch_data=0 2024-05-30T16:18:18.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.490+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896da3b8 2024-05-30T16:18:18.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.490+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.10.10 2024-05-30T16:18:18.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.498+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-05-30T16:18:18.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.498+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=516 2024-05-30T16:18:18.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.498+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=516 stripe_ofs=516 part_ofs=516 rule->part_size=516 2024-05-30T16:18:18.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.498+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-05-30T16:18:18.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.498+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6291972 part_ofs=516 rule->part_size=516 2024-05-30T16:18:18.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.498+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=1032 stripe_ofs=1032 part_ofs=1032 rule->part_size=516 2024-05-30T16:18:18.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.498+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-05-30T16:18:18.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.498+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6292488 part_ofs=1032 rule->part_size=516 2024-05-30T16:18:18.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.498+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=1548 stripe_ofs=1548 part_ofs=1548 rule->part_size=119 2024-05-30T16:18:18.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.498+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=119 rules.size()=2 2024-05-30T16:18:18.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.498+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6293004 part_ofs=1548 rule->part_size=119 2024-05-30T16:18:18.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.498+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=1667 stripe_ofs=1667 part_ofs=1667 rule->part_size=119 2024-05-30T16:18:18.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.498+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12523056908549656105 2024-05-30T16:18:18.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.498+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:18.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.498+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.10__multipart_foo.2~SQVlhgfeYBCxqaPLnUGzte0SNojFJDp.1 2024-05-30T16:18:18.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.498+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 347 2024-05-30T16:18:18.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.498+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.10__multipart_foo.2~SQVlhgfeYBCxqaPLnUGzte0SNojFJDp.2 2024-05-30T16:18:18.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.498+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 593 2024-05-30T16:18:18.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.498+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.10__multipart_foo.2~SQVlhgfeYBCxqaPLnUGzte0SNojFJDp.3 2024-05-30T16:18:18.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.498+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 839 2024-05-30T16:18:18.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.498+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.10__multipart_foo.2~SQVlhgfeYBCxqaPLnUGzte0SNojFJDp.4 2024-05-30T16:18:18.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.498+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1085 2024-05-30T16:18:18.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.498+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:18.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.498+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.6tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12523056908549656105 2024-05-30T16:18:18.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.498+0000 7fbafcff9640 20 handle_completion(): completion ok for obj=foo 2024-05-30T16:18:18.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.502+0000 7fbc1ba93980 20 list_objects_ordered: starting attempt 1 2024-05-30T16:18:18.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.502+0000 7fbc1ba93980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-05-30T16:18:18.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.502+0000 7fbc1ba93980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-05-30T16:18:18.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.502+0000 7fbc1ba93980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-05-30T16:18:18.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.502+0000 7fbc1ba93980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-05-30T16:18:18.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.502+0000 7fbc1ba93980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-05-30T16:18:18.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.502+0000 7fbc1ba93980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-simple.1[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.10])"; uploads.size()=0; is_truncated=0 2024-05-30T16:18:18.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.506+0000 7fbc1ba93980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : miss 2024-05-30T16:18:18.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.506+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.509 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.510+0000 7fbc1ba93980 20 rados_obj.operate() r=0 bl.length=227 2024-05-30T16:18:18.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.510+0000 7fbc1ba93980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 info.flags=0x11 2024-05-30T16:18:18.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.510+0000 7fbc1ba93980 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 to cache LRU end 2024-05-30T16:18:18.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.510+0000 7fbc1ba93980 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 from cache 2024-05-30T16:18:18.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.510+0000 7fbc1ba93980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-simple.1, ofs0, ns] 2024-05-30T16:18:18.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.514+0000 7fbaeffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431586 cookie 94105038836800 notifier 5889 bl.length()=183 2024-05-30T16:18:18.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.518+0000 7fbc1ba93980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.10 from cache 2024-05-30T16:18:18.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.518+0000 7fbc1ba93980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-simple.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.10, ofs0, ns] 2024-05-30T16:18:18.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.518+0000 7fbaeffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431572 cookie 94105038832944 notifier 5889 bl.length()=241 2024-05-30T16:18:18.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.526+0000 7fbc1ba93980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-simple.1/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.10 : miss 2024-05-30T16:18:18.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.526+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.530+0000 7fbc1ba93980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:18:18.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.530+0000 7fbc1ba93980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-simple.1/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.10 info.flags=0x0 2024-05-30T16:18:18.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.530+0000 7fbc1ba93980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-simple.1/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.10 to cache LRU end 2024-05-30T16:18:18.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.534+0000 7fbc1ba93980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16 : miss 2024-05-30T16:18:18.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.534+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.534+0000 7fbc1ba93980 20 rados_obj.operate() r=0 bl.length=447 2024-05-30T16:18:18.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.534+0000 7fbc1ba93980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16 info.flags=0x17 2024-05-30T16:18:18.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.534+0000 7fbc1ba93980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16 to cache LRU end 2024-05-30T16:18:18.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.534+0000 7fbc1ba93980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:18:18.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.534+0000 7fbc1ba93980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-05-30T16:18:18.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.534+0000 7fbc1ba93980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16 2024-05-30T16:18:18.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.538+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo-STANDARD-LUKEWARM 2024-05-30T16:18:18.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.538+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo-LUKEWARM-STANDARD 2024-05-30T16:18:18.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.542+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo-FROZEN 2024-05-30T16:18:18.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.542+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo-FROZEN-LUKEWARM 2024-05-30T16:18:18.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.542+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo-LUKEWARM-FROZEN 2024-05-30T16:18:18.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.542+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo-STANDARD 2024-05-30T16:18:18.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.546+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo-FROZEN-STANDARD 2024-05-30T16:18:18.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.546+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo-LUKEWARM 2024-05-30T16:18:18.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.546+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo-STANDARD-STANDARD 2024-05-30T16:18:18.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.546+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo-LUKEWARM-LUKEWARM 2024-05-30T16:18:18.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.546+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo-STANDARD-FROZEN 2024-05-30T16:18:18.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.546+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo-FROZEN-FROZEN 2024-05-30T16:18:18.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.546+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896cdcb0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-LUKEWARM state=0x5596896c5178 s->prefetch_data=0 2024-05-30T16:18:18.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.550+0000 7fbc1ba93980 10 manifest: total_size = 5242880 2024-05-30T16:18:18.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.550+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to _OcE_IYB-B5quL9-wrpppgcdhg2HuA_T 2024-05-30T16:18:18.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.550+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896cdcb0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-LUKEWARM state=0x5596896c5178 s->prefetch_data=0 2024-05-30T16:18:18.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.550+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896c5178 2024-05-30T16:18:18.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.550+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16.0 2024-05-30T16:18:18.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.554+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-30T16:18:18.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.554+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-05-30T16:18:18.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.554+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: _OcE_IYB-B5quL9-wrpppgcdhg2HuA_T 2024-05-30T16:18:18.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.554+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:18.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.554+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.Znfc6ITYhEqooNv7e4G7MaY3U5V403h_0 2024-05-30T16:18:18.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.554+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 310 2024-05-30T16:18:18.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.554+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:18.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.554+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.25tag is: _OcE_IYB-B5quL9-wrpppgcdhg2HuA_T 2024-05-30T16:18:18.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.558+0000 7fbaeffff640 20 handle_completion(): completion ok for obj=foo-STANDARD-LUKEWARM 2024-05-30T16:18:18.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.558+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596897405f0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-STANDARD state=0x5596896c5178 s->prefetch_data=0 2024-05-30T16:18:18.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.562+0000 7fbc1ba93980 10 manifest: total_size = 5242880 2024-05-30T16:18:18.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.562+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to _kC7FzPcvJAXSqn36qCEfy1ooHRUncgZ 2024-05-30T16:18:18.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.562+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596897405f0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-STANDARD state=0x5596896c5178 s->prefetch_data=0 2024-05-30T16:18:18.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.562+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896c5178 2024-05-30T16:18:18.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.562+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16.2 2024-05-30T16:18:18.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.566+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-30T16:18:18.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.566+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-05-30T16:18:18.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.566+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: _kC7FzPcvJAXSqn36qCEfy1ooHRUncgZ 2024-05-30T16:18:18.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.566+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:18.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.566+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.sAiQcD3_iUscIyjpheJeMPPpllHhzO9_1 2024-05-30T16:18:18.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.566+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 301 2024-05-30T16:18:18.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.566+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:18.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.566+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.17tag is: _kC7FzPcvJAXSqn36qCEfy1ooHRUncgZ 2024-05-30T16:18:18.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.570+0000 7fbafcff9640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-STANDARD 2024-05-30T16:18:18.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.570+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596894befd0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN state=0x5596896c5178 s->prefetch_data=0 2024-05-30T16:18:18.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.574+0000 7fbc1ba93980 10 manifest: total_size = 5242880 2024-05-30T16:18:18.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.574+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12873641608161991758 2024-05-30T16:18:18.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.574+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596894befd0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN state=0x5596896c5178 s->prefetch_data=0 2024-05-30T16:18:18.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.574+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896c5178 2024-05-30T16:18:18.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.574+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16.3 2024-05-30T16:18:18.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.582+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-30T16:18:18.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.582+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-05-30T16:18:18.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.582+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12873641608161991758 2024-05-30T16:18:18.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.582+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:18.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.582+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.hTEeNl7avS8xfv478tR1-RVidOqTxuj_0 2024-05-30T16:18:18.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.582+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 338 2024-05-30T16:18:18.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.582+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:18.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.582+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.11tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12873641608161991758 2024-05-30T16:18:18.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.582+0000 7fbaeffff640 20 handle_completion(): completion ok for obj=foo-FROZEN 2024-05-30T16:18:18.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.586+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596897405f0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-LUKEWARM state=0x5596896c5178 s->prefetch_data=0 2024-05-30T16:18:18.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.586+0000 7fbc1ba93980 10 manifest: total_size = 5242880 2024-05-30T16:18:18.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.586+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to _UV2RhYiS5-EwRIGn2WlcbIOWCjkW94d 2024-05-30T16:18:18.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.586+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596897405f0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-LUKEWARM state=0x5596896c5178 s->prefetch_data=0 2024-05-30T16:18:18.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.586+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896c5178 2024-05-30T16:18:18.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.586+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16.5 2024-05-30T16:18:18.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.594+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-30T16:18:18.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.594+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-05-30T16:18:18.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.594+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: _UV2RhYiS5-EwRIGn2WlcbIOWCjkW94d 2024-05-30T16:18:18.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.594+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:18.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.594+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.RbGtPjlEyaaxk8wiUnUqqJUTuuCpI5__0 2024-05-30T16:18:18.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.594+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 310 2024-05-30T16:18:18.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.594+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:18.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.594+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.6tag is: _UV2RhYiS5-EwRIGn2WlcbIOWCjkW94d 2024-05-30T16:18:18.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.598+0000 7fbafcff9640 20 handle_completion(): completion ok for obj=foo-FROZEN-LUKEWARM 2024-05-30T16:18:18.599 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.598+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896dfc80 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-FROZEN state=0x5596896c5178 s->prefetch_data=0 2024-05-30T16:18:18.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.598+0000 7fbc1ba93980 10 manifest: total_size = 5242880 2024-05-30T16:18:18.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.598+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to _qv0iLHZEhEywlkjgazaBnZp19fc0ofw 2024-05-30T16:18:18.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.598+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896dfc80 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-FROZEN state=0x5596896c5178 s->prefetch_data=0 2024-05-30T16:18:18.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.598+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896c5178 2024-05-30T16:18:18.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.598+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16.5 2024-05-30T16:18:18.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.606+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-30T16:18:18.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.606+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-05-30T16:18:18.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.606+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: _qv0iLHZEhEywlkjgazaBnZp19fc0ofw 2024-05-30T16:18:18.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.606+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:18.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.606+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.M0ITihsC-YD2zOMpJDeMeYhuYGgabJb_0 2024-05-30T16:18:18.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.606+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 308 2024-05-30T16:18:18.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.606+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:18.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.606+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.1tag is: _qv0iLHZEhEywlkjgazaBnZp19fc0ofw 2024-05-30T16:18:18.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.610+0000 7fbaeffff640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-FROZEN 2024-05-30T16:18:18.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.610+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc1009c040 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD state=0x5596896c5178 s->prefetch_data=0 2024-05-30T16:18:18.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.610+0000 7fbc1ba93980 10 manifest: total_size = 5242880 2024-05-30T16:18:18.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.610+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17418197569884719744 2024-05-30T16:18:18.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.610+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc1009c040 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD state=0x5596896c5178 s->prefetch_data=0 2024-05-30T16:18:18.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.610+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896c5178 2024-05-30T16:18:18.613 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.610+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16.5 2024-05-30T16:18:18.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.618+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-30T16:18:18.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.618+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-05-30T16:18:18.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.618+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17418197569884719744 2024-05-30T16:18:18.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.618+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:18.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.618+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.FsErSdszcSbW8ptNqjfN4gica2HNJIg_1 2024-05-30T16:18:18.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.618+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 331 2024-05-30T16:18:18.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.618+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:18.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.618+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.22tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17418197569884719744 2024-05-30T16:18:18.622 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.622+0000 7fbafcff9640 20 handle_completion(): completion ok for obj=foo-STANDARD 2024-05-30T16:18:18.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.622+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc10024050 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-STANDARD state=0x5596896c5178 s->prefetch_data=0 2024-05-30T16:18:18.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.622+0000 7fbc1ba93980 10 manifest: total_size = 5242880 2024-05-30T16:18:18.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.622+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to _8RV0OTyicl_HXdetg1Vl8oWv0mwM22P 2024-05-30T16:18:18.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.622+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc10024050 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-STANDARD state=0x5596896c5178 s->prefetch_data=0 2024-05-30T16:18:18.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.622+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896c5178 2024-05-30T16:18:18.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.622+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16.7 2024-05-30T16:18:18.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.630+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-30T16:18:18.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.630+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-05-30T16:18:18.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.630+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: _8RV0OTyicl_HXdetg1Vl8oWv0mwM22P 2024-05-30T16:18:18.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.630+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:18.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.630+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.WDNKaVYSkoRupdUyl31sobeBcYzQ-1F_1 2024-05-30T16:18:18.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.630+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 301 2024-05-30T16:18:18.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.630+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:18.632 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.630+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.14tag is: _8RV0OTyicl_HXdetg1Vl8oWv0mwM22P 2024-05-30T16:18:18.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.634+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896e02c0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM state=0x5596896c5178 s->prefetch_data=0 2024-05-30T16:18:18.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.634+0000 7fbaeffff640 20 handle_completion(): completion ok for obj=foo-FROZEN-STANDARD 2024-05-30T16:18:18.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.634+0000 7fbc1ba93980 10 manifest: total_size = 5242880 2024-05-30T16:18:18.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.634+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.310794086773851024 2024-05-30T16:18:18.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.634+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896e02c0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM state=0x5596896c5178 s->prefetch_data=0 2024-05-30T16:18:18.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.634+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896c5178 2024-05-30T16:18:18.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.634+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16.7 2024-05-30T16:18:18.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.642+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-30T16:18:18.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.642+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-05-30T16:18:18.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.642+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.310794086773851024 2024-05-30T16:18:18.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.642+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:18.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.642+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.sVnrVYW9-ta8wIRvjaw10ZLJEkUm-k__0 2024-05-30T16:18:18.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.642+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 338 2024-05-30T16:18:18.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.642+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:18.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.642+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.28tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.310794086773851024 2024-05-30T16:18:18.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.646+0000 7fbafcff9640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2024-05-30T16:18:18.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.646+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc1009c040 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-STANDARD state=0x5596896c5178 s->prefetch_data=0 2024-05-30T16:18:18.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.646+0000 7fbc1ba93980 10 manifest: total_size = 5242880 2024-05-30T16:18:18.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.646+0000 7fbc1ba93980 10 generate_fake_tag new tag=601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.FsErSdszcSbW8ptNqjfN4gica2HNJIg_1_98c122ff0375e35c43d882e682fe36ec 2024-05-30T16:18:18.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.646+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.FsErSdszcSbW8ptNqjfN4gica2HNJIg_1_98c122ff0375e35c43d882e682fe36ec 2024-05-30T16:18:18.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.646+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc1009c040 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-STANDARD state=0x5596896c5178 s->prefetch_data=0 2024-05-30T16:18:18.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.646+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896c5178 2024-05-30T16:18:18.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.646+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16.7 2024-05-30T16:18:18.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.654+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-30T16:18:18.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.654+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-05-30T16:18:18.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.654+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.FsErSdszcSbW8ptNqjfN4gica2HNJIg_1_98c122ff0375e35c43d882e682fe36ec 2024-05-30T16:18:18.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.654+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:18.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.654+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.FsErSdszcSbW8ptNqjfN4gica2HNJIg_1 2024-05-30T16:18:18.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.654+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 389 2024-05-30T16:18:18.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.654+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:18.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.654+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.20tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.FsErSdszcSbW8ptNqjfN4gica2HNJIg_1_98c122ff0375e35c43d882e682fe36ec 2024-05-30T16:18:18.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.658+0000 7fbaeffff640 20 handle_completion(): completion ok for obj=foo-STANDARD-STANDARD 2024-05-30T16:18:18.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.658+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc1009c040 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-LUKEWARM state=0x5596896c5178 s->prefetch_data=0 2024-05-30T16:18:18.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.658+0000 7fbc1ba93980 10 manifest: total_size = 5242880 2024-05-30T16:18:18.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.658+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-30T16:18:18.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.658+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-05-30T16:18:18.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.658+0000 7fbc1ba93980 10 generate_fake_tag new tag=601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.sVnrVYW9-ta8wIRvjaw10ZLJEkUm-k__1_a735fe677fd16778bb509a6b8fb3a9a7 2024-05-30T16:18:18.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.658+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.sVnrVYW9-ta8wIRvjaw10ZLJEkUm-k__1_a735fe677fd16778bb509a6b8fb3a9a7 2024-05-30T16:18:18.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.658+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc1009c040 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-LUKEWARM state=0x5596896c5178 s->prefetch_data=0 2024-05-30T16:18:18.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.658+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896c5178 2024-05-30T16:18:18.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.658+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16.8 2024-05-30T16:18:18.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.666+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-30T16:18:18.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.666+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-05-30T16:18:18.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.666+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.sVnrVYW9-ta8wIRvjaw10ZLJEkUm-k__1_a735fe677fd16778bb509a6b8fb3a9a7 2024-05-30T16:18:18.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.666+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:18.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.666+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.sVnrVYW9-ta8wIRvjaw10ZLJEkUm-k__0 2024-05-30T16:18:18.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.666+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 398 2024-05-30T16:18:18.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.666+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:18.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.666+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.10tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.sVnrVYW9-ta8wIRvjaw10ZLJEkUm-k__1_a735fe677fd16778bb509a6b8fb3a9a7 2024-05-30T16:18:18.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.670+0000 7fbafcff9640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-LUKEWARM 2024-05-30T16:18:18.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.670+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc10083f80 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-FROZEN state=0x5596896c5178 s->prefetch_data=0 2024-05-30T16:18:18.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.670+0000 7fbc1ba93980 10 manifest: total_size = 5242880 2024-05-30T16:18:18.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.670+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to _VLaDFQwaCiFDwqOPngb13FJ0EgbZw11 2024-05-30T16:18:18.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.670+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc10083f80 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-FROZEN state=0x5596896c5178 s->prefetch_data=0 2024-05-30T16:18:18.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.670+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896c5178 2024-05-30T16:18:18.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.670+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16.8 2024-05-30T16:18:18.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.678+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-30T16:18:18.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.678+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-05-30T16:18:18.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.678+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: _VLaDFQwaCiFDwqOPngb13FJ0EgbZw11 2024-05-30T16:18:18.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.678+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:18.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.678+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.SVdi7TJiQFTQeBdO0Z3XFH3Rz7vn1A7_0 2024-05-30T16:18:18.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.678+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 308 2024-05-30T16:18:18.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.678+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:18.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.678+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.29tag is: _VLaDFQwaCiFDwqOPngb13FJ0EgbZw11 2024-05-30T16:18:18.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.682+0000 7fbaeffff640 20 handle_completion(): completion ok for obj=foo-STANDARD-FROZEN 2024-05-30T16:18:18.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.682+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc1009c040 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-FROZEN state=0x5596896c5178 s->prefetch_data=0 2024-05-30T16:18:18.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.682+0000 7fbc1ba93980 10 manifest: total_size = 5242880 2024-05-30T16:18:18.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.682+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-30T16:18:18.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.682+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-05-30T16:18:18.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.682+0000 7fbc1ba93980 10 generate_fake_tag new tag=601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.hTEeNl7avS8xfv478tR1-RVidOqTxuj_1_ddf4806003dde406047b90bf47d75905 2024-05-30T16:18:18.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.682+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.hTEeNl7avS8xfv478tR1-RVidOqTxuj_1_ddf4806003dde406047b90bf47d75905 2024-05-30T16:18:18.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.682+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc1009c040 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-FROZEN state=0x5596896c5178 s->prefetch_data=0 2024-05-30T16:18:18.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.682+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896c5178 2024-05-30T16:18:18.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.682+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16.9 2024-05-30T16:18:18.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.690+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-30T16:18:18.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.690+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-05-30T16:18:18.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.690+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.hTEeNl7avS8xfv478tR1-RVidOqTxuj_1_ddf4806003dde406047b90bf47d75905 2024-05-30T16:18:18.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.690+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:18.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.690+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.hTEeNl7avS8xfv478tR1-RVidOqTxuj_0 2024-05-30T16:18:18.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.690+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 396 2024-05-30T16:18:18.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.690+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:18.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.690+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.10tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.hTEeNl7avS8xfv478tR1-RVidOqTxuj_1_ddf4806003dde406047b90bf47d75905 2024-05-30T16:18:18.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.694+0000 7fbafcff9640 20 handle_completion(): completion ok for obj=foo-FROZEN-FROZEN 2024-05-30T16:18:18.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.694+0000 7fbc1ba93980 20 list_objects_ordered: starting attempt 1 2024-05-30T16:18:18.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.694+0000 7fbc1ba93980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-05-30T16:18:18.697 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.698+0000 7fbc1ba93980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-05-30T16:18:18.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.698+0000 7fbc1ba93980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-05-30T16:18:18.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.698+0000 7fbc1ba93980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-05-30T16:18:18.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.698+0000 7fbc1ba93980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-05-30T16:18:18.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.698+0000 7fbc1ba93980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-copy.1[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16])"; uploads.size()=0; is_truncated=0 2024-05-30T16:18:18.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.702+0000 7fbc1ba93980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : miss 2024-05-30T16:18:18.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.702+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.702+0000 7fbc1ba93980 20 rados_obj.operate() r=0 bl.length=233 2024-05-30T16:18:18.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.702+0000 7fbc1ba93980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 info.flags=0x11 2024-05-30T16:18:18.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.702+0000 7fbc1ba93980 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 to cache LRU end 2024-05-30T16:18:18.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.706+0000 7fbc1ba93980 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 from cache 2024-05-30T16:18:18.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.706+0000 7fbc1ba93980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class-copy.1, ofs0, ns] 2024-05-30T16:18:18.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.706+0000 7fbafcff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431573 cookie 94105038832944 notifier 5889 bl.length()=189 2024-05-30T16:18:18.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.714+0000 7fbc1ba93980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16 from cache 2024-05-30T16:18:18.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.714+0000 7fbc1ba93980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16, ofs0, ns] 2024-05-30T16:18:18.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.714+0000 7fbafcff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431587 cookie 94105038836800 notifier 5889 bl.length()=247 2024-05-30T16:18:18.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.722+0000 7fbc1ba93980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-copy.1/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16 : miss 2024-05-30T16:18:18.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.722+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.722+0000 7fbc1ba93980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:18:18.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.722+0000 7fbc1ba93980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-copy.1/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16 info.flags=0x0 2024-05-30T16:18:18.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.722+0000 7fbc1ba93980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-copy.1/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16 to cache LRU end 2024-05-30T16:18:18.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.730+0000 7fbc1ba93980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17 : miss 2024-05-30T16:18:18.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.730+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.730+0000 7fbc1ba93980 20 rados_obj.operate() r=0 bl.length=457 2024-05-30T16:18:18.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.730+0000 7fbc1ba93980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17 info.flags=0x17 2024-05-30T16:18:18.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.730+0000 7fbc1ba93980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17 to cache LRU end 2024-05-30T16:18:18.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.730+0000 7fbc1ba93980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:18:18.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.730+0000 7fbc1ba93980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-05-30T16:18:18.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.730+0000 7fbc1ba93980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17 2024-05-30T16:18:18.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.730+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo-STANDARD-LUKEWARM 2024-05-30T16:18:18.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.734+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo-LUKEWARM-STANDARD 2024-05-30T16:18:18.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.734+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo-FROZEN 2024-05-30T16:18:18.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.738+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo-FROZEN-LUKEWARM 2024-05-30T16:18:18.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.738+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo-LUKEWARM-FROZEN 2024-05-30T16:18:18.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.738+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo-STANDARD 2024-05-30T16:18:18.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.738+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo-FROZEN-STANDARD 2024-05-30T16:18:18.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.738+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo-LUKEWARM 2024-05-30T16:18:18.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.738+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo-STANDARD-STANDARD 2024-05-30T16:18:18.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.738+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo-LUKEWARM-LUKEWARM 2024-05-30T16:18:18.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.738+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo-STANDARD-FROZEN 2024-05-30T16:18:18.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.742+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo-FROZEN-FROZEN 2024-05-30T16:18:18.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.742+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896cdbd0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-LUKEWARM state=0x5596896c5018 s->prefetch_data=0 2024-05-30T16:18:18.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.742+0000 7fbc1ba93980 10 manifest: total_size = 1087 2024-05-30T16:18:18.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.742+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to _Xa0MqTgVK3PtPq9sEAKgmwldHs5USY6 2024-05-30T16:18:18.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.742+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896cdbd0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-LUKEWARM state=0x5596896c5018 s->prefetch_data=0 2024-05-30T16:18:18.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.742+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896c5018 2024-05-30T16:18:18.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.742+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17.0 2024-05-30T16:18:18.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.750+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-30T16:18:18.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.750+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=1087 stripe_ofs=1087 part_ofs=0 rule->part_size=0 2024-05-30T16:18:18.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.750+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: _Xa0MqTgVK3PtPq9sEAKgmwldHs5USY6 2024-05-30T16:18:18.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.750+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:18.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.750+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__shadow_.3x-gEVKbJAF2tDW4EwV63UEa1aUisGd_0 2024-05-30T16:18:18.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.750+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 310 2024-05-30T16:18:18.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.750+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:18.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.750+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.17tag is: _Xa0MqTgVK3PtPq9sEAKgmwldHs5USY6 2024-05-30T16:18:18.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.754+0000 7fbaeffff640 20 handle_completion(): completion ok for obj=foo-STANDARD-LUKEWARM 2024-05-30T16:18:18.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.754+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896a16c0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-STANDARD state=0x5596896c5018 s->prefetch_data=0 2024-05-30T16:18:18.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.754+0000 7fbc1ba93980 10 manifest: total_size = 11534336 2024-05-30T16:18:18.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.754+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to _vnHmXe10Qv_Vs0QKfs4_8YrGQcVcSdx 2024-05-30T16:18:18.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.754+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896a16c0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-STANDARD state=0x5596896c5018 s->prefetch_data=0 2024-05-30T16:18:18.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.754+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896c5018 2024-05-30T16:18:18.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.754+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17.2 2024-05-30T16:18:18.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.762+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-30T16:18:18.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.762+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2024-05-30T16:18:18.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.762+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-30T16:18:18.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.762+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-05-30T16:18:18.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.762+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: _vnHmXe10Qv_Vs0QKfs4_8YrGQcVcSdx 2024-05-30T16:18:18.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.762+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:18.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.762+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__shadow_.2IR4yfvjh4iUsvYhjJUHaDAsLeVi04__1 2024-05-30T16:18:18.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.762+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 301 2024-05-30T16:18:18.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.762+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__shadow_.2IR4yfvjh4iUsvYhjJUHaDAsLeVi04__2 2024-05-30T16:18:18.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.762+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 531 2024-05-30T16:18:18.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.762+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:18.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.762+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.17tag is: _vnHmXe10Qv_Vs0QKfs4_8YrGQcVcSdx 2024-05-30T16:18:18.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.766+0000 7fbafcff9640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-STANDARD 2024-05-30T16:18:18.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.766+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc140561d0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN state=0x5596896c5018 s->prefetch_data=0 2024-05-30T16:18:18.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.766+0000 7fbc1ba93980 10 manifest: total_size = 11534336 2024-05-30T16:18:18.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.766+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17310732107394054150 2024-05-30T16:18:18.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.766+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc140561d0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN state=0x5596896c5018 s->prefetch_data=0 2024-05-30T16:18:18.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.766+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896c5018 2024-05-30T16:18:18.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.766+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17.3 2024-05-30T16:18:18.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.774+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-30T16:18:18.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.774+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-05-30T16:18:18.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.774+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-05-30T16:18:18.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.774+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-30T16:18:18.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.774+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2024-05-30T16:18:18.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.774+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-05-30T16:18:18.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.774+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-05-30T16:18:18.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.774+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2024-05-30T16:18:18.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.774+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-05-30T16:18:18.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.774+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17310732107394054150 2024-05-30T16:18:18.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.774+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:18.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.774+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-FROZEN.2~BFRcb5Q8ovi-IgTqlOD8ZC7hmBrYMTv.1 2024-05-30T16:18:18.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.774+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 368 2024-05-30T16:18:18.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.774+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-FROZEN.2~BFRcb5Q8ovi-IgTqlOD8ZC7hmBrYMTv.2 2024-05-30T16:18:18.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.774+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 635 2024-05-30T16:18:18.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.774+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-FROZEN.2~BFRcb5Q8ovi-IgTqlOD8ZC7hmBrYMTv.3 2024-05-30T16:18:18.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.774+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 902 2024-05-30T16:18:18.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.774+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:18.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.774+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.17tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17310732107394054150 2024-05-30T16:18:18.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.774+0000 7fbaeffff640 20 handle_completion(): completion ok for obj=foo-FROZEN 2024-05-30T16:18:18.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.774+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896e0310 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-LUKEWARM state=0x5596896c5018 s->prefetch_data=0 2024-05-30T16:18:18.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.778+0000 7fbc1ba93980 10 manifest: total_size = 11534336 2024-05-30T16:18:18.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.778+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to _qIFw_2_h-URjgdqm0UZ2AgdHK4WxqD6 2024-05-30T16:18:18.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.778+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896e0310 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-LUKEWARM state=0x5596896c5018 s->prefetch_data=0 2024-05-30T16:18:18.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.778+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896c5018 2024-05-30T16:18:18.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.778+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17.5 2024-05-30T16:18:18.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.782+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-30T16:18:18.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.782+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=6291456 stripe_ofs=6291456 part_ofs=0 rule->part_size=0 2024-05-30T16:18:18.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.782+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-30T16:18:18.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.782+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-05-30T16:18:18.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.782+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: _qIFw_2_h-URjgdqm0UZ2AgdHK4WxqD6 2024-05-30T16:18:18.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.782+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:18.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.782+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__shadow_.T0ycfx6PC5jYOSHRzYF6kP3mqQoeAAT_0 2024-05-30T16:18:18.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.782+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 310 2024-05-30T16:18:18.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.782+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__shadow_.T0ycfx6PC5jYOSHRzYF6kP3mqQoeAAT_1 2024-05-30T16:18:18.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.782+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 549 2024-05-30T16:18:18.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.782+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:18.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.782+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.30tag is: _qIFw_2_h-URjgdqm0UZ2AgdHK4WxqD6 2024-05-30T16:18:18.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.786+0000 7fbafcff9640 20 handle_completion(): completion ok for obj=foo-FROZEN-LUKEWARM 2024-05-30T16:18:18.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.786+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc140561d0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-FROZEN state=0x5596896c5018 s->prefetch_data=0 2024-05-30T16:18:18.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.786+0000 7fbc1ba93980 10 manifest: total_size = 11534336 2024-05-30T16:18:18.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.786+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to _c32z-THpIOEp_EqO3zGgwcYx5kdlsBi 2024-05-30T16:18:18.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.786+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc140561d0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-FROZEN state=0x5596896c5018 s->prefetch_data=0 2024-05-30T16:18:18.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.786+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896c5018 2024-05-30T16:18:18.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.786+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17.5 2024-05-30T16:18:18.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.794+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-30T16:18:18.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.794+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=6291456 stripe_ofs=6291456 part_ofs=0 rule->part_size=0 2024-05-30T16:18:18.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.794+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-30T16:18:18.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.794+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-05-30T16:18:18.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.794+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: _c32z-THpIOEp_EqO3zGgwcYx5kdlsBi 2024-05-30T16:18:18.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.794+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:18.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.794+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__shadow_.OJOkxH1CyfmmDDMChYlCZActQFQ8GS2_0 2024-05-30T16:18:18.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.794+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 308 2024-05-30T16:18:18.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.794+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__shadow_.OJOkxH1CyfmmDDMChYlCZActQFQ8GS2_1 2024-05-30T16:18:18.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.794+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 545 2024-05-30T16:18:18.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.794+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:18.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.794+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.30tag is: _c32z-THpIOEp_EqO3zGgwcYx5kdlsBi 2024-05-30T16:18:18.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.798+0000 7fbaeffff640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-FROZEN 2024-05-30T16:18:18.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.798+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896c47c0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD state=0x5596896c5018 s->prefetch_data=0 2024-05-30T16:18:18.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.798+0000 7fbc1ba93980 10 manifest: total_size = 1087 2024-05-30T16:18:18.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.798+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16322308426137548912 2024-05-30T16:18:18.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.798+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896c47c0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD state=0x5596896c5018 s->prefetch_data=0 2024-05-30T16:18:18.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.798+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896c5018 2024-05-30T16:18:18.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.798+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17.5 2024-05-30T16:18:18.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.806+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-05-30T16:18:18.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.806+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=516 2024-05-30T16:18:18.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.806+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=516 stripe_ofs=516 part_ofs=516 rule->part_size=516 2024-05-30T16:18:18.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.806+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-05-30T16:18:18.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.806+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6291972 part_ofs=516 rule->part_size=516 2024-05-30T16:18:18.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.806+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=1032 stripe_ofs=1032 part_ofs=1032 rule->part_size=55 2024-05-30T16:18:18.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.806+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=55 rules.size()=2 2024-05-30T16:18:18.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.806+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6292488 part_ofs=1032 rule->part_size=55 2024-05-30T16:18:18.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.806+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=1087 stripe_ofs=1087 part_ofs=1087 rule->part_size=55 2024-05-30T16:18:18.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.806+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16322308426137548912 2024-05-30T16:18:18.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.806+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:18.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.806+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-STANDARD.2~9zF-INvfUU3CV2mHkJEI4N-gDYI1hCM.1 2024-05-30T16:18:18.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.806+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 365 2024-05-30T16:18:18.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.806+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-STANDARD.2~9zF-INvfUU3CV2mHkJEI4N-gDYI1hCM.2 2024-05-30T16:18:18.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.806+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 629 2024-05-30T16:18:18.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.806+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-STANDARD.2~9zF-INvfUU3CV2mHkJEI4N-gDYI1hCM.3 2024-05-30T16:18:18.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.806+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 893 2024-05-30T16:18:18.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.806+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:18.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.806+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.26tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16322308426137548912 2024-05-30T16:18:18.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.810+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596897405f0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-STANDARD state=0x5596896c5018 s->prefetch_data=0 2024-05-30T16:18:18.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.810+0000 7fbafcff9640 20 handle_completion(): completion ok for obj=foo-STANDARD 2024-05-30T16:18:18.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.810+0000 7fbc1ba93980 10 manifest: total_size = 11534336 2024-05-30T16:18:18.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.810+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to _V8edkAThlk7MAOlvAqltCrZ2wKS-eS_ 2024-05-30T16:18:18.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.810+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596897405f0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-STANDARD state=0x5596896c5018 s->prefetch_data=0 2024-05-30T16:18:18.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.810+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896c5018 2024-05-30T16:18:18.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.810+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17.7 2024-05-30T16:18:18.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.818+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-30T16:18:18.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.818+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2024-05-30T16:18:18.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.818+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-30T16:18:18.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.818+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-05-30T16:18:18.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.818+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: _V8edkAThlk7MAOlvAqltCrZ2wKS-eS_ 2024-05-30T16:18:18.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.818+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:18.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.818+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__shadow_.w0_wBDs8bF63Tv757aR0tb8_2pXTQyw_1 2024-05-30T16:18:18.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.818+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 301 2024-05-30T16:18:18.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.818+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__shadow_.w0_wBDs8bF63Tv757aR0tb8_2pXTQyw_2 2024-05-30T16:18:18.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.818+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 531 2024-05-30T16:18:18.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.818+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:18.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.818+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.22tag is: _V8edkAThlk7MAOlvAqltCrZ2wKS-eS_ 2024-05-30T16:18:18.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.822+0000 7fbaeffff640 20 handle_completion(): completion ok for obj=foo-FROZEN-STANDARD 2024-05-30T16:18:18.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.822+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc0c0f0480 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM state=0x5596896c5018 s->prefetch_data=0 2024-05-30T16:18:18.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.822+0000 7fbc1ba93980 10 manifest: total_size = 11534336 2024-05-30T16:18:18.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.822+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.1621245897829638086 2024-05-30T16:18:18.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.822+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc0c0f0480 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM state=0x5596896c5018 s->prefetch_data=0 2024-05-30T16:18:18.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.822+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896c5018 2024-05-30T16:18:18.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.822+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17.7 2024-05-30T16:18:18.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.830+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-30T16:18:18.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.830+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-05-30T16:18:18.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.830+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-05-30T16:18:18.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.830+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-30T16:18:18.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.830+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2024-05-30T16:18:18.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.830+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-05-30T16:18:18.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.830+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-05-30T16:18:18.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.830+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2024-05-30T16:18:18.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.830+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-05-30T16:18:18.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.830+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.1621245897829638086 2024-05-30T16:18:18.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.830+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:18.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.830+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-LUKEWARM.2~wajqBmpC82qXXhM9doS3FFBuijhwm7k.1 2024-05-30T16:18:18.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.830+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 373 2024-05-30T16:18:18.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.830+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-LUKEWARM.2~wajqBmpC82qXXhM9doS3FFBuijhwm7k.2 2024-05-30T16:18:18.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.830+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 646 2024-05-30T16:18:18.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.830+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-LUKEWARM.2~wajqBmpC82qXXhM9doS3FFBuijhwm7k.3 2024-05-30T16:18:18.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.830+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 919 2024-05-30T16:18:18.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.830+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:18.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.830+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.1tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.1621245897829638086 2024-05-30T16:18:18.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.830+0000 7fbafcff9640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2024-05-30T16:18:18.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.834+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896cf1f0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-STANDARD state=0x5596896c5018 s->prefetch_data=0 2024-05-30T16:18:18.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.834+0000 7fbc1ba93980 10 manifest: total_size = 1087 2024-05-30T16:18:18.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.834+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-05-30T16:18:18.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.834+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=516 2024-05-30T16:18:18.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.834+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=516 stripe_ofs=516 part_ofs=516 rule->part_size=516 2024-05-30T16:18:18.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.834+0000 7fbc1ba93980 10 generate_fake_tag new tag=601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-STANDARD.2~9zF-INvfUU3CV2mHkJEI4N-gDYI1hCM.2_8bf2befbb85c41f573c5363aefdbcdaa 2024-05-30T16:18:18.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.834+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-STANDARD.2~9zF-INvfUU3CV2mHkJEI4N-gDYI1hCM.2_8bf2befbb85c41f573c5363aefdbcdaa 2024-05-30T16:18:18.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.834+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896cf1f0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-STANDARD state=0x5596896c5018 s->prefetch_data=0 2024-05-30T16:18:18.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.834+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896c5018 2024-05-30T16:18:18.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.834+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17.7 2024-05-30T16:18:18.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.842+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-05-30T16:18:18.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.842+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=516 2024-05-30T16:18:18.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.842+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=516 stripe_ofs=516 part_ofs=516 rule->part_size=516 2024-05-30T16:18:18.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.842+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-05-30T16:18:18.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.842+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6291972 part_ofs=516 rule->part_size=516 2024-05-30T16:18:18.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.842+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=1032 stripe_ofs=1032 part_ofs=1032 rule->part_size=55 2024-05-30T16:18:18.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.842+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=55 rules.size()=2 2024-05-30T16:18:18.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.842+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6292488 part_ofs=1032 rule->part_size=55 2024-05-30T16:18:18.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.842+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=1087 stripe_ofs=1087 part_ofs=1087 rule->part_size=55 2024-05-30T16:18:18.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.842+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-STANDARD.2~9zF-INvfUU3CV2mHkJEI4N-gDYI1hCM.2_8bf2befbb85c41f573c5363aefdbcdaa 2024-05-30T16:18:18.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.842+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:18.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.842+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-STANDARD.2~9zF-INvfUU3CV2mHkJEI4N-gDYI1hCM.1 2024-05-30T16:18:18.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.842+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 440 2024-05-30T16:18:18.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.842+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-STANDARD.2~9zF-INvfUU3CV2mHkJEI4N-gDYI1hCM.2 2024-05-30T16:18:18.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.842+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 704 2024-05-30T16:18:18.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.842+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-STANDARD.2~9zF-INvfUU3CV2mHkJEI4N-gDYI1hCM.3 2024-05-30T16:18:18.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.842+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 968 2024-05-30T16:18:18.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.842+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:18.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.842+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.25tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-STANDARD.2~9zF-INvfUU3CV2mHkJEI4N-gDYI1hCM.2_8bf2befbb85c41f573c5363aefdbcdaa 2024-05-30T16:18:18.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.842+0000 7fbaeffff640 20 handle_completion(): completion ok for obj=foo-STANDARD-STANDARD 2024-05-30T16:18:18.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.846+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896c77d0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-LUKEWARM state=0x5596896c5018 s->prefetch_data=0 2024-05-30T16:18:18.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.846+0000 7fbc1ba93980 10 manifest: total_size = 11534336 2024-05-30T16:18:18.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.846+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-30T16:18:18.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.846+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-05-30T16:18:18.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.846+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-05-30T16:18:18.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.846+0000 7fbc1ba93980 10 generate_fake_tag new tag=601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-LUKEWARM.2~wajqBmpC82qXXhM9doS3FFBuijhwm7k.2_dc86120fc4cbe247e2ec35caf3b7d080 2024-05-30T16:18:18.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.846+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-LUKEWARM.2~wajqBmpC82qXXhM9doS3FFBuijhwm7k.2_dc86120fc4cbe247e2ec35caf3b7d080 2024-05-30T16:18:18.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.846+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896c77d0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-LUKEWARM state=0x5596896c5018 s->prefetch_data=0 2024-05-30T16:18:18.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.846+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896c5018 2024-05-30T16:18:18.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.846+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17.8 2024-05-30T16:18:18.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.854+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-30T16:18:18.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.854+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-05-30T16:18:18.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.854+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-05-30T16:18:18.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.854+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-30T16:18:18.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.854+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2024-05-30T16:18:18.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.854+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-05-30T16:18:18.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.854+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-05-30T16:18:18.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.854+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2024-05-30T16:18:18.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.854+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-05-30T16:18:18.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.854+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-LUKEWARM.2~wajqBmpC82qXXhM9doS3FFBuijhwm7k.2_dc86120fc4cbe247e2ec35caf3b7d080 2024-05-30T16:18:18.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.854+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:18.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.854+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-LUKEWARM.2~wajqBmpC82qXXhM9doS3FFBuijhwm7k.1 2024-05-30T16:18:18.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.854+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 449 2024-05-30T16:18:18.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.854+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-LUKEWARM.2~wajqBmpC82qXXhM9doS3FFBuijhwm7k.2 2024-05-30T16:18:18.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.854+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 722 2024-05-30T16:18:18.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.854+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-LUKEWARM.2~wajqBmpC82qXXhM9doS3FFBuijhwm7k.3 2024-05-30T16:18:18.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.854+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 995 2024-05-30T16:18:18.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.854+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:18.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.854+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.26tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-LUKEWARM.2~wajqBmpC82qXXhM9doS3FFBuijhwm7k.2_dc86120fc4cbe247e2ec35caf3b7d080 2024-05-30T16:18:18.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.854+0000 7fbafcff9640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-LUKEWARM 2024-05-30T16:18:18.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.854+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc1009c040 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-FROZEN state=0x5596896c5018 s->prefetch_data=0 2024-05-30T16:18:18.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.858+0000 7fbc1ba93980 10 manifest: total_size = 1087 2024-05-30T16:18:18.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.858+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to _b_-sVqGh-yH058cG7edPlrV51cWwRdw 2024-05-30T16:18:18.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.858+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc1009c040 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-FROZEN state=0x5596896c5018 s->prefetch_data=0 2024-05-30T16:18:18.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.858+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896c5018 2024-05-30T16:18:18.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.858+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17.8 2024-05-30T16:18:18.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.862+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-30T16:18:18.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.862+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=1087 stripe_ofs=1087 part_ofs=0 rule->part_size=0 2024-05-30T16:18:18.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.862+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: _b_-sVqGh-yH058cG7edPlrV51cWwRdw 2024-05-30T16:18:18.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.862+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:18.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.862+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__shadow_.abmgsR_PDSXz9BE5Oo36g4vJ_Sd7CbP_0 2024-05-30T16:18:18.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.862+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 308 2024-05-30T16:18:18.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.862+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:18.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.862+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.9tag is: _b_-sVqGh-yH058cG7edPlrV51cWwRdw 2024-05-30T16:18:18.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.866+0000 7fbaeffff640 20 handle_completion(): completion ok for obj=foo-STANDARD-FROZEN 2024-05-30T16:18:18.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.866+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc1009c040 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-FROZEN state=0x5596896c5018 s->prefetch_data=0 2024-05-30T16:18:18.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.870+0000 7fbc1ba93980 10 manifest: total_size = 11534336 2024-05-30T16:18:18.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.870+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-30T16:18:18.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.870+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-05-30T16:18:18.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.870+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-05-30T16:18:18.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.870+0000 7fbc1ba93980 10 generate_fake_tag new tag=601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-FROZEN.2~BFRcb5Q8ovi-IgTqlOD8ZC7hmBrYMTv.2_aedc64ca1185a391387b70943af4161d 2024-05-30T16:18:18.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.870+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-FROZEN.2~BFRcb5Q8ovi-IgTqlOD8ZC7hmBrYMTv.2_aedc64ca1185a391387b70943af4161d 2024-05-30T16:18:18.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.870+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc1009c040 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-FROZEN state=0x5596896c5018 s->prefetch_data=0 2024-05-30T16:18:18.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.870+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896c5018 2024-05-30T16:18:18.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.870+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17.9 2024-05-30T16:18:18.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.874+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-30T16:18:18.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.874+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-05-30T16:18:18.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.874+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-05-30T16:18:18.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.874+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-30T16:18:18.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.874+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2024-05-30T16:18:18.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.874+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-05-30T16:18:18.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.874+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-05-30T16:18:18.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.874+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2024-05-30T16:18:18.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.874+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-05-30T16:18:18.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.874+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-FROZEN.2~BFRcb5Q8ovi-IgTqlOD8ZC7hmBrYMTv.2_aedc64ca1185a391387b70943af4161d 2024-05-30T16:18:18.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.874+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:18.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.874+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-FROZEN.2~BFRcb5Q8ovi-IgTqlOD8ZC7hmBrYMTv.1 2024-05-30T16:18:18.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.874+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 441 2024-05-30T16:18:18.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.874+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-FROZEN.2~BFRcb5Q8ovi-IgTqlOD8ZC7hmBrYMTv.2 2024-05-30T16:18:18.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.874+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 708 2024-05-30T16:18:18.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.874+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-FROZEN.2~BFRcb5Q8ovi-IgTqlOD8ZC7hmBrYMTv.3 2024-05-30T16:18:18.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.874+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 975 2024-05-30T16:18:18.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.874+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:18.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.874+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.22tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-FROZEN.2~BFRcb5Q8ovi-IgTqlOD8ZC7hmBrYMTv.2_aedc64ca1185a391387b70943af4161d 2024-05-30T16:18:18.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.878+0000 7fbc1ba93980 20 list_objects_ordered: starting attempt 1 2024-05-30T16:18:18.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.878+0000 7fbc1ba93980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-05-30T16:18:18.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.878+0000 7fbafcff9640 20 handle_completion(): completion ok for obj=foo-FROZEN-FROZEN 2024-05-30T16:18:18.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.882+0000 7fbc1ba93980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-05-30T16:18:18.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.882+0000 7fbc1ba93980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-05-30T16:18:18.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.882+0000 7fbc1ba93980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-05-30T16:18:18.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.882+0000 7fbc1ba93980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-05-30T16:18:18.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.882+0000 7fbc1ba93980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-multipart-copy.1[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17])"; uploads.size()=0; is_truncated=0 2024-05-30T16:18:18.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.886+0000 7fbc1ba93980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : miss 2024-05-30T16:18:18.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.886+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.886+0000 7fbc1ba93980 20 rados_obj.operate() r=0 bl.length=243 2024-05-30T16:18:18.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.886+0000 7fbc1ba93980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 info.flags=0x11 2024-05-30T16:18:18.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.886+0000 7fbc1ba93980 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 to cache LRU end 2024-05-30T16:18:18.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.890+0000 7fbc1ba93980 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 from cache 2024-05-30T16:18:18.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.890+0000 7fbc1ba93980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart-copy.1, ofs0, ns] 2024-05-30T16:18:18.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.890+0000 7fbaeffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431571 cookie 94105038802336 notifier 5889 bl.length()=199 2024-05-30T16:18:18.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.898+0000 7fbc1ba93980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17 from cache 2024-05-30T16:18:18.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.898+0000 7fbc1ba93980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17, ofs0, ns] 2024-05-30T16:18:18.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.898+0000 7fbafcff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431565 cookie 94105038842992 notifier 5889 bl.length()=257 2024-05-30T16:18:18.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.906+0000 7fbc1ba93980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart-copy.1/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17 : miss 2024-05-30T16:18:18.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.906+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.906+0000 7fbc1ba93980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:18:18.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.906+0000 7fbc1ba93980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart-copy.1/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17 info.flags=0x0 2024-05-30T16:18:18.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.906+0000 7fbc1ba93980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart-copy.1/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17 to cache LRU end 2024-05-30T16:18:18.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.914+0000 7fbc1ba93980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15 : miss 2024-05-30T16:18:18.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.914+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.914+0000 7fbc1ba93980 20 rados_obj.operate() r=0 bl.length=452 2024-05-30T16:18:18.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.914+0000 7fbc1ba93980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15 info.flags=0x17 2024-05-30T16:18:18.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.914+0000 7fbc1ba93980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15 to cache LRU end 2024-05-30T16:18:18.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.914+0000 7fbc1ba93980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:18:18.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.914+0000 7fbc1ba93980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-05-30T16:18:18.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.914+0000 7fbc1ba93980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15 2024-05-30T16:18:18.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.918+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo-FROZEN 2024-05-30T16:18:18.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.922+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo-STANDARD 2024-05-30T16:18:18.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.922+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo-LUKEWARM 2024-05-30T16:18:18.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.926+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896c5a60 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-FROZEN state=0x5596896e0828 s->prefetch_data=0 2024-05-30T16:18:18.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.930+0000 7fbc1ba93980 10 manifest: total_size = 11534336 2024-05-30T16:18:18.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.930+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11509641366497265618 2024-05-30T16:18:18.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.930+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896c5a60 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-FROZEN state=0x5596896e0828 s->prefetch_data=0 2024-05-30T16:18:18.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.930+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896e0828 2024-05-30T16:18:18.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.930+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15.3 2024-05-30T16:18:18.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.934+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-30T16:18:18.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.934+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-05-30T16:18:18.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.934+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-05-30T16:18:18.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.934+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-30T16:18:18.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.934+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2024-05-30T16:18:18.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.934+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-05-30T16:18:18.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.934+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-05-30T16:18:18.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.934+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2024-05-30T16:18:18.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.934+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-05-30T16:18:18.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.934+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11509641366497265618 2024-05-30T16:18:18.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.934+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:18.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.934+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15__multipart_foo-FROZEN.2~CW-sTJfcAqjohKEMX8QliHZkV3OM6hr.1 2024-05-30T16:18:18.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.934+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 368 2024-05-30T16:18:18.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.934+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15__multipart_foo-FROZEN.2~CW-sTJfcAqjohKEMX8QliHZkV3OM6hr.2 2024-05-30T16:18:18.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.934+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 635 2024-05-30T16:18:18.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.934+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15__multipart_foo-FROZEN.2~CW-sTJfcAqjohKEMX8QliHZkV3OM6hr.3 2024-05-30T16:18:18.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.934+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 902 2024-05-30T16:18:18.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.934+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:18.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.934+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.23tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11509641366497265618 2024-05-30T16:18:18.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.938+0000 7fbafcff9640 20 handle_completion(): completion ok for obj=foo-FROZEN 2024-05-30T16:18:18.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.938+0000 7fbc1ba93980 20 get_obj_state: octx=0x559689740940 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-STANDARD state=0x5596896e0828 s->prefetch_data=0 2024-05-30T16:18:18.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.942+0000 7fbc1ba93980 10 manifest: total_size = 45360 2024-05-30T16:18:18.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.942+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11005071422508405881 2024-05-30T16:18:18.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.942+0000 7fbc1ba93980 20 get_obj_state: octx=0x559689740940 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-STANDARD state=0x5596896e0828 s->prefetch_data=0 2024-05-30T16:18:18.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.942+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896e0828 2024-05-30T16:18:18.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.942+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15.5 2024-05-30T16:18:18.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.946+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2024-05-30T16:18:18.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.946+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=20613 2024-05-30T16:18:18.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.946+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=20613 stripe_ofs=20613 part_ofs=20613 rule->part_size=20613 2024-05-30T16:18:18.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.946+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2024-05-30T16:18:18.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.946+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6312069 part_ofs=20613 rule->part_size=20613 2024-05-30T16:18:18.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.946+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=41226 stripe_ofs=41226 part_ofs=41226 rule->part_size=4134 2024-05-30T16:18:18.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.946+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=4134 rules.size()=2 2024-05-30T16:18:18.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.946+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6332682 part_ofs=41226 rule->part_size=4134 2024-05-30T16:18:18.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.946+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=45360 stripe_ofs=45360 part_ofs=45360 rule->part_size=4134 2024-05-30T16:18:18.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.946+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11005071422508405881 2024-05-30T16:18:18.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.946+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:18.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.946+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15__multipart_foo-STANDARD.2~gIaO10Qh7XOEWbUAn_wo19sWSzd6og2.1 2024-05-30T16:18:18.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.946+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 365 2024-05-30T16:18:18.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.946+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15__multipart_foo-STANDARD.2~gIaO10Qh7XOEWbUAn_wo19sWSzd6og2.2 2024-05-30T16:18:18.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.946+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 629 2024-05-30T16:18:18.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.946+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15__multipart_foo-STANDARD.2~gIaO10Qh7XOEWbUAn_wo19sWSzd6og2.3 2024-05-30T16:18:18.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.946+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 893 2024-05-30T16:18:18.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.946+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:18.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.946+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.29tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11005071422508405881 2024-05-30T16:18:18.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.950+0000 7fbaeffff640 20 handle_completion(): completion ok for obj=foo-STANDARD 2024-05-30T16:18:18.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.950+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc1009c040 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-LUKEWARM state=0x5596896e0828 s->prefetch_data=0 2024-05-30T16:18:18.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.950+0000 7fbc1ba93980 10 manifest: total_size = 11534336 2024-05-30T16:18:18.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.950+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11488756881640554302 2024-05-30T16:18:18.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.950+0000 7fbc1ba93980 20 get_obj_state: octx=0x7fbc1009c040 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-LUKEWARM state=0x5596896e0828 s->prefetch_data=0 2024-05-30T16:18:18.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.950+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896e0828 2024-05-30T16:18:18.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.950+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15.7 2024-05-30T16:18:18.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.958+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-30T16:18:18.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.958+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-05-30T16:18:18.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.958+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-05-30T16:18:18.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.958+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-30T16:18:18.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.958+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2024-05-30T16:18:18.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.958+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-05-30T16:18:18.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.958+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-05-30T16:18:18.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.958+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2024-05-30T16:18:18.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.958+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-05-30T16:18:18.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.958+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11488756881640554302 2024-05-30T16:18:18.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.958+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:18.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.958+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15__multipart_foo-LUKEWARM.2~Hsrr_GObGFXEobyJ1aV_Kc0udgQ82aw.1 2024-05-30T16:18:18.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.958+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 374 2024-05-30T16:18:18.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.958+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15__multipart_foo-LUKEWARM.2~Hsrr_GObGFXEobyJ1aV_Kc0udgQ82aw.2 2024-05-30T16:18:18.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.958+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 647 2024-05-30T16:18:18.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.958+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15__multipart_foo-LUKEWARM.2~Hsrr_GObGFXEobyJ1aV_Kc0udgQ82aw.3 2024-05-30T16:18:18.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.958+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 920 2024-05-30T16:18:18.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.958+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:18.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.958+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.3tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11488756881640554302 2024-05-30T16:18:18.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.962+0000 7fbafcff9640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2024-05-30T16:18:18.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.962+0000 7fbc1ba93980 20 list_objects_ordered: starting attempt 1 2024-05-30T16:18:18.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.962+0000 7fbc1ba93980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-05-30T16:18:18.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.966+0000 7fbc1ba93980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-05-30T16:18:18.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.966+0000 7fbc1ba93980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-05-30T16:18:18.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.966+0000 7fbc1ba93980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-05-30T16:18:18.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.966+0000 7fbc1ba93980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-05-30T16:18:18.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.966+0000 7fbc1ba93980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-multipart.1[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15])"; uploads.size()=0; is_truncated=0 2024-05-30T16:18:18.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.970+0000 7fbc1ba93980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : miss 2024-05-30T16:18:18.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.970+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.970+0000 7fbc1ba93980 20 rados_obj.operate() r=0 bl.length=238 2024-05-30T16:18:18.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.970+0000 7fbc1ba93980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 info.flags=0x11 2024-05-30T16:18:18.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.970+0000 7fbc1ba93980 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 to cache LRU end 2024-05-30T16:18:18.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.974+0000 7fbc1ba93980 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 from cache 2024-05-30T16:18:18.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.974+0000 7fbc1ba93980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart.1, ofs0, ns] 2024-05-30T16:18:18.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.974+0000 7fbaeffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431617 cookie 94105038840864 notifier 5889 bl.length()=194 2024-05-30T16:18:18.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.978+0000 7fbc1ba93980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15 from cache 2024-05-30T16:18:18.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.978+0000 7fbc1ba93980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15, ofs0, ns] 2024-05-30T16:18:18.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.982+0000 7fbaeffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431566 cookie 94105038842992 notifier 5889 bl.length()=252 2024-05-30T16:18:18.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.990+0000 7fbc1ba93980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart.1/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15 : miss 2024-05-30T16:18:18.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.990+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.990+0000 7fbc1ba93980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:18:18.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.990+0000 7fbc1ba93980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart.1/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15 info.flags=0x0 2024-05-30T16:18:18.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.990+0000 7fbc1ba93980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart.1/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15 to cache LRU end 2024-05-30T16:18:18.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.994+0000 7fbc1ba93980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.14 : miss 2024-05-30T16:18:18.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.994+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:18.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.998+0000 7fbc1ba93980 20 rados_obj.operate() r=0 bl.length=442 2024-05-30T16:18:18.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.998+0000 7fbc1ba93980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.14 info.flags=0x17 2024-05-30T16:18:18.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.998+0000 7fbc1ba93980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.14 to cache LRU end 2024-05-30T16:18:18.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.998+0000 7fbc1ba93980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:18:18.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.998+0000 7fbc1ba93980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-05-30T16:18:18.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:18.998+0000 7fbc1ba93980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.14 2024-05-30T16:18:19.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.002+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo-FROZEN 2024-05-30T16:18:19.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.002+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo-STANDARD 2024-05-30T16:18:19.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.006+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got foo-LUKEWARM 2024-05-30T16:18:19.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.010+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896c5cd0 obj=test-client.0-r-test-obj-storage-class.1:foo-FROZEN state=0x5596896c5018 s->prefetch_data=0 2024-05-30T16:18:19.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.010+0000 7fbc1ba93980 10 manifest: total_size = 10485760 2024-05-30T16:18:19.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.010+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.7737070351589065933 2024-05-30T16:18:19.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.010+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896c5cd0 obj=test-client.0-r-test-obj-storage-class.1:foo-FROZEN state=0x5596896c5018 s->prefetch_data=0 2024-05-30T16:18:19.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.010+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896c5018 2024-05-30T16:18:19.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.010+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.14.3 2024-05-30T16:18:19.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.018+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-30T16:18:19.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.018+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=6291456 stripe_ofs=6291456 part_ofs=0 rule->part_size=0 2024-05-30T16:18:19.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.018+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-30T16:18:19.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.018+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2024-05-30T16:18:19.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.018+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.7737070351589065933 2024-05-30T16:18:19.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.018+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:19.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.018+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.14__shadow_.AXOmusT1Gw2HgSoCl5q-gSncRGm6ftL_0 2024-05-30T16:18:19.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.018+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 337 2024-05-30T16:18:19.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.018+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.14__shadow_.AXOmusT1Gw2HgSoCl5q-gSncRGm6ftL_1 2024-05-30T16:18:19.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.018+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 574 2024-05-30T16:18:19.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.018+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:19.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.018+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.0tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.7737070351589065933 2024-05-30T16:18:19.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.022+0000 7fbafcff9640 20 handle_completion(): completion ok for obj=foo-FROZEN 2024-05-30T16:18:19.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.022+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596894bef50 obj=test-client.0-r-test-obj-storage-class.1:foo-STANDARD state=0x5596896c5018 s->prefetch_data=0 2024-05-30T16:18:19.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.022+0000 7fbc1ba93980 10 manifest: total_size = 3303 2024-05-30T16:18:19.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.022+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.8411597514444599318 2024-05-30T16:18:19.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.022+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596894bef50 obj=test-client.0-r-test-obj-storage-class.1:foo-STANDARD state=0x5596896c5018 s->prefetch_data=0 2024-05-30T16:18:19.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.022+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896c5018 2024-05-30T16:18:19.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.022+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.14.5 2024-05-30T16:18:19.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.030+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596894bef50 obj=test-client.0-r-test-obj-storage-class.1:foo-LUKEWARM state=0x5596896c5018 s->prefetch_data=0 2024-05-30T16:18:19.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.030+0000 7fbc1ba93980 10 manifest: total_size = 10485760 2024-05-30T16:18:19.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.030+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.6717838609504419540 2024-05-30T16:18:19.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.030+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596894bef50 obj=test-client.0-r-test-obj-storage-class.1:foo-LUKEWARM state=0x5596896c5018 s->prefetch_data=0 2024-05-30T16:18:19.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.030+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896c5018 2024-05-30T16:18:19.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.030+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.14.7 2024-05-30T16:18:19.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.034+0000 7fbaeffff640 20 handle_completion(): completion ok for obj=foo-STANDARD 2024-05-30T16:18:19.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.038+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-30T16:18:19.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.038+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=6291456 stripe_ofs=6291456 part_ofs=0 rule->part_size=0 2024-05-30T16:18:19.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.038+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-30T16:18:19.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.038+0000 7fbc1ba93980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2024-05-30T16:18:19.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.038+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.6717838609504419540 2024-05-30T16:18:19.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.038+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-30T16:18:19.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.038+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.14__shadow_.gK8bQIHPaijsLps-ink8QZ-02ky_Yjd_0 2024-05-30T16:18:19.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.038+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 339 2024-05-30T16:18:19.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.038+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.14__shadow_.gK8bQIHPaijsLps-ink8QZ-02ky_Yjd_1 2024-05-30T16:18:19.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.038+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 578 2024-05-30T16:18:19.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.038+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-30T16:18:19.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.038+0000 7fbc1ba93980 20 garbage collection: RGWGC::send_chain - on object name: gc.31tag is: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.6717838609504419540 2024-05-30T16:18:19.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.042+0000 7fbc1ba93980 20 list_objects_ordered: starting attempt 1 2024-05-30T16:18:19.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.042+0000 7fbc1ba93980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-05-30T16:18:19.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.042+0000 7fbafcff9640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2024-05-30T16:18:19.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.042+0000 7fbc1ba93980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-05-30T16:18:19.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.042+0000 7fbc1ba93980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-05-30T16:18:19.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.042+0000 7fbc1ba93980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-05-30T16:18:19.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.042+0000 7fbc1ba93980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-05-30T16:18:19.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.042+0000 7fbc1ba93980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class.1[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.14])"; uploads.size()=0; is_truncated=0 2024-05-30T16:18:19.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.046+0000 7fbc1ba93980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : miss 2024-05-30T16:18:19.049 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.046+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:19.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.050+0000 7fbc1ba93980 20 rados_obj.operate() r=0 bl.length=228 2024-05-30T16:18:19.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.050+0000 7fbc1ba93980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 info.flags=0x11 2024-05-30T16:18:19.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.050+0000 7fbc1ba93980 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 to cache LRU end 2024-05-30T16:18:19.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.050+0000 7fbc1ba93980 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 from cache 2024-05-30T16:18:19.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.050+0000 7fbc1ba93980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class.1, ofs0, ns] 2024-05-30T16:18:19.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.054+0000 7fbaeffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431574 cookie 94105038832944 notifier 5889 bl.length()=184 2024-05-30T16:18:19.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.058+0000 7fbc1ba93980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.14 from cache 2024-05-30T16:18:19.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.058+0000 7fbc1ba93980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.14, ofs0, ns] 2024-05-30T16:18:19.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.062+0000 7fbaeffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431618 cookie 94105038845488 notifier 5889 bl.length()=242 2024-05-30T16:18:19.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.070+0000 7fbc1ba93980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class.1/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.14 : miss 2024-05-30T16:18:19.071 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.070+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:19.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.070+0000 7fbc1ba93980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:18:19.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.070+0000 7fbc1ba93980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class.1/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.14 info.flags=0x0 2024-05-30T16:18:19.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.070+0000 7fbc1ba93980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class.1/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.14 to cache LRU end 2024-05-30T16:18:19.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.074+0000 7fbc1ba93980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.7 : miss 2024-05-30T16:18:19.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.074+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:19.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.078+0000 7fbc1ba93980 20 rados_obj.operate() r=0 bl.length=437 2024-05-30T16:18:19.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.078+0000 7fbc1ba93980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.7 info.flags=0x17 2024-05-30T16:18:19.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.078+0000 7fbc1ba93980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.7 to cache LRU end 2024-05-30T16:18:19.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.078+0000 7fbc1ba93980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:18:19.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.078+0000 7fbc1ba93980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-05-30T16:18:19.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.078+0000 7fbc1ba93980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.7 2024-05-30T16:18:19.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.082+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got ___ 2024-05-30T16:18:19.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.082+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got __ _ 2024-05-30T16:18:19.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.086+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got __ 2024-05-30T16:18:19.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.086+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got obj 2024-05-30T16:18:19.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.090+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896a16c0 obj=test-client.0-r-test-small-obj-data.1:___ state=0x5596896e15b8 s->prefetch_data=0 2024-05-30T16:18:19.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.090+0000 7fbc1ba93980 10 manifest: total_size = 0 2024-05-30T16:18:19.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.090+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.4841706647402776754 2024-05-30T16:18:19.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.090+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896a16c0 obj=test-client.0-r-test-small-obj-data.1:___ state=0x5596896e15b8 s->prefetch_data=0 2024-05-30T16:18:19.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.090+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896e15b8 2024-05-30T16:18:19.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.090+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.7.2 2024-05-30T16:18:19.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.094+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896a16c0 obj=test-client.0-r-test-small-obj-data.1:__ _ state=0x5596896e15b8 s->prefetch_data=0 2024-05-30T16:18:19.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.098+0000 7fbc1ba93980 10 manifest: total_size = 0 2024-05-30T16:18:19.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.098+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12323852631901724667 2024-05-30T16:18:19.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.098+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896a16c0 obj=test-client.0-r-test-small-obj-data.1:__ _ state=0x5596896e15b8 s->prefetch_data=0 2024-05-30T16:18:19.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.098+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896e15b8 2024-05-30T16:18:19.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.098+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.7.3 2024-05-30T16:18:19.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.098+0000 7fbaeffff640 20 handle_completion(): completion ok for obj=___ 2024-05-30T16:18:19.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.102+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896a16c0 obj=test-client.0-r-test-small-obj-data.1:__ state=0x5596896e15b8 s->prefetch_data=0 2024-05-30T16:18:19.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.106+0000 7fbc1ba93980 10 manifest: total_size = 0 2024-05-30T16:18:19.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.106+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15161347617754988604 2024-05-30T16:18:19.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.106+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896a16c0 obj=test-client.0-r-test-small-obj-data.1:__ state=0x5596896e15b8 s->prefetch_data=0 2024-05-30T16:18:19.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.106+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896e15b8 2024-05-30T16:18:19.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.106+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.7.6 2024-05-30T16:18:19.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.106+0000 7fbafcff9640 20 handle_completion(): completion ok for obj=__ _ 2024-05-30T16:18:19.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.110+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896a16c0 obj=test-client.0-r-test-small-obj-data.1:obj state=0x5596896e15b8 s->prefetch_data=0 2024-05-30T16:18:19.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.114+0000 7fbc1ba93980 10 manifest: total_size = 0 2024-05-30T16:18:19.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.114+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.8713233143150967120 2024-05-30T16:18:19.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.114+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896a16c0 obj=test-client.0-r-test-small-obj-data.1:obj state=0x5596896e15b8 s->prefetch_data=0 2024-05-30T16:18:19.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.114+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896e15b8 2024-05-30T16:18:19.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.114+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.7.8 2024-05-30T16:18:19.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.114+0000 7fbaeffff640 20 handle_completion(): completion ok for obj=__ 2024-05-30T16:18:19.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.118+0000 7fbc1ba93980 20 list_objects_ordered: starting attempt 1 2024-05-30T16:18:19.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.118+0000 7fbc1ba93980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-05-30T16:18:19.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.122+0000 7fbaeffff640 20 handle_completion(): completion ok for obj=obj 2024-05-30T16:18:19.125 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.122+0000 7fbc1ba93980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-05-30T16:18:19.125 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.122+0000 7fbc1ba93980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-05-30T16:18:19.125 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.122+0000 7fbc1ba93980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-05-30T16:18:19.125 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.122+0000 7fbc1ba93980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-05-30T16:18:19.125 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.122+0000 7fbc1ba93980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.1[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.7])"; uploads.size()=0; is_truncated=0 2024-05-30T16:18:19.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.130+0000 7fbc1ba93980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : miss 2024-05-30T16:18:19.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.130+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:19.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.130+0000 7fbc1ba93980 20 rados_obj.operate() r=0 bl.length=223 2024-05-30T16:18:19.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.130+0000 7fbc1ba93980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 info.flags=0x11 2024-05-30T16:18:19.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.130+0000 7fbc1ba93980 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 to cache LRU end 2024-05-30T16:18:19.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.134+0000 7fbc1ba93980 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 from cache 2024-05-30T16:18:19.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.134+0000 7fbc1ba93980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-small-obj-data.1, ofs0, ns] 2024-05-30T16:18:19.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.134+0000 7fbafcff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431607 cookie 94105038829744 notifier 5889 bl.length()=181 2024-05-30T16:18:19.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.142+0000 7fbc1ba93980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.7 from cache 2024-05-30T16:18:19.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.142+0000 7fbc1ba93980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.1:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.7, ofs0, ns] 2024-05-30T16:18:19.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.142+0000 7fbafcff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431588 cookie 94105038800128 notifier 5889 bl.length()=238 2024-05-30T16:18:19.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.150+0000 7fbc1ba93980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.1/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.7 : miss 2024-05-30T16:18:19.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.150+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:19.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.154+0000 7fbc1ba93980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:18:19.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.154+0000 7fbc1ba93980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.1/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.7 info.flags=0x0 2024-05-30T16:18:19.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.154+0000 7fbc1ba93980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.1/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.7 to cache LRU end 2024-05-30T16:18:19.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.158+0000 7fbc1ba93980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.8 : miss 2024-05-30T16:18:19.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.158+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:19.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.158+0000 7fbc1ba93980 20 rados_obj.operate() r=0 bl.length=437 2024-05-30T16:18:19.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.158+0000 7fbc1ba93980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.8 info.flags=0x17 2024-05-30T16:18:19.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.158+0000 7fbc1ba93980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.8 to cache LRU end 2024-05-30T16:18:19.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.158+0000 7fbc1ba93980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:18:19.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.158+0000 7fbc1ba93980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-05-30T16:18:19.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.158+0000 7fbc1ba93980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.8 2024-05-30T16:18:19.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.162+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got ___ 2024-05-30T16:18:19.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.162+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got __ _ 2024-05-30T16:18:19.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.166+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got __ 2024-05-30T16:18:19.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.170+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got obj 2024-05-30T16:18:19.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.170+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896cb9a0 obj=test-client.0-r-test-small-obj-data.2:___ state=0x5596896ddc28 s->prefetch_data=0 2024-05-30T16:18:19.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.174+0000 7fbc1ba93980 10 manifest: total_size = 524288 2024-05-30T16:18:19.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.174+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16369010840051949140 2024-05-30T16:18:19.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.174+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896cb9a0 obj=test-client.0-r-test-small-obj-data.2:___ state=0x5596896ddc28 s->prefetch_data=0 2024-05-30T16:18:19.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.174+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896ddc28 2024-05-30T16:18:19.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.174+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.8.2 2024-05-30T16:18:19.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.178+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896cb9a0 obj=test-client.0-r-test-small-obj-data.2:__ _ state=0x5596896ddc28 s->prefetch_data=0 2024-05-30T16:18:19.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.178+0000 7fbc1ba93980 10 manifest: total_size = 524288 2024-05-30T16:18:19.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.178+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.1888049234543746407 2024-05-30T16:18:19.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.178+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896cb9a0 obj=test-client.0-r-test-small-obj-data.2:__ _ state=0x5596896ddc28 s->prefetch_data=0 2024-05-30T16:18:19.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.178+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896ddc28 2024-05-30T16:18:19.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.178+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.8.3 2024-05-30T16:18:19.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.182+0000 7fbaeffff640 20 handle_completion(): completion ok for obj=___ 2024-05-30T16:18:19.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.186+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896cb9a0 obj=test-client.0-r-test-small-obj-data.2:__ state=0x5596896ddc28 s->prefetch_data=0 2024-05-30T16:18:19.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.190+0000 7fbc1ba93980 10 manifest: total_size = 527 2024-05-30T16:18:19.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.190+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16103038695535725340 2024-05-30T16:18:19.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.190+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896cb9a0 obj=test-client.0-r-test-small-obj-data.2:__ state=0x5596896ddc28 s->prefetch_data=0 2024-05-30T16:18:19.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.190+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896ddc28 2024-05-30T16:18:19.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.190+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.8.6 2024-05-30T16:18:19.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.190+0000 7fbafcff9640 20 handle_completion(): completion ok for obj=__ _ 2024-05-30T16:18:19.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.194+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896cb9a0 obj=test-client.0-r-test-small-obj-data.2:obj state=0x5596896ddc28 s->prefetch_data=0 2024-05-30T16:18:19.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.198+0000 7fbc1ba93980 10 manifest: total_size = 38 2024-05-30T16:18:19.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.198+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16888185865902538099 2024-05-30T16:18:19.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.198+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896cb9a0 obj=test-client.0-r-test-small-obj-data.2:obj state=0x5596896ddc28 s->prefetch_data=0 2024-05-30T16:18:19.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.198+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896ddc28 2024-05-30T16:18:19.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.198+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.8.8 2024-05-30T16:18:19.200 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.198+0000 7fbaeffff640 20 handle_completion(): completion ok for obj=__ 2024-05-30T16:18:19.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.202+0000 7fbc1ba93980 20 list_objects_ordered: starting attempt 1 2024-05-30T16:18:19.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.202+0000 7fbc1ba93980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-05-30T16:18:19.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.206+0000 7fbafcff9640 20 handle_completion(): completion ok for obj=obj 2024-05-30T16:18:19.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.206+0000 7fbc1ba93980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-05-30T16:18:19.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.206+0000 7fbc1ba93980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-05-30T16:18:19.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.206+0000 7fbc1ba93980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-05-30T16:18:19.207 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.206+0000 7fbc1ba93980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-05-30T16:18:19.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.206+0000 7fbc1ba93980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.2[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.8])"; uploads.size()=0; is_truncated=0 2024-05-30T16:18:19.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.210+0000 7fbc1ba93980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : miss 2024-05-30T16:18:19.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.210+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:19.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.214+0000 7fbc1ba93980 20 rados_obj.operate() r=0 bl.length=223 2024-05-30T16:18:19.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.214+0000 7fbc1ba93980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 info.flags=0x11 2024-05-30T16:18:19.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.214+0000 7fbc1ba93980 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 to cache LRU end 2024-05-30T16:18:19.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.214+0000 7fbc1ba93980 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 from cache 2024-05-30T16:18:19.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.214+0000 7fbc1ba93980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-small-obj-data.2, ofs0, ns] 2024-05-30T16:18:19.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.214+0000 7fbafcff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431608 cookie 94105038829744 notifier 5889 bl.length()=181 2024-05-30T16:18:19.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.222+0000 7fbc1ba93980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.8 from cache 2024-05-30T16:18:19.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.222+0000 7fbc1ba93980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.2:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.8, ofs0, ns] 2024-05-30T16:18:19.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.222+0000 7fbaeffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431589 cookie 94105038800128 notifier 5889 bl.length()=238 2024-05-30T16:18:19.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.234+0000 7fbc1ba93980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.2/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.8 : miss 2024-05-30T16:18:19.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.234+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:19.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.234+0000 7fbc1ba93980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:18:19.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.234+0000 7fbc1ba93980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.2/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.8 info.flags=0x0 2024-05-30T16:18:19.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.234+0000 7fbc1ba93980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.2/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.8 to cache LRU end 2024-05-30T16:18:19.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.238+0000 7fbc1ba93980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.9 : miss 2024-05-30T16:18:19.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.238+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:19.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.242+0000 7fbc1ba93980 20 rados_obj.operate() r=0 bl.length=437 2024-05-30T16:18:19.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.242+0000 7fbc1ba93980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.9 info.flags=0x17 2024-05-30T16:18:19.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.242+0000 7fbc1ba93980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.9 to cache LRU end 2024-05-30T16:18:19.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.242+0000 7fbc1ba93980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:18:19.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.242+0000 7fbc1ba93980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-05-30T16:18:19.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.242+0000 7fbc1ba93980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.9 2024-05-30T16:18:19.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.242+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got ___ 2024-05-30T16:18:19.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.246+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got __ _ 2024-05-30T16:18:19.248 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.246+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got __ 2024-05-30T16:18:19.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.250+0000 7fbc1ba93980 10 cls_bucket_list_unordered: got obj 2024-05-30T16:18:19.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.250+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896c6200 obj=test-client.0-r-test-small-obj-data.3:___ state=0x5596896ddc28 s->prefetch_data=0 2024-05-30T16:18:19.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.250+0000 7fbc1ba93980 10 manifest: total_size = 4134 2024-05-30T16:18:19.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.250+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.8201868134063333217 2024-05-30T16:18:19.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.250+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896c6200 obj=test-client.0-r-test-small-obj-data.3:___ state=0x5596896ddc28 s->prefetch_data=0 2024-05-30T16:18:19.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.250+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896ddc28 2024-05-30T16:18:19.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.250+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.9.2 2024-05-30T16:18:19.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.258+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896c6200 obj=test-client.0-r-test-small-obj-data.3:__ _ state=0x5596896ddc28 s->prefetch_data=0 2024-05-30T16:18:19.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.262+0000 7fbc1ba93980 10 manifest: total_size = 1035 2024-05-30T16:18:19.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.262+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12645142972887824752 2024-05-30T16:18:19.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.262+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896c6200 obj=test-client.0-r-test-small-obj-data.3:__ _ state=0x5596896ddc28 s->prefetch_data=0 2024-05-30T16:18:19.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.262+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896ddc28 2024-05-30T16:18:19.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.262+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.9.3 2024-05-30T16:18:19.263 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.262+0000 7fbaeffff640 20 handle_completion(): completion ok for obj=___ 2024-05-30T16:18:19.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.266+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896c6200 obj=test-client.0-r-test-small-obj-data.3:__ state=0x5596896ddc28 s->prefetch_data=0 2024-05-30T16:18:19.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.270+0000 7fbc1ba93980 10 manifest: total_size = 55 2024-05-30T16:18:19.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.270+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12136805557184008275 2024-05-30T16:18:19.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.270+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896c6200 obj=test-client.0-r-test-small-obj-data.3:__ state=0x5596896ddc28 s->prefetch_data=0 2024-05-30T16:18:19.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.270+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896ddc28 2024-05-30T16:18:19.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.270+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.9.6 2024-05-30T16:18:19.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.270+0000 7fbafcff9640 20 handle_completion(): completion ok for obj=__ _ 2024-05-30T16:18:19.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.278+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896c6200 obj=test-client.0-r-test-small-obj-data.3:obj state=0x5596896ddc28 s->prefetch_data=0 2024-05-30T16:18:19.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.278+0000 7fbc1ba93980 10 manifest: total_size = 4134 2024-05-30T16:18:19.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.278+0000 7fbc1ba93980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.7021931345951748059 2024-05-30T16:18:19.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.278+0000 7fbc1ba93980 20 get_obj_state: octx=0x5596896c6200 obj=test-client.0-r-test-small-obj-data.3:obj state=0x5596896ddc28 s->prefetch_data=0 2024-05-30T16:18:19.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.278+0000 7fbc1ba93980 20 prepare_atomic_modification: state is not atomic. state=0x5596896ddc28 2024-05-30T16:18:19.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.278+0000 7fbc1ba93980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.9.8 2024-05-30T16:18:19.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.278+0000 7fbaeffff640 20 handle_completion(): completion ok for obj=__ 2024-05-30T16:18:19.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.286+0000 7fbc1ba93980 20 list_objects_ordered: starting attempt 1 2024-05-30T16:18:19.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.286+0000 7fbc1ba93980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-05-30T16:18:19.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.286+0000 7fbafcff9640 20 handle_completion(): completion ok for obj=obj 2024-05-30T16:18:19.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.290+0000 7fbc1ba93980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-05-30T16:18:19.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.290+0000 7fbc1ba93980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-05-30T16:18:19.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.290+0000 7fbc1ba93980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-05-30T16:18:19.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.290+0000 7fbc1ba93980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-05-30T16:18:19.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.290+0000 7fbc1ba93980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.3[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.9])"; uploads.size()=0; is_truncated=0 2024-05-30T16:18:19.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.294+0000 7fbc1ba93980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : miss 2024-05-30T16:18:19.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.294+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:19.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.294+0000 7fbc1ba93980 20 rados_obj.operate() r=0 bl.length=223 2024-05-30T16:18:19.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.294+0000 7fbc1ba93980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 info.flags=0x11 2024-05-30T16:18:19.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.294+0000 7fbc1ba93980 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 to cache LRU end 2024-05-30T16:18:19.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.298+0000 7fbc1ba93980 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 from cache 2024-05-30T16:18:19.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.298+0000 7fbc1ba93980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-small-obj-data.3, ofs0, ns] 2024-05-30T16:18:19.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.298+0000 7fbaeffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431609 cookie 94105038829744 notifier 5889 bl.length()=181 2024-05-30T16:18:19.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.302+0000 7fbc1ba93980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.9 from cache 2024-05-30T16:18:19.305 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.302+0000 7fbc1ba93980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.3:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.9, ofs0, ns] 2024-05-30T16:18:19.305 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.306+0000 7fbaeffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431590 cookie 94105038800128 notifier 5889 bl.length()=238 2024-05-30T16:18:19.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.314+0000 7fbc1ba93980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.3/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.9 : miss 2024-05-30T16:18:19.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.314+0000 7fbc1ba93980 20 rados->read ofs=0 len=0 2024-05-30T16:18:19.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.314+0000 7fbc1ba93980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:18:19.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.314+0000 7fbc1ba93980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.3/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.9 info.flags=0x0 2024-05-30T16:18:19.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.314+0000 7fbc1ba93980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.3/601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.9 to cache LRU end 2024-05-30T16:18:19.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.318+0000 7fbc1ba93980 10 removing key index: SJFTXZIBRWVIGSTRSZSW 2024-05-30T16:18:19.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.322+0000 7fbc1ba93980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:users.keys:SJFTXZIBRWVIGSTRSZSW, ofs0, ns] 2024-05-30T16:18:19.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.322+0000 7fbafcff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431610 cookie 94105038829744 notifier 5889 bl.length()=170 2024-05-30T16:18:19.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.326+0000 7fbc1ba93980 10 removing email index: ragweed.client.0+test@test.test 2024-05-30T16:18:19.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.330+0000 7fbc1ba93980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.email:ragweed.client.0+test@test.test, ofs0, ns] 2024-05-30T16:18:19.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.330+0000 7fbafcff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431575 cookie 94105038832944 notifier 5889 bl.length()=182 2024-05-30T16:18:19.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.334+0000 7fbc1ba93980 10 removing user buckets index 2024-05-30T16:18:19.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.338+0000 7fbc1ba93980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:ragweed.client.0.buckets, ofs0, ns] 2024-05-30T16:18:19.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.338+0000 7fbafcff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431619 cookie 94105038840864 notifier 5889 bl.length()=173 2024-05-30T16:18:19.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.338+0000 7fbc1ba93980 10 removing user index: ragweed.client.0 2024-05-30T16:18:19.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.342+0000 7fbc1ba93980 10 removing default.rgw.meta+users.uid+ragweed.client.0 from cache 2024-05-30T16:18:19.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.342+0000 7fbc1ba93980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.uid:ragweed.client.0, ofs0, ns] 2024-05-30T16:18:19.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.346+0000 7fbaeffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431591 cookie 94105038836800 notifier 5889 bl.length()=165 2024-05-30T16:18:19.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.354+0000 7fbacf7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:18:19.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.362+0000 7fbc1ba93980 20 remove_watcher() i=0 2024-05-30T16:18:19.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.362+0000 7fbc1ba93980 2 removed watcher, disabling cache 2024-05-30T16:18:19.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.362+0000 7fbc1ba93980 20 remove_watcher() i=1 2024-05-30T16:18:19.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.366+0000 7fbc1ba93980 20 remove_watcher() i=2 2024-05-30T16:18:19.370 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.370+0000 7fbc1ba93980 20 remove_watcher() i=3 2024-05-30T16:18:19.374 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.374+0000 7fbc1ba93980 20 remove_watcher() i=4 2024-05-30T16:18:19.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.374+0000 7fbc1ba93980 20 remove_watcher() i=5 2024-05-30T16:18:19.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.378+0000 7fbc1ba93980 20 remove_watcher() i=6 2024-05-30T16:18:19.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.378+0000 7fbc1ba93980 20 remove_watcher() i=7 2024-05-30T16:18:19.402 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid sysuser.client.0 --purge-data 2024-05-30T16:18:19.482 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:18:19.482 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:18:19.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.526+0000 7faf09eee980 20 rados->read ofs=0 len=0 2024-05-30T16:18:19.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.526+0000 7faf09eee980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:18:19.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.526+0000 7faf09eee980 20 realm 2024-05-30T16:18:19.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.526+0000 7faf09eee980 20 rados->read ofs=0 len=0 2024-05-30T16:18:19.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.526+0000 7faf09eee980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:18:19.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.526+0000 7faf09eee980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:18:19.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.526+0000 7faf09eee980 20 rados->read ofs=0 len=0 2024-05-30T16:18:19.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.526+0000 7faf09eee980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:18:19.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.526+0000 7faf09eee980 20 rados->read ofs=0 len=0 2024-05-30T16:18:19.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.526+0000 7faf09eee980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:18:19.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.526+0000 7faf09eee980 20 rados->read ofs=0 len=0 2024-05-30T16:18:19.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.530+0000 7faf09eee980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:18:19.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.530+0000 7faf09eee980 20 searching for the correct realm 2024-05-30T16:18:19.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.534+0000 7faf071fe640 0 -- 172.21.15.6:0/1240441795 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7faef8008050 msgr2=0x7faef808d990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:18:19.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.542+0000 7faf079ff640 0 -- 172.21.15.6:0/1240441795 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7faef40edba0 msgr2=0x7faef41239c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:18:19.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.758+0000 7faf09eee980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:18:19.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.758+0000 7faf09eee980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:18:19.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.758+0000 7faf09eee980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:18:19.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.758+0000 7faf09eee980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:18:19.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.758+0000 7faf09eee980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:18:19.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.758+0000 7faf09eee980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:18:19.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.758+0000 7faf09eee980 20 rados->read ofs=0 len=0 2024-05-30T16:18:19.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.762+0000 7faf09eee980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:18:19.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.762+0000 7faf09eee980 20 rados->read ofs=0 len=0 2024-05-30T16:18:19.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.762+0000 7faf09eee980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:18:19.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.762+0000 7faf09eee980 20 rados->read ofs=0 len=0 2024-05-30T16:18:19.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.762+0000 7faf09eee980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:18:19.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.762+0000 7faf09eee980 20 zone default found 2024-05-30T16:18:19.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.762+0000 7faf09eee980 4 Realm: () 2024-05-30T16:18:19.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.762+0000 7faf09eee980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:18:19.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.762+0000 7faf09eee980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:18:19.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.762+0000 7faf09eee980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:18:19.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.762+0000 7faf09eee980 20 zonegroup default 2024-05-30T16:18:19.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.762+0000 7faf09eee980 20 rados->read ofs=0 len=0 2024-05-30T16:18:19.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.762+0000 7faf09eee980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:18:19.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.762+0000 7faf09eee980 20 rados->read ofs=0 len=0 2024-05-30T16:18:19.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.766+0000 7faf09eee980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:18:19.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.766+0000 7faf09eee980 20 rados->read ofs=0 len=0 2024-05-30T16:18:19.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.766+0000 7faf09eee980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:18:19.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.766+0000 7faf09eee980 20 started sync module instance, tier type = 2024-05-30T16:18:19.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.766+0000 7faf09eee980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:18:19.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.790+0000 7faf09eee980 20 add_watcher() i=0 2024-05-30T16:18:19.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.790+0000 7faf09eee980 20 add_watcher() i=1 2024-05-30T16:18:19.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.790+0000 7faf09eee980 20 add_watcher() i=2 2024-05-30T16:18:19.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.790+0000 7faf09eee980 20 add_watcher() i=3 2024-05-30T16:18:19.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.790+0000 7faf09eee980 20 add_watcher() i=4 2024-05-30T16:18:19.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.790+0000 7faf09eee980 20 add_watcher() i=5 2024-05-30T16:18:19.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.790+0000 7faf09eee980 20 add_watcher() i=6 2024-05-30T16:18:19.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.794+0000 7faf09eee980 20 add_watcher() i=7 2024-05-30T16:18:19.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.794+0000 7faf09eee980 2 all 8 watchers are set, enabling cache 2024-05-30T16:18:19.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.798+0000 7fadbdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:18:19.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.798+0000 7faf09eee980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:18:19.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.798+0000 7faf09eee980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:18:19.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.898+0000 7fadbcfe1640 20 reqs_thread_entry: start 2024-05-30T16:18:19.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.898+0000 7faf09eee980 20 init_complete bucket index max shards: 11 2024-05-30T16:18:19.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.898+0000 7faf09eee980 20 Filter name: none 2024-05-30T16:18:19.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.898+0000 7fad367fc640 20 reqs_thread_entry: start 2024-05-30T16:18:19.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.898+0000 7faf09eee980 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : miss 2024-05-30T16:18:19.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.898+0000 7faf09eee980 20 rados->read ofs=0 len=0 2024-05-30T16:18:19.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.898+0000 7faf09eee980 20 rados_obj.operate() r=0 bl.length=486 2024-05-30T16:18:19.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.898+0000 7faf09eee980 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x13 2024-05-30T16:18:19.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.898+0000 7faf09eee980 10 adding default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2024-05-30T16:18:19.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.898+0000 7faf09eee980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:18:19.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.898+0000 7faf09eee980 10 removing key index: SEQBSIIZTUZARSATLBUJ 2024-05-30T16:18:19.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.902+0000 7faf09eee980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:users.keys:SEQBSIIZTUZARSATLBUJ, ofs0, ns] 2024-05-30T16:18:19.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.902+0000 7fadeaffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431592 cookie 94550961680464 notifier 5675 bl.length()=170 2024-05-30T16:18:19.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.906+0000 7faf09eee980 10 removing email index: sysuser.client.0+test@test.test 2024-05-30T16:18:19.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.906+0000 7faf09eee980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.email:sysuser.client.0+test@test.test, ofs0, ns] 2024-05-30T16:18:19.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.906+0000 7fadeaffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431593 cookie 94550961687920 notifier 5675 bl.length()=182 2024-05-30T16:18:19.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.910+0000 7faf09eee980 10 removing user buckets index 2024-05-30T16:18:19.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.914+0000 7faf09eee980 10 removing user index: sysuser.client.0 2024-05-30T16:18:19.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.914+0000 7faf09eee980 10 removing default.rgw.meta+users.uid+sysuser.client.0 from cache 2024-05-30T16:18:19.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.914+0000 7faf09eee980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.uid:sysuser.client.0, ofs0, ns] 2024-05-30T16:18:19.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.914+0000 7fadeaffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431576 cookie 94550961686864 notifier 5675 bl.length()=165 2024-05-30T16:18:19.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.918+0000 7fadbdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:18:19.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.926+0000 7faf09eee980 20 remove_watcher() i=0 2024-05-30T16:18:19.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.926+0000 7faf09eee980 2 removed watcher, disabling cache 2024-05-30T16:18:19.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.930+0000 7faf09eee980 20 remove_watcher() i=1 2024-05-30T16:18:19.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.930+0000 7faf09eee980 20 remove_watcher() i=2 2024-05-30T16:18:19.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.934+0000 7faf09eee980 20 remove_watcher() i=3 2024-05-30T16:18:19.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.934+0000 7faf09eee980 20 remove_watcher() i=4 2024-05-30T16:18:19.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.938+0000 7faf09eee980 20 remove_watcher() i=5 2024-05-30T16:18:19.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.938+0000 7faf09eee980 20 remove_watcher() i=6 2024-05-30T16:18:19.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:18:19.942+0000 7faf09eee980 20 remove_watcher() i=7 2024-05-30T16:18:19.960 INFO:tasks.ragweed:Removing ragweed... 2024-05-30T16:18:19.960 DEBUG:teuthology.orchestra.run.smithi006:> rm -rf /home/ubuntu/cephtest/ragweed.client.0 2024-05-30T16:18:20.060 INFO:teuthology.run_tasks:Running task workunit... 2024-05-30T16:18:20.071 INFO:tasks.workunit:Pulling workunits from ref da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 2024-05-30T16:18:20.071 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-30T16:18:20.071 DEBUG:teuthology.orchestra.run.smithi006:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-30T16:18:20.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-30T16:18:20.077 INFO:teuthology.orchestra.run.smithi006.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-05-30T16:18:20.078 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-05-30T16:18:20.126 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-05-30T16:18:20.127 DEBUG:teuthology.orchestra.run.smithi006:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-05-30T16:18:20.177 INFO:tasks.workunit:timeout=3h 2024-05-30T16:18:20.177 INFO:tasks.workunit:cleanup=True 2024-05-30T16:18:20.178 DEBUG:teuthology.orchestra.run.smithi006:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 2024-05-30T16:18:20.228 INFO:tasks.workunit.client.0.smithi006.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-30T16:19:45.023 INFO:tasks.workunit.client.0.smithi006.stderr:Updating files: 99% (12247/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-30T16:19:45.374 INFO:tasks.workunit.client.0.smithi006.stderr:Note: switching to 'da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2'. 2024-05-30T16:19:45.374 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:19:45.375 INFO:tasks.workunit.client.0.smithi006.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-30T16:19:45.375 INFO:tasks.workunit.client.0.smithi006.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-30T16:19:45.375 INFO:tasks.workunit.client.0.smithi006.stderr:state without impacting any branches by switching back to a branch. 2024-05-30T16:19:45.375 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:19:45.375 INFO:tasks.workunit.client.0.smithi006.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-30T16:19:45.375 INFO:tasks.workunit.client.0.smithi006.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-30T16:19:45.375 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:19:45.375 INFO:tasks.workunit.client.0.smithi006.stderr: git switch -c 2024-05-30T16:19:45.376 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:19:45.376 INFO:tasks.workunit.client.0.smithi006.stderr:Or undo this operation with: 2024-05-30T16:19:45.376 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:19:45.376 INFO:tasks.workunit.client.0.smithi006.stderr: git switch - 2024-05-30T16:19:45.376 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:19:45.376 INFO:tasks.workunit.client.0.smithi006.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-30T16:19:45.376 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:19:45.376 INFO:tasks.workunit.client.0.smithi006.stderr:HEAD is now at da98024b78b Merge branches 'pr-57079', 'pr-57536', 'pr-57641' and 'pr-57702' into wip-cbodley-testing 2024-05-30T16:19:45.383 DEBUG:teuthology.orchestra.run.smithi006:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-30T16:19:45.432 INFO:tasks.workunit.client.0.smithi006.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-30T16:19:45.436 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-30T16:19:45.436 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-30T16:19:45.492 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-30T16:19:45.550 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-30T16:19:45.596 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-30T16:19:45.597 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-30T16:19:45.598 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-30T16:19:45.643 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-30T16:19:45.647 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T16:19:45.647 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-30T16:19:45.695 INFO:tasks.workunit:Running workunits matching rgw/run-reshard.sh on client.0... 2024-05-30T16:19:45.696 INFO:tasks.workunit:Running workunit rgw/run-reshard.sh... 2024-05-30T16:19:45.696 DEBUG:teuthology.orchestra.run.smithi006:workunit test rgw/run-reshard.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-reshard.sh 2024-05-30T16:19:45.746 INFO:tasks.workunit.client.0.smithi006.stderr:+ ulimit -c 0 2024-05-30T16:19:45.747 INFO:tasks.workunit.client.0.smithi006.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-reshard.sh 2024-05-30T16:19:45.748 INFO:tasks.workunit.client.0.smithi006.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-05-30T16:19:45.748 INFO:tasks.workunit.client.0.smithi006.stderr:+ python3 -m venv /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-05-30T16:19:48.737 INFO:tasks.workunit.client.0.smithi006.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/activate 2024-05-30T16:19:48.737 INFO:tasks.workunit.client.0.smithi006.stderr:++ deactivate nondestructive 2024-05-30T16:19:48.737 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -n '' ']' 2024-05-30T16:19:48.737 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -n '' ']' 2024-05-30T16:19:48.737 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-05-30T16:19:48.737 INFO:tasks.workunit.client.0.smithi006.stderr:++ hash -r 2024-05-30T16:19:48.737 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -n '' ']' 2024-05-30T16:19:48.737 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset VIRTUAL_ENV 2024-05-30T16:19:48.737 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset VIRTUAL_ENV_PROMPT 2024-05-30T16:19:48.737 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '!' nondestructive = nondestructive ']' 2024-05-30T16:19:48.738 INFO:tasks.workunit.client.0.smithi006.stderr:++ VIRTUAL_ENV=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-05-30T16:19:48.738 INFO:tasks.workunit.client.0.smithi006.stderr:++ export VIRTUAL_ENV 2024-05-30T16:19:48.738 INFO:tasks.workunit.client.0.smithi006.stderr:++ _OLD_VIRTUAL_PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-05-30T16:19:48.738 INFO:tasks.workunit.client.0.smithi006.stderr:++ PATH=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-05-30T16:19:48.738 INFO:tasks.workunit.client.0.smithi006.stderr:++ export PATH 2024-05-30T16:19:48.738 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -n '' ']' 2024-05-30T16:19:48.738 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-05-30T16:19:48.738 INFO:tasks.workunit.client.0.smithi006.stderr:++ _OLD_VIRTUAL_PS1= 2024-05-30T16:19:48.738 INFO:tasks.workunit.client.0.smithi006.stderr:++ PS1='(rgw) ' 2024-05-30T16:19:48.738 INFO:tasks.workunit.client.0.smithi006.stderr:++ export PS1 2024-05-30T16:19:48.738 INFO:tasks.workunit.client.0.smithi006.stderr:++ VIRTUAL_ENV_PROMPT='(rgw) ' 2024-05-30T16:19:48.738 INFO:tasks.workunit.client.0.smithi006.stderr:++ export VIRTUAL_ENV_PROMPT 2024-05-30T16:19:48.738 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-05-30T16:19:48.739 INFO:tasks.workunit.client.0.smithi006.stderr:++ hash -r 2024-05-30T16:19:48.739 INFO:tasks.workunit.client.0.smithi006.stderr:+ pip install pip --upgrade 2024-05-30T16:19:49.144 INFO:tasks.workunit.client.0.smithi006.stdout:Requirement already satisfied: pip in /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages (22.0.2) 2024-05-30T16:19:49.358 INFO:tasks.workunit.client.0.smithi006.stdout:Collecting pip 2024-05-30T16:19:49.365 INFO:tasks.workunit.client.0.smithi006.stdout: Using cached pip-24.0-py3-none-any.whl (2.1 MB) 2024-05-30T16:19:49.404 INFO:tasks.workunit.client.0.smithi006.stdout:Installing collected packages: pip 2024-05-30T16:19:49.405 INFO:tasks.workunit.client.0.smithi006.stdout: Attempting uninstall: pip 2024-05-30T16:19:49.405 INFO:tasks.workunit.client.0.smithi006.stdout: Found existing installation: pip 22.0.2 2024-05-30T16:19:49.691 INFO:tasks.workunit.client.0.smithi006.stdout: Uninstalling pip-22.0.2: 2024-05-30T16:19:49.698 INFO:tasks.workunit.client.0.smithi006.stdout: Successfully uninstalled pip-22.0.2 2024-05-30T16:19:50.671 INFO:tasks.workunit.client.0.smithi006.stdout:Successfully installed pip-24.0 2024-05-30T16:19:50.732 INFO:tasks.workunit.client.0.smithi006.stderr:+ pip install boto3 2024-05-30T16:19:51.707 INFO:tasks.workunit.client.0.smithi006.stdout:Collecting boto3 2024-05-30T16:19:51.708 INFO:tasks.workunit.client.0.smithi006.stdout: Using cached boto3-1.34.115-py3-none-any.whl.metadata (6.6 kB) 2024-05-30T16:19:52.299 INFO:tasks.workunit.client.0.smithi006.stdout:Collecting botocore<1.35.0,>=1.34.115 (from boto3) 2024-05-30T16:19:52.300 INFO:tasks.workunit.client.0.smithi006.stdout: Using cached botocore-1.34.115-py3-none-any.whl.metadata (5.7 kB) 2024-05-30T16:19:52.358 INFO:tasks.workunit.client.0.smithi006.stdout:Collecting jmespath<2.0.0,>=0.7.1 (from boto3) 2024-05-30T16:19:52.359 INFO:tasks.workunit.client.0.smithi006.stdout: Using cached jmespath-1.0.1-py3-none-any.whl.metadata (7.6 kB) 2024-05-30T16:19:52.424 INFO:tasks.workunit.client.0.smithi006.stdout:Collecting s3transfer<0.11.0,>=0.10.0 (from boto3) 2024-05-30T16:19:52.426 INFO:tasks.workunit.client.0.smithi006.stdout: Using cached s3transfer-0.10.1-py3-none-any.whl.metadata (1.7 kB) 2024-05-30T16:19:52.514 INFO:tasks.workunit.client.0.smithi006.stdout:Collecting python-dateutil<3.0.0,>=2.1 (from botocore<1.35.0,>=1.34.115->boto3) 2024-05-30T16:19:52.516 INFO:tasks.workunit.client.0.smithi006.stdout: Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB) 2024-05-30T16:19:52.654 INFO:tasks.workunit.client.0.smithi006.stdout:Collecting urllib3!=2.2.0,<3,>=1.25.4 (from botocore<1.35.0,>=1.34.115->boto3) 2024-05-30T16:19:52.656 INFO:tasks.workunit.client.0.smithi006.stdout: Using cached urllib3-2.2.1-py3-none-any.whl.metadata (6.4 kB) 2024-05-30T16:19:52.944 INFO:tasks.workunit.client.0.smithi006.stdout:Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore<1.35.0,>=1.34.115->boto3) 2024-05-30T16:19:52.945 INFO:tasks.workunit.client.0.smithi006.stdout: Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB) 2024-05-30T16:19:52.957 INFO:tasks.workunit.client.0.smithi006.stdout:Using cached boto3-1.34.115-py3-none-any.whl (139 kB) 2024-05-30T16:19:52.958 INFO:tasks.workunit.client.0.smithi006.stdout:Using cached botocore-1.34.115-py3-none-any.whl (12.3 MB) 2024-05-30T16:19:52.979 INFO:tasks.workunit.client.0.smithi006.stdout:Using cached jmespath-1.0.1-py3-none-any.whl (20 kB) 2024-05-30T16:19:52.980 INFO:tasks.workunit.client.0.smithi006.stdout:Using cached s3transfer-0.10.1-py3-none-any.whl (82 kB) 2024-05-30T16:19:52.982 INFO:tasks.workunit.client.0.smithi006.stdout:Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) 2024-05-30T16:19:52.983 INFO:tasks.workunit.client.0.smithi006.stdout:Using cached urllib3-2.2.1-py3-none-any.whl (121 kB) 2024-05-30T16:19:52.985 INFO:tasks.workunit.client.0.smithi006.stdout:Using cached six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-05-30T16:19:53.082 INFO:tasks.workunit.client.0.smithi006.stdout:Installing collected packages: urllib3, six, jmespath, python-dateutil, botocore, s3transfer, boto3 2024-05-30T16:19:54.213 INFO:tasks.workunit.client.0.smithi006.stdout:Successfully installed boto3-1.34.115 botocore-1.34.115 jmespath-1.0.1 python-dateutil-2.9.0.post0 s3transfer-0.10.1 six-1.16.0 urllib3-2.2.1 2024-05-30T16:19:54.386 INFO:tasks.workunit.client.0.smithi006.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/python3 /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_reshard.py 2024-05-30T16:19:54.519 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin user create --uid tester --display-name "Testing" --access-key NX5QOQKC6BH2IDN8HC7A --secret LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn 2024-05-30T16:19:54.562 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:19:54.562 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:19:54.604 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.601+0000 7f6303639980 20 rados->read ofs=0 len=0 2024-05-30T16:19:54.610 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.605+0000 7f6303639980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:54.610 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.605+0000 7f6303639980 20 realm 2024-05-30T16:19:54.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.605+0000 7f6303639980 20 rados->read ofs=0 len=0 2024-05-30T16:19:54.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.605+0000 7f6303639980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:54.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.605+0000 7f6303639980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:19:54.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.605+0000 7f6303639980 20 rados->read ofs=0 len=0 2024-05-30T16:19:54.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.605+0000 7f6303639980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:54.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.605+0000 7f6303639980 20 rados->read ofs=0 len=0 2024-05-30T16:19:54.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.605+0000 7f6303639980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:19:54.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.605+0000 7f6303639980 20 rados->read ofs=0 len=0 2024-05-30T16:19:54.612 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.605+0000 7f6303639980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:19:54.612 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.605+0000 7f6303639980 20 searching for the correct realm 2024-05-30T16:19:54.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.613+0000 7f630194b640 0 -- 172.21.15.6:0/3688161379 >> [v2:172.21.15.6:6808/1897362715,v1:172.21.15.6:6809/1897362715] conn(0x55d889d21aa0 msgr2=0x55d889d23e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:19:54.625 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.621+0000 7f6300949640 0 -- 172.21.15.6:0/3688161379 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f62f40c5040 msgr2=0x7f62f40e2940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:19:54.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.625+0000 7f6303639980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:19:54.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.625+0000 7f6303639980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:19:54.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.625+0000 7f6303639980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:19:54.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.625+0000 7f6303639980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:19:54.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.625+0000 7f6303639980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:19:54.630 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.625+0000 7f6303639980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:19:54.630 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.625+0000 7f6303639980 20 rados->read ofs=0 len=0 2024-05-30T16:19:54.631 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.629+0000 7f6303639980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:54.631 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.629+0000 7f6303639980 20 rados->read ofs=0 len=0 2024-05-30T16:19:54.632 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.629+0000 7f6303639980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:19:54.632 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.629+0000 7f6303639980 20 rados->read ofs=0 len=0 2024-05-30T16:19:54.633 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.629+0000 7f6303639980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:19:54.633 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.629+0000 7f6303639980 20 zone default found 2024-05-30T16:19:54.633 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.629+0000 7f6303639980 4 Realm: () 2024-05-30T16:19:54.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.629+0000 7f6303639980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:19:54.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.629+0000 7f6303639980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:19:54.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.629+0000 7f6303639980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:19:54.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.629+0000 7f6303639980 20 zonegroup default 2024-05-30T16:19:54.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.629+0000 7f6303639980 20 rados->read ofs=0 len=0 2024-05-30T16:19:54.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.629+0000 7f6303639980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:54.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.629+0000 7f6303639980 20 rados->read ofs=0 len=0 2024-05-30T16:19:54.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.629+0000 7f6303639980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:54.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.629+0000 7f6303639980 20 rados->read ofs=0 len=0 2024-05-30T16:19:54.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.629+0000 7f6303639980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:54.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.629+0000 7f6303639980 20 started sync module instance, tier type = 2024-05-30T16:19:54.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.629+0000 7f6303639980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:19:54.655 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.645+0000 7f6303639980 20 add_watcher() i=0 2024-05-30T16:19:54.655 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.645+0000 7f6303639980 20 add_watcher() i=1 2024-05-30T16:19:54.655 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.645+0000 7f6303639980 20 add_watcher() i=2 2024-05-30T16:19:54.655 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.645+0000 7f6303639980 20 add_watcher() i=3 2024-05-30T16:19:54.655 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.645+0000 7f6303639980 20 add_watcher() i=4 2024-05-30T16:19:54.656 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.645+0000 7f6303639980 20 add_watcher() i=5 2024-05-30T16:19:54.656 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.645+0000 7f6303639980 20 add_watcher() i=6 2024-05-30T16:19:54.656 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.645+0000 7f6303639980 20 add_watcher() i=7 2024-05-30T16:19:54.656 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.645+0000 7f6303639980 2 all 8 watchers are set, enabling cache 2024-05-30T16:19:54.657 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.649+0000 7f61b7fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:19:54.657 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.649+0000 7f6303639980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:19:54.657 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.649+0000 7f6303639980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:19:54.657 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.649+0000 7f6303639980 5 note: GC not initialized 2024-05-30T16:19:54.657 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.649+0000 7f61b6fe5640 20 reqs_thread_entry: start 2024-05-30T16:19:54.657 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.649+0000 7f6303639980 20 init_complete bucket index max shards: 11 2024-05-30T16:19:54.658 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.649+0000 7f6303639980 20 Filter name: none 2024-05-30T16:19:54.658 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.649+0000 7f6303639980 10 cache get: name=default.rgw.meta+users.uid+tester : miss 2024-05-30T16:19:54.658 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.649+0000 7f61b4fe1640 20 reqs_thread_entry: start 2024-05-30T16:19:54.658 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.649+0000 7f6303639980 20 rados->read ofs=0 len=0 2024-05-30T16:19:54.658 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.653+0000 7f6303639980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:54.658 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.653+0000 7f6303639980 10 cache put: name=default.rgw.meta+users.uid+tester info.flags=0x0 2024-05-30T16:19:54.658 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.653+0000 7f6303639980 10 adding default.rgw.meta+users.uid+tester to cache LRU end 2024-05-30T16:19:54.658 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.653+0000 7f6303639980 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : miss 2024-05-30T16:19:54.658 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.653+0000 7f6303639980 20 rados->read ofs=0 len=0 2024-05-30T16:19:54.658 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.653+0000 7f6303639980 20 rados_obj.operate() r=0 bl.length=22 2024-05-30T16:19:54.658 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.653+0000 7f6303639980 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x5 2024-05-30T16:19:54.659 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.653+0000 7f6303639980 10 adding default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-05-30T16:19:54.659 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.653+0000 7f6303639980 10 cache get: name=default.rgw.meta+users.uid+rgw_datacache_user : miss 2024-05-30T16:19:54.659 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.653+0000 7f6303639980 20 rados->read ofs=0 len=0 2024-05-30T16:19:54.659 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.653+0000 7f6303639980 20 rados_obj.operate() r=0 bl.length=420 2024-05-30T16:19:54.659 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.653+0000 7f6303639980 10 cache put: name=default.rgw.meta+users.uid+rgw_datacache_user info.flags=0x13 2024-05-30T16:19:54.660 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.653+0000 7f6303639980 10 adding default.rgw.meta+users.uid+rgw_datacache_user to cache LRU end 2024-05-30T16:19:54.660 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.653+0000 7f6303639980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:19:54.660 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.653+0000 7f6303639980 10 chain_cache_entry: cache_locator=default.rgw.meta+users.uid+rgw_datacache_user 2024-05-30T16:19:54.660 INFO:tasks.workunit.client.0.smithi006.stderr:could not create user: 2024-05-30T16:19:54.653+0000 7f6303639980 20 get_user_info_from_index found 0 xattrs 2024-05-30T16:19:54.660 INFO:tasks.workunit.client.0.smithi006.stderr:unable to parse parameters, duplicate key provided 2024-05-30T16:19:54.660 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.653+0000 7f61b7fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:19:54.661 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.657+0000 7f6303639980 20 remove_watcher() i=0 2024-05-30T16:19:54.661 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.657+0000 7f6303639980 2 removed watcher, disabling cache 2024-05-30T16:19:54.663 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.661+0000 7f6303639980 20 remove_watcher() i=1 2024-05-30T16:19:54.665 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.661+0000 7f6303639980 20 remove_watcher() i=2 2024-05-30T16:19:54.667 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.665+0000 7f6303639980 20 remove_watcher() i=3 2024-05-30T16:19:54.669 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.669+0000 7f6303639980 20 remove_watcher() i=4 2024-05-30T16:19:54.671 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.669+0000 7f6303639980 20 remove_watcher() i=5 2024-05-30T16:19:54.672 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.669+0000 7f6303639980 20 remove_watcher() i=6 2024-05-30T16:19:54.673 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:54.669+0000 7f6303639980 20 remove_watcher() i=7 2024-05-30T16:19:55.378 INFO:tasks.workunit.client.0.smithi006.stderr:TEST: reshard add 2024-05-30T16:19:55.379 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:19:55.379 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-05-30T16:19:55.414 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:19:55.415 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:19:55.461 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.457+0000 7effb13c8980 20 rados->read ofs=0 len=0 2024-05-30T16:19:55.462 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.461+0000 7effb13c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:55.462 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.461+0000 7effb13c8980 20 realm 2024-05-30T16:19:55.462 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.461+0000 7effb13c8980 20 rados->read ofs=0 len=0 2024-05-30T16:19:55.463 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.461+0000 7effb13c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:55.463 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.461+0000 7effb13c8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:19:55.463 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.461+0000 7effb13c8980 20 rados->read ofs=0 len=0 2024-05-30T16:19:55.463 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.461+0000 7effb13c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:55.463 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.461+0000 7effb13c8980 20 rados->read ofs=0 len=0 2024-05-30T16:19:55.464 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.461+0000 7effb13c8980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:19:55.464 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.461+0000 7effb13c8980 20 rados->read ofs=0 len=0 2024-05-30T16:19:55.465 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.461+0000 7effb13c8980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:19:55.465 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.461+0000 7effb13c8980 20 searching for the correct realm 2024-05-30T16:19:55.471 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.469+0000 7effaf6da640 0 -- 172.21.15.6:0/2759405164 >> [v2:172.21.15.6:6808/1897362715,v1:172.21.15.6:6809/1897362715] conn(0x563b53cc7620 msgr2=0x563b53cc9a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:19:55.478 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.477+0000 7effae6d8640 0 -- 172.21.15.6:0/2759405164 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7effa0069ad0 msgr2=0x7effa0011450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:19:55.485 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.481+0000 7effb13c8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:19:55.485 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.481+0000 7effb13c8980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:19:55.486 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.481+0000 7effb13c8980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:19:55.486 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.481+0000 7effb13c8980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:19:55.486 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.481+0000 7effb13c8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:19:55.486 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.481+0000 7effb13c8980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:19:55.486 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.481+0000 7effb13c8980 20 rados->read ofs=0 len=0 2024-05-30T16:19:55.486 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.485+0000 7effb13c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:55.486 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.485+0000 7effb13c8980 20 rados->read ofs=0 len=0 2024-05-30T16:19:55.487 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.485+0000 7effb13c8980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:19:55.487 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.485+0000 7effb13c8980 20 rados->read ofs=0 len=0 2024-05-30T16:19:55.487 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.485+0000 7effb13c8980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:19:55.487 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.485+0000 7effb13c8980 20 zone default found 2024-05-30T16:19:55.487 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.485+0000 7effb13c8980 4 Realm: () 2024-05-30T16:19:55.487 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.485+0000 7effb13c8980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:19:55.487 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.485+0000 7effb13c8980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:19:55.487 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.485+0000 7effb13c8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:19:55.488 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.485+0000 7effb13c8980 20 zonegroup default 2024-05-30T16:19:55.488 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.485+0000 7effb13c8980 20 rados->read ofs=0 len=0 2024-05-30T16:19:55.488 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.485+0000 7effb13c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:55.488 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.485+0000 7effb13c8980 20 rados->read ofs=0 len=0 2024-05-30T16:19:55.488 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.485+0000 7effb13c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:55.488 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.485+0000 7effb13c8980 20 rados->read ofs=0 len=0 2024-05-30T16:19:55.488 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.485+0000 7effb13c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:55.488 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.485+0000 7effb13c8980 20 started sync module instance, tier type = 2024-05-30T16:19:55.489 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.485+0000 7effb13c8980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:19:55.504 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.501+0000 7effb13c8980 20 add_watcher() i=0 2024-05-30T16:19:55.504 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.501+0000 7effb13c8980 20 add_watcher() i=1 2024-05-30T16:19:55.504 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.501+0000 7effb13c8980 20 add_watcher() i=2 2024-05-30T16:19:55.504 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.501+0000 7effb13c8980 20 add_watcher() i=3 2024-05-30T16:19:55.504 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.501+0000 7effb13c8980 20 add_watcher() i=4 2024-05-30T16:19:55.504 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.501+0000 7effb13c8980 20 add_watcher() i=5 2024-05-30T16:19:55.504 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.501+0000 7effb13c8980 20 add_watcher() i=6 2024-05-30T16:19:55.505 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.501+0000 7effb13c8980 20 add_watcher() i=7 2024-05-30T16:19:55.505 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.501+0000 7effb13c8980 2 all 8 watchers are set, enabling cache 2024-05-30T16:19:55.510 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.509+0000 7efe657e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:19:55.511 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.509+0000 7effb13c8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:19:55.511 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.509+0000 7effb13c8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:19:55.511 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.509+0000 7effb13c8980 5 note: GC not initialized 2024-05-30T16:19:55.511 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.509+0000 7efddffff640 20 reqs_thread_entry: start 2024-05-30T16:19:55.511 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.509+0000 7effb13c8980 20 init_complete bucket index max shards: 11 2024-05-30T16:19:55.512 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.509+0000 7effb13c8980 20 Filter name: none 2024-05-30T16:19:55.512 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.509+0000 7efdddffb640 20 reqs_thread_entry: start 2024-05-30T16:19:55.512 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.509+0000 7effb13c8980 20 rados->read ofs=0 len=0 2024-05-30T16:19:55.513 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.509+0000 7effb13c8980 20 rados_obj.operate() r=0 bl.length=200 2024-05-30T16:19:55.513 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.509+0000 7effb13c8980 20 rados->read ofs=0 len=0 2024-05-30T16:19:55.513 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.513+0000 7effb13c8980 20 rados_obj.operate() r=0 bl.length=414 2024-05-30T16:19:55.514 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.513+0000 7effb13c8980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:19:55.514 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.513+0000 7effb13c8980 20 rados->read ofs=0 len=0 2024-05-30T16:19:55.515 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.513+0000 7effb13c8980 20 rados_obj.operate() r=0 bl.length=200 2024-05-30T16:19:55.515 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.513+0000 7effb13c8980 20 rados->read ofs=0 len=0 2024-05-30T16:19:55.515 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.513+0000 7effb13c8980 20 rados_obj.operate() r=0 bl.length=414 2024-05-30T16:19:55.515 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.513+0000 7effb13c8980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:19:55.518 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.517+0000 7efe657e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:19:55.523 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.521+0000 7effb13c8980 20 remove_watcher() i=0 2024-05-30T16:19:55.523 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.521+0000 7effb13c8980 2 removed watcher, disabling cache 2024-05-30T16:19:55.525 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.525+0000 7effb13c8980 20 remove_watcher() i=1 2024-05-30T16:19:55.527 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.525+0000 7effb13c8980 20 remove_watcher() i=2 2024-05-30T16:19:55.529 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.529+0000 7effb13c8980 20 remove_watcher() i=3 2024-05-30T16:19:55.531 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.529+0000 7effb13c8980 20 remove_watcher() i=4 2024-05-30T16:19:55.533 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.529+0000 7effb13c8980 20 remove_watcher() i=5 2024-05-30T16:19:55.534 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.533+0000 7effb13c8980 20 remove_watcher() i=6 2024-05-30T16:19:55.535 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.533+0000 7effb13c8980 20 remove_watcher() i=7 2024-05-30T16:19:55.550 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:19:55.550 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:19:55.550 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: a-bucket id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 num_objects: 0 size_kb: 0 num_shards: 11 2024-05-30T16:19:55.550 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:19:55.551 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin reshard add --bucket a-bucket --num-shards 12 2024-05-30T16:19:55.580 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:19:55.580 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:19:55.627 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.625+0000 7ff48251d980 20 rados->read ofs=0 len=0 2024-05-30T16:19:55.628 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.625+0000 7ff48251d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:55.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.625+0000 7ff48251d980 20 realm 2024-05-30T16:19:55.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.625+0000 7ff48251d980 20 rados->read ofs=0 len=0 2024-05-30T16:19:55.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.625+0000 7ff48251d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:55.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.625+0000 7ff48251d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:19:55.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.625+0000 7ff48251d980 20 rados->read ofs=0 len=0 2024-05-30T16:19:55.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.625+0000 7ff48251d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:55.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.625+0000 7ff48251d980 20 rados->read ofs=0 len=0 2024-05-30T16:19:55.631 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.629+0000 7ff48251d980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:19:55.631 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.629+0000 7ff48251d980 20 rados->read ofs=0 len=0 2024-05-30T16:19:55.631 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.629+0000 7ff48251d980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:19:55.632 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.629+0000 7ff48251d980 20 searching for the correct realm 2024-05-30T16:19:55.636 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.633+0000 7ff47f82d640 0 -- 172.21.15.6:0/669547060 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7ff47003c070 msgr2=0x7ff470019410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:19:55.645 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.641+0000 7ff48002e640 0 -- 172.21.15.6:0/669547060 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7ff4740811a0 msgr2=0x7ff474060530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:19:55.860 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.857+0000 7ff48251d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:19:55.861 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.857+0000 7ff48251d980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:19:55.861 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.857+0000 7ff48251d980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:19:55.861 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.857+0000 7ff48251d980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:19:55.861 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.857+0000 7ff48251d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:19:55.861 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.857+0000 7ff48251d980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:19:55.861 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.857+0000 7ff48251d980 20 rados->read ofs=0 len=0 2024-05-30T16:19:55.861 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.857+0000 7ff48251d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:55.861 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.857+0000 7ff48251d980 20 rados->read ofs=0 len=0 2024-05-30T16:19:55.862 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.861+0000 7ff48251d980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:19:55.862 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.861+0000 7ff48251d980 20 rados->read ofs=0 len=0 2024-05-30T16:19:55.863 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.861+0000 7ff48251d980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:19:55.863 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.861+0000 7ff48251d980 20 zone default found 2024-05-30T16:19:55.863 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.861+0000 7ff48251d980 4 Realm: () 2024-05-30T16:19:55.863 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.861+0000 7ff48251d980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:19:55.864 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.861+0000 7ff48251d980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:19:55.864 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.861+0000 7ff48251d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:19:55.864 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.861+0000 7ff48251d980 20 zonegroup default 2024-05-30T16:19:55.864 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.861+0000 7ff48251d980 20 rados->read ofs=0 len=0 2024-05-30T16:19:55.864 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.861+0000 7ff48251d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:55.864 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.861+0000 7ff48251d980 20 rados->read ofs=0 len=0 2024-05-30T16:19:55.865 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.861+0000 7ff48251d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:55.865 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.861+0000 7ff48251d980 20 rados->read ofs=0 len=0 2024-05-30T16:19:55.866 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.865+0000 7ff48251d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:55.866 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.865+0000 7ff48251d980 20 started sync module instance, tier type = 2024-05-30T16:19:55.866 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.865+0000 7ff48251d980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:19:55.890 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.889+0000 7ff48251d980 20 add_watcher() i=0 2024-05-30T16:19:55.890 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.889+0000 7ff48251d980 20 add_watcher() i=1 2024-05-30T16:19:55.890 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.889+0000 7ff48251d980 20 add_watcher() i=2 2024-05-30T16:19:55.890 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.889+0000 7ff48251d980 20 add_watcher() i=3 2024-05-30T16:19:55.891 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.889+0000 7ff48251d980 20 add_watcher() i=4 2024-05-30T16:19:55.891 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.889+0000 7ff48251d980 20 add_watcher() i=5 2024-05-30T16:19:55.891 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.889+0000 7ff48251d980 20 add_watcher() i=6 2024-05-30T16:19:55.892 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.889+0000 7ff48251d980 20 add_watcher() i=7 2024-05-30T16:19:55.892 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.889+0000 7ff48251d980 2 all 8 watchers are set, enabling cache 2024-05-30T16:19:55.897 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.893+0000 7ff3367e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:19:55.898 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.897+0000 7ff48251d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:19:55.898 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.897+0000 7ff48251d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:19:55.898 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.897+0000 7ff48251d980 5 note: GC not initialized 2024-05-30T16:19:55.898 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.897+0000 7ff3357e2640 20 reqs_thread_entry: start 2024-05-30T16:19:55.898 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.897+0000 7ff48251d980 20 init_complete bucket index max shards: 11 2024-05-30T16:19:55.899 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.897+0000 7ff48251d980 20 Filter name: none 2024-05-30T16:19:55.899 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.897+0000 7ff2a6ffd640 20 reqs_thread_entry: start 2024-05-30T16:19:55.899 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.897+0000 7ff48251d980 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-05-30T16:19:55.900 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.897+0000 7ff48251d980 20 rados->read ofs=0 len=0 2024-05-30T16:19:55.900 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.897+0000 7ff48251d980 20 rados_obj.operate() r=0 bl.length=200 2024-05-30T16:19:55.900 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.897+0000 7ff48251d980 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2024-05-30T16:19:55.901 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.897+0000 7ff48251d980 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-05-30T16:19:55.901 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.897+0000 7ff48251d980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 : miss 2024-05-30T16:19:55.901 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.897+0000 7ff48251d980 20 rados->read ofs=0 len=0 2024-05-30T16:19:55.902 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.901+0000 7ff48251d980 20 rados_obj.operate() r=0 bl.length=414 2024-05-30T16:19:55.902 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.901+0000 7ff48251d980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 info.flags=0x17 2024-05-30T16:19:55.902 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.901+0000 7ff48251d980 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 to cache LRU end 2024-05-30T16:19:55.902 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.901+0000 7ff48251d980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:19:55.902 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.901+0000 7ff48251d980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:19:55.902 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.901+0000 7ff48251d980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 2024-05-30T16:19:55.905 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.905+0000 7ff3367e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:19:55.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.909+0000 7ff48251d980 20 remove_watcher() i=0 2024-05-30T16:19:55.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.909+0000 7ff48251d980 2 removed watcher, disabling cache 2024-05-30T16:19:55.914 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.913+0000 7ff48251d980 20 remove_watcher() i=1 2024-05-30T16:19:55.916 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.913+0000 7ff48251d980 20 remove_watcher() i=2 2024-05-30T16:19:55.918 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.917+0000 7ff48251d980 20 remove_watcher() i=3 2024-05-30T16:19:55.920 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.917+0000 7ff48251d980 20 remove_watcher() i=4 2024-05-30T16:19:55.923 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.921+0000 7ff48251d980 20 remove_watcher() i=5 2024-05-30T16:19:55.924 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.921+0000 7ff48251d980 20 remove_watcher() i=6 2024-05-30T16:19:55.926 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:55.925+0000 7ff48251d980 20 remove_watcher() i=7 2024-05-30T16:19:55.943 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin reshard list 2024-05-30T16:19:55.972 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:19:55.972 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:19:56.017 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.017+0000 7f0e8c345980 20 rados->read ofs=0 len=0 2024-05-30T16:19:56.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.017+0000 7f0e8c345980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:56.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.017+0000 7f0e8c345980 20 realm 2024-05-30T16:19:56.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.017+0000 7f0e8c345980 20 rados->read ofs=0 len=0 2024-05-30T16:19:56.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.017+0000 7f0e8c345980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:56.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.017+0000 7f0e8c345980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:19:56.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.017+0000 7f0e8c345980 20 rados->read ofs=0 len=0 2024-05-30T16:19:56.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.017+0000 7f0e8c345980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:56.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.017+0000 7f0e8c345980 20 rados->read ofs=0 len=0 2024-05-30T16:19:56.022 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.021+0000 7f0e8c345980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:19:56.022 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.021+0000 7f0e8c345980 20 rados->read ofs=0 len=0 2024-05-30T16:19:56.023 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.021+0000 7f0e8c345980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:19:56.023 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.021+0000 7f0e8c345980 20 searching for the correct realm 2024-05-30T16:19:56.029 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.025+0000 7f0e89655640 0 -- 172.21.15.6:0/3662433101 >> [v2:172.21.15.6:6819/1141406589,v1:172.21.15.6:6821/1141406589] conn(0x55f095e8ecb0 msgr2=0x55f095e910a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:19:56.035 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.033+0000 7f0e89e56640 0 -- 172.21.15.6:0/3662433101 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f0e80118610 msgr2=0x7f0e800ad9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:19:56.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.041+0000 7f0e8c345980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:19:56.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.041+0000 7f0e8c345980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:19:56.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.041+0000 7f0e8c345980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:19:56.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.041+0000 7f0e8c345980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:19:56.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.041+0000 7f0e8c345980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:19:56.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.041+0000 7f0e8c345980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:19:56.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.041+0000 7f0e8c345980 20 rados->read ofs=0 len=0 2024-05-30T16:19:56.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.041+0000 7f0e8c345980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:56.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.041+0000 7f0e8c345980 20 rados->read ofs=0 len=0 2024-05-30T16:19:56.045 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.041+0000 7f0e8c345980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:19:56.045 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.041+0000 7f0e8c345980 20 rados->read ofs=0 len=0 2024-05-30T16:19:56.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.045+0000 7f0e8c345980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:19:56.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.045+0000 7f0e8c345980 20 zone default found 2024-05-30T16:19:56.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.045+0000 7f0e8c345980 4 Realm: () 2024-05-30T16:19:56.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.045+0000 7f0e8c345980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:19:56.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.045+0000 7f0e8c345980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:19:56.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.045+0000 7f0e8c345980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:19:56.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.045+0000 7f0e8c345980 20 zonegroup default 2024-05-30T16:19:56.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.045+0000 7f0e8c345980 20 rados->read ofs=0 len=0 2024-05-30T16:19:56.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.045+0000 7f0e8c345980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:56.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.045+0000 7f0e8c345980 20 rados->read ofs=0 len=0 2024-05-30T16:19:56.047 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.045+0000 7f0e8c345980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:56.047 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.045+0000 7f0e8c345980 20 rados->read ofs=0 len=0 2024-05-30T16:19:56.047 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.045+0000 7f0e8c345980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:56.047 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.045+0000 7f0e8c345980 20 started sync module instance, tier type = 2024-05-30T16:19:56.047 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.045+0000 7f0e8c345980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:19:56.063 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.061+0000 7f0e8c345980 20 add_watcher() i=0 2024-05-30T16:19:56.063 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.061+0000 7f0e8c345980 20 add_watcher() i=1 2024-05-30T16:19:56.063 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.061+0000 7f0e8c345980 20 add_watcher() i=2 2024-05-30T16:19:56.063 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.061+0000 7f0e8c345980 20 add_watcher() i=3 2024-05-30T16:19:56.063 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.061+0000 7f0e8c345980 20 add_watcher() i=4 2024-05-30T16:19:56.063 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.061+0000 7f0e8c345980 20 add_watcher() i=5 2024-05-30T16:19:56.063 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.061+0000 7f0e8c345980 20 add_watcher() i=6 2024-05-30T16:19:56.064 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.061+0000 7f0e8c345980 20 add_watcher() i=7 2024-05-30T16:19:56.064 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.061+0000 7f0e8c345980 2 all 8 watchers are set, enabling cache 2024-05-30T16:19:56.067 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.065+0000 7f0d3b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:19:56.068 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.065+0000 7f0e8c345980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:19:56.068 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.065+0000 7f0e8c345980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:19:56.068 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.065+0000 7f0e8c345980 5 note: GC not initialized 2024-05-30T16:19:56.068 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.065+0000 7f0d3a7fc640 20 reqs_thread_entry: start 2024-05-30T16:19:56.068 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.065+0000 7f0e8c345980 20 init_complete bucket index max shards: 11 2024-05-30T16:19:56.068 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.065+0000 7f0e8c345980 20 Filter name: none 2024-05-30T16:19:56.068 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.065+0000 7f0cabfff640 20 reqs_thread_entry: start 2024-05-30T16:19:56.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.077+0000 7f0d3b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:19:56.084 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.081+0000 7f0e8c345980 20 remove_watcher() i=0 2024-05-30T16:19:56.084 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.081+0000 7f0e8c345980 2 removed watcher, disabling cache 2024-05-30T16:19:56.086 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.085+0000 7f0e8c345980 20 remove_watcher() i=1 2024-05-30T16:19:56.088 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.085+0000 7f0e8c345980 20 remove_watcher() i=2 2024-05-30T16:19:56.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.089+0000 7f0e8c345980 20 remove_watcher() i=3 2024-05-30T16:19:56.092 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.089+0000 7f0e8c345980 20 remove_watcher() i=4 2024-05-30T16:19:56.094 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.093+0000 7f0e8c345980 20 remove_watcher() i=5 2024-05-30T16:19:56.095 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.093+0000 7f0e8c345980 20 remove_watcher() i=6 2024-05-30T16:19:56.097 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.093+0000 7f0e8c345980 20 remove_watcher() i=7 2024-05-30T16:19:56.112 INFO:tasks.workunit.client.0.smithi006.stderr:bucket name a-bucket 2024-05-30T16:19:56.112 INFO:tasks.workunit.client.0.smithi006.stderr:TEST: reshard process 2024-05-30T16:19:56.112 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:19:56.112 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin reshard process 2024-05-30T16:19:56.137 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:19:56.137 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:19:56.183 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.181+0000 7fc2f5037980 20 rados->read ofs=0 len=0 2024-05-30T16:19:56.185 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.181+0000 7fc2f5037980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:56.185 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.181+0000 7fc2f5037980 20 realm 2024-05-30T16:19:56.185 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.181+0000 7fc2f5037980 20 rados->read ofs=0 len=0 2024-05-30T16:19:56.185 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.181+0000 7fc2f5037980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:56.185 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.181+0000 7fc2f5037980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:19:56.186 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.181+0000 7fc2f5037980 20 rados->read ofs=0 len=0 2024-05-30T16:19:56.186 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.181+0000 7fc2f5037980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:56.186 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.181+0000 7fc2f5037980 20 rados->read ofs=0 len=0 2024-05-30T16:19:56.187 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.185+0000 7fc2f5037980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:19:56.187 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.185+0000 7fc2f5037980 20 rados->read ofs=0 len=0 2024-05-30T16:19:56.187 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.185+0000 7fc2f5037980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:19:56.188 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.185+0000 7fc2f5037980 20 searching for the correct realm 2024-05-30T16:19:56.194 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.193+0000 7fc2f3349640 0 -- 172.21.15.6:0/3080297971 >> [v2:172.21.15.204:6800/3687190393,v1:172.21.15.204:6801/3687190393] conn(0x7fc2e8121a20 msgr2=0x7fc2e80f4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:19:56.412 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.409+0000 7fc2f2b48640 0 -- 172.21.15.6:0/3080297971 >> [v2:172.21.15.204:6816/2677861390,v1:172.21.15.204:6817/2677861390] conn(0x55e5415f8e00 msgr2=0x55e5415fb250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:19:56.419 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.417+0000 7fc2f5037980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:19:56.419 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.417+0000 7fc2f5037980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:19:56.420 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.417+0000 7fc2f5037980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:19:56.420 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.417+0000 7fc2f5037980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:19:56.420 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.417+0000 7fc2f5037980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:19:56.420 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.417+0000 7fc2f5037980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:19:56.420 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.417+0000 7fc2f5037980 20 rados->read ofs=0 len=0 2024-05-30T16:19:56.420 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.417+0000 7fc2f5037980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:56.420 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.417+0000 7fc2f5037980 20 rados->read ofs=0 len=0 2024-05-30T16:19:56.421 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.417+0000 7fc2f5037980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:19:56.421 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.417+0000 7fc2f5037980 20 rados->read ofs=0 len=0 2024-05-30T16:19:56.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.421+0000 7fc2f5037980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:19:56.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.421+0000 7fc2f5037980 20 zone default found 2024-05-30T16:19:56.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.421+0000 7fc2f5037980 4 Realm: () 2024-05-30T16:19:56.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.421+0000 7fc2f5037980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:19:56.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.421+0000 7fc2f5037980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:19:56.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.421+0000 7fc2f5037980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:19:56.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.421+0000 7fc2f5037980 20 zonegroup default 2024-05-30T16:19:56.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.421+0000 7fc2f5037980 20 rados->read ofs=0 len=0 2024-05-30T16:19:56.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.421+0000 7fc2f5037980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:56.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.421+0000 7fc2f5037980 20 rados->read ofs=0 len=0 2024-05-30T16:19:56.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.421+0000 7fc2f5037980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:56.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.421+0000 7fc2f5037980 20 rados->read ofs=0 len=0 2024-05-30T16:19:56.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.421+0000 7fc2f5037980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:19:56.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.421+0000 7fc2f5037980 20 started sync module instance, tier type = 2024-05-30T16:19:56.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.421+0000 7fc2f5037980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:19:56.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.445+0000 7fc2f5037980 20 add_watcher() i=0 2024-05-30T16:19:56.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.445+0000 7fc2f5037980 20 add_watcher() i=1 2024-05-30T16:19:56.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.445+0000 7fc2f5037980 20 add_watcher() i=2 2024-05-30T16:19:56.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.445+0000 7fc2f5037980 20 add_watcher() i=3 2024-05-30T16:19:56.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.445+0000 7fc2f5037980 20 add_watcher() i=4 2024-05-30T16:19:56.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.445+0000 7fc2f5037980 20 add_watcher() i=5 2024-05-30T16:19:56.449 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.445+0000 7fc2f5037980 20 add_watcher() i=6 2024-05-30T16:19:56.451 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.449+0000 7fc2f5037980 20 add_watcher() i=7 2024-05-30T16:19:56.451 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.449+0000 7fc2f5037980 2 all 8 watchers are set, enabling cache 2024-05-30T16:19:56.455 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.453+0000 7fc1a97e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:19:56.455 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.453+0000 7fc2f5037980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:19:56.455 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.453+0000 7fc2f5037980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:19:56.455 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.453+0000 7fc2f5037980 5 note: GC not initialized 2024-05-30T16:19:56.456 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.453+0000 7fc123fff640 20 reqs_thread_entry: start 2024-05-30T16:19:56.457 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.453+0000 7fc2f5037980 20 init_complete bucket index max shards: 11 2024-05-30T16:19:56.457 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.453+0000 7fc2f5037980 20 Filter name: none 2024-05-30T16:19:56.457 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.453+0000 7fc121ffb640 20 reqs_thread_entry: start 2024-05-30T16:19:56.457 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.453+0000 7fc2f5037980 20 processing logshard = reshard.0000000000 2024-05-30T16:19:56.463 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.461+0000 7fc2f5037980 20 finish processing logshard = reshard.0000000000 , ret = 0 2024-05-30T16:19:56.463 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.461+0000 7fc2f5037980 20 processing logshard = reshard.0000000001 2024-05-30T16:19:56.468 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.465+0000 7fc2f5037980 20 finish processing logshard = reshard.0000000001 , ret = 0 2024-05-30T16:19:56.468 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.465+0000 7fc2f5037980 20 processing logshard = reshard.0000000002 2024-05-30T16:19:56.473 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.473+0000 7fc2f5037980 20 finish processing logshard = reshard.0000000002 , ret = 0 2024-05-30T16:19:56.473 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.473+0000 7fc2f5037980 20 processing logshard = reshard.0000000003 2024-05-30T16:19:56.479 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.477+0000 7fc2f5037980 20 finish processing logshard = reshard.0000000003 , ret = 0 2024-05-30T16:19:56.479 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.477+0000 7fc2f5037980 20 processing logshard = reshard.0000000004 2024-05-30T16:19:56.484 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.481+0000 7fc2f5037980 20 finish processing logshard = reshard.0000000004 , ret = 0 2024-05-30T16:19:56.485 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.481+0000 7fc2f5037980 20 processing logshard = reshard.0000000005 2024-05-30T16:19:56.490 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.489+0000 7fc2f5037980 20 finish processing logshard = reshard.0000000005 , ret = 0 2024-05-30T16:19:56.490 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.489+0000 7fc2f5037980 20 processing logshard = reshard.0000000006 2024-05-30T16:19:56.493 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.489+0000 7fc2f5037980 20 process_entry resharding a-bucket 2024-05-30T16:19:56.493 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.489+0000 7fc2f5037980 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-05-30T16:19:56.493 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.489+0000 7fc2f5037980 20 rados->read ofs=0 len=0 2024-05-30T16:19:56.494 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.493+0000 7fc2f5037980 20 rados_obj.operate() r=0 bl.length=200 2024-05-30T16:19:56.494 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.493+0000 7fc2f5037980 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x1 2024-05-30T16:19:56.494 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.493+0000 7fc2f5037980 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-05-30T16:19:56.494 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.493+0000 7fc2f5037980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 : miss 2024-05-30T16:19:56.494 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.493+0000 7fc2f5037980 20 rados->read ofs=0 len=0 2024-05-30T16:19:56.495 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.493+0000 7fc2f5037980 20 rados_obj.operate() r=0 bl.length=414 2024-05-30T16:19:56.495 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.493+0000 7fc2f5037980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 info.flags=0x17 2024-05-30T16:19:56.495 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.493+0000 7fc2f5037980 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 to cache LRU end 2024-05-30T16:19:56.495 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.493+0000 7fc2f5037980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:19:56.495 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.493+0000 7fc2f5037980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:19:56.495 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.493+0000 7fc2f5037980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 2024-05-30T16:19:56.508 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.505+0000 7fc2f5037980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 info.flags=0x17 2024-05-30T16:19:56.508 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.505+0000 7fc2f5037980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 to cache LRU end 2024-05-30T16:19:56.508 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.505+0000 7fc2f5037980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:19:56.508 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.505+0000 7fc2f5037980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18, ofs0, ns] 2024-05-30T16:19:56.508 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.505+0000 7fc1d67fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431612 cookie 94443132687696 notifier 6006 bl.length()=903 2024-05-30T16:19:56.508 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.505+0000 7fc1d67fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 info.flags=0x17 2024-05-30T16:19:56.509 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.505+0000 7fc1d67fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 to cache LRU end 2024-05-30T16:19:56.509 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.505+0000 7fc1d67fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:19:56.513 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.513+0000 7fc2f5037980 20 handle_bi_update(): bucket=:a-bucket[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18]): orig_sources= new_sources= 2024-05-30T16:19:56.513 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.513+0000 7fc2f5037980 20 handle_bi_update(): bucket=:a-bucket[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18]): potential sources added=[] removed=[] 2024-05-30T16:19:56.513 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.513+0000 7fc2f5037980 20 handle_bi_update(): bucket=:a-bucket[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18]): orig_dests= new_dests= 2024-05-30T16:19:56.513 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.513+0000 7fc2f5037980 20 handle_bi_update(): bucket=:a-bucket[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18]): potential dests added=[] removed=[] 2024-05-30T16:19:56.520 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.517+0000 7fc2f5037980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.0 2024-05-30T16:19:56.520 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.517+0000 7fc2f5037980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.0 2024-05-30T16:19:56.520 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.517+0000 7fc2f5037980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.1 2024-05-30T16:19:56.520 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.517+0000 7fc2f5037980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.1 2024-05-30T16:19:56.520 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.517+0000 7fc2f5037980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.2 2024-05-30T16:19:56.520 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.517+0000 7fc2f5037980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.2 2024-05-30T16:19:56.520 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.517+0000 7fc2f5037980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.3 2024-05-30T16:19:56.520 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.517+0000 7fc2f5037980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.3 2024-05-30T16:19:56.521 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.517+0000 7fc2f5037980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.4 2024-05-30T16:19:56.521 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.517+0000 7fc2f5037980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.4 2024-05-30T16:19:56.521 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.517+0000 7fc2f5037980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.5 2024-05-30T16:19:56.521 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.517+0000 7fc2f5037980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.5 2024-05-30T16:19:56.521 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.517+0000 7fc2f5037980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.6 2024-05-30T16:19:56.521 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.517+0000 7fc2f5037980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.6 2024-05-30T16:19:56.521 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.517+0000 7fc2f5037980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.7 2024-05-30T16:19:56.521 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.517+0000 7fc2f5037980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.7 2024-05-30T16:19:56.521 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.517+0000 7fc2f5037980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.8 2024-05-30T16:19:56.521 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.517+0000 7fc2f5037980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.8 2024-05-30T16:19:56.521 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.517+0000 7fc2f5037980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.9 2024-05-30T16:19:56.521 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.517+0000 7fc2f5037980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.9 2024-05-30T16:19:56.521 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.517+0000 7fc2f5037980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.10 2024-05-30T16:19:56.521 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.517+0000 7fc2f5037980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.10 2024-05-30T16:19:56.522 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.517+0000 7fc2f5037980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.11 2024-05-30T16:19:56.522 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.517+0000 7fc2f5037980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.11 2024-05-30T16:19:56.522 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.517+0000 7fc2f5037980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.0 2024-05-30T16:19:56.522 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.517+0000 7fc2f5037980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1 2024-05-30T16:19:56.522 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.521+0000 7fc2f5037980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2 2024-05-30T16:19:56.523 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.521+0000 7fc2f5037980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.3 2024-05-30T16:19:56.525 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.521+0000 7fc2f5037980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.4 2024-05-30T16:19:56.526 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.525+0000 7fc2f5037980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.5 2024-05-30T16:19:56.527 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.525+0000 7fc2f5037980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.6 2024-05-30T16:19:56.528 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.525+0000 7fc2f5037980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.7 2024-05-30T16:19:56.529 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.525+0000 7fc2f5037980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.8 2024-05-30T16:19:56.530 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.529+0000 7fc2f5037980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.9 2024-05-30T16:19:56.531 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.529+0000 7fc2f5037980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.10 2024-05-30T16:19:56.532 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.529+0000 7fc2f5037980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 : hit (requested=0x17, cached=0x17) 2024-05-30T16:19:56.532 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.529+0000 7fc2f5037980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 2024-05-30T16:19:56.535 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.533+0000 7fc2f5037980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 info.flags=0x17 2024-05-30T16:19:56.535 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.533+0000 7fc2f5037980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 to cache LRU end 2024-05-30T16:19:56.535 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.533+0000 7fc2f5037980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:19:56.535 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.533+0000 7fc2f5037980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18, ofs0, ns] 2024-05-30T16:19:56.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.533+0000 7fc1d67fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431613 cookie 94443132687696 notifier 6006 bl.length()=871 2024-05-30T16:19:56.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.533+0000 7fc1d67fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 info.flags=0x17 2024-05-30T16:19:56.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.533+0000 7fc1d67fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 to cache LRU end 2024-05-30T16:19:56.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.533+0000 7fc1d67fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:19:56.540 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.537+0000 7fc2f5037980 20 handle_bi_update(): bucket=:a-bucket[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18]): orig_sources= new_sources= 2024-05-30T16:19:56.540 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.537+0000 7fc2f5037980 20 handle_bi_update(): bucket=:a-bucket[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18]): potential sources added=[] removed=[] 2024-05-30T16:19:56.540 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.537+0000 7fc2f5037980 20 handle_bi_update(): bucket=:a-bucket[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18]): orig_dests= new_dests= 2024-05-30T16:19:56.540 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.537+0000 7fc2f5037980 20 handle_bi_update(): bucket=:a-bucket[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18]): potential dests added=[] removed=[] 2024-05-30T16:19:56.546 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.545+0000 7fc2f5037980 1 execute INFO: reshard of bucket "a-bucket" completed successfully 2024-05-30T16:19:56.548 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.545+0000 7fc2f5037980 20 process_entry removing reshard queue entry for bucket a-bucket 2024-05-30T16:19:56.554 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.553+0000 7fc2f5037980 20 finish processing logshard = reshard.0000000006 , ret = 0 2024-05-30T16:19:56.554 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.553+0000 7fc2f5037980 20 processing logshard = reshard.0000000007 2024-05-30T16:19:56.560 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.557+0000 7fc2f5037980 20 finish processing logshard = reshard.0000000007 , ret = 0 2024-05-30T16:19:56.560 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.557+0000 7fc2f5037980 20 processing logshard = reshard.0000000008 2024-05-30T16:19:56.566 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.565+0000 7fc2f5037980 20 finish processing logshard = reshard.0000000008 , ret = 0 2024-05-30T16:19:56.566 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.565+0000 7fc2f5037980 20 processing logshard = reshard.0000000009 2024-05-30T16:19:56.572 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.569+0000 7fc2f5037980 20 finish processing logshard = reshard.0000000009 , ret = 0 2024-05-30T16:19:56.572 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.569+0000 7fc2f5037980 20 processing logshard = reshard.0000000010 2024-05-30T16:19:56.578 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.577+0000 7fc2f5037980 20 finish processing logshard = reshard.0000000010 , ret = 0 2024-05-30T16:19:56.578 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.577+0000 7fc2f5037980 20 processing logshard = reshard.0000000011 2024-05-30T16:19:56.584 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.581+0000 7fc2f5037980 20 finish processing logshard = reshard.0000000011 , ret = 0 2024-05-30T16:19:56.584 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.581+0000 7fc2f5037980 20 processing logshard = reshard.0000000012 2024-05-30T16:19:56.592 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.589+0000 7fc2f5037980 20 finish processing logshard = reshard.0000000012 , ret = 0 2024-05-30T16:19:56.592 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.589+0000 7fc2f5037980 20 processing logshard = reshard.0000000013 2024-05-30T16:19:56.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.597+0000 7fc2f5037980 20 finish processing logshard = reshard.0000000013 , ret = 0 2024-05-30T16:19:56.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.597+0000 7fc2f5037980 20 processing logshard = reshard.0000000014 2024-05-30T16:19:56.605 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.601+0000 7fc2f5037980 20 finish processing logshard = reshard.0000000014 , ret = 0 2024-05-30T16:19:56.605 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.601+0000 7fc2f5037980 20 processing logshard = reshard.0000000015 2024-05-30T16:19:56.612 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.609+0000 7fc2f5037980 20 finish processing logshard = reshard.0000000015 , ret = 0 2024-05-30T16:19:56.614 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.613+0000 7fc1a97e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:19:56.619 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.617+0000 7fc2f5037980 20 remove_watcher() i=0 2024-05-30T16:19:56.619 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.617+0000 7fc2f5037980 2 removed watcher, disabling cache 2024-05-30T16:19:56.622 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.621+0000 7fc2f5037980 20 remove_watcher() i=1 2024-05-30T16:19:56.625 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.621+0000 7fc2f5037980 20 remove_watcher() i=2 2024-05-30T16:19:56.627 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.625+0000 7fc2f5037980 20 remove_watcher() i=3 2024-05-30T16:19:56.630 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.629+0000 7fc2f5037980 20 remove_watcher() i=4 2024-05-30T16:19:56.632 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.629+0000 7fc2f5037980 20 remove_watcher() i=5 2024-05-30T16:19:56.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.633+0000 7fc2f5037980 20 remove_watcher() i=6 2024-05-30T16:19:56.636 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:19:56.633+0000 7fc2f5037980 20 remove_watcher() i=7 2024-05-30T16:20:01.663 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-05-30T16:20:01.697 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:20:01.697 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:20:01.743 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.741+0000 7f1ff5a6e980 20 rados->read ofs=0 len=0 2024-05-30T16:20:01.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.741+0000 7f1ff5a6e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:01.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.741+0000 7f1ff5a6e980 20 realm 2024-05-30T16:20:01.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.741+0000 7f1ff5a6e980 20 rados->read ofs=0 len=0 2024-05-30T16:20:01.746 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.745+0000 7f1ff5a6e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:01.746 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.745+0000 7f1ff5a6e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:01.746 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.745+0000 7f1ff5a6e980 20 rados->read ofs=0 len=0 2024-05-30T16:20:01.746 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.745+0000 7f1ff5a6e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:01.746 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.745+0000 7f1ff5a6e980 20 rados->read ofs=0 len=0 2024-05-30T16:20:01.747 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.745+0000 7f1ff5a6e980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:01.747 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.745+0000 7f1ff5a6e980 20 rados->read ofs=0 len=0 2024-05-30T16:20:01.748 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.745+0000 7f1ff5a6e980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:01.748 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.745+0000 7f1ff5a6e980 20 searching for the correct realm 2024-05-30T16:20:01.754 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.753+0000 7f1ff3d80640 0 -- 172.21.15.6:0/1978310081 >> [v2:172.21.15.6:6808/1897362715,v1:172.21.15.6:6809/1897362715] conn(0x55caf0a5f3d0 msgr2=0x55caf0a61800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:01.761 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.761+0000 7f1ff2d7e640 0 -- 172.21.15.6:0/1978310081 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f1fe4008040 msgr2=0x7f1fe40663e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:01.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.765+0000 7f1ff5a6e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:01.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.765+0000 7f1ff5a6e980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:01.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.765+0000 7f1ff5a6e980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:01.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.765+0000 7f1ff5a6e980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:01.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.765+0000 7f1ff5a6e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:01.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.765+0000 7f1ff5a6e980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:01.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.765+0000 7f1ff5a6e980 20 rados->read ofs=0 len=0 2024-05-30T16:20:01.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.765+0000 7f1ff5a6e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:01.770 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.765+0000 7f1ff5a6e980 20 rados->read ofs=0 len=0 2024-05-30T16:20:01.770 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.769+0000 7f1ff5a6e980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:01.770 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.769+0000 7f1ff5a6e980 20 rados->read ofs=0 len=0 2024-05-30T16:20:01.770 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.769+0000 7f1ff5a6e980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:01.770 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.769+0000 7f1ff5a6e980 20 zone default found 2024-05-30T16:20:01.770 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.769+0000 7f1ff5a6e980 4 Realm: () 2024-05-30T16:20:01.770 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.769+0000 7f1ff5a6e980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:01.771 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.769+0000 7f1ff5a6e980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:01.771 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.769+0000 7f1ff5a6e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:01.771 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.769+0000 7f1ff5a6e980 20 zonegroup default 2024-05-30T16:20:01.771 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.769+0000 7f1ff5a6e980 20 rados->read ofs=0 len=0 2024-05-30T16:20:01.771 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.769+0000 7f1ff5a6e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:01.771 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.769+0000 7f1ff5a6e980 20 rados->read ofs=0 len=0 2024-05-30T16:20:01.772 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.769+0000 7f1ff5a6e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:01.772 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.769+0000 7f1ff5a6e980 20 rados->read ofs=0 len=0 2024-05-30T16:20:01.772 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.769+0000 7f1ff5a6e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:01.772 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.769+0000 7f1ff5a6e980 20 started sync module instance, tier type = 2024-05-30T16:20:01.772 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.769+0000 7f1ff5a6e980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:01.788 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.785+0000 7f1ff5a6e980 20 add_watcher() i=0 2024-05-30T16:20:01.788 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.785+0000 7f1ff5a6e980 20 add_watcher() i=1 2024-05-30T16:20:01.788 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.785+0000 7f1ff5a6e980 20 add_watcher() i=2 2024-05-30T16:20:01.788 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.785+0000 7f1ff5a6e980 20 add_watcher() i=3 2024-05-30T16:20:01.788 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.785+0000 7f1ff5a6e980 20 add_watcher() i=4 2024-05-30T16:20:01.788 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.785+0000 7f1ff5a6e980 20 add_watcher() i=5 2024-05-30T16:20:01.788 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.785+0000 7f1ff5a6e980 20 add_watcher() i=6 2024-05-30T16:20:01.789 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.789+0000 7f1ff5a6e980 20 add_watcher() i=7 2024-05-30T16:20:01.789 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.789+0000 7f1ff5a6e980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:01.792 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.789+0000 7f1ea9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:01.793 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.789+0000 7f1ff5a6e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:01.793 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.789+0000 7f1ff5a6e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:01.793 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.789+0000 7f1ff5a6e980 5 note: GC not initialized 2024-05-30T16:20:01.793 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.789+0000 7f1ea8fe1640 20 reqs_thread_entry: start 2024-05-30T16:20:01.793 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.789+0000 7f1ff5a6e980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:01.793 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.789+0000 7f1ff5a6e980 20 Filter name: none 2024-05-30T16:20:01.793 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.789+0000 7f1e227fc640 20 reqs_thread_entry: start 2024-05-30T16:20:01.793 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.789+0000 7f1ff5a6e980 20 rados->read ofs=0 len=0 2024-05-30T16:20:01.794 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.793+0000 7f1ff5a6e980 20 rados_obj.operate() r=0 bl.length=200 2024-05-30T16:20:01.794 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.793+0000 7f1ff5a6e980 20 rados->read ofs=0 len=0 2024-05-30T16:20:01.795 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.793+0000 7f1ff5a6e980 20 rados_obj.operate() r=0 bl.length=414 2024-05-30T16:20:01.795 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.793+0000 7f1ff5a6e980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:01.795 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.793+0000 7f1ff5a6e980 20 rados->read ofs=0 len=0 2024-05-30T16:20:01.796 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.793+0000 7f1ff5a6e980 20 rados_obj.operate() r=0 bl.length=200 2024-05-30T16:20:01.796 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.793+0000 7f1ff5a6e980 20 rados->read ofs=0 len=0 2024-05-30T16:20:01.797 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.793+0000 7f1ff5a6e980 20 rados_obj.operate() r=0 bl.length=414 2024-05-30T16:20:01.797 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.793+0000 7f1ff5a6e980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:01.800 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.797+0000 7f1ea9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:01.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.801+0000 7f1ff5a6e980 20 remove_watcher() i=0 2024-05-30T16:20:01.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.801+0000 7f1ff5a6e980 2 removed watcher, disabling cache 2024-05-30T16:20:01.807 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.805+0000 7f1ff5a6e980 20 remove_watcher() i=1 2024-05-30T16:20:01.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.809+0000 7f1ff5a6e980 20 remove_watcher() i=2 2024-05-30T16:20:01.812 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.809+0000 7f1ff5a6e980 20 remove_watcher() i=3 2024-05-30T16:20:01.814 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.813+0000 7f1ff5a6e980 20 remove_watcher() i=4 2024-05-30T16:20:01.815 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.813+0000 7f1ff5a6e980 20 remove_watcher() i=5 2024-05-30T16:20:01.817 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.813+0000 7f1ff5a6e980 20 remove_watcher() i=6 2024-05-30T16:20:01.818 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:01.817+0000 7f1ff5a6e980 20 remove_watcher() i=7 2024-05-30T16:20:01.834 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:01.834 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:20:01.834 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: a-bucket id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 num_objects: 0 size_kb: 0 num_shards: 12 2024-05-30T16:20:01.834 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:01.834 INFO:tasks.workunit.client.0.smithi006.stderr:TEST: reshard add non empty bucket 2024-05-30T16:20:01.835 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:02.512 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-05-30T16:20:02.545 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:20:02.545 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:20:02.592 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.589+0000 7f519e4c2980 20 rados->read ofs=0 len=0 2024-05-30T16:20:02.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.593+0000 7f519e4c2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:02.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.593+0000 7f519e4c2980 20 realm 2024-05-30T16:20:02.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.593+0000 7f519e4c2980 20 rados->read ofs=0 len=0 2024-05-30T16:20:02.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.593+0000 7f519e4c2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:02.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.593+0000 7f519e4c2980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:02.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.593+0000 7f519e4c2980 20 rados->read ofs=0 len=0 2024-05-30T16:20:02.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.593+0000 7f519e4c2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:02.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.593+0000 7f519e4c2980 20 rados->read ofs=0 len=0 2024-05-30T16:20:02.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.593+0000 7f519e4c2980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:02.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.593+0000 7f519e4c2980 20 rados->read ofs=0 len=0 2024-05-30T16:20:02.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.597+0000 7f519e4c2980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:02.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.597+0000 7f519e4c2980 20 searching for the correct realm 2024-05-30T16:20:02.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.601+0000 7f519c7d4640 0 -- 172.21.15.6:0/3076816421 >> [v2:172.21.15.204:6823/3156311527,v1:172.21.15.204:6825/3156311527] conn(0x7f51900c0040 msgr2=0x7f519012a870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:02.609 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.605+0000 7f519bfd3640 0 -- 172.21.15.6:0/3076816421 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f51900c1bf0 msgr2=0x7f51900df040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:02.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.825+0000 7f519e4c2980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:02.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.825+0000 7f519e4c2980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:02.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.825+0000 7f519e4c2980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:02.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.825+0000 7f519e4c2980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:02.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.825+0000 7f519e4c2980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:02.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.825+0000 7f519e4c2980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:02.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.825+0000 7f519e4c2980 20 rados->read ofs=0 len=0 2024-05-30T16:20:02.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.825+0000 7f519e4c2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:02.829 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.825+0000 7f519e4c2980 20 rados->read ofs=0 len=0 2024-05-30T16:20:02.829 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.825+0000 7f519e4c2980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:02.829 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.825+0000 7f519e4c2980 20 rados->read ofs=0 len=0 2024-05-30T16:20:02.830 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.829+0000 7f519e4c2980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:02.830 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.829+0000 7f519e4c2980 20 zone default found 2024-05-30T16:20:02.830 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.829+0000 7f519e4c2980 4 Realm: () 2024-05-30T16:20:02.830 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.829+0000 7f519e4c2980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:02.830 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.829+0000 7f519e4c2980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:02.830 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.829+0000 7f519e4c2980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:02.830 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.829+0000 7f519e4c2980 20 zonegroup default 2024-05-30T16:20:02.831 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.829+0000 7f519e4c2980 20 rados->read ofs=0 len=0 2024-05-30T16:20:02.831 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.829+0000 7f519e4c2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:02.831 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.829+0000 7f519e4c2980 20 rados->read ofs=0 len=0 2024-05-30T16:20:02.831 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.829+0000 7f519e4c2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:02.831 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.829+0000 7f519e4c2980 20 rados->read ofs=0 len=0 2024-05-30T16:20:02.832 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.829+0000 7f519e4c2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:02.832 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.829+0000 7f519e4c2980 20 started sync module instance, tier type = 2024-05-30T16:20:02.832 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.829+0000 7f519e4c2980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:02.856 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.853+0000 7f519e4c2980 20 add_watcher() i=0 2024-05-30T16:20:02.857 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.853+0000 7f519e4c2980 20 add_watcher() i=1 2024-05-30T16:20:02.857 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.853+0000 7f519e4c2980 20 add_watcher() i=2 2024-05-30T16:20:02.857 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.853+0000 7f519e4c2980 20 add_watcher() i=3 2024-05-30T16:20:02.857 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.853+0000 7f519e4c2980 20 add_watcher() i=4 2024-05-30T16:20:02.857 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.853+0000 7f519e4c2980 20 add_watcher() i=5 2024-05-30T16:20:02.857 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.853+0000 7f519e4c2980 20 add_watcher() i=6 2024-05-30T16:20:02.859 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.857+0000 7f519e4c2980 20 add_watcher() i=7 2024-05-30T16:20:02.859 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.857+0000 7f519e4c2980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:02.864 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.861+0000 7f5051fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:02.865 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.861+0000 7f519e4c2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:02.865 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.861+0000 7f519e4c2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:02.865 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.861+0000 7f519e4c2980 5 note: GC not initialized 2024-05-30T16:20:02.865 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.865+0000 7f5050fe1640 20 reqs_thread_entry: start 2024-05-30T16:20:02.866 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.865+0000 7f519e4c2980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:02.866 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.865+0000 7f519e4c2980 20 Filter name: none 2024-05-30T16:20:02.866 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.865+0000 7f4fca7fc640 20 reqs_thread_entry: start 2024-05-30T16:20:02.866 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.865+0000 7f519e4c2980 20 rados->read ofs=0 len=0 2024-05-30T16:20:02.867 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.865+0000 7f519e4c2980 20 rados_obj.operate() r=0 bl.length=200 2024-05-30T16:20:02.868 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.865+0000 7f519e4c2980 20 rados->read ofs=0 len=0 2024-05-30T16:20:02.868 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.865+0000 7f519e4c2980 20 rados_obj.operate() r=0 bl.length=414 2024-05-30T16:20:02.868 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.865+0000 7f519e4c2980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:02.868 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.865+0000 7f519e4c2980 20 rados->read ofs=0 len=0 2024-05-30T16:20:02.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.865+0000 7f519e4c2980 20 rados_obj.operate() r=0 bl.length=200 2024-05-30T16:20:02.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.865+0000 7f519e4c2980 20 rados->read ofs=0 len=0 2024-05-30T16:20:02.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.869+0000 7f519e4c2980 20 rados_obj.operate() r=0 bl.length=414 2024-05-30T16:20:02.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.869+0000 7f519e4c2980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:02.873 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.873+0000 7f5051fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:02.878 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.877+0000 7f519e4c2980 20 remove_watcher() i=0 2024-05-30T16:20:02.878 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.877+0000 7f519e4c2980 2 removed watcher, disabling cache 2024-05-30T16:20:02.881 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.877+0000 7f519e4c2980 20 remove_watcher() i=1 2024-05-30T16:20:02.884 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.881+0000 7f519e4c2980 20 remove_watcher() i=2 2024-05-30T16:20:02.886 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.885+0000 7f519e4c2980 20 remove_watcher() i=3 2024-05-30T16:20:02.888 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.885+0000 7f519e4c2980 20 remove_watcher() i=4 2024-05-30T16:20:02.890 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.889+0000 7f519e4c2980 20 remove_watcher() i=5 2024-05-30T16:20:02.891 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.889+0000 7f519e4c2980 20 remove_watcher() i=6 2024-05-30T16:20:02.893 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.889+0000 7f519e4c2980 20 remove_watcher() i=7 2024-05-30T16:20:02.909 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:02.909 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:20:02.909 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: a-bucket id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 num_objects: 8 size_kb: 1 num_shards: 12 2024-05-30T16:20:02.909 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:02.909 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin reshard add --bucket a-bucket --num-shards 13 2024-05-30T16:20:02.945 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:20:02.945 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:20:02.993 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.989+0000 7f7e23527980 20 rados->read ofs=0 len=0 2024-05-30T16:20:02.994 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.993+0000 7f7e23527980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:02.995 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.993+0000 7f7e23527980 20 realm 2024-05-30T16:20:02.995 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.993+0000 7f7e23527980 20 rados->read ofs=0 len=0 2024-05-30T16:20:02.995 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.993+0000 7f7e23527980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:02.995 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.993+0000 7f7e23527980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:02.995 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.993+0000 7f7e23527980 20 rados->read ofs=0 len=0 2024-05-30T16:20:02.995 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.993+0000 7f7e23527980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:02.995 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.993+0000 7f7e23527980 20 rados->read ofs=0 len=0 2024-05-30T16:20:02.996 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.993+0000 7f7e23527980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:02.996 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.993+0000 7f7e23527980 20 rados->read ofs=0 len=0 2024-05-30T16:20:02.997 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.993+0000 7f7e23527980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:02.997 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.993+0000 7f7e23527980 20 searching for the correct realm 2024-05-30T16:20:03.000 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:02.997+0000 7f7e21038640 0 -- 172.21.15.6:0/2431403482 >> [v2:172.21.15.6:6816/2704410641,v1:172.21.15.6:6817/2704410641] conn(0x55d140b9fd70 msgr2=0x7f7e1c058fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:03.007 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.005+0000 7f7e21038640 0 -- 172.21.15.6:0/2431403482 >> [v2:172.21.15.6:6816/2704410641,v1:172.21.15.6:6817/2704410641] conn(0x7f7cec07e510 msgr2=0x7f7cec080940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:20:03.017 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.017+0000 7f7e23527980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:03.018 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.017+0000 7f7e23527980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:03.018 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.017+0000 7f7e23527980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:03.018 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.017+0000 7f7e23527980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:03.018 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.017+0000 7f7e23527980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:03.018 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.017+0000 7f7e23527980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:03.018 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.017+0000 7f7e23527980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.019 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.017+0000 7f7e23527980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:03.019 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.017+0000 7f7e23527980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.019 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.017+0000 7f7e23527980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:03.019 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.017+0000 7f7e23527980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.017+0000 7f7e23527980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:03.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.017+0000 7f7e23527980 20 zone default found 2024-05-30T16:20:03.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.017+0000 7f7e23527980 4 Realm: () 2024-05-30T16:20:03.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.017+0000 7f7e23527980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:03.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.017+0000 7f7e23527980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:03.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.017+0000 7f7e23527980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:03.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.017+0000 7f7e23527980 20 zonegroup default 2024-05-30T16:20:03.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.017+0000 7f7e23527980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.017+0000 7f7e23527980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:03.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.017+0000 7f7e23527980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.021 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.017+0000 7f7e23527980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:03.021 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.017+0000 7f7e23527980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.021 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.017+0000 7f7e23527980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:03.021 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.017+0000 7f7e23527980 20 started sync module instance, tier type = 2024-05-30T16:20:03.021 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.017+0000 7f7e23527980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:03.036 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.033+0000 7f7e23527980 20 add_watcher() i=0 2024-05-30T16:20:03.037 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.033+0000 7f7e23527980 20 add_watcher() i=1 2024-05-30T16:20:03.037 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.033+0000 7f7e23527980 20 add_watcher() i=2 2024-05-30T16:20:03.037 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.033+0000 7f7e23527980 20 add_watcher() i=3 2024-05-30T16:20:03.037 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.033+0000 7f7e23527980 20 add_watcher() i=4 2024-05-30T16:20:03.037 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.033+0000 7f7e23527980 20 add_watcher() i=5 2024-05-30T16:20:03.037 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.033+0000 7f7e23527980 20 add_watcher() i=6 2024-05-30T16:20:03.037 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.037+0000 7f7e23527980 20 add_watcher() i=7 2024-05-30T16:20:03.037 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.037+0000 7f7e23527980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:03.040 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.037+0000 7f7cd7fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:03.040 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.037+0000 7f7e23527980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:03.040 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.037+0000 7f7e23527980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:03.041 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.037+0000 7f7e23527980 5 note: GC not initialized 2024-05-30T16:20:03.041 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.037+0000 7f7cd6fe5640 20 reqs_thread_entry: start 2024-05-30T16:20:03.041 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.037+0000 7f7e23527980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:03.041 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.037+0000 7f7e23527980 20 Filter name: none 2024-05-30T16:20:03.042 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.037+0000 7f7cd4fe1640 20 reqs_thread_entry: start 2024-05-30T16:20:03.042 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.041+0000 7f7e23527980 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-05-30T16:20:03.042 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.041+0000 7f7e23527980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.042 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.041+0000 7f7e23527980 20 rados_obj.operate() r=0 bl.length=200 2024-05-30T16:20:03.043 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.041+0000 7f7e23527980 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2024-05-30T16:20:03.043 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.041+0000 7f7e23527980 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-05-30T16:20:03.043 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.041+0000 7f7e23527980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 : miss 2024-05-30T16:20:03.043 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.041+0000 7f7e23527980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.043 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.041+0000 7f7e23527980 20 rados_obj.operate() r=0 bl.length=414 2024-05-30T16:20:03.043 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.041+0000 7f7e23527980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 info.flags=0x17 2024-05-30T16:20:03.043 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.041+0000 7f7e23527980 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 to cache LRU end 2024-05-30T16:20:03.043 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.041+0000 7f7e23527980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:20:03.043 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.041+0000 7f7e23527980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:20:03.043 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.041+0000 7f7e23527980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 2024-05-30T16:20:03.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.045+0000 7f7cd7fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:03.051 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.049+0000 7f7e23527980 20 remove_watcher() i=0 2024-05-30T16:20:03.051 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.049+0000 7f7e23527980 2 removed watcher, disabling cache 2024-05-30T16:20:03.053 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.049+0000 7f7e23527980 20 remove_watcher() i=1 2024-05-30T16:20:03.055 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.053+0000 7f7e23527980 20 remove_watcher() i=2 2024-05-30T16:20:03.057 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.053+0000 7f7e23527980 20 remove_watcher() i=3 2024-05-30T16:20:03.060 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.057+0000 7f7e23527980 20 remove_watcher() i=4 2024-05-30T16:20:03.061 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.061+0000 7f7e23527980 20 remove_watcher() i=5 2024-05-30T16:20:03.062 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.061+0000 7f7e23527980 20 remove_watcher() i=6 2024-05-30T16:20:03.064 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.061+0000 7f7e23527980 20 remove_watcher() i=7 2024-05-30T16:20:03.079 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin reshard list 2024-05-30T16:20:03.103 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:20:03.103 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:20:03.146 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.145+0000 7f2a8a706980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.148 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.145+0000 7f2a8a706980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:03.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.145+0000 7f2a8a706980 20 realm 2024-05-30T16:20:03.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.145+0000 7f2a8a706980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.145+0000 7f2a8a706980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:03.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.145+0000 7f2a8a706980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:03.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.145+0000 7f2a8a706980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.145+0000 7f2a8a706980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:03.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.145+0000 7f2a8a706980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.150 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.149+0000 7f2a8a706980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:03.151 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.149+0000 7f2a8a706980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.151 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.149+0000 7f2a8a706980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:03.152 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.149+0000 7f2a8a706980 20 searching for the correct realm 2024-05-30T16:20:03.159 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.157+0000 7f2a88a18640 0 -- 172.21.15.6:0/654402719 >> [v2:172.21.15.204:6800/3687190393,v1:172.21.15.204:6801/3687190393] conn(0x7f2a7c0c3be0 msgr2=0x7f2a7c0c13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:03.376 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.373+0000 7f2a88217640 0 -- 172.21.15.6:0/654402719 >> [v2:172.21.15.204:6816/2677861390,v1:172.21.15.204:6817/2677861390] conn(0x557c93734270 msgr2=0x557c93736660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:03.383 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.381+0000 7f2a8a706980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:03.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.381+0000 7f2a8a706980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:03.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.381+0000 7f2a8a706980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:03.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.381+0000 7f2a8a706980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:03.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.381+0000 7f2a8a706980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:03.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.381+0000 7f2a8a706980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:03.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.381+0000 7f2a8a706980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.381+0000 7f2a8a706980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:03.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.381+0000 7f2a8a706980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.385 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.385+0000 7f2a8a706980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:03.385 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.385+0000 7f2a8a706980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.386 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.385+0000 7f2a8a706980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:03.386 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.385+0000 7f2a8a706980 20 zone default found 2024-05-30T16:20:03.386 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.385+0000 7f2a8a706980 4 Realm: () 2024-05-30T16:20:03.386 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.385+0000 7f2a8a706980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:03.387 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.385+0000 7f2a8a706980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:03.387 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.385+0000 7f2a8a706980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:03.387 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.385+0000 7f2a8a706980 20 zonegroup default 2024-05-30T16:20:03.387 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.385+0000 7f2a8a706980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.387 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.385+0000 7f2a8a706980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:03.387 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.385+0000 7f2a8a706980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.388 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.385+0000 7f2a8a706980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:03.388 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.385+0000 7f2a8a706980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.388 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.385+0000 7f2a8a706980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:03.388 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.385+0000 7f2a8a706980 20 started sync module instance, tier type = 2024-05-30T16:20:03.388 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.385+0000 7f2a8a706980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:03.414 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.413+0000 7f2a8a706980 20 add_watcher() i=0 2024-05-30T16:20:03.414 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.413+0000 7f2a8a706980 20 add_watcher() i=1 2024-05-30T16:20:03.414 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.413+0000 7f2a8a706980 20 add_watcher() i=2 2024-05-30T16:20:03.414 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.413+0000 7f2a8a706980 20 add_watcher() i=3 2024-05-30T16:20:03.414 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.413+0000 7f2a8a706980 20 add_watcher() i=4 2024-05-30T16:20:03.415 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.413+0000 7f2a8a706980 20 add_watcher() i=5 2024-05-30T16:20:03.415 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.413+0000 7f2a8a706980 20 add_watcher() i=6 2024-05-30T16:20:03.416 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.413+0000 7f2a8a706980 20 add_watcher() i=7 2024-05-30T16:20:03.417 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.413+0000 7f2a8a706980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:03.420 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.417+0000 7f293efdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:03.421 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.421+0000 7f2a8a706980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:03.421 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.421+0000 7f2a8a706980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:03.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.421+0000 7f2a8a706980 5 note: GC not initialized 2024-05-30T16:20:03.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.421+0000 7f293dfdb640 20 reqs_thread_entry: start 2024-05-30T16:20:03.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.421+0000 7f2a8a706980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:03.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.421+0000 7f2a8a706980 20 Filter name: none 2024-05-30T16:20:03.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.421+0000 7f28af7fe640 20 reqs_thread_entry: start 2024-05-30T16:20:03.438 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.437+0000 7f293efdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:03.443 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.441+0000 7f2a8a706980 20 remove_watcher() i=0 2024-05-30T16:20:03.443 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.441+0000 7f2a8a706980 2 removed watcher, disabling cache 2024-05-30T16:20:03.446 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.445+0000 7f2a8a706980 20 remove_watcher() i=1 2024-05-30T16:20:03.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.445+0000 7f2a8a706980 20 remove_watcher() i=2 2024-05-30T16:20:03.450 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.449+0000 7f2a8a706980 20 remove_watcher() i=3 2024-05-30T16:20:03.452 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.449+0000 7f2a8a706980 20 remove_watcher() i=4 2024-05-30T16:20:03.454 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.453+0000 7f2a8a706980 20 remove_watcher() i=5 2024-05-30T16:20:03.455 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.453+0000 7f2a8a706980 20 remove_watcher() i=6 2024-05-30T16:20:03.457 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.453+0000 7f2a8a706980 20 remove_watcher() i=7 2024-05-30T16:20:03.474 INFO:tasks.workunit.client.0.smithi006.stderr:TEST: reshard process non empty bucket 2024-05-30T16:20:03.474 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:03.474 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin reshard process 2024-05-30T16:20:03.499 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:20:03.499 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:20:03.547 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.545+0000 7fc8d5ae5980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.548 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.545+0000 7fc8d5ae5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:03.548 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.545+0000 7fc8d5ae5980 20 realm 2024-05-30T16:20:03.548 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.545+0000 7fc8d5ae5980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.549 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.545+0000 7fc8d5ae5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:03.549 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.545+0000 7fc8d5ae5980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:03.549 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.545+0000 7fc8d5ae5980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.549 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.545+0000 7fc8d5ae5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:03.550 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.545+0000 7fc8d5ae5980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.551 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.549+0000 7fc8d5ae5980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:03.551 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.549+0000 7fc8d5ae5980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.552 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.549+0000 7fc8d5ae5980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:03.552 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.549+0000 7fc8d5ae5980 20 searching for the correct realm 2024-05-30T16:20:03.555 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.553+0000 7fc8d35f6640 0 -- 172.21.15.6:0/2777194068 >> [v2:172.21.15.6:6816/2704410641,v1:172.21.15.6:6817/2704410641] conn(0x7fc8cc0c7040 msgr2=0x7fc8cc0c6410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:03.562 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.561+0000 7fc8d3df7640 0 -- 172.21.15.6:0/2777194068 >> [v2:172.21.15.204:6823/3156311527,v1:172.21.15.204:6825/3156311527] conn(0x7fc8c805e040 msgr2=0x7fc8c8063a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:03.572 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.569+0000 7fc8d5ae5980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:03.572 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.569+0000 7fc8d5ae5980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:03.572 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.569+0000 7fc8d5ae5980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:03.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.569+0000 7fc8d5ae5980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:03.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.569+0000 7fc8d5ae5980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:03.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.569+0000 7fc8d5ae5980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:03.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.569+0000 7fc8d5ae5980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.569+0000 7fc8d5ae5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:03.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.569+0000 7fc8d5ae5980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.573+0000 7fc8d5ae5980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:03.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.573+0000 7fc8d5ae5980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.574 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.573+0000 7fc8d5ae5980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:03.574 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.573+0000 7fc8d5ae5980 20 zone default found 2024-05-30T16:20:03.574 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.573+0000 7fc8d5ae5980 4 Realm: () 2024-05-30T16:20:03.574 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.573+0000 7fc8d5ae5980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:03.574 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.573+0000 7fc8d5ae5980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:03.574 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.573+0000 7fc8d5ae5980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:03.574 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.573+0000 7fc8d5ae5980 20 zonegroup default 2024-05-30T16:20:03.574 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.573+0000 7fc8d5ae5980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.575 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.573+0000 7fc8d5ae5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:03.575 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.573+0000 7fc8d5ae5980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.575 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.573+0000 7fc8d5ae5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:03.575 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.573+0000 7fc8d5ae5980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.576 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.573+0000 7fc8d5ae5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:03.576 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.573+0000 7fc8d5ae5980 20 started sync module instance, tier type = 2024-05-30T16:20:03.576 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.573+0000 7fc8d5ae5980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:03.591 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.589+0000 7fc8d5ae5980 20 add_watcher() i=0 2024-05-30T16:20:03.591 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.589+0000 7fc8d5ae5980 20 add_watcher() i=1 2024-05-30T16:20:03.591 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.589+0000 7fc8d5ae5980 20 add_watcher() i=2 2024-05-30T16:20:03.591 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.589+0000 7fc8d5ae5980 20 add_watcher() i=3 2024-05-30T16:20:03.591 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.589+0000 7fc8d5ae5980 20 add_watcher() i=4 2024-05-30T16:20:03.592 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.589+0000 7fc8d5ae5980 20 add_watcher() i=5 2024-05-30T16:20:03.592 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.589+0000 7fc8d5ae5980 20 add_watcher() i=6 2024-05-30T16:20:03.592 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.589+0000 7fc8d5ae5980 20 add_watcher() i=7 2024-05-30T16:20:03.592 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.589+0000 7fc8d5ae5980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:03.594 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.593+0000 7fc785fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:03.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.593+0000 7fc8d5ae5980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:03.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.593+0000 7fc8d5ae5980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:03.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.593+0000 7fc8d5ae5980 5 note: GC not initialized 2024-05-30T16:20:03.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.593+0000 7fc784fe1640 20 reqs_thread_entry: start 2024-05-30T16:20:03.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.593+0000 7fc8d5ae5980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:03.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.593+0000 7fc8d5ae5980 20 Filter name: none 2024-05-30T16:20:03.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.593+0000 7fc6fe7fc640 20 reqs_thread_entry: start 2024-05-30T16:20:03.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.593+0000 7fc8d5ae5980 20 processing logshard = reshard.0000000000 2024-05-30T16:20:03.601 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.597+0000 7fc8d5ae5980 20 finish processing logshard = reshard.0000000000 , ret = 0 2024-05-30T16:20:03.601 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.597+0000 7fc8d5ae5980 20 processing logshard = reshard.0000000001 2024-05-30T16:20:03.605 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.605+0000 7fc8d5ae5980 20 finish processing logshard = reshard.0000000001 , ret = 0 2024-05-30T16:20:03.605 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.605+0000 7fc8d5ae5980 20 processing logshard = reshard.0000000002 2024-05-30T16:20:03.610 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.609+0000 7fc8d5ae5980 20 finish processing logshard = reshard.0000000002 , ret = 0 2024-05-30T16:20:03.610 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.609+0000 7fc8d5ae5980 20 processing logshard = reshard.0000000003 2024-05-30T16:20:03.616 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.613+0000 7fc8d5ae5980 20 finish processing logshard = reshard.0000000003 , ret = 0 2024-05-30T16:20:03.616 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.613+0000 7fc8d5ae5980 20 processing logshard = reshard.0000000004 2024-05-30T16:20:03.622 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.621+0000 7fc8d5ae5980 20 finish processing logshard = reshard.0000000004 , ret = 0 2024-05-30T16:20:03.622 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.621+0000 7fc8d5ae5980 20 processing logshard = reshard.0000000005 2024-05-30T16:20:03.626 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.625+0000 7fc8d5ae5980 20 finish processing logshard = reshard.0000000005 , ret = 0 2024-05-30T16:20:03.627 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.625+0000 7fc8d5ae5980 20 processing logshard = reshard.0000000006 2024-05-30T16:20:03.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.629+0000 7fc8d5ae5980 20 process_entry resharding a-bucket 2024-05-30T16:20:03.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.629+0000 7fc8d5ae5980 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-05-30T16:20:03.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.629+0000 7fc8d5ae5980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.630 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.629+0000 7fc8d5ae5980 20 rados_obj.operate() r=0 bl.length=200 2024-05-30T16:20:03.631 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.629+0000 7fc8d5ae5980 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x1 2024-05-30T16:20:03.631 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.629+0000 7fc8d5ae5980 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-05-30T16:20:03.631 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.629+0000 7fc8d5ae5980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 : miss 2024-05-30T16:20:03.631 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.629+0000 7fc8d5ae5980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.631 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.629+0000 7fc8d5ae5980 20 rados_obj.operate() r=0 bl.length=414 2024-05-30T16:20:03.631 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.629+0000 7fc8d5ae5980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 info.flags=0x17 2024-05-30T16:20:03.631 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.629+0000 7fc8d5ae5980 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 to cache LRU end 2024-05-30T16:20:03.631 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.629+0000 7fc8d5ae5980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:20:03.631 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.629+0000 7fc8d5ae5980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:20:03.631 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.629+0000 7fc8d5ae5980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 2024-05-30T16:20:03.642 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.641+0000 7fc8d5ae5980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 info.flags=0x17 2024-05-30T16:20:03.642 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.641+0000 7fc8d5ae5980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 to cache LRU end 2024-05-30T16:20:03.642 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.641+0000 7fc8d5ae5980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:20:03.642 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.641+0000 7fc8d5ae5980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18, ofs0, ns] 2024-05-30T16:20:03.642 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.641+0000 7fc7b67fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431614 cookie 94099934229792 notifier 6099 bl.length()=903 2024-05-30T16:20:03.642 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.641+0000 7fc7b67fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 info.flags=0x17 2024-05-30T16:20:03.642 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.641+0000 7fc7b67fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 to cache LRU end 2024-05-30T16:20:03.643 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.641+0000 7fc7b67fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:20:03.645 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.641+0000 7fc8d5ae5980 20 handle_bi_update(): bucket=:a-bucket[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18]): orig_sources= new_sources= 2024-05-30T16:20:03.645 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.641+0000 7fc8d5ae5980 20 handle_bi_update(): bucket=:a-bucket[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18]): potential sources added=[] removed=[] 2024-05-30T16:20:03.645 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.641+0000 7fc8d5ae5980 20 handle_bi_update(): bucket=:a-bucket[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18]): orig_dests= new_dests= 2024-05-30T16:20:03.645 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.641+0000 7fc8d5ae5980 20 handle_bi_update(): bucket=:a-bucket[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18]): potential dests added=[] removed=[] 2024-05-30T16:20:03.650 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.0 2024-05-30T16:20:03.650 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.0 2024-05-30T16:20:03.651 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.1 2024-05-30T16:20:03.651 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.1 2024-05-30T16:20:03.651 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.2 2024-05-30T16:20:03.651 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.2 2024-05-30T16:20:03.651 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.3 2024-05-30T16:20:03.651 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.3 2024-05-30T16:20:03.651 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.4 2024-05-30T16:20:03.651 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.4 2024-05-30T16:20:03.651 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.5 2024-05-30T16:20:03.651 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.5 2024-05-30T16:20:03.651 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.6 2024-05-30T16:20:03.651 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.6 2024-05-30T16:20:03.651 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.7 2024-05-30T16:20:03.652 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.7 2024-05-30T16:20:03.652 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.8 2024-05-30T16:20:03.652 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.8 2024-05-30T16:20:03.652 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.9 2024-05-30T16:20:03.652 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.9 2024-05-30T16:20:03.652 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.10 2024-05-30T16:20:03.652 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.10 2024-05-30T16:20:03.652 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.11 2024-05-30T16:20:03.652 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.11 2024-05-30T16:20:03.652 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.12 2024-05-30T16:20:03.652 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.12 2024-05-30T16:20:03.652 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.0 2024-05-30T16:20:03.652 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.0 2024-05-30T16:20:03.653 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.1 2024-05-30T16:20:03.653 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.1 2024-05-30T16:20:03.653 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.2 2024-05-30T16:20:03.653 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.649+0000 7fc8d5ae5980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.2 2024-05-30T16:20:03.654 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.653+0000 7fc8d5ae5980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.3 2024-05-30T16:20:03.654 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.653+0000 7fc8d5ae5980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.3 2024-05-30T16:20:03.654 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.653+0000 7fc8d5ae5980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.4 2024-05-30T16:20:03.654 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.653+0000 7fc8d5ae5980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.4 2024-05-30T16:20:03.655 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.653+0000 7fc8d5ae5980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.5 2024-05-30T16:20:03.655 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.653+0000 7fc8d5ae5980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.5 2024-05-30T16:20:03.656 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.653+0000 7fc8d5ae5980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.6 2024-05-30T16:20:03.656 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.653+0000 7fc8d5ae5980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.6 2024-05-30T16:20:03.657 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.657+0000 7fc8d5ae5980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.7 2024-05-30T16:20:03.657 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.657+0000 7fc8d5ae5980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.7 2024-05-30T16:20:03.658 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.657+0000 7fc8d5ae5980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.8 2024-05-30T16:20:03.658 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.657+0000 7fc8d5ae5980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.8 2024-05-30T16:20:03.658 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.657+0000 7fc8d5ae5980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.9 2024-05-30T16:20:03.658 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.657+0000 7fc8d5ae5980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.9 2024-05-30T16:20:03.660 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.657+0000 7fc8d5ae5980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.10 2024-05-30T16:20:03.660 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.657+0000 7fc8d5ae5980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.10 2024-05-30T16:20:03.661 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.657+0000 7fc8d5ae5980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.11 2024-05-30T16:20:03.661 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.657+0000 7fc8d5ae5980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.1.11 2024-05-30T16:20:03.664 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.661+0000 7fc8d5ae5980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 : hit (requested=0x17, cached=0x17) 2024-05-30T16:20:03.665 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.661+0000 7fc8d5ae5980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 2024-05-30T16:20:03.666 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.665+0000 7fc8d5ae5980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 info.flags=0x17 2024-05-30T16:20:03.666 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.665+0000 7fc8d5ae5980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 to cache LRU end 2024-05-30T16:20:03.666 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.665+0000 7fc8d5ae5980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:20:03.666 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.665+0000 7fc8d5ae5980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18, ofs0, ns] 2024-05-30T16:20:03.667 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.665+0000 7fc7b6ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431615 cookie 94099934229792 notifier 6099 bl.length()=871 2024-05-30T16:20:03.667 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.665+0000 7fc7b6ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 info.flags=0x17 2024-05-30T16:20:03.667 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.665+0000 7fc7b6ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 to cache LRU end 2024-05-30T16:20:03.667 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.665+0000 7fc7b6ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:20:03.669 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.665+0000 7fc8d5ae5980 20 handle_bi_update(): bucket=:a-bucket[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18]): orig_sources= new_sources= 2024-05-30T16:20:03.669 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.665+0000 7fc8d5ae5980 20 handle_bi_update(): bucket=:a-bucket[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18]): potential sources added=[] removed=[] 2024-05-30T16:20:03.669 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.665+0000 7fc8d5ae5980 20 handle_bi_update(): bucket=:a-bucket[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18]): orig_dests= new_dests= 2024-05-30T16:20:03.669 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.665+0000 7fc8d5ae5980 20 handle_bi_update(): bucket=:a-bucket[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18]): potential dests added=[] removed=[] 2024-05-30T16:20:03.674 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.673+0000 7fc8d5ae5980 1 execute INFO: reshard of bucket "a-bucket" completed successfully 2024-05-30T16:20:03.676 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.673+0000 7fc8d5ae5980 20 process_entry removing reshard queue entry for bucket a-bucket 2024-05-30T16:20:03.680 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.677+0000 7fc8d5ae5980 20 finish processing logshard = reshard.0000000006 , ret = 0 2024-05-30T16:20:03.680 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.677+0000 7fc8d5ae5980 20 processing logshard = reshard.0000000007 2024-05-30T16:20:03.685 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.681+0000 7fc8d5ae5980 20 finish processing logshard = reshard.0000000007 , ret = 0 2024-05-30T16:20:03.685 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.681+0000 7fc8d5ae5980 20 processing logshard = reshard.0000000008 2024-05-30T16:20:03.690 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.689+0000 7fc8d5ae5980 20 finish processing logshard = reshard.0000000008 , ret = 0 2024-05-30T16:20:03.690 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.689+0000 7fc8d5ae5980 20 processing logshard = reshard.0000000009 2024-05-30T16:20:03.697 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.693+0000 7fc8d5ae5980 20 finish processing logshard = reshard.0000000009 , ret = 0 2024-05-30T16:20:03.697 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.693+0000 7fc8d5ae5980 20 processing logshard = reshard.0000000010 2024-05-30T16:20:03.700 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.697+0000 7fc8d5ae5980 20 finish processing logshard = reshard.0000000010 , ret = 0 2024-05-30T16:20:03.700 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.697+0000 7fc8d5ae5980 20 processing logshard = reshard.0000000011 2024-05-30T16:20:03.705 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.701+0000 7fc8d5ae5980 20 finish processing logshard = reshard.0000000011 , ret = 0 2024-05-30T16:20:03.705 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.701+0000 7fc8d5ae5980 20 processing logshard = reshard.0000000012 2024-05-30T16:20:03.712 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.709+0000 7fc8d5ae5980 20 finish processing logshard = reshard.0000000012 , ret = 0 2024-05-30T16:20:03.712 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.709+0000 7fc8d5ae5980 20 processing logshard = reshard.0000000013 2024-05-30T16:20:03.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.717+0000 7fc8d5ae5980 20 finish processing logshard = reshard.0000000013 , ret = 0 2024-05-30T16:20:03.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.717+0000 7fc8d5ae5980 20 processing logshard = reshard.0000000014 2024-05-30T16:20:03.721 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.717+0000 7fc8d5ae5980 20 finish processing logshard = reshard.0000000014 , ret = 0 2024-05-30T16:20:03.721 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.717+0000 7fc8d5ae5980 20 processing logshard = reshard.0000000015 2024-05-30T16:20:03.728 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.725+0000 7fc8d5ae5980 20 finish processing logshard = reshard.0000000015 , ret = 0 2024-05-30T16:20:03.729 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.729+0000 7fc785fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:03.735 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.733+0000 7fc8d5ae5980 20 remove_watcher() i=0 2024-05-30T16:20:03.735 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.733+0000 7fc8d5ae5980 2 removed watcher, disabling cache 2024-05-30T16:20:03.737 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.737+0000 7fc8d5ae5980 20 remove_watcher() i=1 2024-05-30T16:20:03.740 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.737+0000 7fc8d5ae5980 20 remove_watcher() i=2 2024-05-30T16:20:03.742 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.741+0000 7fc8d5ae5980 20 remove_watcher() i=3 2024-05-30T16:20:03.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.741+0000 7fc8d5ae5980 20 remove_watcher() i=4 2024-05-30T16:20:03.746 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.745+0000 7fc8d5ae5980 20 remove_watcher() i=5 2024-05-30T16:20:03.747 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.745+0000 7fc8d5ae5980 20 remove_watcher() i=6 2024-05-30T16:20:03.749 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.749+0000 7fc8d5ae5980 20 remove_watcher() i=7 2024-05-30T16:20:03.769 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-05-30T16:20:03.802 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:20:03.802 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:20:03.850 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.849+0000 7fc847fb3980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.849+0000 7fc847fb3980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:03.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.849+0000 7fc847fb3980 20 realm 2024-05-30T16:20:03.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.849+0000 7fc847fb3980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.853 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.849+0000 7fc847fb3980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:03.853 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.849+0000 7fc847fb3980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:03.853 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.849+0000 7fc847fb3980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.853 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.849+0000 7fc847fb3980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:03.853 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.849+0000 7fc847fb3980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.854 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.853+0000 7fc847fb3980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:03.854 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.853+0000 7fc847fb3980 20 rados->read ofs=0 len=0 2024-05-30T16:20:03.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.853+0000 7fc847fb3980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:03.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.853+0000 7fc847fb3980 20 searching for the correct realm 2024-05-30T16:20:03.862 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:03.861+0000 7fc8462c5640 0 -- 172.21.15.6:0/1399870700 >> [v2:172.21.15.204:6800/3687190393,v1:172.21.15.204:6801/3687190393] conn(0x7fc830040440 msgr2=0x7fc8300408c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:04.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.077+0000 7fc845ac4640 0 -- 172.21.15.6:0/1399870700 >> [v2:172.21.15.204:6816/2677861390,v1:172.21.15.204:6817/2677861390] conn(0x55c688af4ad0 msgr2=0x55c688af6f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:04.087 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.085+0000 7fc847fb3980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:04.087 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.085+0000 7fc847fb3980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:04.087 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.085+0000 7fc847fb3980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:04.087 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.085+0000 7fc847fb3980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:04.087 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.085+0000 7fc847fb3980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:04.087 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.085+0000 7fc847fb3980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:04.087 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.085+0000 7fc847fb3980 20 rados->read ofs=0 len=0 2024-05-30T16:20:04.088 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.085+0000 7fc847fb3980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:04.088 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.085+0000 7fc847fb3980 20 rados->read ofs=0 len=0 2024-05-30T16:20:04.088 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.085+0000 7fc847fb3980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:04.089 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.085+0000 7fc847fb3980 20 rados->read ofs=0 len=0 2024-05-30T16:20:04.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.089+0000 7fc847fb3980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:04.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.089+0000 7fc847fb3980 20 zone default found 2024-05-30T16:20:04.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.089+0000 7fc847fb3980 4 Realm: () 2024-05-30T16:20:04.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.089+0000 7fc847fb3980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:04.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.089+0000 7fc847fb3980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:04.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.089+0000 7fc847fb3980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:04.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.089+0000 7fc847fb3980 20 zonegroup default 2024-05-30T16:20:04.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.089+0000 7fc847fb3980 20 rados->read ofs=0 len=0 2024-05-30T16:20:04.091 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.089+0000 7fc847fb3980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:04.091 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.089+0000 7fc847fb3980 20 rados->read ofs=0 len=0 2024-05-30T16:20:04.092 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.089+0000 7fc847fb3980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:04.092 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.089+0000 7fc847fb3980 20 rados->read ofs=0 len=0 2024-05-30T16:20:04.092 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.089+0000 7fc847fb3980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:04.092 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.089+0000 7fc847fb3980 20 started sync module instance, tier type = 2024-05-30T16:20:04.092 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.089+0000 7fc847fb3980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:04.115 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.113+0000 7fc847fb3980 20 add_watcher() i=0 2024-05-30T16:20:04.115 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.113+0000 7fc847fb3980 20 add_watcher() i=1 2024-05-30T16:20:04.116 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.113+0000 7fc847fb3980 20 add_watcher() i=2 2024-05-30T16:20:04.116 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.113+0000 7fc847fb3980 20 add_watcher() i=3 2024-05-30T16:20:04.116 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.113+0000 7fc847fb3980 20 add_watcher() i=4 2024-05-30T16:20:04.116 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.113+0000 7fc847fb3980 20 add_watcher() i=5 2024-05-30T16:20:04.116 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.113+0000 7fc847fb3980 20 add_watcher() i=6 2024-05-30T16:20:04.118 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.117+0000 7fc847fb3980 20 add_watcher() i=7 2024-05-30T16:20:04.118 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.117+0000 7fc847fb3980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:04.121 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.121+0000 7fc6f77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:04.122 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.121+0000 7fc847fb3980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:04.122 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.121+0000 7fc847fb3980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:04.122 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.121+0000 7fc847fb3980 5 note: GC not initialized 2024-05-30T16:20:04.123 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.121+0000 7fc6f67fc640 20 reqs_thread_entry: start 2024-05-30T16:20:04.123 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.121+0000 7fc847fb3980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:04.123 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.121+0000 7fc847fb3980 20 Filter name: none 2024-05-30T16:20:04.123 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.121+0000 7fc66bfff640 20 reqs_thread_entry: start 2024-05-30T16:20:04.123 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.121+0000 7fc847fb3980 20 rados->read ofs=0 len=0 2024-05-30T16:20:04.123 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.121+0000 7fc847fb3980 20 rados_obj.operate() r=0 bl.length=200 2024-05-30T16:20:04.124 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.121+0000 7fc847fb3980 20 rados->read ofs=0 len=0 2024-05-30T16:20:04.125 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.121+0000 7fc847fb3980 20 rados_obj.operate() r=0 bl.length=414 2024-05-30T16:20:04.125 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.121+0000 7fc847fb3980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:04.125 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.121+0000 7fc847fb3980 20 rados->read ofs=0 len=0 2024-05-30T16:20:04.125 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.125+0000 7fc847fb3980 20 rados_obj.operate() r=0 bl.length=200 2024-05-30T16:20:04.125 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.125+0000 7fc847fb3980 20 rados->read ofs=0 len=0 2024-05-30T16:20:04.126 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.125+0000 7fc847fb3980 20 rados_obj.operate() r=0 bl.length=414 2024-05-30T16:20:04.126 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.125+0000 7fc847fb3980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:04.129 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.125+0000 7fc6f77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:04.134 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.133+0000 7fc847fb3980 20 remove_watcher() i=0 2024-05-30T16:20:04.134 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.133+0000 7fc847fb3980 2 removed watcher, disabling cache 2024-05-30T16:20:04.136 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.133+0000 7fc847fb3980 20 remove_watcher() i=1 2024-05-30T16:20:04.138 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.137+0000 7fc847fb3980 20 remove_watcher() i=2 2024-05-30T16:20:04.141 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.137+0000 7fc847fb3980 20 remove_watcher() i=3 2024-05-30T16:20:04.143 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.141+0000 7fc847fb3980 20 remove_watcher() i=4 2024-05-30T16:20:04.145 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.141+0000 7fc847fb3980 20 remove_watcher() i=5 2024-05-30T16:20:04.146 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.145+0000 7fc847fb3980 20 remove_watcher() i=6 2024-05-30T16:20:04.148 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:04.145+0000 7fc847fb3980 20 remove_watcher() i=7 2024-05-30T16:20:04.163 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:04.163 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:20:04.164 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: a-bucket id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 num_objects: 8 size_kb: 1 num_shards: 13 2024-05-30T16:20:04.164 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:04.164 INFO:tasks.workunit.client.0.smithi006.stderr:TEST: reshard bucket with EIO injected at set_target_layout 2024-05-30T16:20:04.164 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:06.223 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-05-30T16:20:06.257 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:20:06.257 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:20:06.303 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.301+0000 7f891a12f980 20 rados->read ofs=0 len=0 2024-05-30T16:20:06.304 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.301+0000 7f891a12f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:06.304 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.301+0000 7f891a12f980 20 realm 2024-05-30T16:20:06.304 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.301+0000 7f891a12f980 20 rados->read ofs=0 len=0 2024-05-30T16:20:06.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.301+0000 7f891a12f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:06.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.301+0000 7f891a12f980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:06.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.301+0000 7f891a12f980 20 rados->read ofs=0 len=0 2024-05-30T16:20:06.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.301+0000 7f891a12f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:06.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.301+0000 7f891a12f980 20 rados->read ofs=0 len=0 2024-05-30T16:20:06.307 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.305+0000 7f891a12f980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:06.307 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.305+0000 7f891a12f980 20 rados->read ofs=0 len=0 2024-05-30T16:20:06.307 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.305+0000 7f891a12f980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:06.307 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.305+0000 7f891a12f980 20 searching for the correct realm 2024-05-30T16:20:06.313 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.313+0000 7f891743f640 0 -- 172.21.15.6:0/490692284 >> [v2:172.21.15.6:6819/1141406589,v1:172.21.15.6:6821/1141406589] conn(0x5646c94b2f30 msgr2=0x5646c94b5320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:06.320 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.317+0000 7f8917c40640 0 -- 172.21.15.6:0/490692284 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f890c05ce80 msgr2=0x7f890c0900b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:06.328 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.325+0000 7f891a12f980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:06.328 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.325+0000 7f891a12f980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:06.328 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.325+0000 7f891a12f980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:06.328 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.325+0000 7f891a12f980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:06.328 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.325+0000 7f891a12f980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:06.328 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.325+0000 7f891a12f980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:06.329 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.325+0000 7f891a12f980 20 rados->read ofs=0 len=0 2024-05-30T16:20:06.329 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.325+0000 7f891a12f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:06.329 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.325+0000 7f891a12f980 20 rados->read ofs=0 len=0 2024-05-30T16:20:06.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.325+0000 7f891a12f980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:06.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.325+0000 7f891a12f980 20 rados->read ofs=0 len=0 2024-05-30T16:20:06.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.329+0000 7f891a12f980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:06.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.329+0000 7f891a12f980 20 zone default found 2024-05-30T16:20:06.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.329+0000 7f891a12f980 4 Realm: () 2024-05-30T16:20:06.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.329+0000 7f891a12f980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:06.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.329+0000 7f891a12f980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:06.331 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.329+0000 7f891a12f980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:06.331 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.329+0000 7f891a12f980 20 zonegroup default 2024-05-30T16:20:06.331 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.329+0000 7f891a12f980 20 rados->read ofs=0 len=0 2024-05-30T16:20:06.331 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.329+0000 7f891a12f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:06.331 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.329+0000 7f891a12f980 20 rados->read ofs=0 len=0 2024-05-30T16:20:06.332 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.329+0000 7f891a12f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:06.332 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.329+0000 7f891a12f980 20 rados->read ofs=0 len=0 2024-05-30T16:20:06.332 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.329+0000 7f891a12f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:06.332 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.329+0000 7f891a12f980 20 started sync module instance, tier type = 2024-05-30T16:20:06.332 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.329+0000 7f891a12f980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:06.347 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.345+0000 7f891a12f980 20 add_watcher() i=0 2024-05-30T16:20:06.347 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.345+0000 7f891a12f980 20 add_watcher() i=1 2024-05-30T16:20:06.347 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.345+0000 7f891a12f980 20 add_watcher() i=2 2024-05-30T16:20:06.347 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.345+0000 7f891a12f980 20 add_watcher() i=3 2024-05-30T16:20:06.347 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.345+0000 7f891a12f980 20 add_watcher() i=4 2024-05-30T16:20:06.347 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.345+0000 7f891a12f980 20 add_watcher() i=5 2024-05-30T16:20:06.348 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.345+0000 7f891a12f980 20 add_watcher() i=6 2024-05-30T16:20:06.348 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.345+0000 7f891a12f980 20 add_watcher() i=7 2024-05-30T16:20:06.348 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.345+0000 7f891a12f980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:06.351 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.349+0000 7f87cdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:06.352 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.349+0000 7f891a12f980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:06.352 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.349+0000 7f891a12f980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:06.352 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.349+0000 7f891a12f980 5 note: GC not initialized 2024-05-30T16:20:06.352 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.349+0000 7f87ccfe1640 20 reqs_thread_entry: start 2024-05-30T16:20:06.352 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.349+0000 7f891a12f980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:06.352 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.349+0000 7f891a12f980 20 Filter name: none 2024-05-30T16:20:06.352 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.349+0000 7f87467fc640 20 reqs_thread_entry: start 2024-05-30T16:20:06.352 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.349+0000 7f891a12f980 20 rados->read ofs=0 len=0 2024-05-30T16:20:06.353 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.349+0000 7f891a12f980 20 rados_obj.operate() r=0 bl.length=218 2024-05-30T16:20:06.353 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.349+0000 7f891a12f980 20 rados->read ofs=0 len=0 2024-05-30T16:20:06.354 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.353+0000 7f891a12f980 20 rados_obj.operate() r=0 bl.length=432 2024-05-30T16:20:06.354 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.353+0000 7f891a12f980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:06.354 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.353+0000 7f891a12f980 20 rados->read ofs=0 len=0 2024-05-30T16:20:06.355 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.353+0000 7f891a12f980 20 rados_obj.operate() r=0 bl.length=218 2024-05-30T16:20:06.355 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.353+0000 7f891a12f980 20 rados->read ofs=0 len=0 2024-05-30T16:20:06.355 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.353+0000 7f891a12f980 20 rados_obj.operate() r=0 bl.length=432 2024-05-30T16:20:06.355 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.353+0000 7f891a12f980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:06.358 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.357+0000 7f87cdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:06.363 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.361+0000 7f891a12f980 20 remove_watcher() i=0 2024-05-30T16:20:06.363 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.361+0000 7f891a12f980 2 removed watcher, disabling cache 2024-05-30T16:20:06.365 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.361+0000 7f891a12f980 20 remove_watcher() i=1 2024-05-30T16:20:06.367 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.365+0000 7f891a12f980 20 remove_watcher() i=2 2024-05-30T16:20:06.369 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.369+0000 7f891a12f980 20 remove_watcher() i=3 2024-05-30T16:20:06.371 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.369+0000 7f891a12f980 20 remove_watcher() i=4 2024-05-30T16:20:06.373 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.369+0000 7f891a12f980 20 remove_watcher() i=5 2024-05-30T16:20:06.374 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.373+0000 7f891a12f980 20 remove_watcher() i=6 2024-05-30T16:20:06.375 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.373+0000 7f891a12f980 20 remove_watcher() i=7 2024-05-30T16:20:06.390 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:06.390 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:20:06.390 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: error-at-set-target-layout id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-30T16:20:06.390 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:06.391 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket reshard --bucket error-at-set-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 --inject-error-at set_target_layout 2024-05-30T16:20:06.422 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:20:06.422 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:20:06.468 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.465+0000 7f6462d50980 20 rados->read ofs=0 len=0 2024-05-30T16:20:06.470 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.469+0000 7f6462d50980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:06.470 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.469+0000 7f6462d50980 20 realm 2024-05-30T16:20:06.470 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.469+0000 7f6462d50980 20 rados->read ofs=0 len=0 2024-05-30T16:20:06.470 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.469+0000 7f6462d50980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:06.470 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.469+0000 7f6462d50980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:06.470 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.469+0000 7f6462d50980 20 rados->read ofs=0 len=0 2024-05-30T16:20:06.471 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.469+0000 7f6462d50980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:06.471 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.469+0000 7f6462d50980 20 rados->read ofs=0 len=0 2024-05-30T16:20:06.472 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.469+0000 7f6462d50980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:06.472 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.469+0000 7f6462d50980 20 rados->read ofs=0 len=0 2024-05-30T16:20:06.472 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.469+0000 7f6462d50980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:06.472 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.469+0000 7f6462d50980 20 searching for the correct realm 2024-05-30T16:20:06.480 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.477+0000 7f6461062640 0 -- 172.21.15.6:0/1574740549 >> [v2:172.21.15.204:6800/3687190393,v1:172.21.15.204:6801/3687190393] conn(0x7f6454069040 msgr2=0x7f64540739e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:20:06.695 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.693+0000 7f6460861640 0 -- 172.21.15.6:0/1574740549 >> [v2:172.21.15.204:6816/2677861390,v1:172.21.15.204:6817/2677861390] conn(0x563995b9fff0 msgr2=0x563995ba2420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:06.903 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.901+0000 7f6462d50980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:06.903 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.901+0000 7f6462d50980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:06.903 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.901+0000 7f6462d50980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:06.903 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.901+0000 7f6462d50980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:06.903 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.901+0000 7f6462d50980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:06.904 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.901+0000 7f6462d50980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:06.904 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.901+0000 7f6462d50980 20 rados->read ofs=0 len=0 2024-05-30T16:20:06.904 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.901+0000 7f6462d50980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:06.904 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.901+0000 7f6462d50980 20 rados->read ofs=0 len=0 2024-05-30T16:20:06.904 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.901+0000 7f6462d50980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:06.904 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.901+0000 7f6462d50980 20 rados->read ofs=0 len=0 2024-05-30T16:20:06.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.905+0000 7f6462d50980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:06.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.905+0000 7f6462d50980 20 zone default found 2024-05-30T16:20:06.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.905+0000 7f6462d50980 4 Realm: () 2024-05-30T16:20:06.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.905+0000 7f6462d50980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:06.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.905+0000 7f6462d50980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:06.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.905+0000 7f6462d50980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:06.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.905+0000 7f6462d50980 20 zonegroup default 2024-05-30T16:20:06.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.905+0000 7f6462d50980 20 rados->read ofs=0 len=0 2024-05-30T16:20:06.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.905+0000 7f6462d50980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:06.907 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.905+0000 7f6462d50980 20 rados->read ofs=0 len=0 2024-05-30T16:20:06.907 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.905+0000 7f6462d50980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:06.907 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.905+0000 7f6462d50980 20 rados->read ofs=0 len=0 2024-05-30T16:20:06.908 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.905+0000 7f6462d50980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:06.908 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.905+0000 7f6462d50980 20 started sync module instance, tier type = 2024-05-30T16:20:06.908 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.905+0000 7f6462d50980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:06.935 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.933+0000 7f6462d50980 20 add_watcher() i=0 2024-05-30T16:20:06.935 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.933+0000 7f6462d50980 20 add_watcher() i=1 2024-05-30T16:20:06.935 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.933+0000 7f6462d50980 20 add_watcher() i=2 2024-05-30T16:20:06.935 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.933+0000 7f6462d50980 20 add_watcher() i=3 2024-05-30T16:20:06.935 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.933+0000 7f6462d50980 20 add_watcher() i=4 2024-05-30T16:20:06.935 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.933+0000 7f6462d50980 20 add_watcher() i=5 2024-05-30T16:20:06.935 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.933+0000 7f6462d50980 20 add_watcher() i=6 2024-05-30T16:20:06.937 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.933+0000 7f6462d50980 20 add_watcher() i=7 2024-05-30T16:20:06.937 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.933+0000 7f6462d50980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:06.941 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.941+0000 7f6316fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:06.942 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.941+0000 7f6462d50980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:06.942 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.941+0000 7f6462d50980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:06.942 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.941+0000 7f6462d50980 5 note: GC not initialized 2024-05-30T16:20:06.943 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.941+0000 7f6315fe3640 20 reqs_thread_entry: start 2024-05-30T16:20:06.943 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.941+0000 7f6462d50980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:06.944 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.941+0000 7f6462d50980 20 Filter name: none 2024-05-30T16:20:06.944 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.941+0000 7f62877fe640 20 reqs_thread_entry: start 2024-05-30T16:20:06.944 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.941+0000 7f6462d50980 10 cache get: name=default.rgw.meta+root+error-at-set-target-layout : miss 2024-05-30T16:20:06.944 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.941+0000 7f6462d50980 20 rados->read ofs=0 len=0 2024-05-30T16:20:06.944 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.941+0000 7f6462d50980 20 rados_obj.operate() r=0 bl.length=218 2024-05-30T16:20:06.945 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.941+0000 7f6462d50980 10 cache put: name=default.rgw.meta+root+error-at-set-target-layout info.flags=0x11 2024-05-30T16:20:06.945 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.941+0000 7f6462d50980 10 adding default.rgw.meta+root+error-at-set-target-layout to cache LRU end 2024-05-30T16:20:06.945 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.941+0000 7f6462d50980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20 : miss 2024-05-30T16:20:06.945 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.941+0000 7f6462d50980 20 rados->read ofs=0 len=0 2024-05-30T16:20:06.946 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.945+0000 7f6462d50980 20 rados_obj.operate() r=0 bl.length=432 2024-05-30T16:20:06.946 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.945+0000 7f6462d50980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20 info.flags=0x17 2024-05-30T16:20:06.946 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.945+0000 7f6462d50980 10 adding default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20 to cache LRU end 2024-05-30T16:20:06.946 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.945+0000 7f6462d50980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:20:06.947 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.945+0000 7f6462d50980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:06.947 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.945+0000 7f6462d50980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20 2024-05-30T16:20:06.954 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.953+0000 7f6462d50980 -1 Injecting error=-5 at location=set_target_layout 2024-05-30T16:20:06.954 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.953+0000 7f6462d50980 0 ERROR: init_target_layout failed to write target index layout to bucket info: (5) Input/output error 2024-05-30T16:20:06.962 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.961+0000 7f6316fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:06.967 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.965+0000 7f6462d50980 20 remove_watcher() i=0 2024-05-30T16:20:06.968 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.965+0000 7f6462d50980 2 removed watcher, disabling cache 2024-05-30T16:20:06.970 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.969+0000 7f6462d50980 20 remove_watcher() i=1 2024-05-30T16:20:06.972 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.969+0000 7f6462d50980 20 remove_watcher() i=2 2024-05-30T16:20:06.974 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.973+0000 7f6462d50980 20 remove_watcher() i=3 2024-05-30T16:20:06.976 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.973+0000 7f6462d50980 20 remove_watcher() i=4 2024-05-30T16:20:06.978 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.977+0000 7f6462d50980 20 remove_watcher() i=5 2024-05-30T16:20:06.980 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.977+0000 7f6462d50980 20 remove_watcher() i=6 2024-05-30T16:20:06.981 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:06.981+0000 7f6462d50980 20 remove_watcher() i=7 2024-05-30T16:20:06.999 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-05-30T16:20:07.026 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:20:07.026 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:20:07.073 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.069+0000 7f129a3c0980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.075 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.073+0000 7f129a3c0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:07.075 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.073+0000 7f129a3c0980 20 realm 2024-05-30T16:20:07.075 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.073+0000 7f129a3c0980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.075 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.073+0000 7f129a3c0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:07.075 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.073+0000 7f129a3c0980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:07.075 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.073+0000 7f129a3c0980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.075 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.073+0000 7f129a3c0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:07.075 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.073+0000 7f129a3c0980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.076 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.073+0000 7f129a3c0980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:07.077 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.073+0000 7f129a3c0980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.077 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.077+0000 7f129a3c0980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:07.077 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.077+0000 7f129a3c0980 20 searching for the correct realm 2024-05-30T16:20:07.083 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.081+0000 7f12986d2640 0 -- 172.21.15.6:0/2974983402 >> [v2:172.21.15.6:6808/1897362715,v1:172.21.15.6:6809/1897362715] conn(0x556aaba2bba0 msgr2=0x556aaba2dfd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:07.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.089+0000 7f1297ed1640 0 -- 172.21.15.6:0/2974983402 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f128c0d03d0 msgr2=0x7f128c1097e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:07.308 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.305+0000 7f129a3c0980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:07.308 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.305+0000 7f129a3c0980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:07.308 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.305+0000 7f129a3c0980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:07.308 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.305+0000 7f129a3c0980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:07.308 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.305+0000 7f129a3c0980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:07.309 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.305+0000 7f129a3c0980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:07.309 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.305+0000 7f129a3c0980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.309 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.305+0000 7f129a3c0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:07.309 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.305+0000 7f129a3c0980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.309 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.309+0000 7f129a3c0980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:07.309 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.309+0000 7f129a3c0980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.310 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.309+0000 7f129a3c0980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:07.310 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.309+0000 7f129a3c0980 20 zone default found 2024-05-30T16:20:07.310 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.309+0000 7f129a3c0980 4 Realm: () 2024-05-30T16:20:07.310 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.309+0000 7f129a3c0980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:07.311 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.309+0000 7f129a3c0980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:07.311 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.309+0000 7f129a3c0980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:07.311 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.309+0000 7f129a3c0980 20 zonegroup default 2024-05-30T16:20:07.311 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.309+0000 7f129a3c0980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.311 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.309+0000 7f129a3c0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:07.311 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.309+0000 7f129a3c0980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.312 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.309+0000 7f129a3c0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:07.312 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.309+0000 7f129a3c0980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.313 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.309+0000 7f129a3c0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:07.313 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.309+0000 7f129a3c0980 20 started sync module instance, tier type = 2024-05-30T16:20:07.313 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.309+0000 7f129a3c0980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:07.338 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.337+0000 7f129a3c0980 20 add_watcher() i=0 2024-05-30T16:20:07.338 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.337+0000 7f129a3c0980 20 add_watcher() i=1 2024-05-30T16:20:07.339 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.337+0000 7f129a3c0980 20 add_watcher() i=2 2024-05-30T16:20:07.339 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.337+0000 7f129a3c0980 20 add_watcher() i=3 2024-05-30T16:20:07.339 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.337+0000 7f129a3c0980 20 add_watcher() i=4 2024-05-30T16:20:07.339 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.337+0000 7f129a3c0980 20 add_watcher() i=5 2024-05-30T16:20:07.339 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.337+0000 7f129a3c0980 20 add_watcher() i=6 2024-05-30T16:20:07.341 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.337+0000 7f129a3c0980 20 add_watcher() i=7 2024-05-30T16:20:07.341 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.337+0000 7f129a3c0980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:07.345 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.341+0000 7f114dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:07.346 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.345+0000 7f129a3c0980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:07.346 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.345+0000 7f129a3c0980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:07.346 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.345+0000 7f129a3c0980 5 note: GC not initialized 2024-05-30T16:20:07.347 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.345+0000 7f114cfe1640 20 reqs_thread_entry: start 2024-05-30T16:20:07.347 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.345+0000 7f129a3c0980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:07.347 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.345+0000 7f129a3c0980 20 Filter name: none 2024-05-30T16:20:07.348 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.345+0000 7f10c67fc640 20 reqs_thread_entry: start 2024-05-30T16:20:07.348 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.345+0000 7f129a3c0980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.348 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.345+0000 7f129a3c0980 20 rados_obj.operate() r=0 bl.length=218 2024-05-30T16:20:07.348 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.345+0000 7f129a3c0980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.350 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.349+0000 7f129a3c0980 20 rados_obj.operate() r=0 bl.length=432 2024-05-30T16:20:07.350 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.349+0000 7f129a3c0980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:07.350 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.349+0000 7f129a3c0980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.351 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.349+0000 7f129a3c0980 20 rados_obj.operate() r=0 bl.length=218 2024-05-30T16:20:07.351 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.349+0000 7f129a3c0980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.352 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.349+0000 7f129a3c0980 20 rados_obj.operate() r=0 bl.length=432 2024-05-30T16:20:07.352 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.349+0000 7f129a3c0980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:07.355 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.353+0000 7f114dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:07.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.357+0000 7f129a3c0980 20 remove_watcher() i=0 2024-05-30T16:20:07.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.357+0000 7f129a3c0980 2 removed watcher, disabling cache 2024-05-30T16:20:07.362 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.361+0000 7f129a3c0980 20 remove_watcher() i=1 2024-05-30T16:20:07.365 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.361+0000 7f129a3c0980 20 remove_watcher() i=2 2024-05-30T16:20:07.367 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.365+0000 7f129a3c0980 20 remove_watcher() i=3 2024-05-30T16:20:07.369 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.369+0000 7f129a3c0980 20 remove_watcher() i=4 2024-05-30T16:20:07.371 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.369+0000 7f129a3c0980 20 remove_watcher() i=5 2024-05-30T16:20:07.373 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.369+0000 7f129a3c0980 20 remove_watcher() i=6 2024-05-30T16:20:07.375 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.373+0000 7f129a3c0980 20 remove_watcher() i=7 2024-05-30T16:20:07.391 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:07.391 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:20:07.391 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: error-at-set-target-layout id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-30T16:20:07.392 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:07.495 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket reshard --bucket error-at-set-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 2024-05-30T16:20:07.531 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:20:07.531 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:20:07.577 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.573+0000 7fd7d90be980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.579 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.577+0000 7fd7d90be980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:07.579 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.577+0000 7fd7d90be980 20 realm 2024-05-30T16:20:07.579 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.577+0000 7fd7d90be980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.579 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.577+0000 7fd7d90be980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:07.579 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.577+0000 7fd7d90be980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:07.579 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.577+0000 7fd7d90be980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.579 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.577+0000 7fd7d90be980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:07.579 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.577+0000 7fd7d90be980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.581 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.577+0000 7fd7d90be980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:07.581 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.577+0000 7fd7d90be980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.581 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.581+0000 7fd7d90be980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:07.581 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.581+0000 7fd7d90be980 20 searching for the correct realm 2024-05-30T16:20:07.587 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.585+0000 7fd7d63ce640 0 -- 172.21.15.6:0/2687171578 >> [v2:172.21.15.6:6808/1897362715,v1:172.21.15.6:6809/1897362715] conn(0x56071dad3ac0 msgr2=0x56071dad5f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:07.593 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.593+0000 7fd7d6bcf640 0 -- 172.21.15.6:0/2687171578 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7fd7cc0203f0 msgr2=0x7fd7cc020870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:07.601 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.597+0000 7fd7d90be980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:07.601 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.597+0000 7fd7d90be980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:07.601 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.597+0000 7fd7d90be980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:07.601 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.597+0000 7fd7d90be980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:07.601 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.597+0000 7fd7d90be980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:07.601 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.597+0000 7fd7d90be980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:07.601 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.597+0000 7fd7d90be980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.601 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.597+0000 7fd7d90be980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:07.602 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.597+0000 7fd7d90be980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.602 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.601+0000 7fd7d90be980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:07.602 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.601+0000 7fd7d90be980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.601+0000 7fd7d90be980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:07.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.601+0000 7fd7d90be980 20 zone default found 2024-05-30T16:20:07.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.601+0000 7fd7d90be980 4 Realm: () 2024-05-30T16:20:07.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.601+0000 7fd7d90be980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:07.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.601+0000 7fd7d90be980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:07.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.601+0000 7fd7d90be980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:07.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.601+0000 7fd7d90be980 20 zonegroup default 2024-05-30T16:20:07.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.601+0000 7fd7d90be980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.601+0000 7fd7d90be980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:07.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.601+0000 7fd7d90be980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.604 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.601+0000 7fd7d90be980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:07.604 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.601+0000 7fd7d90be980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.604 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.601+0000 7fd7d90be980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:07.604 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.601+0000 7fd7d90be980 20 started sync module instance, tier type = 2024-05-30T16:20:07.604 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.601+0000 7fd7d90be980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:07.621 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.617+0000 7fd7d90be980 20 add_watcher() i=0 2024-05-30T16:20:07.621 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.617+0000 7fd7d90be980 20 add_watcher() i=1 2024-05-30T16:20:07.621 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.617+0000 7fd7d90be980 20 add_watcher() i=2 2024-05-30T16:20:07.621 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.617+0000 7fd7d90be980 20 add_watcher() i=3 2024-05-30T16:20:07.621 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.617+0000 7fd7d90be980 20 add_watcher() i=4 2024-05-30T16:20:07.622 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.617+0000 7fd7d90be980 20 add_watcher() i=5 2024-05-30T16:20:07.622 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.621+0000 7fd7d90be980 20 add_watcher() i=6 2024-05-30T16:20:07.622 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.621+0000 7fd7d90be980 20 add_watcher() i=7 2024-05-30T16:20:07.622 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.621+0000 7fd7d90be980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:07.625 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.621+0000 7fd68dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:07.626 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.625+0000 7fd7d90be980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:07.626 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.625+0000 7fd7d90be980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:07.626 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.625+0000 7fd7d90be980 5 note: GC not initialized 2024-05-30T16:20:07.626 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.625+0000 7fd68cfe1640 20 reqs_thread_entry: start 2024-05-30T16:20:07.626 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.625+0000 7fd7d90be980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:07.626 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.625+0000 7fd7d90be980 20 Filter name: none 2024-05-30T16:20:07.627 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.625+0000 7fd685ffb640 20 reqs_thread_entry: start 2024-05-30T16:20:07.627 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.625+0000 7fd7d90be980 10 cache get: name=default.rgw.meta+root+error-at-set-target-layout : miss 2024-05-30T16:20:07.627 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.625+0000 7fd7d90be980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.627 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.625+0000 7fd7d90be980 20 rados_obj.operate() r=0 bl.length=218 2024-05-30T16:20:07.627 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.625+0000 7fd7d90be980 10 cache put: name=default.rgw.meta+root+error-at-set-target-layout info.flags=0x11 2024-05-30T16:20:07.627 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.625+0000 7fd7d90be980 10 adding default.rgw.meta+root+error-at-set-target-layout to cache LRU end 2024-05-30T16:20:07.627 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.625+0000 7fd7d90be980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20 : miss 2024-05-30T16:20:07.627 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.625+0000 7fd7d90be980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.628 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.625+0000 7fd7d90be980 20 rados_obj.operate() r=0 bl.length=432 2024-05-30T16:20:07.628 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.625+0000 7fd7d90be980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20 info.flags=0x17 2024-05-30T16:20:07.628 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.625+0000 7fd7d90be980 10 adding default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20 to cache LRU end 2024-05-30T16:20:07.628 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.625+0000 7fd7d90be980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:20:07.628 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.625+0000 7fd7d90be980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:07.628 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.625+0000 7fd7d90be980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20 2024-05-30T16:20:07.638 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.637+0000 7fd7d90be980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20 info.flags=0x17 2024-05-30T16:20:07.638 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.637+0000 7fd7d90be980 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20 to cache LRU end 2024-05-30T16:20:07.639 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.637+0000 7fd7d90be980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:07.639 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.637+0000 7fd7d90be980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20, ofs0, ns] 2024-05-30T16:20:07.639 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.637+0000 7fd6b9ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431617 cookie 94588562701776 notifier 5930 bl.length()=1001 2024-05-30T16:20:07.639 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.637+0000 7fd6b9ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20 info.flags=0x17 2024-05-30T16:20:07.639 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.637+0000 7fd6b9ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20 to cache LRU end 2024-05-30T16:20:07.639 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.637+0000 7fd6b9ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:07.642 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.641+0000 7fd7d90be980 20 handle_bi_update(): bucket=:error-at-set-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20]): orig_sources= new_sources= 2024-05-30T16:20:07.642 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.641+0000 7fd7d90be980 20 handle_bi_update(): bucket=:error-at-set-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20]): potential sources added=[] removed=[] 2024-05-30T16:20:07.642 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.641+0000 7fd7d90be980 20 handle_bi_update(): bucket=:error-at-set-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20]): orig_dests= new_dests= 2024-05-30T16:20:07.642 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.641+0000 7fd7d90be980 20 handle_bi_update(): bucket=:error-at-set-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20]): potential dests added=[] removed=[] 2024-05-30T16:20:07.647 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.645+0000 7fd7d90be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.1.0 2024-05-30T16:20:07.647 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.645+0000 7fd7d90be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.1.0 2024-05-30T16:20:07.647 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.645+0000 7fd7d90be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.1.1 2024-05-30T16:20:07.647 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.645+0000 7fd7d90be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.1.1 2024-05-30T16:20:07.647 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.645+0000 7fd7d90be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.1.2 2024-05-30T16:20:07.647 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.645+0000 7fd7d90be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.1.2 2024-05-30T16:20:07.647 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.645+0000 7fd7d90be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.1.3 2024-05-30T16:20:07.647 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.645+0000 7fd7d90be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.1.3 2024-05-30T16:20:07.647 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.645+0000 7fd7d90be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.1.4 2024-05-30T16:20:07.648 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.645+0000 7fd7d90be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.1.4 2024-05-30T16:20:07.648 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.645+0000 7fd7d90be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.1.5 2024-05-30T16:20:07.648 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.645+0000 7fd7d90be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.1.5 2024-05-30T16:20:07.648 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.645+0000 7fd7d90be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.1.6 2024-05-30T16:20:07.648 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.645+0000 7fd7d90be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.1.6 2024-05-30T16:20:07.648 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.645+0000 7fd7d90be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.1.7 2024-05-30T16:20:07.648 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.645+0000 7fd7d90be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.1.7 2024-05-30T16:20:07.648 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.645+0000 7fd7d90be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.1.8 2024-05-30T16:20:07.648 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.645+0000 7fd7d90be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.1.8 2024-05-30T16:20:07.648 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.645+0000 7fd7d90be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.1.9 2024-05-30T16:20:07.648 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.645+0000 7fd7d90be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.1.9 2024-05-30T16:20:07.648 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.645+0000 7fd7d90be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.1.10 2024-05-30T16:20:07.648 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.645+0000 7fd7d90be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.1.10 2024-05-30T16:20:07.649 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.645+0000 7fd7d90be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.1.11 2024-05-30T16:20:07.649 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.645+0000 7fd7d90be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.1.11 2024-05-30T16:20:07.649 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.645+0000 7fd7d90be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.0 2024-05-30T16:20:07.649 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.645+0000 7fd7d90be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.1 2024-05-30T16:20:07.649 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.645+0000 7fd7d90be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.2 2024-05-30T16:20:07.649 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.645+0000 7fd7d90be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.3 2024-05-30T16:20:07.650 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.649+0000 7fd7d90be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.4 2024-05-30T16:20:07.650 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.649+0000 7fd7d90be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.5 2024-05-30T16:20:07.651 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.649+0000 7fd7d90be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.6 2024-05-30T16:20:07.653 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.649+0000 7fd7d90be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.7 2024-05-30T16:20:07.654 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.653+0000 7fd7d90be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.8 2024-05-30T16:20:07.654 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.653+0000 7fd7d90be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.9 2024-05-30T16:20:07.655 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.653+0000 7fd7d90be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20.10 2024-05-30T16:20:07.662 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.661+0000 7fd7d90be980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20 : hit (requested=0x17, cached=0x17) 2024-05-30T16:20:07.662 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.661+0000 7fd7d90be980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20 2024-05-30T16:20:07.666 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.661+0000 7fd7d90be980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20 info.flags=0x17 2024-05-30T16:20:07.666 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.661+0000 7fd7d90be980 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20 to cache LRU end 2024-05-30T16:20:07.666 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.661+0000 7fd7d90be980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:07.666 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.661+0000 7fd7d90be980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20, ofs0, ns] 2024-05-30T16:20:07.666 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.665+0000 7fd6b9ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431618 cookie 94588562701776 notifier 5930 bl.length()=969 2024-05-30T16:20:07.666 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.665+0000 7fd6b9ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20 info.flags=0x17 2024-05-30T16:20:07.667 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.665+0000 7fd6b9ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20 to cache LRU end 2024-05-30T16:20:07.667 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.665+0000 7fd6b9ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:07.667 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.665+0000 7fd7d90be980 20 handle_bi_update(): bucket=:error-at-set-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20]): orig_sources= new_sources= 2024-05-30T16:20:07.668 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.665+0000 7fd7d90be980 20 handle_bi_update(): bucket=:error-at-set-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20]): potential sources added=[] removed=[] 2024-05-30T16:20:07.668 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.665+0000 7fd7d90be980 20 handle_bi_update(): bucket=:error-at-set-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20]): orig_dests= new_dests= 2024-05-30T16:20:07.668 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.665+0000 7fd7d90be980 20 handle_bi_update(): bucket=:error-at-set-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20]): potential dests added=[] removed=[] 2024-05-30T16:20:07.673 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.669+0000 7fd7d90be980 1 execute INFO: reshard of bucket "error-at-set-target-layout" completed successfully 2024-05-30T16:20:07.676 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.673+0000 7fd68dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:07.681 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.677+0000 7fd7d90be980 20 remove_watcher() i=0 2024-05-30T16:20:07.681 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.677+0000 7fd7d90be980 2 removed watcher, disabling cache 2024-05-30T16:20:07.683 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.681+0000 7fd7d90be980 20 remove_watcher() i=1 2024-05-30T16:20:07.685 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.685+0000 7fd7d90be980 20 remove_watcher() i=2 2024-05-30T16:20:07.687 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.685+0000 7fd7d90be980 20 remove_watcher() i=3 2024-05-30T16:20:07.689 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.689+0000 7fd7d90be980 20 remove_watcher() i=4 2024-05-30T16:20:07.691 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.689+0000 7fd7d90be980 20 remove_watcher() i=5 2024-05-30T16:20:07.692 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.689+0000 7fd7d90be980 20 remove_watcher() i=6 2024-05-30T16:20:07.693 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.693+0000 7fd7d90be980 20 remove_watcher() i=7 2024-05-30T16:20:07.708 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-05-30T16:20:07.734 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:20:07.734 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:20:07.778 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.777+0000 7fd687ace980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.780 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.777+0000 7fd687ace980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:07.780 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.777+0000 7fd687ace980 20 realm 2024-05-30T16:20:07.780 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.777+0000 7fd687ace980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.780 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.777+0000 7fd687ace980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:07.781 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.777+0000 7fd687ace980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:07.781 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.777+0000 7fd687ace980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.781 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.777+0000 7fd687ace980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:07.781 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.777+0000 7fd687ace980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.782 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.781+0000 7fd687ace980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:07.782 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.781+0000 7fd687ace980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.783 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.781+0000 7fd687ace980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:07.783 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.781+0000 7fd687ace980 20 searching for the correct realm 2024-05-30T16:20:07.789 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.789+0000 7fd684dde640 0 -- 172.21.15.6:0/1326681694 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7fd6781330e0 msgr2=0x7fd678007730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:07.795 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.793+0000 7fd684dde640 0 -- 172.21.15.6:0/1326681694 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7fd55007e510 msgr2=0x7fd550080940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:07.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.801+0000 7fd687ace980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:07.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.801+0000 7fd687ace980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:07.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.801+0000 7fd687ace980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:07.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.801+0000 7fd687ace980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:07.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.801+0000 7fd687ace980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:07.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.801+0000 7fd687ace980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:07.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.801+0000 7fd687ace980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.801+0000 7fd687ace980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:07.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.801+0000 7fd687ace980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.801+0000 7fd687ace980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:07.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.801+0000 7fd687ace980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.804 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.801+0000 7fd687ace980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:07.804 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.801+0000 7fd687ace980 20 zone default found 2024-05-30T16:20:07.804 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.801+0000 7fd687ace980 4 Realm: () 2024-05-30T16:20:07.804 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.801+0000 7fd687ace980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:07.804 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.801+0000 7fd687ace980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:07.804 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.801+0000 7fd687ace980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:07.804 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.801+0000 7fd687ace980 20 zonegroup default 2024-05-30T16:20:07.804 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.801+0000 7fd687ace980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.801+0000 7fd687ace980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:07.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.801+0000 7fd687ace980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.801+0000 7fd687ace980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:07.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.801+0000 7fd687ace980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.805+0000 7fd687ace980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:07.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.805+0000 7fd687ace980 20 started sync module instance, tier type = 2024-05-30T16:20:07.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.805+0000 7fd687ace980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:07.823 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.821+0000 7fd687ace980 20 add_watcher() i=0 2024-05-30T16:20:07.823 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.821+0000 7fd687ace980 20 add_watcher() i=1 2024-05-30T16:20:07.823 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.821+0000 7fd687ace980 20 add_watcher() i=2 2024-05-30T16:20:07.823 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.821+0000 7fd687ace980 20 add_watcher() i=3 2024-05-30T16:20:07.824 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.821+0000 7fd687ace980 20 add_watcher() i=4 2024-05-30T16:20:07.824 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.821+0000 7fd687ace980 20 add_watcher() i=5 2024-05-30T16:20:07.824 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.821+0000 7fd687ace980 20 add_watcher() i=6 2024-05-30T16:20:07.824 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.821+0000 7fd687ace980 20 add_watcher() i=7 2024-05-30T16:20:07.824 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.821+0000 7fd687ace980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:07.827 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.825+0000 7fd53b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:07.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.825+0000 7fd687ace980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:07.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.825+0000 7fd687ace980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:07.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.825+0000 7fd687ace980 5 note: GC not initialized 2024-05-30T16:20:07.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.825+0000 7fd53a7e4640 20 reqs_thread_entry: start 2024-05-30T16:20:07.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.825+0000 7fd687ace980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:07.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.825+0000 7fd687ace980 20 Filter name: none 2024-05-30T16:20:07.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.825+0000 7fd4abfff640 20 reqs_thread_entry: start 2024-05-30T16:20:07.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.825+0000 7fd687ace980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.829 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.825+0000 7fd687ace980 20 rados_obj.operate() r=0 bl.length=218 2024-05-30T16:20:07.829 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.825+0000 7fd687ace980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.829 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.829+0000 7fd687ace980 20 rados_obj.operate() r=0 bl.length=432 2024-05-30T16:20:07.830 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.829+0000 7fd687ace980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:07.830 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.829+0000 7fd687ace980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.830 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.829+0000 7fd687ace980 20 rados_obj.operate() r=0 bl.length=218 2024-05-30T16:20:07.830 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.829+0000 7fd687ace980 20 rados->read ofs=0 len=0 2024-05-30T16:20:07.831 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.829+0000 7fd687ace980 20 rados_obj.operate() r=0 bl.length=432 2024-05-30T16:20:07.831 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.829+0000 7fd687ace980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:07.833 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.833+0000 7fd53b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:07.838 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.837+0000 7fd687ace980 20 remove_watcher() i=0 2024-05-30T16:20:07.838 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.837+0000 7fd687ace980 2 removed watcher, disabling cache 2024-05-30T16:20:07.840 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.837+0000 7fd687ace980 20 remove_watcher() i=1 2024-05-30T16:20:07.842 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.841+0000 7fd687ace980 20 remove_watcher() i=2 2024-05-30T16:20:07.845 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.841+0000 7fd687ace980 20 remove_watcher() i=3 2024-05-30T16:20:07.847 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.845+0000 7fd687ace980 20 remove_watcher() i=4 2024-05-30T16:20:07.848 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.845+0000 7fd687ace980 20 remove_watcher() i=5 2024-05-30T16:20:07.849 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.849+0000 7fd687ace980 20 remove_watcher() i=6 2024-05-30T16:20:07.850 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:07.849+0000 7fd687ace980 20 remove_watcher() i=7 2024-05-30T16:20:07.864 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:07.864 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:20:07.864 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: error-at-set-target-layout id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.20 num_objects: 19 size_kb: 1 num_shards: 12 2024-05-30T16:20:07.864 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:08.750 INFO:tasks.workunit.client.0.smithi006.stderr:TEST: reshard bucket with ECANCELED injected at set_target_layout 2024-05-30T16:20:08.751 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:10.839 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-05-30T16:20:10.873 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:20:10.873 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:20:10.919 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:10.917+0000 7ff0af76a980 20 rados->read ofs=0 len=0 2024-05-30T16:20:10.921 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:10.921+0000 7ff0af76a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:10.921 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:10.921+0000 7ff0af76a980 20 realm 2024-05-30T16:20:10.922 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:10.921+0000 7ff0af76a980 20 rados->read ofs=0 len=0 2024-05-30T16:20:10.922 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:10.921+0000 7ff0af76a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:10.922 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:10.921+0000 7ff0af76a980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:10.922 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:10.921+0000 7ff0af76a980 20 rados->read ofs=0 len=0 2024-05-30T16:20:10.922 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:10.921+0000 7ff0af76a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:10.922 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:10.921+0000 7ff0af76a980 20 rados->read ofs=0 len=0 2024-05-30T16:20:10.923 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:10.921+0000 7ff0af76a980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:10.924 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:10.921+0000 7ff0af76a980 20 rados->read ofs=0 len=0 2024-05-30T16:20:10.924 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:10.921+0000 7ff0af76a980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:10.924 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:10.921+0000 7ff0af76a980 20 searching for the correct realm 2024-05-30T16:20:10.928 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:10.925+0000 7ff0aca7a640 0 -- 172.21.15.6:0/2489507424 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7ff0a00d3070 msgr2=0x7ff0a00ed430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:11.143 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.141+0000 7ff0ad27b640 0 -- 172.21.15.6:0/2489507424 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7ff0a406c390 msgr2=0x7ff0a4061e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:11.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.357+0000 7ff0af76a980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:11.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.357+0000 7ff0af76a980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:11.361 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.357+0000 7ff0af76a980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:11.361 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.357+0000 7ff0af76a980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:11.361 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.357+0000 7ff0af76a980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:11.361 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.357+0000 7ff0af76a980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:11.361 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.357+0000 7ff0af76a980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.361 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.357+0000 7ff0af76a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:11.361 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.357+0000 7ff0af76a980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.362 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.361+0000 7ff0af76a980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:11.362 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.361+0000 7ff0af76a980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.367 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.361+0000 7ff0af76a980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:11.367 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.361+0000 7ff0af76a980 20 zone default found 2024-05-30T16:20:11.367 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.361+0000 7ff0af76a980 4 Realm: () 2024-05-30T16:20:11.367 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.361+0000 7ff0af76a980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:11.367 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.361+0000 7ff0af76a980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:11.367 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.361+0000 7ff0af76a980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:11.367 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.361+0000 7ff0af76a980 20 zonegroup default 2024-05-30T16:20:11.368 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.361+0000 7ff0af76a980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.368 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.361+0000 7ff0af76a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:11.368 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.361+0000 7ff0af76a980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.368 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.361+0000 7ff0af76a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:11.368 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.361+0000 7ff0af76a980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.368 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.365+0000 7ff0af76a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:11.368 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.365+0000 7ff0af76a980 20 started sync module instance, tier type = 2024-05-30T16:20:11.368 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.365+0000 7ff0af76a980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:11.393 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.393+0000 7ff0af76a980 20 add_watcher() i=0 2024-05-30T16:20:11.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.393+0000 7ff0af76a980 20 add_watcher() i=1 2024-05-30T16:20:11.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.393+0000 7ff0af76a980 20 add_watcher() i=2 2024-05-30T16:20:11.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.393+0000 7ff0af76a980 20 add_watcher() i=3 2024-05-30T16:20:11.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.393+0000 7ff0af76a980 20 add_watcher() i=4 2024-05-30T16:20:11.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.393+0000 7ff0af76a980 20 add_watcher() i=5 2024-05-30T16:20:11.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.393+0000 7ff0af76a980 20 add_watcher() i=6 2024-05-30T16:20:11.396 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.393+0000 7ff0af76a980 20 add_watcher() i=7 2024-05-30T16:20:11.396 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.393+0000 7ff0af76a980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:11.400 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.397+0000 7fef637e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:11.401 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.401+0000 7ff0af76a980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:11.401 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.401+0000 7ff0af76a980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:11.401 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.401+0000 7ff0af76a980 5 note: GC not initialized 2024-05-30T16:20:11.402 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.401+0000 7fef627e4640 20 reqs_thread_entry: start 2024-05-30T16:20:11.402 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.401+0000 7ff0af76a980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:11.402 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.401+0000 7ff0af76a980 20 Filter name: none 2024-05-30T16:20:11.403 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.401+0000 7feed3fff640 20 reqs_thread_entry: start 2024-05-30T16:20:11.403 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.401+0000 7ff0af76a980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.403 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.401+0000 7ff0af76a980 20 rados_obj.operate() r=0 bl.length=218 2024-05-30T16:20:11.404 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.401+0000 7ff0af76a980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.404 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.401+0000 7ff0af76a980 20 rados_obj.operate() r=0 bl.length=432 2024-05-30T16:20:11.405 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.401+0000 7ff0af76a980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:11.405 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.401+0000 7ff0af76a980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.405 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.401+0000 7ff0af76a980 20 rados_obj.operate() r=0 bl.length=218 2024-05-30T16:20:11.406 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.401+0000 7ff0af76a980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.406 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.405+0000 7ff0af76a980 20 rados_obj.operate() r=0 bl.length=432 2024-05-30T16:20:11.406 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.405+0000 7ff0af76a980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:11.408 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.405+0000 7fef637e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:11.414 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.413+0000 7ff0af76a980 20 remove_watcher() i=0 2024-05-30T16:20:11.414 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.413+0000 7ff0af76a980 2 removed watcher, disabling cache 2024-05-30T16:20:11.416 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.413+0000 7ff0af76a980 20 remove_watcher() i=1 2024-05-30T16:20:11.418 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.417+0000 7ff0af76a980 20 remove_watcher() i=2 2024-05-30T16:20:11.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.421+0000 7ff0af76a980 20 remove_watcher() i=3 2024-05-30T16:20:11.425 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.425+0000 7ff0af76a980 20 remove_watcher() i=4 2024-05-30T16:20:11.427 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.425+0000 7ff0af76a980 20 remove_watcher() i=5 2024-05-30T16:20:11.429 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.425+0000 7ff0af76a980 20 remove_watcher() i=6 2024-05-30T16:20:11.430 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.429+0000 7ff0af76a980 20 remove_watcher() i=7 2024-05-30T16:20:11.446 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:11.446 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:20:11.447 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: error-at-set-target-layout id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-30T16:20:11.447 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:11.447 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket reshard --bucket error-at-set-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 --inject-error-at set_target_layout --inject-error-code 125 2024-05-30T16:20:11.489 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:20:11.489 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:20:11.534 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.533+0000 7ff846976980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.533+0000 7ff846976980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:11.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.533+0000 7ff846976980 20 realm 2024-05-30T16:20:11.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.533+0000 7ff846976980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.533+0000 7ff846976980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:11.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.533+0000 7ff846976980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:11.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.533+0000 7ff846976980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.533+0000 7ff846976980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:11.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.533+0000 7ff846976980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.537 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.537+0000 7ff846976980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:11.537 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.537+0000 7ff846976980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.538 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.537+0000 7ff846976980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:11.538 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.537+0000 7ff846976980 20 searching for the correct realm 2024-05-30T16:20:11.544 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.541+0000 7ff844c88640 0 -- 172.21.15.6:0/4123319158 >> [v2:172.21.15.6:6808/1897362715,v1:172.21.15.6:6809/1897362715] conn(0x5555f0ff3680 msgr2=0x5555f0ff5ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:11.550 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.549+0000 7ff844487640 0 -- 172.21.15.6:0/4123319158 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7ff82c030410 msgr2=0x7ff82c030890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:11.558 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.557+0000 7ff846976980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:11.559 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.557+0000 7ff846976980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:11.559 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.557+0000 7ff846976980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:11.559 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.557+0000 7ff846976980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:11.559 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.557+0000 7ff846976980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:11.559 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.557+0000 7ff846976980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:11.559 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.557+0000 7ff846976980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.559 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.557+0000 7ff846976980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:11.559 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.557+0000 7ff846976980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.560 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.557+0000 7ff846976980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:11.560 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.557+0000 7ff846976980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.557+0000 7ff846976980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:11.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.557+0000 7ff846976980 20 zone default found 2024-05-30T16:20:11.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.557+0000 7ff846976980 4 Realm: () 2024-05-30T16:20:11.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.557+0000 7ff846976980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:11.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.557+0000 7ff846976980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:11.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.557+0000 7ff846976980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:11.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.557+0000 7ff846976980 20 zonegroup default 2024-05-30T16:20:11.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.557+0000 7ff846976980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.557+0000 7ff846976980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:11.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.557+0000 7ff846976980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.562 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.561+0000 7ff846976980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:11.562 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.561+0000 7ff846976980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.562 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.561+0000 7ff846976980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:11.562 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.561+0000 7ff846976980 20 started sync module instance, tier type = 2024-05-30T16:20:11.562 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.561+0000 7ff846976980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:11.577 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.573+0000 7ff846976980 20 add_watcher() i=0 2024-05-30T16:20:11.577 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.573+0000 7ff846976980 20 add_watcher() i=1 2024-05-30T16:20:11.577 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.573+0000 7ff846976980 20 add_watcher() i=2 2024-05-30T16:20:11.577 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.573+0000 7ff846976980 20 add_watcher() i=3 2024-05-30T16:20:11.577 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.573+0000 7ff846976980 20 add_watcher() i=4 2024-05-30T16:20:11.577 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.573+0000 7ff846976980 20 add_watcher() i=5 2024-05-30T16:20:11.577 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.573+0000 7ff846976980 20 add_watcher() i=6 2024-05-30T16:20:11.578 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.577+0000 7ff846976980 20 add_watcher() i=7 2024-05-30T16:20:11.578 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.577+0000 7ff846976980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:11.580 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.577+0000 7ff6fafdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:11.581 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.577+0000 7ff846976980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:11.582 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.577+0000 7ff846976980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:11.582 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.577+0000 7ff846976980 5 note: GC not initialized 2024-05-30T16:20:11.582 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.577+0000 7ff6f9fdb640 20 reqs_thread_entry: start 2024-05-30T16:20:11.582 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.581+0000 7ff846976980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:11.582 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.581+0000 7ff846976980 20 Filter name: none 2024-05-30T16:20:11.582 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.581+0000 7ff66b7fe640 20 reqs_thread_entry: start 2024-05-30T16:20:11.582 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.581+0000 7ff846976980 10 cache get: name=default.rgw.meta+root+error-at-set-target-layout : miss 2024-05-30T16:20:11.582 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.581+0000 7ff846976980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.583 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.581+0000 7ff846976980 20 rados_obj.operate() r=0 bl.length=218 2024-05-30T16:20:11.583 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.581+0000 7ff846976980 10 cache put: name=default.rgw.meta+root+error-at-set-target-layout info.flags=0x11 2024-05-30T16:20:11.583 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.581+0000 7ff846976980 10 adding default.rgw.meta+root+error-at-set-target-layout to cache LRU end 2024-05-30T16:20:11.583 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.581+0000 7ff846976980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21 : miss 2024-05-30T16:20:11.583 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.581+0000 7ff846976980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.583 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.581+0000 7ff846976980 20 rados_obj.operate() r=0 bl.length=432 2024-05-30T16:20:11.584 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.581+0000 7ff846976980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21 info.flags=0x17 2024-05-30T16:20:11.584 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.581+0000 7ff846976980 10 adding default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21 to cache LRU end 2024-05-30T16:20:11.584 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.581+0000 7ff846976980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:20:11.584 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.581+0000 7ff846976980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:11.584 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.581+0000 7ff846976980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21 2024-05-30T16:20:11.592 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.589+0000 7ff846976980 -1 Injecting error=-125 at location=set_target_layout 2024-05-30T16:20:11.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.593+0000 7ff846976980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21 info.flags=0x17 2024-05-30T16:20:11.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.593+0000 7ff846976980 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21 to cache LRU end 2024-05-30T16:20:11.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.593+0000 7ff846976980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:11.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.593+0000 7ff846976980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21, ofs0, ns] 2024-05-30T16:20:11.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.593+0000 7ff71affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431621 cookie 93827603888656 notifier 5981 bl.length()=1001 2024-05-30T16:20:11.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.593+0000 7ff71affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21 info.flags=0x17 2024-05-30T16:20:11.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.593+0000 7ff71affd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21 to cache LRU end 2024-05-30T16:20:11.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.593+0000 7ff71affd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:11.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.597+0000 7ff846976980 20 handle_bi_update(): bucket=:error-at-set-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21]): orig_sources= new_sources= 2024-05-30T16:20:11.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.597+0000 7ff846976980 20 handle_bi_update(): bucket=:error-at-set-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21]): potential sources added=[] removed=[] 2024-05-30T16:20:11.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.597+0000 7ff846976980 20 handle_bi_update(): bucket=:error-at-set-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21]): orig_dests= new_dests= 2024-05-30T16:20:11.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.597+0000 7ff846976980 20 handle_bi_update(): bucket=:error-at-set-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21]): potential dests added=[] removed=[] 2024-05-30T16:20:11.602 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.601+0000 7ff846976980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.1.0 2024-05-30T16:20:11.602 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.601+0000 7ff846976980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.1.0 2024-05-30T16:20:11.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.601+0000 7ff846976980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.1.1 2024-05-30T16:20:11.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.601+0000 7ff846976980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.1.1 2024-05-30T16:20:11.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.601+0000 7ff846976980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.1.2 2024-05-30T16:20:11.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.601+0000 7ff846976980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.1.2 2024-05-30T16:20:11.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.601+0000 7ff846976980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.1.3 2024-05-30T16:20:11.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.601+0000 7ff846976980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.1.3 2024-05-30T16:20:11.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.601+0000 7ff846976980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.1.4 2024-05-30T16:20:11.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.601+0000 7ff846976980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.1.4 2024-05-30T16:20:11.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.601+0000 7ff846976980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.1.5 2024-05-30T16:20:11.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.601+0000 7ff846976980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.1.5 2024-05-30T16:20:11.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.601+0000 7ff846976980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.1.6 2024-05-30T16:20:11.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.601+0000 7ff846976980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.1.6 2024-05-30T16:20:11.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.601+0000 7ff846976980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.1.7 2024-05-30T16:20:11.604 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.601+0000 7ff846976980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.1.7 2024-05-30T16:20:11.604 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.601+0000 7ff846976980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.1.8 2024-05-30T16:20:11.604 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.601+0000 7ff846976980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.1.8 2024-05-30T16:20:11.604 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.601+0000 7ff846976980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.1.9 2024-05-30T16:20:11.604 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.601+0000 7ff846976980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.1.9 2024-05-30T16:20:11.604 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.601+0000 7ff846976980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.1.10 2024-05-30T16:20:11.604 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.601+0000 7ff846976980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.1.10 2024-05-30T16:20:11.604 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.601+0000 7ff846976980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.1.11 2024-05-30T16:20:11.604 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.601+0000 7ff846976980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.1.11 2024-05-30T16:20:11.604 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.601+0000 7ff846976980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.0 2024-05-30T16:20:11.605 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.601+0000 7ff846976980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.1 2024-05-30T16:20:11.605 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.601+0000 7ff846976980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.2 2024-05-30T16:20:11.605 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.605+0000 7ff846976980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.3 2024-05-30T16:20:11.606 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.605+0000 7ff846976980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.4 2024-05-30T16:20:11.607 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.605+0000 7ff846976980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.5 2024-05-30T16:20:11.607 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.605+0000 7ff846976980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.6 2024-05-30T16:20:11.608 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.605+0000 7ff846976980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.7 2024-05-30T16:20:11.608 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.605+0000 7ff846976980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.8 2024-05-30T16:20:11.609 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.605+0000 7ff846976980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.9 2024-05-30T16:20:11.610 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.609+0000 7ff846976980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21.10 2024-05-30T16:20:11.619 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.617+0000 7ff846976980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21 : hit (requested=0x17, cached=0x17) 2024-05-30T16:20:11.619 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.617+0000 7ff846976980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21 2024-05-30T16:20:11.622 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.621+0000 7ff846976980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21 info.flags=0x17 2024-05-30T16:20:11.622 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.621+0000 7ff846976980 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21 to cache LRU end 2024-05-30T16:20:11.622 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.621+0000 7ff846976980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:11.622 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.621+0000 7ff846976980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21, ofs0, ns] 2024-05-30T16:20:11.623 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.621+0000 7ff71b7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431622 cookie 93827603888656 notifier 5981 bl.length()=969 2024-05-30T16:20:11.623 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.621+0000 7ff71b7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21 info.flags=0x17 2024-05-30T16:20:11.623 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.621+0000 7ff71b7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21 to cache LRU end 2024-05-30T16:20:11.623 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.621+0000 7ff71b7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:11.625 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.625+0000 7ff846976980 20 handle_bi_update(): bucket=:error-at-set-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21]): orig_sources= new_sources= 2024-05-30T16:20:11.625 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.625+0000 7ff846976980 20 handle_bi_update(): bucket=:error-at-set-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21]): potential sources added=[] removed=[] 2024-05-30T16:20:11.625 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.625+0000 7ff846976980 20 handle_bi_update(): bucket=:error-at-set-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21]): orig_dests= new_dests= 2024-05-30T16:20:11.625 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.625+0000 7ff846976980 20 handle_bi_update(): bucket=:error-at-set-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21]): potential dests added=[] removed=[] 2024-05-30T16:20:11.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.629+0000 7ff846976980 1 execute INFO: reshard of bucket "error-at-set-target-layout" completed successfully 2024-05-30T16:20:11.632 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.629+0000 7ff6fafdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:11.637 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.637+0000 7ff846976980 20 remove_watcher() i=0 2024-05-30T16:20:11.637 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.637+0000 7ff846976980 2 removed watcher, disabling cache 2024-05-30T16:20:11.639 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.637+0000 7ff846976980 20 remove_watcher() i=1 2024-05-30T16:20:11.642 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.641+0000 7ff846976980 20 remove_watcher() i=2 2024-05-30T16:20:11.644 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.641+0000 7ff846976980 20 remove_watcher() i=3 2024-05-30T16:20:11.646 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.645+0000 7ff846976980 20 remove_watcher() i=4 2024-05-30T16:20:11.647 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.645+0000 7ff846976980 20 remove_watcher() i=5 2024-05-30T16:20:11.649 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.645+0000 7ff846976980 20 remove_watcher() i=6 2024-05-30T16:20:11.650 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.649+0000 7ff846976980 20 remove_watcher() i=7 2024-05-30T16:20:11.665 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-05-30T16:20:11.692 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:20:11.693 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:20:11.737 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.733+0000 7f1d9abdc980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.739 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.737+0000 7f1d9abdc980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:11.739 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.737+0000 7f1d9abdc980 20 realm 2024-05-30T16:20:11.739 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.737+0000 7f1d9abdc980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.739 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.737+0000 7f1d9abdc980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:11.739 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.737+0000 7f1d9abdc980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:11.740 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.737+0000 7f1d9abdc980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.740 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.737+0000 7f1d9abdc980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:11.740 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.737+0000 7f1d9abdc980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.741 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.741+0000 7f1d9abdc980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:11.741 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.741+0000 7f1d9abdc980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.742 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.741+0000 7f1d9abdc980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:11.742 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.741+0000 7f1d9abdc980 20 searching for the correct realm 2024-05-30T16:20:11.750 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.749+0000 7f1d98eee640 0 -- 172.21.15.6:0/2969110573 >> [v2:172.21.15.204:6800/3687190393,v1:172.21.15.204:6801/3687190393] conn(0x7f1d8c01e410 msgr2=0x7f1d8c01e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:11.967 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.965+0000 7f1d986ed640 0 -- 172.21.15.6:0/2969110573 >> [v2:172.21.15.204:6816/2677861390,v1:172.21.15.204:6817/2677861390] conn(0x56380decab20 msgr2=0x56380deccf10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:11.980 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.973+0000 7f1d9abdc980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:11.980 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.973+0000 7f1d9abdc980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:11.980 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.973+0000 7f1d9abdc980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:11.980 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.973+0000 7f1d9abdc980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:11.980 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.973+0000 7f1d9abdc980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:11.981 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.973+0000 7f1d9abdc980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:11.981 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.973+0000 7f1d9abdc980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.981 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.973+0000 7f1d9abdc980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:11.981 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.973+0000 7f1d9abdc980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.981 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.973+0000 7f1d9abdc980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:11.981 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.973+0000 7f1d9abdc980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.981 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.977+0000 7f1d9abdc980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:11.981 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.977+0000 7f1d9abdc980 20 zone default found 2024-05-30T16:20:11.981 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.977+0000 7f1d9abdc980 4 Realm: () 2024-05-30T16:20:11.981 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.977+0000 7f1d9abdc980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:11.982 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.977+0000 7f1d9abdc980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:11.982 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.977+0000 7f1d9abdc980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:11.982 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.977+0000 7f1d9abdc980 20 zonegroup default 2024-05-30T16:20:11.982 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.977+0000 7f1d9abdc980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.982 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.977+0000 7f1d9abdc980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:11.982 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.977+0000 7f1d9abdc980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.982 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.977+0000 7f1d9abdc980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:11.982 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.977+0000 7f1d9abdc980 20 rados->read ofs=0 len=0 2024-05-30T16:20:11.983 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.977+0000 7f1d9abdc980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:11.983 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.977+0000 7f1d9abdc980 20 started sync module instance, tier type = 2024-05-30T16:20:11.983 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:11.977+0000 7f1d9abdc980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:12.004 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.001+0000 7f1d9abdc980 20 add_watcher() i=0 2024-05-30T16:20:12.004 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.001+0000 7f1d9abdc980 20 add_watcher() i=1 2024-05-30T16:20:12.004 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.001+0000 7f1d9abdc980 20 add_watcher() i=2 2024-05-30T16:20:12.004 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.001+0000 7f1d9abdc980 20 add_watcher() i=3 2024-05-30T16:20:12.004 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.001+0000 7f1d9abdc980 20 add_watcher() i=4 2024-05-30T16:20:12.004 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.001+0000 7f1d9abdc980 20 add_watcher() i=5 2024-05-30T16:20:12.004 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.001+0000 7f1d9abdc980 20 add_watcher() i=6 2024-05-30T16:20:12.006 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.005+0000 7f1d9abdc980 20 add_watcher() i=7 2024-05-30T16:20:12.006 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.005+0000 7f1d9abdc980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:12.010 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.009+0000 7f1c4efdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:12.011 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.009+0000 7f1d9abdc980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:12.011 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.009+0000 7f1d9abdc980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:12.011 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.009+0000 7f1d9abdc980 5 note: GC not initialized 2024-05-30T16:20:12.012 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.009+0000 7f1c4dfdb640 20 reqs_thread_entry: start 2024-05-30T16:20:12.012 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.009+0000 7f1d9abdc980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:12.012 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.009+0000 7f1d9abdc980 20 Filter name: none 2024-05-30T16:20:12.013 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.009+0000 7f1bbf7fe640 20 reqs_thread_entry: start 2024-05-30T16:20:12.013 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.009+0000 7f1d9abdc980 20 rados->read ofs=0 len=0 2024-05-30T16:20:12.013 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.009+0000 7f1d9abdc980 20 rados_obj.operate() r=0 bl.length=218 2024-05-30T16:20:12.013 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.009+0000 7f1d9abdc980 20 rados->read ofs=0 len=0 2024-05-30T16:20:12.014 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.009+0000 7f1d9abdc980 20 rados_obj.operate() r=0 bl.length=432 2024-05-30T16:20:12.014 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.009+0000 7f1d9abdc980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:12.015 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.013+0000 7f1d9abdc980 20 rados->read ofs=0 len=0 2024-05-30T16:20:12.015 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.013+0000 7f1d9abdc980 20 rados_obj.operate() r=0 bl.length=218 2024-05-30T16:20:12.015 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.013+0000 7f1d9abdc980 20 rados->read ofs=0 len=0 2024-05-30T16:20:12.015 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.013+0000 7f1d9abdc980 20 rados_obj.operate() r=0 bl.length=432 2024-05-30T16:20:12.015 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.013+0000 7f1d9abdc980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:12.018 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.017+0000 7f1c4efdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:12.023 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.021+0000 7f1d9abdc980 20 remove_watcher() i=0 2024-05-30T16:20:12.023 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.021+0000 7f1d9abdc980 2 removed watcher, disabling cache 2024-05-30T16:20:12.025 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.025+0000 7f1d9abdc980 20 remove_watcher() i=1 2024-05-30T16:20:12.027 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.025+0000 7f1d9abdc980 20 remove_watcher() i=2 2024-05-30T16:20:12.029 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.029+0000 7f1d9abdc980 20 remove_watcher() i=3 2024-05-30T16:20:12.032 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.029+0000 7f1d9abdc980 20 remove_watcher() i=4 2024-05-30T16:20:12.033 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.029+0000 7f1d9abdc980 20 remove_watcher() i=5 2024-05-30T16:20:12.034 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.033+0000 7f1d9abdc980 20 remove_watcher() i=6 2024-05-30T16:20:12.036 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:12.033+0000 7f1d9abdc980 20 remove_watcher() i=7 2024-05-30T16:20:12.051 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:12.052 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:20:12.052 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: error-at-set-target-layout id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.21 num_objects: 20 size_kb: 1 num_shards: 12 2024-05-30T16:20:12.052 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:12.984 INFO:tasks.workunit.client.0.smithi006.stderr:TEST: reshard bucket with abort at set_target_layout 2024-05-30T16:20:12.984 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:15.032 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket abort-at-set-target-layout 2024-05-30T16:20:15.066 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:20:15.066 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:20:15.112 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.109+0000 7ff6f943a980 20 rados->read ofs=0 len=0 2024-05-30T16:20:15.114 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.113+0000 7ff6f943a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:15.114 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.113+0000 7ff6f943a980 20 realm 2024-05-30T16:20:15.114 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.113+0000 7ff6f943a980 20 rados->read ofs=0 len=0 2024-05-30T16:20:15.115 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.113+0000 7ff6f943a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:15.115 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.113+0000 7ff6f943a980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:15.115 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.113+0000 7ff6f943a980 20 rados->read ofs=0 len=0 2024-05-30T16:20:15.115 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.113+0000 7ff6f943a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:15.115 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.113+0000 7ff6f943a980 20 rados->read ofs=0 len=0 2024-05-30T16:20:15.116 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.113+0000 7ff6f943a980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:15.116 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.113+0000 7ff6f943a980 20 rados->read ofs=0 len=0 2024-05-30T16:20:15.117 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.113+0000 7ff6f943a980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:15.117 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.113+0000 7ff6f943a980 20 searching for the correct realm 2024-05-30T16:20:15.120 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.117+0000 7ff6f674a640 0 -- 172.21.15.6:0/886256966 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7ff6e80e5040 msgr2=0x7ff6e80e4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:15.335 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.333+0000 7ff6f6f4b640 0 -- 172.21.15.6:0/886256966 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7ff6ec032920 msgr2=0x7ff6ec029400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:15.553 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.549+0000 7ff6f943a980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:15.553 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.549+0000 7ff6f943a980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:15.553 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.549+0000 7ff6f943a980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:15.553 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.549+0000 7ff6f943a980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:15.554 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.549+0000 7ff6f943a980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:15.554 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.549+0000 7ff6f943a980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:15.554 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.549+0000 7ff6f943a980 20 rados->read ofs=0 len=0 2024-05-30T16:20:15.554 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.553+0000 7ff6f943a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:15.554 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.553+0000 7ff6f943a980 20 rados->read ofs=0 len=0 2024-05-30T16:20:15.555 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.553+0000 7ff6f943a980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:15.555 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.553+0000 7ff6f943a980 20 rados->read ofs=0 len=0 2024-05-30T16:20:15.556 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.553+0000 7ff6f943a980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:15.556 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.553+0000 7ff6f943a980 20 zone default found 2024-05-30T16:20:15.556 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.553+0000 7ff6f943a980 4 Realm: () 2024-05-30T16:20:15.557 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.553+0000 7ff6f943a980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:15.557 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.553+0000 7ff6f943a980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:15.557 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.553+0000 7ff6f943a980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:15.557 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.553+0000 7ff6f943a980 20 zonegroup default 2024-05-30T16:20:15.557 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.553+0000 7ff6f943a980 20 rados->read ofs=0 len=0 2024-05-30T16:20:15.557 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.553+0000 7ff6f943a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:15.558 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.553+0000 7ff6f943a980 20 rados->read ofs=0 len=0 2024-05-30T16:20:15.558 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.557+0000 7ff6f943a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:15.558 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.557+0000 7ff6f943a980 20 rados->read ofs=0 len=0 2024-05-30T16:20:15.559 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.557+0000 7ff6f943a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:15.559 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.557+0000 7ff6f943a980 20 started sync module instance, tier type = 2024-05-30T16:20:15.559 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.557+0000 7ff6f943a980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:15.584 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.581+0000 7ff6f943a980 20 add_watcher() i=0 2024-05-30T16:20:15.584 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.581+0000 7ff6f943a980 20 add_watcher() i=1 2024-05-30T16:20:15.584 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.581+0000 7ff6f943a980 20 add_watcher() i=2 2024-05-30T16:20:15.584 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.581+0000 7ff6f943a980 20 add_watcher() i=3 2024-05-30T16:20:15.584 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.581+0000 7ff6f943a980 20 add_watcher() i=4 2024-05-30T16:20:15.585 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.581+0000 7ff6f943a980 20 add_watcher() i=5 2024-05-30T16:20:15.585 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.581+0000 7ff6f943a980 20 add_watcher() i=6 2024-05-30T16:20:15.586 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.585+0000 7ff6f943a980 20 add_watcher() i=7 2024-05-30T16:20:15.587 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.585+0000 7ff6f943a980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:15.590 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.589+0000 7ff5ad7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:15.591 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.589+0000 7ff6f943a980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:15.591 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.589+0000 7ff6f943a980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:15.592 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.589+0000 7ff6f943a980 5 note: GC not initialized 2024-05-30T16:20:15.592 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.589+0000 7ff527fff640 20 reqs_thread_entry: start 2024-05-30T16:20:15.592 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.589+0000 7ff6f943a980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:15.593 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.589+0000 7ff6f943a980 20 Filter name: none 2024-05-30T16:20:15.593 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.589+0000 7ff525ffb640 20 reqs_thread_entry: start 2024-05-30T16:20:15.593 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.589+0000 7ff6f943a980 20 rados->read ofs=0 len=0 2024-05-30T16:20:15.594 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.593+0000 7ff6f943a980 20 rados_obj.operate() r=0 bl.length=218 2024-05-30T16:20:15.594 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.593+0000 7ff6f943a980 20 rados->read ofs=0 len=0 2024-05-30T16:20:15.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.593+0000 7ff6f943a980 20 rados_obj.operate() r=0 bl.length=432 2024-05-30T16:20:15.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.593+0000 7ff6f943a980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:15.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.593+0000 7ff6f943a980 20 rados->read ofs=0 len=0 2024-05-30T16:20:15.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.593+0000 7ff6f943a980 20 rados_obj.operate() r=0 bl.length=218 2024-05-30T16:20:15.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.593+0000 7ff6f943a980 20 rados->read ofs=0 len=0 2024-05-30T16:20:15.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.593+0000 7ff6f943a980 20 rados_obj.operate() r=0 bl.length=432 2024-05-30T16:20:15.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.593+0000 7ff6f943a980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:15.600 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.597+0000 7ff5ad7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:15.604 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.601+0000 7ff6f943a980 20 remove_watcher() i=0 2024-05-30T16:20:15.604 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.601+0000 7ff6f943a980 2 removed watcher, disabling cache 2024-05-30T16:20:15.607 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.605+0000 7ff6f943a980 20 remove_watcher() i=1 2024-05-30T16:20:15.609 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.605+0000 7ff6f943a980 20 remove_watcher() i=2 2024-05-30T16:20:15.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.609+0000 7ff6f943a980 20 remove_watcher() i=3 2024-05-30T16:20:15.613 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.613+0000 7ff6f943a980 20 remove_watcher() i=4 2024-05-30T16:20:15.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.613+0000 7ff6f943a980 20 remove_watcher() i=5 2024-05-30T16:20:15.616 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.613+0000 7ff6f943a980 20 remove_watcher() i=6 2024-05-30T16:20:15.618 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.617+0000 7ff6f943a980 20 remove_watcher() i=7 2024-05-30T16:20:15.633 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:15.633 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:20:15.633 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: abort-at-set-target-layout id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-30T16:20:15.634 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:15.634 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket reshard --bucket abort-at-set-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at set_target_layout 2024-05-30T16:20:15.671 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:20:15.671 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:20:15.715 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.713+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:15.717 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.713+0000 7fbc52746980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:15.717 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.713+0000 7fbc52746980 20 realm 2024-05-30T16:20:15.717 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.713+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:15.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.717+0000 7fbc52746980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:15.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.717+0000 7fbc52746980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:15.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.717+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:15.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.717+0000 7fbc52746980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:15.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.717+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:15.719 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.717+0000 7fbc52746980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:15.719 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.717+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:15.720 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.717+0000 7fbc52746980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:15.720 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.717+0000 7fbc52746980 20 searching for the correct realm 2024-05-30T16:20:15.727 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.725+0000 7fbc50a58640 0 -- 172.21.15.6:0/1138350149 >> [v2:172.21.15.204:6800/3687190393,v1:172.21.15.204:6801/3687190393] conn(0x7fbc380e4040 msgr2=0x7fbc380e3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:15.943 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.940+0000 7fbc50257640 0 -- 172.21.15.6:0/1138350149 >> [v2:172.21.15.204:6816/2677861390,v1:172.21.15.204:6817/2677861390] conn(0x559468f78f40 msgr2=0x559468f7b330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:15.951 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.948+0000 7fbc52746980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:15.951 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.948+0000 7fbc52746980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:15.951 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.948+0000 7fbc52746980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:15.951 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.948+0000 7fbc52746980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:15.951 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.948+0000 7fbc52746980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:15.951 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.948+0000 7fbc52746980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:15.951 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.948+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:15.952 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.948+0000 7fbc52746980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:15.952 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.948+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:15.953 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.948+0000 7fbc52746980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:15.953 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.948+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:15.954 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.952+0000 7fbc52746980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:15.954 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.952+0000 7fbc52746980 20 zone default found 2024-05-30T16:20:15.954 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.952+0000 7fbc52746980 4 Realm: () 2024-05-30T16:20:15.954 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.952+0000 7fbc52746980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:15.954 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.952+0000 7fbc52746980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:15.954 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.952+0000 7fbc52746980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:15.954 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.952+0000 7fbc52746980 20 zonegroup default 2024-05-30T16:20:15.954 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.952+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:15.955 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.952+0000 7fbc52746980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:15.955 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.952+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:15.956 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.952+0000 7fbc52746980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:15.956 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.952+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:15.956 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.952+0000 7fbc52746980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:15.956 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.952+0000 7fbc52746980 20 started sync module instance, tier type = 2024-05-30T16:20:15.956 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.952+0000 7fbc52746980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:15.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.976+0000 7fbc52746980 20 add_watcher() i=0 2024-05-30T16:20:15.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.976+0000 7fbc52746980 20 add_watcher() i=1 2024-05-30T16:20:15.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.976+0000 7fbc52746980 20 add_watcher() i=2 2024-05-30T16:20:15.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.976+0000 7fbc52746980 20 add_watcher() i=3 2024-05-30T16:20:15.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.976+0000 7fbc52746980 20 add_watcher() i=4 2024-05-30T16:20:15.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.976+0000 7fbc52746980 20 add_watcher() i=5 2024-05-30T16:20:15.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.976+0000 7fbc52746980 20 add_watcher() i=6 2024-05-30T16:20:15.981 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.976+0000 7fbc52746980 20 add_watcher() i=7 2024-05-30T16:20:15.981 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.976+0000 7fbc52746980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:15.986 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.984+0000 7fbb06fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:15.986 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.984+0000 7fbc52746980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:15.986 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.984+0000 7fbc52746980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:15.987 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.984+0000 7fbc52746980 5 note: GC not initialized 2024-05-30T16:20:15.987 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.984+0000 7fbb05fdb640 20 reqs_thread_entry: start 2024-05-30T16:20:15.987 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.984+0000 7fbc52746980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:15.987 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.984+0000 7fbc52746980 20 Filter name: none 2024-05-30T16:20:15.988 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.984+0000 7fba777fe640 20 reqs_thread_entry: start 2024-05-30T16:20:15.988 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.984+0000 7fbc52746980 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-05-30T16:20:15.988 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.984+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:15.988 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.984+0000 7fbc52746980 20 rados_obj.operate() r=0 bl.length=218 2024-05-30T16:20:15.989 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.984+0000 7fbc52746980 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-05-30T16:20:15.989 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.984+0000 7fbc52746980 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-05-30T16:20:15.989 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.984+0000 7fbc52746980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 : miss 2024-05-30T16:20:15.989 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.984+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:15.989 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.984+0000 7fbc52746980 20 rados_obj.operate() r=0 bl.length=432 2024-05-30T16:20:15.989 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.984+0000 7fbc52746980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 info.flags=0x17 2024-05-30T16:20:15.989 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.984+0000 7fbc52746980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 to cache LRU end 2024-05-30T16:20:15.990 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.984+0000 7fbc52746980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:20:15.990 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.984+0000 7fbc52746980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:15.990 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.984+0000 7fbc52746980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 2024-05-30T16:20:15.998 INFO:tasks.workunit.client.0.smithi006.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fbc52746980 time 2024-05-30T16:20:16.001268+0000 2024-05-30T16:20:15.998 INFO:tasks.workunit.client.0.smithi006.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-30T16:20:15.999 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:20:15.999 INFO:tasks.workunit.client.0.smithi006.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fbc5601dcff] 2024-05-30T16:20:15.999 INFO:tasks.workunit.client.0.smithi006.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7fbc5601deb3] 2024-05-30T16:20:15.999 INFO:tasks.workunit.client.0.smithi006.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x5594675b5f81] 2024-05-30T16:20:15.999 INFO:tasks.workunit.client.0.smithi006.stderr: 4: radosgw-admin(+0x9a792a) [0x5594675c392a] 2024-05-30T16:20:15.999 INFO:tasks.workunit.client.0.smithi006.stderr: 5: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24f) [0x5594675c7e4f] 2024-05-30T16:20:15.999 INFO:tasks.workunit.client.0.smithi006.stderr: 6: main() 2024-05-30T16:20:16.000 INFO:tasks.workunit.client.0.smithi006.stderr: 7: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fbc54e86d90] 2024-05-30T16:20:16.000 INFO:tasks.workunit.client.0.smithi006.stderr: 8: __libc_start_main() 2024-05-30T16:20:16.000 INFO:tasks.workunit.client.0.smithi006.stderr: 9: _start() 2024-05-30T16:20:16.000 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.996+0000 7fbc52746980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fbc52746980 time 2024-05-30T16:20:16.001268+0000 2024-05-30T16:20:16.000 INFO:tasks.workunit.client.0.smithi006.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-30T16:20:16.000 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.000 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:20:16.000 INFO:tasks.workunit.client.0.smithi006.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fbc5601dcff] 2024-05-30T16:20:16.000 INFO:tasks.workunit.client.0.smithi006.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7fbc5601deb3] 2024-05-30T16:20:16.000 INFO:tasks.workunit.client.0.smithi006.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x5594675b5f81] 2024-05-30T16:20:16.000 INFO:tasks.workunit.client.0.smithi006.stderr: 4: radosgw-admin(+0x9a792a) [0x5594675c392a] 2024-05-30T16:20:16.001 INFO:tasks.workunit.client.0.smithi006.stderr: 5: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24f) [0x5594675c7e4f] 2024-05-30T16:20:16.001 INFO:tasks.workunit.client.0.smithi006.stderr: 6: main() 2024-05-30T16:20:16.001 INFO:tasks.workunit.client.0.smithi006.stderr: 7: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fbc54e86d90] 2024-05-30T16:20:16.001 INFO:tasks.workunit.client.0.smithi006.stderr: 8: __libc_start_main() 2024-05-30T16:20:16.001 INFO:tasks.workunit.client.0.smithi006.stderr: 9: _start() 2024-05-30T16:20:16.001 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.001 INFO:tasks.workunit.client.0.smithi006.stderr:*** Caught signal (Aborted) ** 2024-05-30T16:20:16.001 INFO:tasks.workunit.client.0.smithi006.stderr: in thread 7fbc52746980 thread_name:radosgw-admin 2024-05-30T16:20:16.012 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:20:16.012 INFO:tasks.workunit.client.0.smithi006.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fbc54e9f520] 2024-05-30T16:20:16.012 INFO:tasks.workunit.client.0.smithi006.stderr: 2: pthread_kill() 2024-05-30T16:20:16.012 INFO:tasks.workunit.client.0.smithi006.stderr: 3: raise() 2024-05-30T16:20:16.012 INFO:tasks.workunit.client.0.smithi006.stderr: 4: abort() 2024-05-30T16:20:16.012 INFO:tasks.workunit.client.0.smithi006.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fbc5601dd59] 2024-05-30T16:20:16.012 INFO:tasks.workunit.client.0.smithi006.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7fbc5601deb3] 2024-05-30T16:20:16.012 INFO:tasks.workunit.client.0.smithi006.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x5594675b5f81] 2024-05-30T16:20:16.012 INFO:tasks.workunit.client.0.smithi006.stderr: 8: radosgw-admin(+0x9a792a) [0x5594675c392a] 2024-05-30T16:20:16.012 INFO:tasks.workunit.client.0.smithi006.stderr: 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24f) [0x5594675c7e4f] 2024-05-30T16:20:16.013 INFO:tasks.workunit.client.0.smithi006.stderr: 10: main() 2024-05-30T16:20:16.013 INFO:tasks.workunit.client.0.smithi006.stderr: 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fbc54e86d90] 2024-05-30T16:20:16.013 INFO:tasks.workunit.client.0.smithi006.stderr: 12: __libc_start_main() 2024-05-30T16:20:16.013 INFO:tasks.workunit.client.0.smithi006.stderr: 13: _start() 2024-05-30T16:20:16.013 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:15.996+0000 7fbc52746980 -1 *** Caught signal (Aborted) ** 2024-05-30T16:20:16.013 INFO:tasks.workunit.client.0.smithi006.stderr: in thread 7fbc52746980 thread_name:radosgw-admin 2024-05-30T16:20:16.013 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.013 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:20:16.013 INFO:tasks.workunit.client.0.smithi006.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fbc54e9f520] 2024-05-30T16:20:16.013 INFO:tasks.workunit.client.0.smithi006.stderr: 2: pthread_kill() 2024-05-30T16:20:16.013 INFO:tasks.workunit.client.0.smithi006.stderr: 3: raise() 2024-05-30T16:20:16.013 INFO:tasks.workunit.client.0.smithi006.stderr: 4: abort() 2024-05-30T16:20:16.013 INFO:tasks.workunit.client.0.smithi006.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fbc5601dd59] 2024-05-30T16:20:16.014 INFO:tasks.workunit.client.0.smithi006.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7fbc5601deb3] 2024-05-30T16:20:16.014 INFO:tasks.workunit.client.0.smithi006.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x5594675b5f81] 2024-05-30T16:20:16.014 INFO:tasks.workunit.client.0.smithi006.stderr: 8: radosgw-admin(+0x9a792a) [0x5594675c392a] 2024-05-30T16:20:16.014 INFO:tasks.workunit.client.0.smithi006.stderr: 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24f) [0x5594675c7e4f] 2024-05-30T16:20:16.014 INFO:tasks.workunit.client.0.smithi006.stderr: 10: main() 2024-05-30T16:20:16.014 INFO:tasks.workunit.client.0.smithi006.stderr: 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fbc54e86d90] 2024-05-30T16:20:16.014 INFO:tasks.workunit.client.0.smithi006.stderr: 12: __libc_start_main() 2024-05-30T16:20:16.014 INFO:tasks.workunit.client.0.smithi006.stderr: 13: _start() 2024-05-30T16:20:16.014 INFO:tasks.workunit.client.0.smithi006.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-30T16:20:16.014 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.014 INFO:tasks.workunit.client.0.smithi006.stderr:--- begin dump of recent events --- 2024-05-30T16:20:16.014 INFO:tasks.workunit.client.0.smithi006.stderr: -499> 2024-05-30T16:20:15.689+0000 7fbc50a58640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:16.015 INFO:tasks.workunit.client.0.smithi006.stderr: -498> 2024-05-30T16:20:15.689+0000 7fbc4fa56640 10 monclient(hunting): get_auth_request con 0x559468dd6b20 auth_method 0 2024-05-30T16:20:16.015 INFO:tasks.workunit.client.0.smithi006.stderr: -497> 2024-05-30T16:20:15.689+0000 7fbc4fa56640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:16.015 INFO:tasks.workunit.client.0.smithi006.stderr: -496> 2024-05-30T16:20:15.689+0000 7fbc4fa56640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:16.015 INFO:tasks.workunit.client.0.smithi006.stderr: -495> 2024-05-30T16:20:15.689+0000 7fbc4fa56640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:16.015 INFO:tasks.workunit.client.0.smithi006.stderr: -494> 2024-05-30T16:20:15.689+0000 7fbc50a58640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:16.015 INFO:tasks.workunit.client.0.smithi006.stderr: -493> 2024-05-30T16:20:15.689+0000 7fbc50a58640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:16.015 INFO:tasks.workunit.client.0.smithi006.stderr: -492> 2024-05-30T16:20:15.689+0000 7fbc50a58640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:16.015 INFO:tasks.workunit.client.0.smithi006.stderr: -491> 2024-05-30T16:20:15.689+0000 7fbc4fa56640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:16.015 INFO:tasks.workunit.client.0.smithi006.stderr: -490> 2024-05-30T16:20:15.689+0000 7fbc4fa56640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:16.016 INFO:tasks.workunit.client.0.smithi006.stderr: -489> 2024-05-30T16:20:15.689+0000 7fbc4fa56640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:16.016 INFO:tasks.workunit.client.0.smithi006.stderr: -488> 2024-05-30T16:20:15.689+0000 7fbc50a58640 10 monclient(hunting): handle_auth_done global_id 6017 payload 274 2024-05-30T16:20:16.016 INFO:tasks.workunit.client.0.smithi006.stderr: -487> 2024-05-30T16:20:15.689+0000 7fbc50a58640 10 monclient: _finish_hunting 0 2024-05-30T16:20:16.016 INFO:tasks.workunit.client.0.smithi006.stderr: -486> 2024-05-30T16:20:15.689+0000 7fbc50a58640 1 monclient: found mon.noname-c 2024-05-30T16:20:16.016 INFO:tasks.workunit.client.0.smithi006.stderr: -485> 2024-05-30T16:20:15.689+0000 7fbc50a58640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.6:3301/0 2024-05-30T16:20:16.016 INFO:tasks.workunit.client.0.smithi006.stderr: -484> 2024-05-30T16:20:15.689+0000 7fbc50257640 10 monclient: get_auth_request con 0x559468dd5e40 auth_method 0 2024-05-30T16:20:16.016 INFO:tasks.workunit.client.0.smithi006.stderr: -483> 2024-05-30T16:20:15.689+0000 7fbb3e60c640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:16.016 INFO:tasks.workunit.client.0.smithi006.stderr: -482> 2024-05-30T16:20:15.689+0000 7fbb3e60c640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-30T16:20:16.016 INFO:tasks.workunit.client.0.smithi006.stderr: -481> 2024-05-30T16:20:15.689+0000 7fbb3e60c640 10 monclient: dump: 2024-05-30T16:20:16.016 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:16.016 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:16.016 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.016 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.017 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:16.017 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:16.017 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:16.017 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:16.017 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:16.017 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.017 INFO:tasks.workunit.client.0.smithi006.stderr: -480> 2024-05-30T16:20:15.689+0000 7fbb3e60c640 10 monclient: _finish_auth 0 2024-05-30T16:20:16.017 INFO:tasks.workunit.client.0.smithi006.stderr: -479> 2024-05-30T16:20:15.689+0000 7fbb3e60c640 10 monclient: _check_auth_tickets 2024-05-30T16:20:16.017 INFO:tasks.workunit.client.0.smithi006.stderr: -478> 2024-05-30T16:20:15.689+0000 7fbb3e60c640 10 monclient: handle_config config(0 keys) 2024-05-30T16:20:16.018 INFO:tasks.workunit.client.0.smithi006.stderr: -477> 2024-05-30T16:20:15.689+0000 7fbb3e60c640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:16.018 INFO:tasks.workunit.client.0.smithi006.stderr: -476> 2024-05-30T16:20:15.689+0000 7fbb3e60c640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-30T16:20:16.018 INFO:tasks.workunit.client.0.smithi006.stderr: -475> 2024-05-30T16:20:15.689+0000 7fbb3e60c640 10 monclient: dump: 2024-05-30T16:20:16.018 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:16.018 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:16.018 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.018 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.018 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:16.018 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:16.018 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:16.018 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:16.018 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:16.018 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.019 INFO:tasks.workunit.client.0.smithi006.stderr: -474> 2024-05-30T16:20:15.689+0000 7fbc52746980 5 monclient: authenticate success, global_id 6017 2024-05-30T16:20:16.019 INFO:tasks.workunit.client.0.smithi006.stderr: -473> 2024-05-30T16:20:15.689+0000 7fbc52746980 10 monclient: get_monmap_and_config success 2024-05-30T16:20:16.019 INFO:tasks.workunit.client.0.smithi006.stderr: -472> 2024-05-30T16:20:15.689+0000 7fbc52746980 4 set_mon_vals no callback set 2024-05-30T16:20:16.019 INFO:tasks.workunit.client.0.smithi006.stderr: -471> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient: shutdown 2024-05-30T16:20:16.019 INFO:tasks.workunit.client.0.smithi006.stderr: -470> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 asok(0x559468c7f790) unregister_commands rotate-key 2024-05-30T16:20:16.019 INFO:tasks.workunit.client.0.smithi006.stderr: -469> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient: build_initial_monmap 2024-05-30T16:20:16.019 INFO:tasks.workunit.client.0.smithi006.stderr: -468> 2024-05-30T16:20:15.693+0000 7fbc52746980 1 build_initial for_mkfs: 0 2024-05-30T16:20:16.019 INFO:tasks.workunit.client.0.smithi006.stderr: -467> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient: monmap: 2024-05-30T16:20:16.019 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 0 2024-05-30T16:20:16.020 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-30T16:20:16.020 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 0.000000 2024-05-30T16:20:16.020 INFO:tasks.workunit.client.0.smithi006.stderr:created 0.000000 2024-05-30T16:20:16.020 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 0 (unknown) 2024-05-30T16:20:16.020 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:16.020 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.noname-a 2024-05-30T16:20:16.020 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-b 2024-05-30T16:20:16.020 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.noname-c 2024-05-30T16:20:16.020 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.020 INFO:tasks.workunit.client.0.smithi006.stderr: -466> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e909e0) adding auth protocol: cephx 2024-05-30T16:20:16.020 INFO:tasks.workunit.client.0.smithi006.stderr: -465> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e909e0) adding auth protocol: cephx 2024-05-30T16:20:16.020 INFO:tasks.workunit.client.0.smithi006.stderr: -464> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e909e0) adding auth protocol: cephx 2024-05-30T16:20:16.020 INFO:tasks.workunit.client.0.smithi006.stderr: -463> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e909e0) adding con mode: secure 2024-05-30T16:20:16.021 INFO:tasks.workunit.client.0.smithi006.stderr: -462> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e909e0) adding con mode: crc 2024-05-30T16:20:16.021 INFO:tasks.workunit.client.0.smithi006.stderr: -461> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e909e0) adding con mode: secure 2024-05-30T16:20:16.021 INFO:tasks.workunit.client.0.smithi006.stderr: -460> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e909e0) adding con mode: crc 2024-05-30T16:20:16.021 INFO:tasks.workunit.client.0.smithi006.stderr: -459> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e909e0) adding con mode: secure 2024-05-30T16:20:16.021 INFO:tasks.workunit.client.0.smithi006.stderr: -458> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e909e0) adding con mode: crc 2024-05-30T16:20:16.021 INFO:tasks.workunit.client.0.smithi006.stderr: -457> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e909e0) adding con mode: crc 2024-05-30T16:20:16.021 INFO:tasks.workunit.client.0.smithi006.stderr: -456> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e909e0) adding con mode: secure 2024-05-30T16:20:16.022 INFO:tasks.workunit.client.0.smithi006.stderr: -455> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e909e0) adding con mode: crc 2024-05-30T16:20:16.022 INFO:tasks.workunit.client.0.smithi006.stderr: -454> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e909e0) adding con mode: secure 2024-05-30T16:20:16.022 INFO:tasks.workunit.client.0.smithi006.stderr: -453> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e909e0) adding con mode: crc 2024-05-30T16:20:16.022 INFO:tasks.workunit.client.0.smithi006.stderr: -452> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e909e0) adding con mode: secure 2024-05-30T16:20:16.022 INFO:tasks.workunit.client.0.smithi006.stderr: -451> 2024-05-30T16:20:15.693+0000 7fbc52746980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:16.022 INFO:tasks.workunit.client.0.smithi006.stderr: -450> 2024-05-30T16:20:15.693+0000 7fbc52746980 1 librados: starting msgr at 2024-05-30T16:20:16.022 INFO:tasks.workunit.client.0.smithi006.stderr: -449> 2024-05-30T16:20:15.693+0000 7fbc52746980 1 librados: starting objecter 2024-05-30T16:20:16.022 INFO:tasks.workunit.client.0.smithi006.stderr: -448> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 asok(0x559468c7f790) register_command objecter_requests hook 0x559468bfaa30 2024-05-30T16:20:16.022 INFO:tasks.workunit.client.0.smithi006.stderr: -447> 2024-05-30T16:20:15.693+0000 7fbc52746980 1 librados: setting wanted keys 2024-05-30T16:20:16.022 INFO:tasks.workunit.client.0.smithi006.stderr: -446> 2024-05-30T16:20:15.693+0000 7fbc52746980 1 librados: calling monclient init 2024-05-30T16:20:16.022 INFO:tasks.workunit.client.0.smithi006.stderr: -445> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient: init 2024-05-30T16:20:16.023 INFO:tasks.workunit.client.0.smithi006.stderr: -444> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e8b4b0) adding auth protocol: cephx 2024-05-30T16:20:16.023 INFO:tasks.workunit.client.0.smithi006.stderr: -443> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e8b4b0) adding auth protocol: cephx 2024-05-30T16:20:16.023 INFO:tasks.workunit.client.0.smithi006.stderr: -442> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e8b4b0) adding auth protocol: cephx 2024-05-30T16:20:16.023 INFO:tasks.workunit.client.0.smithi006.stderr: -441> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e8b4b0) adding con mode: secure 2024-05-30T16:20:16.023 INFO:tasks.workunit.client.0.smithi006.stderr: -440> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e8b4b0) adding con mode: crc 2024-05-30T16:20:16.023 INFO:tasks.workunit.client.0.smithi006.stderr: -439> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e8b4b0) adding con mode: secure 2024-05-30T16:20:16.023 INFO:tasks.workunit.client.0.smithi006.stderr: -438> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e8b4b0) adding con mode: crc 2024-05-30T16:20:16.023 INFO:tasks.workunit.client.0.smithi006.stderr: -437> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e8b4b0) adding con mode: secure 2024-05-30T16:20:16.023 INFO:tasks.workunit.client.0.smithi006.stderr: -436> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e8b4b0) adding con mode: crc 2024-05-30T16:20:16.023 INFO:tasks.workunit.client.0.smithi006.stderr: -435> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e8b4b0) adding con mode: crc 2024-05-30T16:20:16.024 INFO:tasks.workunit.client.0.smithi006.stderr: -434> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e8b4b0) adding con mode: secure 2024-05-30T16:20:16.024 INFO:tasks.workunit.client.0.smithi006.stderr: -433> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e8b4b0) adding con mode: crc 2024-05-30T16:20:16.024 INFO:tasks.workunit.client.0.smithi006.stderr: -432> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e8b4b0) adding con mode: secure 2024-05-30T16:20:16.024 INFO:tasks.workunit.client.0.smithi006.stderr: -431> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e8b4b0) adding con mode: crc 2024-05-30T16:20:16.024 INFO:tasks.workunit.client.0.smithi006.stderr: -430> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e8b4b0) adding con mode: secure 2024-05-30T16:20:16.024 INFO:tasks.workunit.client.0.smithi006.stderr: -429> 2024-05-30T16:20:15.693+0000 7fbc52746980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:16.024 INFO:tasks.workunit.client.0.smithi006.stderr: -428> 2024-05-30T16:20:15.693+0000 7fbc52746980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:16.024 INFO:tasks.workunit.client.0.smithi006.stderr: -427> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 asok(0x559468c7f790) register_command rotate-key hook 0x559468e8b5f8 2024-05-30T16:20:16.024 INFO:tasks.workunit.client.0.smithi006.stderr: -426> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient: _reopen_session rank -1 2024-05-30T16:20:16.024 INFO:tasks.workunit.client.0.smithi006.stderr: -425> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient: _add_conns ranks=[0,1,2] 2024-05-30T16:20:16.025 INFO:tasks.workunit.client.0.smithi006.stderr: -424> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient(hunting): picked mon.noname-a con 0x559468dd5e40 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:20:16.025 INFO:tasks.workunit.client.0.smithi006.stderr: -423> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient(hunting): picked mon.noname-b con 0x559468dd6b20 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:20:16.025 INFO:tasks.workunit.client.0.smithi006.stderr: -422> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient(hunting): picked mon.noname-c con 0x559468eb8920 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:20:16.025 INFO:tasks.workunit.client.0.smithi006.stderr: -421> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:16.025 INFO:tasks.workunit.client.0.smithi006.stderr: -420> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:16.025 INFO:tasks.workunit.client.0.smithi006.stderr: -419> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:16.025 INFO:tasks.workunit.client.0.smithi006.stderr: -418> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient(hunting): _renew_subs 2024-05-30T16:20:16.025 INFO:tasks.workunit.client.0.smithi006.stderr: -417> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient(hunting): authenticate will time out at 2555.629599s 2024-05-30T16:20:16.026 INFO:tasks.workunit.client.0.smithi006.stderr: -416> 2024-05-30T16:20:15.693+0000 7fbc50257640 10 monclient(hunting): get_auth_request con 0x559468dd5e40 auth_method 0 2024-05-30T16:20:16.026 INFO:tasks.workunit.client.0.smithi006.stderr: -415> 2024-05-30T16:20:15.693+0000 7fbc50257640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:16.026 INFO:tasks.workunit.client.0.smithi006.stderr: -414> 2024-05-30T16:20:15.693+0000 7fbc50257640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:16.026 INFO:tasks.workunit.client.0.smithi006.stderr: -413> 2024-05-30T16:20:15.693+0000 7fbc50257640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:16.026 INFO:tasks.workunit.client.0.smithi006.stderr: -412> 2024-05-30T16:20:15.693+0000 7fbc50a58640 10 monclient(hunting): get_auth_request con 0x559468eb8920 auth_method 0 2024-05-30T16:20:16.026 INFO:tasks.workunit.client.0.smithi006.stderr: -411> 2024-05-30T16:20:15.693+0000 7fbc50a58640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:16.026 INFO:tasks.workunit.client.0.smithi006.stderr: -410> 2024-05-30T16:20:15.693+0000 7fbc50a58640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:16.026 INFO:tasks.workunit.client.0.smithi006.stderr: -409> 2024-05-30T16:20:15.693+0000 7fbc50a58640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:16.026 INFO:tasks.workunit.client.0.smithi006.stderr: -408> 2024-05-30T16:20:15.693+0000 7fbc50257640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:16.027 INFO:tasks.workunit.client.0.smithi006.stderr: -407> 2024-05-30T16:20:15.693+0000 7fbc50257640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:16.027 INFO:tasks.workunit.client.0.smithi006.stderr: -406> 2024-05-30T16:20:15.693+0000 7fbc50257640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:16.027 INFO:tasks.workunit.client.0.smithi006.stderr: -405> 2024-05-30T16:20:15.693+0000 7fbc50a58640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:16.027 INFO:tasks.workunit.client.0.smithi006.stderr: -404> 2024-05-30T16:20:15.693+0000 7fbc50a58640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:16.027 INFO:tasks.workunit.client.0.smithi006.stderr: -403> 2024-05-30T16:20:15.693+0000 7fbc50a58640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:16.027 INFO:tasks.workunit.client.0.smithi006.stderr: -402> 2024-05-30T16:20:15.693+0000 7fbc50257640 10 monclient(hunting): handle_auth_done global_id 6285 payload 824 2024-05-30T16:20:16.027 INFO:tasks.workunit.client.0.smithi006.stderr: -401> 2024-05-30T16:20:15.693+0000 7fbc50257640 10 monclient: _finish_hunting 0 2024-05-30T16:20:16.027 INFO:tasks.workunit.client.0.smithi006.stderr: -400> 2024-05-30T16:20:15.693+0000 7fbc50257640 1 monclient: found mon.noname-a 2024-05-30T16:20:16.027 INFO:tasks.workunit.client.0.smithi006.stderr: -399> 2024-05-30T16:20:15.693+0000 7fbc50257640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.6:3300/0 2024-05-30T16:20:16.027 INFO:tasks.workunit.client.0.smithi006.stderr: -398> 2024-05-30T16:20:15.693+0000 7fbc4fa56640 10 monclient: get_auth_request con 0x559468dd6b20 auth_method 0 2024-05-30T16:20:16.028 INFO:tasks.workunit.client.0.smithi006.stderr: -397> 2024-05-30T16:20:15.693+0000 7fbb3ce09640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:16.028 INFO:tasks.workunit.client.0.smithi006.stderr: -396> 2024-05-30T16:20:15.693+0000 7fbb3ce09640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-30T16:20:16.028 INFO:tasks.workunit.client.0.smithi006.stderr: -395> 2024-05-30T16:20:15.693+0000 7fbb3ce09640 10 monclient: dump: 2024-05-30T16:20:16.028 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:16.028 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:16.028 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.028 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.028 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:16.028 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:16.028 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:16.028 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:16.028 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:16.028 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.029 INFO:tasks.workunit.client.0.smithi006.stderr: -394> 2024-05-30T16:20:15.693+0000 7fbb3ce09640 10 monclient: _finish_auth 0 2024-05-30T16:20:16.029 INFO:tasks.workunit.client.0.smithi006.stderr: -393> 2024-05-30T16:20:15.693+0000 7fbb3ce09640 10 monclient: _check_auth_tickets 2024-05-30T16:20:16.029 INFO:tasks.workunit.client.0.smithi006.stderr: -392> 2024-05-30T16:20:15.693+0000 7fbb3ce09640 10 monclient: handle_config config(0 keys) 2024-05-30T16:20:16.029 INFO:tasks.workunit.client.0.smithi006.stderr: -391> 2024-05-30T16:20:15.693+0000 7fbb3ce09640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:16.029 INFO:tasks.workunit.client.0.smithi006.stderr: -390> 2024-05-30T16:20:15.693+0000 7fbb3ce09640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-30T16:20:16.029 INFO:tasks.workunit.client.0.smithi006.stderr: -389> 2024-05-30T16:20:15.693+0000 7fbb3ce09640 10 monclient: dump: 2024-05-30T16:20:16.029 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:16.029 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:16.029 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.029 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.029 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:16.029 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:16.029 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:16.029 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:16.030 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:16.030 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.030 INFO:tasks.workunit.client.0.smithi006.stderr: -388> 2024-05-30T16:20:15.693+0000 7fbb3e60c640 4 set_mon_vals no callback set 2024-05-30T16:20:16.030 INFO:tasks.workunit.client.0.smithi006.stderr: -387> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 monclient: authenticate success, global_id 6285 2024-05-30T16:20:16.030 INFO:tasks.workunit.client.0.smithi006.stderr: -386> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient: _renew_subs 2024-05-30T16:20:16.030 INFO:tasks.workunit.client.0.smithi006.stderr: -385> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:20:16.030 INFO:tasks.workunit.client.0.smithi006.stderr: -384> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient: _renew_subs 2024-05-30T16:20:16.030 INFO:tasks.workunit.client.0.smithi006.stderr: -383> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:20:16.030 INFO:tasks.workunit.client.0.smithi006.stderr: -382> 2024-05-30T16:20:15.693+0000 7fbc52746980 1 librados: init done 2024-05-30T16:20:16.030 INFO:tasks.workunit.client.0.smithi006.stderr: -381> 2024-05-30T16:20:15.697+0000 7fbb3ce09640 4 mgrc handle_mgr_map Got map version 5 2024-05-30T16:20:16.030 INFO:tasks.workunit.client.0.smithi006.stderr: -380> 2024-05-30T16:20:15.697+0000 7fbb3ce09640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:20:16.030 INFO:tasks.workunit.client.0.smithi006.stderr: -379> 2024-05-30T16:20:15.697+0000 7fbb3ce09640 4 mgrc reconnect Starting new session with [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:20:16.031 INFO:tasks.workunit.client.0.smithi006.stderr: -378> 2024-05-30T16:20:15.697+0000 7fbc4fa56640 10 monclient: get_auth_request con 0x7fbb3806de10 auth_method 0 2024-05-30T16:20:16.031 INFO:tasks.workunit.client.0.smithi006.stderr: -377> 2024-05-30T16:20:15.697+0000 7fbc50a58640 10 monclient: get_auth_request con 0x559468ec8930 auth_method 0 2024-05-30T16:20:16.031 INFO:tasks.workunit.client.0.smithi006.stderr: -376> 2024-05-30T16:20:15.697+0000 7fbc50257640 10 monclient: get_auth_request con 0x559468ed5f80 auth_method 0 2024-05-30T16:20:16.031 INFO:tasks.workunit.client.0.smithi006.stderr: -375> 2024-05-30T16:20:15.701+0000 7fbc4fa56640 10 monclient: get_auth_request con 0x559468edb010 auth_method 0 2024-05-30T16:20:16.031 INFO:tasks.workunit.client.0.smithi006.stderr: -374> 2024-05-30T16:20:15.701+0000 7fbc52746980 10 monclient: build_initial_monmap 2024-05-30T16:20:16.031 INFO:tasks.workunit.client.0.smithi006.stderr: -373> 2024-05-30T16:20:15.701+0000 7fbc52746980 1 build_initial for_mkfs: 0 2024-05-30T16:20:16.031 INFO:tasks.workunit.client.0.smithi006.stderr: -372> 2024-05-30T16:20:15.701+0000 7fbc52746980 10 monclient: monmap: 2024-05-30T16:20:16.031 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 0 2024-05-30T16:20:16.031 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-30T16:20:16.031 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 0.000000 2024-05-30T16:20:16.031 INFO:tasks.workunit.client.0.smithi006.stderr:created 0.000000 2024-05-30T16:20:16.031 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 0 (unknown) 2024-05-30T16:20:16.032 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:16.032 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.noname-a 2024-05-30T16:20:16.032 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-b 2024-05-30T16:20:16.032 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.noname-c 2024-05-30T16:20:16.032 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.032 INFO:tasks.workunit.client.0.smithi006.stderr: -371> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 AuthRegistry(0x559468edee90) adding auth protocol: cephx 2024-05-30T16:20:16.032 INFO:tasks.workunit.client.0.smithi006.stderr: -370> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 AuthRegistry(0x559468edee90) adding auth protocol: cephx 2024-05-30T16:20:16.032 INFO:tasks.workunit.client.0.smithi006.stderr: -369> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 AuthRegistry(0x559468edee90) adding auth protocol: cephx 2024-05-30T16:20:16.032 INFO:tasks.workunit.client.0.smithi006.stderr: -368> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 AuthRegistry(0x559468edee90) adding con mode: secure 2024-05-30T16:20:16.032 INFO:tasks.workunit.client.0.smithi006.stderr: -367> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 AuthRegistry(0x559468edee90) adding con mode: crc 2024-05-30T16:20:16.032 INFO:tasks.workunit.client.0.smithi006.stderr: -366> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 AuthRegistry(0x559468edee90) adding con mode: secure 2024-05-30T16:20:16.032 INFO:tasks.workunit.client.0.smithi006.stderr: -365> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 AuthRegistry(0x559468edee90) adding con mode: crc 2024-05-30T16:20:16.032 INFO:tasks.workunit.client.0.smithi006.stderr: -364> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 AuthRegistry(0x559468edee90) adding con mode: secure 2024-05-30T16:20:16.032 INFO:tasks.workunit.client.0.smithi006.stderr: -363> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 AuthRegistry(0x559468edee90) adding con mode: crc 2024-05-30T16:20:16.033 INFO:tasks.workunit.client.0.smithi006.stderr: -362> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 AuthRegistry(0x559468edee90) adding con mode: crc 2024-05-30T16:20:16.033 INFO:tasks.workunit.client.0.smithi006.stderr: -361> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 AuthRegistry(0x559468edee90) adding con mode: secure 2024-05-30T16:20:16.033 INFO:tasks.workunit.client.0.smithi006.stderr: -360> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 AuthRegistry(0x559468edee90) adding con mode: crc 2024-05-30T16:20:16.033 INFO:tasks.workunit.client.0.smithi006.stderr: -359> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 AuthRegistry(0x559468edee90) adding con mode: secure 2024-05-30T16:20:16.033 INFO:tasks.workunit.client.0.smithi006.stderr: -358> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 AuthRegistry(0x559468edee90) adding con mode: crc 2024-05-30T16:20:16.033 INFO:tasks.workunit.client.0.smithi006.stderr: -357> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 AuthRegistry(0x559468edee90) adding con mode: secure 2024-05-30T16:20:16.033 INFO:tasks.workunit.client.0.smithi006.stderr: -356> 2024-05-30T16:20:15.701+0000 7fbc52746980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:16.033 INFO:tasks.workunit.client.0.smithi006.stderr: -355> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 asok(0x559468c7f790) register_command objecter_requests cmddesc objecter_requests hook 0x559468ee2490 EEXIST 2024-05-30T16:20:16.033 INFO:tasks.workunit.client.0.smithi006.stderr: -354> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient: init 2024-05-30T16:20:16.033 INFO:tasks.workunit.client.0.smithi006.stderr: -353> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468edde28) adding auth protocol: cephx 2024-05-30T16:20:16.033 INFO:tasks.workunit.client.0.smithi006.stderr: -352> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468edde28) adding auth protocol: cephx 2024-05-30T16:20:16.033 INFO:tasks.workunit.client.0.smithi006.stderr: -351> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468edde28) adding auth protocol: cephx 2024-05-30T16:20:16.034 INFO:tasks.workunit.client.0.smithi006.stderr: -350> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468edde28) adding con mode: secure 2024-05-30T16:20:16.034 INFO:tasks.workunit.client.0.smithi006.stderr: -349> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468edde28) adding con mode: crc 2024-05-30T16:20:16.034 INFO:tasks.workunit.client.0.smithi006.stderr: -348> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468edde28) adding con mode: secure 2024-05-30T16:20:16.034 INFO:tasks.workunit.client.0.smithi006.stderr: -347> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468edde28) adding con mode: crc 2024-05-30T16:20:16.034 INFO:tasks.workunit.client.0.smithi006.stderr: -346> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468edde28) adding con mode: secure 2024-05-30T16:20:16.034 INFO:tasks.workunit.client.0.smithi006.stderr: -345> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468edde28) adding con mode: crc 2024-05-30T16:20:16.034 INFO:tasks.workunit.client.0.smithi006.stderr: -344> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468edde28) adding con mode: crc 2024-05-30T16:20:16.034 INFO:tasks.workunit.client.0.smithi006.stderr: -343> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468edde28) adding con mode: secure 2024-05-30T16:20:16.034 INFO:tasks.workunit.client.0.smithi006.stderr: -342> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468edde28) adding con mode: crc 2024-05-30T16:20:16.034 INFO:tasks.workunit.client.0.smithi006.stderr: -341> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468edde28) adding con mode: secure 2024-05-30T16:20:16.034 INFO:tasks.workunit.client.0.smithi006.stderr: -340> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468edde28) adding con mode: crc 2024-05-30T16:20:16.034 INFO:tasks.workunit.client.0.smithi006.stderr: -339> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468edde28) adding con mode: secure 2024-05-30T16:20:16.034 INFO:tasks.workunit.client.0.smithi006.stderr: -338> 2024-05-30T16:20:15.705+0000 7fbc52746980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:16.035 INFO:tasks.workunit.client.0.smithi006.stderr: -337> 2024-05-30T16:20:15.705+0000 7fbc52746980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:16.035 INFO:tasks.workunit.client.0.smithi006.stderr: -336> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 asok(0x559468c7f790) register_command rotate-key cmddesc rotate-key hook 0x559468eddf70 EEXIST 2024-05-30T16:20:16.035 INFO:tasks.workunit.client.0.smithi006.stderr: -335> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient: _reopen_session rank -1 2024-05-30T16:20:16.035 INFO:tasks.workunit.client.0.smithi006.stderr: -334> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient: _add_conns ranks=[2,1,0] 2024-05-30T16:20:16.035 INFO:tasks.workunit.client.0.smithi006.stderr: -333> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient(hunting): picked mon.noname-c con 0x559468ee8a20 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:20:16.035 INFO:tasks.workunit.client.0.smithi006.stderr: -332> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient(hunting): picked mon.noname-b con 0x559468ee9350 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:20:16.035 INFO:tasks.workunit.client.0.smithi006.stderr: -331> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient(hunting): picked mon.noname-a con 0x559468ee8e00 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:20:16.035 INFO:tasks.workunit.client.0.smithi006.stderr: -330> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:16.035 INFO:tasks.workunit.client.0.smithi006.stderr: -329> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:16.035 INFO:tasks.workunit.client.0.smithi006.stderr: -328> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:16.035 INFO:tasks.workunit.client.0.smithi006.stderr: -327> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient(hunting): _renew_subs 2024-05-30T16:20:16.035 INFO:tasks.workunit.client.0.smithi006.stderr: -326> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient(hunting): authenticate will time out at 2555.640608s 2024-05-30T16:20:16.035 INFO:tasks.workunit.client.0.smithi006.stderr: -325> 2024-05-30T16:20:15.705+0000 7fbc50a58640 10 monclient(hunting): get_auth_request con 0x559468ee8e00 auth_method 0 2024-05-30T16:20:16.036 INFO:tasks.workunit.client.0.smithi006.stderr: -324> 2024-05-30T16:20:15.705+0000 7fbc50a58640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:16.036 INFO:tasks.workunit.client.0.smithi006.stderr: -323> 2024-05-30T16:20:15.705+0000 7fbc50a58640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:16.036 INFO:tasks.workunit.client.0.smithi006.stderr: -322> 2024-05-30T16:20:15.705+0000 7fbc50a58640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:16.036 INFO:tasks.workunit.client.0.smithi006.stderr: -321> 2024-05-30T16:20:15.705+0000 7fbc50257640 10 monclient(hunting): get_auth_request con 0x559468ee8a20 auth_method 0 2024-05-30T16:20:16.036 INFO:tasks.workunit.client.0.smithi006.stderr: -320> 2024-05-30T16:20:15.705+0000 7fbc50257640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:16.036 INFO:tasks.workunit.client.0.smithi006.stderr: -319> 2024-05-30T16:20:15.705+0000 7fbc50257640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:16.036 INFO:tasks.workunit.client.0.smithi006.stderr: -318> 2024-05-30T16:20:15.705+0000 7fbc50257640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:16.036 INFO:tasks.workunit.client.0.smithi006.stderr: -317> 2024-05-30T16:20:15.705+0000 7fbc50a58640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:16.036 INFO:tasks.workunit.client.0.smithi006.stderr: -316> 2024-05-30T16:20:15.705+0000 7fbc50a58640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:16.036 INFO:tasks.workunit.client.0.smithi006.stderr: -315> 2024-05-30T16:20:15.705+0000 7fbc50a58640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:16.036 INFO:tasks.workunit.client.0.smithi006.stderr: -314> 2024-05-30T16:20:15.705+0000 7fbc50257640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:16.036 INFO:tasks.workunit.client.0.smithi006.stderr: -313> 2024-05-30T16:20:15.705+0000 7fbc50257640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:16.036 INFO:tasks.workunit.client.0.smithi006.stderr: -312> 2024-05-30T16:20:15.705+0000 7fbc50257640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:16.037 INFO:tasks.workunit.client.0.smithi006.stderr: -311> 2024-05-30T16:20:15.705+0000 7fbc4fa56640 10 monclient(hunting): get_auth_request con 0x559468ee9350 auth_method 0 2024-05-30T16:20:16.037 INFO:tasks.workunit.client.0.smithi006.stderr: -310> 2024-05-30T16:20:15.705+0000 7fbc4fa56640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:16.037 INFO:tasks.workunit.client.0.smithi006.stderr: -309> 2024-05-30T16:20:15.705+0000 7fbc4fa56640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:16.037 INFO:tasks.workunit.client.0.smithi006.stderr: -308> 2024-05-30T16:20:15.705+0000 7fbc4fa56640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:16.037 INFO:tasks.workunit.client.0.smithi006.stderr: -307> 2024-05-30T16:20:15.705+0000 7fbc50a58640 10 monclient(hunting): handle_auth_done global_id 6288 payload 824 2024-05-30T16:20:16.037 INFO:tasks.workunit.client.0.smithi006.stderr: -306> 2024-05-30T16:20:15.705+0000 7fbc50a58640 10 monclient: _finish_hunting 0 2024-05-30T16:20:16.037 INFO:tasks.workunit.client.0.smithi006.stderr: -305> 2024-05-30T16:20:15.705+0000 7fbc50a58640 1 monclient: found mon.noname-a 2024-05-30T16:20:16.037 INFO:tasks.workunit.client.0.smithi006.stderr: -304> 2024-05-30T16:20:15.705+0000 7fbc50a58640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.6:3300/0 2024-05-30T16:20:16.037 INFO:tasks.workunit.client.0.smithi006.stderr: -303> 2024-05-30T16:20:15.705+0000 7fbb35ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:16.037 INFO:tasks.workunit.client.0.smithi006.stderr: -302> 2024-05-30T16:20:15.705+0000 7fbb35ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-30T16:20:16.037 INFO:tasks.workunit.client.0.smithi006.stderr: -301> 2024-05-30T16:20:15.705+0000 7fbb35ffb640 10 monclient: dump: 2024-05-30T16:20:16.037 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:16.037 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:16.038 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.038 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.038 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:16.038 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:16.038 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:16.038 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:16.038 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:16.038 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.038 INFO:tasks.workunit.client.0.smithi006.stderr: -300> 2024-05-30T16:20:15.705+0000 7fbb35ffb640 10 monclient: _finish_auth 0 2024-05-30T16:20:16.038 INFO:tasks.workunit.client.0.smithi006.stderr: -299> 2024-05-30T16:20:15.705+0000 7fbb35ffb640 10 monclient: _check_auth_tickets 2024-05-30T16:20:16.038 INFO:tasks.workunit.client.0.smithi006.stderr: -298> 2024-05-30T16:20:15.705+0000 7fbb35ffb640 10 monclient: handle_config config(0 keys) 2024-05-30T16:20:16.038 INFO:tasks.workunit.client.0.smithi006.stderr: -297> 2024-05-30T16:20:15.705+0000 7fbb35ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:16.038 INFO:tasks.workunit.client.0.smithi006.stderr: -296> 2024-05-30T16:20:15.705+0000 7fbb35ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-30T16:20:16.039 INFO:tasks.workunit.client.0.smithi006.stderr: -295> 2024-05-30T16:20:15.705+0000 7fbb35ffb640 10 monclient: dump: 2024-05-30T16:20:16.039 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:16.039 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:16.039 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.039 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.039 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:16.039 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:16.039 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:16.039 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:16.039 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:16.039 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.039 INFO:tasks.workunit.client.0.smithi006.stderr: -294> 2024-05-30T16:20:15.705+0000 7fbc51259640 4 set_mon_vals no callback set 2024-05-30T16:20:16.039 INFO:tasks.workunit.client.0.smithi006.stderr: -293> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 monclient: authenticate success, global_id 6288 2024-05-30T16:20:16.040 INFO:tasks.workunit.client.0.smithi006.stderr: -292> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient: _renew_subs 2024-05-30T16:20:16.040 INFO:tasks.workunit.client.0.smithi006.stderr: -291> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:20:16.040 INFO:tasks.workunit.client.0.smithi006.stderr: -290> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient: _renew_subs 2024-05-30T16:20:16.040 INFO:tasks.workunit.client.0.smithi006.stderr: -289> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:20:16.040 INFO:tasks.workunit.client.0.smithi006.stderr: -288> 2024-05-30T16:20:15.705+0000 7fbb35ffb640 4 mgrc handle_mgr_map Got map version 5 2024-05-30T16:20:16.040 INFO:tasks.workunit.client.0.smithi006.stderr: -287> 2024-05-30T16:20:15.705+0000 7fbb35ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:20:16.040 INFO:tasks.workunit.client.0.smithi006.stderr: -286> 2024-05-30T16:20:15.705+0000 7fbb35ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:20:16.040 INFO:tasks.workunit.client.0.smithi006.stderr: -285> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient: get_monmap_and_config 2024-05-30T16:20:16.040 INFO:tasks.workunit.client.0.smithi006.stderr: -284> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient: build_initial_monmap 2024-05-30T16:20:16.040 INFO:tasks.workunit.client.0.smithi006.stderr: -283> 2024-05-30T16:20:15.705+0000 7fbc52746980 1 build_initial for_mkfs: 0 2024-05-30T16:20:16.040 INFO:tasks.workunit.client.0.smithi006.stderr: -282> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient: monmap: 2024-05-30T16:20:16.040 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 0 2024-05-30T16:20:16.040 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-30T16:20:16.041 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 0.000000 2024-05-30T16:20:16.041 INFO:tasks.workunit.client.0.smithi006.stderr:created 0.000000 2024-05-30T16:20:16.041 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 0 (unknown) 2024-05-30T16:20:16.041 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:16.041 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.noname-a 2024-05-30T16:20:16.041 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-b 2024-05-30T16:20:16.041 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.noname-c 2024-05-30T16:20:16.041 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.041 INFO:tasks.workunit.client.0.smithi006.stderr: -281> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468f26220) adding auth protocol: cephx 2024-05-30T16:20:16.041 INFO:tasks.workunit.client.0.smithi006.stderr: -280> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468f26220) adding auth protocol: cephx 2024-05-30T16:20:16.041 INFO:tasks.workunit.client.0.smithi006.stderr: -279> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468f26220) adding auth protocol: cephx 2024-05-30T16:20:16.041 INFO:tasks.workunit.client.0.smithi006.stderr: -278> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468f26220) adding con mode: secure 2024-05-30T16:20:16.041 INFO:tasks.workunit.client.0.smithi006.stderr: -277> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468f26220) adding con mode: crc 2024-05-30T16:20:16.041 INFO:tasks.workunit.client.0.smithi006.stderr: -276> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468f26220) adding con mode: secure 2024-05-30T16:20:16.042 INFO:tasks.workunit.client.0.smithi006.stderr: -275> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468f26220) adding con mode: crc 2024-05-30T16:20:16.042 INFO:tasks.workunit.client.0.smithi006.stderr: -274> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468f26220) adding con mode: secure 2024-05-30T16:20:16.042 INFO:tasks.workunit.client.0.smithi006.stderr: -273> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468f26220) adding con mode: crc 2024-05-30T16:20:16.042 INFO:tasks.workunit.client.0.smithi006.stderr: -272> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468f26220) adding con mode: crc 2024-05-30T16:20:16.042 INFO:tasks.workunit.client.0.smithi006.stderr: -271> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468f26220) adding con mode: secure 2024-05-30T16:20:16.042 INFO:tasks.workunit.client.0.smithi006.stderr: -270> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468f26220) adding con mode: crc 2024-05-30T16:20:16.042 INFO:tasks.workunit.client.0.smithi006.stderr: -269> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468f26220) adding con mode: secure 2024-05-30T16:20:16.042 INFO:tasks.workunit.client.0.smithi006.stderr: -268> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468f26220) adding con mode: crc 2024-05-30T16:20:16.042 INFO:tasks.workunit.client.0.smithi006.stderr: -267> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468f26220) adding con mode: secure 2024-05-30T16:20:16.042 INFO:tasks.workunit.client.0.smithi006.stderr: -266> 2024-05-30T16:20:15.705+0000 7fbc50257640 10 monclient: get_auth_request con 0x7fbb2806dcf0 auth_method 0 2024-05-30T16:20:16.042 INFO:tasks.workunit.client.0.smithi006.stderr: -265> 2024-05-30T16:20:15.709+0000 7fbc52746980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:16.042 INFO:tasks.workunit.client.0.smithi006.stderr: -264> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient: init 2024-05-30T16:20:16.042 INFO:tasks.workunit.client.0.smithi006.stderr: -263> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3aa0) adding auth protocol: cephx 2024-05-30T16:20:16.043 INFO:tasks.workunit.client.0.smithi006.stderr: -262> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3aa0) adding auth protocol: cephx 2024-05-30T16:20:16.043 INFO:tasks.workunit.client.0.smithi006.stderr: -261> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3aa0) adding auth protocol: cephx 2024-05-30T16:20:16.043 INFO:tasks.workunit.client.0.smithi006.stderr: -260> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3aa0) adding con mode: secure 2024-05-30T16:20:16.043 INFO:tasks.workunit.client.0.smithi006.stderr: -259> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3aa0) adding con mode: crc 2024-05-30T16:20:16.043 INFO:tasks.workunit.client.0.smithi006.stderr: -258> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3aa0) adding con mode: secure 2024-05-30T16:20:16.043 INFO:tasks.workunit.client.0.smithi006.stderr: -257> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3aa0) adding con mode: crc 2024-05-30T16:20:16.043 INFO:tasks.workunit.client.0.smithi006.stderr: -256> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3aa0) adding con mode: secure 2024-05-30T16:20:16.043 INFO:tasks.workunit.client.0.smithi006.stderr: -255> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3aa0) adding con mode: crc 2024-05-30T16:20:16.043 INFO:tasks.workunit.client.0.smithi006.stderr: -254> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3aa0) adding con mode: crc 2024-05-30T16:20:16.043 INFO:tasks.workunit.client.0.smithi006.stderr: -253> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3aa0) adding con mode: secure 2024-05-30T16:20:16.043 INFO:tasks.workunit.client.0.smithi006.stderr: -252> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3aa0) adding con mode: crc 2024-05-30T16:20:16.043 INFO:tasks.workunit.client.0.smithi006.stderr: -251> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3aa0) adding con mode: secure 2024-05-30T16:20:16.043 INFO:tasks.workunit.client.0.smithi006.stderr: -250> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3aa0) adding con mode: crc 2024-05-30T16:20:16.043 INFO:tasks.workunit.client.0.smithi006.stderr: -249> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3aa0) adding con mode: secure 2024-05-30T16:20:16.044 INFO:tasks.workunit.client.0.smithi006.stderr: -248> 2024-05-30T16:20:15.709+0000 7fbc52746980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:16.044 INFO:tasks.workunit.client.0.smithi006.stderr: -247> 2024-05-30T16:20:15.709+0000 7fbc52746980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:16.044 INFO:tasks.workunit.client.0.smithi006.stderr: -246> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 asok(0x559468c7f790) register_command rotate-key cmddesc rotate-key hook 0x7ffd6b8c3be8 EEXIST 2024-05-30T16:20:16.044 INFO:tasks.workunit.client.0.smithi006.stderr: -245> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient: _reopen_session rank -1 2024-05-30T16:20:16.044 INFO:tasks.workunit.client.0.smithi006.stderr: -244> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient: _add_conns ranks=[1,0,2] 2024-05-30T16:20:16.044 INFO:tasks.workunit.client.0.smithi006.stderr: -243> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): picked mon.noname-b con 0x559468f560a0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:20:16.044 INFO:tasks.workunit.client.0.smithi006.stderr: -242> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): picked mon.noname-a con 0x559468f56480 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:20:16.044 INFO:tasks.workunit.client.0.smithi006.stderr: -241> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): picked mon.noname-c con 0x559468f5f3b0 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:20:16.044 INFO:tasks.workunit.client.0.smithi006.stderr: -240> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:16.044 INFO:tasks.workunit.client.0.smithi006.stderr: -239> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:16.044 INFO:tasks.workunit.client.0.smithi006.stderr: -238> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:16.044 INFO:tasks.workunit.client.0.smithi006.stderr: -237> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): _renew_subs 2024-05-30T16:20:16.045 INFO:tasks.workunit.client.0.smithi006.stderr: -236> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): authenticate will time out at 2555.644849s 2024-05-30T16:20:16.045 INFO:tasks.workunit.client.0.smithi006.stderr: -235> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient(hunting): get_auth_request con 0x559468f56480 auth_method 0 2024-05-30T16:20:16.045 INFO:tasks.workunit.client.0.smithi006.stderr: -234> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:16.045 INFO:tasks.workunit.client.0.smithi006.stderr: -233> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:16.045 INFO:tasks.workunit.client.0.smithi006.stderr: -232> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:16.045 INFO:tasks.workunit.client.0.smithi006.stderr: -231> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient(hunting): get_auth_request con 0x559468f5f3b0 auth_method 0 2024-05-30T16:20:16.045 INFO:tasks.workunit.client.0.smithi006.stderr: -230> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:16.045 INFO:tasks.workunit.client.0.smithi006.stderr: -229> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:16.045 INFO:tasks.workunit.client.0.smithi006.stderr: -228> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:16.045 INFO:tasks.workunit.client.0.smithi006.stderr: -227> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:16.045 INFO:tasks.workunit.client.0.smithi006.stderr: -226> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:16.045 INFO:tasks.workunit.client.0.smithi006.stderr: -225> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:16.045 INFO:tasks.workunit.client.0.smithi006.stderr: -224> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:16.046 INFO:tasks.workunit.client.0.smithi006.stderr: -223> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:16.046 INFO:tasks.workunit.client.0.smithi006.stderr: -222> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:16.046 INFO:tasks.workunit.client.0.smithi006.stderr: -221> 2024-05-30T16:20:15.709+0000 7fbc50257640 10 monclient(hunting): get_auth_request con 0x559468f560a0 auth_method 0 2024-05-30T16:20:16.046 INFO:tasks.workunit.client.0.smithi006.stderr: -220> 2024-05-30T16:20:15.709+0000 7fbc50257640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:16.046 INFO:tasks.workunit.client.0.smithi006.stderr: -219> 2024-05-30T16:20:15.709+0000 7fbc50257640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:16.046 INFO:tasks.workunit.client.0.smithi006.stderr: -218> 2024-05-30T16:20:15.709+0000 7fbc50257640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:16.046 INFO:tasks.workunit.client.0.smithi006.stderr: -217> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient(hunting): handle_auth_done global_id 6291 payload 274 2024-05-30T16:20:16.046 INFO:tasks.workunit.client.0.smithi006.stderr: -216> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient: _finish_hunting 0 2024-05-30T16:20:16.046 INFO:tasks.workunit.client.0.smithi006.stderr: -215> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 1 monclient: found mon.noname-a 2024-05-30T16:20:16.046 INFO:tasks.workunit.client.0.smithi006.stderr: -214> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.6:3300/0 2024-05-30T16:20:16.046 INFO:tasks.workunit.client.0.smithi006.stderr: -213> 2024-05-30T16:20:15.709+0000 7fbb277fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:16.046 INFO:tasks.workunit.client.0.smithi006.stderr: -212> 2024-05-30T16:20:15.709+0000 7fbb277fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-30T16:20:16.046 INFO:tasks.workunit.client.0.smithi006.stderr: -211> 2024-05-30T16:20:15.709+0000 7fbb277fe640 10 monclient: dump: 2024-05-30T16:20:16.047 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:16.047 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:16.047 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.047 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.047 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:16.047 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:16.047 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:16.047 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:16.047 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:16.047 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.047 INFO:tasks.workunit.client.0.smithi006.stderr: -210> 2024-05-30T16:20:15.709+0000 7fbb277fe640 10 monclient: _finish_auth 0 2024-05-30T16:20:16.048 INFO:tasks.workunit.client.0.smithi006.stderr: -209> 2024-05-30T16:20:15.709+0000 7fbb277fe640 10 monclient: _check_auth_tickets 2024-05-30T16:20:16.048 INFO:tasks.workunit.client.0.smithi006.stderr: -208> 2024-05-30T16:20:15.709+0000 7fbb277fe640 10 monclient: handle_config config(0 keys) 2024-05-30T16:20:16.048 INFO:tasks.workunit.client.0.smithi006.stderr: -207> 2024-05-30T16:20:15.709+0000 7fbb277fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:16.048 INFO:tasks.workunit.client.0.smithi006.stderr: -206> 2024-05-30T16:20:15.709+0000 7fbb277fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-30T16:20:16.048 INFO:tasks.workunit.client.0.smithi006.stderr: -205> 2024-05-30T16:20:15.709+0000 7fbb277fe640 10 monclient: dump: 2024-05-30T16:20:16.048 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:16.048 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:16.048 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.048 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.048 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:16.049 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:16.049 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:16.049 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:16.049 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:16.049 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.049 INFO:tasks.workunit.client.0.smithi006.stderr: -204> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 monclient: authenticate success, global_id 6291 2024-05-30T16:20:16.049 INFO:tasks.workunit.client.0.smithi006.stderr: -203> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient: get_monmap_and_config success 2024-05-30T16:20:16.049 INFO:tasks.workunit.client.0.smithi006.stderr: -202> 2024-05-30T16:20:15.709+0000 7fbc52746980 4 set_mon_vals no callback set 2024-05-30T16:20:16.049 INFO:tasks.workunit.client.0.smithi006.stderr: -201> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient: shutdown 2024-05-30T16:20:16.049 INFO:tasks.workunit.client.0.smithi006.stderr: -200> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient: build_initial_monmap 2024-05-30T16:20:16.049 INFO:tasks.workunit.client.0.smithi006.stderr: -199> 2024-05-30T16:20:15.709+0000 7fbc52746980 1 build_initial for_mkfs: 0 2024-05-30T16:20:16.049 INFO:tasks.workunit.client.0.smithi006.stderr: -198> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient: monmap: 2024-05-30T16:20:16.049 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 0 2024-05-30T16:20:16.050 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-30T16:20:16.050 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 0.000000 2024-05-30T16:20:16.050 INFO:tasks.workunit.client.0.smithi006.stderr:created 0.000000 2024-05-30T16:20:16.050 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 0 (unknown) 2024-05-30T16:20:16.050 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:16.050 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.noname-a 2024-05-30T16:20:16.050 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-b 2024-05-30T16:20:16.050 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.noname-c 2024-05-30T16:20:16.050 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.051 INFO:tasks.workunit.client.0.smithi006.stderr: -197> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f56980) adding auth protocol: cephx 2024-05-30T16:20:16.051 INFO:tasks.workunit.client.0.smithi006.stderr: -196> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f56980) adding auth protocol: cephx 2024-05-30T16:20:16.051 INFO:tasks.workunit.client.0.smithi006.stderr: -195> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f56980) adding auth protocol: cephx 2024-05-30T16:20:16.051 INFO:tasks.workunit.client.0.smithi006.stderr: -194> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f56980) adding con mode: secure 2024-05-30T16:20:16.051 INFO:tasks.workunit.client.0.smithi006.stderr: -193> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f56980) adding con mode: crc 2024-05-30T16:20:16.051 INFO:tasks.workunit.client.0.smithi006.stderr: -192> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f56980) adding con mode: secure 2024-05-30T16:20:16.051 INFO:tasks.workunit.client.0.smithi006.stderr: -191> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f56980) adding con mode: crc 2024-05-30T16:20:16.051 INFO:tasks.workunit.client.0.smithi006.stderr: -190> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f56980) adding con mode: secure 2024-05-30T16:20:16.051 INFO:tasks.workunit.client.0.smithi006.stderr: -189> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f56980) adding con mode: crc 2024-05-30T16:20:16.051 INFO:tasks.workunit.client.0.smithi006.stderr: -188> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f56980) adding con mode: crc 2024-05-30T16:20:16.051 INFO:tasks.workunit.client.0.smithi006.stderr: -187> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f56980) adding con mode: secure 2024-05-30T16:20:16.051 INFO:tasks.workunit.client.0.smithi006.stderr: -186> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f56980) adding con mode: crc 2024-05-30T16:20:16.051 INFO:tasks.workunit.client.0.smithi006.stderr: -185> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f56980) adding con mode: secure 2024-05-30T16:20:16.052 INFO:tasks.workunit.client.0.smithi006.stderr: -184> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f56980) adding con mode: crc 2024-05-30T16:20:16.052 INFO:tasks.workunit.client.0.smithi006.stderr: -183> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f56980) adding con mode: secure 2024-05-30T16:20:16.052 INFO:tasks.workunit.client.0.smithi006.stderr: -182> 2024-05-30T16:20:15.709+0000 7fbc52746980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:16.052 INFO:tasks.workunit.client.0.smithi006.stderr: -181> 2024-05-30T16:20:15.709+0000 7fbc52746980 1 librados: starting msgr at 2024-05-30T16:20:16.052 INFO:tasks.workunit.client.0.smithi006.stderr: -180> 2024-05-30T16:20:15.709+0000 7fbc52746980 1 librados: starting objecter 2024-05-30T16:20:16.052 INFO:tasks.workunit.client.0.smithi006.stderr: -179> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 asok(0x559468c7f790) register_command objecter_requests cmddesc objecter_requests hook 0x559468f2a410 EEXIST 2024-05-30T16:20:16.052 INFO:tasks.workunit.client.0.smithi006.stderr: -178> 2024-05-30T16:20:15.709+0000 7fbc52746980 1 librados: setting wanted keys 2024-05-30T16:20:16.052 INFO:tasks.workunit.client.0.smithi006.stderr: -177> 2024-05-30T16:20:15.709+0000 7fbc52746980 1 librados: calling monclient init 2024-05-30T16:20:16.052 INFO:tasks.workunit.client.0.smithi006.stderr: -176> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient: init 2024-05-30T16:20:16.053 INFO:tasks.workunit.client.0.smithi006.stderr: -175> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f1e480) adding auth protocol: cephx 2024-05-30T16:20:16.053 INFO:tasks.workunit.client.0.smithi006.stderr: -174> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f1e480) adding auth protocol: cephx 2024-05-30T16:20:16.053 INFO:tasks.workunit.client.0.smithi006.stderr: -173> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f1e480) adding auth protocol: cephx 2024-05-30T16:20:16.053 INFO:tasks.workunit.client.0.smithi006.stderr: -172> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f1e480) adding con mode: secure 2024-05-30T16:20:16.053 INFO:tasks.workunit.client.0.smithi006.stderr: -171> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f1e480) adding con mode: crc 2024-05-30T16:20:16.053 INFO:tasks.workunit.client.0.smithi006.stderr: -170> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f1e480) adding con mode: secure 2024-05-30T16:20:16.053 INFO:tasks.workunit.client.0.smithi006.stderr: -169> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f1e480) adding con mode: crc 2024-05-30T16:20:16.053 INFO:tasks.workunit.client.0.smithi006.stderr: -168> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f1e480) adding con mode: secure 2024-05-30T16:20:16.053 INFO:tasks.workunit.client.0.smithi006.stderr: -167> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f1e480) adding con mode: crc 2024-05-30T16:20:16.053 INFO:tasks.workunit.client.0.smithi006.stderr: -166> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f1e480) adding con mode: crc 2024-05-30T16:20:16.053 INFO:tasks.workunit.client.0.smithi006.stderr: -165> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f1e480) adding con mode: secure 2024-05-30T16:20:16.053 INFO:tasks.workunit.client.0.smithi006.stderr: -164> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f1e480) adding con mode: crc 2024-05-30T16:20:16.054 INFO:tasks.workunit.client.0.smithi006.stderr: -163> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f1e480) adding con mode: secure 2024-05-30T16:20:16.054 INFO:tasks.workunit.client.0.smithi006.stderr: -162> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f1e480) adding con mode: crc 2024-05-30T16:20:16.054 INFO:tasks.workunit.client.0.smithi006.stderr: -161> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f1e480) adding con mode: secure 2024-05-30T16:20:16.054 INFO:tasks.workunit.client.0.smithi006.stderr: -160> 2024-05-30T16:20:15.709+0000 7fbc52746980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:16.054 INFO:tasks.workunit.client.0.smithi006.stderr: -159> 2024-05-30T16:20:15.709+0000 7fbc52746980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:16.054 INFO:tasks.workunit.client.0.smithi006.stderr: -158> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 asok(0x559468c7f790) register_command rotate-key cmddesc rotate-key hook 0x559468f1e5c8 EEXIST 2024-05-30T16:20:16.054 INFO:tasks.workunit.client.0.smithi006.stderr: -157> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient: _reopen_session rank -1 2024-05-30T16:20:16.054 INFO:tasks.workunit.client.0.smithi006.stderr: -156> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient: _add_conns ranks=[1,0,2] 2024-05-30T16:20:16.054 INFO:tasks.workunit.client.0.smithi006.stderr: -155> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): picked mon.noname-b con 0x559468f560a0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:20:16.055 INFO:tasks.workunit.client.0.smithi006.stderr: -154> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): picked mon.noname-a con 0x559468f5f3b0 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:20:16.055 INFO:tasks.workunit.client.0.smithi006.stderr: -153> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): picked mon.noname-c con 0x559468f278b0 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:20:16.055 INFO:tasks.workunit.client.0.smithi006.stderr: -152> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:16.055 INFO:tasks.workunit.client.0.smithi006.stderr: -151> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:16.055 INFO:tasks.workunit.client.0.smithi006.stderr: -150> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:16.055 INFO:tasks.workunit.client.0.smithi006.stderr: -149> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): _renew_subs 2024-05-30T16:20:16.055 INFO:tasks.workunit.client.0.smithi006.stderr: -148> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): authenticate will time out at 2555.647460s 2024-05-30T16:20:16.055 INFO:tasks.workunit.client.0.smithi006.stderr: -147> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient(hunting): get_auth_request con 0x559468f5f3b0 auth_method 0 2024-05-30T16:20:16.055 INFO:tasks.workunit.client.0.smithi006.stderr: -146> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:16.055 INFO:tasks.workunit.client.0.smithi006.stderr: -145> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:16.056 INFO:tasks.workunit.client.0.smithi006.stderr: -144> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:16.056 INFO:tasks.workunit.client.0.smithi006.stderr: -143> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient(hunting): get_auth_request con 0x559468f278b0 auth_method 0 2024-05-30T16:20:16.056 INFO:tasks.workunit.client.0.smithi006.stderr: -142> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:16.056 INFO:tasks.workunit.client.0.smithi006.stderr: -141> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:16.056 INFO:tasks.workunit.client.0.smithi006.stderr: -140> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:16.056 INFO:tasks.workunit.client.0.smithi006.stderr: -139> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:16.056 INFO:tasks.workunit.client.0.smithi006.stderr: -138> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:16.056 INFO:tasks.workunit.client.0.smithi006.stderr: -137> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:16.056 INFO:tasks.workunit.client.0.smithi006.stderr: -136> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:16.056 INFO:tasks.workunit.client.0.smithi006.stderr: -135> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:16.056 INFO:tasks.workunit.client.0.smithi006.stderr: -134> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:16.057 INFO:tasks.workunit.client.0.smithi006.stderr: -133> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient(hunting): handle_auth_done global_id 6029 payload 824 2024-05-30T16:20:16.057 INFO:tasks.workunit.client.0.smithi006.stderr: -132> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient: _finish_hunting 0 2024-05-30T16:20:16.057 INFO:tasks.workunit.client.0.smithi006.stderr: -131> 2024-05-30T16:20:15.709+0000 7fbc50a58640 1 monclient: found mon.noname-c 2024-05-30T16:20:16.057 INFO:tasks.workunit.client.0.smithi006.stderr: -130> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.6:3301/0 2024-05-30T16:20:16.057 INFO:tasks.workunit.client.0.smithi006.stderr: -129> 2024-05-30T16:20:15.713+0000 7fbb25ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:16.057 INFO:tasks.workunit.client.0.smithi006.stderr: -128> 2024-05-30T16:20:15.713+0000 7fbb25ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-30T16:20:16.057 INFO:tasks.workunit.client.0.smithi006.stderr: -127> 2024-05-30T16:20:15.713+0000 7fbb25ffb640 10 monclient: dump: 2024-05-30T16:20:16.057 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:16.057 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:16.057 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.057 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.057 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:16.057 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:16.058 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:16.058 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:16.058 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:16.058 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.058 INFO:tasks.workunit.client.0.smithi006.stderr: -126> 2024-05-30T16:20:15.713+0000 7fbb25ffb640 10 monclient: _finish_auth 0 2024-05-30T16:20:16.058 INFO:tasks.workunit.client.0.smithi006.stderr: -125> 2024-05-30T16:20:15.713+0000 7fbb25ffb640 10 monclient: _check_auth_tickets 2024-05-30T16:20:16.058 INFO:tasks.workunit.client.0.smithi006.stderr: -124> 2024-05-30T16:20:15.713+0000 7fbb25ffb640 10 monclient: handle_config config(0 keys) 2024-05-30T16:20:16.058 INFO:tasks.workunit.client.0.smithi006.stderr: -123> 2024-05-30T16:20:15.713+0000 7fbb25ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:16.058 INFO:tasks.workunit.client.0.smithi006.stderr: -122> 2024-05-30T16:20:15.713+0000 7fbb25ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-30T16:20:16.058 INFO:tasks.workunit.client.0.smithi006.stderr: -121> 2024-05-30T16:20:15.713+0000 7fbb25ffb640 10 monclient: dump: 2024-05-30T16:20:16.058 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:16.058 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:16.058 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.059 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.059 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:16.059 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:16.059 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:16.059 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:16.059 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:16.059 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.059 INFO:tasks.workunit.client.0.smithi006.stderr: -120> 2024-05-30T16:20:15.713+0000 7fbb277fe640 4 set_mon_vals no callback set 2024-05-30T16:20:16.059 INFO:tasks.workunit.client.0.smithi006.stderr: -119> 2024-05-30T16:20:15.713+0000 7fbc52746980 5 monclient: authenticate success, global_id 6029 2024-05-30T16:20:16.059 INFO:tasks.workunit.client.0.smithi006.stderr: -118> 2024-05-30T16:20:15.713+0000 7fbc52746980 10 monclient: _renew_subs 2024-05-30T16:20:16.059 INFO:tasks.workunit.client.0.smithi006.stderr: -117> 2024-05-30T16:20:15.713+0000 7fbc52746980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.6:3301/0 2024-05-30T16:20:16.060 INFO:tasks.workunit.client.0.smithi006.stderr: -116> 2024-05-30T16:20:15.713+0000 7fbc52746980 10 monclient: _renew_subs 2024-05-30T16:20:16.060 INFO:tasks.workunit.client.0.smithi006.stderr: -115> 2024-05-30T16:20:15.713+0000 7fbc52746980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.6:3301/0 2024-05-30T16:20:16.060 INFO:tasks.workunit.client.0.smithi006.stderr: -114> 2024-05-30T16:20:15.713+0000 7fbc52746980 1 librados: init done 2024-05-30T16:20:16.060 INFO:tasks.workunit.client.0.smithi006.stderr: -113> 2024-05-30T16:20:15.713+0000 7fbc52746980 5 asok(0x559468c7f790) register_command cr dump hook 0x559468f262e8 2024-05-30T16:20:16.060 INFO:tasks.workunit.client.0.smithi006.stderr: -112> 2024-05-30T16:20:15.713+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.060 INFO:tasks.workunit.client.0.smithi006.stderr: -111> 2024-05-30T16:20:15.713+0000 7fbb25ffb640 4 mgrc handle_mgr_map Got map version 5 2024-05-30T16:20:16.060 INFO:tasks.workunit.client.0.smithi006.stderr: -110> 2024-05-30T16:20:15.713+0000 7fbb25ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:20:16.060 INFO:tasks.workunit.client.0.smithi006.stderr: -109> 2024-05-30T16:20:15.713+0000 7fbb25ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:20:16.060 INFO:tasks.workunit.client.0.smithi006.stderr: -108> 2024-05-30T16:20:15.713+0000 7fbc4fa56640 10 monclient: get_auth_request con 0x559468f74b60 auth_method 0 2024-05-30T16:20:16.061 INFO:tasks.workunit.client.0.smithi006.stderr: -107> 2024-05-30T16:20:15.713+0000 7fbc50257640 10 monclient: get_auth_request con 0x7fbb1806dde0 auth_method 0 2024-05-30T16:20:16.061 INFO:tasks.workunit.client.0.smithi006.stderr: -106> 2024-05-30T16:20:15.713+0000 7fbc52746980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:16.061 INFO:tasks.workunit.client.0.smithi006.stderr: -105> 2024-05-30T16:20:15.713+0000 7fbc52746980 20 realm 2024-05-30T16:20:16.061 INFO:tasks.workunit.client.0.smithi006.stderr: -104> 2024-05-30T16:20:15.713+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.061 INFO:tasks.workunit.client.0.smithi006.stderr: -103> 2024-05-30T16:20:15.717+0000 7fbc52746980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:16.061 INFO:tasks.workunit.client.0.smithi006.stderr: -102> 2024-05-30T16:20:15.717+0000 7fbc52746980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:16.061 INFO:tasks.workunit.client.0.smithi006.stderr: -101> 2024-05-30T16:20:15.717+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.061 INFO:tasks.workunit.client.0.smithi006.stderr: -100> 2024-05-30T16:20:15.717+0000 7fbc52746980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:16.062 INFO:tasks.workunit.client.0.smithi006.stderr: -99> 2024-05-30T16:20:15.717+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.062 INFO:tasks.workunit.client.0.smithi006.stderr: -98> 2024-05-30T16:20:15.717+0000 7fbc50257640 10 monclient: get_auth_request con 0x559468f78f40 auth_method 0 2024-05-30T16:20:16.062 INFO:tasks.workunit.client.0.smithi006.stderr: -97> 2024-05-30T16:20:15.717+0000 7fbc52746980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:16.062 INFO:tasks.workunit.client.0.smithi006.stderr: -96> 2024-05-30T16:20:15.717+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.062 INFO:tasks.workunit.client.0.smithi006.stderr: -95> 2024-05-30T16:20:15.717+0000 7fbc52746980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:16.062 INFO:tasks.workunit.client.0.smithi006.stderr: -94> 2024-05-30T16:20:15.717+0000 7fbc52746980 20 searching for the correct realm 2024-05-30T16:20:16.062 INFO:tasks.workunit.client.0.smithi006.stderr: -93> 2024-05-30T16:20:15.717+0000 7fbc4fa56640 10 monclient: get_auth_request con 0x559468f7e0c0 auth_method 0 2024-05-30T16:20:16.062 INFO:tasks.workunit.client.0.smithi006.stderr: -92> 2024-05-30T16:20:15.721+0000 7fbc50a58640 10 monclient: get_auth_request con 0x7fbc380b9b30 auth_method 0 2024-05-30T16:20:16.063 INFO:tasks.workunit.client.0.smithi006.stderr: -91> 2024-05-30T16:20:15.721+0000 7fbc50257640 10 monclient: get_auth_request con 0x7fbc48015070 auth_method 0 2024-05-30T16:20:16.063 INFO:tasks.workunit.client.0.smithi006.stderr: -90> 2024-05-30T16:20:15.721+0000 7fbc4fa56640 10 monclient: get_auth_request con 0x7fbc4000e040 auth_method 0 2024-05-30T16:20:16.063 INFO:tasks.workunit.client.0.smithi006.stderr: -89> 2024-05-30T16:20:15.725+0000 7fbc50a58640 0 -- 172.21.15.6:0/1138350149 >> [v2:172.21.15.204:6800/3687190393,v1:172.21.15.204:6801/3687190393] conn(0x7fbc380e4040 msgr2=0x7fbc380e3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:16.063 INFO:tasks.workunit.client.0.smithi006.stderr: -88> 2024-05-30T16:20:15.928+0000 7fbc50a58640 10 monclient: get_auth_request con 0x7fbc380e4040 auth_method 0 2024-05-30T16:20:16.063 INFO:tasks.workunit.client.0.smithi006.stderr: -87> 2024-05-30T16:20:15.932+0000 7fbc50257640 10 monclient: get_auth_request con 0x7fbc380f4400 auth_method 0 2024-05-30T16:20:16.063 INFO:tasks.workunit.client.0.smithi006.stderr: -86> 2024-05-30T16:20:15.940+0000 7fbc50257640 0 -- 172.21.15.6:0/1138350149 >> [v2:172.21.15.204:6816/2677861390,v1:172.21.15.204:6817/2677861390] conn(0x559468f78f40 msgr2=0x559468f7b330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:16.063 INFO:tasks.workunit.client.0.smithi006.stderr: -85> 2024-05-30T16:20:15.940+0000 7fbb25ffb640 1 client.6029.objecter ms_handle_reset 0x559468f78f40 session 0x559468f77e00 osd.5 2024-05-30T16:20:16.063 INFO:tasks.workunit.client.0.smithi006.stderr: -84> 2024-05-30T16:20:15.940+0000 7fbb25ffb640 10 monclient: _renew_subs 2024-05-30T16:20:16.063 INFO:tasks.workunit.client.0.smithi006.stderr: -83> 2024-05-30T16:20:15.940+0000 7fbb25ffb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.6:3301/0 2024-05-30T16:20:16.063 INFO:tasks.workunit.client.0.smithi006.stderr: -82> 2024-05-30T16:20:15.940+0000 7fbc50257640 10 monclient: get_auth_request con 0x7fbb1807e510 auth_method 0 2024-05-30T16:20:16.063 INFO:tasks.workunit.client.0.smithi006.stderr: -81> 2024-05-30T16:20:15.948+0000 7fbc52746980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:16.063 INFO:tasks.workunit.client.0.smithi006.stderr: -80> 2024-05-30T16:20:15.948+0000 7fbc52746980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:16.064 INFO:tasks.workunit.client.0.smithi006.stderr: -79> 2024-05-30T16:20:15.948+0000 7fbc52746980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:16.064 INFO:tasks.workunit.client.0.smithi006.stderr: -78> 2024-05-30T16:20:15.948+0000 7fbc52746980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:16.064 INFO:tasks.workunit.client.0.smithi006.stderr: -77> 2024-05-30T16:20:15.948+0000 7fbc52746980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:16.064 INFO:tasks.workunit.client.0.smithi006.stderr: -76> 2024-05-30T16:20:15.948+0000 7fbc52746980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:16.064 INFO:tasks.workunit.client.0.smithi006.stderr: -75> 2024-05-30T16:20:15.948+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.064 INFO:tasks.workunit.client.0.smithi006.stderr: -74> 2024-05-30T16:20:15.948+0000 7fbc52746980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:16.064 INFO:tasks.workunit.client.0.smithi006.stderr: -73> 2024-05-30T16:20:15.948+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.064 INFO:tasks.workunit.client.0.smithi006.stderr: -72> 2024-05-30T16:20:15.948+0000 7fbc52746980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:16.064 INFO:tasks.workunit.client.0.smithi006.stderr: -71> 2024-05-30T16:20:15.948+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.064 INFO:tasks.workunit.client.0.smithi006.stderr: -70> 2024-05-30T16:20:15.952+0000 7fbc52746980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:16.064 INFO:tasks.workunit.client.0.smithi006.stderr: -69> 2024-05-30T16:20:15.952+0000 7fbc52746980 20 zone default found 2024-05-30T16:20:16.064 INFO:tasks.workunit.client.0.smithi006.stderr: -68> 2024-05-30T16:20:15.952+0000 7fbc52746980 4 Realm: () 2024-05-30T16:20:16.064 INFO:tasks.workunit.client.0.smithi006.stderr: -67> 2024-05-30T16:20:15.952+0000 7fbc52746980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:16.065 INFO:tasks.workunit.client.0.smithi006.stderr: -66> 2024-05-30T16:20:15.952+0000 7fbc52746980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:16.065 INFO:tasks.workunit.client.0.smithi006.stderr: -65> 2024-05-30T16:20:15.952+0000 7fbc52746980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:16.065 INFO:tasks.workunit.client.0.smithi006.stderr: -64> 2024-05-30T16:20:15.952+0000 7fbc52746980 20 zonegroup default 2024-05-30T16:20:16.065 INFO:tasks.workunit.client.0.smithi006.stderr: -63> 2024-05-30T16:20:15.952+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.065 INFO:tasks.workunit.client.0.smithi006.stderr: -62> 2024-05-30T16:20:15.952+0000 7fbc52746980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:16.065 INFO:tasks.workunit.client.0.smithi006.stderr: -61> 2024-05-30T16:20:15.952+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.065 INFO:tasks.workunit.client.0.smithi006.stderr: -60> 2024-05-30T16:20:15.952+0000 7fbc52746980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:16.065 INFO:tasks.workunit.client.0.smithi006.stderr: -59> 2024-05-30T16:20:15.952+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.065 INFO:tasks.workunit.client.0.smithi006.stderr: -58> 2024-05-30T16:20:15.952+0000 7fbc52746980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:16.065 INFO:tasks.workunit.client.0.smithi006.stderr: -57> 2024-05-30T16:20:15.952+0000 7fbc52746980 20 started sync module instance, tier type = 2024-05-30T16:20:16.065 INFO:tasks.workunit.client.0.smithi006.stderr: -56> 2024-05-30T16:20:15.952+0000 7fbc52746980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:16.065 INFO:tasks.workunit.client.0.smithi006.stderr: -55> 2024-05-30T16:20:15.976+0000 7fbc52746980 20 add_watcher() i=0 2024-05-30T16:20:16.065 INFO:tasks.workunit.client.0.smithi006.stderr: -54> 2024-05-30T16:20:15.976+0000 7fbc52746980 20 add_watcher() i=1 2024-05-30T16:20:16.066 INFO:tasks.workunit.client.0.smithi006.stderr: -53> 2024-05-30T16:20:15.976+0000 7fbc52746980 20 add_watcher() i=2 2024-05-30T16:20:16.066 INFO:tasks.workunit.client.0.smithi006.stderr: -52> 2024-05-30T16:20:15.976+0000 7fbc52746980 20 add_watcher() i=3 2024-05-30T16:20:16.066 INFO:tasks.workunit.client.0.smithi006.stderr: -51> 2024-05-30T16:20:15.976+0000 7fbc52746980 20 add_watcher() i=4 2024-05-30T16:20:16.066 INFO:tasks.workunit.client.0.smithi006.stderr: -50> 2024-05-30T16:20:15.976+0000 7fbc52746980 20 add_watcher() i=5 2024-05-30T16:20:16.066 INFO:tasks.workunit.client.0.smithi006.stderr: -49> 2024-05-30T16:20:15.976+0000 7fbc52746980 20 add_watcher() i=6 2024-05-30T16:20:16.066 INFO:tasks.workunit.client.0.smithi006.stderr: -48> 2024-05-30T16:20:15.976+0000 7fbc52746980 20 add_watcher() i=7 2024-05-30T16:20:16.066 INFO:tasks.workunit.client.0.smithi006.stderr: -47> 2024-05-30T16:20:15.976+0000 7fbc52746980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:16.066 INFO:tasks.workunit.client.0.smithi006.stderr: -46> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3f80) adding auth protocol: cephx 2024-05-30T16:20:16.066 INFO:tasks.workunit.client.0.smithi006.stderr: -45> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3f80) adding auth protocol: cephx 2024-05-30T16:20:16.066 INFO:tasks.workunit.client.0.smithi006.stderr: -44> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3f80) adding auth protocol: cephx 2024-05-30T16:20:16.066 INFO:tasks.workunit.client.0.smithi006.stderr: -43> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3f80) adding con mode: secure 2024-05-30T16:20:16.066 INFO:tasks.workunit.client.0.smithi006.stderr: -42> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3f80) adding con mode: crc 2024-05-30T16:20:16.066 INFO:tasks.workunit.client.0.smithi006.stderr: -41> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3f80) adding con mode: secure 2024-05-30T16:20:16.067 INFO:tasks.workunit.client.0.smithi006.stderr: -40> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3f80) adding con mode: crc 2024-05-30T16:20:16.067 INFO:tasks.workunit.client.0.smithi006.stderr: -39> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3f80) adding con mode: secure 2024-05-30T16:20:16.067 INFO:tasks.workunit.client.0.smithi006.stderr: -38> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3f80) adding con mode: crc 2024-05-30T16:20:16.067 INFO:tasks.workunit.client.0.smithi006.stderr: -37> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3f80) adding con mode: crc 2024-05-30T16:20:16.067 INFO:tasks.workunit.client.0.smithi006.stderr: -36> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3f80) adding con mode: secure 2024-05-30T16:20:16.067 INFO:tasks.workunit.client.0.smithi006.stderr: -35> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3f80) adding con mode: crc 2024-05-30T16:20:16.067 INFO:tasks.workunit.client.0.smithi006.stderr: -34> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3f80) adding con mode: secure 2024-05-30T16:20:16.067 INFO:tasks.workunit.client.0.smithi006.stderr: -33> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3f80) adding con mode: crc 2024-05-30T16:20:16.067 INFO:tasks.workunit.client.0.smithi006.stderr: -32> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3f80) adding con mode: secure 2024-05-30T16:20:16.067 INFO:tasks.workunit.client.0.smithi006.stderr: -31> 2024-05-30T16:20:15.984+0000 7fbb06fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:16.067 INFO:tasks.workunit.client.0.smithi006.stderr: -30> 2024-05-30T16:20:15.984+0000 7fbc52746980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:16.067 INFO:tasks.workunit.client.0.smithi006.stderr: -29> 2024-05-30T16:20:15.984+0000 7fbc52746980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:16.067 INFO:tasks.workunit.client.0.smithi006.stderr: -28> 2024-05-30T16:20:15.984+0000 7fbc52746980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:16.067 INFO:tasks.workunit.client.0.smithi006.stderr: -27> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 asok(0x559468c7f790) register_command cache list hook 0x559468f26c20 2024-05-30T16:20:16.068 INFO:tasks.workunit.client.0.smithi006.stderr: -26> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 asok(0x559468c7f790) register_command cache inspect hook 0x559468f26c20 2024-05-30T16:20:16.068 INFO:tasks.workunit.client.0.smithi006.stderr: -25> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 asok(0x559468c7f790) register_command cache erase hook 0x559468f26c20 2024-05-30T16:20:16.068 INFO:tasks.workunit.client.0.smithi006.stderr: -24> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 asok(0x559468c7f790) register_command cache zap hook 0x559468f26c20 2024-05-30T16:20:16.068 INFO:tasks.workunit.client.0.smithi006.stderr: -23> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 note: GC not initialized 2024-05-30T16:20:16.068 INFO:tasks.workunit.client.0.smithi006.stderr: -22> 2024-05-30T16:20:15.984+0000 7fbb05fdb640 20 reqs_thread_entry: start 2024-05-30T16:20:16.068 INFO:tasks.workunit.client.0.smithi006.stderr: -21> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 asok(0x559468c7f790) register_command sync trace show hook 0x7fbc380acda0 2024-05-30T16:20:16.068 INFO:tasks.workunit.client.0.smithi006.stderr: -20> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 asok(0x559468c7f790) register_command sync trace history hook 0x7fbc380acda0 2024-05-30T16:20:16.068 INFO:tasks.workunit.client.0.smithi006.stderr: -19> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 asok(0x559468c7f790) register_command sync trace active hook 0x7fbc380acda0 2024-05-30T16:20:16.068 INFO:tasks.workunit.client.0.smithi006.stderr: -18> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 asok(0x559468c7f790) register_command sync trace active_short hook 0x7fbc380acda0 2024-05-30T16:20:16.068 INFO:tasks.workunit.client.0.smithi006.stderr: -17> 2024-05-30T16:20:15.984+0000 7fbc52746980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:16.068 INFO:tasks.workunit.client.0.smithi006.stderr: -16> 2024-05-30T16:20:15.984+0000 7fbc52746980 20 Filter name: none 2024-05-30T16:20:16.068 INFO:tasks.workunit.client.0.smithi006.stderr: -15> 2024-05-30T16:20:15.984+0000 7fba777fe640 20 reqs_thread_entry: start 2024-05-30T16:20:16.068 INFO:tasks.workunit.client.0.smithi006.stderr: -14> 2024-05-30T16:20:15.984+0000 7fbc52746980 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-05-30T16:20:16.068 INFO:tasks.workunit.client.0.smithi006.stderr: -13> 2024-05-30T16:20:15.984+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.069 INFO:tasks.workunit.client.0.smithi006.stderr: -12> 2024-05-30T16:20:15.984+0000 7fbc52746980 20 rados_obj.operate() r=0 bl.length=218 2024-05-30T16:20:16.069 INFO:tasks.workunit.client.0.smithi006.stderr: -11> 2024-05-30T16:20:15.984+0000 7fbc52746980 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-05-30T16:20:16.069 INFO:tasks.workunit.client.0.smithi006.stderr: -10> 2024-05-30T16:20:15.984+0000 7fbc52746980 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-05-30T16:20:16.069 INFO:tasks.workunit.client.0.smithi006.stderr: -9> 2024-05-30T16:20:15.984+0000 7fbc52746980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 : miss 2024-05-30T16:20:16.069 INFO:tasks.workunit.client.0.smithi006.stderr: -8> 2024-05-30T16:20:15.984+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.069 INFO:tasks.workunit.client.0.smithi006.stderr: -7> 2024-05-30T16:20:15.984+0000 7fbc52746980 20 rados_obj.operate() r=0 bl.length=432 2024-05-30T16:20:16.069 INFO:tasks.workunit.client.0.smithi006.stderr: -6> 2024-05-30T16:20:15.984+0000 7fbc52746980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 info.flags=0x17 2024-05-30T16:20:16.069 INFO:tasks.workunit.client.0.smithi006.stderr: -5> 2024-05-30T16:20:15.984+0000 7fbc52746980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 to cache LRU end 2024-05-30T16:20:16.069 INFO:tasks.workunit.client.0.smithi006.stderr: -4> 2024-05-30T16:20:15.984+0000 7fbc52746980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:20:16.069 INFO:tasks.workunit.client.0.smithi006.stderr: -3> 2024-05-30T16:20:15.984+0000 7fbc52746980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:16.069 INFO:tasks.workunit.client.0.smithi006.stderr: -2> 2024-05-30T16:20:15.984+0000 7fbc52746980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 2024-05-30T16:20:16.069 INFO:tasks.workunit.client.0.smithi006.stderr: -1> 2024-05-30T16:20:15.996+0000 7fbc52746980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fbc52746980 time 2024-05-30T16:20:16.001268+0000 2024-05-30T16:20:16.070 INFO:tasks.workunit.client.0.smithi006.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-30T16:20:16.070 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.070 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:20:16.070 INFO:tasks.workunit.client.0.smithi006.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fbc5601dcff] 2024-05-30T16:20:16.070 INFO:tasks.workunit.client.0.smithi006.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7fbc5601deb3] 2024-05-30T16:20:16.070 INFO:tasks.workunit.client.0.smithi006.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x5594675b5f81] 2024-05-30T16:20:16.070 INFO:tasks.workunit.client.0.smithi006.stderr: 4: radosgw-admin(+0x9a792a) [0x5594675c392a] 2024-05-30T16:20:16.070 INFO:tasks.workunit.client.0.smithi006.stderr: 5: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24f) [0x5594675c7e4f] 2024-05-30T16:20:16.070 INFO:tasks.workunit.client.0.smithi006.stderr: 6: main() 2024-05-30T16:20:16.070 INFO:tasks.workunit.client.0.smithi006.stderr: 7: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fbc54e86d90] 2024-05-30T16:20:16.070 INFO:tasks.workunit.client.0.smithi006.stderr: 8: __libc_start_main() 2024-05-30T16:20:16.070 INFO:tasks.workunit.client.0.smithi006.stderr: 9: _start() 2024-05-30T16:20:16.070 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.071 INFO:tasks.workunit.client.0.smithi006.stderr: 0> 2024-05-30T16:20:15.996+0000 7fbc52746980 -1 *** Caught signal (Aborted) ** 2024-05-30T16:20:16.071 INFO:tasks.workunit.client.0.smithi006.stderr: in thread 7fbc52746980 thread_name:radosgw-admin 2024-05-30T16:20:16.071 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.071 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:20:16.071 INFO:tasks.workunit.client.0.smithi006.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fbc54e9f520] 2024-05-30T16:20:16.071 INFO:tasks.workunit.client.0.smithi006.stderr: 2: pthread_kill() 2024-05-30T16:20:16.071 INFO:tasks.workunit.client.0.smithi006.stderr: 3: raise() 2024-05-30T16:20:16.071 INFO:tasks.workunit.client.0.smithi006.stderr: 4: abort() 2024-05-30T16:20:16.071 INFO:tasks.workunit.client.0.smithi006.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fbc5601dd59] 2024-05-30T16:20:16.071 INFO:tasks.workunit.client.0.smithi006.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7fbc5601deb3] 2024-05-30T16:20:16.071 INFO:tasks.workunit.client.0.smithi006.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x5594675b5f81] 2024-05-30T16:20:16.071 INFO:tasks.workunit.client.0.smithi006.stderr: 8: radosgw-admin(+0x9a792a) [0x5594675c392a] 2024-05-30T16:20:16.071 INFO:tasks.workunit.client.0.smithi006.stderr: 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24f) [0x5594675c7e4f] 2024-05-30T16:20:16.071 INFO:tasks.workunit.client.0.smithi006.stderr: 10: main() 2024-05-30T16:20:16.072 INFO:tasks.workunit.client.0.smithi006.stderr: 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fbc54e86d90] 2024-05-30T16:20:16.072 INFO:tasks.workunit.client.0.smithi006.stderr: 12: __libc_start_main() 2024-05-30T16:20:16.072 INFO:tasks.workunit.client.0.smithi006.stderr: 13: _start() 2024-05-30T16:20:16.072 INFO:tasks.workunit.client.0.smithi006.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-30T16:20:16.074 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.074 INFO:tasks.workunit.client.0.smithi006.stderr:--- logging levels --- 2024-05-30T16:20:16.074 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 none 2024-05-30T16:20:16.074 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 lockdep 2024-05-30T16:20:16.074 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 context 2024-05-30T16:20:16.075 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 crush 2024-05-30T16:20:16.075 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds 2024-05-30T16:20:16.075 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_balancer 2024-05-30T16:20:16.075 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_locker 2024-05-30T16:20:16.075 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_log 2024-05-30T16:20:16.075 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_log_expire 2024-05-30T16:20:16.075 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_migrator 2024-05-30T16:20:16.075 INFO:tasks.workunit.client.0.smithi006.stderr: 3/ 5 mds_quiesce 2024-05-30T16:20:16.075 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 buffer 2024-05-30T16:20:16.075 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 timer 2024-05-30T16:20:16.075 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 filer 2024-05-30T16:20:16.075 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 striper 2024-05-30T16:20:16.075 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 objecter 2024-05-30T16:20:16.075 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rados 2024-05-30T16:20:16.076 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd 2024-05-30T16:20:16.076 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd_mirror 2024-05-30T16:20:16.076 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd_replay 2024-05-30T16:20:16.076 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd_pwl 2024-05-30T16:20:16.076 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 journaler 2024-05-30T16:20:16.076 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 objectcacher 2024-05-30T16:20:16.076 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 immutable_obj_cache 2024-05-30T16:20:16.076 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 client 2024-05-30T16:20:16.076 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 osd 2024-05-30T16:20:16.076 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 optracker 2024-05-30T16:20:16.076 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 objclass 2024-05-30T16:20:16.076 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 3 filestore 2024-05-30T16:20:16.076 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 3 journal 2024-05-30T16:20:16.076 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 0 ms 2024-05-30T16:20:16.076 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mon 2024-05-30T16:20:16.077 INFO:tasks.workunit.client.0.smithi006.stderr: 0/10 monc 2024-05-30T16:20:16.077 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 paxos 2024-05-30T16:20:16.077 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 tp 2024-05-30T16:20:16.077 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 auth 2024-05-30T16:20:16.077 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 crypto 2024-05-30T16:20:16.077 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 finisher 2024-05-30T16:20:16.077 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 reserver 2024-05-30T16:20:16.077 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 heartbeatmap 2024-05-30T16:20:16.077 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 perfcounter 2024-05-30T16:20:16.077 INFO:tasks.workunit.client.0.smithi006.stderr: 20/20 rgw 2024-05-30T16:20:16.077 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_sync 2024-05-30T16:20:16.077 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_datacache 2024-05-30T16:20:16.077 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_access 2024-05-30T16:20:16.077 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_dbstore 2024-05-30T16:20:16.078 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_flight 2024-05-30T16:20:16.078 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_lifecycle 2024-05-30T16:20:16.078 INFO:tasks.workunit.client.0.smithi006.stderr: 20/20 rgw_notification 2024-05-30T16:20:16.078 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 javaclient 2024-05-30T16:20:16.078 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 asok 2024-05-30T16:20:16.078 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 throttle 2024-05-30T16:20:16.078 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 0 refs 2024-05-30T16:20:16.078 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 compressor 2024-05-30T16:20:16.078 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 bluestore 2024-05-30T16:20:16.078 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 bluefs 2024-05-30T16:20:16.078 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 3 bdev 2024-05-30T16:20:16.078 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 kstore 2024-05-30T16:20:16.078 INFO:tasks.workunit.client.0.smithi006.stderr: 4/ 5 rocksdb 2024-05-30T16:20:16.078 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 fuse 2024-05-30T16:20:16.079 INFO:tasks.workunit.client.0.smithi006.stderr: 2/ 5 mgr 2024-05-30T16:20:16.079 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mgrc 2024-05-30T16:20:16.079 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 dpdk 2024-05-30T16:20:16.079 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 eventtrace 2024-05-30T16:20:16.079 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 prioritycache 2024-05-30T16:20:16.079 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 test 2024-05-30T16:20:16.079 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 cephfs_mirror 2024-05-30T16:20:16.079 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 cephsqlite 2024-05-30T16:20:16.079 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 crimson_interrupt 2024-05-30T16:20:16.079 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore 2024-05-30T16:20:16.079 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_onode 2024-05-30T16:20:16.079 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_odata 2024-05-30T16:20:16.079 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_omap 2024-05-30T16:20:16.079 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_tm 2024-05-30T16:20:16.080 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_t 2024-05-30T16:20:16.080 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_cleaner 2024-05-30T16:20:16.080 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_epm 2024-05-30T16:20:16.080 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_lba 2024-05-30T16:20:16.080 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_fixedkv_tree 2024-05-30T16:20:16.080 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_cache 2024-05-30T16:20:16.080 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_journal 2024-05-30T16:20:16.080 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_device 2024-05-30T16:20:16.080 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_backref 2024-05-30T16:20:16.080 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 alienstore 2024-05-30T16:20:16.080 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mclock 2024-05-30T16:20:16.080 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 cyanstore 2024-05-30T16:20:16.080 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 ceph_exporter 2024-05-30T16:20:16.080 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 memstore 2024-05-30T16:20:16.081 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 trace 2024-05-30T16:20:16.081 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 ceph_dedup 2024-05-30T16:20:16.081 INFO:tasks.workunit.client.0.smithi006.stderr: -2/-2 (syslog threshold) 2024-05-30T16:20:16.081 INFO:tasks.workunit.client.0.smithi006.stderr: 99/99 (stderr threshold) 2024-05-30T16:20:16.081 INFO:tasks.workunit.client.0.smithi006.stderr:--- pthread ID / name mapping for recent threads --- 2024-05-30T16:20:16.081 INFO:tasks.workunit.client.0.smithi006.stderr: 7fba777fe640 / http_manager 2024-05-30T16:20:16.081 INFO:tasks.workunit.client.0.smithi006.stderr: 7fbb05fdb640 / http_manager 2024-05-30T16:20:16.081 INFO:tasks.workunit.client.0.smithi006.stderr: 7fbb06fdd640 / rgw_dt_lg_renew 2024-05-30T16:20:16.081 INFO:tasks.workunit.client.0.smithi006.stderr: 7fbb25ffb640 / ms_dispatch 2024-05-30T16:20:16.081 INFO:tasks.workunit.client.0.smithi006.stderr: 7fbb277fe640 / io_context_pool 2024-05-30T16:20:16.081 INFO:tasks.workunit.client.0.smithi006.stderr: 7fbb35ffb640 / ms_dispatch 2024-05-30T16:20:16.081 INFO:tasks.workunit.client.0.smithi006.stderr: 7fbb3ce09640 / ms_dispatch 2024-05-30T16:20:16.081 INFO:tasks.workunit.client.0.smithi006.stderr: 7fbb3e60c640 / io_context_pool 2024-05-30T16:20:16.081 INFO:tasks.workunit.client.0.smithi006.stderr: 7fbc4fa56640 / msgr-worker-2 2024-05-30T16:20:16.082 INFO:tasks.workunit.client.0.smithi006.stderr: 7fbc50257640 / msgr-worker-1 2024-05-30T16:20:16.082 INFO:tasks.workunit.client.0.smithi006.stderr: 7fbc50a58640 / msgr-worker-0 2024-05-30T16:20:16.082 INFO:tasks.workunit.client.0.smithi006.stderr: 7fbc51259640 / io_context_pool 2024-05-30T16:20:16.082 INFO:tasks.workunit.client.0.smithi006.stderr: 7fbc52746980 / radosgw-admin 2024-05-30T16:20:16.082 INFO:tasks.workunit.client.0.smithi006.stderr: max_recent 500 2024-05-30T16:20:16.082 INFO:tasks.workunit.client.0.smithi006.stderr: max_new 1000 2024-05-30T16:20:16.082 INFO:tasks.workunit.client.0.smithi006.stderr: log_file /var/log/ceph/ceph-client.admin.81021.log 2024-05-30T16:20:16.082 INFO:tasks.workunit.client.0.smithi006.stderr:--- end dump of recent events --- 2024-05-30T16:20:16.082 INFO:tasks.workunit.client.0.smithi006.stderr:--- begin dump of recent events --- 2024-05-30T16:20:16.082 INFO:tasks.workunit.client.0.smithi006.stderr: -499> 2024-05-30T16:20:15.689+0000 7fbc50a58640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:16.082 INFO:tasks.workunit.client.0.smithi006.stderr: -498> 2024-05-30T16:20:15.689+0000 7fbc4fa56640 10 monclient(hunting): get_auth_request con 0x559468dd6b20 auth_method 0 2024-05-30T16:20:16.082 INFO:tasks.workunit.client.0.smithi006.stderr: -497> 2024-05-30T16:20:15.689+0000 7fbc4fa56640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:16.082 INFO:tasks.workunit.client.0.smithi006.stderr: -496> 2024-05-30T16:20:15.689+0000 7fbc4fa56640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:16.083 INFO:tasks.workunit.client.0.smithi006.stderr: -495> 2024-05-30T16:20:15.689+0000 7fbc4fa56640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:16.083 INFO:tasks.workunit.client.0.smithi006.stderr: -494> 2024-05-30T16:20:15.689+0000 7fbc50a58640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:16.083 INFO:tasks.workunit.client.0.smithi006.stderr: -493> 2024-05-30T16:20:15.689+0000 7fbc50a58640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:16.083 INFO:tasks.workunit.client.0.smithi006.stderr: -492> 2024-05-30T16:20:15.689+0000 7fbc50a58640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:16.083 INFO:tasks.workunit.client.0.smithi006.stderr: -491> 2024-05-30T16:20:15.689+0000 7fbc4fa56640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:16.083 INFO:tasks.workunit.client.0.smithi006.stderr: -490> 2024-05-30T16:20:15.689+0000 7fbc4fa56640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:16.083 INFO:tasks.workunit.client.0.smithi006.stderr: -489> 2024-05-30T16:20:15.689+0000 7fbc4fa56640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:16.083 INFO:tasks.workunit.client.0.smithi006.stderr: -488> 2024-05-30T16:20:15.689+0000 7fbc50a58640 10 monclient(hunting): handle_auth_done global_id 6017 payload 274 2024-05-30T16:20:16.083 INFO:tasks.workunit.client.0.smithi006.stderr: -487> 2024-05-30T16:20:15.689+0000 7fbc50a58640 10 monclient: _finish_hunting 0 2024-05-30T16:20:16.083 INFO:tasks.workunit.client.0.smithi006.stderr: -486> 2024-05-30T16:20:15.689+0000 7fbc50a58640 1 monclient: found mon.noname-c 2024-05-30T16:20:16.083 INFO:tasks.workunit.client.0.smithi006.stderr: -485> 2024-05-30T16:20:15.689+0000 7fbc50a58640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.6:3301/0 2024-05-30T16:20:16.083 INFO:tasks.workunit.client.0.smithi006.stderr: -484> 2024-05-30T16:20:15.689+0000 7fbc50257640 10 monclient: get_auth_request con 0x559468dd5e40 auth_method 0 2024-05-30T16:20:16.083 INFO:tasks.workunit.client.0.smithi006.stderr: -483> 2024-05-30T16:20:15.689+0000 7fbb3e60c640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:16.084 INFO:tasks.workunit.client.0.smithi006.stderr: -482> 2024-05-30T16:20:15.689+0000 7fbb3e60c640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-30T16:20:16.084 INFO:tasks.workunit.client.0.smithi006.stderr: -481> 2024-05-30T16:20:15.689+0000 7fbb3e60c640 10 monclient: dump: 2024-05-30T16:20:16.084 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:16.084 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:16.084 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.084 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.084 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:16.084 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:16.084 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:16.084 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:16.084 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:16.084 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.084 INFO:tasks.workunit.client.0.smithi006.stderr: -480> 2024-05-30T16:20:15.689+0000 7fbb3e60c640 10 monclient: _finish_auth 0 2024-05-30T16:20:16.085 INFO:tasks.workunit.client.0.smithi006.stderr: -479> 2024-05-30T16:20:15.689+0000 7fbb3e60c640 10 monclient: _check_auth_tickets 2024-05-30T16:20:16.085 INFO:tasks.workunit.client.0.smithi006.stderr: -478> 2024-05-30T16:20:15.689+0000 7fbb3e60c640 10 monclient: handle_config config(0 keys) 2024-05-30T16:20:16.085 INFO:tasks.workunit.client.0.smithi006.stderr: -477> 2024-05-30T16:20:15.689+0000 7fbb3e60c640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:16.085 INFO:tasks.workunit.client.0.smithi006.stderr: -476> 2024-05-30T16:20:15.689+0000 7fbb3e60c640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-30T16:20:16.085 INFO:tasks.workunit.client.0.smithi006.stderr: -475> 2024-05-30T16:20:15.689+0000 7fbb3e60c640 10 monclient: dump: 2024-05-30T16:20:16.085 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:16.085 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:16.085 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.085 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.085 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:16.085 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:16.085 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:16.085 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:16.086 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:16.086 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.086 INFO:tasks.workunit.client.0.smithi006.stderr: -474> 2024-05-30T16:20:15.689+0000 7fbc52746980 5 monclient: authenticate success, global_id 6017 2024-05-30T16:20:16.086 INFO:tasks.workunit.client.0.smithi006.stderr: -473> 2024-05-30T16:20:15.689+0000 7fbc52746980 10 monclient: get_monmap_and_config success 2024-05-30T16:20:16.086 INFO:tasks.workunit.client.0.smithi006.stderr: -472> 2024-05-30T16:20:15.689+0000 7fbc52746980 4 set_mon_vals no callback set 2024-05-30T16:20:16.086 INFO:tasks.workunit.client.0.smithi006.stderr: -471> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient: shutdown 2024-05-30T16:20:16.086 INFO:tasks.workunit.client.0.smithi006.stderr: -470> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 asok(0x559468c7f790) unregister_commands rotate-key 2024-05-30T16:20:16.086 INFO:tasks.workunit.client.0.smithi006.stderr: -469> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient: build_initial_monmap 2024-05-30T16:20:16.086 INFO:tasks.workunit.client.0.smithi006.stderr: -468> 2024-05-30T16:20:15.693+0000 7fbc52746980 1 build_initial for_mkfs: 0 2024-05-30T16:20:16.086 INFO:tasks.workunit.client.0.smithi006.stderr: -467> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient: monmap: 2024-05-30T16:20:16.086 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 0 2024-05-30T16:20:16.086 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-30T16:20:16.087 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 0.000000 2024-05-30T16:20:16.087 INFO:tasks.workunit.client.0.smithi006.stderr:created 0.000000 2024-05-30T16:20:16.087 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 0 (unknown) 2024-05-30T16:20:16.087 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:16.087 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.noname-a 2024-05-30T16:20:16.087 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-b 2024-05-30T16:20:16.087 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.noname-c 2024-05-30T16:20:16.087 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.087 INFO:tasks.workunit.client.0.smithi006.stderr: -466> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e909e0) adding auth protocol: cephx 2024-05-30T16:20:16.087 INFO:tasks.workunit.client.0.smithi006.stderr: -465> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e909e0) adding auth protocol: cephx 2024-05-30T16:20:16.088 INFO:tasks.workunit.client.0.smithi006.stderr: -464> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e909e0) adding auth protocol: cephx 2024-05-30T16:20:16.088 INFO:tasks.workunit.client.0.smithi006.stderr: -463> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e909e0) adding con mode: secure 2024-05-30T16:20:16.088 INFO:tasks.workunit.client.0.smithi006.stderr: -462> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e909e0) adding con mode: crc 2024-05-30T16:20:16.088 INFO:tasks.workunit.client.0.smithi006.stderr: -461> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e909e0) adding con mode: secure 2024-05-30T16:20:16.088 INFO:tasks.workunit.client.0.smithi006.stderr: -460> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e909e0) adding con mode: crc 2024-05-30T16:20:16.088 INFO:tasks.workunit.client.0.smithi006.stderr: -459> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e909e0) adding con mode: secure 2024-05-30T16:20:16.088 INFO:tasks.workunit.client.0.smithi006.stderr: -458> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e909e0) adding con mode: crc 2024-05-30T16:20:16.088 INFO:tasks.workunit.client.0.smithi006.stderr: -457> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e909e0) adding con mode: crc 2024-05-30T16:20:16.088 INFO:tasks.workunit.client.0.smithi006.stderr: -456> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e909e0) adding con mode: secure 2024-05-30T16:20:16.088 INFO:tasks.workunit.client.0.smithi006.stderr: -455> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e909e0) adding con mode: crc 2024-05-30T16:20:16.088 INFO:tasks.workunit.client.0.smithi006.stderr: -454> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e909e0) adding con mode: secure 2024-05-30T16:20:16.089 INFO:tasks.workunit.client.0.smithi006.stderr: -453> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e909e0) adding con mode: crc 2024-05-30T16:20:16.089 INFO:tasks.workunit.client.0.smithi006.stderr: -452> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e909e0) adding con mode: secure 2024-05-30T16:20:16.089 INFO:tasks.workunit.client.0.smithi006.stderr: -451> 2024-05-30T16:20:15.693+0000 7fbc52746980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:16.089 INFO:tasks.workunit.client.0.smithi006.stderr: -450> 2024-05-30T16:20:15.693+0000 7fbc52746980 1 librados: starting msgr at 2024-05-30T16:20:16.089 INFO:tasks.workunit.client.0.smithi006.stderr: -449> 2024-05-30T16:20:15.693+0000 7fbc52746980 1 librados: starting objecter 2024-05-30T16:20:16.089 INFO:tasks.workunit.client.0.smithi006.stderr: -448> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 asok(0x559468c7f790) register_command objecter_requests hook 0x559468bfaa30 2024-05-30T16:20:16.089 INFO:tasks.workunit.client.0.smithi006.stderr: -447> 2024-05-30T16:20:15.693+0000 7fbc52746980 1 librados: setting wanted keys 2024-05-30T16:20:16.089 INFO:tasks.workunit.client.0.smithi006.stderr: -446> 2024-05-30T16:20:15.693+0000 7fbc52746980 1 librados: calling monclient init 2024-05-30T16:20:16.089 INFO:tasks.workunit.client.0.smithi006.stderr: -445> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient: init 2024-05-30T16:20:16.089 INFO:tasks.workunit.client.0.smithi006.stderr: -444> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e8b4b0) adding auth protocol: cephx 2024-05-30T16:20:16.089 INFO:tasks.workunit.client.0.smithi006.stderr: -443> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e8b4b0) adding auth protocol: cephx 2024-05-30T16:20:16.090 INFO:tasks.workunit.client.0.smithi006.stderr: -442> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e8b4b0) adding auth protocol: cephx 2024-05-30T16:20:16.090 INFO:tasks.workunit.client.0.smithi006.stderr: -441> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e8b4b0) adding con mode: secure 2024-05-30T16:20:16.090 INFO:tasks.workunit.client.0.smithi006.stderr: -440> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e8b4b0) adding con mode: crc 2024-05-30T16:20:16.090 INFO:tasks.workunit.client.0.smithi006.stderr: -439> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e8b4b0) adding con mode: secure 2024-05-30T16:20:16.090 INFO:tasks.workunit.client.0.smithi006.stderr: -438> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e8b4b0) adding con mode: crc 2024-05-30T16:20:16.090 INFO:tasks.workunit.client.0.smithi006.stderr: -437> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e8b4b0) adding con mode: secure 2024-05-30T16:20:16.090 INFO:tasks.workunit.client.0.smithi006.stderr: -436> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e8b4b0) adding con mode: crc 2024-05-30T16:20:16.090 INFO:tasks.workunit.client.0.smithi006.stderr: -435> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e8b4b0) adding con mode: crc 2024-05-30T16:20:16.090 INFO:tasks.workunit.client.0.smithi006.stderr: -434> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e8b4b0) adding con mode: secure 2024-05-30T16:20:16.090 INFO:tasks.workunit.client.0.smithi006.stderr: -433> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e8b4b0) adding con mode: crc 2024-05-30T16:20:16.090 INFO:tasks.workunit.client.0.smithi006.stderr: -432> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e8b4b0) adding con mode: secure 2024-05-30T16:20:16.091 INFO:tasks.workunit.client.0.smithi006.stderr: -431> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e8b4b0) adding con mode: crc 2024-05-30T16:20:16.091 INFO:tasks.workunit.client.0.smithi006.stderr: -430> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 AuthRegistry(0x559468e8b4b0) adding con mode: secure 2024-05-30T16:20:16.091 INFO:tasks.workunit.client.0.smithi006.stderr: -429> 2024-05-30T16:20:15.693+0000 7fbc52746980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:16.091 INFO:tasks.workunit.client.0.smithi006.stderr: -428> 2024-05-30T16:20:15.693+0000 7fbc52746980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:16.091 INFO:tasks.workunit.client.0.smithi006.stderr: -427> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 asok(0x559468c7f790) register_command rotate-key hook 0x559468e8b5f8 2024-05-30T16:20:16.091 INFO:tasks.workunit.client.0.smithi006.stderr: -426> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient: _reopen_session rank -1 2024-05-30T16:20:16.091 INFO:tasks.workunit.client.0.smithi006.stderr: -425> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient: _add_conns ranks=[0,1,2] 2024-05-30T16:20:16.091 INFO:tasks.workunit.client.0.smithi006.stderr: -424> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient(hunting): picked mon.noname-a con 0x559468dd5e40 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:20:16.091 INFO:tasks.workunit.client.0.smithi006.stderr: -423> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient(hunting): picked mon.noname-b con 0x559468dd6b20 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:20:16.091 INFO:tasks.workunit.client.0.smithi006.stderr: -422> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient(hunting): picked mon.noname-c con 0x559468eb8920 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:20:16.091 INFO:tasks.workunit.client.0.smithi006.stderr: -421> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:16.091 INFO:tasks.workunit.client.0.smithi006.stderr: -420> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:16.092 INFO:tasks.workunit.client.0.smithi006.stderr: -419> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:16.092 INFO:tasks.workunit.client.0.smithi006.stderr: -418> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient(hunting): _renew_subs 2024-05-30T16:20:16.092 INFO:tasks.workunit.client.0.smithi006.stderr: -417> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient(hunting): authenticate will time out at 2555.629599s 2024-05-30T16:20:16.092 INFO:tasks.workunit.client.0.smithi006.stderr: -416> 2024-05-30T16:20:15.693+0000 7fbc50257640 10 monclient(hunting): get_auth_request con 0x559468dd5e40 auth_method 0 2024-05-30T16:20:16.092 INFO:tasks.workunit.client.0.smithi006.stderr: -415> 2024-05-30T16:20:15.693+0000 7fbc50257640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:16.092 INFO:tasks.workunit.client.0.smithi006.stderr: -414> 2024-05-30T16:20:15.693+0000 7fbc50257640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:16.092 INFO:tasks.workunit.client.0.smithi006.stderr: -413> 2024-05-30T16:20:15.693+0000 7fbc50257640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:16.092 INFO:tasks.workunit.client.0.smithi006.stderr: -412> 2024-05-30T16:20:15.693+0000 7fbc50a58640 10 monclient(hunting): get_auth_request con 0x559468eb8920 auth_method 0 2024-05-30T16:20:16.092 INFO:tasks.workunit.client.0.smithi006.stderr: -411> 2024-05-30T16:20:15.693+0000 7fbc50a58640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:16.092 INFO:tasks.workunit.client.0.smithi006.stderr: -410> 2024-05-30T16:20:15.693+0000 7fbc50a58640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:16.093 INFO:tasks.workunit.client.0.smithi006.stderr: -409> 2024-05-30T16:20:15.693+0000 7fbc50a58640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:16.093 INFO:tasks.workunit.client.0.smithi006.stderr: -408> 2024-05-30T16:20:15.693+0000 7fbc50257640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:16.093 INFO:tasks.workunit.client.0.smithi006.stderr: -407> 2024-05-30T16:20:15.693+0000 7fbc50257640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:16.093 INFO:tasks.workunit.client.0.smithi006.stderr: -406> 2024-05-30T16:20:15.693+0000 7fbc50257640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:16.093 INFO:tasks.workunit.client.0.smithi006.stderr: -405> 2024-05-30T16:20:15.693+0000 7fbc50a58640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:16.093 INFO:tasks.workunit.client.0.smithi006.stderr: -404> 2024-05-30T16:20:15.693+0000 7fbc50a58640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:16.093 INFO:tasks.workunit.client.0.smithi006.stderr: -403> 2024-05-30T16:20:15.693+0000 7fbc50a58640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:16.093 INFO:tasks.workunit.client.0.smithi006.stderr: -402> 2024-05-30T16:20:15.693+0000 7fbc50257640 10 monclient(hunting): handle_auth_done global_id 6285 payload 824 2024-05-30T16:20:16.093 INFO:tasks.workunit.client.0.smithi006.stderr: -401> 2024-05-30T16:20:15.693+0000 7fbc50257640 10 monclient: _finish_hunting 0 2024-05-30T16:20:16.093 INFO:tasks.workunit.client.0.smithi006.stderr: -400> 2024-05-30T16:20:15.693+0000 7fbc50257640 1 monclient: found mon.noname-a 2024-05-30T16:20:16.093 INFO:tasks.workunit.client.0.smithi006.stderr: -399> 2024-05-30T16:20:15.693+0000 7fbc50257640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.6:3300/0 2024-05-30T16:20:16.094 INFO:tasks.workunit.client.0.smithi006.stderr: -398> 2024-05-30T16:20:15.693+0000 7fbc4fa56640 10 monclient: get_auth_request con 0x559468dd6b20 auth_method 0 2024-05-30T16:20:16.094 INFO:tasks.workunit.client.0.smithi006.stderr: -397> 2024-05-30T16:20:15.693+0000 7fbb3ce09640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:16.094 INFO:tasks.workunit.client.0.smithi006.stderr: -396> 2024-05-30T16:20:15.693+0000 7fbb3ce09640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-30T16:20:16.094 INFO:tasks.workunit.client.0.smithi006.stderr: -395> 2024-05-30T16:20:15.693+0000 7fbb3ce09640 10 monclient: dump: 2024-05-30T16:20:16.094 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:16.094 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:16.094 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.094 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.094 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:16.094 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:16.094 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:16.094 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:16.095 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:16.095 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.095 INFO:tasks.workunit.client.0.smithi006.stderr: -394> 2024-05-30T16:20:15.693+0000 7fbb3ce09640 10 monclient: _finish_auth 0 2024-05-30T16:20:16.095 INFO:tasks.workunit.client.0.smithi006.stderr: -393> 2024-05-30T16:20:15.693+0000 7fbb3ce09640 10 monclient: _check_auth_tickets 2024-05-30T16:20:16.095 INFO:tasks.workunit.client.0.smithi006.stderr: -392> 2024-05-30T16:20:15.693+0000 7fbb3ce09640 10 monclient: handle_config config(0 keys) 2024-05-30T16:20:16.095 INFO:tasks.workunit.client.0.smithi006.stderr: -391> 2024-05-30T16:20:15.693+0000 7fbb3ce09640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:16.095 INFO:tasks.workunit.client.0.smithi006.stderr: -390> 2024-05-30T16:20:15.693+0000 7fbb3ce09640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-30T16:20:16.095 INFO:tasks.workunit.client.0.smithi006.stderr: -389> 2024-05-30T16:20:15.693+0000 7fbb3ce09640 10 monclient: dump: 2024-05-30T16:20:16.095 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:16.095 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:16.096 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.096 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.096 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:16.096 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:16.096 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:16.096 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:16.096 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:16.096 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.096 INFO:tasks.workunit.client.0.smithi006.stderr: -388> 2024-05-30T16:20:15.693+0000 7fbb3e60c640 4 set_mon_vals no callback set 2024-05-30T16:20:16.097 INFO:tasks.workunit.client.0.smithi006.stderr: -387> 2024-05-30T16:20:15.693+0000 7fbc52746980 5 monclient: authenticate success, global_id 6285 2024-05-30T16:20:16.097 INFO:tasks.workunit.client.0.smithi006.stderr: -386> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient: _renew_subs 2024-05-30T16:20:16.097 INFO:tasks.workunit.client.0.smithi006.stderr: -385> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:20:16.097 INFO:tasks.workunit.client.0.smithi006.stderr: -384> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient: _renew_subs 2024-05-30T16:20:16.097 INFO:tasks.workunit.client.0.smithi006.stderr: -383> 2024-05-30T16:20:15.693+0000 7fbc52746980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:20:16.097 INFO:tasks.workunit.client.0.smithi006.stderr: -382> 2024-05-30T16:20:15.693+0000 7fbc52746980 1 librados: init done 2024-05-30T16:20:16.097 INFO:tasks.workunit.client.0.smithi006.stderr: -381> 2024-05-30T16:20:15.697+0000 7fbb3ce09640 4 mgrc handle_mgr_map Got map version 5 2024-05-30T16:20:16.097 INFO:tasks.workunit.client.0.smithi006.stderr: -380> 2024-05-30T16:20:15.697+0000 7fbb3ce09640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:20:16.097 INFO:tasks.workunit.client.0.smithi006.stderr: -379> 2024-05-30T16:20:15.697+0000 7fbb3ce09640 4 mgrc reconnect Starting new session with [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:20:16.097 INFO:tasks.workunit.client.0.smithi006.stderr: -378> 2024-05-30T16:20:15.697+0000 7fbc4fa56640 10 monclient: get_auth_request con 0x7fbb3806de10 auth_method 0 2024-05-30T16:20:16.097 INFO:tasks.workunit.client.0.smithi006.stderr: -377> 2024-05-30T16:20:15.697+0000 7fbc50a58640 10 monclient: get_auth_request con 0x559468ec8930 auth_method 0 2024-05-30T16:20:16.097 INFO:tasks.workunit.client.0.smithi006.stderr: -376> 2024-05-30T16:20:15.697+0000 7fbc50257640 10 monclient: get_auth_request con 0x559468ed5f80 auth_method 0 2024-05-30T16:20:16.098 INFO:tasks.workunit.client.0.smithi006.stderr: -375> 2024-05-30T16:20:15.701+0000 7fbc4fa56640 10 monclient: get_auth_request con 0x559468edb010 auth_method 0 2024-05-30T16:20:16.098 INFO:tasks.workunit.client.0.smithi006.stderr: -374> 2024-05-30T16:20:15.701+0000 7fbc52746980 10 monclient: build_initial_monmap 2024-05-30T16:20:16.098 INFO:tasks.workunit.client.0.smithi006.stderr: -373> 2024-05-30T16:20:15.701+0000 7fbc52746980 1 build_initial for_mkfs: 0 2024-05-30T16:20:16.098 INFO:tasks.workunit.client.0.smithi006.stderr: -372> 2024-05-30T16:20:15.701+0000 7fbc52746980 10 monclient: monmap: 2024-05-30T16:20:16.098 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 0 2024-05-30T16:20:16.098 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-30T16:20:16.099 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 0.000000 2024-05-30T16:20:16.099 INFO:tasks.workunit.client.0.smithi006.stderr:created 0.000000 2024-05-30T16:20:16.099 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 0 (unknown) 2024-05-30T16:20:16.099 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:16.099 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.noname-a 2024-05-30T16:20:16.099 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-b 2024-05-30T16:20:16.099 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.noname-c 2024-05-30T16:20:16.099 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.099 INFO:tasks.workunit.client.0.smithi006.stderr: -371> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 AuthRegistry(0x559468edee90) adding auth protocol: cephx 2024-05-30T16:20:16.100 INFO:tasks.workunit.client.0.smithi006.stderr: -370> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 AuthRegistry(0x559468edee90) adding auth protocol: cephx 2024-05-30T16:20:16.100 INFO:tasks.workunit.client.0.smithi006.stderr: -369> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 AuthRegistry(0x559468edee90) adding auth protocol: cephx 2024-05-30T16:20:16.100 INFO:tasks.workunit.client.0.smithi006.stderr: -368> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 AuthRegistry(0x559468edee90) adding con mode: secure 2024-05-30T16:20:16.100 INFO:tasks.workunit.client.0.smithi006.stderr: -367> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 AuthRegistry(0x559468edee90) adding con mode: crc 2024-05-30T16:20:16.100 INFO:tasks.workunit.client.0.smithi006.stderr: -366> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 AuthRegistry(0x559468edee90) adding con mode: secure 2024-05-30T16:20:16.100 INFO:tasks.workunit.client.0.smithi006.stderr: -365> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 AuthRegistry(0x559468edee90) adding con mode: crc 2024-05-30T16:20:16.100 INFO:tasks.workunit.client.0.smithi006.stderr: -364> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 AuthRegistry(0x559468edee90) adding con mode: secure 2024-05-30T16:20:16.100 INFO:tasks.workunit.client.0.smithi006.stderr: -363> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 AuthRegistry(0x559468edee90) adding con mode: crc 2024-05-30T16:20:16.100 INFO:tasks.workunit.client.0.smithi006.stderr: -362> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 AuthRegistry(0x559468edee90) adding con mode: crc 2024-05-30T16:20:16.100 INFO:tasks.workunit.client.0.smithi006.stderr: -361> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 AuthRegistry(0x559468edee90) adding con mode: secure 2024-05-30T16:20:16.100 INFO:tasks.workunit.client.0.smithi006.stderr: -360> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 AuthRegistry(0x559468edee90) adding con mode: crc 2024-05-30T16:20:16.101 INFO:tasks.workunit.client.0.smithi006.stderr: -359> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 AuthRegistry(0x559468edee90) adding con mode: secure 2024-05-30T16:20:16.101 INFO:tasks.workunit.client.0.smithi006.stderr: -358> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 AuthRegistry(0x559468edee90) adding con mode: crc 2024-05-30T16:20:16.101 INFO:tasks.workunit.client.0.smithi006.stderr: -357> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 AuthRegistry(0x559468edee90) adding con mode: secure 2024-05-30T16:20:16.101 INFO:tasks.workunit.client.0.smithi006.stderr: -356> 2024-05-30T16:20:15.701+0000 7fbc52746980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:16.101 INFO:tasks.workunit.client.0.smithi006.stderr: -355> 2024-05-30T16:20:15.701+0000 7fbc52746980 5 asok(0x559468c7f790) register_command objecter_requests cmddesc objecter_requests hook 0x559468ee2490 EEXIST 2024-05-30T16:20:16.101 INFO:tasks.workunit.client.0.smithi006.stderr: -354> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient: init 2024-05-30T16:20:16.101 INFO:tasks.workunit.client.0.smithi006.stderr: -353> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468edde28) adding auth protocol: cephx 2024-05-30T16:20:16.101 INFO:tasks.workunit.client.0.smithi006.stderr: -352> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468edde28) adding auth protocol: cephx 2024-05-30T16:20:16.101 INFO:tasks.workunit.client.0.smithi006.stderr: -351> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468edde28) adding auth protocol: cephx 2024-05-30T16:20:16.101 INFO:tasks.workunit.client.0.smithi006.stderr: -350> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468edde28) adding con mode: secure 2024-05-30T16:20:16.101 INFO:tasks.workunit.client.0.smithi006.stderr: -349> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468edde28) adding con mode: crc 2024-05-30T16:20:16.101 INFO:tasks.workunit.client.0.smithi006.stderr: -348> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468edde28) adding con mode: secure 2024-05-30T16:20:16.102 INFO:tasks.workunit.client.0.smithi006.stderr: -347> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468edde28) adding con mode: crc 2024-05-30T16:20:16.102 INFO:tasks.workunit.client.0.smithi006.stderr: -346> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468edde28) adding con mode: secure 2024-05-30T16:20:16.102 INFO:tasks.workunit.client.0.smithi006.stderr: -345> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468edde28) adding con mode: crc 2024-05-30T16:20:16.102 INFO:tasks.workunit.client.0.smithi006.stderr: -344> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468edde28) adding con mode: crc 2024-05-30T16:20:16.102 INFO:tasks.workunit.client.0.smithi006.stderr: -343> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468edde28) adding con mode: secure 2024-05-30T16:20:16.102 INFO:tasks.workunit.client.0.smithi006.stderr: -342> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468edde28) adding con mode: crc 2024-05-30T16:20:16.102 INFO:tasks.workunit.client.0.smithi006.stderr: -341> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468edde28) adding con mode: secure 2024-05-30T16:20:16.102 INFO:tasks.workunit.client.0.smithi006.stderr: -340> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468edde28) adding con mode: crc 2024-05-30T16:20:16.102 INFO:tasks.workunit.client.0.smithi006.stderr: -339> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468edde28) adding con mode: secure 2024-05-30T16:20:16.102 INFO:tasks.workunit.client.0.smithi006.stderr: -338> 2024-05-30T16:20:15.705+0000 7fbc52746980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:16.102 INFO:tasks.workunit.client.0.smithi006.stderr: -337> 2024-05-30T16:20:15.705+0000 7fbc52746980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:16.103 INFO:tasks.workunit.client.0.smithi006.stderr: -336> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 asok(0x559468c7f790) register_command rotate-key cmddesc rotate-key hook 0x559468eddf70 EEXIST 2024-05-30T16:20:16.103 INFO:tasks.workunit.client.0.smithi006.stderr: -335> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient: _reopen_session rank -1 2024-05-30T16:20:16.103 INFO:tasks.workunit.client.0.smithi006.stderr: -334> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient: _add_conns ranks=[2,1,0] 2024-05-30T16:20:16.103 INFO:tasks.workunit.client.0.smithi006.stderr: -333> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient(hunting): picked mon.noname-c con 0x559468ee8a20 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:20:16.103 INFO:tasks.workunit.client.0.smithi006.stderr: -332> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient(hunting): picked mon.noname-b con 0x559468ee9350 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:20:16.103 INFO:tasks.workunit.client.0.smithi006.stderr: -331> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient(hunting): picked mon.noname-a con 0x559468ee8e00 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:20:16.103 INFO:tasks.workunit.client.0.smithi006.stderr: -330> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:16.103 INFO:tasks.workunit.client.0.smithi006.stderr: -329> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:16.103 INFO:tasks.workunit.client.0.smithi006.stderr: -328> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:16.103 INFO:tasks.workunit.client.0.smithi006.stderr: -327> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient(hunting): _renew_subs 2024-05-30T16:20:16.103 INFO:tasks.workunit.client.0.smithi006.stderr: -326> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient(hunting): authenticate will time out at 2555.640608s 2024-05-30T16:20:16.103 INFO:tasks.workunit.client.0.smithi006.stderr: -325> 2024-05-30T16:20:15.705+0000 7fbc50a58640 10 monclient(hunting): get_auth_request con 0x559468ee8e00 auth_method 0 2024-05-30T16:20:16.103 INFO:tasks.workunit.client.0.smithi006.stderr: -324> 2024-05-30T16:20:15.705+0000 7fbc50a58640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:16.104 INFO:tasks.workunit.client.0.smithi006.stderr: -323> 2024-05-30T16:20:15.705+0000 7fbc50a58640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:16.104 INFO:tasks.workunit.client.0.smithi006.stderr: -322> 2024-05-30T16:20:15.705+0000 7fbc50a58640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:16.104 INFO:tasks.workunit.client.0.smithi006.stderr: -321> 2024-05-30T16:20:15.705+0000 7fbc50257640 10 monclient(hunting): get_auth_request con 0x559468ee8a20 auth_method 0 2024-05-30T16:20:16.104 INFO:tasks.workunit.client.0.smithi006.stderr: -320> 2024-05-30T16:20:15.705+0000 7fbc50257640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:16.104 INFO:tasks.workunit.client.0.smithi006.stderr: -319> 2024-05-30T16:20:15.705+0000 7fbc50257640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:16.104 INFO:tasks.workunit.client.0.smithi006.stderr: -318> 2024-05-30T16:20:15.705+0000 7fbc50257640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:16.104 INFO:tasks.workunit.client.0.smithi006.stderr: -317> 2024-05-30T16:20:15.705+0000 7fbc50a58640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:16.104 INFO:tasks.workunit.client.0.smithi006.stderr: -316> 2024-05-30T16:20:15.705+0000 7fbc50a58640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:16.104 INFO:tasks.workunit.client.0.smithi006.stderr: -315> 2024-05-30T16:20:15.705+0000 7fbc50a58640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:16.104 INFO:tasks.workunit.client.0.smithi006.stderr: -314> 2024-05-30T16:20:15.705+0000 7fbc50257640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:16.104 INFO:tasks.workunit.client.0.smithi006.stderr: -313> 2024-05-30T16:20:15.705+0000 7fbc50257640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:16.104 INFO:tasks.workunit.client.0.smithi006.stderr: -312> 2024-05-30T16:20:15.705+0000 7fbc50257640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:16.104 INFO:tasks.workunit.client.0.smithi006.stderr: -311> 2024-05-30T16:20:15.705+0000 7fbc4fa56640 10 monclient(hunting): get_auth_request con 0x559468ee9350 auth_method 0 2024-05-30T16:20:16.105 INFO:tasks.workunit.client.0.smithi006.stderr: -310> 2024-05-30T16:20:15.705+0000 7fbc4fa56640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:16.105 INFO:tasks.workunit.client.0.smithi006.stderr: -309> 2024-05-30T16:20:15.705+0000 7fbc4fa56640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:16.105 INFO:tasks.workunit.client.0.smithi006.stderr: -308> 2024-05-30T16:20:15.705+0000 7fbc4fa56640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:16.105 INFO:tasks.workunit.client.0.smithi006.stderr: -307> 2024-05-30T16:20:15.705+0000 7fbc50a58640 10 monclient(hunting): handle_auth_done global_id 6288 payload 824 2024-05-30T16:20:16.105 INFO:tasks.workunit.client.0.smithi006.stderr: -306> 2024-05-30T16:20:15.705+0000 7fbc50a58640 10 monclient: _finish_hunting 0 2024-05-30T16:20:16.105 INFO:tasks.workunit.client.0.smithi006.stderr: -305> 2024-05-30T16:20:15.705+0000 7fbc50a58640 1 monclient: found mon.noname-a 2024-05-30T16:20:16.105 INFO:tasks.workunit.client.0.smithi006.stderr: -304> 2024-05-30T16:20:15.705+0000 7fbc50a58640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.6:3300/0 2024-05-30T16:20:16.105 INFO:tasks.workunit.client.0.smithi006.stderr: -303> 2024-05-30T16:20:15.705+0000 7fbb35ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:16.105 INFO:tasks.workunit.client.0.smithi006.stderr: -302> 2024-05-30T16:20:15.705+0000 7fbb35ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-30T16:20:16.105 INFO:tasks.workunit.client.0.smithi006.stderr: -301> 2024-05-30T16:20:15.705+0000 7fbb35ffb640 10 monclient: dump: 2024-05-30T16:20:16.105 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:16.105 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:16.106 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.106 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.106 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:16.106 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:16.106 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:16.106 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:16.106 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:16.106 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.106 INFO:tasks.workunit.client.0.smithi006.stderr: -300> 2024-05-30T16:20:15.705+0000 7fbb35ffb640 10 monclient: _finish_auth 0 2024-05-30T16:20:16.106 INFO:tasks.workunit.client.0.smithi006.stderr: -299> 2024-05-30T16:20:15.705+0000 7fbb35ffb640 10 monclient: _check_auth_tickets 2024-05-30T16:20:16.106 INFO:tasks.workunit.client.0.smithi006.stderr: -298> 2024-05-30T16:20:15.705+0000 7fbb35ffb640 10 monclient: handle_config config(0 keys) 2024-05-30T16:20:16.106 INFO:tasks.workunit.client.0.smithi006.stderr: -297> 2024-05-30T16:20:15.705+0000 7fbb35ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:16.106 INFO:tasks.workunit.client.0.smithi006.stderr: -296> 2024-05-30T16:20:15.705+0000 7fbb35ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-30T16:20:16.107 INFO:tasks.workunit.client.0.smithi006.stderr: -295> 2024-05-30T16:20:15.705+0000 7fbb35ffb640 10 monclient: dump: 2024-05-30T16:20:16.107 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:16.107 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:16.107 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.107 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.107 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:16.107 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:16.107 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:16.107 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:16.107 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:16.107 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.107 INFO:tasks.workunit.client.0.smithi006.stderr: -294> 2024-05-30T16:20:15.705+0000 7fbc51259640 4 set_mon_vals no callback set 2024-05-30T16:20:16.107 INFO:tasks.workunit.client.0.smithi006.stderr: -293> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 monclient: authenticate success, global_id 6288 2024-05-30T16:20:16.108 INFO:tasks.workunit.client.0.smithi006.stderr: -292> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient: _renew_subs 2024-05-30T16:20:16.108 INFO:tasks.workunit.client.0.smithi006.stderr: -291> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:20:16.108 INFO:tasks.workunit.client.0.smithi006.stderr: -290> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient: _renew_subs 2024-05-30T16:20:16.108 INFO:tasks.workunit.client.0.smithi006.stderr: -289> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:20:16.108 INFO:tasks.workunit.client.0.smithi006.stderr: -288> 2024-05-30T16:20:15.705+0000 7fbb35ffb640 4 mgrc handle_mgr_map Got map version 5 2024-05-30T16:20:16.108 INFO:tasks.workunit.client.0.smithi006.stderr: -287> 2024-05-30T16:20:15.705+0000 7fbb35ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:20:16.108 INFO:tasks.workunit.client.0.smithi006.stderr: -286> 2024-05-30T16:20:15.705+0000 7fbb35ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:20:16.108 INFO:tasks.workunit.client.0.smithi006.stderr: -285> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient: get_monmap_and_config 2024-05-30T16:20:16.108 INFO:tasks.workunit.client.0.smithi006.stderr: -284> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient: build_initial_monmap 2024-05-30T16:20:16.108 INFO:tasks.workunit.client.0.smithi006.stderr: -283> 2024-05-30T16:20:15.705+0000 7fbc52746980 1 build_initial for_mkfs: 0 2024-05-30T16:20:16.108 INFO:tasks.workunit.client.0.smithi006.stderr: -282> 2024-05-30T16:20:15.705+0000 7fbc52746980 10 monclient: monmap: 2024-05-30T16:20:16.108 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 0 2024-05-30T16:20:16.109 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-30T16:20:16.109 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 0.000000 2024-05-30T16:20:16.109 INFO:tasks.workunit.client.0.smithi006.stderr:created 0.000000 2024-05-30T16:20:16.109 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 0 (unknown) 2024-05-30T16:20:16.109 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:16.109 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.noname-a 2024-05-30T16:20:16.109 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-b 2024-05-30T16:20:16.109 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.noname-c 2024-05-30T16:20:16.109 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.109 INFO:tasks.workunit.client.0.smithi006.stderr: -281> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468f26220) adding auth protocol: cephx 2024-05-30T16:20:16.109 INFO:tasks.workunit.client.0.smithi006.stderr: -280> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468f26220) adding auth protocol: cephx 2024-05-30T16:20:16.109 INFO:tasks.workunit.client.0.smithi006.stderr: -279> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468f26220) adding auth protocol: cephx 2024-05-30T16:20:16.109 INFO:tasks.workunit.client.0.smithi006.stderr: -278> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468f26220) adding con mode: secure 2024-05-30T16:20:16.109 INFO:tasks.workunit.client.0.smithi006.stderr: -277> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468f26220) adding con mode: crc 2024-05-30T16:20:16.110 INFO:tasks.workunit.client.0.smithi006.stderr: -276> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468f26220) adding con mode: secure 2024-05-30T16:20:16.110 INFO:tasks.workunit.client.0.smithi006.stderr: -275> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468f26220) adding con mode: crc 2024-05-30T16:20:16.110 INFO:tasks.workunit.client.0.smithi006.stderr: -274> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468f26220) adding con mode: secure 2024-05-30T16:20:16.110 INFO:tasks.workunit.client.0.smithi006.stderr: -273> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468f26220) adding con mode: crc 2024-05-30T16:20:16.110 INFO:tasks.workunit.client.0.smithi006.stderr: -272> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468f26220) adding con mode: crc 2024-05-30T16:20:16.110 INFO:tasks.workunit.client.0.smithi006.stderr: -271> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468f26220) adding con mode: secure 2024-05-30T16:20:16.110 INFO:tasks.workunit.client.0.smithi006.stderr: -270> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468f26220) adding con mode: crc 2024-05-30T16:20:16.110 INFO:tasks.workunit.client.0.smithi006.stderr: -269> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468f26220) adding con mode: secure 2024-05-30T16:20:16.110 INFO:tasks.workunit.client.0.smithi006.stderr: -268> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468f26220) adding con mode: crc 2024-05-30T16:20:16.110 INFO:tasks.workunit.client.0.smithi006.stderr: -267> 2024-05-30T16:20:15.705+0000 7fbc52746980 5 AuthRegistry(0x559468f26220) adding con mode: secure 2024-05-30T16:20:16.110 INFO:tasks.workunit.client.0.smithi006.stderr: -266> 2024-05-30T16:20:15.705+0000 7fbc50257640 10 monclient: get_auth_request con 0x7fbb2806dcf0 auth_method 0 2024-05-30T16:20:16.110 INFO:tasks.workunit.client.0.smithi006.stderr: -265> 2024-05-30T16:20:15.709+0000 7fbc52746980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:16.111 INFO:tasks.workunit.client.0.smithi006.stderr: -264> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient: init 2024-05-30T16:20:16.111 INFO:tasks.workunit.client.0.smithi006.stderr: -263> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3aa0) adding auth protocol: cephx 2024-05-30T16:20:16.111 INFO:tasks.workunit.client.0.smithi006.stderr: -262> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3aa0) adding auth protocol: cephx 2024-05-30T16:20:16.111 INFO:tasks.workunit.client.0.smithi006.stderr: -261> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3aa0) adding auth protocol: cephx 2024-05-30T16:20:16.111 INFO:tasks.workunit.client.0.smithi006.stderr: -260> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3aa0) adding con mode: secure 2024-05-30T16:20:16.111 INFO:tasks.workunit.client.0.smithi006.stderr: -259> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3aa0) adding con mode: crc 2024-05-30T16:20:16.111 INFO:tasks.workunit.client.0.smithi006.stderr: -258> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3aa0) adding con mode: secure 2024-05-30T16:20:16.111 INFO:tasks.workunit.client.0.smithi006.stderr: -257> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3aa0) adding con mode: crc 2024-05-30T16:20:16.111 INFO:tasks.workunit.client.0.smithi006.stderr: -256> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3aa0) adding con mode: secure 2024-05-30T16:20:16.111 INFO:tasks.workunit.client.0.smithi006.stderr: -255> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3aa0) adding con mode: crc 2024-05-30T16:20:16.111 INFO:tasks.workunit.client.0.smithi006.stderr: -254> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3aa0) adding con mode: crc 2024-05-30T16:20:16.111 INFO:tasks.workunit.client.0.smithi006.stderr: -253> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3aa0) adding con mode: secure 2024-05-30T16:20:16.111 INFO:tasks.workunit.client.0.smithi006.stderr: -252> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3aa0) adding con mode: crc 2024-05-30T16:20:16.112 INFO:tasks.workunit.client.0.smithi006.stderr: -251> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3aa0) adding con mode: secure 2024-05-30T16:20:16.112 INFO:tasks.workunit.client.0.smithi006.stderr: -250> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3aa0) adding con mode: crc 2024-05-30T16:20:16.112 INFO:tasks.workunit.client.0.smithi006.stderr: -249> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3aa0) adding con mode: secure 2024-05-30T16:20:16.112 INFO:tasks.workunit.client.0.smithi006.stderr: -248> 2024-05-30T16:20:15.709+0000 7fbc52746980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:16.112 INFO:tasks.workunit.client.0.smithi006.stderr: -247> 2024-05-30T16:20:15.709+0000 7fbc52746980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:16.112 INFO:tasks.workunit.client.0.smithi006.stderr: -246> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 asok(0x559468c7f790) register_command rotate-key cmddesc rotate-key hook 0x7ffd6b8c3be8 EEXIST 2024-05-30T16:20:16.112 INFO:tasks.workunit.client.0.smithi006.stderr: -245> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient: _reopen_session rank -1 2024-05-30T16:20:16.112 INFO:tasks.workunit.client.0.smithi006.stderr: -244> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient: _add_conns ranks=[1,0,2] 2024-05-30T16:20:16.112 INFO:tasks.workunit.client.0.smithi006.stderr: -243> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): picked mon.noname-b con 0x559468f560a0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:20:16.112 INFO:tasks.workunit.client.0.smithi006.stderr: -242> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): picked mon.noname-a con 0x559468f56480 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:20:16.112 INFO:tasks.workunit.client.0.smithi006.stderr: -241> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): picked mon.noname-c con 0x559468f5f3b0 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:20:16.112 INFO:tasks.workunit.client.0.smithi006.stderr: -240> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:16.112 INFO:tasks.workunit.client.0.smithi006.stderr: -239> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:16.112 INFO:tasks.workunit.client.0.smithi006.stderr: -238> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:16.113 INFO:tasks.workunit.client.0.smithi006.stderr: -237> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): _renew_subs 2024-05-30T16:20:16.113 INFO:tasks.workunit.client.0.smithi006.stderr: -236> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): authenticate will time out at 2555.644849s 2024-05-30T16:20:16.113 INFO:tasks.workunit.client.0.smithi006.stderr: -235> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient(hunting): get_auth_request con 0x559468f56480 auth_method 0 2024-05-30T16:20:16.113 INFO:tasks.workunit.client.0.smithi006.stderr: -234> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:16.113 INFO:tasks.workunit.client.0.smithi006.stderr: -233> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:16.113 INFO:tasks.workunit.client.0.smithi006.stderr: -232> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:16.113 INFO:tasks.workunit.client.0.smithi006.stderr: -231> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient(hunting): get_auth_request con 0x559468f5f3b0 auth_method 0 2024-05-30T16:20:16.113 INFO:tasks.workunit.client.0.smithi006.stderr: -230> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:16.113 INFO:tasks.workunit.client.0.smithi006.stderr: -229> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:16.113 INFO:tasks.workunit.client.0.smithi006.stderr: -228> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:16.113 INFO:tasks.workunit.client.0.smithi006.stderr: -227> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:16.113 INFO:tasks.workunit.client.0.smithi006.stderr: -226> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:16.113 INFO:tasks.workunit.client.0.smithi006.stderr: -225> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:16.114 INFO:tasks.workunit.client.0.smithi006.stderr: -224> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:16.114 INFO:tasks.workunit.client.0.smithi006.stderr: -223> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:16.114 INFO:tasks.workunit.client.0.smithi006.stderr: -222> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:16.114 INFO:tasks.workunit.client.0.smithi006.stderr: -221> 2024-05-30T16:20:15.709+0000 7fbc50257640 10 monclient(hunting): get_auth_request con 0x559468f560a0 auth_method 0 2024-05-30T16:20:16.114 INFO:tasks.workunit.client.0.smithi006.stderr: -220> 2024-05-30T16:20:15.709+0000 7fbc50257640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:16.114 INFO:tasks.workunit.client.0.smithi006.stderr: -219> 2024-05-30T16:20:15.709+0000 7fbc50257640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:16.114 INFO:tasks.workunit.client.0.smithi006.stderr: -218> 2024-05-30T16:20:15.709+0000 7fbc50257640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:16.114 INFO:tasks.workunit.client.0.smithi006.stderr: -217> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient(hunting): handle_auth_done global_id 6291 payload 274 2024-05-30T16:20:16.114 INFO:tasks.workunit.client.0.smithi006.stderr: -216> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient: _finish_hunting 0 2024-05-30T16:20:16.114 INFO:tasks.workunit.client.0.smithi006.stderr: -215> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 1 monclient: found mon.noname-a 2024-05-30T16:20:16.114 INFO:tasks.workunit.client.0.smithi006.stderr: -214> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.6:3300/0 2024-05-30T16:20:16.114 INFO:tasks.workunit.client.0.smithi006.stderr: -213> 2024-05-30T16:20:15.709+0000 7fbb277fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:16.115 INFO:tasks.workunit.client.0.smithi006.stderr: -212> 2024-05-30T16:20:15.709+0000 7fbb277fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-30T16:20:16.115 INFO:tasks.workunit.client.0.smithi006.stderr: -211> 2024-05-30T16:20:15.709+0000 7fbb277fe640 10 monclient: dump: 2024-05-30T16:20:16.115 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:16.115 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:16.115 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.115 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.115 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:16.115 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:16.115 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:16.115 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:16.115 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:16.115 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.115 INFO:tasks.workunit.client.0.smithi006.stderr: -210> 2024-05-30T16:20:15.709+0000 7fbb277fe640 10 monclient: _finish_auth 0 2024-05-30T16:20:16.115 INFO:tasks.workunit.client.0.smithi006.stderr: -209> 2024-05-30T16:20:15.709+0000 7fbb277fe640 10 monclient: _check_auth_tickets 2024-05-30T16:20:16.115 INFO:tasks.workunit.client.0.smithi006.stderr: -208> 2024-05-30T16:20:15.709+0000 7fbb277fe640 10 monclient: handle_config config(0 keys) 2024-05-30T16:20:16.116 INFO:tasks.workunit.client.0.smithi006.stderr: -207> 2024-05-30T16:20:15.709+0000 7fbb277fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:16.116 INFO:tasks.workunit.client.0.smithi006.stderr: -206> 2024-05-30T16:20:15.709+0000 7fbb277fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-30T16:20:16.116 INFO:tasks.workunit.client.0.smithi006.stderr: -205> 2024-05-30T16:20:15.709+0000 7fbb277fe640 10 monclient: dump: 2024-05-30T16:20:16.116 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:16.116 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:16.116 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.116 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.116 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:16.116 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:16.116 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:16.116 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:16.116 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:16.116 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.116 INFO:tasks.workunit.client.0.smithi006.stderr: -204> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 monclient: authenticate success, global_id 6291 2024-05-30T16:20:16.116 INFO:tasks.workunit.client.0.smithi006.stderr: -203> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient: get_monmap_and_config success 2024-05-30T16:20:16.117 INFO:tasks.workunit.client.0.smithi006.stderr: -202> 2024-05-30T16:20:15.709+0000 7fbc52746980 4 set_mon_vals no callback set 2024-05-30T16:20:16.117 INFO:tasks.workunit.client.0.smithi006.stderr: -201> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient: shutdown 2024-05-30T16:20:16.117 INFO:tasks.workunit.client.0.smithi006.stderr: -200> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient: build_initial_monmap 2024-05-30T16:20:16.117 INFO:tasks.workunit.client.0.smithi006.stderr: -199> 2024-05-30T16:20:15.709+0000 7fbc52746980 1 build_initial for_mkfs: 0 2024-05-30T16:20:16.117 INFO:tasks.workunit.client.0.smithi006.stderr: -198> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient: monmap: 2024-05-30T16:20:16.117 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 0 2024-05-30T16:20:16.117 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-30T16:20:16.117 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 0.000000 2024-05-30T16:20:16.117 INFO:tasks.workunit.client.0.smithi006.stderr:created 0.000000 2024-05-30T16:20:16.117 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 0 (unknown) 2024-05-30T16:20:16.117 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:16.117 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.noname-a 2024-05-30T16:20:16.117 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-b 2024-05-30T16:20:16.117 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.noname-c 2024-05-30T16:20:16.118 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.118 INFO:tasks.workunit.client.0.smithi006.stderr: -197> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f56980) adding auth protocol: cephx 2024-05-30T16:20:16.118 INFO:tasks.workunit.client.0.smithi006.stderr: -196> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f56980) adding auth protocol: cephx 2024-05-30T16:20:16.118 INFO:tasks.workunit.client.0.smithi006.stderr: -195> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f56980) adding auth protocol: cephx 2024-05-30T16:20:16.118 INFO:tasks.workunit.client.0.smithi006.stderr: -194> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f56980) adding con mode: secure 2024-05-30T16:20:16.118 INFO:tasks.workunit.client.0.smithi006.stderr: -193> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f56980) adding con mode: crc 2024-05-30T16:20:16.118 INFO:tasks.workunit.client.0.smithi006.stderr: -192> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f56980) adding con mode: secure 2024-05-30T16:20:16.118 INFO:tasks.workunit.client.0.smithi006.stderr: -191> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f56980) adding con mode: crc 2024-05-30T16:20:16.118 INFO:tasks.workunit.client.0.smithi006.stderr: -190> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f56980) adding con mode: secure 2024-05-30T16:20:16.118 INFO:tasks.workunit.client.0.smithi006.stderr: -189> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f56980) adding con mode: crc 2024-05-30T16:20:16.118 INFO:tasks.workunit.client.0.smithi006.stderr: -188> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f56980) adding con mode: crc 2024-05-30T16:20:16.118 INFO:tasks.workunit.client.0.smithi006.stderr: -187> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f56980) adding con mode: secure 2024-05-30T16:20:16.118 INFO:tasks.workunit.client.0.smithi006.stderr: -186> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f56980) adding con mode: crc 2024-05-30T16:20:16.118 INFO:tasks.workunit.client.0.smithi006.stderr: -185> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f56980) adding con mode: secure 2024-05-30T16:20:16.119 INFO:tasks.workunit.client.0.smithi006.stderr: -184> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f56980) adding con mode: crc 2024-05-30T16:20:16.119 INFO:tasks.workunit.client.0.smithi006.stderr: -183> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f56980) adding con mode: secure 2024-05-30T16:20:16.119 INFO:tasks.workunit.client.0.smithi006.stderr: -182> 2024-05-30T16:20:15.709+0000 7fbc52746980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:16.119 INFO:tasks.workunit.client.0.smithi006.stderr: -181> 2024-05-30T16:20:15.709+0000 7fbc52746980 1 librados: starting msgr at 2024-05-30T16:20:16.119 INFO:tasks.workunit.client.0.smithi006.stderr: -180> 2024-05-30T16:20:15.709+0000 7fbc52746980 1 librados: starting objecter 2024-05-30T16:20:16.119 INFO:tasks.workunit.client.0.smithi006.stderr: -179> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 asok(0x559468c7f790) register_command objecter_requests cmddesc objecter_requests hook 0x559468f2a410 EEXIST 2024-05-30T16:20:16.119 INFO:tasks.workunit.client.0.smithi006.stderr: -178> 2024-05-30T16:20:15.709+0000 7fbc52746980 1 librados: setting wanted keys 2024-05-30T16:20:16.119 INFO:tasks.workunit.client.0.smithi006.stderr: -177> 2024-05-30T16:20:15.709+0000 7fbc52746980 1 librados: calling monclient init 2024-05-30T16:20:16.119 INFO:tasks.workunit.client.0.smithi006.stderr: -176> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient: init 2024-05-30T16:20:16.119 INFO:tasks.workunit.client.0.smithi006.stderr: -175> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f1e480) adding auth protocol: cephx 2024-05-30T16:20:16.119 INFO:tasks.workunit.client.0.smithi006.stderr: -174> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f1e480) adding auth protocol: cephx 2024-05-30T16:20:16.119 INFO:tasks.workunit.client.0.smithi006.stderr: -173> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f1e480) adding auth protocol: cephx 2024-05-30T16:20:16.119 INFO:tasks.workunit.client.0.smithi006.stderr: -172> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f1e480) adding con mode: secure 2024-05-30T16:20:16.119 INFO:tasks.workunit.client.0.smithi006.stderr: -171> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f1e480) adding con mode: crc 2024-05-30T16:20:16.120 INFO:tasks.workunit.client.0.smithi006.stderr: -170> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f1e480) adding con mode: secure 2024-05-30T16:20:16.120 INFO:tasks.workunit.client.0.smithi006.stderr: -169> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f1e480) adding con mode: crc 2024-05-30T16:20:16.120 INFO:tasks.workunit.client.0.smithi006.stderr: -168> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f1e480) adding con mode: secure 2024-05-30T16:20:16.120 INFO:tasks.workunit.client.0.smithi006.stderr: -167> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f1e480) adding con mode: crc 2024-05-30T16:20:16.120 INFO:tasks.workunit.client.0.smithi006.stderr: -166> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f1e480) adding con mode: crc 2024-05-30T16:20:16.120 INFO:tasks.workunit.client.0.smithi006.stderr: -165> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f1e480) adding con mode: secure 2024-05-30T16:20:16.120 INFO:tasks.workunit.client.0.smithi006.stderr: -164> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f1e480) adding con mode: crc 2024-05-30T16:20:16.120 INFO:tasks.workunit.client.0.smithi006.stderr: -163> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f1e480) adding con mode: secure 2024-05-30T16:20:16.120 INFO:tasks.workunit.client.0.smithi006.stderr: -162> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f1e480) adding con mode: crc 2024-05-30T16:20:16.121 INFO:tasks.workunit.client.0.smithi006.stderr: -161> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 AuthRegistry(0x559468f1e480) adding con mode: secure 2024-05-30T16:20:16.121 INFO:tasks.workunit.client.0.smithi006.stderr: -160> 2024-05-30T16:20:15.709+0000 7fbc52746980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:16.121 INFO:tasks.workunit.client.0.smithi006.stderr: -159> 2024-05-30T16:20:15.709+0000 7fbc52746980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:16.121 INFO:tasks.workunit.client.0.smithi006.stderr: -158> 2024-05-30T16:20:15.709+0000 7fbc52746980 5 asok(0x559468c7f790) register_command rotate-key cmddesc rotate-key hook 0x559468f1e5c8 EEXIST 2024-05-30T16:20:16.121 INFO:tasks.workunit.client.0.smithi006.stderr: -157> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient: _reopen_session rank -1 2024-05-30T16:20:16.121 INFO:tasks.workunit.client.0.smithi006.stderr: -156> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient: _add_conns ranks=[1,0,2] 2024-05-30T16:20:16.121 INFO:tasks.workunit.client.0.smithi006.stderr: -155> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): picked mon.noname-b con 0x559468f560a0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:20:16.121 INFO:tasks.workunit.client.0.smithi006.stderr: -154> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): picked mon.noname-a con 0x559468f5f3b0 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:20:16.121 INFO:tasks.workunit.client.0.smithi006.stderr: -153> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): picked mon.noname-c con 0x559468f278b0 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:20:16.121 INFO:tasks.workunit.client.0.smithi006.stderr: -152> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:16.121 INFO:tasks.workunit.client.0.smithi006.stderr: -151> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:16.121 INFO:tasks.workunit.client.0.smithi006.stderr: -150> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:16.121 INFO:tasks.workunit.client.0.smithi006.stderr: -149> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): _renew_subs 2024-05-30T16:20:16.122 INFO:tasks.workunit.client.0.smithi006.stderr: -148> 2024-05-30T16:20:15.709+0000 7fbc52746980 10 monclient(hunting): authenticate will time out at 2555.647460s 2024-05-30T16:20:16.122 INFO:tasks.workunit.client.0.smithi006.stderr: -147> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient(hunting): get_auth_request con 0x559468f5f3b0 auth_method 0 2024-05-30T16:20:16.122 INFO:tasks.workunit.client.0.smithi006.stderr: -146> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:16.122 INFO:tasks.workunit.client.0.smithi006.stderr: -145> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:16.122 INFO:tasks.workunit.client.0.smithi006.stderr: -144> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:16.122 INFO:tasks.workunit.client.0.smithi006.stderr: -143> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient(hunting): get_auth_request con 0x559468f278b0 auth_method 0 2024-05-30T16:20:16.122 INFO:tasks.workunit.client.0.smithi006.stderr: -142> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:16.122 INFO:tasks.workunit.client.0.smithi006.stderr: -141> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:16.122 INFO:tasks.workunit.client.0.smithi006.stderr: -140> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:16.123 INFO:tasks.workunit.client.0.smithi006.stderr: -139> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:16.123 INFO:tasks.workunit.client.0.smithi006.stderr: -138> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:16.123 INFO:tasks.workunit.client.0.smithi006.stderr: -137> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:16.123 INFO:tasks.workunit.client.0.smithi006.stderr: -136> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:16.123 INFO:tasks.workunit.client.0.smithi006.stderr: -135> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:16.123 INFO:tasks.workunit.client.0.smithi006.stderr: -134> 2024-05-30T16:20:15.709+0000 7fbc4fa56640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:16.123 INFO:tasks.workunit.client.0.smithi006.stderr: -133> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient(hunting): handle_auth_done global_id 6029 payload 824 2024-05-30T16:20:16.123 INFO:tasks.workunit.client.0.smithi006.stderr: -132> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient: _finish_hunting 0 2024-05-30T16:20:16.123 INFO:tasks.workunit.client.0.smithi006.stderr: -131> 2024-05-30T16:20:15.709+0000 7fbc50a58640 1 monclient: found mon.noname-c 2024-05-30T16:20:16.123 INFO:tasks.workunit.client.0.smithi006.stderr: -130> 2024-05-30T16:20:15.709+0000 7fbc50a58640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.6:3301/0 2024-05-30T16:20:16.123 INFO:tasks.workunit.client.0.smithi006.stderr: -129> 2024-05-30T16:20:15.713+0000 7fbb25ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:16.124 INFO:tasks.workunit.client.0.smithi006.stderr: -128> 2024-05-30T16:20:15.713+0000 7fbb25ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-30T16:20:16.124 INFO:tasks.workunit.client.0.smithi006.stderr: -127> 2024-05-30T16:20:15.713+0000 7fbb25ffb640 10 monclient: dump: 2024-05-30T16:20:16.124 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:16.124 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:16.124 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.124 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.124 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:16.124 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:16.124 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:16.124 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:16.124 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:16.125 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.125 INFO:tasks.workunit.client.0.smithi006.stderr: -126> 2024-05-30T16:20:15.713+0000 7fbb25ffb640 10 monclient: _finish_auth 0 2024-05-30T16:20:16.125 INFO:tasks.workunit.client.0.smithi006.stderr: -125> 2024-05-30T16:20:15.713+0000 7fbb25ffb640 10 monclient: _check_auth_tickets 2024-05-30T16:20:16.125 INFO:tasks.workunit.client.0.smithi006.stderr: -124> 2024-05-30T16:20:15.713+0000 7fbb25ffb640 10 monclient: handle_config config(0 keys) 2024-05-30T16:20:16.125 INFO:tasks.workunit.client.0.smithi006.stderr: -123> 2024-05-30T16:20:15.713+0000 7fbb25ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:16.125 INFO:tasks.workunit.client.0.smithi006.stderr: -122> 2024-05-30T16:20:15.713+0000 7fbb25ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-30T16:20:16.125 INFO:tasks.workunit.client.0.smithi006.stderr: -121> 2024-05-30T16:20:15.713+0000 7fbb25ffb640 10 monclient: dump: 2024-05-30T16:20:16.125 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:16.125 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:16.125 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.125 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:16.125 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:16.126 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:16.126 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:16.126 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:16.126 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:16.126 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.126 INFO:tasks.workunit.client.0.smithi006.stderr: -120> 2024-05-30T16:20:15.713+0000 7fbb277fe640 4 set_mon_vals no callback set 2024-05-30T16:20:16.126 INFO:tasks.workunit.client.0.smithi006.stderr: -119> 2024-05-30T16:20:15.713+0000 7fbc52746980 5 monclient: authenticate success, global_id 6029 2024-05-30T16:20:16.126 INFO:tasks.workunit.client.0.smithi006.stderr: -118> 2024-05-30T16:20:15.713+0000 7fbc52746980 10 monclient: _renew_subs 2024-05-30T16:20:16.126 INFO:tasks.workunit.client.0.smithi006.stderr: -117> 2024-05-30T16:20:15.713+0000 7fbc52746980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.6:3301/0 2024-05-30T16:20:16.126 INFO:tasks.workunit.client.0.smithi006.stderr: -116> 2024-05-30T16:20:15.713+0000 7fbc52746980 10 monclient: _renew_subs 2024-05-30T16:20:16.127 INFO:tasks.workunit.client.0.smithi006.stderr: -115> 2024-05-30T16:20:15.713+0000 7fbc52746980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.6:3301/0 2024-05-30T16:20:16.127 INFO:tasks.workunit.client.0.smithi006.stderr: -114> 2024-05-30T16:20:15.713+0000 7fbc52746980 1 librados: init done 2024-05-30T16:20:16.127 INFO:tasks.workunit.client.0.smithi006.stderr: -113> 2024-05-30T16:20:15.713+0000 7fbc52746980 5 asok(0x559468c7f790) register_command cr dump hook 0x559468f262e8 2024-05-30T16:20:16.127 INFO:tasks.workunit.client.0.smithi006.stderr: -112> 2024-05-30T16:20:15.713+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.127 INFO:tasks.workunit.client.0.smithi006.stderr: -111> 2024-05-30T16:20:15.713+0000 7fbb25ffb640 4 mgrc handle_mgr_map Got map version 5 2024-05-30T16:20:16.127 INFO:tasks.workunit.client.0.smithi006.stderr: -110> 2024-05-30T16:20:15.713+0000 7fbb25ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:20:16.127 INFO:tasks.workunit.client.0.smithi006.stderr: -109> 2024-05-30T16:20:15.713+0000 7fbb25ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:20:16.127 INFO:tasks.workunit.client.0.smithi006.stderr: -108> 2024-05-30T16:20:15.713+0000 7fbc4fa56640 10 monclient: get_auth_request con 0x559468f74b60 auth_method 0 2024-05-30T16:20:16.127 INFO:tasks.workunit.client.0.smithi006.stderr: -107> 2024-05-30T16:20:15.713+0000 7fbc50257640 10 monclient: get_auth_request con 0x7fbb1806dde0 auth_method 0 2024-05-30T16:20:16.127 INFO:tasks.workunit.client.0.smithi006.stderr: -106> 2024-05-30T16:20:15.713+0000 7fbc52746980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:16.127 INFO:tasks.workunit.client.0.smithi006.stderr: -105> 2024-05-30T16:20:15.713+0000 7fbc52746980 20 realm 2024-05-30T16:20:16.128 INFO:tasks.workunit.client.0.smithi006.stderr: -104> 2024-05-30T16:20:15.713+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.128 INFO:tasks.workunit.client.0.smithi006.stderr: -103> 2024-05-30T16:20:15.717+0000 7fbc52746980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:16.128 INFO:tasks.workunit.client.0.smithi006.stderr: -102> 2024-05-30T16:20:15.717+0000 7fbc52746980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:16.128 INFO:tasks.workunit.client.0.smithi006.stderr: -101> 2024-05-30T16:20:15.717+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.128 INFO:tasks.workunit.client.0.smithi006.stderr: -100> 2024-05-30T16:20:15.717+0000 7fbc52746980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:16.128 INFO:tasks.workunit.client.0.smithi006.stderr: -99> 2024-05-30T16:20:15.717+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.128 INFO:tasks.workunit.client.0.smithi006.stderr: -98> 2024-05-30T16:20:15.717+0000 7fbc50257640 10 monclient: get_auth_request con 0x559468f78f40 auth_method 0 2024-05-30T16:20:16.128 INFO:tasks.workunit.client.0.smithi006.stderr: -97> 2024-05-30T16:20:15.717+0000 7fbc52746980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:16.128 INFO:tasks.workunit.client.0.smithi006.stderr: -96> 2024-05-30T16:20:15.717+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.129 INFO:tasks.workunit.client.0.smithi006.stderr: -95> 2024-05-30T16:20:15.717+0000 7fbc52746980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:16.129 INFO:tasks.workunit.client.0.smithi006.stderr: -94> 2024-05-30T16:20:15.717+0000 7fbc52746980 20 searching for the correct realm 2024-05-30T16:20:16.129 INFO:tasks.workunit.client.0.smithi006.stderr: -93> 2024-05-30T16:20:15.717+0000 7fbc4fa56640 10 monclient: get_auth_request con 0x559468f7e0c0 auth_method 0 2024-05-30T16:20:16.129 INFO:tasks.workunit.client.0.smithi006.stderr: -92> 2024-05-30T16:20:15.721+0000 7fbc50a58640 10 monclient: get_auth_request con 0x7fbc380b9b30 auth_method 0 2024-05-30T16:20:16.129 INFO:tasks.workunit.client.0.smithi006.stderr: -91> 2024-05-30T16:20:15.721+0000 7fbc50257640 10 monclient: get_auth_request con 0x7fbc48015070 auth_method 0 2024-05-30T16:20:16.129 INFO:tasks.workunit.client.0.smithi006.stderr: -90> 2024-05-30T16:20:15.721+0000 7fbc4fa56640 10 monclient: get_auth_request con 0x7fbc4000e040 auth_method 0 2024-05-30T16:20:16.129 INFO:tasks.workunit.client.0.smithi006.stderr: -89> 2024-05-30T16:20:15.725+0000 7fbc50a58640 0 -- 172.21.15.6:0/1138350149 >> [v2:172.21.15.204:6800/3687190393,v1:172.21.15.204:6801/3687190393] conn(0x7fbc380e4040 msgr2=0x7fbc380e3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:16.129 INFO:tasks.workunit.client.0.smithi006.stderr: -88> 2024-05-30T16:20:15.928+0000 7fbc50a58640 10 monclient: get_auth_request con 0x7fbc380e4040 auth_method 0 2024-05-30T16:20:16.129 INFO:tasks.workunit.client.0.smithi006.stderr: -87> 2024-05-30T16:20:15.932+0000 7fbc50257640 10 monclient: get_auth_request con 0x7fbc380f4400 auth_method 0 2024-05-30T16:20:16.129 INFO:tasks.workunit.client.0.smithi006.stderr: -86> 2024-05-30T16:20:15.940+0000 7fbc50257640 0 -- 172.21.15.6:0/1138350149 >> [v2:172.21.15.204:6816/2677861390,v1:172.21.15.204:6817/2677861390] conn(0x559468f78f40 msgr2=0x559468f7b330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:16.129 INFO:tasks.workunit.client.0.smithi006.stderr: -85> 2024-05-30T16:20:15.940+0000 7fbb25ffb640 1 client.6029.objecter ms_handle_reset 0x559468f78f40 session 0x559468f77e00 osd.5 2024-05-30T16:20:16.129 INFO:tasks.workunit.client.0.smithi006.stderr: -84> 2024-05-30T16:20:15.940+0000 7fbb25ffb640 10 monclient: _renew_subs 2024-05-30T16:20:16.130 INFO:tasks.workunit.client.0.smithi006.stderr: -83> 2024-05-30T16:20:15.940+0000 7fbb25ffb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.6:3301/0 2024-05-30T16:20:16.130 INFO:tasks.workunit.client.0.smithi006.stderr: -82> 2024-05-30T16:20:15.940+0000 7fbc50257640 10 monclient: get_auth_request con 0x7fbb1807e510 auth_method 0 2024-05-30T16:20:16.130 INFO:tasks.workunit.client.0.smithi006.stderr: -81> 2024-05-30T16:20:15.948+0000 7fbc52746980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:16.130 INFO:tasks.workunit.client.0.smithi006.stderr: -80> 2024-05-30T16:20:15.948+0000 7fbc52746980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:16.130 INFO:tasks.workunit.client.0.smithi006.stderr: -79> 2024-05-30T16:20:15.948+0000 7fbc52746980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:16.130 INFO:tasks.workunit.client.0.smithi006.stderr: -78> 2024-05-30T16:20:15.948+0000 7fbc52746980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:16.130 INFO:tasks.workunit.client.0.smithi006.stderr: -77> 2024-05-30T16:20:15.948+0000 7fbc52746980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:16.130 INFO:tasks.workunit.client.0.smithi006.stderr: -76> 2024-05-30T16:20:15.948+0000 7fbc52746980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:16.130 INFO:tasks.workunit.client.0.smithi006.stderr: -75> 2024-05-30T16:20:15.948+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.131 INFO:tasks.workunit.client.0.smithi006.stderr: -74> 2024-05-30T16:20:15.948+0000 7fbc52746980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:16.131 INFO:tasks.workunit.client.0.smithi006.stderr: -73> 2024-05-30T16:20:15.948+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.131 INFO:tasks.workunit.client.0.smithi006.stderr: -72> 2024-05-30T16:20:15.948+0000 7fbc52746980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:16.131 INFO:tasks.workunit.client.0.smithi006.stderr: -71> 2024-05-30T16:20:15.948+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.131 INFO:tasks.workunit.client.0.smithi006.stderr: -70> 2024-05-30T16:20:15.952+0000 7fbc52746980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:16.131 INFO:tasks.workunit.client.0.smithi006.stderr: -69> 2024-05-30T16:20:15.952+0000 7fbc52746980 20 zone default found 2024-05-30T16:20:16.131 INFO:tasks.workunit.client.0.smithi006.stderr: -68> 2024-05-30T16:20:15.952+0000 7fbc52746980 4 Realm: () 2024-05-30T16:20:16.131 INFO:tasks.workunit.client.0.smithi006.stderr: -67> 2024-05-30T16:20:15.952+0000 7fbc52746980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:16.131 INFO:tasks.workunit.client.0.smithi006.stderr: -66> 2024-05-30T16:20:15.952+0000 7fbc52746980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:16.131 INFO:tasks.workunit.client.0.smithi006.stderr: -65> 2024-05-30T16:20:15.952+0000 7fbc52746980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:16.131 INFO:tasks.workunit.client.0.smithi006.stderr: -64> 2024-05-30T16:20:15.952+0000 7fbc52746980 20 zonegroup default 2024-05-30T16:20:16.131 INFO:tasks.workunit.client.0.smithi006.stderr: -63> 2024-05-30T16:20:15.952+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.131 INFO:tasks.workunit.client.0.smithi006.stderr: -62> 2024-05-30T16:20:15.952+0000 7fbc52746980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:16.131 INFO:tasks.workunit.client.0.smithi006.stderr: -61> 2024-05-30T16:20:15.952+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.132 INFO:tasks.workunit.client.0.smithi006.stderr: -60> 2024-05-30T16:20:15.952+0000 7fbc52746980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:16.132 INFO:tasks.workunit.client.0.smithi006.stderr: -59> 2024-05-30T16:20:15.952+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.132 INFO:tasks.workunit.client.0.smithi006.stderr: -58> 2024-05-30T16:20:15.952+0000 7fbc52746980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:16.132 INFO:tasks.workunit.client.0.smithi006.stderr: -57> 2024-05-30T16:20:15.952+0000 7fbc52746980 20 started sync module instance, tier type = 2024-05-30T16:20:16.132 INFO:tasks.workunit.client.0.smithi006.stderr: -56> 2024-05-30T16:20:15.952+0000 7fbc52746980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:16.132 INFO:tasks.workunit.client.0.smithi006.stderr: -55> 2024-05-30T16:20:15.976+0000 7fbc52746980 20 add_watcher() i=0 2024-05-30T16:20:16.132 INFO:tasks.workunit.client.0.smithi006.stderr: -54> 2024-05-30T16:20:15.976+0000 7fbc52746980 20 add_watcher() i=1 2024-05-30T16:20:16.132 INFO:tasks.workunit.client.0.smithi006.stderr: -53> 2024-05-30T16:20:15.976+0000 7fbc52746980 20 add_watcher() i=2 2024-05-30T16:20:16.132 INFO:tasks.workunit.client.0.smithi006.stderr: -52> 2024-05-30T16:20:15.976+0000 7fbc52746980 20 add_watcher() i=3 2024-05-30T16:20:16.132 INFO:tasks.workunit.client.0.smithi006.stderr: -51> 2024-05-30T16:20:15.976+0000 7fbc52746980 20 add_watcher() i=4 2024-05-30T16:20:16.132 INFO:tasks.workunit.client.0.smithi006.stderr: -50> 2024-05-30T16:20:15.976+0000 7fbc52746980 20 add_watcher() i=5 2024-05-30T16:20:16.132 INFO:tasks.workunit.client.0.smithi006.stderr: -49> 2024-05-30T16:20:15.976+0000 7fbc52746980 20 add_watcher() i=6 2024-05-30T16:20:16.132 INFO:tasks.workunit.client.0.smithi006.stderr: -48> 2024-05-30T16:20:15.976+0000 7fbc52746980 20 add_watcher() i=7 2024-05-30T16:20:16.133 INFO:tasks.workunit.client.0.smithi006.stderr: -47> 2024-05-30T16:20:15.976+0000 7fbc52746980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:16.133 INFO:tasks.workunit.client.0.smithi006.stderr: -46> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3f80) adding auth protocol: cephx 2024-05-30T16:20:16.133 INFO:tasks.workunit.client.0.smithi006.stderr: -45> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3f80) adding auth protocol: cephx 2024-05-30T16:20:16.133 INFO:tasks.workunit.client.0.smithi006.stderr: -44> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3f80) adding auth protocol: cephx 2024-05-30T16:20:16.133 INFO:tasks.workunit.client.0.smithi006.stderr: -43> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3f80) adding con mode: secure 2024-05-30T16:20:16.133 INFO:tasks.workunit.client.0.smithi006.stderr: -42> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3f80) adding con mode: crc 2024-05-30T16:20:16.133 INFO:tasks.workunit.client.0.smithi006.stderr: -41> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3f80) adding con mode: secure 2024-05-30T16:20:16.133 INFO:tasks.workunit.client.0.smithi006.stderr: -40> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3f80) adding con mode: crc 2024-05-30T16:20:16.133 INFO:tasks.workunit.client.0.smithi006.stderr: -39> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3f80) adding con mode: secure 2024-05-30T16:20:16.133 INFO:tasks.workunit.client.0.smithi006.stderr: -38> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3f80) adding con mode: crc 2024-05-30T16:20:16.133 INFO:tasks.workunit.client.0.smithi006.stderr: -37> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3f80) adding con mode: crc 2024-05-30T16:20:16.133 INFO:tasks.workunit.client.0.smithi006.stderr: -36> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3f80) adding con mode: secure 2024-05-30T16:20:16.133 INFO:tasks.workunit.client.0.smithi006.stderr: -35> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3f80) adding con mode: crc 2024-05-30T16:20:16.134 INFO:tasks.workunit.client.0.smithi006.stderr: -34> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3f80) adding con mode: secure 2024-05-30T16:20:16.134 INFO:tasks.workunit.client.0.smithi006.stderr: -33> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3f80) adding con mode: crc 2024-05-30T16:20:16.134 INFO:tasks.workunit.client.0.smithi006.stderr: -32> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 AuthRegistry(0x7ffd6b8c3f80) adding con mode: secure 2024-05-30T16:20:16.134 INFO:tasks.workunit.client.0.smithi006.stderr: -31> 2024-05-30T16:20:15.984+0000 7fbb06fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:16.134 INFO:tasks.workunit.client.0.smithi006.stderr: -30> 2024-05-30T16:20:15.984+0000 7fbc52746980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:16.134 INFO:tasks.workunit.client.0.smithi006.stderr: -29> 2024-05-30T16:20:15.984+0000 7fbc52746980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:16.134 INFO:tasks.workunit.client.0.smithi006.stderr: -28> 2024-05-30T16:20:15.984+0000 7fbc52746980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:16.134 INFO:tasks.workunit.client.0.smithi006.stderr: -27> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 asok(0x559468c7f790) register_command cache list hook 0x559468f26c20 2024-05-30T16:20:16.134 INFO:tasks.workunit.client.0.smithi006.stderr: -26> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 asok(0x559468c7f790) register_command cache inspect hook 0x559468f26c20 2024-05-30T16:20:16.134 INFO:tasks.workunit.client.0.smithi006.stderr: -25> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 asok(0x559468c7f790) register_command cache erase hook 0x559468f26c20 2024-05-30T16:20:16.134 INFO:tasks.workunit.client.0.smithi006.stderr: -24> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 asok(0x559468c7f790) register_command cache zap hook 0x559468f26c20 2024-05-30T16:20:16.134 INFO:tasks.workunit.client.0.smithi006.stderr: -23> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 note: GC not initialized 2024-05-30T16:20:16.135 INFO:tasks.workunit.client.0.smithi006.stderr: -22> 2024-05-30T16:20:15.984+0000 7fbb05fdb640 20 reqs_thread_entry: start 2024-05-30T16:20:16.135 INFO:tasks.workunit.client.0.smithi006.stderr: -21> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 asok(0x559468c7f790) register_command sync trace show hook 0x7fbc380acda0 2024-05-30T16:20:16.135 INFO:tasks.workunit.client.0.smithi006.stderr: -20> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 asok(0x559468c7f790) register_command sync trace history hook 0x7fbc380acda0 2024-05-30T16:20:16.135 INFO:tasks.workunit.client.0.smithi006.stderr: -19> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 asok(0x559468c7f790) register_command sync trace active hook 0x7fbc380acda0 2024-05-30T16:20:16.135 INFO:tasks.workunit.client.0.smithi006.stderr: -18> 2024-05-30T16:20:15.984+0000 7fbc52746980 5 asok(0x559468c7f790) register_command sync trace active_short hook 0x7fbc380acda0 2024-05-30T16:20:16.135 INFO:tasks.workunit.client.0.smithi006.stderr: -17> 2024-05-30T16:20:15.984+0000 7fbc52746980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:16.135 INFO:tasks.workunit.client.0.smithi006.stderr: -16> 2024-05-30T16:20:15.984+0000 7fbc52746980 20 Filter name: none 2024-05-30T16:20:16.135 INFO:tasks.workunit.client.0.smithi006.stderr: -15> 2024-05-30T16:20:15.984+0000 7fba777fe640 20 reqs_thread_entry: start 2024-05-30T16:20:16.135 INFO:tasks.workunit.client.0.smithi006.stderr: -14> 2024-05-30T16:20:15.984+0000 7fbc52746980 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-05-30T16:20:16.135 INFO:tasks.workunit.client.0.smithi006.stderr: -13> 2024-05-30T16:20:15.984+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.135 INFO:tasks.workunit.client.0.smithi006.stderr: -12> 2024-05-30T16:20:15.984+0000 7fbc52746980 20 rados_obj.operate() r=0 bl.length=218 2024-05-30T16:20:16.135 INFO:tasks.workunit.client.0.smithi006.stderr: -11> 2024-05-30T16:20:15.984+0000 7fbc52746980 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-05-30T16:20:16.135 INFO:tasks.workunit.client.0.smithi006.stderr: -10> 2024-05-30T16:20:15.984+0000 7fbc52746980 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-05-30T16:20:16.135 INFO:tasks.workunit.client.0.smithi006.stderr: -9> 2024-05-30T16:20:15.984+0000 7fbc52746980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 : miss 2024-05-30T16:20:16.136 INFO:tasks.workunit.client.0.smithi006.stderr: -8> 2024-05-30T16:20:15.984+0000 7fbc52746980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.136 INFO:tasks.workunit.client.0.smithi006.stderr: -7> 2024-05-30T16:20:15.984+0000 7fbc52746980 20 rados_obj.operate() r=0 bl.length=432 2024-05-30T16:20:16.136 INFO:tasks.workunit.client.0.smithi006.stderr: -6> 2024-05-30T16:20:15.984+0000 7fbc52746980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 info.flags=0x17 2024-05-30T16:20:16.136 INFO:tasks.workunit.client.0.smithi006.stderr: -5> 2024-05-30T16:20:15.984+0000 7fbc52746980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 to cache LRU end 2024-05-30T16:20:16.136 INFO:tasks.workunit.client.0.smithi006.stderr: -4> 2024-05-30T16:20:15.984+0000 7fbc52746980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:20:16.136 INFO:tasks.workunit.client.0.smithi006.stderr: -3> 2024-05-30T16:20:15.984+0000 7fbc52746980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:16.136 INFO:tasks.workunit.client.0.smithi006.stderr: -2> 2024-05-30T16:20:15.984+0000 7fbc52746980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 2024-05-30T16:20:16.136 INFO:tasks.workunit.client.0.smithi006.stderr: -1> 2024-05-30T16:20:15.996+0000 7fbc52746980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fbc52746980 time 2024-05-30T16:20:16.001268+0000 2024-05-30T16:20:16.136 INFO:tasks.workunit.client.0.smithi006.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-30T16:20:16.136 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.136 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:20:16.136 INFO:tasks.workunit.client.0.smithi006.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fbc5601dcff] 2024-05-30T16:20:16.137 INFO:tasks.workunit.client.0.smithi006.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7fbc5601deb3] 2024-05-30T16:20:16.137 INFO:tasks.workunit.client.0.smithi006.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x5594675b5f81] 2024-05-30T16:20:16.137 INFO:tasks.workunit.client.0.smithi006.stderr: 4: radosgw-admin(+0x9a792a) [0x5594675c392a] 2024-05-30T16:20:16.137 INFO:tasks.workunit.client.0.smithi006.stderr: 5: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24f) [0x5594675c7e4f] 2024-05-30T16:20:16.137 INFO:tasks.workunit.client.0.smithi006.stderr: 6: main() 2024-05-30T16:20:16.137 INFO:tasks.workunit.client.0.smithi006.stderr: 7: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fbc54e86d90] 2024-05-30T16:20:16.137 INFO:tasks.workunit.client.0.smithi006.stderr: 8: __libc_start_main() 2024-05-30T16:20:16.137 INFO:tasks.workunit.client.0.smithi006.stderr: 9: _start() 2024-05-30T16:20:16.137 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.138 INFO:tasks.workunit.client.0.smithi006.stderr: 0> 2024-05-30T16:20:15.996+0000 7fbc52746980 -1 *** Caught signal (Aborted) ** 2024-05-30T16:20:16.138 INFO:tasks.workunit.client.0.smithi006.stderr: in thread 7fbc52746980 thread_name:radosgw-admin 2024-05-30T16:20:16.138 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.138 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:20:16.138 INFO:tasks.workunit.client.0.smithi006.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fbc54e9f520] 2024-05-30T16:20:16.138 INFO:tasks.workunit.client.0.smithi006.stderr: 2: pthread_kill() 2024-05-30T16:20:16.138 INFO:tasks.workunit.client.0.smithi006.stderr: 3: raise() 2024-05-30T16:20:16.138 INFO:tasks.workunit.client.0.smithi006.stderr: 4: abort() 2024-05-30T16:20:16.138 INFO:tasks.workunit.client.0.smithi006.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fbc5601dd59] 2024-05-30T16:20:16.138 INFO:tasks.workunit.client.0.smithi006.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7fbc5601deb3] 2024-05-30T16:20:16.139 INFO:tasks.workunit.client.0.smithi006.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x5594675b5f81] 2024-05-30T16:20:16.139 INFO:tasks.workunit.client.0.smithi006.stderr: 8: radosgw-admin(+0x9a792a) [0x5594675c392a] 2024-05-30T16:20:16.139 INFO:tasks.workunit.client.0.smithi006.stderr: 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24f) [0x5594675c7e4f] 2024-05-30T16:20:16.139 INFO:tasks.workunit.client.0.smithi006.stderr: 10: main() 2024-05-30T16:20:16.139 INFO:tasks.workunit.client.0.smithi006.stderr: 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fbc54e86d90] 2024-05-30T16:20:16.139 INFO:tasks.workunit.client.0.smithi006.stderr: 12: __libc_start_main() 2024-05-30T16:20:16.139 INFO:tasks.workunit.client.0.smithi006.stderr: 13: _start() 2024-05-30T16:20:16.139 INFO:tasks.workunit.client.0.smithi006.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-30T16:20:16.139 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.139 INFO:tasks.workunit.client.0.smithi006.stderr:--- logging levels --- 2024-05-30T16:20:16.140 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 none 2024-05-30T16:20:16.140 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 lockdep 2024-05-30T16:20:16.140 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 context 2024-05-30T16:20:16.140 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 crush 2024-05-30T16:20:16.140 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds 2024-05-30T16:20:16.140 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_balancer 2024-05-30T16:20:16.140 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_locker 2024-05-30T16:20:16.140 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_log 2024-05-30T16:20:16.140 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_log_expire 2024-05-30T16:20:16.140 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_migrator 2024-05-30T16:20:16.140 INFO:tasks.workunit.client.0.smithi006.stderr: 3/ 5 mds_quiesce 2024-05-30T16:20:16.140 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 buffer 2024-05-30T16:20:16.140 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 timer 2024-05-30T16:20:16.140 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 filer 2024-05-30T16:20:16.141 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 striper 2024-05-30T16:20:16.141 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 objecter 2024-05-30T16:20:16.141 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rados 2024-05-30T16:20:16.141 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd 2024-05-30T16:20:16.141 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd_mirror 2024-05-30T16:20:16.141 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd_replay 2024-05-30T16:20:16.141 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd_pwl 2024-05-30T16:20:16.141 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 journaler 2024-05-30T16:20:16.141 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 objectcacher 2024-05-30T16:20:16.141 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 immutable_obj_cache 2024-05-30T16:20:16.141 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 client 2024-05-30T16:20:16.141 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 osd 2024-05-30T16:20:16.141 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 optracker 2024-05-30T16:20:16.141 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 objclass 2024-05-30T16:20:16.142 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 3 filestore 2024-05-30T16:20:16.142 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 3 journal 2024-05-30T16:20:16.142 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 0 ms 2024-05-30T16:20:16.142 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mon 2024-05-30T16:20:16.142 INFO:tasks.workunit.client.0.smithi006.stderr: 0/10 monc 2024-05-30T16:20:16.142 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 paxos 2024-05-30T16:20:16.142 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 tp 2024-05-30T16:20:16.142 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 auth 2024-05-30T16:20:16.142 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 crypto 2024-05-30T16:20:16.142 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 finisher 2024-05-30T16:20:16.142 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 reserver 2024-05-30T16:20:16.142 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 heartbeatmap 2024-05-30T16:20:16.142 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 perfcounter 2024-05-30T16:20:16.142 INFO:tasks.workunit.client.0.smithi006.stderr: 20/20 rgw 2024-05-30T16:20:16.143 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_sync 2024-05-30T16:20:16.143 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_datacache 2024-05-30T16:20:16.143 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_access 2024-05-30T16:20:16.143 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_dbstore 2024-05-30T16:20:16.143 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_flight 2024-05-30T16:20:16.143 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_lifecycle 2024-05-30T16:20:16.143 INFO:tasks.workunit.client.0.smithi006.stderr: 20/20 rgw_notification 2024-05-30T16:20:16.143 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 javaclient 2024-05-30T16:20:16.143 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 asok 2024-05-30T16:20:16.143 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 throttle 2024-05-30T16:20:16.143 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 0 refs 2024-05-30T16:20:16.143 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 compressor 2024-05-30T16:20:16.143 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 bluestore 2024-05-30T16:20:16.143 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 bluefs 2024-05-30T16:20:16.144 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 3 bdev 2024-05-30T16:20:16.144 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 kstore 2024-05-30T16:20:16.144 INFO:tasks.workunit.client.0.smithi006.stderr: 4/ 5 rocksdb 2024-05-30T16:20:16.144 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 fuse 2024-05-30T16:20:16.144 INFO:tasks.workunit.client.0.smithi006.stderr: 2/ 5 mgr 2024-05-30T16:20:16.144 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mgrc 2024-05-30T16:20:16.144 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 dpdk 2024-05-30T16:20:16.144 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 eventtrace 2024-05-30T16:20:16.144 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 prioritycache 2024-05-30T16:20:16.144 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 test 2024-05-30T16:20:16.144 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 cephfs_mirror 2024-05-30T16:20:16.144 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 cephsqlite 2024-05-30T16:20:16.144 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 crimson_interrupt 2024-05-30T16:20:16.145 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore 2024-05-30T16:20:16.145 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_onode 2024-05-30T16:20:16.145 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_odata 2024-05-30T16:20:16.145 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_omap 2024-05-30T16:20:16.145 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_tm 2024-05-30T16:20:16.145 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_t 2024-05-30T16:20:16.145 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_cleaner 2024-05-30T16:20:16.145 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_epm 2024-05-30T16:20:16.145 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_lba 2024-05-30T16:20:16.145 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_fixedkv_tree 2024-05-30T16:20:16.145 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_cache 2024-05-30T16:20:16.145 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_journal 2024-05-30T16:20:16.145 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_device 2024-05-30T16:20:16.145 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_backref 2024-05-30T16:20:16.146 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 alienstore 2024-05-30T16:20:16.146 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mclock 2024-05-30T16:20:16.146 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 cyanstore 2024-05-30T16:20:16.146 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 ceph_exporter 2024-05-30T16:20:16.146 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 memstore 2024-05-30T16:20:16.146 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 trace 2024-05-30T16:20:16.146 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 ceph_dedup 2024-05-30T16:20:16.146 INFO:tasks.workunit.client.0.smithi006.stderr: -2/-2 (syslog threshold) 2024-05-30T16:20:16.146 INFO:tasks.workunit.client.0.smithi006.stderr: 99/99 (stderr threshold) 2024-05-30T16:20:16.146 INFO:tasks.workunit.client.0.smithi006.stderr:--- pthread ID / name mapping for recent threads --- 2024-05-30T16:20:16.146 INFO:tasks.workunit.client.0.smithi006.stderr: 7fba777fe640 / http_manager 2024-05-30T16:20:16.146 INFO:tasks.workunit.client.0.smithi006.stderr: 7fbb05fdb640 / http_manager 2024-05-30T16:20:16.146 INFO:tasks.workunit.client.0.smithi006.stderr: 7fbb06fdd640 / rgw_dt_lg_renew 2024-05-30T16:20:16.146 INFO:tasks.workunit.client.0.smithi006.stderr: 7fbb25ffb640 / ms_dispatch 2024-05-30T16:20:16.147 INFO:tasks.workunit.client.0.smithi006.stderr: 7fbb277fe640 / io_context_pool 2024-05-30T16:20:16.147 INFO:tasks.workunit.client.0.smithi006.stderr: 7fbb35ffb640 / ms_dispatch 2024-05-30T16:20:16.147 INFO:tasks.workunit.client.0.smithi006.stderr: 7fbb3ce09640 / ms_dispatch 2024-05-30T16:20:16.147 INFO:tasks.workunit.client.0.smithi006.stderr: 7fbb3e60c640 / io_context_pool 2024-05-30T16:20:16.147 INFO:tasks.workunit.client.0.smithi006.stderr: 7fbc4fa56640 / msgr-worker-2 2024-05-30T16:20:16.147 INFO:tasks.workunit.client.0.smithi006.stderr: 7fbc50257640 / msgr-worker-1 2024-05-30T16:20:16.147 INFO:tasks.workunit.client.0.smithi006.stderr: 7fbc50a58640 / msgr-worker-0 2024-05-30T16:20:16.147 INFO:tasks.workunit.client.0.smithi006.stderr: 7fbc51259640 / io_context_pool 2024-05-30T16:20:16.147 INFO:tasks.workunit.client.0.smithi006.stderr: 7fbc52746980 / radosgw-admin 2024-05-30T16:20:16.147 INFO:tasks.workunit.client.0.smithi006.stderr: max_recent 500 2024-05-30T16:20:16.147 INFO:tasks.workunit.client.0.smithi006.stderr: max_new 1000 2024-05-30T16:20:16.147 INFO:tasks.workunit.client.0.smithi006.stderr: log_file /var/lib/ceph/crash/2024-05-30T16:20:16.003680Z_3a0c8c51-22be-4313-82bf-7761b613a53a/log 2024-05-30T16:20:16.147 INFO:tasks.workunit.client.0.smithi006.stderr:--- end dump of recent events --- 2024-05-30T16:20:16.147 INFO:tasks.workunit.client.0.smithi006.stderr:Aborted 2024-05-30T16:20:16.148 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket abort-at-set-target-layout 2024-05-30T16:20:16.149 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:20:16.149 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:20:16.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.080+0000 7ff093827980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.084+0000 7ff093827980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:16.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.084+0000 7ff093827980 20 realm 2024-05-30T16:20:16.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.084+0000 7ff093827980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.084+0000 7ff093827980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:16.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.084+0000 7ff093827980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:16.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.084+0000 7ff093827980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.150 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.084+0000 7ff093827980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:16.150 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.084+0000 7ff093827980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.150 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.084+0000 7ff093827980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:16.150 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.084+0000 7ff093827980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.150 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.084+0000 7ff093827980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:16.150 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.084+0000 7ff093827980 20 searching for the correct realm 2024-05-30T16:20:16.150 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.092+0000 7ff091b39640 0 -- 172.21.15.6:0/2290231659 >> [v2:172.21.15.204:6823/3156311527,v1:172.21.15.204:6825/3156311527] conn(0x7ff07c008040 msgr2=0x7ff07c080110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:16.150 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.100+0000 7ff091338640 0 -- 172.21.15.6:0/2290231659 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7ff07c064c40 msgr2=0x7ff07c056510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:16.321 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.316+0000 7ff093827980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:16.321 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.316+0000 7ff093827980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:16.322 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.316+0000 7ff093827980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:16.322 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.316+0000 7ff093827980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:16.322 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.316+0000 7ff093827980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:16.322 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.316+0000 7ff093827980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:16.322 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.316+0000 7ff093827980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.322 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.320+0000 7ff093827980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:16.322 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.320+0000 7ff093827980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.323 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.320+0000 7ff093827980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:16.323 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.320+0000 7ff093827980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.324 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.320+0000 7ff093827980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:16.324 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.320+0000 7ff093827980 20 zone default found 2024-05-30T16:20:16.324 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.320+0000 7ff093827980 4 Realm: () 2024-05-30T16:20:16.324 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.320+0000 7ff093827980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:16.324 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.320+0000 7ff093827980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:16.324 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.320+0000 7ff093827980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:16.325 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.320+0000 7ff093827980 20 zonegroup default 2024-05-30T16:20:16.325 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.320+0000 7ff093827980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.325 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.320+0000 7ff093827980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:16.325 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.320+0000 7ff093827980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.326 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.324+0000 7ff093827980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:16.326 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.324+0000 7ff093827980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.327 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.324+0000 7ff093827980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:16.327 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.324+0000 7ff093827980 20 started sync module instance, tier type = 2024-05-30T16:20:16.327 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.324+0000 7ff093827980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:16.352 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.348+0000 7ff093827980 20 add_watcher() i=0 2024-05-30T16:20:16.352 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.348+0000 7ff093827980 20 add_watcher() i=1 2024-05-30T16:20:16.352 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.348+0000 7ff093827980 20 add_watcher() i=2 2024-05-30T16:20:16.352 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.348+0000 7ff093827980 20 add_watcher() i=3 2024-05-30T16:20:16.352 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.348+0000 7ff093827980 20 add_watcher() i=4 2024-05-30T16:20:16.352 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.348+0000 7ff093827980 20 add_watcher() i=5 2024-05-30T16:20:16.352 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.348+0000 7ff093827980 20 add_watcher() i=6 2024-05-30T16:20:16.354 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.352+0000 7ff093827980 20 add_watcher() i=7 2024-05-30T16:20:16.354 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.352+0000 7ff093827980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:16.358 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.356+0000 7fef47fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:16.359 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.356+0000 7ff093827980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:16.359 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.356+0000 7ff093827980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:16.359 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.356+0000 7ff093827980 5 note: GC not initialized 2024-05-30T16:20:16.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.356+0000 7fef46fe5640 20 reqs_thread_entry: start 2024-05-30T16:20:16.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.356+0000 7ff093827980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:16.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.356+0000 7ff093827980 20 Filter name: none 2024-05-30T16:20:16.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.356+0000 7fef44fe1640 20 reqs_thread_entry: start 2024-05-30T16:20:16.361 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.356+0000 7ff093827980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.361 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.360+0000 7ff093827980 20 rados_obj.operate() r=0 bl.length=218 2024-05-30T16:20:16.361 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.360+0000 7ff093827980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.362 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.360+0000 7ff093827980 20 rados_obj.operate() r=0 bl.length=432 2024-05-30T16:20:16.362 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.360+0000 7ff093827980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:16.362 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.360+0000 7ff093827980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.363 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.360+0000 7ff093827980 20 rados_obj.operate() r=0 bl.length=218 2024-05-30T16:20:16.364 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.360+0000 7ff093827980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.364 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.360+0000 7ff093827980 20 rados_obj.operate() r=0 bl.length=432 2024-05-30T16:20:16.364 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.360+0000 7ff093827980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:16.367 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.364+0000 7fef47fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:16.372 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.368+0000 7ff093827980 20 remove_watcher() i=0 2024-05-30T16:20:16.372 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.368+0000 7ff093827980 2 removed watcher, disabling cache 2024-05-30T16:20:16.375 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.372+0000 7ff093827980 20 remove_watcher() i=1 2024-05-30T16:20:16.377 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.376+0000 7ff093827980 20 remove_watcher() i=2 2024-05-30T16:20:16.379 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.376+0000 7ff093827980 20 remove_watcher() i=3 2024-05-30T16:20:16.382 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.380+0000 7ff093827980 20 remove_watcher() i=4 2024-05-30T16:20:16.383 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.380+0000 7ff093827980 20 remove_watcher() i=5 2024-05-30T16:20:16.385 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.380+0000 7ff093827980 20 remove_watcher() i=6 2024-05-30T16:20:16.386 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.384+0000 7ff093827980 20 remove_watcher() i=7 2024-05-30T16:20:16.405 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.405 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:20:16.405 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: abort-at-set-target-layout id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-30T16:20:16.405 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:16.523 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket reshard --bucket abort-at-set-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 2024-05-30T16:20:16.562 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:20:16.562 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:20:16.608 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.604+0000 7fde2fa60980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.609 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.608+0000 7fde2fa60980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:16.609 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.608+0000 7fde2fa60980 20 realm 2024-05-30T16:20:16.609 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.608+0000 7fde2fa60980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.610 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.608+0000 7fde2fa60980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:16.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.608+0000 7fde2fa60980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:16.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.608+0000 7fde2fa60980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.608+0000 7fde2fa60980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:16.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.608+0000 7fde2fa60980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.612 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.608+0000 7fde2fa60980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:16.612 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.608+0000 7fde2fa60980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.612 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.608+0000 7fde2fa60980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:16.613 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.608+0000 7fde2fa60980 20 searching for the correct realm 2024-05-30T16:20:16.618 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.616+0000 7fde2dd72640 0 -- 172.21.15.6:0/3203876808 >> [v2:172.21.15.204:6823/3156311527,v1:172.21.15.204:6825/3156311527] conn(0x561d2c653e30 msgr2=0x7fde2405b7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:16.624 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.620+0000 7fde2d571640 0 -- 172.21.15.6:0/3203876808 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7fde2407d040 msgr2=0x7fde24087870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:16.840 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.836+0000 7fde2fa60980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:16.840 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.836+0000 7fde2fa60980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:16.841 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.836+0000 7fde2fa60980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:16.841 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.836+0000 7fde2fa60980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:16.841 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.836+0000 7fde2fa60980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:16.841 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.836+0000 7fde2fa60980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:16.841 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.836+0000 7fde2fa60980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.841 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.836+0000 7fde2fa60980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:16.841 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.836+0000 7fde2fa60980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.842 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.840+0000 7fde2fa60980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:16.842 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.840+0000 7fde2fa60980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.842 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.840+0000 7fde2fa60980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:16.843 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.840+0000 7fde2fa60980 20 zone default found 2024-05-30T16:20:16.843 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.840+0000 7fde2fa60980 4 Realm: () 2024-05-30T16:20:16.843 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.840+0000 7fde2fa60980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:16.843 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.840+0000 7fde2fa60980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:16.843 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.840+0000 7fde2fa60980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:16.843 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.840+0000 7fde2fa60980 20 zonegroup default 2024-05-30T16:20:16.843 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.840+0000 7fde2fa60980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.843 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.840+0000 7fde2fa60980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:16.843 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.840+0000 7fde2fa60980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.844 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.840+0000 7fde2fa60980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:16.844 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.840+0000 7fde2fa60980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.845 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.840+0000 7fde2fa60980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:16.845 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.840+0000 7fde2fa60980 20 started sync module instance, tier type = 2024-05-30T16:20:16.845 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.840+0000 7fde2fa60980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:16.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.864+0000 7fde2fa60980 20 add_watcher() i=0 2024-05-30T16:20:16.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.864+0000 7fde2fa60980 20 add_watcher() i=1 2024-05-30T16:20:16.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.864+0000 7fde2fa60980 20 add_watcher() i=2 2024-05-30T16:20:16.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.864+0000 7fde2fa60980 20 add_watcher() i=3 2024-05-30T16:20:16.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.864+0000 7fde2fa60980 20 add_watcher() i=4 2024-05-30T16:20:16.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.864+0000 7fde2fa60980 20 add_watcher() i=5 2024-05-30T16:20:16.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.864+0000 7fde2fa60980 20 add_watcher() i=6 2024-05-30T16:20:16.871 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.868+0000 7fde2fa60980 20 add_watcher() i=7 2024-05-30T16:20:16.871 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.868+0000 7fde2fa60980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:16.875 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.872+0000 7fdce3fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:16.876 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.872+0000 7fde2fa60980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:16.876 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.872+0000 7fde2fa60980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:16.876 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.872+0000 7fde2fa60980 5 note: GC not initialized 2024-05-30T16:20:16.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.872+0000 7fdce2fe5640 20 reqs_thread_entry: start 2024-05-30T16:20:16.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.872+0000 7fde2fa60980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:16.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.872+0000 7fde2fa60980 20 Filter name: none 2024-05-30T16:20:16.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.872+0000 7fdce0fe1640 20 reqs_thread_entry: start 2024-05-30T16:20:16.878 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.876+0000 7fde2fa60980 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-05-30T16:20:16.878 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.876+0000 7fde2fa60980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.878 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.876+0000 7fde2fa60980 20 rados_obj.operate() r=0 bl.length=218 2024-05-30T16:20:16.878 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.876+0000 7fde2fa60980 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-05-30T16:20:16.878 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.876+0000 7fde2fa60980 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-05-30T16:20:16.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.876+0000 7fde2fa60980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 : miss 2024-05-30T16:20:16.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.876+0000 7fde2fa60980 20 rados->read ofs=0 len=0 2024-05-30T16:20:16.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.876+0000 7fde2fa60980 20 rados_obj.operate() r=0 bl.length=432 2024-05-30T16:20:16.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.876+0000 7fde2fa60980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 info.flags=0x17 2024-05-30T16:20:16.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.876+0000 7fde2fa60980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 to cache LRU end 2024-05-30T16:20:16.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.876+0000 7fde2fa60980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:20:16.880 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.876+0000 7fde2fa60980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:16.880 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.876+0000 7fde2fa60980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 2024-05-30T16:20:16.882 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.880+0000 7fde2fa60980 0 INFO: RGWReshardLock::lock found lock on abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 to be held by another RGW process; skipping for now 2024-05-30T16:20:16.883 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.880+0000 7fdce3fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:16.889 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.884+0000 7fde2fa60980 20 remove_watcher() i=0 2024-05-30T16:20:16.889 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.884+0000 7fde2fa60980 2 removed watcher, disabling cache 2024-05-30T16:20:16.892 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.888+0000 7fde2fa60980 20 remove_watcher() i=1 2024-05-30T16:20:16.894 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.892+0000 7fde2fa60980 20 remove_watcher() i=2 2024-05-30T16:20:16.896 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.892+0000 7fde2fa60980 20 remove_watcher() i=3 2024-05-30T16:20:16.899 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.896+0000 7fde2fa60980 20 remove_watcher() i=4 2024-05-30T16:20:16.900 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.896+0000 7fde2fa60980 20 remove_watcher() i=5 2024-05-30T16:20:16.902 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.900+0000 7fde2fa60980 20 remove_watcher() i=6 2024-05-30T16:20:16.904 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:16.900+0000 7fde2fa60980 20 remove_watcher() i=7 2024-05-30T16:20:16.922 INFO:tasks.workunit.client.0.smithi006.stderr:waiting 30 seconds for reshard lock to expire... 2024-05-30T16:20:46.954 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket reshard --bucket abort-at-set-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 2024-05-30T16:20:46.985 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:20:46.985 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:20:47.031 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.028+0000 7fd698d78980 20 rados->read ofs=0 len=0 2024-05-30T16:20:47.033 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.028+0000 7fd698d78980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:47.033 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.028+0000 7fd698d78980 20 realm 2024-05-30T16:20:47.033 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.028+0000 7fd698d78980 20 rados->read ofs=0 len=0 2024-05-30T16:20:47.034 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.032+0000 7fd698d78980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:47.034 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.032+0000 7fd698d78980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:47.034 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.032+0000 7fd698d78980 20 rados->read ofs=0 len=0 2024-05-30T16:20:47.034 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.032+0000 7fd698d78980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:47.034 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.032+0000 7fd698d78980 20 rados->read ofs=0 len=0 2024-05-30T16:20:47.035 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.032+0000 7fd698d78980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:47.035 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.032+0000 7fd698d78980 20 rados->read ofs=0 len=0 2024-05-30T16:20:47.036 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.032+0000 7fd698d78980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:47.036 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.032+0000 7fd698d78980 20 searching for the correct realm 2024-05-30T16:20:47.041 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.040+0000 7fd696088640 0 -- 172.21.15.6:0/1942919271 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7fd688043040 msgr2=0x7fd688045950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:47.048 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.044+0000 7fd696889640 0 -- 172.21.15.6:0/1942919271 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7fd68c040420 msgr2=0x7fd68c0408a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:47.265 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.260+0000 7fd698d78980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:47.266 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.260+0000 7fd698d78980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:47.266 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.260+0000 7fd698d78980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:47.266 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.260+0000 7fd698d78980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:47.266 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.260+0000 7fd698d78980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:47.266 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.260+0000 7fd698d78980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:47.266 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.260+0000 7fd698d78980 20 rados->read ofs=0 len=0 2024-05-30T16:20:47.266 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.264+0000 7fd698d78980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:47.266 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.264+0000 7fd698d78980 20 rados->read ofs=0 len=0 2024-05-30T16:20:47.267 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.264+0000 7fd698d78980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:47.267 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.264+0000 7fd698d78980 20 rados->read ofs=0 len=0 2024-05-30T16:20:47.268 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.264+0000 7fd698d78980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:47.268 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.264+0000 7fd698d78980 20 zone default found 2024-05-30T16:20:47.268 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.264+0000 7fd698d78980 4 Realm: () 2024-05-30T16:20:47.268 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.264+0000 7fd698d78980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:47.268 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.264+0000 7fd698d78980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:47.268 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.264+0000 7fd698d78980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:47.268 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.264+0000 7fd698d78980 20 zonegroup default 2024-05-30T16:20:47.268 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.264+0000 7fd698d78980 20 rados->read ofs=0 len=0 2024-05-30T16:20:47.269 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.264+0000 7fd698d78980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:47.269 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.264+0000 7fd698d78980 20 rados->read ofs=0 len=0 2024-05-30T16:20:47.269 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.268+0000 7fd698d78980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:47.270 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.268+0000 7fd698d78980 20 rados->read ofs=0 len=0 2024-05-30T16:20:47.270 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.268+0000 7fd698d78980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:47.270 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.268+0000 7fd698d78980 20 started sync module instance, tier type = 2024-05-30T16:20:47.270 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.268+0000 7fd698d78980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:47.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.288+0000 7fd698d78980 20 add_watcher() i=0 2024-05-30T16:20:47.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.288+0000 7fd698d78980 20 add_watcher() i=1 2024-05-30T16:20:47.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.288+0000 7fd698d78980 20 add_watcher() i=2 2024-05-30T16:20:47.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.288+0000 7fd698d78980 20 add_watcher() i=3 2024-05-30T16:20:47.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.288+0000 7fd698d78980 20 add_watcher() i=4 2024-05-30T16:20:47.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.288+0000 7fd698d78980 20 add_watcher() i=5 2024-05-30T16:20:47.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.288+0000 7fd698d78980 20 add_watcher() i=6 2024-05-30T16:20:47.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.296+0000 7fd698d78980 20 add_watcher() i=7 2024-05-30T16:20:47.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.296+0000 7fd698d78980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:47.304 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.300+0000 7fd54dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:47.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.300+0000 7fd698d78980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:47.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.300+0000 7fd698d78980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:47.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.300+0000 7fd698d78980 5 note: GC not initialized 2024-05-30T16:20:47.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.300+0000 7fd54cfe1640 20 reqs_thread_entry: start 2024-05-30T16:20:47.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.304+0000 7fd698d78980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:47.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.304+0000 7fd698d78980 20 Filter name: none 2024-05-30T16:20:47.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.304+0000 7fd545ffb640 20 reqs_thread_entry: start 2024-05-30T16:20:47.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.304+0000 7fd698d78980 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-05-30T16:20:47.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.304+0000 7fd698d78980 20 rados->read ofs=0 len=0 2024-05-30T16:20:47.307 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.304+0000 7fd698d78980 20 rados_obj.operate() r=0 bl.length=218 2024-05-30T16:20:47.307 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.304+0000 7fd698d78980 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-05-30T16:20:47.307 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.304+0000 7fd698d78980 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-05-30T16:20:47.307 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.304+0000 7fd698d78980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 : miss 2024-05-30T16:20:47.308 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.304+0000 7fd698d78980 20 rados->read ofs=0 len=0 2024-05-30T16:20:47.308 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.304+0000 7fd698d78980 20 rados_obj.operate() r=0 bl.length=432 2024-05-30T16:20:47.308 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.304+0000 7fd698d78980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 info.flags=0x17 2024-05-30T16:20:47.308 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.304+0000 7fd698d78980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 to cache LRU end 2024-05-30T16:20:47.308 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.304+0000 7fd698d78980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:20:47.308 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.304+0000 7fd698d78980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:47.309 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.304+0000 7fd698d78980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 2024-05-30T16:20:47.317 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.316+0000 7fd698d78980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 info.flags=0x17 2024-05-30T16:20:47.318 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.316+0000 7fd698d78980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 to cache LRU end 2024-05-30T16:20:47.318 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.316+0000 7fd698d78980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:47.318 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.316+0000 7fd698d78980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22, ofs0, ns] 2024-05-30T16:20:47.318 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.316+0000 7fd57a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431621 cookie 94117129002512 notifier 6083 bl.length()=1001 2024-05-30T16:20:47.318 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.316+0000 7fd57a7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 info.flags=0x17 2024-05-30T16:20:47.318 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.316+0000 7fd57a7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 to cache LRU end 2024-05-30T16:20:47.318 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.316+0000 7fd57a7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:47.321 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.316+0000 7fd698d78980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22]): orig_sources= new_sources= 2024-05-30T16:20:47.321 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.316+0000 7fd698d78980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22]): potential sources added=[] removed=[] 2024-05-30T16:20:47.321 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.316+0000 7fd698d78980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22]): orig_dests= new_dests= 2024-05-30T16:20:47.321 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.316+0000 7fd698d78980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22]): potential dests added=[] removed=[] 2024-05-30T16:20:47.327 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.324+0000 7fd698d78980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.1.0 2024-05-30T16:20:47.327 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.324+0000 7fd698d78980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.1.0 2024-05-30T16:20:47.327 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.324+0000 7fd698d78980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.1.1 2024-05-30T16:20:47.327 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.324+0000 7fd698d78980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.1.1 2024-05-30T16:20:47.327 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.324+0000 7fd698d78980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.1.2 2024-05-30T16:20:47.327 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.324+0000 7fd698d78980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.1.2 2024-05-30T16:20:47.327 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.324+0000 7fd698d78980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.1.3 2024-05-30T16:20:47.327 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.324+0000 7fd698d78980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.1.3 2024-05-30T16:20:47.327 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.324+0000 7fd698d78980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.1.4 2024-05-30T16:20:47.327 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.324+0000 7fd698d78980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.1.4 2024-05-30T16:20:47.327 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.324+0000 7fd698d78980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.1.5 2024-05-30T16:20:47.328 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.324+0000 7fd698d78980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.1.5 2024-05-30T16:20:47.328 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.324+0000 7fd698d78980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.1.6 2024-05-30T16:20:47.328 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.324+0000 7fd698d78980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.1.6 2024-05-30T16:20:47.328 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.324+0000 7fd698d78980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.1.7 2024-05-30T16:20:47.328 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.324+0000 7fd698d78980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.1.7 2024-05-30T16:20:47.328 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.324+0000 7fd698d78980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.1.8 2024-05-30T16:20:47.328 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.324+0000 7fd698d78980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.1.8 2024-05-30T16:20:47.328 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.324+0000 7fd698d78980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.1.9 2024-05-30T16:20:47.328 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.324+0000 7fd698d78980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.1.9 2024-05-30T16:20:47.328 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.324+0000 7fd698d78980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.1.10 2024-05-30T16:20:47.328 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.324+0000 7fd698d78980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.1.10 2024-05-30T16:20:47.328 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.324+0000 7fd698d78980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.1.11 2024-05-30T16:20:47.328 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.324+0000 7fd698d78980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.1.11 2024-05-30T16:20:47.328 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.324+0000 7fd698d78980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.0 2024-05-30T16:20:47.329 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.324+0000 7fd698d78980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.1 2024-05-30T16:20:47.329 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.324+0000 7fd698d78980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.2 2024-05-30T16:20:47.329 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.324+0000 7fd698d78980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.3 2024-05-30T16:20:47.329 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.324+0000 7fd698d78980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.4 2024-05-30T16:20:47.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.328+0000 7fd698d78980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.5 2024-05-30T16:20:47.331 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.328+0000 7fd698d78980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.6 2024-05-30T16:20:47.332 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.328+0000 7fd698d78980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.7 2024-05-30T16:20:47.333 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.328+0000 7fd698d78980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.8 2024-05-30T16:20:47.333 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.332+0000 7fd698d78980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.9 2024-05-30T16:20:47.335 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.332+0000 7fd698d78980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22.10 2024-05-30T16:20:47.342 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.340+0000 7fd698d78980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 : hit (requested=0x17, cached=0x17) 2024-05-30T16:20:47.342 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.340+0000 7fd698d78980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 2024-05-30T16:20:47.345 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.340+0000 7fd698d78980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 info.flags=0x17 2024-05-30T16:20:47.345 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.340+0000 7fd698d78980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 to cache LRU end 2024-05-30T16:20:47.345 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.340+0000 7fd698d78980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:47.345 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.340+0000 7fd698d78980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22, ofs0, ns] 2024-05-30T16:20:47.345 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.344+0000 7fd57a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431622 cookie 94117129002512 notifier 6083 bl.length()=969 2024-05-30T16:20:47.346 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.344+0000 7fd57a7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 info.flags=0x17 2024-05-30T16:20:47.346 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.344+0000 7fd57a7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 to cache LRU end 2024-05-30T16:20:47.346 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.344+0000 7fd57a7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:47.348 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.344+0000 7fd698d78980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22]): orig_sources= new_sources= 2024-05-30T16:20:47.348 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.344+0000 7fd698d78980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22]): potential sources added=[] removed=[] 2024-05-30T16:20:47.348 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.344+0000 7fd698d78980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22]): orig_dests= new_dests= 2024-05-30T16:20:47.348 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.344+0000 7fd698d78980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22]): potential dests added=[] removed=[] 2024-05-30T16:20:47.354 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.352+0000 7fd698d78980 1 execute INFO: reshard of bucket "abort-at-set-target-layout" completed successfully 2024-05-30T16:20:47.358 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.356+0000 7fd54dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:47.363 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.360+0000 7fd698d78980 20 remove_watcher() i=0 2024-05-30T16:20:47.363 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.360+0000 7fd698d78980 2 removed watcher, disabling cache 2024-05-30T16:20:47.366 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.364+0000 7fd698d78980 20 remove_watcher() i=1 2024-05-30T16:20:47.368 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.364+0000 7fd698d78980 20 remove_watcher() i=2 2024-05-30T16:20:47.371 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.368+0000 7fd698d78980 20 remove_watcher() i=3 2024-05-30T16:20:47.373 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.368+0000 7fd698d78980 20 remove_watcher() i=4 2024-05-30T16:20:47.376 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.372+0000 7fd698d78980 20 remove_watcher() i=5 2024-05-30T16:20:47.377 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.372+0000 7fd698d78980 20 remove_watcher() i=6 2024-05-30T16:20:47.378 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.376+0000 7fd698d78980 20 remove_watcher() i=7 2024-05-30T16:20:47.395 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket abort-at-set-target-layout 2024-05-30T16:20:47.427 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:20:47.427 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:20:47.471 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.468+0000 7fc4ec3ce980 20 rados->read ofs=0 len=0 2024-05-30T16:20:47.473 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.468+0000 7fc4ec3ce980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:47.473 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.468+0000 7fc4ec3ce980 20 realm 2024-05-30T16:20:47.473 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.468+0000 7fc4ec3ce980 20 rados->read ofs=0 len=0 2024-05-30T16:20:47.474 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.472+0000 7fc4ec3ce980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:47.474 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.472+0000 7fc4ec3ce980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:47.474 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.472+0000 7fc4ec3ce980 20 rados->read ofs=0 len=0 2024-05-30T16:20:47.474 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.472+0000 7fc4ec3ce980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:47.474 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.472+0000 7fc4ec3ce980 20 rados->read ofs=0 len=0 2024-05-30T16:20:47.475 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.472+0000 7fc4ec3ce980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:47.475 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.472+0000 7fc4ec3ce980 20 rados->read ofs=0 len=0 2024-05-30T16:20:47.476 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.472+0000 7fc4ec3ce980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:47.476 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.472+0000 7fc4ec3ce980 20 searching for the correct realm 2024-05-30T16:20:47.482 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.480+0000 7fc4e96de640 0 -- 172.21.15.6:0/2133799203 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7fc4dc008050 msgr2=0x7fc4dc0113c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:20:47.490 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.488+0000 7fc4e96de640 0 -- 172.21.15.6:0/2133799203 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7fc3b407e540 msgr2=0x7fc3b4080970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:47.497 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.492+0000 7fc4ec3ce980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:47.497 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.492+0000 7fc4ec3ce980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:47.498 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.492+0000 7fc4ec3ce980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:47.498 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.492+0000 7fc4ec3ce980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:47.498 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.492+0000 7fc4ec3ce980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:47.498 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.492+0000 7fc4ec3ce980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:47.498 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.492+0000 7fc4ec3ce980 20 rados->read ofs=0 len=0 2024-05-30T16:20:47.498 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.496+0000 7fc4ec3ce980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:47.498 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.496+0000 7fc4ec3ce980 20 rados->read ofs=0 len=0 2024-05-30T16:20:47.498 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.496+0000 7fc4ec3ce980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:47.498 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.496+0000 7fc4ec3ce980 20 rados->read ofs=0 len=0 2024-05-30T16:20:47.499 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.496+0000 7fc4ec3ce980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:47.499 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.496+0000 7fc4ec3ce980 20 zone default found 2024-05-30T16:20:47.499 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.496+0000 7fc4ec3ce980 4 Realm: () 2024-05-30T16:20:47.499 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.496+0000 7fc4ec3ce980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:47.499 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.496+0000 7fc4ec3ce980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:47.499 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.496+0000 7fc4ec3ce980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:47.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.496+0000 7fc4ec3ce980 20 zonegroup default 2024-05-30T16:20:47.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.496+0000 7fc4ec3ce980 20 rados->read ofs=0 len=0 2024-05-30T16:20:47.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.496+0000 7fc4ec3ce980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:47.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.496+0000 7fc4ec3ce980 20 rados->read ofs=0 len=0 2024-05-30T16:20:47.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.496+0000 7fc4ec3ce980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:47.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.496+0000 7fc4ec3ce980 20 rados->read ofs=0 len=0 2024-05-30T16:20:47.501 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.496+0000 7fc4ec3ce980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:47.501 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.496+0000 7fc4ec3ce980 20 started sync module instance, tier type = 2024-05-30T16:20:47.501 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.496+0000 7fc4ec3ce980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:47.516 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.512+0000 7fc4ec3ce980 20 add_watcher() i=0 2024-05-30T16:20:47.516 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.512+0000 7fc4ec3ce980 20 add_watcher() i=1 2024-05-30T16:20:47.516 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.512+0000 7fc4ec3ce980 20 add_watcher() i=2 2024-05-30T16:20:47.516 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.512+0000 7fc4ec3ce980 20 add_watcher() i=3 2024-05-30T16:20:47.516 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.512+0000 7fc4ec3ce980 20 add_watcher() i=4 2024-05-30T16:20:47.516 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.512+0000 7fc4ec3ce980 20 add_watcher() i=5 2024-05-30T16:20:47.516 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.512+0000 7fc4ec3ce980 20 add_watcher() i=6 2024-05-30T16:20:47.517 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.512+0000 7fc4ec3ce980 20 add_watcher() i=7 2024-05-30T16:20:47.517 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.512+0000 7fc4ec3ce980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:47.520 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.516+0000 7fc39ffe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:47.521 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.516+0000 7fc4ec3ce980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:47.521 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.516+0000 7fc4ec3ce980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:47.521 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.516+0000 7fc4ec3ce980 5 note: GC not initialized 2024-05-30T16:20:47.521 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.516+0000 7fc39efe5640 20 reqs_thread_entry: start 2024-05-30T16:20:47.521 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.516+0000 7fc4ec3ce980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:47.521 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.516+0000 7fc4ec3ce980 20 Filter name: none 2024-05-30T16:20:47.522 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.516+0000 7fc39cfe1640 20 reqs_thread_entry: start 2024-05-30T16:20:47.522 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.516+0000 7fc4ec3ce980 20 rados->read ofs=0 len=0 2024-05-30T16:20:47.522 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.520+0000 7fc4ec3ce980 20 rados_obj.operate() r=0 bl.length=218 2024-05-30T16:20:47.523 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.520+0000 7fc4ec3ce980 20 rados->read ofs=0 len=0 2024-05-30T16:20:47.523 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.520+0000 7fc4ec3ce980 20 rados_obj.operate() r=0 bl.length=432 2024-05-30T16:20:47.523 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.520+0000 7fc4ec3ce980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:47.523 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.520+0000 7fc4ec3ce980 20 rados->read ofs=0 len=0 2024-05-30T16:20:47.524 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.520+0000 7fc4ec3ce980 20 rados_obj.operate() r=0 bl.length=218 2024-05-30T16:20:47.524 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.520+0000 7fc4ec3ce980 20 rados->read ofs=0 len=0 2024-05-30T16:20:47.524 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.520+0000 7fc4ec3ce980 20 rados_obj.operate() r=0 bl.length=432 2024-05-30T16:20:47.524 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.520+0000 7fc4ec3ce980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:47.526 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.524+0000 7fc39ffe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:47.530 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.528+0000 7fc4ec3ce980 20 remove_watcher() i=0 2024-05-30T16:20:47.531 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.528+0000 7fc4ec3ce980 2 removed watcher, disabling cache 2024-05-30T16:20:47.532 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.528+0000 7fc4ec3ce980 20 remove_watcher() i=1 2024-05-30T16:20:47.535 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.532+0000 7fc4ec3ce980 20 remove_watcher() i=2 2024-05-30T16:20:47.537 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.532+0000 7fc4ec3ce980 20 remove_watcher() i=3 2024-05-30T16:20:47.539 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.536+0000 7fc4ec3ce980 20 remove_watcher() i=4 2024-05-30T16:20:47.540 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.536+0000 7fc4ec3ce980 20 remove_watcher() i=5 2024-05-30T16:20:47.542 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.540+0000 7fc4ec3ce980 20 remove_watcher() i=6 2024-05-30T16:20:47.543 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:47.540+0000 7fc4ec3ce980 20 remove_watcher() i=7 2024-05-30T16:20:47.556 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:47.556 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:20:47.556 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: abort-at-set-target-layout id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.22 num_objects: 19 size_kb: 1 num_shards: 12 2024-05-30T16:20:47.556 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:48.377 INFO:tasks.workunit.client.0.smithi006.stderr:TEST: reshard bucket with EIO injected at block_writes 2024-05-30T16:20:48.377 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:50.434 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket error-at-block-writes 2024-05-30T16:20:50.468 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:20:50.468 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:20:50.514 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.512+0000 7f0b0758b980 20 rados->read ofs=0 len=0 2024-05-30T16:20:50.516 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.512+0000 7f0b0758b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:50.516 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.512+0000 7f0b0758b980 20 realm 2024-05-30T16:20:50.516 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.512+0000 7f0b0758b980 20 rados->read ofs=0 len=0 2024-05-30T16:20:50.517 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.512+0000 7f0b0758b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:50.517 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.512+0000 7f0b0758b980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:50.517 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.512+0000 7f0b0758b980 20 rados->read ofs=0 len=0 2024-05-30T16:20:50.517 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.512+0000 7f0b0758b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:50.517 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.512+0000 7f0b0758b980 20 rados->read ofs=0 len=0 2024-05-30T16:20:50.518 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.516+0000 7f0b0758b980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:50.518 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.516+0000 7f0b0758b980 20 rados->read ofs=0 len=0 2024-05-30T16:20:50.519 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.516+0000 7f0b0758b980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:50.519 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.516+0000 7f0b0758b980 20 searching for the correct realm 2024-05-30T16:20:50.525 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.520+0000 7f0b0589d640 0 -- 172.21.15.6:0/2373281598 >> [v2:172.21.15.204:6823/3156311527,v1:172.21.15.204:6825/3156311527] conn(0x7f0afc037070 msgr2=0x7f0afc047920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:50.531 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.528+0000 7f0b0509c640 0 -- 172.21.15.6:0/2373281598 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f0afc019920 msgr2=0x7f0afc038400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:50.749 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.744+0000 7f0b0758b980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:50.749 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.744+0000 7f0b0758b980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:50.749 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.744+0000 7f0b0758b980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:50.749 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.744+0000 7f0b0758b980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:50.750 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.744+0000 7f0b0758b980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:50.750 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.744+0000 7f0b0758b980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:50.750 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.744+0000 7f0b0758b980 20 rados->read ofs=0 len=0 2024-05-30T16:20:50.750 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.744+0000 7f0b0758b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:50.751 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.744+0000 7f0b0758b980 20 rados->read ofs=0 len=0 2024-05-30T16:20:50.751 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.748+0000 7f0b0758b980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:50.751 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.748+0000 7f0b0758b980 20 rados->read ofs=0 len=0 2024-05-30T16:20:50.752 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.748+0000 7f0b0758b980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:50.752 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.748+0000 7f0b0758b980 20 zone default found 2024-05-30T16:20:50.752 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.748+0000 7f0b0758b980 4 Realm: () 2024-05-30T16:20:50.752 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.748+0000 7f0b0758b980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:50.752 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.748+0000 7f0b0758b980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:50.752 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.748+0000 7f0b0758b980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:50.752 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.748+0000 7f0b0758b980 20 zonegroup default 2024-05-30T16:20:50.752 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.748+0000 7f0b0758b980 20 rados->read ofs=0 len=0 2024-05-30T16:20:50.753 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.748+0000 7f0b0758b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:50.753 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.748+0000 7f0b0758b980 20 rados->read ofs=0 len=0 2024-05-30T16:20:50.753 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.748+0000 7f0b0758b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:50.753 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.748+0000 7f0b0758b980 20 rados->read ofs=0 len=0 2024-05-30T16:20:50.754 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.752+0000 7f0b0758b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:50.754 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.752+0000 7f0b0758b980 20 started sync module instance, tier type = 2024-05-30T16:20:50.754 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.752+0000 7f0b0758b980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:50.778 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.776+0000 7f0b0758b980 20 add_watcher() i=0 2024-05-30T16:20:50.778 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.776+0000 7f0b0758b980 20 add_watcher() i=1 2024-05-30T16:20:50.778 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.776+0000 7f0b0758b980 20 add_watcher() i=2 2024-05-30T16:20:50.778 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.776+0000 7f0b0758b980 20 add_watcher() i=3 2024-05-30T16:20:50.778 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.776+0000 7f0b0758b980 20 add_watcher() i=4 2024-05-30T16:20:50.778 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.776+0000 7f0b0758b980 20 add_watcher() i=5 2024-05-30T16:20:50.778 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.776+0000 7f0b0758b980 20 add_watcher() i=6 2024-05-30T16:20:50.780 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.776+0000 7f0b0758b980 20 add_watcher() i=7 2024-05-30T16:20:50.780 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.776+0000 7f0b0758b980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:50.784 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.780+0000 7f09bb7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:50.785 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.784+0000 7f0b0758b980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:50.785 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.784+0000 7f0b0758b980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:50.786 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.784+0000 7f0b0758b980 5 note: GC not initialized 2024-05-30T16:20:50.786 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.784+0000 7f09ba7e4640 20 reqs_thread_entry: start 2024-05-30T16:20:50.786 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.784+0000 7f0b0758b980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:50.787 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.784+0000 7f0b0758b980 20 Filter name: none 2024-05-30T16:20:50.787 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.784+0000 7f092bfff640 20 reqs_thread_entry: start 2024-05-30T16:20:50.787 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.784+0000 7f0b0758b980 20 rados->read ofs=0 len=0 2024-05-30T16:20:50.788 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.784+0000 7f0b0758b980 20 rados_obj.operate() r=0 bl.length=213 2024-05-30T16:20:50.788 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.784+0000 7f0b0758b980 20 rados->read ofs=0 len=0 2024-05-30T16:20:50.790 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.788+0000 7f0b0758b980 20 rados_obj.operate() r=0 bl.length=427 2024-05-30T16:20:50.790 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.788+0000 7f0b0758b980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:50.790 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.788+0000 7f0b0758b980 20 rados->read ofs=0 len=0 2024-05-30T16:20:50.791 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.788+0000 7f0b0758b980 20 rados_obj.operate() r=0 bl.length=213 2024-05-30T16:20:50.791 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.788+0000 7f0b0758b980 20 rados->read ofs=0 len=0 2024-05-30T16:20:50.792 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.788+0000 7f0b0758b980 20 rados_obj.operate() r=0 bl.length=427 2024-05-30T16:20:50.792 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.788+0000 7f0b0758b980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:50.795 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.792+0000 7f09bb7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:50.800 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.796+0000 7f0b0758b980 20 remove_watcher() i=0 2024-05-30T16:20:50.800 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.796+0000 7f0b0758b980 2 removed watcher, disabling cache 2024-05-30T16:20:50.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.800+0000 7f0b0758b980 20 remove_watcher() i=1 2024-05-30T16:20:50.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.800+0000 7f0b0758b980 20 remove_watcher() i=2 2024-05-30T16:20:50.807 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.804+0000 7f0b0758b980 20 remove_watcher() i=3 2024-05-30T16:20:50.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.808+0000 7f0b0758b980 20 remove_watcher() i=4 2024-05-30T16:20:50.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.808+0000 7f0b0758b980 20 remove_watcher() i=5 2024-05-30T16:20:50.813 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.808+0000 7f0b0758b980 20 remove_watcher() i=6 2024-05-30T16:20:50.815 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:50.812+0000 7f0b0758b980 20 remove_watcher() i=7 2024-05-30T16:20:50.831 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:50.831 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:20:50.831 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: error-at-block-writes id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-30T16:20:50.831 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:50.831 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket reshard --bucket error-at-block-writes --num-shards 12 --rgw-reshard-bucket-lock-duration 30 --inject-error-at block_writes 2024-05-30T16:20:50.864 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:20:50.864 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:20:51.117 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.112+0000 7fb0eb590980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.118 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.116+0000 7fb0eb590980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:51.119 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.116+0000 7fb0eb590980 20 realm 2024-05-30T16:20:51.119 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.116+0000 7fb0eb590980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.119 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.116+0000 7fb0eb590980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:51.119 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.116+0000 7fb0eb590980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:51.119 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.116+0000 7fb0eb590980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.120 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.116+0000 7fb0eb590980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:51.120 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.116+0000 7fb0eb590980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.121 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.116+0000 7fb0eb590980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:51.121 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.116+0000 7fb0eb590980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.121 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.120+0000 7fb0eb590980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:51.122 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.120+0000 7fb0eb590980 20 searching for the correct realm 2024-05-30T16:20:51.126 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.124+0000 7fb0e88a0640 0 -- 172.21.15.6:0/966810371 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7fb0dc0bab30 msgr2=0x7fb0dc0fbf10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:51.135 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.132+0000 7fb0e90a1640 0 -- 172.21.15.6:0/966810371 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7fb0e007b840 msgr2=0x7fb0e0072770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:51.354 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.352+0000 7fb0eb590980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:51.355 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.352+0000 7fb0eb590980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:51.355 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.352+0000 7fb0eb590980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:51.355 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.352+0000 7fb0eb590980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:51.355 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.352+0000 7fb0eb590980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:51.355 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.352+0000 7fb0eb590980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:51.355 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.352+0000 7fb0eb590980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.356 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.352+0000 7fb0eb590980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:51.356 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.352+0000 7fb0eb590980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.356 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.352+0000 7fb0eb590980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:51.356 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.352+0000 7fb0eb590980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.357 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.352+0000 7fb0eb590980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:51.357 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.352+0000 7fb0eb590980 20 zone default found 2024-05-30T16:20:51.357 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.352+0000 7fb0eb590980 4 Realm: () 2024-05-30T16:20:51.358 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.352+0000 7fb0eb590980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:51.358 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.352+0000 7fb0eb590980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:51.358 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.352+0000 7fb0eb590980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:51.358 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.352+0000 7fb0eb590980 20 zonegroup default 2024-05-30T16:20:51.358 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.352+0000 7fb0eb590980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.359 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.356+0000 7fb0eb590980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:51.359 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.356+0000 7fb0eb590980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.359 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.356+0000 7fb0eb590980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:51.359 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.356+0000 7fb0eb590980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.356+0000 7fb0eb590980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:51.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.356+0000 7fb0eb590980 20 started sync module instance, tier type = 2024-05-30T16:20:51.360 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.356+0000 7fb0eb590980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:51.386 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.384+0000 7fb0eb590980 20 add_watcher() i=0 2024-05-30T16:20:51.386 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.384+0000 7fb0eb590980 20 add_watcher() i=1 2024-05-30T16:20:51.386 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.384+0000 7fb0eb590980 20 add_watcher() i=2 2024-05-30T16:20:51.386 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.384+0000 7fb0eb590980 20 add_watcher() i=3 2024-05-30T16:20:51.387 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.384+0000 7fb0eb590980 20 add_watcher() i=4 2024-05-30T16:20:51.387 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.384+0000 7fb0eb590980 20 add_watcher() i=5 2024-05-30T16:20:51.387 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.384+0000 7fb0eb590980 20 add_watcher() i=6 2024-05-30T16:20:51.388 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.384+0000 7fb0eb590980 20 add_watcher() i=7 2024-05-30T16:20:51.389 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.384+0000 7fb0eb590980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:51.393 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.388+0000 7faf9ffe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:51.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.392+0000 7fb0eb590980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:51.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.392+0000 7fb0eb590980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:51.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.392+0000 7fb0eb590980 5 note: GC not initialized 2024-05-30T16:20:51.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.392+0000 7faf9efe5640 20 reqs_thread_entry: start 2024-05-30T16:20:51.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.392+0000 7fb0eb590980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:51.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.392+0000 7fb0eb590980 20 Filter name: none 2024-05-30T16:20:51.396 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.392+0000 7faf9cfe1640 20 reqs_thread_entry: start 2024-05-30T16:20:51.396 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.392+0000 7fb0eb590980 10 cache get: name=default.rgw.meta+root+error-at-block-writes : miss 2024-05-30T16:20:51.396 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.392+0000 7fb0eb590980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.397 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.392+0000 7fb0eb590980 20 rados_obj.operate() r=0 bl.length=213 2024-05-30T16:20:51.397 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.392+0000 7fb0eb590980 10 cache put: name=default.rgw.meta+root+error-at-block-writes info.flags=0x11 2024-05-30T16:20:51.397 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.392+0000 7fb0eb590980 10 adding default.rgw.meta+root+error-at-block-writes to cache LRU end 2024-05-30T16:20:51.397 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.392+0000 7fb0eb590980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23 : miss 2024-05-30T16:20:51.397 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.392+0000 7fb0eb590980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.398 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.396+0000 7fb0eb590980 20 rados_obj.operate() r=0 bl.length=427 2024-05-30T16:20:51.398 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.396+0000 7fb0eb590980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23 info.flags=0x17 2024-05-30T16:20:51.398 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.396+0000 7fb0eb590980 10 adding default.rgw.meta+root+.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23 to cache LRU end 2024-05-30T16:20:51.398 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.396+0000 7fb0eb590980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:20:51.399 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.396+0000 7fb0eb590980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:51.399 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.396+0000 7fb0eb590980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23 2024-05-30T16:20:51.409 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.408+0000 7fb0eb590980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23 info.flags=0x17 2024-05-30T16:20:51.409 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.408+0000 7fb0eb590980 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23 to cache LRU end 2024-05-30T16:20:51.410 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.408+0000 7fb0eb590980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:51.410 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.408+0000 7fb0eb590980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23, ofs0, ns] 2024-05-30T16:20:51.410 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.408+0000 7fafbffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431568 cookie 93892174067024 notifier 6137 bl.length()=991 2024-05-30T16:20:51.411 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.408+0000 7fafbffff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23 info.flags=0x17 2024-05-30T16:20:51.411 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.408+0000 7fafbffff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23 to cache LRU end 2024-05-30T16:20:51.411 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.408+0000 7fafbffff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:51.412 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.408+0000 7fb0eb590980 20 handle_bi_update(): bucket=:error-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23]): orig_sources= new_sources= 2024-05-30T16:20:51.412 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.408+0000 7fb0eb590980 20 handle_bi_update(): bucket=:error-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23]): potential sources added=[] removed=[] 2024-05-30T16:20:51.413 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.408+0000 7fb0eb590980 20 handle_bi_update(): bucket=:error-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23]): orig_dests= new_dests= 2024-05-30T16:20:51.413 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.408+0000 7fb0eb590980 20 handle_bi_update(): bucket=:error-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23]): potential dests added=[] removed=[] 2024-05-30T16:20:51.413 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.408+0000 7fb0eb590980 -1 Injecting error=-5 at location=block_writes 2024-05-30T16:20:51.413 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.408+0000 7fb0eb590980 0 ERROR: init_reshard failed to pause writes to the current index: (5) Input/output error 2024-05-30T16:20:51.417 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.412+0000 7fb0eb590980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23 : hit (requested=0x17, cached=0x17) 2024-05-30T16:20:51.417 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.412+0000 7fb0eb590980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23 2024-05-30T16:20:51.420 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.416+0000 7fb0eb590980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23 info.flags=0x17 2024-05-30T16:20:51.420 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.416+0000 7fb0eb590980 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23 to cache LRU end 2024-05-30T16:20:51.420 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.416+0000 7fb0eb590980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:51.420 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.416+0000 7fb0eb590980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23, ofs0, ns] 2024-05-30T16:20:51.421 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.416+0000 7fafccff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431569 cookie 93892174067024 notifier 6137 bl.length()=959 2024-05-30T16:20:51.421 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.416+0000 7fafccff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23 info.flags=0x17 2024-05-30T16:20:51.421 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.416+0000 7fafccff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23 to cache LRU end 2024-05-30T16:20:51.421 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.416+0000 7fafccff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:51.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.420+0000 7fb0eb590980 20 handle_bi_update(): bucket=:error-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23]): orig_sources= new_sources= 2024-05-30T16:20:51.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.420+0000 7fb0eb590980 20 handle_bi_update(): bucket=:error-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23]): potential sources added=[] removed=[] 2024-05-30T16:20:51.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.420+0000 7fb0eb590980 20 handle_bi_update(): bucket=:error-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23]): orig_dests= new_dests= 2024-05-30T16:20:51.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.420+0000 7fb0eb590980 20 handle_bi_update(): bucket=:error-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23]): potential dests added=[] removed=[] 2024-05-30T16:20:51.426 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.424+0000 7faf9ffe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:51.432 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.428+0000 7fb0eb590980 20 remove_watcher() i=0 2024-05-30T16:20:51.432 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.428+0000 7fb0eb590980 2 removed watcher, disabling cache 2024-05-30T16:20:51.434 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.432+0000 7fb0eb590980 20 remove_watcher() i=1 2024-05-30T16:20:51.438 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.432+0000 7fb0eb590980 20 remove_watcher() i=2 2024-05-30T16:20:51.439 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.436+0000 7fb0eb590980 20 remove_watcher() i=3 2024-05-30T16:20:51.441 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.440+0000 7fb0eb590980 20 remove_watcher() i=4 2024-05-30T16:20:51.443 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.440+0000 7fb0eb590980 20 remove_watcher() i=5 2024-05-30T16:20:51.444 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.440+0000 7fb0eb590980 20 remove_watcher() i=6 2024-05-30T16:20:51.446 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.444+0000 7fb0eb590980 20 remove_watcher() i=7 2024-05-30T16:20:51.461 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket error-at-block-writes 2024-05-30T16:20:51.489 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:20:51.489 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:20:51.534 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.532+0000 7f9d4f494980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.532+0000 7f9d4f494980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:51.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.532+0000 7f9d4f494980 20 realm 2024-05-30T16:20:51.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.532+0000 7f9d4f494980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.537 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.532+0000 7f9d4f494980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:51.537 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.532+0000 7f9d4f494980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:51.537 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.532+0000 7f9d4f494980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.537 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.532+0000 7f9d4f494980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:51.537 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.532+0000 7f9d4f494980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.538 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.536+0000 7f9d4f494980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:51.538 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.536+0000 7f9d4f494980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.539 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.536+0000 7f9d4f494980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:51.539 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.536+0000 7f9d4f494980 20 searching for the correct realm 2024-05-30T16:20:51.542 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.540+0000 7f9d4cfa5640 0 -- 172.21.15.6:0/41269864 >> [v2:172.21.15.6:6816/2704410641,v1:172.21.15.6:6817/2704410641] conn(0x7f9d4401a040 msgr2=0x7f9d4401b940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:51.548 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.544+0000 7f9d4d7a6640 0 -- 172.21.15.6:0/41269864 >> [v2:172.21.15.204:6823/3156311527,v1:172.21.15.204:6825/3156311527] conn(0x7f9d34003100 msgr2=0x7f9d400dd480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:20:51.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.556+0000 7f9d4f494980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:51.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.556+0000 7f9d4f494980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:51.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.556+0000 7f9d4f494980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:51.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.556+0000 7f9d4f494980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:51.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.556+0000 7f9d4f494980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:51.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.556+0000 7f9d4f494980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:51.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.556+0000 7f9d4f494980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.556+0000 7f9d4f494980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:51.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.556+0000 7f9d4f494980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.562 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.560+0000 7f9d4f494980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:51.562 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.560+0000 7f9d4f494980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.562 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.560+0000 7f9d4f494980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:51.562 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.560+0000 7f9d4f494980 20 zone default found 2024-05-30T16:20:51.562 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.560+0000 7f9d4f494980 4 Realm: () 2024-05-30T16:20:51.562 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.560+0000 7f9d4f494980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:51.562 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.560+0000 7f9d4f494980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:51.563 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.560+0000 7f9d4f494980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:51.563 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.560+0000 7f9d4f494980 20 zonegroup default 2024-05-30T16:20:51.563 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.560+0000 7f9d4f494980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.563 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.560+0000 7f9d4f494980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:51.563 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.560+0000 7f9d4f494980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.563 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.560+0000 7f9d4f494980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:51.563 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.560+0000 7f9d4f494980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.564 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.560+0000 7f9d4f494980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:51.564 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.560+0000 7f9d4f494980 20 started sync module instance, tier type = 2024-05-30T16:20:51.564 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.560+0000 7f9d4f494980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:51.580 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.576+0000 7f9d4f494980 20 add_watcher() i=0 2024-05-30T16:20:51.580 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.576+0000 7f9d4f494980 20 add_watcher() i=1 2024-05-30T16:20:51.580 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.576+0000 7f9d4f494980 20 add_watcher() i=2 2024-05-30T16:20:51.580 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.576+0000 7f9d4f494980 20 add_watcher() i=3 2024-05-30T16:20:51.580 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.576+0000 7f9d4f494980 20 add_watcher() i=4 2024-05-30T16:20:51.580 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.576+0000 7f9d4f494980 20 add_watcher() i=5 2024-05-30T16:20:51.580 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.576+0000 7f9d4f494980 20 add_watcher() i=6 2024-05-30T16:20:51.581 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.576+0000 7f9d4f494980 20 add_watcher() i=7 2024-05-30T16:20:51.581 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.576+0000 7f9d4f494980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:51.583 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.580+0000 7f9c037e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:51.584 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.580+0000 7f9d4f494980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:51.585 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.580+0000 7f9d4f494980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:51.585 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.580+0000 7f9d4f494980 5 note: GC not initialized 2024-05-30T16:20:51.585 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.580+0000 7f9c027e4640 20 reqs_thread_entry: start 2024-05-30T16:20:51.585 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.580+0000 7f9d4f494980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:51.585 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.580+0000 7f9d4f494980 20 Filter name: none 2024-05-30T16:20:51.585 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.580+0000 7f9bfb7fe640 20 reqs_thread_entry: start 2024-05-30T16:20:51.585 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.580+0000 7f9d4f494980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.585 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.584+0000 7f9d4f494980 20 rados_obj.operate() r=0 bl.length=213 2024-05-30T16:20:51.585 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.584+0000 7f9d4f494980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.587 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.584+0000 7f9d4f494980 20 rados_obj.operate() r=0 bl.length=427 2024-05-30T16:20:51.587 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.584+0000 7f9d4f494980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:51.587 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.584+0000 7f9d4f494980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.587 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.584+0000 7f9d4f494980 20 rados_obj.operate() r=0 bl.length=213 2024-05-30T16:20:51.587 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.584+0000 7f9d4f494980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.588 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.584+0000 7f9d4f494980 20 rados_obj.operate() r=0 bl.length=427 2024-05-30T16:20:51.588 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.584+0000 7f9d4f494980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:51.591 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.588+0000 7f9c037e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:51.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.592+0000 7f9d4f494980 20 remove_watcher() i=0 2024-05-30T16:20:51.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.592+0000 7f9d4f494980 2 removed watcher, disabling cache 2024-05-30T16:20:51.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.596+0000 7f9d4f494980 20 remove_watcher() i=1 2024-05-30T16:20:51.600 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.596+0000 7f9d4f494980 20 remove_watcher() i=2 2024-05-30T16:20:51.602 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.600+0000 7f9d4f494980 20 remove_watcher() i=3 2024-05-30T16:20:51.604 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.600+0000 7f9d4f494980 20 remove_watcher() i=4 2024-05-30T16:20:51.605 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.604+0000 7f9d4f494980 20 remove_watcher() i=5 2024-05-30T16:20:51.607 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.604+0000 7f9d4f494980 20 remove_watcher() i=6 2024-05-30T16:20:51.608 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.604+0000 7f9d4f494980 20 remove_watcher() i=7 2024-05-30T16:20:51.622 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:51.622 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:20:51.622 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: error-at-block-writes id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-30T16:20:51.622 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:51.719 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket reshard --bucket error-at-block-writes --num-shards 12 --rgw-reshard-bucket-lock-duration 30 2024-05-30T16:20:51.755 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:20:51.755 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:20:51.800 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.796+0000 7f41afe54980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.800+0000 7f41afe54980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:51.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.800+0000 7f41afe54980 20 realm 2024-05-30T16:20:51.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.800+0000 7f41afe54980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.800+0000 7f41afe54980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:51.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.800+0000 7f41afe54980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:51.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.800+0000 7f41afe54980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.800+0000 7f41afe54980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:51.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.800+0000 7f41afe54980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.804 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.800+0000 7f41afe54980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:51.804 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.800+0000 7f41afe54980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.800+0000 7f41afe54980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:51.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.800+0000 7f41afe54980 20 searching for the correct realm 2024-05-30T16:20:51.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.808+0000 7f41ad164640 0 -- 172.21.15.6:0/2821595306 >> [v2:172.21.15.6:6808/1897362715,v1:172.21.15.6:6809/1897362715] conn(0x5596ac471fc0 msgr2=0x5596ac474450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:20:51.817 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.812+0000 7f41ad965640 0 -- 172.21.15.6:0/2821595306 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f41a4048430 msgr2=0x7f41a40488b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:51.825 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.820+0000 7f41afe54980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:51.825 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.820+0000 7f41afe54980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:51.825 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.820+0000 7f41afe54980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:51.825 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.820+0000 7f41afe54980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:51.825 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.820+0000 7f41afe54980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:51.825 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.820+0000 7f41afe54980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:51.825 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.820+0000 7f41afe54980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.826 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.820+0000 7f41afe54980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:51.826 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.820+0000 7f41afe54980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.826 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.824+0000 7f41afe54980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:51.826 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.824+0000 7f41afe54980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.827 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.824+0000 7f41afe54980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:51.827 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.824+0000 7f41afe54980 20 zone default found 2024-05-30T16:20:51.827 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.824+0000 7f41afe54980 4 Realm: () 2024-05-30T16:20:51.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.824+0000 7f41afe54980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:51.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.824+0000 7f41afe54980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:51.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.824+0000 7f41afe54980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:51.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.824+0000 7f41afe54980 20 zonegroup default 2024-05-30T16:20:51.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.824+0000 7f41afe54980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.824+0000 7f41afe54980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:51.828 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.824+0000 7f41afe54980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.829 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.824+0000 7f41afe54980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:51.829 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.824+0000 7f41afe54980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.829 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.824+0000 7f41afe54980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:51.829 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.824+0000 7f41afe54980 20 started sync module instance, tier type = 2024-05-30T16:20:51.829 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.824+0000 7f41afe54980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:51.843 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.840+0000 7f41afe54980 20 add_watcher() i=0 2024-05-30T16:20:51.844 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.840+0000 7f41afe54980 20 add_watcher() i=1 2024-05-30T16:20:51.844 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.840+0000 7f41afe54980 20 add_watcher() i=2 2024-05-30T16:20:51.844 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.840+0000 7f41afe54980 20 add_watcher() i=3 2024-05-30T16:20:51.844 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.840+0000 7f41afe54980 20 add_watcher() i=4 2024-05-30T16:20:51.844 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.840+0000 7f41afe54980 20 add_watcher() i=5 2024-05-30T16:20:51.844 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.840+0000 7f41afe54980 20 add_watcher() i=6 2024-05-30T16:20:51.845 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.840+0000 7f41afe54980 20 add_watcher() i=7 2024-05-30T16:20:51.845 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.840+0000 7f41afe54980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:51.849 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.844+0000 7f405f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:51.851 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.844+0000 7f41afe54980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:51.851 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.844+0000 7f41afe54980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:51.851 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.844+0000 7f41afe54980 5 note: GC not initialized 2024-05-30T16:20:51.851 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.844+0000 7f405e7fc640 20 reqs_thread_entry: start 2024-05-30T16:20:51.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.844+0000 7f41afe54980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:51.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.844+0000 7f41afe54980 20 Filter name: none 2024-05-30T16:20:51.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.844+0000 7f3fcffff640 20 reqs_thread_entry: start 2024-05-30T16:20:51.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.844+0000 7f41afe54980 10 cache get: name=default.rgw.meta+root+error-at-block-writes : miss 2024-05-30T16:20:51.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.844+0000 7f41afe54980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.848+0000 7f41afe54980 20 rados_obj.operate() r=0 bl.length=213 2024-05-30T16:20:51.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.848+0000 7f41afe54980 10 cache put: name=default.rgw.meta+root+error-at-block-writes info.flags=0x11 2024-05-30T16:20:51.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.848+0000 7f41afe54980 10 adding default.rgw.meta+root+error-at-block-writes to cache LRU end 2024-05-30T16:20:51.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.848+0000 7f41afe54980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23 : miss 2024-05-30T16:20:51.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.848+0000 7f41afe54980 20 rados->read ofs=0 len=0 2024-05-30T16:20:51.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.848+0000 7f41afe54980 20 rados_obj.operate() r=0 bl.length=427 2024-05-30T16:20:51.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.848+0000 7f41afe54980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23 info.flags=0x17 2024-05-30T16:20:51.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.848+0000 7f41afe54980 10 adding default.rgw.meta+root+.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23 to cache LRU end 2024-05-30T16:20:51.853 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.848+0000 7f41afe54980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:20:51.853 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.848+0000 7f41afe54980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:51.853 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.848+0000 7f41afe54980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23 2024-05-30T16:20:51.861 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.856+0000 7f41afe54980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23 info.flags=0x17 2024-05-30T16:20:51.861 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.856+0000 7f41afe54980 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23 to cache LRU end 2024-05-30T16:20:51.861 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.856+0000 7f41afe54980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:51.861 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.856+0000 7f41afe54980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23, ofs0, ns] 2024-05-30T16:20:51.862 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.860+0000 7f40917fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431570 cookie 94105623859568 notifier 6170 bl.length()=991 2024-05-30T16:20:51.862 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.860+0000 7f40917fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23 info.flags=0x17 2024-05-30T16:20:51.862 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.860+0000 7f40917fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23 to cache LRU end 2024-05-30T16:20:51.862 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.860+0000 7f40917fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:51.864 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.860+0000 7f41afe54980 20 handle_bi_update(): bucket=:error-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23]): orig_sources= new_sources= 2024-05-30T16:20:51.864 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.860+0000 7f41afe54980 20 handle_bi_update(): bucket=:error-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23]): potential sources added=[] removed=[] 2024-05-30T16:20:51.864 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.860+0000 7f41afe54980 20 handle_bi_update(): bucket=:error-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23]): orig_dests= new_dests= 2024-05-30T16:20:51.864 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.860+0000 7f41afe54980 20 handle_bi_update(): bucket=:error-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23]): potential dests added=[] removed=[] 2024-05-30T16:20:51.868 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.864+0000 7f41afe54980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.1.0 2024-05-30T16:20:51.868 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.864+0000 7f41afe54980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.1.0 2024-05-30T16:20:51.868 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.864+0000 7f41afe54980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.1.1 2024-05-30T16:20:51.868 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.864+0000 7f41afe54980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.1.1 2024-05-30T16:20:51.868 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.864+0000 7f41afe54980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.1.2 2024-05-30T16:20:51.868 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.864+0000 7f41afe54980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.1.2 2024-05-30T16:20:51.868 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.864+0000 7f41afe54980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.1.3 2024-05-30T16:20:51.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.864+0000 7f41afe54980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.1.3 2024-05-30T16:20:51.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.864+0000 7f41afe54980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.1.4 2024-05-30T16:20:51.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.864+0000 7f41afe54980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.1.4 2024-05-30T16:20:51.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.864+0000 7f41afe54980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.1.5 2024-05-30T16:20:51.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.864+0000 7f41afe54980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.1.5 2024-05-30T16:20:51.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.864+0000 7f41afe54980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.1.6 2024-05-30T16:20:51.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.864+0000 7f41afe54980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.1.6 2024-05-30T16:20:51.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.864+0000 7f41afe54980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.1.7 2024-05-30T16:20:51.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.864+0000 7f41afe54980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.1.7 2024-05-30T16:20:51.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.864+0000 7f41afe54980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.1.8 2024-05-30T16:20:51.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.864+0000 7f41afe54980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.1.8 2024-05-30T16:20:51.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.864+0000 7f41afe54980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.1.9 2024-05-30T16:20:51.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.864+0000 7f41afe54980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.1.9 2024-05-30T16:20:51.870 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.864+0000 7f41afe54980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.1.10 2024-05-30T16:20:51.870 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.864+0000 7f41afe54980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.1.10 2024-05-30T16:20:51.870 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.864+0000 7f41afe54980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.1.11 2024-05-30T16:20:51.870 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.864+0000 7f41afe54980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.1.11 2024-05-30T16:20:51.870 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.864+0000 7f41afe54980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.0 2024-05-30T16:20:51.871 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.864+0000 7f41afe54980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.1 2024-05-30T16:20:51.871 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.868+0000 7f41afe54980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.2 2024-05-30T16:20:51.871 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.868+0000 7f41afe54980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.3 2024-05-30T16:20:51.871 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.868+0000 7f41afe54980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.4 2024-05-30T16:20:51.871 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.868+0000 7f41afe54980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.5 2024-05-30T16:20:51.872 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.868+0000 7f41afe54980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.6 2024-05-30T16:20:51.872 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.868+0000 7f41afe54980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.7 2024-05-30T16:20:51.873 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.868+0000 7f41afe54980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.8 2024-05-30T16:20:51.874 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.872+0000 7f41afe54980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.9 2024-05-30T16:20:51.875 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.872+0000 7f41afe54980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23.10 2024-05-30T16:20:51.881 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.876+0000 7f41afe54980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23 : hit (requested=0x17, cached=0x17) 2024-05-30T16:20:51.881 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.876+0000 7f41afe54980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23 2024-05-30T16:20:51.884 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.880+0000 7f41afe54980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23 info.flags=0x17 2024-05-30T16:20:51.884 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.880+0000 7f41afe54980 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23 to cache LRU end 2024-05-30T16:20:51.884 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.880+0000 7f41afe54980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:51.884 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.880+0000 7f41afe54980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23, ofs0, ns] 2024-05-30T16:20:51.885 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.880+0000 7f4090ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431571 cookie 94105623859568 notifier 6170 bl.length()=959 2024-05-30T16:20:51.885 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.880+0000 7f4090ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23 info.flags=0x17 2024-05-30T16:20:51.885 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.880+0000 7f4090ff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23 to cache LRU end 2024-05-30T16:20:51.885 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.880+0000 7f4090ff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:51.886 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.884+0000 7f41afe54980 20 handle_bi_update(): bucket=:error-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23]): orig_sources= new_sources= 2024-05-30T16:20:51.886 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.884+0000 7f41afe54980 20 handle_bi_update(): bucket=:error-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23]): potential sources added=[] removed=[] 2024-05-30T16:20:51.886 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.884+0000 7f41afe54980 20 handle_bi_update(): bucket=:error-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23]): orig_dests= new_dests= 2024-05-30T16:20:51.886 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.884+0000 7f41afe54980 20 handle_bi_update(): bucket=:error-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23]): potential dests added=[] removed=[] 2024-05-30T16:20:51.890 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.888+0000 7f41afe54980 1 execute INFO: reshard of bucket "error-at-block-writes" completed successfully 2024-05-30T16:20:51.893 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.892+0000 7f405f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:51.898 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.896+0000 7f41afe54980 20 remove_watcher() i=0 2024-05-30T16:20:51.898 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.896+0000 7f41afe54980 2 removed watcher, disabling cache 2024-05-30T16:20:51.900 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.896+0000 7f41afe54980 20 remove_watcher() i=1 2024-05-30T16:20:51.902 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.900+0000 7f41afe54980 20 remove_watcher() i=2 2024-05-30T16:20:51.904 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.900+0000 7f41afe54980 20 remove_watcher() i=3 2024-05-30T16:20:51.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.904+0000 7f41afe54980 20 remove_watcher() i=4 2024-05-30T16:20:51.907 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.904+0000 7f41afe54980 20 remove_watcher() i=5 2024-05-30T16:20:51.909 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.904+0000 7f41afe54980 20 remove_watcher() i=6 2024-05-30T16:20:51.910 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.908+0000 7f41afe54980 20 remove_watcher() i=7 2024-05-30T16:20:51.925 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket error-at-block-writes 2024-05-30T16:20:51.955 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:20:51.955 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:20:51.998 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.996+0000 7f45e66bc980 20 rados->read ofs=0 len=0 2024-05-30T16:20:52.000 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.996+0000 7f45e66bc980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:52.000 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.996+0000 7f45e66bc980 20 realm 2024-05-30T16:20:52.000 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.996+0000 7f45e66bc980 20 rados->read ofs=0 len=0 2024-05-30T16:20:52.000 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.996+0000 7f45e66bc980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:52.000 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.996+0000 7f45e66bc980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:52.000 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.996+0000 7f45e66bc980 20 rados->read ofs=0 len=0 2024-05-30T16:20:52.000 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.996+0000 7f45e66bc980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:52.001 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:51.996+0000 7f45e66bc980 20 rados->read ofs=0 len=0 2024-05-30T16:20:52.002 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.000+0000 7f45e66bc980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:52.002 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.000+0000 7f45e66bc980 20 rados->read ofs=0 len=0 2024-05-30T16:20:52.002 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.000+0000 7f45e66bc980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:52.003 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.000+0000 7f45e66bc980 20 searching for the correct realm 2024-05-30T16:20:52.008 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.004+0000 7f45e49ce640 0 -- 172.21.15.6:0/2328723933 >> [v2:172.21.15.204:6823/3156311527,v1:172.21.15.204:6825/3156311527] conn(0x7f45d8046040 msgr2=0x7f45d8039970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:52.014 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.012+0000 7f45e41cd640 0 -- 172.21.15.6:0/2328723933 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f45d8059040 msgr2=0x7f45d8059aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:20:52.232 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.228+0000 7f45e66bc980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:52.232 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.228+0000 7f45e66bc980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:52.233 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.228+0000 7f45e66bc980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:52.233 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.228+0000 7f45e66bc980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:52.233 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.228+0000 7f45e66bc980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:52.233 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.228+0000 7f45e66bc980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:52.233 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.228+0000 7f45e66bc980 20 rados->read ofs=0 len=0 2024-05-30T16:20:52.233 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.228+0000 7f45e66bc980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:52.233 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.228+0000 7f45e66bc980 20 rados->read ofs=0 len=0 2024-05-30T16:20:52.234 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.232+0000 7f45e66bc980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:52.234 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.232+0000 7f45e66bc980 20 rados->read ofs=0 len=0 2024-05-30T16:20:52.235 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.232+0000 7f45e66bc980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:52.235 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.232+0000 7f45e66bc980 20 zone default found 2024-05-30T16:20:52.235 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.232+0000 7f45e66bc980 4 Realm: () 2024-05-30T16:20:52.235 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.232+0000 7f45e66bc980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:52.235 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.232+0000 7f45e66bc980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:52.235 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.232+0000 7f45e66bc980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:52.235 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.232+0000 7f45e66bc980 20 zonegroup default 2024-05-30T16:20:52.235 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.232+0000 7f45e66bc980 20 rados->read ofs=0 len=0 2024-05-30T16:20:52.236 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.232+0000 7f45e66bc980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:52.236 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.232+0000 7f45e66bc980 20 rados->read ofs=0 len=0 2024-05-30T16:20:52.236 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.232+0000 7f45e66bc980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:52.236 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.232+0000 7f45e66bc980 20 rados->read ofs=0 len=0 2024-05-30T16:20:52.237 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.232+0000 7f45e66bc980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:52.237 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.232+0000 7f45e66bc980 20 started sync module instance, tier type = 2024-05-30T16:20:52.237 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.232+0000 7f45e66bc980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:52.263 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.260+0000 7f45e66bc980 20 add_watcher() i=0 2024-05-30T16:20:52.263 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.260+0000 7f45e66bc980 20 add_watcher() i=1 2024-05-30T16:20:52.263 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.260+0000 7f45e66bc980 20 add_watcher() i=2 2024-05-30T16:20:52.263 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.260+0000 7f45e66bc980 20 add_watcher() i=3 2024-05-30T16:20:52.263 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.260+0000 7f45e66bc980 20 add_watcher() i=4 2024-05-30T16:20:52.263 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.260+0000 7f45e66bc980 20 add_watcher() i=5 2024-05-30T16:20:52.263 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.260+0000 7f45e66bc980 20 add_watcher() i=6 2024-05-30T16:20:52.265 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.264+0000 7f45e66bc980 20 add_watcher() i=7 2024-05-30T16:20:52.266 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.264+0000 7f45e66bc980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:52.270 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.268+0000 7f449a7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:52.271 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.268+0000 7f45e66bc980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:52.271 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.268+0000 7f45e66bc980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:52.271 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.268+0000 7f45e66bc980 5 note: GC not initialized 2024-05-30T16:20:52.271 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.268+0000 7f44997e2640 20 reqs_thread_entry: start 2024-05-30T16:20:52.271 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.268+0000 7f45e66bc980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:52.272 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.268+0000 7f45e66bc980 20 Filter name: none 2024-05-30T16:20:52.272 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.268+0000 7f440affd640 20 reqs_thread_entry: start 2024-05-30T16:20:52.272 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.268+0000 7f45e66bc980 20 rados->read ofs=0 len=0 2024-05-30T16:20:52.273 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.268+0000 7f45e66bc980 20 rados_obj.operate() r=0 bl.length=213 2024-05-30T16:20:52.273 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.268+0000 7f45e66bc980 20 rados->read ofs=0 len=0 2024-05-30T16:20:52.274 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.272+0000 7f45e66bc980 20 rados_obj.operate() r=0 bl.length=427 2024-05-30T16:20:52.274 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.272+0000 7f45e66bc980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:52.275 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.272+0000 7f45e66bc980 20 rados->read ofs=0 len=0 2024-05-30T16:20:52.275 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.272+0000 7f45e66bc980 20 rados_obj.operate() r=0 bl.length=213 2024-05-30T16:20:52.275 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.272+0000 7f45e66bc980 20 rados->read ofs=0 len=0 2024-05-30T16:20:52.276 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.272+0000 7f45e66bc980 20 rados_obj.operate() r=0 bl.length=427 2024-05-30T16:20:52.276 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.272+0000 7f45e66bc980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:52.279 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.276+0000 7f449a7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:52.284 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.280+0000 7f45e66bc980 20 remove_watcher() i=0 2024-05-30T16:20:52.284 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.280+0000 7f45e66bc980 2 removed watcher, disabling cache 2024-05-30T16:20:52.287 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.284+0000 7f45e66bc980 20 remove_watcher() i=1 2024-05-30T16:20:52.290 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.288+0000 7f45e66bc980 20 remove_watcher() i=2 2024-05-30T16:20:52.292 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.288+0000 7f45e66bc980 20 remove_watcher() i=3 2024-05-30T16:20:52.295 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.292+0000 7f45e66bc980 20 remove_watcher() i=4 2024-05-30T16:20:52.297 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.292+0000 7f45e66bc980 20 remove_watcher() i=5 2024-05-30T16:20:52.298 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.296+0000 7f45e66bc980 20 remove_watcher() i=6 2024-05-30T16:20:52.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:52.296+0000 7f45e66bc980 20 remove_watcher() i=7 2024-05-30T16:20:52.316 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:52.316 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:20:52.316 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: error-at-block-writes id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.23 num_objects: 19 size_kb: 1 num_shards: 12 2024-05-30T16:20:52.316 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:53.224 INFO:tasks.workunit.client.0.smithi006.stderr:TEST: reshard bucket with abort at block_writes 2024-05-30T16:20:53.224 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.281 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket abort-at-block-writes 2024-05-30T16:20:55.318 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:20:55.318 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:20:55.365 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.364+0000 7ff703efb980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.367 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.364+0000 7ff703efb980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.367 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.364+0000 7ff703efb980 20 realm 2024-05-30T16:20:55.367 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.364+0000 7ff703efb980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.367 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.364+0000 7ff703efb980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.367 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.364+0000 7ff703efb980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:55.367 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.364+0000 7ff703efb980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.368 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.364+0000 7ff703efb980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.368 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.364+0000 7ff703efb980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.369 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.364+0000 7ff703efb980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:55.369 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.364+0000 7ff703efb980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.370 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.368+0000 7ff703efb980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:55.370 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.368+0000 7ff703efb980 20 searching for the correct realm 2024-05-30T16:20:55.376 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.372+0000 7ff70120b640 0 -- 172.21.15.6:0/1145032886 >> [v2:172.21.15.6:6819/1141406589,v1:172.21.15.6:6821/1141406589] conn(0x5556665708b0 msgr2=0x555666572ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:55.383 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.380+0000 7ff701a0c640 0 -- 172.21.15.6:0/1145032886 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7ff6f801f920 msgr2=0x7ff6f803d410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:55.390 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.388+0000 7ff703efb980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:55.391 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.388+0000 7ff703efb980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:55.391 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.388+0000 7ff703efb980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:55.391 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.388+0000 7ff703efb980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:55.391 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.388+0000 7ff703efb980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:55.391 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.388+0000 7ff703efb980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:55.391 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.388+0000 7ff703efb980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.392 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.388+0000 7ff703efb980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.392 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.388+0000 7ff703efb980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.392 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.388+0000 7ff703efb980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:55.392 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.388+0000 7ff703efb980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.393 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.388+0000 7ff703efb980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:55.393 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.388+0000 7ff703efb980 20 zone default found 2024-05-30T16:20:55.393 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.388+0000 7ff703efb980 4 Realm: () 2024-05-30T16:20:55.393 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.388+0000 7ff703efb980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:55.393 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.388+0000 7ff703efb980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:55.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.388+0000 7ff703efb980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:55.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.388+0000 7ff703efb980 20 zonegroup default 2024-05-30T16:20:55.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.388+0000 7ff703efb980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.388+0000 7ff703efb980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.388+0000 7ff703efb980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.392+0000 7ff703efb980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.392+0000 7ff703efb980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.392+0000 7ff703efb980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.392+0000 7ff703efb980 20 started sync module instance, tier type = 2024-05-30T16:20:55.396 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.392+0000 7ff703efb980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:55.410 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.408+0000 7ff703efb980 20 add_watcher() i=0 2024-05-30T16:20:55.410 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.408+0000 7ff703efb980 20 add_watcher() i=1 2024-05-30T16:20:55.410 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.408+0000 7ff703efb980 20 add_watcher() i=2 2024-05-30T16:20:55.410 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.408+0000 7ff703efb980 20 add_watcher() i=3 2024-05-30T16:20:55.411 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.408+0000 7ff703efb980 20 add_watcher() i=4 2024-05-30T16:20:55.411 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.408+0000 7ff703efb980 20 add_watcher() i=5 2024-05-30T16:20:55.411 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.408+0000 7ff703efb980 20 add_watcher() i=6 2024-05-30T16:20:55.411 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.408+0000 7ff703efb980 20 add_watcher() i=7 2024-05-30T16:20:55.411 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.408+0000 7ff703efb980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:55.414 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.412+0000 7ff5b37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:55.415 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.412+0000 7ff703efb980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:55.415 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.412+0000 7ff703efb980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:55.415 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.412+0000 7ff703efb980 5 note: GC not initialized 2024-05-30T16:20:55.415 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.412+0000 7ff5b27fc640 20 reqs_thread_entry: start 2024-05-30T16:20:55.415 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.412+0000 7ff703efb980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:55.415 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.412+0000 7ff703efb980 20 Filter name: none 2024-05-30T16:20:55.415 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.412+0000 7ff703efb980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.416 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.412+0000 7ff523fff640 20 reqs_thread_entry: start 2024-05-30T16:20:55.416 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.412+0000 7ff703efb980 20 rados_obj.operate() r=0 bl.length=213 2024-05-30T16:20:55.416 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.412+0000 7ff703efb980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.417 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.412+0000 7ff703efb980 20 rados_obj.operate() r=0 bl.length=427 2024-05-30T16:20:55.418 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.412+0000 7ff703efb980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:55.418 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.412+0000 7ff703efb980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.418 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.416+0000 7ff703efb980 20 rados_obj.operate() r=0 bl.length=213 2024-05-30T16:20:55.418 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.416+0000 7ff703efb980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.419 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.416+0000 7ff703efb980 20 rados_obj.operate() r=0 bl.length=427 2024-05-30T16:20:55.419 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.416+0000 7ff703efb980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:55.421 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.416+0000 7ff5b37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:55.426 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.424+0000 7ff703efb980 20 remove_watcher() i=0 2024-05-30T16:20:55.426 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.424+0000 7ff703efb980 2 removed watcher, disabling cache 2024-05-30T16:20:55.428 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.424+0000 7ff703efb980 20 remove_watcher() i=1 2024-05-30T16:20:55.430 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.432+0000 7ff703efb980 20 remove_watcher() i=2 2024-05-30T16:20:55.432 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.432+0000 7ff703efb980 20 remove_watcher() i=3 2024-05-30T16:20:55.434 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.436+0000 7ff703efb980 20 remove_watcher() i=4 2024-05-30T16:20:55.435 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.436+0000 7ff703efb980 20 remove_watcher() i=5 2024-05-30T16:20:55.437 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.436+0000 7ff703efb980 20 remove_watcher() i=6 2024-05-30T16:20:55.438 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.436+0000 7ff703efb980 20 remove_watcher() i=7 2024-05-30T16:20:55.452 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.452 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:20:55.452 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: abort-at-block-writes id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-30T16:20:55.452 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.452 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket reshard --bucket abort-at-block-writes --num-shards 12 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at block_writes 2024-05-30T16:20:55.484 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:20:55.484 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:20:55.707 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.708+0000 7fa7dc054640 0 --2- 172.21.15.6:0/4289678576 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x565264e23190 0x565264f78fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-05-30T16:20:55.736 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.732+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.738 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.736+0000 7fa7de543980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.738 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.736+0000 7fa7de543980 20 realm 2024-05-30T16:20:55.738 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.736+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.738 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.736+0000 7fa7de543980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.738 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.736+0000 7fa7de543980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:55.738 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.736+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.739 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.736+0000 7fa7de543980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.739 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.736+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.740 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.736+0000 7fa7de543980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:55.740 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.736+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.741 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.736+0000 7fa7de543980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:55.741 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.736+0000 7fa7de543980 20 searching for the correct realm 2024-05-30T16:20:55.744 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.744+0000 7fa7dc054640 0 -- 172.21.15.6:0/2873706297 >> [v2:172.21.15.6:6816/2704410641,v1:172.21.15.6:6817/2704410641] conn(0x7fa7d4017920 msgr2=0x7fa7d4029410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:55.751 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.748+0000 7fa7dc054640 0 -- 172.21.15.6:0/2873706297 >> [v2:172.21.15.6:6816/2704410641,v1:172.21.15.6:6817/2704410641] conn(0x7fa6a407e440 msgr2=0x7fa6a4080870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:55.764 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.760+0000 7fa7de543980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:55.764 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.760+0000 7fa7de543980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:55.764 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.760+0000 7fa7de543980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:55.764 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.760+0000 7fa7de543980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:55.764 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.760+0000 7fa7de543980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:55.764 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.760+0000 7fa7de543980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:55.764 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.760+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.764 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.760+0000 7fa7de543980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.764 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.760+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.765 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.760+0000 7fa7de543980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:55.765 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.760+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.766 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.768+0000 7fa7de543980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:55.766 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.768+0000 7fa7de543980 20 zone default found 2024-05-30T16:20:55.766 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.768+0000 7fa7de543980 4 Realm: () 2024-05-30T16:20:55.766 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.768+0000 7fa7de543980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:55.766 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.768+0000 7fa7de543980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:55.766 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.768+0000 7fa7de543980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:55.766 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.768+0000 7fa7de543980 20 zonegroup default 2024-05-30T16:20:55.766 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.768+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.766 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.768+0000 7fa7de543980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.766 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.768+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.767 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.768+0000 7fa7de543980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.767 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.768+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.767 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.768+0000 7fa7de543980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.767 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.768+0000 7fa7de543980 20 started sync module instance, tier type = 2024-05-30T16:20:55.767 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.768+0000 7fa7de543980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:55.784 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.780+0000 7fa7de543980 20 add_watcher() i=0 2024-05-30T16:20:55.784 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.780+0000 7fa7de543980 20 add_watcher() i=1 2024-05-30T16:20:55.784 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.780+0000 7fa7de543980 20 add_watcher() i=2 2024-05-30T16:20:55.784 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.780+0000 7fa7de543980 20 add_watcher() i=3 2024-05-30T16:20:55.784 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.780+0000 7fa7de543980 20 add_watcher() i=4 2024-05-30T16:20:55.784 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.780+0000 7fa7de543980 20 add_watcher() i=5 2024-05-30T16:20:55.784 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.780+0000 7fa7de543980 20 add_watcher() i=6 2024-05-30T16:20:55.785 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.780+0000 7fa7de543980 20 add_watcher() i=7 2024-05-30T16:20:55.785 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.780+0000 7fa7de543980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:55.788 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.784+0000 7fa692fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:55.788 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.784+0000 7fa7de543980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:55.789 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.784+0000 7fa7de543980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:55.789 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.784+0000 7fa7de543980 5 note: GC not initialized 2024-05-30T16:20:55.789 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.784+0000 7fa691fdb640 20 reqs_thread_entry: start 2024-05-30T16:20:55.789 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.784+0000 7fa7de543980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:55.789 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.784+0000 7fa7de543980 20 Filter name: none 2024-05-30T16:20:55.789 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.784+0000 7fa6037fe640 20 reqs_thread_entry: start 2024-05-30T16:20:55.789 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.784+0000 7fa7de543980 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-05-30T16:20:55.789 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.784+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.790 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.784+0000 7fa7de543980 20 rados_obj.operate() r=0 bl.length=213 2024-05-30T16:20:55.790 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.784+0000 7fa7de543980 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-05-30T16:20:55.790 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.784+0000 7fa7de543980 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-05-30T16:20:55.790 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.784+0000 7fa7de543980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 : miss 2024-05-30T16:20:55.790 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.784+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.791 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.788+0000 7fa7de543980 20 rados_obj.operate() r=0 bl.length=427 2024-05-30T16:20:55.791 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.788+0000 7fa7de543980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 info.flags=0x17 2024-05-30T16:20:55.791 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.788+0000 7fa7de543980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 to cache LRU end 2024-05-30T16:20:55.791 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.788+0000 7fa7de543980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:20:55.791 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.788+0000 7fa7de543980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:55.791 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.788+0000 7fa7de543980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 2024-05-30T16:20:55.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.800+0000 7fa7de543980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 info.flags=0x17 2024-05-30T16:20:55.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.800+0000 7fa7de543980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 to cache LRU end 2024-05-30T16:20:55.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.800+0000 7fa7de543980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:55.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.800+0000 7fa7de543980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24, ofs0, ns] 2024-05-30T16:20:55.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.800+0000 7fa6b2ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431625 cookie 94911882257936 notifier 6215 bl.length()=991 2024-05-30T16:20:55.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.800+0000 7fa6b2ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 info.flags=0x17 2024-05-30T16:20:55.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.800+0000 7fa6b2ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 to cache LRU end 2024-05-30T16:20:55.804 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.800+0000 7fa6b2ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:55.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.800+0000 7fa7de543980 20 handle_bi_update(): bucket=:abort-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24]): orig_sources= new_sources= 2024-05-30T16:20:55.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.800+0000 7fa7de543980 20 handle_bi_update(): bucket=:abort-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24]): potential sources added=[] removed=[] 2024-05-30T16:20:55.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.800+0000 7fa7de543980 20 handle_bi_update(): bucket=:abort-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24]): orig_dests= new_dests= 2024-05-30T16:20:55.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.800+0000 7fa7de543980 20 handle_bi_update(): bucket=:abort-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24]): potential dests added=[] removed=[] 2024-05-30T16:20:55.806 INFO:tasks.workunit.client.0.smithi006.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fa7de543980 time 2024-05-30T16:20:55.808517+0000 2024-05-30T16:20:55.806 INFO:tasks.workunit.client.0.smithi006.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-30T16:20:55.807 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:20:55.807 INFO:tasks.workunit.client.0.smithi006.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fa7e1e1acff] 2024-05-30T16:20:55.807 INFO:tasks.workunit.client.0.smithi006.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7fa7e1e1aeb3] 2024-05-30T16:20:55.807 INFO:tasks.workunit.client.0.smithi006.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x565262955f81] 2024-05-30T16:20:55.807 INFO:tasks.workunit.client.0.smithi006.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2ab) [0x565262967eab] 2024-05-30T16:20:55.807 INFO:tasks.workunit.client.0.smithi006.stderr: 5: main() 2024-05-30T16:20:55.807 INFO:tasks.workunit.client.0.smithi006.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fa7e0c83d90] 2024-05-30T16:20:55.807 INFO:tasks.workunit.client.0.smithi006.stderr: 7: __libc_start_main() 2024-05-30T16:20:55.807 INFO:tasks.workunit.client.0.smithi006.stderr: 8: _start() 2024-05-30T16:20:55.808 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.804+0000 7fa7de543980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fa7de543980 time 2024-05-30T16:20:55.808517+0000 2024-05-30T16:20:55.808 INFO:tasks.workunit.client.0.smithi006.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-30T16:20:55.808 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.808 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:20:55.808 INFO:tasks.workunit.client.0.smithi006.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fa7e1e1acff] 2024-05-30T16:20:55.808 INFO:tasks.workunit.client.0.smithi006.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7fa7e1e1aeb3] 2024-05-30T16:20:55.808 INFO:tasks.workunit.client.0.smithi006.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x565262955f81] 2024-05-30T16:20:55.808 INFO:tasks.workunit.client.0.smithi006.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2ab) [0x565262967eab] 2024-05-30T16:20:55.808 INFO:tasks.workunit.client.0.smithi006.stderr: 5: main() 2024-05-30T16:20:55.808 INFO:tasks.workunit.client.0.smithi006.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fa7e0c83d90] 2024-05-30T16:20:55.809 INFO:tasks.workunit.client.0.smithi006.stderr: 7: __libc_start_main() 2024-05-30T16:20:55.809 INFO:tasks.workunit.client.0.smithi006.stderr: 8: _start() 2024-05-30T16:20:55.809 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.809 INFO:tasks.workunit.client.0.smithi006.stderr:*** Caught signal (Aborted) ** 2024-05-30T16:20:55.809 INFO:tasks.workunit.client.0.smithi006.stderr: in thread 7fa7de543980 thread_name:radosgw-admin 2024-05-30T16:20:55.822 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:20:55.822 INFO:tasks.workunit.client.0.smithi006.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fa7e0c9c520] 2024-05-30T16:20:55.822 INFO:tasks.workunit.client.0.smithi006.stderr: 2: pthread_kill() 2024-05-30T16:20:55.822 INFO:tasks.workunit.client.0.smithi006.stderr: 3: raise() 2024-05-30T16:20:55.823 INFO:tasks.workunit.client.0.smithi006.stderr: 4: abort() 2024-05-30T16:20:55.823 INFO:tasks.workunit.client.0.smithi006.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fa7e1e1ad59] 2024-05-30T16:20:55.823 INFO:tasks.workunit.client.0.smithi006.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7fa7e1e1aeb3] 2024-05-30T16:20:55.823 INFO:tasks.workunit.client.0.smithi006.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x565262955f81] 2024-05-30T16:20:55.823 INFO:tasks.workunit.client.0.smithi006.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2ab) [0x565262967eab] 2024-05-30T16:20:55.823 INFO:tasks.workunit.client.0.smithi006.stderr: 9: main() 2024-05-30T16:20:55.823 INFO:tasks.workunit.client.0.smithi006.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fa7e0c83d90] 2024-05-30T16:20:55.823 INFO:tasks.workunit.client.0.smithi006.stderr: 11: __libc_start_main() 2024-05-30T16:20:55.823 INFO:tasks.workunit.client.0.smithi006.stderr: 12: _start() 2024-05-30T16:20:55.823 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.804+0000 7fa7de543980 -1 *** Caught signal (Aborted) ** 2024-05-30T16:20:55.823 INFO:tasks.workunit.client.0.smithi006.stderr: in thread 7fa7de543980 thread_name:radosgw-admin 2024-05-30T16:20:55.823 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.824 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:20:55.824 INFO:tasks.workunit.client.0.smithi006.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fa7e0c9c520] 2024-05-30T16:20:55.824 INFO:tasks.workunit.client.0.smithi006.stderr: 2: pthread_kill() 2024-05-30T16:20:55.824 INFO:tasks.workunit.client.0.smithi006.stderr: 3: raise() 2024-05-30T16:20:55.824 INFO:tasks.workunit.client.0.smithi006.stderr: 4: abort() 2024-05-30T16:20:55.824 INFO:tasks.workunit.client.0.smithi006.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fa7e1e1ad59] 2024-05-30T16:20:55.824 INFO:tasks.workunit.client.0.smithi006.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7fa7e1e1aeb3] 2024-05-30T16:20:55.824 INFO:tasks.workunit.client.0.smithi006.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x565262955f81] 2024-05-30T16:20:55.824 INFO:tasks.workunit.client.0.smithi006.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2ab) [0x565262967eab] 2024-05-30T16:20:55.824 INFO:tasks.workunit.client.0.smithi006.stderr: 9: main() 2024-05-30T16:20:55.824 INFO:tasks.workunit.client.0.smithi006.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fa7e0c83d90] 2024-05-30T16:20:55.824 INFO:tasks.workunit.client.0.smithi006.stderr: 11: __libc_start_main() 2024-05-30T16:20:55.824 INFO:tasks.workunit.client.0.smithi006.stderr: 12: _start() 2024-05-30T16:20:55.825 INFO:tasks.workunit.client.0.smithi006.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-30T16:20:55.825 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.825 INFO:tasks.workunit.client.0.smithi006.stderr:--- begin dump of recent events --- 2024-05-30T16:20:55.825 INFO:tasks.workunit.client.0.smithi006.stderr: -499> 2024-05-30T16:20:55.708+0000 7fa6ca60c640 10 monclient: handle_config config(0 keys) 2024-05-30T16:20:55.825 INFO:tasks.workunit.client.0.smithi006.stderr: -498> 2024-05-30T16:20:55.708+0000 7fa6ca60c640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:55.825 INFO:tasks.workunit.client.0.smithi006.stderr: -497> 2024-05-30T16:20:55.708+0000 7fa6ca60c640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-30T16:20:55.825 INFO:tasks.workunit.client.0.smithi006.stderr: -496> 2024-05-30T16:20:55.708+0000 7fa6ca60c640 10 monclient: dump: 2024-05-30T16:20:55.825 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:55.825 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:55.825 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.825 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.825 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:55.825 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:55.825 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:55.826 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:55.826 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:55.826 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.826 INFO:tasks.workunit.client.0.smithi006.stderr: -495> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 monclient: authenticate success, global_id 6203 2024-05-30T16:20:55.826 INFO:tasks.workunit.client.0.smithi006.stderr: -494> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient: get_monmap_and_config success 2024-05-30T16:20:55.826 INFO:tasks.workunit.client.0.smithi006.stderr: -493> 2024-05-30T16:20:55.708+0000 7fa7de543980 4 set_mon_vals no callback set 2024-05-30T16:20:55.826 INFO:tasks.workunit.client.0.smithi006.stderr: -492> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient: shutdown 2024-05-30T16:20:55.826 INFO:tasks.workunit.client.0.smithi006.stderr: -491> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 asok(0x565264d67790) unregister_commands rotate-key 2024-05-30T16:20:55.826 INFO:tasks.workunit.client.0.smithi006.stderr: -490> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient: build_initial_monmap 2024-05-30T16:20:55.826 INFO:tasks.workunit.client.0.smithi006.stderr: -489> 2024-05-30T16:20:55.708+0000 7fa7de543980 1 build_initial for_mkfs: 0 2024-05-30T16:20:55.827 INFO:tasks.workunit.client.0.smithi006.stderr: -488> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient: monmap: 2024-05-30T16:20:55.827 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 0 2024-05-30T16:20:55.827 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-30T16:20:55.827 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 0.000000 2024-05-30T16:20:55.827 INFO:tasks.workunit.client.0.smithi006.stderr:created 0.000000 2024-05-30T16:20:55.827 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 0 (unknown) 2024-05-30T16:20:55.827 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:55.827 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.noname-a 2024-05-30T16:20:55.827 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-b 2024-05-30T16:20:55.827 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.noname-c 2024-05-30T16:20:55.827 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.827 INFO:tasks.workunit.client.0.smithi006.stderr: -487> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f75ce0) adding auth protocol: cephx 2024-05-30T16:20:55.828 INFO:tasks.workunit.client.0.smithi006.stderr: -486> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f75ce0) adding auth protocol: cephx 2024-05-30T16:20:55.828 INFO:tasks.workunit.client.0.smithi006.stderr: -485> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f75ce0) adding auth protocol: cephx 2024-05-30T16:20:55.828 INFO:tasks.workunit.client.0.smithi006.stderr: -484> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f75ce0) adding con mode: secure 2024-05-30T16:20:55.828 INFO:tasks.workunit.client.0.smithi006.stderr: -483> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f75ce0) adding con mode: crc 2024-05-30T16:20:55.828 INFO:tasks.workunit.client.0.smithi006.stderr: -482> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f75ce0) adding con mode: secure 2024-05-30T16:20:55.828 INFO:tasks.workunit.client.0.smithi006.stderr: -481> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f75ce0) adding con mode: crc 2024-05-30T16:20:55.828 INFO:tasks.workunit.client.0.smithi006.stderr: -480> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f75ce0) adding con mode: secure 2024-05-30T16:20:55.828 INFO:tasks.workunit.client.0.smithi006.stderr: -479> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f75ce0) adding con mode: crc 2024-05-30T16:20:55.828 INFO:tasks.workunit.client.0.smithi006.stderr: -478> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f75ce0) adding con mode: crc 2024-05-30T16:20:55.828 INFO:tasks.workunit.client.0.smithi006.stderr: -477> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f75ce0) adding con mode: secure 2024-05-30T16:20:55.828 INFO:tasks.workunit.client.0.smithi006.stderr: -476> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f75ce0) adding con mode: crc 2024-05-30T16:20:55.828 INFO:tasks.workunit.client.0.smithi006.stderr: -475> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f75ce0) adding con mode: secure 2024-05-30T16:20:55.828 INFO:tasks.workunit.client.0.smithi006.stderr: -474> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f75ce0) adding con mode: crc 2024-05-30T16:20:55.829 INFO:tasks.workunit.client.0.smithi006.stderr: -473> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f75ce0) adding con mode: secure 2024-05-30T16:20:55.829 INFO:tasks.workunit.client.0.smithi006.stderr: -472> 2024-05-30T16:20:55.708+0000 7fa7de543980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:55.829 INFO:tasks.workunit.client.0.smithi006.stderr: -471> 2024-05-30T16:20:55.708+0000 7fa7de543980 1 librados: starting msgr at 2024-05-30T16:20:55.829 INFO:tasks.workunit.client.0.smithi006.stderr: -470> 2024-05-30T16:20:55.708+0000 7fa7de543980 1 librados: starting objecter 2024-05-30T16:20:55.829 INFO:tasks.workunit.client.0.smithi006.stderr: -469> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 asok(0x565264d67790) register_command objecter_requests hook 0x565264ce6360 2024-05-30T16:20:55.829 INFO:tasks.workunit.client.0.smithi006.stderr: -468> 2024-05-30T16:20:55.708+0000 7fa7de543980 1 librados: setting wanted keys 2024-05-30T16:20:55.829 INFO:tasks.workunit.client.0.smithi006.stderr: -467> 2024-05-30T16:20:55.708+0000 7fa7de543980 1 librados: calling monclient init 2024-05-30T16:20:55.829 INFO:tasks.workunit.client.0.smithi006.stderr: -466> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient: init 2024-05-30T16:20:55.829 INFO:tasks.workunit.client.0.smithi006.stderr: -465> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f72930) adding auth protocol: cephx 2024-05-30T16:20:55.829 INFO:tasks.workunit.client.0.smithi006.stderr: -464> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f72930) adding auth protocol: cephx 2024-05-30T16:20:55.829 INFO:tasks.workunit.client.0.smithi006.stderr: -463> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f72930) adding auth protocol: cephx 2024-05-30T16:20:55.830 INFO:tasks.workunit.client.0.smithi006.stderr: -462> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f72930) adding con mode: secure 2024-05-30T16:20:55.830 INFO:tasks.workunit.client.0.smithi006.stderr: -461> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f72930) adding con mode: crc 2024-05-30T16:20:55.830 INFO:tasks.workunit.client.0.smithi006.stderr: -460> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f72930) adding con mode: secure 2024-05-30T16:20:55.830 INFO:tasks.workunit.client.0.smithi006.stderr: -459> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f72930) adding con mode: crc 2024-05-30T16:20:55.830 INFO:tasks.workunit.client.0.smithi006.stderr: -458> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f72930) adding con mode: secure 2024-05-30T16:20:55.830 INFO:tasks.workunit.client.0.smithi006.stderr: -457> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f72930) adding con mode: crc 2024-05-30T16:20:55.830 INFO:tasks.workunit.client.0.smithi006.stderr: -456> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f72930) adding con mode: crc 2024-05-30T16:20:55.830 INFO:tasks.workunit.client.0.smithi006.stderr: -455> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f72930) adding con mode: secure 2024-05-30T16:20:55.830 INFO:tasks.workunit.client.0.smithi006.stderr: -454> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f72930) adding con mode: crc 2024-05-30T16:20:55.830 INFO:tasks.workunit.client.0.smithi006.stderr: -453> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f72930) adding con mode: secure 2024-05-30T16:20:55.830 INFO:tasks.workunit.client.0.smithi006.stderr: -452> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f72930) adding con mode: crc 2024-05-30T16:20:55.830 INFO:tasks.workunit.client.0.smithi006.stderr: -451> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f72930) adding con mode: secure 2024-05-30T16:20:55.831 INFO:tasks.workunit.client.0.smithi006.stderr: -450> 2024-05-30T16:20:55.708+0000 7fa7de543980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:55.831 INFO:tasks.workunit.client.0.smithi006.stderr: -449> 2024-05-30T16:20:55.708+0000 7fa7de543980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:55.831 INFO:tasks.workunit.client.0.smithi006.stderr: -448> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 asok(0x565264d67790) register_command rotate-key hook 0x565264f72a78 2024-05-30T16:20:55.831 INFO:tasks.workunit.client.0.smithi006.stderr: -447> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient: _reopen_session rank -1 2024-05-30T16:20:55.831 INFO:tasks.workunit.client.0.smithi006.stderr: -446> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient: _add_conns ranks=[1,0,2] 2024-05-30T16:20:55.831 INFO:tasks.workunit.client.0.smithi006.stderr: -445> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient(hunting): picked mon.noname-b con 0x565264e2d0b0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:20:55.831 INFO:tasks.workunit.client.0.smithi006.stderr: -444> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient(hunting): picked mon.noname-a con 0x565264f79580 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:20:55.831 INFO:tasks.workunit.client.0.smithi006.stderr: -443> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient(hunting): picked mon.noname-c con 0x565264fadb80 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:20:55.831 INFO:tasks.workunit.client.0.smithi006.stderr: -442> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:55.831 INFO:tasks.workunit.client.0.smithi006.stderr: -441> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:55.831 INFO:tasks.workunit.client.0.smithi006.stderr: -440> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:55.831 INFO:tasks.workunit.client.0.smithi006.stderr: -439> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient(hunting): _renew_subs 2024-05-30T16:20:55.832 INFO:tasks.workunit.client.0.smithi006.stderr: -438> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient(hunting): authenticate will time out at 2595.646644s 2024-05-30T16:20:55.832 INFO:tasks.workunit.client.0.smithi006.stderr: -437> 2024-05-30T16:20:55.708+0000 7fa7db853640 10 monclient(hunting): get_auth_request con 0x565264f79580 auth_method 0 2024-05-30T16:20:55.832 INFO:tasks.workunit.client.0.smithi006.stderr: -436> 2024-05-30T16:20:55.708+0000 7fa7db853640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:55.832 INFO:tasks.workunit.client.0.smithi006.stderr: -435> 2024-05-30T16:20:55.708+0000 7fa7db853640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:55.832 INFO:tasks.workunit.client.0.smithi006.stderr: -434> 2024-05-30T16:20:55.708+0000 7fa7db853640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:55.832 INFO:tasks.workunit.client.0.smithi006.stderr: -433> 2024-05-30T16:20:55.708+0000 7fa7db853640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:55.832 INFO:tasks.workunit.client.0.smithi006.stderr: -432> 2024-05-30T16:20:55.708+0000 7fa7db853640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:55.832 INFO:tasks.workunit.client.0.smithi006.stderr: -431> 2024-05-30T16:20:55.708+0000 7fa7db853640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:55.832 INFO:tasks.workunit.client.0.smithi006.stderr: -430> 2024-05-30T16:20:55.708+0000 7fa7dc855640 10 monclient(hunting): get_auth_request con 0x565264fadb80 auth_method 0 2024-05-30T16:20:55.832 INFO:tasks.workunit.client.0.smithi006.stderr: -429> 2024-05-30T16:20:55.708+0000 7fa7dc855640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:55.832 INFO:tasks.workunit.client.0.smithi006.stderr: -428> 2024-05-30T16:20:55.708+0000 7fa7dc855640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:55.832 INFO:tasks.workunit.client.0.smithi006.stderr: -427> 2024-05-30T16:20:55.708+0000 7fa7dc855640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:55.833 INFO:tasks.workunit.client.0.smithi006.stderr: -426> 2024-05-30T16:20:55.708+0000 7fa7dc054640 10 monclient(hunting): get_auth_request con 0x565264e2d0b0 auth_method 0 2024-05-30T16:20:55.833 INFO:tasks.workunit.client.0.smithi006.stderr: -425> 2024-05-30T16:20:55.708+0000 7fa7dc054640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:55.833 INFO:tasks.workunit.client.0.smithi006.stderr: -424> 2024-05-30T16:20:55.708+0000 7fa7dc054640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:55.833 INFO:tasks.workunit.client.0.smithi006.stderr: -423> 2024-05-30T16:20:55.708+0000 7fa7dc054640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:55.833 INFO:tasks.workunit.client.0.smithi006.stderr: -422> 2024-05-30T16:20:55.708+0000 7fa7dc855640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:55.833 INFO:tasks.workunit.client.0.smithi006.stderr: -421> 2024-05-30T16:20:55.708+0000 7fa7dc855640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:55.833 INFO:tasks.workunit.client.0.smithi006.stderr: -420> 2024-05-30T16:20:55.708+0000 7fa7dc855640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:55.833 INFO:tasks.workunit.client.0.smithi006.stderr: -419> 2024-05-30T16:20:55.708+0000 7fa7db853640 10 monclient(hunting): handle_auth_done global_id 6489 payload 824 2024-05-30T16:20:55.833 INFO:tasks.workunit.client.0.smithi006.stderr: -418> 2024-05-30T16:20:55.708+0000 7fa7db853640 10 monclient: _finish_hunting 0 2024-05-30T16:20:55.833 INFO:tasks.workunit.client.0.smithi006.stderr: -417> 2024-05-30T16:20:55.708+0000 7fa7db853640 1 monclient: found mon.noname-a 2024-05-30T16:20:55.833 INFO:tasks.workunit.client.0.smithi006.stderr: -416> 2024-05-30T16:20:55.708+0000 7fa7db853640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.6:3300/0 2024-05-30T16:20:55.833 INFO:tasks.workunit.client.0.smithi006.stderr: -415> 2024-05-30T16:20:55.708+0000 7fa6c8e09640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:55.833 INFO:tasks.workunit.client.0.smithi006.stderr: -414> 2024-05-30T16:20:55.708+0000 7fa6c8e09640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-30T16:20:55.833 INFO:tasks.workunit.client.0.smithi006.stderr: -413> 2024-05-30T16:20:55.708+0000 7fa6c8e09640 10 monclient: dump: 2024-05-30T16:20:55.834 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:55.834 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:55.834 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.834 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.834 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:55.834 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:55.834 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:55.834 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:55.834 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:55.834 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.834 INFO:tasks.workunit.client.0.smithi006.stderr: -412> 2024-05-30T16:20:55.708+0000 7fa6c8e09640 10 monclient: _finish_auth 0 2024-05-30T16:20:55.834 INFO:tasks.workunit.client.0.smithi006.stderr: -411> 2024-05-30T16:20:55.708+0000 7fa6c8e09640 10 monclient: _check_auth_tickets 2024-05-30T16:20:55.834 INFO:tasks.workunit.client.0.smithi006.stderr: -410> 2024-05-30T16:20:55.708+0000 7fa6c8e09640 10 monclient: handle_config config(0 keys) 2024-05-30T16:20:55.835 INFO:tasks.workunit.client.0.smithi006.stderr: -409> 2024-05-30T16:20:55.708+0000 7fa6c8e09640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:55.835 INFO:tasks.workunit.client.0.smithi006.stderr: -408> 2024-05-30T16:20:55.708+0000 7fa6c8e09640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-30T16:20:55.835 INFO:tasks.workunit.client.0.smithi006.stderr: -407> 2024-05-30T16:20:55.708+0000 7fa6c8e09640 10 monclient: dump: 2024-05-30T16:20:55.835 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:55.835 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:55.835 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.835 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.835 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:55.835 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:55.835 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:55.836 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:55.836 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:55.836 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.836 INFO:tasks.workunit.client.0.smithi006.stderr: -406> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 monclient: authenticate success, global_id 6489 2024-05-30T16:20:55.836 INFO:tasks.workunit.client.0.smithi006.stderr: -405> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient: _renew_subs 2024-05-30T16:20:55.836 INFO:tasks.workunit.client.0.smithi006.stderr: -404> 2024-05-30T16:20:55.708+0000 7fa6ca60c640 4 set_mon_vals no callback set 2024-05-30T16:20:55.836 INFO:tasks.workunit.client.0.smithi006.stderr: -403> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:20:55.836 INFO:tasks.workunit.client.0.smithi006.stderr: -402> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient: _renew_subs 2024-05-30T16:20:55.836 INFO:tasks.workunit.client.0.smithi006.stderr: -401> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:20:55.836 INFO:tasks.workunit.client.0.smithi006.stderr: -400> 2024-05-30T16:20:55.708+0000 7fa7de543980 1 librados: init done 2024-05-30T16:20:55.836 INFO:tasks.workunit.client.0.smithi006.stderr: -399> 2024-05-30T16:20:55.712+0000 7fa6c8e09640 4 mgrc handle_mgr_map Got map version 5 2024-05-30T16:20:55.836 INFO:tasks.workunit.client.0.smithi006.stderr: -398> 2024-05-30T16:20:55.712+0000 7fa6c8e09640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:20:55.837 INFO:tasks.workunit.client.0.smithi006.stderr: -397> 2024-05-30T16:20:55.712+0000 7fa6c8e09640 4 mgrc reconnect Starting new session with [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:20:55.837 INFO:tasks.workunit.client.0.smithi006.stderr: -396> 2024-05-30T16:20:55.712+0000 7fa7dc054640 10 monclient: get_auth_request con 0x7fa6c406eb40 auth_method 0 2024-05-30T16:20:55.837 INFO:tasks.workunit.client.0.smithi006.stderr: -395> 2024-05-30T16:20:55.712+0000 7fa7dc855640 10 monclient: get_auth_request con 0x565264fb3b60 auth_method 0 2024-05-30T16:20:55.837 INFO:tasks.workunit.client.0.smithi006.stderr: -394> 2024-05-30T16:20:55.716+0000 7fa7dc054640 10 monclient: get_auth_request con 0x565264fbd470 auth_method 0 2024-05-30T16:20:55.837 INFO:tasks.workunit.client.0.smithi006.stderr: -393> 2024-05-30T16:20:55.716+0000 7fa7db853640 10 monclient: get_auth_request con 0x565264fc2600 auth_method 0 2024-05-30T16:20:55.837 INFO:tasks.workunit.client.0.smithi006.stderr: -392> 2024-05-30T16:20:55.720+0000 7fa7de543980 10 monclient: build_initial_monmap 2024-05-30T16:20:55.838 INFO:tasks.workunit.client.0.smithi006.stderr: -391> 2024-05-30T16:20:55.720+0000 7fa7de543980 1 build_initial for_mkfs: 0 2024-05-30T16:20:55.838 INFO:tasks.workunit.client.0.smithi006.stderr: -390> 2024-05-30T16:20:55.720+0000 7fa7de543980 10 monclient: monmap: 2024-05-30T16:20:55.838 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 0 2024-05-30T16:20:55.838 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-30T16:20:55.838 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 0.000000 2024-05-30T16:20:55.838 INFO:tasks.workunit.client.0.smithi006.stderr:created 0.000000 2024-05-30T16:20:55.838 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 0 (unknown) 2024-05-30T16:20:55.838 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:55.838 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.noname-a 2024-05-30T16:20:55.838 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-b 2024-05-30T16:20:55.838 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.noname-c 2024-05-30T16:20:55.838 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.838 INFO:tasks.workunit.client.0.smithi006.stderr: -389> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc6730) adding auth protocol: cephx 2024-05-30T16:20:55.839 INFO:tasks.workunit.client.0.smithi006.stderr: -388> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc6730) adding auth protocol: cephx 2024-05-30T16:20:55.839 INFO:tasks.workunit.client.0.smithi006.stderr: -387> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc6730) adding auth protocol: cephx 2024-05-30T16:20:55.839 INFO:tasks.workunit.client.0.smithi006.stderr: -386> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc6730) adding con mode: secure 2024-05-30T16:20:55.839 INFO:tasks.workunit.client.0.smithi006.stderr: -385> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc6730) adding con mode: crc 2024-05-30T16:20:55.839 INFO:tasks.workunit.client.0.smithi006.stderr: -384> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc6730) adding con mode: secure 2024-05-30T16:20:55.839 INFO:tasks.workunit.client.0.smithi006.stderr: -383> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc6730) adding con mode: crc 2024-05-30T16:20:55.839 INFO:tasks.workunit.client.0.smithi006.stderr: -382> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc6730) adding con mode: secure 2024-05-30T16:20:55.839 INFO:tasks.workunit.client.0.smithi006.stderr: -381> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc6730) adding con mode: crc 2024-05-30T16:20:55.840 INFO:tasks.workunit.client.0.smithi006.stderr: -380> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc6730) adding con mode: crc 2024-05-30T16:20:55.840 INFO:tasks.workunit.client.0.smithi006.stderr: -379> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc6730) adding con mode: secure 2024-05-30T16:20:55.840 INFO:tasks.workunit.client.0.smithi006.stderr: -378> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc6730) adding con mode: crc 2024-05-30T16:20:55.840 INFO:tasks.workunit.client.0.smithi006.stderr: -377> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc6730) adding con mode: secure 2024-05-30T16:20:55.840 INFO:tasks.workunit.client.0.smithi006.stderr: -376> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc6730) adding con mode: crc 2024-05-30T16:20:55.840 INFO:tasks.workunit.client.0.smithi006.stderr: -375> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc6730) adding con mode: secure 2024-05-30T16:20:55.840 INFO:tasks.workunit.client.0.smithi006.stderr: -374> 2024-05-30T16:20:55.720+0000 7fa7de543980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:55.840 INFO:tasks.workunit.client.0.smithi006.stderr: -373> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 asok(0x565264d67790) register_command objecter_requests cmddesc objecter_requests hook 0x565264fc9c60 EEXIST 2024-05-30T16:20:55.840 INFO:tasks.workunit.client.0.smithi006.stderr: -372> 2024-05-30T16:20:55.720+0000 7fa7de543980 10 monclient: init 2024-05-30T16:20:55.840 INFO:tasks.workunit.client.0.smithi006.stderr: -371> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc5628) adding auth protocol: cephx 2024-05-30T16:20:55.841 INFO:tasks.workunit.client.0.smithi006.stderr: -370> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc5628) adding auth protocol: cephx 2024-05-30T16:20:55.841 INFO:tasks.workunit.client.0.smithi006.stderr: -369> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc5628) adding auth protocol: cephx 2024-05-30T16:20:55.841 INFO:tasks.workunit.client.0.smithi006.stderr: -368> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc5628) adding con mode: secure 2024-05-30T16:20:55.841 INFO:tasks.workunit.client.0.smithi006.stderr: -367> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc5628) adding con mode: crc 2024-05-30T16:20:55.841 INFO:tasks.workunit.client.0.smithi006.stderr: -366> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc5628) adding con mode: secure 2024-05-30T16:20:55.841 INFO:tasks.workunit.client.0.smithi006.stderr: -365> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc5628) adding con mode: crc 2024-05-30T16:20:55.841 INFO:tasks.workunit.client.0.smithi006.stderr: -364> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc5628) adding con mode: secure 2024-05-30T16:20:55.841 INFO:tasks.workunit.client.0.smithi006.stderr: -363> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc5628) adding con mode: crc 2024-05-30T16:20:55.841 INFO:tasks.workunit.client.0.smithi006.stderr: -362> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc5628) adding con mode: crc 2024-05-30T16:20:55.841 INFO:tasks.workunit.client.0.smithi006.stderr: -361> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc5628) adding con mode: secure 2024-05-30T16:20:55.842 INFO:tasks.workunit.client.0.smithi006.stderr: -360> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc5628) adding con mode: crc 2024-05-30T16:20:55.842 INFO:tasks.workunit.client.0.smithi006.stderr: -359> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc5628) adding con mode: secure 2024-05-30T16:20:55.842 INFO:tasks.workunit.client.0.smithi006.stderr: -358> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc5628) adding con mode: crc 2024-05-30T16:20:55.842 INFO:tasks.workunit.client.0.smithi006.stderr: -357> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc5628) adding con mode: secure 2024-05-30T16:20:55.842 INFO:tasks.workunit.client.0.smithi006.stderr: -356> 2024-05-30T16:20:55.720+0000 7fa7de543980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:55.842 INFO:tasks.workunit.client.0.smithi006.stderr: -355> 2024-05-30T16:20:55.720+0000 7fa7de543980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:55.842 INFO:tasks.workunit.client.0.smithi006.stderr: -354> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 asok(0x565264d67790) register_command rotate-key cmddesc rotate-key hook 0x565264fc5770 EEXIST 2024-05-30T16:20:55.842 INFO:tasks.workunit.client.0.smithi006.stderr: -353> 2024-05-30T16:20:55.720+0000 7fa7de543980 10 monclient: _reopen_session rank -1 2024-05-30T16:20:55.842 INFO:tasks.workunit.client.0.smithi006.stderr: -352> 2024-05-30T16:20:55.720+0000 7fa7de543980 10 monclient: _add_conns ranks=[2,0,1] 2024-05-30T16:20:55.842 INFO:tasks.workunit.client.0.smithi006.stderr: -351> 2024-05-30T16:20:55.720+0000 7fa7de543980 10 monclient(hunting): picked mon.noname-c con 0x565264fd01f0 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:20:55.842 INFO:tasks.workunit.client.0.smithi006.stderr: -350> 2024-05-30T16:20:55.720+0000 7fa7de543980 10 monclient(hunting): picked mon.noname-a con 0x565264fd0b20 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:20:55.842 INFO:tasks.workunit.client.0.smithi006.stderr: -349> 2024-05-30T16:20:55.720+0000 7fa7de543980 10 monclient(hunting): picked mon.noname-b con 0x565264fd05d0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:20:55.843 INFO:tasks.workunit.client.0.smithi006.stderr: -348> 2024-05-30T16:20:55.720+0000 7fa7de543980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:55.843 INFO:tasks.workunit.client.0.smithi006.stderr: -347> 2024-05-30T16:20:55.720+0000 7fa7de543980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:55.843 INFO:tasks.workunit.client.0.smithi006.stderr: -346> 2024-05-30T16:20:55.720+0000 7fa7de543980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:55.843 INFO:tasks.workunit.client.0.smithi006.stderr: -345> 2024-05-30T16:20:55.720+0000 7fa7de543980 10 monclient(hunting): _renew_subs 2024-05-30T16:20:55.843 INFO:tasks.workunit.client.0.smithi006.stderr: -344> 2024-05-30T16:20:55.720+0000 7fa7de543980 10 monclient(hunting): authenticate will time out at 2595.659499s 2024-05-30T16:20:55.843 INFO:tasks.workunit.client.0.smithi006.stderr: -343> 2024-05-30T16:20:55.720+0000 7fa7dc054640 10 monclient(hunting): get_auth_request con 0x565264fd01f0 auth_method 0 2024-05-30T16:20:55.843 INFO:tasks.workunit.client.0.smithi006.stderr: -342> 2024-05-30T16:20:55.720+0000 7fa7dc054640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:55.843 INFO:tasks.workunit.client.0.smithi006.stderr: -341> 2024-05-30T16:20:55.720+0000 7fa7dc054640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:55.843 INFO:tasks.workunit.client.0.smithi006.stderr: -340> 2024-05-30T16:20:55.720+0000 7fa7dc054640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:55.843 INFO:tasks.workunit.client.0.smithi006.stderr: -339> 2024-05-30T16:20:55.720+0000 7fa7db853640 10 monclient(hunting): get_auth_request con 0x565264fd0b20 auth_method 0 2024-05-30T16:20:55.844 INFO:tasks.workunit.client.0.smithi006.stderr: -338> 2024-05-30T16:20:55.720+0000 7fa7db853640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:55.844 INFO:tasks.workunit.client.0.smithi006.stderr: -337> 2024-05-30T16:20:55.720+0000 7fa7db853640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:55.844 INFO:tasks.workunit.client.0.smithi006.stderr: -336> 2024-05-30T16:20:55.720+0000 7fa7db853640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:55.844 INFO:tasks.workunit.client.0.smithi006.stderr: -335> 2024-05-30T16:20:55.720+0000 7fa7dc054640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:55.844 INFO:tasks.workunit.client.0.smithi006.stderr: -334> 2024-05-30T16:20:55.720+0000 7fa7dc054640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:55.844 INFO:tasks.workunit.client.0.smithi006.stderr: -333> 2024-05-30T16:20:55.720+0000 7fa7dc054640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:55.844 INFO:tasks.workunit.client.0.smithi006.stderr: -332> 2024-05-30T16:20:55.720+0000 7fa7db853640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:55.844 INFO:tasks.workunit.client.0.smithi006.stderr: -331> 2024-05-30T16:20:55.720+0000 7fa7db853640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:55.844 INFO:tasks.workunit.client.0.smithi006.stderr: -330> 2024-05-30T16:20:55.720+0000 7fa7db853640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:55.844 INFO:tasks.workunit.client.0.smithi006.stderr: -329> 2024-05-30T16:20:55.724+0000 7fa7dc855640 10 monclient(hunting): get_auth_request con 0x565264fd05d0 auth_method 0 2024-05-30T16:20:55.844 INFO:tasks.workunit.client.0.smithi006.stderr: -328> 2024-05-30T16:20:55.724+0000 7fa7dc855640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:55.845 INFO:tasks.workunit.client.0.smithi006.stderr: -327> 2024-05-30T16:20:55.724+0000 7fa7dc855640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:55.845 INFO:tasks.workunit.client.0.smithi006.stderr: -326> 2024-05-30T16:20:55.724+0000 7fa7dc855640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:55.845 INFO:tasks.workunit.client.0.smithi006.stderr: -325> 2024-05-30T16:20:55.724+0000 7fa7db853640 10 monclient(hunting): handle_auth_done global_id 6492 payload 824 2024-05-30T16:20:55.845 INFO:tasks.workunit.client.0.smithi006.stderr: -324> 2024-05-30T16:20:55.724+0000 7fa7db853640 10 monclient: _finish_hunting 0 2024-05-30T16:20:55.845 INFO:tasks.workunit.client.0.smithi006.stderr: -323> 2024-05-30T16:20:55.724+0000 7fa7db853640 1 monclient: found mon.noname-a 2024-05-30T16:20:55.845 INFO:tasks.workunit.client.0.smithi006.stderr: -322> 2024-05-30T16:20:55.724+0000 7fa7db853640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.6:3300/0 2024-05-30T16:20:55.845 INFO:tasks.workunit.client.0.smithi006.stderr: -321> 2024-05-30T16:20:55.724+0000 7fa6c1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:55.845 INFO:tasks.workunit.client.0.smithi006.stderr: -320> 2024-05-30T16:20:55.724+0000 7fa6c1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-30T16:20:55.845 INFO:tasks.workunit.client.0.smithi006.stderr: -319> 2024-05-30T16:20:55.724+0000 7fa6c1ffb640 10 monclient: dump: 2024-05-30T16:20:55.846 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:55.846 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:55.846 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.846 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.846 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:55.846 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:55.846 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:55.846 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:55.846 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:55.847 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.847 INFO:tasks.workunit.client.0.smithi006.stderr: -318> 2024-05-30T16:20:55.724+0000 7fa6c1ffb640 10 monclient: _finish_auth 0 2024-05-30T16:20:55.847 INFO:tasks.workunit.client.0.smithi006.stderr: -317> 2024-05-30T16:20:55.724+0000 7fa6c1ffb640 10 monclient: _check_auth_tickets 2024-05-30T16:20:55.847 INFO:tasks.workunit.client.0.smithi006.stderr: -316> 2024-05-30T16:20:55.724+0000 7fa6c1ffb640 10 monclient: handle_config config(0 keys) 2024-05-30T16:20:55.847 INFO:tasks.workunit.client.0.smithi006.stderr: -315> 2024-05-30T16:20:55.724+0000 7fa6c1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:55.847 INFO:tasks.workunit.client.0.smithi006.stderr: -314> 2024-05-30T16:20:55.724+0000 7fa6c1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-30T16:20:55.847 INFO:tasks.workunit.client.0.smithi006.stderr: -313> 2024-05-30T16:20:55.724+0000 7fa6c1ffb640 10 monclient: dump: 2024-05-30T16:20:55.847 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:55.848 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:55.848 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.848 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.848 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:55.848 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:55.848 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:55.848 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:55.848 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:55.848 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.849 INFO:tasks.workunit.client.0.smithi006.stderr: -312> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 monclient: authenticate success, global_id 6492 2024-05-30T16:20:55.849 INFO:tasks.workunit.client.0.smithi006.stderr: -311> 2024-05-30T16:20:55.724+0000 7fa7dd056640 4 set_mon_vals no callback set 2024-05-30T16:20:55.849 INFO:tasks.workunit.client.0.smithi006.stderr: -310> 2024-05-30T16:20:55.724+0000 7fa7de543980 10 monclient: _renew_subs 2024-05-30T16:20:55.849 INFO:tasks.workunit.client.0.smithi006.stderr: -309> 2024-05-30T16:20:55.724+0000 7fa7de543980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:20:55.849 INFO:tasks.workunit.client.0.smithi006.stderr: -308> 2024-05-30T16:20:55.724+0000 7fa7de543980 10 monclient: _renew_subs 2024-05-30T16:20:55.849 INFO:tasks.workunit.client.0.smithi006.stderr: -307> 2024-05-30T16:20:55.724+0000 7fa7de543980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:20:55.849 INFO:tasks.workunit.client.0.smithi006.stderr: -306> 2024-05-30T16:20:55.724+0000 7fa6c1ffb640 4 mgrc handle_mgr_map Got map version 5 2024-05-30T16:20:55.849 INFO:tasks.workunit.client.0.smithi006.stderr: -305> 2024-05-30T16:20:55.724+0000 7fa6c1ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:20:55.849 INFO:tasks.workunit.client.0.smithi006.stderr: -304> 2024-05-30T16:20:55.724+0000 7fa6c1ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:20:55.849 INFO:tasks.workunit.client.0.smithi006.stderr: -303> 2024-05-30T16:20:55.724+0000 7fa7de543980 10 monclient: get_monmap_and_config 2024-05-30T16:20:55.849 INFO:tasks.workunit.client.0.smithi006.stderr: -302> 2024-05-30T16:20:55.724+0000 7fa7de543980 10 monclient: build_initial_monmap 2024-05-30T16:20:55.849 INFO:tasks.workunit.client.0.smithi006.stderr: -301> 2024-05-30T16:20:55.724+0000 7fa7de543980 1 build_initial for_mkfs: 0 2024-05-30T16:20:55.849 INFO:tasks.workunit.client.0.smithi006.stderr: -300> 2024-05-30T16:20:55.724+0000 7fa7de543980 10 monclient: monmap: 2024-05-30T16:20:55.850 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 0 2024-05-30T16:20:55.850 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-30T16:20:55.850 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 0.000000 2024-05-30T16:20:55.850 INFO:tasks.workunit.client.0.smithi006.stderr:created 0.000000 2024-05-30T16:20:55.850 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 0 (unknown) 2024-05-30T16:20:55.850 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:55.850 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.noname-a 2024-05-30T16:20:55.850 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-b 2024-05-30T16:20:55.850 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.noname-c 2024-05-30T16:20:55.850 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.850 INFO:tasks.workunit.client.0.smithi006.stderr: -299> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x56526500dd30) adding auth protocol: cephx 2024-05-30T16:20:55.850 INFO:tasks.workunit.client.0.smithi006.stderr: -298> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x56526500dd30) adding auth protocol: cephx 2024-05-30T16:20:55.850 INFO:tasks.workunit.client.0.smithi006.stderr: -297> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x56526500dd30) adding auth protocol: cephx 2024-05-30T16:20:55.850 INFO:tasks.workunit.client.0.smithi006.stderr: -296> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x56526500dd30) adding con mode: secure 2024-05-30T16:20:55.851 INFO:tasks.workunit.client.0.smithi006.stderr: -295> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x56526500dd30) adding con mode: crc 2024-05-30T16:20:55.851 INFO:tasks.workunit.client.0.smithi006.stderr: -294> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x56526500dd30) adding con mode: secure 2024-05-30T16:20:55.851 INFO:tasks.workunit.client.0.smithi006.stderr: -293> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x56526500dd30) adding con mode: crc 2024-05-30T16:20:55.851 INFO:tasks.workunit.client.0.smithi006.stderr: -292> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x56526500dd30) adding con mode: secure 2024-05-30T16:20:55.851 INFO:tasks.workunit.client.0.smithi006.stderr: -291> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x56526500dd30) adding con mode: crc 2024-05-30T16:20:55.851 INFO:tasks.workunit.client.0.smithi006.stderr: -290> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x56526500dd30) adding con mode: crc 2024-05-30T16:20:55.851 INFO:tasks.workunit.client.0.smithi006.stderr: -289> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x56526500dd30) adding con mode: secure 2024-05-30T16:20:55.851 INFO:tasks.workunit.client.0.smithi006.stderr: -288> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x56526500dd30) adding con mode: crc 2024-05-30T16:20:55.851 INFO:tasks.workunit.client.0.smithi006.stderr: -287> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x56526500dd30) adding con mode: secure 2024-05-30T16:20:55.851 INFO:tasks.workunit.client.0.smithi006.stderr: -286> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x56526500dd30) adding con mode: crc 2024-05-30T16:20:55.851 INFO:tasks.workunit.client.0.smithi006.stderr: -285> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x56526500dd30) adding con mode: secure 2024-05-30T16:20:55.851 INFO:tasks.workunit.client.0.smithi006.stderr: -284> 2024-05-30T16:20:55.724+0000 7fa7dc054640 10 monclient: get_auth_request con 0x7fa6b406dcf0 auth_method 0 2024-05-30T16:20:55.851 INFO:tasks.workunit.client.0.smithi006.stderr: -283> 2024-05-30T16:20:55.724+0000 7fa7de543980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:55.852 INFO:tasks.workunit.client.0.smithi006.stderr: -282> 2024-05-30T16:20:55.724+0000 7fa7de543980 10 monclient: init 2024-05-30T16:20:55.852 INFO:tasks.workunit.client.0.smithi006.stderr: -281> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c1da0) adding auth protocol: cephx 2024-05-30T16:20:55.852 INFO:tasks.workunit.client.0.smithi006.stderr: -280> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c1da0) adding auth protocol: cephx 2024-05-30T16:20:55.852 INFO:tasks.workunit.client.0.smithi006.stderr: -279> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c1da0) adding auth protocol: cephx 2024-05-30T16:20:55.852 INFO:tasks.workunit.client.0.smithi006.stderr: -278> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c1da0) adding con mode: secure 2024-05-30T16:20:55.852 INFO:tasks.workunit.client.0.smithi006.stderr: -277> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c1da0) adding con mode: crc 2024-05-30T16:20:55.852 INFO:tasks.workunit.client.0.smithi006.stderr: -276> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c1da0) adding con mode: secure 2024-05-30T16:20:55.852 INFO:tasks.workunit.client.0.smithi006.stderr: -275> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c1da0) adding con mode: crc 2024-05-30T16:20:55.852 INFO:tasks.workunit.client.0.smithi006.stderr: -274> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c1da0) adding con mode: secure 2024-05-30T16:20:55.852 INFO:tasks.workunit.client.0.smithi006.stderr: -273> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c1da0) adding con mode: crc 2024-05-30T16:20:55.852 INFO:tasks.workunit.client.0.smithi006.stderr: -272> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c1da0) adding con mode: crc 2024-05-30T16:20:55.852 INFO:tasks.workunit.client.0.smithi006.stderr: -271> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c1da0) adding con mode: secure 2024-05-30T16:20:55.852 INFO:tasks.workunit.client.0.smithi006.stderr: -270> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c1da0) adding con mode: crc 2024-05-30T16:20:55.852 INFO:tasks.workunit.client.0.smithi006.stderr: -269> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c1da0) adding con mode: secure 2024-05-30T16:20:55.853 INFO:tasks.workunit.client.0.smithi006.stderr: -268> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c1da0) adding con mode: crc 2024-05-30T16:20:55.853 INFO:tasks.workunit.client.0.smithi006.stderr: -267> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c1da0) adding con mode: secure 2024-05-30T16:20:55.853 INFO:tasks.workunit.client.0.smithi006.stderr: -266> 2024-05-30T16:20:55.724+0000 7fa7de543980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:55.853 INFO:tasks.workunit.client.0.smithi006.stderr: -265> 2024-05-30T16:20:55.728+0000 7fa7de543980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:55.853 INFO:tasks.workunit.client.0.smithi006.stderr: -264> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 asok(0x565264d67790) register_command rotate-key cmddesc rotate-key hook 0x7ffef71c1ee8 EEXIST 2024-05-30T16:20:55.853 INFO:tasks.workunit.client.0.smithi006.stderr: -263> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient: _reopen_session rank -1 2024-05-30T16:20:55.853 INFO:tasks.workunit.client.0.smithi006.stderr: -262> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient: _add_conns ranks=[2,0,1] 2024-05-30T16:20:55.853 INFO:tasks.workunit.client.0.smithi006.stderr: -261> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): picked mon.noname-c con 0x5652650421c0 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:20:55.853 INFO:tasks.workunit.client.0.smithi006.stderr: -260> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): picked mon.noname-a con 0x56526503ded0 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:20:55.853 INFO:tasks.workunit.client.0.smithi006.stderr: -259> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): picked mon.noname-b con 0x56526503c790 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:20:55.853 INFO:tasks.workunit.client.0.smithi006.stderr: -258> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:55.853 INFO:tasks.workunit.client.0.smithi006.stderr: -257> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:55.853 INFO:tasks.workunit.client.0.smithi006.stderr: -256> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:55.854 INFO:tasks.workunit.client.0.smithi006.stderr: -255> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): _renew_subs 2024-05-30T16:20:55.854 INFO:tasks.workunit.client.0.smithi006.stderr: -254> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): authenticate will time out at 2595.664196s 2024-05-30T16:20:55.854 INFO:tasks.workunit.client.0.smithi006.stderr: -253> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): get_auth_request con 0x5652650421c0 auth_method 0 2024-05-30T16:20:55.854 INFO:tasks.workunit.client.0.smithi006.stderr: -252> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:55.854 INFO:tasks.workunit.client.0.smithi006.stderr: -251> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:55.854 INFO:tasks.workunit.client.0.smithi006.stderr: -250> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:55.854 INFO:tasks.workunit.client.0.smithi006.stderr: -249> 2024-05-30T16:20:55.728+0000 7fa7db853640 10 monclient(hunting): get_auth_request con 0x56526503ded0 auth_method 0 2024-05-30T16:20:55.854 INFO:tasks.workunit.client.0.smithi006.stderr: -248> 2024-05-30T16:20:55.728+0000 7fa7db853640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:55.854 INFO:tasks.workunit.client.0.smithi006.stderr: -247> 2024-05-30T16:20:55.728+0000 7fa7db853640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:55.854 INFO:tasks.workunit.client.0.smithi006.stderr: -246> 2024-05-30T16:20:55.728+0000 7fa7db853640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:55.854 INFO:tasks.workunit.client.0.smithi006.stderr: -245> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:55.854 INFO:tasks.workunit.client.0.smithi006.stderr: -244> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:55.854 INFO:tasks.workunit.client.0.smithi006.stderr: -243> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:55.855 INFO:tasks.workunit.client.0.smithi006.stderr: -242> 2024-05-30T16:20:55.728+0000 7fa7db853640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:55.855 INFO:tasks.workunit.client.0.smithi006.stderr: -241> 2024-05-30T16:20:55.728+0000 7fa7db853640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:55.855 INFO:tasks.workunit.client.0.smithi006.stderr: -240> 2024-05-30T16:20:55.728+0000 7fa7db853640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:55.855 INFO:tasks.workunit.client.0.smithi006.stderr: -239> 2024-05-30T16:20:55.728+0000 7fa7dc855640 10 monclient(hunting): get_auth_request con 0x56526503c790 auth_method 0 2024-05-30T16:20:55.855 INFO:tasks.workunit.client.0.smithi006.stderr: -238> 2024-05-30T16:20:55.728+0000 7fa7dc855640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:55.855 INFO:tasks.workunit.client.0.smithi006.stderr: -237> 2024-05-30T16:20:55.728+0000 7fa7dc855640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:55.855 INFO:tasks.workunit.client.0.smithi006.stderr: -236> 2024-05-30T16:20:55.728+0000 7fa7dc855640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:55.855 INFO:tasks.workunit.client.0.smithi006.stderr: -235> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): handle_auth_done global_id 6212 payload 274 2024-05-30T16:20:55.855 INFO:tasks.workunit.client.0.smithi006.stderr: -234> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient: _finish_hunting 0 2024-05-30T16:20:55.855 INFO:tasks.workunit.client.0.smithi006.stderr: -233> 2024-05-30T16:20:55.728+0000 7fa7dc054640 1 monclient: found mon.noname-c 2024-05-30T16:20:55.855 INFO:tasks.workunit.client.0.smithi006.stderr: -232> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.6:3301/0 2024-05-30T16:20:55.855 INFO:tasks.workunit.client.0.smithi006.stderr: -231> 2024-05-30T16:20:55.728+0000 7fa6b37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:55.855 INFO:tasks.workunit.client.0.smithi006.stderr: -230> 2024-05-30T16:20:55.728+0000 7fa6b37fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-30T16:20:55.856 INFO:tasks.workunit.client.0.smithi006.stderr: -229> 2024-05-30T16:20:55.728+0000 7fa6b37fe640 10 monclient: dump: 2024-05-30T16:20:55.856 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:55.856 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:55.856 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.856 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.856 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:55.856 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:55.856 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:55.856 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:55.856 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:55.856 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.856 INFO:tasks.workunit.client.0.smithi006.stderr: -228> 2024-05-30T16:20:55.728+0000 7fa6b37fe640 10 monclient: _finish_auth 0 2024-05-30T16:20:55.856 INFO:tasks.workunit.client.0.smithi006.stderr: -227> 2024-05-30T16:20:55.728+0000 7fa6b37fe640 10 monclient: _check_auth_tickets 2024-05-30T16:20:55.857 INFO:tasks.workunit.client.0.smithi006.stderr: -226> 2024-05-30T16:20:55.728+0000 7fa6b37fe640 10 monclient: handle_config config(0 keys) 2024-05-30T16:20:55.857 INFO:tasks.workunit.client.0.smithi006.stderr: -225> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 monclient: authenticate success, global_id 6212 2024-05-30T16:20:55.857 INFO:tasks.workunit.client.0.smithi006.stderr: -224> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient: get_monmap_and_config success 2024-05-30T16:20:55.857 INFO:tasks.workunit.client.0.smithi006.stderr: -223> 2024-05-30T16:20:55.728+0000 7fa7de543980 4 set_mon_vals no callback set 2024-05-30T16:20:55.857 INFO:tasks.workunit.client.0.smithi006.stderr: -222> 2024-05-30T16:20:55.728+0000 7fa6b37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:55.857 INFO:tasks.workunit.client.0.smithi006.stderr: -221> 2024-05-30T16:20:55.728+0000 7fa6b37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-30T16:20:55.857 INFO:tasks.workunit.client.0.smithi006.stderr: -220> 2024-05-30T16:20:55.728+0000 7fa6b37fe640 10 monclient: dump: 2024-05-30T16:20:55.857 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:55.857 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:55.857 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.857 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.857 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:55.857 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:55.857 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:55.858 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:55.858 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:55.858 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.858 INFO:tasks.workunit.client.0.smithi006.stderr: -219> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient: shutdown 2024-05-30T16:20:55.858 INFO:tasks.workunit.client.0.smithi006.stderr: -218> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient: build_initial_monmap 2024-05-30T16:20:55.858 INFO:tasks.workunit.client.0.smithi006.stderr: -217> 2024-05-30T16:20:55.728+0000 7fa7de543980 1 build_initial for_mkfs: 0 2024-05-30T16:20:55.858 INFO:tasks.workunit.client.0.smithi006.stderr: -216> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient: monmap: 2024-05-30T16:20:55.858 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 0 2024-05-30T16:20:55.858 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-30T16:20:55.858 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 0.000000 2024-05-30T16:20:55.859 INFO:tasks.workunit.client.0.smithi006.stderr:created 0.000000 2024-05-30T16:20:55.859 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 0 (unknown) 2024-05-30T16:20:55.859 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:55.859 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.noname-a 2024-05-30T16:20:55.859 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-b 2024-05-30T16:20:55.859 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.noname-c 2024-05-30T16:20:55.859 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.859 INFO:tasks.workunit.client.0.smithi006.stderr: -215> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x56526500dce0) adding auth protocol: cephx 2024-05-30T16:20:55.859 INFO:tasks.workunit.client.0.smithi006.stderr: -214> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x56526500dce0) adding auth protocol: cephx 2024-05-30T16:20:55.859 INFO:tasks.workunit.client.0.smithi006.stderr: -213> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x56526500dce0) adding auth protocol: cephx 2024-05-30T16:20:55.859 INFO:tasks.workunit.client.0.smithi006.stderr: -212> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x56526500dce0) adding con mode: secure 2024-05-30T16:20:55.859 INFO:tasks.workunit.client.0.smithi006.stderr: -211> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x56526500dce0) adding con mode: crc 2024-05-30T16:20:55.860 INFO:tasks.workunit.client.0.smithi006.stderr: -210> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x56526500dce0) adding con mode: secure 2024-05-30T16:20:55.860 INFO:tasks.workunit.client.0.smithi006.stderr: -209> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x56526500dce0) adding con mode: crc 2024-05-30T16:20:55.860 INFO:tasks.workunit.client.0.smithi006.stderr: -208> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x56526500dce0) adding con mode: secure 2024-05-30T16:20:55.860 INFO:tasks.workunit.client.0.smithi006.stderr: -207> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x56526500dce0) adding con mode: crc 2024-05-30T16:20:55.860 INFO:tasks.workunit.client.0.smithi006.stderr: -206> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x56526500dce0) adding con mode: crc 2024-05-30T16:20:55.860 INFO:tasks.workunit.client.0.smithi006.stderr: -205> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x56526500dce0) adding con mode: secure 2024-05-30T16:20:55.860 INFO:tasks.workunit.client.0.smithi006.stderr: -204> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x56526500dce0) adding con mode: crc 2024-05-30T16:20:55.860 INFO:tasks.workunit.client.0.smithi006.stderr: -203> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x56526500dce0) adding con mode: secure 2024-05-30T16:20:55.860 INFO:tasks.workunit.client.0.smithi006.stderr: -202> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x56526500dce0) adding con mode: crc 2024-05-30T16:20:55.860 INFO:tasks.workunit.client.0.smithi006.stderr: -201> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x56526500dce0) adding con mode: secure 2024-05-30T16:20:55.860 INFO:tasks.workunit.client.0.smithi006.stderr: -200> 2024-05-30T16:20:55.728+0000 7fa7de543980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:55.860 INFO:tasks.workunit.client.0.smithi006.stderr: -199> 2024-05-30T16:20:55.728+0000 7fa7de543980 1 librados: starting msgr at 2024-05-30T16:20:55.860 INFO:tasks.workunit.client.0.smithi006.stderr: -198> 2024-05-30T16:20:55.728+0000 7fa7de543980 1 librados: starting objecter 2024-05-30T16:20:55.860 INFO:tasks.workunit.client.0.smithi006.stderr: -197> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 asok(0x565264d67790) register_command objecter_requests cmddesc objecter_requests hook 0x565265012cb0 EEXIST 2024-05-30T16:20:55.861 INFO:tasks.workunit.client.0.smithi006.stderr: -196> 2024-05-30T16:20:55.728+0000 7fa7de543980 1 librados: setting wanted keys 2024-05-30T16:20:55.861 INFO:tasks.workunit.client.0.smithi006.stderr: -195> 2024-05-30T16:20:55.728+0000 7fa7de543980 1 librados: calling monclient init 2024-05-30T16:20:55.861 INFO:tasks.workunit.client.0.smithi006.stderr: -194> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient: init 2024-05-30T16:20:55.861 INFO:tasks.workunit.client.0.smithi006.stderr: -193> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x565265005c50) adding auth protocol: cephx 2024-05-30T16:20:55.861 INFO:tasks.workunit.client.0.smithi006.stderr: -192> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x565265005c50) adding auth protocol: cephx 2024-05-30T16:20:55.861 INFO:tasks.workunit.client.0.smithi006.stderr: -191> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x565265005c50) adding auth protocol: cephx 2024-05-30T16:20:55.861 INFO:tasks.workunit.client.0.smithi006.stderr: -190> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x565265005c50) adding con mode: secure 2024-05-30T16:20:55.861 INFO:tasks.workunit.client.0.smithi006.stderr: -189> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x565265005c50) adding con mode: crc 2024-05-30T16:20:55.861 INFO:tasks.workunit.client.0.smithi006.stderr: -188> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x565265005c50) adding con mode: secure 2024-05-30T16:20:55.861 INFO:tasks.workunit.client.0.smithi006.stderr: -187> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x565265005c50) adding con mode: crc 2024-05-30T16:20:55.861 INFO:tasks.workunit.client.0.smithi006.stderr: -186> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x565265005c50) adding con mode: secure 2024-05-30T16:20:55.861 INFO:tasks.workunit.client.0.smithi006.stderr: -185> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x565265005c50) adding con mode: crc 2024-05-30T16:20:55.861 INFO:tasks.workunit.client.0.smithi006.stderr: -184> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x565265005c50) adding con mode: crc 2024-05-30T16:20:55.861 INFO:tasks.workunit.client.0.smithi006.stderr: -183> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x565265005c50) adding con mode: secure 2024-05-30T16:20:55.862 INFO:tasks.workunit.client.0.smithi006.stderr: -182> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x565265005c50) adding con mode: crc 2024-05-30T16:20:55.862 INFO:tasks.workunit.client.0.smithi006.stderr: -181> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x565265005c50) adding con mode: secure 2024-05-30T16:20:55.862 INFO:tasks.workunit.client.0.smithi006.stderr: -180> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x565265005c50) adding con mode: crc 2024-05-30T16:20:55.862 INFO:tasks.workunit.client.0.smithi006.stderr: -179> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x565265005c50) adding con mode: secure 2024-05-30T16:20:55.862 INFO:tasks.workunit.client.0.smithi006.stderr: -178> 2024-05-30T16:20:55.728+0000 7fa7de543980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:55.862 INFO:tasks.workunit.client.0.smithi006.stderr: -177> 2024-05-30T16:20:55.728+0000 7fa7de543980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:55.862 INFO:tasks.workunit.client.0.smithi006.stderr: -176> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 asok(0x565264d67790) register_command rotate-key cmddesc rotate-key hook 0x565265005d98 EEXIST 2024-05-30T16:20:55.862 INFO:tasks.workunit.client.0.smithi006.stderr: -175> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient: _reopen_session rank -1 2024-05-30T16:20:55.862 INFO:tasks.workunit.client.0.smithi006.stderr: -174> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient: _add_conns ranks=[2,1,0] 2024-05-30T16:20:55.862 INFO:tasks.workunit.client.0.smithi006.stderr: -173> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): picked mon.noname-c con 0x56526503c790 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:20:55.862 INFO:tasks.workunit.client.0.smithi006.stderr: -172> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): picked mon.noname-b con 0x56526503ded0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:20:55.862 INFO:tasks.workunit.client.0.smithi006.stderr: -171> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): picked mon.noname-a con 0x5652650421c0 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:20:55.862 INFO:tasks.workunit.client.0.smithi006.stderr: -170> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:55.863 INFO:tasks.workunit.client.0.smithi006.stderr: -169> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:55.863 INFO:tasks.workunit.client.0.smithi006.stderr: -168> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:55.863 INFO:tasks.workunit.client.0.smithi006.stderr: -167> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): _renew_subs 2024-05-30T16:20:55.863 INFO:tasks.workunit.client.0.smithi006.stderr: -166> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): authenticate will time out at 2595.667336s 2024-05-30T16:20:55.863 INFO:tasks.workunit.client.0.smithi006.stderr: -165> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): get_auth_request con 0x56526503c790 auth_method 0 2024-05-30T16:20:55.863 INFO:tasks.workunit.client.0.smithi006.stderr: -164> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:55.863 INFO:tasks.workunit.client.0.smithi006.stderr: -163> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:55.863 INFO:tasks.workunit.client.0.smithi006.stderr: -162> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:55.863 INFO:tasks.workunit.client.0.smithi006.stderr: -161> 2024-05-30T16:20:55.728+0000 7fa7dc855640 10 monclient(hunting): get_auth_request con 0x5652650421c0 auth_method 0 2024-05-30T16:20:55.863 INFO:tasks.workunit.client.0.smithi006.stderr: -160> 2024-05-30T16:20:55.728+0000 7fa7dc855640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:55.863 INFO:tasks.workunit.client.0.smithi006.stderr: -159> 2024-05-30T16:20:55.728+0000 7fa7dc855640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:55.863 INFO:tasks.workunit.client.0.smithi006.stderr: -158> 2024-05-30T16:20:55.728+0000 7fa7dc855640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:55.863 INFO:tasks.workunit.client.0.smithi006.stderr: -157> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:55.864 INFO:tasks.workunit.client.0.smithi006.stderr: -156> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:55.864 INFO:tasks.workunit.client.0.smithi006.stderr: -155> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:55.864 INFO:tasks.workunit.client.0.smithi006.stderr: -154> 2024-05-30T16:20:55.728+0000 7fa7dc855640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:55.864 INFO:tasks.workunit.client.0.smithi006.stderr: -153> 2024-05-30T16:20:55.728+0000 7fa7dc855640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:55.864 INFO:tasks.workunit.client.0.smithi006.stderr: -152> 2024-05-30T16:20:55.728+0000 7fa7dc855640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:55.864 INFO:tasks.workunit.client.0.smithi006.stderr: -151> 2024-05-30T16:20:55.728+0000 7fa7db853640 10 monclient(hunting): get_auth_request con 0x56526503ded0 auth_method 0 2024-05-30T16:20:55.864 INFO:tasks.workunit.client.0.smithi006.stderr: -150> 2024-05-30T16:20:55.728+0000 7fa7db853640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:55.864 INFO:tasks.workunit.client.0.smithi006.stderr: -149> 2024-05-30T16:20:55.728+0000 7fa7db853640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:55.864 INFO:tasks.workunit.client.0.smithi006.stderr: -148> 2024-05-30T16:20:55.728+0000 7fa7db853640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:55.864 INFO:tasks.workunit.client.0.smithi006.stderr: -147> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): handle_auth_done global_id 6215 payload 824 2024-05-30T16:20:55.864 INFO:tasks.workunit.client.0.smithi006.stderr: -146> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient: _finish_hunting 0 2024-05-30T16:20:55.864 INFO:tasks.workunit.client.0.smithi006.stderr: -145> 2024-05-30T16:20:55.728+0000 7fa7dc054640 1 monclient: found mon.noname-c 2024-05-30T16:20:55.864 INFO:tasks.workunit.client.0.smithi006.stderr: -144> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.6:3301/0 2024-05-30T16:20:55.865 INFO:tasks.workunit.client.0.smithi006.stderr: -143> 2024-05-30T16:20:55.732+0000 7fa6b1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:55.865 INFO:tasks.workunit.client.0.smithi006.stderr: -142> 2024-05-30T16:20:55.732+0000 7fa6b1ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-30T16:20:55.865 INFO:tasks.workunit.client.0.smithi006.stderr: -141> 2024-05-30T16:20:55.732+0000 7fa6b1ffb640 10 monclient: dump: 2024-05-30T16:20:55.865 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:55.865 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:55.865 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.865 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.865 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:55.865 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:55.865 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:55.865 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:55.865 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:55.865 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.866 INFO:tasks.workunit.client.0.smithi006.stderr: -140> 2024-05-30T16:20:55.732+0000 7fa6b1ffb640 10 monclient: _finish_auth 0 2024-05-30T16:20:55.866 INFO:tasks.workunit.client.0.smithi006.stderr: -139> 2024-05-30T16:20:55.732+0000 7fa6b1ffb640 10 monclient: _check_auth_tickets 2024-05-30T16:20:55.866 INFO:tasks.workunit.client.0.smithi006.stderr: -138> 2024-05-30T16:20:55.732+0000 7fa6b1ffb640 10 monclient: handle_config config(0 keys) 2024-05-30T16:20:55.866 INFO:tasks.workunit.client.0.smithi006.stderr: -137> 2024-05-30T16:20:55.732+0000 7fa6b1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:55.866 INFO:tasks.workunit.client.0.smithi006.stderr: -136> 2024-05-30T16:20:55.732+0000 7fa6b1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-30T16:20:55.866 INFO:tasks.workunit.client.0.smithi006.stderr: -135> 2024-05-30T16:20:55.732+0000 7fa6b1ffb640 10 monclient: dump: 2024-05-30T16:20:55.866 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:55.866 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:55.866 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.866 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.866 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:55.867 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:55.867 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:55.867 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:55.867 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:55.867 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.867 INFO:tasks.workunit.client.0.smithi006.stderr: -134> 2024-05-30T16:20:55.732+0000 7fa7de543980 5 monclient: authenticate success, global_id 6215 2024-05-30T16:20:55.867 INFO:tasks.workunit.client.0.smithi006.stderr: -133> 2024-05-30T16:20:55.732+0000 7fa6b37fe640 4 set_mon_vals no callback set 2024-05-30T16:20:55.867 INFO:tasks.workunit.client.0.smithi006.stderr: -132> 2024-05-30T16:20:55.732+0000 7fa7de543980 10 monclient: _renew_subs 2024-05-30T16:20:55.867 INFO:tasks.workunit.client.0.smithi006.stderr: -131> 2024-05-30T16:20:55.732+0000 7fa7de543980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.6:3301/0 2024-05-30T16:20:55.867 INFO:tasks.workunit.client.0.smithi006.stderr: -130> 2024-05-30T16:20:55.732+0000 7fa7de543980 10 monclient: _renew_subs 2024-05-30T16:20:55.867 INFO:tasks.workunit.client.0.smithi006.stderr: -129> 2024-05-30T16:20:55.732+0000 7fa7de543980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.6:3301/0 2024-05-30T16:20:55.867 INFO:tasks.workunit.client.0.smithi006.stderr: -128> 2024-05-30T16:20:55.732+0000 7fa7de543980 1 librados: init done 2024-05-30T16:20:55.867 INFO:tasks.workunit.client.0.smithi006.stderr: -127> 2024-05-30T16:20:55.732+0000 7fa7de543980 5 asok(0x565264d67790) register_command cr dump hook 0x56526503d9c8 2024-05-30T16:20:55.867 INFO:tasks.workunit.client.0.smithi006.stderr: -126> 2024-05-30T16:20:55.732+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.868 INFO:tasks.workunit.client.0.smithi006.stderr: -125> 2024-05-30T16:20:55.732+0000 7fa6b1ffb640 4 mgrc handle_mgr_map Got map version 5 2024-05-30T16:20:55.868 INFO:tasks.workunit.client.0.smithi006.stderr: -124> 2024-05-30T16:20:55.732+0000 7fa6b1ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:20:55.868 INFO:tasks.workunit.client.0.smithi006.stderr: -123> 2024-05-30T16:20:55.732+0000 7fa6b1ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:20:55.868 INFO:tasks.workunit.client.0.smithi006.stderr: -122> 2024-05-30T16:20:55.732+0000 7fa7dc855640 10 monclient: get_auth_request con 0x56526505c160 auth_method 0 2024-05-30T16:20:55.868 INFO:tasks.workunit.client.0.smithi006.stderr: -121> 2024-05-30T16:20:55.732+0000 7fa7db853640 10 monclient: get_auth_request con 0x7fa6a406dde0 auth_method 0 2024-05-30T16:20:55.868 INFO:tasks.workunit.client.0.smithi006.stderr: -120> 2024-05-30T16:20:55.736+0000 7fa7de543980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.868 INFO:tasks.workunit.client.0.smithi006.stderr: -119> 2024-05-30T16:20:55.736+0000 7fa7de543980 20 realm 2024-05-30T16:20:55.868 INFO:tasks.workunit.client.0.smithi006.stderr: -118> 2024-05-30T16:20:55.736+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.868 INFO:tasks.workunit.client.0.smithi006.stderr: -117> 2024-05-30T16:20:55.736+0000 7fa7de543980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.868 INFO:tasks.workunit.client.0.smithi006.stderr: -116> 2024-05-30T16:20:55.736+0000 7fa7de543980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:55.869 INFO:tasks.workunit.client.0.smithi006.stderr: -115> 2024-05-30T16:20:55.736+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.869 INFO:tasks.workunit.client.0.smithi006.stderr: -114> 2024-05-30T16:20:55.736+0000 7fa7de543980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.869 INFO:tasks.workunit.client.0.smithi006.stderr: -113> 2024-05-30T16:20:55.736+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.869 INFO:tasks.workunit.client.0.smithi006.stderr: -112> 2024-05-30T16:20:55.736+0000 7fa7dc054640 10 monclient: get_auth_request con 0x565265060350 auth_method 0 2024-05-30T16:20:55.869 INFO:tasks.workunit.client.0.smithi006.stderr: -111> 2024-05-30T16:20:55.736+0000 7fa7de543980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:55.869 INFO:tasks.workunit.client.0.smithi006.stderr: -110> 2024-05-30T16:20:55.736+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.869 INFO:tasks.workunit.client.0.smithi006.stderr: -109> 2024-05-30T16:20:55.736+0000 7fa7de543980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:55.869 INFO:tasks.workunit.client.0.smithi006.stderr: -108> 2024-05-30T16:20:55.736+0000 7fa7de543980 20 searching for the correct realm 2024-05-30T16:20:55.870 INFO:tasks.workunit.client.0.smithi006.stderr: -107> 2024-05-30T16:20:55.736+0000 7fa7db853640 10 monclient: get_auth_request con 0x5652650656c0 auth_method 0 2024-05-30T16:20:55.870 INFO:tasks.workunit.client.0.smithi006.stderr: -106> 2024-05-30T16:20:55.744+0000 7fa7dc855640 10 monclient: get_auth_request con 0x7fa7d0021060 auth_method 0 2024-05-30T16:20:55.870 INFO:tasks.workunit.client.0.smithi006.stderr: -105> 2024-05-30T16:20:55.744+0000 7fa7dc054640 10 monclient: get_auth_request con 0x7fa7d4017920 auth_method 0 2024-05-30T16:20:55.870 INFO:tasks.workunit.client.0.smithi006.stderr: -104> 2024-05-30T16:20:55.744+0000 7fa7dc054640 0 -- 172.21.15.6:0/2873706297 >> [v2:172.21.15.6:6816/2704410641,v1:172.21.15.6:6817/2704410641] conn(0x7fa7d4017920 msgr2=0x7fa7d4029410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:55.870 INFO:tasks.workunit.client.0.smithi006.stderr: -103> 2024-05-30T16:20:55.744+0000 7fa6b1ffb640 1 client.6215.objecter ms_handle_reset 0x7fa7d4017920 session 0x7fa7d400f040 osd.2 2024-05-30T16:20:55.870 INFO:tasks.workunit.client.0.smithi006.stderr: -102> 2024-05-30T16:20:55.744+0000 7fa6b1ffb640 10 monclient: _renew_subs 2024-05-30T16:20:55.870 INFO:tasks.workunit.client.0.smithi006.stderr: -101> 2024-05-30T16:20:55.744+0000 7fa6b1ffb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.6:3301/0 2024-05-30T16:20:55.870 INFO:tasks.workunit.client.0.smithi006.stderr: -100> 2024-05-30T16:20:55.744+0000 7fa7dc054640 10 monclient: get_auth_request con 0x7fa6a407e440 auth_method 0 2024-05-30T16:20:55.870 INFO:tasks.workunit.client.0.smithi006.stderr: -99> 2024-05-30T16:20:55.744+0000 7fa7db853640 10 monclient: get_auth_request con 0x7fa6c4001dd0 auth_method 0 2024-05-30T16:20:55.870 INFO:tasks.workunit.client.0.smithi006.stderr: -98> 2024-05-30T16:20:55.744+0000 7fa7dc855640 10 monclient: get_auth_request con 0x7fa7d0063b10 auth_method 0 2024-05-30T16:20:55.871 INFO:tasks.workunit.client.0.smithi006.stderr: -97> 2024-05-30T16:20:55.748+0000 7fa7dc054640 0 -- 172.21.15.6:0/2873706297 >> [v2:172.21.15.6:6816/2704410641,v1:172.21.15.6:6817/2704410641] conn(0x7fa6a407e440 msgr2=0x7fa6a4080870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:55.871 INFO:tasks.workunit.client.0.smithi006.stderr: -96> 2024-05-30T16:20:55.748+0000 7fa6b1ffb640 1 client.6215.objecter ms_handle_reset 0x7fa6a407e440 session 0x7fa7d400f040 osd.2 2024-05-30T16:20:55.871 INFO:tasks.workunit.client.0.smithi006.stderr: -95> 2024-05-30T16:20:55.748+0000 7fa7dc054640 10 monclient: get_auth_request con 0x7fa7d4017920 auth_method 0 2024-05-30T16:20:55.871 INFO:tasks.workunit.client.0.smithi006.stderr: -94> 2024-05-30T16:20:55.752+0000 7fa7dc054640 10 monclient: get_auth_request con 0x7fa7d00673f0 auth_method 0 2024-05-30T16:20:55.871 INFO:tasks.workunit.client.0.smithi006.stderr: -93> 2024-05-30T16:20:55.760+0000 7fa7de543980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:55.871 INFO:tasks.workunit.client.0.smithi006.stderr: -92> 2024-05-30T16:20:55.760+0000 7fa7de543980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:55.871 INFO:tasks.workunit.client.0.smithi006.stderr: -91> 2024-05-30T16:20:55.760+0000 7fa7de543980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:55.872 INFO:tasks.workunit.client.0.smithi006.stderr: -90> 2024-05-30T16:20:55.760+0000 7fa7de543980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:55.872 INFO:tasks.workunit.client.0.smithi006.stderr: -89> 2024-05-30T16:20:55.760+0000 7fa7de543980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:55.872 INFO:tasks.workunit.client.0.smithi006.stderr: -88> 2024-05-30T16:20:55.760+0000 7fa7de543980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:55.872 INFO:tasks.workunit.client.0.smithi006.stderr: -87> 2024-05-30T16:20:55.760+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.872 INFO:tasks.workunit.client.0.smithi006.stderr: -86> 2024-05-30T16:20:55.760+0000 7fa7de543980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.872 INFO:tasks.workunit.client.0.smithi006.stderr: -85> 2024-05-30T16:20:55.760+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.872 INFO:tasks.workunit.client.0.smithi006.stderr: -84> 2024-05-30T16:20:55.760+0000 7fa7de543980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:55.872 INFO:tasks.workunit.client.0.smithi006.stderr: -83> 2024-05-30T16:20:55.760+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.873 INFO:tasks.workunit.client.0.smithi006.stderr: -82> 2024-05-30T16:20:55.768+0000 7fa7de543980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:55.873 INFO:tasks.workunit.client.0.smithi006.stderr: -81> 2024-05-30T16:20:55.768+0000 7fa7de543980 20 zone default found 2024-05-30T16:20:55.873 INFO:tasks.workunit.client.0.smithi006.stderr: -80> 2024-05-30T16:20:55.768+0000 7fa7de543980 4 Realm: () 2024-05-30T16:20:55.873 INFO:tasks.workunit.client.0.smithi006.stderr: -79> 2024-05-30T16:20:55.768+0000 7fa7de543980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:55.873 INFO:tasks.workunit.client.0.smithi006.stderr: -78> 2024-05-30T16:20:55.768+0000 7fa7de543980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:55.873 INFO:tasks.workunit.client.0.smithi006.stderr: -77> 2024-05-30T16:20:55.768+0000 7fa7de543980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:55.873 INFO:tasks.workunit.client.0.smithi006.stderr: -76> 2024-05-30T16:20:55.768+0000 7fa7de543980 20 zonegroup default 2024-05-30T16:20:55.873 INFO:tasks.workunit.client.0.smithi006.stderr: -75> 2024-05-30T16:20:55.768+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.874 INFO:tasks.workunit.client.0.smithi006.stderr: -74> 2024-05-30T16:20:55.768+0000 7fa7de543980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.874 INFO:tasks.workunit.client.0.smithi006.stderr: -73> 2024-05-30T16:20:55.768+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.874 INFO:tasks.workunit.client.0.smithi006.stderr: -72> 2024-05-30T16:20:55.768+0000 7fa7de543980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.874 INFO:tasks.workunit.client.0.smithi006.stderr: -71> 2024-05-30T16:20:55.768+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.874 INFO:tasks.workunit.client.0.smithi006.stderr: -70> 2024-05-30T16:20:55.768+0000 7fa7de543980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.874 INFO:tasks.workunit.client.0.smithi006.stderr: -69> 2024-05-30T16:20:55.768+0000 7fa7de543980 20 started sync module instance, tier type = 2024-05-30T16:20:55.874 INFO:tasks.workunit.client.0.smithi006.stderr: -68> 2024-05-30T16:20:55.768+0000 7fa7de543980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:55.874 INFO:tasks.workunit.client.0.smithi006.stderr: -67> 2024-05-30T16:20:55.780+0000 7fa7de543980 20 add_watcher() i=0 2024-05-30T16:20:55.874 INFO:tasks.workunit.client.0.smithi006.stderr: -66> 2024-05-30T16:20:55.780+0000 7fa7de543980 20 add_watcher() i=1 2024-05-30T16:20:55.875 INFO:tasks.workunit.client.0.smithi006.stderr: -65> 2024-05-30T16:20:55.780+0000 7fa7de543980 20 add_watcher() i=2 2024-05-30T16:20:55.875 INFO:tasks.workunit.client.0.smithi006.stderr: -64> 2024-05-30T16:20:55.780+0000 7fa7de543980 20 add_watcher() i=3 2024-05-30T16:20:55.875 INFO:tasks.workunit.client.0.smithi006.stderr: -63> 2024-05-30T16:20:55.780+0000 7fa7de543980 20 add_watcher() i=4 2024-05-30T16:20:55.875 INFO:tasks.workunit.client.0.smithi006.stderr: -62> 2024-05-30T16:20:55.780+0000 7fa7de543980 20 add_watcher() i=5 2024-05-30T16:20:55.875 INFO:tasks.workunit.client.0.smithi006.stderr: -61> 2024-05-30T16:20:55.780+0000 7fa7de543980 20 add_watcher() i=6 2024-05-30T16:20:55.875 INFO:tasks.workunit.client.0.smithi006.stderr: -60> 2024-05-30T16:20:55.780+0000 7fa7de543980 20 add_watcher() i=7 2024-05-30T16:20:55.875 INFO:tasks.workunit.client.0.smithi006.stderr: -59> 2024-05-30T16:20:55.780+0000 7fa7de543980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:55.875 INFO:tasks.workunit.client.0.smithi006.stderr: -58> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c2280) adding auth protocol: cephx 2024-05-30T16:20:55.875 INFO:tasks.workunit.client.0.smithi006.stderr: -57> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c2280) adding auth protocol: cephx 2024-05-30T16:20:55.875 INFO:tasks.workunit.client.0.smithi006.stderr: -56> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c2280) adding auth protocol: cephx 2024-05-30T16:20:55.876 INFO:tasks.workunit.client.0.smithi006.stderr: -55> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c2280) adding con mode: secure 2024-05-30T16:20:55.876 INFO:tasks.workunit.client.0.smithi006.stderr: -54> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c2280) adding con mode: crc 2024-05-30T16:20:55.876 INFO:tasks.workunit.client.0.smithi006.stderr: -53> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c2280) adding con mode: secure 2024-05-30T16:20:55.876 INFO:tasks.workunit.client.0.smithi006.stderr: -52> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c2280) adding con mode: crc 2024-05-30T16:20:55.876 INFO:tasks.workunit.client.0.smithi006.stderr: -51> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c2280) adding con mode: secure 2024-05-30T16:20:55.876 INFO:tasks.workunit.client.0.smithi006.stderr: -50> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c2280) adding con mode: crc 2024-05-30T16:20:55.876 INFO:tasks.workunit.client.0.smithi006.stderr: -49> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c2280) adding con mode: crc 2024-05-30T16:20:55.876 INFO:tasks.workunit.client.0.smithi006.stderr: -48> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c2280) adding con mode: secure 2024-05-30T16:20:55.876 INFO:tasks.workunit.client.0.smithi006.stderr: -47> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c2280) adding con mode: crc 2024-05-30T16:20:55.876 INFO:tasks.workunit.client.0.smithi006.stderr: -46> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c2280) adding con mode: secure 2024-05-30T16:20:55.877 INFO:tasks.workunit.client.0.smithi006.stderr: -45> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c2280) adding con mode: crc 2024-05-30T16:20:55.877 INFO:tasks.workunit.client.0.smithi006.stderr: -44> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c2280) adding con mode: secure 2024-05-30T16:20:55.877 INFO:tasks.workunit.client.0.smithi006.stderr: -43> 2024-05-30T16:20:55.784+0000 7fa692fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:55.877 INFO:tasks.workunit.client.0.smithi006.stderr: -42> 2024-05-30T16:20:55.784+0000 7fa7de543980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:55.877 INFO:tasks.workunit.client.0.smithi006.stderr: -41> 2024-05-30T16:20:55.784+0000 7fa7de543980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:55.877 INFO:tasks.workunit.client.0.smithi006.stderr: -40> 2024-05-30T16:20:55.784+0000 7fa7de543980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:55.877 INFO:tasks.workunit.client.0.smithi006.stderr: -39> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 asok(0x565264d67790) register_command cache list hook 0x5652650452a0 2024-05-30T16:20:55.877 INFO:tasks.workunit.client.0.smithi006.stderr: -38> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 asok(0x565264d67790) register_command cache inspect hook 0x5652650452a0 2024-05-30T16:20:55.877 INFO:tasks.workunit.client.0.smithi006.stderr: -37> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 asok(0x565264d67790) register_command cache erase hook 0x5652650452a0 2024-05-30T16:20:55.877 INFO:tasks.workunit.client.0.smithi006.stderr: -36> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 asok(0x565264d67790) register_command cache zap hook 0x5652650452a0 2024-05-30T16:20:55.877 INFO:tasks.workunit.client.0.smithi006.stderr: -35> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 note: GC not initialized 2024-05-30T16:20:55.877 INFO:tasks.workunit.client.0.smithi006.stderr: -34> 2024-05-30T16:20:55.784+0000 7fa691fdb640 20 reqs_thread_entry: start 2024-05-30T16:20:55.877 INFO:tasks.workunit.client.0.smithi006.stderr: -33> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 asok(0x565264d67790) register_command sync trace show hook 0x7fa6c4004870 2024-05-30T16:20:55.878 INFO:tasks.workunit.client.0.smithi006.stderr: -32> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 asok(0x565264d67790) register_command sync trace history hook 0x7fa6c4004870 2024-05-30T16:20:55.878 INFO:tasks.workunit.client.0.smithi006.stderr: -31> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 asok(0x565264d67790) register_command sync trace active hook 0x7fa6c4004870 2024-05-30T16:20:55.878 INFO:tasks.workunit.client.0.smithi006.stderr: -30> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 asok(0x565264d67790) register_command sync trace active_short hook 0x7fa6c4004870 2024-05-30T16:20:55.878 INFO:tasks.workunit.client.0.smithi006.stderr: -29> 2024-05-30T16:20:55.784+0000 7fa7de543980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:55.878 INFO:tasks.workunit.client.0.smithi006.stderr: -28> 2024-05-30T16:20:55.784+0000 7fa7de543980 20 Filter name: none 2024-05-30T16:20:55.878 INFO:tasks.workunit.client.0.smithi006.stderr: -27> 2024-05-30T16:20:55.784+0000 7fa6037fe640 20 reqs_thread_entry: start 2024-05-30T16:20:55.878 INFO:tasks.workunit.client.0.smithi006.stderr: -26> 2024-05-30T16:20:55.784+0000 7fa7de543980 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-05-30T16:20:55.878 INFO:tasks.workunit.client.0.smithi006.stderr: -25> 2024-05-30T16:20:55.784+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.879 INFO:tasks.workunit.client.0.smithi006.stderr: -24> 2024-05-30T16:20:55.784+0000 7fa7de543980 20 rados_obj.operate() r=0 bl.length=213 2024-05-30T16:20:55.879 INFO:tasks.workunit.client.0.smithi006.stderr: -23> 2024-05-30T16:20:55.784+0000 7fa7de543980 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-05-30T16:20:55.879 INFO:tasks.workunit.client.0.smithi006.stderr: -22> 2024-05-30T16:20:55.784+0000 7fa7de543980 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-05-30T16:20:55.879 INFO:tasks.workunit.client.0.smithi006.stderr: -21> 2024-05-30T16:20:55.784+0000 7fa7de543980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 : miss 2024-05-30T16:20:55.879 INFO:tasks.workunit.client.0.smithi006.stderr: -20> 2024-05-30T16:20:55.784+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.879 INFO:tasks.workunit.client.0.smithi006.stderr: -19> 2024-05-30T16:20:55.788+0000 7fa7de543980 20 rados_obj.operate() r=0 bl.length=427 2024-05-30T16:20:55.879 INFO:tasks.workunit.client.0.smithi006.stderr: -18> 2024-05-30T16:20:55.788+0000 7fa7de543980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 info.flags=0x17 2024-05-30T16:20:55.879 INFO:tasks.workunit.client.0.smithi006.stderr: -17> 2024-05-30T16:20:55.788+0000 7fa7de543980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 to cache LRU end 2024-05-30T16:20:55.880 INFO:tasks.workunit.client.0.smithi006.stderr: -16> 2024-05-30T16:20:55.788+0000 7fa7de543980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:20:55.880 INFO:tasks.workunit.client.0.smithi006.stderr: -15> 2024-05-30T16:20:55.788+0000 7fa7de543980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:55.880 INFO:tasks.workunit.client.0.smithi006.stderr: -14> 2024-05-30T16:20:55.788+0000 7fa7de543980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 2024-05-30T16:20:55.880 INFO:tasks.workunit.client.0.smithi006.stderr: -13> 2024-05-30T16:20:55.800+0000 7fa7de543980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 info.flags=0x17 2024-05-30T16:20:55.880 INFO:tasks.workunit.client.0.smithi006.stderr: -12> 2024-05-30T16:20:55.800+0000 7fa7de543980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 to cache LRU end 2024-05-30T16:20:55.880 INFO:tasks.workunit.client.0.smithi006.stderr: -11> 2024-05-30T16:20:55.800+0000 7fa7de543980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:55.881 INFO:tasks.workunit.client.0.smithi006.stderr: -10> 2024-05-30T16:20:55.800+0000 7fa7de543980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24, ofs0, ns] 2024-05-30T16:20:55.881 INFO:tasks.workunit.client.0.smithi006.stderr: -9> 2024-05-30T16:20:55.800+0000 7fa6b2ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431625 cookie 94911882257936 notifier 6215 bl.length()=991 2024-05-30T16:20:55.881 INFO:tasks.workunit.client.0.smithi006.stderr: -8> 2024-05-30T16:20:55.800+0000 7fa6b2ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 info.flags=0x17 2024-05-30T16:20:55.881 INFO:tasks.workunit.client.0.smithi006.stderr: -7> 2024-05-30T16:20:55.800+0000 7fa6b2ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 to cache LRU end 2024-05-30T16:20:55.881 INFO:tasks.workunit.client.0.smithi006.stderr: -6> 2024-05-30T16:20:55.800+0000 7fa6b2ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:55.881 INFO:tasks.workunit.client.0.smithi006.stderr: -5> 2024-05-30T16:20:55.800+0000 7fa7de543980 20 handle_bi_update(): bucket=:abort-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24]): orig_sources= new_sources= 2024-05-30T16:20:55.881 INFO:tasks.workunit.client.0.smithi006.stderr: -4> 2024-05-30T16:20:55.800+0000 7fa7de543980 20 handle_bi_update(): bucket=:abort-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24]): potential sources added=[] removed=[] 2024-05-30T16:20:55.881 INFO:tasks.workunit.client.0.smithi006.stderr: -3> 2024-05-30T16:20:55.800+0000 7fa7de543980 20 handle_bi_update(): bucket=:abort-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24]): orig_dests= new_dests= 2024-05-30T16:20:55.881 INFO:tasks.workunit.client.0.smithi006.stderr: -2> 2024-05-30T16:20:55.800+0000 7fa7de543980 20 handle_bi_update(): bucket=:abort-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24]): potential dests added=[] removed=[] 2024-05-30T16:20:55.881 INFO:tasks.workunit.client.0.smithi006.stderr: -1> 2024-05-30T16:20:55.804+0000 7fa7de543980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fa7de543980 time 2024-05-30T16:20:55.808517+0000 2024-05-30T16:20:55.881 INFO:tasks.workunit.client.0.smithi006.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-30T16:20:55.882 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.882 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:20:55.882 INFO:tasks.workunit.client.0.smithi006.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fa7e1e1acff] 2024-05-30T16:20:55.882 INFO:tasks.workunit.client.0.smithi006.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7fa7e1e1aeb3] 2024-05-30T16:20:55.882 INFO:tasks.workunit.client.0.smithi006.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x565262955f81] 2024-05-30T16:20:55.882 INFO:tasks.workunit.client.0.smithi006.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2ab) [0x565262967eab] 2024-05-30T16:20:55.882 INFO:tasks.workunit.client.0.smithi006.stderr: 5: main() 2024-05-30T16:20:55.882 INFO:tasks.workunit.client.0.smithi006.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fa7e0c83d90] 2024-05-30T16:20:55.883 INFO:tasks.workunit.client.0.smithi006.stderr: 7: __libc_start_main() 2024-05-30T16:20:55.883 INFO:tasks.workunit.client.0.smithi006.stderr: 8: _start() 2024-05-30T16:20:55.883 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.883 INFO:tasks.workunit.client.0.smithi006.stderr: 0> 2024-05-30T16:20:55.804+0000 7fa7de543980 -1 *** Caught signal (Aborted) ** 2024-05-30T16:20:55.883 INFO:tasks.workunit.client.0.smithi006.stderr: in thread 7fa7de543980 thread_name:radosgw-admin 2024-05-30T16:20:55.883 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.884 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:20:55.884 INFO:tasks.workunit.client.0.smithi006.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fa7e0c9c520] 2024-05-30T16:20:55.884 INFO:tasks.workunit.client.0.smithi006.stderr: 2: pthread_kill() 2024-05-30T16:20:55.884 INFO:tasks.workunit.client.0.smithi006.stderr: 3: raise() 2024-05-30T16:20:55.884 INFO:tasks.workunit.client.0.smithi006.stderr: 4: abort() 2024-05-30T16:20:55.885 INFO:tasks.workunit.client.0.smithi006.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fa7e1e1ad59] 2024-05-30T16:20:55.885 INFO:tasks.workunit.client.0.smithi006.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7fa7e1e1aeb3] 2024-05-30T16:20:55.885 INFO:tasks.workunit.client.0.smithi006.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x565262955f81] 2024-05-30T16:20:55.885 INFO:tasks.workunit.client.0.smithi006.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2ab) [0x565262967eab] 2024-05-30T16:20:55.885 INFO:tasks.workunit.client.0.smithi006.stderr: 9: main() 2024-05-30T16:20:55.886 INFO:tasks.workunit.client.0.smithi006.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fa7e0c83d90] 2024-05-30T16:20:55.886 INFO:tasks.workunit.client.0.smithi006.stderr: 11: __libc_start_main() 2024-05-30T16:20:55.886 INFO:tasks.workunit.client.0.smithi006.stderr: 12: _start() 2024-05-30T16:20:55.886 INFO:tasks.workunit.client.0.smithi006.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-30T16:20:55.886 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.886 INFO:tasks.workunit.client.0.smithi006.stderr:--- logging levels --- 2024-05-30T16:20:55.886 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 none 2024-05-30T16:20:55.887 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 lockdep 2024-05-30T16:20:55.887 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 context 2024-05-30T16:20:55.887 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 crush 2024-05-30T16:20:55.887 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds 2024-05-30T16:20:55.887 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_balancer 2024-05-30T16:20:55.887 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_locker 2024-05-30T16:20:55.887 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_log 2024-05-30T16:20:55.888 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_log_expire 2024-05-30T16:20:55.888 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_migrator 2024-05-30T16:20:55.888 INFO:tasks.workunit.client.0.smithi006.stderr: 3/ 5 mds_quiesce 2024-05-30T16:20:55.888 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 buffer 2024-05-30T16:20:55.888 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 timer 2024-05-30T16:20:55.888 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 filer 2024-05-30T16:20:55.888 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 striper 2024-05-30T16:20:55.888 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 objecter 2024-05-30T16:20:55.888 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rados 2024-05-30T16:20:55.888 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd 2024-05-30T16:20:55.889 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd_mirror 2024-05-30T16:20:55.889 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd_replay 2024-05-30T16:20:55.889 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd_pwl 2024-05-30T16:20:55.889 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 journaler 2024-05-30T16:20:55.889 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 objectcacher 2024-05-30T16:20:55.889 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 immutable_obj_cache 2024-05-30T16:20:55.889 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 client 2024-05-30T16:20:55.889 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 osd 2024-05-30T16:20:55.889 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 optracker 2024-05-30T16:20:55.889 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 objclass 2024-05-30T16:20:55.889 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 3 filestore 2024-05-30T16:20:55.889 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 3 journal 2024-05-30T16:20:55.889 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 0 ms 2024-05-30T16:20:55.889 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mon 2024-05-30T16:20:55.890 INFO:tasks.workunit.client.0.smithi006.stderr: 0/10 monc 2024-05-30T16:20:55.890 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 paxos 2024-05-30T16:20:55.890 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 tp 2024-05-30T16:20:55.890 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 auth 2024-05-30T16:20:55.890 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 crypto 2024-05-30T16:20:55.890 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 finisher 2024-05-30T16:20:55.890 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 reserver 2024-05-30T16:20:55.890 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 heartbeatmap 2024-05-30T16:20:55.891 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 perfcounter 2024-05-30T16:20:55.891 INFO:tasks.workunit.client.0.smithi006.stderr: 20/20 rgw 2024-05-30T16:20:55.891 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_sync 2024-05-30T16:20:55.891 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_datacache 2024-05-30T16:20:55.891 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_access 2024-05-30T16:20:55.891 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_dbstore 2024-05-30T16:20:55.891 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_flight 2024-05-30T16:20:55.891 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_lifecycle 2024-05-30T16:20:55.891 INFO:tasks.workunit.client.0.smithi006.stderr: 20/20 rgw_notification 2024-05-30T16:20:55.891 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 javaclient 2024-05-30T16:20:55.891 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 asok 2024-05-30T16:20:55.891 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 throttle 2024-05-30T16:20:55.892 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 0 refs 2024-05-30T16:20:55.892 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 compressor 2024-05-30T16:20:55.892 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 bluestore 2024-05-30T16:20:55.892 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 bluefs 2024-05-30T16:20:55.892 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 3 bdev 2024-05-30T16:20:55.892 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 kstore 2024-05-30T16:20:55.892 INFO:tasks.workunit.client.0.smithi006.stderr: 4/ 5 rocksdb 2024-05-30T16:20:55.892 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 fuse 2024-05-30T16:20:55.892 INFO:tasks.workunit.client.0.smithi006.stderr: 2/ 5 mgr 2024-05-30T16:20:55.892 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mgrc 2024-05-30T16:20:55.892 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 dpdk 2024-05-30T16:20:55.892 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 eventtrace 2024-05-30T16:20:55.892 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 prioritycache 2024-05-30T16:20:55.892 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 test 2024-05-30T16:20:55.893 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 cephfs_mirror 2024-05-30T16:20:55.893 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 cephsqlite 2024-05-30T16:20:55.893 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 crimson_interrupt 2024-05-30T16:20:55.893 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore 2024-05-30T16:20:55.893 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_onode 2024-05-30T16:20:55.893 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_odata 2024-05-30T16:20:55.893 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_omap 2024-05-30T16:20:55.893 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_tm 2024-05-30T16:20:55.893 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_t 2024-05-30T16:20:55.893 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_cleaner 2024-05-30T16:20:55.893 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_epm 2024-05-30T16:20:55.893 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_lba 2024-05-30T16:20:55.893 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_fixedkv_tree 2024-05-30T16:20:55.893 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_cache 2024-05-30T16:20:55.894 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_journal 2024-05-30T16:20:55.894 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_device 2024-05-30T16:20:55.894 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_backref 2024-05-30T16:20:55.894 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 alienstore 2024-05-30T16:20:55.894 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mclock 2024-05-30T16:20:55.894 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 cyanstore 2024-05-30T16:20:55.894 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 ceph_exporter 2024-05-30T16:20:55.894 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 memstore 2024-05-30T16:20:55.894 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 trace 2024-05-30T16:20:55.894 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 ceph_dedup 2024-05-30T16:20:55.894 INFO:tasks.workunit.client.0.smithi006.stderr: -2/-2 (syslog threshold) 2024-05-30T16:20:55.894 INFO:tasks.workunit.client.0.smithi006.stderr: 99/99 (stderr threshold) 2024-05-30T16:20:55.895 INFO:tasks.workunit.client.0.smithi006.stderr:--- pthread ID / name mapping for recent threads --- 2024-05-30T16:20:55.895 INFO:tasks.workunit.client.0.smithi006.stderr: 7fa6037fe640 / http_manager 2024-05-30T16:20:55.895 INFO:tasks.workunit.client.0.smithi006.stderr: 7fa691fdb640 / http_manager 2024-05-30T16:20:55.895 INFO:tasks.workunit.client.0.smithi006.stderr: 7fa692fdd640 / rgw_dt_lg_renew 2024-05-30T16:20:55.895 INFO:tasks.workunit.client.0.smithi006.stderr: 7fa6b1ffb640 / ms_dispatch 2024-05-30T16:20:55.895 INFO:tasks.workunit.client.0.smithi006.stderr: 7fa6b2ffd640 / io_context_pool 2024-05-30T16:20:55.895 INFO:tasks.workunit.client.0.smithi006.stderr: 7fa6b37fe640 / io_context_pool 2024-05-30T16:20:55.895 INFO:tasks.workunit.client.0.smithi006.stderr: 7fa6c1ffb640 / ms_dispatch 2024-05-30T16:20:55.895 INFO:tasks.workunit.client.0.smithi006.stderr: 7fa6c8e09640 / ms_dispatch 2024-05-30T16:20:55.895 INFO:tasks.workunit.client.0.smithi006.stderr: 7fa6ca60c640 / io_context_pool 2024-05-30T16:20:55.895 INFO:tasks.workunit.client.0.smithi006.stderr: 7fa7db853640 / msgr-worker-2 2024-05-30T16:20:55.895 INFO:tasks.workunit.client.0.smithi006.stderr: 7fa7dc054640 / msgr-worker-1 2024-05-30T16:20:55.896 INFO:tasks.workunit.client.0.smithi006.stderr: 7fa7dc855640 / msgr-worker-0 2024-05-30T16:20:55.896 INFO:tasks.workunit.client.0.smithi006.stderr: 7fa7dd056640 / io_context_pool 2024-05-30T16:20:55.896 INFO:tasks.workunit.client.0.smithi006.stderr: 7fa7de543980 / radosgw-admin 2024-05-30T16:20:55.896 INFO:tasks.workunit.client.0.smithi006.stderr: max_recent 500 2024-05-30T16:20:55.896 INFO:tasks.workunit.client.0.smithi006.stderr: max_new 1000 2024-05-30T16:20:55.896 INFO:tasks.workunit.client.0.smithi006.stderr: log_file /var/log/ceph/ceph-client.admin.87511.log 2024-05-30T16:20:55.896 INFO:tasks.workunit.client.0.smithi006.stderr:--- end dump of recent events --- 2024-05-30T16:20:55.896 INFO:tasks.workunit.client.0.smithi006.stderr:--- begin dump of recent events --- 2024-05-30T16:20:55.896 INFO:tasks.workunit.client.0.smithi006.stderr: -499> 2024-05-30T16:20:55.708+0000 7fa6ca60c640 10 monclient: handle_config config(0 keys) 2024-05-30T16:20:55.896 INFO:tasks.workunit.client.0.smithi006.stderr: -498> 2024-05-30T16:20:55.708+0000 7fa6ca60c640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:55.896 INFO:tasks.workunit.client.0.smithi006.stderr: -497> 2024-05-30T16:20:55.708+0000 7fa6ca60c640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-30T16:20:55.896 INFO:tasks.workunit.client.0.smithi006.stderr: -496> 2024-05-30T16:20:55.708+0000 7fa6ca60c640 10 monclient: dump: 2024-05-30T16:20:55.896 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:55.897 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:55.897 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.897 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.897 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:55.897 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:55.897 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:55.897 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:55.897 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:55.897 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.897 INFO:tasks.workunit.client.0.smithi006.stderr: -495> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 monclient: authenticate success, global_id 6203 2024-05-30T16:20:55.897 INFO:tasks.workunit.client.0.smithi006.stderr: -494> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient: get_monmap_and_config success 2024-05-30T16:20:55.897 INFO:tasks.workunit.client.0.smithi006.stderr: -493> 2024-05-30T16:20:55.708+0000 7fa7de543980 4 set_mon_vals no callback set 2024-05-30T16:20:55.898 INFO:tasks.workunit.client.0.smithi006.stderr: -492> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient: shutdown 2024-05-30T16:20:55.898 INFO:tasks.workunit.client.0.smithi006.stderr: -491> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 asok(0x565264d67790) unregister_commands rotate-key 2024-05-30T16:20:55.898 INFO:tasks.workunit.client.0.smithi006.stderr: -490> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient: build_initial_monmap 2024-05-30T16:20:55.898 INFO:tasks.workunit.client.0.smithi006.stderr: -489> 2024-05-30T16:20:55.708+0000 7fa7de543980 1 build_initial for_mkfs: 0 2024-05-30T16:20:55.898 INFO:tasks.workunit.client.0.smithi006.stderr: -488> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient: monmap: 2024-05-30T16:20:55.898 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 0 2024-05-30T16:20:55.898 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-30T16:20:55.898 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 0.000000 2024-05-30T16:20:55.898 INFO:tasks.workunit.client.0.smithi006.stderr:created 0.000000 2024-05-30T16:20:55.898 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 0 (unknown) 2024-05-30T16:20:55.898 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:55.898 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.noname-a 2024-05-30T16:20:55.898 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-b 2024-05-30T16:20:55.899 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.noname-c 2024-05-30T16:20:55.899 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.899 INFO:tasks.workunit.client.0.smithi006.stderr: -487> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f75ce0) adding auth protocol: cephx 2024-05-30T16:20:55.899 INFO:tasks.workunit.client.0.smithi006.stderr: -486> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f75ce0) adding auth protocol: cephx 2024-05-30T16:20:55.899 INFO:tasks.workunit.client.0.smithi006.stderr: -485> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f75ce0) adding auth protocol: cephx 2024-05-30T16:20:55.899 INFO:tasks.workunit.client.0.smithi006.stderr: -484> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f75ce0) adding con mode: secure 2024-05-30T16:20:55.899 INFO:tasks.workunit.client.0.smithi006.stderr: -483> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f75ce0) adding con mode: crc 2024-05-30T16:20:55.899 INFO:tasks.workunit.client.0.smithi006.stderr: -482> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f75ce0) adding con mode: secure 2024-05-30T16:20:55.899 INFO:tasks.workunit.client.0.smithi006.stderr: -481> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f75ce0) adding con mode: crc 2024-05-30T16:20:55.899 INFO:tasks.workunit.client.0.smithi006.stderr: -480> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f75ce0) adding con mode: secure 2024-05-30T16:20:55.899 INFO:tasks.workunit.client.0.smithi006.stderr: -479> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f75ce0) adding con mode: crc 2024-05-30T16:20:55.899 INFO:tasks.workunit.client.0.smithi006.stderr: -478> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f75ce0) adding con mode: crc 2024-05-30T16:20:55.899 INFO:tasks.workunit.client.0.smithi006.stderr: -477> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f75ce0) adding con mode: secure 2024-05-30T16:20:55.900 INFO:tasks.workunit.client.0.smithi006.stderr: -476> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f75ce0) adding con mode: crc 2024-05-30T16:20:55.900 INFO:tasks.workunit.client.0.smithi006.stderr: -475> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f75ce0) adding con mode: secure 2024-05-30T16:20:55.900 INFO:tasks.workunit.client.0.smithi006.stderr: -474> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f75ce0) adding con mode: crc 2024-05-30T16:20:55.900 INFO:tasks.workunit.client.0.smithi006.stderr: -473> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f75ce0) adding con mode: secure 2024-05-30T16:20:55.900 INFO:tasks.workunit.client.0.smithi006.stderr: -472> 2024-05-30T16:20:55.708+0000 7fa7de543980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:55.900 INFO:tasks.workunit.client.0.smithi006.stderr: -471> 2024-05-30T16:20:55.708+0000 7fa7de543980 1 librados: starting msgr at 2024-05-30T16:20:55.900 INFO:tasks.workunit.client.0.smithi006.stderr: -470> 2024-05-30T16:20:55.708+0000 7fa7de543980 1 librados: starting objecter 2024-05-30T16:20:55.900 INFO:tasks.workunit.client.0.smithi006.stderr: -469> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 asok(0x565264d67790) register_command objecter_requests hook 0x565264ce6360 2024-05-30T16:20:55.900 INFO:tasks.workunit.client.0.smithi006.stderr: -468> 2024-05-30T16:20:55.708+0000 7fa7de543980 1 librados: setting wanted keys 2024-05-30T16:20:55.900 INFO:tasks.workunit.client.0.smithi006.stderr: -467> 2024-05-30T16:20:55.708+0000 7fa7de543980 1 librados: calling monclient init 2024-05-30T16:20:55.900 INFO:tasks.workunit.client.0.smithi006.stderr: -466> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient: init 2024-05-30T16:20:55.900 INFO:tasks.workunit.client.0.smithi006.stderr: -465> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f72930) adding auth protocol: cephx 2024-05-30T16:20:55.900 INFO:tasks.workunit.client.0.smithi006.stderr: -464> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f72930) adding auth protocol: cephx 2024-05-30T16:20:55.901 INFO:tasks.workunit.client.0.smithi006.stderr: -463> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f72930) adding auth protocol: cephx 2024-05-30T16:20:55.901 INFO:tasks.workunit.client.0.smithi006.stderr: -462> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f72930) adding con mode: secure 2024-05-30T16:20:55.901 INFO:tasks.workunit.client.0.smithi006.stderr: -461> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f72930) adding con mode: crc 2024-05-30T16:20:55.901 INFO:tasks.workunit.client.0.smithi006.stderr: -460> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f72930) adding con mode: secure 2024-05-30T16:20:55.901 INFO:tasks.workunit.client.0.smithi006.stderr: -459> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f72930) adding con mode: crc 2024-05-30T16:20:55.902 INFO:tasks.workunit.client.0.smithi006.stderr: -458> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f72930) adding con mode: secure 2024-05-30T16:20:55.902 INFO:tasks.workunit.client.0.smithi006.stderr: -457> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f72930) adding con mode: crc 2024-05-30T16:20:55.902 INFO:tasks.workunit.client.0.smithi006.stderr: -456> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f72930) adding con mode: crc 2024-05-30T16:20:55.902 INFO:tasks.workunit.client.0.smithi006.stderr: -455> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f72930) adding con mode: secure 2024-05-30T16:20:55.902 INFO:tasks.workunit.client.0.smithi006.stderr: -454> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f72930) adding con mode: crc 2024-05-30T16:20:55.902 INFO:tasks.workunit.client.0.smithi006.stderr: -453> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f72930) adding con mode: secure 2024-05-30T16:20:55.902 INFO:tasks.workunit.client.0.smithi006.stderr: -452> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f72930) adding con mode: crc 2024-05-30T16:20:55.902 INFO:tasks.workunit.client.0.smithi006.stderr: -451> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 AuthRegistry(0x565264f72930) adding con mode: secure 2024-05-30T16:20:55.902 INFO:tasks.workunit.client.0.smithi006.stderr: -450> 2024-05-30T16:20:55.708+0000 7fa7de543980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:55.902 INFO:tasks.workunit.client.0.smithi006.stderr: -449> 2024-05-30T16:20:55.708+0000 7fa7de543980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:55.902 INFO:tasks.workunit.client.0.smithi006.stderr: -448> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 asok(0x565264d67790) register_command rotate-key hook 0x565264f72a78 2024-05-30T16:20:55.903 INFO:tasks.workunit.client.0.smithi006.stderr: -447> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient: _reopen_session rank -1 2024-05-30T16:20:55.903 INFO:tasks.workunit.client.0.smithi006.stderr: -446> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient: _add_conns ranks=[1,0,2] 2024-05-30T16:20:55.903 INFO:tasks.workunit.client.0.smithi006.stderr: -445> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient(hunting): picked mon.noname-b con 0x565264e2d0b0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:20:55.903 INFO:tasks.workunit.client.0.smithi006.stderr: -444> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient(hunting): picked mon.noname-a con 0x565264f79580 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:20:55.903 INFO:tasks.workunit.client.0.smithi006.stderr: -443> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient(hunting): picked mon.noname-c con 0x565264fadb80 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:20:55.903 INFO:tasks.workunit.client.0.smithi006.stderr: -442> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:55.903 INFO:tasks.workunit.client.0.smithi006.stderr: -441> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:55.903 INFO:tasks.workunit.client.0.smithi006.stderr: -440> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:55.903 INFO:tasks.workunit.client.0.smithi006.stderr: -439> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient(hunting): _renew_subs 2024-05-30T16:20:55.903 INFO:tasks.workunit.client.0.smithi006.stderr: -438> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient(hunting): authenticate will time out at 2595.646644s 2024-05-30T16:20:55.903 INFO:tasks.workunit.client.0.smithi006.stderr: -437> 2024-05-30T16:20:55.708+0000 7fa7db853640 10 monclient(hunting): get_auth_request con 0x565264f79580 auth_method 0 2024-05-30T16:20:55.903 INFO:tasks.workunit.client.0.smithi006.stderr: -436> 2024-05-30T16:20:55.708+0000 7fa7db853640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:55.903 INFO:tasks.workunit.client.0.smithi006.stderr: -435> 2024-05-30T16:20:55.708+0000 7fa7db853640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:55.903 INFO:tasks.workunit.client.0.smithi006.stderr: -434> 2024-05-30T16:20:55.708+0000 7fa7db853640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:55.904 INFO:tasks.workunit.client.0.smithi006.stderr: -433> 2024-05-30T16:20:55.708+0000 7fa7db853640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:55.904 INFO:tasks.workunit.client.0.smithi006.stderr: -432> 2024-05-30T16:20:55.708+0000 7fa7db853640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:55.904 INFO:tasks.workunit.client.0.smithi006.stderr: -431> 2024-05-30T16:20:55.708+0000 7fa7db853640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:55.904 INFO:tasks.workunit.client.0.smithi006.stderr: -430> 2024-05-30T16:20:55.708+0000 7fa7dc855640 10 monclient(hunting): get_auth_request con 0x565264fadb80 auth_method 0 2024-05-30T16:20:55.904 INFO:tasks.workunit.client.0.smithi006.stderr: -429> 2024-05-30T16:20:55.708+0000 7fa7dc855640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:55.904 INFO:tasks.workunit.client.0.smithi006.stderr: -428> 2024-05-30T16:20:55.708+0000 7fa7dc855640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:55.904 INFO:tasks.workunit.client.0.smithi006.stderr: -427> 2024-05-30T16:20:55.708+0000 7fa7dc855640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:55.904 INFO:tasks.workunit.client.0.smithi006.stderr: -426> 2024-05-30T16:20:55.708+0000 7fa7dc054640 10 monclient(hunting): get_auth_request con 0x565264e2d0b0 auth_method 0 2024-05-30T16:20:55.904 INFO:tasks.workunit.client.0.smithi006.stderr: -425> 2024-05-30T16:20:55.708+0000 7fa7dc054640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:55.904 INFO:tasks.workunit.client.0.smithi006.stderr: -424> 2024-05-30T16:20:55.708+0000 7fa7dc054640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:55.904 INFO:tasks.workunit.client.0.smithi006.stderr: -423> 2024-05-30T16:20:55.708+0000 7fa7dc054640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:55.904 INFO:tasks.workunit.client.0.smithi006.stderr: -422> 2024-05-30T16:20:55.708+0000 7fa7dc855640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:55.904 INFO:tasks.workunit.client.0.smithi006.stderr: -421> 2024-05-30T16:20:55.708+0000 7fa7dc855640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:55.905 INFO:tasks.workunit.client.0.smithi006.stderr: -420> 2024-05-30T16:20:55.708+0000 7fa7dc855640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:55.905 INFO:tasks.workunit.client.0.smithi006.stderr: -419> 2024-05-30T16:20:55.708+0000 7fa7db853640 10 monclient(hunting): handle_auth_done global_id 6489 payload 824 2024-05-30T16:20:55.905 INFO:tasks.workunit.client.0.smithi006.stderr: -418> 2024-05-30T16:20:55.708+0000 7fa7db853640 10 monclient: _finish_hunting 0 2024-05-30T16:20:55.905 INFO:tasks.workunit.client.0.smithi006.stderr: -417> 2024-05-30T16:20:55.708+0000 7fa7db853640 1 monclient: found mon.noname-a 2024-05-30T16:20:55.905 INFO:tasks.workunit.client.0.smithi006.stderr: -416> 2024-05-30T16:20:55.708+0000 7fa7db853640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.6:3300/0 2024-05-30T16:20:55.905 INFO:tasks.workunit.client.0.smithi006.stderr: -415> 2024-05-30T16:20:55.708+0000 7fa6c8e09640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:55.905 INFO:tasks.workunit.client.0.smithi006.stderr: -414> 2024-05-30T16:20:55.708+0000 7fa6c8e09640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-30T16:20:55.905 INFO:tasks.workunit.client.0.smithi006.stderr: -413> 2024-05-30T16:20:55.708+0000 7fa6c8e09640 10 monclient: dump: 2024-05-30T16:20:55.905 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:55.905 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:55.905 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.905 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.905 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:55.906 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:55.906 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:55.906 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:55.906 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:55.906 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.906 INFO:tasks.workunit.client.0.smithi006.stderr: -412> 2024-05-30T16:20:55.708+0000 7fa6c8e09640 10 monclient: _finish_auth 0 2024-05-30T16:20:55.906 INFO:tasks.workunit.client.0.smithi006.stderr: -411> 2024-05-30T16:20:55.708+0000 7fa6c8e09640 10 monclient: _check_auth_tickets 2024-05-30T16:20:55.906 INFO:tasks.workunit.client.0.smithi006.stderr: -410> 2024-05-30T16:20:55.708+0000 7fa6c8e09640 10 monclient: handle_config config(0 keys) 2024-05-30T16:20:55.907 INFO:tasks.workunit.client.0.smithi006.stderr: -409> 2024-05-30T16:20:55.708+0000 7fa6c8e09640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:55.907 INFO:tasks.workunit.client.0.smithi006.stderr: -408> 2024-05-30T16:20:55.708+0000 7fa6c8e09640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-30T16:20:55.907 INFO:tasks.workunit.client.0.smithi006.stderr: -407> 2024-05-30T16:20:55.708+0000 7fa6c8e09640 10 monclient: dump: 2024-05-30T16:20:55.907 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:55.907 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:55.907 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.907 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.907 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:55.907 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:55.907 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:55.907 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:55.907 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:55.908 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.908 INFO:tasks.workunit.client.0.smithi006.stderr: -406> 2024-05-30T16:20:55.708+0000 7fa7de543980 5 monclient: authenticate success, global_id 6489 2024-05-30T16:20:55.908 INFO:tasks.workunit.client.0.smithi006.stderr: -405> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient: _renew_subs 2024-05-30T16:20:55.908 INFO:tasks.workunit.client.0.smithi006.stderr: -404> 2024-05-30T16:20:55.708+0000 7fa6ca60c640 4 set_mon_vals no callback set 2024-05-30T16:20:55.908 INFO:tasks.workunit.client.0.smithi006.stderr: -403> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:20:55.908 INFO:tasks.workunit.client.0.smithi006.stderr: -402> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient: _renew_subs 2024-05-30T16:20:55.908 INFO:tasks.workunit.client.0.smithi006.stderr: -401> 2024-05-30T16:20:55.708+0000 7fa7de543980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:20:55.908 INFO:tasks.workunit.client.0.smithi006.stderr: -400> 2024-05-30T16:20:55.708+0000 7fa7de543980 1 librados: init done 2024-05-30T16:20:55.908 INFO:tasks.workunit.client.0.smithi006.stderr: -399> 2024-05-30T16:20:55.712+0000 7fa6c8e09640 4 mgrc handle_mgr_map Got map version 5 2024-05-30T16:20:55.908 INFO:tasks.workunit.client.0.smithi006.stderr: -398> 2024-05-30T16:20:55.712+0000 7fa6c8e09640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:20:55.908 INFO:tasks.workunit.client.0.smithi006.stderr: -397> 2024-05-30T16:20:55.712+0000 7fa6c8e09640 4 mgrc reconnect Starting new session with [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:20:55.908 INFO:tasks.workunit.client.0.smithi006.stderr: -396> 2024-05-30T16:20:55.712+0000 7fa7dc054640 10 monclient: get_auth_request con 0x7fa6c406eb40 auth_method 0 2024-05-30T16:20:55.909 INFO:tasks.workunit.client.0.smithi006.stderr: -395> 2024-05-30T16:20:55.712+0000 7fa7dc855640 10 monclient: get_auth_request con 0x565264fb3b60 auth_method 0 2024-05-30T16:20:55.909 INFO:tasks.workunit.client.0.smithi006.stderr: -394> 2024-05-30T16:20:55.716+0000 7fa7dc054640 10 monclient: get_auth_request con 0x565264fbd470 auth_method 0 2024-05-30T16:20:55.909 INFO:tasks.workunit.client.0.smithi006.stderr: -393> 2024-05-30T16:20:55.716+0000 7fa7db853640 10 monclient: get_auth_request con 0x565264fc2600 auth_method 0 2024-05-30T16:20:55.909 INFO:tasks.workunit.client.0.smithi006.stderr: -392> 2024-05-30T16:20:55.720+0000 7fa7de543980 10 monclient: build_initial_monmap 2024-05-30T16:20:55.909 INFO:tasks.workunit.client.0.smithi006.stderr: -391> 2024-05-30T16:20:55.720+0000 7fa7de543980 1 build_initial for_mkfs: 0 2024-05-30T16:20:55.909 INFO:tasks.workunit.client.0.smithi006.stderr: -390> 2024-05-30T16:20:55.720+0000 7fa7de543980 10 monclient: monmap: 2024-05-30T16:20:55.909 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 0 2024-05-30T16:20:55.909 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-30T16:20:55.909 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 0.000000 2024-05-30T16:20:55.909 INFO:tasks.workunit.client.0.smithi006.stderr:created 0.000000 2024-05-30T16:20:55.909 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 0 (unknown) 2024-05-30T16:20:55.909 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:55.909 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.noname-a 2024-05-30T16:20:55.910 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-b 2024-05-30T16:20:55.910 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.noname-c 2024-05-30T16:20:55.910 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.910 INFO:tasks.workunit.client.0.smithi006.stderr: -389> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc6730) adding auth protocol: cephx 2024-05-30T16:20:55.910 INFO:tasks.workunit.client.0.smithi006.stderr: -388> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc6730) adding auth protocol: cephx 2024-05-30T16:20:55.910 INFO:tasks.workunit.client.0.smithi006.stderr: -387> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc6730) adding auth protocol: cephx 2024-05-30T16:20:55.910 INFO:tasks.workunit.client.0.smithi006.stderr: -386> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc6730) adding con mode: secure 2024-05-30T16:20:55.911 INFO:tasks.workunit.client.0.smithi006.stderr: -385> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc6730) adding con mode: crc 2024-05-30T16:20:55.911 INFO:tasks.workunit.client.0.smithi006.stderr: -384> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc6730) adding con mode: secure 2024-05-30T16:20:55.911 INFO:tasks.workunit.client.0.smithi006.stderr: -383> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc6730) adding con mode: crc 2024-05-30T16:20:55.911 INFO:tasks.workunit.client.0.smithi006.stderr: -382> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc6730) adding con mode: secure 2024-05-30T16:20:55.911 INFO:tasks.workunit.client.0.smithi006.stderr: -381> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc6730) adding con mode: crc 2024-05-30T16:20:55.911 INFO:tasks.workunit.client.0.smithi006.stderr: -380> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc6730) adding con mode: crc 2024-05-30T16:20:55.911 INFO:tasks.workunit.client.0.smithi006.stderr: -379> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc6730) adding con mode: secure 2024-05-30T16:20:55.911 INFO:tasks.workunit.client.0.smithi006.stderr: -378> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc6730) adding con mode: crc 2024-05-30T16:20:55.912 INFO:tasks.workunit.client.0.smithi006.stderr: -377> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc6730) adding con mode: secure 2024-05-30T16:20:55.912 INFO:tasks.workunit.client.0.smithi006.stderr: -376> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc6730) adding con mode: crc 2024-05-30T16:20:55.912 INFO:tasks.workunit.client.0.smithi006.stderr: -375> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc6730) adding con mode: secure 2024-05-30T16:20:55.912 INFO:tasks.workunit.client.0.smithi006.stderr: -374> 2024-05-30T16:20:55.720+0000 7fa7de543980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:55.912 INFO:tasks.workunit.client.0.smithi006.stderr: -373> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 asok(0x565264d67790) register_command objecter_requests cmddesc objecter_requests hook 0x565264fc9c60 EEXIST 2024-05-30T16:20:55.912 INFO:tasks.workunit.client.0.smithi006.stderr: -372> 2024-05-30T16:20:55.720+0000 7fa7de543980 10 monclient: init 2024-05-30T16:20:55.912 INFO:tasks.workunit.client.0.smithi006.stderr: -371> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc5628) adding auth protocol: cephx 2024-05-30T16:20:55.912 INFO:tasks.workunit.client.0.smithi006.stderr: -370> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc5628) adding auth protocol: cephx 2024-05-30T16:20:55.913 INFO:tasks.workunit.client.0.smithi006.stderr: -369> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc5628) adding auth protocol: cephx 2024-05-30T16:20:55.913 INFO:tasks.workunit.client.0.smithi006.stderr: -368> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc5628) adding con mode: secure 2024-05-30T16:20:55.913 INFO:tasks.workunit.client.0.smithi006.stderr: -367> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc5628) adding con mode: crc 2024-05-30T16:20:55.913 INFO:tasks.workunit.client.0.smithi006.stderr: -366> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc5628) adding con mode: secure 2024-05-30T16:20:55.913 INFO:tasks.workunit.client.0.smithi006.stderr: -365> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc5628) adding con mode: crc 2024-05-30T16:20:55.913 INFO:tasks.workunit.client.0.smithi006.stderr: -364> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc5628) adding con mode: secure 2024-05-30T16:20:55.913 INFO:tasks.workunit.client.0.smithi006.stderr: -363> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc5628) adding con mode: crc 2024-05-30T16:20:55.913 INFO:tasks.workunit.client.0.smithi006.stderr: -362> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc5628) adding con mode: crc 2024-05-30T16:20:55.913 INFO:tasks.workunit.client.0.smithi006.stderr: -361> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc5628) adding con mode: secure 2024-05-30T16:20:55.913 INFO:tasks.workunit.client.0.smithi006.stderr: -360> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc5628) adding con mode: crc 2024-05-30T16:20:55.914 INFO:tasks.workunit.client.0.smithi006.stderr: -359> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc5628) adding con mode: secure 2024-05-30T16:20:55.914 INFO:tasks.workunit.client.0.smithi006.stderr: -358> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc5628) adding con mode: crc 2024-05-30T16:20:55.914 INFO:tasks.workunit.client.0.smithi006.stderr: -357> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 AuthRegistry(0x565264fc5628) adding con mode: secure 2024-05-30T16:20:55.914 INFO:tasks.workunit.client.0.smithi006.stderr: -356> 2024-05-30T16:20:55.720+0000 7fa7de543980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:55.914 INFO:tasks.workunit.client.0.smithi006.stderr: -355> 2024-05-30T16:20:55.720+0000 7fa7de543980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:55.914 INFO:tasks.workunit.client.0.smithi006.stderr: -354> 2024-05-30T16:20:55.720+0000 7fa7de543980 5 asok(0x565264d67790) register_command rotate-key cmddesc rotate-key hook 0x565264fc5770 EEXIST 2024-05-30T16:20:55.914 INFO:tasks.workunit.client.0.smithi006.stderr: -353> 2024-05-30T16:20:55.720+0000 7fa7de543980 10 monclient: _reopen_session rank -1 2024-05-30T16:20:55.914 INFO:tasks.workunit.client.0.smithi006.stderr: -352> 2024-05-30T16:20:55.720+0000 7fa7de543980 10 monclient: _add_conns ranks=[2,0,1] 2024-05-30T16:20:55.914 INFO:tasks.workunit.client.0.smithi006.stderr: -351> 2024-05-30T16:20:55.720+0000 7fa7de543980 10 monclient(hunting): picked mon.noname-c con 0x565264fd01f0 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:20:55.914 INFO:tasks.workunit.client.0.smithi006.stderr: -350> 2024-05-30T16:20:55.720+0000 7fa7de543980 10 monclient(hunting): picked mon.noname-a con 0x565264fd0b20 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:20:55.915 INFO:tasks.workunit.client.0.smithi006.stderr: -349> 2024-05-30T16:20:55.720+0000 7fa7de543980 10 monclient(hunting): picked mon.noname-b con 0x565264fd05d0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:20:55.915 INFO:tasks.workunit.client.0.smithi006.stderr: -348> 2024-05-30T16:20:55.720+0000 7fa7de543980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:55.915 INFO:tasks.workunit.client.0.smithi006.stderr: -347> 2024-05-30T16:20:55.720+0000 7fa7de543980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:55.915 INFO:tasks.workunit.client.0.smithi006.stderr: -346> 2024-05-30T16:20:55.720+0000 7fa7de543980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:55.915 INFO:tasks.workunit.client.0.smithi006.stderr: -345> 2024-05-30T16:20:55.720+0000 7fa7de543980 10 monclient(hunting): _renew_subs 2024-05-30T16:20:55.915 INFO:tasks.workunit.client.0.smithi006.stderr: -344> 2024-05-30T16:20:55.720+0000 7fa7de543980 10 monclient(hunting): authenticate will time out at 2595.659499s 2024-05-30T16:20:55.915 INFO:tasks.workunit.client.0.smithi006.stderr: -343> 2024-05-30T16:20:55.720+0000 7fa7dc054640 10 monclient(hunting): get_auth_request con 0x565264fd01f0 auth_method 0 2024-05-30T16:20:55.915 INFO:tasks.workunit.client.0.smithi006.stderr: -342> 2024-05-30T16:20:55.720+0000 7fa7dc054640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:55.915 INFO:tasks.workunit.client.0.smithi006.stderr: -341> 2024-05-30T16:20:55.720+0000 7fa7dc054640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:55.915 INFO:tasks.workunit.client.0.smithi006.stderr: -340> 2024-05-30T16:20:55.720+0000 7fa7dc054640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:55.915 INFO:tasks.workunit.client.0.smithi006.stderr: -339> 2024-05-30T16:20:55.720+0000 7fa7db853640 10 monclient(hunting): get_auth_request con 0x565264fd0b20 auth_method 0 2024-05-30T16:20:55.916 INFO:tasks.workunit.client.0.smithi006.stderr: -338> 2024-05-30T16:20:55.720+0000 7fa7db853640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:55.916 INFO:tasks.workunit.client.0.smithi006.stderr: -337> 2024-05-30T16:20:55.720+0000 7fa7db853640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:55.916 INFO:tasks.workunit.client.0.smithi006.stderr: -336> 2024-05-30T16:20:55.720+0000 7fa7db853640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:55.916 INFO:tasks.workunit.client.0.smithi006.stderr: -335> 2024-05-30T16:20:55.720+0000 7fa7dc054640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:55.916 INFO:tasks.workunit.client.0.smithi006.stderr: -334> 2024-05-30T16:20:55.720+0000 7fa7dc054640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:55.916 INFO:tasks.workunit.client.0.smithi006.stderr: -333> 2024-05-30T16:20:55.720+0000 7fa7dc054640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:55.916 INFO:tasks.workunit.client.0.smithi006.stderr: -332> 2024-05-30T16:20:55.720+0000 7fa7db853640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:55.916 INFO:tasks.workunit.client.0.smithi006.stderr: -331> 2024-05-30T16:20:55.720+0000 7fa7db853640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:55.916 INFO:tasks.workunit.client.0.smithi006.stderr: -330> 2024-05-30T16:20:55.720+0000 7fa7db853640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:55.917 INFO:tasks.workunit.client.0.smithi006.stderr: -329> 2024-05-30T16:20:55.724+0000 7fa7dc855640 10 monclient(hunting): get_auth_request con 0x565264fd05d0 auth_method 0 2024-05-30T16:20:55.917 INFO:tasks.workunit.client.0.smithi006.stderr: -328> 2024-05-30T16:20:55.724+0000 7fa7dc855640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:55.917 INFO:tasks.workunit.client.0.smithi006.stderr: -327> 2024-05-30T16:20:55.724+0000 7fa7dc855640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:55.917 INFO:tasks.workunit.client.0.smithi006.stderr: -326> 2024-05-30T16:20:55.724+0000 7fa7dc855640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:55.917 INFO:tasks.workunit.client.0.smithi006.stderr: -325> 2024-05-30T16:20:55.724+0000 7fa7db853640 10 monclient(hunting): handle_auth_done global_id 6492 payload 824 2024-05-30T16:20:55.917 INFO:tasks.workunit.client.0.smithi006.stderr: -324> 2024-05-30T16:20:55.724+0000 7fa7db853640 10 monclient: _finish_hunting 0 2024-05-30T16:20:55.917 INFO:tasks.workunit.client.0.smithi006.stderr: -323> 2024-05-30T16:20:55.724+0000 7fa7db853640 1 monclient: found mon.noname-a 2024-05-30T16:20:55.917 INFO:tasks.workunit.client.0.smithi006.stderr: -322> 2024-05-30T16:20:55.724+0000 7fa7db853640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.6:3300/0 2024-05-30T16:20:55.917 INFO:tasks.workunit.client.0.smithi006.stderr: -321> 2024-05-30T16:20:55.724+0000 7fa6c1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:55.918 INFO:tasks.workunit.client.0.smithi006.stderr: -320> 2024-05-30T16:20:55.724+0000 7fa6c1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-30T16:20:55.918 INFO:tasks.workunit.client.0.smithi006.stderr: -319> 2024-05-30T16:20:55.724+0000 7fa6c1ffb640 10 monclient: dump: 2024-05-30T16:20:55.918 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:55.918 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:55.918 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.918 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.918 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:55.918 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:55.918 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:55.918 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:55.918 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:55.919 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.919 INFO:tasks.workunit.client.0.smithi006.stderr: -318> 2024-05-30T16:20:55.724+0000 7fa6c1ffb640 10 monclient: _finish_auth 0 2024-05-30T16:20:55.919 INFO:tasks.workunit.client.0.smithi006.stderr: -317> 2024-05-30T16:20:55.724+0000 7fa6c1ffb640 10 monclient: _check_auth_tickets 2024-05-30T16:20:55.919 INFO:tasks.workunit.client.0.smithi006.stderr: -316> 2024-05-30T16:20:55.724+0000 7fa6c1ffb640 10 monclient: handle_config config(0 keys) 2024-05-30T16:20:55.919 INFO:tasks.workunit.client.0.smithi006.stderr: -315> 2024-05-30T16:20:55.724+0000 7fa6c1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:55.919 INFO:tasks.workunit.client.0.smithi006.stderr: -314> 2024-05-30T16:20:55.724+0000 7fa6c1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-30T16:20:55.919 INFO:tasks.workunit.client.0.smithi006.stderr: -313> 2024-05-30T16:20:55.724+0000 7fa6c1ffb640 10 monclient: dump: 2024-05-30T16:20:55.919 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:55.919 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:55.919 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.919 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.919 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:55.919 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:55.920 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:55.920 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:55.920 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:55.920 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.920 INFO:tasks.workunit.client.0.smithi006.stderr: -312> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 monclient: authenticate success, global_id 6492 2024-05-30T16:20:55.920 INFO:tasks.workunit.client.0.smithi006.stderr: -311> 2024-05-30T16:20:55.724+0000 7fa7dd056640 4 set_mon_vals no callback set 2024-05-30T16:20:55.920 INFO:tasks.workunit.client.0.smithi006.stderr: -310> 2024-05-30T16:20:55.724+0000 7fa7de543980 10 monclient: _renew_subs 2024-05-30T16:20:55.920 INFO:tasks.workunit.client.0.smithi006.stderr: -309> 2024-05-30T16:20:55.724+0000 7fa7de543980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:20:55.920 INFO:tasks.workunit.client.0.smithi006.stderr: -308> 2024-05-30T16:20:55.724+0000 7fa7de543980 10 monclient: _renew_subs 2024-05-30T16:20:55.920 INFO:tasks.workunit.client.0.smithi006.stderr: -307> 2024-05-30T16:20:55.724+0000 7fa7de543980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:20:55.920 INFO:tasks.workunit.client.0.smithi006.stderr: -306> 2024-05-30T16:20:55.724+0000 7fa6c1ffb640 4 mgrc handle_mgr_map Got map version 5 2024-05-30T16:20:55.920 INFO:tasks.workunit.client.0.smithi006.stderr: -305> 2024-05-30T16:20:55.724+0000 7fa6c1ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:20:55.920 INFO:tasks.workunit.client.0.smithi006.stderr: -304> 2024-05-30T16:20:55.724+0000 7fa6c1ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:20:55.921 INFO:tasks.workunit.client.0.smithi006.stderr: -303> 2024-05-30T16:20:55.724+0000 7fa7de543980 10 monclient: get_monmap_and_config 2024-05-30T16:20:55.921 INFO:tasks.workunit.client.0.smithi006.stderr: -302> 2024-05-30T16:20:55.724+0000 7fa7de543980 10 monclient: build_initial_monmap 2024-05-30T16:20:55.921 INFO:tasks.workunit.client.0.smithi006.stderr: -301> 2024-05-30T16:20:55.724+0000 7fa7de543980 1 build_initial for_mkfs: 0 2024-05-30T16:20:55.921 INFO:tasks.workunit.client.0.smithi006.stderr: -300> 2024-05-30T16:20:55.724+0000 7fa7de543980 10 monclient: monmap: 2024-05-30T16:20:55.921 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 0 2024-05-30T16:20:55.921 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-30T16:20:55.921 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 0.000000 2024-05-30T16:20:55.921 INFO:tasks.workunit.client.0.smithi006.stderr:created 0.000000 2024-05-30T16:20:55.921 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 0 (unknown) 2024-05-30T16:20:55.921 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:55.921 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.noname-a 2024-05-30T16:20:55.921 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-b 2024-05-30T16:20:55.921 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.noname-c 2024-05-30T16:20:55.922 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.922 INFO:tasks.workunit.client.0.smithi006.stderr: -299> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x56526500dd30) adding auth protocol: cephx 2024-05-30T16:20:55.922 INFO:tasks.workunit.client.0.smithi006.stderr: -298> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x56526500dd30) adding auth protocol: cephx 2024-05-30T16:20:55.922 INFO:tasks.workunit.client.0.smithi006.stderr: -297> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x56526500dd30) adding auth protocol: cephx 2024-05-30T16:20:55.922 INFO:tasks.workunit.client.0.smithi006.stderr: -296> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x56526500dd30) adding con mode: secure 2024-05-30T16:20:55.922 INFO:tasks.workunit.client.0.smithi006.stderr: -295> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x56526500dd30) adding con mode: crc 2024-05-30T16:20:55.922 INFO:tasks.workunit.client.0.smithi006.stderr: -294> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x56526500dd30) adding con mode: secure 2024-05-30T16:20:55.922 INFO:tasks.workunit.client.0.smithi006.stderr: -293> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x56526500dd30) adding con mode: crc 2024-05-30T16:20:55.923 INFO:tasks.workunit.client.0.smithi006.stderr: -292> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x56526500dd30) adding con mode: secure 2024-05-30T16:20:55.923 INFO:tasks.workunit.client.0.smithi006.stderr: -291> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x56526500dd30) adding con mode: crc 2024-05-30T16:20:55.923 INFO:tasks.workunit.client.0.smithi006.stderr: -290> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x56526500dd30) adding con mode: crc 2024-05-30T16:20:55.923 INFO:tasks.workunit.client.0.smithi006.stderr: -289> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x56526500dd30) adding con mode: secure 2024-05-30T16:20:55.923 INFO:tasks.workunit.client.0.smithi006.stderr: -288> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x56526500dd30) adding con mode: crc 2024-05-30T16:20:55.923 INFO:tasks.workunit.client.0.smithi006.stderr: -287> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x56526500dd30) adding con mode: secure 2024-05-30T16:20:55.923 INFO:tasks.workunit.client.0.smithi006.stderr: -286> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x56526500dd30) adding con mode: crc 2024-05-30T16:20:55.923 INFO:tasks.workunit.client.0.smithi006.stderr: -285> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x56526500dd30) adding con mode: secure 2024-05-30T16:20:55.923 INFO:tasks.workunit.client.0.smithi006.stderr: -284> 2024-05-30T16:20:55.724+0000 7fa7dc054640 10 monclient: get_auth_request con 0x7fa6b406dcf0 auth_method 0 2024-05-30T16:20:55.923 INFO:tasks.workunit.client.0.smithi006.stderr: -283> 2024-05-30T16:20:55.724+0000 7fa7de543980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:55.923 INFO:tasks.workunit.client.0.smithi006.stderr: -282> 2024-05-30T16:20:55.724+0000 7fa7de543980 10 monclient: init 2024-05-30T16:20:55.923 INFO:tasks.workunit.client.0.smithi006.stderr: -281> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c1da0) adding auth protocol: cephx 2024-05-30T16:20:55.923 INFO:tasks.workunit.client.0.smithi006.stderr: -280> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c1da0) adding auth protocol: cephx 2024-05-30T16:20:55.924 INFO:tasks.workunit.client.0.smithi006.stderr: -279> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c1da0) adding auth protocol: cephx 2024-05-30T16:20:55.924 INFO:tasks.workunit.client.0.smithi006.stderr: -278> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c1da0) adding con mode: secure 2024-05-30T16:20:55.924 INFO:tasks.workunit.client.0.smithi006.stderr: -277> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c1da0) adding con mode: crc 2024-05-30T16:20:55.924 INFO:tasks.workunit.client.0.smithi006.stderr: -276> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c1da0) adding con mode: secure 2024-05-30T16:20:55.924 INFO:tasks.workunit.client.0.smithi006.stderr: -275> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c1da0) adding con mode: crc 2024-05-30T16:20:55.924 INFO:tasks.workunit.client.0.smithi006.stderr: -274> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c1da0) adding con mode: secure 2024-05-30T16:20:55.924 INFO:tasks.workunit.client.0.smithi006.stderr: -273> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c1da0) adding con mode: crc 2024-05-30T16:20:55.924 INFO:tasks.workunit.client.0.smithi006.stderr: -272> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c1da0) adding con mode: crc 2024-05-30T16:20:55.924 INFO:tasks.workunit.client.0.smithi006.stderr: -271> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c1da0) adding con mode: secure 2024-05-30T16:20:55.924 INFO:tasks.workunit.client.0.smithi006.stderr: -270> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c1da0) adding con mode: crc 2024-05-30T16:20:55.924 INFO:tasks.workunit.client.0.smithi006.stderr: -269> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c1da0) adding con mode: secure 2024-05-30T16:20:55.924 INFO:tasks.workunit.client.0.smithi006.stderr: -268> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c1da0) adding con mode: crc 2024-05-30T16:20:55.924 INFO:tasks.workunit.client.0.smithi006.stderr: -267> 2024-05-30T16:20:55.724+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c1da0) adding con mode: secure 2024-05-30T16:20:55.925 INFO:tasks.workunit.client.0.smithi006.stderr: -266> 2024-05-30T16:20:55.724+0000 7fa7de543980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:55.925 INFO:tasks.workunit.client.0.smithi006.stderr: -265> 2024-05-30T16:20:55.728+0000 7fa7de543980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:55.925 INFO:tasks.workunit.client.0.smithi006.stderr: -264> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 asok(0x565264d67790) register_command rotate-key cmddesc rotate-key hook 0x7ffef71c1ee8 EEXIST 2024-05-30T16:20:55.925 INFO:tasks.workunit.client.0.smithi006.stderr: -263> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient: _reopen_session rank -1 2024-05-30T16:20:55.925 INFO:tasks.workunit.client.0.smithi006.stderr: -262> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient: _add_conns ranks=[2,0,1] 2024-05-30T16:20:55.925 INFO:tasks.workunit.client.0.smithi006.stderr: -261> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): picked mon.noname-c con 0x5652650421c0 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:20:55.925 INFO:tasks.workunit.client.0.smithi006.stderr: -260> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): picked mon.noname-a con 0x56526503ded0 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:20:55.925 INFO:tasks.workunit.client.0.smithi006.stderr: -259> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): picked mon.noname-b con 0x56526503c790 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:20:55.925 INFO:tasks.workunit.client.0.smithi006.stderr: -258> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:55.925 INFO:tasks.workunit.client.0.smithi006.stderr: -257> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:55.925 INFO:tasks.workunit.client.0.smithi006.stderr: -256> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:55.925 INFO:tasks.workunit.client.0.smithi006.stderr: -255> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): _renew_subs 2024-05-30T16:20:55.925 INFO:tasks.workunit.client.0.smithi006.stderr: -254> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): authenticate will time out at 2595.664196s 2024-05-30T16:20:55.925 INFO:tasks.workunit.client.0.smithi006.stderr: -253> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): get_auth_request con 0x5652650421c0 auth_method 0 2024-05-30T16:20:55.926 INFO:tasks.workunit.client.0.smithi006.stderr: -252> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:55.926 INFO:tasks.workunit.client.0.smithi006.stderr: -251> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:55.926 INFO:tasks.workunit.client.0.smithi006.stderr: -250> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:55.926 INFO:tasks.workunit.client.0.smithi006.stderr: -249> 2024-05-30T16:20:55.728+0000 7fa7db853640 10 monclient(hunting): get_auth_request con 0x56526503ded0 auth_method 0 2024-05-30T16:20:55.926 INFO:tasks.workunit.client.0.smithi006.stderr: -248> 2024-05-30T16:20:55.728+0000 7fa7db853640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:55.926 INFO:tasks.workunit.client.0.smithi006.stderr: -247> 2024-05-30T16:20:55.728+0000 7fa7db853640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:55.926 INFO:tasks.workunit.client.0.smithi006.stderr: -246> 2024-05-30T16:20:55.728+0000 7fa7db853640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:55.926 INFO:tasks.workunit.client.0.smithi006.stderr: -245> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:55.926 INFO:tasks.workunit.client.0.smithi006.stderr: -244> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:55.926 INFO:tasks.workunit.client.0.smithi006.stderr: -243> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:55.926 INFO:tasks.workunit.client.0.smithi006.stderr: -242> 2024-05-30T16:20:55.728+0000 7fa7db853640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:55.926 INFO:tasks.workunit.client.0.smithi006.stderr: -241> 2024-05-30T16:20:55.728+0000 7fa7db853640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:55.926 INFO:tasks.workunit.client.0.smithi006.stderr: -240> 2024-05-30T16:20:55.728+0000 7fa7db853640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:55.926 INFO:tasks.workunit.client.0.smithi006.stderr: -239> 2024-05-30T16:20:55.728+0000 7fa7dc855640 10 monclient(hunting): get_auth_request con 0x56526503c790 auth_method 0 2024-05-30T16:20:55.927 INFO:tasks.workunit.client.0.smithi006.stderr: -238> 2024-05-30T16:20:55.728+0000 7fa7dc855640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:55.927 INFO:tasks.workunit.client.0.smithi006.stderr: -237> 2024-05-30T16:20:55.728+0000 7fa7dc855640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:55.927 INFO:tasks.workunit.client.0.smithi006.stderr: -236> 2024-05-30T16:20:55.728+0000 7fa7dc855640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:55.927 INFO:tasks.workunit.client.0.smithi006.stderr: -235> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): handle_auth_done global_id 6212 payload 274 2024-05-30T16:20:55.927 INFO:tasks.workunit.client.0.smithi006.stderr: -234> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient: _finish_hunting 0 2024-05-30T16:20:55.927 INFO:tasks.workunit.client.0.smithi006.stderr: -233> 2024-05-30T16:20:55.728+0000 7fa7dc054640 1 monclient: found mon.noname-c 2024-05-30T16:20:55.927 INFO:tasks.workunit.client.0.smithi006.stderr: -232> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.6:3301/0 2024-05-30T16:20:55.927 INFO:tasks.workunit.client.0.smithi006.stderr: -231> 2024-05-30T16:20:55.728+0000 7fa6b37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:55.927 INFO:tasks.workunit.client.0.smithi006.stderr: -230> 2024-05-30T16:20:55.728+0000 7fa6b37fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-30T16:20:55.927 INFO:tasks.workunit.client.0.smithi006.stderr: -229> 2024-05-30T16:20:55.728+0000 7fa6b37fe640 10 monclient: dump: 2024-05-30T16:20:55.927 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:55.927 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:55.927 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.927 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.928 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:55.928 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:55.928 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:55.928 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:55.928 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:55.928 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.928 INFO:tasks.workunit.client.0.smithi006.stderr: -228> 2024-05-30T16:20:55.728+0000 7fa6b37fe640 10 monclient: _finish_auth 0 2024-05-30T16:20:55.930 INFO:tasks.workunit.client.0.smithi006.stderr: -227> 2024-05-30T16:20:55.728+0000 7fa6b37fe640 10 monclient: _check_auth_tickets 2024-05-30T16:20:55.930 INFO:tasks.workunit.client.0.smithi006.stderr: -226> 2024-05-30T16:20:55.728+0000 7fa6b37fe640 10 monclient: handle_config config(0 keys) 2024-05-30T16:20:55.930 INFO:tasks.workunit.client.0.smithi006.stderr: -225> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 monclient: authenticate success, global_id 6212 2024-05-30T16:20:55.930 INFO:tasks.workunit.client.0.smithi006.stderr: -224> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient: get_monmap_and_config success 2024-05-30T16:20:55.930 INFO:tasks.workunit.client.0.smithi006.stderr: -223> 2024-05-30T16:20:55.728+0000 7fa7de543980 4 set_mon_vals no callback set 2024-05-30T16:20:55.930 INFO:tasks.workunit.client.0.smithi006.stderr: -222> 2024-05-30T16:20:55.728+0000 7fa6b37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:55.930 INFO:tasks.workunit.client.0.smithi006.stderr: -221> 2024-05-30T16:20:55.728+0000 7fa6b37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-30T16:20:55.930 INFO:tasks.workunit.client.0.smithi006.stderr: -220> 2024-05-30T16:20:55.728+0000 7fa6b37fe640 10 monclient: dump: 2024-05-30T16:20:55.930 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:55.930 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:55.930 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.930 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.930 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:55.931 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:55.931 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:55.931 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:55.931 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:55.931 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.931 INFO:tasks.workunit.client.0.smithi006.stderr: -219> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient: shutdown 2024-05-30T16:20:55.931 INFO:tasks.workunit.client.0.smithi006.stderr: -218> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient: build_initial_monmap 2024-05-30T16:20:55.931 INFO:tasks.workunit.client.0.smithi006.stderr: -217> 2024-05-30T16:20:55.728+0000 7fa7de543980 1 build_initial for_mkfs: 0 2024-05-30T16:20:55.931 INFO:tasks.workunit.client.0.smithi006.stderr: -216> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient: monmap: 2024-05-30T16:20:55.931 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 0 2024-05-30T16:20:55.931 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-30T16:20:55.931 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 0.000000 2024-05-30T16:20:55.931 INFO:tasks.workunit.client.0.smithi006.stderr:created 0.000000 2024-05-30T16:20:55.931 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 0 (unknown) 2024-05-30T16:20:55.932 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:55.932 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.noname-a 2024-05-30T16:20:55.932 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-b 2024-05-30T16:20:55.932 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.noname-c 2024-05-30T16:20:55.932 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.932 INFO:tasks.workunit.client.0.smithi006.stderr: -215> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x56526500dce0) adding auth protocol: cephx 2024-05-30T16:20:55.932 INFO:tasks.workunit.client.0.smithi006.stderr: -214> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x56526500dce0) adding auth protocol: cephx 2024-05-30T16:20:55.932 INFO:tasks.workunit.client.0.smithi006.stderr: -213> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x56526500dce0) adding auth protocol: cephx 2024-05-30T16:20:55.932 INFO:tasks.workunit.client.0.smithi006.stderr: -212> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x56526500dce0) adding con mode: secure 2024-05-30T16:20:55.932 INFO:tasks.workunit.client.0.smithi006.stderr: -211> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x56526500dce0) adding con mode: crc 2024-05-30T16:20:55.932 INFO:tasks.workunit.client.0.smithi006.stderr: -210> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x56526500dce0) adding con mode: secure 2024-05-30T16:20:55.932 INFO:tasks.workunit.client.0.smithi006.stderr: -209> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x56526500dce0) adding con mode: crc 2024-05-30T16:20:55.932 INFO:tasks.workunit.client.0.smithi006.stderr: -208> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x56526500dce0) adding con mode: secure 2024-05-30T16:20:55.932 INFO:tasks.workunit.client.0.smithi006.stderr: -207> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x56526500dce0) adding con mode: crc 2024-05-30T16:20:55.933 INFO:tasks.workunit.client.0.smithi006.stderr: -206> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x56526500dce0) adding con mode: crc 2024-05-30T16:20:55.933 INFO:tasks.workunit.client.0.smithi006.stderr: -205> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x56526500dce0) adding con mode: secure 2024-05-30T16:20:55.933 INFO:tasks.workunit.client.0.smithi006.stderr: -204> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x56526500dce0) adding con mode: crc 2024-05-30T16:20:55.933 INFO:tasks.workunit.client.0.smithi006.stderr: -203> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x56526500dce0) adding con mode: secure 2024-05-30T16:20:55.933 INFO:tasks.workunit.client.0.smithi006.stderr: -202> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x56526500dce0) adding con mode: crc 2024-05-30T16:20:55.933 INFO:tasks.workunit.client.0.smithi006.stderr: -201> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x56526500dce0) adding con mode: secure 2024-05-30T16:20:55.933 INFO:tasks.workunit.client.0.smithi006.stderr: -200> 2024-05-30T16:20:55.728+0000 7fa7de543980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:55.933 INFO:tasks.workunit.client.0.smithi006.stderr: -199> 2024-05-30T16:20:55.728+0000 7fa7de543980 1 librados: starting msgr at 2024-05-30T16:20:55.933 INFO:tasks.workunit.client.0.smithi006.stderr: -198> 2024-05-30T16:20:55.728+0000 7fa7de543980 1 librados: starting objecter 2024-05-30T16:20:55.934 INFO:tasks.workunit.client.0.smithi006.stderr: -197> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 asok(0x565264d67790) register_command objecter_requests cmddesc objecter_requests hook 0x565265012cb0 EEXIST 2024-05-30T16:20:55.934 INFO:tasks.workunit.client.0.smithi006.stderr: -196> 2024-05-30T16:20:55.728+0000 7fa7de543980 1 librados: setting wanted keys 2024-05-30T16:20:55.934 INFO:tasks.workunit.client.0.smithi006.stderr: -195> 2024-05-30T16:20:55.728+0000 7fa7de543980 1 librados: calling monclient init 2024-05-30T16:20:55.934 INFO:tasks.workunit.client.0.smithi006.stderr: -194> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient: init 2024-05-30T16:20:55.934 INFO:tasks.workunit.client.0.smithi006.stderr: -193> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x565265005c50) adding auth protocol: cephx 2024-05-30T16:20:55.934 INFO:tasks.workunit.client.0.smithi006.stderr: -192> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x565265005c50) adding auth protocol: cephx 2024-05-30T16:20:55.934 INFO:tasks.workunit.client.0.smithi006.stderr: -191> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x565265005c50) adding auth protocol: cephx 2024-05-30T16:20:55.934 INFO:tasks.workunit.client.0.smithi006.stderr: -190> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x565265005c50) adding con mode: secure 2024-05-30T16:20:55.934 INFO:tasks.workunit.client.0.smithi006.stderr: -189> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x565265005c50) adding con mode: crc 2024-05-30T16:20:55.934 INFO:tasks.workunit.client.0.smithi006.stderr: -188> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x565265005c50) adding con mode: secure 2024-05-30T16:20:55.934 INFO:tasks.workunit.client.0.smithi006.stderr: -187> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x565265005c50) adding con mode: crc 2024-05-30T16:20:55.935 INFO:tasks.workunit.client.0.smithi006.stderr: -186> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x565265005c50) adding con mode: secure 2024-05-30T16:20:55.935 INFO:tasks.workunit.client.0.smithi006.stderr: -185> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x565265005c50) adding con mode: crc 2024-05-30T16:20:55.935 INFO:tasks.workunit.client.0.smithi006.stderr: -184> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x565265005c50) adding con mode: crc 2024-05-30T16:20:55.935 INFO:tasks.workunit.client.0.smithi006.stderr: -183> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x565265005c50) adding con mode: secure 2024-05-30T16:20:55.935 INFO:tasks.workunit.client.0.smithi006.stderr: -182> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x565265005c50) adding con mode: crc 2024-05-30T16:20:55.935 INFO:tasks.workunit.client.0.smithi006.stderr: -181> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x565265005c50) adding con mode: secure 2024-05-30T16:20:55.935 INFO:tasks.workunit.client.0.smithi006.stderr: -180> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x565265005c50) adding con mode: crc 2024-05-30T16:20:55.935 INFO:tasks.workunit.client.0.smithi006.stderr: -179> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 AuthRegistry(0x565265005c50) adding con mode: secure 2024-05-30T16:20:55.935 INFO:tasks.workunit.client.0.smithi006.stderr: -178> 2024-05-30T16:20:55.728+0000 7fa7de543980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:55.935 INFO:tasks.workunit.client.0.smithi006.stderr: -177> 2024-05-30T16:20:55.728+0000 7fa7de543980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:55.935 INFO:tasks.workunit.client.0.smithi006.stderr: -176> 2024-05-30T16:20:55.728+0000 7fa7de543980 5 asok(0x565264d67790) register_command rotate-key cmddesc rotate-key hook 0x565265005d98 EEXIST 2024-05-30T16:20:55.935 INFO:tasks.workunit.client.0.smithi006.stderr: -175> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient: _reopen_session rank -1 2024-05-30T16:20:55.936 INFO:tasks.workunit.client.0.smithi006.stderr: -174> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient: _add_conns ranks=[2,1,0] 2024-05-30T16:20:55.936 INFO:tasks.workunit.client.0.smithi006.stderr: -173> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): picked mon.noname-c con 0x56526503c790 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:20:55.936 INFO:tasks.workunit.client.0.smithi006.stderr: -172> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): picked mon.noname-b con 0x56526503ded0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:20:55.936 INFO:tasks.workunit.client.0.smithi006.stderr: -171> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): picked mon.noname-a con 0x5652650421c0 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:20:55.936 INFO:tasks.workunit.client.0.smithi006.stderr: -170> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:55.936 INFO:tasks.workunit.client.0.smithi006.stderr: -169> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:55.936 INFO:tasks.workunit.client.0.smithi006.stderr: -168> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): start opening mon connection 2024-05-30T16:20:55.936 INFO:tasks.workunit.client.0.smithi006.stderr: -167> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): _renew_subs 2024-05-30T16:20:55.936 INFO:tasks.workunit.client.0.smithi006.stderr: -166> 2024-05-30T16:20:55.728+0000 7fa7de543980 10 monclient(hunting): authenticate will time out at 2595.667336s 2024-05-30T16:20:55.936 INFO:tasks.workunit.client.0.smithi006.stderr: -165> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): get_auth_request con 0x56526503c790 auth_method 0 2024-05-30T16:20:55.936 INFO:tasks.workunit.client.0.smithi006.stderr: -164> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:55.936 INFO:tasks.workunit.client.0.smithi006.stderr: -163> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:55.936 INFO:tasks.workunit.client.0.smithi006.stderr: -162> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:55.937 INFO:tasks.workunit.client.0.smithi006.stderr: -161> 2024-05-30T16:20:55.728+0000 7fa7dc855640 10 monclient(hunting): get_auth_request con 0x5652650421c0 auth_method 0 2024-05-30T16:20:55.937 INFO:tasks.workunit.client.0.smithi006.stderr: -160> 2024-05-30T16:20:55.728+0000 7fa7dc855640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:55.937 INFO:tasks.workunit.client.0.smithi006.stderr: -159> 2024-05-30T16:20:55.728+0000 7fa7dc855640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:55.937 INFO:tasks.workunit.client.0.smithi006.stderr: -158> 2024-05-30T16:20:55.728+0000 7fa7dc855640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:55.937 INFO:tasks.workunit.client.0.smithi006.stderr: -157> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:55.937 INFO:tasks.workunit.client.0.smithi006.stderr: -156> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:55.937 INFO:tasks.workunit.client.0.smithi006.stderr: -155> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:55.937 INFO:tasks.workunit.client.0.smithi006.stderr: -154> 2024-05-30T16:20:55.728+0000 7fa7dc855640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:20:55.938 INFO:tasks.workunit.client.0.smithi006.stderr: -153> 2024-05-30T16:20:55.728+0000 7fa7dc855640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:20:55.938 INFO:tasks.workunit.client.0.smithi006.stderr: -152> 2024-05-30T16:20:55.728+0000 7fa7dc855640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:20:55.938 INFO:tasks.workunit.client.0.smithi006.stderr: -151> 2024-05-30T16:20:55.728+0000 7fa7db853640 10 monclient(hunting): get_auth_request con 0x56526503ded0 auth_method 0 2024-05-30T16:20:55.938 INFO:tasks.workunit.client.0.smithi006.stderr: -150> 2024-05-30T16:20:55.728+0000 7fa7db853640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:20:55.938 INFO:tasks.workunit.client.0.smithi006.stderr: -149> 2024-05-30T16:20:55.728+0000 7fa7db853640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:20:55.938 INFO:tasks.workunit.client.0.smithi006.stderr: -148> 2024-05-30T16:20:55.728+0000 7fa7db853640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:20:55.938 INFO:tasks.workunit.client.0.smithi006.stderr: -147> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient(hunting): handle_auth_done global_id 6215 payload 824 2024-05-30T16:20:55.938 INFO:tasks.workunit.client.0.smithi006.stderr: -146> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient: _finish_hunting 0 2024-05-30T16:20:55.938 INFO:tasks.workunit.client.0.smithi006.stderr: -145> 2024-05-30T16:20:55.728+0000 7fa7dc054640 1 monclient: found mon.noname-c 2024-05-30T16:20:55.938 INFO:tasks.workunit.client.0.smithi006.stderr: -144> 2024-05-30T16:20:55.728+0000 7fa7dc054640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.6:3301/0 2024-05-30T16:20:55.939 INFO:tasks.workunit.client.0.smithi006.stderr: -143> 2024-05-30T16:20:55.732+0000 7fa6b1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:55.939 INFO:tasks.workunit.client.0.smithi006.stderr: -142> 2024-05-30T16:20:55.732+0000 7fa6b1ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-30T16:20:55.939 INFO:tasks.workunit.client.0.smithi006.stderr: -141> 2024-05-30T16:20:55.732+0000 7fa6b1ffb640 10 monclient: dump: 2024-05-30T16:20:55.939 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:55.939 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:55.939 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.939 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.940 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:55.940 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:55.940 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:55.940 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:55.940 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:55.940 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.941 INFO:tasks.workunit.client.0.smithi006.stderr: -140> 2024-05-30T16:20:55.732+0000 7fa6b1ffb640 10 monclient: _finish_auth 0 2024-05-30T16:20:55.941 INFO:tasks.workunit.client.0.smithi006.stderr: -139> 2024-05-30T16:20:55.732+0000 7fa6b1ffb640 10 monclient: _check_auth_tickets 2024-05-30T16:20:55.941 INFO:tasks.workunit.client.0.smithi006.stderr: -138> 2024-05-30T16:20:55.732+0000 7fa6b1ffb640 10 monclient: handle_config config(0 keys) 2024-05-30T16:20:55.941 INFO:tasks.workunit.client.0.smithi006.stderr: -137> 2024-05-30T16:20:55.732+0000 7fa6b1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:20:55.942 INFO:tasks.workunit.client.0.smithi006.stderr: -136> 2024-05-30T16:20:55.732+0000 7fa6b1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-30T16:20:55.942 INFO:tasks.workunit.client.0.smithi006.stderr: -135> 2024-05-30T16:20:55.732+0000 7fa6b1ffb640 10 monclient: dump: 2024-05-30T16:20:55.942 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:20:55.942 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:20:55.942 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.942 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:20:55.942 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:20:55.942 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:20:55.942 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:20:55.942 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:20:55.942 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:20:55.942 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.942 INFO:tasks.workunit.client.0.smithi006.stderr: -134> 2024-05-30T16:20:55.732+0000 7fa7de543980 5 monclient: authenticate success, global_id 6215 2024-05-30T16:20:55.943 INFO:tasks.workunit.client.0.smithi006.stderr: -133> 2024-05-30T16:20:55.732+0000 7fa6b37fe640 4 set_mon_vals no callback set 2024-05-30T16:20:55.943 INFO:tasks.workunit.client.0.smithi006.stderr: -132> 2024-05-30T16:20:55.732+0000 7fa7de543980 10 monclient: _renew_subs 2024-05-30T16:20:55.943 INFO:tasks.workunit.client.0.smithi006.stderr: -131> 2024-05-30T16:20:55.732+0000 7fa7de543980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.6:3301/0 2024-05-30T16:20:55.943 INFO:tasks.workunit.client.0.smithi006.stderr: -130> 2024-05-30T16:20:55.732+0000 7fa7de543980 10 monclient: _renew_subs 2024-05-30T16:20:55.943 INFO:tasks.workunit.client.0.smithi006.stderr: -129> 2024-05-30T16:20:55.732+0000 7fa7de543980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.6:3301/0 2024-05-30T16:20:55.943 INFO:tasks.workunit.client.0.smithi006.stderr: -128> 2024-05-30T16:20:55.732+0000 7fa7de543980 1 librados: init done 2024-05-30T16:20:55.943 INFO:tasks.workunit.client.0.smithi006.stderr: -127> 2024-05-30T16:20:55.732+0000 7fa7de543980 5 asok(0x565264d67790) register_command cr dump hook 0x56526503d9c8 2024-05-30T16:20:55.944 INFO:tasks.workunit.client.0.smithi006.stderr: -126> 2024-05-30T16:20:55.732+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.944 INFO:tasks.workunit.client.0.smithi006.stderr: -125> 2024-05-30T16:20:55.732+0000 7fa6b1ffb640 4 mgrc handle_mgr_map Got map version 5 2024-05-30T16:20:55.944 INFO:tasks.workunit.client.0.smithi006.stderr: -124> 2024-05-30T16:20:55.732+0000 7fa6b1ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:20:55.944 INFO:tasks.workunit.client.0.smithi006.stderr: -123> 2024-05-30T16:20:55.732+0000 7fa6b1ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:20:55.944 INFO:tasks.workunit.client.0.smithi006.stderr: -122> 2024-05-30T16:20:55.732+0000 7fa7dc855640 10 monclient: get_auth_request con 0x56526505c160 auth_method 0 2024-05-30T16:20:55.944 INFO:tasks.workunit.client.0.smithi006.stderr: -121> 2024-05-30T16:20:55.732+0000 7fa7db853640 10 monclient: get_auth_request con 0x7fa6a406dde0 auth_method 0 2024-05-30T16:20:55.944 INFO:tasks.workunit.client.0.smithi006.stderr: -120> 2024-05-30T16:20:55.736+0000 7fa7de543980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.944 INFO:tasks.workunit.client.0.smithi006.stderr: -119> 2024-05-30T16:20:55.736+0000 7fa7de543980 20 realm 2024-05-30T16:20:55.944 INFO:tasks.workunit.client.0.smithi006.stderr: -118> 2024-05-30T16:20:55.736+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.944 INFO:tasks.workunit.client.0.smithi006.stderr: -117> 2024-05-30T16:20:55.736+0000 7fa7de543980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.944 INFO:tasks.workunit.client.0.smithi006.stderr: -116> 2024-05-30T16:20:55.736+0000 7fa7de543980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:55.945 INFO:tasks.workunit.client.0.smithi006.stderr: -115> 2024-05-30T16:20:55.736+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.945 INFO:tasks.workunit.client.0.smithi006.stderr: -114> 2024-05-30T16:20:55.736+0000 7fa7de543980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.945 INFO:tasks.workunit.client.0.smithi006.stderr: -113> 2024-05-30T16:20:55.736+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.945 INFO:tasks.workunit.client.0.smithi006.stderr: -112> 2024-05-30T16:20:55.736+0000 7fa7dc054640 10 monclient: get_auth_request con 0x565265060350 auth_method 0 2024-05-30T16:20:55.945 INFO:tasks.workunit.client.0.smithi006.stderr: -111> 2024-05-30T16:20:55.736+0000 7fa7de543980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:55.945 INFO:tasks.workunit.client.0.smithi006.stderr: -110> 2024-05-30T16:20:55.736+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.945 INFO:tasks.workunit.client.0.smithi006.stderr: -109> 2024-05-30T16:20:55.736+0000 7fa7de543980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:55.945 INFO:tasks.workunit.client.0.smithi006.stderr: -108> 2024-05-30T16:20:55.736+0000 7fa7de543980 20 searching for the correct realm 2024-05-30T16:20:55.945 INFO:tasks.workunit.client.0.smithi006.stderr: -107> 2024-05-30T16:20:55.736+0000 7fa7db853640 10 monclient: get_auth_request con 0x5652650656c0 auth_method 0 2024-05-30T16:20:55.945 INFO:tasks.workunit.client.0.smithi006.stderr: -106> 2024-05-30T16:20:55.744+0000 7fa7dc855640 10 monclient: get_auth_request con 0x7fa7d0021060 auth_method 0 2024-05-30T16:20:55.946 INFO:tasks.workunit.client.0.smithi006.stderr: -105> 2024-05-30T16:20:55.744+0000 7fa7dc054640 10 monclient: get_auth_request con 0x7fa7d4017920 auth_method 0 2024-05-30T16:20:55.946 INFO:tasks.workunit.client.0.smithi006.stderr: -104> 2024-05-30T16:20:55.744+0000 7fa7dc054640 0 -- 172.21.15.6:0/2873706297 >> [v2:172.21.15.6:6816/2704410641,v1:172.21.15.6:6817/2704410641] conn(0x7fa7d4017920 msgr2=0x7fa7d4029410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:55.946 INFO:tasks.workunit.client.0.smithi006.stderr: -103> 2024-05-30T16:20:55.744+0000 7fa6b1ffb640 1 client.6215.objecter ms_handle_reset 0x7fa7d4017920 session 0x7fa7d400f040 osd.2 2024-05-30T16:20:55.946 INFO:tasks.workunit.client.0.smithi006.stderr: -102> 2024-05-30T16:20:55.744+0000 7fa6b1ffb640 10 monclient: _renew_subs 2024-05-30T16:20:55.946 INFO:tasks.workunit.client.0.smithi006.stderr: -101> 2024-05-30T16:20:55.744+0000 7fa6b1ffb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.6:3301/0 2024-05-30T16:20:55.946 INFO:tasks.workunit.client.0.smithi006.stderr: -100> 2024-05-30T16:20:55.744+0000 7fa7dc054640 10 monclient: get_auth_request con 0x7fa6a407e440 auth_method 0 2024-05-30T16:20:55.946 INFO:tasks.workunit.client.0.smithi006.stderr: -99> 2024-05-30T16:20:55.744+0000 7fa7db853640 10 monclient: get_auth_request con 0x7fa6c4001dd0 auth_method 0 2024-05-30T16:20:55.946 INFO:tasks.workunit.client.0.smithi006.stderr: -98> 2024-05-30T16:20:55.744+0000 7fa7dc855640 10 monclient: get_auth_request con 0x7fa7d0063b10 auth_method 0 2024-05-30T16:20:55.946 INFO:tasks.workunit.client.0.smithi006.stderr: -97> 2024-05-30T16:20:55.748+0000 7fa7dc054640 0 -- 172.21.15.6:0/2873706297 >> [v2:172.21.15.6:6816/2704410641,v1:172.21.15.6:6817/2704410641] conn(0x7fa6a407e440 msgr2=0x7fa6a4080870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:55.947 INFO:tasks.workunit.client.0.smithi006.stderr: -96> 2024-05-30T16:20:55.748+0000 7fa6b1ffb640 1 client.6215.objecter ms_handle_reset 0x7fa6a407e440 session 0x7fa7d400f040 osd.2 2024-05-30T16:20:55.947 INFO:tasks.workunit.client.0.smithi006.stderr: -95> 2024-05-30T16:20:55.748+0000 7fa7dc054640 10 monclient: get_auth_request con 0x7fa7d4017920 auth_method 0 2024-05-30T16:20:55.947 INFO:tasks.workunit.client.0.smithi006.stderr: -94> 2024-05-30T16:20:55.752+0000 7fa7dc054640 10 monclient: get_auth_request con 0x7fa7d00673f0 auth_method 0 2024-05-30T16:20:55.947 INFO:tasks.workunit.client.0.smithi006.stderr: -93> 2024-05-30T16:20:55.760+0000 7fa7de543980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:55.947 INFO:tasks.workunit.client.0.smithi006.stderr: -92> 2024-05-30T16:20:55.760+0000 7fa7de543980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:55.947 INFO:tasks.workunit.client.0.smithi006.stderr: -91> 2024-05-30T16:20:55.760+0000 7fa7de543980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:55.947 INFO:tasks.workunit.client.0.smithi006.stderr: -90> 2024-05-30T16:20:55.760+0000 7fa7de543980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:55.947 INFO:tasks.workunit.client.0.smithi006.stderr: -89> 2024-05-30T16:20:55.760+0000 7fa7de543980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:55.947 INFO:tasks.workunit.client.0.smithi006.stderr: -88> 2024-05-30T16:20:55.760+0000 7fa7de543980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:55.947 INFO:tasks.workunit.client.0.smithi006.stderr: -87> 2024-05-30T16:20:55.760+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.948 INFO:tasks.workunit.client.0.smithi006.stderr: -86> 2024-05-30T16:20:55.760+0000 7fa7de543980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.948 INFO:tasks.workunit.client.0.smithi006.stderr: -85> 2024-05-30T16:20:55.760+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.948 INFO:tasks.workunit.client.0.smithi006.stderr: -84> 2024-05-30T16:20:55.760+0000 7fa7de543980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:55.948 INFO:tasks.workunit.client.0.smithi006.stderr: -83> 2024-05-30T16:20:55.760+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.948 INFO:tasks.workunit.client.0.smithi006.stderr: -82> 2024-05-30T16:20:55.768+0000 7fa7de543980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:55.948 INFO:tasks.workunit.client.0.smithi006.stderr: -81> 2024-05-30T16:20:55.768+0000 7fa7de543980 20 zone default found 2024-05-30T16:20:55.948 INFO:tasks.workunit.client.0.smithi006.stderr: -80> 2024-05-30T16:20:55.768+0000 7fa7de543980 4 Realm: () 2024-05-30T16:20:55.948 INFO:tasks.workunit.client.0.smithi006.stderr: -79> 2024-05-30T16:20:55.768+0000 7fa7de543980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:55.948 INFO:tasks.workunit.client.0.smithi006.stderr: -78> 2024-05-30T16:20:55.768+0000 7fa7de543980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:55.948 INFO:tasks.workunit.client.0.smithi006.stderr: -77> 2024-05-30T16:20:55.768+0000 7fa7de543980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:55.949 INFO:tasks.workunit.client.0.smithi006.stderr: -76> 2024-05-30T16:20:55.768+0000 7fa7de543980 20 zonegroup default 2024-05-30T16:20:55.949 INFO:tasks.workunit.client.0.smithi006.stderr: -75> 2024-05-30T16:20:55.768+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.949 INFO:tasks.workunit.client.0.smithi006.stderr: -74> 2024-05-30T16:20:55.768+0000 7fa7de543980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.949 INFO:tasks.workunit.client.0.smithi006.stderr: -73> 2024-05-30T16:20:55.768+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.949 INFO:tasks.workunit.client.0.smithi006.stderr: -72> 2024-05-30T16:20:55.768+0000 7fa7de543980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.949 INFO:tasks.workunit.client.0.smithi006.stderr: -71> 2024-05-30T16:20:55.768+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.949 INFO:tasks.workunit.client.0.smithi006.stderr: -70> 2024-05-30T16:20:55.768+0000 7fa7de543980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.949 INFO:tasks.workunit.client.0.smithi006.stderr: -69> 2024-05-30T16:20:55.768+0000 7fa7de543980 20 started sync module instance, tier type = 2024-05-30T16:20:55.949 INFO:tasks.workunit.client.0.smithi006.stderr: -68> 2024-05-30T16:20:55.768+0000 7fa7de543980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:55.949 INFO:tasks.workunit.client.0.smithi006.stderr: -67> 2024-05-30T16:20:55.780+0000 7fa7de543980 20 add_watcher() i=0 2024-05-30T16:20:55.950 INFO:tasks.workunit.client.0.smithi006.stderr: -66> 2024-05-30T16:20:55.780+0000 7fa7de543980 20 add_watcher() i=1 2024-05-30T16:20:55.950 INFO:tasks.workunit.client.0.smithi006.stderr: -65> 2024-05-30T16:20:55.780+0000 7fa7de543980 20 add_watcher() i=2 2024-05-30T16:20:55.950 INFO:tasks.workunit.client.0.smithi006.stderr: -64> 2024-05-30T16:20:55.780+0000 7fa7de543980 20 add_watcher() i=3 2024-05-30T16:20:55.950 INFO:tasks.workunit.client.0.smithi006.stderr: -63> 2024-05-30T16:20:55.780+0000 7fa7de543980 20 add_watcher() i=4 2024-05-30T16:20:55.950 INFO:tasks.workunit.client.0.smithi006.stderr: -62> 2024-05-30T16:20:55.780+0000 7fa7de543980 20 add_watcher() i=5 2024-05-30T16:20:55.950 INFO:tasks.workunit.client.0.smithi006.stderr: -61> 2024-05-30T16:20:55.780+0000 7fa7de543980 20 add_watcher() i=6 2024-05-30T16:20:55.950 INFO:tasks.workunit.client.0.smithi006.stderr: -60> 2024-05-30T16:20:55.780+0000 7fa7de543980 20 add_watcher() i=7 2024-05-30T16:20:55.950 INFO:tasks.workunit.client.0.smithi006.stderr: -59> 2024-05-30T16:20:55.780+0000 7fa7de543980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:55.950 INFO:tasks.workunit.client.0.smithi006.stderr: -58> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c2280) adding auth protocol: cephx 2024-05-30T16:20:55.951 INFO:tasks.workunit.client.0.smithi006.stderr: -57> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c2280) adding auth protocol: cephx 2024-05-30T16:20:55.951 INFO:tasks.workunit.client.0.smithi006.stderr: -56> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c2280) adding auth protocol: cephx 2024-05-30T16:20:55.951 INFO:tasks.workunit.client.0.smithi006.stderr: -55> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c2280) adding con mode: secure 2024-05-30T16:20:55.951 INFO:tasks.workunit.client.0.smithi006.stderr: -54> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c2280) adding con mode: crc 2024-05-30T16:20:55.951 INFO:tasks.workunit.client.0.smithi006.stderr: -53> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c2280) adding con mode: secure 2024-05-30T16:20:55.951 INFO:tasks.workunit.client.0.smithi006.stderr: -52> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c2280) adding con mode: crc 2024-05-30T16:20:55.951 INFO:tasks.workunit.client.0.smithi006.stderr: -51> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c2280) adding con mode: secure 2024-05-30T16:20:55.951 INFO:tasks.workunit.client.0.smithi006.stderr: -50> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c2280) adding con mode: crc 2024-05-30T16:20:55.952 INFO:tasks.workunit.client.0.smithi006.stderr: -49> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c2280) adding con mode: crc 2024-05-30T16:20:55.952 INFO:tasks.workunit.client.0.smithi006.stderr: -48> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c2280) adding con mode: secure 2024-05-30T16:20:55.952 INFO:tasks.workunit.client.0.smithi006.stderr: -47> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c2280) adding con mode: crc 2024-05-30T16:20:55.952 INFO:tasks.workunit.client.0.smithi006.stderr: -46> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c2280) adding con mode: secure 2024-05-30T16:20:55.952 INFO:tasks.workunit.client.0.smithi006.stderr: -45> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c2280) adding con mode: crc 2024-05-30T16:20:55.952 INFO:tasks.workunit.client.0.smithi006.stderr: -44> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 AuthRegistry(0x7ffef71c2280) adding con mode: secure 2024-05-30T16:20:55.952 INFO:tasks.workunit.client.0.smithi006.stderr: -43> 2024-05-30T16:20:55.784+0000 7fa692fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:55.953 INFO:tasks.workunit.client.0.smithi006.stderr: -42> 2024-05-30T16:20:55.784+0000 7fa7de543980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:20:55.953 INFO:tasks.workunit.client.0.smithi006.stderr: -41> 2024-05-30T16:20:55.784+0000 7fa7de543980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:55.953 INFO:tasks.workunit.client.0.smithi006.stderr: -40> 2024-05-30T16:20:55.784+0000 7fa7de543980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:55.953 INFO:tasks.workunit.client.0.smithi006.stderr: -39> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 asok(0x565264d67790) register_command cache list hook 0x5652650452a0 2024-05-30T16:20:55.953 INFO:tasks.workunit.client.0.smithi006.stderr: -38> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 asok(0x565264d67790) register_command cache inspect hook 0x5652650452a0 2024-05-30T16:20:55.953 INFO:tasks.workunit.client.0.smithi006.stderr: -37> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 asok(0x565264d67790) register_command cache erase hook 0x5652650452a0 2024-05-30T16:20:55.953 INFO:tasks.workunit.client.0.smithi006.stderr: -36> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 asok(0x565264d67790) register_command cache zap hook 0x5652650452a0 2024-05-30T16:20:55.953 INFO:tasks.workunit.client.0.smithi006.stderr: -35> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 note: GC not initialized 2024-05-30T16:20:55.953 INFO:tasks.workunit.client.0.smithi006.stderr: -34> 2024-05-30T16:20:55.784+0000 7fa691fdb640 20 reqs_thread_entry: start 2024-05-30T16:20:55.953 INFO:tasks.workunit.client.0.smithi006.stderr: -33> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 asok(0x565264d67790) register_command sync trace show hook 0x7fa6c4004870 2024-05-30T16:20:55.953 INFO:tasks.workunit.client.0.smithi006.stderr: -32> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 asok(0x565264d67790) register_command sync trace history hook 0x7fa6c4004870 2024-05-30T16:20:55.953 INFO:tasks.workunit.client.0.smithi006.stderr: -31> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 asok(0x565264d67790) register_command sync trace active hook 0x7fa6c4004870 2024-05-30T16:20:55.954 INFO:tasks.workunit.client.0.smithi006.stderr: -30> 2024-05-30T16:20:55.784+0000 7fa7de543980 5 asok(0x565264d67790) register_command sync trace active_short hook 0x7fa6c4004870 2024-05-30T16:20:55.954 INFO:tasks.workunit.client.0.smithi006.stderr: -29> 2024-05-30T16:20:55.784+0000 7fa7de543980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:55.954 INFO:tasks.workunit.client.0.smithi006.stderr: -28> 2024-05-30T16:20:55.784+0000 7fa7de543980 20 Filter name: none 2024-05-30T16:20:55.954 INFO:tasks.workunit.client.0.smithi006.stderr: -27> 2024-05-30T16:20:55.784+0000 7fa6037fe640 20 reqs_thread_entry: start 2024-05-30T16:20:55.954 INFO:tasks.workunit.client.0.smithi006.stderr: -26> 2024-05-30T16:20:55.784+0000 7fa7de543980 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-05-30T16:20:55.954 INFO:tasks.workunit.client.0.smithi006.stderr: -25> 2024-05-30T16:20:55.784+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.955 INFO:tasks.workunit.client.0.smithi006.stderr: -24> 2024-05-30T16:20:55.784+0000 7fa7de543980 20 rados_obj.operate() r=0 bl.length=213 2024-05-30T16:20:55.955 INFO:tasks.workunit.client.0.smithi006.stderr: -23> 2024-05-30T16:20:55.784+0000 7fa7de543980 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-05-30T16:20:55.955 INFO:tasks.workunit.client.0.smithi006.stderr: -22> 2024-05-30T16:20:55.784+0000 7fa7de543980 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-05-30T16:20:55.955 INFO:tasks.workunit.client.0.smithi006.stderr: -21> 2024-05-30T16:20:55.784+0000 7fa7de543980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 : miss 2024-05-30T16:20:55.955 INFO:tasks.workunit.client.0.smithi006.stderr: -20> 2024-05-30T16:20:55.784+0000 7fa7de543980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.955 INFO:tasks.workunit.client.0.smithi006.stderr: -19> 2024-05-30T16:20:55.788+0000 7fa7de543980 20 rados_obj.operate() r=0 bl.length=427 2024-05-30T16:20:55.955 INFO:tasks.workunit.client.0.smithi006.stderr: -18> 2024-05-30T16:20:55.788+0000 7fa7de543980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 info.flags=0x17 2024-05-30T16:20:55.955 INFO:tasks.workunit.client.0.smithi006.stderr: -17> 2024-05-30T16:20:55.788+0000 7fa7de543980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 to cache LRU end 2024-05-30T16:20:55.955 INFO:tasks.workunit.client.0.smithi006.stderr: -16> 2024-05-30T16:20:55.788+0000 7fa7de543980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:20:55.955 INFO:tasks.workunit.client.0.smithi006.stderr: -15> 2024-05-30T16:20:55.788+0000 7fa7de543980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:55.955 INFO:tasks.workunit.client.0.smithi006.stderr: -14> 2024-05-30T16:20:55.788+0000 7fa7de543980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 2024-05-30T16:20:55.956 INFO:tasks.workunit.client.0.smithi006.stderr: -13> 2024-05-30T16:20:55.800+0000 7fa7de543980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 info.flags=0x17 2024-05-30T16:20:55.956 INFO:tasks.workunit.client.0.smithi006.stderr: -12> 2024-05-30T16:20:55.800+0000 7fa7de543980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 to cache LRU end 2024-05-30T16:20:55.956 INFO:tasks.workunit.client.0.smithi006.stderr: -11> 2024-05-30T16:20:55.800+0000 7fa7de543980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:55.956 INFO:tasks.workunit.client.0.smithi006.stderr: -10> 2024-05-30T16:20:55.800+0000 7fa7de543980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24, ofs0, ns] 2024-05-30T16:20:55.956 INFO:tasks.workunit.client.0.smithi006.stderr: -9> 2024-05-30T16:20:55.800+0000 7fa6b2ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431625 cookie 94911882257936 notifier 6215 bl.length()=991 2024-05-30T16:20:55.956 INFO:tasks.workunit.client.0.smithi006.stderr: -8> 2024-05-30T16:20:55.800+0000 7fa6b2ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 info.flags=0x17 2024-05-30T16:20:55.956 INFO:tasks.workunit.client.0.smithi006.stderr: -7> 2024-05-30T16:20:55.800+0000 7fa6b2ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 to cache LRU end 2024-05-30T16:20:55.956 INFO:tasks.workunit.client.0.smithi006.stderr: -6> 2024-05-30T16:20:55.800+0000 7fa6b2ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:55.956 INFO:tasks.workunit.client.0.smithi006.stderr: -5> 2024-05-30T16:20:55.800+0000 7fa7de543980 20 handle_bi_update(): bucket=:abort-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24]): orig_sources= new_sources= 2024-05-30T16:20:55.956 INFO:tasks.workunit.client.0.smithi006.stderr: -4> 2024-05-30T16:20:55.800+0000 7fa7de543980 20 handle_bi_update(): bucket=:abort-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24]): potential sources added=[] removed=[] 2024-05-30T16:20:55.956 INFO:tasks.workunit.client.0.smithi006.stderr: -3> 2024-05-30T16:20:55.800+0000 7fa7de543980 20 handle_bi_update(): bucket=:abort-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24]): orig_dests= new_dests= 2024-05-30T16:20:55.956 INFO:tasks.workunit.client.0.smithi006.stderr: -2> 2024-05-30T16:20:55.800+0000 7fa7de543980 20 handle_bi_update(): bucket=:abort-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24]): potential dests added=[] removed=[] 2024-05-30T16:20:55.956 INFO:tasks.workunit.client.0.smithi006.stderr: -1> 2024-05-30T16:20:55.804+0000 7fa7de543980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fa7de543980 time 2024-05-30T16:20:55.808517+0000 2024-05-30T16:20:55.957 INFO:tasks.workunit.client.0.smithi006.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-30T16:20:55.957 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.957 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:20:55.957 INFO:tasks.workunit.client.0.smithi006.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fa7e1e1acff] 2024-05-30T16:20:55.957 INFO:tasks.workunit.client.0.smithi006.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7fa7e1e1aeb3] 2024-05-30T16:20:55.957 INFO:tasks.workunit.client.0.smithi006.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x565262955f81] 2024-05-30T16:20:55.957 INFO:tasks.workunit.client.0.smithi006.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2ab) [0x565262967eab] 2024-05-30T16:20:55.957 INFO:tasks.workunit.client.0.smithi006.stderr: 5: main() 2024-05-30T16:20:55.957 INFO:tasks.workunit.client.0.smithi006.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fa7e0c83d90] 2024-05-30T16:20:55.957 INFO:tasks.workunit.client.0.smithi006.stderr: 7: __libc_start_main() 2024-05-30T16:20:55.957 INFO:tasks.workunit.client.0.smithi006.stderr: 8: _start() 2024-05-30T16:20:55.957 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.957 INFO:tasks.workunit.client.0.smithi006.stderr: 0> 2024-05-30T16:20:55.804+0000 7fa7de543980 -1 *** Caught signal (Aborted) ** 2024-05-30T16:20:55.958 INFO:tasks.workunit.client.0.smithi006.stderr: in thread 7fa7de543980 thread_name:radosgw-admin 2024-05-30T16:20:55.958 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.958 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:20:55.958 INFO:tasks.workunit.client.0.smithi006.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fa7e0c9c520] 2024-05-30T16:20:55.958 INFO:tasks.workunit.client.0.smithi006.stderr: 2: pthread_kill() 2024-05-30T16:20:55.958 INFO:tasks.workunit.client.0.smithi006.stderr: 3: raise() 2024-05-30T16:20:55.958 INFO:tasks.workunit.client.0.smithi006.stderr: 4: abort() 2024-05-30T16:20:55.958 INFO:tasks.workunit.client.0.smithi006.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fa7e1e1ad59] 2024-05-30T16:20:55.958 INFO:tasks.workunit.client.0.smithi006.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7fa7e1e1aeb3] 2024-05-30T16:20:55.958 INFO:tasks.workunit.client.0.smithi006.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x565262955f81] 2024-05-30T16:20:55.958 INFO:tasks.workunit.client.0.smithi006.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2ab) [0x565262967eab] 2024-05-30T16:20:55.958 INFO:tasks.workunit.client.0.smithi006.stderr: 9: main() 2024-05-30T16:20:55.959 INFO:tasks.workunit.client.0.smithi006.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fa7e0c83d90] 2024-05-30T16:20:55.959 INFO:tasks.workunit.client.0.smithi006.stderr: 11: __libc_start_main() 2024-05-30T16:20:55.959 INFO:tasks.workunit.client.0.smithi006.stderr: 12: _start() 2024-05-30T16:20:55.959 INFO:tasks.workunit.client.0.smithi006.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-30T16:20:55.959 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.959 INFO:tasks.workunit.client.0.smithi006.stderr:--- logging levels --- 2024-05-30T16:20:55.959 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 none 2024-05-30T16:20:55.959 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 lockdep 2024-05-30T16:20:55.959 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 context 2024-05-30T16:20:55.959 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 crush 2024-05-30T16:20:55.959 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds 2024-05-30T16:20:55.959 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_balancer 2024-05-30T16:20:55.959 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_locker 2024-05-30T16:20:55.960 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_log 2024-05-30T16:20:55.960 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_log_expire 2024-05-30T16:20:55.960 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_migrator 2024-05-30T16:20:55.960 INFO:tasks.workunit.client.0.smithi006.stderr: 3/ 5 mds_quiesce 2024-05-30T16:20:55.960 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 buffer 2024-05-30T16:20:55.960 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 timer 2024-05-30T16:20:55.960 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 filer 2024-05-30T16:20:55.960 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 striper 2024-05-30T16:20:55.960 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 objecter 2024-05-30T16:20:55.960 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rados 2024-05-30T16:20:55.960 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd 2024-05-30T16:20:55.960 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd_mirror 2024-05-30T16:20:55.960 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd_replay 2024-05-30T16:20:55.960 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd_pwl 2024-05-30T16:20:55.961 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 journaler 2024-05-30T16:20:55.961 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 objectcacher 2024-05-30T16:20:55.961 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 immutable_obj_cache 2024-05-30T16:20:55.961 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 client 2024-05-30T16:20:55.961 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 osd 2024-05-30T16:20:55.961 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 optracker 2024-05-30T16:20:55.961 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 objclass 2024-05-30T16:20:55.961 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 3 filestore 2024-05-30T16:20:55.961 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 3 journal 2024-05-30T16:20:55.961 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 0 ms 2024-05-30T16:20:55.961 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mon 2024-05-30T16:20:55.961 INFO:tasks.workunit.client.0.smithi006.stderr: 0/10 monc 2024-05-30T16:20:55.961 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 paxos 2024-05-30T16:20:55.961 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 tp 2024-05-30T16:20:55.962 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 auth 2024-05-30T16:20:55.962 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 crypto 2024-05-30T16:20:55.962 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 finisher 2024-05-30T16:20:55.962 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 reserver 2024-05-30T16:20:55.962 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 heartbeatmap 2024-05-30T16:20:55.962 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 perfcounter 2024-05-30T16:20:55.962 INFO:tasks.workunit.client.0.smithi006.stderr: 20/20 rgw 2024-05-30T16:20:55.962 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_sync 2024-05-30T16:20:55.962 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_datacache 2024-05-30T16:20:55.963 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_access 2024-05-30T16:20:55.963 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_dbstore 2024-05-30T16:20:55.963 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_flight 2024-05-30T16:20:55.963 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_lifecycle 2024-05-30T16:20:55.963 INFO:tasks.workunit.client.0.smithi006.stderr: 20/20 rgw_notification 2024-05-30T16:20:55.963 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 javaclient 2024-05-30T16:20:55.963 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 asok 2024-05-30T16:20:55.963 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 throttle 2024-05-30T16:20:55.963 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 0 refs 2024-05-30T16:20:55.963 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 compressor 2024-05-30T16:20:55.963 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 bluestore 2024-05-30T16:20:55.963 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 bluefs 2024-05-30T16:20:55.963 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 3 bdev 2024-05-30T16:20:55.964 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 kstore 2024-05-30T16:20:55.964 INFO:tasks.workunit.client.0.smithi006.stderr: 4/ 5 rocksdb 2024-05-30T16:20:55.964 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 fuse 2024-05-30T16:20:55.964 INFO:tasks.workunit.client.0.smithi006.stderr: 2/ 5 mgr 2024-05-30T16:20:55.964 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mgrc 2024-05-30T16:20:55.964 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 dpdk 2024-05-30T16:20:55.964 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 eventtrace 2024-05-30T16:20:55.964 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 prioritycache 2024-05-30T16:20:55.964 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 test 2024-05-30T16:20:55.964 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 cephfs_mirror 2024-05-30T16:20:55.964 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 cephsqlite 2024-05-30T16:20:55.964 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 crimson_interrupt 2024-05-30T16:20:55.965 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore 2024-05-30T16:20:55.965 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_onode 2024-05-30T16:20:55.965 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_odata 2024-05-30T16:20:55.965 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_omap 2024-05-30T16:20:55.965 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_tm 2024-05-30T16:20:55.965 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_t 2024-05-30T16:20:55.965 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_cleaner 2024-05-30T16:20:55.965 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_epm 2024-05-30T16:20:55.965 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_lba 2024-05-30T16:20:55.965 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_fixedkv_tree 2024-05-30T16:20:55.966 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_cache 2024-05-30T16:20:55.966 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_journal 2024-05-30T16:20:55.966 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_device 2024-05-30T16:20:55.966 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_backref 2024-05-30T16:20:55.966 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 alienstore 2024-05-30T16:20:55.966 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mclock 2024-05-30T16:20:55.966 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 cyanstore 2024-05-30T16:20:55.966 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 ceph_exporter 2024-05-30T16:20:55.966 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 memstore 2024-05-30T16:20:55.966 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 trace 2024-05-30T16:20:55.966 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 ceph_dedup 2024-05-30T16:20:55.966 INFO:tasks.workunit.client.0.smithi006.stderr: -2/-2 (syslog threshold) 2024-05-30T16:20:55.966 INFO:tasks.workunit.client.0.smithi006.stderr: 99/99 (stderr threshold) 2024-05-30T16:20:55.967 INFO:tasks.workunit.client.0.smithi006.stderr:--- pthread ID / name mapping for recent threads --- 2024-05-30T16:20:55.967 INFO:tasks.workunit.client.0.smithi006.stderr: 7fa6037fe640 / http_manager 2024-05-30T16:20:55.967 INFO:tasks.workunit.client.0.smithi006.stderr: 7fa691fdb640 / http_manager 2024-05-30T16:20:55.967 INFO:tasks.workunit.client.0.smithi006.stderr: 7fa692fdd640 / rgw_dt_lg_renew 2024-05-30T16:20:55.967 INFO:tasks.workunit.client.0.smithi006.stderr: 7fa6b1ffb640 / ms_dispatch 2024-05-30T16:20:55.967 INFO:tasks.workunit.client.0.smithi006.stderr: 7fa6b2ffd640 / io_context_pool 2024-05-30T16:20:55.967 INFO:tasks.workunit.client.0.smithi006.stderr: 7fa6b37fe640 / io_context_pool 2024-05-30T16:20:55.967 INFO:tasks.workunit.client.0.smithi006.stderr: 7fa6c1ffb640 / ms_dispatch 2024-05-30T16:20:55.967 INFO:tasks.workunit.client.0.smithi006.stderr: 7fa6c8e09640 / ms_dispatch 2024-05-30T16:20:55.967 INFO:tasks.workunit.client.0.smithi006.stderr: 7fa6ca60c640 / io_context_pool 2024-05-30T16:20:55.967 INFO:tasks.workunit.client.0.smithi006.stderr: 7fa7db853640 / msgr-worker-2 2024-05-30T16:20:55.967 INFO:tasks.workunit.client.0.smithi006.stderr: 7fa7dc054640 / msgr-worker-1 2024-05-30T16:20:55.968 INFO:tasks.workunit.client.0.smithi006.stderr: 7fa7dc855640 / msgr-worker-0 2024-05-30T16:20:55.968 INFO:tasks.workunit.client.0.smithi006.stderr: 7fa7dd056640 / io_context_pool 2024-05-30T16:20:55.968 INFO:tasks.workunit.client.0.smithi006.stderr: 7fa7de543980 / radosgw-admin 2024-05-30T16:20:55.968 INFO:tasks.workunit.client.0.smithi006.stderr: max_recent 500 2024-05-30T16:20:55.968 INFO:tasks.workunit.client.0.smithi006.stderr: max_new 1000 2024-05-30T16:20:55.968 INFO:tasks.workunit.client.0.smithi006.stderr: log_file /var/lib/ceph/crash/2024-05-30T16:20:55.810564Z_353105a0-2818-47c0-aae8-0dd6042dfab9/log 2024-05-30T16:20:55.968 INFO:tasks.workunit.client.0.smithi006.stderr:--- end dump of recent events --- 2024-05-30T16:20:55.968 INFO:tasks.workunit.client.0.smithi006.stderr:Aborted 2024-05-30T16:20:55.968 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket abort-at-block-writes 2024-05-30T16:20:55.973 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:20:55.973 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:20:55.973 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.888+0000 7f9cb8ffd980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.973 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.892+0000 7f9cb8ffd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.973 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.892+0000 7f9cb8ffd980 20 realm 2024-05-30T16:20:55.973 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.892+0000 7f9cb8ffd980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.973 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.892+0000 7f9cb8ffd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.973 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.892+0000 7f9cb8ffd980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:55.973 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.892+0000 7f9cb8ffd980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.974 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.892+0000 7f9cb8ffd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.974 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.892+0000 7f9cb8ffd980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.974 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.892+0000 7f9cb8ffd980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:55.974 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.892+0000 7f9cb8ffd980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.974 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.896+0000 7f9cb8ffd980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:55.974 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.896+0000 7f9cb8ffd980 20 searching for the correct realm 2024-05-30T16:20:55.974 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.900+0000 7f9cb630d640 0 -- 172.21.15.6:0/540624326 >> [v2:172.21.15.6:6819/1141406589,v1:172.21.15.6:6821/1141406589] conn(0x5573414bb800 msgr2=0x5573414bdbf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:55.974 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.904+0000 7f9cb6b0e640 0 -- 172.21.15.6:0/540624326 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f9cac07dbc0 msgr2=0x7f9cac077810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:55.974 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.916+0000 7f9cb8ffd980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:55.974 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.916+0000 7f9cb8ffd980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:55.975 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.916+0000 7f9cb8ffd980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:55.975 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.916+0000 7f9cb8ffd980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:55.975 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.916+0000 7f9cb8ffd980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:55.975 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.916+0000 7f9cb8ffd980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:55.975 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.916+0000 7f9cb8ffd980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.975 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.916+0000 7f9cb8ffd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.975 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.916+0000 7f9cb8ffd980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.976 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.916+0000 7f9cb8ffd980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:55.976 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.916+0000 7f9cb8ffd980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.976 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.916+0000 7f9cb8ffd980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:55.976 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.916+0000 7f9cb8ffd980 20 zone default found 2024-05-30T16:20:55.976 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.916+0000 7f9cb8ffd980 4 Realm: () 2024-05-30T16:20:55.976 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.916+0000 7f9cb8ffd980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:55.976 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.916+0000 7f9cb8ffd980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:55.976 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.916+0000 7f9cb8ffd980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:55.976 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.916+0000 7f9cb8ffd980 20 zonegroup default 2024-05-30T16:20:55.976 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.916+0000 7f9cb8ffd980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.976 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.916+0000 7f9cb8ffd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.977 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.916+0000 7f9cb8ffd980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.977 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.916+0000 7f9cb8ffd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.977 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.916+0000 7f9cb8ffd980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.977 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.916+0000 7f9cb8ffd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:55.977 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.916+0000 7f9cb8ffd980 20 started sync module instance, tier type = 2024-05-30T16:20:55.977 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.916+0000 7f9cb8ffd980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:55.977 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.932+0000 7f9cb8ffd980 20 add_watcher() i=0 2024-05-30T16:20:55.977 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.932+0000 7f9cb8ffd980 20 add_watcher() i=1 2024-05-30T16:20:55.977 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.932+0000 7f9cb8ffd980 20 add_watcher() i=2 2024-05-30T16:20:55.978 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.932+0000 7f9cb8ffd980 20 add_watcher() i=3 2024-05-30T16:20:55.978 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.932+0000 7f9cb8ffd980 20 add_watcher() i=4 2024-05-30T16:20:55.978 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.932+0000 7f9cb8ffd980 20 add_watcher() i=5 2024-05-30T16:20:55.978 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.932+0000 7f9cb8ffd980 20 add_watcher() i=6 2024-05-30T16:20:55.978 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.936+0000 7f9cb8ffd980 20 add_watcher() i=7 2024-05-30T16:20:55.978 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.936+0000 7f9cb8ffd980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:55.978 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.936+0000 7f9b6d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:55.978 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.936+0000 7f9cb8ffd980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:55.978 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.936+0000 7f9cb8ffd980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:55.978 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.936+0000 7f9cb8ffd980 5 note: GC not initialized 2024-05-30T16:20:55.978 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.940+0000 7f9ae7fff640 20 reqs_thread_entry: start 2024-05-30T16:20:55.978 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.940+0000 7f9cb8ffd980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:55.978 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.940+0000 7f9cb8ffd980 20 Filter name: none 2024-05-30T16:20:55.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.940+0000 7f9ae5ffb640 20 reqs_thread_entry: start 2024-05-30T16:20:55.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.940+0000 7f9cb8ffd980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.940+0000 7f9cb8ffd980 20 rados_obj.operate() r=0 bl.length=213 2024-05-30T16:20:55.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.940+0000 7f9cb8ffd980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.940+0000 7f9cb8ffd980 20 rados_obj.operate() r=0 bl.length=459 2024-05-30T16:20:55.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.940+0000 7f9cb8ffd980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:55.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.940+0000 7f9cb8ffd980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.940+0000 7f9cb8ffd980 20 rados_obj.operate() r=0 bl.length=213 2024-05-30T16:20:55.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.940+0000 7f9cb8ffd980 20 rados->read ofs=0 len=0 2024-05-30T16:20:55.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.940+0000 7f9cb8ffd980 20 rados_obj.operate() r=0 bl.length=459 2024-05-30T16:20:55.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.940+0000 7f9cb8ffd980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:20:55.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.944+0000 7f9b6d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:55.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.948+0000 7f9cb8ffd980 20 remove_watcher() i=0 2024-05-30T16:20:55.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.948+0000 7f9cb8ffd980 2 removed watcher, disabling cache 2024-05-30T16:20:55.980 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.952+0000 7f9cb8ffd980 20 remove_watcher() i=1 2024-05-30T16:20:55.980 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.952+0000 7f9cb8ffd980 20 remove_watcher() i=2 2024-05-30T16:20:55.980 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.960+0000 7f9cb8ffd980 20 remove_watcher() i=3 2024-05-30T16:20:55.980 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.960+0000 7f9cb8ffd980 20 remove_watcher() i=4 2024-05-30T16:20:55.980 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.960+0000 7f9cb8ffd980 20 remove_watcher() i=5 2024-05-30T16:20:55.980 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.960+0000 7f9cb8ffd980 20 remove_watcher() i=6 2024-05-30T16:20:55.980 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:55.960+0000 7f9cb8ffd980 20 remove_watcher() i=7 2024-05-30T16:20:55.980 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:55.980 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:20:55.981 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: abort-at-block-writes id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-30T16:20:55.981 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:20:56.079 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket reshard --bucket abort-at-block-writes --num-shards 12 --rgw-reshard-bucket-lock-duration 30 2024-05-30T16:20:56.116 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:20:56.116 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:20:56.162 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.160+0000 7fe2b08b2980 20 rados->read ofs=0 len=0 2024-05-30T16:20:56.164 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.160+0000 7fe2b08b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:56.164 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.160+0000 7fe2b08b2980 20 realm 2024-05-30T16:20:56.164 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.160+0000 7fe2b08b2980 20 rados->read ofs=0 len=0 2024-05-30T16:20:56.164 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.160+0000 7fe2b08b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:56.164 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.160+0000 7fe2b08b2980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:20:56.164 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.160+0000 7fe2b08b2980 20 rados->read ofs=0 len=0 2024-05-30T16:20:56.165 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.160+0000 7fe2b08b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:56.165 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.160+0000 7fe2b08b2980 20 rados->read ofs=0 len=0 2024-05-30T16:20:56.166 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.164+0000 7fe2b08b2980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:56.166 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.164+0000 7fe2b08b2980 20 rados->read ofs=0 len=0 2024-05-30T16:20:56.167 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.164+0000 7fe2b08b2980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:20:56.167 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.164+0000 7fe2b08b2980 20 searching for the correct realm 2024-05-30T16:20:56.173 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.168+0000 7fe2adbc2640 0 -- 172.21.15.6:0/402822817 >> [v2:172.21.15.6:6819/1141406589,v1:172.21.15.6:6821/1141406589] conn(0x55ef9d506600 msgr2=0x55ef9d508ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:56.179 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.176+0000 7fe2ae3c3640 0 -- 172.21.15.6:0/402822817 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7fe2a400f3d0 msgr2=0x7fe2a400f850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:20:56.188 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.188+0000 7fe2b08b2980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:20:56.188 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.188+0000 7fe2b08b2980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:20:56.188 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.188+0000 7fe2b08b2980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:20:56.188 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.188+0000 7fe2b08b2980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:20:56.188 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.188+0000 7fe2b08b2980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:20:56.188 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.188+0000 7fe2b08b2980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:20:56.189 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.188+0000 7fe2b08b2980 20 rados->read ofs=0 len=0 2024-05-30T16:20:56.189 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.188+0000 7fe2b08b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:56.189 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.188+0000 7fe2b08b2980 20 rados->read ofs=0 len=0 2024-05-30T16:20:56.190 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.188+0000 7fe2b08b2980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:20:56.190 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.188+0000 7fe2b08b2980 20 rados->read ofs=0 len=0 2024-05-30T16:20:56.190 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.192+0000 7fe2b08b2980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:20:56.190 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.192+0000 7fe2b08b2980 20 zone default found 2024-05-30T16:20:56.190 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.192+0000 7fe2b08b2980 4 Realm: () 2024-05-30T16:20:56.190 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.192+0000 7fe2b08b2980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:20:56.191 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.192+0000 7fe2b08b2980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:20:56.191 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.192+0000 7fe2b08b2980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:20:56.191 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.192+0000 7fe2b08b2980 20 zonegroup default 2024-05-30T16:20:56.191 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.192+0000 7fe2b08b2980 20 rados->read ofs=0 len=0 2024-05-30T16:20:56.191 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.192+0000 7fe2b08b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:56.191 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.192+0000 7fe2b08b2980 20 rados->read ofs=0 len=0 2024-05-30T16:20:56.192 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.192+0000 7fe2b08b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:56.192 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.192+0000 7fe2b08b2980 20 rados->read ofs=0 len=0 2024-05-30T16:20:56.192 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.192+0000 7fe2b08b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:20:56.192 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.192+0000 7fe2b08b2980 20 started sync module instance, tier type = 2024-05-30T16:20:56.192 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.192+0000 7fe2b08b2980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:20:56.206 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.204+0000 7fe2b08b2980 20 add_watcher() i=0 2024-05-30T16:20:56.206 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.204+0000 7fe2b08b2980 20 add_watcher() i=1 2024-05-30T16:20:56.207 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.204+0000 7fe2b08b2980 20 add_watcher() i=2 2024-05-30T16:20:56.207 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.204+0000 7fe2b08b2980 20 add_watcher() i=3 2024-05-30T16:20:56.207 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.204+0000 7fe2b08b2980 20 add_watcher() i=4 2024-05-30T16:20:56.207 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.204+0000 7fe2b08b2980 20 add_watcher() i=5 2024-05-30T16:20:56.207 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.204+0000 7fe2b08b2980 20 add_watcher() i=6 2024-05-30T16:20:56.207 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.204+0000 7fe2b08b2980 20 add_watcher() i=7 2024-05-30T16:20:56.207 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.204+0000 7fe2b08b2980 2 all 8 watchers are set, enabling cache 2024-05-30T16:20:56.210 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.208+0000 7fe164fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:56.211 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.208+0000 7fe2b08b2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:20:56.211 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.208+0000 7fe2b08b2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:20:56.211 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.208+0000 7fe2b08b2980 5 note: GC not initialized 2024-05-30T16:20:56.211 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.208+0000 7fe0df7fe640 20 reqs_thread_entry: start 2024-05-30T16:20:56.212 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.208+0000 7fe2b08b2980 20 init_complete bucket index max shards: 11 2024-05-30T16:20:56.212 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.208+0000 7fe2b08b2980 20 Filter name: none 2024-05-30T16:20:56.212 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.208+0000 7fe0dd7fa640 20 reqs_thread_entry: start 2024-05-30T16:20:56.212 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.208+0000 7fe2b08b2980 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-05-30T16:20:56.212 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.208+0000 7fe2b08b2980 20 rados->read ofs=0 len=0 2024-05-30T16:20:56.213 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.208+0000 7fe2b08b2980 20 rados_obj.operate() r=0 bl.length=213 2024-05-30T16:20:56.213 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.208+0000 7fe2b08b2980 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-05-30T16:20:56.213 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.208+0000 7fe2b08b2980 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-05-30T16:20:56.213 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.208+0000 7fe2b08b2980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 : miss 2024-05-30T16:20:56.214 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.208+0000 7fe2b08b2980 20 rados->read ofs=0 len=0 2024-05-30T16:20:56.214 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.208+0000 7fe2b08b2980 20 rados_obj.operate() r=0 bl.length=459 2024-05-30T16:20:56.214 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.208+0000 7fe2b08b2980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 info.flags=0x17 2024-05-30T16:20:56.214 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.208+0000 7fe2b08b2980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 to cache LRU end 2024-05-30T16:20:56.214 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.208+0000 7fe2b08b2980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:20:56.214 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.208+0000 7fe2b08b2980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:20:56.214 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.208+0000 7fe2b08b2980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 2024-05-30T16:20:56.215 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.212+0000 7fe2b08b2980 0 INFO: RGWReshardLock::lock found lock on abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 to be held by another RGW process; skipping for now 2024-05-30T16:20:56.217 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.212+0000 7fe164fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:20:56.221 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.224+0000 7fe2b08b2980 20 remove_watcher() i=0 2024-05-30T16:20:56.221 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.224+0000 7fe2b08b2980 2 removed watcher, disabling cache 2024-05-30T16:20:56.223 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.224+0000 7fe2b08b2980 20 remove_watcher() i=1 2024-05-30T16:20:56.226 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.224+0000 7fe2b08b2980 20 remove_watcher() i=2 2024-05-30T16:20:56.228 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.224+0000 7fe2b08b2980 20 remove_watcher() i=3 2024-05-30T16:20:56.229 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.228+0000 7fe2b08b2980 20 remove_watcher() i=4 2024-05-30T16:20:56.231 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.228+0000 7fe2b08b2980 20 remove_watcher() i=5 2024-05-30T16:20:56.232 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.228+0000 7fe2b08b2980 20 remove_watcher() i=6 2024-05-30T16:20:56.233 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:20:56.228+0000 7fe2b08b2980 20 remove_watcher() i=7 2024-05-30T16:20:56.247 INFO:tasks.workunit.client.0.smithi006.stderr:waiting 30 seconds for reshard lock to expire... 2024-05-30T16:21:26.251 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket reshard --bucket abort-at-block-writes --num-shards 12 --rgw-reshard-bucket-lock-duration 30 2024-05-30T16:21:26.290 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:21:26.290 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:21:26.334 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.336+0000 7f26c6f36980 20 rados->read ofs=0 len=0 2024-05-30T16:21:26.539 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.540+0000 7f26c6f36980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:26.539 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.540+0000 7f26c6f36980 20 realm 2024-05-30T16:21:26.539 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.540+0000 7f26c6f36980 20 rados->read ofs=0 len=0 2024-05-30T16:21:26.540 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.540+0000 7f26c6f36980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:26.540 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.540+0000 7f26c6f36980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:21:26.540 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.540+0000 7f26c6f36980 20 rados->read ofs=0 len=0 2024-05-30T16:21:26.540 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.540+0000 7f26c6f36980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:26.541 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.540+0000 7f26c6f36980 20 rados->read ofs=0 len=0 2024-05-30T16:21:26.543 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.544+0000 7f26c6f36980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:21:26.543 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.544+0000 7f26c6f36980 20 rados->read ofs=0 len=0 2024-05-30T16:21:26.544 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.544+0000 7f26c6f36980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:21:26.544 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.544+0000 7f26c6f36980 20 searching for the correct realm 2024-05-30T16:21:26.552 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.552+0000 7f26c5248640 0 -- 172.21.15.6:0/3863331811 >> [v2:172.21.15.204:6823/3156311527,v1:172.21.15.204:6825/3156311527] conn(0x7f26ac043040 msgr2=0x7f26ac043420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:21:26.562 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.564+0000 7f26c4a47640 0 -- 172.21.15.6:0/3863331811 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f26ac008040 msgr2=0x7f26ac054410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:21:26.780 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.780+0000 7f26c6f36980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:21:26.780 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.780+0000 7f26c6f36980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:21:26.780 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.780+0000 7f26c6f36980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:21:26.781 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.780+0000 7f26c6f36980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:21:26.781 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.780+0000 7f26c6f36980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:21:26.781 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.780+0000 7f26c6f36980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:21:26.781 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.780+0000 7f26c6f36980 20 rados->read ofs=0 len=0 2024-05-30T16:21:26.781 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.780+0000 7f26c6f36980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:26.781 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.780+0000 7f26c6f36980 20 rados->read ofs=0 len=0 2024-05-30T16:21:26.781 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.784+0000 7f26c6f36980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:21:26.782 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.784+0000 7f26c6f36980 20 rados->read ofs=0 len=0 2024-05-30T16:21:26.782 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.784+0000 7f26c6f36980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:21:26.783 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.784+0000 7f26c6f36980 20 zone default found 2024-05-30T16:21:26.783 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.784+0000 7f26c6f36980 4 Realm: () 2024-05-30T16:21:26.783 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.784+0000 7f26c6f36980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:21:26.783 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.784+0000 7f26c6f36980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:21:26.783 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.784+0000 7f26c6f36980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:21:26.783 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.784+0000 7f26c6f36980 20 zonegroup default 2024-05-30T16:21:26.783 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.784+0000 7f26c6f36980 20 rados->read ofs=0 len=0 2024-05-30T16:21:26.783 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.784+0000 7f26c6f36980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:26.784 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.784+0000 7f26c6f36980 20 rados->read ofs=0 len=0 2024-05-30T16:21:26.784 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.784+0000 7f26c6f36980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:26.784 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.784+0000 7f26c6f36980 20 rados->read ofs=0 len=0 2024-05-30T16:21:26.785 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.784+0000 7f26c6f36980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:26.785 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.784+0000 7f26c6f36980 20 started sync module instance, tier type = 2024-05-30T16:21:26.785 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.784+0000 7f26c6f36980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:21:26.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.808+0000 7f26c6f36980 20 add_watcher() i=0 2024-05-30T16:21:26.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.808+0000 7f26c6f36980 20 add_watcher() i=1 2024-05-30T16:21:26.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.808+0000 7f26c6f36980 20 add_watcher() i=2 2024-05-30T16:21:26.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.808+0000 7f26c6f36980 20 add_watcher() i=3 2024-05-30T16:21:26.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.808+0000 7f26c6f36980 20 add_watcher() i=4 2024-05-30T16:21:26.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.808+0000 7f26c6f36980 20 add_watcher() i=5 2024-05-30T16:21:26.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.808+0000 7f26c6f36980 20 add_watcher() i=6 2024-05-30T16:21:26.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.812+0000 7f26c6f36980 20 add_watcher() i=7 2024-05-30T16:21:26.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.812+0000 7f26c6f36980 2 all 8 watchers are set, enabling cache 2024-05-30T16:21:26.815 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.816+0000 7f257afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:21:26.817 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.816+0000 7f26c6f36980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:21:26.817 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.816+0000 7f26c6f36980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:21:26.817 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.816+0000 7f26c6f36980 5 note: GC not initialized 2024-05-30T16:21:26.818 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.816+0000 7f2579fe3640 20 reqs_thread_entry: start 2024-05-30T16:21:26.818 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.820+0000 7f26c6f36980 20 init_complete bucket index max shards: 11 2024-05-30T16:21:26.818 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.820+0000 7f26c6f36980 20 Filter name: none 2024-05-30T16:21:26.818 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.820+0000 7f24eb7fe640 20 reqs_thread_entry: start 2024-05-30T16:21:26.819 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.820+0000 7f26c6f36980 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-05-30T16:21:26.819 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.820+0000 7f26c6f36980 20 rados->read ofs=0 len=0 2024-05-30T16:21:26.819 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.820+0000 7f26c6f36980 20 rados_obj.operate() r=0 bl.length=213 2024-05-30T16:21:26.819 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.820+0000 7f26c6f36980 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-05-30T16:21:26.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.820+0000 7f26c6f36980 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-05-30T16:21:26.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.820+0000 7f26c6f36980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 : miss 2024-05-30T16:21:26.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.820+0000 7f26c6f36980 20 rados->read ofs=0 len=0 2024-05-30T16:21:26.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.820+0000 7f26c6f36980 20 rados_obj.operate() r=0 bl.length=459 2024-05-30T16:21:26.821 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.820+0000 7f26c6f36980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 info.flags=0x17 2024-05-30T16:21:26.821 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.820+0000 7f26c6f36980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 to cache LRU end 2024-05-30T16:21:26.821 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.820+0000 7f26c6f36980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:21:26.821 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.820+0000 7f26c6f36980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:21:26.821 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.820+0000 7f26c6f36980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 2024-05-30T16:21:26.823 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.824+0000 7f26c6f36980 10 init_target_layout removing existing target index objects from a previous reshard attempt 2024-05-30T16:21:26.839 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.840+0000 7f26c6f36980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 info.flags=0x17 2024-05-30T16:21:26.839 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.840+0000 7f26c6f36980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 to cache LRU end 2024-05-30T16:21:26.839 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.840+0000 7f26c6f36980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:21:26.839 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.840+0000 7f26c6f36980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24, ofs0, ns] 2024-05-30T16:21:26.839 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.840+0000 7f259bfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431626 cookie 94743172962640 notifier 6561 bl.length()=991 2024-05-30T16:21:26.840 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.840+0000 7f259bfff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 info.flags=0x17 2024-05-30T16:21:26.840 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.840+0000 7f259bfff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 to cache LRU end 2024-05-30T16:21:26.840 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.840+0000 7f259bfff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:21:26.842 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.844+0000 7f26c6f36980 20 handle_bi_update(): bucket=:abort-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24]): orig_sources= new_sources= 2024-05-30T16:21:26.842 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.844+0000 7f26c6f36980 20 handle_bi_update(): bucket=:abort-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24]): potential sources added=[] removed=[] 2024-05-30T16:21:26.842 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.844+0000 7f26c6f36980 20 handle_bi_update(): bucket=:abort-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24]): orig_dests= new_dests= 2024-05-30T16:21:26.842 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.844+0000 7f26c6f36980 20 handle_bi_update(): bucket=:abort-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24]): potential dests added=[] removed=[] 2024-05-30T16:21:26.847 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.848+0000 7f26c6f36980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.2.0 2024-05-30T16:21:26.848 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.848+0000 7f26c6f36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.2.0 2024-05-30T16:21:26.848 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.848+0000 7f26c6f36980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.2.1 2024-05-30T16:21:26.848 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.848+0000 7f26c6f36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.2.1 2024-05-30T16:21:26.848 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.848+0000 7f26c6f36980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.2.2 2024-05-30T16:21:26.848 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.848+0000 7f26c6f36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.2.2 2024-05-30T16:21:26.848 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.848+0000 7f26c6f36980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.2.3 2024-05-30T16:21:26.848 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.848+0000 7f26c6f36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.2.3 2024-05-30T16:21:26.848 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.848+0000 7f26c6f36980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.2.4 2024-05-30T16:21:26.848 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.848+0000 7f26c6f36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.2.4 2024-05-30T16:21:26.848 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.848+0000 7f26c6f36980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.2.5 2024-05-30T16:21:26.848 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.848+0000 7f26c6f36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.2.5 2024-05-30T16:21:26.849 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.848+0000 7f26c6f36980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.2.6 2024-05-30T16:21:26.849 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.848+0000 7f26c6f36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.2.6 2024-05-30T16:21:26.849 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.848+0000 7f26c6f36980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.2.7 2024-05-30T16:21:26.849 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.848+0000 7f26c6f36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.2.7 2024-05-30T16:21:26.849 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.848+0000 7f26c6f36980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.2.8 2024-05-30T16:21:26.849 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.848+0000 7f26c6f36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.2.8 2024-05-30T16:21:26.849 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.848+0000 7f26c6f36980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.2.9 2024-05-30T16:21:26.849 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.848+0000 7f26c6f36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.2.9 2024-05-30T16:21:26.849 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.848+0000 7f26c6f36980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.2.10 2024-05-30T16:21:26.849 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.848+0000 7f26c6f36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.2.10 2024-05-30T16:21:26.850 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.848+0000 7f26c6f36980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.2.11 2024-05-30T16:21:26.850 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.848+0000 7f26c6f36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.2.11 2024-05-30T16:21:26.850 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.848+0000 7f26c6f36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.0 2024-05-30T16:21:26.851 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.848+0000 7f26c6f36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.1 2024-05-30T16:21:26.851 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.848+0000 7f26c6f36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.2 2024-05-30T16:21:26.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.852+0000 7f26c6f36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.3 2024-05-30T16:21:26.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.852+0000 7f26c6f36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.4 2024-05-30T16:21:26.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.852+0000 7f26c6f36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.5 2024-05-30T16:21:26.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.852+0000 7f26c6f36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.6 2024-05-30T16:21:26.854 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.856+0000 7f26c6f36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.7 2024-05-30T16:21:26.854 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.856+0000 7f26c6f36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.8 2024-05-30T16:21:26.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.856+0000 7f26c6f36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.9 2024-05-30T16:21:26.856 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.856+0000 7f26c6f36980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24.10 2024-05-30T16:21:26.864 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.864+0000 7f26c6f36980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 : hit (requested=0x17, cached=0x17) 2024-05-30T16:21:26.864 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.864+0000 7f26c6f36980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 2024-05-30T16:21:26.867 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.868+0000 7f26c6f36980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 info.flags=0x17 2024-05-30T16:21:26.867 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.868+0000 7f26c6f36980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 to cache LRU end 2024-05-30T16:21:26.868 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.868+0000 7f26c6f36980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:21:26.868 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.868+0000 7f26c6f36980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24, ofs0, ns] 2024-05-30T16:21:26.868 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.868+0000 7f259bfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431627 cookie 94743172962640 notifier 6561 bl.length()=959 2024-05-30T16:21:26.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.868+0000 7f259bfff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 info.flags=0x17 2024-05-30T16:21:26.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.868+0000 7f259bfff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 to cache LRU end 2024-05-30T16:21:26.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.868+0000 7f259bfff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:21:26.871 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.872+0000 7f26c6f36980 20 handle_bi_update(): bucket=:abort-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24]): orig_sources= new_sources= 2024-05-30T16:21:26.871 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.872+0000 7f26c6f36980 20 handle_bi_update(): bucket=:abort-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24]): potential sources added=[] removed=[] 2024-05-30T16:21:26.871 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.872+0000 7f26c6f36980 20 handle_bi_update(): bucket=:abort-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24]): orig_dests= new_dests= 2024-05-30T16:21:26.871 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.872+0000 7f26c6f36980 20 handle_bi_update(): bucket=:abort-at-block-writes[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24]): potential dests added=[] removed=[] 2024-05-30T16:21:26.875 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.876+0000 7f26c6f36980 1 execute INFO: reshard of bucket "abort-at-block-writes" completed successfully 2024-05-30T16:21:26.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.880+0000 7f257afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:21:26.884 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.884+0000 7f26c6f36980 20 remove_watcher() i=0 2024-05-30T16:21:26.885 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.884+0000 7f26c6f36980 2 removed watcher, disabling cache 2024-05-30T16:21:26.887 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.888+0000 7f26c6f36980 20 remove_watcher() i=1 2024-05-30T16:21:26.890 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.892+0000 7f26c6f36980 20 remove_watcher() i=2 2024-05-30T16:21:26.892 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.892+0000 7f26c6f36980 20 remove_watcher() i=3 2024-05-30T16:21:26.894 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.896+0000 7f26c6f36980 20 remove_watcher() i=4 2024-05-30T16:21:26.895 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.896+0000 7f26c6f36980 20 remove_watcher() i=5 2024-05-30T16:21:26.897 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.896+0000 7f26c6f36980 20 remove_watcher() i=6 2024-05-30T16:21:26.899 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.900+0000 7f26c6f36980 20 remove_watcher() i=7 2024-05-30T16:21:26.916 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket abort-at-block-writes 2024-05-30T16:21:26.943 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:21:26.944 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:21:26.990 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.992+0000 7fbb22174980 20 rados->read ofs=0 len=0 2024-05-30T16:21:26.992 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.992+0000 7fbb22174980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:26.992 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.992+0000 7fbb22174980 20 realm 2024-05-30T16:21:26.992 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.992+0000 7fbb22174980 20 rados->read ofs=0 len=0 2024-05-30T16:21:26.993 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.992+0000 7fbb22174980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:26.993 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.992+0000 7fbb22174980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:21:26.993 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.992+0000 7fbb22174980 20 rados->read ofs=0 len=0 2024-05-30T16:21:26.993 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.992+0000 7fbb22174980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:26.993 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.992+0000 7fbb22174980 20 rados->read ofs=0 len=0 2024-05-30T16:21:26.994 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.996+0000 7fbb22174980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:21:26.994 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.996+0000 7fbb22174980 20 rados->read ofs=0 len=0 2024-05-30T16:21:26.995 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.996+0000 7fbb22174980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:21:26.995 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:26.996+0000 7fbb22174980 20 searching for the correct realm 2024-05-30T16:21:26.999 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.000+0000 7fbb1f484640 0 -- 172.21.15.6:0/1971284322 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7fbb100d5070 msgr2=0x7fbb100ed920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:21:27.214 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.216+0000 7fbb1f484640 0 -- 172.21.15.6:0/1971284322 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7fb9e807e590 msgr2=0x7fb9e80809c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:21:27.435 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.436+0000 7fbb22174980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:21:27.435 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.436+0000 7fbb22174980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:21:27.435 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.436+0000 7fbb22174980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:21:27.435 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.436+0000 7fbb22174980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:21:27.435 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.436+0000 7fbb22174980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:21:27.435 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.436+0000 7fbb22174980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:21:27.435 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.436+0000 7fbb22174980 20 rados->read ofs=0 len=0 2024-05-30T16:21:27.436 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.436+0000 7fbb22174980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:27.436 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.436+0000 7fbb22174980 20 rados->read ofs=0 len=0 2024-05-30T16:21:27.436 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.436+0000 7fbb22174980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:21:27.436 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.436+0000 7fbb22174980 20 rados->read ofs=0 len=0 2024-05-30T16:21:27.437 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.436+0000 7fbb22174980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:21:27.437 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.436+0000 7fbb22174980 20 zone default found 2024-05-30T16:21:27.437 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.436+0000 7fbb22174980 4 Realm: () 2024-05-30T16:21:27.437 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.436+0000 7fbb22174980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:21:27.437 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.436+0000 7fbb22174980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:21:27.437 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.436+0000 7fbb22174980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:21:27.438 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.436+0000 7fbb22174980 20 zonegroup default 2024-05-30T16:21:27.438 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.436+0000 7fbb22174980 20 rados->read ofs=0 len=0 2024-05-30T16:21:27.439 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.440+0000 7fbb22174980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:27.439 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.440+0000 7fbb22174980 20 rados->read ofs=0 len=0 2024-05-30T16:21:27.439 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.440+0000 7fbb22174980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:27.439 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.440+0000 7fbb22174980 20 rados->read ofs=0 len=0 2024-05-30T16:21:27.440 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.440+0000 7fbb22174980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:27.440 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.440+0000 7fbb22174980 20 started sync module instance, tier type = 2024-05-30T16:21:27.440 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.440+0000 7fbb22174980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:21:27.464 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.464+0000 7fbb22174980 20 add_watcher() i=0 2024-05-30T16:21:27.464 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.464+0000 7fbb22174980 20 add_watcher() i=1 2024-05-30T16:21:27.464 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.464+0000 7fbb22174980 20 add_watcher() i=2 2024-05-30T16:21:27.464 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.464+0000 7fbb22174980 20 add_watcher() i=3 2024-05-30T16:21:27.464 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.464+0000 7fbb22174980 20 add_watcher() i=4 2024-05-30T16:21:27.464 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.464+0000 7fbb22174980 20 add_watcher() i=5 2024-05-30T16:21:27.464 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.464+0000 7fbb22174980 20 add_watcher() i=6 2024-05-30T16:21:27.466 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.468+0000 7fbb22174980 20 add_watcher() i=7 2024-05-30T16:21:27.466 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.468+0000 7fbb22174980 2 all 8 watchers are set, enabling cache 2024-05-30T16:21:27.471 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.472+0000 7fb9d5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:21:27.472 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.472+0000 7fbb22174980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:21:27.472 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.472+0000 7fbb22174980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:21:27.472 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.472+0000 7fbb22174980 5 note: GC not initialized 2024-05-30T16:21:27.472 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.472+0000 7fb9d4fe1640 20 reqs_thread_entry: start 2024-05-30T16:21:27.473 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.472+0000 7fbb22174980 20 init_complete bucket index max shards: 11 2024-05-30T16:21:27.473 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.472+0000 7fbb22174980 20 Filter name: none 2024-05-30T16:21:27.473 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.472+0000 7fb94e7fc640 20 reqs_thread_entry: start 2024-05-30T16:21:27.474 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.476+0000 7fbb22174980 20 rados->read ofs=0 len=0 2024-05-30T16:21:27.474 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.476+0000 7fbb22174980 20 rados_obj.operate() r=0 bl.length=213 2024-05-30T16:21:27.474 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.476+0000 7fbb22174980 20 rados->read ofs=0 len=0 2024-05-30T16:21:27.475 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.476+0000 7fbb22174980 20 rados_obj.operate() r=0 bl.length=427 2024-05-30T16:21:27.475 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.476+0000 7fbb22174980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:21:27.476 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.476+0000 7fbb22174980 20 rados->read ofs=0 len=0 2024-05-30T16:21:27.476 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.476+0000 7fbb22174980 20 rados_obj.operate() r=0 bl.length=213 2024-05-30T16:21:27.476 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.476+0000 7fbb22174980 20 rados->read ofs=0 len=0 2024-05-30T16:21:27.477 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.476+0000 7fbb22174980 20 rados_obj.operate() r=0 bl.length=427 2024-05-30T16:21:27.477 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.476+0000 7fbb22174980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:21:27.481 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.480+0000 7fb9d5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:21:27.486 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.488+0000 7fbb22174980 20 remove_watcher() i=0 2024-05-30T16:21:27.486 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.488+0000 7fbb22174980 2 removed watcher, disabling cache 2024-05-30T16:21:27.489 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.488+0000 7fbb22174980 20 remove_watcher() i=1 2024-05-30T16:21:27.491 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.492+0000 7fbb22174980 20 remove_watcher() i=2 2024-05-30T16:21:27.493 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.492+0000 7fbb22174980 20 remove_watcher() i=3 2024-05-30T16:21:27.496 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.496+0000 7fbb22174980 20 remove_watcher() i=4 2024-05-30T16:21:27.497 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.500+0000 7fbb22174980 20 remove_watcher() i=5 2024-05-30T16:21:27.499 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.500+0000 7fbb22174980 20 remove_watcher() i=6 2024-05-30T16:21:27.501 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:27.500+0000 7fbb22174980 20 remove_watcher() i=7 2024-05-30T16:21:27.516 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:27.516 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:21:27.516 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: abort-at-block-writes id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.24 num_objects: 19 size_kb: 1 num_shards: 12 2024-05-30T16:21:27.516 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:28.473 INFO:tasks.workunit.client.0.smithi006.stderr:TEST: reshard bucket with EIO injected at commit_target_layout 2024-05-30T16:21:28.473 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:30.549 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-05-30T16:21:30.583 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:21:30.584 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:21:30.630 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:30.632+0000 7fd3fc00e980 20 rados->read ofs=0 len=0 2024-05-30T16:21:30.835 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:30.836+0000 7fd3fc00e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:30.835 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:30.836+0000 7fd3fc00e980 20 realm 2024-05-30T16:21:30.835 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:30.836+0000 7fd3fc00e980 20 rados->read ofs=0 len=0 2024-05-30T16:21:30.836 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:30.836+0000 7fd3fc00e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:30.836 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:30.836+0000 7fd3fc00e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:21:30.836 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:30.836+0000 7fd3fc00e980 20 rados->read ofs=0 len=0 2024-05-30T16:21:30.836 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:30.836+0000 7fd3fc00e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:30.836 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:30.836+0000 7fd3fc00e980 20 rados->read ofs=0 len=0 2024-05-30T16:21:30.838 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:30.840+0000 7fd3fc00e980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:21:30.838 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:30.840+0000 7fd3fc00e980 20 rados->read ofs=0 len=0 2024-05-30T16:21:30.839 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:30.840+0000 7fd3fc00e980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:21:30.839 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:30.840+0000 7fd3fc00e980 20 searching for the correct realm 2024-05-30T16:21:30.847 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:30.848+0000 7fd3f931e640 0 -- 172.21.15.6:0/1493237798 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7fd3ec050070 msgr2=0x7fd3ec016390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:21:30.858 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:30.860+0000 7fd3f9b1f640 0 -- 172.21.15.6:0/1493237798 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7fd3f00273d0 msgr2=0x7fd3f0027830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:21:31.076 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.076+0000 7fd3fc00e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:21:31.076 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.076+0000 7fd3fc00e980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:21:31.076 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.076+0000 7fd3fc00e980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:21:31.076 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.076+0000 7fd3fc00e980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:21:31.077 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.076+0000 7fd3fc00e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:21:31.077 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.076+0000 7fd3fc00e980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:21:31.077 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.076+0000 7fd3fc00e980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.077 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.076+0000 7fd3fc00e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:31.077 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.076+0000 7fd3fc00e980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.077 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.080+0000 7fd3fc00e980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:21:31.078 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.080+0000 7fd3fc00e980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.080+0000 7fd3fc00e980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:21:31.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.080+0000 7fd3fc00e980 20 zone default found 2024-05-30T16:21:31.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.080+0000 7fd3fc00e980 4 Realm: () 2024-05-30T16:21:31.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.080+0000 7fd3fc00e980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:21:31.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.080+0000 7fd3fc00e980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:21:31.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.080+0000 7fd3fc00e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:21:31.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.080+0000 7fd3fc00e980 20 zonegroup default 2024-05-30T16:21:31.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.080+0000 7fd3fc00e980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.080 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.080+0000 7fd3fc00e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:31.080 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.080+0000 7fd3fc00e980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.080 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.080+0000 7fd3fc00e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:31.081 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.080+0000 7fd3fc00e980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.082 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.084+0000 7fd3fc00e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:31.082 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.084+0000 7fd3fc00e980 20 started sync module instance, tier type = 2024-05-30T16:21:31.082 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.084+0000 7fd3fc00e980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:21:31.107 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.108+0000 7fd3fc00e980 20 add_watcher() i=0 2024-05-30T16:21:31.107 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.108+0000 7fd3fc00e980 20 add_watcher() i=1 2024-05-30T16:21:31.107 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.108+0000 7fd3fc00e980 20 add_watcher() i=2 2024-05-30T16:21:31.107 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.108+0000 7fd3fc00e980 20 add_watcher() i=3 2024-05-30T16:21:31.107 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.108+0000 7fd3fc00e980 20 add_watcher() i=4 2024-05-30T16:21:31.107 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.108+0000 7fd3fc00e980 20 add_watcher() i=5 2024-05-30T16:21:31.107 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.108+0000 7fd3fc00e980 20 add_watcher() i=6 2024-05-30T16:21:31.109 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.108+0000 7fd3fc00e980 20 add_watcher() i=7 2024-05-30T16:21:31.109 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.108+0000 7fd3fc00e980 2 all 8 watchers are set, enabling cache 2024-05-30T16:21:31.114 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.116+0000 7fd2ab7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:21:31.114 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.116+0000 7fd3fc00e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:21:31.115 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.116+0000 7fd3fc00e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:21:31.115 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.116+0000 7fd3fc00e980 5 note: GC not initialized 2024-05-30T16:21:31.115 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.116+0000 7fd2aa7fc640 20 reqs_thread_entry: start 2024-05-30T16:21:31.115 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.116+0000 7fd3fc00e980 20 init_complete bucket index max shards: 11 2024-05-30T16:21:31.115 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.116+0000 7fd3fc00e980 20 Filter name: none 2024-05-30T16:21:31.116 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.116+0000 7fd21bfff640 20 reqs_thread_entry: start 2024-05-30T16:21:31.116 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.116+0000 7fd3fc00e980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.117 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.116+0000 7fd3fc00e980 20 rados_obj.operate() r=0 bl.length=221 2024-05-30T16:21:31.117 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.116+0000 7fd3fc00e980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.118 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.120+0000 7fd3fc00e980 20 rados_obj.operate() r=0 bl.length=435 2024-05-30T16:21:31.118 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.120+0000 7fd3fc00e980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:21:31.118 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.120+0000 7fd3fc00e980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.119 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.120+0000 7fd3fc00e980 20 rados_obj.operate() r=0 bl.length=221 2024-05-30T16:21:31.119 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.120+0000 7fd3fc00e980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.120 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.120+0000 7fd3fc00e980 20 rados_obj.operate() r=0 bl.length=435 2024-05-30T16:21:31.120 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.120+0000 7fd3fc00e980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:21:31.123 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.124+0000 7fd2ab7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:21:31.128 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.128+0000 7fd3fc00e980 20 remove_watcher() i=0 2024-05-30T16:21:31.128 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.128+0000 7fd3fc00e980 2 removed watcher, disabling cache 2024-05-30T16:21:31.130 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.132+0000 7fd3fc00e980 20 remove_watcher() i=1 2024-05-30T16:21:31.133 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.132+0000 7fd3fc00e980 20 remove_watcher() i=2 2024-05-30T16:21:31.135 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.136+0000 7fd3fc00e980 20 remove_watcher() i=3 2024-05-30T16:21:31.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.140+0000 7fd3fc00e980 20 remove_watcher() i=4 2024-05-30T16:21:31.139 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.140+0000 7fd3fc00e980 20 remove_watcher() i=5 2024-05-30T16:21:31.141 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.140+0000 7fd3fc00e980 20 remove_watcher() i=6 2024-05-30T16:21:31.142 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.144+0000 7fd3fc00e980 20 remove_watcher() i=7 2024-05-30T16:21:31.158 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:31.158 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:21:31.158 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: error-at-commit-target-layout id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-30T16:21:31.158 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:31.159 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket reshard --bucket error-at-commit-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 --inject-error-at commit_target_layout 2024-05-30T16:21:31.196 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:21:31.196 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:21:31.242 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.244+0000 7fbb44c32980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.244+0000 7fbb44c32980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:31.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.244+0000 7fbb44c32980 20 realm 2024-05-30T16:21:31.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.244+0000 7fbb44c32980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.244+0000 7fbb44c32980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:31.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.244+0000 7fbb44c32980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:21:31.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.244+0000 7fbb44c32980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.245 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.244+0000 7fbb44c32980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:31.245 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.244+0000 7fbb44c32980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.246 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.248+0000 7fbb44c32980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:21:31.246 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.248+0000 7fbb44c32980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.247 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.248+0000 7fbb44c32980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:21:31.247 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.248+0000 7fbb44c32980 20 searching for the correct realm 2024-05-30T16:21:31.253 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.256+0000 7fbb41f42640 0 -- 172.21.15.6:0/642765141 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7fbb34083020 msgr2=0x7fbb34007960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:21:31.259 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.260+0000 7fbb42f44640 0 -- 172.21.15.6:0/642765141 >> [v2:172.21.15.204:6800/3687190393,v1:172.21.15.204:6801/3687190393] conn(0x7fbb2c0c6030 msgr2=0x7fbb2c0ac000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:21:31.267 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.268+0000 7fbb44c32980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:21:31.267 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.268+0000 7fbb44c32980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:21:31.267 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.268+0000 7fbb44c32980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:21:31.267 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.268+0000 7fbb44c32980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:21:31.267 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.268+0000 7fbb44c32980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:21:31.268 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.268+0000 7fbb44c32980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:21:31.268 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.268+0000 7fbb44c32980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.268 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.268+0000 7fbb44c32980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:31.268 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.268+0000 7fbb44c32980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.268 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.268+0000 7fbb44c32980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:21:31.268 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.268+0000 7fbb44c32980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.269 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.268+0000 7fbb44c32980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:21:31.269 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.268+0000 7fbb44c32980 20 zone default found 2024-05-30T16:21:31.269 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.268+0000 7fbb44c32980 4 Realm: () 2024-05-30T16:21:31.270 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.268+0000 7fbb44c32980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:21:31.270 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.268+0000 7fbb44c32980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:21:31.270 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.268+0000 7fbb44c32980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:21:31.270 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.268+0000 7fbb44c32980 20 zonegroup default 2024-05-30T16:21:31.270 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.268+0000 7fbb44c32980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.270 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.272+0000 7fbb44c32980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:31.270 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.272+0000 7fbb44c32980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.271 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.272+0000 7fbb44c32980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:31.271 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.272+0000 7fbb44c32980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.271 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.272+0000 7fbb44c32980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:31.271 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.272+0000 7fbb44c32980 20 started sync module instance, tier type = 2024-05-30T16:21:31.271 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.272+0000 7fbb44c32980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:21:31.287 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.288+0000 7fbb44c32980 20 add_watcher() i=0 2024-05-30T16:21:31.287 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.288+0000 7fbb44c32980 20 add_watcher() i=1 2024-05-30T16:21:31.287 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.288+0000 7fbb44c32980 20 add_watcher() i=2 2024-05-30T16:21:31.288 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.288+0000 7fbb44c32980 20 add_watcher() i=3 2024-05-30T16:21:31.288 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.288+0000 7fbb44c32980 20 add_watcher() i=4 2024-05-30T16:21:31.288 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.288+0000 7fbb44c32980 20 add_watcher() i=5 2024-05-30T16:21:31.288 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.288+0000 7fbb44c32980 20 add_watcher() i=6 2024-05-30T16:21:31.288 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.288+0000 7fbb44c32980 20 add_watcher() i=7 2024-05-30T16:21:31.288 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.288+0000 7fbb44c32980 2 all 8 watchers are set, enabling cache 2024-05-30T16:21:31.291 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.292+0000 7fb9f8fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:21:31.292 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.292+0000 7fbb44c32980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:21:31.292 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.292+0000 7fbb44c32980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:21:31.292 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.292+0000 7fbb44c32980 5 note: GC not initialized 2024-05-30T16:21:31.292 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.292+0000 7fb9737fe640 20 reqs_thread_entry: start 2024-05-30T16:21:31.292 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.292+0000 7fbb44c32980 20 init_complete bucket index max shards: 11 2024-05-30T16:21:31.292 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.292+0000 7fbb44c32980 20 Filter name: none 2024-05-30T16:21:31.292 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.292+0000 7fb9717fa640 20 reqs_thread_entry: start 2024-05-30T16:21:31.292 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.292+0000 7fbb44c32980 10 cache get: name=default.rgw.meta+root+error-at-commit-target-layout : miss 2024-05-30T16:21:31.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.292+0000 7fbb44c32980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.294 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.292+0000 7fbb44c32980 20 rados_obj.operate() r=0 bl.length=221 2024-05-30T16:21:31.294 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.292+0000 7fbb44c32980 10 cache put: name=default.rgw.meta+root+error-at-commit-target-layout info.flags=0x11 2024-05-30T16:21:31.294 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.292+0000 7fbb44c32980 10 adding default.rgw.meta+root+error-at-commit-target-layout to cache LRU end 2024-05-30T16:21:31.294 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.292+0000 7fbb44c32980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25 : miss 2024-05-30T16:21:31.294 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.292+0000 7fbb44c32980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.294 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.296+0000 7fbb44c32980 20 rados_obj.operate() r=0 bl.length=435 2024-05-30T16:21:31.294 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.296+0000 7fbb44c32980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25 info.flags=0x17 2024-05-30T16:21:31.294 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.296+0000 7fbb44c32980 10 adding default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25 to cache LRU end 2024-05-30T16:21:31.294 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.296+0000 7fbb44c32980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:21:31.294 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.296+0000 7fbb44c32980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:21:31.294 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.296+0000 7fbb44c32980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25 2024-05-30T16:21:31.304 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.304+0000 7fbb44c32980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25 info.flags=0x17 2024-05-30T16:21:31.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.304+0000 7fbb44c32980 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25 to cache LRU end 2024-05-30T16:21:31.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.304+0000 7fbb44c32980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:21:31.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.304+0000 7fbb44c32980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25, ofs0, ns] 2024-05-30T16:21:31.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.308+0000 7fba25ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431582 cookie 94278828725152 notifier 6618 bl.length()=1007 2024-05-30T16:21:31.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.308+0000 7fba25ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25 info.flags=0x17 2024-05-30T16:21:31.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.308+0000 7fba25ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25 to cache LRU end 2024-05-30T16:21:31.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.308+0000 7fba25ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:21:31.307 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.308+0000 7fbb44c32980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25]): orig_sources= new_sources= 2024-05-30T16:21:31.308 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.308+0000 7fbb44c32980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25]): potential sources added=[] removed=[] 2024-05-30T16:21:31.308 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.308+0000 7fbb44c32980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25]): orig_dests= new_dests= 2024-05-30T16:21:31.308 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.308+0000 7fbb44c32980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25]): potential dests added=[] removed=[] 2024-05-30T16:21:31.314 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.316+0000 7fbb44c32980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.1.0 2024-05-30T16:21:31.314 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.316+0000 7fbb44c32980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.1.0 2024-05-30T16:21:31.314 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.316+0000 7fbb44c32980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.1.1 2024-05-30T16:21:31.314 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.316+0000 7fbb44c32980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.1.1 2024-05-30T16:21:31.314 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.316+0000 7fbb44c32980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.1.2 2024-05-30T16:21:31.314 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.316+0000 7fbb44c32980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.1.2 2024-05-30T16:21:31.314 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.316+0000 7fbb44c32980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.1.3 2024-05-30T16:21:31.314 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.316+0000 7fbb44c32980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.1.3 2024-05-30T16:21:31.314 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.316+0000 7fbb44c32980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.1.4 2024-05-30T16:21:31.314 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.316+0000 7fbb44c32980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.1.4 2024-05-30T16:21:31.314 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.316+0000 7fbb44c32980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.1.5 2024-05-30T16:21:31.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.316+0000 7fbb44c32980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.1.5 2024-05-30T16:21:31.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.316+0000 7fbb44c32980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.1.6 2024-05-30T16:21:31.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.316+0000 7fbb44c32980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.1.6 2024-05-30T16:21:31.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.316+0000 7fbb44c32980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.1.7 2024-05-30T16:21:31.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.316+0000 7fbb44c32980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.1.7 2024-05-30T16:21:31.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.316+0000 7fbb44c32980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.1.8 2024-05-30T16:21:31.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.316+0000 7fbb44c32980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.1.8 2024-05-30T16:21:31.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.316+0000 7fbb44c32980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.1.9 2024-05-30T16:21:31.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.316+0000 7fbb44c32980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.1.9 2024-05-30T16:21:31.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.316+0000 7fbb44c32980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.1.10 2024-05-30T16:21:31.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.316+0000 7fbb44c32980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.1.10 2024-05-30T16:21:31.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.316+0000 7fbb44c32980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.1.11 2024-05-30T16:21:31.315 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.316+0000 7fbb44c32980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.1.11 2024-05-30T16:21:31.316 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.316+0000 7fbb44c32980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.0 2024-05-30T16:21:31.316 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.316+0000 7fbb44c32980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.1 2024-05-30T16:21:31.316 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.316+0000 7fbb44c32980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.2 2024-05-30T16:21:31.316 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.316+0000 7fbb44c32980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.3 2024-05-30T16:21:31.317 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.316+0000 7fbb44c32980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.4 2024-05-30T16:21:31.318 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.320+0000 7fbb44c32980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.5 2024-05-30T16:21:31.319 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.320+0000 7fbb44c32980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.6 2024-05-30T16:21:31.319 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.320+0000 7fbb44c32980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.7 2024-05-30T16:21:31.320 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.320+0000 7fbb44c32980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.8 2024-05-30T16:21:31.321 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.324+0000 7fbb44c32980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.9 2024-05-30T16:21:31.323 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.324+0000 7fbb44c32980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.10 2024-05-30T16:21:31.329 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.328+0000 7fbb44c32980 -1 Injecting error=-5 at location=commit_target_layout 2024-05-30T16:21:31.329 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.328+0000 7fbb44c32980 0 ERROR: commit_reshard failed to commit target index layout: (5) Input/output error 2024-05-30T16:21:31.338 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.340+0000 7fb9f8fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:21:31.343 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.344+0000 7fbb44c32980 20 remove_watcher() i=0 2024-05-30T16:21:31.343 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.344+0000 7fbb44c32980 2 removed watcher, disabling cache 2024-05-30T16:21:31.345 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.348+0000 7fbb44c32980 20 remove_watcher() i=1 2024-05-30T16:21:31.347 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.348+0000 7fbb44c32980 20 remove_watcher() i=2 2024-05-30T16:21:31.349 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.352+0000 7fbb44c32980 20 remove_watcher() i=3 2024-05-30T16:21:31.352 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.352+0000 7fbb44c32980 20 remove_watcher() i=4 2024-05-30T16:21:31.353 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.352+0000 7fbb44c32980 20 remove_watcher() i=5 2024-05-30T16:21:31.354 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.356+0000 7fbb44c32980 20 remove_watcher() i=6 2024-05-30T16:21:31.355 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.356+0000 7fbb44c32980 20 remove_watcher() i=7 2024-05-30T16:21:31.371 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-05-30T16:21:31.398 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:21:31.398 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:21:31.442 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.444+0000 7f2bda870980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.444 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.444+0000 7f2bda870980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:31.444 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.444+0000 7f2bda870980 20 realm 2024-05-30T16:21:31.444 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.444+0000 7f2bda870980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.444 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.444+0000 7f2bda870980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:31.444 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.444+0000 7f2bda870980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:21:31.444 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.444+0000 7f2bda870980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.444 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.444+0000 7f2bda870980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:31.444 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.444+0000 7f2bda870980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.446 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.448+0000 7f2bda870980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:21:31.446 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.448+0000 7f2bda870980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.446 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.448+0000 7f2bda870980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:21:31.446 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.448+0000 7f2bda870980 20 searching for the correct realm 2024-05-30T16:21:31.453 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.452+0000 7f2bd7b80640 0 -- 172.21.15.6:0/2191242110 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f2bc8064a20 msgr2=0x7f2bc807b040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:21:31.459 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.460+0000 7f2bd7b80640 0 -- 172.21.15.6:0/2191242110 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f2aa007e4c0 msgr2=0x7f2aa00808f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:21:31.466 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.468+0000 7f2bda870980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:21:31.467 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.468+0000 7f2bda870980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:21:31.467 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.468+0000 7f2bda870980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:21:31.467 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.468+0000 7f2bda870980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:21:31.467 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.468+0000 7f2bda870980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:21:31.467 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.468+0000 7f2bda870980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:21:31.467 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.468+0000 7f2bda870980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.467 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.468+0000 7f2bda870980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:31.468 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.468+0000 7f2bda870980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.468 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.468+0000 7f2bda870980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:21:31.469 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.468+0000 7f2bda870980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.469 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.468+0000 7f2bda870980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:21:31.469 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.468+0000 7f2bda870980 20 zone default found 2024-05-30T16:21:31.469 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.468+0000 7f2bda870980 4 Realm: () 2024-05-30T16:21:31.469 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.468+0000 7f2bda870980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:21:31.469 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.468+0000 7f2bda870980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:21:31.469 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.468+0000 7f2bda870980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:21:31.470 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.468+0000 7f2bda870980 20 zonegroup default 2024-05-30T16:21:31.470 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.468+0000 7f2bda870980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.470 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.468+0000 7f2bda870980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:31.470 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.468+0000 7f2bda870980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.471 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.468+0000 7f2bda870980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:31.471 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.468+0000 7f2bda870980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.471 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.468+0000 7f2bda870980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:31.472 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.468+0000 7f2bda870980 20 started sync module instance, tier type = 2024-05-30T16:21:31.472 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.468+0000 7f2bda870980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:21:31.485 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.484+0000 7f2bda870980 20 add_watcher() i=0 2024-05-30T16:21:31.485 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.484+0000 7f2bda870980 20 add_watcher() i=1 2024-05-30T16:21:31.485 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.484+0000 7f2bda870980 20 add_watcher() i=2 2024-05-30T16:21:31.485 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.484+0000 7f2bda870980 20 add_watcher() i=3 2024-05-30T16:21:31.485 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.484+0000 7f2bda870980 20 add_watcher() i=4 2024-05-30T16:21:31.486 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.484+0000 7f2bda870980 20 add_watcher() i=5 2024-05-30T16:21:31.486 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.484+0000 7f2bda870980 20 add_watcher() i=6 2024-05-30T16:21:31.486 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.488+0000 7f2bda870980 20 add_watcher() i=7 2024-05-30T16:21:31.486 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.488+0000 7f2bda870980 2 all 8 watchers are set, enabling cache 2024-05-30T16:21:31.489 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.488+0000 7f2a8e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:21:31.490 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.492+0000 7f2bda870980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:21:31.490 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.492+0000 7f2bda870980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:21:31.490 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.492+0000 7f2bda870980 5 note: GC not initialized 2024-05-30T16:21:31.490 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.492+0000 7f2a8d7e2640 20 reqs_thread_entry: start 2024-05-30T16:21:31.490 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.492+0000 7f2bda870980 20 init_complete bucket index max shards: 11 2024-05-30T16:21:31.490 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.492+0000 7f2bda870980 20 Filter name: none 2024-05-30T16:21:31.490 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.492+0000 7f2a06ffd640 20 reqs_thread_entry: start 2024-05-30T16:21:31.490 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.492+0000 7f2bda870980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.491 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.492+0000 7f2bda870980 20 rados_obj.operate() r=0 bl.length=221 2024-05-30T16:21:31.491 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.492+0000 7f2bda870980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.492 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.492+0000 7f2bda870980 20 rados_obj.operate() r=0 bl.length=467 2024-05-30T16:21:31.492 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.492+0000 7f2bda870980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:21:31.492 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.492+0000 7f2bda870980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.492 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.492+0000 7f2bda870980 20 rados_obj.operate() r=0 bl.length=221 2024-05-30T16:21:31.492 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.492+0000 7f2bda870980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.493 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.492+0000 7f2bda870980 20 rados_obj.operate() r=0 bl.length=467 2024-05-30T16:21:31.493 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.492+0000 7f2bda870980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:21:31.495 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.496+0000 7f2a8e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:21:31.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.500+0000 7f2bda870980 20 remove_watcher() i=0 2024-05-30T16:21:31.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.500+0000 7f2bda870980 2 removed watcher, disabling cache 2024-05-30T16:21:31.502 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.504+0000 7f2bda870980 20 remove_watcher() i=1 2024-05-30T16:21:31.504 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.504+0000 7f2bda870980 20 remove_watcher() i=2 2024-05-30T16:21:31.506 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.508+0000 7f2bda870980 20 remove_watcher() i=3 2024-05-30T16:21:31.508 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.508+0000 7f2bda870980 20 remove_watcher() i=4 2024-05-30T16:21:31.509 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.512+0000 7f2bda870980 20 remove_watcher() i=5 2024-05-30T16:21:31.511 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.512+0000 7f2bda870980 20 remove_watcher() i=6 2024-05-30T16:21:31.512 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.512+0000 7f2bda870980 20 remove_watcher() i=7 2024-05-30T16:21:31.526 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:31.526 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:21:31.526 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: error-at-commit-target-layout id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-30T16:21:31.526 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:31.623 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket reshard --bucket error-at-commit-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 2024-05-30T16:21:31.659 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:21:31.660 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:21:31.705 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.704+0000 7f91970f1980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.706 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.708+0000 7f91970f1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:31.706 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.708+0000 7f91970f1980 20 realm 2024-05-30T16:21:31.707 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.708+0000 7f91970f1980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.707 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.708+0000 7f91970f1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:31.707 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.708+0000 7f91970f1980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:21:31.707 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.708+0000 7f91970f1980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.707 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.708+0000 7f91970f1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:31.707 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.708+0000 7f91970f1980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.708 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.708+0000 7f91970f1980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:21:31.708 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.708+0000 7f91970f1980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.709 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.708+0000 7f91970f1980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:21:31.709 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.708+0000 7f91970f1980 20 searching for the correct realm 2024-05-30T16:21:31.717 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.716+0000 7f9195403640 0 -- 172.21.15.6:0/906732933 >> [v2:172.21.15.204:6800/3687190393,v1:172.21.15.204:6801/3687190393] conn(0x7f9188037040 msgr2=0x7f9188044410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:21:31.931 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.932+0000 7f9194c02640 0 -- 172.21.15.6:0/906732933 >> [v2:172.21.15.204:6816/2677861390,v1:172.21.15.204:6817/2677861390] conn(0x559280d5ce70 msgr2=0x559280d5f2a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:21:31.939 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.940+0000 7f91970f1980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:21:31.939 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.940+0000 7f91970f1980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:21:31.939 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.940+0000 7f91970f1980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:21:31.939 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.940+0000 7f91970f1980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:21:31.939 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.940+0000 7f91970f1980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:21:31.939 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.940+0000 7f91970f1980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:21:31.940 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.940+0000 7f91970f1980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.940 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.940+0000 7f91970f1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:31.940 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.940+0000 7f91970f1980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.940 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.940+0000 7f91970f1980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:21:31.941 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.940+0000 7f91970f1980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.942 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.944+0000 7f91970f1980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:21:31.942 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.944+0000 7f91970f1980 20 zone default found 2024-05-30T16:21:31.942 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.944+0000 7f91970f1980 4 Realm: () 2024-05-30T16:21:31.942 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.944+0000 7f91970f1980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:21:31.942 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.944+0000 7f91970f1980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:21:31.942 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.944+0000 7f91970f1980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:21:31.943 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.944+0000 7f91970f1980 20 zonegroup default 2024-05-30T16:21:31.943 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.944+0000 7f91970f1980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.943 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.944+0000 7f91970f1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:31.943 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.944+0000 7f91970f1980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.944 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.944+0000 7f91970f1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:31.944 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.944+0000 7f91970f1980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.945 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.944+0000 7f91970f1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:31.945 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.944+0000 7f91970f1980 20 started sync module instance, tier type = 2024-05-30T16:21:31.945 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.944+0000 7f91970f1980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:21:31.970 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.972+0000 7f91970f1980 20 add_watcher() i=0 2024-05-30T16:21:31.970 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.972+0000 7f91970f1980 20 add_watcher() i=1 2024-05-30T16:21:31.970 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.972+0000 7f91970f1980 20 add_watcher() i=2 2024-05-30T16:21:31.970 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.972+0000 7f91970f1980 20 add_watcher() i=3 2024-05-30T16:21:31.970 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.972+0000 7f91970f1980 20 add_watcher() i=4 2024-05-30T16:21:31.970 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.972+0000 7f91970f1980 20 add_watcher() i=5 2024-05-30T16:21:31.970 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.972+0000 7f91970f1980 20 add_watcher() i=6 2024-05-30T16:21:31.972 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.972+0000 7f91970f1980 20 add_watcher() i=7 2024-05-30T16:21:31.972 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.972+0000 7f91970f1980 2 all 8 watchers are set, enabling cache 2024-05-30T16:21:31.977 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.976+0000 7f904b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:21:31.978 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.980+0000 7f91970f1980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:21:31.978 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.980+0000 7f91970f1980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:21:31.978 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.980+0000 7f91970f1980 5 note: GC not initialized 2024-05-30T16:21:31.978 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.980+0000 7f904a7e4640 20 reqs_thread_entry: start 2024-05-30T16:21:31.978 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.980+0000 7f91970f1980 20 init_complete bucket index max shards: 11 2024-05-30T16:21:31.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.980+0000 7f91970f1980 20 Filter name: none 2024-05-30T16:21:31.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.980+0000 7f9043fff640 20 reqs_thread_entry: start 2024-05-30T16:21:31.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.980+0000 7f91970f1980 10 cache get: name=default.rgw.meta+root+error-at-commit-target-layout : miss 2024-05-30T16:21:31.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.980+0000 7f91970f1980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.981 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.980+0000 7f91970f1980 20 rados_obj.operate() r=0 bl.length=221 2024-05-30T16:21:31.981 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.980+0000 7f91970f1980 10 cache put: name=default.rgw.meta+root+error-at-commit-target-layout info.flags=0x11 2024-05-30T16:21:31.981 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.980+0000 7f91970f1980 10 adding default.rgw.meta+root+error-at-commit-target-layout to cache LRU end 2024-05-30T16:21:31.981 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.980+0000 7f91970f1980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25 : miss 2024-05-30T16:21:31.981 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.980+0000 7f91970f1980 20 rados->read ofs=0 len=0 2024-05-30T16:21:31.982 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.980+0000 7f91970f1980 20 rados_obj.operate() r=0 bl.length=467 2024-05-30T16:21:31.982 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.980+0000 7f91970f1980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25 info.flags=0x17 2024-05-30T16:21:31.982 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.980+0000 7f91970f1980 10 adding default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25 to cache LRU end 2024-05-30T16:21:31.982 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.980+0000 7f91970f1980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:21:31.982 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.980+0000 7f91970f1980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:21:31.982 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.980+0000 7f91970f1980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25 2024-05-30T16:21:31.984 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:31.984+0000 7f91970f1980 10 init_target_layout removing existing target index objects from a previous reshard attempt 2024-05-30T16:21:31.998 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.000+0000 7f91970f1980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25 info.flags=0x17 2024-05-30T16:21:31.998 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.000+0000 7f91970f1980 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25 to cache LRU end 2024-05-30T16:21:31.998 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.000+0000 7f91970f1980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:21:31.999 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.000+0000 7f91970f1980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25, ofs0, ns] 2024-05-30T16:21:31.999 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.000+0000 7f906bfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431583 cookie 94087715117616 notifier 6651 bl.length()=1007 2024-05-30T16:21:31.999 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.000+0000 7f906bfff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25 info.flags=0x17 2024-05-30T16:21:31.999 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.000+0000 7f906bfff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25 to cache LRU end 2024-05-30T16:21:31.999 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.000+0000 7f906bfff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:21:32.002 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.004+0000 7f91970f1980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25]): orig_sources= new_sources= 2024-05-30T16:21:32.002 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.004+0000 7f91970f1980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25]): potential sources added=[] removed=[] 2024-05-30T16:21:32.002 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.004+0000 7f91970f1980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25]): orig_dests= new_dests= 2024-05-30T16:21:32.002 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.004+0000 7f91970f1980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25]): potential dests added=[] removed=[] 2024-05-30T16:21:32.008 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.008+0000 7f91970f1980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.2.0 2024-05-30T16:21:32.008 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.008+0000 7f91970f1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.2.0 2024-05-30T16:21:32.008 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.008+0000 7f91970f1980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.2.1 2024-05-30T16:21:32.008 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.008+0000 7f91970f1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.2.1 2024-05-30T16:21:32.009 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.008+0000 7f91970f1980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.2.2 2024-05-30T16:21:32.009 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.008+0000 7f91970f1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.2.2 2024-05-30T16:21:32.009 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.008+0000 7f91970f1980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.2.3 2024-05-30T16:21:32.009 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.008+0000 7f91970f1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.2.3 2024-05-30T16:21:32.009 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.008+0000 7f91970f1980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.2.4 2024-05-30T16:21:32.009 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.008+0000 7f91970f1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.2.4 2024-05-30T16:21:32.009 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.008+0000 7f91970f1980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.2.5 2024-05-30T16:21:32.009 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.008+0000 7f91970f1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.2.5 2024-05-30T16:21:32.009 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.008+0000 7f91970f1980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.2.6 2024-05-30T16:21:32.009 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.008+0000 7f91970f1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.2.6 2024-05-30T16:21:32.009 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.008+0000 7f91970f1980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.2.7 2024-05-30T16:21:32.009 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.008+0000 7f91970f1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.2.7 2024-05-30T16:21:32.009 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.008+0000 7f91970f1980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.2.8 2024-05-30T16:21:32.009 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.008+0000 7f91970f1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.2.8 2024-05-30T16:21:32.010 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.008+0000 7f91970f1980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.2.9 2024-05-30T16:21:32.010 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.008+0000 7f91970f1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.2.9 2024-05-30T16:21:32.010 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.008+0000 7f91970f1980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.2.10 2024-05-30T16:21:32.010 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.008+0000 7f91970f1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.2.10 2024-05-30T16:21:32.010 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.008+0000 7f91970f1980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.2.11 2024-05-30T16:21:32.010 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.008+0000 7f91970f1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.2.11 2024-05-30T16:21:32.010 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.008+0000 7f91970f1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.0 2024-05-30T16:21:32.011 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.008+0000 7f91970f1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.1 2024-05-30T16:21:32.011 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.012+0000 7f91970f1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.2 2024-05-30T16:21:32.011 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.012+0000 7f91970f1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.3 2024-05-30T16:21:32.012 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.012+0000 7f91970f1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.4 2024-05-30T16:21:32.013 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.012+0000 7f91970f1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.5 2024-05-30T16:21:32.014 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.016+0000 7f91970f1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.6 2024-05-30T16:21:32.014 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.016+0000 7f91970f1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.7 2024-05-30T16:21:32.015 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.016+0000 7f91970f1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.8 2024-05-30T16:21:32.016 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.016+0000 7f91970f1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.9 2024-05-30T16:21:32.018 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.020+0000 7f91970f1980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25.10 2024-05-30T16:21:32.024 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.024+0000 7f91970f1980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25 : hit (requested=0x17, cached=0x17) 2024-05-30T16:21:32.024 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.024+0000 7f91970f1980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25 2024-05-30T16:21:32.027 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.028+0000 7f91970f1980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25 info.flags=0x17 2024-05-30T16:21:32.027 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.028+0000 7f91970f1980 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25 to cache LRU end 2024-05-30T16:21:32.027 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.028+0000 7f91970f1980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:21:32.028 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.028+0000 7f91970f1980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25, ofs0, ns] 2024-05-30T16:21:32.028 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.028+0000 7f906bfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431584 cookie 94087715117616 notifier 6651 bl.length()=975 2024-05-30T16:21:32.028 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.028+0000 7f906bfff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25 info.flags=0x17 2024-05-30T16:21:32.028 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.028+0000 7f906bfff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25 to cache LRU end 2024-05-30T16:21:32.029 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.028+0000 7f906bfff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:21:32.031 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.032+0000 7f91970f1980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25]): orig_sources= new_sources= 2024-05-30T16:21:32.031 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.032+0000 7f91970f1980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25]): potential sources added=[] removed=[] 2024-05-30T16:21:32.032 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.032+0000 7f91970f1980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25]): orig_dests= new_dests= 2024-05-30T16:21:32.032 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.032+0000 7f91970f1980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25]): potential dests added=[] removed=[] 2024-05-30T16:21:32.036 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.036+0000 7f91970f1980 1 execute INFO: reshard of bucket "error-at-commit-target-layout" completed successfully 2024-05-30T16:21:32.040 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.040+0000 7f904b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:21:32.045 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.048+0000 7f91970f1980 20 remove_watcher() i=0 2024-05-30T16:21:32.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.048+0000 7f91970f1980 2 removed watcher, disabling cache 2024-05-30T16:21:32.048 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.048+0000 7f91970f1980 20 remove_watcher() i=1 2024-05-30T16:21:32.051 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.052+0000 7f91970f1980 20 remove_watcher() i=2 2024-05-30T16:21:32.053 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.052+0000 7f91970f1980 20 remove_watcher() i=3 2024-05-30T16:21:32.056 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.056+0000 7f91970f1980 20 remove_watcher() i=4 2024-05-30T16:21:32.057 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.060+0000 7f91970f1980 20 remove_watcher() i=5 2024-05-30T16:21:32.059 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.060+0000 7f91970f1980 20 remove_watcher() i=6 2024-05-30T16:21:32.060 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.060+0000 7f91970f1980 20 remove_watcher() i=7 2024-05-30T16:21:32.076 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-05-30T16:21:32.108 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:21:32.108 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:21:32.154 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.156+0000 7f8642266980 20 rados->read ofs=0 len=0 2024-05-30T16:21:32.156 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.156+0000 7f8642266980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:32.156 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.156+0000 7f8642266980 20 realm 2024-05-30T16:21:32.156 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.156+0000 7f8642266980 20 rados->read ofs=0 len=0 2024-05-30T16:21:32.157 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.156+0000 7f8642266980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:32.157 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.156+0000 7f8642266980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:21:32.157 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.156+0000 7f8642266980 20 rados->read ofs=0 len=0 2024-05-30T16:21:32.157 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.156+0000 7f8642266980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:32.157 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.156+0000 7f8642266980 20 rados->read ofs=0 len=0 2024-05-30T16:21:32.159 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.160+0000 7f8642266980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:21:32.159 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.160+0000 7f8642266980 20 rados->read ofs=0 len=0 2024-05-30T16:21:32.159 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.160+0000 7f8642266980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:21:32.159 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.160+0000 7f8642266980 20 searching for the correct realm 2024-05-30T16:21:32.165 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.164+0000 7f8640578640 0 -- 172.21.15.6:0/3116704704 >> [v2:172.21.15.204:6823/3156311527,v1:172.21.15.204:6825/3156311527] conn(0x7f8634092040 msgr2=0x7f8634059740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:21:32.171 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.172+0000 7f863fd77640 0 -- 172.21.15.6:0/3116704704 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f8634041980 msgr2=0x7f863406b910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:21:32.180 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.180+0000 7f8642266980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:21:32.180 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.180+0000 7f8642266980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:21:32.180 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.180+0000 7f8642266980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:21:32.180 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.180+0000 7f8642266980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:21:32.180 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.180+0000 7f8642266980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:21:32.180 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.180+0000 7f8642266980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:21:32.180 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.180+0000 7f8642266980 20 rados->read ofs=0 len=0 2024-05-30T16:21:32.180 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.180+0000 7f8642266980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:32.180 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.180+0000 7f8642266980 20 rados->read ofs=0 len=0 2024-05-30T16:21:32.181 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.180+0000 7f8642266980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:21:32.181 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.180+0000 7f8642266980 20 rados->read ofs=0 len=0 2024-05-30T16:21:32.182 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.184+0000 7f8642266980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:21:32.182 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.184+0000 7f8642266980 20 zone default found 2024-05-30T16:21:32.182 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.184+0000 7f8642266980 4 Realm: () 2024-05-30T16:21:32.182 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.184+0000 7f8642266980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:21:32.182 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.184+0000 7f8642266980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:21:32.182 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.184+0000 7f8642266980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:21:32.182 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.184+0000 7f8642266980 20 zonegroup default 2024-05-30T16:21:32.182 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.184+0000 7f8642266980 20 rados->read ofs=0 len=0 2024-05-30T16:21:32.182 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.184+0000 7f8642266980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:32.182 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.184+0000 7f8642266980 20 rados->read ofs=0 len=0 2024-05-30T16:21:32.183 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.184+0000 7f8642266980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:32.183 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.184+0000 7f8642266980 20 rados->read ofs=0 len=0 2024-05-30T16:21:32.183 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.184+0000 7f8642266980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:32.183 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.184+0000 7f8642266980 20 started sync module instance, tier type = 2024-05-30T16:21:32.183 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.184+0000 7f8642266980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:21:32.198 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.200+0000 7f8642266980 20 add_watcher() i=0 2024-05-30T16:21:32.198 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.200+0000 7f8642266980 20 add_watcher() i=1 2024-05-30T16:21:32.198 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.200+0000 7f8642266980 20 add_watcher() i=2 2024-05-30T16:21:32.198 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.200+0000 7f8642266980 20 add_watcher() i=3 2024-05-30T16:21:32.198 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.200+0000 7f8642266980 20 add_watcher() i=4 2024-05-30T16:21:32.199 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.200+0000 7f8642266980 20 add_watcher() i=5 2024-05-30T16:21:32.199 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.200+0000 7f8642266980 20 add_watcher() i=6 2024-05-30T16:21:32.199 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.200+0000 7f8642266980 20 add_watcher() i=7 2024-05-30T16:21:32.199 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.200+0000 7f8642266980 2 all 8 watchers are set, enabling cache 2024-05-30T16:21:32.202 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.204+0000 7f84f5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:21:32.203 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.204+0000 7f8642266980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:21:32.203 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.204+0000 7f8642266980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:21:32.203 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.204+0000 7f8642266980 5 note: GC not initialized 2024-05-30T16:21:32.203 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.204+0000 7f84f4fe1640 20 reqs_thread_entry: start 2024-05-30T16:21:32.203 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.204+0000 7f8642266980 20 init_complete bucket index max shards: 11 2024-05-30T16:21:32.203 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.204+0000 7f8642266980 20 Filter name: none 2024-05-30T16:21:32.204 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.204+0000 7f846e7fc640 20 reqs_thread_entry: start 2024-05-30T16:21:32.204 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.204+0000 7f8642266980 20 rados->read ofs=0 len=0 2024-05-30T16:21:32.204 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.204+0000 7f8642266980 20 rados_obj.operate() r=0 bl.length=221 2024-05-30T16:21:32.205 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.204+0000 7f8642266980 20 rados->read ofs=0 len=0 2024-05-30T16:21:32.205 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.204+0000 7f8642266980 20 rados_obj.operate() r=0 bl.length=435 2024-05-30T16:21:32.205 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.204+0000 7f8642266980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:21:32.205 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.204+0000 7f8642266980 20 rados->read ofs=0 len=0 2024-05-30T16:21:32.206 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.208+0000 7f8642266980 20 rados_obj.operate() r=0 bl.length=221 2024-05-30T16:21:32.206 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.208+0000 7f8642266980 20 rados->read ofs=0 len=0 2024-05-30T16:21:32.207 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.208+0000 7f8642266980 20 rados_obj.operate() r=0 bl.length=435 2024-05-30T16:21:32.207 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.208+0000 7f8642266980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:21:32.210 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.212+0000 7f84f5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:21:32.214 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.216+0000 7f8642266980 20 remove_watcher() i=0 2024-05-30T16:21:32.215 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.216+0000 7f8642266980 2 removed watcher, disabling cache 2024-05-30T16:21:32.217 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.216+0000 7f8642266980 20 remove_watcher() i=1 2024-05-30T16:21:32.219 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.220+0000 7f8642266980 20 remove_watcher() i=2 2024-05-30T16:21:32.221 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.220+0000 7f8642266980 20 remove_watcher() i=3 2024-05-30T16:21:32.223 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.224+0000 7f8642266980 20 remove_watcher() i=4 2024-05-30T16:21:32.224 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.224+0000 7f8642266980 20 remove_watcher() i=5 2024-05-30T16:21:32.225 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.228+0000 7f8642266980 20 remove_watcher() i=6 2024-05-30T16:21:32.227 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:32.228+0000 7f8642266980 20 remove_watcher() i=7 2024-05-30T16:21:32.241 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:32.241 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:21:32.241 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: error-at-commit-target-layout id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.25 num_objects: 19 size_kb: 1 num_shards: 12 2024-05-30T16:21:32.241 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:33.103 INFO:tasks.workunit.client.0.smithi006.stderr:TEST: reshard bucket with ECANCELED injected at commit_target_layout 2024-05-30T16:21:33.103 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:35.159 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-05-30T16:21:35.194 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:21:35.194 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:21:35.241 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.240+0000 7f87b7926980 20 rados->read ofs=0 len=0 2024-05-30T16:21:35.243 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.244+0000 7f87b7926980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:35.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.244+0000 7f87b7926980 20 realm 2024-05-30T16:21:35.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.244+0000 7f87b7926980 20 rados->read ofs=0 len=0 2024-05-30T16:21:35.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.244+0000 7f87b7926980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:35.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.244+0000 7f87b7926980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:21:35.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.244+0000 7f87b7926980 20 rados->read ofs=0 len=0 2024-05-30T16:21:35.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.244+0000 7f87b7926980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:35.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.244+0000 7f87b7926980 20 rados->read ofs=0 len=0 2024-05-30T16:21:35.245 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.244+0000 7f87b7926980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:21:35.245 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.244+0000 7f87b7926980 20 rados->read ofs=0 len=0 2024-05-30T16:21:35.245 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.248+0000 7f87b7926980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:21:35.246 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.248+0000 7f87b7926980 20 searching for the correct realm 2024-05-30T16:21:35.251 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.252+0000 7f87b5c38640 0 -- 172.21.15.6:0/2059499601 >> [v2:172.21.15.204:6823/3156311527,v1:172.21.15.204:6825/3156311527] conn(0x7f87ac0e4040 msgr2=0x7f87ac0b7af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:21:35.257 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.260+0000 7f87b5437640 0 -- 172.21.15.6:0/2059499601 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f87ac12db80 msgr2=0x7f87ac007cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:21:35.477 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.476+0000 7f87b7926980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:21:35.477 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.476+0000 7f87b7926980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:21:35.477 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.476+0000 7f87b7926980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:21:35.477 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.476+0000 7f87b7926980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:21:35.477 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.476+0000 7f87b7926980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:21:35.477 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.476+0000 7f87b7926980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:21:35.477 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.476+0000 7f87b7926980 20 rados->read ofs=0 len=0 2024-05-30T16:21:35.478 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.476+0000 7f87b7926980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:35.478 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.476+0000 7f87b7926980 20 rados->read ofs=0 len=0 2024-05-30T16:21:35.478 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.480+0000 7f87b7926980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:21:35.478 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.480+0000 7f87b7926980 20 rados->read ofs=0 len=0 2024-05-30T16:21:35.479 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.480+0000 7f87b7926980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:21:35.479 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.480+0000 7f87b7926980 20 zone default found 2024-05-30T16:21:35.479 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.480+0000 7f87b7926980 4 Realm: () 2024-05-30T16:21:35.479 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.480+0000 7f87b7926980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:21:35.479 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.480+0000 7f87b7926980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:21:35.480 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.480+0000 7f87b7926980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:21:35.480 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.480+0000 7f87b7926980 20 zonegroup default 2024-05-30T16:21:35.480 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.480+0000 7f87b7926980 20 rados->read ofs=0 len=0 2024-05-30T16:21:35.480 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.480+0000 7f87b7926980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:35.480 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.480+0000 7f87b7926980 20 rados->read ofs=0 len=0 2024-05-30T16:21:35.481 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.480+0000 7f87b7926980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:35.481 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.480+0000 7f87b7926980 20 rados->read ofs=0 len=0 2024-05-30T16:21:35.481 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.484+0000 7f87b7926980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:35.482 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.484+0000 7f87b7926980 20 started sync module instance, tier type = 2024-05-30T16:21:35.482 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.484+0000 7f87b7926980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:21:35.506 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.508+0000 7f87b7926980 20 add_watcher() i=0 2024-05-30T16:21:35.506 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.508+0000 7f87b7926980 20 add_watcher() i=1 2024-05-30T16:21:35.506 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.508+0000 7f87b7926980 20 add_watcher() i=2 2024-05-30T16:21:35.506 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.508+0000 7f87b7926980 20 add_watcher() i=3 2024-05-30T16:21:35.507 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.508+0000 7f87b7926980 20 add_watcher() i=4 2024-05-30T16:21:35.507 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.508+0000 7f87b7926980 20 add_watcher() i=5 2024-05-30T16:21:35.507 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.508+0000 7f87b7926980 20 add_watcher() i=6 2024-05-30T16:21:35.509 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.508+0000 7f87b7926980 20 add_watcher() i=7 2024-05-30T16:21:35.509 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.508+0000 7f87b7926980 2 all 8 watchers are set, enabling cache 2024-05-30T16:21:35.514 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.516+0000 7f866b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:21:35.515 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.516+0000 7f87b7926980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:21:35.515 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.516+0000 7f87b7926980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:21:35.515 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.516+0000 7f87b7926980 5 note: GC not initialized 2024-05-30T16:21:35.515 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.516+0000 7f866a7e4640 20 reqs_thread_entry: start 2024-05-30T16:21:35.515 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.516+0000 7f87b7926980 20 init_complete bucket index max shards: 11 2024-05-30T16:21:35.516 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.516+0000 7f87b7926980 20 Filter name: none 2024-05-30T16:21:35.516 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.516+0000 7f85dbfff640 20 reqs_thread_entry: start 2024-05-30T16:21:35.516 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.516+0000 7f87b7926980 20 rados->read ofs=0 len=0 2024-05-30T16:21:35.517 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.520+0000 7f87b7926980 20 rados_obj.operate() r=0 bl.length=221 2024-05-30T16:21:35.518 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.520+0000 7f87b7926980 20 rados->read ofs=0 len=0 2024-05-30T16:21:35.519 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.520+0000 7f87b7926980 20 rados_obj.operate() r=0 bl.length=435 2024-05-30T16:21:35.519 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.520+0000 7f87b7926980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:21:35.519 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.520+0000 7f87b7926980 20 rados->read ofs=0 len=0 2024-05-30T16:21:35.521 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.520+0000 7f87b7926980 20 rados_obj.operate() r=0 bl.length=221 2024-05-30T16:21:35.521 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.520+0000 7f87b7926980 20 rados->read ofs=0 len=0 2024-05-30T16:21:35.522 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.524+0000 7f87b7926980 20 rados_obj.operate() r=0 bl.length=435 2024-05-30T16:21:35.522 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.524+0000 7f87b7926980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:21:35.525 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.524+0000 7f866b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:21:35.530 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.532+0000 7f87b7926980 20 remove_watcher() i=0 2024-05-30T16:21:35.531 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.532+0000 7f87b7926980 2 removed watcher, disabling cache 2024-05-30T16:21:35.533 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.532+0000 7f87b7926980 20 remove_watcher() i=1 2024-05-30T16:21:35.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.536+0000 7f87b7926980 20 remove_watcher() i=2 2024-05-30T16:21:35.538 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.540+0000 7f87b7926980 20 remove_watcher() i=3 2024-05-30T16:21:35.541 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.540+0000 7f87b7926980 20 remove_watcher() i=4 2024-05-30T16:21:35.543 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.544+0000 7f87b7926980 20 remove_watcher() i=5 2024-05-30T16:21:35.544 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.544+0000 7f87b7926980 20 remove_watcher() i=6 2024-05-30T16:21:35.546 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.548+0000 7f87b7926980 20 remove_watcher() i=7 2024-05-30T16:21:35.561 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:35.561 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:21:35.561 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: error-at-commit-target-layout id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-30T16:21:35.561 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:35.561 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket reshard --bucket error-at-commit-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 --inject-error-at commit_target_layout --inject-error-code 125 2024-05-30T16:21:35.598 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:21:35.598 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:21:35.643 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.644+0000 7f8bda078980 20 rados->read ofs=0 len=0 2024-05-30T16:21:35.645 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.644+0000 7f8bda078980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:35.645 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.644+0000 7f8bda078980 20 realm 2024-05-30T16:21:35.646 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.644+0000 7f8bda078980 20 rados->read ofs=0 len=0 2024-05-30T16:21:35.646 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.644+0000 7f8bda078980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:35.646 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.648+0000 7f8bda078980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:21:35.646 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.648+0000 7f8bda078980 20 rados->read ofs=0 len=0 2024-05-30T16:21:35.646 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.648+0000 7f8bda078980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:35.646 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.648+0000 7f8bda078980 20 rados->read ofs=0 len=0 2024-05-30T16:21:35.647 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.648+0000 7f8bda078980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:21:35.647 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.648+0000 7f8bda078980 20 rados->read ofs=0 len=0 2024-05-30T16:21:35.648 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.648+0000 7f8bda078980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:21:35.648 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.648+0000 7f8bda078980 20 searching for the correct realm 2024-05-30T16:21:35.654 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.656+0000 7f8bd838a640 0 -- 172.21.15.6:0/1995690087 >> [v2:172.21.15.204:6823/3156311527,v1:172.21.15.204:6825/3156311527] conn(0x7f8bcc0b9070 msgr2=0x7f8bcc0bb920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:21:35.660 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.660+0000 7f8bd7b89640 0 -- 172.21.15.6:0/1995690087 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f8bcc0ef920 msgr2=0x7f8bcc0b9450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:21:35.876 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.876+0000 7f8bda078980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:21:35.876 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.876+0000 7f8bda078980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:21:35.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.876+0000 7f8bda078980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:21:35.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.876+0000 7f8bda078980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:21:35.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.876+0000 7f8bda078980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:21:35.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.876+0000 7f8bda078980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:21:35.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.876+0000 7f8bda078980 20 rados->read ofs=0 len=0 2024-05-30T16:21:35.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.876+0000 7f8bda078980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:35.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.876+0000 7f8bda078980 20 rados->read ofs=0 len=0 2024-05-30T16:21:35.878 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.880+0000 7f8bda078980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:21:35.878 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.880+0000 7f8bda078980 20 rados->read ofs=0 len=0 2024-05-30T16:21:35.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.880+0000 7f8bda078980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:21:35.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.880+0000 7f8bda078980 20 zone default found 2024-05-30T16:21:35.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.880+0000 7f8bda078980 4 Realm: () 2024-05-30T16:21:35.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.880+0000 7f8bda078980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:21:35.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.880+0000 7f8bda078980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:21:35.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.880+0000 7f8bda078980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:21:35.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.880+0000 7f8bda078980 20 zonegroup default 2024-05-30T16:21:35.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.880+0000 7f8bda078980 20 rados->read ofs=0 len=0 2024-05-30T16:21:35.880 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.880+0000 7f8bda078980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:35.880 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.880+0000 7f8bda078980 20 rados->read ofs=0 len=0 2024-05-30T16:21:35.880 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.880+0000 7f8bda078980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:35.881 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.880+0000 7f8bda078980 20 rados->read ofs=0 len=0 2024-05-30T16:21:35.881 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.880+0000 7f8bda078980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:35.881 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.880+0000 7f8bda078980 20 started sync module instance, tier type = 2024-05-30T16:21:35.881 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.880+0000 7f8bda078980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:21:35.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.908+0000 7f8bda078980 20 add_watcher() i=0 2024-05-30T16:21:35.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.908+0000 7f8bda078980 20 add_watcher() i=1 2024-05-30T16:21:35.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.908+0000 7f8bda078980 20 add_watcher() i=2 2024-05-30T16:21:35.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.908+0000 7f8bda078980 20 add_watcher() i=3 2024-05-30T16:21:35.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.908+0000 7f8bda078980 20 add_watcher() i=4 2024-05-30T16:21:35.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.908+0000 7f8bda078980 20 add_watcher() i=5 2024-05-30T16:21:35.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.908+0000 7f8bda078980 20 add_watcher() i=6 2024-05-30T16:21:35.908 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.908+0000 7f8bda078980 20 add_watcher() i=7 2024-05-30T16:21:35.908 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.908+0000 7f8bda078980 2 all 8 watchers are set, enabling cache 2024-05-30T16:21:35.912 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.912+0000 7f8a8dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:21:35.913 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.912+0000 7f8bda078980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:21:35.913 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.912+0000 7f8bda078980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:21:35.914 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.916+0000 7f8bda078980 5 note: GC not initialized 2024-05-30T16:21:35.914 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.916+0000 7f8a8cfe1640 20 reqs_thread_entry: start 2024-05-30T16:21:35.914 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.916+0000 7f8bda078980 20 init_complete bucket index max shards: 11 2024-05-30T16:21:35.915 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.916+0000 7f8bda078980 20 Filter name: none 2024-05-30T16:21:35.915 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.916+0000 7f8a067fc640 20 reqs_thread_entry: start 2024-05-30T16:21:35.915 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.916+0000 7f8bda078980 10 cache get: name=default.rgw.meta+root+error-at-commit-target-layout : miss 2024-05-30T16:21:35.915 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.916+0000 7f8bda078980 20 rados->read ofs=0 len=0 2024-05-30T16:21:35.916 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.916+0000 7f8bda078980 20 rados_obj.operate() r=0 bl.length=221 2024-05-30T16:21:35.916 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.916+0000 7f8bda078980 10 cache put: name=default.rgw.meta+root+error-at-commit-target-layout info.flags=0x11 2024-05-30T16:21:35.916 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.916+0000 7f8bda078980 10 adding default.rgw.meta+root+error-at-commit-target-layout to cache LRU end 2024-05-30T16:21:35.917 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.916+0000 7f8bda078980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26 : miss 2024-05-30T16:21:35.917 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.916+0000 7f8bda078980 20 rados->read ofs=0 len=0 2024-05-30T16:21:35.917 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.916+0000 7f8bda078980 20 rados_obj.operate() r=0 bl.length=435 2024-05-30T16:21:35.918 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.916+0000 7f8bda078980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26 info.flags=0x17 2024-05-30T16:21:35.918 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.916+0000 7f8bda078980 10 adding default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26 to cache LRU end 2024-05-30T16:21:35.918 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.916+0000 7f8bda078980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:21:35.918 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.916+0000 7f8bda078980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:21:35.918 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.920+0000 7f8bda078980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26 2024-05-30T16:21:35.928 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.928+0000 7f8bda078980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26 info.flags=0x17 2024-05-30T16:21:35.929 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.928+0000 7f8bda078980 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26 to cache LRU end 2024-05-30T16:21:35.929 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.928+0000 7f8bda078980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:21:35.929 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.928+0000 7f8bda078980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26, ofs0, ns] 2024-05-30T16:21:35.929 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.928+0000 7f8ab67fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431587 cookie 94661652201680 notifier 6711 bl.length()=1007 2024-05-30T16:21:35.929 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.928+0000 7f8ab67fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26 info.flags=0x17 2024-05-30T16:21:35.929 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.928+0000 7f8ab67fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26 to cache LRU end 2024-05-30T16:21:35.930 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.928+0000 7f8ab67fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:21:35.934 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.936+0000 7f8bda078980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26]): orig_sources= new_sources= 2024-05-30T16:21:35.934 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.936+0000 7f8bda078980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26]): potential sources added=[] removed=[] 2024-05-30T16:21:35.934 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.936+0000 7f8bda078980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26]): orig_dests= new_dests= 2024-05-30T16:21:35.934 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.936+0000 7f8bda078980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26]): potential dests added=[] removed=[] 2024-05-30T16:21:35.939 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.940+0000 7f8bda078980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.1.0 2024-05-30T16:21:35.939 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.940+0000 7f8bda078980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.1.0 2024-05-30T16:21:35.940 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.940+0000 7f8bda078980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.1.1 2024-05-30T16:21:35.940 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.940+0000 7f8bda078980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.1.1 2024-05-30T16:21:35.940 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.940+0000 7f8bda078980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.1.2 2024-05-30T16:21:35.940 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.940+0000 7f8bda078980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.1.2 2024-05-30T16:21:35.940 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.940+0000 7f8bda078980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.1.3 2024-05-30T16:21:35.940 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.940+0000 7f8bda078980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.1.3 2024-05-30T16:21:35.940 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.940+0000 7f8bda078980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.1.4 2024-05-30T16:21:35.940 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.940+0000 7f8bda078980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.1.4 2024-05-30T16:21:35.940 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.940+0000 7f8bda078980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.1.5 2024-05-30T16:21:35.940 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.940+0000 7f8bda078980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.1.5 2024-05-30T16:21:35.940 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.940+0000 7f8bda078980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.1.6 2024-05-30T16:21:35.940 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.940+0000 7f8bda078980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.1.6 2024-05-30T16:21:35.940 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.940+0000 7f8bda078980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.1.7 2024-05-30T16:21:35.941 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.940+0000 7f8bda078980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.1.7 2024-05-30T16:21:35.941 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.940+0000 7f8bda078980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.1.8 2024-05-30T16:21:35.941 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.940+0000 7f8bda078980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.1.8 2024-05-30T16:21:35.941 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.940+0000 7f8bda078980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.1.9 2024-05-30T16:21:35.941 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.940+0000 7f8bda078980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.1.9 2024-05-30T16:21:35.941 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.940+0000 7f8bda078980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.1.10 2024-05-30T16:21:35.941 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.940+0000 7f8bda078980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.1.10 2024-05-30T16:21:35.941 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.940+0000 7f8bda078980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.1.11 2024-05-30T16:21:35.941 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.940+0000 7f8bda078980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.1.11 2024-05-30T16:21:35.941 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.940+0000 7f8bda078980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.0 2024-05-30T16:21:35.941 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.940+0000 7f8bda078980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.1 2024-05-30T16:21:35.942 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.940+0000 7f8bda078980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.2 2024-05-30T16:21:35.942 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.944+0000 7f8bda078980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.3 2024-05-30T16:21:35.942 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.944+0000 7f8bda078980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.4 2024-05-30T16:21:35.943 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.944+0000 7f8bda078980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.5 2024-05-30T16:21:35.945 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.944+0000 7f8bda078980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.6 2024-05-30T16:21:35.945 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.948+0000 7f8bda078980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.7 2024-05-30T16:21:35.947 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.948+0000 7f8bda078980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.8 2024-05-30T16:21:35.948 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.948+0000 7f8bda078980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.9 2024-05-30T16:21:35.949 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.952+0000 7f8bda078980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26.10 2024-05-30T16:21:35.955 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.956+0000 7f8bda078980 -1 Injecting error=-125 at location=commit_target_layout 2024-05-30T16:21:35.956 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.956+0000 7f8bda078980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26 : hit (requested=0x17, cached=0x17) 2024-05-30T16:21:35.956 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.956+0000 7f8bda078980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26 2024-05-30T16:21:35.958 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.960+0000 7f8bda078980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26 info.flags=0x17 2024-05-30T16:21:35.958 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.960+0000 7f8bda078980 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26 to cache LRU end 2024-05-30T16:21:35.958 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.960+0000 7f8bda078980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:21:35.959 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.960+0000 7f8bda078980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26, ofs0, ns] 2024-05-30T16:21:35.959 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.960+0000 7f8ab6ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431588 cookie 94661652201680 notifier 6711 bl.length()=975 2024-05-30T16:21:35.959 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.960+0000 7f8ab6ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26 info.flags=0x17 2024-05-30T16:21:35.959 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.960+0000 7f8ab6ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26 to cache LRU end 2024-05-30T16:21:35.960 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.960+0000 7f8ab6ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:21:35.962 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.964+0000 7f8bda078980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26]): orig_sources= new_sources= 2024-05-30T16:21:35.962 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.964+0000 7f8bda078980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26]): potential sources added=[] removed=[] 2024-05-30T16:21:35.962 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.964+0000 7f8bda078980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26]): orig_dests= new_dests= 2024-05-30T16:21:35.962 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.964+0000 7f8bda078980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26]): potential dests added=[] removed=[] 2024-05-30T16:21:35.968 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.968+0000 7f8bda078980 1 execute INFO: reshard of bucket "error-at-commit-target-layout" completed successfully 2024-05-30T16:21:35.971 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.972+0000 7f8a8dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:21:35.977 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.976+0000 7f8bda078980 20 remove_watcher() i=0 2024-05-30T16:21:35.977 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.976+0000 7f8bda078980 2 removed watcher, disabling cache 2024-05-30T16:21:35.979 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.980+0000 7f8bda078980 20 remove_watcher() i=1 2024-05-30T16:21:35.982 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.984+0000 7f8bda078980 20 remove_watcher() i=2 2024-05-30T16:21:35.984 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.984+0000 7f8bda078980 20 remove_watcher() i=3 2024-05-30T16:21:35.986 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.988+0000 7f8bda078980 20 remove_watcher() i=4 2024-05-30T16:21:35.988 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.988+0000 7f8bda078980 20 remove_watcher() i=5 2024-05-30T16:21:35.990 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.992+0000 7f8bda078980 20 remove_watcher() i=6 2024-05-30T16:21:35.992 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:35.992+0000 7f8bda078980 20 remove_watcher() i=7 2024-05-30T16:21:36.009 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-05-30T16:21:36.041 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:21:36.041 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:21:36.086 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.088+0000 7f8323276980 20 rados->read ofs=0 len=0 2024-05-30T16:21:36.088 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.088+0000 7f8323276980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:36.088 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.088+0000 7f8323276980 20 realm 2024-05-30T16:21:36.088 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.088+0000 7f8323276980 20 rados->read ofs=0 len=0 2024-05-30T16:21:36.088 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.088+0000 7f8323276980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:36.088 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.088+0000 7f8323276980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:21:36.088 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.088+0000 7f8323276980 20 rados->read ofs=0 len=0 2024-05-30T16:21:36.088 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.088+0000 7f8323276980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:36.088 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.088+0000 7f8323276980 20 rados->read ofs=0 len=0 2024-05-30T16:21:36.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.092+0000 7f8323276980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:21:36.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.092+0000 7f8323276980 20 rados->read ofs=0 len=0 2024-05-30T16:21:36.091 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.092+0000 7f8323276980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:21:36.091 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.092+0000 7f8323276980 20 searching for the correct realm 2024-05-30T16:21:36.097 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.096+0000 7f8320586640 0 -- 172.21.15.6:0/234877689 >> [v2:172.21.15.6:6819/1141406589,v1:172.21.15.6:6821/1141406589] conn(0x560d5e4cd8c0 msgr2=0x560d5e4cfcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:21:36.103 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.104+0000 7f8320d87640 0 -- 172.21.15.6:0/234877689 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f8314039a90 msgr2=0x7f8314049400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:21:36.111 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.112+0000 7f8323276980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:21:36.111 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.112+0000 7f8323276980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:21:36.111 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.112+0000 7f8323276980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:21:36.111 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.112+0000 7f8323276980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:21:36.111 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.112+0000 7f8323276980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:21:36.112 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.112+0000 7f8323276980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:21:36.112 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.112+0000 7f8323276980 20 rados->read ofs=0 len=0 2024-05-30T16:21:36.112 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.112+0000 7f8323276980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:36.112 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.112+0000 7f8323276980 20 rados->read ofs=0 len=0 2024-05-30T16:21:36.112 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.112+0000 7f8323276980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:21:36.112 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.112+0000 7f8323276980 20 rados->read ofs=0 len=0 2024-05-30T16:21:36.113 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.112+0000 7f8323276980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:21:36.114 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.112+0000 7f8323276980 20 zone default found 2024-05-30T16:21:36.114 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.112+0000 7f8323276980 4 Realm: () 2024-05-30T16:21:36.114 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.112+0000 7f8323276980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:21:36.114 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.112+0000 7f8323276980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:21:36.114 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.112+0000 7f8323276980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:21:36.114 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.112+0000 7f8323276980 20 zonegroup default 2024-05-30T16:21:36.114 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.112+0000 7f8323276980 20 rados->read ofs=0 len=0 2024-05-30T16:21:36.114 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.116+0000 7f8323276980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:36.114 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.116+0000 7f8323276980 20 rados->read ofs=0 len=0 2024-05-30T16:21:36.115 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.116+0000 7f8323276980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:36.115 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.116+0000 7f8323276980 20 rados->read ofs=0 len=0 2024-05-30T16:21:36.115 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.116+0000 7f8323276980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:36.115 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.116+0000 7f8323276980 20 started sync module instance, tier type = 2024-05-30T16:21:36.115 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.116+0000 7f8323276980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:21:36.131 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.132+0000 7f8323276980 20 add_watcher() i=0 2024-05-30T16:21:36.132 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.132+0000 7f8323276980 20 add_watcher() i=1 2024-05-30T16:21:36.132 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.132+0000 7f8323276980 20 add_watcher() i=2 2024-05-30T16:21:36.132 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.132+0000 7f8323276980 20 add_watcher() i=3 2024-05-30T16:21:36.132 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.132+0000 7f8323276980 20 add_watcher() i=4 2024-05-30T16:21:36.132 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.132+0000 7f8323276980 20 add_watcher() i=5 2024-05-30T16:21:36.132 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.132+0000 7f8323276980 20 add_watcher() i=6 2024-05-30T16:21:36.132 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.132+0000 7f8323276980 20 add_watcher() i=7 2024-05-30T16:21:36.132 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.132+0000 7f8323276980 2 all 8 watchers are set, enabling cache 2024-05-30T16:21:36.135 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.136+0000 7f81d6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:21:36.136 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.136+0000 7f8323276980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:21:36.136 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.136+0000 7f8323276980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:21:36.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.136+0000 7f8323276980 5 note: GC not initialized 2024-05-30T16:21:36.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.136+0000 7f81d5fe3640 20 reqs_thread_entry: start 2024-05-30T16:21:36.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.136+0000 7f8323276980 20 init_complete bucket index max shards: 11 2024-05-30T16:21:36.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.136+0000 7f8323276980 20 Filter name: none 2024-05-30T16:21:36.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.136+0000 7f81477fe640 20 reqs_thread_entry: start 2024-05-30T16:21:36.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.136+0000 7f8323276980 20 rados->read ofs=0 len=0 2024-05-30T16:21:36.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.136+0000 7f8323276980 20 rados_obj.operate() r=0 bl.length=221 2024-05-30T16:21:36.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.140+0000 7f8323276980 20 rados->read ofs=0 len=0 2024-05-30T16:21:36.139 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.140+0000 7f8323276980 20 rados_obj.operate() r=0 bl.length=435 2024-05-30T16:21:36.139 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.140+0000 7f8323276980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:21:36.139 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.140+0000 7f8323276980 20 rados->read ofs=0 len=0 2024-05-30T16:21:36.140 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.140+0000 7f8323276980 20 rados_obj.operate() r=0 bl.length=221 2024-05-30T16:21:36.140 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.140+0000 7f8323276980 20 rados->read ofs=0 len=0 2024-05-30T16:21:36.141 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.140+0000 7f8323276980 20 rados_obj.operate() r=0 bl.length=435 2024-05-30T16:21:36.141 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.140+0000 7f8323276980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:21:36.143 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.144+0000 7f81d6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:21:36.148 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.148+0000 7f8323276980 20 remove_watcher() i=0 2024-05-30T16:21:36.148 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.148+0000 7f8323276980 2 removed watcher, disabling cache 2024-05-30T16:21:36.150 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.152+0000 7f8323276980 20 remove_watcher() i=1 2024-05-30T16:21:36.152 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.152+0000 7f8323276980 20 remove_watcher() i=2 2024-05-30T16:21:36.155 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.156+0000 7f8323276980 20 remove_watcher() i=3 2024-05-30T16:21:36.157 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.156+0000 7f8323276980 20 remove_watcher() i=4 2024-05-30T16:21:36.158 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.160+0000 7f8323276980 20 remove_watcher() i=5 2024-05-30T16:21:36.160 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.160+0000 7f8323276980 20 remove_watcher() i=6 2024-05-30T16:21:36.161 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:36.160+0000 7f8323276980 20 remove_watcher() i=7 2024-05-30T16:21:36.175 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:36.175 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:21:36.176 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: error-at-commit-target-layout id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.26 num_objects: 20 size_kb: 1 num_shards: 12 2024-05-30T16:21:36.176 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:37.154 INFO:tasks.workunit.client.0.smithi006.stderr:TEST: reshard bucket with abort at commit_target_layout 2024-05-30T16:21:37.154 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:39.298 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket abort-at-commit-target-layout 2024-05-30T16:21:39.332 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:21:39.332 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:21:39.380 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.380+0000 7f7ce41a5980 20 rados->read ofs=0 len=0 2024-05-30T16:21:39.382 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.384+0000 7f7ce41a5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:39.382 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.384+0000 7f7ce41a5980 20 realm 2024-05-30T16:21:39.382 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.384+0000 7f7ce41a5980 20 rados->read ofs=0 len=0 2024-05-30T16:21:39.382 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.384+0000 7f7ce41a5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:39.382 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.384+0000 7f7ce41a5980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:21:39.383 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.384+0000 7f7ce41a5980 20 rados->read ofs=0 len=0 2024-05-30T16:21:39.383 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.384+0000 7f7ce41a5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:39.383 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.384+0000 7f7ce41a5980 20 rados->read ofs=0 len=0 2024-05-30T16:21:39.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.384+0000 7f7ce41a5980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:21:39.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.384+0000 7f7ce41a5980 20 rados->read ofs=0 len=0 2024-05-30T16:21:39.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.384+0000 7f7ce41a5980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:21:39.385 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.384+0000 7f7ce41a5980 20 searching for the correct realm 2024-05-30T16:21:39.390 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.392+0000 7f7ce14b5640 0 -- 172.21.15.6:0/3564568806 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f7cd4084050 msgr2=0x7f7cd4083ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:21:39.397 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.396+0000 7f7ce1cb6640 0 -- 172.21.15.6:0/3564568806 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f7cd80c6040 msgr2=0x7f7cd80ff400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:21:39.612 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.612+0000 7f7ce41a5980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:21:39.613 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.612+0000 7f7ce41a5980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:21:39.613 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.612+0000 7f7ce41a5980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:21:39.613 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.612+0000 7f7ce41a5980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:21:39.613 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.612+0000 7f7ce41a5980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:21:39.613 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.612+0000 7f7ce41a5980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:21:39.613 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.612+0000 7f7ce41a5980 20 rados->read ofs=0 len=0 2024-05-30T16:21:39.613 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.612+0000 7f7ce41a5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:39.613 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.612+0000 7f7ce41a5980 20 rados->read ofs=0 len=0 2024-05-30T16:21:39.614 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.616+0000 7f7ce41a5980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:21:39.614 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.616+0000 7f7ce41a5980 20 rados->read ofs=0 len=0 2024-05-30T16:21:39.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.616+0000 7f7ce41a5980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:21:39.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.616+0000 7f7ce41a5980 20 zone default found 2024-05-30T16:21:39.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.616+0000 7f7ce41a5980 4 Realm: () 2024-05-30T16:21:39.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.616+0000 7f7ce41a5980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:21:39.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.616+0000 7f7ce41a5980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:21:39.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.616+0000 7f7ce41a5980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:21:39.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.616+0000 7f7ce41a5980 20 zonegroup default 2024-05-30T16:21:39.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.616+0000 7f7ce41a5980 20 rados->read ofs=0 len=0 2024-05-30T16:21:39.616 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.616+0000 7f7ce41a5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:39.616 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.616+0000 7f7ce41a5980 20 rados->read ofs=0 len=0 2024-05-30T16:21:39.616 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.616+0000 7f7ce41a5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:39.617 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.616+0000 7f7ce41a5980 20 rados->read ofs=0 len=0 2024-05-30T16:21:39.617 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.616+0000 7f7ce41a5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:39.617 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.616+0000 7f7ce41a5980 20 started sync module instance, tier type = 2024-05-30T16:21:39.617 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.616+0000 7f7ce41a5980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:21:39.640 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.640+0000 7f7ce41a5980 20 add_watcher() i=0 2024-05-30T16:21:39.640 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.640+0000 7f7ce41a5980 20 add_watcher() i=1 2024-05-30T16:21:39.640 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.640+0000 7f7ce41a5980 20 add_watcher() i=2 2024-05-30T16:21:39.640 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.640+0000 7f7ce41a5980 20 add_watcher() i=3 2024-05-30T16:21:39.640 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.640+0000 7f7ce41a5980 20 add_watcher() i=4 2024-05-30T16:21:39.640 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.640+0000 7f7ce41a5980 20 add_watcher() i=5 2024-05-30T16:21:39.640 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.640+0000 7f7ce41a5980 20 add_watcher() i=6 2024-05-30T16:21:39.643 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.644+0000 7f7ce41a5980 20 add_watcher() i=7 2024-05-30T16:21:39.643 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.644+0000 7f7ce41a5980 2 all 8 watchers are set, enabling cache 2024-05-30T16:21:39.648 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.648+0000 7f7b937fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:21:39.649 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.648+0000 7f7ce41a5980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:21:39.649 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.648+0000 7f7ce41a5980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:21:39.649 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.648+0000 7f7ce41a5980 5 note: GC not initialized 2024-05-30T16:21:39.650 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.648+0000 7f7b927fc640 20 reqs_thread_entry: start 2024-05-30T16:21:39.650 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.652+0000 7f7ce41a5980 20 init_complete bucket index max shards: 11 2024-05-30T16:21:39.650 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.652+0000 7f7ce41a5980 20 Filter name: none 2024-05-30T16:21:39.651 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.652+0000 7f7b03fff640 20 reqs_thread_entry: start 2024-05-30T16:21:39.651 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.652+0000 7f7ce41a5980 20 rados->read ofs=0 len=0 2024-05-30T16:21:39.651 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.652+0000 7f7ce41a5980 20 rados_obj.operate() r=0 bl.length=221 2024-05-30T16:21:39.651 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.652+0000 7f7ce41a5980 20 rados->read ofs=0 len=0 2024-05-30T16:21:39.653 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.652+0000 7f7ce41a5980 20 rados_obj.operate() r=0 bl.length=435 2024-05-30T16:21:39.653 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.652+0000 7f7ce41a5980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:21:39.653 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.652+0000 7f7ce41a5980 20 rados->read ofs=0 len=0 2024-05-30T16:21:39.653 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.652+0000 7f7ce41a5980 20 rados_obj.operate() r=0 bl.length=221 2024-05-30T16:21:39.654 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.652+0000 7f7ce41a5980 20 rados->read ofs=0 len=0 2024-05-30T16:21:39.654 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.656+0000 7f7ce41a5980 20 rados_obj.operate() r=0 bl.length=435 2024-05-30T16:21:39.655 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.656+0000 7f7ce41a5980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:21:39.657 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.656+0000 7f7b937fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:21:39.662 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.664+0000 7f7ce41a5980 20 remove_watcher() i=0 2024-05-30T16:21:39.662 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.664+0000 7f7ce41a5980 2 removed watcher, disabling cache 2024-05-30T16:21:39.665 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.664+0000 7f7ce41a5980 20 remove_watcher() i=1 2024-05-30T16:21:39.667 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.668+0000 7f7ce41a5980 20 remove_watcher() i=2 2024-05-30T16:21:39.670 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.672+0000 7f7ce41a5980 20 remove_watcher() i=3 2024-05-30T16:21:39.672 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.672+0000 7f7ce41a5980 20 remove_watcher() i=4 2024-05-30T16:21:39.674 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.676+0000 7f7ce41a5980 20 remove_watcher() i=5 2024-05-30T16:21:39.676 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.676+0000 7f7ce41a5980 20 remove_watcher() i=6 2024-05-30T16:21:39.678 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.680+0000 7f7ce41a5980 20 remove_watcher() i=7 2024-05-30T16:21:39.694 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:39.694 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:21:39.694 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: abort-at-commit-target-layout id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-30T16:21:39.694 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:39.694 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket reshard --bucket abort-at-commit-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at commit_target_layout 2024-05-30T16:21:39.733 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:21:39.733 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:21:39.779 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.780+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:39.781 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.780+0000 7f7ffd551980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:39.781 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.780+0000 7f7ffd551980 20 realm 2024-05-30T16:21:39.781 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.780+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:39.781 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.780+0000 7f7ffd551980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:39.782 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.780+0000 7f7ffd551980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:21:39.782 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.780+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:39.782 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.780+0000 7f7ffd551980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:39.782 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.780+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:39.783 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.784+0000 7f7ffd551980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:21:39.783 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.784+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:39.784 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.784+0000 7f7ffd551980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:21:39.784 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.784+0000 7f7ffd551980 20 searching for the correct realm 2024-05-30T16:21:39.791 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:39.792+0000 7f7ffb863640 0 -- 172.21.15.6:0/308574472 >> [v2:172.21.15.204:6800/3687190393,v1:172.21.15.204:6801/3687190393] conn(0x7f7ff004c040 msgr2=0x7f7ff001a410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:21:40.007 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.008+0000 7f7ffb062640 0 -- 172.21.15.6:0/308574472 >> [v2:172.21.15.204:6816/2677861390,v1:172.21.15.204:6817/2677861390] conn(0x5607c4be17f0 msgr2=0x5607c4be3be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:21:40.016 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.016+0000 7f7ffd551980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:21:40.016 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.016+0000 7f7ffd551980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:21:40.016 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.016+0000 7f7ffd551980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:21:40.016 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.016+0000 7f7ffd551980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:21:40.016 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.016+0000 7f7ffd551980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:21:40.016 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.016+0000 7f7ffd551980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:21:40.016 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.016+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.016 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.016+0000 7f7ffd551980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:40.017 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.016+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.017 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.016+0000 7f7ffd551980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:21:40.017 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.016+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.018 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.020+0000 7f7ffd551980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:21:40.018 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.020+0000 7f7ffd551980 20 zone default found 2024-05-30T16:21:40.018 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.020+0000 7f7ffd551980 4 Realm: () 2024-05-30T16:21:40.018 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.020+0000 7f7ffd551980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:21:40.018 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.020+0000 7f7ffd551980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:21:40.018 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.020+0000 7f7ffd551980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:21:40.019 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.020+0000 7f7ffd551980 20 zonegroup default 2024-05-30T16:21:40.019 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.020+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.019 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.020+0000 7f7ffd551980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:40.019 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.020+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.020+0000 7f7ffd551980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:40.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.020+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.021 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.020+0000 7f7ffd551980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:40.021 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.020+0000 7f7ffd551980 20 started sync module instance, tier type = 2024-05-30T16:21:40.021 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.020+0000 7f7ffd551980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:21:40.045 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.044+0000 7f7ffd551980 20 add_watcher() i=0 2024-05-30T16:21:40.045 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.044+0000 7f7ffd551980 20 add_watcher() i=1 2024-05-30T16:21:40.045 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.044+0000 7f7ffd551980 20 add_watcher() i=2 2024-05-30T16:21:40.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.044+0000 7f7ffd551980 20 add_watcher() i=3 2024-05-30T16:21:40.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.044+0000 7f7ffd551980 20 add_watcher() i=4 2024-05-30T16:21:40.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.044+0000 7f7ffd551980 20 add_watcher() i=5 2024-05-30T16:21:40.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.048+0000 7f7ffd551980 20 add_watcher() i=6 2024-05-30T16:21:40.048 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.048+0000 7f7ffd551980 20 add_watcher() i=7 2024-05-30T16:21:40.048 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.048+0000 7f7ffd551980 2 all 8 watchers are set, enabling cache 2024-05-30T16:21:40.052 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.052+0000 7f7eb1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:21:40.053 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.052+0000 7f7ffd551980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:21:40.053 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.052+0000 7f7ffd551980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:21:40.053 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.052+0000 7f7ffd551980 5 note: GC not initialized 2024-05-30T16:21:40.054 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.056+0000 7f7eb0fe1640 20 reqs_thread_entry: start 2024-05-30T16:21:40.054 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.056+0000 7f7ffd551980 20 init_complete bucket index max shards: 11 2024-05-30T16:21:40.054 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.056+0000 7f7ffd551980 20 Filter name: none 2024-05-30T16:21:40.054 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.056+0000 7f7e2a7fc640 20 reqs_thread_entry: start 2024-05-30T16:21:40.055 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.056+0000 7f7ffd551980 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2024-05-30T16:21:40.055 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.056+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.055 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.056+0000 7f7ffd551980 20 rados_obj.operate() r=0 bl.length=221 2024-05-30T16:21:40.055 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.056+0000 7f7ffd551980 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2024-05-30T16:21:40.055 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.056+0000 7f7ffd551980 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2024-05-30T16:21:40.056 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.056+0000 7f7ffd551980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 : miss 2024-05-30T16:21:40.056 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.056+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.056 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.056+0000 7f7ffd551980 20 rados_obj.operate() r=0 bl.length=435 2024-05-30T16:21:40.056 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.056+0000 7f7ffd551980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 info.flags=0x17 2024-05-30T16:21:40.056 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.056+0000 7f7ffd551980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 to cache LRU end 2024-05-30T16:21:40.057 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.056+0000 7f7ffd551980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:21:40.057 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.056+0000 7f7ffd551980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:21:40.057 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.056+0000 7f7ffd551980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 2024-05-30T16:21:40.068 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.068+0000 7f7ffd551980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 info.flags=0x17 2024-05-30T16:21:40.068 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.068+0000 7f7ffd551980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 to cache LRU end 2024-05-30T16:21:40.068 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.068+0000 7f7ffd551980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:21:40.068 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.068+0000 7f7ffd551980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27, ofs0, ns] 2024-05-30T16:21:40.069 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.068+0000 7f7edeffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431626 cookie 94591365588048 notifier 6768 bl.length()=1007 2024-05-30T16:21:40.069 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.068+0000 7f7edeffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 info.flags=0x17 2024-05-30T16:21:40.069 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.068+0000 7f7edeffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 to cache LRU end 2024-05-30T16:21:40.069 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.068+0000 7f7edeffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:21:40.072 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.072+0000 7f7ffd551980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27]): orig_sources= new_sources= 2024-05-30T16:21:40.072 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.072+0000 7f7ffd551980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27]): potential sources added=[] removed=[] 2024-05-30T16:21:40.072 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.072+0000 7f7ffd551980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27]): orig_dests= new_dests= 2024-05-30T16:21:40.072 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.072+0000 7f7ffd551980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27]): potential dests added=[] removed=[] 2024-05-30T16:21:40.077 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.0 2024-05-30T16:21:40.077 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.0 2024-05-30T16:21:40.077 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.1 2024-05-30T16:21:40.078 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.1 2024-05-30T16:21:40.078 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.2 2024-05-30T16:21:40.078 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.2 2024-05-30T16:21:40.078 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.3 2024-05-30T16:21:40.078 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.3 2024-05-30T16:21:40.078 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.4 2024-05-30T16:21:40.078 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.4 2024-05-30T16:21:40.078 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.5 2024-05-30T16:21:40.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.5 2024-05-30T16:21:40.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.6 2024-05-30T16:21:40.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.6 2024-05-30T16:21:40.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.7 2024-05-30T16:21:40.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.7 2024-05-30T16:21:40.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.8 2024-05-30T16:21:40.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.8 2024-05-30T16:21:40.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.9 2024-05-30T16:21:40.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.9 2024-05-30T16:21:40.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.10 2024-05-30T16:21:40.080 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.10 2024-05-30T16:21:40.080 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.11 2024-05-30T16:21:40.080 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.11 2024-05-30T16:21:40.080 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.0 2024-05-30T16:21:40.081 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.080+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1 2024-05-30T16:21:40.081 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.080+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.2 2024-05-30T16:21:40.081 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.080+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.3 2024-05-30T16:21:40.081 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.080+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.4 2024-05-30T16:21:40.082 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.084+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.5 2024-05-30T16:21:40.084 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.084+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.6 2024-05-30T16:21:40.085 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.084+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.7 2024-05-30T16:21:40.086 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.088+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.8 2024-05-30T16:21:40.086 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.088+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.9 2024-05-30T16:21:40.087 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.088+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.10 2024-05-30T16:21:40.093 INFO:tasks.workunit.client.0.smithi006.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f7ffd551980 time 2024-05-30T16:21:40.096090+0000 2024-05-30T16:21:40.093 INFO:tasks.workunit.client.0.smithi006.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-30T16:21:40.094 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:21:40.094 INFO:tasks.workunit.client.0.smithi006.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f8000e28cff] 2024-05-30T16:21:40.094 INFO:tasks.workunit.client.0.smithi006.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7f8000e28eb3] 2024-05-30T16:21:40.094 INFO:tasks.workunit.client.0.smithi006.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x5607c26f5f81] 2024-05-30T16:21:40.094 INFO:tasks.workunit.client.0.smithi006.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x56f) [0x5607c270816f] 2024-05-30T16:21:40.095 INFO:tasks.workunit.client.0.smithi006.stderr: 5: main() 2024-05-30T16:21:40.095 INFO:tasks.workunit.client.0.smithi006.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f7fffc91d90] 2024-05-30T16:21:40.095 INFO:tasks.workunit.client.0.smithi006.stderr: 7: __libc_start_main() 2024-05-30T16:21:40.095 INFO:tasks.workunit.client.0.smithi006.stderr: 8: _start() 2024-05-30T16:21:40.095 INFO:tasks.workunit.client.0.smithi006.stderr:*** Caught signal (Aborted) ** 2024-05-30T16:21:40.095 INFO:tasks.workunit.client.0.smithi006.stderr: in thread 7f7ffd551980 thread_name:radosgw-admin 2024-05-30T16:21:40.095 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.096+0000 7f7ffd551980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f7ffd551980 time 2024-05-30T16:21:40.096090+0000 2024-05-30T16:21:40.095 INFO:tasks.workunit.client.0.smithi006.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-30T16:21:40.095 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.095 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:21:40.095 INFO:tasks.workunit.client.0.smithi006.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f8000e28cff] 2024-05-30T16:21:40.095 INFO:tasks.workunit.client.0.smithi006.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7f8000e28eb3] 2024-05-30T16:21:40.096 INFO:tasks.workunit.client.0.smithi006.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x5607c26f5f81] 2024-05-30T16:21:40.096 INFO:tasks.workunit.client.0.smithi006.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x56f) [0x5607c270816f] 2024-05-30T16:21:40.096 INFO:tasks.workunit.client.0.smithi006.stderr: 5: main() 2024-05-30T16:21:40.096 INFO:tasks.workunit.client.0.smithi006.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f7fffc91d90] 2024-05-30T16:21:40.096 INFO:tasks.workunit.client.0.smithi006.stderr: 7: __libc_start_main() 2024-05-30T16:21:40.096 INFO:tasks.workunit.client.0.smithi006.stderr: 8: _start() 2024-05-30T16:21:40.096 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.113 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:21:40.113 INFO:tasks.workunit.client.0.smithi006.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f7fffcaa520] 2024-05-30T16:21:40.113 INFO:tasks.workunit.client.0.smithi006.stderr: 2: pthread_kill() 2024-05-30T16:21:40.113 INFO:tasks.workunit.client.0.smithi006.stderr: 3: raise() 2024-05-30T16:21:40.113 INFO:tasks.workunit.client.0.smithi006.stderr: 4: abort() 2024-05-30T16:21:40.113 INFO:tasks.workunit.client.0.smithi006.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f8000e28d59] 2024-05-30T16:21:40.114 INFO:tasks.workunit.client.0.smithi006.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7f8000e28eb3] 2024-05-30T16:21:40.114 INFO:tasks.workunit.client.0.smithi006.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x5607c26f5f81] 2024-05-30T16:21:40.114 INFO:tasks.workunit.client.0.smithi006.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x56f) [0x5607c270816f] 2024-05-30T16:21:40.114 INFO:tasks.workunit.client.0.smithi006.stderr: 9: main() 2024-05-30T16:21:40.114 INFO:tasks.workunit.client.0.smithi006.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f7fffc91d90] 2024-05-30T16:21:40.114 INFO:tasks.workunit.client.0.smithi006.stderr: 11: __libc_start_main() 2024-05-30T16:21:40.114 INFO:tasks.workunit.client.0.smithi006.stderr: 12: _start() 2024-05-30T16:21:40.114 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.096+0000 7f7ffd551980 -1 *** Caught signal (Aborted) ** 2024-05-30T16:21:40.115 INFO:tasks.workunit.client.0.smithi006.stderr: in thread 7f7ffd551980 thread_name:radosgw-admin 2024-05-30T16:21:40.115 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.115 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:21:40.115 INFO:tasks.workunit.client.0.smithi006.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f7fffcaa520] 2024-05-30T16:21:40.115 INFO:tasks.workunit.client.0.smithi006.stderr: 2: pthread_kill() 2024-05-30T16:21:40.115 INFO:tasks.workunit.client.0.smithi006.stderr: 3: raise() 2024-05-30T16:21:40.115 INFO:tasks.workunit.client.0.smithi006.stderr: 4: abort() 2024-05-30T16:21:40.115 INFO:tasks.workunit.client.0.smithi006.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f8000e28d59] 2024-05-30T16:21:40.116 INFO:tasks.workunit.client.0.smithi006.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7f8000e28eb3] 2024-05-30T16:21:40.116 INFO:tasks.workunit.client.0.smithi006.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x5607c26f5f81] 2024-05-30T16:21:40.116 INFO:tasks.workunit.client.0.smithi006.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x56f) [0x5607c270816f] 2024-05-30T16:21:40.116 INFO:tasks.workunit.client.0.smithi006.stderr: 9: main() 2024-05-30T16:21:40.116 INFO:tasks.workunit.client.0.smithi006.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f7fffc91d90] 2024-05-30T16:21:40.116 INFO:tasks.workunit.client.0.smithi006.stderr: 11: __libc_start_main() 2024-05-30T16:21:40.116 INFO:tasks.workunit.client.0.smithi006.stderr: 12: _start() 2024-05-30T16:21:40.116 INFO:tasks.workunit.client.0.smithi006.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-30T16:21:40.116 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.116 INFO:tasks.workunit.client.0.smithi006.stderr:--- begin dump of recent events --- 2024-05-30T16:21:40.117 INFO:tasks.workunit.client.0.smithi006.stderr: -499> 2024-05-30T16:21:39.756+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af92e0) adding con mode: crc 2024-05-30T16:21:40.117 INFO:tasks.workunit.client.0.smithi006.stderr: -498> 2024-05-30T16:21:39.756+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af92e0) adding con mode: secure 2024-05-30T16:21:40.117 INFO:tasks.workunit.client.0.smithi006.stderr: -497> 2024-05-30T16:21:39.756+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af92e0) adding con mode: crc 2024-05-30T16:21:40.117 INFO:tasks.workunit.client.0.smithi006.stderr: -496> 2024-05-30T16:21:39.756+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af92e0) adding con mode: secure 2024-05-30T16:21:40.117 INFO:tasks.workunit.client.0.smithi006.stderr: -495> 2024-05-30T16:21:39.760+0000 7f7ffd551980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:21:40.117 INFO:tasks.workunit.client.0.smithi006.stderr: -494> 2024-05-30T16:21:39.760+0000 7f7ffd551980 1 librados: starting msgr at 2024-05-30T16:21:40.117 INFO:tasks.workunit.client.0.smithi006.stderr: -493> 2024-05-30T16:21:39.760+0000 7f7ffd551980 1 librados: starting objecter 2024-05-30T16:21:40.117 INFO:tasks.workunit.client.0.smithi006.stderr: -492> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command objecter_requests hook 0x5607c48d9a50 2024-05-30T16:21:40.117 INFO:tasks.workunit.client.0.smithi006.stderr: -491> 2024-05-30T16:21:39.760+0000 7f7ffd551980 1 librados: setting wanted keys 2024-05-30T16:21:40.117 INFO:tasks.workunit.client.0.smithi006.stderr: -490> 2024-05-30T16:21:39.760+0000 7f7ffd551980 1 librados: calling monclient init 2024-05-30T16:21:40.117 INFO:tasks.workunit.client.0.smithi006.stderr: -489> 2024-05-30T16:21:39.760+0000 7f7ffd551980 10 monclient: init 2024-05-30T16:21:40.117 INFO:tasks.workunit.client.0.smithi006.stderr: -488> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af3e70) adding auth protocol: cephx 2024-05-30T16:21:40.118 INFO:tasks.workunit.client.0.smithi006.stderr: -487> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af3e70) adding auth protocol: cephx 2024-05-30T16:21:40.118 INFO:tasks.workunit.client.0.smithi006.stderr: -486> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af3e70) adding auth protocol: cephx 2024-05-30T16:21:40.118 INFO:tasks.workunit.client.0.smithi006.stderr: -485> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af3e70) adding con mode: secure 2024-05-30T16:21:40.118 INFO:tasks.workunit.client.0.smithi006.stderr: -484> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af3e70) adding con mode: crc 2024-05-30T16:21:40.118 INFO:tasks.workunit.client.0.smithi006.stderr: -483> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af3e70) adding con mode: secure 2024-05-30T16:21:40.118 INFO:tasks.workunit.client.0.smithi006.stderr: -482> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af3e70) adding con mode: crc 2024-05-30T16:21:40.118 INFO:tasks.workunit.client.0.smithi006.stderr: -481> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af3e70) adding con mode: secure 2024-05-30T16:21:40.118 INFO:tasks.workunit.client.0.smithi006.stderr: -480> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af3e70) adding con mode: crc 2024-05-30T16:21:40.118 INFO:tasks.workunit.client.0.smithi006.stderr: -479> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af3e70) adding con mode: crc 2024-05-30T16:21:40.118 INFO:tasks.workunit.client.0.smithi006.stderr: -478> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af3e70) adding con mode: secure 2024-05-30T16:21:40.118 INFO:tasks.workunit.client.0.smithi006.stderr: -477> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af3e70) adding con mode: crc 2024-05-30T16:21:40.118 INFO:tasks.workunit.client.0.smithi006.stderr: -476> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af3e70) adding con mode: secure 2024-05-30T16:21:40.119 INFO:tasks.workunit.client.0.smithi006.stderr: -475> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af3e70) adding con mode: crc 2024-05-30T16:21:40.119 INFO:tasks.workunit.client.0.smithi006.stderr: -474> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af3e70) adding con mode: secure 2024-05-30T16:21:40.119 INFO:tasks.workunit.client.0.smithi006.stderr: -473> 2024-05-30T16:21:39.760+0000 7f7ffd551980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:21:40.119 INFO:tasks.workunit.client.0.smithi006.stderr: -472> 2024-05-30T16:21:39.760+0000 7f7ffd551980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:21:40.119 INFO:tasks.workunit.client.0.smithi006.stderr: -471> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command rotate-key hook 0x5607c4af3fb8 2024-05-30T16:21:40.119 INFO:tasks.workunit.client.0.smithi006.stderr: -470> 2024-05-30T16:21:39.760+0000 7f7ffd551980 10 monclient: _reopen_session rank -1 2024-05-30T16:21:40.119 INFO:tasks.workunit.client.0.smithi006.stderr: -469> 2024-05-30T16:21:39.760+0000 7f7ffd551980 10 monclient: _add_conns ranks=[1,0,2] 2024-05-30T16:21:40.119 INFO:tasks.workunit.client.0.smithi006.stderr: -468> 2024-05-30T16:21:39.760+0000 7f7ffd551980 10 monclient(hunting): picked mon.noname-b con 0x5607c4985220 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:21:40.119 INFO:tasks.workunit.client.0.smithi006.stderr: -467> 2024-05-30T16:21:39.760+0000 7f7ffd551980 10 monclient(hunting): picked mon.noname-a con 0x5607c49a4070 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:21:40.119 INFO:tasks.workunit.client.0.smithi006.stderr: -466> 2024-05-30T16:21:39.760+0000 7f7ffd551980 10 monclient(hunting): picked mon.noname-c con 0x5607c4b21430 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:21:40.119 INFO:tasks.workunit.client.0.smithi006.stderr: -465> 2024-05-30T16:21:39.760+0000 7f7ffd551980 10 monclient(hunting): start opening mon connection 2024-05-30T16:21:40.119 INFO:tasks.workunit.client.0.smithi006.stderr: -464> 2024-05-30T16:21:39.760+0000 7f7ffd551980 10 monclient(hunting): start opening mon connection 2024-05-30T16:21:40.119 INFO:tasks.workunit.client.0.smithi006.stderr: -463> 2024-05-30T16:21:39.760+0000 7f7ffd551980 10 monclient(hunting): start opening mon connection 2024-05-30T16:21:40.120 INFO:tasks.workunit.client.0.smithi006.stderr: -462> 2024-05-30T16:21:39.760+0000 7f7ffd551980 10 monclient(hunting): _renew_subs 2024-05-30T16:21:40.120 INFO:tasks.workunit.client.0.smithi006.stderr: -461> 2024-05-30T16:21:39.760+0000 7f7ffd551980 10 monclient(hunting): authenticate will time out at 2639.692976s 2024-05-30T16:21:40.120 INFO:tasks.workunit.client.0.smithi006.stderr: -460> 2024-05-30T16:21:39.760+0000 7f7ffb863640 10 monclient(hunting): get_auth_request con 0x5607c4b21430 auth_method 0 2024-05-30T16:21:40.120 INFO:tasks.workunit.client.0.smithi006.stderr: -459> 2024-05-30T16:21:39.760+0000 7f7ffb863640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:21:40.120 INFO:tasks.workunit.client.0.smithi006.stderr: -458> 2024-05-30T16:21:39.760+0000 7f7ffb863640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:21:40.120 INFO:tasks.workunit.client.0.smithi006.stderr: -457> 2024-05-30T16:21:39.760+0000 7f7ffb863640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:21:40.120 INFO:tasks.workunit.client.0.smithi006.stderr: -456> 2024-05-30T16:21:39.760+0000 7f7ffa861640 10 monclient(hunting): get_auth_request con 0x5607c49a4070 auth_method 0 2024-05-30T16:21:40.120 INFO:tasks.workunit.client.0.smithi006.stderr: -455> 2024-05-30T16:21:39.760+0000 7f7ffa861640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:21:40.120 INFO:tasks.workunit.client.0.smithi006.stderr: -454> 2024-05-30T16:21:39.760+0000 7f7ffa861640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:21:40.120 INFO:tasks.workunit.client.0.smithi006.stderr: -453> 2024-05-30T16:21:39.760+0000 7f7ffa861640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:21:40.120 INFO:tasks.workunit.client.0.smithi006.stderr: -452> 2024-05-30T16:21:39.760+0000 7f7ffb863640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:21:40.120 INFO:tasks.workunit.client.0.smithi006.stderr: -451> 2024-05-30T16:21:39.760+0000 7f7ffb863640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:21:40.120 INFO:tasks.workunit.client.0.smithi006.stderr: -450> 2024-05-30T16:21:39.760+0000 7f7ffb863640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:21:40.121 INFO:tasks.workunit.client.0.smithi006.stderr: -449> 2024-05-30T16:21:39.760+0000 7f7ffa861640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:21:40.121 INFO:tasks.workunit.client.0.smithi006.stderr: -448> 2024-05-30T16:21:39.760+0000 7f7ffa861640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:21:40.121 INFO:tasks.workunit.client.0.smithi006.stderr: -447> 2024-05-30T16:21:39.760+0000 7f7ffa861640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:21:40.121 INFO:tasks.workunit.client.0.smithi006.stderr: -446> 2024-05-30T16:21:39.760+0000 7f7ffb863640 10 monclient(hunting): handle_auth_done global_id 6440 payload 824 2024-05-30T16:21:40.121 INFO:tasks.workunit.client.0.smithi006.stderr: -445> 2024-05-30T16:21:39.760+0000 7f7ffb863640 10 monclient: _finish_hunting 0 2024-05-30T16:21:40.121 INFO:tasks.workunit.client.0.smithi006.stderr: -444> 2024-05-30T16:21:39.760+0000 7f7ffb863640 1 monclient: found mon.noname-c 2024-05-30T16:21:40.121 INFO:tasks.workunit.client.0.smithi006.stderr: -443> 2024-05-30T16:21:39.760+0000 7f7ffb863640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.6:3301/0 2024-05-30T16:21:40.121 INFO:tasks.workunit.client.0.smithi006.stderr: -442> 2024-05-30T16:21:39.760+0000 7f7ee7e07640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:21:40.121 INFO:tasks.workunit.client.0.smithi006.stderr: -441> 2024-05-30T16:21:39.760+0000 7f7ee7e07640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-30T16:21:40.121 INFO:tasks.workunit.client.0.smithi006.stderr: -440> 2024-05-30T16:21:39.760+0000 7f7ee7e07640 10 monclient: dump: 2024-05-30T16:21:40.121 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:21:40.121 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:21:40.121 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.122 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.122 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:21:40.122 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:21:40.122 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:21:40.122 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:21:40.122 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:21:40.122 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.122 INFO:tasks.workunit.client.0.smithi006.stderr: -439> 2024-05-30T16:21:39.760+0000 7f7ee7e07640 10 monclient: _finish_auth 0 2024-05-30T16:21:40.122 INFO:tasks.workunit.client.0.smithi006.stderr: -438> 2024-05-30T16:21:39.760+0000 7f7ee7e07640 10 monclient: _check_auth_tickets 2024-05-30T16:21:40.122 INFO:tasks.workunit.client.0.smithi006.stderr: -437> 2024-05-30T16:21:39.760+0000 7f7ee7e07640 10 monclient: handle_config config(0 keys) 2024-05-30T16:21:40.122 INFO:tasks.workunit.client.0.smithi006.stderr: -436> 2024-05-30T16:21:39.760+0000 7f7ee7e07640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:21:40.122 INFO:tasks.workunit.client.0.smithi006.stderr: -435> 2024-05-30T16:21:39.760+0000 7f7ee7e07640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-30T16:21:40.123 INFO:tasks.workunit.client.0.smithi006.stderr: -434> 2024-05-30T16:21:39.760+0000 7f7ee7e07640 10 monclient: dump: 2024-05-30T16:21:40.123 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:21:40.123 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:21:40.123 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.123 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.123 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:21:40.123 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:21:40.123 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:21:40.123 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:21:40.123 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:21:40.123 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.123 INFO:tasks.workunit.client.0.smithi006.stderr: -433> 2024-05-30T16:21:39.760+0000 7f7ee960a640 4 set_mon_vals no callback set 2024-05-30T16:21:40.123 INFO:tasks.workunit.client.0.smithi006.stderr: -432> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 monclient: authenticate success, global_id 6440 2024-05-30T16:21:40.124 INFO:tasks.workunit.client.0.smithi006.stderr: -431> 2024-05-30T16:21:39.760+0000 7f7ffd551980 10 monclient: _renew_subs 2024-05-30T16:21:40.124 INFO:tasks.workunit.client.0.smithi006.stderr: -430> 2024-05-30T16:21:39.760+0000 7f7ffd551980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.6:3301/0 2024-05-30T16:21:40.124 INFO:tasks.workunit.client.0.smithi006.stderr: -429> 2024-05-30T16:21:39.760+0000 7f7ffd551980 10 monclient: _renew_subs 2024-05-30T16:21:40.124 INFO:tasks.workunit.client.0.smithi006.stderr: -428> 2024-05-30T16:21:39.760+0000 7f7ffd551980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.6:3301/0 2024-05-30T16:21:40.124 INFO:tasks.workunit.client.0.smithi006.stderr: -427> 2024-05-30T16:21:39.760+0000 7f7ffd551980 1 librados: init done 2024-05-30T16:21:40.124 INFO:tasks.workunit.client.0.smithi006.stderr: -426> 2024-05-30T16:21:39.764+0000 7f7ee7e07640 4 mgrc handle_mgr_map Got map version 5 2024-05-30T16:21:40.124 INFO:tasks.workunit.client.0.smithi006.stderr: -425> 2024-05-30T16:21:39.764+0000 7f7ee7e07640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:21:40.124 INFO:tasks.workunit.client.0.smithi006.stderr: -424> 2024-05-30T16:21:39.764+0000 7f7ee7e07640 4 mgrc reconnect Starting new session with [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:21:40.124 INFO:tasks.workunit.client.0.smithi006.stderr: -423> 2024-05-30T16:21:39.764+0000 7f7ffb062640 10 monclient: get_auth_request con 0x7f7ee006dde0 auth_method 0 2024-05-30T16:21:40.124 INFO:tasks.workunit.client.0.smithi006.stderr: -422> 2024-05-30T16:21:39.764+0000 7f7ffa861640 10 monclient: get_auth_request con 0x5607c4982b40 auth_method 0 2024-05-30T16:21:40.124 INFO:tasks.workunit.client.0.smithi006.stderr: -421> 2024-05-30T16:21:39.764+0000 7f7ffb062640 10 monclient: get_auth_request con 0x5607c4b3fca0 auth_method 0 2024-05-30T16:21:40.124 INFO:tasks.workunit.client.0.smithi006.stderr: -420> 2024-05-30T16:21:39.768+0000 7f7ffa861640 10 monclient: get_auth_request con 0x5607c4b43410 auth_method 0 2024-05-30T16:21:40.124 INFO:tasks.workunit.client.0.smithi006.stderr: -419> 2024-05-30T16:21:39.768+0000 7f7ffd551980 10 monclient: build_initial_monmap 2024-05-30T16:21:40.125 INFO:tasks.workunit.client.0.smithi006.stderr: -418> 2024-05-30T16:21:39.768+0000 7f7ffd551980 1 build_initial for_mkfs: 0 2024-05-30T16:21:40.125 INFO:tasks.workunit.client.0.smithi006.stderr: -417> 2024-05-30T16:21:39.768+0000 7f7ffd551980 10 monclient: monmap: 2024-05-30T16:21:40.125 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 0 2024-05-30T16:21:40.125 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-30T16:21:40.125 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 0.000000 2024-05-30T16:21:40.125 INFO:tasks.workunit.client.0.smithi006.stderr:created 0.000000 2024-05-30T16:21:40.125 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 0 (unknown) 2024-05-30T16:21:40.125 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:21:40.125 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.noname-a 2024-05-30T16:21:40.125 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-b 2024-05-30T16:21:40.125 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.noname-c 2024-05-30T16:21:40.125 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.125 INFO:tasks.workunit.client.0.smithi006.stderr: -416> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b488b0) adding auth protocol: cephx 2024-05-30T16:21:40.126 INFO:tasks.workunit.client.0.smithi006.stderr: -415> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b488b0) adding auth protocol: cephx 2024-05-30T16:21:40.126 INFO:tasks.workunit.client.0.smithi006.stderr: -414> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b488b0) adding auth protocol: cephx 2024-05-30T16:21:40.126 INFO:tasks.workunit.client.0.smithi006.stderr: -413> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b488b0) adding con mode: secure 2024-05-30T16:21:40.126 INFO:tasks.workunit.client.0.smithi006.stderr: -412> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b488b0) adding con mode: crc 2024-05-30T16:21:40.126 INFO:tasks.workunit.client.0.smithi006.stderr: -411> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b488b0) adding con mode: secure 2024-05-30T16:21:40.126 INFO:tasks.workunit.client.0.smithi006.stderr: -410> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b488b0) adding con mode: crc 2024-05-30T16:21:40.126 INFO:tasks.workunit.client.0.smithi006.stderr: -409> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b488b0) adding con mode: secure 2024-05-30T16:21:40.126 INFO:tasks.workunit.client.0.smithi006.stderr: -408> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b488b0) adding con mode: crc 2024-05-30T16:21:40.126 INFO:tasks.workunit.client.0.smithi006.stderr: -407> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b488b0) adding con mode: crc 2024-05-30T16:21:40.126 INFO:tasks.workunit.client.0.smithi006.stderr: -406> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b488b0) adding con mode: secure 2024-05-30T16:21:40.126 INFO:tasks.workunit.client.0.smithi006.stderr: -405> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b488b0) adding con mode: crc 2024-05-30T16:21:40.126 INFO:tasks.workunit.client.0.smithi006.stderr: -404> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b488b0) adding con mode: secure 2024-05-30T16:21:40.127 INFO:tasks.workunit.client.0.smithi006.stderr: -403> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b488b0) adding con mode: crc 2024-05-30T16:21:40.127 INFO:tasks.workunit.client.0.smithi006.stderr: -402> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b488b0) adding con mode: secure 2024-05-30T16:21:40.127 INFO:tasks.workunit.client.0.smithi006.stderr: -401> 2024-05-30T16:21:39.768+0000 7f7ffd551980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:21:40.127 INFO:tasks.workunit.client.0.smithi006.stderr: -400> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command objecter_requests cmddesc objecter_requests hook 0x5607c4b4bda0 EEXIST 2024-05-30T16:21:40.127 INFO:tasks.workunit.client.0.smithi006.stderr: -399> 2024-05-30T16:21:39.768+0000 7f7ffd551980 10 monclient: init 2024-05-30T16:21:40.127 INFO:tasks.workunit.client.0.smithi006.stderr: -398> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b477a8) adding auth protocol: cephx 2024-05-30T16:21:40.127 INFO:tasks.workunit.client.0.smithi006.stderr: -397> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b477a8) adding auth protocol: cephx 2024-05-30T16:21:40.127 INFO:tasks.workunit.client.0.smithi006.stderr: -396> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b477a8) adding auth protocol: cephx 2024-05-30T16:21:40.127 INFO:tasks.workunit.client.0.smithi006.stderr: -395> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b477a8) adding con mode: secure 2024-05-30T16:21:40.127 INFO:tasks.workunit.client.0.smithi006.stderr: -394> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b477a8) adding con mode: crc 2024-05-30T16:21:40.127 INFO:tasks.workunit.client.0.smithi006.stderr: -393> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b477a8) adding con mode: secure 2024-05-30T16:21:40.127 INFO:tasks.workunit.client.0.smithi006.stderr: -392> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b477a8) adding con mode: crc 2024-05-30T16:21:40.127 INFO:tasks.workunit.client.0.smithi006.stderr: -391> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b477a8) adding con mode: secure 2024-05-30T16:21:40.128 INFO:tasks.workunit.client.0.smithi006.stderr: -390> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b477a8) adding con mode: crc 2024-05-30T16:21:40.128 INFO:tasks.workunit.client.0.smithi006.stderr: -389> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b477a8) adding con mode: crc 2024-05-30T16:21:40.128 INFO:tasks.workunit.client.0.smithi006.stderr: -388> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b477a8) adding con mode: secure 2024-05-30T16:21:40.128 INFO:tasks.workunit.client.0.smithi006.stderr: -387> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b477a8) adding con mode: crc 2024-05-30T16:21:40.128 INFO:tasks.workunit.client.0.smithi006.stderr: -386> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b477a8) adding con mode: secure 2024-05-30T16:21:40.128 INFO:tasks.workunit.client.0.smithi006.stderr: -385> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b477a8) adding con mode: crc 2024-05-30T16:21:40.128 INFO:tasks.workunit.client.0.smithi006.stderr: -384> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b477a8) adding con mode: secure 2024-05-30T16:21:40.128 INFO:tasks.workunit.client.0.smithi006.stderr: -383> 2024-05-30T16:21:39.768+0000 7f7ffd551980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:21:40.128 INFO:tasks.workunit.client.0.smithi006.stderr: -382> 2024-05-30T16:21:39.772+0000 7f7ffd551980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:21:40.128 INFO:tasks.workunit.client.0.smithi006.stderr: -381> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command rotate-key cmddesc rotate-key hook 0x5607c4b478f0 EEXIST 2024-05-30T16:21:40.128 INFO:tasks.workunit.client.0.smithi006.stderr: -380> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient: _reopen_session rank -1 2024-05-30T16:21:40.128 INFO:tasks.workunit.client.0.smithi006.stderr: -379> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient: _add_conns ranks=[0,1,2] 2024-05-30T16:21:40.128 INFO:tasks.workunit.client.0.smithi006.stderr: -378> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient(hunting): picked mon.noname-a con 0x5607c4b52330 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:21:40.129 INFO:tasks.workunit.client.0.smithi006.stderr: -377> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient(hunting): picked mon.noname-b con 0x5607c4b52c60 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:21:40.129 INFO:tasks.workunit.client.0.smithi006.stderr: -376> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient(hunting): picked mon.noname-c con 0x5607c4b52710 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:21:40.129 INFO:tasks.workunit.client.0.smithi006.stderr: -375> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient(hunting): start opening mon connection 2024-05-30T16:21:40.129 INFO:tasks.workunit.client.0.smithi006.stderr: -374> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient(hunting): start opening mon connection 2024-05-30T16:21:40.129 INFO:tasks.workunit.client.0.smithi006.stderr: -373> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient(hunting): start opening mon connection 2024-05-30T16:21:40.129 INFO:tasks.workunit.client.0.smithi006.stderr: -372> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient(hunting): _renew_subs 2024-05-30T16:21:40.129 INFO:tasks.workunit.client.0.smithi006.stderr: -371> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient(hunting): authenticate will time out at 2639.704304s 2024-05-30T16:21:40.129 INFO:tasks.workunit.client.0.smithi006.stderr: -370> 2024-05-30T16:21:39.772+0000 7f7ffb062640 10 monclient(hunting): get_auth_request con 0x5607c4b52330 auth_method 0 2024-05-30T16:21:40.129 INFO:tasks.workunit.client.0.smithi006.stderr: -369> 2024-05-30T16:21:39.772+0000 7f7ffb062640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:21:40.129 INFO:tasks.workunit.client.0.smithi006.stderr: -368> 2024-05-30T16:21:39.772+0000 7f7ffb062640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:21:40.129 INFO:tasks.workunit.client.0.smithi006.stderr: -367> 2024-05-30T16:21:39.772+0000 7f7ffb062640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:21:40.129 INFO:tasks.workunit.client.0.smithi006.stderr: -366> 2024-05-30T16:21:39.772+0000 7f7ffb863640 10 monclient(hunting): get_auth_request con 0x5607c4b52710 auth_method 0 2024-05-30T16:21:40.130 INFO:tasks.workunit.client.0.smithi006.stderr: -365> 2024-05-30T16:21:39.772+0000 7f7ffb863640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:21:40.130 INFO:tasks.workunit.client.0.smithi006.stderr: -364> 2024-05-30T16:21:39.772+0000 7f7ffb863640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:21:40.130 INFO:tasks.workunit.client.0.smithi006.stderr: -363> 2024-05-30T16:21:39.772+0000 7f7ffb863640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:21:40.130 INFO:tasks.workunit.client.0.smithi006.stderr: -362> 2024-05-30T16:21:39.772+0000 7f7ffb062640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:21:40.130 INFO:tasks.workunit.client.0.smithi006.stderr: -361> 2024-05-30T16:21:39.772+0000 7f7ffb062640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:21:40.130 INFO:tasks.workunit.client.0.smithi006.stderr: -360> 2024-05-30T16:21:39.772+0000 7f7ffb062640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:21:40.130 INFO:tasks.workunit.client.0.smithi006.stderr: -359> 2024-05-30T16:21:39.772+0000 7f7ffb863640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:21:40.130 INFO:tasks.workunit.client.0.smithi006.stderr: -358> 2024-05-30T16:21:39.772+0000 7f7ffb863640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:21:40.130 INFO:tasks.workunit.client.0.smithi006.stderr: -357> 2024-05-30T16:21:39.772+0000 7f7ffb863640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:21:40.130 INFO:tasks.workunit.client.0.smithi006.stderr: -356> 2024-05-30T16:21:39.772+0000 7f7ffa861640 10 monclient(hunting): get_auth_request con 0x5607c4b52c60 auth_method 0 2024-05-30T16:21:40.130 INFO:tasks.workunit.client.0.smithi006.stderr: -355> 2024-05-30T16:21:39.772+0000 7f7ffa861640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:21:40.130 INFO:tasks.workunit.client.0.smithi006.stderr: -354> 2024-05-30T16:21:39.772+0000 7f7ffa861640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:21:40.130 INFO:tasks.workunit.client.0.smithi006.stderr: -353> 2024-05-30T16:21:39.772+0000 7f7ffa861640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:21:40.131 INFO:tasks.workunit.client.0.smithi006.stderr: -352> 2024-05-30T16:21:39.772+0000 7f7ffb062640 10 monclient(hunting): handle_auth_done global_id 6762 payload 824 2024-05-30T16:21:40.131 INFO:tasks.workunit.client.0.smithi006.stderr: -351> 2024-05-30T16:21:39.772+0000 7f7ffb062640 10 monclient: _finish_hunting 0 2024-05-30T16:21:40.131 INFO:tasks.workunit.client.0.smithi006.stderr: -350> 2024-05-30T16:21:39.772+0000 7f7ffb062640 1 monclient: found mon.noname-a 2024-05-30T16:21:40.131 INFO:tasks.workunit.client.0.smithi006.stderr: -349> 2024-05-30T16:21:39.772+0000 7f7ffb062640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.6:3300/0 2024-05-30T16:21:40.131 INFO:tasks.workunit.client.0.smithi006.stderr: -348> 2024-05-30T16:21:39.772+0000 7f7ee5602640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:21:40.131 INFO:tasks.workunit.client.0.smithi006.stderr: -347> 2024-05-30T16:21:39.772+0000 7f7ee5602640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-30T16:21:40.131 INFO:tasks.workunit.client.0.smithi006.stderr: -346> 2024-05-30T16:21:39.772+0000 7f7ee5602640 10 monclient: dump: 2024-05-30T16:21:40.131 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:21:40.131 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:21:40.131 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.131 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.131 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:21:40.131 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:21:40.131 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:21:40.132 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:21:40.132 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:21:40.132 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.132 INFO:tasks.workunit.client.0.smithi006.stderr: -345> 2024-05-30T16:21:39.772+0000 7f7ee5602640 10 monclient: _finish_auth 0 2024-05-30T16:21:40.132 INFO:tasks.workunit.client.0.smithi006.stderr: -344> 2024-05-30T16:21:39.772+0000 7f7ee5602640 10 monclient: _check_auth_tickets 2024-05-30T16:21:40.132 INFO:tasks.workunit.client.0.smithi006.stderr: -343> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 monclient: authenticate success, global_id 6762 2024-05-30T16:21:40.132 INFO:tasks.workunit.client.0.smithi006.stderr: -342> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient: _renew_subs 2024-05-30T16:21:40.132 INFO:tasks.workunit.client.0.smithi006.stderr: -341> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:21:40.132 INFO:tasks.workunit.client.0.smithi006.stderr: -340> 2024-05-30T16:21:39.772+0000 7f7ee5602640 10 monclient: handle_config config(0 keys) 2024-05-30T16:21:40.132 INFO:tasks.workunit.client.0.smithi006.stderr: -339> 2024-05-30T16:21:39.772+0000 7f7ee5602640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:21:40.132 INFO:tasks.workunit.client.0.smithi006.stderr: -338> 2024-05-30T16:21:39.772+0000 7f7ee5602640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-30T16:21:40.132 INFO:tasks.workunit.client.0.smithi006.stderr: -337> 2024-05-30T16:21:39.772+0000 7f7ee5602640 10 monclient: dump: 2024-05-30T16:21:40.132 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:21:40.132 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:21:40.133 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.133 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.133 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:21:40.133 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:21:40.133 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:21:40.133 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:21:40.133 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:21:40.133 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.133 INFO:tasks.workunit.client.0.smithi006.stderr: -336> 2024-05-30T16:21:39.772+0000 7f7ffc064640 4 set_mon_vals no callback set 2024-05-30T16:21:40.133 INFO:tasks.workunit.client.0.smithi006.stderr: -335> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient: _renew_subs 2024-05-30T16:21:40.133 INFO:tasks.workunit.client.0.smithi006.stderr: -334> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:21:40.133 INFO:tasks.workunit.client.0.smithi006.stderr: -333> 2024-05-30T16:21:39.772+0000 7f7ee5602640 4 mgrc handle_mgr_map Got map version 5 2024-05-30T16:21:40.133 INFO:tasks.workunit.client.0.smithi006.stderr: -332> 2024-05-30T16:21:39.772+0000 7f7ee5602640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:21:40.133 INFO:tasks.workunit.client.0.smithi006.stderr: -331> 2024-05-30T16:21:39.772+0000 7f7ee5602640 4 mgrc reconnect Starting new session with [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:21:40.134 INFO:tasks.workunit.client.0.smithi006.stderr: -330> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient: get_monmap_and_config 2024-05-30T16:21:40.134 INFO:tasks.workunit.client.0.smithi006.stderr: -329> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient: build_initial_monmap 2024-05-30T16:21:40.134 INFO:tasks.workunit.client.0.smithi006.stderr: -328> 2024-05-30T16:21:39.772+0000 7f7ffd551980 1 build_initial for_mkfs: 0 2024-05-30T16:21:40.134 INFO:tasks.workunit.client.0.smithi006.stderr: -327> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient: monmap: 2024-05-30T16:21:40.134 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 0 2024-05-30T16:21:40.134 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-30T16:21:40.134 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 0.000000 2024-05-30T16:21:40.134 INFO:tasks.workunit.client.0.smithi006.stderr:created 0.000000 2024-05-30T16:21:40.134 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 0 (unknown) 2024-05-30T16:21:40.134 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:21:40.134 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.noname-a 2024-05-30T16:21:40.134 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-b 2024-05-30T16:21:40.134 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.noname-c 2024-05-30T16:21:40.135 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.135 INFO:tasks.workunit.client.0.smithi006.stderr: -326> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b8fe10) adding auth protocol: cephx 2024-05-30T16:21:40.135 INFO:tasks.workunit.client.0.smithi006.stderr: -325> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b8fe10) adding auth protocol: cephx 2024-05-30T16:21:40.135 INFO:tasks.workunit.client.0.smithi006.stderr: -324> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b8fe10) adding auth protocol: cephx 2024-05-30T16:21:40.135 INFO:tasks.workunit.client.0.smithi006.stderr: -323> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b8fe10) adding con mode: secure 2024-05-30T16:21:40.135 INFO:tasks.workunit.client.0.smithi006.stderr: -322> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b8fe10) adding con mode: crc 2024-05-30T16:21:40.135 INFO:tasks.workunit.client.0.smithi006.stderr: -321> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b8fe10) adding con mode: secure 2024-05-30T16:21:40.135 INFO:tasks.workunit.client.0.smithi006.stderr: -320> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b8fe10) adding con mode: crc 2024-05-30T16:21:40.135 INFO:tasks.workunit.client.0.smithi006.stderr: -319> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b8fe10) adding con mode: secure 2024-05-30T16:21:40.135 INFO:tasks.workunit.client.0.smithi006.stderr: -318> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b8fe10) adding con mode: crc 2024-05-30T16:21:40.135 INFO:tasks.workunit.client.0.smithi006.stderr: -317> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b8fe10) adding con mode: crc 2024-05-30T16:21:40.135 INFO:tasks.workunit.client.0.smithi006.stderr: -316> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b8fe10) adding con mode: secure 2024-05-30T16:21:40.135 INFO:tasks.workunit.client.0.smithi006.stderr: -315> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b8fe10) adding con mode: crc 2024-05-30T16:21:40.135 INFO:tasks.workunit.client.0.smithi006.stderr: -314> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b8fe10) adding con mode: secure 2024-05-30T16:21:40.136 INFO:tasks.workunit.client.0.smithi006.stderr: -313> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b8fe10) adding con mode: crc 2024-05-30T16:21:40.136 INFO:tasks.workunit.client.0.smithi006.stderr: -312> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b8fe10) adding con mode: secure 2024-05-30T16:21:40.136 INFO:tasks.workunit.client.0.smithi006.stderr: -311> 2024-05-30T16:21:39.772+0000 7f7ffa861640 10 monclient: get_auth_request con 0x7f7ed406dcf0 auth_method 0 2024-05-30T16:21:40.136 INFO:tasks.workunit.client.0.smithi006.stderr: -310> 2024-05-30T16:21:39.772+0000 7f7ffd551980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:21:40.136 INFO:tasks.workunit.client.0.smithi006.stderr: -309> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient: init 2024-05-30T16:21:40.136 INFO:tasks.workunit.client.0.smithi006.stderr: -308> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc37e0) adding auth protocol: cephx 2024-05-30T16:21:40.136 INFO:tasks.workunit.client.0.smithi006.stderr: -307> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc37e0) adding auth protocol: cephx 2024-05-30T16:21:40.137 INFO:tasks.workunit.client.0.smithi006.stderr: -306> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc37e0) adding auth protocol: cephx 2024-05-30T16:21:40.137 INFO:tasks.workunit.client.0.smithi006.stderr: -305> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc37e0) adding con mode: secure 2024-05-30T16:21:40.137 INFO:tasks.workunit.client.0.smithi006.stderr: -304> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc37e0) adding con mode: crc 2024-05-30T16:21:40.137 INFO:tasks.workunit.client.0.smithi006.stderr: -303> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc37e0) adding con mode: secure 2024-05-30T16:21:40.137 INFO:tasks.workunit.client.0.smithi006.stderr: -302> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc37e0) adding con mode: crc 2024-05-30T16:21:40.137 INFO:tasks.workunit.client.0.smithi006.stderr: -301> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc37e0) adding con mode: secure 2024-05-30T16:21:40.137 INFO:tasks.workunit.client.0.smithi006.stderr: -300> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc37e0) adding con mode: crc 2024-05-30T16:21:40.137 INFO:tasks.workunit.client.0.smithi006.stderr: -299> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc37e0) adding con mode: crc 2024-05-30T16:21:40.137 INFO:tasks.workunit.client.0.smithi006.stderr: -298> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc37e0) adding con mode: secure 2024-05-30T16:21:40.137 INFO:tasks.workunit.client.0.smithi006.stderr: -297> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc37e0) adding con mode: crc 2024-05-30T16:21:40.137 INFO:tasks.workunit.client.0.smithi006.stderr: -296> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc37e0) adding con mode: secure 2024-05-30T16:21:40.137 INFO:tasks.workunit.client.0.smithi006.stderr: -295> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc37e0) adding con mode: crc 2024-05-30T16:21:40.137 INFO:tasks.workunit.client.0.smithi006.stderr: -294> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc37e0) adding con mode: secure 2024-05-30T16:21:40.137 INFO:tasks.workunit.client.0.smithi006.stderr: -293> 2024-05-30T16:21:39.776+0000 7f7ffd551980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:21:40.138 INFO:tasks.workunit.client.0.smithi006.stderr: -292> 2024-05-30T16:21:39.776+0000 7f7ffd551980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:21:40.138 INFO:tasks.workunit.client.0.smithi006.stderr: -291> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command rotate-key cmddesc rotate-key hook 0x7ffe4bdc3928 EEXIST 2024-05-30T16:21:40.138 INFO:tasks.workunit.client.0.smithi006.stderr: -290> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient: _reopen_session rank -1 2024-05-30T16:21:40.138 INFO:tasks.workunit.client.0.smithi006.stderr: -289> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient: _add_conns ranks=[0,2,1] 2024-05-30T16:21:40.138 INFO:tasks.workunit.client.0.smithi006.stderr: -288> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): picked mon.noname-a con 0x5607c4bbe640 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:21:40.138 INFO:tasks.workunit.client.0.smithi006.stderr: -287> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): picked mon.noname-c con 0x5607c4bc9be0 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:21:40.138 INFO:tasks.workunit.client.0.smithi006.stderr: -286> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): picked mon.noname-b con 0x5607c4bb65a0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:21:40.138 INFO:tasks.workunit.client.0.smithi006.stderr: -285> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): start opening mon connection 2024-05-30T16:21:40.138 INFO:tasks.workunit.client.0.smithi006.stderr: -284> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): start opening mon connection 2024-05-30T16:21:40.139 INFO:tasks.workunit.client.0.smithi006.stderr: -283> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): start opening mon connection 2024-05-30T16:21:40.139 INFO:tasks.workunit.client.0.smithi006.stderr: -282> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): _renew_subs 2024-05-30T16:21:40.139 INFO:tasks.workunit.client.0.smithi006.stderr: -281> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): authenticate will time out at 2639.708422s 2024-05-30T16:21:40.139 INFO:tasks.workunit.client.0.smithi006.stderr: -280> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): get_auth_request con 0x5607c4bbe640 auth_method 0 2024-05-30T16:21:40.139 INFO:tasks.workunit.client.0.smithi006.stderr: -279> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:21:40.139 INFO:tasks.workunit.client.0.smithi006.stderr: -278> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:21:40.139 INFO:tasks.workunit.client.0.smithi006.stderr: -277> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:21:40.139 INFO:tasks.workunit.client.0.smithi006.stderr: -276> 2024-05-30T16:21:39.776+0000 7f7ffa861640 10 monclient(hunting): get_auth_request con 0x5607c4bc9be0 auth_method 0 2024-05-30T16:21:40.139 INFO:tasks.workunit.client.0.smithi006.stderr: -275> 2024-05-30T16:21:39.776+0000 7f7ffa861640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:21:40.139 INFO:tasks.workunit.client.0.smithi006.stderr: -274> 2024-05-30T16:21:39.776+0000 7f7ffa861640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:21:40.139 INFO:tasks.workunit.client.0.smithi006.stderr: -273> 2024-05-30T16:21:39.776+0000 7f7ffa861640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:21:40.139 INFO:tasks.workunit.client.0.smithi006.stderr: -272> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:21:40.140 INFO:tasks.workunit.client.0.smithi006.stderr: -271> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:21:40.140 INFO:tasks.workunit.client.0.smithi006.stderr: -270> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:21:40.140 INFO:tasks.workunit.client.0.smithi006.stderr: -269> 2024-05-30T16:21:39.776+0000 7f7ffa861640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:21:40.140 INFO:tasks.workunit.client.0.smithi006.stderr: -268> 2024-05-30T16:21:39.776+0000 7f7ffa861640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:21:40.140 INFO:tasks.workunit.client.0.smithi006.stderr: -267> 2024-05-30T16:21:39.776+0000 7f7ffa861640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:21:40.140 INFO:tasks.workunit.client.0.smithi006.stderr: -266> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): handle_auth_done global_id 6765 payload 274 2024-05-30T16:21:40.140 INFO:tasks.workunit.client.0.smithi006.stderr: -265> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient: _finish_hunting 0 2024-05-30T16:21:40.140 INFO:tasks.workunit.client.0.smithi006.stderr: -264> 2024-05-30T16:21:39.776+0000 7f7ffb062640 1 monclient: found mon.noname-a 2024-05-30T16:21:40.140 INFO:tasks.workunit.client.0.smithi006.stderr: -263> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.6:3300/0 2024-05-30T16:21:40.140 INFO:tasks.workunit.client.0.smithi006.stderr: -262> 2024-05-30T16:21:39.776+0000 7f7ffb863640 10 monclient: get_auth_request con 0x5607c4bb65a0 auth_method 0 2024-05-30T16:21:40.141 INFO:tasks.workunit.client.0.smithi006.stderr: -261> 2024-05-30T16:21:39.776+0000 7f7edeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:21:40.141 INFO:tasks.workunit.client.0.smithi006.stderr: -260> 2024-05-30T16:21:39.776+0000 7f7edeffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-30T16:21:40.141 INFO:tasks.workunit.client.0.smithi006.stderr: -259> 2024-05-30T16:21:39.776+0000 7f7edeffd640 10 monclient: dump: 2024-05-30T16:21:40.141 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:21:40.141 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:21:40.141 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.141 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.141 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:21:40.141 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:21:40.141 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:21:40.141 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:21:40.141 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:21:40.142 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.142 INFO:tasks.workunit.client.0.smithi006.stderr: -258> 2024-05-30T16:21:39.776+0000 7f7edeffd640 10 monclient: _finish_auth 0 2024-05-30T16:21:40.142 INFO:tasks.workunit.client.0.smithi006.stderr: -257> 2024-05-30T16:21:39.776+0000 7f7edeffd640 10 monclient: _check_auth_tickets 2024-05-30T16:21:40.142 INFO:tasks.workunit.client.0.smithi006.stderr: -256> 2024-05-30T16:21:39.776+0000 7f7edeffd640 10 monclient: handle_config config(0 keys) 2024-05-30T16:21:40.142 INFO:tasks.workunit.client.0.smithi006.stderr: -255> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 monclient: authenticate success, global_id 6765 2024-05-30T16:21:40.142 INFO:tasks.workunit.client.0.smithi006.stderr: -254> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient: get_monmap_and_config success 2024-05-30T16:21:40.142 INFO:tasks.workunit.client.0.smithi006.stderr: -253> 2024-05-30T16:21:39.776+0000 7f7edeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:21:40.142 INFO:tasks.workunit.client.0.smithi006.stderr: -252> 2024-05-30T16:21:39.776+0000 7f7ffd551980 4 set_mon_vals no callback set 2024-05-30T16:21:40.142 INFO:tasks.workunit.client.0.smithi006.stderr: -251> 2024-05-30T16:21:39.776+0000 7f7edeffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-30T16:21:40.142 INFO:tasks.workunit.client.0.smithi006.stderr: -250> 2024-05-30T16:21:39.776+0000 7f7edeffd640 10 monclient: dump: 2024-05-30T16:21:40.143 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:21:40.143 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:21:40.143 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.143 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.143 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:21:40.143 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:21:40.143 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:21:40.143 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:21:40.143 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:21:40.143 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.143 INFO:tasks.workunit.client.0.smithi006.stderr: -249> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient: shutdown 2024-05-30T16:21:40.144 INFO:tasks.workunit.client.0.smithi006.stderr: -248> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient: build_initial_monmap 2024-05-30T16:21:40.144 INFO:tasks.workunit.client.0.smithi006.stderr: -247> 2024-05-30T16:21:39.776+0000 7f7ffd551980 1 build_initial for_mkfs: 0 2024-05-30T16:21:40.144 INFO:tasks.workunit.client.0.smithi006.stderr: -246> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient: monmap: 2024-05-30T16:21:40.144 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 0 2024-05-30T16:21:40.144 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-30T16:21:40.144 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 0.000000 2024-05-30T16:21:40.144 INFO:tasks.workunit.client.0.smithi006.stderr:created 0.000000 2024-05-30T16:21:40.144 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 0 (unknown) 2024-05-30T16:21:40.144 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:21:40.144 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.noname-a 2024-05-30T16:21:40.145 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-b 2024-05-30T16:21:40.145 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.noname-c 2024-05-30T16:21:40.145 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.145 INFO:tasks.workunit.client.0.smithi006.stderr: -245> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b93f20) adding auth protocol: cephx 2024-05-30T16:21:40.145 INFO:tasks.workunit.client.0.smithi006.stderr: -244> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b93f20) adding auth protocol: cephx 2024-05-30T16:21:40.145 INFO:tasks.workunit.client.0.smithi006.stderr: -243> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b93f20) adding auth protocol: cephx 2024-05-30T16:21:40.145 INFO:tasks.workunit.client.0.smithi006.stderr: -242> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b93f20) adding con mode: secure 2024-05-30T16:21:40.145 INFO:tasks.workunit.client.0.smithi006.stderr: -241> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b93f20) adding con mode: crc 2024-05-30T16:21:40.146 INFO:tasks.workunit.client.0.smithi006.stderr: -240> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b93f20) adding con mode: secure 2024-05-30T16:21:40.146 INFO:tasks.workunit.client.0.smithi006.stderr: -239> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b93f20) adding con mode: crc 2024-05-30T16:21:40.146 INFO:tasks.workunit.client.0.smithi006.stderr: -238> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b93f20) adding con mode: secure 2024-05-30T16:21:40.146 INFO:tasks.workunit.client.0.smithi006.stderr: -237> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b93f20) adding con mode: crc 2024-05-30T16:21:40.146 INFO:tasks.workunit.client.0.smithi006.stderr: -236> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b93f20) adding con mode: crc 2024-05-30T16:21:40.146 INFO:tasks.workunit.client.0.smithi006.stderr: -235> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b93f20) adding con mode: secure 2024-05-30T16:21:40.146 INFO:tasks.workunit.client.0.smithi006.stderr: -234> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b93f20) adding con mode: crc 2024-05-30T16:21:40.146 INFO:tasks.workunit.client.0.smithi006.stderr: -233> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b93f20) adding con mode: secure 2024-05-30T16:21:40.146 INFO:tasks.workunit.client.0.smithi006.stderr: -232> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b93f20) adding con mode: crc 2024-05-30T16:21:40.146 INFO:tasks.workunit.client.0.smithi006.stderr: -231> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b93f20) adding con mode: secure 2024-05-30T16:21:40.147 INFO:tasks.workunit.client.0.smithi006.stderr: -230> 2024-05-30T16:21:39.776+0000 7f7ffd551980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:21:40.147 INFO:tasks.workunit.client.0.smithi006.stderr: -229> 2024-05-30T16:21:39.776+0000 7f7ffd551980 1 librados: starting msgr at 2024-05-30T16:21:40.147 INFO:tasks.workunit.client.0.smithi006.stderr: -228> 2024-05-30T16:21:39.776+0000 7f7ffd551980 1 librados: starting objecter 2024-05-30T16:21:40.147 INFO:tasks.workunit.client.0.smithi006.stderr: -227> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command objecter_requests cmddesc objecter_requests hook 0x5607c4b93db0 EEXIST 2024-05-30T16:21:40.147 INFO:tasks.workunit.client.0.smithi006.stderr: -226> 2024-05-30T16:21:39.776+0000 7f7ffd551980 1 librados: setting wanted keys 2024-05-30T16:21:40.147 INFO:tasks.workunit.client.0.smithi006.stderr: -225> 2024-05-30T16:21:39.776+0000 7f7ffd551980 1 librados: calling monclient init 2024-05-30T16:21:40.147 INFO:tasks.workunit.client.0.smithi006.stderr: -224> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient: init 2024-05-30T16:21:40.147 INFO:tasks.workunit.client.0.smithi006.stderr: -223> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b87d90) adding auth protocol: cephx 2024-05-30T16:21:40.148 INFO:tasks.workunit.client.0.smithi006.stderr: -222> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b87d90) adding auth protocol: cephx 2024-05-30T16:21:40.148 INFO:tasks.workunit.client.0.smithi006.stderr: -221> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b87d90) adding auth protocol: cephx 2024-05-30T16:21:40.148 INFO:tasks.workunit.client.0.smithi006.stderr: -220> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b87d90) adding con mode: secure 2024-05-30T16:21:40.148 INFO:tasks.workunit.client.0.smithi006.stderr: -219> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b87d90) adding con mode: crc 2024-05-30T16:21:40.148 INFO:tasks.workunit.client.0.smithi006.stderr: -218> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b87d90) adding con mode: secure 2024-05-30T16:21:40.148 INFO:tasks.workunit.client.0.smithi006.stderr: -217> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b87d90) adding con mode: crc 2024-05-30T16:21:40.148 INFO:tasks.workunit.client.0.smithi006.stderr: -216> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b87d90) adding con mode: secure 2024-05-30T16:21:40.148 INFO:tasks.workunit.client.0.smithi006.stderr: -215> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b87d90) adding con mode: crc 2024-05-30T16:21:40.148 INFO:tasks.workunit.client.0.smithi006.stderr: -214> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b87d90) adding con mode: crc 2024-05-30T16:21:40.148 INFO:tasks.workunit.client.0.smithi006.stderr: -213> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b87d90) adding con mode: secure 2024-05-30T16:21:40.149 INFO:tasks.workunit.client.0.smithi006.stderr: -212> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b87d90) adding con mode: crc 2024-05-30T16:21:40.149 INFO:tasks.workunit.client.0.smithi006.stderr: -211> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b87d90) adding con mode: secure 2024-05-30T16:21:40.149 INFO:tasks.workunit.client.0.smithi006.stderr: -210> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b87d90) adding con mode: crc 2024-05-30T16:21:40.149 INFO:tasks.workunit.client.0.smithi006.stderr: -209> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b87d90) adding con mode: secure 2024-05-30T16:21:40.149 INFO:tasks.workunit.client.0.smithi006.stderr: -208> 2024-05-30T16:21:39.776+0000 7f7ffd551980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:21:40.149 INFO:tasks.workunit.client.0.smithi006.stderr: -207> 2024-05-30T16:21:39.776+0000 7f7ffd551980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:21:40.149 INFO:tasks.workunit.client.0.smithi006.stderr: -206> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command rotate-key cmddesc rotate-key hook 0x5607c4b87ed8 EEXIST 2024-05-30T16:21:40.149 INFO:tasks.workunit.client.0.smithi006.stderr: -205> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient: _reopen_session rank -1 2024-05-30T16:21:40.149 INFO:tasks.workunit.client.0.smithi006.stderr: -204> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient: _add_conns ranks=[0,1,2] 2024-05-30T16:21:40.149 INFO:tasks.workunit.client.0.smithi006.stderr: -203> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): picked mon.noname-a con 0x5607c4bb65a0 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:21:40.150 INFO:tasks.workunit.client.0.smithi006.stderr: -202> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): picked mon.noname-b con 0x5607c4bbe640 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:21:40.150 INFO:tasks.workunit.client.0.smithi006.stderr: -201> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): picked mon.noname-c con 0x5607c4bc9be0 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:21:40.150 INFO:tasks.workunit.client.0.smithi006.stderr: -200> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): start opening mon connection 2024-05-30T16:21:40.150 INFO:tasks.workunit.client.0.smithi006.stderr: -199> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): start opening mon connection 2024-05-30T16:21:40.150 INFO:tasks.workunit.client.0.smithi006.stderr: -198> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): start opening mon connection 2024-05-30T16:21:40.150 INFO:tasks.workunit.client.0.smithi006.stderr: -197> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): _renew_subs 2024-05-30T16:21:40.150 INFO:tasks.workunit.client.0.smithi006.stderr: -196> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): authenticate will time out at 2639.710987s 2024-05-30T16:21:40.150 INFO:tasks.workunit.client.0.smithi006.stderr: -195> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): get_auth_request con 0x5607c4bb65a0 auth_method 0 2024-05-30T16:21:40.150 INFO:tasks.workunit.client.0.smithi006.stderr: -194> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:21:40.150 INFO:tasks.workunit.client.0.smithi006.stderr: -193> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:21:40.150 INFO:tasks.workunit.client.0.smithi006.stderr: -192> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:21:40.150 INFO:tasks.workunit.client.0.smithi006.stderr: -191> 2024-05-30T16:21:39.776+0000 7f7ffb863640 10 monclient(hunting): get_auth_request con 0x5607c4bc9be0 auth_method 0 2024-05-30T16:21:40.151 INFO:tasks.workunit.client.0.smithi006.stderr: -190> 2024-05-30T16:21:39.776+0000 7f7ffb863640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:21:40.151 INFO:tasks.workunit.client.0.smithi006.stderr: -189> 2024-05-30T16:21:39.776+0000 7f7ffb863640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:21:40.151 INFO:tasks.workunit.client.0.smithi006.stderr: -188> 2024-05-30T16:21:39.776+0000 7f7ffb863640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:21:40.151 INFO:tasks.workunit.client.0.smithi006.stderr: -187> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:21:40.151 INFO:tasks.workunit.client.0.smithi006.stderr: -186> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:21:40.151 INFO:tasks.workunit.client.0.smithi006.stderr: -185> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:21:40.151 INFO:tasks.workunit.client.0.smithi006.stderr: -184> 2024-05-30T16:21:39.776+0000 7f7ffb863640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:21:40.151 INFO:tasks.workunit.client.0.smithi006.stderr: -183> 2024-05-30T16:21:39.776+0000 7f7ffb863640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:21:40.151 INFO:tasks.workunit.client.0.smithi006.stderr: -182> 2024-05-30T16:21:39.776+0000 7f7ffb863640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:21:40.151 INFO:tasks.workunit.client.0.smithi006.stderr: -181> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): handle_auth_done global_id 6768 payload 824 2024-05-30T16:21:40.151 INFO:tasks.workunit.client.0.smithi006.stderr: -180> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient: _finish_hunting 0 2024-05-30T16:21:40.151 INFO:tasks.workunit.client.0.smithi006.stderr: -179> 2024-05-30T16:21:39.776+0000 7f7ffb062640 1 monclient: found mon.noname-a 2024-05-30T16:21:40.151 INFO:tasks.workunit.client.0.smithi006.stderr: -178> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.6:3300/0 2024-05-30T16:21:40.152 INFO:tasks.workunit.client.0.smithi006.stderr: -177> 2024-05-30T16:21:39.776+0000 7f7edd7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:21:40.152 INFO:tasks.workunit.client.0.smithi006.stderr: -176> 2024-05-30T16:21:39.776+0000 7f7edd7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-30T16:21:40.152 INFO:tasks.workunit.client.0.smithi006.stderr: -175> 2024-05-30T16:21:39.776+0000 7f7edd7fa640 10 monclient: dump: 2024-05-30T16:21:40.152 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:21:40.152 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:21:40.152 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.152 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.152 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:21:40.152 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:21:40.152 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:21:40.152 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:21:40.152 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:21:40.152 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.153 INFO:tasks.workunit.client.0.smithi006.stderr: -174> 2024-05-30T16:21:39.776+0000 7f7edd7fa640 10 monclient: _finish_auth 0 2024-05-30T16:21:40.153 INFO:tasks.workunit.client.0.smithi006.stderr: -173> 2024-05-30T16:21:39.776+0000 7f7edd7fa640 10 monclient: _check_auth_tickets 2024-05-30T16:21:40.153 INFO:tasks.workunit.client.0.smithi006.stderr: -172> 2024-05-30T16:21:39.776+0000 7f7edd7fa640 10 monclient: handle_config config(0 keys) 2024-05-30T16:21:40.153 INFO:tasks.workunit.client.0.smithi006.stderr: -171> 2024-05-30T16:21:39.776+0000 7f7edd7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:21:40.153 INFO:tasks.workunit.client.0.smithi006.stderr: -170> 2024-05-30T16:21:39.776+0000 7f7edd7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-30T16:21:40.153 INFO:tasks.workunit.client.0.smithi006.stderr: -169> 2024-05-30T16:21:39.776+0000 7f7edd7fa640 10 monclient: dump: 2024-05-30T16:21:40.153 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:21:40.153 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:21:40.153 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.153 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.153 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:21:40.153 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:21:40.154 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:21:40.154 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:21:40.154 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:21:40.154 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.154 INFO:tasks.workunit.client.0.smithi006.stderr: -168> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 monclient: authenticate success, global_id 6768 2024-05-30T16:21:40.154 INFO:tasks.workunit.client.0.smithi006.stderr: -167> 2024-05-30T16:21:39.776+0000 7f7edeffd640 4 set_mon_vals no callback set 2024-05-30T16:21:40.154 INFO:tasks.workunit.client.0.smithi006.stderr: -166> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient: _renew_subs 2024-05-30T16:21:40.154 INFO:tasks.workunit.client.0.smithi006.stderr: -165> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:21:40.154 INFO:tasks.workunit.client.0.smithi006.stderr: -164> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient: _renew_subs 2024-05-30T16:21:40.154 INFO:tasks.workunit.client.0.smithi006.stderr: -163> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:21:40.154 INFO:tasks.workunit.client.0.smithi006.stderr: -162> 2024-05-30T16:21:39.776+0000 7f7ffd551980 1 librados: init done 2024-05-30T16:21:40.155 INFO:tasks.workunit.client.0.smithi006.stderr: -161> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command cr dump hook 0x5607c4bbea38 2024-05-30T16:21:40.155 INFO:tasks.workunit.client.0.smithi006.stderr: -160> 2024-05-30T16:21:39.780+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.155 INFO:tasks.workunit.client.0.smithi006.stderr: -159> 2024-05-30T16:21:39.780+0000 7f7edd7fa640 4 mgrc handle_mgr_map Got map version 5 2024-05-30T16:21:40.155 INFO:tasks.workunit.client.0.smithi006.stderr: -158> 2024-05-30T16:21:39.780+0000 7f7edd7fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:21:40.155 INFO:tasks.workunit.client.0.smithi006.stderr: -157> 2024-05-30T16:21:39.780+0000 7f7edd7fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:21:40.155 INFO:tasks.workunit.client.0.smithi006.stderr: -156> 2024-05-30T16:21:39.780+0000 7f7ffb863640 10 monclient: get_auth_request con 0x5607c4bdd460 auth_method 0 2024-05-30T16:21:40.155 INFO:tasks.workunit.client.0.smithi006.stderr: -155> 2024-05-30T16:21:39.780+0000 7f7ffa861640 10 monclient: get_auth_request con 0x7f7ec406dde0 auth_method 0 2024-05-30T16:21:40.155 INFO:tasks.workunit.client.0.smithi006.stderr: -154> 2024-05-30T16:21:39.780+0000 7f7ffd551980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:40.155 INFO:tasks.workunit.client.0.smithi006.stderr: -153> 2024-05-30T16:21:39.780+0000 7f7ffd551980 20 realm 2024-05-30T16:21:40.155 INFO:tasks.workunit.client.0.smithi006.stderr: -152> 2024-05-30T16:21:39.780+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.155 INFO:tasks.workunit.client.0.smithi006.stderr: -151> 2024-05-30T16:21:39.780+0000 7f7ffd551980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:40.155 INFO:tasks.workunit.client.0.smithi006.stderr: -150> 2024-05-30T16:21:39.780+0000 7f7ffd551980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:21:40.156 INFO:tasks.workunit.client.0.smithi006.stderr: -149> 2024-05-30T16:21:39.780+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.156 INFO:tasks.workunit.client.0.smithi006.stderr: -148> 2024-05-30T16:21:39.780+0000 7f7ffd551980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:40.156 INFO:tasks.workunit.client.0.smithi006.stderr: -147> 2024-05-30T16:21:39.780+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.156 INFO:tasks.workunit.client.0.smithi006.stderr: -146> 2024-05-30T16:21:39.784+0000 7f7ffb062640 10 monclient: get_auth_request con 0x5607c4be17f0 auth_method 0 2024-05-30T16:21:40.156 INFO:tasks.workunit.client.0.smithi006.stderr: -145> 2024-05-30T16:21:39.784+0000 7f7ffd551980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:21:40.156 INFO:tasks.workunit.client.0.smithi006.stderr: -144> 2024-05-30T16:21:39.784+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.156 INFO:tasks.workunit.client.0.smithi006.stderr: -143> 2024-05-30T16:21:39.784+0000 7f7ffd551980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:21:40.156 INFO:tasks.workunit.client.0.smithi006.stderr: -142> 2024-05-30T16:21:39.784+0000 7f7ffd551980 20 searching for the correct realm 2024-05-30T16:21:40.156 INFO:tasks.workunit.client.0.smithi006.stderr: -141> 2024-05-30T16:21:39.784+0000 7f7ffa861640 10 monclient: get_auth_request con 0x5607c4be69b0 auth_method 0 2024-05-30T16:21:40.156 INFO:tasks.workunit.client.0.smithi006.stderr: -140> 2024-05-30T16:21:39.784+0000 7f7ffb863640 10 monclient: get_auth_request con 0x7f7ff0024040 auth_method 0 2024-05-30T16:21:40.156 INFO:tasks.workunit.client.0.smithi006.stderr: -139> 2024-05-30T16:21:39.788+0000 7f7ffb062640 10 monclient: get_auth_request con 0x7f7ff40bd070 auth_method 0 2024-05-30T16:21:40.156 INFO:tasks.workunit.client.0.smithi006.stderr: -138> 2024-05-30T16:21:39.788+0000 7f7ffa861640 10 monclient: get_auth_request con 0x7f7fec026040 auth_method 0 2024-05-30T16:21:40.156 INFO:tasks.workunit.client.0.smithi006.stderr: -137> 2024-05-30T16:21:39.792+0000 7f7ffb863640 10 monclient: get_auth_request con 0x7f7ff004c040 auth_method 0 2024-05-30T16:21:40.157 INFO:tasks.workunit.client.0.smithi006.stderr: -136> 2024-05-30T16:21:39.792+0000 7f7ffb863640 0 -- 172.21.15.6:0/308574472 >> [v2:172.21.15.204:6800/3687190393,v1:172.21.15.204:6801/3687190393] conn(0x7f7ff004c040 msgr2=0x7f7ff001a410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:21:40.157 INFO:tasks.workunit.client.0.smithi006.stderr: -135> 2024-05-30T16:21:39.996+0000 7f7ffb863640 10 monclient: get_auth_request con 0x7f7ff004c040 auth_method 0 2024-05-30T16:21:40.157 INFO:tasks.workunit.client.0.smithi006.stderr: -134> 2024-05-30T16:21:40.000+0000 7f7ffb062640 10 monclient: get_auth_request con 0x7f7ff0008040 auth_method 0 2024-05-30T16:21:40.157 INFO:tasks.workunit.client.0.smithi006.stderr: -133> 2024-05-30T16:21:40.008+0000 7f7ffb062640 0 -- 172.21.15.6:0/308574472 >> [v2:172.21.15.204:6816/2677861390,v1:172.21.15.204:6817/2677861390] conn(0x5607c4be17f0 msgr2=0x5607c4be3be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:21:40.157 INFO:tasks.workunit.client.0.smithi006.stderr: -132> 2024-05-30T16:21:40.008+0000 7f7edd7fa640 1 client.6768.objecter ms_handle_reset 0x5607c4be17f0 session 0x5607c4be06e0 osd.5 2024-05-30T16:21:40.157 INFO:tasks.workunit.client.0.smithi006.stderr: -131> 2024-05-30T16:21:40.008+0000 7f7edd7fa640 10 monclient: _renew_subs 2024-05-30T16:21:40.157 INFO:tasks.workunit.client.0.smithi006.stderr: -130> 2024-05-30T16:21:40.008+0000 7f7edd7fa640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:21:40.157 INFO:tasks.workunit.client.0.smithi006.stderr: -129> 2024-05-30T16:21:40.008+0000 7f7ffb062640 10 monclient: get_auth_request con 0x7f7ec407e510 auth_method 0 2024-05-30T16:21:40.157 INFO:tasks.workunit.client.0.smithi006.stderr: -128> 2024-05-30T16:21:40.016+0000 7f7ffd551980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:21:40.157 INFO:tasks.workunit.client.0.smithi006.stderr: -127> 2024-05-30T16:21:40.016+0000 7f7ffd551980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:21:40.157 INFO:tasks.workunit.client.0.smithi006.stderr: -126> 2024-05-30T16:21:40.016+0000 7f7ffd551980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:21:40.157 INFO:tasks.workunit.client.0.smithi006.stderr: -125> 2024-05-30T16:21:40.016+0000 7f7ffd551980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:21:40.157 INFO:tasks.workunit.client.0.smithi006.stderr: -124> 2024-05-30T16:21:40.016+0000 7f7ffd551980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:21:40.158 INFO:tasks.workunit.client.0.smithi006.stderr: -123> 2024-05-30T16:21:40.016+0000 7f7ffd551980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:21:40.158 INFO:tasks.workunit.client.0.smithi006.stderr: -122> 2024-05-30T16:21:40.016+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.158 INFO:tasks.workunit.client.0.smithi006.stderr: -121> 2024-05-30T16:21:40.016+0000 7f7ffd551980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:40.158 INFO:tasks.workunit.client.0.smithi006.stderr: -120> 2024-05-30T16:21:40.016+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.158 INFO:tasks.workunit.client.0.smithi006.stderr: -119> 2024-05-30T16:21:40.016+0000 7f7ffd551980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:21:40.158 INFO:tasks.workunit.client.0.smithi006.stderr: -118> 2024-05-30T16:21:40.016+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.158 INFO:tasks.workunit.client.0.smithi006.stderr: -117> 2024-05-30T16:21:40.020+0000 7f7ffd551980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:21:40.158 INFO:tasks.workunit.client.0.smithi006.stderr: -116> 2024-05-30T16:21:40.020+0000 7f7ffd551980 20 zone default found 2024-05-30T16:21:40.158 INFO:tasks.workunit.client.0.smithi006.stderr: -115> 2024-05-30T16:21:40.020+0000 7f7ffd551980 4 Realm: () 2024-05-30T16:21:40.158 INFO:tasks.workunit.client.0.smithi006.stderr: -114> 2024-05-30T16:21:40.020+0000 7f7ffd551980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:21:40.158 INFO:tasks.workunit.client.0.smithi006.stderr: -113> 2024-05-30T16:21:40.020+0000 7f7ffd551980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:21:40.158 INFO:tasks.workunit.client.0.smithi006.stderr: -112> 2024-05-30T16:21:40.020+0000 7f7ffd551980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:21:40.159 INFO:tasks.workunit.client.0.smithi006.stderr: -111> 2024-05-30T16:21:40.020+0000 7f7ffd551980 20 zonegroup default 2024-05-30T16:21:40.159 INFO:tasks.workunit.client.0.smithi006.stderr: -110> 2024-05-30T16:21:40.020+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.159 INFO:tasks.workunit.client.0.smithi006.stderr: -109> 2024-05-30T16:21:40.020+0000 7f7ffd551980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:40.159 INFO:tasks.workunit.client.0.smithi006.stderr: -108> 2024-05-30T16:21:40.020+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.159 INFO:tasks.workunit.client.0.smithi006.stderr: -107> 2024-05-30T16:21:40.020+0000 7f7ffd551980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:40.159 INFO:tasks.workunit.client.0.smithi006.stderr: -106> 2024-05-30T16:21:40.020+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.159 INFO:tasks.workunit.client.0.smithi006.stderr: -105> 2024-05-30T16:21:40.020+0000 7f7ffd551980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:40.159 INFO:tasks.workunit.client.0.smithi006.stderr: -104> 2024-05-30T16:21:40.020+0000 7f7ffd551980 20 started sync module instance, tier type = 2024-05-30T16:21:40.159 INFO:tasks.workunit.client.0.smithi006.stderr: -103> 2024-05-30T16:21:40.020+0000 7f7ffd551980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:21:40.159 INFO:tasks.workunit.client.0.smithi006.stderr: -102> 2024-05-30T16:21:40.044+0000 7f7ffd551980 20 add_watcher() i=0 2024-05-30T16:21:40.159 INFO:tasks.workunit.client.0.smithi006.stderr: -101> 2024-05-30T16:21:40.044+0000 7f7ffd551980 20 add_watcher() i=1 2024-05-30T16:21:40.159 INFO:tasks.workunit.client.0.smithi006.stderr: -100> 2024-05-30T16:21:40.044+0000 7f7ffd551980 20 add_watcher() i=2 2024-05-30T16:21:40.159 INFO:tasks.workunit.client.0.smithi006.stderr: -99> 2024-05-30T16:21:40.044+0000 7f7ffd551980 20 add_watcher() i=3 2024-05-30T16:21:40.159 INFO:tasks.workunit.client.0.smithi006.stderr: -98> 2024-05-30T16:21:40.044+0000 7f7ffd551980 20 add_watcher() i=4 2024-05-30T16:21:40.160 INFO:tasks.workunit.client.0.smithi006.stderr: -97> 2024-05-30T16:21:40.044+0000 7f7ffd551980 20 add_watcher() i=5 2024-05-30T16:21:40.160 INFO:tasks.workunit.client.0.smithi006.stderr: -96> 2024-05-30T16:21:40.048+0000 7f7ffd551980 20 add_watcher() i=6 2024-05-30T16:21:40.160 INFO:tasks.workunit.client.0.smithi006.stderr: -95> 2024-05-30T16:21:40.048+0000 7f7ffd551980 20 add_watcher() i=7 2024-05-30T16:21:40.160 INFO:tasks.workunit.client.0.smithi006.stderr: -94> 2024-05-30T16:21:40.048+0000 7f7ffd551980 2 all 8 watchers are set, enabling cache 2024-05-30T16:21:40.160 INFO:tasks.workunit.client.0.smithi006.stderr: -93> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc3cc0) adding auth protocol: cephx 2024-05-30T16:21:40.160 INFO:tasks.workunit.client.0.smithi006.stderr: -92> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc3cc0) adding auth protocol: cephx 2024-05-30T16:21:40.160 INFO:tasks.workunit.client.0.smithi006.stderr: -91> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc3cc0) adding auth protocol: cephx 2024-05-30T16:21:40.160 INFO:tasks.workunit.client.0.smithi006.stderr: -90> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc3cc0) adding con mode: secure 2024-05-30T16:21:40.160 INFO:tasks.workunit.client.0.smithi006.stderr: -89> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc3cc0) adding con mode: crc 2024-05-30T16:21:40.160 INFO:tasks.workunit.client.0.smithi006.stderr: -88> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc3cc0) adding con mode: secure 2024-05-30T16:21:40.160 INFO:tasks.workunit.client.0.smithi006.stderr: -87> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc3cc0) adding con mode: crc 2024-05-30T16:21:40.160 INFO:tasks.workunit.client.0.smithi006.stderr: -86> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc3cc0) adding con mode: secure 2024-05-30T16:21:40.160 INFO:tasks.workunit.client.0.smithi006.stderr: -85> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc3cc0) adding con mode: crc 2024-05-30T16:21:40.161 INFO:tasks.workunit.client.0.smithi006.stderr: -84> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc3cc0) adding con mode: crc 2024-05-30T16:21:40.161 INFO:tasks.workunit.client.0.smithi006.stderr: -83> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc3cc0) adding con mode: secure 2024-05-30T16:21:40.161 INFO:tasks.workunit.client.0.smithi006.stderr: -82> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc3cc0) adding con mode: crc 2024-05-30T16:21:40.161 INFO:tasks.workunit.client.0.smithi006.stderr: -81> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc3cc0) adding con mode: secure 2024-05-30T16:21:40.161 INFO:tasks.workunit.client.0.smithi006.stderr: -80> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc3cc0) adding con mode: crc 2024-05-30T16:21:40.161 INFO:tasks.workunit.client.0.smithi006.stderr: -79> 2024-05-30T16:21:40.052+0000 7f7eb1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:21:40.161 INFO:tasks.workunit.client.0.smithi006.stderr: -78> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc3cc0) adding con mode: secure 2024-05-30T16:21:40.161 INFO:tasks.workunit.client.0.smithi006.stderr: -77> 2024-05-30T16:21:40.052+0000 7f7ffd551980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:21:40.161 INFO:tasks.workunit.client.0.smithi006.stderr: -76> 2024-05-30T16:21:40.052+0000 7f7ffd551980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:21:40.161 INFO:tasks.workunit.client.0.smithi006.stderr: -75> 2024-05-30T16:21:40.052+0000 7f7ffd551980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:21:40.161 INFO:tasks.workunit.client.0.smithi006.stderr: -74> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command cache list hook 0x5607c4b965b0 2024-05-30T16:21:40.161 INFO:tasks.workunit.client.0.smithi006.stderr: -73> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command cache inspect hook 0x5607c4b965b0 2024-05-30T16:21:40.161 INFO:tasks.workunit.client.0.smithi006.stderr: -72> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command cache erase hook 0x5607c4b965b0 2024-05-30T16:21:40.162 INFO:tasks.workunit.client.0.smithi006.stderr: -71> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command cache zap hook 0x5607c4b965b0 2024-05-30T16:21:40.162 INFO:tasks.workunit.client.0.smithi006.stderr: -70> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 note: GC not initialized 2024-05-30T16:21:40.162 INFO:tasks.workunit.client.0.smithi006.stderr: -69> 2024-05-30T16:21:40.056+0000 7f7eb0fe1640 20 reqs_thread_entry: start 2024-05-30T16:21:40.162 INFO:tasks.workunit.client.0.smithi006.stderr: -68> 2024-05-30T16:21:40.056+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command sync trace show hook 0x7f7ff000f040 2024-05-30T16:21:40.162 INFO:tasks.workunit.client.0.smithi006.stderr: -67> 2024-05-30T16:21:40.056+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command sync trace history hook 0x7f7ff000f040 2024-05-30T16:21:40.162 INFO:tasks.workunit.client.0.smithi006.stderr: -66> 2024-05-30T16:21:40.056+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command sync trace active hook 0x7f7ff000f040 2024-05-30T16:21:40.162 INFO:tasks.workunit.client.0.smithi006.stderr: -65> 2024-05-30T16:21:40.056+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command sync trace active_short hook 0x7f7ff000f040 2024-05-30T16:21:40.162 INFO:tasks.workunit.client.0.smithi006.stderr: -64> 2024-05-30T16:21:40.056+0000 7f7ffd551980 20 init_complete bucket index max shards: 11 2024-05-30T16:21:40.162 INFO:tasks.workunit.client.0.smithi006.stderr: -63> 2024-05-30T16:21:40.056+0000 7f7ffd551980 20 Filter name: none 2024-05-30T16:21:40.162 INFO:tasks.workunit.client.0.smithi006.stderr: -62> 2024-05-30T16:21:40.056+0000 7f7e2a7fc640 20 reqs_thread_entry: start 2024-05-30T16:21:40.162 INFO:tasks.workunit.client.0.smithi006.stderr: -61> 2024-05-30T16:21:40.056+0000 7f7ffd551980 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2024-05-30T16:21:40.162 INFO:tasks.workunit.client.0.smithi006.stderr: -60> 2024-05-30T16:21:40.056+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.162 INFO:tasks.workunit.client.0.smithi006.stderr: -59> 2024-05-30T16:21:40.056+0000 7f7ffd551980 20 rados_obj.operate() r=0 bl.length=221 2024-05-30T16:21:40.163 INFO:tasks.workunit.client.0.smithi006.stderr: -58> 2024-05-30T16:21:40.056+0000 7f7ffd551980 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2024-05-30T16:21:40.163 INFO:tasks.workunit.client.0.smithi006.stderr: -57> 2024-05-30T16:21:40.056+0000 7f7ffd551980 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2024-05-30T16:21:40.163 INFO:tasks.workunit.client.0.smithi006.stderr: -56> 2024-05-30T16:21:40.056+0000 7f7ffd551980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 : miss 2024-05-30T16:21:40.163 INFO:tasks.workunit.client.0.smithi006.stderr: -55> 2024-05-30T16:21:40.056+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.163 INFO:tasks.workunit.client.0.smithi006.stderr: -54> 2024-05-30T16:21:40.056+0000 7f7ffd551980 20 rados_obj.operate() r=0 bl.length=435 2024-05-30T16:21:40.163 INFO:tasks.workunit.client.0.smithi006.stderr: -53> 2024-05-30T16:21:40.056+0000 7f7ffd551980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 info.flags=0x17 2024-05-30T16:21:40.163 INFO:tasks.workunit.client.0.smithi006.stderr: -52> 2024-05-30T16:21:40.056+0000 7f7ffd551980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 to cache LRU end 2024-05-30T16:21:40.163 INFO:tasks.workunit.client.0.smithi006.stderr: -51> 2024-05-30T16:21:40.056+0000 7f7ffd551980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:21:40.163 INFO:tasks.workunit.client.0.smithi006.stderr: -50> 2024-05-30T16:21:40.056+0000 7f7ffd551980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:21:40.163 INFO:tasks.workunit.client.0.smithi006.stderr: -49> 2024-05-30T16:21:40.056+0000 7f7ffd551980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 2024-05-30T16:21:40.163 INFO:tasks.workunit.client.0.smithi006.stderr: -48> 2024-05-30T16:21:40.068+0000 7f7ffd551980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 info.flags=0x17 2024-05-30T16:21:40.163 INFO:tasks.workunit.client.0.smithi006.stderr: -47> 2024-05-30T16:21:40.068+0000 7f7ffd551980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 to cache LRU end 2024-05-30T16:21:40.163 INFO:tasks.workunit.client.0.smithi006.stderr: -46> 2024-05-30T16:21:40.068+0000 7f7ffd551980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:21:40.164 INFO:tasks.workunit.client.0.smithi006.stderr: -45> 2024-05-30T16:21:40.068+0000 7f7ffd551980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27, ofs0, ns] 2024-05-30T16:21:40.164 INFO:tasks.workunit.client.0.smithi006.stderr: -44> 2024-05-30T16:21:40.068+0000 7f7edeffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431626 cookie 94591365588048 notifier 6768 bl.length()=1007 2024-05-30T16:21:40.164 INFO:tasks.workunit.client.0.smithi006.stderr: -43> 2024-05-30T16:21:40.068+0000 7f7edeffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 info.flags=0x17 2024-05-30T16:21:40.164 INFO:tasks.workunit.client.0.smithi006.stderr: -42> 2024-05-30T16:21:40.068+0000 7f7edeffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 to cache LRU end 2024-05-30T16:21:40.164 INFO:tasks.workunit.client.0.smithi006.stderr: -41> 2024-05-30T16:21:40.068+0000 7f7edeffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:21:40.164 INFO:tasks.workunit.client.0.smithi006.stderr: -40> 2024-05-30T16:21:40.072+0000 7f7ffd551980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27]): orig_sources= new_sources= 2024-05-30T16:21:40.164 INFO:tasks.workunit.client.0.smithi006.stderr: -39> 2024-05-30T16:21:40.072+0000 7f7ffd551980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27]): potential sources added=[] removed=[] 2024-05-30T16:21:40.164 INFO:tasks.workunit.client.0.smithi006.stderr: -38> 2024-05-30T16:21:40.072+0000 7f7ffd551980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27]): orig_dests= new_dests= 2024-05-30T16:21:40.164 INFO:tasks.workunit.client.0.smithi006.stderr: -37> 2024-05-30T16:21:40.072+0000 7f7ffd551980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27]): potential dests added=[] removed=[] 2024-05-30T16:21:40.164 INFO:tasks.workunit.client.0.smithi006.stderr: -36> 2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.0 2024-05-30T16:21:40.164 INFO:tasks.workunit.client.0.smithi006.stderr: -35> 2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.0 2024-05-30T16:21:40.164 INFO:tasks.workunit.client.0.smithi006.stderr: -34> 2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.1 2024-05-30T16:21:40.164 INFO:tasks.workunit.client.0.smithi006.stderr: -33> 2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.1 2024-05-30T16:21:40.165 INFO:tasks.workunit.client.0.smithi006.stderr: -32> 2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.2 2024-05-30T16:21:40.165 INFO:tasks.workunit.client.0.smithi006.stderr: -31> 2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.2 2024-05-30T16:21:40.165 INFO:tasks.workunit.client.0.smithi006.stderr: -30> 2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.3 2024-05-30T16:21:40.165 INFO:tasks.workunit.client.0.smithi006.stderr: -29> 2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.3 2024-05-30T16:21:40.165 INFO:tasks.workunit.client.0.smithi006.stderr: -28> 2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.4 2024-05-30T16:21:40.165 INFO:tasks.workunit.client.0.smithi006.stderr: -27> 2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.4 2024-05-30T16:21:40.165 INFO:tasks.workunit.client.0.smithi006.stderr: -26> 2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.5 2024-05-30T16:21:40.165 INFO:tasks.workunit.client.0.smithi006.stderr: -25> 2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.5 2024-05-30T16:21:40.165 INFO:tasks.workunit.client.0.smithi006.stderr: -24> 2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.6 2024-05-30T16:21:40.165 INFO:tasks.workunit.client.0.smithi006.stderr: -23> 2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.6 2024-05-30T16:21:40.165 INFO:tasks.workunit.client.0.smithi006.stderr: -22> 2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.7 2024-05-30T16:21:40.165 INFO:tasks.workunit.client.0.smithi006.stderr: -21> 2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.7 2024-05-30T16:21:40.165 INFO:tasks.workunit.client.0.smithi006.stderr: -20> 2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.8 2024-05-30T16:21:40.166 INFO:tasks.workunit.client.0.smithi006.stderr: -19> 2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.8 2024-05-30T16:21:40.166 INFO:tasks.workunit.client.0.smithi006.stderr: -18> 2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.9 2024-05-30T16:21:40.166 INFO:tasks.workunit.client.0.smithi006.stderr: -17> 2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.9 2024-05-30T16:21:40.166 INFO:tasks.workunit.client.0.smithi006.stderr: -16> 2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.10 2024-05-30T16:21:40.166 INFO:tasks.workunit.client.0.smithi006.stderr: -15> 2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.10 2024-05-30T16:21:40.166 INFO:tasks.workunit.client.0.smithi006.stderr: -14> 2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.11 2024-05-30T16:21:40.166 INFO:tasks.workunit.client.0.smithi006.stderr: -13> 2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.11 2024-05-30T16:21:40.166 INFO:tasks.workunit.client.0.smithi006.stderr: -12> 2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.0 2024-05-30T16:21:40.166 INFO:tasks.workunit.client.0.smithi006.stderr: -11> 2024-05-30T16:21:40.080+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1 2024-05-30T16:21:40.166 INFO:tasks.workunit.client.0.smithi006.stderr: -10> 2024-05-30T16:21:40.080+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.2 2024-05-30T16:21:40.166 INFO:tasks.workunit.client.0.smithi006.stderr: -9> 2024-05-30T16:21:40.080+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.3 2024-05-30T16:21:40.166 INFO:tasks.workunit.client.0.smithi006.stderr: -8> 2024-05-30T16:21:40.080+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.4 2024-05-30T16:21:40.166 INFO:tasks.workunit.client.0.smithi006.stderr: -7> 2024-05-30T16:21:40.084+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.5 2024-05-30T16:21:40.167 INFO:tasks.workunit.client.0.smithi006.stderr: -6> 2024-05-30T16:21:40.084+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.6 2024-05-30T16:21:40.167 INFO:tasks.workunit.client.0.smithi006.stderr: -5> 2024-05-30T16:21:40.084+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.7 2024-05-30T16:21:40.167 INFO:tasks.workunit.client.0.smithi006.stderr: -4> 2024-05-30T16:21:40.088+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.8 2024-05-30T16:21:40.167 INFO:tasks.workunit.client.0.smithi006.stderr: -3> 2024-05-30T16:21:40.088+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.9 2024-05-30T16:21:40.167 INFO:tasks.workunit.client.0.smithi006.stderr: -2> 2024-05-30T16:21:40.088+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.10 2024-05-30T16:21:40.167 INFO:tasks.workunit.client.0.smithi006.stderr: -1> 2024-05-30T16:21:40.096+0000 7f7ffd551980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f7ffd551980 time 2024-05-30T16:21:40.096090+0000 2024-05-30T16:21:40.167 INFO:tasks.workunit.client.0.smithi006.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-30T16:21:40.167 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.167 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:21:40.167 INFO:tasks.workunit.client.0.smithi006.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f8000e28cff] 2024-05-30T16:21:40.167 INFO:tasks.workunit.client.0.smithi006.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7f8000e28eb3] 2024-05-30T16:21:40.167 INFO:tasks.workunit.client.0.smithi006.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x5607c26f5f81] 2024-05-30T16:21:40.167 INFO:tasks.workunit.client.0.smithi006.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x56f) [0x5607c270816f] 2024-05-30T16:21:40.168 INFO:tasks.workunit.client.0.smithi006.stderr: 5: main() 2024-05-30T16:21:40.168 INFO:tasks.workunit.client.0.smithi006.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f7fffc91d90] 2024-05-30T16:21:40.168 INFO:tasks.workunit.client.0.smithi006.stderr: 7: __libc_start_main() 2024-05-30T16:21:40.168 INFO:tasks.workunit.client.0.smithi006.stderr: 8: _start() 2024-05-30T16:21:40.168 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.168 INFO:tasks.workunit.client.0.smithi006.stderr: 0> 2024-05-30T16:21:40.096+0000 7f7ffd551980 -1 *** Caught signal (Aborted) ** 2024-05-30T16:21:40.168 INFO:tasks.workunit.client.0.smithi006.stderr: in thread 7f7ffd551980 thread_name:radosgw-admin 2024-05-30T16:21:40.168 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.168 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:21:40.168 INFO:tasks.workunit.client.0.smithi006.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f7fffcaa520] 2024-05-30T16:21:40.168 INFO:tasks.workunit.client.0.smithi006.stderr: 2: pthread_kill() 2024-05-30T16:21:40.168 INFO:tasks.workunit.client.0.smithi006.stderr: 3: raise() 2024-05-30T16:21:40.168 INFO:tasks.workunit.client.0.smithi006.stderr: 4: abort() 2024-05-30T16:21:40.168 INFO:tasks.workunit.client.0.smithi006.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f8000e28d59] 2024-05-30T16:21:40.169 INFO:tasks.workunit.client.0.smithi006.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7f8000e28eb3] 2024-05-30T16:21:40.169 INFO:tasks.workunit.client.0.smithi006.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x5607c26f5f81] 2024-05-30T16:21:40.169 INFO:tasks.workunit.client.0.smithi006.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x56f) [0x5607c270816f] 2024-05-30T16:21:40.169 INFO:tasks.workunit.client.0.smithi006.stderr: 9: main() 2024-05-30T16:21:40.169 INFO:tasks.workunit.client.0.smithi006.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f7fffc91d90] 2024-05-30T16:21:40.169 INFO:tasks.workunit.client.0.smithi006.stderr: 11: __libc_start_main() 2024-05-30T16:21:40.169 INFO:tasks.workunit.client.0.smithi006.stderr: 12: _start() 2024-05-30T16:21:40.169 INFO:tasks.workunit.client.0.smithi006.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-30T16:21:40.170 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.170 INFO:tasks.workunit.client.0.smithi006.stderr:--- logging levels --- 2024-05-30T16:21:40.170 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 none 2024-05-30T16:21:40.170 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 lockdep 2024-05-30T16:21:40.170 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 context 2024-05-30T16:21:40.170 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 crush 2024-05-30T16:21:40.170 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds 2024-05-30T16:21:40.170 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_balancer 2024-05-30T16:21:40.170 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_locker 2024-05-30T16:21:40.170 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_log 2024-05-30T16:21:40.170 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_log_expire 2024-05-30T16:21:40.170 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_migrator 2024-05-30T16:21:40.170 INFO:tasks.workunit.client.0.smithi006.stderr: 3/ 5 mds_quiesce 2024-05-30T16:21:40.171 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 buffer 2024-05-30T16:21:40.171 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 timer 2024-05-30T16:21:40.171 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 filer 2024-05-30T16:21:40.171 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 striper 2024-05-30T16:21:40.171 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 objecter 2024-05-30T16:21:40.171 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rados 2024-05-30T16:21:40.171 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd 2024-05-30T16:21:40.171 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd_mirror 2024-05-30T16:21:40.171 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd_replay 2024-05-30T16:21:40.171 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd_pwl 2024-05-30T16:21:40.171 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 journaler 2024-05-30T16:21:40.172 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 objectcacher 2024-05-30T16:21:40.172 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 immutable_obj_cache 2024-05-30T16:21:40.172 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 client 2024-05-30T16:21:40.172 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 osd 2024-05-30T16:21:40.172 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 optracker 2024-05-30T16:21:40.172 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 objclass 2024-05-30T16:21:40.172 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 3 filestore 2024-05-30T16:21:40.172 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 3 journal 2024-05-30T16:21:40.172 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 0 ms 2024-05-30T16:21:40.172 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mon 2024-05-30T16:21:40.172 INFO:tasks.workunit.client.0.smithi006.stderr: 0/10 monc 2024-05-30T16:21:40.172 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 paxos 2024-05-30T16:21:40.173 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 tp 2024-05-30T16:21:40.173 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 auth 2024-05-30T16:21:40.173 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 crypto 2024-05-30T16:21:40.173 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 finisher 2024-05-30T16:21:40.173 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 reserver 2024-05-30T16:21:40.173 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 heartbeatmap 2024-05-30T16:21:40.173 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 perfcounter 2024-05-30T16:21:40.173 INFO:tasks.workunit.client.0.smithi006.stderr: 20/20 rgw 2024-05-30T16:21:40.173 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_sync 2024-05-30T16:21:40.173 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_datacache 2024-05-30T16:21:40.173 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_access 2024-05-30T16:21:40.173 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_dbstore 2024-05-30T16:21:40.174 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_flight 2024-05-30T16:21:40.174 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_lifecycle 2024-05-30T16:21:40.174 INFO:tasks.workunit.client.0.smithi006.stderr: 20/20 rgw_notification 2024-05-30T16:21:40.174 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 javaclient 2024-05-30T16:21:40.174 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 asok 2024-05-30T16:21:40.174 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 throttle 2024-05-30T16:21:40.174 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 0 refs 2024-05-30T16:21:40.174 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 compressor 2024-05-30T16:21:40.174 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 bluestore 2024-05-30T16:21:40.174 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 bluefs 2024-05-30T16:21:40.174 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 3 bdev 2024-05-30T16:21:40.175 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 kstore 2024-05-30T16:21:40.175 INFO:tasks.workunit.client.0.smithi006.stderr: 4/ 5 rocksdb 2024-05-30T16:21:40.175 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 fuse 2024-05-30T16:21:40.175 INFO:tasks.workunit.client.0.smithi006.stderr: 2/ 5 mgr 2024-05-30T16:21:40.175 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mgrc 2024-05-30T16:21:40.175 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 dpdk 2024-05-30T16:21:40.175 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 eventtrace 2024-05-30T16:21:40.175 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 prioritycache 2024-05-30T16:21:40.175 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 test 2024-05-30T16:21:40.175 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 cephfs_mirror 2024-05-30T16:21:40.175 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 cephsqlite 2024-05-30T16:21:40.176 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 crimson_interrupt 2024-05-30T16:21:40.176 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore 2024-05-30T16:21:40.176 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_onode 2024-05-30T16:21:40.176 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_odata 2024-05-30T16:21:40.176 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_omap 2024-05-30T16:21:40.176 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_tm 2024-05-30T16:21:40.176 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_t 2024-05-30T16:21:40.176 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_cleaner 2024-05-30T16:21:40.176 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_epm 2024-05-30T16:21:40.176 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_lba 2024-05-30T16:21:40.176 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_fixedkv_tree 2024-05-30T16:21:40.177 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_cache 2024-05-30T16:21:40.177 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_journal 2024-05-30T16:21:40.177 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_device 2024-05-30T16:21:40.177 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_backref 2024-05-30T16:21:40.177 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 alienstore 2024-05-30T16:21:40.177 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mclock 2024-05-30T16:21:40.177 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 cyanstore 2024-05-30T16:21:40.177 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 ceph_exporter 2024-05-30T16:21:40.177 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 memstore 2024-05-30T16:21:40.177 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 trace 2024-05-30T16:21:40.178 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 ceph_dedup 2024-05-30T16:21:40.178 INFO:tasks.workunit.client.0.smithi006.stderr: -2/-2 (syslog threshold) 2024-05-30T16:21:40.178 INFO:tasks.workunit.client.0.smithi006.stderr: 99/99 (stderr threshold) 2024-05-30T16:21:40.178 INFO:tasks.workunit.client.0.smithi006.stderr:--- pthread ID / name mapping for recent threads --- 2024-05-30T16:21:40.178 INFO:tasks.workunit.client.0.smithi006.stderr: 7f7e2a7fc640 / http_manager 2024-05-30T16:21:40.178 INFO:tasks.workunit.client.0.smithi006.stderr: 7f7eb0fe1640 / http_manager 2024-05-30T16:21:40.178 INFO:tasks.workunit.client.0.smithi006.stderr: 7f7eb1fe3640 / rgw_dt_lg_renew 2024-05-30T16:21:40.178 INFO:tasks.workunit.client.0.smithi006.stderr: 7f7edd7fa640 / ms_dispatch 2024-05-30T16:21:40.178 INFO:tasks.workunit.client.0.smithi006.stderr: 7f7edeffd640 / io_context_pool 2024-05-30T16:21:40.178 INFO:tasks.workunit.client.0.smithi006.stderr: 7f7ee5602640 / ms_dispatch 2024-05-30T16:21:40.178 INFO:tasks.workunit.client.0.smithi006.stderr: 7f7ee7e07640 / ms_dispatch 2024-05-30T16:21:40.178 INFO:tasks.workunit.client.0.smithi006.stderr: 7f7ee960a640 / io_context_pool 2024-05-30T16:21:40.179 INFO:tasks.workunit.client.0.smithi006.stderr: 7f7ffa861640 / msgr-worker-2 2024-05-30T16:21:40.179 INFO:tasks.workunit.client.0.smithi006.stderr: 7f7ffb062640 / msgr-worker-1 2024-05-30T16:21:40.179 INFO:tasks.workunit.client.0.smithi006.stderr: 7f7ffb863640 / msgr-worker-0 2024-05-30T16:21:40.179 INFO:tasks.workunit.client.0.smithi006.stderr: 7f7ffc064640 / io_context_pool 2024-05-30T16:21:40.179 INFO:tasks.workunit.client.0.smithi006.stderr: 7f7ffd551980 / radosgw-admin 2024-05-30T16:21:40.179 INFO:tasks.workunit.client.0.smithi006.stderr: max_recent 500 2024-05-30T16:21:40.179 INFO:tasks.workunit.client.0.smithi006.stderr: max_new 1000 2024-05-30T16:21:40.179 INFO:tasks.workunit.client.0.smithi006.stderr: log_file /var/log/ceph/ceph-client.admin.95772.log 2024-05-30T16:21:40.179 INFO:tasks.workunit.client.0.smithi006.stderr:--- end dump of recent events --- 2024-05-30T16:21:40.179 INFO:tasks.workunit.client.0.smithi006.stderr:--- begin dump of recent events --- 2024-05-30T16:21:40.179 INFO:tasks.workunit.client.0.smithi006.stderr: -499> 2024-05-30T16:21:39.756+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af92e0) adding con mode: crc 2024-05-30T16:21:40.179 INFO:tasks.workunit.client.0.smithi006.stderr: -498> 2024-05-30T16:21:39.756+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af92e0) adding con mode: secure 2024-05-30T16:21:40.179 INFO:tasks.workunit.client.0.smithi006.stderr: -497> 2024-05-30T16:21:39.756+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af92e0) adding con mode: crc 2024-05-30T16:21:40.179 INFO:tasks.workunit.client.0.smithi006.stderr: -496> 2024-05-30T16:21:39.756+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af92e0) adding con mode: secure 2024-05-30T16:21:40.180 INFO:tasks.workunit.client.0.smithi006.stderr: -495> 2024-05-30T16:21:39.760+0000 7f7ffd551980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:21:40.180 INFO:tasks.workunit.client.0.smithi006.stderr: -494> 2024-05-30T16:21:39.760+0000 7f7ffd551980 1 librados: starting msgr at 2024-05-30T16:21:40.180 INFO:tasks.workunit.client.0.smithi006.stderr: -493> 2024-05-30T16:21:39.760+0000 7f7ffd551980 1 librados: starting objecter 2024-05-30T16:21:40.180 INFO:tasks.workunit.client.0.smithi006.stderr: -492> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command objecter_requests hook 0x5607c48d9a50 2024-05-30T16:21:40.180 INFO:tasks.workunit.client.0.smithi006.stderr: -491> 2024-05-30T16:21:39.760+0000 7f7ffd551980 1 librados: setting wanted keys 2024-05-30T16:21:40.180 INFO:tasks.workunit.client.0.smithi006.stderr: -490> 2024-05-30T16:21:39.760+0000 7f7ffd551980 1 librados: calling monclient init 2024-05-30T16:21:40.180 INFO:tasks.workunit.client.0.smithi006.stderr: -489> 2024-05-30T16:21:39.760+0000 7f7ffd551980 10 monclient: init 2024-05-30T16:21:40.180 INFO:tasks.workunit.client.0.smithi006.stderr: -488> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af3e70) adding auth protocol: cephx 2024-05-30T16:21:40.180 INFO:tasks.workunit.client.0.smithi006.stderr: -487> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af3e70) adding auth protocol: cephx 2024-05-30T16:21:40.180 INFO:tasks.workunit.client.0.smithi006.stderr: -486> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af3e70) adding auth protocol: cephx 2024-05-30T16:21:40.180 INFO:tasks.workunit.client.0.smithi006.stderr: -485> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af3e70) adding con mode: secure 2024-05-30T16:21:40.180 INFO:tasks.workunit.client.0.smithi006.stderr: -484> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af3e70) adding con mode: crc 2024-05-30T16:21:40.180 INFO:tasks.workunit.client.0.smithi006.stderr: -483> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af3e70) adding con mode: secure 2024-05-30T16:21:40.181 INFO:tasks.workunit.client.0.smithi006.stderr: -482> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af3e70) adding con mode: crc 2024-05-30T16:21:40.181 INFO:tasks.workunit.client.0.smithi006.stderr: -481> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af3e70) adding con mode: secure 2024-05-30T16:21:40.181 INFO:tasks.workunit.client.0.smithi006.stderr: -480> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af3e70) adding con mode: crc 2024-05-30T16:21:40.181 INFO:tasks.workunit.client.0.smithi006.stderr: -479> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af3e70) adding con mode: crc 2024-05-30T16:21:40.181 INFO:tasks.workunit.client.0.smithi006.stderr: -478> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af3e70) adding con mode: secure 2024-05-30T16:21:40.181 INFO:tasks.workunit.client.0.smithi006.stderr: -477> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af3e70) adding con mode: crc 2024-05-30T16:21:40.181 INFO:tasks.workunit.client.0.smithi006.stderr: -476> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af3e70) adding con mode: secure 2024-05-30T16:21:40.181 INFO:tasks.workunit.client.0.smithi006.stderr: -475> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af3e70) adding con mode: crc 2024-05-30T16:21:40.181 INFO:tasks.workunit.client.0.smithi006.stderr: -474> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 AuthRegistry(0x5607c4af3e70) adding con mode: secure 2024-05-30T16:21:40.181 INFO:tasks.workunit.client.0.smithi006.stderr: -473> 2024-05-30T16:21:39.760+0000 7f7ffd551980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:21:40.181 INFO:tasks.workunit.client.0.smithi006.stderr: -472> 2024-05-30T16:21:39.760+0000 7f7ffd551980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:21:40.182 INFO:tasks.workunit.client.0.smithi006.stderr: -471> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command rotate-key hook 0x5607c4af3fb8 2024-05-30T16:21:40.182 INFO:tasks.workunit.client.0.smithi006.stderr: -470> 2024-05-30T16:21:39.760+0000 7f7ffd551980 10 monclient: _reopen_session rank -1 2024-05-30T16:21:40.182 INFO:tasks.workunit.client.0.smithi006.stderr: -469> 2024-05-30T16:21:39.760+0000 7f7ffd551980 10 monclient: _add_conns ranks=[1,0,2] 2024-05-30T16:21:40.182 INFO:tasks.workunit.client.0.smithi006.stderr: -468> 2024-05-30T16:21:39.760+0000 7f7ffd551980 10 monclient(hunting): picked mon.noname-b con 0x5607c4985220 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:21:40.182 INFO:tasks.workunit.client.0.smithi006.stderr: -467> 2024-05-30T16:21:39.760+0000 7f7ffd551980 10 monclient(hunting): picked mon.noname-a con 0x5607c49a4070 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:21:40.182 INFO:tasks.workunit.client.0.smithi006.stderr: -466> 2024-05-30T16:21:39.760+0000 7f7ffd551980 10 monclient(hunting): picked mon.noname-c con 0x5607c4b21430 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:21:40.182 INFO:tasks.workunit.client.0.smithi006.stderr: -465> 2024-05-30T16:21:39.760+0000 7f7ffd551980 10 monclient(hunting): start opening mon connection 2024-05-30T16:21:40.182 INFO:tasks.workunit.client.0.smithi006.stderr: -464> 2024-05-30T16:21:39.760+0000 7f7ffd551980 10 monclient(hunting): start opening mon connection 2024-05-30T16:21:40.183 INFO:tasks.workunit.client.0.smithi006.stderr: -463> 2024-05-30T16:21:39.760+0000 7f7ffd551980 10 monclient(hunting): start opening mon connection 2024-05-30T16:21:40.183 INFO:tasks.workunit.client.0.smithi006.stderr: -462> 2024-05-30T16:21:39.760+0000 7f7ffd551980 10 monclient(hunting): _renew_subs 2024-05-30T16:21:40.183 INFO:tasks.workunit.client.0.smithi006.stderr: -461> 2024-05-30T16:21:39.760+0000 7f7ffd551980 10 monclient(hunting): authenticate will time out at 2639.692976s 2024-05-30T16:21:40.183 INFO:tasks.workunit.client.0.smithi006.stderr: -460> 2024-05-30T16:21:39.760+0000 7f7ffb863640 10 monclient(hunting): get_auth_request con 0x5607c4b21430 auth_method 0 2024-05-30T16:21:40.183 INFO:tasks.workunit.client.0.smithi006.stderr: -459> 2024-05-30T16:21:39.760+0000 7f7ffb863640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:21:40.183 INFO:tasks.workunit.client.0.smithi006.stderr: -458> 2024-05-30T16:21:39.760+0000 7f7ffb863640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:21:40.183 INFO:tasks.workunit.client.0.smithi006.stderr: -457> 2024-05-30T16:21:39.760+0000 7f7ffb863640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:21:40.183 INFO:tasks.workunit.client.0.smithi006.stderr: -456> 2024-05-30T16:21:39.760+0000 7f7ffa861640 10 monclient(hunting): get_auth_request con 0x5607c49a4070 auth_method 0 2024-05-30T16:21:40.183 INFO:tasks.workunit.client.0.smithi006.stderr: -455> 2024-05-30T16:21:39.760+0000 7f7ffa861640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:21:40.184 INFO:tasks.workunit.client.0.smithi006.stderr: -454> 2024-05-30T16:21:39.760+0000 7f7ffa861640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:21:40.184 INFO:tasks.workunit.client.0.smithi006.stderr: -453> 2024-05-30T16:21:39.760+0000 7f7ffa861640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:21:40.184 INFO:tasks.workunit.client.0.smithi006.stderr: -452> 2024-05-30T16:21:39.760+0000 7f7ffb863640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:21:40.184 INFO:tasks.workunit.client.0.smithi006.stderr: -451> 2024-05-30T16:21:39.760+0000 7f7ffb863640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:21:40.184 INFO:tasks.workunit.client.0.smithi006.stderr: -450> 2024-05-30T16:21:39.760+0000 7f7ffb863640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:21:40.184 INFO:tasks.workunit.client.0.smithi006.stderr: -449> 2024-05-30T16:21:39.760+0000 7f7ffa861640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:21:40.184 INFO:tasks.workunit.client.0.smithi006.stderr: -448> 2024-05-30T16:21:39.760+0000 7f7ffa861640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:21:40.184 INFO:tasks.workunit.client.0.smithi006.stderr: -447> 2024-05-30T16:21:39.760+0000 7f7ffa861640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:21:40.185 INFO:tasks.workunit.client.0.smithi006.stderr: -446> 2024-05-30T16:21:39.760+0000 7f7ffb863640 10 monclient(hunting): handle_auth_done global_id 6440 payload 824 2024-05-30T16:21:40.185 INFO:tasks.workunit.client.0.smithi006.stderr: -445> 2024-05-30T16:21:39.760+0000 7f7ffb863640 10 monclient: _finish_hunting 0 2024-05-30T16:21:40.185 INFO:tasks.workunit.client.0.smithi006.stderr: -444> 2024-05-30T16:21:39.760+0000 7f7ffb863640 1 monclient: found mon.noname-c 2024-05-30T16:21:40.185 INFO:tasks.workunit.client.0.smithi006.stderr: -443> 2024-05-30T16:21:39.760+0000 7f7ffb863640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.6:3301/0 2024-05-30T16:21:40.185 INFO:tasks.workunit.client.0.smithi006.stderr: -442> 2024-05-30T16:21:39.760+0000 7f7ee7e07640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:21:40.185 INFO:tasks.workunit.client.0.smithi006.stderr: -441> 2024-05-30T16:21:39.760+0000 7f7ee7e07640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-30T16:21:40.185 INFO:tasks.workunit.client.0.smithi006.stderr: -440> 2024-05-30T16:21:39.760+0000 7f7ee7e07640 10 monclient: dump: 2024-05-30T16:21:40.185 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:21:40.185 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:21:40.185 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.185 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.185 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:21:40.185 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:21:40.186 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:21:40.186 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:21:40.186 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:21:40.186 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.186 INFO:tasks.workunit.client.0.smithi006.stderr: -439> 2024-05-30T16:21:39.760+0000 7f7ee7e07640 10 monclient: _finish_auth 0 2024-05-30T16:21:40.186 INFO:tasks.workunit.client.0.smithi006.stderr: -438> 2024-05-30T16:21:39.760+0000 7f7ee7e07640 10 monclient: _check_auth_tickets 2024-05-30T16:21:40.186 INFO:tasks.workunit.client.0.smithi006.stderr: -437> 2024-05-30T16:21:39.760+0000 7f7ee7e07640 10 monclient: handle_config config(0 keys) 2024-05-30T16:21:40.186 INFO:tasks.workunit.client.0.smithi006.stderr: -436> 2024-05-30T16:21:39.760+0000 7f7ee7e07640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:21:40.186 INFO:tasks.workunit.client.0.smithi006.stderr: -435> 2024-05-30T16:21:39.760+0000 7f7ee7e07640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-30T16:21:40.186 INFO:tasks.workunit.client.0.smithi006.stderr: -434> 2024-05-30T16:21:39.760+0000 7f7ee7e07640 10 monclient: dump: 2024-05-30T16:21:40.186 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:21:40.186 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:21:40.187 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.187 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.187 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:21:40.187 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:21:40.187 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:21:40.187 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:21:40.187 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:21:40.187 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.187 INFO:tasks.workunit.client.0.smithi006.stderr: -433> 2024-05-30T16:21:39.760+0000 7f7ee960a640 4 set_mon_vals no callback set 2024-05-30T16:21:40.187 INFO:tasks.workunit.client.0.smithi006.stderr: -432> 2024-05-30T16:21:39.760+0000 7f7ffd551980 5 monclient: authenticate success, global_id 6440 2024-05-30T16:21:40.187 INFO:tasks.workunit.client.0.smithi006.stderr: -431> 2024-05-30T16:21:39.760+0000 7f7ffd551980 10 monclient: _renew_subs 2024-05-30T16:21:40.187 INFO:tasks.workunit.client.0.smithi006.stderr: -430> 2024-05-30T16:21:39.760+0000 7f7ffd551980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.6:3301/0 2024-05-30T16:21:40.187 INFO:tasks.workunit.client.0.smithi006.stderr: -429> 2024-05-30T16:21:39.760+0000 7f7ffd551980 10 monclient: _renew_subs 2024-05-30T16:21:40.187 INFO:tasks.workunit.client.0.smithi006.stderr: -428> 2024-05-30T16:21:39.760+0000 7f7ffd551980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.6:3301/0 2024-05-30T16:21:40.188 INFO:tasks.workunit.client.0.smithi006.stderr: -427> 2024-05-30T16:21:39.760+0000 7f7ffd551980 1 librados: init done 2024-05-30T16:21:40.188 INFO:tasks.workunit.client.0.smithi006.stderr: -426> 2024-05-30T16:21:39.764+0000 7f7ee7e07640 4 mgrc handle_mgr_map Got map version 5 2024-05-30T16:21:40.188 INFO:tasks.workunit.client.0.smithi006.stderr: -425> 2024-05-30T16:21:39.764+0000 7f7ee7e07640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:21:40.188 INFO:tasks.workunit.client.0.smithi006.stderr: -424> 2024-05-30T16:21:39.764+0000 7f7ee7e07640 4 mgrc reconnect Starting new session with [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:21:40.188 INFO:tasks.workunit.client.0.smithi006.stderr: -423> 2024-05-30T16:21:39.764+0000 7f7ffb062640 10 monclient: get_auth_request con 0x7f7ee006dde0 auth_method 0 2024-05-30T16:21:40.188 INFO:tasks.workunit.client.0.smithi006.stderr: -422> 2024-05-30T16:21:39.764+0000 7f7ffa861640 10 monclient: get_auth_request con 0x5607c4982b40 auth_method 0 2024-05-30T16:21:40.188 INFO:tasks.workunit.client.0.smithi006.stderr: -421> 2024-05-30T16:21:39.764+0000 7f7ffb062640 10 monclient: get_auth_request con 0x5607c4b3fca0 auth_method 0 2024-05-30T16:21:40.188 INFO:tasks.workunit.client.0.smithi006.stderr: -420> 2024-05-30T16:21:39.768+0000 7f7ffa861640 10 monclient: get_auth_request con 0x5607c4b43410 auth_method 0 2024-05-30T16:21:40.188 INFO:tasks.workunit.client.0.smithi006.stderr: -419> 2024-05-30T16:21:39.768+0000 7f7ffd551980 10 monclient: build_initial_monmap 2024-05-30T16:21:40.188 INFO:tasks.workunit.client.0.smithi006.stderr: -418> 2024-05-30T16:21:39.768+0000 7f7ffd551980 1 build_initial for_mkfs: 0 2024-05-30T16:21:40.188 INFO:tasks.workunit.client.0.smithi006.stderr: -417> 2024-05-30T16:21:39.768+0000 7f7ffd551980 10 monclient: monmap: 2024-05-30T16:21:40.188 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 0 2024-05-30T16:21:40.188 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-30T16:21:40.189 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 0.000000 2024-05-30T16:21:40.189 INFO:tasks.workunit.client.0.smithi006.stderr:created 0.000000 2024-05-30T16:21:40.189 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 0 (unknown) 2024-05-30T16:21:40.189 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:21:40.189 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.noname-a 2024-05-30T16:21:40.189 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-b 2024-05-30T16:21:40.189 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.noname-c 2024-05-30T16:21:40.189 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.189 INFO:tasks.workunit.client.0.smithi006.stderr: -416> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b488b0) adding auth protocol: cephx 2024-05-30T16:21:40.189 INFO:tasks.workunit.client.0.smithi006.stderr: -415> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b488b0) adding auth protocol: cephx 2024-05-30T16:21:40.189 INFO:tasks.workunit.client.0.smithi006.stderr: -414> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b488b0) adding auth protocol: cephx 2024-05-30T16:21:40.189 INFO:tasks.workunit.client.0.smithi006.stderr: -413> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b488b0) adding con mode: secure 2024-05-30T16:21:40.189 INFO:tasks.workunit.client.0.smithi006.stderr: -412> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b488b0) adding con mode: crc 2024-05-30T16:21:40.190 INFO:tasks.workunit.client.0.smithi006.stderr: -411> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b488b0) adding con mode: secure 2024-05-30T16:21:40.190 INFO:tasks.workunit.client.0.smithi006.stderr: -410> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b488b0) adding con mode: crc 2024-05-30T16:21:40.190 INFO:tasks.workunit.client.0.smithi006.stderr: -409> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b488b0) adding con mode: secure 2024-05-30T16:21:40.190 INFO:tasks.workunit.client.0.smithi006.stderr: -408> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b488b0) adding con mode: crc 2024-05-30T16:21:40.190 INFO:tasks.workunit.client.0.smithi006.stderr: -407> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b488b0) adding con mode: crc 2024-05-30T16:21:40.190 INFO:tasks.workunit.client.0.smithi006.stderr: -406> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b488b0) adding con mode: secure 2024-05-30T16:21:40.190 INFO:tasks.workunit.client.0.smithi006.stderr: -405> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b488b0) adding con mode: crc 2024-05-30T16:21:40.190 INFO:tasks.workunit.client.0.smithi006.stderr: -404> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b488b0) adding con mode: secure 2024-05-30T16:21:40.190 INFO:tasks.workunit.client.0.smithi006.stderr: -403> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b488b0) adding con mode: crc 2024-05-30T16:21:40.190 INFO:tasks.workunit.client.0.smithi006.stderr: -402> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b488b0) adding con mode: secure 2024-05-30T16:21:40.190 INFO:tasks.workunit.client.0.smithi006.stderr: -401> 2024-05-30T16:21:39.768+0000 7f7ffd551980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:21:40.190 INFO:tasks.workunit.client.0.smithi006.stderr: -400> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command objecter_requests cmddesc objecter_requests hook 0x5607c4b4bda0 EEXIST 2024-05-30T16:21:40.191 INFO:tasks.workunit.client.0.smithi006.stderr: -399> 2024-05-30T16:21:39.768+0000 7f7ffd551980 10 monclient: init 2024-05-30T16:21:40.191 INFO:tasks.workunit.client.0.smithi006.stderr: -398> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b477a8) adding auth protocol: cephx 2024-05-30T16:21:40.191 INFO:tasks.workunit.client.0.smithi006.stderr: -397> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b477a8) adding auth protocol: cephx 2024-05-30T16:21:40.191 INFO:tasks.workunit.client.0.smithi006.stderr: -396> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b477a8) adding auth protocol: cephx 2024-05-30T16:21:40.191 INFO:tasks.workunit.client.0.smithi006.stderr: -395> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b477a8) adding con mode: secure 2024-05-30T16:21:40.191 INFO:tasks.workunit.client.0.smithi006.stderr: -394> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b477a8) adding con mode: crc 2024-05-30T16:21:40.191 INFO:tasks.workunit.client.0.smithi006.stderr: -393> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b477a8) adding con mode: secure 2024-05-30T16:21:40.191 INFO:tasks.workunit.client.0.smithi006.stderr: -392> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b477a8) adding con mode: crc 2024-05-30T16:21:40.191 INFO:tasks.workunit.client.0.smithi006.stderr: -391> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b477a8) adding con mode: secure 2024-05-30T16:21:40.191 INFO:tasks.workunit.client.0.smithi006.stderr: -390> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b477a8) adding con mode: crc 2024-05-30T16:21:40.191 INFO:tasks.workunit.client.0.smithi006.stderr: -389> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b477a8) adding con mode: crc 2024-05-30T16:21:40.191 INFO:tasks.workunit.client.0.smithi006.stderr: -388> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b477a8) adding con mode: secure 2024-05-30T16:21:40.192 INFO:tasks.workunit.client.0.smithi006.stderr: -387> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b477a8) adding con mode: crc 2024-05-30T16:21:40.192 INFO:tasks.workunit.client.0.smithi006.stderr: -386> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b477a8) adding con mode: secure 2024-05-30T16:21:40.192 INFO:tasks.workunit.client.0.smithi006.stderr: -385> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b477a8) adding con mode: crc 2024-05-30T16:21:40.192 INFO:tasks.workunit.client.0.smithi006.stderr: -384> 2024-05-30T16:21:39.768+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b477a8) adding con mode: secure 2024-05-30T16:21:40.192 INFO:tasks.workunit.client.0.smithi006.stderr: -383> 2024-05-30T16:21:39.768+0000 7f7ffd551980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:21:40.192 INFO:tasks.workunit.client.0.smithi006.stderr: -382> 2024-05-30T16:21:39.772+0000 7f7ffd551980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:21:40.192 INFO:tasks.workunit.client.0.smithi006.stderr: -381> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command rotate-key cmddesc rotate-key hook 0x5607c4b478f0 EEXIST 2024-05-30T16:21:40.192 INFO:tasks.workunit.client.0.smithi006.stderr: -380> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient: _reopen_session rank -1 2024-05-30T16:21:40.192 INFO:tasks.workunit.client.0.smithi006.stderr: -379> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient: _add_conns ranks=[0,1,2] 2024-05-30T16:21:40.192 INFO:tasks.workunit.client.0.smithi006.stderr: -378> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient(hunting): picked mon.noname-a con 0x5607c4b52330 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:21:40.192 INFO:tasks.workunit.client.0.smithi006.stderr: -377> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient(hunting): picked mon.noname-b con 0x5607c4b52c60 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:21:40.192 INFO:tasks.workunit.client.0.smithi006.stderr: -376> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient(hunting): picked mon.noname-c con 0x5607c4b52710 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:21:40.192 INFO:tasks.workunit.client.0.smithi006.stderr: -375> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient(hunting): start opening mon connection 2024-05-30T16:21:40.192 INFO:tasks.workunit.client.0.smithi006.stderr: -374> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient(hunting): start opening mon connection 2024-05-30T16:21:40.193 INFO:tasks.workunit.client.0.smithi006.stderr: -373> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient(hunting): start opening mon connection 2024-05-30T16:21:40.193 INFO:tasks.workunit.client.0.smithi006.stderr: -372> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient(hunting): _renew_subs 2024-05-30T16:21:40.193 INFO:tasks.workunit.client.0.smithi006.stderr: -371> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient(hunting): authenticate will time out at 2639.704304s 2024-05-30T16:21:40.193 INFO:tasks.workunit.client.0.smithi006.stderr: -370> 2024-05-30T16:21:39.772+0000 7f7ffb062640 10 monclient(hunting): get_auth_request con 0x5607c4b52330 auth_method 0 2024-05-30T16:21:40.193 INFO:tasks.workunit.client.0.smithi006.stderr: -369> 2024-05-30T16:21:39.772+0000 7f7ffb062640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:21:40.193 INFO:tasks.workunit.client.0.smithi006.stderr: -368> 2024-05-30T16:21:39.772+0000 7f7ffb062640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:21:40.193 INFO:tasks.workunit.client.0.smithi006.stderr: -367> 2024-05-30T16:21:39.772+0000 7f7ffb062640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:21:40.193 INFO:tasks.workunit.client.0.smithi006.stderr: -366> 2024-05-30T16:21:39.772+0000 7f7ffb863640 10 monclient(hunting): get_auth_request con 0x5607c4b52710 auth_method 0 2024-05-30T16:21:40.193 INFO:tasks.workunit.client.0.smithi006.stderr: -365> 2024-05-30T16:21:39.772+0000 7f7ffb863640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:21:40.193 INFO:tasks.workunit.client.0.smithi006.stderr: -364> 2024-05-30T16:21:39.772+0000 7f7ffb863640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:21:40.193 INFO:tasks.workunit.client.0.smithi006.stderr: -363> 2024-05-30T16:21:39.772+0000 7f7ffb863640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:21:40.193 INFO:tasks.workunit.client.0.smithi006.stderr: -362> 2024-05-30T16:21:39.772+0000 7f7ffb062640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:21:40.193 INFO:tasks.workunit.client.0.smithi006.stderr: -361> 2024-05-30T16:21:39.772+0000 7f7ffb062640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:21:40.194 INFO:tasks.workunit.client.0.smithi006.stderr: -360> 2024-05-30T16:21:39.772+0000 7f7ffb062640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:21:40.194 INFO:tasks.workunit.client.0.smithi006.stderr: -359> 2024-05-30T16:21:39.772+0000 7f7ffb863640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:21:40.194 INFO:tasks.workunit.client.0.smithi006.stderr: -358> 2024-05-30T16:21:39.772+0000 7f7ffb863640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:21:40.194 INFO:tasks.workunit.client.0.smithi006.stderr: -357> 2024-05-30T16:21:39.772+0000 7f7ffb863640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:21:40.194 INFO:tasks.workunit.client.0.smithi006.stderr: -356> 2024-05-30T16:21:39.772+0000 7f7ffa861640 10 monclient(hunting): get_auth_request con 0x5607c4b52c60 auth_method 0 2024-05-30T16:21:40.194 INFO:tasks.workunit.client.0.smithi006.stderr: -355> 2024-05-30T16:21:39.772+0000 7f7ffa861640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:21:40.194 INFO:tasks.workunit.client.0.smithi006.stderr: -354> 2024-05-30T16:21:39.772+0000 7f7ffa861640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:21:40.194 INFO:tasks.workunit.client.0.smithi006.stderr: -353> 2024-05-30T16:21:39.772+0000 7f7ffa861640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:21:40.194 INFO:tasks.workunit.client.0.smithi006.stderr: -352> 2024-05-30T16:21:39.772+0000 7f7ffb062640 10 monclient(hunting): handle_auth_done global_id 6762 payload 824 2024-05-30T16:21:40.194 INFO:tasks.workunit.client.0.smithi006.stderr: -351> 2024-05-30T16:21:39.772+0000 7f7ffb062640 10 monclient: _finish_hunting 0 2024-05-30T16:21:40.194 INFO:tasks.workunit.client.0.smithi006.stderr: -350> 2024-05-30T16:21:39.772+0000 7f7ffb062640 1 monclient: found mon.noname-a 2024-05-30T16:21:40.194 INFO:tasks.workunit.client.0.smithi006.stderr: -349> 2024-05-30T16:21:39.772+0000 7f7ffb062640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.6:3300/0 2024-05-30T16:21:40.195 INFO:tasks.workunit.client.0.smithi006.stderr: -348> 2024-05-30T16:21:39.772+0000 7f7ee5602640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:21:40.195 INFO:tasks.workunit.client.0.smithi006.stderr: -347> 2024-05-30T16:21:39.772+0000 7f7ee5602640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-30T16:21:40.195 INFO:tasks.workunit.client.0.smithi006.stderr: -346> 2024-05-30T16:21:39.772+0000 7f7ee5602640 10 monclient: dump: 2024-05-30T16:21:40.195 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:21:40.195 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:21:40.195 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.195 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.195 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:21:40.195 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:21:40.195 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:21:40.195 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:21:40.195 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:21:40.195 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.196 INFO:tasks.workunit.client.0.smithi006.stderr: -345> 2024-05-30T16:21:39.772+0000 7f7ee5602640 10 monclient: _finish_auth 0 2024-05-30T16:21:40.196 INFO:tasks.workunit.client.0.smithi006.stderr: -344> 2024-05-30T16:21:39.772+0000 7f7ee5602640 10 monclient: _check_auth_tickets 2024-05-30T16:21:40.196 INFO:tasks.workunit.client.0.smithi006.stderr: -343> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 monclient: authenticate success, global_id 6762 2024-05-30T16:21:40.196 INFO:tasks.workunit.client.0.smithi006.stderr: -342> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient: _renew_subs 2024-05-30T16:21:40.196 INFO:tasks.workunit.client.0.smithi006.stderr: -341> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:21:40.196 INFO:tasks.workunit.client.0.smithi006.stderr: -340> 2024-05-30T16:21:39.772+0000 7f7ee5602640 10 monclient: handle_config config(0 keys) 2024-05-30T16:21:40.196 INFO:tasks.workunit.client.0.smithi006.stderr: -339> 2024-05-30T16:21:39.772+0000 7f7ee5602640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:21:40.196 INFO:tasks.workunit.client.0.smithi006.stderr: -338> 2024-05-30T16:21:39.772+0000 7f7ee5602640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-30T16:21:40.196 INFO:tasks.workunit.client.0.smithi006.stderr: -337> 2024-05-30T16:21:39.772+0000 7f7ee5602640 10 monclient: dump: 2024-05-30T16:21:40.196 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:21:40.196 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:21:40.196 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.196 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.196 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:21:40.197 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:21:40.197 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:21:40.197 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:21:40.197 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:21:40.197 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.197 INFO:tasks.workunit.client.0.smithi006.stderr: -336> 2024-05-30T16:21:39.772+0000 7f7ffc064640 4 set_mon_vals no callback set 2024-05-30T16:21:40.197 INFO:tasks.workunit.client.0.smithi006.stderr: -335> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient: _renew_subs 2024-05-30T16:21:40.197 INFO:tasks.workunit.client.0.smithi006.stderr: -334> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:21:40.197 INFO:tasks.workunit.client.0.smithi006.stderr: -333> 2024-05-30T16:21:39.772+0000 7f7ee5602640 4 mgrc handle_mgr_map Got map version 5 2024-05-30T16:21:40.197 INFO:tasks.workunit.client.0.smithi006.stderr: -332> 2024-05-30T16:21:39.772+0000 7f7ee5602640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:21:40.197 INFO:tasks.workunit.client.0.smithi006.stderr: -331> 2024-05-30T16:21:39.772+0000 7f7ee5602640 4 mgrc reconnect Starting new session with [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:21:40.197 INFO:tasks.workunit.client.0.smithi006.stderr: -330> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient: get_monmap_and_config 2024-05-30T16:21:40.197 INFO:tasks.workunit.client.0.smithi006.stderr: -329> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient: build_initial_monmap 2024-05-30T16:21:40.198 INFO:tasks.workunit.client.0.smithi006.stderr: -328> 2024-05-30T16:21:39.772+0000 7f7ffd551980 1 build_initial for_mkfs: 0 2024-05-30T16:21:40.198 INFO:tasks.workunit.client.0.smithi006.stderr: -327> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient: monmap: 2024-05-30T16:21:40.198 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 0 2024-05-30T16:21:40.198 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-30T16:21:40.198 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 0.000000 2024-05-30T16:21:40.198 INFO:tasks.workunit.client.0.smithi006.stderr:created 0.000000 2024-05-30T16:21:40.198 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 0 (unknown) 2024-05-30T16:21:40.198 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:21:40.198 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.noname-a 2024-05-30T16:21:40.198 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-b 2024-05-30T16:21:40.198 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.noname-c 2024-05-30T16:21:40.198 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.198 INFO:tasks.workunit.client.0.smithi006.stderr: -326> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b8fe10) adding auth protocol: cephx 2024-05-30T16:21:40.199 INFO:tasks.workunit.client.0.smithi006.stderr: -325> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b8fe10) adding auth protocol: cephx 2024-05-30T16:21:40.199 INFO:tasks.workunit.client.0.smithi006.stderr: -324> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b8fe10) adding auth protocol: cephx 2024-05-30T16:21:40.199 INFO:tasks.workunit.client.0.smithi006.stderr: -323> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b8fe10) adding con mode: secure 2024-05-30T16:21:40.199 INFO:tasks.workunit.client.0.smithi006.stderr: -322> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b8fe10) adding con mode: crc 2024-05-30T16:21:40.199 INFO:tasks.workunit.client.0.smithi006.stderr: -321> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b8fe10) adding con mode: secure 2024-05-30T16:21:40.199 INFO:tasks.workunit.client.0.smithi006.stderr: -320> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b8fe10) adding con mode: crc 2024-05-30T16:21:40.199 INFO:tasks.workunit.client.0.smithi006.stderr: -319> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b8fe10) adding con mode: secure 2024-05-30T16:21:40.199 INFO:tasks.workunit.client.0.smithi006.stderr: -318> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b8fe10) adding con mode: crc 2024-05-30T16:21:40.199 INFO:tasks.workunit.client.0.smithi006.stderr: -317> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b8fe10) adding con mode: crc 2024-05-30T16:21:40.199 INFO:tasks.workunit.client.0.smithi006.stderr: -316> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b8fe10) adding con mode: secure 2024-05-30T16:21:40.199 INFO:tasks.workunit.client.0.smithi006.stderr: -315> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b8fe10) adding con mode: crc 2024-05-30T16:21:40.199 INFO:tasks.workunit.client.0.smithi006.stderr: -314> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b8fe10) adding con mode: secure 2024-05-30T16:21:40.199 INFO:tasks.workunit.client.0.smithi006.stderr: -313> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b8fe10) adding con mode: crc 2024-05-30T16:21:40.200 INFO:tasks.workunit.client.0.smithi006.stderr: -312> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b8fe10) adding con mode: secure 2024-05-30T16:21:40.200 INFO:tasks.workunit.client.0.smithi006.stderr: -311> 2024-05-30T16:21:39.772+0000 7f7ffa861640 10 monclient: get_auth_request con 0x7f7ed406dcf0 auth_method 0 2024-05-30T16:21:40.200 INFO:tasks.workunit.client.0.smithi006.stderr: -310> 2024-05-30T16:21:39.772+0000 7f7ffd551980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:21:40.200 INFO:tasks.workunit.client.0.smithi006.stderr: -309> 2024-05-30T16:21:39.772+0000 7f7ffd551980 10 monclient: init 2024-05-30T16:21:40.200 INFO:tasks.workunit.client.0.smithi006.stderr: -308> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc37e0) adding auth protocol: cephx 2024-05-30T16:21:40.200 INFO:tasks.workunit.client.0.smithi006.stderr: -307> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc37e0) adding auth protocol: cephx 2024-05-30T16:21:40.200 INFO:tasks.workunit.client.0.smithi006.stderr: -306> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc37e0) adding auth protocol: cephx 2024-05-30T16:21:40.200 INFO:tasks.workunit.client.0.smithi006.stderr: -305> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc37e0) adding con mode: secure 2024-05-30T16:21:40.200 INFO:tasks.workunit.client.0.smithi006.stderr: -304> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc37e0) adding con mode: crc 2024-05-30T16:21:40.200 INFO:tasks.workunit.client.0.smithi006.stderr: -303> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc37e0) adding con mode: secure 2024-05-30T16:21:40.200 INFO:tasks.workunit.client.0.smithi006.stderr: -302> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc37e0) adding con mode: crc 2024-05-30T16:21:40.200 INFO:tasks.workunit.client.0.smithi006.stderr: -301> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc37e0) adding con mode: secure 2024-05-30T16:21:40.200 INFO:tasks.workunit.client.0.smithi006.stderr: -300> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc37e0) adding con mode: crc 2024-05-30T16:21:40.201 INFO:tasks.workunit.client.0.smithi006.stderr: -299> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc37e0) adding con mode: crc 2024-05-30T16:21:40.201 INFO:tasks.workunit.client.0.smithi006.stderr: -298> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc37e0) adding con mode: secure 2024-05-30T16:21:40.201 INFO:tasks.workunit.client.0.smithi006.stderr: -297> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc37e0) adding con mode: crc 2024-05-30T16:21:40.201 INFO:tasks.workunit.client.0.smithi006.stderr: -296> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc37e0) adding con mode: secure 2024-05-30T16:21:40.201 INFO:tasks.workunit.client.0.smithi006.stderr: -295> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc37e0) adding con mode: crc 2024-05-30T16:21:40.201 INFO:tasks.workunit.client.0.smithi006.stderr: -294> 2024-05-30T16:21:39.772+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc37e0) adding con mode: secure 2024-05-30T16:21:40.201 INFO:tasks.workunit.client.0.smithi006.stderr: -293> 2024-05-30T16:21:39.776+0000 7f7ffd551980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:21:40.201 INFO:tasks.workunit.client.0.smithi006.stderr: -292> 2024-05-30T16:21:39.776+0000 7f7ffd551980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:21:40.201 INFO:tasks.workunit.client.0.smithi006.stderr: -291> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command rotate-key cmddesc rotate-key hook 0x7ffe4bdc3928 EEXIST 2024-05-30T16:21:40.201 INFO:tasks.workunit.client.0.smithi006.stderr: -290> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient: _reopen_session rank -1 2024-05-30T16:21:40.201 INFO:tasks.workunit.client.0.smithi006.stderr: -289> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient: _add_conns ranks=[0,2,1] 2024-05-30T16:21:40.201 INFO:tasks.workunit.client.0.smithi006.stderr: -288> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): picked mon.noname-a con 0x5607c4bbe640 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:21:40.201 INFO:tasks.workunit.client.0.smithi006.stderr: -287> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): picked mon.noname-c con 0x5607c4bc9be0 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:21:40.202 INFO:tasks.workunit.client.0.smithi006.stderr: -286> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): picked mon.noname-b con 0x5607c4bb65a0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:21:40.202 INFO:tasks.workunit.client.0.smithi006.stderr: -285> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): start opening mon connection 2024-05-30T16:21:40.202 INFO:tasks.workunit.client.0.smithi006.stderr: -284> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): start opening mon connection 2024-05-30T16:21:40.202 INFO:tasks.workunit.client.0.smithi006.stderr: -283> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): start opening mon connection 2024-05-30T16:21:40.202 INFO:tasks.workunit.client.0.smithi006.stderr: -282> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): _renew_subs 2024-05-30T16:21:40.202 INFO:tasks.workunit.client.0.smithi006.stderr: -281> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): authenticate will time out at 2639.708422s 2024-05-30T16:21:40.202 INFO:tasks.workunit.client.0.smithi006.stderr: -280> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): get_auth_request con 0x5607c4bbe640 auth_method 0 2024-05-30T16:21:40.202 INFO:tasks.workunit.client.0.smithi006.stderr: -279> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:21:40.202 INFO:tasks.workunit.client.0.smithi006.stderr: -278> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:21:40.202 INFO:tasks.workunit.client.0.smithi006.stderr: -277> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:21:40.202 INFO:tasks.workunit.client.0.smithi006.stderr: -276> 2024-05-30T16:21:39.776+0000 7f7ffa861640 10 monclient(hunting): get_auth_request con 0x5607c4bc9be0 auth_method 0 2024-05-30T16:21:40.202 INFO:tasks.workunit.client.0.smithi006.stderr: -275> 2024-05-30T16:21:39.776+0000 7f7ffa861640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:21:40.203 INFO:tasks.workunit.client.0.smithi006.stderr: -274> 2024-05-30T16:21:39.776+0000 7f7ffa861640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:21:40.203 INFO:tasks.workunit.client.0.smithi006.stderr: -273> 2024-05-30T16:21:39.776+0000 7f7ffa861640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:21:40.203 INFO:tasks.workunit.client.0.smithi006.stderr: -272> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:21:40.203 INFO:tasks.workunit.client.0.smithi006.stderr: -271> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:21:40.203 INFO:tasks.workunit.client.0.smithi006.stderr: -270> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:21:40.203 INFO:tasks.workunit.client.0.smithi006.stderr: -269> 2024-05-30T16:21:39.776+0000 7f7ffa861640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:21:40.203 INFO:tasks.workunit.client.0.smithi006.stderr: -268> 2024-05-30T16:21:39.776+0000 7f7ffa861640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:21:40.203 INFO:tasks.workunit.client.0.smithi006.stderr: -267> 2024-05-30T16:21:39.776+0000 7f7ffa861640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:21:40.203 INFO:tasks.workunit.client.0.smithi006.stderr: -266> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): handle_auth_done global_id 6765 payload 274 2024-05-30T16:21:40.203 INFO:tasks.workunit.client.0.smithi006.stderr: -265> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient: _finish_hunting 0 2024-05-30T16:21:40.203 INFO:tasks.workunit.client.0.smithi006.stderr: -264> 2024-05-30T16:21:39.776+0000 7f7ffb062640 1 monclient: found mon.noname-a 2024-05-30T16:21:40.203 INFO:tasks.workunit.client.0.smithi006.stderr: -263> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.6:3300/0 2024-05-30T16:21:40.203 INFO:tasks.workunit.client.0.smithi006.stderr: -262> 2024-05-30T16:21:39.776+0000 7f7ffb863640 10 monclient: get_auth_request con 0x5607c4bb65a0 auth_method 0 2024-05-30T16:21:40.204 INFO:tasks.workunit.client.0.smithi006.stderr: -261> 2024-05-30T16:21:39.776+0000 7f7edeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:21:40.204 INFO:tasks.workunit.client.0.smithi006.stderr: -260> 2024-05-30T16:21:39.776+0000 7f7edeffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-30T16:21:40.204 INFO:tasks.workunit.client.0.smithi006.stderr: -259> 2024-05-30T16:21:39.776+0000 7f7edeffd640 10 monclient: dump: 2024-05-30T16:21:40.204 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:21:40.204 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:21:40.204 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.204 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.204 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:21:40.204 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:21:40.204 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:21:40.204 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:21:40.204 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:21:40.205 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.205 INFO:tasks.workunit.client.0.smithi006.stderr: -258> 2024-05-30T16:21:39.776+0000 7f7edeffd640 10 monclient: _finish_auth 0 2024-05-30T16:21:40.205 INFO:tasks.workunit.client.0.smithi006.stderr: -257> 2024-05-30T16:21:39.776+0000 7f7edeffd640 10 monclient: _check_auth_tickets 2024-05-30T16:21:40.205 INFO:tasks.workunit.client.0.smithi006.stderr: -256> 2024-05-30T16:21:39.776+0000 7f7edeffd640 10 monclient: handle_config config(0 keys) 2024-05-30T16:21:40.205 INFO:tasks.workunit.client.0.smithi006.stderr: -255> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 monclient: authenticate success, global_id 6765 2024-05-30T16:21:40.205 INFO:tasks.workunit.client.0.smithi006.stderr: -254> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient: get_monmap_and_config success 2024-05-30T16:21:40.205 INFO:tasks.workunit.client.0.smithi006.stderr: -253> 2024-05-30T16:21:39.776+0000 7f7edeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:21:40.205 INFO:tasks.workunit.client.0.smithi006.stderr: -252> 2024-05-30T16:21:39.776+0000 7f7ffd551980 4 set_mon_vals no callback set 2024-05-30T16:21:40.205 INFO:tasks.workunit.client.0.smithi006.stderr: -251> 2024-05-30T16:21:39.776+0000 7f7edeffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-30T16:21:40.205 INFO:tasks.workunit.client.0.smithi006.stderr: -250> 2024-05-30T16:21:39.776+0000 7f7edeffd640 10 monclient: dump: 2024-05-30T16:21:40.205 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:21:40.206 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:21:40.206 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.206 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.206 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:21:40.206 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:21:40.206 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:21:40.206 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:21:40.206 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:21:40.206 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.206 INFO:tasks.workunit.client.0.smithi006.stderr: -249> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient: shutdown 2024-05-30T16:21:40.206 INFO:tasks.workunit.client.0.smithi006.stderr: -248> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient: build_initial_monmap 2024-05-30T16:21:40.206 INFO:tasks.workunit.client.0.smithi006.stderr: -247> 2024-05-30T16:21:39.776+0000 7f7ffd551980 1 build_initial for_mkfs: 0 2024-05-30T16:21:40.206 INFO:tasks.workunit.client.0.smithi006.stderr: -246> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient: monmap: 2024-05-30T16:21:40.207 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 0 2024-05-30T16:21:40.207 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-30T16:21:40.207 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 0.000000 2024-05-30T16:21:40.207 INFO:tasks.workunit.client.0.smithi006.stderr:created 0.000000 2024-05-30T16:21:40.207 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 0 (unknown) 2024-05-30T16:21:40.207 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:21:40.207 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.noname-a 2024-05-30T16:21:40.207 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-b 2024-05-30T16:21:40.207 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.noname-c 2024-05-30T16:21:40.207 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.208 INFO:tasks.workunit.client.0.smithi006.stderr: -245> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b93f20) adding auth protocol: cephx 2024-05-30T16:21:40.208 INFO:tasks.workunit.client.0.smithi006.stderr: -244> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b93f20) adding auth protocol: cephx 2024-05-30T16:21:40.208 INFO:tasks.workunit.client.0.smithi006.stderr: -243> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b93f20) adding auth protocol: cephx 2024-05-30T16:21:40.208 INFO:tasks.workunit.client.0.smithi006.stderr: -242> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b93f20) adding con mode: secure 2024-05-30T16:21:40.208 INFO:tasks.workunit.client.0.smithi006.stderr: -241> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b93f20) adding con mode: crc 2024-05-30T16:21:40.208 INFO:tasks.workunit.client.0.smithi006.stderr: -240> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b93f20) adding con mode: secure 2024-05-30T16:21:40.208 INFO:tasks.workunit.client.0.smithi006.stderr: -239> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b93f20) adding con mode: crc 2024-05-30T16:21:40.208 INFO:tasks.workunit.client.0.smithi006.stderr: -238> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b93f20) adding con mode: secure 2024-05-30T16:21:40.208 INFO:tasks.workunit.client.0.smithi006.stderr: -237> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b93f20) adding con mode: crc 2024-05-30T16:21:40.208 INFO:tasks.workunit.client.0.smithi006.stderr: -236> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b93f20) adding con mode: crc 2024-05-30T16:21:40.208 INFO:tasks.workunit.client.0.smithi006.stderr: -235> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b93f20) adding con mode: secure 2024-05-30T16:21:40.208 INFO:tasks.workunit.client.0.smithi006.stderr: -234> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b93f20) adding con mode: crc 2024-05-30T16:21:40.208 INFO:tasks.workunit.client.0.smithi006.stderr: -233> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b93f20) adding con mode: secure 2024-05-30T16:21:40.208 INFO:tasks.workunit.client.0.smithi006.stderr: -232> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b93f20) adding con mode: crc 2024-05-30T16:21:40.209 INFO:tasks.workunit.client.0.smithi006.stderr: -231> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b93f20) adding con mode: secure 2024-05-30T16:21:40.209 INFO:tasks.workunit.client.0.smithi006.stderr: -230> 2024-05-30T16:21:39.776+0000 7f7ffd551980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:21:40.209 INFO:tasks.workunit.client.0.smithi006.stderr: -229> 2024-05-30T16:21:39.776+0000 7f7ffd551980 1 librados: starting msgr at 2024-05-30T16:21:40.209 INFO:tasks.workunit.client.0.smithi006.stderr: -228> 2024-05-30T16:21:39.776+0000 7f7ffd551980 1 librados: starting objecter 2024-05-30T16:21:40.209 INFO:tasks.workunit.client.0.smithi006.stderr: -227> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command objecter_requests cmddesc objecter_requests hook 0x5607c4b93db0 EEXIST 2024-05-30T16:21:40.209 INFO:tasks.workunit.client.0.smithi006.stderr: -226> 2024-05-30T16:21:39.776+0000 7f7ffd551980 1 librados: setting wanted keys 2024-05-30T16:21:40.209 INFO:tasks.workunit.client.0.smithi006.stderr: -225> 2024-05-30T16:21:39.776+0000 7f7ffd551980 1 librados: calling monclient init 2024-05-30T16:21:40.209 INFO:tasks.workunit.client.0.smithi006.stderr: -224> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient: init 2024-05-30T16:21:40.209 INFO:tasks.workunit.client.0.smithi006.stderr: -223> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b87d90) adding auth protocol: cephx 2024-05-30T16:21:40.209 INFO:tasks.workunit.client.0.smithi006.stderr: -222> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b87d90) adding auth protocol: cephx 2024-05-30T16:21:40.210 INFO:tasks.workunit.client.0.smithi006.stderr: -221> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b87d90) adding auth protocol: cephx 2024-05-30T16:21:40.210 INFO:tasks.workunit.client.0.smithi006.stderr: -220> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b87d90) adding con mode: secure 2024-05-30T16:21:40.210 INFO:tasks.workunit.client.0.smithi006.stderr: -219> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b87d90) adding con mode: crc 2024-05-30T16:21:40.210 INFO:tasks.workunit.client.0.smithi006.stderr: -218> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b87d90) adding con mode: secure 2024-05-30T16:21:40.210 INFO:tasks.workunit.client.0.smithi006.stderr: -217> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b87d90) adding con mode: crc 2024-05-30T16:21:40.210 INFO:tasks.workunit.client.0.smithi006.stderr: -216> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b87d90) adding con mode: secure 2024-05-30T16:21:40.210 INFO:tasks.workunit.client.0.smithi006.stderr: -215> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b87d90) adding con mode: crc 2024-05-30T16:21:40.210 INFO:tasks.workunit.client.0.smithi006.stderr: -214> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b87d90) adding con mode: crc 2024-05-30T16:21:40.210 INFO:tasks.workunit.client.0.smithi006.stderr: -213> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b87d90) adding con mode: secure 2024-05-30T16:21:40.211 INFO:tasks.workunit.client.0.smithi006.stderr: -212> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b87d90) adding con mode: crc 2024-05-30T16:21:40.211 INFO:tasks.workunit.client.0.smithi006.stderr: -211> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b87d90) adding con mode: secure 2024-05-30T16:21:40.211 INFO:tasks.workunit.client.0.smithi006.stderr: -210> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b87d90) adding con mode: crc 2024-05-30T16:21:40.211 INFO:tasks.workunit.client.0.smithi006.stderr: -209> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 AuthRegistry(0x5607c4b87d90) adding con mode: secure 2024-05-30T16:21:40.211 INFO:tasks.workunit.client.0.smithi006.stderr: -208> 2024-05-30T16:21:39.776+0000 7f7ffd551980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:21:40.211 INFO:tasks.workunit.client.0.smithi006.stderr: -207> 2024-05-30T16:21:39.776+0000 7f7ffd551980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:21:40.211 INFO:tasks.workunit.client.0.smithi006.stderr: -206> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command rotate-key cmddesc rotate-key hook 0x5607c4b87ed8 EEXIST 2024-05-30T16:21:40.211 INFO:tasks.workunit.client.0.smithi006.stderr: -205> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient: _reopen_session rank -1 2024-05-30T16:21:40.211 INFO:tasks.workunit.client.0.smithi006.stderr: -204> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient: _add_conns ranks=[0,1,2] 2024-05-30T16:21:40.211 INFO:tasks.workunit.client.0.smithi006.stderr: -203> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): picked mon.noname-a con 0x5607c4bb65a0 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:21:40.212 INFO:tasks.workunit.client.0.smithi006.stderr: -202> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): picked mon.noname-b con 0x5607c4bbe640 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:21:40.212 INFO:tasks.workunit.client.0.smithi006.stderr: -201> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): picked mon.noname-c con 0x5607c4bc9be0 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:21:40.212 INFO:tasks.workunit.client.0.smithi006.stderr: -200> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): start opening mon connection 2024-05-30T16:21:40.212 INFO:tasks.workunit.client.0.smithi006.stderr: -199> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): start opening mon connection 2024-05-30T16:21:40.212 INFO:tasks.workunit.client.0.smithi006.stderr: -198> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): start opening mon connection 2024-05-30T16:21:40.212 INFO:tasks.workunit.client.0.smithi006.stderr: -197> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): _renew_subs 2024-05-30T16:21:40.212 INFO:tasks.workunit.client.0.smithi006.stderr: -196> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient(hunting): authenticate will time out at 2639.710987s 2024-05-30T16:21:40.212 INFO:tasks.workunit.client.0.smithi006.stderr: -195> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): get_auth_request con 0x5607c4bb65a0 auth_method 0 2024-05-30T16:21:40.212 INFO:tasks.workunit.client.0.smithi006.stderr: -194> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:21:40.212 INFO:tasks.workunit.client.0.smithi006.stderr: -193> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:21:40.213 INFO:tasks.workunit.client.0.smithi006.stderr: -192> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:21:40.213 INFO:tasks.workunit.client.0.smithi006.stderr: -191> 2024-05-30T16:21:39.776+0000 7f7ffb863640 10 monclient(hunting): get_auth_request con 0x5607c4bc9be0 auth_method 0 2024-05-30T16:21:40.213 INFO:tasks.workunit.client.0.smithi006.stderr: -190> 2024-05-30T16:21:39.776+0000 7f7ffb863640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:21:40.213 INFO:tasks.workunit.client.0.smithi006.stderr: -189> 2024-05-30T16:21:39.776+0000 7f7ffb863640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:21:40.213 INFO:tasks.workunit.client.0.smithi006.stderr: -188> 2024-05-30T16:21:39.776+0000 7f7ffb863640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:21:40.213 INFO:tasks.workunit.client.0.smithi006.stderr: -187> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:21:40.213 INFO:tasks.workunit.client.0.smithi006.stderr: -186> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:21:40.213 INFO:tasks.workunit.client.0.smithi006.stderr: -185> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:21:40.213 INFO:tasks.workunit.client.0.smithi006.stderr: -184> 2024-05-30T16:21:39.776+0000 7f7ffb863640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:21:40.213 INFO:tasks.workunit.client.0.smithi006.stderr: -183> 2024-05-30T16:21:39.776+0000 7f7ffb863640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:21:40.213 INFO:tasks.workunit.client.0.smithi006.stderr: -182> 2024-05-30T16:21:39.776+0000 7f7ffb863640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:21:40.214 INFO:tasks.workunit.client.0.smithi006.stderr: -181> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient(hunting): handle_auth_done global_id 6768 payload 824 2024-05-30T16:21:40.214 INFO:tasks.workunit.client.0.smithi006.stderr: -180> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient: _finish_hunting 0 2024-05-30T16:21:40.214 INFO:tasks.workunit.client.0.smithi006.stderr: -179> 2024-05-30T16:21:39.776+0000 7f7ffb062640 1 monclient: found mon.noname-a 2024-05-30T16:21:40.214 INFO:tasks.workunit.client.0.smithi006.stderr: -178> 2024-05-30T16:21:39.776+0000 7f7ffb062640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.6:3300/0 2024-05-30T16:21:40.214 INFO:tasks.workunit.client.0.smithi006.stderr: -177> 2024-05-30T16:21:39.776+0000 7f7edd7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:21:40.214 INFO:tasks.workunit.client.0.smithi006.stderr: -176> 2024-05-30T16:21:39.776+0000 7f7edd7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-30T16:21:40.214 INFO:tasks.workunit.client.0.smithi006.stderr: -175> 2024-05-30T16:21:39.776+0000 7f7edd7fa640 10 monclient: dump: 2024-05-30T16:21:40.214 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:21:40.215 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:21:40.215 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.215 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.215 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:21:40.215 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:21:40.215 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:21:40.215 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:21:40.215 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:21:40.215 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.216 INFO:tasks.workunit.client.0.smithi006.stderr: -174> 2024-05-30T16:21:39.776+0000 7f7edd7fa640 10 monclient: _finish_auth 0 2024-05-30T16:21:40.216 INFO:tasks.workunit.client.0.smithi006.stderr: -173> 2024-05-30T16:21:39.776+0000 7f7edd7fa640 10 monclient: _check_auth_tickets 2024-05-30T16:21:40.216 INFO:tasks.workunit.client.0.smithi006.stderr: -172> 2024-05-30T16:21:39.776+0000 7f7edd7fa640 10 monclient: handle_config config(0 keys) 2024-05-30T16:21:40.216 INFO:tasks.workunit.client.0.smithi006.stderr: -171> 2024-05-30T16:21:39.776+0000 7f7edd7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:21:40.216 INFO:tasks.workunit.client.0.smithi006.stderr: -170> 2024-05-30T16:21:39.776+0000 7f7edd7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-30T16:21:40.216 INFO:tasks.workunit.client.0.smithi006.stderr: -169> 2024-05-30T16:21:39.776+0000 7f7edd7fa640 10 monclient: dump: 2024-05-30T16:21:40.216 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:21:40.216 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:21:40.216 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.216 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:21:40.216 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:21:40.216 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:21:40.217 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:21:40.217 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:21:40.217 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:21:40.217 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.217 INFO:tasks.workunit.client.0.smithi006.stderr: -168> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 monclient: authenticate success, global_id 6768 2024-05-30T16:21:40.217 INFO:tasks.workunit.client.0.smithi006.stderr: -167> 2024-05-30T16:21:39.776+0000 7f7edeffd640 4 set_mon_vals no callback set 2024-05-30T16:21:40.217 INFO:tasks.workunit.client.0.smithi006.stderr: -166> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient: _renew_subs 2024-05-30T16:21:40.217 INFO:tasks.workunit.client.0.smithi006.stderr: -165> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:21:40.217 INFO:tasks.workunit.client.0.smithi006.stderr: -164> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient: _renew_subs 2024-05-30T16:21:40.217 INFO:tasks.workunit.client.0.smithi006.stderr: -163> 2024-05-30T16:21:39.776+0000 7f7ffd551980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:21:40.217 INFO:tasks.workunit.client.0.smithi006.stderr: -162> 2024-05-30T16:21:39.776+0000 7f7ffd551980 1 librados: init done 2024-05-30T16:21:40.217 INFO:tasks.workunit.client.0.smithi006.stderr: -161> 2024-05-30T16:21:39.776+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command cr dump hook 0x5607c4bbea38 2024-05-30T16:21:40.217 INFO:tasks.workunit.client.0.smithi006.stderr: -160> 2024-05-30T16:21:39.780+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.218 INFO:tasks.workunit.client.0.smithi006.stderr: -159> 2024-05-30T16:21:39.780+0000 7f7edd7fa640 4 mgrc handle_mgr_map Got map version 5 2024-05-30T16:21:40.218 INFO:tasks.workunit.client.0.smithi006.stderr: -158> 2024-05-30T16:21:39.780+0000 7f7edd7fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:21:40.218 INFO:tasks.workunit.client.0.smithi006.stderr: -157> 2024-05-30T16:21:39.780+0000 7f7edd7fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:21:40.218 INFO:tasks.workunit.client.0.smithi006.stderr: -156> 2024-05-30T16:21:39.780+0000 7f7ffb863640 10 monclient: get_auth_request con 0x5607c4bdd460 auth_method 0 2024-05-30T16:21:40.218 INFO:tasks.workunit.client.0.smithi006.stderr: -155> 2024-05-30T16:21:39.780+0000 7f7ffa861640 10 monclient: get_auth_request con 0x7f7ec406dde0 auth_method 0 2024-05-30T16:21:40.218 INFO:tasks.workunit.client.0.smithi006.stderr: -154> 2024-05-30T16:21:39.780+0000 7f7ffd551980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:40.218 INFO:tasks.workunit.client.0.smithi006.stderr: -153> 2024-05-30T16:21:39.780+0000 7f7ffd551980 20 realm 2024-05-30T16:21:40.218 INFO:tasks.workunit.client.0.smithi006.stderr: -152> 2024-05-30T16:21:39.780+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.219 INFO:tasks.workunit.client.0.smithi006.stderr: -151> 2024-05-30T16:21:39.780+0000 7f7ffd551980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:40.219 INFO:tasks.workunit.client.0.smithi006.stderr: -150> 2024-05-30T16:21:39.780+0000 7f7ffd551980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:21:40.219 INFO:tasks.workunit.client.0.smithi006.stderr: -149> 2024-05-30T16:21:39.780+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.219 INFO:tasks.workunit.client.0.smithi006.stderr: -148> 2024-05-30T16:21:39.780+0000 7f7ffd551980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:40.219 INFO:tasks.workunit.client.0.smithi006.stderr: -147> 2024-05-30T16:21:39.780+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.219 INFO:tasks.workunit.client.0.smithi006.stderr: -146> 2024-05-30T16:21:39.784+0000 7f7ffb062640 10 monclient: get_auth_request con 0x5607c4be17f0 auth_method 0 2024-05-30T16:21:40.219 INFO:tasks.workunit.client.0.smithi006.stderr: -145> 2024-05-30T16:21:39.784+0000 7f7ffd551980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:21:40.219 INFO:tasks.workunit.client.0.smithi006.stderr: -144> 2024-05-30T16:21:39.784+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.219 INFO:tasks.workunit.client.0.smithi006.stderr: -143> 2024-05-30T16:21:39.784+0000 7f7ffd551980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:21:40.219 INFO:tasks.workunit.client.0.smithi006.stderr: -142> 2024-05-30T16:21:39.784+0000 7f7ffd551980 20 searching for the correct realm 2024-05-30T16:21:40.219 INFO:tasks.workunit.client.0.smithi006.stderr: -141> 2024-05-30T16:21:39.784+0000 7f7ffa861640 10 monclient: get_auth_request con 0x5607c4be69b0 auth_method 0 2024-05-30T16:21:40.219 INFO:tasks.workunit.client.0.smithi006.stderr: -140> 2024-05-30T16:21:39.784+0000 7f7ffb863640 10 monclient: get_auth_request con 0x7f7ff0024040 auth_method 0 2024-05-30T16:21:40.219 INFO:tasks.workunit.client.0.smithi006.stderr: -139> 2024-05-30T16:21:39.788+0000 7f7ffb062640 10 monclient: get_auth_request con 0x7f7ff40bd070 auth_method 0 2024-05-30T16:21:40.220 INFO:tasks.workunit.client.0.smithi006.stderr: -138> 2024-05-30T16:21:39.788+0000 7f7ffa861640 10 monclient: get_auth_request con 0x7f7fec026040 auth_method 0 2024-05-30T16:21:40.220 INFO:tasks.workunit.client.0.smithi006.stderr: -137> 2024-05-30T16:21:39.792+0000 7f7ffb863640 10 monclient: get_auth_request con 0x7f7ff004c040 auth_method 0 2024-05-30T16:21:40.220 INFO:tasks.workunit.client.0.smithi006.stderr: -136> 2024-05-30T16:21:39.792+0000 7f7ffb863640 0 -- 172.21.15.6:0/308574472 >> [v2:172.21.15.204:6800/3687190393,v1:172.21.15.204:6801/3687190393] conn(0x7f7ff004c040 msgr2=0x7f7ff001a410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:21:40.220 INFO:tasks.workunit.client.0.smithi006.stderr: -135> 2024-05-30T16:21:39.996+0000 7f7ffb863640 10 monclient: get_auth_request con 0x7f7ff004c040 auth_method 0 2024-05-30T16:21:40.220 INFO:tasks.workunit.client.0.smithi006.stderr: -134> 2024-05-30T16:21:40.000+0000 7f7ffb062640 10 monclient: get_auth_request con 0x7f7ff0008040 auth_method 0 2024-05-30T16:21:40.220 INFO:tasks.workunit.client.0.smithi006.stderr: -133> 2024-05-30T16:21:40.008+0000 7f7ffb062640 0 -- 172.21.15.6:0/308574472 >> [v2:172.21.15.204:6816/2677861390,v1:172.21.15.204:6817/2677861390] conn(0x5607c4be17f0 msgr2=0x5607c4be3be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:21:40.220 INFO:tasks.workunit.client.0.smithi006.stderr: -132> 2024-05-30T16:21:40.008+0000 7f7edd7fa640 1 client.6768.objecter ms_handle_reset 0x5607c4be17f0 session 0x5607c4be06e0 osd.5 2024-05-30T16:21:40.220 INFO:tasks.workunit.client.0.smithi006.stderr: -131> 2024-05-30T16:21:40.008+0000 7f7edd7fa640 10 monclient: _renew_subs 2024-05-30T16:21:40.220 INFO:tasks.workunit.client.0.smithi006.stderr: -130> 2024-05-30T16:21:40.008+0000 7f7edd7fa640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:21:40.220 INFO:tasks.workunit.client.0.smithi006.stderr: -129> 2024-05-30T16:21:40.008+0000 7f7ffb062640 10 monclient: get_auth_request con 0x7f7ec407e510 auth_method 0 2024-05-30T16:21:40.220 INFO:tasks.workunit.client.0.smithi006.stderr: -128> 2024-05-30T16:21:40.016+0000 7f7ffd551980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:21:40.220 INFO:tasks.workunit.client.0.smithi006.stderr: -127> 2024-05-30T16:21:40.016+0000 7f7ffd551980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:21:40.221 INFO:tasks.workunit.client.0.smithi006.stderr: -126> 2024-05-30T16:21:40.016+0000 7f7ffd551980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:21:40.221 INFO:tasks.workunit.client.0.smithi006.stderr: -125> 2024-05-30T16:21:40.016+0000 7f7ffd551980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:21:40.221 INFO:tasks.workunit.client.0.smithi006.stderr: -124> 2024-05-30T16:21:40.016+0000 7f7ffd551980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:21:40.221 INFO:tasks.workunit.client.0.smithi006.stderr: -123> 2024-05-30T16:21:40.016+0000 7f7ffd551980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:21:40.221 INFO:tasks.workunit.client.0.smithi006.stderr: -122> 2024-05-30T16:21:40.016+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.221 INFO:tasks.workunit.client.0.smithi006.stderr: -121> 2024-05-30T16:21:40.016+0000 7f7ffd551980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:40.221 INFO:tasks.workunit.client.0.smithi006.stderr: -120> 2024-05-30T16:21:40.016+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.221 INFO:tasks.workunit.client.0.smithi006.stderr: -119> 2024-05-30T16:21:40.016+0000 7f7ffd551980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:21:40.221 INFO:tasks.workunit.client.0.smithi006.stderr: -118> 2024-05-30T16:21:40.016+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.221 INFO:tasks.workunit.client.0.smithi006.stderr: -117> 2024-05-30T16:21:40.020+0000 7f7ffd551980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:21:40.221 INFO:tasks.workunit.client.0.smithi006.stderr: -116> 2024-05-30T16:21:40.020+0000 7f7ffd551980 20 zone default found 2024-05-30T16:21:40.221 INFO:tasks.workunit.client.0.smithi006.stderr: -115> 2024-05-30T16:21:40.020+0000 7f7ffd551980 4 Realm: () 2024-05-30T16:21:40.221 INFO:tasks.workunit.client.0.smithi006.stderr: -114> 2024-05-30T16:21:40.020+0000 7f7ffd551980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:21:40.221 INFO:tasks.workunit.client.0.smithi006.stderr: -113> 2024-05-30T16:21:40.020+0000 7f7ffd551980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:21:40.222 INFO:tasks.workunit.client.0.smithi006.stderr: -112> 2024-05-30T16:21:40.020+0000 7f7ffd551980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:21:40.222 INFO:tasks.workunit.client.0.smithi006.stderr: -111> 2024-05-30T16:21:40.020+0000 7f7ffd551980 20 zonegroup default 2024-05-30T16:21:40.222 INFO:tasks.workunit.client.0.smithi006.stderr: -110> 2024-05-30T16:21:40.020+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.222 INFO:tasks.workunit.client.0.smithi006.stderr: -109> 2024-05-30T16:21:40.020+0000 7f7ffd551980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:40.222 INFO:tasks.workunit.client.0.smithi006.stderr: -108> 2024-05-30T16:21:40.020+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.222 INFO:tasks.workunit.client.0.smithi006.stderr: -107> 2024-05-30T16:21:40.020+0000 7f7ffd551980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:40.222 INFO:tasks.workunit.client.0.smithi006.stderr: -106> 2024-05-30T16:21:40.020+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.222 INFO:tasks.workunit.client.0.smithi006.stderr: -105> 2024-05-30T16:21:40.020+0000 7f7ffd551980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:40.222 INFO:tasks.workunit.client.0.smithi006.stderr: -104> 2024-05-30T16:21:40.020+0000 7f7ffd551980 20 started sync module instance, tier type = 2024-05-30T16:21:40.222 INFO:tasks.workunit.client.0.smithi006.stderr: -103> 2024-05-30T16:21:40.020+0000 7f7ffd551980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:21:40.222 INFO:tasks.workunit.client.0.smithi006.stderr: -102> 2024-05-30T16:21:40.044+0000 7f7ffd551980 20 add_watcher() i=0 2024-05-30T16:21:40.222 INFO:tasks.workunit.client.0.smithi006.stderr: -101> 2024-05-30T16:21:40.044+0000 7f7ffd551980 20 add_watcher() i=1 2024-05-30T16:21:40.222 INFO:tasks.workunit.client.0.smithi006.stderr: -100> 2024-05-30T16:21:40.044+0000 7f7ffd551980 20 add_watcher() i=2 2024-05-30T16:21:40.223 INFO:tasks.workunit.client.0.smithi006.stderr: -99> 2024-05-30T16:21:40.044+0000 7f7ffd551980 20 add_watcher() i=3 2024-05-30T16:21:40.223 INFO:tasks.workunit.client.0.smithi006.stderr: -98> 2024-05-30T16:21:40.044+0000 7f7ffd551980 20 add_watcher() i=4 2024-05-30T16:21:40.223 INFO:tasks.workunit.client.0.smithi006.stderr: -97> 2024-05-30T16:21:40.044+0000 7f7ffd551980 20 add_watcher() i=5 2024-05-30T16:21:40.223 INFO:tasks.workunit.client.0.smithi006.stderr: -96> 2024-05-30T16:21:40.048+0000 7f7ffd551980 20 add_watcher() i=6 2024-05-30T16:21:40.223 INFO:tasks.workunit.client.0.smithi006.stderr: -95> 2024-05-30T16:21:40.048+0000 7f7ffd551980 20 add_watcher() i=7 2024-05-30T16:21:40.223 INFO:tasks.workunit.client.0.smithi006.stderr: -94> 2024-05-30T16:21:40.048+0000 7f7ffd551980 2 all 8 watchers are set, enabling cache 2024-05-30T16:21:40.223 INFO:tasks.workunit.client.0.smithi006.stderr: -93> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc3cc0) adding auth protocol: cephx 2024-05-30T16:21:40.223 INFO:tasks.workunit.client.0.smithi006.stderr: -92> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc3cc0) adding auth protocol: cephx 2024-05-30T16:21:40.223 INFO:tasks.workunit.client.0.smithi006.stderr: -91> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc3cc0) adding auth protocol: cephx 2024-05-30T16:21:40.223 INFO:tasks.workunit.client.0.smithi006.stderr: -90> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc3cc0) adding con mode: secure 2024-05-30T16:21:40.223 INFO:tasks.workunit.client.0.smithi006.stderr: -89> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc3cc0) adding con mode: crc 2024-05-30T16:21:40.223 INFO:tasks.workunit.client.0.smithi006.stderr: -88> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc3cc0) adding con mode: secure 2024-05-30T16:21:40.223 INFO:tasks.workunit.client.0.smithi006.stderr: -87> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc3cc0) adding con mode: crc 2024-05-30T16:21:40.223 INFO:tasks.workunit.client.0.smithi006.stderr: -86> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc3cc0) adding con mode: secure 2024-05-30T16:21:40.224 INFO:tasks.workunit.client.0.smithi006.stderr: -85> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc3cc0) adding con mode: crc 2024-05-30T16:21:40.224 INFO:tasks.workunit.client.0.smithi006.stderr: -84> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc3cc0) adding con mode: crc 2024-05-30T16:21:40.224 INFO:tasks.workunit.client.0.smithi006.stderr: -83> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc3cc0) adding con mode: secure 2024-05-30T16:21:40.224 INFO:tasks.workunit.client.0.smithi006.stderr: -82> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc3cc0) adding con mode: crc 2024-05-30T16:21:40.224 INFO:tasks.workunit.client.0.smithi006.stderr: -81> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc3cc0) adding con mode: secure 2024-05-30T16:21:40.224 INFO:tasks.workunit.client.0.smithi006.stderr: -80> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc3cc0) adding con mode: crc 2024-05-30T16:21:40.224 INFO:tasks.workunit.client.0.smithi006.stderr: -79> 2024-05-30T16:21:40.052+0000 7f7eb1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:21:40.224 INFO:tasks.workunit.client.0.smithi006.stderr: -78> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 AuthRegistry(0x7ffe4bdc3cc0) adding con mode: secure 2024-05-30T16:21:40.224 INFO:tasks.workunit.client.0.smithi006.stderr: -77> 2024-05-30T16:21:40.052+0000 7f7ffd551980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:21:40.224 INFO:tasks.workunit.client.0.smithi006.stderr: -76> 2024-05-30T16:21:40.052+0000 7f7ffd551980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:21:40.224 INFO:tasks.workunit.client.0.smithi006.stderr: -75> 2024-05-30T16:21:40.052+0000 7f7ffd551980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:21:40.224 INFO:tasks.workunit.client.0.smithi006.stderr: -74> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command cache list hook 0x5607c4b965b0 2024-05-30T16:21:40.224 INFO:tasks.workunit.client.0.smithi006.stderr: -73> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command cache inspect hook 0x5607c4b965b0 2024-05-30T16:21:40.225 INFO:tasks.workunit.client.0.smithi006.stderr: -72> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command cache erase hook 0x5607c4b965b0 2024-05-30T16:21:40.225 INFO:tasks.workunit.client.0.smithi006.stderr: -71> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command cache zap hook 0x5607c4b965b0 2024-05-30T16:21:40.225 INFO:tasks.workunit.client.0.smithi006.stderr: -70> 2024-05-30T16:21:40.052+0000 7f7ffd551980 5 note: GC not initialized 2024-05-30T16:21:40.225 INFO:tasks.workunit.client.0.smithi006.stderr: -69> 2024-05-30T16:21:40.056+0000 7f7eb0fe1640 20 reqs_thread_entry: start 2024-05-30T16:21:40.225 INFO:tasks.workunit.client.0.smithi006.stderr: -68> 2024-05-30T16:21:40.056+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command sync trace show hook 0x7f7ff000f040 2024-05-30T16:21:40.225 INFO:tasks.workunit.client.0.smithi006.stderr: -67> 2024-05-30T16:21:40.056+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command sync trace history hook 0x7f7ff000f040 2024-05-30T16:21:40.225 INFO:tasks.workunit.client.0.smithi006.stderr: -66> 2024-05-30T16:21:40.056+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command sync trace active hook 0x7f7ff000f040 2024-05-30T16:21:40.225 INFO:tasks.workunit.client.0.smithi006.stderr: -65> 2024-05-30T16:21:40.056+0000 7f7ffd551980 5 asok(0x5607c48e9790) register_command sync trace active_short hook 0x7f7ff000f040 2024-05-30T16:21:40.225 INFO:tasks.workunit.client.0.smithi006.stderr: -64> 2024-05-30T16:21:40.056+0000 7f7ffd551980 20 init_complete bucket index max shards: 11 2024-05-30T16:21:40.225 INFO:tasks.workunit.client.0.smithi006.stderr: -63> 2024-05-30T16:21:40.056+0000 7f7ffd551980 20 Filter name: none 2024-05-30T16:21:40.225 INFO:tasks.workunit.client.0.smithi006.stderr: -62> 2024-05-30T16:21:40.056+0000 7f7e2a7fc640 20 reqs_thread_entry: start 2024-05-30T16:21:40.225 INFO:tasks.workunit.client.0.smithi006.stderr: -61> 2024-05-30T16:21:40.056+0000 7f7ffd551980 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2024-05-30T16:21:40.225 INFO:tasks.workunit.client.0.smithi006.stderr: -60> 2024-05-30T16:21:40.056+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.226 INFO:tasks.workunit.client.0.smithi006.stderr: -59> 2024-05-30T16:21:40.056+0000 7f7ffd551980 20 rados_obj.operate() r=0 bl.length=221 2024-05-30T16:21:40.226 INFO:tasks.workunit.client.0.smithi006.stderr: -58> 2024-05-30T16:21:40.056+0000 7f7ffd551980 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2024-05-30T16:21:40.226 INFO:tasks.workunit.client.0.smithi006.stderr: -57> 2024-05-30T16:21:40.056+0000 7f7ffd551980 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2024-05-30T16:21:40.226 INFO:tasks.workunit.client.0.smithi006.stderr: -56> 2024-05-30T16:21:40.056+0000 7f7ffd551980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 : miss 2024-05-30T16:21:40.226 INFO:tasks.workunit.client.0.smithi006.stderr: -55> 2024-05-30T16:21:40.056+0000 7f7ffd551980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.226 INFO:tasks.workunit.client.0.smithi006.stderr: -54> 2024-05-30T16:21:40.056+0000 7f7ffd551980 20 rados_obj.operate() r=0 bl.length=435 2024-05-30T16:21:40.226 INFO:tasks.workunit.client.0.smithi006.stderr: -53> 2024-05-30T16:21:40.056+0000 7f7ffd551980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 info.flags=0x17 2024-05-30T16:21:40.226 INFO:tasks.workunit.client.0.smithi006.stderr: -52> 2024-05-30T16:21:40.056+0000 7f7ffd551980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 to cache LRU end 2024-05-30T16:21:40.226 INFO:tasks.workunit.client.0.smithi006.stderr: -51> 2024-05-30T16:21:40.056+0000 7f7ffd551980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:21:40.226 INFO:tasks.workunit.client.0.smithi006.stderr: -50> 2024-05-30T16:21:40.056+0000 7f7ffd551980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:21:40.226 INFO:tasks.workunit.client.0.smithi006.stderr: -49> 2024-05-30T16:21:40.056+0000 7f7ffd551980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 2024-05-30T16:21:40.226 INFO:tasks.workunit.client.0.smithi006.stderr: -48> 2024-05-30T16:21:40.068+0000 7f7ffd551980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 info.flags=0x17 2024-05-30T16:21:40.226 INFO:tasks.workunit.client.0.smithi006.stderr: -47> 2024-05-30T16:21:40.068+0000 7f7ffd551980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 to cache LRU end 2024-05-30T16:21:40.227 INFO:tasks.workunit.client.0.smithi006.stderr: -46> 2024-05-30T16:21:40.068+0000 7f7ffd551980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:21:40.227 INFO:tasks.workunit.client.0.smithi006.stderr: -45> 2024-05-30T16:21:40.068+0000 7f7ffd551980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27, ofs0, ns] 2024-05-30T16:21:40.227 INFO:tasks.workunit.client.0.smithi006.stderr: -44> 2024-05-30T16:21:40.068+0000 7f7edeffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431626 cookie 94591365588048 notifier 6768 bl.length()=1007 2024-05-30T16:21:40.227 INFO:tasks.workunit.client.0.smithi006.stderr: -43> 2024-05-30T16:21:40.068+0000 7f7edeffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 info.flags=0x17 2024-05-30T16:21:40.227 INFO:tasks.workunit.client.0.smithi006.stderr: -42> 2024-05-30T16:21:40.068+0000 7f7edeffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 to cache LRU end 2024-05-30T16:21:40.227 INFO:tasks.workunit.client.0.smithi006.stderr: -41> 2024-05-30T16:21:40.068+0000 7f7edeffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:21:40.227 INFO:tasks.workunit.client.0.smithi006.stderr: -40> 2024-05-30T16:21:40.072+0000 7f7ffd551980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27]): orig_sources= new_sources= 2024-05-30T16:21:40.227 INFO:tasks.workunit.client.0.smithi006.stderr: -39> 2024-05-30T16:21:40.072+0000 7f7ffd551980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27]): potential sources added=[] removed=[] 2024-05-30T16:21:40.227 INFO:tasks.workunit.client.0.smithi006.stderr: -38> 2024-05-30T16:21:40.072+0000 7f7ffd551980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27]): orig_dests= new_dests= 2024-05-30T16:21:40.227 INFO:tasks.workunit.client.0.smithi006.stderr: -37> 2024-05-30T16:21:40.072+0000 7f7ffd551980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27]): potential dests added=[] removed=[] 2024-05-30T16:21:40.227 INFO:tasks.workunit.client.0.smithi006.stderr: -36> 2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.0 2024-05-30T16:21:40.227 INFO:tasks.workunit.client.0.smithi006.stderr: -35> 2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.0 2024-05-30T16:21:40.227 INFO:tasks.workunit.client.0.smithi006.stderr: -34> 2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.1 2024-05-30T16:21:40.228 INFO:tasks.workunit.client.0.smithi006.stderr: -33> 2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.1 2024-05-30T16:21:40.228 INFO:tasks.workunit.client.0.smithi006.stderr: -32> 2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.2 2024-05-30T16:21:40.228 INFO:tasks.workunit.client.0.smithi006.stderr: -31> 2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.2 2024-05-30T16:21:40.228 INFO:tasks.workunit.client.0.smithi006.stderr: -30> 2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.3 2024-05-30T16:21:40.228 INFO:tasks.workunit.client.0.smithi006.stderr: -29> 2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.3 2024-05-30T16:21:40.228 INFO:tasks.workunit.client.0.smithi006.stderr: -28> 2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.4 2024-05-30T16:21:40.228 INFO:tasks.workunit.client.0.smithi006.stderr: -27> 2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.4 2024-05-30T16:21:40.228 INFO:tasks.workunit.client.0.smithi006.stderr: -26> 2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.5 2024-05-30T16:21:40.228 INFO:tasks.workunit.client.0.smithi006.stderr: -25> 2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.5 2024-05-30T16:21:40.228 INFO:tasks.workunit.client.0.smithi006.stderr: -24> 2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.6 2024-05-30T16:21:40.228 INFO:tasks.workunit.client.0.smithi006.stderr: -23> 2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.6 2024-05-30T16:21:40.228 INFO:tasks.workunit.client.0.smithi006.stderr: -22> 2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.7 2024-05-30T16:21:40.228 INFO:tasks.workunit.client.0.smithi006.stderr: -21> 2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.7 2024-05-30T16:21:40.228 INFO:tasks.workunit.client.0.smithi006.stderr: -20> 2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.8 2024-05-30T16:21:40.229 INFO:tasks.workunit.client.0.smithi006.stderr: -19> 2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.8 2024-05-30T16:21:40.229 INFO:tasks.workunit.client.0.smithi006.stderr: -18> 2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.9 2024-05-30T16:21:40.229 INFO:tasks.workunit.client.0.smithi006.stderr: -17> 2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.9 2024-05-30T16:21:40.229 INFO:tasks.workunit.client.0.smithi006.stderr: -16> 2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.10 2024-05-30T16:21:40.229 INFO:tasks.workunit.client.0.smithi006.stderr: -15> 2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.10 2024-05-30T16:21:40.229 INFO:tasks.workunit.client.0.smithi006.stderr: -14> 2024-05-30T16:21:40.076+0000 7f7ffd551980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.11 2024-05-30T16:21:40.229 INFO:tasks.workunit.client.0.smithi006.stderr: -13> 2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.11 2024-05-30T16:21:40.229 INFO:tasks.workunit.client.0.smithi006.stderr: -12> 2024-05-30T16:21:40.076+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.0 2024-05-30T16:21:40.229 INFO:tasks.workunit.client.0.smithi006.stderr: -11> 2024-05-30T16:21:40.080+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1 2024-05-30T16:21:40.229 INFO:tasks.workunit.client.0.smithi006.stderr: -10> 2024-05-30T16:21:40.080+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.2 2024-05-30T16:21:40.229 INFO:tasks.workunit.client.0.smithi006.stderr: -9> 2024-05-30T16:21:40.080+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.3 2024-05-30T16:21:40.229 INFO:tasks.workunit.client.0.smithi006.stderr: -8> 2024-05-30T16:21:40.080+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.4 2024-05-30T16:21:40.229 INFO:tasks.workunit.client.0.smithi006.stderr: -7> 2024-05-30T16:21:40.084+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.5 2024-05-30T16:21:40.230 INFO:tasks.workunit.client.0.smithi006.stderr: -6> 2024-05-30T16:21:40.084+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.6 2024-05-30T16:21:40.230 INFO:tasks.workunit.client.0.smithi006.stderr: -5> 2024-05-30T16:21:40.084+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.7 2024-05-30T16:21:40.230 INFO:tasks.workunit.client.0.smithi006.stderr: -4> 2024-05-30T16:21:40.088+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.8 2024-05-30T16:21:40.230 INFO:tasks.workunit.client.0.smithi006.stderr: -3> 2024-05-30T16:21:40.088+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.9 2024-05-30T16:21:40.230 INFO:tasks.workunit.client.0.smithi006.stderr: -2> 2024-05-30T16:21:40.088+0000 7f7ffd551980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.10 2024-05-30T16:21:40.230 INFO:tasks.workunit.client.0.smithi006.stderr: -1> 2024-05-30T16:21:40.096+0000 7f7ffd551980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f7ffd551980 time 2024-05-30T16:21:40.096090+0000 2024-05-30T16:21:40.230 INFO:tasks.workunit.client.0.smithi006.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-30T16:21:40.230 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.230 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:21:40.230 INFO:tasks.workunit.client.0.smithi006.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f8000e28cff] 2024-05-30T16:21:40.230 INFO:tasks.workunit.client.0.smithi006.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7f8000e28eb3] 2024-05-30T16:21:40.230 INFO:tasks.workunit.client.0.smithi006.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x5607c26f5f81] 2024-05-30T16:21:40.230 INFO:tasks.workunit.client.0.smithi006.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x56f) [0x5607c270816f] 2024-05-30T16:21:40.231 INFO:tasks.workunit.client.0.smithi006.stderr: 5: main() 2024-05-30T16:21:40.231 INFO:tasks.workunit.client.0.smithi006.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f7fffc91d90] 2024-05-30T16:21:40.231 INFO:tasks.workunit.client.0.smithi006.stderr: 7: __libc_start_main() 2024-05-30T16:21:40.231 INFO:tasks.workunit.client.0.smithi006.stderr: 8: _start() 2024-05-30T16:21:40.231 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.231 INFO:tasks.workunit.client.0.smithi006.stderr: 0> 2024-05-30T16:21:40.096+0000 7f7ffd551980 -1 *** Caught signal (Aborted) ** 2024-05-30T16:21:40.231 INFO:tasks.workunit.client.0.smithi006.stderr: in thread 7f7ffd551980 thread_name:radosgw-admin 2024-05-30T16:21:40.231 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.231 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:21:40.231 INFO:tasks.workunit.client.0.smithi006.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f7fffcaa520] 2024-05-30T16:21:40.231 INFO:tasks.workunit.client.0.smithi006.stderr: 2: pthread_kill() 2024-05-30T16:21:40.231 INFO:tasks.workunit.client.0.smithi006.stderr: 3: raise() 2024-05-30T16:21:40.231 INFO:tasks.workunit.client.0.smithi006.stderr: 4: abort() 2024-05-30T16:21:40.232 INFO:tasks.workunit.client.0.smithi006.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f8000e28d59] 2024-05-30T16:21:40.232 INFO:tasks.workunit.client.0.smithi006.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7f8000e28eb3] 2024-05-30T16:21:40.232 INFO:tasks.workunit.client.0.smithi006.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x5607c26f5f81] 2024-05-30T16:21:40.232 INFO:tasks.workunit.client.0.smithi006.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x56f) [0x5607c270816f] 2024-05-30T16:21:40.232 INFO:tasks.workunit.client.0.smithi006.stderr: 9: main() 2024-05-30T16:21:40.232 INFO:tasks.workunit.client.0.smithi006.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f7fffc91d90] 2024-05-30T16:21:40.232 INFO:tasks.workunit.client.0.smithi006.stderr: 11: __libc_start_main() 2024-05-30T16:21:40.232 INFO:tasks.workunit.client.0.smithi006.stderr: 12: _start() 2024-05-30T16:21:40.232 INFO:tasks.workunit.client.0.smithi006.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-30T16:21:40.232 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.232 INFO:tasks.workunit.client.0.smithi006.stderr:--- logging levels --- 2024-05-30T16:21:40.232 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 none 2024-05-30T16:21:40.232 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 lockdep 2024-05-30T16:21:40.232 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 context 2024-05-30T16:21:40.233 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 crush 2024-05-30T16:21:40.233 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds 2024-05-30T16:21:40.233 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_balancer 2024-05-30T16:21:40.233 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_locker 2024-05-30T16:21:40.233 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_log 2024-05-30T16:21:40.233 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_log_expire 2024-05-30T16:21:40.233 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_migrator 2024-05-30T16:21:40.233 INFO:tasks.workunit.client.0.smithi006.stderr: 3/ 5 mds_quiesce 2024-05-30T16:21:40.233 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 buffer 2024-05-30T16:21:40.233 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 timer 2024-05-30T16:21:40.233 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 filer 2024-05-30T16:21:40.233 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 striper 2024-05-30T16:21:40.233 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 objecter 2024-05-30T16:21:40.233 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rados 2024-05-30T16:21:40.233 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd 2024-05-30T16:21:40.234 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd_mirror 2024-05-30T16:21:40.234 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd_replay 2024-05-30T16:21:40.234 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd_pwl 2024-05-30T16:21:40.234 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 journaler 2024-05-30T16:21:40.234 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 objectcacher 2024-05-30T16:21:40.234 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 immutable_obj_cache 2024-05-30T16:21:40.234 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 client 2024-05-30T16:21:40.234 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 osd 2024-05-30T16:21:40.234 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 optracker 2024-05-30T16:21:40.234 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 objclass 2024-05-30T16:21:40.234 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 3 filestore 2024-05-30T16:21:40.234 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 3 journal 2024-05-30T16:21:40.234 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 0 ms 2024-05-30T16:21:40.235 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mon 2024-05-30T16:21:40.235 INFO:tasks.workunit.client.0.smithi006.stderr: 0/10 monc 2024-05-30T16:21:40.235 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 paxos 2024-05-30T16:21:40.235 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 tp 2024-05-30T16:21:40.235 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 auth 2024-05-30T16:21:40.235 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 crypto 2024-05-30T16:21:40.235 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 finisher 2024-05-30T16:21:40.235 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 reserver 2024-05-30T16:21:40.235 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 heartbeatmap 2024-05-30T16:21:40.235 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 perfcounter 2024-05-30T16:21:40.235 INFO:tasks.workunit.client.0.smithi006.stderr: 20/20 rgw 2024-05-30T16:21:40.235 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_sync 2024-05-30T16:21:40.235 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_datacache 2024-05-30T16:21:40.235 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_access 2024-05-30T16:21:40.236 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_dbstore 2024-05-30T16:21:40.236 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_flight 2024-05-30T16:21:40.236 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_lifecycle 2024-05-30T16:21:40.236 INFO:tasks.workunit.client.0.smithi006.stderr: 20/20 rgw_notification 2024-05-30T16:21:40.236 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 javaclient 2024-05-30T16:21:40.236 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 asok 2024-05-30T16:21:40.236 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 throttle 2024-05-30T16:21:40.236 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 0 refs 2024-05-30T16:21:40.236 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 compressor 2024-05-30T16:21:40.236 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 bluestore 2024-05-30T16:21:40.236 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 bluefs 2024-05-30T16:21:40.237 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 3 bdev 2024-05-30T16:21:40.237 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 kstore 2024-05-30T16:21:40.237 INFO:tasks.workunit.client.0.smithi006.stderr: 4/ 5 rocksdb 2024-05-30T16:21:40.237 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 fuse 2024-05-30T16:21:40.237 INFO:tasks.workunit.client.0.smithi006.stderr: 2/ 5 mgr 2024-05-30T16:21:40.237 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mgrc 2024-05-30T16:21:40.237 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 dpdk 2024-05-30T16:21:40.237 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 eventtrace 2024-05-30T16:21:40.237 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 prioritycache 2024-05-30T16:21:40.237 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 test 2024-05-30T16:21:40.238 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 cephfs_mirror 2024-05-30T16:21:40.238 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 cephsqlite 2024-05-30T16:21:40.238 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 crimson_interrupt 2024-05-30T16:21:40.238 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore 2024-05-30T16:21:40.238 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_onode 2024-05-30T16:21:40.238 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_odata 2024-05-30T16:21:40.238 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_omap 2024-05-30T16:21:40.238 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_tm 2024-05-30T16:21:40.238 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_t 2024-05-30T16:21:40.238 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_cleaner 2024-05-30T16:21:40.238 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_epm 2024-05-30T16:21:40.238 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_lba 2024-05-30T16:21:40.238 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_fixedkv_tree 2024-05-30T16:21:40.239 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_cache 2024-05-30T16:21:40.239 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_journal 2024-05-30T16:21:40.239 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_device 2024-05-30T16:21:40.239 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_backref 2024-05-30T16:21:40.239 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 alienstore 2024-05-30T16:21:40.239 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mclock 2024-05-30T16:21:40.239 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 cyanstore 2024-05-30T16:21:40.239 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 ceph_exporter 2024-05-30T16:21:40.240 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 memstore 2024-05-30T16:21:40.240 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 trace 2024-05-30T16:21:40.240 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 ceph_dedup 2024-05-30T16:21:40.240 INFO:tasks.workunit.client.0.smithi006.stderr: -2/-2 (syslog threshold) 2024-05-30T16:21:40.240 INFO:tasks.workunit.client.0.smithi006.stderr: 99/99 (stderr threshold) 2024-05-30T16:21:40.240 INFO:tasks.workunit.client.0.smithi006.stderr:--- pthread ID / name mapping for recent threads --- 2024-05-30T16:21:40.240 INFO:tasks.workunit.client.0.smithi006.stderr: 7f7e2a7fc640 / http_manager 2024-05-30T16:21:40.240 INFO:tasks.workunit.client.0.smithi006.stderr: 7f7eb0fe1640 / http_manager 2024-05-30T16:21:40.240 INFO:tasks.workunit.client.0.smithi006.stderr: 7f7eb1fe3640 / rgw_dt_lg_renew 2024-05-30T16:21:40.240 INFO:tasks.workunit.client.0.smithi006.stderr: 7f7edd7fa640 / ms_dispatch 2024-05-30T16:21:40.240 INFO:tasks.workunit.client.0.smithi006.stderr: 7f7edeffd640 / io_context_pool 2024-05-30T16:21:40.240 INFO:tasks.workunit.client.0.smithi006.stderr: 7f7ee5602640 / ms_dispatch 2024-05-30T16:21:40.240 INFO:tasks.workunit.client.0.smithi006.stderr: 7f7ee7e07640 / ms_dispatch 2024-05-30T16:21:40.240 INFO:tasks.workunit.client.0.smithi006.stderr: 7f7ee960a640 / io_context_pool 2024-05-30T16:21:40.241 INFO:tasks.workunit.client.0.smithi006.stderr: 7f7ffa861640 / msgr-worker-2 2024-05-30T16:21:40.241 INFO:tasks.workunit.client.0.smithi006.stderr: 7f7ffb062640 / msgr-worker-1 2024-05-30T16:21:40.241 INFO:tasks.workunit.client.0.smithi006.stderr: 7f7ffb863640 / msgr-worker-0 2024-05-30T16:21:40.241 INFO:tasks.workunit.client.0.smithi006.stderr: 7f7ffc064640 / io_context_pool 2024-05-30T16:21:40.241 INFO:tasks.workunit.client.0.smithi006.stderr: 7f7ffd551980 / radosgw-admin 2024-05-30T16:21:40.241 INFO:tasks.workunit.client.0.smithi006.stderr: max_recent 500 2024-05-30T16:21:40.241 INFO:tasks.workunit.client.0.smithi006.stderr: max_new 1000 2024-05-30T16:21:40.241 INFO:tasks.workunit.client.0.smithi006.stderr: log_file /var/lib/ceph/crash/2024-05-30T16:21:40.098719Z_6cc6c344-b857-4460-bd54-c983b802a079/log 2024-05-30T16:21:40.241 INFO:tasks.workunit.client.0.smithi006.stderr:--- end dump of recent events --- 2024-05-30T16:21:40.242 INFO:tasks.workunit.client.0.smithi006.stderr:Aborted 2024-05-30T16:21:40.242 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket abort-at-commit-target-layout 2024-05-30T16:21:40.243 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:21:40.243 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:21:40.243 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.184+0000 7f837d7b8980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.243 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.184+0000 7f837d7b8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:40.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.184+0000 7f837d7b8980 20 realm 2024-05-30T16:21:40.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.184+0000 7f837d7b8980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.184+0000 7f837d7b8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:40.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.184+0000 7f837d7b8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:21:40.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.184+0000 7f837d7b8980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.184+0000 7f837d7b8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:40.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.184+0000 7f837d7b8980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.188+0000 7f837d7b8980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:21:40.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.188+0000 7f837d7b8980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.188+0000 7f837d7b8980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:21:40.244 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.188+0000 7f837d7b8980 20 searching for the correct realm 2024-05-30T16:21:40.245 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.192+0000 7f837b2c9640 0 -- 172.21.15.6:0/203183196 >> [v2:172.21.15.6:6816/2704410641,v1:172.21.15.6:6817/2704410641] conn(0x7f8374018040 msgr2=0x7f8374022950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:21:40.405 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.404+0000 7f837baca640 0 -- 172.21.15.6:0/203183196 >> [v2:172.21.15.204:6823/3156311527,v1:172.21.15.204:6825/3156311527] conn(0x7f83700c4040 msgr2=0x7f83700faf40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:21:40.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.424+0000 7f837d7b8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:21:40.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.424+0000 7f837d7b8980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:21:40.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.424+0000 7f837d7b8980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:21:40.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.424+0000 7f837d7b8980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:21:40.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.424+0000 7f837d7b8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:21:40.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.424+0000 7f837d7b8980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:21:40.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.424+0000 7f837d7b8980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.424+0000 7f837d7b8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:40.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.424+0000 7f837d7b8980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.424+0000 7f837d7b8980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:21:40.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.424+0000 7f837d7b8980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.425 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.424+0000 7f837d7b8980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:21:40.425 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.424+0000 7f837d7b8980 20 zone default found 2024-05-30T16:21:40.425 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.424+0000 7f837d7b8980 4 Realm: () 2024-05-30T16:21:40.425 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.424+0000 7f837d7b8980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:21:40.425 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.424+0000 7f837d7b8980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:21:40.425 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.424+0000 7f837d7b8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:21:40.425 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.424+0000 7f837d7b8980 20 zonegroup default 2024-05-30T16:21:40.426 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.424+0000 7f837d7b8980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.426 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.428+0000 7f837d7b8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:40.426 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.428+0000 7f837d7b8980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.426 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.428+0000 7f837d7b8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:40.427 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.428+0000 7f837d7b8980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.427 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.428+0000 7f837d7b8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:21:40.427 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.428+0000 7f837d7b8980 20 started sync module instance, tier type = 2024-05-30T16:21:40.428 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.428+0000 7f837d7b8980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:21:40.450 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.452+0000 7f837d7b8980 20 add_watcher() i=0 2024-05-30T16:21:40.450 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.452+0000 7f837d7b8980 20 add_watcher() i=1 2024-05-30T16:21:40.450 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.452+0000 7f837d7b8980 20 add_watcher() i=2 2024-05-30T16:21:40.450 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.452+0000 7f837d7b8980 20 add_watcher() i=3 2024-05-30T16:21:40.450 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.452+0000 7f837d7b8980 20 add_watcher() i=4 2024-05-30T16:21:40.450 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.452+0000 7f837d7b8980 20 add_watcher() i=5 2024-05-30T16:21:40.451 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.452+0000 7f837d7b8980 20 add_watcher() i=6 2024-05-30T16:21:40.452 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.452+0000 7f837d7b8980 20 add_watcher() i=7 2024-05-30T16:21:40.452 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.452+0000 7f837d7b8980 2 all 8 watchers are set, enabling cache 2024-05-30T16:21:40.456 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.456+0000 7f822dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:21:40.457 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.456+0000 7f837d7b8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:21:40.457 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.456+0000 7f837d7b8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:21:40.458 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.460+0000 7f837d7b8980 5 note: GC not initialized 2024-05-30T16:21:40.458 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.460+0000 7f822cfe1640 20 reqs_thread_entry: start 2024-05-30T16:21:40.458 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.460+0000 7f837d7b8980 20 init_complete bucket index max shards: 11 2024-05-30T16:21:40.459 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.460+0000 7f837d7b8980 20 Filter name: none 2024-05-30T16:21:40.459 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.460+0000 7f81a67fc640 20 reqs_thread_entry: start 2024-05-30T16:21:40.459 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.460+0000 7f837d7b8980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.460 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.460+0000 7f837d7b8980 20 rados_obj.operate() r=0 bl.length=221 2024-05-30T16:21:40.460 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.460+0000 7f837d7b8980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.461 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.460+0000 7f837d7b8980 20 rados_obj.operate() r=0 bl.length=467 2024-05-30T16:21:40.461 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.460+0000 7f837d7b8980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:21:40.462 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.460+0000 7f837d7b8980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.462 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.464+0000 7f837d7b8980 20 rados_obj.operate() r=0 bl.length=221 2024-05-30T16:21:40.462 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.464+0000 7f837d7b8980 20 rados->read ofs=0 len=0 2024-05-30T16:21:40.463 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.464+0000 7f837d7b8980 20 rados_obj.operate() r=0 bl.length=467 2024-05-30T16:21:40.463 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.464+0000 7f837d7b8980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:21:40.466 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.468+0000 7f822dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:21:40.471 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.472+0000 7f837d7b8980 20 remove_watcher() i=0 2024-05-30T16:21:40.471 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.472+0000 7f837d7b8980 2 removed watcher, disabling cache 2024-05-30T16:21:40.473 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.476+0000 7f837d7b8980 20 remove_watcher() i=1 2024-05-30T16:21:40.476 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.476+0000 7f837d7b8980 20 remove_watcher() i=2 2024-05-30T16:21:40.478 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.480+0000 7f837d7b8980 20 remove_watcher() i=3 2024-05-30T16:21:40.480 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.480+0000 7f837d7b8980 20 remove_watcher() i=4 2024-05-30T16:21:40.482 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.484+0000 7f837d7b8980 20 remove_watcher() i=5 2024-05-30T16:21:40.483 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.484+0000 7f837d7b8980 20 remove_watcher() i=6 2024-05-30T16:21:40.485 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:21:40.484+0000 7f837d7b8980 20 remove_watcher() i=7 2024-05-30T16:21:40.510 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:21:40.510 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:21:40.510 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: abort-at-commit-target-layout id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-30T16:21:40.510 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:10.916 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket reshard --bucket abort-at-commit-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 2024-05-30T16:22:10.953 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:22:10.953 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:22:10.999 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:10.999+0000 7fcc4dda4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:11.001 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:10.999+0000 7fcc4dda4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:11.001 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:10.999+0000 7fcc4dda4980 20 realm 2024-05-30T16:22:11.001 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:10.999+0000 7fcc4dda4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:11.001 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:10.999+0000 7fcc4dda4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:11.001 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:10.999+0000 7fcc4dda4980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:22:11.002 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:10.999+0000 7fcc4dda4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:11.002 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:10.999+0000 7fcc4dda4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:11.002 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:10.999+0000 7fcc4dda4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:11.003 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.003+0000 7fcc4dda4980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:11.003 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.003+0000 7fcc4dda4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:11.004 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.003+0000 7fcc4dda4980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:22:11.004 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.003+0000 7fcc4dda4980 20 searching for the correct realm 2024-05-30T16:22:11.010 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.011+0000 7fcc4c0b6640 0 -- 172.21.15.6:0/385460712 >> [v2:172.21.15.6:6808/1897362715,v1:172.21.15.6:6809/1897362715] conn(0x55bfe2a06d90 msgr2=0x55bfe2a091e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:22:11.016 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.015+0000 7fcc4b8b5640 0 -- 172.21.15.6:0/385460712 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7fcc40044930 msgr2=0x7fcc40052400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:11.024 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.023+0000 7fcc4dda4980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:22:11.025 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.023+0000 7fcc4dda4980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:22:11.025 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.023+0000 7fcc4dda4980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:22:11.025 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.023+0000 7fcc4dda4980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:22:11.025 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.023+0000 7fcc4dda4980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:22:11.025 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.023+0000 7fcc4dda4980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:22:11.025 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.023+0000 7fcc4dda4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:11.025 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.023+0000 7fcc4dda4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:11.025 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.023+0000 7fcc4dda4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:11.026 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.023+0000 7fcc4dda4980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:11.026 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.023+0000 7fcc4dda4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:11.027 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.027+0000 7fcc4dda4980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:22:11.027 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.027+0000 7fcc4dda4980 20 zone default found 2024-05-30T16:22:11.027 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.027+0000 7fcc4dda4980 4 Realm: () 2024-05-30T16:22:11.027 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.027+0000 7fcc4dda4980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:22:11.027 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.027+0000 7fcc4dda4980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:22:11.027 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.027+0000 7fcc4dda4980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:22:11.027 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.027+0000 7fcc4dda4980 20 zonegroup default 2024-05-30T16:22:11.027 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.027+0000 7fcc4dda4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:11.027 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.027+0000 7fcc4dda4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:11.027 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.027+0000 7fcc4dda4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:11.028 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.027+0000 7fcc4dda4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:11.028 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.027+0000 7fcc4dda4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:11.028 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.027+0000 7fcc4dda4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:11.028 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.027+0000 7fcc4dda4980 20 started sync module instance, tier type = 2024-05-30T16:22:11.028 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.027+0000 7fcc4dda4980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:22:11.045 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.043+0000 7fcc4dda4980 20 add_watcher() i=0 2024-05-30T16:22:11.045 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.043+0000 7fcc4dda4980 20 add_watcher() i=1 2024-05-30T16:22:11.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.043+0000 7fcc4dda4980 20 add_watcher() i=2 2024-05-30T16:22:11.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.043+0000 7fcc4dda4980 20 add_watcher() i=3 2024-05-30T16:22:11.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.043+0000 7fcc4dda4980 20 add_watcher() i=4 2024-05-30T16:22:11.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.043+0000 7fcc4dda4980 20 add_watcher() i=5 2024-05-30T16:22:11.047 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.047+0000 7fcc4dda4980 20 add_watcher() i=6 2024-05-30T16:22:11.047 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.047+0000 7fcc4dda4980 20 add_watcher() i=7 2024-05-30T16:22:11.047 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.047+0000 7fcc4dda4980 2 all 8 watchers are set, enabling cache 2024-05-30T16:22:11.049 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.051+0000 7fcb01fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:11.050 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.051+0000 7fcc4dda4980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:22:11.050 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.051+0000 7fcc4dda4980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:22:11.050 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.051+0000 7fcc4dda4980 5 note: GC not initialized 2024-05-30T16:22:11.051 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.051+0000 7fcb00fe1640 20 reqs_thread_entry: start 2024-05-30T16:22:11.051 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.051+0000 7fcc4dda4980 20 init_complete bucket index max shards: 11 2024-05-30T16:22:11.051 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.051+0000 7fcc4dda4980 20 Filter name: none 2024-05-30T16:22:11.051 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.051+0000 7fca7a7fc640 20 reqs_thread_entry: start 2024-05-30T16:22:11.051 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.051+0000 7fcc4dda4980 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2024-05-30T16:22:11.051 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.051+0000 7fcc4dda4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:11.052 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.051+0000 7fcc4dda4980 20 rados_obj.operate() r=0 bl.length=221 2024-05-30T16:22:11.052 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.051+0000 7fcc4dda4980 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2024-05-30T16:22:11.052 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.051+0000 7fcc4dda4980 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2024-05-30T16:22:11.052 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.051+0000 7fcc4dda4980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 : miss 2024-05-30T16:22:11.052 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.051+0000 7fcc4dda4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:11.052 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.051+0000 7fcc4dda4980 20 rados_obj.operate() r=0 bl.length=435 2024-05-30T16:22:11.052 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.051+0000 7fcc4dda4980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 info.flags=0x17 2024-05-30T16:22:11.053 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.051+0000 7fcc4dda4980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 to cache LRU end 2024-05-30T16:22:11.053 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.051+0000 7fcc4dda4980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:22:11.053 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.051+0000 7fcc4dda4980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:22:11.053 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.051+0000 7fcc4dda4980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 2024-05-30T16:22:11.063 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.063+0000 7fcc4dda4980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 info.flags=0x17 2024-05-30T16:22:11.063 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.063+0000 7fcc4dda4980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 to cache LRU end 2024-05-30T16:22:11.063 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.063+0000 7fcc4dda4980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:22:11.063 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.063+0000 7fcc4dda4980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27, ofs0, ns] 2024-05-30T16:22:11.064 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.063+0000 7fcb2affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431628 cookie 94282629330336 notifier 6479 bl.length()=1007 2024-05-30T16:22:11.064 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.063+0000 7fcb2affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 info.flags=0x17 2024-05-30T16:22:11.064 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.063+0000 7fcb2affd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 to cache LRU end 2024-05-30T16:22:11.064 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.063+0000 7fcb2affd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:22:11.066 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.067+0000 7fcc4dda4980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27]): orig_sources= new_sources= 2024-05-30T16:22:11.066 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.067+0000 7fcc4dda4980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27]): potential sources added=[] removed=[] 2024-05-30T16:22:11.066 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.067+0000 7fcc4dda4980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27]): orig_dests= new_dests= 2024-05-30T16:22:11.066 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.067+0000 7fcc4dda4980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27]): potential dests added=[] removed=[] 2024-05-30T16:22:11.071 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.071+0000 7fcc4dda4980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.0 2024-05-30T16:22:11.071 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.071+0000 7fcc4dda4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.0 2024-05-30T16:22:11.072 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.071+0000 7fcc4dda4980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.1 2024-05-30T16:22:11.072 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.071+0000 7fcc4dda4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.1 2024-05-30T16:22:11.072 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.071+0000 7fcc4dda4980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.2 2024-05-30T16:22:11.072 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.071+0000 7fcc4dda4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.2 2024-05-30T16:22:11.072 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.071+0000 7fcc4dda4980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.3 2024-05-30T16:22:11.072 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.071+0000 7fcc4dda4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.3 2024-05-30T16:22:11.072 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.071+0000 7fcc4dda4980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.4 2024-05-30T16:22:11.072 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.071+0000 7fcc4dda4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.4 2024-05-30T16:22:11.072 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.071+0000 7fcc4dda4980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.5 2024-05-30T16:22:11.072 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.071+0000 7fcc4dda4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.5 2024-05-30T16:22:11.072 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.071+0000 7fcc4dda4980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.6 2024-05-30T16:22:11.072 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.071+0000 7fcc4dda4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.6 2024-05-30T16:22:11.072 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.071+0000 7fcc4dda4980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.7 2024-05-30T16:22:11.072 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.071+0000 7fcc4dda4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.7 2024-05-30T16:22:11.073 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.071+0000 7fcc4dda4980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.8 2024-05-30T16:22:11.073 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.071+0000 7fcc4dda4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.8 2024-05-30T16:22:11.073 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.071+0000 7fcc4dda4980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.9 2024-05-30T16:22:11.073 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.071+0000 7fcc4dda4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.9 2024-05-30T16:22:11.073 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.071+0000 7fcc4dda4980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.10 2024-05-30T16:22:11.073 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.071+0000 7fcc4dda4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.10 2024-05-30T16:22:11.073 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.071+0000 7fcc4dda4980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.11 2024-05-30T16:22:11.073 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.071+0000 7fcc4dda4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1.11 2024-05-30T16:22:11.073 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.071+0000 7fcc4dda4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.0 2024-05-30T16:22:11.073 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.071+0000 7fcc4dda4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.1 2024-05-30T16:22:11.074 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.075+0000 7fcc4dda4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.2 2024-05-30T16:22:11.075 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.075+0000 7fcc4dda4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.3 2024-05-30T16:22:11.075 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.075+0000 7fcc4dda4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.4 2024-05-30T16:22:11.076 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.075+0000 7fcc4dda4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.5 2024-05-30T16:22:11.078 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.079+0000 7fcc4dda4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.6 2024-05-30T16:22:11.080 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.079+0000 7fcc4dda4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.7 2024-05-30T16:22:11.080 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.079+0000 7fcc4dda4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.8 2024-05-30T16:22:11.081 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.079+0000 7fcc4dda4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.9 2024-05-30T16:22:11.081 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.083+0000 7fcc4dda4980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27.10 2024-05-30T16:22:11.086 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.087+0000 7fcc4dda4980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 : hit (requested=0x17, cached=0x17) 2024-05-30T16:22:11.086 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.087+0000 7fcc4dda4980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 2024-05-30T16:22:11.089 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.091+0000 7fcc4dda4980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 info.flags=0x17 2024-05-30T16:22:11.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.091+0000 7fcc4dda4980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 to cache LRU end 2024-05-30T16:22:11.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.091+0000 7fcc4dda4980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:22:11.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.091+0000 7fcc4dda4980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27, ofs0, ns] 2024-05-30T16:22:11.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.091+0000 7fcb2a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431629 cookie 94282629330336 notifier 6479 bl.length()=975 2024-05-30T16:22:11.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.091+0000 7fcb2a7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 info.flags=0x17 2024-05-30T16:22:11.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.091+0000 7fcb2a7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 to cache LRU end 2024-05-30T16:22:11.091 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.091+0000 7fcb2a7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:22:11.093 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.091+0000 7fcc4dda4980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27]): orig_sources= new_sources= 2024-05-30T16:22:11.093 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.091+0000 7fcc4dda4980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27]): potential sources added=[] removed=[] 2024-05-30T16:22:11.093 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.091+0000 7fcc4dda4980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27]): orig_dests= new_dests= 2024-05-30T16:22:11.093 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.091+0000 7fcc4dda4980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27]): potential dests added=[] removed=[] 2024-05-30T16:22:11.097 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.099+0000 7fcc4dda4980 1 execute INFO: reshard of bucket "abort-at-commit-target-layout" completed successfully 2024-05-30T16:22:11.101 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.099+0000 7fcb01fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:11.105 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.107+0000 7fcc4dda4980 20 remove_watcher() i=0 2024-05-30T16:22:11.106 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.107+0000 7fcc4dda4980 2 removed watcher, disabling cache 2024-05-30T16:22:11.108 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.107+0000 7fcc4dda4980 20 remove_watcher() i=1 2024-05-30T16:22:11.110 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.111+0000 7fcc4dda4980 20 remove_watcher() i=2 2024-05-30T16:22:11.113 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.111+0000 7fcc4dda4980 20 remove_watcher() i=3 2024-05-30T16:22:11.115 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.115+0000 7fcc4dda4980 20 remove_watcher() i=4 2024-05-30T16:22:11.116 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.115+0000 7fcc4dda4980 20 remove_watcher() i=5 2024-05-30T16:22:11.117 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.119+0000 7fcc4dda4980 20 remove_watcher() i=6 2024-05-30T16:22:11.120 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.119+0000 7fcc4dda4980 20 remove_watcher() i=7 2024-05-30T16:22:11.134 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket abort-at-commit-target-layout 2024-05-30T16:22:11.162 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:22:11.162 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:22:11.206 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.207+0000 7f99c13d4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:11.209 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.207+0000 7f99c13d4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:11.209 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.207+0000 7f99c13d4980 20 realm 2024-05-30T16:22:11.209 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.207+0000 7f99c13d4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:11.209 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.207+0000 7f99c13d4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:11.209 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.207+0000 7f99c13d4980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:22:11.209 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.207+0000 7f99c13d4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:11.209 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.207+0000 7f99c13d4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:11.209 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.207+0000 7f99c13d4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:11.211 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.211+0000 7f99c13d4980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:11.211 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.211+0000 7f99c13d4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:11.211 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.211+0000 7f99c13d4980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:22:11.212 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.211+0000 7f99c13d4980 20 searching for the correct realm 2024-05-30T16:22:11.217 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.215+0000 7f99bf6e6640 0 -- 172.21.15.6:0/3089406178 >> [v2:172.21.15.204:6823/3156311527,v1:172.21.15.204:6825/3156311527] conn(0x7f99b408c1c0 msgr2=0x7f99b4059780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:11.223 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.223+0000 7f99beee5640 0 -- 172.21.15.6:0/3089406178 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f99b40851d0 msgr2=0x7f99b40858d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:11.441 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.439+0000 7f99c13d4980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:22:11.441 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.439+0000 7f99c13d4980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:22:11.441 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.439+0000 7f99c13d4980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:22:11.441 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.439+0000 7f99c13d4980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:22:11.441 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.439+0000 7f99c13d4980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:22:11.442 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.439+0000 7f99c13d4980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:22:11.442 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.439+0000 7f99c13d4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:11.442 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.443+0000 7f99c13d4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:11.442 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.443+0000 7f99c13d4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:11.442 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.443+0000 7f99c13d4980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:11.442 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.443+0000 7f99c13d4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:11.443 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.443+0000 7f99c13d4980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:22:11.444 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.443+0000 7f99c13d4980 20 zone default found 2024-05-30T16:22:11.444 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.443+0000 7f99c13d4980 4 Realm: () 2024-05-30T16:22:11.444 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.443+0000 7f99c13d4980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:22:11.444 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.443+0000 7f99c13d4980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:22:11.444 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.443+0000 7f99c13d4980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:22:11.444 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.443+0000 7f99c13d4980 20 zonegroup default 2024-05-30T16:22:11.444 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.443+0000 7f99c13d4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:11.445 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.443+0000 7f99c13d4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:11.445 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.443+0000 7f99c13d4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:11.445 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.443+0000 7f99c13d4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:11.445 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.443+0000 7f99c13d4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:11.446 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.447+0000 7f99c13d4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:11.446 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.447+0000 7f99c13d4980 20 started sync module instance, tier type = 2024-05-30T16:22:11.446 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.447+0000 7f99c13d4980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:22:11.468 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.467+0000 7f99c13d4980 20 add_watcher() i=0 2024-05-30T16:22:11.468 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.467+0000 7f99c13d4980 20 add_watcher() i=1 2024-05-30T16:22:11.468 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.467+0000 7f99c13d4980 20 add_watcher() i=2 2024-05-30T16:22:11.468 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.467+0000 7f99c13d4980 20 add_watcher() i=3 2024-05-30T16:22:11.468 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.467+0000 7f99c13d4980 20 add_watcher() i=4 2024-05-30T16:22:11.468 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.467+0000 7f99c13d4980 20 add_watcher() i=5 2024-05-30T16:22:11.468 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.467+0000 7f99c13d4980 20 add_watcher() i=6 2024-05-30T16:22:11.470 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.471+0000 7f99c13d4980 20 add_watcher() i=7 2024-05-30T16:22:11.471 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.471+0000 7f99c13d4980 2 all 8 watchers are set, enabling cache 2024-05-30T16:22:11.475 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.475+0000 7f98757e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:11.475 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.475+0000 7f99c13d4980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:22:11.475 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.475+0000 7f99c13d4980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:22:11.475 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.475+0000 7f99c13d4980 5 note: GC not initialized 2024-05-30T16:22:11.476 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.475+0000 7f97effff640 20 reqs_thread_entry: start 2024-05-30T16:22:11.476 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.475+0000 7f99c13d4980 20 init_complete bucket index max shards: 11 2024-05-30T16:22:11.476 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.475+0000 7f99c13d4980 20 Filter name: none 2024-05-30T16:22:11.476 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.475+0000 7f97edffb640 20 reqs_thread_entry: start 2024-05-30T16:22:11.476 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.475+0000 7f99c13d4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:11.477 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.475+0000 7f99c13d4980 20 rados_obj.operate() r=0 bl.length=221 2024-05-30T16:22:11.477 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.475+0000 7f99c13d4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:11.478 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.479+0000 7f99c13d4980 20 rados_obj.operate() r=0 bl.length=435 2024-05-30T16:22:11.478 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.479+0000 7f99c13d4980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:22:11.478 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.479+0000 7f99c13d4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:11.478 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.479+0000 7f99c13d4980 20 rados_obj.operate() r=0 bl.length=221 2024-05-30T16:22:11.479 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.479+0000 7f99c13d4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:11.480 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.479+0000 7f99c13d4980 20 rados_obj.operate() r=0 bl.length=435 2024-05-30T16:22:11.480 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.479+0000 7f99c13d4980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:22:11.483 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.483+0000 7f98757e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:11.488 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.487+0000 7f99c13d4980 20 remove_watcher() i=0 2024-05-30T16:22:11.488 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.487+0000 7f99c13d4980 2 removed watcher, disabling cache 2024-05-30T16:22:11.491 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.491+0000 7f99c13d4980 20 remove_watcher() i=1 2024-05-30T16:22:11.493 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.491+0000 7f99c13d4980 20 remove_watcher() i=2 2024-05-30T16:22:11.495 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.495+0000 7f99c13d4980 20 remove_watcher() i=3 2024-05-30T16:22:11.497 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.495+0000 7f99c13d4980 20 remove_watcher() i=4 2024-05-30T16:22:11.498 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.499+0000 7f99c13d4980 20 remove_watcher() i=5 2024-05-30T16:22:11.499 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.499+0000 7f99c13d4980 20 remove_watcher() i=6 2024-05-30T16:22:11.501 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:11.499+0000 7f99c13d4980 20 remove_watcher() i=7 2024-05-30T16:22:11.517 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:11.517 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:22:11.517 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: abort-at-commit-target-layout id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.27 num_objects: 19 size_kb: 1 num_shards: 12 2024-05-30T16:22:11.517 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:12.387 INFO:tasks.workunit.client.0.smithi006.stderr:TEST: reshard bucket with EIO injected at do_reshard 2024-05-30T16:22:12.387 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:14.501 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket error-at-do-reshard 2024-05-30T16:22:14.536 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:22:14.536 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:22:14.583 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.583+0000 7f04e440d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.585 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.583+0000 7f04e440d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:14.585 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.583+0000 7f04e440d980 20 realm 2024-05-30T16:22:14.585 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.583+0000 7f04e440d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.585 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.583+0000 7f04e440d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:14.585 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.583+0000 7f04e440d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:22:14.585 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.583+0000 7f04e440d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.586 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.583+0000 7f04e440d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:14.586 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.583+0000 7f04e440d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.587 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.587+0000 7f04e440d980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:14.587 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.587+0000 7f04e440d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.588 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.587+0000 7f04e440d980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:22:14.588 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.587+0000 7f04e440d980 20 searching for the correct realm 2024-05-30T16:22:14.594 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.595+0000 7f04e171d640 0 -- 172.21.15.6:0/304831480 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f04d4029070 msgr2=0x7f04d4063700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:14.601 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.599+0000 7f04e171d640 0 -- 172.21.15.6:0/304831480 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f03ac07e510 msgr2=0x7f03ac080940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:14.608 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.607+0000 7f04e440d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:22:14.608 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.607+0000 7f04e440d980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:22:14.608 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.607+0000 7f04e440d980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:22:14.608 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.607+0000 7f04e440d980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:22:14.608 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.607+0000 7f04e440d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:22:14.608 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.607+0000 7f04e440d980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:22:14.608 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.607+0000 7f04e440d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.608 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.607+0000 7f04e440d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:14.609 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.607+0000 7f04e440d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.609 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.607+0000 7f04e440d980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:14.609 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.607+0000 7f04e440d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.610 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.611+0000 7f04e440d980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:22:14.610 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.611+0000 7f04e440d980 20 zone default found 2024-05-30T16:22:14.610 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.611+0000 7f04e440d980 4 Realm: () 2024-05-30T16:22:14.610 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.611+0000 7f04e440d980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:22:14.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.611+0000 7f04e440d980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:22:14.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.611+0000 7f04e440d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:22:14.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.611+0000 7f04e440d980 20 zonegroup default 2024-05-30T16:22:14.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.611+0000 7f04e440d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.611+0000 7f04e440d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:14.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.611+0000 7f04e440d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.611+0000 7f04e440d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:14.612 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.611+0000 7f04e440d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.612 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.611+0000 7f04e440d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:14.612 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.611+0000 7f04e440d980 20 started sync module instance, tier type = 2024-05-30T16:22:14.612 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.611+0000 7f04e440d980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:22:14.630 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.627+0000 7f04e440d980 20 add_watcher() i=0 2024-05-30T16:22:14.630 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.627+0000 7f04e440d980 20 add_watcher() i=1 2024-05-30T16:22:14.630 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.627+0000 7f04e440d980 20 add_watcher() i=2 2024-05-30T16:22:14.630 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.627+0000 7f04e440d980 20 add_watcher() i=3 2024-05-30T16:22:14.630 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.627+0000 7f04e440d980 20 add_watcher() i=4 2024-05-30T16:22:14.630 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.627+0000 7f04e440d980 20 add_watcher() i=5 2024-05-30T16:22:14.630 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.627+0000 7f04e440d980 20 add_watcher() i=6 2024-05-30T16:22:14.630 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.631+0000 7f04e440d980 20 add_watcher() i=7 2024-05-30T16:22:14.631 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.631+0000 7f04e440d980 2 all 8 watchers are set, enabling cache 2024-05-30T16:22:14.633 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.631+0000 7f03937fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:14.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.635+0000 7f04e440d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:22:14.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.635+0000 7f04e440d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:22:14.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.635+0000 7f04e440d980 5 note: GC not initialized 2024-05-30T16:22:14.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.635+0000 7f03927fc640 20 reqs_thread_entry: start 2024-05-30T16:22:14.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.635+0000 7f04e440d980 20 init_complete bucket index max shards: 11 2024-05-30T16:22:14.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.635+0000 7f04e440d980 20 Filter name: none 2024-05-30T16:22:14.635 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.635+0000 7f0303fff640 20 reqs_thread_entry: start 2024-05-30T16:22:14.635 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.635+0000 7f04e440d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.635 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.635+0000 7f04e440d980 20 rados_obj.operate() r=0 bl.length=211 2024-05-30T16:22:14.635 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.635+0000 7f04e440d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.636 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.635+0000 7f04e440d980 20 rados_obj.operate() r=0 bl.length=425 2024-05-30T16:22:14.636 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.635+0000 7f04e440d980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:22:14.636 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.635+0000 7f04e440d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.636 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.635+0000 7f04e440d980 20 rados_obj.operate() r=0 bl.length=211 2024-05-30T16:22:14.636 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.635+0000 7f04e440d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.636 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.635+0000 7f04e440d980 20 rados_obj.operate() r=0 bl.length=425 2024-05-30T16:22:14.637 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.635+0000 7f04e440d980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:22:14.639 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.639+0000 7f03937fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:14.650 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.647+0000 7f04e440d980 20 remove_watcher() i=0 2024-05-30T16:22:14.650 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.647+0000 7f04e440d980 2 removed watcher, disabling cache 2024-05-30T16:22:14.650 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.651+0000 7f04e440d980 20 remove_watcher() i=1 2024-05-30T16:22:14.655 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.655+0000 7f04e440d980 20 remove_watcher() i=2 2024-05-30T16:22:14.657 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.655+0000 7f04e440d980 20 remove_watcher() i=3 2024-05-30T16:22:14.661 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.663+0000 7f04e440d980 20 remove_watcher() i=4 2024-05-30T16:22:14.663 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.663+0000 7f04e440d980 20 remove_watcher() i=5 2024-05-30T16:22:14.664 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.663+0000 7f04e440d980 20 remove_watcher() i=6 2024-05-30T16:22:14.666 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.667+0000 7f04e440d980 20 remove_watcher() i=7 2024-05-30T16:22:14.681 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:14.682 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:22:14.682 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: error-at-do-reshard id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-30T16:22:14.682 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:14.682 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket reshard --bucket error-at-do-reshard --num-shards 12 --rgw-reshard-bucket-lock-duration 30 --inject-error-at do_reshard 2024-05-30T16:22:14.713 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:22:14.713 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:22:14.757 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.755+0000 7f0b6b850980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.759 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.759+0000 7f0b6b850980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:14.759 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.759+0000 7f0b6b850980 20 realm 2024-05-30T16:22:14.759 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.759+0000 7f0b6b850980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.759 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.759+0000 7f0b6b850980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:14.759 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.759+0000 7f0b6b850980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:22:14.760 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.759+0000 7f0b6b850980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.760 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.759+0000 7f0b6b850980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:14.760 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.759+0000 7f0b6b850980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.761 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.759+0000 7f0b6b850980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:14.761 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.759+0000 7f0b6b850980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.761 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.759+0000 7f0b6b850980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:22:14.762 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.759+0000 7f0b6b850980 20 searching for the correct realm 2024-05-30T16:22:14.767 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.767+0000 7f0b68b60640 0 -- 172.21.15.6:0/774755868 >> [v2:172.21.15.6:6819/1141406589,v1:172.21.15.6:6821/1141406589] conn(0x55fce1094d90 msgr2=0x55fce1097250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:14.773 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.771+0000 7f0b69361640 0 -- 172.21.15.6:0/774755868 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f0b600f78b0 msgr2=0x7f0b600fa140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:22:14.782 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.783+0000 7f0b6b850980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:22:14.782 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.783+0000 7f0b6b850980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:22:14.782 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.783+0000 7f0b6b850980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:22:14.782 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.783+0000 7f0b6b850980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:22:14.783 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.783+0000 7f0b6b850980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:22:14.783 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.783+0000 7f0b6b850980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:22:14.783 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.783+0000 7f0b6b850980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.783 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.783+0000 7f0b6b850980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:14.783 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.783+0000 7f0b6b850980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.783 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.783+0000 7f0b6b850980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:14.783 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.783+0000 7f0b6b850980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.784 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.783+0000 7f0b6b850980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:22:14.784 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.783+0000 7f0b6b850980 20 zone default found 2024-05-30T16:22:14.784 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.783+0000 7f0b6b850980 4 Realm: () 2024-05-30T16:22:14.784 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.783+0000 7f0b6b850980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:22:14.784 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.783+0000 7f0b6b850980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:22:14.784 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.783+0000 7f0b6b850980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:22:14.784 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.783+0000 7f0b6b850980 20 zonegroup default 2024-05-30T16:22:14.784 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.783+0000 7f0b6b850980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.785 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.783+0000 7f0b6b850980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:14.785 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.783+0000 7f0b6b850980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.785 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.783+0000 7f0b6b850980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:14.786 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.783+0000 7f0b6b850980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.786 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.787+0000 7f0b6b850980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:14.786 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.787+0000 7f0b6b850980 20 started sync module instance, tier type = 2024-05-30T16:22:14.786 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.787+0000 7f0b6b850980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:22:14.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.803+0000 7f0b6b850980 20 add_watcher() i=0 2024-05-30T16:22:14.804 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.803+0000 7f0b6b850980 20 add_watcher() i=1 2024-05-30T16:22:14.804 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.803+0000 7f0b6b850980 20 add_watcher() i=2 2024-05-30T16:22:14.804 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.803+0000 7f0b6b850980 20 add_watcher() i=3 2024-05-30T16:22:14.804 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.803+0000 7f0b6b850980 20 add_watcher() i=4 2024-05-30T16:22:14.804 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.803+0000 7f0b6b850980 20 add_watcher() i=5 2024-05-30T16:22:14.804 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.803+0000 7f0b6b850980 20 add_watcher() i=6 2024-05-30T16:22:14.804 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.803+0000 7f0b6b850980 20 add_watcher() i=7 2024-05-30T16:22:14.804 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.803+0000 7f0b6b850980 2 all 8 watchers are set, enabling cache 2024-05-30T16:22:14.807 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.807+0000 7f0a1ffe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:14.808 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.807+0000 7f0b6b850980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:22:14.808 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.807+0000 7f0b6b850980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:22:14.808 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.807+0000 7f0b6b850980 5 note: GC not initialized 2024-05-30T16:22:14.808 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.807+0000 7f0a1efe5640 20 reqs_thread_entry: start 2024-05-30T16:22:14.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.807+0000 7f0b6b850980 20 init_complete bucket index max shards: 11 2024-05-30T16:22:14.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.807+0000 7f0b6b850980 20 Filter name: none 2024-05-30T16:22:14.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.807+0000 7f0a1cfe1640 20 reqs_thread_entry: start 2024-05-30T16:22:14.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.807+0000 7f0b6b850980 10 cache get: name=default.rgw.meta+root+error-at-do-reshard : miss 2024-05-30T16:22:14.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.807+0000 7f0b6b850980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.807+0000 7f0b6b850980 20 rados_obj.operate() r=0 bl.length=211 2024-05-30T16:22:14.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.807+0000 7f0b6b850980 10 cache put: name=default.rgw.meta+root+error-at-do-reshard info.flags=0x11 2024-05-30T16:22:14.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.807+0000 7f0b6b850980 10 adding default.rgw.meta+root+error-at-do-reshard to cache LRU end 2024-05-30T16:22:14.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.807+0000 7f0b6b850980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28 : miss 2024-05-30T16:22:14.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.807+0000 7f0b6b850980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.807+0000 7f0b6b850980 20 rados_obj.operate() r=0 bl.length=425 2024-05-30T16:22:14.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.807+0000 7f0b6b850980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28 info.flags=0x17 2024-05-30T16:22:14.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.807+0000 7f0b6b850980 10 adding default.rgw.meta+root+.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28 to cache LRU end 2024-05-30T16:22:14.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.807+0000 7f0b6b850980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:22:14.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.807+0000 7f0b6b850980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:22:14.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.807+0000 7f0b6b850980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28 2024-05-30T16:22:14.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.819+0000 7f0b6b850980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28 info.flags=0x17 2024-05-30T16:22:14.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.819+0000 7f0b6b850980 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28 to cache LRU end 2024-05-30T16:22:14.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.819+0000 7f0b6b850980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:22:14.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.819+0000 7f0b6b850980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28, ofs0, ns] 2024-05-30T16:22:14.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.819+0000 7f0a3bfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431630 cookie 94544595624176 notifier 6530 bl.length()=987 2024-05-30T16:22:14.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.819+0000 7f0a3bfff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28 info.flags=0x17 2024-05-30T16:22:14.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.819+0000 7f0a3bfff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28 to cache LRU end 2024-05-30T16:22:14.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.819+0000 7f0a3bfff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:22:14.822 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.823+0000 7f0b6b850980 20 handle_bi_update(): bucket=:error-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28]): orig_sources= new_sources= 2024-05-30T16:22:14.822 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.823+0000 7f0b6b850980 20 handle_bi_update(): bucket=:error-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28]): potential sources added=[] removed=[] 2024-05-30T16:22:14.822 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.823+0000 7f0b6b850980 20 handle_bi_update(): bucket=:error-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28]): orig_dests= new_dests= 2024-05-30T16:22:14.822 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.823+0000 7f0b6b850980 20 handle_bi_update(): bucket=:error-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28]): potential dests added=[] removed=[] 2024-05-30T16:22:14.827 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.827+0000 7f0b6b850980 -1 Injecting error=-5 at location=do_reshard 2024-05-30T16:22:14.838 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.839+0000 7f0b6b850980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28 : hit (requested=0x17, cached=0x17) 2024-05-30T16:22:14.838 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.839+0000 7f0b6b850980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28 2024-05-30T16:22:14.839 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.839+0000 7f0b6b850980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28 info.flags=0x17 2024-05-30T16:22:14.840 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.839+0000 7f0b6b850980 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28 to cache LRU end 2024-05-30T16:22:14.840 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.839+0000 7f0b6b850980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:22:14.840 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.839+0000 7f0b6b850980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28, ofs0, ns] 2024-05-30T16:22:14.840 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.839+0000 7f0a3bfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431631 cookie 94544595624176 notifier 6530 bl.length()=955 2024-05-30T16:22:14.840 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.839+0000 7f0a3bfff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28 info.flags=0x17 2024-05-30T16:22:14.840 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.839+0000 7f0a3bfff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28 to cache LRU end 2024-05-30T16:22:14.840 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.839+0000 7f0a3bfff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:22:14.842 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.843+0000 7f0b6b850980 20 handle_bi_update(): bucket=:error-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28]): orig_sources= new_sources= 2024-05-30T16:22:14.842 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.843+0000 7f0b6b850980 20 handle_bi_update(): bucket=:error-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28]): potential sources added=[] removed=[] 2024-05-30T16:22:14.842 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.843+0000 7f0b6b850980 20 handle_bi_update(): bucket=:error-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28]): orig_dests= new_dests= 2024-05-30T16:22:14.842 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.843+0000 7f0b6b850980 20 handle_bi_update(): bucket=:error-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28]): potential dests added=[] removed=[] 2024-05-30T16:22:14.842 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.843+0000 7f0b6b850980 1 execute INFO: reshard of bucket "error-at-do-reshard" canceled due to errors 2024-05-30T16:22:14.845 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.843+0000 7f0a1ffe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:14.850 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.851+0000 7f0b6b850980 20 remove_watcher() i=0 2024-05-30T16:22:14.850 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.851+0000 7f0b6b850980 2 removed watcher, disabling cache 2024-05-30T16:22:14.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.851+0000 7f0b6b850980 20 remove_watcher() i=1 2024-05-30T16:22:14.854 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.855+0000 7f0b6b850980 20 remove_watcher() i=2 2024-05-30T16:22:14.857 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.855+0000 7f0b6b850980 20 remove_watcher() i=3 2024-05-30T16:22:14.858 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.859+0000 7f0b6b850980 20 remove_watcher() i=4 2024-05-30T16:22:14.860 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.859+0000 7f0b6b850980 20 remove_watcher() i=5 2024-05-30T16:22:14.861 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.859+0000 7f0b6b850980 20 remove_watcher() i=6 2024-05-30T16:22:14.862 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.863+0000 7f0b6b850980 20 remove_watcher() i=7 2024-05-30T16:22:14.876 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket error-at-do-reshard 2024-05-30T16:22:14.905 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:22:14.905 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:22:14.950 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.951+0000 7f07f9f13980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.952 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.951+0000 7f07f9f13980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:14.952 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.951+0000 7f07f9f13980 20 realm 2024-05-30T16:22:14.952 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.951+0000 7f07f9f13980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.952 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.951+0000 7f07f9f13980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:14.952 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.951+0000 7f07f9f13980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:22:14.952 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.951+0000 7f07f9f13980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.952 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.951+0000 7f07f9f13980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:14.952 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.951+0000 7f07f9f13980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.954 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.955+0000 7f07f9f13980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:14.954 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.955+0000 7f07f9f13980 20 rados->read ofs=0 len=0 2024-05-30T16:22:14.954 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.955+0000 7f07f9f13980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:22:14.954 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.955+0000 7f07f9f13980 20 searching for the correct realm 2024-05-30T16:22:14.958 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:14.959+0000 7f07f7223640 0 -- 172.21.15.6:0/198797578 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f07e80db070 msgr2=0x7f07e80eb930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:15.174 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.175+0000 7f07f7a24640 0 -- 172.21.15.6:0/198797578 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f07ec064e90 msgr2=0x7f07ec0859c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:15.393 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.391+0000 7f07f9f13980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:22:15.393 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.391+0000 7f07f9f13980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:22:15.393 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.391+0000 7f07f9f13980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:22:15.393 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.391+0000 7f07f9f13980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:22:15.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.391+0000 7f07f9f13980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:22:15.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.391+0000 7f07f9f13980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:22:15.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.391+0000 7f07f9f13980 20 rados->read ofs=0 len=0 2024-05-30T16:22:15.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.395+0000 7f07f9f13980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:15.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.395+0000 7f07f9f13980 20 rados->read ofs=0 len=0 2024-05-30T16:22:15.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.395+0000 7f07f9f13980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:15.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.395+0000 7f07f9f13980 20 rados->read ofs=0 len=0 2024-05-30T16:22:15.396 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.395+0000 7f07f9f13980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:22:15.396 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.395+0000 7f07f9f13980 20 zone default found 2024-05-30T16:22:15.396 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.395+0000 7f07f9f13980 4 Realm: () 2024-05-30T16:22:15.396 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.395+0000 7f07f9f13980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:22:15.397 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.395+0000 7f07f9f13980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:22:15.397 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.395+0000 7f07f9f13980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:22:15.397 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.395+0000 7f07f9f13980 20 zonegroup default 2024-05-30T16:22:15.397 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.395+0000 7f07f9f13980 20 rados->read ofs=0 len=0 2024-05-30T16:22:15.397 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.395+0000 7f07f9f13980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:15.397 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.395+0000 7f07f9f13980 20 rados->read ofs=0 len=0 2024-05-30T16:22:15.398 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.399+0000 7f07f9f13980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:15.398 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.399+0000 7f07f9f13980 20 rados->read ofs=0 len=0 2024-05-30T16:22:15.399 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.399+0000 7f07f9f13980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:15.399 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.399+0000 7f07f9f13980 20 started sync module instance, tier type = 2024-05-30T16:22:15.400 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.399+0000 7f07f9f13980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:22:15.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.423+0000 7f07f9f13980 20 add_watcher() i=0 2024-05-30T16:22:15.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.423+0000 7f07f9f13980 20 add_watcher() i=1 2024-05-30T16:22:15.425 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.423+0000 7f07f9f13980 20 add_watcher() i=2 2024-05-30T16:22:15.425 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.423+0000 7f07f9f13980 20 add_watcher() i=3 2024-05-30T16:22:15.425 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.423+0000 7f07f9f13980 20 add_watcher() i=4 2024-05-30T16:22:15.425 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.423+0000 7f07f9f13980 20 add_watcher() i=5 2024-05-30T16:22:15.425 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.423+0000 7f07f9f13980 20 add_watcher() i=6 2024-05-30T16:22:15.426 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.427+0000 7f07f9f13980 20 add_watcher() i=7 2024-05-30T16:22:15.426 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.427+0000 7f07f9f13980 2 all 8 watchers are set, enabling cache 2024-05-30T16:22:15.433 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.431+0000 7f06adfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:15.434 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.435+0000 7f07f9f13980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:22:15.434 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.435+0000 7f07f9f13980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:22:15.434 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.435+0000 7f07f9f13980 5 note: GC not initialized 2024-05-30T16:22:15.435 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.435+0000 7f06acfe1640 20 reqs_thread_entry: start 2024-05-30T16:22:15.435 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.435+0000 7f07f9f13980 20 init_complete bucket index max shards: 11 2024-05-30T16:22:15.435 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.435+0000 7f07f9f13980 20 Filter name: none 2024-05-30T16:22:15.436 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.435+0000 7f06267fc640 20 reqs_thread_entry: start 2024-05-30T16:22:15.436 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.435+0000 7f07f9f13980 20 rados->read ofs=0 len=0 2024-05-30T16:22:15.436 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.435+0000 7f07f9f13980 20 rados_obj.operate() r=0 bl.length=211 2024-05-30T16:22:15.436 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.435+0000 7f07f9f13980 20 rados->read ofs=0 len=0 2024-05-30T16:22:15.437 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.435+0000 7f07f9f13980 20 rados_obj.operate() r=0 bl.length=425 2024-05-30T16:22:15.437 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.435+0000 7f07f9f13980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:22:15.437 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.435+0000 7f07f9f13980 20 rados->read ofs=0 len=0 2024-05-30T16:22:15.438 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.439+0000 7f07f9f13980 20 rados_obj.operate() r=0 bl.length=211 2024-05-30T16:22:15.438 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.439+0000 7f07f9f13980 20 rados->read ofs=0 len=0 2024-05-30T16:22:15.438 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.439+0000 7f07f9f13980 20 rados_obj.operate() r=0 bl.length=425 2024-05-30T16:22:15.438 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.439+0000 7f07f9f13980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:22:15.441 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.439+0000 7f06adfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:15.446 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.447+0000 7f07f9f13980 20 remove_watcher() i=0 2024-05-30T16:22:15.446 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.447+0000 7f07f9f13980 2 removed watcher, disabling cache 2024-05-30T16:22:15.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.447+0000 7f07f9f13980 20 remove_watcher() i=1 2024-05-30T16:22:15.451 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.451+0000 7f07f9f13980 20 remove_watcher() i=2 2024-05-30T16:22:15.453 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.451+0000 7f07f9f13980 20 remove_watcher() i=3 2024-05-30T16:22:15.455 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.455+0000 7f07f9f13980 20 remove_watcher() i=4 2024-05-30T16:22:15.457 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.455+0000 7f07f9f13980 20 remove_watcher() i=5 2024-05-30T16:22:15.459 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.459+0000 7f07f9f13980 20 remove_watcher() i=6 2024-05-30T16:22:15.461 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.459+0000 7f07f9f13980 20 remove_watcher() i=7 2024-05-30T16:22:15.486 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:15.486 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:22:15.486 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: error-at-do-reshard id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-30T16:22:15.486 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:15.607 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket reshard --bucket error-at-do-reshard --num-shards 12 --rgw-reshard-bucket-lock-duration 30 2024-05-30T16:22:15.644 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:22:15.644 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:22:15.691 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.691+0000 7f2982e2c980 20 rados->read ofs=0 len=0 2024-05-30T16:22:15.692 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.691+0000 7f2982e2c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:15.692 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.691+0000 7f2982e2c980 20 realm 2024-05-30T16:22:15.692 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.691+0000 7f2982e2c980 20 rados->read ofs=0 len=0 2024-05-30T16:22:15.693 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.691+0000 7f2982e2c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:15.693 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.691+0000 7f2982e2c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:22:15.693 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.691+0000 7f2982e2c980 20 rados->read ofs=0 len=0 2024-05-30T16:22:15.693 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.691+0000 7f2982e2c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:15.693 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.691+0000 7f2982e2c980 20 rados->read ofs=0 len=0 2024-05-30T16:22:15.695 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.695+0000 7f2982e2c980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:15.695 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.695+0000 7f2982e2c980 20 rados->read ofs=0 len=0 2024-05-30T16:22:15.696 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.695+0000 7f2982e2c980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:22:15.696 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.695+0000 7f2982e2c980 20 searching for the correct realm 2024-05-30T16:22:15.701 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.703+0000 7f298113e640 0 -- 172.21.15.6:0/2390990682 >> [v2:172.21.15.204:6823/3156311527,v1:172.21.15.204:6825/3156311527] conn(0x7f297409b400 msgr2=0x7f2974080a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:15.708 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:15.707+0000 7f298093d640 0 -- 172.21.15.6:0/2390990682 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f297408f5e0 msgr2=0x7f2974080250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:22:16.329 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.327+0000 7f2982e2c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:22:16.329 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.327+0000 7f2982e2c980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:22:16.329 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.327+0000 7f2982e2c980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:22:16.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.327+0000 7f2982e2c980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:22:16.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.327+0000 7f2982e2c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:22:16.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.327+0000 7f2982e2c980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:22:16.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.327+0000 7f2982e2c980 20 rados->read ofs=0 len=0 2024-05-30T16:22:16.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.331+0000 7f2982e2c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:16.330 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.331+0000 7f2982e2c980 20 rados->read ofs=0 len=0 2024-05-30T16:22:16.331 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.331+0000 7f2982e2c980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:16.331 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.331+0000 7f2982e2c980 20 rados->read ofs=0 len=0 2024-05-30T16:22:16.332 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.331+0000 7f2982e2c980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:22:16.332 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.331+0000 7f2982e2c980 20 zone default found 2024-05-30T16:22:16.332 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.331+0000 7f2982e2c980 4 Realm: () 2024-05-30T16:22:16.332 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.331+0000 7f2982e2c980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:22:16.332 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.331+0000 7f2982e2c980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:22:16.332 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.331+0000 7f2982e2c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:22:16.332 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.331+0000 7f2982e2c980 20 zonegroup default 2024-05-30T16:22:16.332 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.331+0000 7f2982e2c980 20 rados->read ofs=0 len=0 2024-05-30T16:22:16.333 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.331+0000 7f2982e2c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:16.333 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.331+0000 7f2982e2c980 20 rados->read ofs=0 len=0 2024-05-30T16:22:16.333 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.331+0000 7f2982e2c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:16.333 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.335+0000 7f2982e2c980 20 rados->read ofs=0 len=0 2024-05-30T16:22:16.334 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.335+0000 7f2982e2c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:16.334 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.335+0000 7f2982e2c980 20 started sync module instance, tier type = 2024-05-30T16:22:16.334 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.335+0000 7f2982e2c980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:22:16.358 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.359+0000 7f2982e2c980 20 add_watcher() i=0 2024-05-30T16:22:16.358 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.359+0000 7f2982e2c980 20 add_watcher() i=1 2024-05-30T16:22:16.358 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.359+0000 7f2982e2c980 20 add_watcher() i=2 2024-05-30T16:22:16.358 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.359+0000 7f2982e2c980 20 add_watcher() i=3 2024-05-30T16:22:16.358 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.359+0000 7f2982e2c980 20 add_watcher() i=4 2024-05-30T16:22:16.359 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.359+0000 7f2982e2c980 20 add_watcher() i=5 2024-05-30T16:22:16.359 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.359+0000 7f2982e2c980 20 add_watcher() i=6 2024-05-30T16:22:16.361 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.359+0000 7f2982e2c980 20 add_watcher() i=7 2024-05-30T16:22:16.361 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.359+0000 7f2982e2c980 2 all 8 watchers are set, enabling cache 2024-05-30T16:22:16.366 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.367+0000 7f2836fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:16.367 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.367+0000 7f2982e2c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:22:16.367 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.367+0000 7f2982e2c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:22:16.367 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.367+0000 7f2982e2c980 5 note: GC not initialized 2024-05-30T16:22:16.367 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.367+0000 7f2835fe3640 20 reqs_thread_entry: start 2024-05-30T16:22:16.367 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.367+0000 7f2982e2c980 20 init_complete bucket index max shards: 11 2024-05-30T16:22:16.368 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.367+0000 7f2982e2c980 20 Filter name: none 2024-05-30T16:22:16.368 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.367+0000 7f27a77fe640 20 reqs_thread_entry: start 2024-05-30T16:22:16.368 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.367+0000 7f2982e2c980 10 cache get: name=default.rgw.meta+root+error-at-do-reshard : miss 2024-05-30T16:22:16.368 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.367+0000 7f2982e2c980 20 rados->read ofs=0 len=0 2024-05-30T16:22:16.369 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.367+0000 7f2982e2c980 20 rados_obj.operate() r=0 bl.length=211 2024-05-30T16:22:16.369 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.367+0000 7f2982e2c980 10 cache put: name=default.rgw.meta+root+error-at-do-reshard info.flags=0x11 2024-05-30T16:22:16.369 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.367+0000 7f2982e2c980 10 adding default.rgw.meta+root+error-at-do-reshard to cache LRU end 2024-05-30T16:22:16.369 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.367+0000 7f2982e2c980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28 : miss 2024-05-30T16:22:16.369 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.367+0000 7f2982e2c980 20 rados->read ofs=0 len=0 2024-05-30T16:22:16.370 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.367+0000 7f2982e2c980 20 rados_obj.operate() r=0 bl.length=425 2024-05-30T16:22:16.370 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.367+0000 7f2982e2c980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28 info.flags=0x17 2024-05-30T16:22:16.370 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.367+0000 7f2982e2c980 10 adding default.rgw.meta+root+.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28 to cache LRU end 2024-05-30T16:22:16.370 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.367+0000 7f2982e2c980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:22:16.370 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.367+0000 7f2982e2c980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:22:16.370 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.367+0000 7f2982e2c980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28 2024-05-30T16:22:16.380 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.379+0000 7f2982e2c980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28 info.flags=0x17 2024-05-30T16:22:16.380 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.379+0000 7f2982e2c980 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28 to cache LRU end 2024-05-30T16:22:16.381 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.379+0000 7f2982e2c980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:22:16.381 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.379+0000 7f2982e2c980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28, ofs0, ns] 2024-05-30T16:22:16.381 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.379+0000 7f2857fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431632 cookie 93899185802832 notifier 6909 bl.length()=987 2024-05-30T16:22:16.381 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.379+0000 7f2857fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28 info.flags=0x17 2024-05-30T16:22:16.381 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.379+0000 7f2857fff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28 to cache LRU end 2024-05-30T16:22:16.381 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.379+0000 7f2857fff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:22:16.383 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.383+0000 7f2982e2c980 20 handle_bi_update(): bucket=:error-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28]): orig_sources= new_sources= 2024-05-30T16:22:16.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.383+0000 7f2982e2c980 20 handle_bi_update(): bucket=:error-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28]): potential sources added=[] removed=[] 2024-05-30T16:22:16.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.383+0000 7f2982e2c980 20 handle_bi_update(): bucket=:error-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28]): orig_dests= new_dests= 2024-05-30T16:22:16.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.383+0000 7f2982e2c980 20 handle_bi_update(): bucket=:error-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28]): potential dests added=[] removed=[] 2024-05-30T16:22:16.388 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.387+0000 7f2982e2c980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.1.0 2024-05-30T16:22:16.388 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.387+0000 7f2982e2c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.1.0 2024-05-30T16:22:16.388 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.387+0000 7f2982e2c980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.1.1 2024-05-30T16:22:16.389 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.387+0000 7f2982e2c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.1.1 2024-05-30T16:22:16.389 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.387+0000 7f2982e2c980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.1.2 2024-05-30T16:22:16.389 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.387+0000 7f2982e2c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.1.2 2024-05-30T16:22:16.389 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.387+0000 7f2982e2c980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.1.3 2024-05-30T16:22:16.389 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.387+0000 7f2982e2c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.1.3 2024-05-30T16:22:16.389 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.387+0000 7f2982e2c980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.1.4 2024-05-30T16:22:16.389 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.387+0000 7f2982e2c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.1.4 2024-05-30T16:22:16.389 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.387+0000 7f2982e2c980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.1.5 2024-05-30T16:22:16.389 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.387+0000 7f2982e2c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.1.5 2024-05-30T16:22:16.389 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.387+0000 7f2982e2c980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.1.6 2024-05-30T16:22:16.389 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.387+0000 7f2982e2c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.1.6 2024-05-30T16:22:16.389 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.387+0000 7f2982e2c980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.1.7 2024-05-30T16:22:16.390 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.387+0000 7f2982e2c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.1.7 2024-05-30T16:22:16.390 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.387+0000 7f2982e2c980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.1.8 2024-05-30T16:22:16.390 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.387+0000 7f2982e2c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.1.8 2024-05-30T16:22:16.390 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.387+0000 7f2982e2c980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.1.9 2024-05-30T16:22:16.390 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.387+0000 7f2982e2c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.1.9 2024-05-30T16:22:16.390 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.387+0000 7f2982e2c980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.1.10 2024-05-30T16:22:16.390 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.387+0000 7f2982e2c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.1.10 2024-05-30T16:22:16.390 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.387+0000 7f2982e2c980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.1.11 2024-05-30T16:22:16.390 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.387+0000 7f2982e2c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.1.11 2024-05-30T16:22:16.390 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.387+0000 7f2982e2c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.0 2024-05-30T16:22:16.391 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.387+0000 7f2982e2c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.1 2024-05-30T16:22:16.391 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.391+0000 7f2982e2c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.2 2024-05-30T16:22:16.392 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.391+0000 7f2982e2c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.3 2024-05-30T16:22:16.393 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.391+0000 7f2982e2c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.4 2024-05-30T16:22:16.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.395+0000 7f2982e2c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.5 2024-05-30T16:22:16.396 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.395+0000 7f2982e2c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.6 2024-05-30T16:22:16.396 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.395+0000 7f2982e2c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.7 2024-05-30T16:22:16.398 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.399+0000 7f2982e2c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.8 2024-05-30T16:22:16.399 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.399+0000 7f2982e2c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.9 2024-05-30T16:22:16.399 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.399+0000 7f2982e2c980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28.10 2024-05-30T16:22:16.407 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.407+0000 7f2982e2c980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28 : hit (requested=0x17, cached=0x17) 2024-05-30T16:22:16.407 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.407+0000 7f2982e2c980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28 2024-05-30T16:22:16.409 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.407+0000 7f2982e2c980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28 info.flags=0x17 2024-05-30T16:22:16.409 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.407+0000 7f2982e2c980 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28 to cache LRU end 2024-05-30T16:22:16.409 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.407+0000 7f2982e2c980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:22:16.409 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.407+0000 7f2982e2c980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28, ofs0, ns] 2024-05-30T16:22:16.410 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.411+0000 7f28577fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431633 cookie 93899185802832 notifier 6909 bl.length()=955 2024-05-30T16:22:16.410 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.411+0000 7f28577fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28 info.flags=0x17 2024-05-30T16:22:16.410 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.411+0000 7f28577fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28 to cache LRU end 2024-05-30T16:22:16.410 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.411+0000 7f28577fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:22:16.412 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.411+0000 7f2982e2c980 20 handle_bi_update(): bucket=:error-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28]): orig_sources= new_sources= 2024-05-30T16:22:16.412 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.411+0000 7f2982e2c980 20 handle_bi_update(): bucket=:error-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28]): potential sources added=[] removed=[] 2024-05-30T16:22:16.412 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.411+0000 7f2982e2c980 20 handle_bi_update(): bucket=:error-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28]): orig_dests= new_dests= 2024-05-30T16:22:16.412 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.411+0000 7f2982e2c980 20 handle_bi_update(): bucket=:error-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28]): potential dests added=[] removed=[] 2024-05-30T16:22:16.417 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.415+0000 7f2982e2c980 1 execute INFO: reshard of bucket "error-at-do-reshard" completed successfully 2024-05-30T16:22:16.421 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.419+0000 7f2836fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:16.425 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.427+0000 7f2982e2c980 20 remove_watcher() i=0 2024-05-30T16:22:16.426 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.427+0000 7f2982e2c980 2 removed watcher, disabling cache 2024-05-30T16:22:16.427 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.427+0000 7f2982e2c980 20 remove_watcher() i=1 2024-05-30T16:22:16.430 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.431+0000 7f2982e2c980 20 remove_watcher() i=2 2024-05-30T16:22:16.432 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.431+0000 7f2982e2c980 20 remove_watcher() i=3 2024-05-30T16:22:16.434 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.435+0000 7f2982e2c980 20 remove_watcher() i=4 2024-05-30T16:22:16.436 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.435+0000 7f2982e2c980 20 remove_watcher() i=5 2024-05-30T16:22:16.437 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.435+0000 7f2982e2c980 20 remove_watcher() i=6 2024-05-30T16:22:16.438 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.439+0000 7f2982e2c980 20 remove_watcher() i=7 2024-05-30T16:22:16.454 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket error-at-do-reshard 2024-05-30T16:22:16.483 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:22:16.483 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:22:16.528 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.527+0000 7f9b33c1e980 20 rados->read ofs=0 len=0 2024-05-30T16:22:16.530 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.531+0000 7f9b33c1e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:16.530 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.531+0000 7f9b33c1e980 20 realm 2024-05-30T16:22:16.530 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.531+0000 7f9b33c1e980 20 rados->read ofs=0 len=0 2024-05-30T16:22:16.530 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.531+0000 7f9b33c1e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:16.530 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.531+0000 7f9b33c1e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:22:16.530 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.531+0000 7f9b33c1e980 20 rados->read ofs=0 len=0 2024-05-30T16:22:16.530 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.531+0000 7f9b33c1e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:16.530 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.531+0000 7f9b33c1e980 20 rados->read ofs=0 len=0 2024-05-30T16:22:16.532 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.531+0000 7f9b33c1e980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:16.532 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.531+0000 7f9b33c1e980 20 rados->read ofs=0 len=0 2024-05-30T16:22:16.532 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.531+0000 7f9b33c1e980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:22:16.532 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.531+0000 7f9b33c1e980 20 searching for the correct realm 2024-05-30T16:22:16.538 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.539+0000 7f9b30f2e640 0 -- 172.21.15.6:0/1996664460 >> [v2:172.21.15.6:6819/1141406589,v1:172.21.15.6:6821/1141406589] conn(0x556e90ffd090 msgr2=0x556e90fff480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:16.545 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.543+0000 7f9b3172f640 0 -- 172.21.15.6:0/1996664460 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f9b280639b0 msgr2=0x7f9b280947e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:16.553 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.551+0000 7f9b33c1e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:22:16.554 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.551+0000 7f9b33c1e980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:22:16.554 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.551+0000 7f9b33c1e980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:22:16.554 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.551+0000 7f9b33c1e980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:22:16.554 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.551+0000 7f9b33c1e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:22:16.554 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.551+0000 7f9b33c1e980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:22:16.554 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.551+0000 7f9b33c1e980 20 rados->read ofs=0 len=0 2024-05-30T16:22:16.554 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.555+0000 7f9b33c1e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:16.554 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.555+0000 7f9b33c1e980 20 rados->read ofs=0 len=0 2024-05-30T16:22:16.554 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.555+0000 7f9b33c1e980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:16.555 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.555+0000 7f9b33c1e980 20 rados->read ofs=0 len=0 2024-05-30T16:22:16.555 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.555+0000 7f9b33c1e980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:22:16.556 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.555+0000 7f9b33c1e980 20 zone default found 2024-05-30T16:22:16.556 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.555+0000 7f9b33c1e980 4 Realm: () 2024-05-30T16:22:16.556 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.555+0000 7f9b33c1e980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:22:16.556 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.555+0000 7f9b33c1e980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:22:16.556 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.555+0000 7f9b33c1e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:22:16.556 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.555+0000 7f9b33c1e980 20 zonegroup default 2024-05-30T16:22:16.556 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.555+0000 7f9b33c1e980 20 rados->read ofs=0 len=0 2024-05-30T16:22:16.556 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.555+0000 7f9b33c1e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:16.556 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.555+0000 7f9b33c1e980 20 rados->read ofs=0 len=0 2024-05-30T16:22:16.557 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.555+0000 7f9b33c1e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:16.557 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.555+0000 7f9b33c1e980 20 rados->read ofs=0 len=0 2024-05-30T16:22:16.557 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.555+0000 7f9b33c1e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:16.557 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.555+0000 7f9b33c1e980 20 started sync module instance, tier type = 2024-05-30T16:22:16.557 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.555+0000 7f9b33c1e980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:22:16.572 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.571+0000 7f9b33c1e980 20 add_watcher() i=0 2024-05-30T16:22:16.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.571+0000 7f9b33c1e980 20 add_watcher() i=1 2024-05-30T16:22:16.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.571+0000 7f9b33c1e980 20 add_watcher() i=2 2024-05-30T16:22:16.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.571+0000 7f9b33c1e980 20 add_watcher() i=3 2024-05-30T16:22:16.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.571+0000 7f9b33c1e980 20 add_watcher() i=4 2024-05-30T16:22:16.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.571+0000 7f9b33c1e980 20 add_watcher() i=5 2024-05-30T16:22:16.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.571+0000 7f9b33c1e980 20 add_watcher() i=6 2024-05-30T16:22:16.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.575+0000 7f9b33c1e980 20 add_watcher() i=7 2024-05-30T16:22:16.574 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.575+0000 7f9b33c1e980 2 all 8 watchers are set, enabling cache 2024-05-30T16:22:16.576 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.575+0000 7f99e7fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:16.577 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.575+0000 7f9b33c1e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:22:16.577 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.575+0000 7f9b33c1e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:22:16.577 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.575+0000 7f9b33c1e980 5 note: GC not initialized 2024-05-30T16:22:16.577 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.575+0000 7f99e6fe5640 20 reqs_thread_entry: start 2024-05-30T16:22:16.577 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.575+0000 7f9b33c1e980 20 init_complete bucket index max shards: 11 2024-05-30T16:22:16.577 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.575+0000 7f9b33c1e980 20 Filter name: none 2024-05-30T16:22:16.578 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.575+0000 7f99e4fe1640 20 reqs_thread_entry: start 2024-05-30T16:22:16.578 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.575+0000 7f9b33c1e980 20 rados->read ofs=0 len=0 2024-05-30T16:22:16.578 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.579+0000 7f9b33c1e980 20 rados_obj.operate() r=0 bl.length=211 2024-05-30T16:22:16.579 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.579+0000 7f9b33c1e980 20 rados->read ofs=0 len=0 2024-05-30T16:22:16.579 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.579+0000 7f9b33c1e980 20 rados_obj.operate() r=0 bl.length=425 2024-05-30T16:22:16.579 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.579+0000 7f9b33c1e980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:22:16.579 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.579+0000 7f9b33c1e980 20 rados->read ofs=0 len=0 2024-05-30T16:22:16.579 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.579+0000 7f9b33c1e980 20 rados_obj.operate() r=0 bl.length=211 2024-05-30T16:22:16.579 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.579+0000 7f9b33c1e980 20 rados->read ofs=0 len=0 2024-05-30T16:22:16.579 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.579+0000 7f9b33c1e980 20 rados_obj.operate() r=0 bl.length=425 2024-05-30T16:22:16.579 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.579+0000 7f9b33c1e980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:22:16.582 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.583+0000 7f99e7fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:16.587 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.587+0000 7f9b33c1e980 20 remove_watcher() i=0 2024-05-30T16:22:16.587 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.587+0000 7f9b33c1e980 2 removed watcher, disabling cache 2024-05-30T16:22:16.589 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.587+0000 7f9b33c1e980 20 remove_watcher() i=1 2024-05-30T16:22:16.591 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.591+0000 7f9b33c1e980 20 remove_watcher() i=2 2024-05-30T16:22:16.593 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.591+0000 7f9b33c1e980 20 remove_watcher() i=3 2024-05-30T16:22:16.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.595+0000 7f9b33c1e980 20 remove_watcher() i=4 2024-05-30T16:22:16.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.595+0000 7f9b33c1e980 20 remove_watcher() i=5 2024-05-30T16:22:16.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.599+0000 7f9b33c1e980 20 remove_watcher() i=6 2024-05-30T16:22:16.599 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:16.599+0000 7f9b33c1e980 20 remove_watcher() i=7 2024-05-30T16:22:16.613 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:16.613 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:22:16.613 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: error-at-do-reshard id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.28 num_objects: 19 size_kb: 1 num_shards: 12 2024-05-30T16:22:16.613 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:17.400 INFO:tasks.workunit.client.0.smithi006.stderr:TEST: reshard bucket with abort at do_reshard 2024-05-30T16:22:17.400 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:19.501 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket abort-at-do-reshard 2024-05-30T16:22:19.536 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:22:19.537 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:22:19.584 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.583+0000 7f2aa466d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:19.586 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.587+0000 7f2aa466d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:19.586 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.587+0000 7f2aa466d980 20 realm 2024-05-30T16:22:19.586 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.587+0000 7f2aa466d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:19.586 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.587+0000 7f2aa466d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:19.586 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.587+0000 7f2aa466d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:22:19.586 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.587+0000 7f2aa466d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:19.586 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.587+0000 7f2aa466d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:19.586 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.587+0000 7f2aa466d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:19.587 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.587+0000 7f2aa466d980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:19.588 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.587+0000 7f2aa466d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:19.588 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.587+0000 7f2aa466d980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:22:19.588 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.587+0000 7f2aa466d980 20 searching for the correct realm 2024-05-30T16:22:19.593 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.591+0000 7f2a9bfff640 0 -- 172.21.15.6:0/3166968300 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f2a9411a040 msgr2=0x7f2a94128900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:19.600 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.599+0000 7f2aa217e640 0 -- 172.21.15.6:0/3166968300 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f2a8c05a040 msgr2=0x7f2a8c05a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:19.817 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.815+0000 7f2aa466d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:22:19.818 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.815+0000 7f2aa466d980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:22:19.818 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.815+0000 7f2aa466d980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:22:19.818 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.815+0000 7f2aa466d980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:22:19.818 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.815+0000 7f2aa466d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:22:19.818 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.815+0000 7f2aa466d980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:22:19.818 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.815+0000 7f2aa466d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:19.818 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.819+0000 7f2aa466d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:19.819 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.819+0000 7f2aa466d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:19.819 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.819+0000 7f2aa466d980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:19.819 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.819+0000 7f2aa466d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:19.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.819+0000 7f2aa466d980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:22:19.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.819+0000 7f2aa466d980 20 zone default found 2024-05-30T16:22:19.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.819+0000 7f2aa466d980 4 Realm: () 2024-05-30T16:22:19.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.819+0000 7f2aa466d980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:22:19.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.819+0000 7f2aa466d980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:22:19.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.819+0000 7f2aa466d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:22:19.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.819+0000 7f2aa466d980 20 zonegroup default 2024-05-30T16:22:19.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.819+0000 7f2aa466d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:19.821 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.819+0000 7f2aa466d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:19.821 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.819+0000 7f2aa466d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:19.821 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.819+0000 7f2aa466d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:19.821 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.819+0000 7f2aa466d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:19.822 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.823+0000 7f2aa466d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:19.822 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.823+0000 7f2aa466d980 20 started sync module instance, tier type = 2024-05-30T16:22:19.822 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.823+0000 7f2aa466d980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:22:19.847 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.847+0000 7f2aa466d980 20 add_watcher() i=0 2024-05-30T16:22:19.847 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.847+0000 7f2aa466d980 20 add_watcher() i=1 2024-05-30T16:22:19.847 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.847+0000 7f2aa466d980 20 add_watcher() i=2 2024-05-30T16:22:19.847 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.847+0000 7f2aa466d980 20 add_watcher() i=3 2024-05-30T16:22:19.847 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.847+0000 7f2aa466d980 20 add_watcher() i=4 2024-05-30T16:22:19.847 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.847+0000 7f2aa466d980 20 add_watcher() i=5 2024-05-30T16:22:19.847 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.847+0000 7f2aa466d980 20 add_watcher() i=6 2024-05-30T16:22:19.850 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.851+0000 7f2aa466d980 20 add_watcher() i=7 2024-05-30T16:22:19.850 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.851+0000 7f2aa466d980 2 all 8 watchers are set, enabling cache 2024-05-30T16:22:19.854 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.855+0000 7f2958fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:19.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.855+0000 7f2aa466d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:22:19.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.855+0000 7f2aa466d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:22:19.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.855+0000 7f2aa466d980 5 note: GC not initialized 2024-05-30T16:22:19.856 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.855+0000 7f28d37fe640 20 reqs_thread_entry: start 2024-05-30T16:22:19.856 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.855+0000 7f2aa466d980 20 init_complete bucket index max shards: 11 2024-05-30T16:22:19.856 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.855+0000 7f2aa466d980 20 Filter name: none 2024-05-30T16:22:19.856 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.855+0000 7f28d17fa640 20 reqs_thread_entry: start 2024-05-30T16:22:19.856 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.855+0000 7f2aa466d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:19.857 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.859+0000 7f2aa466d980 20 rados_obj.operate() r=0 bl.length=211 2024-05-30T16:22:19.857 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.859+0000 7f2aa466d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:19.859 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.859+0000 7f2aa466d980 20 rados_obj.operate() r=0 bl.length=425 2024-05-30T16:22:19.859 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.859+0000 7f2aa466d980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:22:19.859 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.859+0000 7f2aa466d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:19.859 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.859+0000 7f2aa466d980 20 rados_obj.operate() r=0 bl.length=211 2024-05-30T16:22:19.859 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.859+0000 7f2aa466d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:19.860 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.859+0000 7f2aa466d980 20 rados_obj.operate() r=0 bl.length=425 2024-05-30T16:22:19.860 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.859+0000 7f2aa466d980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:22:19.863 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.863+0000 7f2958fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:19.868 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.867+0000 7f2aa466d980 20 remove_watcher() i=0 2024-05-30T16:22:19.868 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.867+0000 7f2aa466d980 2 removed watcher, disabling cache 2024-05-30T16:22:19.871 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.871+0000 7f2aa466d980 20 remove_watcher() i=1 2024-05-30T16:22:19.873 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.871+0000 7f2aa466d980 20 remove_watcher() i=2 2024-05-30T16:22:19.875 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.875+0000 7f2aa466d980 20 remove_watcher() i=3 2024-05-30T16:22:19.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.879+0000 7f2aa466d980 20 remove_watcher() i=4 2024-05-30T16:22:19.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.879+0000 7f2aa466d980 20 remove_watcher() i=5 2024-05-30T16:22:19.880 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.879+0000 7f2aa466d980 20 remove_watcher() i=6 2024-05-30T16:22:19.882 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.883+0000 7f2aa466d980 20 remove_watcher() i=7 2024-05-30T16:22:19.900 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:19.900 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:22:19.900 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: abort-at-do-reshard id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-30T16:22:19.900 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:19.900 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket reshard --bucket abort-at-do-reshard --num-shards 12 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at do_reshard 2024-05-30T16:22:19.935 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:22:19.935 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:22:19.981 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.979+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:19.983 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.983+0000 7f6942fc2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:19.983 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.983+0000 7f6942fc2980 20 realm 2024-05-30T16:22:19.983 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.983+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:19.983 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.983+0000 7f6942fc2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:19.983 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.983+0000 7f6942fc2980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:22:19.983 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.983+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:19.983 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.983+0000 7f6942fc2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:19.984 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.983+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:19.985 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.983+0000 7f6942fc2980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:19.985 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.983+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:19.985 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.983+0000 7f6942fc2980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:22:19.985 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.983+0000 7f6942fc2980 20 searching for the correct realm 2024-05-30T16:22:19.989 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:19.991+0000 7f69402d2640 0 -- 172.21.15.6:0/1226585167 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f6930045040 msgr2=0x7f6930045420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:20.207 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.207+0000 7f6940ad3640 0 -- 172.21.15.6:0/1226585167 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f69340093d0 msgr2=0x7f69340c56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:20.425 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.423+0000 7f6942fc2980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:22:20.425 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.423+0000 7f6942fc2980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:22:20.425 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.423+0000 7f6942fc2980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:22:20.426 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.423+0000 7f6942fc2980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:22:20.426 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.423+0000 7f6942fc2980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:22:20.426 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.423+0000 7f6942fc2980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:22:20.426 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.423+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.426 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.427+0000 7f6942fc2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:20.426 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.427+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.427 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.427+0000 7f6942fc2980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:20.427 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.427+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.428 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.427+0000 7f6942fc2980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:22:20.428 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.427+0000 7f6942fc2980 20 zone default found 2024-05-30T16:22:20.428 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.427+0000 7f6942fc2980 4 Realm: () 2024-05-30T16:22:20.428 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.427+0000 7f6942fc2980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:22:20.428 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.427+0000 7f6942fc2980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:22:20.428 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.427+0000 7f6942fc2980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:22:20.428 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.427+0000 7f6942fc2980 20 zonegroup default 2024-05-30T16:22:20.428 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.427+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.428 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.427+0000 7f6942fc2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:20.429 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.427+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.429 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.427+0000 7f6942fc2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:20.429 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.427+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.430 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.431+0000 7f6942fc2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:20.430 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.431+0000 7f6942fc2980 20 started sync module instance, tier type = 2024-05-30T16:22:20.430 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.431+0000 7f6942fc2980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:22:20.452 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.451+0000 7f6942fc2980 20 add_watcher() i=0 2024-05-30T16:22:20.452 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.451+0000 7f6942fc2980 20 add_watcher() i=1 2024-05-30T16:22:20.452 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.451+0000 7f6942fc2980 20 add_watcher() i=2 2024-05-30T16:22:20.452 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.451+0000 7f6942fc2980 20 add_watcher() i=3 2024-05-30T16:22:20.452 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.451+0000 7f6942fc2980 20 add_watcher() i=4 2024-05-30T16:22:20.453 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.451+0000 7f6942fc2980 20 add_watcher() i=5 2024-05-30T16:22:20.453 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.451+0000 7f6942fc2980 20 add_watcher() i=6 2024-05-30T16:22:20.454 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.455+0000 7f6942fc2980 20 add_watcher() i=7 2024-05-30T16:22:20.455 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.455+0000 7f6942fc2980 2 all 8 watchers are set, enabling cache 2024-05-30T16:22:20.459 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.459+0000 7f67f6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:20.460 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.459+0000 7f6942fc2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:22:20.460 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.459+0000 7f6942fc2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:22:20.460 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.459+0000 7f6942fc2980 5 note: GC not initialized 2024-05-30T16:22:20.460 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.459+0000 7f67f5fe3640 20 reqs_thread_entry: start 2024-05-30T16:22:20.460 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.459+0000 7f6942fc2980 20 init_complete bucket index max shards: 11 2024-05-30T16:22:20.460 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.459+0000 7f6942fc2980 20 Filter name: none 2024-05-30T16:22:20.460 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.459+0000 7f67677fe640 20 reqs_thread_entry: start 2024-05-30T16:22:20.460 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.459+0000 7f6942fc2980 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2024-05-30T16:22:20.461 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.459+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.461 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.459+0000 7f6942fc2980 20 rados_obj.operate() r=0 bl.length=211 2024-05-30T16:22:20.461 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.459+0000 7f6942fc2980 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2024-05-30T16:22:20.461 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.459+0000 7f6942fc2980 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2024-05-30T16:22:20.461 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.459+0000 7f6942fc2980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 : miss 2024-05-30T16:22:20.461 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.459+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.462 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.463+0000 7f6942fc2980 20 rados_obj.operate() r=0 bl.length=425 2024-05-30T16:22:20.462 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.463+0000 7f6942fc2980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 info.flags=0x17 2024-05-30T16:22:20.462 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.463+0000 7f6942fc2980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 to cache LRU end 2024-05-30T16:22:20.462 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.463+0000 7f6942fc2980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:22:20.462 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.463+0000 7f6942fc2980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:22:20.462 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.463+0000 7f6942fc2980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 2024-05-30T16:22:20.472 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.471+0000 7f6942fc2980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 info.flags=0x17 2024-05-30T16:22:20.472 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.471+0000 7f6942fc2980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 to cache LRU end 2024-05-30T16:22:20.472 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.471+0000 7f6942fc2980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:22:20.472 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.471+0000 7f6942fc2980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29, ofs0, ns] 2024-05-30T16:22:20.472 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.471+0000 7f6817fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431637 cookie 94387223527648 notifier 6966 bl.length()=987 2024-05-30T16:22:20.472 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.471+0000 7f6817fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 info.flags=0x17 2024-05-30T16:22:20.473 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.471+0000 7f6817fff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 to cache LRU end 2024-05-30T16:22:20.473 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.471+0000 7f6817fff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:22:20.474 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.475+0000 7f6942fc2980 20 handle_bi_update(): bucket=:abort-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29]): orig_sources= new_sources= 2024-05-30T16:22:20.474 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.475+0000 7f6942fc2980 20 handle_bi_update(): bucket=:abort-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29]): potential sources added=[] removed=[] 2024-05-30T16:22:20.474 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.475+0000 7f6942fc2980 20 handle_bi_update(): bucket=:abort-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29]): orig_dests= new_dests= 2024-05-30T16:22:20.474 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.475+0000 7f6942fc2980 20 handle_bi_update(): bucket=:abort-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29]): potential dests added=[] removed=[] 2024-05-30T16:22:20.479 INFO:tasks.workunit.client.0.smithi006.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f6942fc2980 time 2024-05-30T16:22:20.482496+0000 2024-05-30T16:22:20.479 INFO:tasks.workunit.client.0.smithi006.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-30T16:22:20.480 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:22:20.481 INFO:tasks.workunit.client.0.smithi006.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f6946899cff] 2024-05-30T16:22:20.481 INFO:tasks.workunit.client.0.smithi006.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7f6946899eb3] 2024-05-30T16:22:20.481 INFO:tasks.workunit.client.0.smithi006.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55d83b4d0f81] 2024-05-30T16:22:20.481 INFO:tasks.workunit.client.0.smithi006.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x31b) [0x55d83b4e2f1b] 2024-05-30T16:22:20.481 INFO:tasks.workunit.client.0.smithi006.stderr: 5: main() 2024-05-30T16:22:20.481 INFO:tasks.workunit.client.0.smithi006.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f6945702d90] 2024-05-30T16:22:20.481 INFO:tasks.workunit.client.0.smithi006.stderr: 7: __libc_start_main() 2024-05-30T16:22:20.481 INFO:tasks.workunit.client.0.smithi006.stderr: 8: _start() 2024-05-30T16:22:20.481 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.479+0000 7f6942fc2980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f6942fc2980 time 2024-05-30T16:22:20.482496+0000 2024-05-30T16:22:20.481 INFO:tasks.workunit.client.0.smithi006.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-30T16:22:20.481 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.481 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:22:20.482 INFO:tasks.workunit.client.0.smithi006.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f6946899cff] 2024-05-30T16:22:20.482 INFO:tasks.workunit.client.0.smithi006.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7f6946899eb3] 2024-05-30T16:22:20.482 INFO:tasks.workunit.client.0.smithi006.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55d83b4d0f81] 2024-05-30T16:22:20.482 INFO:tasks.workunit.client.0.smithi006.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x31b) [0x55d83b4e2f1b] 2024-05-30T16:22:20.482 INFO:tasks.workunit.client.0.smithi006.stderr: 5: main() 2024-05-30T16:22:20.482 INFO:tasks.workunit.client.0.smithi006.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f6945702d90] 2024-05-30T16:22:20.482 INFO:tasks.workunit.client.0.smithi006.stderr: 7: __libc_start_main() 2024-05-30T16:22:20.482 INFO:tasks.workunit.client.0.smithi006.stderr: 8: _start() 2024-05-30T16:22:20.482 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.482 INFO:tasks.workunit.client.0.smithi006.stderr:*** Caught signal (Aborted) ** 2024-05-30T16:22:20.482 INFO:tasks.workunit.client.0.smithi006.stderr: in thread 7f6942fc2980 thread_name:radosgw-admin 2024-05-30T16:22:20.501 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:22:20.501 INFO:tasks.workunit.client.0.smithi006.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f694571b520] 2024-05-30T16:22:20.501 INFO:tasks.workunit.client.0.smithi006.stderr: 2: pthread_kill() 2024-05-30T16:22:20.501 INFO:tasks.workunit.client.0.smithi006.stderr: 3: raise() 2024-05-30T16:22:20.501 INFO:tasks.workunit.client.0.smithi006.stderr: 4: abort() 2024-05-30T16:22:20.501 INFO:tasks.workunit.client.0.smithi006.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f6946899d59] 2024-05-30T16:22:20.501 INFO:tasks.workunit.client.0.smithi006.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7f6946899eb3] 2024-05-30T16:22:20.501 INFO:tasks.workunit.client.0.smithi006.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55d83b4d0f81] 2024-05-30T16:22:20.501 INFO:tasks.workunit.client.0.smithi006.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x31b) [0x55d83b4e2f1b] 2024-05-30T16:22:20.501 INFO:tasks.workunit.client.0.smithi006.stderr: 9: main() 2024-05-30T16:22:20.501 INFO:tasks.workunit.client.0.smithi006.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f6945702d90] 2024-05-30T16:22:20.502 INFO:tasks.workunit.client.0.smithi006.stderr: 11: __libc_start_main() 2024-05-30T16:22:20.502 INFO:tasks.workunit.client.0.smithi006.stderr: 12: _start() 2024-05-30T16:22:20.502 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.479+0000 7f6942fc2980 -1 *** Caught signal (Aborted) ** 2024-05-30T16:22:20.502 INFO:tasks.workunit.client.0.smithi006.stderr: in thread 7f6942fc2980 thread_name:radosgw-admin 2024-05-30T16:22:20.502 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.502 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:22:20.502 INFO:tasks.workunit.client.0.smithi006.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f694571b520] 2024-05-30T16:22:20.502 INFO:tasks.workunit.client.0.smithi006.stderr: 2: pthread_kill() 2024-05-30T16:22:20.502 INFO:tasks.workunit.client.0.smithi006.stderr: 3: raise() 2024-05-30T16:22:20.502 INFO:tasks.workunit.client.0.smithi006.stderr: 4: abort() 2024-05-30T16:22:20.502 INFO:tasks.workunit.client.0.smithi006.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f6946899d59] 2024-05-30T16:22:20.502 INFO:tasks.workunit.client.0.smithi006.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7f6946899eb3] 2024-05-30T16:22:20.503 INFO:tasks.workunit.client.0.smithi006.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55d83b4d0f81] 2024-05-30T16:22:20.503 INFO:tasks.workunit.client.0.smithi006.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x31b) [0x55d83b4e2f1b] 2024-05-30T16:22:20.503 INFO:tasks.workunit.client.0.smithi006.stderr: 9: main() 2024-05-30T16:22:20.503 INFO:tasks.workunit.client.0.smithi006.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f6945702d90] 2024-05-30T16:22:20.503 INFO:tasks.workunit.client.0.smithi006.stderr: 11: __libc_start_main() 2024-05-30T16:22:20.503 INFO:tasks.workunit.client.0.smithi006.stderr: 12: _start() 2024-05-30T16:22:20.503 INFO:tasks.workunit.client.0.smithi006.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-30T16:22:20.503 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.503 INFO:tasks.workunit.client.0.smithi006.stderr:--- begin dump of recent events --- 2024-05-30T16:22:20.503 INFO:tasks.workunit.client.0.smithi006.stderr: -499> 2024-05-30T16:22:19.959+0000 7f682ee0d640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-30T16:22:20.503 INFO:tasks.workunit.client.0.smithi006.stderr: -498> 2024-05-30T16:22:19.959+0000 7f682ee0d640 10 monclient: dump: 2024-05-30T16:22:20.503 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:22:20.503 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:22:20.504 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.504 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.504 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:22:20.504 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:22:20.504 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:22:20.504 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:22:20.504 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:22:20.504 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.504 INFO:tasks.workunit.client.0.smithi006.stderr: -497> 2024-05-30T16:22:19.959+0000 7f682ee0d640 10 monclient: _finish_auth 0 2024-05-30T16:22:20.504 INFO:tasks.workunit.client.0.smithi006.stderr: -496> 2024-05-30T16:22:19.959+0000 7f682ee0d640 10 monclient: _check_auth_tickets 2024-05-30T16:22:20.504 INFO:tasks.workunit.client.0.smithi006.stderr: -495> 2024-05-30T16:22:19.959+0000 7f682ee0d640 10 monclient: handle_config config(0 keys) 2024-05-30T16:22:20.504 INFO:tasks.workunit.client.0.smithi006.stderr: -494> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 monclient: authenticate success, global_id 6605 2024-05-30T16:22:20.504 INFO:tasks.workunit.client.0.smithi006.stderr: -493> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient: get_monmap_and_config success 2024-05-30T16:22:20.505 INFO:tasks.workunit.client.0.smithi006.stderr: -492> 2024-05-30T16:22:19.959+0000 7f6942fc2980 4 set_mon_vals no callback set 2024-05-30T16:22:20.505 INFO:tasks.workunit.client.0.smithi006.stderr: -491> 2024-05-30T16:22:19.959+0000 7f682ee0d640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:22:20.505 INFO:tasks.workunit.client.0.smithi006.stderr: -490> 2024-05-30T16:22:19.959+0000 7f682ee0d640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-30T16:22:20.505 INFO:tasks.workunit.client.0.smithi006.stderr: -489> 2024-05-30T16:22:19.959+0000 7f682ee0d640 10 monclient: dump: 2024-05-30T16:22:20.505 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:22:20.505 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:22:20.505 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.505 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.505 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:22:20.505 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:22:20.505 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:22:20.505 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:22:20.505 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:22:20.506 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.506 INFO:tasks.workunit.client.0.smithi006.stderr: -488> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient: shutdown 2024-05-30T16:22:20.506 INFO:tasks.workunit.client.0.smithi006.stderr: -487> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 asok(0x55d83cbda790) unregister_commands rotate-key 2024-05-30T16:22:20.506 INFO:tasks.workunit.client.0.smithi006.stderr: -486> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient: build_initial_monmap 2024-05-30T16:22:20.506 INFO:tasks.workunit.client.0.smithi006.stderr: -485> 2024-05-30T16:22:19.959+0000 7f6942fc2980 1 build_initial for_mkfs: 0 2024-05-30T16:22:20.506 INFO:tasks.workunit.client.0.smithi006.stderr: -484> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient: monmap: 2024-05-30T16:22:20.506 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 0 2024-05-30T16:22:20.506 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-30T16:22:20.506 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 0.000000 2024-05-30T16:22:20.506 INFO:tasks.workunit.client.0.smithi006.stderr:created 0.000000 2024-05-30T16:22:20.506 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 0 (unknown) 2024-05-30T16:22:20.506 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:22:20.506 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.noname-a 2024-05-30T16:22:20.507 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-b 2024-05-30T16:22:20.507 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.noname-c 2024-05-30T16:22:20.507 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.507 INFO:tasks.workunit.client.0.smithi006.stderr: -483> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cdea080) adding auth protocol: cephx 2024-05-30T16:22:20.507 INFO:tasks.workunit.client.0.smithi006.stderr: -482> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cdea080) adding auth protocol: cephx 2024-05-30T16:22:20.507 INFO:tasks.workunit.client.0.smithi006.stderr: -481> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cdea080) adding auth protocol: cephx 2024-05-30T16:22:20.507 INFO:tasks.workunit.client.0.smithi006.stderr: -480> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cdea080) adding con mode: secure 2024-05-30T16:22:20.507 INFO:tasks.workunit.client.0.smithi006.stderr: -479> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cdea080) adding con mode: crc 2024-05-30T16:22:20.507 INFO:tasks.workunit.client.0.smithi006.stderr: -478> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cdea080) adding con mode: secure 2024-05-30T16:22:20.507 INFO:tasks.workunit.client.0.smithi006.stderr: -477> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cdea080) adding con mode: crc 2024-05-30T16:22:20.507 INFO:tasks.workunit.client.0.smithi006.stderr: -476> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cdea080) adding con mode: secure 2024-05-30T16:22:20.507 INFO:tasks.workunit.client.0.smithi006.stderr: -475> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cdea080) adding con mode: crc 2024-05-30T16:22:20.507 INFO:tasks.workunit.client.0.smithi006.stderr: -474> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cdea080) adding con mode: crc 2024-05-30T16:22:20.508 INFO:tasks.workunit.client.0.smithi006.stderr: -473> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cdea080) adding con mode: secure 2024-05-30T16:22:20.508 INFO:tasks.workunit.client.0.smithi006.stderr: -472> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cdea080) adding con mode: crc 2024-05-30T16:22:20.508 INFO:tasks.workunit.client.0.smithi006.stderr: -471> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cdea080) adding con mode: secure 2024-05-30T16:22:20.508 INFO:tasks.workunit.client.0.smithi006.stderr: -470> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cdea080) adding con mode: crc 2024-05-30T16:22:20.508 INFO:tasks.workunit.client.0.smithi006.stderr: -469> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cdea080) adding con mode: secure 2024-05-30T16:22:20.508 INFO:tasks.workunit.client.0.smithi006.stderr: -468> 2024-05-30T16:22:19.959+0000 7f6942fc2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:22:20.508 INFO:tasks.workunit.client.0.smithi006.stderr: -467> 2024-05-30T16:22:19.959+0000 7f6942fc2980 1 librados: starting msgr at 2024-05-30T16:22:20.508 INFO:tasks.workunit.client.0.smithi006.stderr: -466> 2024-05-30T16:22:19.959+0000 7f6942fc2980 1 librados: starting objecter 2024-05-30T16:22:20.508 INFO:tasks.workunit.client.0.smithi006.stderr: -465> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command objecter_requests hook 0x55d83cc9fa10 2024-05-30T16:22:20.508 INFO:tasks.workunit.client.0.smithi006.stderr: -464> 2024-05-30T16:22:19.959+0000 7f6942fc2980 1 librados: setting wanted keys 2024-05-30T16:22:20.508 INFO:tasks.workunit.client.0.smithi006.stderr: -463> 2024-05-30T16:22:19.959+0000 7f6942fc2980 1 librados: calling monclient init 2024-05-30T16:22:20.508 INFO:tasks.workunit.client.0.smithi006.stderr: -462> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient: init 2024-05-30T16:22:20.508 INFO:tasks.workunit.client.0.smithi006.stderr: -461> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cde8290) adding auth protocol: cephx 2024-05-30T16:22:20.509 INFO:tasks.workunit.client.0.smithi006.stderr: -460> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cde8290) adding auth protocol: cephx 2024-05-30T16:22:20.509 INFO:tasks.workunit.client.0.smithi006.stderr: -459> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cde8290) adding auth protocol: cephx 2024-05-30T16:22:20.509 INFO:tasks.workunit.client.0.smithi006.stderr: -458> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cde8290) adding con mode: secure 2024-05-30T16:22:20.509 INFO:tasks.workunit.client.0.smithi006.stderr: -457> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cde8290) adding con mode: crc 2024-05-30T16:22:20.509 INFO:tasks.workunit.client.0.smithi006.stderr: -456> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cde8290) adding con mode: secure 2024-05-30T16:22:20.509 INFO:tasks.workunit.client.0.smithi006.stderr: -455> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cde8290) adding con mode: crc 2024-05-30T16:22:20.509 INFO:tasks.workunit.client.0.smithi006.stderr: -454> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cde8290) adding con mode: secure 2024-05-30T16:22:20.509 INFO:tasks.workunit.client.0.smithi006.stderr: -453> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cde8290) adding con mode: crc 2024-05-30T16:22:20.509 INFO:tasks.workunit.client.0.smithi006.stderr: -452> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cde8290) adding con mode: crc 2024-05-30T16:22:20.509 INFO:tasks.workunit.client.0.smithi006.stderr: -451> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cde8290) adding con mode: secure 2024-05-30T16:22:20.509 INFO:tasks.workunit.client.0.smithi006.stderr: -450> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cde8290) adding con mode: crc 2024-05-30T16:22:20.509 INFO:tasks.workunit.client.0.smithi006.stderr: -449> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cde8290) adding con mode: secure 2024-05-30T16:22:20.509 INFO:tasks.workunit.client.0.smithi006.stderr: -448> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cde8290) adding con mode: crc 2024-05-30T16:22:20.510 INFO:tasks.workunit.client.0.smithi006.stderr: -447> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cde8290) adding con mode: secure 2024-05-30T16:22:20.510 INFO:tasks.workunit.client.0.smithi006.stderr: -446> 2024-05-30T16:22:19.959+0000 7f6942fc2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:22:20.510 INFO:tasks.workunit.client.0.smithi006.stderr: -445> 2024-05-30T16:22:19.959+0000 7f6942fc2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:22:20.510 INFO:tasks.workunit.client.0.smithi006.stderr: -444> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command rotate-key hook 0x55d83cde83d8 2024-05-30T16:22:20.510 INFO:tasks.workunit.client.0.smithi006.stderr: -443> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient: _reopen_session rank -1 2024-05-30T16:22:20.510 INFO:tasks.workunit.client.0.smithi006.stderr: -442> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient: _add_conns ranks=[1,2,0] 2024-05-30T16:22:20.510 INFO:tasks.workunit.client.0.smithi006.stderr: -441> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient(hunting): picked mon.noname-b con 0x55d83cca14d0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:22:20.510 INFO:tasks.workunit.client.0.smithi006.stderr: -440> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient(hunting): picked mon.noname-c con 0x55d83cdeb690 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:22:20.510 INFO:tasks.workunit.client.0.smithi006.stderr: -439> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient(hunting): picked mon.noname-a con 0x55d83ce29a50 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:22:20.510 INFO:tasks.workunit.client.0.smithi006.stderr: -438> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient(hunting): start opening mon connection 2024-05-30T16:22:20.510 INFO:tasks.workunit.client.0.smithi006.stderr: -437> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient(hunting): start opening mon connection 2024-05-30T16:22:20.510 INFO:tasks.workunit.client.0.smithi006.stderr: -436> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient(hunting): start opening mon connection 2024-05-30T16:22:20.510 INFO:tasks.workunit.client.0.smithi006.stderr: -435> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient(hunting): _renew_subs 2024-05-30T16:22:20.511 INFO:tasks.workunit.client.0.smithi006.stderr: -434> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient(hunting): authenticate will time out at 2679.894862s 2024-05-30T16:22:20.511 INFO:tasks.workunit.client.0.smithi006.stderr: -433> 2024-05-30T16:22:19.959+0000 7f69412d4640 10 monclient(hunting): get_auth_request con 0x55d83ce29a50 auth_method 0 2024-05-30T16:22:20.511 INFO:tasks.workunit.client.0.smithi006.stderr: -432> 2024-05-30T16:22:19.959+0000 7f69412d4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:22:20.511 INFO:tasks.workunit.client.0.smithi006.stderr: -431> 2024-05-30T16:22:19.959+0000 7f69412d4640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:22:20.511 INFO:tasks.workunit.client.0.smithi006.stderr: -430> 2024-05-30T16:22:19.959+0000 7f69412d4640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:22:20.511 INFO:tasks.workunit.client.0.smithi006.stderr: -429> 2024-05-30T16:22:19.959+0000 7f69402d2640 10 monclient(hunting): get_auth_request con 0x55d83cdeb690 auth_method 0 2024-05-30T16:22:20.511 INFO:tasks.workunit.client.0.smithi006.stderr: -428> 2024-05-30T16:22:19.959+0000 7f69402d2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:22:20.511 INFO:tasks.workunit.client.0.smithi006.stderr: -427> 2024-05-30T16:22:19.959+0000 7f69402d2640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:22:20.511 INFO:tasks.workunit.client.0.smithi006.stderr: -426> 2024-05-30T16:22:19.959+0000 7f69402d2640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:22:20.511 INFO:tasks.workunit.client.0.smithi006.stderr: -425> 2024-05-30T16:22:19.959+0000 7f69412d4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:22:20.511 INFO:tasks.workunit.client.0.smithi006.stderr: -424> 2024-05-30T16:22:19.959+0000 7f69412d4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:22:20.511 INFO:tasks.workunit.client.0.smithi006.stderr: -423> 2024-05-30T16:22:19.959+0000 7f69412d4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:22:20.511 INFO:tasks.workunit.client.0.smithi006.stderr: -422> 2024-05-30T16:22:19.959+0000 7f69402d2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:22:20.512 INFO:tasks.workunit.client.0.smithi006.stderr: -421> 2024-05-30T16:22:19.959+0000 7f69402d2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:22:20.512 INFO:tasks.workunit.client.0.smithi006.stderr: -420> 2024-05-30T16:22:19.959+0000 7f69402d2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:22:20.512 INFO:tasks.workunit.client.0.smithi006.stderr: -419> 2024-05-30T16:22:19.959+0000 7f6940ad3640 10 monclient(hunting): get_auth_request con 0x55d83cca14d0 auth_method 0 2024-05-30T16:22:20.512 INFO:tasks.workunit.client.0.smithi006.stderr: -418> 2024-05-30T16:22:19.959+0000 7f6940ad3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:22:20.512 INFO:tasks.workunit.client.0.smithi006.stderr: -417> 2024-05-30T16:22:19.959+0000 7f6940ad3640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:22:20.512 INFO:tasks.workunit.client.0.smithi006.stderr: -416> 2024-05-30T16:22:19.959+0000 7f6940ad3640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:22:20.512 INFO:tasks.workunit.client.0.smithi006.stderr: -415> 2024-05-30T16:22:19.959+0000 7f69412d4640 10 monclient(hunting): handle_auth_done global_id 6957 payload 824 2024-05-30T16:22:20.512 INFO:tasks.workunit.client.0.smithi006.stderr: -414> 2024-05-30T16:22:19.959+0000 7f69412d4640 10 monclient: _finish_hunting 0 2024-05-30T16:22:20.512 INFO:tasks.workunit.client.0.smithi006.stderr: -413> 2024-05-30T16:22:19.959+0000 7f69412d4640 1 monclient: found mon.noname-a 2024-05-30T16:22:20.512 INFO:tasks.workunit.client.0.smithi006.stderr: -412> 2024-05-30T16:22:19.959+0000 7f69412d4640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.6:3300/0 2024-05-30T16:22:20.512 INFO:tasks.workunit.client.0.smithi006.stderr: -411> 2024-05-30T16:22:19.959+0000 7f682d60a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:22:20.512 INFO:tasks.workunit.client.0.smithi006.stderr: -410> 2024-05-30T16:22:19.959+0000 7f682d60a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-30T16:22:20.512 INFO:tasks.workunit.client.0.smithi006.stderr: -409> 2024-05-30T16:22:19.959+0000 7f682d60a640 10 monclient: dump: 2024-05-30T16:22:20.512 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:22:20.513 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:22:20.513 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.513 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.513 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:22:20.513 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:22:20.513 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:22:20.513 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:22:20.513 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:22:20.513 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.513 INFO:tasks.workunit.client.0.smithi006.stderr: -408> 2024-05-30T16:22:19.959+0000 7f682d60a640 10 monclient: _finish_auth 0 2024-05-30T16:22:20.513 INFO:tasks.workunit.client.0.smithi006.stderr: -407> 2024-05-30T16:22:19.959+0000 7f682d60a640 10 monclient: _check_auth_tickets 2024-05-30T16:22:20.513 INFO:tasks.workunit.client.0.smithi006.stderr: -406> 2024-05-30T16:22:19.959+0000 7f682d60a640 10 monclient: handle_config config(0 keys) 2024-05-30T16:22:20.513 INFO:tasks.workunit.client.0.smithi006.stderr: -405> 2024-05-30T16:22:19.959+0000 7f682d60a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:22:20.514 INFO:tasks.workunit.client.0.smithi006.stderr: -404> 2024-05-30T16:22:19.959+0000 7f682d60a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-30T16:22:20.514 INFO:tasks.workunit.client.0.smithi006.stderr: -403> 2024-05-30T16:22:19.959+0000 7f682d60a640 10 monclient: dump: 2024-05-30T16:22:20.514 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:22:20.514 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:22:20.514 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.514 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.514 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:22:20.514 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:22:20.514 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:22:20.514 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:22:20.514 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:22:20.514 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.514 INFO:tasks.workunit.client.0.smithi006.stderr: -402> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 monclient: authenticate success, global_id 6957 2024-05-30T16:22:20.515 INFO:tasks.workunit.client.0.smithi006.stderr: -401> 2024-05-30T16:22:19.959+0000 7f682ee0d640 4 set_mon_vals no callback set 2024-05-30T16:22:20.515 INFO:tasks.workunit.client.0.smithi006.stderr: -400> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient: _renew_subs 2024-05-30T16:22:20.515 INFO:tasks.workunit.client.0.smithi006.stderr: -399> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:22:20.515 INFO:tasks.workunit.client.0.smithi006.stderr: -398> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient: _renew_subs 2024-05-30T16:22:20.515 INFO:tasks.workunit.client.0.smithi006.stderr: -397> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:22:20.515 INFO:tasks.workunit.client.0.smithi006.stderr: -396> 2024-05-30T16:22:19.959+0000 7f6942fc2980 1 librados: init done 2024-05-30T16:22:20.515 INFO:tasks.workunit.client.0.smithi006.stderr: -395> 2024-05-30T16:22:19.963+0000 7f682d60a640 4 mgrc handle_mgr_map Got map version 5 2024-05-30T16:22:20.515 INFO:tasks.workunit.client.0.smithi006.stderr: -394> 2024-05-30T16:22:19.963+0000 7f682d60a640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:22:20.515 INFO:tasks.workunit.client.0.smithi006.stderr: -393> 2024-05-30T16:22:19.963+0000 7f682d60a640 4 mgrc reconnect Starting new session with [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:22:20.515 INFO:tasks.workunit.client.0.smithi006.stderr: -392> 2024-05-30T16:22:19.963+0000 7f6940ad3640 10 monclient: get_auth_request con 0x7f682806dde0 auth_method 0 2024-05-30T16:22:20.515 INFO:tasks.workunit.client.0.smithi006.stderr: -391> 2024-05-30T16:22:19.963+0000 7f69402d2640 10 monclient: get_auth_request con 0x55d83ce263c0 auth_method 0 2024-05-30T16:22:20.515 INFO:tasks.workunit.client.0.smithi006.stderr: -390> 2024-05-30T16:22:19.963+0000 7f682d60a640 1 client.6957.objecter ms_handle_reset 0x55d83ce263c0 session 0x55d83ce25d40 osd.1 2024-05-30T16:22:20.515 INFO:tasks.workunit.client.0.smithi006.stderr: -389> 2024-05-30T16:22:19.963+0000 7f682d60a640 10 monclient: _renew_subs 2024-05-30T16:22:20.515 INFO:tasks.workunit.client.0.smithi006.stderr: -388> 2024-05-30T16:22:19.963+0000 7f682d60a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:22:20.516 INFO:tasks.workunit.client.0.smithi006.stderr: -387> 2024-05-30T16:22:19.963+0000 7f69402d2640 10 monclient: get_auth_request con 0x7f682807e510 auth_method 0 2024-05-30T16:22:20.516 INFO:tasks.workunit.client.0.smithi006.stderr: -386> 2024-05-30T16:22:19.963+0000 7f6940ad3640 10 monclient: get_auth_request con 0x55d83ce30750 auth_method 0 2024-05-30T16:22:20.516 INFO:tasks.workunit.client.0.smithi006.stderr: -385> 2024-05-30T16:22:19.967+0000 7f69402d2640 10 monclient: get_auth_request con 0x55d83ce35d80 auth_method 0 2024-05-30T16:22:20.516 INFO:tasks.workunit.client.0.smithi006.stderr: -384> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient: build_initial_monmap 2024-05-30T16:22:20.516 INFO:tasks.workunit.client.0.smithi006.stderr: -383> 2024-05-30T16:22:19.971+0000 7f6942fc2980 1 build_initial for_mkfs: 0 2024-05-30T16:22:20.516 INFO:tasks.workunit.client.0.smithi006.stderr: -382> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient: monmap: 2024-05-30T16:22:20.516 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 0 2024-05-30T16:22:20.516 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-30T16:22:20.516 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 0.000000 2024-05-30T16:22:20.516 INFO:tasks.workunit.client.0.smithi006.stderr:created 0.000000 2024-05-30T16:22:20.516 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 0 (unknown) 2024-05-30T16:22:20.516 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:22:20.516 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.noname-a 2024-05-30T16:22:20.516 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-b 2024-05-30T16:22:20.517 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.noname-c 2024-05-30T16:22:20.517 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.517 INFO:tasks.workunit.client.0.smithi006.stderr: -381> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce39c30) adding auth protocol: cephx 2024-05-30T16:22:20.517 INFO:tasks.workunit.client.0.smithi006.stderr: -380> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce39c30) adding auth protocol: cephx 2024-05-30T16:22:20.517 INFO:tasks.workunit.client.0.smithi006.stderr: -379> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce39c30) adding auth protocol: cephx 2024-05-30T16:22:20.517 INFO:tasks.workunit.client.0.smithi006.stderr: -378> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce39c30) adding con mode: secure 2024-05-30T16:22:20.517 INFO:tasks.workunit.client.0.smithi006.stderr: -377> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce39c30) adding con mode: crc 2024-05-30T16:22:20.517 INFO:tasks.workunit.client.0.smithi006.stderr: -376> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce39c30) adding con mode: secure 2024-05-30T16:22:20.517 INFO:tasks.workunit.client.0.smithi006.stderr: -375> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce39c30) adding con mode: crc 2024-05-30T16:22:20.517 INFO:tasks.workunit.client.0.smithi006.stderr: -374> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce39c30) adding con mode: secure 2024-05-30T16:22:20.517 INFO:tasks.workunit.client.0.smithi006.stderr: -373> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce39c30) adding con mode: crc 2024-05-30T16:22:20.517 INFO:tasks.workunit.client.0.smithi006.stderr: -372> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce39c30) adding con mode: crc 2024-05-30T16:22:20.517 INFO:tasks.workunit.client.0.smithi006.stderr: -371> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce39c30) adding con mode: secure 2024-05-30T16:22:20.517 INFO:tasks.workunit.client.0.smithi006.stderr: -370> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce39c30) adding con mode: crc 2024-05-30T16:22:20.518 INFO:tasks.workunit.client.0.smithi006.stderr: -369> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce39c30) adding con mode: secure 2024-05-30T16:22:20.518 INFO:tasks.workunit.client.0.smithi006.stderr: -368> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce39c30) adding con mode: crc 2024-05-30T16:22:20.518 INFO:tasks.workunit.client.0.smithi006.stderr: -367> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce39c30) adding con mode: secure 2024-05-30T16:22:20.518 INFO:tasks.workunit.client.0.smithi006.stderr: -366> 2024-05-30T16:22:19.971+0000 7f6942fc2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:22:20.518 INFO:tasks.workunit.client.0.smithi006.stderr: -365> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command objecter_requests cmddesc objecter_requests hook 0x55d83ce3d1b0 EEXIST 2024-05-30T16:22:20.518 INFO:tasks.workunit.client.0.smithi006.stderr: -364> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient: init 2024-05-30T16:22:20.518 INFO:tasks.workunit.client.0.smithi006.stderr: -363> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce38be8) adding auth protocol: cephx 2024-05-30T16:22:20.518 INFO:tasks.workunit.client.0.smithi006.stderr: -362> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce38be8) adding auth protocol: cephx 2024-05-30T16:22:20.518 INFO:tasks.workunit.client.0.smithi006.stderr: -361> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce38be8) adding auth protocol: cephx 2024-05-30T16:22:20.518 INFO:tasks.workunit.client.0.smithi006.stderr: -360> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce38be8) adding con mode: secure 2024-05-30T16:22:20.518 INFO:tasks.workunit.client.0.smithi006.stderr: -359> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce38be8) adding con mode: crc 2024-05-30T16:22:20.518 INFO:tasks.workunit.client.0.smithi006.stderr: -358> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce38be8) adding con mode: secure 2024-05-30T16:22:20.519 INFO:tasks.workunit.client.0.smithi006.stderr: -357> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce38be8) adding con mode: crc 2024-05-30T16:22:20.519 INFO:tasks.workunit.client.0.smithi006.stderr: -356> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce38be8) adding con mode: secure 2024-05-30T16:22:20.519 INFO:tasks.workunit.client.0.smithi006.stderr: -355> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce38be8) adding con mode: crc 2024-05-30T16:22:20.519 INFO:tasks.workunit.client.0.smithi006.stderr: -354> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce38be8) adding con mode: crc 2024-05-30T16:22:20.519 INFO:tasks.workunit.client.0.smithi006.stderr: -353> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce38be8) adding con mode: secure 2024-05-30T16:22:20.519 INFO:tasks.workunit.client.0.smithi006.stderr: -352> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce38be8) adding con mode: crc 2024-05-30T16:22:20.519 INFO:tasks.workunit.client.0.smithi006.stderr: -351> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce38be8) adding con mode: secure 2024-05-30T16:22:20.519 INFO:tasks.workunit.client.0.smithi006.stderr: -350> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce38be8) adding con mode: crc 2024-05-30T16:22:20.519 INFO:tasks.workunit.client.0.smithi006.stderr: -349> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce38be8) adding con mode: secure 2024-05-30T16:22:20.519 INFO:tasks.workunit.client.0.smithi006.stderr: -348> 2024-05-30T16:22:19.971+0000 7f6942fc2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:22:20.519 INFO:tasks.workunit.client.0.smithi006.stderr: -347> 2024-05-30T16:22:19.971+0000 7f6942fc2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:22:20.519 INFO:tasks.workunit.client.0.smithi006.stderr: -346> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command rotate-key cmddesc rotate-key hook 0x55d83ce38d30 EEXIST 2024-05-30T16:22:20.519 INFO:tasks.workunit.client.0.smithi006.stderr: -345> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient: _reopen_session rank -1 2024-05-30T16:22:20.520 INFO:tasks.workunit.client.0.smithi006.stderr: -344> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient: _add_conns ranks=[1,0,2] 2024-05-30T16:22:20.520 INFO:tasks.workunit.client.0.smithi006.stderr: -343> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient(hunting): picked mon.noname-b con 0x55d83ce43740 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:22:20.520 INFO:tasks.workunit.client.0.smithi006.stderr: -342> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient(hunting): picked mon.noname-a con 0x55d83ce44070 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:22:20.520 INFO:tasks.workunit.client.0.smithi006.stderr: -341> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient(hunting): picked mon.noname-c con 0x55d83ce43b20 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:22:20.520 INFO:tasks.workunit.client.0.smithi006.stderr: -340> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient(hunting): start opening mon connection 2024-05-30T16:22:20.520 INFO:tasks.workunit.client.0.smithi006.stderr: -339> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient(hunting): start opening mon connection 2024-05-30T16:22:20.520 INFO:tasks.workunit.client.0.smithi006.stderr: -338> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient(hunting): start opening mon connection 2024-05-30T16:22:20.520 INFO:tasks.workunit.client.0.smithi006.stderr: -337> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient(hunting): _renew_subs 2024-05-30T16:22:20.520 INFO:tasks.workunit.client.0.smithi006.stderr: -336> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient(hunting): authenticate will time out at 2679.905855s 2024-05-30T16:22:20.520 INFO:tasks.workunit.client.0.smithi006.stderr: -335> 2024-05-30T16:22:19.971+0000 7f69402d2640 10 monclient(hunting): get_auth_request con 0x55d83ce44070 auth_method 0 2024-05-30T16:22:20.520 INFO:tasks.workunit.client.0.smithi006.stderr: -334> 2024-05-30T16:22:19.971+0000 7f69402d2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:22:20.520 INFO:tasks.workunit.client.0.smithi006.stderr: -333> 2024-05-30T16:22:19.971+0000 7f69402d2640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:22:20.520 INFO:tasks.workunit.client.0.smithi006.stderr: -332> 2024-05-30T16:22:19.971+0000 7f69402d2640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:22:20.521 INFO:tasks.workunit.client.0.smithi006.stderr: -331> 2024-05-30T16:22:19.971+0000 7f69412d4640 10 monclient(hunting): get_auth_request con 0x55d83ce43b20 auth_method 0 2024-05-30T16:22:20.521 INFO:tasks.workunit.client.0.smithi006.stderr: -330> 2024-05-30T16:22:19.971+0000 7f69412d4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:22:20.521 INFO:tasks.workunit.client.0.smithi006.stderr: -329> 2024-05-30T16:22:19.971+0000 7f69412d4640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:22:20.521 INFO:tasks.workunit.client.0.smithi006.stderr: -328> 2024-05-30T16:22:19.971+0000 7f69412d4640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:22:20.521 INFO:tasks.workunit.client.0.smithi006.stderr: -327> 2024-05-30T16:22:19.971+0000 7f69402d2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:22:20.521 INFO:tasks.workunit.client.0.smithi006.stderr: -326> 2024-05-30T16:22:19.971+0000 7f69402d2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:22:20.521 INFO:tasks.workunit.client.0.smithi006.stderr: -325> 2024-05-30T16:22:19.971+0000 7f69402d2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:22:20.521 INFO:tasks.workunit.client.0.smithi006.stderr: -324> 2024-05-30T16:22:19.971+0000 7f69412d4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:22:20.521 INFO:tasks.workunit.client.0.smithi006.stderr: -323> 2024-05-30T16:22:19.971+0000 7f69412d4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:22:20.521 INFO:tasks.workunit.client.0.smithi006.stderr: -322> 2024-05-30T16:22:19.971+0000 7f69412d4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:22:20.521 INFO:tasks.workunit.client.0.smithi006.stderr: -321> 2024-05-30T16:22:19.971+0000 7f6940ad3640 10 monclient(hunting): get_auth_request con 0x55d83ce43740 auth_method 0 2024-05-30T16:22:20.521 INFO:tasks.workunit.client.0.smithi006.stderr: -320> 2024-05-30T16:22:19.971+0000 7f6940ad3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:22:20.521 INFO:tasks.workunit.client.0.smithi006.stderr: -319> 2024-05-30T16:22:19.971+0000 7f6940ad3640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:22:20.522 INFO:tasks.workunit.client.0.smithi006.stderr: -318> 2024-05-30T16:22:19.971+0000 7f6940ad3640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:22:20.522 INFO:tasks.workunit.client.0.smithi006.stderr: -317> 2024-05-30T16:22:19.971+0000 7f69412d4640 10 monclient(hunting): handle_auth_done global_id 6611 payload 824 2024-05-30T16:22:20.522 INFO:tasks.workunit.client.0.smithi006.stderr: -316> 2024-05-30T16:22:19.971+0000 7f69412d4640 10 monclient: _finish_hunting 0 2024-05-30T16:22:20.522 INFO:tasks.workunit.client.0.smithi006.stderr: -315> 2024-05-30T16:22:19.971+0000 7f69412d4640 1 monclient: found mon.noname-c 2024-05-30T16:22:20.522 INFO:tasks.workunit.client.0.smithi006.stderr: -314> 2024-05-30T16:22:19.971+0000 7f69412d4640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.6:3301/0 2024-05-30T16:22:20.522 INFO:tasks.workunit.client.0.smithi006.stderr: -313> 2024-05-30T16:22:19.971+0000 7f68267fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:22:20.522 INFO:tasks.workunit.client.0.smithi006.stderr: -312> 2024-05-30T16:22:19.971+0000 7f68267fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-30T16:22:20.522 INFO:tasks.workunit.client.0.smithi006.stderr: -311> 2024-05-30T16:22:19.971+0000 7f68267fc640 10 monclient: dump: 2024-05-30T16:22:20.522 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:22:20.522 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:22:20.522 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.522 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.523 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:22:20.523 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:22:20.523 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:22:20.523 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:22:20.523 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:22:20.523 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.523 INFO:tasks.workunit.client.0.smithi006.stderr: -310> 2024-05-30T16:22:19.971+0000 7f68267fc640 10 monclient: _finish_auth 0 2024-05-30T16:22:20.523 INFO:tasks.workunit.client.0.smithi006.stderr: -309> 2024-05-30T16:22:19.971+0000 7f68267fc640 10 monclient: _check_auth_tickets 2024-05-30T16:22:20.523 INFO:tasks.workunit.client.0.smithi006.stderr: -308> 2024-05-30T16:22:19.971+0000 7f68267fc640 10 monclient: handle_config config(0 keys) 2024-05-30T16:22:20.523 INFO:tasks.workunit.client.0.smithi006.stderr: -307> 2024-05-30T16:22:19.971+0000 7f68267fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:22:20.523 INFO:tasks.workunit.client.0.smithi006.stderr: -306> 2024-05-30T16:22:19.971+0000 7f68267fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-30T16:22:20.523 INFO:tasks.workunit.client.0.smithi006.stderr: -305> 2024-05-30T16:22:19.971+0000 7f68267fc640 10 monclient: dump: 2024-05-30T16:22:20.523 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:22:20.524 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:22:20.524 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.524 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.524 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:22:20.524 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:22:20.524 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:22:20.524 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:22:20.524 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:22:20.524 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.524 INFO:tasks.workunit.client.0.smithi006.stderr: -304> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 monclient: authenticate success, global_id 6611 2024-05-30T16:22:20.524 INFO:tasks.workunit.client.0.smithi006.stderr: -303> 2024-05-30T16:22:19.971+0000 7f6941ad5640 4 set_mon_vals no callback set 2024-05-30T16:22:20.524 INFO:tasks.workunit.client.0.smithi006.stderr: -302> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient: _renew_subs 2024-05-30T16:22:20.524 INFO:tasks.workunit.client.0.smithi006.stderr: -301> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.6:3301/0 2024-05-30T16:22:20.524 INFO:tasks.workunit.client.0.smithi006.stderr: -300> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient: _renew_subs 2024-05-30T16:22:20.525 INFO:tasks.workunit.client.0.smithi006.stderr: -299> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.6:3301/0 2024-05-30T16:22:20.525 INFO:tasks.workunit.client.0.smithi006.stderr: -298> 2024-05-30T16:22:19.975+0000 7f68267fc640 4 mgrc handle_mgr_map Got map version 5 2024-05-30T16:22:20.525 INFO:tasks.workunit.client.0.smithi006.stderr: -297> 2024-05-30T16:22:19.975+0000 7f68267fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:22:20.525 INFO:tasks.workunit.client.0.smithi006.stderr: -296> 2024-05-30T16:22:19.975+0000 7f68267fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:22:20.525 INFO:tasks.workunit.client.0.smithi006.stderr: -295> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient: get_monmap_and_config 2024-05-30T16:22:20.525 INFO:tasks.workunit.client.0.smithi006.stderr: -294> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient: build_initial_monmap 2024-05-30T16:22:20.525 INFO:tasks.workunit.client.0.smithi006.stderr: -293> 2024-05-30T16:22:19.975+0000 7f6942fc2980 1 build_initial for_mkfs: 0 2024-05-30T16:22:20.525 INFO:tasks.workunit.client.0.smithi006.stderr: -292> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient: monmap: 2024-05-30T16:22:20.525 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 0 2024-05-30T16:22:20.525 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-30T16:22:20.525 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 0.000000 2024-05-30T16:22:20.525 INFO:tasks.workunit.client.0.smithi006.stderr:created 0.000000 2024-05-30T16:22:20.525 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 0 (unknown) 2024-05-30T16:22:20.525 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:22:20.526 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.noname-a 2024-05-30T16:22:20.526 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-b 2024-05-30T16:22:20.526 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.noname-c 2024-05-30T16:22:20.526 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.526 INFO:tasks.workunit.client.0.smithi006.stderr: -291> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce81190) adding auth protocol: cephx 2024-05-30T16:22:20.526 INFO:tasks.workunit.client.0.smithi006.stderr: -290> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce81190) adding auth protocol: cephx 2024-05-30T16:22:20.526 INFO:tasks.workunit.client.0.smithi006.stderr: -289> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce81190) adding auth protocol: cephx 2024-05-30T16:22:20.526 INFO:tasks.workunit.client.0.smithi006.stderr: -288> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce81190) adding con mode: secure 2024-05-30T16:22:20.526 INFO:tasks.workunit.client.0.smithi006.stderr: -287> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce81190) adding con mode: crc 2024-05-30T16:22:20.527 INFO:tasks.workunit.client.0.smithi006.stderr: -286> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce81190) adding con mode: secure 2024-05-30T16:22:20.527 INFO:tasks.workunit.client.0.smithi006.stderr: -285> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce81190) adding con mode: crc 2024-05-30T16:22:20.527 INFO:tasks.workunit.client.0.smithi006.stderr: -284> 2024-05-30T16:22:19.975+0000 7f6940ad3640 10 monclient: get_auth_request con 0x7f681806dcf0 auth_method 0 2024-05-30T16:22:20.527 INFO:tasks.workunit.client.0.smithi006.stderr: -283> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce81190) adding con mode: secure 2024-05-30T16:22:20.527 INFO:tasks.workunit.client.0.smithi006.stderr: -282> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce81190) adding con mode: crc 2024-05-30T16:22:20.527 INFO:tasks.workunit.client.0.smithi006.stderr: -281> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce81190) adding con mode: crc 2024-05-30T16:22:20.527 INFO:tasks.workunit.client.0.smithi006.stderr: -280> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce81190) adding con mode: secure 2024-05-30T16:22:20.527 INFO:tasks.workunit.client.0.smithi006.stderr: -279> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce81190) adding con mode: crc 2024-05-30T16:22:20.527 INFO:tasks.workunit.client.0.smithi006.stderr: -278> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce81190) adding con mode: secure 2024-05-30T16:22:20.527 INFO:tasks.workunit.client.0.smithi006.stderr: -277> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce81190) adding con mode: crc 2024-05-30T16:22:20.527 INFO:tasks.workunit.client.0.smithi006.stderr: -276> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce81190) adding con mode: secure 2024-05-30T16:22:20.527 INFO:tasks.workunit.client.0.smithi006.stderr: -275> 2024-05-30T16:22:19.975+0000 7f6942fc2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:22:20.527 INFO:tasks.workunit.client.0.smithi006.stderr: -274> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient: init 2024-05-30T16:22:20.528 INFO:tasks.workunit.client.0.smithi006.stderr: -273> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf665a0) adding auth protocol: cephx 2024-05-30T16:22:20.528 INFO:tasks.workunit.client.0.smithi006.stderr: -272> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf665a0) adding auth protocol: cephx 2024-05-30T16:22:20.528 INFO:tasks.workunit.client.0.smithi006.stderr: -271> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf665a0) adding auth protocol: cephx 2024-05-30T16:22:20.528 INFO:tasks.workunit.client.0.smithi006.stderr: -270> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf665a0) adding con mode: secure 2024-05-30T16:22:20.528 INFO:tasks.workunit.client.0.smithi006.stderr: -269> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf665a0) adding con mode: crc 2024-05-30T16:22:20.528 INFO:tasks.workunit.client.0.smithi006.stderr: -268> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf665a0) adding con mode: secure 2024-05-30T16:22:20.528 INFO:tasks.workunit.client.0.smithi006.stderr: -267> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf665a0) adding con mode: crc 2024-05-30T16:22:20.528 INFO:tasks.workunit.client.0.smithi006.stderr: -266> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf665a0) adding con mode: secure 2024-05-30T16:22:20.528 INFO:tasks.workunit.client.0.smithi006.stderr: -265> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf665a0) adding con mode: crc 2024-05-30T16:22:20.528 INFO:tasks.workunit.client.0.smithi006.stderr: -264> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf665a0) adding con mode: crc 2024-05-30T16:22:20.528 INFO:tasks.workunit.client.0.smithi006.stderr: -263> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf665a0) adding con mode: secure 2024-05-30T16:22:20.528 INFO:tasks.workunit.client.0.smithi006.stderr: -262> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf665a0) adding con mode: crc 2024-05-30T16:22:20.528 INFO:tasks.workunit.client.0.smithi006.stderr: -261> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf665a0) adding con mode: secure 2024-05-30T16:22:20.528 INFO:tasks.workunit.client.0.smithi006.stderr: -260> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf665a0) adding con mode: crc 2024-05-30T16:22:20.529 INFO:tasks.workunit.client.0.smithi006.stderr: -259> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf665a0) adding con mode: secure 2024-05-30T16:22:20.529 INFO:tasks.workunit.client.0.smithi006.stderr: -258> 2024-05-30T16:22:19.975+0000 7f6942fc2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:22:20.529 INFO:tasks.workunit.client.0.smithi006.stderr: -257> 2024-05-30T16:22:19.975+0000 7f6942fc2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:22:20.529 INFO:tasks.workunit.client.0.smithi006.stderr: -256> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command rotate-key cmddesc rotate-key hook 0x7ffc9bf666e8 EEXIST 2024-05-30T16:22:20.529 INFO:tasks.workunit.client.0.smithi006.stderr: -255> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient: _reopen_session rank -1 2024-05-30T16:22:20.529 INFO:tasks.workunit.client.0.smithi006.stderr: -254> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient: _add_conns ranks=[1,0,2] 2024-05-30T16:22:20.529 INFO:tasks.workunit.client.0.smithi006.stderr: -253> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient(hunting): picked mon.noname-b con 0x55d83ce833d0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:22:20.529 INFO:tasks.workunit.client.0.smithi006.stderr: -252> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient(hunting): picked mon.noname-a con 0x55d83ce868b0 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:22:20.529 INFO:tasks.workunit.client.0.smithi006.stderr: -251> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient(hunting): picked mon.noname-c con 0x55d83ceafcd0 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:22:20.529 INFO:tasks.workunit.client.0.smithi006.stderr: -250> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient(hunting): start opening mon connection 2024-05-30T16:22:20.529 INFO:tasks.workunit.client.0.smithi006.stderr: -249> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient(hunting): start opening mon connection 2024-05-30T16:22:20.529 INFO:tasks.workunit.client.0.smithi006.stderr: -248> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient(hunting): start opening mon connection 2024-05-30T16:22:20.529 INFO:tasks.workunit.client.0.smithi006.stderr: -247> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient(hunting): _renew_subs 2024-05-30T16:22:20.530 INFO:tasks.workunit.client.0.smithi006.stderr: -246> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient(hunting): authenticate will time out at 2679.909976s 2024-05-30T16:22:20.530 INFO:tasks.workunit.client.0.smithi006.stderr: -245> 2024-05-30T16:22:19.975+0000 7f69402d2640 10 monclient(hunting): get_auth_request con 0x55d83ce868b0 auth_method 0 2024-05-30T16:22:20.530 INFO:tasks.workunit.client.0.smithi006.stderr: -244> 2024-05-30T16:22:19.975+0000 7f69402d2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:22:20.530 INFO:tasks.workunit.client.0.smithi006.stderr: -243> 2024-05-30T16:22:19.975+0000 7f69402d2640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:22:20.530 INFO:tasks.workunit.client.0.smithi006.stderr: -242> 2024-05-30T16:22:19.975+0000 7f69402d2640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:22:20.530 INFO:tasks.workunit.client.0.smithi006.stderr: -241> 2024-05-30T16:22:19.975+0000 7f69412d4640 10 monclient(hunting): get_auth_request con 0x55d83ceafcd0 auth_method 0 2024-05-30T16:22:20.530 INFO:tasks.workunit.client.0.smithi006.stderr: -240> 2024-05-30T16:22:19.975+0000 7f69412d4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:22:20.530 INFO:tasks.workunit.client.0.smithi006.stderr: -239> 2024-05-30T16:22:19.975+0000 7f69412d4640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:22:20.530 INFO:tasks.workunit.client.0.smithi006.stderr: -238> 2024-05-30T16:22:19.975+0000 7f69412d4640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:22:20.530 INFO:tasks.workunit.client.0.smithi006.stderr: -237> 2024-05-30T16:22:19.975+0000 7f69402d2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:22:20.530 INFO:tasks.workunit.client.0.smithi006.stderr: -236> 2024-05-30T16:22:19.975+0000 7f69402d2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:22:20.530 INFO:tasks.workunit.client.0.smithi006.stderr: -235> 2024-05-30T16:22:19.975+0000 7f69402d2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:22:20.530 INFO:tasks.workunit.client.0.smithi006.stderr: -234> 2024-05-30T16:22:19.975+0000 7f69412d4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:22:20.531 INFO:tasks.workunit.client.0.smithi006.stderr: -233> 2024-05-30T16:22:19.975+0000 7f69412d4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:22:20.531 INFO:tasks.workunit.client.0.smithi006.stderr: -232> 2024-05-30T16:22:19.975+0000 7f69412d4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:22:20.531 INFO:tasks.workunit.client.0.smithi006.stderr: -231> 2024-05-30T16:22:19.975+0000 7f69402d2640 10 monclient(hunting): handle_auth_done global_id 6963 payload 274 2024-05-30T16:22:20.531 INFO:tasks.workunit.client.0.smithi006.stderr: -230> 2024-05-30T16:22:19.975+0000 7f69402d2640 10 monclient: _finish_hunting 0 2024-05-30T16:22:20.531 INFO:tasks.workunit.client.0.smithi006.stderr: -229> 2024-05-30T16:22:19.975+0000 7f69402d2640 1 monclient: found mon.noname-a 2024-05-30T16:22:20.531 INFO:tasks.workunit.client.0.smithi006.stderr: -228> 2024-05-30T16:22:19.975+0000 7f69402d2640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.6:3300/0 2024-05-30T16:22:20.531 INFO:tasks.workunit.client.0.smithi006.stderr: -227> 2024-05-30T16:22:19.975+0000 7f6817fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:22:20.531 INFO:tasks.workunit.client.0.smithi006.stderr: -226> 2024-05-30T16:22:19.975+0000 7f6817fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-30T16:22:20.531 INFO:tasks.workunit.client.0.smithi006.stderr: -225> 2024-05-30T16:22:19.975+0000 7f6817fff640 10 monclient: dump: 2024-05-30T16:22:20.531 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:22:20.531 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:22:20.531 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.531 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.532 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:22:20.532 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:22:20.532 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:22:20.532 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:22:20.532 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:22:20.532 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.532 INFO:tasks.workunit.client.0.smithi006.stderr: -224> 2024-05-30T16:22:19.975+0000 7f6817fff640 10 monclient: _finish_auth 0 2024-05-30T16:22:20.532 INFO:tasks.workunit.client.0.smithi006.stderr: -223> 2024-05-30T16:22:19.975+0000 7f6817fff640 10 monclient: _check_auth_tickets 2024-05-30T16:22:20.532 INFO:tasks.workunit.client.0.smithi006.stderr: -222> 2024-05-30T16:22:19.975+0000 7f6817fff640 10 monclient: handle_config config(0 keys) 2024-05-30T16:22:20.532 INFO:tasks.workunit.client.0.smithi006.stderr: -221> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 monclient: authenticate success, global_id 6963 2024-05-30T16:22:20.532 INFO:tasks.workunit.client.0.smithi006.stderr: -220> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient: get_monmap_and_config success 2024-05-30T16:22:20.532 INFO:tasks.workunit.client.0.smithi006.stderr: -219> 2024-05-30T16:22:19.975+0000 7f6942fc2980 4 set_mon_vals no callback set 2024-05-30T16:22:20.532 INFO:tasks.workunit.client.0.smithi006.stderr: -218> 2024-05-30T16:22:19.975+0000 7f6817fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:22:20.533 INFO:tasks.workunit.client.0.smithi006.stderr: -217> 2024-05-30T16:22:19.975+0000 7f6817fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-30T16:22:20.533 INFO:tasks.workunit.client.0.smithi006.stderr: -216> 2024-05-30T16:22:19.975+0000 7f6817fff640 10 monclient: dump: 2024-05-30T16:22:20.533 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:22:20.533 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:22:20.533 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.533 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.533 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:22:20.533 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:22:20.533 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:22:20.533 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:22:20.533 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:22:20.533 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.533 INFO:tasks.workunit.client.0.smithi006.stderr: -215> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient: shutdown 2024-05-30T16:22:20.534 INFO:tasks.workunit.client.0.smithi006.stderr: -214> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient: build_initial_monmap 2024-05-30T16:22:20.534 INFO:tasks.workunit.client.0.smithi006.stderr: -213> 2024-05-30T16:22:19.975+0000 7f6942fc2980 1 build_initial for_mkfs: 0 2024-05-30T16:22:20.534 INFO:tasks.workunit.client.0.smithi006.stderr: -212> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient: monmap: 2024-05-30T16:22:20.534 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 0 2024-05-30T16:22:20.534 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-30T16:22:20.534 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 0.000000 2024-05-30T16:22:20.534 INFO:tasks.workunit.client.0.smithi006.stderr:created 0.000000 2024-05-30T16:22:20.534 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 0 (unknown) 2024-05-30T16:22:20.534 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:22:20.534 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.noname-a 2024-05-30T16:22:20.534 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-b 2024-05-30T16:22:20.534 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.noname-c 2024-05-30T16:22:20.535 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.535 INFO:tasks.workunit.client.0.smithi006.stderr: -211> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce85de0) adding auth protocol: cephx 2024-05-30T16:22:20.535 INFO:tasks.workunit.client.0.smithi006.stderr: -210> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce85de0) adding auth protocol: cephx 2024-05-30T16:22:20.535 INFO:tasks.workunit.client.0.smithi006.stderr: -209> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce85de0) adding auth protocol: cephx 2024-05-30T16:22:20.535 INFO:tasks.workunit.client.0.smithi006.stderr: -208> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce85de0) adding con mode: secure 2024-05-30T16:22:20.535 INFO:tasks.workunit.client.0.smithi006.stderr: -207> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce85de0) adding con mode: crc 2024-05-30T16:22:20.535 INFO:tasks.workunit.client.0.smithi006.stderr: -206> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce85de0) adding con mode: secure 2024-05-30T16:22:20.535 INFO:tasks.workunit.client.0.smithi006.stderr: -205> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce85de0) adding con mode: crc 2024-05-30T16:22:20.535 INFO:tasks.workunit.client.0.smithi006.stderr: -204> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce85de0) adding con mode: secure 2024-05-30T16:22:20.535 INFO:tasks.workunit.client.0.smithi006.stderr: -203> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce85de0) adding con mode: crc 2024-05-30T16:22:20.535 INFO:tasks.workunit.client.0.smithi006.stderr: -202> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce85de0) adding con mode: crc 2024-05-30T16:22:20.535 INFO:tasks.workunit.client.0.smithi006.stderr: -201> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce85de0) adding con mode: secure 2024-05-30T16:22:20.535 INFO:tasks.workunit.client.0.smithi006.stderr: -200> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce85de0) adding con mode: crc 2024-05-30T16:22:20.536 INFO:tasks.workunit.client.0.smithi006.stderr: -199> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce85de0) adding con mode: secure 2024-05-30T16:22:20.536 INFO:tasks.workunit.client.0.smithi006.stderr: -198> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce85de0) adding con mode: crc 2024-05-30T16:22:20.536 INFO:tasks.workunit.client.0.smithi006.stderr: -197> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce85de0) adding con mode: secure 2024-05-30T16:22:20.536 INFO:tasks.workunit.client.0.smithi006.stderr: -196> 2024-05-30T16:22:19.975+0000 7f6942fc2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:22:20.536 INFO:tasks.workunit.client.0.smithi006.stderr: -195> 2024-05-30T16:22:19.975+0000 7f6942fc2980 1 librados: starting msgr at 2024-05-30T16:22:20.536 INFO:tasks.workunit.client.0.smithi006.stderr: -194> 2024-05-30T16:22:19.975+0000 7f6942fc2980 1 librados: starting objecter 2024-05-30T16:22:20.536 INFO:tasks.workunit.client.0.smithi006.stderr: -193> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command objecter_requests cmddesc objecter_requests hook 0x55d83ceb2990 EEXIST 2024-05-30T16:22:20.536 INFO:tasks.workunit.client.0.smithi006.stderr: -192> 2024-05-30T16:22:19.979+0000 7f6942fc2980 1 librados: setting wanted keys 2024-05-30T16:22:20.536 INFO:tasks.workunit.client.0.smithi006.stderr: -191> 2024-05-30T16:22:19.979+0000 7f6942fc2980 1 librados: calling monclient init 2024-05-30T16:22:20.536 INFO:tasks.workunit.client.0.smithi006.stderr: -190> 2024-05-30T16:22:19.979+0000 7f6942fc2980 10 monclient: init 2024-05-30T16:22:20.536 INFO:tasks.workunit.client.0.smithi006.stderr: -189> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce791a0) adding auth protocol: cephx 2024-05-30T16:22:20.536 INFO:tasks.workunit.client.0.smithi006.stderr: -188> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce791a0) adding auth protocol: cephx 2024-05-30T16:22:20.536 INFO:tasks.workunit.client.0.smithi006.stderr: -187> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce791a0) adding auth protocol: cephx 2024-05-30T16:22:20.537 INFO:tasks.workunit.client.0.smithi006.stderr: -186> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce791a0) adding con mode: secure 2024-05-30T16:22:20.537 INFO:tasks.workunit.client.0.smithi006.stderr: -185> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce791a0) adding con mode: crc 2024-05-30T16:22:20.537 INFO:tasks.workunit.client.0.smithi006.stderr: -184> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce791a0) adding con mode: secure 2024-05-30T16:22:20.537 INFO:tasks.workunit.client.0.smithi006.stderr: -183> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce791a0) adding con mode: crc 2024-05-30T16:22:20.537 INFO:tasks.workunit.client.0.smithi006.stderr: -182> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce791a0) adding con mode: secure 2024-05-30T16:22:20.537 INFO:tasks.workunit.client.0.smithi006.stderr: -181> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce791a0) adding con mode: crc 2024-05-30T16:22:20.537 INFO:tasks.workunit.client.0.smithi006.stderr: -180> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce791a0) adding con mode: crc 2024-05-30T16:22:20.537 INFO:tasks.workunit.client.0.smithi006.stderr: -179> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce791a0) adding con mode: secure 2024-05-30T16:22:20.537 INFO:tasks.workunit.client.0.smithi006.stderr: -178> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce791a0) adding con mode: crc 2024-05-30T16:22:20.537 INFO:tasks.workunit.client.0.smithi006.stderr: -177> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce791a0) adding con mode: secure 2024-05-30T16:22:20.537 INFO:tasks.workunit.client.0.smithi006.stderr: -176> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce791a0) adding con mode: crc 2024-05-30T16:22:20.537 INFO:tasks.workunit.client.0.smithi006.stderr: -175> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce791a0) adding con mode: secure 2024-05-30T16:22:20.538 INFO:tasks.workunit.client.0.smithi006.stderr: -174> 2024-05-30T16:22:19.979+0000 7f6942fc2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:22:20.538 INFO:tasks.workunit.client.0.smithi006.stderr: -173> 2024-05-30T16:22:19.979+0000 7f6942fc2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:22:20.538 INFO:tasks.workunit.client.0.smithi006.stderr: -172> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command rotate-key cmddesc rotate-key hook 0x55d83ce792e8 EEXIST 2024-05-30T16:22:20.538 INFO:tasks.workunit.client.0.smithi006.stderr: -171> 2024-05-30T16:22:19.979+0000 7f6942fc2980 10 monclient: _reopen_session rank -1 2024-05-30T16:22:20.538 INFO:tasks.workunit.client.0.smithi006.stderr: -170> 2024-05-30T16:22:19.979+0000 7f6942fc2980 10 monclient: _add_conns ranks=[1,0,2] 2024-05-30T16:22:20.538 INFO:tasks.workunit.client.0.smithi006.stderr: -169> 2024-05-30T16:22:19.979+0000 7f6942fc2980 10 monclient(hunting): picked mon.noname-b con 0x55d83ce833d0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:22:20.538 INFO:tasks.workunit.client.0.smithi006.stderr: -168> 2024-05-30T16:22:19.979+0000 7f6942fc2980 10 monclient(hunting): picked mon.noname-a con 0x55d83ce868b0 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:22:20.538 INFO:tasks.workunit.client.0.smithi006.stderr: -167> 2024-05-30T16:22:19.979+0000 7f6942fc2980 10 monclient(hunting): picked mon.noname-c con 0x55d83ceafcd0 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:22:20.538 INFO:tasks.workunit.client.0.smithi006.stderr: -166> 2024-05-30T16:22:19.979+0000 7f6942fc2980 10 monclient(hunting): start opening mon connection 2024-05-30T16:22:20.538 INFO:tasks.workunit.client.0.smithi006.stderr: -165> 2024-05-30T16:22:19.979+0000 7f6942fc2980 10 monclient(hunting): start opening mon connection 2024-05-30T16:22:20.538 INFO:tasks.workunit.client.0.smithi006.stderr: -164> 2024-05-30T16:22:19.979+0000 7f6942fc2980 10 monclient(hunting): start opening mon connection 2024-05-30T16:22:20.538 INFO:tasks.workunit.client.0.smithi006.stderr: -163> 2024-05-30T16:22:19.979+0000 7f6942fc2980 10 monclient(hunting): _renew_subs 2024-05-30T16:22:20.539 INFO:tasks.workunit.client.0.smithi006.stderr: -162> 2024-05-30T16:22:19.979+0000 7f6942fc2980 10 monclient(hunting): authenticate will time out at 2679.912623s 2024-05-30T16:22:20.539 INFO:tasks.workunit.client.0.smithi006.stderr: -161> 2024-05-30T16:22:19.979+0000 7f69402d2640 10 monclient(hunting): get_auth_request con 0x55d83ce868b0 auth_method 0 2024-05-30T16:22:20.539 INFO:tasks.workunit.client.0.smithi006.stderr: -160> 2024-05-30T16:22:19.979+0000 7f69402d2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:22:20.539 INFO:tasks.workunit.client.0.smithi006.stderr: -159> 2024-05-30T16:22:19.979+0000 7f69402d2640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:22:20.539 INFO:tasks.workunit.client.0.smithi006.stderr: -158> 2024-05-30T16:22:19.979+0000 7f69402d2640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:22:20.539 INFO:tasks.workunit.client.0.smithi006.stderr: -157> 2024-05-30T16:22:19.979+0000 7f69412d4640 10 monclient(hunting): get_auth_request con 0x55d83ceafcd0 auth_method 0 2024-05-30T16:22:20.539 INFO:tasks.workunit.client.0.smithi006.stderr: -156> 2024-05-30T16:22:19.979+0000 7f69412d4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:22:20.539 INFO:tasks.workunit.client.0.smithi006.stderr: -155> 2024-05-30T16:22:19.979+0000 7f69412d4640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:22:20.539 INFO:tasks.workunit.client.0.smithi006.stderr: -154> 2024-05-30T16:22:19.979+0000 7f69412d4640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:22:20.539 INFO:tasks.workunit.client.0.smithi006.stderr: -153> 2024-05-30T16:22:19.979+0000 7f69402d2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:22:20.539 INFO:tasks.workunit.client.0.smithi006.stderr: -152> 2024-05-30T16:22:19.979+0000 7f69402d2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:22:20.539 INFO:tasks.workunit.client.0.smithi006.stderr: -151> 2024-05-30T16:22:19.979+0000 7f69402d2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:22:20.539 INFO:tasks.workunit.client.0.smithi006.stderr: -150> 2024-05-30T16:22:19.979+0000 7f69412d4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:22:20.540 INFO:tasks.workunit.client.0.smithi006.stderr: -149> 2024-05-30T16:22:19.979+0000 7f69412d4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:22:20.540 INFO:tasks.workunit.client.0.smithi006.stderr: -148> 2024-05-30T16:22:19.979+0000 7f69412d4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:22:20.540 INFO:tasks.workunit.client.0.smithi006.stderr: -147> 2024-05-30T16:22:19.979+0000 7f6940ad3640 10 monclient(hunting): get_auth_request con 0x55d83ce833d0 auth_method 0 2024-05-30T16:22:20.540 INFO:tasks.workunit.client.0.smithi006.stderr: -146> 2024-05-30T16:22:19.979+0000 7f6940ad3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:22:20.540 INFO:tasks.workunit.client.0.smithi006.stderr: -145> 2024-05-30T16:22:19.979+0000 7f6940ad3640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:22:20.540 INFO:tasks.workunit.client.0.smithi006.stderr: -144> 2024-05-30T16:22:19.979+0000 7f6940ad3640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:22:20.540 INFO:tasks.workunit.client.0.smithi006.stderr: -143> 2024-05-30T16:22:19.979+0000 7f69402d2640 10 monclient(hunting): handle_auth_done global_id 6966 payload 824 2024-05-30T16:22:20.540 INFO:tasks.workunit.client.0.smithi006.stderr: -142> 2024-05-30T16:22:19.979+0000 7f69402d2640 10 monclient: _finish_hunting 0 2024-05-30T16:22:20.540 INFO:tasks.workunit.client.0.smithi006.stderr: -141> 2024-05-30T16:22:19.979+0000 7f69402d2640 1 monclient: found mon.noname-a 2024-05-30T16:22:20.540 INFO:tasks.workunit.client.0.smithi006.stderr: -140> 2024-05-30T16:22:19.979+0000 7f69402d2640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.6:3300/0 2024-05-30T16:22:20.540 INFO:tasks.workunit.client.0.smithi006.stderr: -139> 2024-05-30T16:22:19.979+0000 7f68167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:22:20.540 INFO:tasks.workunit.client.0.smithi006.stderr: -138> 2024-05-30T16:22:19.979+0000 7f68167fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-30T16:22:20.541 INFO:tasks.workunit.client.0.smithi006.stderr: -137> 2024-05-30T16:22:19.979+0000 7f68167fc640 10 monclient: dump: 2024-05-30T16:22:20.541 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:22:20.541 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:22:20.541 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.541 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.541 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:22:20.541 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:22:20.541 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:22:20.541 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:22:20.541 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:22:20.541 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.541 INFO:tasks.workunit.client.0.smithi006.stderr: -136> 2024-05-30T16:22:19.979+0000 7f68167fc640 10 monclient: _finish_auth 0 2024-05-30T16:22:20.541 INFO:tasks.workunit.client.0.smithi006.stderr: -135> 2024-05-30T16:22:19.979+0000 7f68167fc640 10 monclient: _check_auth_tickets 2024-05-30T16:22:20.542 INFO:tasks.workunit.client.0.smithi006.stderr: -134> 2024-05-30T16:22:19.979+0000 7f68167fc640 10 monclient: handle_config config(0 keys) 2024-05-30T16:22:20.542 INFO:tasks.workunit.client.0.smithi006.stderr: -133> 2024-05-30T16:22:19.979+0000 7f68167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:22:20.542 INFO:tasks.workunit.client.0.smithi006.stderr: -132> 2024-05-30T16:22:19.979+0000 7f68167fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-30T16:22:20.542 INFO:tasks.workunit.client.0.smithi006.stderr: -131> 2024-05-30T16:22:19.979+0000 7f68167fc640 10 monclient: dump: 2024-05-30T16:22:20.542 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:22:20.542 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:22:20.542 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.542 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.542 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:22:20.542 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:22:20.542 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:22:20.542 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:22:20.542 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:22:20.543 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.543 INFO:tasks.workunit.client.0.smithi006.stderr: -130> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 monclient: authenticate success, global_id 6966 2024-05-30T16:22:20.543 INFO:tasks.workunit.client.0.smithi006.stderr: -129> 2024-05-30T16:22:19.979+0000 7f6817fff640 4 set_mon_vals no callback set 2024-05-30T16:22:20.543 INFO:tasks.workunit.client.0.smithi006.stderr: -128> 2024-05-30T16:22:19.979+0000 7f6942fc2980 10 monclient: _renew_subs 2024-05-30T16:22:20.543 INFO:tasks.workunit.client.0.smithi006.stderr: -127> 2024-05-30T16:22:19.979+0000 7f6942fc2980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:22:20.543 INFO:tasks.workunit.client.0.smithi006.stderr: -126> 2024-05-30T16:22:19.979+0000 7f6942fc2980 10 monclient: _renew_subs 2024-05-30T16:22:20.543 INFO:tasks.workunit.client.0.smithi006.stderr: -125> 2024-05-30T16:22:19.979+0000 7f6942fc2980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:22:20.543 INFO:tasks.workunit.client.0.smithi006.stderr: -124> 2024-05-30T16:22:19.979+0000 7f6942fc2980 1 librados: init done 2024-05-30T16:22:20.543 INFO:tasks.workunit.client.0.smithi006.stderr: -123> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command cr dump hook 0x55d83cea6608 2024-05-30T16:22:20.543 INFO:tasks.workunit.client.0.smithi006.stderr: -122> 2024-05-30T16:22:19.979+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.543 INFO:tasks.workunit.client.0.smithi006.stderr: -121> 2024-05-30T16:22:19.979+0000 7f68167fc640 4 mgrc handle_mgr_map Got map version 5 2024-05-30T16:22:20.543 INFO:tasks.workunit.client.0.smithi006.stderr: -120> 2024-05-30T16:22:19.979+0000 7f68167fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:22:20.544 INFO:tasks.workunit.client.0.smithi006.stderr: -119> 2024-05-30T16:22:19.979+0000 7f68167fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:22:20.544 INFO:tasks.workunit.client.0.smithi006.stderr: -118> 2024-05-30T16:22:19.983+0000 7f69412d4640 10 monclient: get_auth_request con 0x55d83cecee00 auth_method 0 2024-05-30T16:22:20.544 INFO:tasks.workunit.client.0.smithi006.stderr: -117> 2024-05-30T16:22:19.983+0000 7f6940ad3640 10 monclient: get_auth_request con 0x7f680c06dde0 auth_method 0 2024-05-30T16:22:20.544 INFO:tasks.workunit.client.0.smithi006.stderr: -116> 2024-05-30T16:22:19.983+0000 7f6942fc2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:20.544 INFO:tasks.workunit.client.0.smithi006.stderr: -115> 2024-05-30T16:22:19.983+0000 7f6942fc2980 20 realm 2024-05-30T16:22:20.544 INFO:tasks.workunit.client.0.smithi006.stderr: -114> 2024-05-30T16:22:19.983+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.544 INFO:tasks.workunit.client.0.smithi006.stderr: -113> 2024-05-30T16:22:19.983+0000 7f6942fc2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:20.544 INFO:tasks.workunit.client.0.smithi006.stderr: -112> 2024-05-30T16:22:19.983+0000 7f6942fc2980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:22:20.544 INFO:tasks.workunit.client.0.smithi006.stderr: -111> 2024-05-30T16:22:19.983+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.544 INFO:tasks.workunit.client.0.smithi006.stderr: -110> 2024-05-30T16:22:19.983+0000 7f6942fc2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:20.544 INFO:tasks.workunit.client.0.smithi006.stderr: -109> 2024-05-30T16:22:19.983+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.544 INFO:tasks.workunit.client.0.smithi006.stderr: -108> 2024-05-30T16:22:19.983+0000 7f6940ad3640 10 monclient: get_auth_request con 0x55d83ced2ff0 auth_method 0 2024-05-30T16:22:20.544 INFO:tasks.workunit.client.0.smithi006.stderr: -107> 2024-05-30T16:22:19.983+0000 7f6942fc2980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:20.544 INFO:tasks.workunit.client.0.smithi006.stderr: -106> 2024-05-30T16:22:19.983+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.545 INFO:tasks.workunit.client.0.smithi006.stderr: -105> 2024-05-30T16:22:19.983+0000 7f6942fc2980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:22:20.545 INFO:tasks.workunit.client.0.smithi006.stderr: -104> 2024-05-30T16:22:19.983+0000 7f6942fc2980 20 searching for the correct realm 2024-05-30T16:22:20.545 INFO:tasks.workunit.client.0.smithi006.stderr: -103> 2024-05-30T16:22:19.987+0000 7f69402d2640 10 monclient: get_auth_request con 0x55d83ced82e0 auth_method 0 2024-05-30T16:22:20.545 INFO:tasks.workunit.client.0.smithi006.stderr: -102> 2024-05-30T16:22:19.987+0000 7f69412d4640 10 monclient: get_auth_request con 0x7f69340f2070 auth_method 0 2024-05-30T16:22:20.545 INFO:tasks.workunit.client.0.smithi006.stderr: -101> 2024-05-30T16:22:19.987+0000 7f6940ad3640 10 monclient: get_auth_request con 0x7f6938059c90 auth_method 0 2024-05-30T16:22:20.545 INFO:tasks.workunit.client.0.smithi006.stderr: -100> 2024-05-30T16:22:19.987+0000 7f69402d2640 10 monclient: get_auth_request con 0x7f6930045040 auth_method 0 2024-05-30T16:22:20.545 INFO:tasks.workunit.client.0.smithi006.stderr: -99> 2024-05-30T16:22:19.991+0000 7f69402d2640 0 -- 172.21.15.6:0/1226585167 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f6930045040 msgr2=0x7f6930045420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:20.545 INFO:tasks.workunit.client.0.smithi006.stderr: -98> 2024-05-30T16:22:20.195+0000 7f69402d2640 10 monclient: get_auth_request con 0x7f6930045040 auth_method 0 2024-05-30T16:22:20.545 INFO:tasks.workunit.client.0.smithi006.stderr: -97> 2024-05-30T16:22:20.199+0000 7f69412d4640 10 monclient: get_auth_request con 0x7f6934003030 auth_method 0 2024-05-30T16:22:20.545 INFO:tasks.workunit.client.0.smithi006.stderr: -96> 2024-05-30T16:22:20.207+0000 7f6940ad3640 10 monclient: get_auth_request con 0x7f69340093d0 auth_method 0 2024-05-30T16:22:20.545 INFO:tasks.workunit.client.0.smithi006.stderr: -95> 2024-05-30T16:22:20.207+0000 7f6940ad3640 0 -- 172.21.15.6:0/1226585167 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f69340093d0 msgr2=0x7f69340c56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:20.545 INFO:tasks.workunit.client.0.smithi006.stderr: -94> 2024-05-30T16:22:20.411+0000 7f6940ad3640 10 monclient: get_auth_request con 0x7f69340093d0 auth_method 0 2024-05-30T16:22:20.546 INFO:tasks.workunit.client.0.smithi006.stderr: -93> 2024-05-30T16:22:20.423+0000 7f6942fc2980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:22:20.546 INFO:tasks.workunit.client.0.smithi006.stderr: -92> 2024-05-30T16:22:20.423+0000 7f6942fc2980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:22:20.546 INFO:tasks.workunit.client.0.smithi006.stderr: -91> 2024-05-30T16:22:20.423+0000 7f6942fc2980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:22:20.546 INFO:tasks.workunit.client.0.smithi006.stderr: -90> 2024-05-30T16:22:20.423+0000 7f6942fc2980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:22:20.546 INFO:tasks.workunit.client.0.smithi006.stderr: -89> 2024-05-30T16:22:20.423+0000 7f6942fc2980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:22:20.546 INFO:tasks.workunit.client.0.smithi006.stderr: -88> 2024-05-30T16:22:20.423+0000 7f6942fc2980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:22:20.546 INFO:tasks.workunit.client.0.smithi006.stderr: -87> 2024-05-30T16:22:20.423+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.546 INFO:tasks.workunit.client.0.smithi006.stderr: -86> 2024-05-30T16:22:20.427+0000 7f6942fc2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:20.546 INFO:tasks.workunit.client.0.smithi006.stderr: -85> 2024-05-30T16:22:20.427+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.546 INFO:tasks.workunit.client.0.smithi006.stderr: -84> 2024-05-30T16:22:20.427+0000 7f6942fc2980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:20.546 INFO:tasks.workunit.client.0.smithi006.stderr: -83> 2024-05-30T16:22:20.427+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.546 INFO:tasks.workunit.client.0.smithi006.stderr: -82> 2024-05-30T16:22:20.427+0000 7f6942fc2980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:22:20.546 INFO:tasks.workunit.client.0.smithi006.stderr: -81> 2024-05-30T16:22:20.427+0000 7f6942fc2980 20 zone default found 2024-05-30T16:22:20.547 INFO:tasks.workunit.client.0.smithi006.stderr: -80> 2024-05-30T16:22:20.427+0000 7f6942fc2980 4 Realm: () 2024-05-30T16:22:20.547 INFO:tasks.workunit.client.0.smithi006.stderr: -79> 2024-05-30T16:22:20.427+0000 7f6942fc2980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:22:20.547 INFO:tasks.workunit.client.0.smithi006.stderr: -78> 2024-05-30T16:22:20.427+0000 7f6942fc2980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:22:20.547 INFO:tasks.workunit.client.0.smithi006.stderr: -77> 2024-05-30T16:22:20.427+0000 7f6942fc2980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:22:20.547 INFO:tasks.workunit.client.0.smithi006.stderr: -76> 2024-05-30T16:22:20.427+0000 7f6942fc2980 20 zonegroup default 2024-05-30T16:22:20.547 INFO:tasks.workunit.client.0.smithi006.stderr: -75> 2024-05-30T16:22:20.427+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.547 INFO:tasks.workunit.client.0.smithi006.stderr: -74> 2024-05-30T16:22:20.427+0000 7f6942fc2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:20.547 INFO:tasks.workunit.client.0.smithi006.stderr: -73> 2024-05-30T16:22:20.427+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.547 INFO:tasks.workunit.client.0.smithi006.stderr: -72> 2024-05-30T16:22:20.427+0000 7f6942fc2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:20.547 INFO:tasks.workunit.client.0.smithi006.stderr: -71> 2024-05-30T16:22:20.427+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.547 INFO:tasks.workunit.client.0.smithi006.stderr: -70> 2024-05-30T16:22:20.431+0000 7f6942fc2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:20.547 INFO:tasks.workunit.client.0.smithi006.stderr: -69> 2024-05-30T16:22:20.431+0000 7f6942fc2980 20 started sync module instance, tier type = 2024-05-30T16:22:20.547 INFO:tasks.workunit.client.0.smithi006.stderr: -68> 2024-05-30T16:22:20.431+0000 7f6942fc2980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:22:20.548 INFO:tasks.workunit.client.0.smithi006.stderr: -67> 2024-05-30T16:22:20.451+0000 7f6942fc2980 20 add_watcher() i=0 2024-05-30T16:22:20.548 INFO:tasks.workunit.client.0.smithi006.stderr: -66> 2024-05-30T16:22:20.451+0000 7f6942fc2980 20 add_watcher() i=1 2024-05-30T16:22:20.548 INFO:tasks.workunit.client.0.smithi006.stderr: -65> 2024-05-30T16:22:20.451+0000 7f6942fc2980 20 add_watcher() i=2 2024-05-30T16:22:20.548 INFO:tasks.workunit.client.0.smithi006.stderr: -64> 2024-05-30T16:22:20.451+0000 7f6942fc2980 20 add_watcher() i=3 2024-05-30T16:22:20.548 INFO:tasks.workunit.client.0.smithi006.stderr: -63> 2024-05-30T16:22:20.451+0000 7f6942fc2980 20 add_watcher() i=4 2024-05-30T16:22:20.548 INFO:tasks.workunit.client.0.smithi006.stderr: -62> 2024-05-30T16:22:20.451+0000 7f6942fc2980 20 add_watcher() i=5 2024-05-30T16:22:20.548 INFO:tasks.workunit.client.0.smithi006.stderr: -61> 2024-05-30T16:22:20.451+0000 7f6942fc2980 20 add_watcher() i=6 2024-05-30T16:22:20.548 INFO:tasks.workunit.client.0.smithi006.stderr: -60> 2024-05-30T16:22:20.455+0000 7f6942fc2980 20 add_watcher() i=7 2024-05-30T16:22:20.548 INFO:tasks.workunit.client.0.smithi006.stderr: -59> 2024-05-30T16:22:20.455+0000 7f6942fc2980 2 all 8 watchers are set, enabling cache 2024-05-30T16:22:20.548 INFO:tasks.workunit.client.0.smithi006.stderr: -58> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf66a80) adding auth protocol: cephx 2024-05-30T16:22:20.548 INFO:tasks.workunit.client.0.smithi006.stderr: -57> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf66a80) adding auth protocol: cephx 2024-05-30T16:22:20.548 INFO:tasks.workunit.client.0.smithi006.stderr: -56> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf66a80) adding auth protocol: cephx 2024-05-30T16:22:20.548 INFO:tasks.workunit.client.0.smithi006.stderr: -55> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf66a80) adding con mode: secure 2024-05-30T16:22:20.548 INFO:tasks.workunit.client.0.smithi006.stderr: -54> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf66a80) adding con mode: crc 2024-05-30T16:22:20.549 INFO:tasks.workunit.client.0.smithi006.stderr: -53> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf66a80) adding con mode: secure 2024-05-30T16:22:20.549 INFO:tasks.workunit.client.0.smithi006.stderr: -52> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf66a80) adding con mode: crc 2024-05-30T16:22:20.549 INFO:tasks.workunit.client.0.smithi006.stderr: -51> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf66a80) adding con mode: secure 2024-05-30T16:22:20.549 INFO:tasks.workunit.client.0.smithi006.stderr: -50> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf66a80) adding con mode: crc 2024-05-30T16:22:20.549 INFO:tasks.workunit.client.0.smithi006.stderr: -49> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf66a80) adding con mode: crc 2024-05-30T16:22:20.549 INFO:tasks.workunit.client.0.smithi006.stderr: -48> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf66a80) adding con mode: secure 2024-05-30T16:22:20.549 INFO:tasks.workunit.client.0.smithi006.stderr: -47> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf66a80) adding con mode: crc 2024-05-30T16:22:20.549 INFO:tasks.workunit.client.0.smithi006.stderr: -46> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf66a80) adding con mode: secure 2024-05-30T16:22:20.549 INFO:tasks.workunit.client.0.smithi006.stderr: -45> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf66a80) adding con mode: crc 2024-05-30T16:22:20.549 INFO:tasks.workunit.client.0.smithi006.stderr: -44> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf66a80) adding con mode: secure 2024-05-30T16:22:20.549 INFO:tasks.workunit.client.0.smithi006.stderr: -43> 2024-05-30T16:22:20.459+0000 7f67f6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:20.549 INFO:tasks.workunit.client.0.smithi006.stderr: -42> 2024-05-30T16:22:20.459+0000 7f6942fc2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:22:20.549 INFO:tasks.workunit.client.0.smithi006.stderr: -41> 2024-05-30T16:22:20.459+0000 7f6942fc2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:22:20.550 INFO:tasks.workunit.client.0.smithi006.stderr: -40> 2024-05-30T16:22:20.459+0000 7f6942fc2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:22:20.550 INFO:tasks.workunit.client.0.smithi006.stderr: -39> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command cache list hook 0x55d83cea4ba0 2024-05-30T16:22:20.550 INFO:tasks.workunit.client.0.smithi006.stderr: -38> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command cache inspect hook 0x55d83cea4ba0 2024-05-30T16:22:20.550 INFO:tasks.workunit.client.0.smithi006.stderr: -37> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command cache erase hook 0x55d83cea4ba0 2024-05-30T16:22:20.550 INFO:tasks.workunit.client.0.smithi006.stderr: -36> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command cache zap hook 0x55d83cea4ba0 2024-05-30T16:22:20.550 INFO:tasks.workunit.client.0.smithi006.stderr: -35> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 note: GC not initialized 2024-05-30T16:22:20.550 INFO:tasks.workunit.client.0.smithi006.stderr: -34> 2024-05-30T16:22:20.459+0000 7f67f5fe3640 20 reqs_thread_entry: start 2024-05-30T16:22:20.550 INFO:tasks.workunit.client.0.smithi006.stderr: -33> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command sync trace show hook 0x7f69340abef0 2024-05-30T16:22:20.550 INFO:tasks.workunit.client.0.smithi006.stderr: -32> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command sync trace history hook 0x7f69340abef0 2024-05-30T16:22:20.550 INFO:tasks.workunit.client.0.smithi006.stderr: -31> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command sync trace active hook 0x7f69340abef0 2024-05-30T16:22:20.550 INFO:tasks.workunit.client.0.smithi006.stderr: -30> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command sync trace active_short hook 0x7f69340abef0 2024-05-30T16:22:20.550 INFO:tasks.workunit.client.0.smithi006.stderr: -29> 2024-05-30T16:22:20.459+0000 7f6942fc2980 20 init_complete bucket index max shards: 11 2024-05-30T16:22:20.550 INFO:tasks.workunit.client.0.smithi006.stderr: -28> 2024-05-30T16:22:20.459+0000 7f6942fc2980 20 Filter name: none 2024-05-30T16:22:20.551 INFO:tasks.workunit.client.0.smithi006.stderr: -27> 2024-05-30T16:22:20.459+0000 7f67677fe640 20 reqs_thread_entry: start 2024-05-30T16:22:20.551 INFO:tasks.workunit.client.0.smithi006.stderr: -26> 2024-05-30T16:22:20.459+0000 7f6942fc2980 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2024-05-30T16:22:20.551 INFO:tasks.workunit.client.0.smithi006.stderr: -25> 2024-05-30T16:22:20.459+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.551 INFO:tasks.workunit.client.0.smithi006.stderr: -24> 2024-05-30T16:22:20.459+0000 7f6942fc2980 20 rados_obj.operate() r=0 bl.length=211 2024-05-30T16:22:20.551 INFO:tasks.workunit.client.0.smithi006.stderr: -23> 2024-05-30T16:22:20.459+0000 7f6942fc2980 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2024-05-30T16:22:20.551 INFO:tasks.workunit.client.0.smithi006.stderr: -22> 2024-05-30T16:22:20.459+0000 7f6942fc2980 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2024-05-30T16:22:20.551 INFO:tasks.workunit.client.0.smithi006.stderr: -21> 2024-05-30T16:22:20.459+0000 7f6942fc2980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 : miss 2024-05-30T16:22:20.551 INFO:tasks.workunit.client.0.smithi006.stderr: -20> 2024-05-30T16:22:20.459+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.551 INFO:tasks.workunit.client.0.smithi006.stderr: -19> 2024-05-30T16:22:20.463+0000 7f6942fc2980 20 rados_obj.operate() r=0 bl.length=425 2024-05-30T16:22:20.551 INFO:tasks.workunit.client.0.smithi006.stderr: -18> 2024-05-30T16:22:20.463+0000 7f6942fc2980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 info.flags=0x17 2024-05-30T16:22:20.551 INFO:tasks.workunit.client.0.smithi006.stderr: -17> 2024-05-30T16:22:20.463+0000 7f6942fc2980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 to cache LRU end 2024-05-30T16:22:20.551 INFO:tasks.workunit.client.0.smithi006.stderr: -16> 2024-05-30T16:22:20.463+0000 7f6942fc2980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:22:20.551 INFO:tasks.workunit.client.0.smithi006.stderr: -15> 2024-05-30T16:22:20.463+0000 7f6942fc2980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:22:20.551 INFO:tasks.workunit.client.0.smithi006.stderr: -14> 2024-05-30T16:22:20.463+0000 7f6942fc2980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 2024-05-30T16:22:20.552 INFO:tasks.workunit.client.0.smithi006.stderr: -13> 2024-05-30T16:22:20.471+0000 7f6942fc2980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 info.flags=0x17 2024-05-30T16:22:20.552 INFO:tasks.workunit.client.0.smithi006.stderr: -12> 2024-05-30T16:22:20.471+0000 7f6942fc2980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 to cache LRU end 2024-05-30T16:22:20.552 INFO:tasks.workunit.client.0.smithi006.stderr: -11> 2024-05-30T16:22:20.471+0000 7f6942fc2980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:22:20.552 INFO:tasks.workunit.client.0.smithi006.stderr: -10> 2024-05-30T16:22:20.471+0000 7f6942fc2980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29, ofs0, ns] 2024-05-30T16:22:20.552 INFO:tasks.workunit.client.0.smithi006.stderr: -9> 2024-05-30T16:22:20.471+0000 7f6817fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431637 cookie 94387223527648 notifier 6966 bl.length()=987 2024-05-30T16:22:20.552 INFO:tasks.workunit.client.0.smithi006.stderr: -8> 2024-05-30T16:22:20.471+0000 7f6817fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 info.flags=0x17 2024-05-30T16:22:20.552 INFO:tasks.workunit.client.0.smithi006.stderr: -7> 2024-05-30T16:22:20.471+0000 7f6817fff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 to cache LRU end 2024-05-30T16:22:20.552 INFO:tasks.workunit.client.0.smithi006.stderr: -6> 2024-05-30T16:22:20.471+0000 7f6817fff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:22:20.552 INFO:tasks.workunit.client.0.smithi006.stderr: -5> 2024-05-30T16:22:20.475+0000 7f6942fc2980 20 handle_bi_update(): bucket=:abort-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29]): orig_sources= new_sources= 2024-05-30T16:22:20.552 INFO:tasks.workunit.client.0.smithi006.stderr: -4> 2024-05-30T16:22:20.475+0000 7f6942fc2980 20 handle_bi_update(): bucket=:abort-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29]): potential sources added=[] removed=[] 2024-05-30T16:22:20.552 INFO:tasks.workunit.client.0.smithi006.stderr: -3> 2024-05-30T16:22:20.475+0000 7f6942fc2980 20 handle_bi_update(): bucket=:abort-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29]): orig_dests= new_dests= 2024-05-30T16:22:20.552 INFO:tasks.workunit.client.0.smithi006.stderr: -2> 2024-05-30T16:22:20.475+0000 7f6942fc2980 20 handle_bi_update(): bucket=:abort-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29]): potential dests added=[] removed=[] 2024-05-30T16:22:20.552 INFO:tasks.workunit.client.0.smithi006.stderr: -1> 2024-05-30T16:22:20.479+0000 7f6942fc2980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f6942fc2980 time 2024-05-30T16:22:20.482496+0000 2024-05-30T16:22:20.553 INFO:tasks.workunit.client.0.smithi006.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-30T16:22:20.553 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.553 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:22:20.553 INFO:tasks.workunit.client.0.smithi006.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f6946899cff] 2024-05-30T16:22:20.553 INFO:tasks.workunit.client.0.smithi006.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7f6946899eb3] 2024-05-30T16:22:20.553 INFO:tasks.workunit.client.0.smithi006.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55d83b4d0f81] 2024-05-30T16:22:20.553 INFO:tasks.workunit.client.0.smithi006.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x31b) [0x55d83b4e2f1b] 2024-05-30T16:22:20.553 INFO:tasks.workunit.client.0.smithi006.stderr: 5: main() 2024-05-30T16:22:20.553 INFO:tasks.workunit.client.0.smithi006.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f6945702d90] 2024-05-30T16:22:20.553 INFO:tasks.workunit.client.0.smithi006.stderr: 7: __libc_start_main() 2024-05-30T16:22:20.553 INFO:tasks.workunit.client.0.smithi006.stderr: 8: _start() 2024-05-30T16:22:20.553 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.553 INFO:tasks.workunit.client.0.smithi006.stderr: 0> 2024-05-30T16:22:20.479+0000 7f6942fc2980 -1 *** Caught signal (Aborted) ** 2024-05-30T16:22:20.554 INFO:tasks.workunit.client.0.smithi006.stderr: in thread 7f6942fc2980 thread_name:radosgw-admin 2024-05-30T16:22:20.554 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.554 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:22:20.554 INFO:tasks.workunit.client.0.smithi006.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f694571b520] 2024-05-30T16:22:20.554 INFO:tasks.workunit.client.0.smithi006.stderr: 2: pthread_kill() 2024-05-30T16:22:20.554 INFO:tasks.workunit.client.0.smithi006.stderr: 3: raise() 2024-05-30T16:22:20.554 INFO:tasks.workunit.client.0.smithi006.stderr: 4: abort() 2024-05-30T16:22:20.554 INFO:tasks.workunit.client.0.smithi006.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f6946899d59] 2024-05-30T16:22:20.554 INFO:tasks.workunit.client.0.smithi006.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7f6946899eb3] 2024-05-30T16:22:20.554 INFO:tasks.workunit.client.0.smithi006.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55d83b4d0f81] 2024-05-30T16:22:20.554 INFO:tasks.workunit.client.0.smithi006.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x31b) [0x55d83b4e2f1b] 2024-05-30T16:22:20.554 INFO:tasks.workunit.client.0.smithi006.stderr: 9: main() 2024-05-30T16:22:20.555 INFO:tasks.workunit.client.0.smithi006.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f6945702d90] 2024-05-30T16:22:20.555 INFO:tasks.workunit.client.0.smithi006.stderr: 11: __libc_start_main() 2024-05-30T16:22:20.555 INFO:tasks.workunit.client.0.smithi006.stderr: 12: _start() 2024-05-30T16:22:20.555 INFO:tasks.workunit.client.0.smithi006.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-30T16:22:20.555 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.555 INFO:tasks.workunit.client.0.smithi006.stderr:--- logging levels --- 2024-05-30T16:22:20.555 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 none 2024-05-30T16:22:20.555 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 lockdep 2024-05-30T16:22:20.555 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 context 2024-05-30T16:22:20.555 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 crush 2024-05-30T16:22:20.555 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds 2024-05-30T16:22:20.555 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_balancer 2024-05-30T16:22:20.555 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_locker 2024-05-30T16:22:20.556 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_log 2024-05-30T16:22:20.556 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_log_expire 2024-05-30T16:22:20.556 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_migrator 2024-05-30T16:22:20.556 INFO:tasks.workunit.client.0.smithi006.stderr: 3/ 5 mds_quiesce 2024-05-30T16:22:20.556 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 buffer 2024-05-30T16:22:20.556 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 timer 2024-05-30T16:22:20.556 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 filer 2024-05-30T16:22:20.556 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 striper 2024-05-30T16:22:20.556 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 objecter 2024-05-30T16:22:20.556 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rados 2024-05-30T16:22:20.557 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd 2024-05-30T16:22:20.557 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd_mirror 2024-05-30T16:22:20.557 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd_replay 2024-05-30T16:22:20.557 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd_pwl 2024-05-30T16:22:20.557 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 journaler 2024-05-30T16:22:20.557 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 objectcacher 2024-05-30T16:22:20.557 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 immutable_obj_cache 2024-05-30T16:22:20.557 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 client 2024-05-30T16:22:20.557 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 osd 2024-05-30T16:22:20.557 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 optracker 2024-05-30T16:22:20.557 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 objclass 2024-05-30T16:22:20.557 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 3 filestore 2024-05-30T16:22:20.557 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 3 journal 2024-05-30T16:22:20.557 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 0 ms 2024-05-30T16:22:20.558 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mon 2024-05-30T16:22:20.558 INFO:tasks.workunit.client.0.smithi006.stderr: 0/10 monc 2024-05-30T16:22:20.558 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 paxos 2024-05-30T16:22:20.558 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 tp 2024-05-30T16:22:20.558 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 auth 2024-05-30T16:22:20.558 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 crypto 2024-05-30T16:22:20.558 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 finisher 2024-05-30T16:22:20.558 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 reserver 2024-05-30T16:22:20.558 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 heartbeatmap 2024-05-30T16:22:20.558 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 perfcounter 2024-05-30T16:22:20.558 INFO:tasks.workunit.client.0.smithi006.stderr: 20/20 rgw 2024-05-30T16:22:20.558 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_sync 2024-05-30T16:22:20.558 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_datacache 2024-05-30T16:22:20.559 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_access 2024-05-30T16:22:20.559 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_dbstore 2024-05-30T16:22:20.559 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_flight 2024-05-30T16:22:20.559 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_lifecycle 2024-05-30T16:22:20.559 INFO:tasks.workunit.client.0.smithi006.stderr: 20/20 rgw_notification 2024-05-30T16:22:20.559 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 javaclient 2024-05-30T16:22:20.559 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 asok 2024-05-30T16:22:20.559 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 throttle 2024-05-30T16:22:20.559 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 0 refs 2024-05-30T16:22:20.559 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 compressor 2024-05-30T16:22:20.559 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 bluestore 2024-05-30T16:22:20.559 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 bluefs 2024-05-30T16:22:20.559 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 3 bdev 2024-05-30T16:22:20.559 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 kstore 2024-05-30T16:22:20.560 INFO:tasks.workunit.client.0.smithi006.stderr: 4/ 5 rocksdb 2024-05-30T16:22:20.560 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 fuse 2024-05-30T16:22:20.560 INFO:tasks.workunit.client.0.smithi006.stderr: 2/ 5 mgr 2024-05-30T16:22:20.560 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mgrc 2024-05-30T16:22:20.560 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 dpdk 2024-05-30T16:22:20.560 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 eventtrace 2024-05-30T16:22:20.560 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 prioritycache 2024-05-30T16:22:20.560 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 test 2024-05-30T16:22:20.560 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 cephfs_mirror 2024-05-30T16:22:20.560 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 cephsqlite 2024-05-30T16:22:20.560 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 crimson_interrupt 2024-05-30T16:22:20.561 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore 2024-05-30T16:22:20.561 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_onode 2024-05-30T16:22:20.561 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_odata 2024-05-30T16:22:20.561 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_omap 2024-05-30T16:22:20.561 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_tm 2024-05-30T16:22:20.561 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_t 2024-05-30T16:22:20.561 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_cleaner 2024-05-30T16:22:20.561 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_epm 2024-05-30T16:22:20.561 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_lba 2024-05-30T16:22:20.561 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_fixedkv_tree 2024-05-30T16:22:20.561 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_cache 2024-05-30T16:22:20.561 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_journal 2024-05-30T16:22:20.561 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_device 2024-05-30T16:22:20.562 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_backref 2024-05-30T16:22:20.562 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 alienstore 2024-05-30T16:22:20.562 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mclock 2024-05-30T16:22:20.562 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 cyanstore 2024-05-30T16:22:20.562 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 ceph_exporter 2024-05-30T16:22:20.562 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 memstore 2024-05-30T16:22:20.562 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 trace 2024-05-30T16:22:20.562 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 ceph_dedup 2024-05-30T16:22:20.562 INFO:tasks.workunit.client.0.smithi006.stderr: -2/-2 (syslog threshold) 2024-05-30T16:22:20.562 INFO:tasks.workunit.client.0.smithi006.stderr: 99/99 (stderr threshold) 2024-05-30T16:22:20.562 INFO:tasks.workunit.client.0.smithi006.stderr:--- pthread ID / name mapping for recent threads --- 2024-05-30T16:22:20.562 INFO:tasks.workunit.client.0.smithi006.stderr: 7f67677fe640 / http_manager 2024-05-30T16:22:20.563 INFO:tasks.workunit.client.0.smithi006.stderr: 7f67f5fe3640 / http_manager 2024-05-30T16:22:20.563 INFO:tasks.workunit.client.0.smithi006.stderr: 7f67f6fe5640 / rgw_dt_lg_renew 2024-05-30T16:22:20.563 INFO:tasks.workunit.client.0.smithi006.stderr: 7f68167fc640 / ms_dispatch 2024-05-30T16:22:20.563 INFO:tasks.workunit.client.0.smithi006.stderr: 7f6817fff640 / io_context_pool 2024-05-30T16:22:20.563 INFO:tasks.workunit.client.0.smithi006.stderr: 7f68267fc640 / ms_dispatch 2024-05-30T16:22:20.563 INFO:tasks.workunit.client.0.smithi006.stderr: 7f682d60a640 / ms_dispatch 2024-05-30T16:22:20.563 INFO:tasks.workunit.client.0.smithi006.stderr: 7f682ee0d640 / io_context_pool 2024-05-30T16:22:20.563 INFO:tasks.workunit.client.0.smithi006.stderr: 7f69402d2640 / msgr-worker-2 2024-05-30T16:22:20.563 INFO:tasks.workunit.client.0.smithi006.stderr: 7f6940ad3640 / msgr-worker-1 2024-05-30T16:22:20.563 INFO:tasks.workunit.client.0.smithi006.stderr: 7f69412d4640 / msgr-worker-0 2024-05-30T16:22:20.563 INFO:tasks.workunit.client.0.smithi006.stderr: 7f6941ad5640 / io_context_pool 2024-05-30T16:22:20.563 INFO:tasks.workunit.client.0.smithi006.stderr: 7f6942fc2980 / radosgw-admin 2024-05-30T16:22:20.563 INFO:tasks.workunit.client.0.smithi006.stderr: max_recent 500 2024-05-30T16:22:20.564 INFO:tasks.workunit.client.0.smithi006.stderr: max_new 1000 2024-05-30T16:22:20.564 INFO:tasks.workunit.client.0.smithi006.stderr: log_file /var/log/ceph/ceph-client.admin.101672.log 2024-05-30T16:22:20.564 INFO:tasks.workunit.client.0.smithi006.stderr:--- end dump of recent events --- 2024-05-30T16:22:20.564 INFO:tasks.workunit.client.0.smithi006.stderr:--- begin dump of recent events --- 2024-05-30T16:22:20.564 INFO:tasks.workunit.client.0.smithi006.stderr: -499> 2024-05-30T16:22:19.959+0000 7f682ee0d640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-30T16:22:20.564 INFO:tasks.workunit.client.0.smithi006.stderr: -498> 2024-05-30T16:22:19.959+0000 7f682ee0d640 10 monclient: dump: 2024-05-30T16:22:20.564 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:22:20.564 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:22:20.564 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.564 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.564 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:22:20.564 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:22:20.564 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:22:20.565 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:22:20.565 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:22:20.565 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.565 INFO:tasks.workunit.client.0.smithi006.stderr: -497> 2024-05-30T16:22:19.959+0000 7f682ee0d640 10 monclient: _finish_auth 0 2024-05-30T16:22:20.565 INFO:tasks.workunit.client.0.smithi006.stderr: -496> 2024-05-30T16:22:19.959+0000 7f682ee0d640 10 monclient: _check_auth_tickets 2024-05-30T16:22:20.565 INFO:tasks.workunit.client.0.smithi006.stderr: -495> 2024-05-30T16:22:19.959+0000 7f682ee0d640 10 monclient: handle_config config(0 keys) 2024-05-30T16:22:20.565 INFO:tasks.workunit.client.0.smithi006.stderr: -494> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 monclient: authenticate success, global_id 6605 2024-05-30T16:22:20.565 INFO:tasks.workunit.client.0.smithi006.stderr: -493> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient: get_monmap_and_config success 2024-05-30T16:22:20.565 INFO:tasks.workunit.client.0.smithi006.stderr: -492> 2024-05-30T16:22:19.959+0000 7f6942fc2980 4 set_mon_vals no callback set 2024-05-30T16:22:20.565 INFO:tasks.workunit.client.0.smithi006.stderr: -491> 2024-05-30T16:22:19.959+0000 7f682ee0d640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:22:20.565 INFO:tasks.workunit.client.0.smithi006.stderr: -490> 2024-05-30T16:22:19.959+0000 7f682ee0d640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-30T16:22:20.565 INFO:tasks.workunit.client.0.smithi006.stderr: -489> 2024-05-30T16:22:19.959+0000 7f682ee0d640 10 monclient: dump: 2024-05-30T16:22:20.565 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:22:20.565 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:22:20.566 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.566 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.566 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:22:20.566 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:22:20.566 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:22:20.566 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:22:20.566 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:22:20.566 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.566 INFO:tasks.workunit.client.0.smithi006.stderr: -488> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient: shutdown 2024-05-30T16:22:20.566 INFO:tasks.workunit.client.0.smithi006.stderr: -487> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 asok(0x55d83cbda790) unregister_commands rotate-key 2024-05-30T16:22:20.566 INFO:tasks.workunit.client.0.smithi006.stderr: -486> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient: build_initial_monmap 2024-05-30T16:22:20.566 INFO:tasks.workunit.client.0.smithi006.stderr: -485> 2024-05-30T16:22:19.959+0000 7f6942fc2980 1 build_initial for_mkfs: 0 2024-05-30T16:22:20.566 INFO:tasks.workunit.client.0.smithi006.stderr: -484> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient: monmap: 2024-05-30T16:22:20.567 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 0 2024-05-30T16:22:20.567 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-30T16:22:20.567 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 0.000000 2024-05-30T16:22:20.567 INFO:tasks.workunit.client.0.smithi006.stderr:created 0.000000 2024-05-30T16:22:20.567 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 0 (unknown) 2024-05-30T16:22:20.567 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:22:20.567 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.noname-a 2024-05-30T16:22:20.567 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-b 2024-05-30T16:22:20.567 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.noname-c 2024-05-30T16:22:20.567 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.567 INFO:tasks.workunit.client.0.smithi006.stderr: -483> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cdea080) adding auth protocol: cephx 2024-05-30T16:22:20.568 INFO:tasks.workunit.client.0.smithi006.stderr: -482> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cdea080) adding auth protocol: cephx 2024-05-30T16:22:20.568 INFO:tasks.workunit.client.0.smithi006.stderr: -481> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cdea080) adding auth protocol: cephx 2024-05-30T16:22:20.568 INFO:tasks.workunit.client.0.smithi006.stderr: -480> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cdea080) adding con mode: secure 2024-05-30T16:22:20.568 INFO:tasks.workunit.client.0.smithi006.stderr: -479> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cdea080) adding con mode: crc 2024-05-30T16:22:20.568 INFO:tasks.workunit.client.0.smithi006.stderr: -478> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cdea080) adding con mode: secure 2024-05-30T16:22:20.568 INFO:tasks.workunit.client.0.smithi006.stderr: -477> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cdea080) adding con mode: crc 2024-05-30T16:22:20.568 INFO:tasks.workunit.client.0.smithi006.stderr: -476> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cdea080) adding con mode: secure 2024-05-30T16:22:20.568 INFO:tasks.workunit.client.0.smithi006.stderr: -475> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cdea080) adding con mode: crc 2024-05-30T16:22:20.568 INFO:tasks.workunit.client.0.smithi006.stderr: -474> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cdea080) adding con mode: crc 2024-05-30T16:22:20.568 INFO:tasks.workunit.client.0.smithi006.stderr: -473> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cdea080) adding con mode: secure 2024-05-30T16:22:20.568 INFO:tasks.workunit.client.0.smithi006.stderr: -472> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cdea080) adding con mode: crc 2024-05-30T16:22:20.568 INFO:tasks.workunit.client.0.smithi006.stderr: -471> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cdea080) adding con mode: secure 2024-05-30T16:22:20.568 INFO:tasks.workunit.client.0.smithi006.stderr: -470> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cdea080) adding con mode: crc 2024-05-30T16:22:20.569 INFO:tasks.workunit.client.0.smithi006.stderr: -469> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cdea080) adding con mode: secure 2024-05-30T16:22:20.569 INFO:tasks.workunit.client.0.smithi006.stderr: -468> 2024-05-30T16:22:19.959+0000 7f6942fc2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:22:20.569 INFO:tasks.workunit.client.0.smithi006.stderr: -467> 2024-05-30T16:22:19.959+0000 7f6942fc2980 1 librados: starting msgr at 2024-05-30T16:22:20.569 INFO:tasks.workunit.client.0.smithi006.stderr: -466> 2024-05-30T16:22:19.959+0000 7f6942fc2980 1 librados: starting objecter 2024-05-30T16:22:20.569 INFO:tasks.workunit.client.0.smithi006.stderr: -465> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command objecter_requests hook 0x55d83cc9fa10 2024-05-30T16:22:20.569 INFO:tasks.workunit.client.0.smithi006.stderr: -464> 2024-05-30T16:22:19.959+0000 7f6942fc2980 1 librados: setting wanted keys 2024-05-30T16:22:20.569 INFO:tasks.workunit.client.0.smithi006.stderr: -463> 2024-05-30T16:22:19.959+0000 7f6942fc2980 1 librados: calling monclient init 2024-05-30T16:22:20.569 INFO:tasks.workunit.client.0.smithi006.stderr: -462> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient: init 2024-05-30T16:22:20.569 INFO:tasks.workunit.client.0.smithi006.stderr: -461> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cde8290) adding auth protocol: cephx 2024-05-30T16:22:20.569 INFO:tasks.workunit.client.0.smithi006.stderr: -460> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cde8290) adding auth protocol: cephx 2024-05-30T16:22:20.569 INFO:tasks.workunit.client.0.smithi006.stderr: -459> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cde8290) adding auth protocol: cephx 2024-05-30T16:22:20.569 INFO:tasks.workunit.client.0.smithi006.stderr: -458> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cde8290) adding con mode: secure 2024-05-30T16:22:20.569 INFO:tasks.workunit.client.0.smithi006.stderr: -457> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cde8290) adding con mode: crc 2024-05-30T16:22:20.570 INFO:tasks.workunit.client.0.smithi006.stderr: -456> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cde8290) adding con mode: secure 2024-05-30T16:22:20.570 INFO:tasks.workunit.client.0.smithi006.stderr: -455> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cde8290) adding con mode: crc 2024-05-30T16:22:20.570 INFO:tasks.workunit.client.0.smithi006.stderr: -454> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cde8290) adding con mode: secure 2024-05-30T16:22:20.570 INFO:tasks.workunit.client.0.smithi006.stderr: -453> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cde8290) adding con mode: crc 2024-05-30T16:22:20.570 INFO:tasks.workunit.client.0.smithi006.stderr: -452> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cde8290) adding con mode: crc 2024-05-30T16:22:20.570 INFO:tasks.workunit.client.0.smithi006.stderr: -451> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cde8290) adding con mode: secure 2024-05-30T16:22:20.570 INFO:tasks.workunit.client.0.smithi006.stderr: -450> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cde8290) adding con mode: crc 2024-05-30T16:22:20.570 INFO:tasks.workunit.client.0.smithi006.stderr: -449> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cde8290) adding con mode: secure 2024-05-30T16:22:20.570 INFO:tasks.workunit.client.0.smithi006.stderr: -448> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cde8290) adding con mode: crc 2024-05-30T16:22:20.570 INFO:tasks.workunit.client.0.smithi006.stderr: -447> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 AuthRegistry(0x55d83cde8290) adding con mode: secure 2024-05-30T16:22:20.570 INFO:tasks.workunit.client.0.smithi006.stderr: -446> 2024-05-30T16:22:19.959+0000 7f6942fc2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:22:20.570 INFO:tasks.workunit.client.0.smithi006.stderr: -445> 2024-05-30T16:22:19.959+0000 7f6942fc2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:22:20.570 INFO:tasks.workunit.client.0.smithi006.stderr: -444> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command rotate-key hook 0x55d83cde83d8 2024-05-30T16:22:20.571 INFO:tasks.workunit.client.0.smithi006.stderr: -443> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient: _reopen_session rank -1 2024-05-30T16:22:20.571 INFO:tasks.workunit.client.0.smithi006.stderr: -442> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient: _add_conns ranks=[1,2,0] 2024-05-30T16:22:20.571 INFO:tasks.workunit.client.0.smithi006.stderr: -441> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient(hunting): picked mon.noname-b con 0x55d83cca14d0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:22:20.571 INFO:tasks.workunit.client.0.smithi006.stderr: -440> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient(hunting): picked mon.noname-c con 0x55d83cdeb690 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:22:20.571 INFO:tasks.workunit.client.0.smithi006.stderr: -439> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient(hunting): picked mon.noname-a con 0x55d83ce29a50 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:22:20.571 INFO:tasks.workunit.client.0.smithi006.stderr: -438> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient(hunting): start opening mon connection 2024-05-30T16:22:20.571 INFO:tasks.workunit.client.0.smithi006.stderr: -437> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient(hunting): start opening mon connection 2024-05-30T16:22:20.571 INFO:tasks.workunit.client.0.smithi006.stderr: -436> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient(hunting): start opening mon connection 2024-05-30T16:22:20.571 INFO:tasks.workunit.client.0.smithi006.stderr: -435> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient(hunting): _renew_subs 2024-05-30T16:22:20.571 INFO:tasks.workunit.client.0.smithi006.stderr: -434> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient(hunting): authenticate will time out at 2679.894862s 2024-05-30T16:22:20.571 INFO:tasks.workunit.client.0.smithi006.stderr: -433> 2024-05-30T16:22:19.959+0000 7f69412d4640 10 monclient(hunting): get_auth_request con 0x55d83ce29a50 auth_method 0 2024-05-30T16:22:20.571 INFO:tasks.workunit.client.0.smithi006.stderr: -432> 2024-05-30T16:22:19.959+0000 7f69412d4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:22:20.571 INFO:tasks.workunit.client.0.smithi006.stderr: -431> 2024-05-30T16:22:19.959+0000 7f69412d4640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:22:20.572 INFO:tasks.workunit.client.0.smithi006.stderr: -430> 2024-05-30T16:22:19.959+0000 7f69412d4640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:22:20.572 INFO:tasks.workunit.client.0.smithi006.stderr: -429> 2024-05-30T16:22:19.959+0000 7f69402d2640 10 monclient(hunting): get_auth_request con 0x55d83cdeb690 auth_method 0 2024-05-30T16:22:20.572 INFO:tasks.workunit.client.0.smithi006.stderr: -428> 2024-05-30T16:22:19.959+0000 7f69402d2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:22:20.572 INFO:tasks.workunit.client.0.smithi006.stderr: -427> 2024-05-30T16:22:19.959+0000 7f69402d2640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:22:20.572 INFO:tasks.workunit.client.0.smithi006.stderr: -426> 2024-05-30T16:22:19.959+0000 7f69402d2640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:22:20.572 INFO:tasks.workunit.client.0.smithi006.stderr: -425> 2024-05-30T16:22:19.959+0000 7f69412d4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:22:20.572 INFO:tasks.workunit.client.0.smithi006.stderr: -424> 2024-05-30T16:22:19.959+0000 7f69412d4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:22:20.572 INFO:tasks.workunit.client.0.smithi006.stderr: -423> 2024-05-30T16:22:19.959+0000 7f69412d4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:22:20.572 INFO:tasks.workunit.client.0.smithi006.stderr: -422> 2024-05-30T16:22:19.959+0000 7f69402d2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:22:20.572 INFO:tasks.workunit.client.0.smithi006.stderr: -421> 2024-05-30T16:22:19.959+0000 7f69402d2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:22:20.572 INFO:tasks.workunit.client.0.smithi006.stderr: -420> 2024-05-30T16:22:19.959+0000 7f69402d2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:22:20.572 INFO:tasks.workunit.client.0.smithi006.stderr: -419> 2024-05-30T16:22:19.959+0000 7f6940ad3640 10 monclient(hunting): get_auth_request con 0x55d83cca14d0 auth_method 0 2024-05-30T16:22:20.572 INFO:tasks.workunit.client.0.smithi006.stderr: -418> 2024-05-30T16:22:19.959+0000 7f6940ad3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:22:20.573 INFO:tasks.workunit.client.0.smithi006.stderr: -417> 2024-05-30T16:22:19.959+0000 7f6940ad3640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:22:20.573 INFO:tasks.workunit.client.0.smithi006.stderr: -416> 2024-05-30T16:22:19.959+0000 7f6940ad3640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:22:20.573 INFO:tasks.workunit.client.0.smithi006.stderr: -415> 2024-05-30T16:22:19.959+0000 7f69412d4640 10 monclient(hunting): handle_auth_done global_id 6957 payload 824 2024-05-30T16:22:20.573 INFO:tasks.workunit.client.0.smithi006.stderr: -414> 2024-05-30T16:22:19.959+0000 7f69412d4640 10 monclient: _finish_hunting 0 2024-05-30T16:22:20.573 INFO:tasks.workunit.client.0.smithi006.stderr: -413> 2024-05-30T16:22:19.959+0000 7f69412d4640 1 monclient: found mon.noname-a 2024-05-30T16:22:20.573 INFO:tasks.workunit.client.0.smithi006.stderr: -412> 2024-05-30T16:22:19.959+0000 7f69412d4640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.6:3300/0 2024-05-30T16:22:20.573 INFO:tasks.workunit.client.0.smithi006.stderr: -411> 2024-05-30T16:22:19.959+0000 7f682d60a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:22:20.573 INFO:tasks.workunit.client.0.smithi006.stderr: -410> 2024-05-30T16:22:19.959+0000 7f682d60a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-30T16:22:20.573 INFO:tasks.workunit.client.0.smithi006.stderr: -409> 2024-05-30T16:22:19.959+0000 7f682d60a640 10 monclient: dump: 2024-05-30T16:22:20.573 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:22:20.573 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:22:20.573 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.573 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.573 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:22:20.574 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:22:20.574 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:22:20.574 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:22:20.574 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:22:20.574 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.574 INFO:tasks.workunit.client.0.smithi006.stderr: -408> 2024-05-30T16:22:19.959+0000 7f682d60a640 10 monclient: _finish_auth 0 2024-05-30T16:22:20.574 INFO:tasks.workunit.client.0.smithi006.stderr: -407> 2024-05-30T16:22:19.959+0000 7f682d60a640 10 monclient: _check_auth_tickets 2024-05-30T16:22:20.574 INFO:tasks.workunit.client.0.smithi006.stderr: -406> 2024-05-30T16:22:19.959+0000 7f682d60a640 10 monclient: handle_config config(0 keys) 2024-05-30T16:22:20.574 INFO:tasks.workunit.client.0.smithi006.stderr: -405> 2024-05-30T16:22:19.959+0000 7f682d60a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:22:20.574 INFO:tasks.workunit.client.0.smithi006.stderr: -404> 2024-05-30T16:22:19.959+0000 7f682d60a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-30T16:22:20.574 INFO:tasks.workunit.client.0.smithi006.stderr: -403> 2024-05-30T16:22:19.959+0000 7f682d60a640 10 monclient: dump: 2024-05-30T16:22:20.574 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:22:20.575 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:22:20.575 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.575 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.575 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:22:20.575 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:22:20.575 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:22:20.575 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:22:20.575 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:22:20.575 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.575 INFO:tasks.workunit.client.0.smithi006.stderr: -402> 2024-05-30T16:22:19.959+0000 7f6942fc2980 5 monclient: authenticate success, global_id 6957 2024-05-30T16:22:20.575 INFO:tasks.workunit.client.0.smithi006.stderr: -401> 2024-05-30T16:22:19.959+0000 7f682ee0d640 4 set_mon_vals no callback set 2024-05-30T16:22:20.575 INFO:tasks.workunit.client.0.smithi006.stderr: -400> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient: _renew_subs 2024-05-30T16:22:20.575 INFO:tasks.workunit.client.0.smithi006.stderr: -399> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:22:20.575 INFO:tasks.workunit.client.0.smithi006.stderr: -398> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient: _renew_subs 2024-05-30T16:22:20.576 INFO:tasks.workunit.client.0.smithi006.stderr: -397> 2024-05-30T16:22:19.959+0000 7f6942fc2980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:22:20.576 INFO:tasks.workunit.client.0.smithi006.stderr: -396> 2024-05-30T16:22:19.959+0000 7f6942fc2980 1 librados: init done 2024-05-30T16:22:20.576 INFO:tasks.workunit.client.0.smithi006.stderr: -395> 2024-05-30T16:22:19.963+0000 7f682d60a640 4 mgrc handle_mgr_map Got map version 5 2024-05-30T16:22:20.576 INFO:tasks.workunit.client.0.smithi006.stderr: -394> 2024-05-30T16:22:19.963+0000 7f682d60a640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:22:20.576 INFO:tasks.workunit.client.0.smithi006.stderr: -393> 2024-05-30T16:22:19.963+0000 7f682d60a640 4 mgrc reconnect Starting new session with [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:22:20.576 INFO:tasks.workunit.client.0.smithi006.stderr: -392> 2024-05-30T16:22:19.963+0000 7f6940ad3640 10 monclient: get_auth_request con 0x7f682806dde0 auth_method 0 2024-05-30T16:22:20.576 INFO:tasks.workunit.client.0.smithi006.stderr: -391> 2024-05-30T16:22:19.963+0000 7f69402d2640 10 monclient: get_auth_request con 0x55d83ce263c0 auth_method 0 2024-05-30T16:22:20.576 INFO:tasks.workunit.client.0.smithi006.stderr: -390> 2024-05-30T16:22:19.963+0000 7f682d60a640 1 client.6957.objecter ms_handle_reset 0x55d83ce263c0 session 0x55d83ce25d40 osd.1 2024-05-30T16:22:20.576 INFO:tasks.workunit.client.0.smithi006.stderr: -389> 2024-05-30T16:22:19.963+0000 7f682d60a640 10 monclient: _renew_subs 2024-05-30T16:22:20.576 INFO:tasks.workunit.client.0.smithi006.stderr: -388> 2024-05-30T16:22:19.963+0000 7f682d60a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:22:20.576 INFO:tasks.workunit.client.0.smithi006.stderr: -387> 2024-05-30T16:22:19.963+0000 7f69402d2640 10 monclient: get_auth_request con 0x7f682807e510 auth_method 0 2024-05-30T16:22:20.576 INFO:tasks.workunit.client.0.smithi006.stderr: -386> 2024-05-30T16:22:19.963+0000 7f6940ad3640 10 monclient: get_auth_request con 0x55d83ce30750 auth_method 0 2024-05-30T16:22:20.576 INFO:tasks.workunit.client.0.smithi006.stderr: -385> 2024-05-30T16:22:19.967+0000 7f69402d2640 10 monclient: get_auth_request con 0x55d83ce35d80 auth_method 0 2024-05-30T16:22:20.577 INFO:tasks.workunit.client.0.smithi006.stderr: -384> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient: build_initial_monmap 2024-05-30T16:22:20.577 INFO:tasks.workunit.client.0.smithi006.stderr: -383> 2024-05-30T16:22:19.971+0000 7f6942fc2980 1 build_initial for_mkfs: 0 2024-05-30T16:22:20.577 INFO:tasks.workunit.client.0.smithi006.stderr: -382> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient: monmap: 2024-05-30T16:22:20.577 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 0 2024-05-30T16:22:20.577 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-30T16:22:20.577 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 0.000000 2024-05-30T16:22:20.577 INFO:tasks.workunit.client.0.smithi006.stderr:created 0.000000 2024-05-30T16:22:20.577 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 0 (unknown) 2024-05-30T16:22:20.577 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:22:20.577 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.noname-a 2024-05-30T16:22:20.577 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-b 2024-05-30T16:22:20.577 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.noname-c 2024-05-30T16:22:20.578 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.578 INFO:tasks.workunit.client.0.smithi006.stderr: -381> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce39c30) adding auth protocol: cephx 2024-05-30T16:22:20.578 INFO:tasks.workunit.client.0.smithi006.stderr: -380> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce39c30) adding auth protocol: cephx 2024-05-30T16:22:20.578 INFO:tasks.workunit.client.0.smithi006.stderr: -379> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce39c30) adding auth protocol: cephx 2024-05-30T16:22:20.578 INFO:tasks.workunit.client.0.smithi006.stderr: -378> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce39c30) adding con mode: secure 2024-05-30T16:22:20.578 INFO:tasks.workunit.client.0.smithi006.stderr: -377> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce39c30) adding con mode: crc 2024-05-30T16:22:20.578 INFO:tasks.workunit.client.0.smithi006.stderr: -376> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce39c30) adding con mode: secure 2024-05-30T16:22:20.578 INFO:tasks.workunit.client.0.smithi006.stderr: -375> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce39c30) adding con mode: crc 2024-05-30T16:22:20.578 INFO:tasks.workunit.client.0.smithi006.stderr: -374> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce39c30) adding con mode: secure 2024-05-30T16:22:20.578 INFO:tasks.workunit.client.0.smithi006.stderr: -373> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce39c30) adding con mode: crc 2024-05-30T16:22:20.578 INFO:tasks.workunit.client.0.smithi006.stderr: -372> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce39c30) adding con mode: crc 2024-05-30T16:22:20.579 INFO:tasks.workunit.client.0.smithi006.stderr: -371> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce39c30) adding con mode: secure 2024-05-30T16:22:20.579 INFO:tasks.workunit.client.0.smithi006.stderr: -370> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce39c30) adding con mode: crc 2024-05-30T16:22:20.579 INFO:tasks.workunit.client.0.smithi006.stderr: -369> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce39c30) adding con mode: secure 2024-05-30T16:22:20.579 INFO:tasks.workunit.client.0.smithi006.stderr: -368> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce39c30) adding con mode: crc 2024-05-30T16:22:20.579 INFO:tasks.workunit.client.0.smithi006.stderr: -367> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce39c30) adding con mode: secure 2024-05-30T16:22:20.579 INFO:tasks.workunit.client.0.smithi006.stderr: -366> 2024-05-30T16:22:19.971+0000 7f6942fc2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:22:20.579 INFO:tasks.workunit.client.0.smithi006.stderr: -365> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command objecter_requests cmddesc objecter_requests hook 0x55d83ce3d1b0 EEXIST 2024-05-30T16:22:20.579 INFO:tasks.workunit.client.0.smithi006.stderr: -364> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient: init 2024-05-30T16:22:20.579 INFO:tasks.workunit.client.0.smithi006.stderr: -363> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce38be8) adding auth protocol: cephx 2024-05-30T16:22:20.579 INFO:tasks.workunit.client.0.smithi006.stderr: -362> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce38be8) adding auth protocol: cephx 2024-05-30T16:22:20.579 INFO:tasks.workunit.client.0.smithi006.stderr: -361> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce38be8) adding auth protocol: cephx 2024-05-30T16:22:20.580 INFO:tasks.workunit.client.0.smithi006.stderr: -360> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce38be8) adding con mode: secure 2024-05-30T16:22:20.580 INFO:tasks.workunit.client.0.smithi006.stderr: -359> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce38be8) adding con mode: crc 2024-05-30T16:22:20.580 INFO:tasks.workunit.client.0.smithi006.stderr: -358> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce38be8) adding con mode: secure 2024-05-30T16:22:20.580 INFO:tasks.workunit.client.0.smithi006.stderr: -357> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce38be8) adding con mode: crc 2024-05-30T16:22:20.580 INFO:tasks.workunit.client.0.smithi006.stderr: -356> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce38be8) adding con mode: secure 2024-05-30T16:22:20.580 INFO:tasks.workunit.client.0.smithi006.stderr: -355> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce38be8) adding con mode: crc 2024-05-30T16:22:20.580 INFO:tasks.workunit.client.0.smithi006.stderr: -354> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce38be8) adding con mode: crc 2024-05-30T16:22:20.580 INFO:tasks.workunit.client.0.smithi006.stderr: -353> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce38be8) adding con mode: secure 2024-05-30T16:22:20.580 INFO:tasks.workunit.client.0.smithi006.stderr: -352> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce38be8) adding con mode: crc 2024-05-30T16:22:20.580 INFO:tasks.workunit.client.0.smithi006.stderr: -351> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce38be8) adding con mode: secure 2024-05-30T16:22:20.580 INFO:tasks.workunit.client.0.smithi006.stderr: -350> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce38be8) adding con mode: crc 2024-05-30T16:22:20.581 INFO:tasks.workunit.client.0.smithi006.stderr: -349> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce38be8) adding con mode: secure 2024-05-30T16:22:20.581 INFO:tasks.workunit.client.0.smithi006.stderr: -348> 2024-05-30T16:22:19.971+0000 7f6942fc2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:22:20.581 INFO:tasks.workunit.client.0.smithi006.stderr: -347> 2024-05-30T16:22:19.971+0000 7f6942fc2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:22:20.581 INFO:tasks.workunit.client.0.smithi006.stderr: -346> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command rotate-key cmddesc rotate-key hook 0x55d83ce38d30 EEXIST 2024-05-30T16:22:20.581 INFO:tasks.workunit.client.0.smithi006.stderr: -345> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient: _reopen_session rank -1 2024-05-30T16:22:20.581 INFO:tasks.workunit.client.0.smithi006.stderr: -344> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient: _add_conns ranks=[1,0,2] 2024-05-30T16:22:20.581 INFO:tasks.workunit.client.0.smithi006.stderr: -343> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient(hunting): picked mon.noname-b con 0x55d83ce43740 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:22:20.581 INFO:tasks.workunit.client.0.smithi006.stderr: -342> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient(hunting): picked mon.noname-a con 0x55d83ce44070 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:22:20.581 INFO:tasks.workunit.client.0.smithi006.stderr: -341> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient(hunting): picked mon.noname-c con 0x55d83ce43b20 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:22:20.581 INFO:tasks.workunit.client.0.smithi006.stderr: -340> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient(hunting): start opening mon connection 2024-05-30T16:22:20.581 INFO:tasks.workunit.client.0.smithi006.stderr: -339> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient(hunting): start opening mon connection 2024-05-30T16:22:20.581 INFO:tasks.workunit.client.0.smithi006.stderr: -338> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient(hunting): start opening mon connection 2024-05-30T16:22:20.581 INFO:tasks.workunit.client.0.smithi006.stderr: -337> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient(hunting): _renew_subs 2024-05-30T16:22:20.582 INFO:tasks.workunit.client.0.smithi006.stderr: -336> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient(hunting): authenticate will time out at 2679.905855s 2024-05-30T16:22:20.582 INFO:tasks.workunit.client.0.smithi006.stderr: -335> 2024-05-30T16:22:19.971+0000 7f69402d2640 10 monclient(hunting): get_auth_request con 0x55d83ce44070 auth_method 0 2024-05-30T16:22:20.582 INFO:tasks.workunit.client.0.smithi006.stderr: -334> 2024-05-30T16:22:19.971+0000 7f69402d2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:22:20.582 INFO:tasks.workunit.client.0.smithi006.stderr: -333> 2024-05-30T16:22:19.971+0000 7f69402d2640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:22:20.582 INFO:tasks.workunit.client.0.smithi006.stderr: -332> 2024-05-30T16:22:19.971+0000 7f69402d2640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:22:20.582 INFO:tasks.workunit.client.0.smithi006.stderr: -331> 2024-05-30T16:22:19.971+0000 7f69412d4640 10 monclient(hunting): get_auth_request con 0x55d83ce43b20 auth_method 0 2024-05-30T16:22:20.582 INFO:tasks.workunit.client.0.smithi006.stderr: -330> 2024-05-30T16:22:19.971+0000 7f69412d4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:22:20.582 INFO:tasks.workunit.client.0.smithi006.stderr: -329> 2024-05-30T16:22:19.971+0000 7f69412d4640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:22:20.582 INFO:tasks.workunit.client.0.smithi006.stderr: -328> 2024-05-30T16:22:19.971+0000 7f69412d4640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:22:20.582 INFO:tasks.workunit.client.0.smithi006.stderr: -327> 2024-05-30T16:22:19.971+0000 7f69402d2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:22:20.582 INFO:tasks.workunit.client.0.smithi006.stderr: -326> 2024-05-30T16:22:19.971+0000 7f69402d2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:22:20.582 INFO:tasks.workunit.client.0.smithi006.stderr: -325> 2024-05-30T16:22:19.971+0000 7f69402d2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:22:20.582 INFO:tasks.workunit.client.0.smithi006.stderr: -324> 2024-05-30T16:22:19.971+0000 7f69412d4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:22:20.583 INFO:tasks.workunit.client.0.smithi006.stderr: -323> 2024-05-30T16:22:19.971+0000 7f69412d4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:22:20.583 INFO:tasks.workunit.client.0.smithi006.stderr: -322> 2024-05-30T16:22:19.971+0000 7f69412d4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:22:20.583 INFO:tasks.workunit.client.0.smithi006.stderr: -321> 2024-05-30T16:22:19.971+0000 7f6940ad3640 10 monclient(hunting): get_auth_request con 0x55d83ce43740 auth_method 0 2024-05-30T16:22:20.583 INFO:tasks.workunit.client.0.smithi006.stderr: -320> 2024-05-30T16:22:19.971+0000 7f6940ad3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:22:20.583 INFO:tasks.workunit.client.0.smithi006.stderr: -319> 2024-05-30T16:22:19.971+0000 7f6940ad3640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:22:20.583 INFO:tasks.workunit.client.0.smithi006.stderr: -318> 2024-05-30T16:22:19.971+0000 7f6940ad3640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:22:20.583 INFO:tasks.workunit.client.0.smithi006.stderr: -317> 2024-05-30T16:22:19.971+0000 7f69412d4640 10 monclient(hunting): handle_auth_done global_id 6611 payload 824 2024-05-30T16:22:20.583 INFO:tasks.workunit.client.0.smithi006.stderr: -316> 2024-05-30T16:22:19.971+0000 7f69412d4640 10 monclient: _finish_hunting 0 2024-05-30T16:22:20.583 INFO:tasks.workunit.client.0.smithi006.stderr: -315> 2024-05-30T16:22:19.971+0000 7f69412d4640 1 monclient: found mon.noname-c 2024-05-30T16:22:20.583 INFO:tasks.workunit.client.0.smithi006.stderr: -314> 2024-05-30T16:22:19.971+0000 7f69412d4640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.6:3301/0 2024-05-30T16:22:20.583 INFO:tasks.workunit.client.0.smithi006.stderr: -313> 2024-05-30T16:22:19.971+0000 7f68267fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:22:20.583 INFO:tasks.workunit.client.0.smithi006.stderr: -312> 2024-05-30T16:22:19.971+0000 7f68267fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-30T16:22:20.583 INFO:tasks.workunit.client.0.smithi006.stderr: -311> 2024-05-30T16:22:19.971+0000 7f68267fc640 10 monclient: dump: 2024-05-30T16:22:20.584 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:22:20.584 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:22:20.584 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.584 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.584 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:22:20.584 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:22:20.584 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:22:20.584 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:22:20.584 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:22:20.584 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.584 INFO:tasks.workunit.client.0.smithi006.stderr: -310> 2024-05-30T16:22:19.971+0000 7f68267fc640 10 monclient: _finish_auth 0 2024-05-30T16:22:20.584 INFO:tasks.workunit.client.0.smithi006.stderr: -309> 2024-05-30T16:22:19.971+0000 7f68267fc640 10 monclient: _check_auth_tickets 2024-05-30T16:22:20.584 INFO:tasks.workunit.client.0.smithi006.stderr: -308> 2024-05-30T16:22:19.971+0000 7f68267fc640 10 monclient: handle_config config(0 keys) 2024-05-30T16:22:20.585 INFO:tasks.workunit.client.0.smithi006.stderr: -307> 2024-05-30T16:22:19.971+0000 7f68267fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:22:20.585 INFO:tasks.workunit.client.0.smithi006.stderr: -306> 2024-05-30T16:22:19.971+0000 7f68267fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-30T16:22:20.585 INFO:tasks.workunit.client.0.smithi006.stderr: -305> 2024-05-30T16:22:19.971+0000 7f68267fc640 10 monclient: dump: 2024-05-30T16:22:20.585 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:22:20.585 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:22:20.585 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.585 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.585 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:22:20.585 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:22:20.585 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:22:20.585 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:22:20.585 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:22:20.585 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.585 INFO:tasks.workunit.client.0.smithi006.stderr: -304> 2024-05-30T16:22:19.971+0000 7f6942fc2980 5 monclient: authenticate success, global_id 6611 2024-05-30T16:22:20.586 INFO:tasks.workunit.client.0.smithi006.stderr: -303> 2024-05-30T16:22:19.971+0000 7f6941ad5640 4 set_mon_vals no callback set 2024-05-30T16:22:20.586 INFO:tasks.workunit.client.0.smithi006.stderr: -302> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient: _renew_subs 2024-05-30T16:22:20.586 INFO:tasks.workunit.client.0.smithi006.stderr: -301> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.6:3301/0 2024-05-30T16:22:20.586 INFO:tasks.workunit.client.0.smithi006.stderr: -300> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient: _renew_subs 2024-05-30T16:22:20.586 INFO:tasks.workunit.client.0.smithi006.stderr: -299> 2024-05-30T16:22:19.971+0000 7f6942fc2980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.6:3301/0 2024-05-30T16:22:20.586 INFO:tasks.workunit.client.0.smithi006.stderr: -298> 2024-05-30T16:22:19.975+0000 7f68267fc640 4 mgrc handle_mgr_map Got map version 5 2024-05-30T16:22:20.586 INFO:tasks.workunit.client.0.smithi006.stderr: -297> 2024-05-30T16:22:19.975+0000 7f68267fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:22:20.586 INFO:tasks.workunit.client.0.smithi006.stderr: -296> 2024-05-30T16:22:19.975+0000 7f68267fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:22:20.586 INFO:tasks.workunit.client.0.smithi006.stderr: -295> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient: get_monmap_and_config 2024-05-30T16:22:20.586 INFO:tasks.workunit.client.0.smithi006.stderr: -294> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient: build_initial_monmap 2024-05-30T16:22:20.586 INFO:tasks.workunit.client.0.smithi006.stderr: -293> 2024-05-30T16:22:19.975+0000 7f6942fc2980 1 build_initial for_mkfs: 0 2024-05-30T16:22:20.586 INFO:tasks.workunit.client.0.smithi006.stderr: -292> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient: monmap: 2024-05-30T16:22:20.587 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 0 2024-05-30T16:22:20.587 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-30T16:22:20.587 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 0.000000 2024-05-30T16:22:20.587 INFO:tasks.workunit.client.0.smithi006.stderr:created 0.000000 2024-05-30T16:22:20.587 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 0 (unknown) 2024-05-30T16:22:20.587 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:22:20.587 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.noname-a 2024-05-30T16:22:20.587 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-b 2024-05-30T16:22:20.587 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.noname-c 2024-05-30T16:22:20.587 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.587 INFO:tasks.workunit.client.0.smithi006.stderr: -291> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce81190) adding auth protocol: cephx 2024-05-30T16:22:20.587 INFO:tasks.workunit.client.0.smithi006.stderr: -290> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce81190) adding auth protocol: cephx 2024-05-30T16:22:20.587 INFO:tasks.workunit.client.0.smithi006.stderr: -289> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce81190) adding auth protocol: cephx 2024-05-30T16:22:20.588 INFO:tasks.workunit.client.0.smithi006.stderr: -288> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce81190) adding con mode: secure 2024-05-30T16:22:20.588 INFO:tasks.workunit.client.0.smithi006.stderr: -287> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce81190) adding con mode: crc 2024-05-30T16:22:20.588 INFO:tasks.workunit.client.0.smithi006.stderr: -286> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce81190) adding con mode: secure 2024-05-30T16:22:20.588 INFO:tasks.workunit.client.0.smithi006.stderr: -285> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce81190) adding con mode: crc 2024-05-30T16:22:20.588 INFO:tasks.workunit.client.0.smithi006.stderr: -284> 2024-05-30T16:22:19.975+0000 7f6940ad3640 10 monclient: get_auth_request con 0x7f681806dcf0 auth_method 0 2024-05-30T16:22:20.588 INFO:tasks.workunit.client.0.smithi006.stderr: -283> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce81190) adding con mode: secure 2024-05-30T16:22:20.588 INFO:tasks.workunit.client.0.smithi006.stderr: -282> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce81190) adding con mode: crc 2024-05-30T16:22:20.588 INFO:tasks.workunit.client.0.smithi006.stderr: -281> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce81190) adding con mode: crc 2024-05-30T16:22:20.588 INFO:tasks.workunit.client.0.smithi006.stderr: -280> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce81190) adding con mode: secure 2024-05-30T16:22:20.588 INFO:tasks.workunit.client.0.smithi006.stderr: -279> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce81190) adding con mode: crc 2024-05-30T16:22:20.588 INFO:tasks.workunit.client.0.smithi006.stderr: -278> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce81190) adding con mode: secure 2024-05-30T16:22:20.588 INFO:tasks.workunit.client.0.smithi006.stderr: -277> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce81190) adding con mode: crc 2024-05-30T16:22:20.588 INFO:tasks.workunit.client.0.smithi006.stderr: -276> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce81190) adding con mode: secure 2024-05-30T16:22:20.588 INFO:tasks.workunit.client.0.smithi006.stderr: -275> 2024-05-30T16:22:19.975+0000 7f6942fc2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:22:20.589 INFO:tasks.workunit.client.0.smithi006.stderr: -274> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient: init 2024-05-30T16:22:20.589 INFO:tasks.workunit.client.0.smithi006.stderr: -273> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf665a0) adding auth protocol: cephx 2024-05-30T16:22:20.589 INFO:tasks.workunit.client.0.smithi006.stderr: -272> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf665a0) adding auth protocol: cephx 2024-05-30T16:22:20.589 INFO:tasks.workunit.client.0.smithi006.stderr: -271> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf665a0) adding auth protocol: cephx 2024-05-30T16:22:20.589 INFO:tasks.workunit.client.0.smithi006.stderr: -270> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf665a0) adding con mode: secure 2024-05-30T16:22:20.589 INFO:tasks.workunit.client.0.smithi006.stderr: -269> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf665a0) adding con mode: crc 2024-05-30T16:22:20.589 INFO:tasks.workunit.client.0.smithi006.stderr: -268> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf665a0) adding con mode: secure 2024-05-30T16:22:20.589 INFO:tasks.workunit.client.0.smithi006.stderr: -267> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf665a0) adding con mode: crc 2024-05-30T16:22:20.589 INFO:tasks.workunit.client.0.smithi006.stderr: -266> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf665a0) adding con mode: secure 2024-05-30T16:22:20.589 INFO:tasks.workunit.client.0.smithi006.stderr: -265> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf665a0) adding con mode: crc 2024-05-30T16:22:20.589 INFO:tasks.workunit.client.0.smithi006.stderr: -264> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf665a0) adding con mode: crc 2024-05-30T16:22:20.589 INFO:tasks.workunit.client.0.smithi006.stderr: -263> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf665a0) adding con mode: secure 2024-05-30T16:22:20.589 INFO:tasks.workunit.client.0.smithi006.stderr: -262> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf665a0) adding con mode: crc 2024-05-30T16:22:20.590 INFO:tasks.workunit.client.0.smithi006.stderr: -261> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf665a0) adding con mode: secure 2024-05-30T16:22:20.590 INFO:tasks.workunit.client.0.smithi006.stderr: -260> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf665a0) adding con mode: crc 2024-05-30T16:22:20.590 INFO:tasks.workunit.client.0.smithi006.stderr: -259> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf665a0) adding con mode: secure 2024-05-30T16:22:20.590 INFO:tasks.workunit.client.0.smithi006.stderr: -258> 2024-05-30T16:22:19.975+0000 7f6942fc2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:22:20.590 INFO:tasks.workunit.client.0.smithi006.stderr: -257> 2024-05-30T16:22:19.975+0000 7f6942fc2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:22:20.590 INFO:tasks.workunit.client.0.smithi006.stderr: -256> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command rotate-key cmddesc rotate-key hook 0x7ffc9bf666e8 EEXIST 2024-05-30T16:22:20.590 INFO:tasks.workunit.client.0.smithi006.stderr: -255> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient: _reopen_session rank -1 2024-05-30T16:22:20.590 INFO:tasks.workunit.client.0.smithi006.stderr: -254> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient: _add_conns ranks=[1,0,2] 2024-05-30T16:22:20.590 INFO:tasks.workunit.client.0.smithi006.stderr: -253> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient(hunting): picked mon.noname-b con 0x55d83ce833d0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:22:20.590 INFO:tasks.workunit.client.0.smithi006.stderr: -252> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient(hunting): picked mon.noname-a con 0x55d83ce868b0 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:22:20.590 INFO:tasks.workunit.client.0.smithi006.stderr: -251> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient(hunting): picked mon.noname-c con 0x55d83ceafcd0 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:22:20.590 INFO:tasks.workunit.client.0.smithi006.stderr: -250> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient(hunting): start opening mon connection 2024-05-30T16:22:20.590 INFO:tasks.workunit.client.0.smithi006.stderr: -249> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient(hunting): start opening mon connection 2024-05-30T16:22:20.590 INFO:tasks.workunit.client.0.smithi006.stderr: -248> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient(hunting): start opening mon connection 2024-05-30T16:22:20.591 INFO:tasks.workunit.client.0.smithi006.stderr: -247> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient(hunting): _renew_subs 2024-05-30T16:22:20.591 INFO:tasks.workunit.client.0.smithi006.stderr: -246> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient(hunting): authenticate will time out at 2679.909976s 2024-05-30T16:22:20.591 INFO:tasks.workunit.client.0.smithi006.stderr: -245> 2024-05-30T16:22:19.975+0000 7f69402d2640 10 monclient(hunting): get_auth_request con 0x55d83ce868b0 auth_method 0 2024-05-30T16:22:20.591 INFO:tasks.workunit.client.0.smithi006.stderr: -244> 2024-05-30T16:22:19.975+0000 7f69402d2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:22:20.591 INFO:tasks.workunit.client.0.smithi006.stderr: -243> 2024-05-30T16:22:19.975+0000 7f69402d2640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:22:20.591 INFO:tasks.workunit.client.0.smithi006.stderr: -242> 2024-05-30T16:22:19.975+0000 7f69402d2640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:22:20.591 INFO:tasks.workunit.client.0.smithi006.stderr: -241> 2024-05-30T16:22:19.975+0000 7f69412d4640 10 monclient(hunting): get_auth_request con 0x55d83ceafcd0 auth_method 0 2024-05-30T16:22:20.591 INFO:tasks.workunit.client.0.smithi006.stderr: -240> 2024-05-30T16:22:19.975+0000 7f69412d4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:22:20.591 INFO:tasks.workunit.client.0.smithi006.stderr: -239> 2024-05-30T16:22:19.975+0000 7f69412d4640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:22:20.591 INFO:tasks.workunit.client.0.smithi006.stderr: -238> 2024-05-30T16:22:19.975+0000 7f69412d4640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:22:20.593 INFO:tasks.workunit.client.0.smithi006.stderr: -237> 2024-05-30T16:22:19.975+0000 7f69402d2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:22:20.593 INFO:tasks.workunit.client.0.smithi006.stderr: -236> 2024-05-30T16:22:19.975+0000 7f69402d2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:22:20.593 INFO:tasks.workunit.client.0.smithi006.stderr: -235> 2024-05-30T16:22:19.975+0000 7f69402d2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:22:20.593 INFO:tasks.workunit.client.0.smithi006.stderr: -234> 2024-05-30T16:22:19.975+0000 7f69412d4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:22:20.593 INFO:tasks.workunit.client.0.smithi006.stderr: -233> 2024-05-30T16:22:19.975+0000 7f69412d4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:22:20.594 INFO:tasks.workunit.client.0.smithi006.stderr: -232> 2024-05-30T16:22:19.975+0000 7f69412d4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:22:20.594 INFO:tasks.workunit.client.0.smithi006.stderr: -231> 2024-05-30T16:22:19.975+0000 7f69402d2640 10 monclient(hunting): handle_auth_done global_id 6963 payload 274 2024-05-30T16:22:20.594 INFO:tasks.workunit.client.0.smithi006.stderr: -230> 2024-05-30T16:22:19.975+0000 7f69402d2640 10 monclient: _finish_hunting 0 2024-05-30T16:22:20.594 INFO:tasks.workunit.client.0.smithi006.stderr: -229> 2024-05-30T16:22:19.975+0000 7f69402d2640 1 monclient: found mon.noname-a 2024-05-30T16:22:20.594 INFO:tasks.workunit.client.0.smithi006.stderr: -228> 2024-05-30T16:22:19.975+0000 7f69402d2640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.6:3300/0 2024-05-30T16:22:20.594 INFO:tasks.workunit.client.0.smithi006.stderr: -227> 2024-05-30T16:22:19.975+0000 7f6817fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:22:20.594 INFO:tasks.workunit.client.0.smithi006.stderr: -226> 2024-05-30T16:22:19.975+0000 7f6817fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-30T16:22:20.594 INFO:tasks.workunit.client.0.smithi006.stderr: -225> 2024-05-30T16:22:19.975+0000 7f6817fff640 10 monclient: dump: 2024-05-30T16:22:20.594 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:22:20.594 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:22:20.594 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.594 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.595 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:22:20.595 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:22:20.595 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:22:20.595 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:22:20.595 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:22:20.595 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.595 INFO:tasks.workunit.client.0.smithi006.stderr: -224> 2024-05-30T16:22:19.975+0000 7f6817fff640 10 monclient: _finish_auth 0 2024-05-30T16:22:20.595 INFO:tasks.workunit.client.0.smithi006.stderr: -223> 2024-05-30T16:22:19.975+0000 7f6817fff640 10 monclient: _check_auth_tickets 2024-05-30T16:22:20.595 INFO:tasks.workunit.client.0.smithi006.stderr: -222> 2024-05-30T16:22:19.975+0000 7f6817fff640 10 monclient: handle_config config(0 keys) 2024-05-30T16:22:20.595 INFO:tasks.workunit.client.0.smithi006.stderr: -221> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 monclient: authenticate success, global_id 6963 2024-05-30T16:22:20.595 INFO:tasks.workunit.client.0.smithi006.stderr: -220> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient: get_monmap_and_config success 2024-05-30T16:22:20.595 INFO:tasks.workunit.client.0.smithi006.stderr: -219> 2024-05-30T16:22:19.975+0000 7f6942fc2980 4 set_mon_vals no callback set 2024-05-30T16:22:20.595 INFO:tasks.workunit.client.0.smithi006.stderr: -218> 2024-05-30T16:22:19.975+0000 7f6817fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:22:20.596 INFO:tasks.workunit.client.0.smithi006.stderr: -217> 2024-05-30T16:22:19.975+0000 7f6817fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-30T16:22:20.596 INFO:tasks.workunit.client.0.smithi006.stderr: -216> 2024-05-30T16:22:19.975+0000 7f6817fff640 10 monclient: dump: 2024-05-30T16:22:20.596 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:22:20.596 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:22:20.596 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.596 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.596 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:22:20.596 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:22:20.596 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:22:20.596 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:22:20.596 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:22:20.596 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.596 INFO:tasks.workunit.client.0.smithi006.stderr: -215> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient: shutdown 2024-05-30T16:22:20.596 INFO:tasks.workunit.client.0.smithi006.stderr: -214> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient: build_initial_monmap 2024-05-30T16:22:20.597 INFO:tasks.workunit.client.0.smithi006.stderr: -213> 2024-05-30T16:22:19.975+0000 7f6942fc2980 1 build_initial for_mkfs: 0 2024-05-30T16:22:20.597 INFO:tasks.workunit.client.0.smithi006.stderr: -212> 2024-05-30T16:22:19.975+0000 7f6942fc2980 10 monclient: monmap: 2024-05-30T16:22:20.597 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 0 2024-05-30T16:22:20.597 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-30T16:22:20.597 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 0.000000 2024-05-30T16:22:20.597 INFO:tasks.workunit.client.0.smithi006.stderr:created 0.000000 2024-05-30T16:22:20.597 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 0 (unknown) 2024-05-30T16:22:20.597 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:22:20.597 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.noname-a 2024-05-30T16:22:20.597 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-b 2024-05-30T16:22:20.597 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.noname-c 2024-05-30T16:22:20.597 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.598 INFO:tasks.workunit.client.0.smithi006.stderr: -211> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce85de0) adding auth protocol: cephx 2024-05-30T16:22:20.598 INFO:tasks.workunit.client.0.smithi006.stderr: -210> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce85de0) adding auth protocol: cephx 2024-05-30T16:22:20.598 INFO:tasks.workunit.client.0.smithi006.stderr: -209> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce85de0) adding auth protocol: cephx 2024-05-30T16:22:20.598 INFO:tasks.workunit.client.0.smithi006.stderr: -208> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce85de0) adding con mode: secure 2024-05-30T16:22:20.598 INFO:tasks.workunit.client.0.smithi006.stderr: -207> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce85de0) adding con mode: crc 2024-05-30T16:22:20.598 INFO:tasks.workunit.client.0.smithi006.stderr: -206> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce85de0) adding con mode: secure 2024-05-30T16:22:20.598 INFO:tasks.workunit.client.0.smithi006.stderr: -205> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce85de0) adding con mode: crc 2024-05-30T16:22:20.598 INFO:tasks.workunit.client.0.smithi006.stderr: -204> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce85de0) adding con mode: secure 2024-05-30T16:22:20.598 INFO:tasks.workunit.client.0.smithi006.stderr: -203> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce85de0) adding con mode: crc 2024-05-30T16:22:20.598 INFO:tasks.workunit.client.0.smithi006.stderr: -202> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce85de0) adding con mode: crc 2024-05-30T16:22:20.598 INFO:tasks.workunit.client.0.smithi006.stderr: -201> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce85de0) adding con mode: secure 2024-05-30T16:22:20.598 INFO:tasks.workunit.client.0.smithi006.stderr: -200> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce85de0) adding con mode: crc 2024-05-30T16:22:20.599 INFO:tasks.workunit.client.0.smithi006.stderr: -199> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce85de0) adding con mode: secure 2024-05-30T16:22:20.599 INFO:tasks.workunit.client.0.smithi006.stderr: -198> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce85de0) adding con mode: crc 2024-05-30T16:22:20.599 INFO:tasks.workunit.client.0.smithi006.stderr: -197> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce85de0) adding con mode: secure 2024-05-30T16:22:20.599 INFO:tasks.workunit.client.0.smithi006.stderr: -196> 2024-05-30T16:22:19.975+0000 7f6942fc2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:22:20.599 INFO:tasks.workunit.client.0.smithi006.stderr: -195> 2024-05-30T16:22:19.975+0000 7f6942fc2980 1 librados: starting msgr at 2024-05-30T16:22:20.599 INFO:tasks.workunit.client.0.smithi006.stderr: -194> 2024-05-30T16:22:19.975+0000 7f6942fc2980 1 librados: starting objecter 2024-05-30T16:22:20.599 INFO:tasks.workunit.client.0.smithi006.stderr: -193> 2024-05-30T16:22:19.975+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command objecter_requests cmddesc objecter_requests hook 0x55d83ceb2990 EEXIST 2024-05-30T16:22:20.599 INFO:tasks.workunit.client.0.smithi006.stderr: -192> 2024-05-30T16:22:19.979+0000 7f6942fc2980 1 librados: setting wanted keys 2024-05-30T16:22:20.599 INFO:tasks.workunit.client.0.smithi006.stderr: -191> 2024-05-30T16:22:19.979+0000 7f6942fc2980 1 librados: calling monclient init 2024-05-30T16:22:20.599 INFO:tasks.workunit.client.0.smithi006.stderr: -190> 2024-05-30T16:22:19.979+0000 7f6942fc2980 10 monclient: init 2024-05-30T16:22:20.599 INFO:tasks.workunit.client.0.smithi006.stderr: -189> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce791a0) adding auth protocol: cephx 2024-05-30T16:22:20.599 INFO:tasks.workunit.client.0.smithi006.stderr: -188> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce791a0) adding auth protocol: cephx 2024-05-30T16:22:20.599 INFO:tasks.workunit.client.0.smithi006.stderr: -187> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce791a0) adding auth protocol: cephx 2024-05-30T16:22:20.600 INFO:tasks.workunit.client.0.smithi006.stderr: -186> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce791a0) adding con mode: secure 2024-05-30T16:22:20.600 INFO:tasks.workunit.client.0.smithi006.stderr: -185> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce791a0) adding con mode: crc 2024-05-30T16:22:20.600 INFO:tasks.workunit.client.0.smithi006.stderr: -184> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce791a0) adding con mode: secure 2024-05-30T16:22:20.600 INFO:tasks.workunit.client.0.smithi006.stderr: -183> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce791a0) adding con mode: crc 2024-05-30T16:22:20.600 INFO:tasks.workunit.client.0.smithi006.stderr: -182> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce791a0) adding con mode: secure 2024-05-30T16:22:20.600 INFO:tasks.workunit.client.0.smithi006.stderr: -181> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce791a0) adding con mode: crc 2024-05-30T16:22:20.600 INFO:tasks.workunit.client.0.smithi006.stderr: -180> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce791a0) adding con mode: crc 2024-05-30T16:22:20.600 INFO:tasks.workunit.client.0.smithi006.stderr: -179> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce791a0) adding con mode: secure 2024-05-30T16:22:20.600 INFO:tasks.workunit.client.0.smithi006.stderr: -178> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce791a0) adding con mode: crc 2024-05-30T16:22:20.600 INFO:tasks.workunit.client.0.smithi006.stderr: -177> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce791a0) adding con mode: secure 2024-05-30T16:22:20.600 INFO:tasks.workunit.client.0.smithi006.stderr: -176> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce791a0) adding con mode: crc 2024-05-30T16:22:20.600 INFO:tasks.workunit.client.0.smithi006.stderr: -175> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 AuthRegistry(0x55d83ce791a0) adding con mode: secure 2024-05-30T16:22:20.601 INFO:tasks.workunit.client.0.smithi006.stderr: -174> 2024-05-30T16:22:19.979+0000 7f6942fc2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:22:20.601 INFO:tasks.workunit.client.0.smithi006.stderr: -173> 2024-05-30T16:22:19.979+0000 7f6942fc2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:22:20.601 INFO:tasks.workunit.client.0.smithi006.stderr: -172> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command rotate-key cmddesc rotate-key hook 0x55d83ce792e8 EEXIST 2024-05-30T16:22:20.601 INFO:tasks.workunit.client.0.smithi006.stderr: -171> 2024-05-30T16:22:19.979+0000 7f6942fc2980 10 monclient: _reopen_session rank -1 2024-05-30T16:22:20.601 INFO:tasks.workunit.client.0.smithi006.stderr: -170> 2024-05-30T16:22:19.979+0000 7f6942fc2980 10 monclient: _add_conns ranks=[1,0,2] 2024-05-30T16:22:20.601 INFO:tasks.workunit.client.0.smithi006.stderr: -169> 2024-05-30T16:22:19.979+0000 7f6942fc2980 10 monclient(hunting): picked mon.noname-b con 0x55d83ce833d0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-30T16:22:20.601 INFO:tasks.workunit.client.0.smithi006.stderr: -168> 2024-05-30T16:22:19.979+0000 7f6942fc2980 10 monclient(hunting): picked mon.noname-a con 0x55d83ce868b0 addr [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] 2024-05-30T16:22:20.601 INFO:tasks.workunit.client.0.smithi006.stderr: -167> 2024-05-30T16:22:19.979+0000 7f6942fc2980 10 monclient(hunting): picked mon.noname-c con 0x55d83ceafcd0 addr [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-05-30T16:22:20.601 INFO:tasks.workunit.client.0.smithi006.stderr: -166> 2024-05-30T16:22:19.979+0000 7f6942fc2980 10 monclient(hunting): start opening mon connection 2024-05-30T16:22:20.601 INFO:tasks.workunit.client.0.smithi006.stderr: -165> 2024-05-30T16:22:19.979+0000 7f6942fc2980 10 monclient(hunting): start opening mon connection 2024-05-30T16:22:20.601 INFO:tasks.workunit.client.0.smithi006.stderr: -164> 2024-05-30T16:22:19.979+0000 7f6942fc2980 10 monclient(hunting): start opening mon connection 2024-05-30T16:22:20.601 INFO:tasks.workunit.client.0.smithi006.stderr: -163> 2024-05-30T16:22:19.979+0000 7f6942fc2980 10 monclient(hunting): _renew_subs 2024-05-30T16:22:20.601 INFO:tasks.workunit.client.0.smithi006.stderr: -162> 2024-05-30T16:22:19.979+0000 7f6942fc2980 10 monclient(hunting): authenticate will time out at 2679.912623s 2024-05-30T16:22:20.601 INFO:tasks.workunit.client.0.smithi006.stderr: -161> 2024-05-30T16:22:19.979+0000 7f69402d2640 10 monclient(hunting): get_auth_request con 0x55d83ce868b0 auth_method 0 2024-05-30T16:22:20.602 INFO:tasks.workunit.client.0.smithi006.stderr: -160> 2024-05-30T16:22:19.979+0000 7f69402d2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:22:20.602 INFO:tasks.workunit.client.0.smithi006.stderr: -159> 2024-05-30T16:22:19.979+0000 7f69402d2640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:22:20.602 INFO:tasks.workunit.client.0.smithi006.stderr: -158> 2024-05-30T16:22:19.979+0000 7f69402d2640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:22:20.602 INFO:tasks.workunit.client.0.smithi006.stderr: -157> 2024-05-30T16:22:19.979+0000 7f69412d4640 10 monclient(hunting): get_auth_request con 0x55d83ceafcd0 auth_method 0 2024-05-30T16:22:20.602 INFO:tasks.workunit.client.0.smithi006.stderr: -156> 2024-05-30T16:22:19.979+0000 7f69412d4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:22:20.602 INFO:tasks.workunit.client.0.smithi006.stderr: -155> 2024-05-30T16:22:19.979+0000 7f69412d4640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:22:20.602 INFO:tasks.workunit.client.0.smithi006.stderr: -154> 2024-05-30T16:22:19.979+0000 7f69412d4640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:22:20.602 INFO:tasks.workunit.client.0.smithi006.stderr: -153> 2024-05-30T16:22:19.979+0000 7f69402d2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:22:20.602 INFO:tasks.workunit.client.0.smithi006.stderr: -152> 2024-05-30T16:22:19.979+0000 7f69402d2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:22:20.602 INFO:tasks.workunit.client.0.smithi006.stderr: -151> 2024-05-30T16:22:19.979+0000 7f69402d2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:22:20.602 INFO:tasks.workunit.client.0.smithi006.stderr: -150> 2024-05-30T16:22:19.979+0000 7f69412d4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-30T16:22:20.602 INFO:tasks.workunit.client.0.smithi006.stderr: -149> 2024-05-30T16:22:19.979+0000 7f69412d4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-30T16:22:20.603 INFO:tasks.workunit.client.0.smithi006.stderr: -148> 2024-05-30T16:22:19.979+0000 7f69412d4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-30T16:22:20.603 INFO:tasks.workunit.client.0.smithi006.stderr: -147> 2024-05-30T16:22:19.979+0000 7f6940ad3640 10 monclient(hunting): get_auth_request con 0x55d83ce833d0 auth_method 0 2024-05-30T16:22:20.603 INFO:tasks.workunit.client.0.smithi006.stderr: -146> 2024-05-30T16:22:19.979+0000 7f6940ad3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-30T16:22:20.603 INFO:tasks.workunit.client.0.smithi006.stderr: -145> 2024-05-30T16:22:19.979+0000 7f6940ad3640 10 monclient(hunting): _init_auth method 2 2024-05-30T16:22:20.603 INFO:tasks.workunit.client.0.smithi006.stderr: -144> 2024-05-30T16:22:19.979+0000 7f6940ad3640 10 monclient(hunting): _init_auth creating new auth 2024-05-30T16:22:20.603 INFO:tasks.workunit.client.0.smithi006.stderr: -143> 2024-05-30T16:22:19.979+0000 7f69402d2640 10 monclient(hunting): handle_auth_done global_id 6966 payload 824 2024-05-30T16:22:20.603 INFO:tasks.workunit.client.0.smithi006.stderr: -142> 2024-05-30T16:22:19.979+0000 7f69402d2640 10 monclient: _finish_hunting 0 2024-05-30T16:22:20.603 INFO:tasks.workunit.client.0.smithi006.stderr: -141> 2024-05-30T16:22:19.979+0000 7f69402d2640 1 monclient: found mon.noname-a 2024-05-30T16:22:20.603 INFO:tasks.workunit.client.0.smithi006.stderr: -140> 2024-05-30T16:22:19.979+0000 7f69402d2640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.6:3300/0 2024-05-30T16:22:20.603 INFO:tasks.workunit.client.0.smithi006.stderr: -139> 2024-05-30T16:22:19.979+0000 7f68167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:22:20.603 INFO:tasks.workunit.client.0.smithi006.stderr: -138> 2024-05-30T16:22:19.979+0000 7f68167fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-30T16:22:20.603 INFO:tasks.workunit.client.0.smithi006.stderr: -137> 2024-05-30T16:22:19.979+0000 7f68167fc640 10 monclient: dump: 2024-05-30T16:22:20.604 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:22:20.604 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:22:20.604 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.604 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.604 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:22:20.604 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:22:20.604 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:22:20.604 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:22:20.604 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:22:20.604 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.604 INFO:tasks.workunit.client.0.smithi006.stderr: -136> 2024-05-30T16:22:19.979+0000 7f68167fc640 10 monclient: _finish_auth 0 2024-05-30T16:22:20.604 INFO:tasks.workunit.client.0.smithi006.stderr: -135> 2024-05-30T16:22:19.979+0000 7f68167fc640 10 monclient: _check_auth_tickets 2024-05-30T16:22:20.604 INFO:tasks.workunit.client.0.smithi006.stderr: -134> 2024-05-30T16:22:19.979+0000 7f68167fc640 10 monclient: handle_config config(0 keys) 2024-05-30T16:22:20.604 INFO:tasks.workunit.client.0.smithi006.stderr: -133> 2024-05-30T16:22:19.979+0000 7f68167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-30T16:22:20.605 INFO:tasks.workunit.client.0.smithi006.stderr: -132> 2024-05-30T16:22:19.979+0000 7f68167fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-30T16:22:20.605 INFO:tasks.workunit.client.0.smithi006.stderr: -131> 2024-05-30T16:22:19.979+0000 7f68167fc640 10 monclient: dump: 2024-05-30T16:22:20.605 INFO:tasks.workunit.client.0.smithi006.stderr:epoch 1 2024-05-30T16:22:20.605 INFO:tasks.workunit.client.0.smithi006.stderr:fsid 87f2315a-4430-477b-ae32-1664164ce9e8 2024-05-30T16:22:20.605 INFO:tasks.workunit.client.0.smithi006.stderr:last_changed 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.605 INFO:tasks.workunit.client.0.smithi006.stderr:created 2024-05-30T15:53:09.772574+0000 2024-05-30T16:22:20.605 INFO:tasks.workunit.client.0.smithi006.stderr:min_mon_release 19 (squid) 2024-05-30T16:22:20.605 INFO:tasks.workunit.client.0.smithi006.stderr:election_strategy: 1 2024-05-30T16:22:20.605 INFO:tasks.workunit.client.0.smithi006.stderr:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2024-05-30T16:22:20.605 INFO:tasks.workunit.client.0.smithi006.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-30T16:22:20.605 INFO:tasks.workunit.client.0.smithi006.stderr:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2024-05-30T16:22:20.605 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.605 INFO:tasks.workunit.client.0.smithi006.stderr: -130> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 monclient: authenticate success, global_id 6966 2024-05-30T16:22:20.606 INFO:tasks.workunit.client.0.smithi006.stderr: -129> 2024-05-30T16:22:19.979+0000 7f6817fff640 4 set_mon_vals no callback set 2024-05-30T16:22:20.606 INFO:tasks.workunit.client.0.smithi006.stderr: -128> 2024-05-30T16:22:19.979+0000 7f6942fc2980 10 monclient: _renew_subs 2024-05-30T16:22:20.606 INFO:tasks.workunit.client.0.smithi006.stderr: -127> 2024-05-30T16:22:19.979+0000 7f6942fc2980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:22:20.606 INFO:tasks.workunit.client.0.smithi006.stderr: -126> 2024-05-30T16:22:19.979+0000 7f6942fc2980 10 monclient: _renew_subs 2024-05-30T16:22:20.606 INFO:tasks.workunit.client.0.smithi006.stderr: -125> 2024-05-30T16:22:19.979+0000 7f6942fc2980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.6:3300/0 2024-05-30T16:22:20.606 INFO:tasks.workunit.client.0.smithi006.stderr: -124> 2024-05-30T16:22:19.979+0000 7f6942fc2980 1 librados: init done 2024-05-30T16:22:20.606 INFO:tasks.workunit.client.0.smithi006.stderr: -123> 2024-05-30T16:22:19.979+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command cr dump hook 0x55d83cea6608 2024-05-30T16:22:20.606 INFO:tasks.workunit.client.0.smithi006.stderr: -122> 2024-05-30T16:22:19.979+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.606 INFO:tasks.workunit.client.0.smithi006.stderr: -121> 2024-05-30T16:22:19.979+0000 7f68167fc640 4 mgrc handle_mgr_map Got map version 5 2024-05-30T16:22:20.606 INFO:tasks.workunit.client.0.smithi006.stderr: -120> 2024-05-30T16:22:19.979+0000 7f68167fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:22:20.606 INFO:tasks.workunit.client.0.smithi006.stderr: -119> 2024-05-30T16:22:19.979+0000 7f68167fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.204:6832/2105440097,v1:172.21.15.204:6833/2105440097] 2024-05-30T16:22:20.606 INFO:tasks.workunit.client.0.smithi006.stderr: -118> 2024-05-30T16:22:19.983+0000 7f69412d4640 10 monclient: get_auth_request con 0x55d83cecee00 auth_method 0 2024-05-30T16:22:20.607 INFO:tasks.workunit.client.0.smithi006.stderr: -117> 2024-05-30T16:22:19.983+0000 7f6940ad3640 10 monclient: get_auth_request con 0x7f680c06dde0 auth_method 0 2024-05-30T16:22:20.607 INFO:tasks.workunit.client.0.smithi006.stderr: -116> 2024-05-30T16:22:19.983+0000 7f6942fc2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:20.607 INFO:tasks.workunit.client.0.smithi006.stderr: -115> 2024-05-30T16:22:19.983+0000 7f6942fc2980 20 realm 2024-05-30T16:22:20.607 INFO:tasks.workunit.client.0.smithi006.stderr: -114> 2024-05-30T16:22:19.983+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.607 INFO:tasks.workunit.client.0.smithi006.stderr: -113> 2024-05-30T16:22:19.983+0000 7f6942fc2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:20.607 INFO:tasks.workunit.client.0.smithi006.stderr: -112> 2024-05-30T16:22:19.983+0000 7f6942fc2980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:22:20.607 INFO:tasks.workunit.client.0.smithi006.stderr: -111> 2024-05-30T16:22:19.983+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.607 INFO:tasks.workunit.client.0.smithi006.stderr: -110> 2024-05-30T16:22:19.983+0000 7f6942fc2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:20.607 INFO:tasks.workunit.client.0.smithi006.stderr: -109> 2024-05-30T16:22:19.983+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.607 INFO:tasks.workunit.client.0.smithi006.stderr: -108> 2024-05-30T16:22:19.983+0000 7f6940ad3640 10 monclient: get_auth_request con 0x55d83ced2ff0 auth_method 0 2024-05-30T16:22:20.607 INFO:tasks.workunit.client.0.smithi006.stderr: -107> 2024-05-30T16:22:19.983+0000 7f6942fc2980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:20.607 INFO:tasks.workunit.client.0.smithi006.stderr: -106> 2024-05-30T16:22:19.983+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.607 INFO:tasks.workunit.client.0.smithi006.stderr: -105> 2024-05-30T16:22:19.983+0000 7f6942fc2980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:22:20.607 INFO:tasks.workunit.client.0.smithi006.stderr: -104> 2024-05-30T16:22:19.983+0000 7f6942fc2980 20 searching for the correct realm 2024-05-30T16:22:20.608 INFO:tasks.workunit.client.0.smithi006.stderr: -103> 2024-05-30T16:22:19.987+0000 7f69402d2640 10 monclient: get_auth_request con 0x55d83ced82e0 auth_method 0 2024-05-30T16:22:20.608 INFO:tasks.workunit.client.0.smithi006.stderr: -102> 2024-05-30T16:22:19.987+0000 7f69412d4640 10 monclient: get_auth_request con 0x7f69340f2070 auth_method 0 2024-05-30T16:22:20.608 INFO:tasks.workunit.client.0.smithi006.stderr: -101> 2024-05-30T16:22:19.987+0000 7f6940ad3640 10 monclient: get_auth_request con 0x7f6938059c90 auth_method 0 2024-05-30T16:22:20.608 INFO:tasks.workunit.client.0.smithi006.stderr: -100> 2024-05-30T16:22:19.987+0000 7f69402d2640 10 monclient: get_auth_request con 0x7f6930045040 auth_method 0 2024-05-30T16:22:20.608 INFO:tasks.workunit.client.0.smithi006.stderr: -99> 2024-05-30T16:22:19.991+0000 7f69402d2640 0 -- 172.21.15.6:0/1226585167 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f6930045040 msgr2=0x7f6930045420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:20.608 INFO:tasks.workunit.client.0.smithi006.stderr: -98> 2024-05-30T16:22:20.195+0000 7f69402d2640 10 monclient: get_auth_request con 0x7f6930045040 auth_method 0 2024-05-30T16:22:20.608 INFO:tasks.workunit.client.0.smithi006.stderr: -97> 2024-05-30T16:22:20.199+0000 7f69412d4640 10 monclient: get_auth_request con 0x7f6934003030 auth_method 0 2024-05-30T16:22:20.608 INFO:tasks.workunit.client.0.smithi006.stderr: -96> 2024-05-30T16:22:20.207+0000 7f6940ad3640 10 monclient: get_auth_request con 0x7f69340093d0 auth_method 0 2024-05-30T16:22:20.608 INFO:tasks.workunit.client.0.smithi006.stderr: -95> 2024-05-30T16:22:20.207+0000 7f6940ad3640 0 -- 172.21.15.6:0/1226585167 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f69340093d0 msgr2=0x7f69340c56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:20.608 INFO:tasks.workunit.client.0.smithi006.stderr: -94> 2024-05-30T16:22:20.411+0000 7f6940ad3640 10 monclient: get_auth_request con 0x7f69340093d0 auth_method 0 2024-05-30T16:22:20.608 INFO:tasks.workunit.client.0.smithi006.stderr: -93> 2024-05-30T16:22:20.423+0000 7f6942fc2980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:22:20.608 INFO:tasks.workunit.client.0.smithi006.stderr: -92> 2024-05-30T16:22:20.423+0000 7f6942fc2980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:22:20.608 INFO:tasks.workunit.client.0.smithi006.stderr: -91> 2024-05-30T16:22:20.423+0000 7f6942fc2980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:22:20.609 INFO:tasks.workunit.client.0.smithi006.stderr: -90> 2024-05-30T16:22:20.423+0000 7f6942fc2980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:22:20.609 INFO:tasks.workunit.client.0.smithi006.stderr: -89> 2024-05-30T16:22:20.423+0000 7f6942fc2980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:22:20.609 INFO:tasks.workunit.client.0.smithi006.stderr: -88> 2024-05-30T16:22:20.423+0000 7f6942fc2980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:22:20.609 INFO:tasks.workunit.client.0.smithi006.stderr: -87> 2024-05-30T16:22:20.423+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.609 INFO:tasks.workunit.client.0.smithi006.stderr: -86> 2024-05-30T16:22:20.427+0000 7f6942fc2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:20.609 INFO:tasks.workunit.client.0.smithi006.stderr: -85> 2024-05-30T16:22:20.427+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.609 INFO:tasks.workunit.client.0.smithi006.stderr: -84> 2024-05-30T16:22:20.427+0000 7f6942fc2980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:20.609 INFO:tasks.workunit.client.0.smithi006.stderr: -83> 2024-05-30T16:22:20.427+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.609 INFO:tasks.workunit.client.0.smithi006.stderr: -82> 2024-05-30T16:22:20.427+0000 7f6942fc2980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:22:20.609 INFO:tasks.workunit.client.0.smithi006.stderr: -81> 2024-05-30T16:22:20.427+0000 7f6942fc2980 20 zone default found 2024-05-30T16:22:20.610 INFO:tasks.workunit.client.0.smithi006.stderr: -80> 2024-05-30T16:22:20.427+0000 7f6942fc2980 4 Realm: () 2024-05-30T16:22:20.610 INFO:tasks.workunit.client.0.smithi006.stderr: -79> 2024-05-30T16:22:20.427+0000 7f6942fc2980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:22:20.610 INFO:tasks.workunit.client.0.smithi006.stderr: -78> 2024-05-30T16:22:20.427+0000 7f6942fc2980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:22:20.610 INFO:tasks.workunit.client.0.smithi006.stderr: -77> 2024-05-30T16:22:20.427+0000 7f6942fc2980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:22:20.610 INFO:tasks.workunit.client.0.smithi006.stderr: -76> 2024-05-30T16:22:20.427+0000 7f6942fc2980 20 zonegroup default 2024-05-30T16:22:20.610 INFO:tasks.workunit.client.0.smithi006.stderr: -75> 2024-05-30T16:22:20.427+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.610 INFO:tasks.workunit.client.0.smithi006.stderr: -74> 2024-05-30T16:22:20.427+0000 7f6942fc2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:20.610 INFO:tasks.workunit.client.0.smithi006.stderr: -73> 2024-05-30T16:22:20.427+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.610 INFO:tasks.workunit.client.0.smithi006.stderr: -72> 2024-05-30T16:22:20.427+0000 7f6942fc2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:20.610 INFO:tasks.workunit.client.0.smithi006.stderr: -71> 2024-05-30T16:22:20.427+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.610 INFO:tasks.workunit.client.0.smithi006.stderr: -70> 2024-05-30T16:22:20.431+0000 7f6942fc2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:20.610 INFO:tasks.workunit.client.0.smithi006.stderr: -69> 2024-05-30T16:22:20.431+0000 7f6942fc2980 20 started sync module instance, tier type = 2024-05-30T16:22:20.611 INFO:tasks.workunit.client.0.smithi006.stderr: -68> 2024-05-30T16:22:20.431+0000 7f6942fc2980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:22:20.611 INFO:tasks.workunit.client.0.smithi006.stderr: -67> 2024-05-30T16:22:20.451+0000 7f6942fc2980 20 add_watcher() i=0 2024-05-30T16:22:20.611 INFO:tasks.workunit.client.0.smithi006.stderr: -66> 2024-05-30T16:22:20.451+0000 7f6942fc2980 20 add_watcher() i=1 2024-05-30T16:22:20.611 INFO:tasks.workunit.client.0.smithi006.stderr: -65> 2024-05-30T16:22:20.451+0000 7f6942fc2980 20 add_watcher() i=2 2024-05-30T16:22:20.611 INFO:tasks.workunit.client.0.smithi006.stderr: -64> 2024-05-30T16:22:20.451+0000 7f6942fc2980 20 add_watcher() i=3 2024-05-30T16:22:20.611 INFO:tasks.workunit.client.0.smithi006.stderr: -63> 2024-05-30T16:22:20.451+0000 7f6942fc2980 20 add_watcher() i=4 2024-05-30T16:22:20.611 INFO:tasks.workunit.client.0.smithi006.stderr: -62> 2024-05-30T16:22:20.451+0000 7f6942fc2980 20 add_watcher() i=5 2024-05-30T16:22:20.611 INFO:tasks.workunit.client.0.smithi006.stderr: -61> 2024-05-30T16:22:20.451+0000 7f6942fc2980 20 add_watcher() i=6 2024-05-30T16:22:20.611 INFO:tasks.workunit.client.0.smithi006.stderr: -60> 2024-05-30T16:22:20.455+0000 7f6942fc2980 20 add_watcher() i=7 2024-05-30T16:22:20.611 INFO:tasks.workunit.client.0.smithi006.stderr: -59> 2024-05-30T16:22:20.455+0000 7f6942fc2980 2 all 8 watchers are set, enabling cache 2024-05-30T16:22:20.611 INFO:tasks.workunit.client.0.smithi006.stderr: -58> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf66a80) adding auth protocol: cephx 2024-05-30T16:22:20.611 INFO:tasks.workunit.client.0.smithi006.stderr: -57> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf66a80) adding auth protocol: cephx 2024-05-30T16:22:20.611 INFO:tasks.workunit.client.0.smithi006.stderr: -56> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf66a80) adding auth protocol: cephx 2024-05-30T16:22:20.612 INFO:tasks.workunit.client.0.smithi006.stderr: -55> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf66a80) adding con mode: secure 2024-05-30T16:22:20.612 INFO:tasks.workunit.client.0.smithi006.stderr: -54> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf66a80) adding con mode: crc 2024-05-30T16:22:20.612 INFO:tasks.workunit.client.0.smithi006.stderr: -53> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf66a80) adding con mode: secure 2024-05-30T16:22:20.612 INFO:tasks.workunit.client.0.smithi006.stderr: -52> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf66a80) adding con mode: crc 2024-05-30T16:22:20.612 INFO:tasks.workunit.client.0.smithi006.stderr: -51> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf66a80) adding con mode: secure 2024-05-30T16:22:20.612 INFO:tasks.workunit.client.0.smithi006.stderr: -50> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf66a80) adding con mode: crc 2024-05-30T16:22:20.612 INFO:tasks.workunit.client.0.smithi006.stderr: -49> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf66a80) adding con mode: crc 2024-05-30T16:22:20.612 INFO:tasks.workunit.client.0.smithi006.stderr: -48> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf66a80) adding con mode: secure 2024-05-30T16:22:20.612 INFO:tasks.workunit.client.0.smithi006.stderr: -47> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf66a80) adding con mode: crc 2024-05-30T16:22:20.612 INFO:tasks.workunit.client.0.smithi006.stderr: -46> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf66a80) adding con mode: secure 2024-05-30T16:22:20.612 INFO:tasks.workunit.client.0.smithi006.stderr: -45> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf66a80) adding con mode: crc 2024-05-30T16:22:20.612 INFO:tasks.workunit.client.0.smithi006.stderr: -44> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 AuthRegistry(0x7ffc9bf66a80) adding con mode: secure 2024-05-30T16:22:20.612 INFO:tasks.workunit.client.0.smithi006.stderr: -43> 2024-05-30T16:22:20.459+0000 7f67f6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:20.613 INFO:tasks.workunit.client.0.smithi006.stderr: -42> 2024-05-30T16:22:20.459+0000 7f6942fc2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-30T16:22:20.613 INFO:tasks.workunit.client.0.smithi006.stderr: -41> 2024-05-30T16:22:20.459+0000 7f6942fc2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:22:20.613 INFO:tasks.workunit.client.0.smithi006.stderr: -40> 2024-05-30T16:22:20.459+0000 7f6942fc2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:22:20.613 INFO:tasks.workunit.client.0.smithi006.stderr: -39> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command cache list hook 0x55d83cea4ba0 2024-05-30T16:22:20.613 INFO:tasks.workunit.client.0.smithi006.stderr: -38> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command cache inspect hook 0x55d83cea4ba0 2024-05-30T16:22:20.613 INFO:tasks.workunit.client.0.smithi006.stderr: -37> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command cache erase hook 0x55d83cea4ba0 2024-05-30T16:22:20.613 INFO:tasks.workunit.client.0.smithi006.stderr: -36> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command cache zap hook 0x55d83cea4ba0 2024-05-30T16:22:20.613 INFO:tasks.workunit.client.0.smithi006.stderr: -35> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 note: GC not initialized 2024-05-30T16:22:20.613 INFO:tasks.workunit.client.0.smithi006.stderr: -34> 2024-05-30T16:22:20.459+0000 7f67f5fe3640 20 reqs_thread_entry: start 2024-05-30T16:22:20.613 INFO:tasks.workunit.client.0.smithi006.stderr: -33> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command sync trace show hook 0x7f69340abef0 2024-05-30T16:22:20.613 INFO:tasks.workunit.client.0.smithi006.stderr: -32> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command sync trace history hook 0x7f69340abef0 2024-05-30T16:22:20.613 INFO:tasks.workunit.client.0.smithi006.stderr: -31> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command sync trace active hook 0x7f69340abef0 2024-05-30T16:22:20.613 INFO:tasks.workunit.client.0.smithi006.stderr: -30> 2024-05-30T16:22:20.459+0000 7f6942fc2980 5 asok(0x55d83cbda790) register_command sync trace active_short hook 0x7f69340abef0 2024-05-30T16:22:20.614 INFO:tasks.workunit.client.0.smithi006.stderr: -29> 2024-05-30T16:22:20.459+0000 7f6942fc2980 20 init_complete bucket index max shards: 11 2024-05-30T16:22:20.614 INFO:tasks.workunit.client.0.smithi006.stderr: -28> 2024-05-30T16:22:20.459+0000 7f6942fc2980 20 Filter name: none 2024-05-30T16:22:20.614 INFO:tasks.workunit.client.0.smithi006.stderr: -27> 2024-05-30T16:22:20.459+0000 7f67677fe640 20 reqs_thread_entry: start 2024-05-30T16:22:20.614 INFO:tasks.workunit.client.0.smithi006.stderr: -26> 2024-05-30T16:22:20.459+0000 7f6942fc2980 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2024-05-30T16:22:20.614 INFO:tasks.workunit.client.0.smithi006.stderr: -25> 2024-05-30T16:22:20.459+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.614 INFO:tasks.workunit.client.0.smithi006.stderr: -24> 2024-05-30T16:22:20.459+0000 7f6942fc2980 20 rados_obj.operate() r=0 bl.length=211 2024-05-30T16:22:20.614 INFO:tasks.workunit.client.0.smithi006.stderr: -23> 2024-05-30T16:22:20.459+0000 7f6942fc2980 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2024-05-30T16:22:20.614 INFO:tasks.workunit.client.0.smithi006.stderr: -22> 2024-05-30T16:22:20.459+0000 7f6942fc2980 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2024-05-30T16:22:20.614 INFO:tasks.workunit.client.0.smithi006.stderr: -21> 2024-05-30T16:22:20.459+0000 7f6942fc2980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 : miss 2024-05-30T16:22:20.614 INFO:tasks.workunit.client.0.smithi006.stderr: -20> 2024-05-30T16:22:20.459+0000 7f6942fc2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.614 INFO:tasks.workunit.client.0.smithi006.stderr: -19> 2024-05-30T16:22:20.463+0000 7f6942fc2980 20 rados_obj.operate() r=0 bl.length=425 2024-05-30T16:22:20.614 INFO:tasks.workunit.client.0.smithi006.stderr: -18> 2024-05-30T16:22:20.463+0000 7f6942fc2980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 info.flags=0x17 2024-05-30T16:22:20.614 INFO:tasks.workunit.client.0.smithi006.stderr: -17> 2024-05-30T16:22:20.463+0000 7f6942fc2980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 to cache LRU end 2024-05-30T16:22:20.615 INFO:tasks.workunit.client.0.smithi006.stderr: -16> 2024-05-30T16:22:20.463+0000 7f6942fc2980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:22:20.615 INFO:tasks.workunit.client.0.smithi006.stderr: -15> 2024-05-30T16:22:20.463+0000 7f6942fc2980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:22:20.615 INFO:tasks.workunit.client.0.smithi006.stderr: -14> 2024-05-30T16:22:20.463+0000 7f6942fc2980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 2024-05-30T16:22:20.615 INFO:tasks.workunit.client.0.smithi006.stderr: -13> 2024-05-30T16:22:20.471+0000 7f6942fc2980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 info.flags=0x17 2024-05-30T16:22:20.615 INFO:tasks.workunit.client.0.smithi006.stderr: -12> 2024-05-30T16:22:20.471+0000 7f6942fc2980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 to cache LRU end 2024-05-30T16:22:20.615 INFO:tasks.workunit.client.0.smithi006.stderr: -11> 2024-05-30T16:22:20.471+0000 7f6942fc2980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:22:20.615 INFO:tasks.workunit.client.0.smithi006.stderr: -10> 2024-05-30T16:22:20.471+0000 7f6942fc2980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29, ofs0, ns] 2024-05-30T16:22:20.615 INFO:tasks.workunit.client.0.smithi006.stderr: -9> 2024-05-30T16:22:20.471+0000 7f6817fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431637 cookie 94387223527648 notifier 6966 bl.length()=987 2024-05-30T16:22:20.615 INFO:tasks.workunit.client.0.smithi006.stderr: -8> 2024-05-30T16:22:20.471+0000 7f6817fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 info.flags=0x17 2024-05-30T16:22:20.615 INFO:tasks.workunit.client.0.smithi006.stderr: -7> 2024-05-30T16:22:20.471+0000 7f6817fff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 to cache LRU end 2024-05-30T16:22:20.615 INFO:tasks.workunit.client.0.smithi006.stderr: -6> 2024-05-30T16:22:20.471+0000 7f6817fff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:22:20.615 INFO:tasks.workunit.client.0.smithi006.stderr: -5> 2024-05-30T16:22:20.475+0000 7f6942fc2980 20 handle_bi_update(): bucket=:abort-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29]): orig_sources= new_sources= 2024-05-30T16:22:20.615 INFO:tasks.workunit.client.0.smithi006.stderr: -4> 2024-05-30T16:22:20.475+0000 7f6942fc2980 20 handle_bi_update(): bucket=:abort-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29]): potential sources added=[] removed=[] 2024-05-30T16:22:20.616 INFO:tasks.workunit.client.0.smithi006.stderr: -3> 2024-05-30T16:22:20.475+0000 7f6942fc2980 20 handle_bi_update(): bucket=:abort-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29]): orig_dests= new_dests= 2024-05-30T16:22:20.616 INFO:tasks.workunit.client.0.smithi006.stderr: -2> 2024-05-30T16:22:20.475+0000 7f6942fc2980 20 handle_bi_update(): bucket=:abort-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29]): potential dests added=[] removed=[] 2024-05-30T16:22:20.616 INFO:tasks.workunit.client.0.smithi006.stderr: -1> 2024-05-30T16:22:20.479+0000 7f6942fc2980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f6942fc2980 time 2024-05-30T16:22:20.482496+0000 2024-05-30T16:22:20.616 INFO:tasks.workunit.client.0.smithi006.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-30T16:22:20.616 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.616 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:22:20.616 INFO:tasks.workunit.client.0.smithi006.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f6946899cff] 2024-05-30T16:22:20.616 INFO:tasks.workunit.client.0.smithi006.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7f6946899eb3] 2024-05-30T16:22:20.616 INFO:tasks.workunit.client.0.smithi006.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55d83b4d0f81] 2024-05-30T16:22:20.616 INFO:tasks.workunit.client.0.smithi006.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x31b) [0x55d83b4e2f1b] 2024-05-30T16:22:20.616 INFO:tasks.workunit.client.0.smithi006.stderr: 5: main() 2024-05-30T16:22:20.616 INFO:tasks.workunit.client.0.smithi006.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f6945702d90] 2024-05-30T16:22:20.616 INFO:tasks.workunit.client.0.smithi006.stderr: 7: __libc_start_main() 2024-05-30T16:22:20.617 INFO:tasks.workunit.client.0.smithi006.stderr: 8: _start() 2024-05-30T16:22:20.617 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.617 INFO:tasks.workunit.client.0.smithi006.stderr: 0> 2024-05-30T16:22:20.479+0000 7f6942fc2980 -1 *** Caught signal (Aborted) ** 2024-05-30T16:22:20.617 INFO:tasks.workunit.client.0.smithi006.stderr: in thread 7f6942fc2980 thread_name:radosgw-admin 2024-05-30T16:22:20.617 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.617 INFO:tasks.workunit.client.0.smithi006.stderr: ceph version 19.0.0-4011-gda98024b (da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2) squid (dev) 2024-05-30T16:22:20.617 INFO:tasks.workunit.client.0.smithi006.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f694571b520] 2024-05-30T16:22:20.617 INFO:tasks.workunit.client.0.smithi006.stderr: 2: pthread_kill() 2024-05-30T16:22:20.617 INFO:tasks.workunit.client.0.smithi006.stderr: 3: raise() 2024-05-30T16:22:20.617 INFO:tasks.workunit.client.0.smithi006.stderr: 4: abort() 2024-05-30T16:22:20.617 INFO:tasks.workunit.client.0.smithi006.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f6946899d59] 2024-05-30T16:22:20.617 INFO:tasks.workunit.client.0.smithi006.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182eb3) [0x7f6946899eb3] 2024-05-30T16:22:20.617 INFO:tasks.workunit.client.0.smithi006.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55d83b4d0f81] 2024-05-30T16:22:20.618 INFO:tasks.workunit.client.0.smithi006.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x31b) [0x55d83b4e2f1b] 2024-05-30T16:22:20.618 INFO:tasks.workunit.client.0.smithi006.stderr: 9: main() 2024-05-30T16:22:20.618 INFO:tasks.workunit.client.0.smithi006.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f6945702d90] 2024-05-30T16:22:20.618 INFO:tasks.workunit.client.0.smithi006.stderr: 11: __libc_start_main() 2024-05-30T16:22:20.618 INFO:tasks.workunit.client.0.smithi006.stderr: 12: _start() 2024-05-30T16:22:20.618 INFO:tasks.workunit.client.0.smithi006.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-30T16:22:20.618 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.618 INFO:tasks.workunit.client.0.smithi006.stderr:--- logging levels --- 2024-05-30T16:22:20.618 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 none 2024-05-30T16:22:20.618 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 lockdep 2024-05-30T16:22:20.618 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 context 2024-05-30T16:22:20.618 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 crush 2024-05-30T16:22:20.619 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds 2024-05-30T16:22:20.619 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_balancer 2024-05-30T16:22:20.619 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_locker 2024-05-30T16:22:20.619 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_log 2024-05-30T16:22:20.619 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_log_expire 2024-05-30T16:22:20.619 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mds_migrator 2024-05-30T16:22:20.619 INFO:tasks.workunit.client.0.smithi006.stderr: 3/ 5 mds_quiesce 2024-05-30T16:22:20.619 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 buffer 2024-05-30T16:22:20.619 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 timer 2024-05-30T16:22:20.619 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 filer 2024-05-30T16:22:20.619 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 striper 2024-05-30T16:22:20.619 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 1 objecter 2024-05-30T16:22:20.619 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rados 2024-05-30T16:22:20.620 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd 2024-05-30T16:22:20.620 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd_mirror 2024-05-30T16:22:20.620 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd_replay 2024-05-30T16:22:20.620 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 rbd_pwl 2024-05-30T16:22:20.620 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 journaler 2024-05-30T16:22:20.620 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 objectcacher 2024-05-30T16:22:20.620 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 immutable_obj_cache 2024-05-30T16:22:20.620 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 client 2024-05-30T16:22:20.620 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 osd 2024-05-30T16:22:20.620 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 optracker 2024-05-30T16:22:20.620 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 objclass 2024-05-30T16:22:20.620 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 3 filestore 2024-05-30T16:22:20.620 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 3 journal 2024-05-30T16:22:20.620 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 0 ms 2024-05-30T16:22:20.621 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mon 2024-05-30T16:22:20.621 INFO:tasks.workunit.client.0.smithi006.stderr: 0/10 monc 2024-05-30T16:22:20.621 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 paxos 2024-05-30T16:22:20.621 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 tp 2024-05-30T16:22:20.621 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 auth 2024-05-30T16:22:20.621 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 crypto 2024-05-30T16:22:20.621 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 finisher 2024-05-30T16:22:20.621 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 reserver 2024-05-30T16:22:20.621 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 heartbeatmap 2024-05-30T16:22:20.621 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 perfcounter 2024-05-30T16:22:20.621 INFO:tasks.workunit.client.0.smithi006.stderr: 20/20 rgw 2024-05-30T16:22:20.621 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_sync 2024-05-30T16:22:20.621 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_datacache 2024-05-30T16:22:20.621 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_access 2024-05-30T16:22:20.622 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_dbstore 2024-05-30T16:22:20.622 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_flight 2024-05-30T16:22:20.622 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 rgw_lifecycle 2024-05-30T16:22:20.622 INFO:tasks.workunit.client.0.smithi006.stderr: 20/20 rgw_notification 2024-05-30T16:22:20.622 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 javaclient 2024-05-30T16:22:20.622 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 asok 2024-05-30T16:22:20.622 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 1 throttle 2024-05-30T16:22:20.622 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 0 refs 2024-05-30T16:22:20.622 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 compressor 2024-05-30T16:22:20.622 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 bluestore 2024-05-30T16:22:20.622 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 bluefs 2024-05-30T16:22:20.622 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 3 bdev 2024-05-30T16:22:20.622 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 kstore 2024-05-30T16:22:20.623 INFO:tasks.workunit.client.0.smithi006.stderr: 4/ 5 rocksdb 2024-05-30T16:22:20.623 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 fuse 2024-05-30T16:22:20.623 INFO:tasks.workunit.client.0.smithi006.stderr: 2/ 5 mgr 2024-05-30T16:22:20.623 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mgrc 2024-05-30T16:22:20.623 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 dpdk 2024-05-30T16:22:20.623 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 eventtrace 2024-05-30T16:22:20.623 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 prioritycache 2024-05-30T16:22:20.623 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 test 2024-05-30T16:22:20.623 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 cephfs_mirror 2024-05-30T16:22:20.623 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 cephsqlite 2024-05-30T16:22:20.623 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 crimson_interrupt 2024-05-30T16:22:20.623 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore 2024-05-30T16:22:20.623 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_onode 2024-05-30T16:22:20.624 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_odata 2024-05-30T16:22:20.624 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_omap 2024-05-30T16:22:20.624 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_tm 2024-05-30T16:22:20.624 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_t 2024-05-30T16:22:20.624 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_cleaner 2024-05-30T16:22:20.624 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_epm 2024-05-30T16:22:20.624 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_lba 2024-05-30T16:22:20.624 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_fixedkv_tree 2024-05-30T16:22:20.624 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_cache 2024-05-30T16:22:20.624 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_journal 2024-05-30T16:22:20.624 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_device 2024-05-30T16:22:20.624 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 seastore_backref 2024-05-30T16:22:20.624 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 alienstore 2024-05-30T16:22:20.624 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 mclock 2024-05-30T16:22:20.625 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 cyanstore 2024-05-30T16:22:20.625 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 ceph_exporter 2024-05-30T16:22:20.625 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 memstore 2024-05-30T16:22:20.625 INFO:tasks.workunit.client.0.smithi006.stderr: 1/ 5 trace 2024-05-30T16:22:20.625 INFO:tasks.workunit.client.0.smithi006.stderr: 0/ 5 ceph_dedup 2024-05-30T16:22:20.625 INFO:tasks.workunit.client.0.smithi006.stderr: -2/-2 (syslog threshold) 2024-05-30T16:22:20.625 INFO:tasks.workunit.client.0.smithi006.stderr: 99/99 (stderr threshold) 2024-05-30T16:22:20.625 INFO:tasks.workunit.client.0.smithi006.stderr:--- pthread ID / name mapping for recent threads --- 2024-05-30T16:22:20.625 INFO:tasks.workunit.client.0.smithi006.stderr: 7f67677fe640 / http_manager 2024-05-30T16:22:20.625 INFO:tasks.workunit.client.0.smithi006.stderr: 7f67f5fe3640 / http_manager 2024-05-30T16:22:20.625 INFO:tasks.workunit.client.0.smithi006.stderr: 7f67f6fe5640 / rgw_dt_lg_renew 2024-05-30T16:22:20.625 INFO:tasks.workunit.client.0.smithi006.stderr: 7f68167fc640 / ms_dispatch 2024-05-30T16:22:20.625 INFO:tasks.workunit.client.0.smithi006.stderr: 7f6817fff640 / io_context_pool 2024-05-30T16:22:20.626 INFO:tasks.workunit.client.0.smithi006.stderr: 7f68267fc640 / ms_dispatch 2024-05-30T16:22:20.626 INFO:tasks.workunit.client.0.smithi006.stderr: 7f682d60a640 / ms_dispatch 2024-05-30T16:22:20.626 INFO:tasks.workunit.client.0.smithi006.stderr: 7f682ee0d640 / io_context_pool 2024-05-30T16:22:20.626 INFO:tasks.workunit.client.0.smithi006.stderr: 7f69402d2640 / msgr-worker-2 2024-05-30T16:22:20.626 INFO:tasks.workunit.client.0.smithi006.stderr: 7f6940ad3640 / msgr-worker-1 2024-05-30T16:22:20.626 INFO:tasks.workunit.client.0.smithi006.stderr: 7f69412d4640 / msgr-worker-0 2024-05-30T16:22:20.626 INFO:tasks.workunit.client.0.smithi006.stderr: 7f6941ad5640 / io_context_pool 2024-05-30T16:22:20.626 INFO:tasks.workunit.client.0.smithi006.stderr: 7f6942fc2980 / radosgw-admin 2024-05-30T16:22:20.626 INFO:tasks.workunit.client.0.smithi006.stderr: max_recent 500 2024-05-30T16:22:20.626 INFO:tasks.workunit.client.0.smithi006.stderr: max_new 1000 2024-05-30T16:22:20.626 INFO:tasks.workunit.client.0.smithi006.stderr: log_file /var/lib/ceph/crash/2024-05-30T16:22:20.484567Z_b22a18bc-dcde-4f49-b9af-ac06a3b9f94f/log 2024-05-30T16:22:20.626 INFO:tasks.workunit.client.0.smithi006.stderr:--- end dump of recent events --- 2024-05-30T16:22:20.626 INFO:tasks.workunit.client.0.smithi006.stderr:Aborted 2024-05-30T16:22:20.626 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket abort-at-do-reshard 2024-05-30T16:22:20.628 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:22:20.628 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:22:20.628 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.563+0000 7faa0b3fe980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.628 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.567+0000 7faa0b3fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:20.628 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.567+0000 7faa0b3fe980 20 realm 2024-05-30T16:22:20.628 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.567+0000 7faa0b3fe980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.628 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.567+0000 7faa0b3fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:20.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.567+0000 7faa0b3fe980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:22:20.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.567+0000 7faa0b3fe980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.567+0000 7faa0b3fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:20.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.567+0000 7faa0b3fe980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.571+0000 7faa0b3fe980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:20.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.571+0000 7faa0b3fe980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.571+0000 7faa0b3fe980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:22:20.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.571+0000 7faa0b3fe980 20 searching for the correct realm 2024-05-30T16:22:20.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.575+0000 7faa09710640 0 -- 172.21.15.6:0/1340554551 >> [v2:172.21.15.204:6823/3156311527,v1:172.21.15.204:6825/3156311527] conn(0x7fa9fc075c20 msgr2=0x7fa9fc077020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:20.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.583+0000 7faa08f0f640 0 -- 172.21.15.6:0/1340554551 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7fa9fc008070 msgr2=0x7fa9fc071880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:22:20.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.803+0000 7faa0b3fe980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:22:20.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.803+0000 7faa0b3fe980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:22:20.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.803+0000 7faa0b3fe980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:22:20.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.803+0000 7faa0b3fe980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:22:20.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.803+0000 7faa0b3fe980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:22:20.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.803+0000 7faa0b3fe980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:22:20.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.803+0000 7faa0b3fe980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.803+0000 7faa0b3fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:20.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.803+0000 7faa0b3fe980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.803+0000 7faa0b3fe980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:20.804 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.803+0000 7faa0b3fe980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.804 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.803+0000 7faa0b3fe980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:22:20.804 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.803+0000 7faa0b3fe980 20 zone default found 2024-05-30T16:22:20.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.803+0000 7faa0b3fe980 4 Realm: () 2024-05-30T16:22:20.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.803+0000 7faa0b3fe980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:22:20.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.803+0000 7faa0b3fe980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:22:20.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.803+0000 7faa0b3fe980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:22:20.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.803+0000 7faa0b3fe980 20 zonegroup default 2024-05-30T16:22:20.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.803+0000 7faa0b3fe980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.803+0000 7faa0b3fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:20.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.803+0000 7faa0b3fe980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.807+0000 7faa0b3fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:20.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.807+0000 7faa0b3fe980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.807 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.807+0000 7faa0b3fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:20.807 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.807+0000 7faa0b3fe980 20 started sync module instance, tier type = 2024-05-30T16:22:20.807 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.807+0000 7faa0b3fe980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:22:20.831 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.831+0000 7faa0b3fe980 20 add_watcher() i=0 2024-05-30T16:22:20.832 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.831+0000 7faa0b3fe980 20 add_watcher() i=1 2024-05-30T16:22:20.832 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.831+0000 7faa0b3fe980 20 add_watcher() i=2 2024-05-30T16:22:20.832 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.831+0000 7faa0b3fe980 20 add_watcher() i=3 2024-05-30T16:22:20.832 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.831+0000 7faa0b3fe980 20 add_watcher() i=4 2024-05-30T16:22:20.832 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.831+0000 7faa0b3fe980 20 add_watcher() i=5 2024-05-30T16:22:20.832 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.831+0000 7faa0b3fe980 20 add_watcher() i=6 2024-05-30T16:22:20.834 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.835+0000 7faa0b3fe980 20 add_watcher() i=7 2024-05-30T16:22:20.834 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.835+0000 7faa0b3fe980 2 all 8 watchers are set, enabling cache 2024-05-30T16:22:20.838 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.839+0000 7fa8bf7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:20.839 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.839+0000 7faa0b3fe980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:22:20.839 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.839+0000 7faa0b3fe980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:22:20.839 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.839+0000 7faa0b3fe980 5 note: GC not initialized 2024-05-30T16:22:20.840 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.839+0000 7fa8be7e4640 20 reqs_thread_entry: start 2024-05-30T16:22:20.840 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.839+0000 7faa0b3fe980 20 init_complete bucket index max shards: 11 2024-05-30T16:22:20.840 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.839+0000 7faa0b3fe980 20 Filter name: none 2024-05-30T16:22:20.840 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.839+0000 7fa8b77fe640 20 reqs_thread_entry: start 2024-05-30T16:22:20.840 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.839+0000 7faa0b3fe980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.841 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.843+0000 7faa0b3fe980 20 rados_obj.operate() r=0 bl.length=211 2024-05-30T16:22:20.842 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.843+0000 7faa0b3fe980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.842 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.843+0000 7faa0b3fe980 20 rados_obj.operate() r=0 bl.length=457 2024-05-30T16:22:20.842 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.843+0000 7faa0b3fe980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:22:20.843 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.843+0000 7faa0b3fe980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.843 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.843+0000 7faa0b3fe980 20 rados_obj.operate() r=0 bl.length=211 2024-05-30T16:22:20.843 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.843+0000 7faa0b3fe980 20 rados->read ofs=0 len=0 2024-05-30T16:22:20.844 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.843+0000 7faa0b3fe980 20 rados_obj.operate() r=0 bl.length=457 2024-05-30T16:22:20.844 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.843+0000 7faa0b3fe980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:22:20.847 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.847+0000 7fa8bf7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:20.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.851+0000 7faa0b3fe980 20 remove_watcher() i=0 2024-05-30T16:22:20.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.851+0000 7faa0b3fe980 2 removed watcher, disabling cache 2024-05-30T16:22:20.854 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.855+0000 7faa0b3fe980 20 remove_watcher() i=1 2024-05-30T16:22:20.857 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.855+0000 7faa0b3fe980 20 remove_watcher() i=2 2024-05-30T16:22:20.859 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.859+0000 7faa0b3fe980 20 remove_watcher() i=3 2024-05-30T16:22:20.861 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.859+0000 7faa0b3fe980 20 remove_watcher() i=4 2024-05-30T16:22:20.863 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.863+0000 7faa0b3fe980 20 remove_watcher() i=5 2024-05-30T16:22:20.865 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.863+0000 7faa0b3fe980 20 remove_watcher() i=6 2024-05-30T16:22:20.866 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:20.867+0000 7faa0b3fe980 20 remove_watcher() i=7 2024-05-30T16:22:20.883 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:20.884 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:22:20.884 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: abort-at-do-reshard id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-30T16:22:20.884 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:51.208 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket reshard --bucket abort-at-do-reshard --num-shards 12 --rgw-reshard-bucket-lock-duration 30 2024-05-30T16:22:51.246 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:22:51.247 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:22:51.294 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.295+0000 7f1dcbba3980 20 rados->read ofs=0 len=0 2024-05-30T16:22:51.296 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.295+0000 7f1dcbba3980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:51.296 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.295+0000 7f1dcbba3980 20 realm 2024-05-30T16:22:51.297 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.295+0000 7f1dcbba3980 20 rados->read ofs=0 len=0 2024-05-30T16:22:51.297 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.295+0000 7f1dcbba3980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:51.297 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.295+0000 7f1dcbba3980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:22:51.297 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.295+0000 7f1dcbba3980 20 rados->read ofs=0 len=0 2024-05-30T16:22:51.297 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.295+0000 7f1dcbba3980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:51.298 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.295+0000 7f1dcbba3980 20 rados->read ofs=0 len=0 2024-05-30T16:22:51.298 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.299+0000 7f1dcbba3980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:51.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.299+0000 7f1dcbba3980 20 rados->read ofs=0 len=0 2024-05-30T16:22:51.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.299+0000 7f1dcbba3980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:22:51.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.299+0000 7f1dcbba3980 20 searching for the correct realm 2024-05-30T16:22:51.301 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.303+0000 7f1dc96b4640 0 -- 172.21.15.6:0/219169349 >> [v2:172.21.15.6:6816/2704410641,v1:172.21.15.6:6817/2704410641] conn(0x55e11c9319b0 msgr2=0x7f1dc4024440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:51.518 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.519+0000 7f1dc96b4640 0 -- 172.21.15.6:0/219169349 >> [v2:172.21.15.6:6816/2704410641,v1:172.21.15.6:6817/2704410641] conn(0x55e11c9319b0 msgr2=0x7f1dc4024440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:22:51.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.535+0000 7f1dcbba3980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:22:51.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.535+0000 7f1dcbba3980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:22:51.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.535+0000 7f1dcbba3980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:22:51.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.535+0000 7f1dcbba3980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:22:51.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.535+0000 7f1dcbba3980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:22:51.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.535+0000 7f1dcbba3980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:22:51.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.535+0000 7f1dcbba3980 20 rados->read ofs=0 len=0 2024-05-30T16:22:51.537 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.535+0000 7f1dcbba3980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:51.537 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.535+0000 7f1dcbba3980 20 rados->read ofs=0 len=0 2024-05-30T16:22:51.537 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.535+0000 7f1dcbba3980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:51.537 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.535+0000 7f1dcbba3980 20 rados->read ofs=0 len=0 2024-05-30T16:22:51.538 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.539+0000 7f1dcbba3980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:22:51.538 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.539+0000 7f1dcbba3980 20 zone default found 2024-05-30T16:22:51.539 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.539+0000 7f1dcbba3980 4 Realm: () 2024-05-30T16:22:51.539 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.539+0000 7f1dcbba3980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:22:51.539 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.539+0000 7f1dcbba3980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:22:51.539 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.539+0000 7f1dcbba3980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:22:51.539 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.539+0000 7f1dcbba3980 20 zonegroup default 2024-05-30T16:22:51.539 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.539+0000 7f1dcbba3980 20 rados->read ofs=0 len=0 2024-05-30T16:22:51.539 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.539+0000 7f1dcbba3980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:51.539 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.539+0000 7f1dcbba3980 20 rados->read ofs=0 len=0 2024-05-30T16:22:51.540 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.539+0000 7f1dcbba3980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:51.540 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.539+0000 7f1dcbba3980 20 rados->read ofs=0 len=0 2024-05-30T16:22:51.540 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.539+0000 7f1dcbba3980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:51.541 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.539+0000 7f1dcbba3980 20 started sync module instance, tier type = 2024-05-30T16:22:51.541 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.539+0000 7f1dcbba3980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:22:51.565 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.563+0000 7f1dcbba3980 20 add_watcher() i=0 2024-05-30T16:22:51.565 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.563+0000 7f1dcbba3980 20 add_watcher() i=1 2024-05-30T16:22:51.565 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.563+0000 7f1dcbba3980 20 add_watcher() i=2 2024-05-30T16:22:51.566 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.563+0000 7f1dcbba3980 20 add_watcher() i=3 2024-05-30T16:22:51.566 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.563+0000 7f1dcbba3980 20 add_watcher() i=4 2024-05-30T16:22:51.566 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.563+0000 7f1dcbba3980 20 add_watcher() i=5 2024-05-30T16:22:51.566 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.563+0000 7f1dcbba3980 20 add_watcher() i=6 2024-05-30T16:22:51.567 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.567+0000 7f1dcbba3980 20 add_watcher() i=7 2024-05-30T16:22:51.567 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.567+0000 7f1dcbba3980 2 all 8 watchers are set, enabling cache 2024-05-30T16:22:51.571 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.571+0000 7f1c7f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:51.572 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.571+0000 7f1dcbba3980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:22:51.572 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.571+0000 7f1dcbba3980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:22:51.572 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.571+0000 7f1dcbba3980 5 note: GC not initialized 2024-05-30T16:22:51.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.571+0000 7f1c7e7e4640 20 reqs_thread_entry: start 2024-05-30T16:22:51.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.571+0000 7f1dcbba3980 20 init_complete bucket index max shards: 11 2024-05-30T16:22:51.573 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.571+0000 7f1dcbba3980 20 Filter name: none 2024-05-30T16:22:51.574 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.571+0000 7f1beffff640 20 reqs_thread_entry: start 2024-05-30T16:22:51.574 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.571+0000 7f1dcbba3980 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2024-05-30T16:22:51.574 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.571+0000 7f1dcbba3980 20 rados->read ofs=0 len=0 2024-05-30T16:22:51.575 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.575+0000 7f1dcbba3980 20 rados_obj.operate() r=0 bl.length=211 2024-05-30T16:22:51.575 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.575+0000 7f1dcbba3980 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2024-05-30T16:22:51.575 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.575+0000 7f1dcbba3980 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2024-05-30T16:22:51.575 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.575+0000 7f1dcbba3980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 : miss 2024-05-30T16:22:51.575 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.575+0000 7f1dcbba3980 20 rados->read ofs=0 len=0 2024-05-30T16:22:51.575 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.575+0000 7f1dcbba3980 20 rados_obj.operate() r=0 bl.length=425 2024-05-30T16:22:51.576 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.575+0000 7f1dcbba3980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 info.flags=0x17 2024-05-30T16:22:51.576 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.575+0000 7f1dcbba3980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 to cache LRU end 2024-05-30T16:22:51.576 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.575+0000 7f1dcbba3980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:22:51.576 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.575+0000 7f1dcbba3980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:22:51.576 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.575+0000 7f1dcbba3980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 2024-05-30T16:22:51.586 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.587+0000 7f1dcbba3980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 info.flags=0x17 2024-05-30T16:22:51.586 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.587+0000 7f1dcbba3980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 to cache LRU end 2024-05-30T16:22:51.586 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.587+0000 7f1dcbba3980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:22:51.586 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.587+0000 7f1dcbba3980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29, ofs0, ns] 2024-05-30T16:22:51.586 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.587+0000 7f1c9bfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431639 cookie 94425337803584 notifier 7002 bl.length()=987 2024-05-30T16:22:51.586 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.587+0000 7f1c9bfff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 info.flags=0x17 2024-05-30T16:22:51.586 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.587+0000 7f1c9bfff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 to cache LRU end 2024-05-30T16:22:51.587 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.587+0000 7f1c9bfff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:22:51.589 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.587+0000 7f1dcbba3980 20 handle_bi_update(): bucket=:abort-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29]): orig_sources= new_sources= 2024-05-30T16:22:51.589 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.587+0000 7f1dcbba3980 20 handle_bi_update(): bucket=:abort-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29]): potential sources added=[] removed=[] 2024-05-30T16:22:51.590 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.587+0000 7f1dcbba3980 20 handle_bi_update(): bucket=:abort-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29]): orig_dests= new_dests= 2024-05-30T16:22:51.590 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.587+0000 7f1dcbba3980 20 handle_bi_update(): bucket=:abort-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29]): potential dests added=[] removed=[] 2024-05-30T16:22:51.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.595+0000 7f1dcbba3980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.1.0 2024-05-30T16:22:51.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.595+0000 7f1dcbba3980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.1.0 2024-05-30T16:22:51.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.595+0000 7f1dcbba3980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.1.1 2024-05-30T16:22:51.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.595+0000 7f1dcbba3980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.1.1 2024-05-30T16:22:51.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.595+0000 7f1dcbba3980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.1.2 2024-05-30T16:22:51.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.595+0000 7f1dcbba3980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.1.2 2024-05-30T16:22:51.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.595+0000 7f1dcbba3980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.1.3 2024-05-30T16:22:51.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.595+0000 7f1dcbba3980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.1.3 2024-05-30T16:22:51.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.595+0000 7f1dcbba3980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.1.4 2024-05-30T16:22:51.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.595+0000 7f1dcbba3980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.1.4 2024-05-30T16:22:51.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.595+0000 7f1dcbba3980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.1.5 2024-05-30T16:22:51.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.595+0000 7f1dcbba3980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.1.5 2024-05-30T16:22:51.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.595+0000 7f1dcbba3980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.1.6 2024-05-30T16:22:51.596 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.595+0000 7f1dcbba3980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.1.6 2024-05-30T16:22:51.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.595+0000 7f1dcbba3980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.1.7 2024-05-30T16:22:51.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.595+0000 7f1dcbba3980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.1.7 2024-05-30T16:22:51.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.595+0000 7f1dcbba3980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.1.8 2024-05-30T16:22:51.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.595+0000 7f1dcbba3980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.1.8 2024-05-30T16:22:51.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.595+0000 7f1dcbba3980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.1.9 2024-05-30T16:22:51.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.595+0000 7f1dcbba3980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.1.9 2024-05-30T16:22:51.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.595+0000 7f1dcbba3980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.1.10 2024-05-30T16:22:51.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.595+0000 7f1dcbba3980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.1.10 2024-05-30T16:22:51.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.595+0000 7f1dcbba3980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.1.11 2024-05-30T16:22:51.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.595+0000 7f1dcbba3980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.1.11 2024-05-30T16:22:51.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.595+0000 7f1dcbba3980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.0 2024-05-30T16:22:51.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.595+0000 7f1dcbba3980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.1 2024-05-30T16:22:51.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.595+0000 7f1dcbba3980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.2 2024-05-30T16:22:51.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.595+0000 7f1dcbba3980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.3 2024-05-30T16:22:51.598 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.599+0000 7f1dcbba3980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.4 2024-05-30T16:22:51.599 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.599+0000 7f1dcbba3980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.5 2024-05-30T16:22:51.600 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.599+0000 7f1dcbba3980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.6 2024-05-30T16:22:51.600 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.599+0000 7f1dcbba3980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.7 2024-05-30T16:22:51.601 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.599+0000 7f1dcbba3980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.8 2024-05-30T16:22:51.602 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.603+0000 7f1dcbba3980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.9 2024-05-30T16:22:51.603 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.603+0000 7f1dcbba3980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29.10 2024-05-30T16:22:51.610 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.611+0000 7f1dcbba3980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 : hit (requested=0x17, cached=0x17) 2024-05-30T16:22:51.610 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.611+0000 7f1dcbba3980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 2024-05-30T16:22:51.612 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.611+0000 7f1dcbba3980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 info.flags=0x17 2024-05-30T16:22:51.612 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.611+0000 7f1dcbba3980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 to cache LRU end 2024-05-30T16:22:51.612 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.611+0000 7f1dcbba3980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:22:51.612 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.611+0000 7f1dcbba3980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29, ofs0, ns] 2024-05-30T16:22:51.613 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.611+0000 7f1cacff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431640 cookie 94425337803584 notifier 7002 bl.length()=955 2024-05-30T16:22:51.613 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.611+0000 7f1cacff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 info.flags=0x17 2024-05-30T16:22:51.613 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.611+0000 7f1cacff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 to cache LRU end 2024-05-30T16:22:51.613 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.611+0000 7f1cacff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-05-30T16:22:51.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.615+0000 7f1dcbba3980 20 handle_bi_update(): bucket=:abort-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29]): orig_sources= new_sources= 2024-05-30T16:22:51.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.615+0000 7f1dcbba3980 20 handle_bi_update(): bucket=:abort-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29]): potential sources added=[] removed=[] 2024-05-30T16:22:51.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.615+0000 7f1dcbba3980 20 handle_bi_update(): bucket=:abort-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29]): orig_dests= new_dests= 2024-05-30T16:22:51.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.615+0000 7f1dcbba3980 20 handle_bi_update(): bucket=:abort-at-do-reshard[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29]): potential dests added=[] removed=[] 2024-05-30T16:22:51.619 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.619+0000 7f1dcbba3980 1 execute INFO: reshard of bucket "abort-at-do-reshard" completed successfully 2024-05-30T16:22:51.624 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.623+0000 7f1c7f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:51.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.627+0000 7f1dcbba3980 20 remove_watcher() i=0 2024-05-30T16:22:51.629 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.627+0000 7f1dcbba3980 2 removed watcher, disabling cache 2024-05-30T16:22:51.632 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.631+0000 7f1dcbba3980 20 remove_watcher() i=1 2024-05-30T16:22:51.634 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.635+0000 7f1dcbba3980 20 remove_watcher() i=2 2024-05-30T16:22:51.636 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.635+0000 7f1dcbba3980 20 remove_watcher() i=3 2024-05-30T16:22:51.639 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.639+0000 7f1dcbba3980 20 remove_watcher() i=4 2024-05-30T16:22:51.640 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.639+0000 7f1dcbba3980 20 remove_watcher() i=5 2024-05-30T16:22:51.641 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.639+0000 7f1dcbba3980 20 remove_watcher() i=6 2024-05-30T16:22:51.643 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.643+0000 7f1dcbba3980 20 remove_watcher() i=7 2024-05-30T16:22:51.658 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket abort-at-do-reshard 2024-05-30T16:22:51.687 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:22:51.687 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:22:51.733 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.731+0000 7f41b9cb4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:51.735 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.735+0000 7f41b9cb4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:51.735 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.735+0000 7f41b9cb4980 20 realm 2024-05-30T16:22:51.735 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.735+0000 7f41b9cb4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:51.736 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.735+0000 7f41b9cb4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:51.736 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.735+0000 7f41b9cb4980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:22:51.736 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.735+0000 7f41b9cb4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:51.736 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.735+0000 7f41b9cb4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:51.736 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.735+0000 7f41b9cb4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:51.737 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.735+0000 7f41b9cb4980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:51.737 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.735+0000 7f41b9cb4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:51.738 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.739+0000 7f41b9cb4980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:22:51.738 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.739+0000 7f41b9cb4980 20 searching for the correct realm 2024-05-30T16:22:51.742 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.743+0000 7f41b6fc4640 0 -- 172.21.15.6:0/180053256 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f41a8051070 msgr2=0x7f41a8051960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:22:51.959 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:51.959+0000 7f41b77c5640 0 -- 172.21.15.6:0/180053256 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f41ac076120 msgr2=0x7f41ac07a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:52.177 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.175+0000 7f41b9cb4980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:22:52.178 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.175+0000 7f41b9cb4980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:22:52.178 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.175+0000 7f41b9cb4980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:22:52.178 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.175+0000 7f41b9cb4980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:22:52.178 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.175+0000 7f41b9cb4980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:22:52.178 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.175+0000 7f41b9cb4980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:22:52.178 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.175+0000 7f41b9cb4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:52.179 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.179+0000 7f41b9cb4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:52.179 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.179+0000 7f41b9cb4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:52.179 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.179+0000 7f41b9cb4980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:52.179 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.179+0000 7f41b9cb4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:52.180 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.179+0000 7f41b9cb4980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:22:52.180 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.179+0000 7f41b9cb4980 20 zone default found 2024-05-30T16:22:52.180 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.179+0000 7f41b9cb4980 4 Realm: () 2024-05-30T16:22:52.180 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.179+0000 7f41b9cb4980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:22:52.180 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.179+0000 7f41b9cb4980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:22:52.180 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.179+0000 7f41b9cb4980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:22:52.180 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.179+0000 7f41b9cb4980 20 zonegroup default 2024-05-30T16:22:52.180 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.179+0000 7f41b9cb4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:52.181 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.179+0000 7f41b9cb4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:52.181 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.179+0000 7f41b9cb4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:52.181 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.179+0000 7f41b9cb4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:52.182 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.179+0000 7f41b9cb4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:52.182 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.183+0000 7f41b9cb4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:52.182 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.183+0000 7f41b9cb4980 20 started sync module instance, tier type = 2024-05-30T16:22:52.183 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.183+0000 7f41b9cb4980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:22:52.207 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.207+0000 7f41b9cb4980 20 add_watcher() i=0 2024-05-30T16:22:52.207 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.207+0000 7f41b9cb4980 20 add_watcher() i=1 2024-05-30T16:22:52.207 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.207+0000 7f41b9cb4980 20 add_watcher() i=2 2024-05-30T16:22:52.207 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.207+0000 7f41b9cb4980 20 add_watcher() i=3 2024-05-30T16:22:52.207 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.207+0000 7f41b9cb4980 20 add_watcher() i=4 2024-05-30T16:22:52.207 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.207+0000 7f41b9cb4980 20 add_watcher() i=5 2024-05-30T16:22:52.207 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.207+0000 7f41b9cb4980 20 add_watcher() i=6 2024-05-30T16:22:52.209 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.207+0000 7f41b9cb4980 20 add_watcher() i=7 2024-05-30T16:22:52.209 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.207+0000 7f41b9cb4980 2 all 8 watchers are set, enabling cache 2024-05-30T16:22:52.214 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.215+0000 7f406dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:52.215 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.215+0000 7f41b9cb4980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:22:52.215 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.215+0000 7f41b9cb4980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:22:52.215 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.215+0000 7f41b9cb4980 5 note: GC not initialized 2024-05-30T16:22:52.215 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.215+0000 7f406cfe1640 20 reqs_thread_entry: start 2024-05-30T16:22:52.215 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.215+0000 7f41b9cb4980 20 init_complete bucket index max shards: 11 2024-05-30T16:22:52.216 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.215+0000 7f41b9cb4980 20 Filter name: none 2024-05-30T16:22:52.216 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.215+0000 7f3fe67fc640 20 reqs_thread_entry: start 2024-05-30T16:22:52.216 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.215+0000 7f41b9cb4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:52.217 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.215+0000 7f41b9cb4980 20 rados_obj.operate() r=0 bl.length=211 2024-05-30T16:22:52.217 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.215+0000 7f41b9cb4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:52.218 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.219+0000 7f41b9cb4980 20 rados_obj.operate() r=0 bl.length=425 2024-05-30T16:22:52.218 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.219+0000 7f41b9cb4980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:22:52.218 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.219+0000 7f41b9cb4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:52.219 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.219+0000 7f41b9cb4980 20 rados_obj.operate() r=0 bl.length=211 2024-05-30T16:22:52.219 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.219+0000 7f41b9cb4980 20 rados->read ofs=0 len=0 2024-05-30T16:22:52.220 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.219+0000 7f41b9cb4980 20 rados_obj.operate() r=0 bl.length=425 2024-05-30T16:22:52.220 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.219+0000 7f41b9cb4980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:22:52.224 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.223+0000 7f406dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:52.228 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.227+0000 7f41b9cb4980 20 remove_watcher() i=0 2024-05-30T16:22:52.228 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.227+0000 7f41b9cb4980 2 removed watcher, disabling cache 2024-05-30T16:22:52.230 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.231+0000 7f41b9cb4980 20 remove_watcher() i=1 2024-05-30T16:22:52.233 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.231+0000 7f41b9cb4980 20 remove_watcher() i=2 2024-05-30T16:22:52.235 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.235+0000 7f41b9cb4980 20 remove_watcher() i=3 2024-05-30T16:22:52.237 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.235+0000 7f41b9cb4980 20 remove_watcher() i=4 2024-05-30T16:22:52.238 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.239+0000 7f41b9cb4980 20 remove_watcher() i=5 2024-05-30T16:22:52.239 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.239+0000 7f41b9cb4980 20 remove_watcher() i=6 2024-05-30T16:22:52.241 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:52.239+0000 7f41b9cb4980 20 remove_watcher() i=7 2024-05-30T16:22:52.255 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:52.255 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:22:52.255 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: abort-at-do-reshard id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.29 num_objects: 19 size_kb: 1 num_shards: 12 2024-05-30T16:22:52.256 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:53.211 INFO:tasks.workunit.client.0.smithi006.stderr: test: reshard versioned bucket 2024-05-30T16:22:53.211 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket myver 2024-05-30T16:22:53.247 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:22:53.247 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:22:53.294 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.295+0000 7fbd2f3ea980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.296 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.295+0000 7fbd2f3ea980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:53.296 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.295+0000 7fbd2f3ea980 20 realm 2024-05-30T16:22:53.296 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.295+0000 7fbd2f3ea980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.296 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.295+0000 7fbd2f3ea980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:53.296 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.295+0000 7fbd2f3ea980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:22:53.296 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.295+0000 7fbd2f3ea980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.296 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.295+0000 7fbd2f3ea980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:53.296 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.295+0000 7fbd2f3ea980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.298 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.299+0000 7fbd2f3ea980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:53.298 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.299+0000 7fbd2f3ea980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.298 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.299+0000 7fbd2f3ea980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:22:53.298 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.299+0000 7fbd2f3ea980 20 searching for the correct realm 2024-05-30T16:22:53.304 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.303+0000 7fbd2d6fc640 0 -- 172.21.15.6:0/1770315018 >> [v2:172.21.15.204:6823/3156311527,v1:172.21.15.204:6825/3156311527] conn(0x7fbd200c0020 msgr2=0x7fbd200f4bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:53.310 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.311+0000 7fbd2cefb640 0 -- 172.21.15.6:0/1770315018 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7fbd200f42b0 msgr2=0x7fbd200f6870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:53.530 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.531+0000 7fbd2f3ea980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:22:53.530 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.531+0000 7fbd2f3ea980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:22:53.531 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.531+0000 7fbd2f3ea980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:22:53.531 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.531+0000 7fbd2f3ea980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:22:53.531 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.531+0000 7fbd2f3ea980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:22:53.531 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.531+0000 7fbd2f3ea980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:22:53.531 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.531+0000 7fbd2f3ea980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.531 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.531+0000 7fbd2f3ea980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:53.531 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.531+0000 7fbd2f3ea980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.532 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.531+0000 7fbd2f3ea980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:53.532 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.531+0000 7fbd2f3ea980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.533 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.531+0000 7fbd2f3ea980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:22:53.533 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.531+0000 7fbd2f3ea980 20 zone default found 2024-05-30T16:22:53.533 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.531+0000 7fbd2f3ea980 4 Realm: () 2024-05-30T16:22:53.533 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.531+0000 7fbd2f3ea980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:22:53.533 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.531+0000 7fbd2f3ea980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:22:53.533 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.531+0000 7fbd2f3ea980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:22:53.533 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.531+0000 7fbd2f3ea980 20 zonegroup default 2024-05-30T16:22:53.533 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.531+0000 7fbd2f3ea980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.534 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.535+0000 7fbd2f3ea980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:53.534 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.535+0000 7fbd2f3ea980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.534 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.535+0000 7fbd2f3ea980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:53.534 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.535+0000 7fbd2f3ea980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.538 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.535+0000 7fbd2f3ea980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:53.538 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.535+0000 7fbd2f3ea980 20 started sync module instance, tier type = 2024-05-30T16:22:53.538 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.535+0000 7fbd2f3ea980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:22:53.558 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.559+0000 7fbd2f3ea980 20 add_watcher() i=0 2024-05-30T16:22:53.559 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.559+0000 7fbd2f3ea980 20 add_watcher() i=1 2024-05-30T16:22:53.559 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.559+0000 7fbd2f3ea980 20 add_watcher() i=2 2024-05-30T16:22:53.559 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.559+0000 7fbd2f3ea980 20 add_watcher() i=3 2024-05-30T16:22:53.559 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.559+0000 7fbd2f3ea980 20 add_watcher() i=4 2024-05-30T16:22:53.559 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.559+0000 7fbd2f3ea980 20 add_watcher() i=5 2024-05-30T16:22:53.559 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.559+0000 7fbd2f3ea980 20 add_watcher() i=6 2024-05-30T16:22:53.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.559+0000 7fbd2f3ea980 20 add_watcher() i=7 2024-05-30T16:22:53.561 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.559+0000 7fbd2f3ea980 2 all 8 watchers are set, enabling cache 2024-05-30T16:22:53.565 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.563+0000 7fbbe2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:53.566 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.567+0000 7fbd2f3ea980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:22:53.566 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.567+0000 7fbd2f3ea980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:22:53.566 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.567+0000 7fbd2f3ea980 5 note: GC not initialized 2024-05-30T16:22:53.567 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.567+0000 7fbbe1fe3640 20 reqs_thread_entry: start 2024-05-30T16:22:53.567 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.567+0000 7fbd2f3ea980 20 init_complete bucket index max shards: 11 2024-05-30T16:22:53.567 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.567+0000 7fbd2f3ea980 20 Filter name: none 2024-05-30T16:22:53.567 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.567+0000 7fbb537fe640 20 reqs_thread_entry: start 2024-05-30T16:22:53.568 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.567+0000 7fbd2f3ea980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.568 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.567+0000 7fbd2f3ea980 20 rados_obj.operate() r=0 bl.length=197 2024-05-30T16:22:53.568 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.567+0000 7fbd2f3ea980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.570 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.571+0000 7fbd2f3ea980 20 rados_obj.operate() r=0 bl.length=411 2024-05-30T16:22:53.570 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.571+0000 7fbd2f3ea980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:22:53.570 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.571+0000 7fbd2f3ea980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.571 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.571+0000 7fbd2f3ea980 20 rados_obj.operate() r=0 bl.length=197 2024-05-30T16:22:53.571 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.571+0000 7fbd2f3ea980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.572 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.571+0000 7fbd2f3ea980 20 rados_obj.operate() r=0 bl.length=411 2024-05-30T16:22:53.572 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.571+0000 7fbd2f3ea980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:22:53.576 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.575+0000 7fbbe2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:53.581 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.579+0000 7fbd2f3ea980 20 remove_watcher() i=0 2024-05-30T16:22:53.581 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.579+0000 7fbd2f3ea980 2 removed watcher, disabling cache 2024-05-30T16:22:53.584 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.583+0000 7fbd2f3ea980 20 remove_watcher() i=1 2024-05-30T16:22:53.586 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.587+0000 7fbd2f3ea980 20 remove_watcher() i=2 2024-05-30T16:22:53.588 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.587+0000 7fbd2f3ea980 20 remove_watcher() i=3 2024-05-30T16:22:53.591 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.591+0000 7fbd2f3ea980 20 remove_watcher() i=4 2024-05-30T16:22:53.593 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.591+0000 7fbd2f3ea980 20 remove_watcher() i=5 2024-05-30T16:22:53.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.595+0000 7fbd2f3ea980 20 remove_watcher() i=6 2024-05-30T16:22:53.597 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.595+0000 7fbd2f3ea980 20 remove_watcher() i=7 2024-05-30T16:22:53.614 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:53.614 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:22:53.615 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: myver id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 num_objects: 0 size_kb: 0 num_shards: 11 2024-05-30T16:22:53.615 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:53.615 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket reshard --bucket myver --num-shards 12 2024-05-30T16:22:53.649 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:22:53.650 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:22:53.698 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.699+0000 7f814ef86980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.700 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.699+0000 7f814ef86980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:53.700 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.699+0000 7f814ef86980 20 realm 2024-05-30T16:22:53.700 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.699+0000 7f814ef86980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.700 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.699+0000 7f814ef86980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:53.700 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.699+0000 7f814ef86980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:22:53.700 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.699+0000 7f814ef86980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.700 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.699+0000 7f814ef86980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:53.700 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.699+0000 7f814ef86980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.702 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.703+0000 7f814ef86980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:53.702 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.703+0000 7f814ef86980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.702 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.703+0000 7f814ef86980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:22:53.702 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.703+0000 7f814ef86980 20 searching for the correct realm 2024-05-30T16:22:53.708 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.707+0000 7f814c296640 0 -- 172.21.15.6:0/3207263937 >> [v2:172.21.15.6:6819/1141406589,v1:172.21.15.6:6821/1141406589] conn(0x557914249010 msgr2=0x55791424b4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:53.715 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.715+0000 7f814ca97640 0 -- 172.21.15.6:0/3207263937 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f8140078bc0 msgr2=0x7f814008eac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:53.722 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.723+0000 7f814ef86980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:22:53.722 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.723+0000 7f814ef86980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:22:53.722 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.723+0000 7f814ef86980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:22:53.722 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.723+0000 7f814ef86980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:22:53.722 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.723+0000 7f814ef86980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:22:53.722 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.723+0000 7f814ef86980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:22:53.722 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.723+0000 7f814ef86980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.722 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.723+0000 7f814ef86980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:53.723 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.723+0000 7f814ef86980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.723 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.723+0000 7f814ef86980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:53.723 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.723+0000 7f814ef86980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.724 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.723+0000 7f814ef86980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:22:53.724 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.723+0000 7f814ef86980 20 zone default found 2024-05-30T16:22:53.724 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.723+0000 7f814ef86980 4 Realm: () 2024-05-30T16:22:53.724 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.723+0000 7f814ef86980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:22:53.724 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.723+0000 7f814ef86980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:22:53.724 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.723+0000 7f814ef86980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:22:53.724 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.723+0000 7f814ef86980 20 zonegroup default 2024-05-30T16:22:53.724 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.723+0000 7f814ef86980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.724 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.723+0000 7f814ef86980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:53.724 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.723+0000 7f814ef86980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.725 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.723+0000 7f814ef86980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:53.725 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.723+0000 7f814ef86980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.725 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.723+0000 7f814ef86980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:53.725 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.723+0000 7f814ef86980 20 started sync module instance, tier type = 2024-05-30T16:22:53.725 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.723+0000 7f814ef86980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:22:53.740 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.739+0000 7f814ef86980 20 add_watcher() i=0 2024-05-30T16:22:53.740 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.739+0000 7f814ef86980 20 add_watcher() i=1 2024-05-30T16:22:53.740 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.739+0000 7f814ef86980 20 add_watcher() i=2 2024-05-30T16:22:53.740 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.739+0000 7f814ef86980 20 add_watcher() i=3 2024-05-30T16:22:53.741 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.739+0000 7f814ef86980 20 add_watcher() i=4 2024-05-30T16:22:53.741 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.739+0000 7f814ef86980 20 add_watcher() i=5 2024-05-30T16:22:53.741 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.739+0000 7f814ef86980 20 add_watcher() i=6 2024-05-30T16:22:53.741 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.739+0000 7f814ef86980 20 add_watcher() i=7 2024-05-30T16:22:53.741 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.739+0000 7f814ef86980 2 all 8 watchers are set, enabling cache 2024-05-30T16:22:53.744 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.743+0000 7f80037e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:53.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.743+0000 7f814ef86980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:22:53.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.743+0000 7f814ef86980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:22:53.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.743+0000 7f814ef86980 5 note: GC not initialized 2024-05-30T16:22:53.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.743+0000 7f80027e4640 20 reqs_thread_entry: start 2024-05-30T16:22:53.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.743+0000 7f814ef86980 20 init_complete bucket index max shards: 11 2024-05-30T16:22:53.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.743+0000 7f814ef86980 20 Filter name: none 2024-05-30T16:22:53.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.743+0000 7f7ffb7fe640 20 reqs_thread_entry: start 2024-05-30T16:22:53.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.743+0000 7f814ef86980 10 cache get: name=default.rgw.meta+root+myver : miss 2024-05-30T16:22:53.745 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.743+0000 7f814ef86980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.746 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.743+0000 7f814ef86980 20 rados_obj.operate() r=0 bl.length=197 2024-05-30T16:22:53.746 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.747+0000 7f814ef86980 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2024-05-30T16:22:53.746 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.747+0000 7f814ef86980 10 adding default.rgw.meta+root+myver to cache LRU end 2024-05-30T16:22:53.746 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.747+0000 7f814ef86980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 : miss 2024-05-30T16:22:53.746 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.747+0000 7f814ef86980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.747 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.747+0000 7f814ef86980 20 rados_obj.operate() r=0 bl.length=411 2024-05-30T16:22:53.747 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.747+0000 7f814ef86980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 info.flags=0x17 2024-05-30T16:22:53.747 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.747+0000 7f814ef86980 10 adding default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 to cache LRU end 2024-05-30T16:22:53.747 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.747+0000 7f814ef86980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:22:53.747 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.747+0000 7f814ef86980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:22:53.747 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.747+0000 7f814ef86980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 2024-05-30T16:22:53.758 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.759+0000 7f814ef86980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 info.flags=0x17 2024-05-30T16:22:53.758 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.759+0000 7f814ef86980 10 moving default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 to cache LRU end 2024-05-30T16:22:53.758 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.759+0000 7f814ef86980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:22:53.758 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.759+0000 7f814ef86980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19, ofs0, ns] 2024-05-30T16:22:53.759 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.759+0000 7f8023fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431590 cookie 93978517363056 notifier 7056 bl.length()=897 2024-05-30T16:22:53.759 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.759+0000 7f8023fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 info.flags=0x17 2024-05-30T16:22:53.759 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.759+0000 7f8023fff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 to cache LRU end 2024-05-30T16:22:53.759 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.759+0000 7f8023fff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:22:53.761 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.759+0000 7f814ef86980 20 handle_bi_update(): bucket=:myver[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19]): orig_sources= new_sources= 2024-05-30T16:22:53.761 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.759+0000 7f814ef86980 20 handle_bi_update(): bucket=:myver[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19]): potential sources added=[] removed=[] 2024-05-30T16:22:53.761 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.759+0000 7f814ef86980 20 handle_bi_update(): bucket=:myver[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19]): orig_dests= new_dests= 2024-05-30T16:22:53.761 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.759+0000 7f814ef86980 20 handle_bi_update(): bucket=:myver[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19]): potential dests added=[] removed=[] 2024-05-30T16:22:53.768 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.767+0000 7f814ef86980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.0 2024-05-30T16:22:53.768 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.767+0000 7f814ef86980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.0 2024-05-30T16:22:53.768 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.767+0000 7f814ef86980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.1 2024-05-30T16:22:53.768 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.767+0000 7f814ef86980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.1 2024-05-30T16:22:53.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.767+0000 7f814ef86980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.2 2024-05-30T16:22:53.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.767+0000 7f814ef86980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.2 2024-05-30T16:22:53.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.767+0000 7f814ef86980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.3 2024-05-30T16:22:53.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.767+0000 7f814ef86980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.3 2024-05-30T16:22:53.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.767+0000 7f814ef86980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.4 2024-05-30T16:22:53.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.767+0000 7f814ef86980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.4 2024-05-30T16:22:53.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.767+0000 7f814ef86980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.5 2024-05-30T16:22:53.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.767+0000 7f814ef86980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.5 2024-05-30T16:22:53.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.767+0000 7f814ef86980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.6 2024-05-30T16:22:53.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.767+0000 7f814ef86980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.6 2024-05-30T16:22:53.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.767+0000 7f814ef86980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.7 2024-05-30T16:22:53.769 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.767+0000 7f814ef86980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.7 2024-05-30T16:22:53.770 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.767+0000 7f814ef86980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.8 2024-05-30T16:22:53.770 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.767+0000 7f814ef86980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.8 2024-05-30T16:22:53.770 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.767+0000 7f814ef86980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.9 2024-05-30T16:22:53.770 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.767+0000 7f814ef86980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.9 2024-05-30T16:22:53.770 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.767+0000 7f814ef86980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.10 2024-05-30T16:22:53.770 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.767+0000 7f814ef86980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.10 2024-05-30T16:22:53.770 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.767+0000 7f814ef86980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.11 2024-05-30T16:22:53.770 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.767+0000 7f814ef86980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.11 2024-05-30T16:22:53.770 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.767+0000 7f814ef86980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.0 2024-05-30T16:22:53.771 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.767+0000 7f814ef86980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1 2024-05-30T16:22:53.771 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.771+0000 7f814ef86980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2 2024-05-30T16:22:53.771 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.771+0000 7f814ef86980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3 2024-05-30T16:22:53.772 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.771+0000 7f814ef86980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.4 2024-05-30T16:22:53.773 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.771+0000 7f814ef86980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.5 2024-05-30T16:22:53.774 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.775+0000 7f814ef86980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.6 2024-05-30T16:22:53.775 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.775+0000 7f814ef86980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.7 2024-05-30T16:22:53.776 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.775+0000 7f814ef86980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.8 2024-05-30T16:22:53.777 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.775+0000 7f814ef86980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.9 2024-05-30T16:22:53.778 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.779+0000 7f814ef86980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.10 2024-05-30T16:22:53.779 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.779+0000 7f814ef86980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 : hit (requested=0x17, cached=0x17) 2024-05-30T16:22:53.779 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.779+0000 7f814ef86980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 2024-05-30T16:22:53.782 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.783+0000 7f814ef86980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 info.flags=0x17 2024-05-30T16:22:53.782 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.783+0000 7f814ef86980 10 moving default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 to cache LRU end 2024-05-30T16:22:53.782 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.783+0000 7f814ef86980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:22:53.782 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.783+0000 7f814ef86980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19, ofs0, ns] 2024-05-30T16:22:53.783 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.783+0000 7f80237fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431591 cookie 93978517363056 notifier 7056 bl.length()=865 2024-05-30T16:22:53.783 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.783+0000 7f80237fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 info.flags=0x17 2024-05-30T16:22:53.783 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.783+0000 7f80237fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 to cache LRU end 2024-05-30T16:22:53.783 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.783+0000 7f80237fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:22:53.785 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.783+0000 7f814ef86980 20 handle_bi_update(): bucket=:myver[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19]): orig_sources= new_sources= 2024-05-30T16:22:53.785 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.783+0000 7f814ef86980 20 handle_bi_update(): bucket=:myver[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19]): potential sources added=[] removed=[] 2024-05-30T16:22:53.785 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.783+0000 7f814ef86980 20 handle_bi_update(): bucket=:myver[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19]): orig_dests= new_dests= 2024-05-30T16:22:53.786 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.783+0000 7f814ef86980 20 handle_bi_update(): bucket=:myver[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19]): potential dests added=[] removed=[] 2024-05-30T16:22:53.791 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.791+0000 7f814ef86980 1 execute INFO: reshard of bucket "myver" completed successfully 2024-05-30T16:22:53.794 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.795+0000 7f80037e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:53.799 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.799+0000 7f814ef86980 20 remove_watcher() i=0 2024-05-30T16:22:53.799 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.799+0000 7f814ef86980 2 removed watcher, disabling cache 2024-05-30T16:22:53.801 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.803+0000 7f814ef86980 20 remove_watcher() i=1 2024-05-30T16:22:53.804 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.803+0000 7f814ef86980 20 remove_watcher() i=2 2024-05-30T16:22:53.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.807+0000 7f814ef86980 20 remove_watcher() i=3 2024-05-30T16:22:53.808 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.807+0000 7f814ef86980 20 remove_watcher() i=4 2024-05-30T16:22:53.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.807+0000 7f814ef86980 20 remove_watcher() i=5 2024-05-30T16:22:53.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.811+0000 7f814ef86980 20 remove_watcher() i=6 2024-05-30T16:22:53.812 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.811+0000 7f814ef86980 20 remove_watcher() i=7 2024-05-30T16:22:53.827 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket myver 2024-05-30T16:22:53.854 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:22:53.854 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:22:53.899 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.899+0000 7f44cccca980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.901 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.899+0000 7f44cccca980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:53.901 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.899+0000 7f44cccca980 20 realm 2024-05-30T16:22:53.901 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.899+0000 7f44cccca980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.901 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.899+0000 7f44cccca980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:53.901 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.899+0000 7f44cccca980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:22:53.901 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.899+0000 7f44cccca980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.901 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.899+0000 7f44cccca980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:53.901 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.899+0000 7f44cccca980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.903 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.903+0000 7f44cccca980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:53.903 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.903+0000 7f44cccca980 20 rados->read ofs=0 len=0 2024-05-30T16:22:53.903 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.903+0000 7f44cccca980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:22:53.903 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.903+0000 7f44cccca980 20 searching for the correct realm 2024-05-30T16:22:53.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:53.911+0000 7f44cafdc640 0 -- 172.21.15.6:0/4155893666 >> [v2:172.21.15.204:6800/3687190393,v1:172.21.15.204:6801/3687190393] conn(0x7f44c0014070 msgr2=0x7f44c00897e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:54.128 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.127+0000 7f44ca7db640 0 -- 172.21.15.6:0/4155893666 >> [v2:172.21.15.204:6816/2677861390,v1:172.21.15.204:6817/2677861390] conn(0x5614fbc6c3f0 msgr2=0x5614fbc6e800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:54.136 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.135+0000 7f44cccca980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:22:54.136 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.135+0000 7f44cccca980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:22:54.136 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.135+0000 7f44cccca980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:22:54.136 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.135+0000 7f44cccca980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:22:54.136 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.135+0000 7f44cccca980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:22:54.136 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.135+0000 7f44cccca980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:22:54.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.135+0000 7f44cccca980 20 rados->read ofs=0 len=0 2024-05-30T16:22:54.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.135+0000 7f44cccca980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:54.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.135+0000 7f44cccca980 20 rados->read ofs=0 len=0 2024-05-30T16:22:54.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.135+0000 7f44cccca980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:54.137 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.135+0000 7f44cccca980 20 rados->read ofs=0 len=0 2024-05-30T16:22:54.139 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.139+0000 7f44cccca980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:22:54.139 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.139+0000 7f44cccca980 20 zone default found 2024-05-30T16:22:54.139 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.139+0000 7f44cccca980 4 Realm: () 2024-05-30T16:22:54.139 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.139+0000 7f44cccca980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:22:54.139 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.139+0000 7f44cccca980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:22:54.139 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.139+0000 7f44cccca980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:22:54.139 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.139+0000 7f44cccca980 20 zonegroup default 2024-05-30T16:22:54.139 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.139+0000 7f44cccca980 20 rados->read ofs=0 len=0 2024-05-30T16:22:54.140 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.139+0000 7f44cccca980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:54.140 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.139+0000 7f44cccca980 20 rados->read ofs=0 len=0 2024-05-30T16:22:54.140 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.139+0000 7f44cccca980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:54.140 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.139+0000 7f44cccca980 20 rados->read ofs=0 len=0 2024-05-30T16:22:54.141 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.139+0000 7f44cccca980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:54.141 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.139+0000 7f44cccca980 20 started sync module instance, tier type = 2024-05-30T16:22:54.141 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.139+0000 7f44cccca980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:22:54.165 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.163+0000 7f44cccca980 20 add_watcher() i=0 2024-05-30T16:22:54.165 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.163+0000 7f44cccca980 20 add_watcher() i=1 2024-05-30T16:22:54.165 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.163+0000 7f44cccca980 20 add_watcher() i=2 2024-05-30T16:22:54.165 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.163+0000 7f44cccca980 20 add_watcher() i=3 2024-05-30T16:22:54.165 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.163+0000 7f44cccca980 20 add_watcher() i=4 2024-05-30T16:22:54.165 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.163+0000 7f44cccca980 20 add_watcher() i=5 2024-05-30T16:22:54.166 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.163+0000 7f44cccca980 20 add_watcher() i=6 2024-05-30T16:22:54.168 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.167+0000 7f44cccca980 20 add_watcher() i=7 2024-05-30T16:22:54.168 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.167+0000 7f44cccca980 2 all 8 watchers are set, enabling cache 2024-05-30T16:22:54.172 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.171+0000 7f43817e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:54.173 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.171+0000 7f44cccca980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:22:54.173 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.171+0000 7f44cccca980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:22:54.173 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.171+0000 7f44cccca980 5 note: GC not initialized 2024-05-30T16:22:54.173 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.171+0000 7f437b7fe640 20 reqs_thread_entry: start 2024-05-30T16:22:54.173 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.171+0000 7f44cccca980 20 init_complete bucket index max shards: 11 2024-05-30T16:22:54.174 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.175+0000 7f44cccca980 20 Filter name: none 2024-05-30T16:22:54.174 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.175+0000 7f43797fa640 20 reqs_thread_entry: start 2024-05-30T16:22:54.174 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.175+0000 7f44cccca980 20 rados->read ofs=0 len=0 2024-05-30T16:22:54.175 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.175+0000 7f44cccca980 20 rados_obj.operate() r=0 bl.length=197 2024-05-30T16:22:54.175 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.175+0000 7f44cccca980 20 rados->read ofs=0 len=0 2024-05-30T16:22:54.176 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.175+0000 7f44cccca980 20 rados_obj.operate() r=0 bl.length=411 2024-05-30T16:22:54.176 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.175+0000 7f44cccca980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:22:54.176 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.175+0000 7f44cccca980 20 rados->read ofs=0 len=0 2024-05-30T16:22:54.177 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.175+0000 7f44cccca980 20 rados_obj.operate() r=0 bl.length=197 2024-05-30T16:22:54.177 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.175+0000 7f44cccca980 20 rados->read ofs=0 len=0 2024-05-30T16:22:54.178 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.179+0000 7f44cccca980 20 rados_obj.operate() r=0 bl.length=411 2024-05-30T16:22:54.178 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.179+0000 7f44cccca980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:22:54.181 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.179+0000 7f43817e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:54.186 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.187+0000 7f44cccca980 20 remove_watcher() i=0 2024-05-30T16:22:54.186 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.187+0000 7f44cccca980 2 removed watcher, disabling cache 2024-05-30T16:22:54.190 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.187+0000 7f44cccca980 20 remove_watcher() i=1 2024-05-30T16:22:54.192 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.191+0000 7f44cccca980 20 remove_watcher() i=2 2024-05-30T16:22:54.195 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.195+0000 7f44cccca980 20 remove_watcher() i=3 2024-05-30T16:22:54.197 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.195+0000 7f44cccca980 20 remove_watcher() i=4 2024-05-30T16:22:54.198 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.199+0000 7f44cccca980 20 remove_watcher() i=5 2024-05-30T16:22:54.200 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.199+0000 7f44cccca980 20 remove_watcher() i=6 2024-05-30T16:22:54.201 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.199+0000 7f44cccca980 20 remove_watcher() i=7 2024-05-30T16:22:54.225 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:54.225 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:22:54.225 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: myver id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 num_objects: 0 size_kb: 0 num_shards: 12 2024-05-30T16:22:54.225 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:54.306 INFO:tasks.workunit.client.0.smithi006.stderr: test: reshard removes olh entries with empty name 2024-05-30T16:22:54.729 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-05-30T16:22:54.761 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:22:54.761 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:22:54.807 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.807+0000 7ffb6bb4f980 20 rados->read ofs=0 len=0 2024-05-30T16:22:54.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.811+0000 7ffb6bb4f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:54.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.811+0000 7ffb6bb4f980 20 realm 2024-05-30T16:22:54.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.811+0000 7ffb6bb4f980 20 rados->read ofs=0 len=0 2024-05-30T16:22:54.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.811+0000 7ffb6bb4f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:54.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.811+0000 7ffb6bb4f980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:22:54.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.811+0000 7ffb6bb4f980 20 rados->read ofs=0 len=0 2024-05-30T16:22:54.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.811+0000 7ffb6bb4f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:54.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.811+0000 7ffb6bb4f980 20 rados->read ofs=0 len=0 2024-05-30T16:22:54.812 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.811+0000 7ffb6bb4f980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:54.812 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.811+0000 7ffb6bb4f980 20 rados->read ofs=0 len=0 2024-05-30T16:22:54.813 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.811+0000 7ffb6bb4f980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:22:54.813 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.811+0000 7ffb6bb4f980 20 searching for the correct realm 2024-05-30T16:22:54.819 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.819+0000 7ffb69e61640 0 -- 172.21.15.6:0/3728795362 >> [v2:172.21.15.204:6823/3156311527,v1:172.21.15.204:6825/3156311527] conn(0x7ffb6001c410 msgr2=0x7ffb6001c850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:54.825 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:54.823+0000 7ffb69660640 0 -- 172.21.15.6:0/3728795362 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7ffb60042910 msgr2=0x7ffb600323a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:55.041 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.039+0000 7ffb6bb4f980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:22:55.041 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.039+0000 7ffb6bb4f980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:22:55.041 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.039+0000 7ffb6bb4f980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:22:55.041 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.039+0000 7ffb6bb4f980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:22:55.041 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.039+0000 7ffb6bb4f980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:22:55.041 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.039+0000 7ffb6bb4f980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:22:55.041 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.039+0000 7ffb6bb4f980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.042 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.039+0000 7ffb6bb4f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:55.042 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.039+0000 7ffb6bb4f980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.042 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.043+0000 7ffb6bb4f980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:55.043 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.043+0000 7ffb6bb4f980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.043+0000 7ffb6bb4f980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:22:55.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.043+0000 7ffb6bb4f980 20 zone default found 2024-05-30T16:22:55.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.043+0000 7ffb6bb4f980 4 Realm: () 2024-05-30T16:22:55.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.043+0000 7ffb6bb4f980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:22:55.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.043+0000 7ffb6bb4f980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:22:55.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.043+0000 7ffb6bb4f980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:22:55.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.043+0000 7ffb6bb4f980 20 zonegroup default 2024-05-30T16:22:55.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.043+0000 7ffb6bb4f980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.045 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.043+0000 7ffb6bb4f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:55.045 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.043+0000 7ffb6bb4f980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.045 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.043+0000 7ffb6bb4f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:55.045 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.043+0000 7ffb6bb4f980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.047+0000 7ffb6bb4f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:55.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.047+0000 7ffb6bb4f980 20 started sync module instance, tier type = 2024-05-30T16:22:55.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.047+0000 7ffb6bb4f980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:22:55.071 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.071+0000 7ffb6bb4f980 20 add_watcher() i=0 2024-05-30T16:22:55.071 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.071+0000 7ffb6bb4f980 20 add_watcher() i=1 2024-05-30T16:22:55.071 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.071+0000 7ffb6bb4f980 20 add_watcher() i=2 2024-05-30T16:22:55.071 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.071+0000 7ffb6bb4f980 20 add_watcher() i=3 2024-05-30T16:22:55.071 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.071+0000 7ffb6bb4f980 20 add_watcher() i=4 2024-05-30T16:22:55.071 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.071+0000 7ffb6bb4f980 20 add_watcher() i=5 2024-05-30T16:22:55.071 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.071+0000 7ffb6bb4f980 20 add_watcher() i=6 2024-05-30T16:22:55.073 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.071+0000 7ffb6bb4f980 20 add_watcher() i=7 2024-05-30T16:22:55.073 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.071+0000 7ffb6bb4f980 2 all 8 watchers are set, enabling cache 2024-05-30T16:22:55.077 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.075+0000 7ffa1f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:55.078 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.079+0000 7ffb6bb4f980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:22:55.078 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.079+0000 7ffb6bb4f980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:22:55.078 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.079+0000 7ffb6bb4f980 5 note: GC not initialized 2024-05-30T16:22:55.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.079+0000 7ffa1e7e4640 20 reqs_thread_entry: start 2024-05-30T16:22:55.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.079+0000 7ffb6bb4f980 20 init_complete bucket index max shards: 11 2024-05-30T16:22:55.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.079+0000 7ffb6bb4f980 20 Filter name: none 2024-05-30T16:22:55.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.079+0000 7ff98ffff640 20 reqs_thread_entry: start 2024-05-30T16:22:55.080 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.079+0000 7ffb6bb4f980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.080 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.079+0000 7ffb6bb4f980 20 rados_obj.operate() r=0 bl.length=200 2024-05-30T16:22:55.081 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.079+0000 7ffb6bb4f980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.082 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.083+0000 7ffb6bb4f980 20 rados_obj.operate() r=0 bl.length=414 2024-05-30T16:22:55.082 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.083+0000 7ffb6bb4f980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:22:55.082 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.083+0000 7ffb6bb4f980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.082 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.083+0000 7ffb6bb4f980 20 rados_obj.operate() r=0 bl.length=200 2024-05-30T16:22:55.083 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.083+0000 7ffb6bb4f980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.083 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.083+0000 7ffb6bb4f980 20 rados_obj.operate() r=0 bl.length=414 2024-05-30T16:22:55.083 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.083+0000 7ffb6bb4f980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:22:55.086 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.087+0000 7ffa1f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:55.092 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.091+0000 7ffb6bb4f980 20 remove_watcher() i=0 2024-05-30T16:22:55.092 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.091+0000 7ffb6bb4f980 2 removed watcher, disabling cache 2024-05-30T16:22:55.094 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.095+0000 7ffb6bb4f980 20 remove_watcher() i=1 2024-05-30T16:22:55.096 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.095+0000 7ffb6bb4f980 20 remove_watcher() i=2 2024-05-30T16:22:55.099 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.099+0000 7ffb6bb4f980 20 remove_watcher() i=3 2024-05-30T16:22:55.102 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.103+0000 7ffb6bb4f980 20 remove_watcher() i=4 2024-05-30T16:22:55.103 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.103+0000 7ffb6bb4f980 20 remove_watcher() i=5 2024-05-30T16:22:55.104 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.103+0000 7ffb6bb4f980 20 remove_watcher() i=6 2024-05-30T16:22:55.106 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.107+0000 7ffb6bb4f980 20 remove_watcher() i=7 2024-05-30T16:22:55.123 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:55.123 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:22:55.123 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: a-bucket id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 num_objects: 0 size_kb: 0 num_shards: 13 2024-05-30T16:22:55.123 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:55.124 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket layout --bucket a-bucket 2024-05-30T16:22:55.151 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:22:55.151 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:22:55.199 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.199+0000 7f39e1f54980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.200 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.199+0000 7f39e1f54980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:55.200 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.199+0000 7f39e1f54980 20 realm 2024-05-30T16:22:55.200 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.199+0000 7f39e1f54980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.201 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.199+0000 7f39e1f54980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:55.201 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.199+0000 7f39e1f54980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:22:55.201 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.199+0000 7f39e1f54980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.201 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.199+0000 7f39e1f54980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:55.201 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.199+0000 7f39e1f54980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.203 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.203+0000 7f39e1f54980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:55.203 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.203+0000 7f39e1f54980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.204 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.203+0000 7f39e1f54980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:22:55.204 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.203+0000 7f39e1f54980 20 searching for the correct realm 2024-05-30T16:22:55.210 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.211+0000 7f39e0266640 0 -- 172.21.15.6:0/3047574945 >> [v2:172.21.15.6:6808/1897362715,v1:172.21.15.6:6809/1897362715] conn(0x55725c78a150 msgr2=0x55725c78c610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:55.216 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.215+0000 7f39df264640 0 -- 172.21.15.6:0/3047574945 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f39d00d8060 msgr2=0x7f39d012e350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:22:55.226 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.227+0000 7f39e1f54980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:22:55.226 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.227+0000 7f39e1f54980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:22:55.226 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.227+0000 7f39e1f54980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:22:55.226 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.227+0000 7f39e1f54980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:22:55.226 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.227+0000 7f39e1f54980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:22:55.226 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.227+0000 7f39e1f54980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:22:55.226 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.227+0000 7f39e1f54980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.226 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.227+0000 7f39e1f54980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:55.226 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.227+0000 7f39e1f54980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.227 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.227+0000 7f39e1f54980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:55.227 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.227+0000 7f39e1f54980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.228 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.227+0000 7f39e1f54980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:22:55.228 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.227+0000 7f39e1f54980 20 zone default found 2024-05-30T16:22:55.228 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.227+0000 7f39e1f54980 4 Realm: () 2024-05-30T16:22:55.228 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.227+0000 7f39e1f54980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:22:55.228 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.227+0000 7f39e1f54980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:22:55.228 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.227+0000 7f39e1f54980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:22:55.228 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.227+0000 7f39e1f54980 20 zonegroup default 2024-05-30T16:22:55.228 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.227+0000 7f39e1f54980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.228 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.227+0000 7f39e1f54980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:55.228 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.227+0000 7f39e1f54980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.229 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.227+0000 7f39e1f54980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:55.229 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.227+0000 7f39e1f54980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.229 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.227+0000 7f39e1f54980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:55.229 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.227+0000 7f39e1f54980 20 started sync module instance, tier type = 2024-05-30T16:22:55.229 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.227+0000 7f39e1f54980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:22:55.246 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.243+0000 7f39e1f54980 20 add_watcher() i=0 2024-05-30T16:22:55.246 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.243+0000 7f39e1f54980 20 add_watcher() i=1 2024-05-30T16:22:55.246 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.243+0000 7f39e1f54980 20 add_watcher() i=2 2024-05-30T16:22:55.246 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.243+0000 7f39e1f54980 20 add_watcher() i=3 2024-05-30T16:22:55.246 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.243+0000 7f39e1f54980 20 add_watcher() i=4 2024-05-30T16:22:55.246 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.247+0000 7f39e1f54980 20 add_watcher() i=5 2024-05-30T16:22:55.246 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.247+0000 7f39e1f54980 20 add_watcher() i=6 2024-05-30T16:22:55.247 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.247+0000 7f39e1f54980 20 add_watcher() i=7 2024-05-30T16:22:55.247 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.247+0000 7f39e1f54980 2 all 8 watchers are set, enabling cache 2024-05-30T16:22:55.249 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.247+0000 7f3895fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:55.250 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.251+0000 7f39e1f54980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:22:55.250 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.251+0000 7f39e1f54980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:22:55.250 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.251+0000 7f39e1f54980 5 note: GC not initialized 2024-05-30T16:22:55.251 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.251+0000 7f3894fe1640 20 reqs_thread_entry: start 2024-05-30T16:22:55.251 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.251+0000 7f39e1f54980 20 init_complete bucket index max shards: 11 2024-05-30T16:22:55.251 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.251+0000 7f39e1f54980 20 Filter name: none 2024-05-30T16:22:55.251 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.251+0000 7f380e7fc640 20 reqs_thread_entry: start 2024-05-30T16:22:55.251 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.251+0000 7f39e1f54980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.252 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.251+0000 7f39e1f54980 20 rados_obj.operate() r=0 bl.length=200 2024-05-30T16:22:55.252 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.251+0000 7f39e1f54980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.252 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.251+0000 7f39e1f54980 20 rados_obj.operate() r=0 bl.length=414 2024-05-30T16:22:55.252 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.251+0000 7f39e1f54980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:22:55.253 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.251+0000 7f3895fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:55.258 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.259+0000 7f39e1f54980 20 remove_watcher() i=0 2024-05-30T16:22:55.258 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.259+0000 7f39e1f54980 2 removed watcher, disabling cache 2024-05-30T16:22:55.260 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.259+0000 7f39e1f54980 20 remove_watcher() i=1 2024-05-30T16:22:55.262 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.263+0000 7f39e1f54980 20 remove_watcher() i=2 2024-05-30T16:22:55.264 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.263+0000 7f39e1f54980 20 remove_watcher() i=3 2024-05-30T16:22:55.266 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.267+0000 7f39e1f54980 20 remove_watcher() i=4 2024-05-30T16:22:55.267 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.267+0000 7f39e1f54980 20 remove_watcher() i=5 2024-05-30T16:22:55.268 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.267+0000 7f39e1f54980 20 remove_watcher() i=6 2024-05-30T16:22:55.270 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.271+0000 7f39e1f54980 20 remove_watcher() i=7 2024-05-30T16:22:55.284 INFO:tasks.workunit.client.0.smithi006.stderr:rados -p default.rgw.buckets.index setomapval .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.0 --omap-key-file /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/olh_noname_key < /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/olh_noname_val 2024-05-30T16:22:55.310 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:22:55.310 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:22:55.329 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bi list --bucket a-bucket 2024-05-30T16:22:55.355 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:22:55.355 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:22:55.397 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.395+0000 7f0c30b14980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.400 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.399+0000 7f0c30b14980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:55.400 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.399+0000 7f0c30b14980 20 realm 2024-05-30T16:22:55.400 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.399+0000 7f0c30b14980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.400 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.399+0000 7f0c30b14980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:55.400 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.399+0000 7f0c30b14980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:22:55.400 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.399+0000 7f0c30b14980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.400 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.399+0000 7f0c30b14980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:55.400 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.399+0000 7f0c30b14980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.401 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.403+0000 7f0c30b14980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:55.402 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.403+0000 7f0c30b14980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.402 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.403+0000 7f0c30b14980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:22:55.402 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.403+0000 7f0c30b14980 20 searching for the correct realm 2024-05-30T16:22:55.408 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.407+0000 7f0c2ee26640 0 -- 172.21.15.6:0/839005660 >> [v2:172.21.15.6:6808/1897362715,v1:172.21.15.6:6809/1897362715] conn(0x563c1ae82cd0 msgr2=0x563c1ae85190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:55.415 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.415+0000 7f0c2de24640 0 -- 172.21.15.6:0/839005660 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f0c2001a040 msgr2=0x7f0c200983d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:55.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.423+0000 7f0c30b14980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:22:55.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.423+0000 7f0c30b14980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:22:55.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.423+0000 7f0c30b14980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:22:55.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.423+0000 7f0c30b14980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:22:55.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.423+0000 7f0c30b14980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:22:55.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.423+0000 7f0c30b14980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:22:55.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.423+0000 7f0c30b14980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.423+0000 7f0c30b14980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:55.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.423+0000 7f0c30b14980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.423+0000 7f0c30b14980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:55.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.423+0000 7f0c30b14980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.423+0000 7f0c30b14980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:22:55.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.423+0000 7f0c30b14980 20 zone default found 2024-05-30T16:22:55.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.423+0000 7f0c30b14980 4 Realm: () 2024-05-30T16:22:55.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.423+0000 7f0c30b14980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:22:55.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.423+0000 7f0c30b14980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:22:55.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.423+0000 7f0c30b14980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:22:55.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.423+0000 7f0c30b14980 20 zonegroup default 2024-05-30T16:22:55.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.423+0000 7f0c30b14980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.423+0000 7f0c30b14980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:55.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.423+0000 7f0c30b14980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.425 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.423+0000 7f0c30b14980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:55.425 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.423+0000 7f0c30b14980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.425 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.423+0000 7f0c30b14980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:55.425 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.423+0000 7f0c30b14980 20 started sync module instance, tier type = 2024-05-30T16:22:55.425 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.423+0000 7f0c30b14980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:22:55.441 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.439+0000 7f0c30b14980 20 add_watcher() i=0 2024-05-30T16:22:55.441 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.439+0000 7f0c30b14980 20 add_watcher() i=1 2024-05-30T16:22:55.441 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.439+0000 7f0c30b14980 20 add_watcher() i=2 2024-05-30T16:22:55.441 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.439+0000 7f0c30b14980 20 add_watcher() i=3 2024-05-30T16:22:55.441 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.439+0000 7f0c30b14980 20 add_watcher() i=4 2024-05-30T16:22:55.441 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.439+0000 7f0c30b14980 20 add_watcher() i=5 2024-05-30T16:22:55.441 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.439+0000 7f0c30b14980 20 add_watcher() i=6 2024-05-30T16:22:55.442 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.443+0000 7f0c30b14980 20 add_watcher() i=7 2024-05-30T16:22:55.442 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.443+0000 7f0c30b14980 2 all 8 watchers are set, enabling cache 2024-05-30T16:22:55.444 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.443+0000 7f0ae4fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:55.445 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.443+0000 7f0c30b14980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:22:55.445 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.443+0000 7f0c30b14980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:22:55.445 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.443+0000 7f0c30b14980 5 note: GC not initialized 2024-05-30T16:22:55.445 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.443+0000 7f0a5f7fe640 20 reqs_thread_entry: start 2024-05-30T16:22:55.445 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.443+0000 7f0c30b14980 20 init_complete bucket index max shards: 11 2024-05-30T16:22:55.446 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.443+0000 7f0c30b14980 20 Filter name: none 2024-05-30T16:22:55.446 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.443+0000 7f0a5d7fa640 20 reqs_thread_entry: start 2024-05-30T16:22:55.446 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.443+0000 7f0c30b14980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.447 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.447+0000 7f0c30b14980 20 rados_obj.operate() r=0 bl.length=200 2024-05-30T16:22:55.447 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.447+0000 7f0c30b14980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.447 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.447+0000 7f0c30b14980 20 rados_obj.operate() r=0 bl.length=414 2024-05-30T16:22:55.447 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.447+0000 7f0c30b14980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:22:55.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.447+0000 7f0c30b14980 20 INFO: main: max_entries=1000, index=gen=2, max_shards=13 2024-05-30T16:22:55.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.447+0000 7f0c30b14980 20 INFO: main: starting shard=0 2024-05-30T16:22:55.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.447+0000 7f0c30b14980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.0 2024-05-30T16:22:55.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.447+0000 7f0c30b14980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.0 2024-05-30T16:22:55.449 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.447+0000 7f0c30b14980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-05-30T16:22:55.449 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.447+0000 7f0c30b14980 20 INFO: main: starting shard=1 2024-05-30T16:22:55.449 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.447+0000 7f0c30b14980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.1 2024-05-30T16:22:55.449 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.447+0000 7f0c30b14980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.1 2024-05-30T16:22:55.450 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.451+0000 7f0c30b14980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:22:55.450 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.451+0000 7f0c30b14980 20 INFO: main: starting shard=2 2024-05-30T16:22:55.450 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.451+0000 7f0c30b14980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.2 2024-05-30T16:22:55.450 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.451+0000 7f0c30b14980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.2 2024-05-30T16:22:55.451 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.451+0000 7f0c30b14980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:22:55.451 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.451+0000 7f0c30b14980 20 INFO: main: starting shard=3 2024-05-30T16:22:55.451 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.451+0000 7f0c30b14980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.3 2024-05-30T16:22:55.451 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.451+0000 7f0c30b14980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.3 2024-05-30T16:22:55.452 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.451+0000 7f0c30b14980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:22:55.452 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.451+0000 7f0c30b14980 20 INFO: main: starting shard=4 2024-05-30T16:22:55.452 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.451+0000 7f0c30b14980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.4 2024-05-30T16:22:55.452 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.451+0000 7f0c30b14980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.4 2024-05-30T16:22:55.452 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.451+0000 7f0c30b14980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:22:55.453 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.451+0000 7f0c30b14980 20 INFO: main: starting shard=5 2024-05-30T16:22:55.453 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.451+0000 7f0c30b14980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.5 2024-05-30T16:22:55.453 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.451+0000 7f0c30b14980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.5 2024-05-30T16:22:55.453 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.451+0000 7f0c30b14980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:22:55.453 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.451+0000 7f0c30b14980 20 INFO: main: starting shard=6 2024-05-30T16:22:55.453 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.451+0000 7f0c30b14980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.6 2024-05-30T16:22:55.453 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.451+0000 7f0c30b14980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.6 2024-05-30T16:22:55.454 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.455+0000 7f0c30b14980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:22:55.454 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.455+0000 7f0c30b14980 20 INFO: main: starting shard=7 2024-05-30T16:22:55.454 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.455+0000 7f0c30b14980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.7 2024-05-30T16:22:55.454 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.455+0000 7f0c30b14980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.7 2024-05-30T16:22:55.456 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.455+0000 7f0c30b14980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:22:55.456 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.455+0000 7f0c30b14980 20 INFO: main: starting shard=8 2024-05-30T16:22:55.456 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.455+0000 7f0c30b14980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.8 2024-05-30T16:22:55.456 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.455+0000 7f0c30b14980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.8 2024-05-30T16:22:55.456 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.455+0000 7f0c30b14980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:22:55.456 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.455+0000 7f0c30b14980 20 INFO: main: starting shard=9 2024-05-30T16:22:55.456 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.455+0000 7f0c30b14980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.9 2024-05-30T16:22:55.457 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.455+0000 7f0c30b14980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.9 2024-05-30T16:22:55.457 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.455+0000 7f0c30b14980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:22:55.457 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.455+0000 7f0c30b14980 20 INFO: main: starting shard=10 2024-05-30T16:22:55.457 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.455+0000 7f0c30b14980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.10 2024-05-30T16:22:55.457 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.455+0000 7f0c30b14980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.10 2024-05-30T16:22:55.457 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.455+0000 7f0c30b14980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:22:55.458 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.455+0000 7f0c30b14980 20 INFO: main: starting shard=11 2024-05-30T16:22:55.458 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.455+0000 7f0c30b14980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.11 2024-05-30T16:22:55.458 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.455+0000 7f0c30b14980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.11 2024-05-30T16:22:55.459 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.459+0000 7f0c30b14980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:22:55.459 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.459+0000 7f0c30b14980 20 INFO: main: starting shard=12 2024-05-30T16:22:55.459 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.459+0000 7f0c30b14980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.12 2024-05-30T16:22:55.459 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.459+0000 7f0c30b14980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.12 2024-05-30T16:22:55.459 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.459+0000 7f0c30b14980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:22:55.459 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.459+0000 7f0c30b14980 20 INFO: main: done 2024-05-30T16:22:55.459 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.459+0000 7f0ae4fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:55.464 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.463+0000 7f0c30b14980 20 remove_watcher() i=0 2024-05-30T16:22:55.464 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.463+0000 7f0c30b14980 2 removed watcher, disabling cache 2024-05-30T16:22:55.466 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.467+0000 7f0c30b14980 20 remove_watcher() i=1 2024-05-30T16:22:55.469 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.467+0000 7f0c30b14980 20 remove_watcher() i=2 2024-05-30T16:22:55.471 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.471+0000 7f0c30b14980 20 remove_watcher() i=3 2024-05-30T16:22:55.473 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.471+0000 7f0c30b14980 20 remove_watcher() i=4 2024-05-30T16:22:55.474 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.475+0000 7f0c30b14980 20 remove_watcher() i=5 2024-05-30T16:22:55.475 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.475+0000 7f0c30b14980 20 remove_watcher() i=6 2024-05-30T16:22:55.477 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.475+0000 7f0c30b14980 20 remove_watcher() i=7 2024-05-30T16:22:55.491 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket reshard --bucket a-bucket --num-shards 1 --yes-i-really-mean-it 2024-05-30T16:22:55.522 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:22:55.522 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:22:55.567 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.567+0000 7faf5b5d2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.569 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.567+0000 7faf5b5d2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:55.569 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.567+0000 7faf5b5d2980 20 realm 2024-05-30T16:22:55.569 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.567+0000 7faf5b5d2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.569 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.567+0000 7faf5b5d2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:55.570 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.567+0000 7faf5b5d2980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:22:55.570 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.567+0000 7faf5b5d2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.570 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.567+0000 7faf5b5d2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:55.570 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.567+0000 7faf5b5d2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.571 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.571+0000 7faf5b5d2980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:55.571 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.571+0000 7faf5b5d2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.572 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.571+0000 7faf5b5d2980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:22:55.572 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.571+0000 7faf5b5d2980 20 searching for the correct realm 2024-05-30T16:22:55.581 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.579+0000 7faf598e4640 0 -- 172.21.15.6:0/3161970432 >> [v2:172.21.15.204:6800/3687190393,v1:172.21.15.204:6801/3687190393] conn(0x7faf50128540 msgr2=0x7faf500b7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:55.794 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.795+0000 7faf598e4640 0 -- 172.21.15.6:0/3161970432 >> [v2:172.21.15.204:6800/3687190393,v1:172.21.15.204:6801/3687190393] conn(0x7faf50128540 msgr2=0x7faf500b7910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:55.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.803+0000 7faf5b5d2980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:22:55.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.803+0000 7faf5b5d2980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:22:55.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.803+0000 7faf5b5d2980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:22:55.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.803+0000 7faf5b5d2980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:22:55.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.803+0000 7faf5b5d2980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:22:55.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.803+0000 7faf5b5d2980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:22:55.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.803+0000 7faf5b5d2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.807+0000 7faf5b5d2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:55.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.807+0000 7faf5b5d2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.807+0000 7faf5b5d2980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:55.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.807+0000 7faf5b5d2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.808 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.807+0000 7faf5b5d2980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:22:55.808 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.807+0000 7faf5b5d2980 20 zone default found 2024-05-30T16:22:55.808 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.807+0000 7faf5b5d2980 4 Realm: () 2024-05-30T16:22:55.808 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.807+0000 7faf5b5d2980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:22:55.808 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.807+0000 7faf5b5d2980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:22:55.808 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.807+0000 7faf5b5d2980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:22:55.808 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.807+0000 7faf5b5d2980 20 zonegroup default 2024-05-30T16:22:55.808 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.807+0000 7faf5b5d2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.807+0000 7faf5b5d2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:55.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.807+0000 7faf5b5d2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.807+0000 7faf5b5d2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:55.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.807+0000 7faf5b5d2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.811+0000 7faf5b5d2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:55.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.811+0000 7faf5b5d2980 20 started sync module instance, tier type = 2024-05-30T16:22:55.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.811+0000 7faf5b5d2980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:22:55.834 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.835+0000 7faf5b5d2980 20 add_watcher() i=0 2024-05-30T16:22:55.834 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.835+0000 7faf5b5d2980 20 add_watcher() i=1 2024-05-30T16:22:55.834 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.835+0000 7faf5b5d2980 20 add_watcher() i=2 2024-05-30T16:22:55.834 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.835+0000 7faf5b5d2980 20 add_watcher() i=3 2024-05-30T16:22:55.834 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.835+0000 7faf5b5d2980 20 add_watcher() i=4 2024-05-30T16:22:55.834 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.835+0000 7faf5b5d2980 20 add_watcher() i=5 2024-05-30T16:22:55.834 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.835+0000 7faf5b5d2980 20 add_watcher() i=6 2024-05-30T16:22:55.836 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.835+0000 7faf5b5d2980 20 add_watcher() i=7 2024-05-30T16:22:55.837 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.835+0000 7faf5b5d2980 2 all 8 watchers are set, enabling cache 2024-05-30T16:22:55.841 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.839+0000 7fae0f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:55.842 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.843+0000 7faf5b5d2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:22:55.842 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.843+0000 7faf5b5d2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:22:55.842 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.843+0000 7faf5b5d2980 5 note: GC not initialized 2024-05-30T16:22:55.843 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.843+0000 7fae0e7e4640 20 reqs_thread_entry: start 2024-05-30T16:22:55.843 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.843+0000 7faf5b5d2980 20 init_complete bucket index max shards: 11 2024-05-30T16:22:55.843 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.843+0000 7faf5b5d2980 20 Filter name: none 2024-05-30T16:22:55.843 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.843+0000 7fad7ffff640 20 reqs_thread_entry: start 2024-05-30T16:22:55.843 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.843+0000 7faf5b5d2980 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-05-30T16:22:55.843 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.843+0000 7faf5b5d2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.844 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.843+0000 7faf5b5d2980 20 rados_obj.operate() r=0 bl.length=200 2024-05-30T16:22:55.844 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.843+0000 7faf5b5d2980 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2024-05-30T16:22:55.844 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.843+0000 7faf5b5d2980 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-05-30T16:22:55.844 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.843+0000 7faf5b5d2980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 : miss 2024-05-30T16:22:55.844 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.843+0000 7faf5b5d2980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.845 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.843+0000 7faf5b5d2980 20 rados_obj.operate() r=0 bl.length=414 2024-05-30T16:22:55.845 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.843+0000 7faf5b5d2980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 info.flags=0x17 2024-05-30T16:22:55.845 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.843+0000 7faf5b5d2980 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 to cache LRU end 2024-05-30T16:22:55.845 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.843+0000 7faf5b5d2980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:22:55.845 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.843+0000 7faf5b5d2980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:22:55.845 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.843+0000 7faf5b5d2980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 2024-05-30T16:22:55.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.851+0000 7faf5b5d2980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 info.flags=0x17 2024-05-30T16:22:55.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.851+0000 7faf5b5d2980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 to cache LRU end 2024-05-30T16:22:55.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.851+0000 7faf5b5d2980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:22:55.853 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.851+0000 7faf5b5d2980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18, ofs0, ns] 2024-05-30T16:22:55.853 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.851+0000 7fae2bfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431632 cookie 93911161453728 notifier 6788 bl.length()=903 2024-05-30T16:22:55.853 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.851+0000 7fae2bfff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 info.flags=0x17 2024-05-30T16:22:55.853 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.851+0000 7fae2bfff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 to cache LRU end 2024-05-30T16:22:55.853 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.851+0000 7fae2bfff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:22:55.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.855+0000 7faf5b5d2980 20 handle_bi_update(): bucket=:a-bucket[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18]): orig_sources= new_sources= 2024-05-30T16:22:55.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.855+0000 7faf5b5d2980 20 handle_bi_update(): bucket=:a-bucket[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18]): potential sources added=[] removed=[] 2024-05-30T16:22:55.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.855+0000 7faf5b5d2980 20 handle_bi_update(): bucket=:a-bucket[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18]): orig_dests= new_dests= 2024-05-30T16:22:55.856 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.855+0000 7faf5b5d2980 20 handle_bi_update(): bucket=:a-bucket[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18]): potential dests added=[] removed=[] 2024-05-30T16:22:55.861 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.859+0000 7faf5b5d2980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.3.0 2024-05-30T16:22:55.861 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.859+0000 7faf5b5d2980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.3.0 2024-05-30T16:22:55.861 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.859+0000 7faf5b5d2980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.0 2024-05-30T16:22:55.861 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.859+0000 7faf5b5d2980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.0 2024-05-30T16:22:55.862 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.863+0000 7faf5b5d2980 10 Dropping entry with empty name, idx=�1001_04/57/0457f727ec113e418d5b16d206b200ed068c0533554883ce811df7c932a3df68/2018_12_11/2889999/3386469/metadata.gz 2024-05-30T16:22:55.862 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.863+0000 7faf5b5d2980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.1 2024-05-30T16:22:55.863 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.863+0000 7faf5b5d2980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.1 2024-05-30T16:22:55.863 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.863+0000 7faf5b5d2980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.2 2024-05-30T16:22:55.864 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.863+0000 7faf5b5d2980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.2 2024-05-30T16:22:55.865 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.863+0000 7faf5b5d2980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.3 2024-05-30T16:22:55.865 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.863+0000 7faf5b5d2980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.3 2024-05-30T16:22:55.866 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.867+0000 7faf5b5d2980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.4 2024-05-30T16:22:55.866 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.867+0000 7faf5b5d2980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.4 2024-05-30T16:22:55.866 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.867+0000 7faf5b5d2980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.5 2024-05-30T16:22:55.867 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.867+0000 7faf5b5d2980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.5 2024-05-30T16:22:55.867 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.867+0000 7faf5b5d2980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.6 2024-05-30T16:22:55.867 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.867+0000 7faf5b5d2980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.6 2024-05-30T16:22:55.868 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.867+0000 7faf5b5d2980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.7 2024-05-30T16:22:55.868 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.867+0000 7faf5b5d2980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.7 2024-05-30T16:22:55.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.867+0000 7faf5b5d2980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.8 2024-05-30T16:22:55.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.867+0000 7faf5b5d2980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.8 2024-05-30T16:22:55.870 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.871+0000 7faf5b5d2980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.9 2024-05-30T16:22:55.871 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.871+0000 7faf5b5d2980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.9 2024-05-30T16:22:55.871 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.871+0000 7faf5b5d2980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.10 2024-05-30T16:22:55.871 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.871+0000 7faf5b5d2980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.10 2024-05-30T16:22:55.871 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.871+0000 7faf5b5d2980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.11 2024-05-30T16:22:55.872 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.871+0000 7faf5b5d2980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.11 2024-05-30T16:22:55.872 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.871+0000 7faf5b5d2980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.12 2024-05-30T16:22:55.872 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.871+0000 7faf5b5d2980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.2.12 2024-05-30T16:22:55.873 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.871+0000 7faf5b5d2980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 : hit (requested=0x17, cached=0x17) 2024-05-30T16:22:55.873 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.871+0000 7faf5b5d2980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 2024-05-30T16:22:55.875 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.875+0000 7faf5b5d2980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 info.flags=0x17 2024-05-30T16:22:55.875 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.875+0000 7faf5b5d2980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 to cache LRU end 2024-05-30T16:22:55.875 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.875+0000 7faf5b5d2980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:22:55.875 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.875+0000 7faf5b5d2980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18, ofs0, ns] 2024-05-30T16:22:55.876 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.875+0000 7fae2bfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431633 cookie 93911161453728 notifier 6788 bl.length()=871 2024-05-30T16:22:55.876 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.875+0000 7fae2bfff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 info.flags=0x17 2024-05-30T16:22:55.876 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.875+0000 7fae2bfff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18 to cache LRU end 2024-05-30T16:22:55.876 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.875+0000 7fae2bfff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:22:55.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.879+0000 7faf5b5d2980 20 handle_bi_update(): bucket=:a-bucket[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18]): orig_sources= new_sources= 2024-05-30T16:22:55.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.879+0000 7faf5b5d2980 20 handle_bi_update(): bucket=:a-bucket[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18]): potential sources added=[] removed=[] 2024-05-30T16:22:55.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.879+0000 7faf5b5d2980 20 handle_bi_update(): bucket=:a-bucket[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18]): orig_dests= new_dests= 2024-05-30T16:22:55.879 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.879+0000 7faf5b5d2980 20 handle_bi_update(): bucket=:a-bucket[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18]): potential dests added=[] removed=[] 2024-05-30T16:22:55.884 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.883+0000 7faf5b5d2980 1 execute INFO: reshard of bucket "a-bucket" completed successfully 2024-05-30T16:22:55.887 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.887+0000 7fae0f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:55.892 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.891+0000 7faf5b5d2980 20 remove_watcher() i=0 2024-05-30T16:22:55.892 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.891+0000 7faf5b5d2980 2 removed watcher, disabling cache 2024-05-30T16:22:55.894 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.895+0000 7faf5b5d2980 20 remove_watcher() i=1 2024-05-30T16:22:55.897 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.895+0000 7faf5b5d2980 20 remove_watcher() i=2 2024-05-30T16:22:55.899 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.899+0000 7faf5b5d2980 20 remove_watcher() i=3 2024-05-30T16:22:55.901 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.899+0000 7faf5b5d2980 20 remove_watcher() i=4 2024-05-30T16:22:55.903 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.903+0000 7faf5b5d2980 20 remove_watcher() i=5 2024-05-30T16:22:55.904 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.903+0000 7faf5b5d2980 20 remove_watcher() i=6 2024-05-30T16:22:55.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.907+0000 7faf5b5d2980 20 remove_watcher() i=7 2024-05-30T16:22:55.923 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bi list --bucket a-bucket 2024-05-30T16:22:55.950 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:22:55.950 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:22:55.994 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.995+0000 7fb3e764d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.996 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.995+0000 7fb3e764d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:55.996 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.995+0000 7fb3e764d980 20 realm 2024-05-30T16:22:55.996 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.995+0000 7fb3e764d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.997 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.995+0000 7fb3e764d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:55.997 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.995+0000 7fb3e764d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:22:55.997 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.995+0000 7fb3e764d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.997 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.995+0000 7fb3e764d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:55.997 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.995+0000 7fb3e764d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.998 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.999+0000 7fb3e764d980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:55.998 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.999+0000 7fb3e764d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:55.999 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.999+0000 7fb3e764d980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:22:55.999 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:55.999+0000 7fb3e764d980 20 searching for the correct realm 2024-05-30T16:22:56.005 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.007+0000 7fb3e495d640 0 -- 172.21.15.6:0/3903736040 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7fb3d801f040 msgr2=0x7fb3d8094310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:22:56.013 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.011+0000 7fb3e495d640 0 -- 172.21.15.6:0/3903736040 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7fb2b007e540 msgr2=0x7fb2b0080970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:56.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.019+0000 7fb3e764d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:22:56.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.019+0000 7fb3e764d980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:22:56.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.019+0000 7fb3e764d980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:22:56.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.019+0000 7fb3e764d980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:22:56.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.019+0000 7fb3e764d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:22:56.021 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.019+0000 7fb3e764d980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:22:56.021 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.019+0000 7fb3e764d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:56.021 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.019+0000 7fb3e764d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:56.021 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.019+0000 7fb3e764d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:56.021 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.023+0000 7fb3e764d980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:56.022 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.023+0000 7fb3e764d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:56.022 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.023+0000 7fb3e764d980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:22:56.022 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.023+0000 7fb3e764d980 20 zone default found 2024-05-30T16:22:56.022 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.023+0000 7fb3e764d980 4 Realm: () 2024-05-30T16:22:56.022 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.023+0000 7fb3e764d980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:22:56.023 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.023+0000 7fb3e764d980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:22:56.023 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.023+0000 7fb3e764d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:22:56.023 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.023+0000 7fb3e764d980 20 zonegroup default 2024-05-30T16:22:56.023 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.023+0000 7fb3e764d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:56.024 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.023+0000 7fb3e764d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:56.024 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.023+0000 7fb3e764d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:56.024 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.023+0000 7fb3e764d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:56.024 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.023+0000 7fb3e764d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:56.024 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.023+0000 7fb3e764d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:56.024 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.023+0000 7fb3e764d980 20 started sync module instance, tier type = 2024-05-30T16:22:56.024 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.023+0000 7fb3e764d980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:22:56.040 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.039+0000 7fb3e764d980 20 add_watcher() i=0 2024-05-30T16:22:56.040 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.039+0000 7fb3e764d980 20 add_watcher() i=1 2024-05-30T16:22:56.040 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.039+0000 7fb3e764d980 20 add_watcher() i=2 2024-05-30T16:22:56.040 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.039+0000 7fb3e764d980 20 add_watcher() i=3 2024-05-30T16:22:56.040 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.039+0000 7fb3e764d980 20 add_watcher() i=4 2024-05-30T16:22:56.040 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.039+0000 7fb3e764d980 20 add_watcher() i=5 2024-05-30T16:22:56.040 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.039+0000 7fb3e764d980 20 add_watcher() i=6 2024-05-30T16:22:56.041 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.039+0000 7fb3e764d980 20 add_watcher() i=7 2024-05-30T16:22:56.041 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.039+0000 7fb3e764d980 2 all 8 watchers are set, enabling cache 2024-05-30T16:22:56.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.043+0000 7fb29b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:56.044 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.043+0000 7fb3e764d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:22:56.045 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.043+0000 7fb3e764d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:22:56.045 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.043+0000 7fb3e764d980 5 note: GC not initialized 2024-05-30T16:22:56.045 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.043+0000 7fb29a7e4640 20 reqs_thread_entry: start 2024-05-30T16:22:56.045 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.043+0000 7fb3e764d980 20 init_complete bucket index max shards: 11 2024-05-30T16:22:56.045 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.043+0000 7fb3e764d980 20 Filter name: none 2024-05-30T16:22:56.045 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.043+0000 7fb20bfff640 20 reqs_thread_entry: start 2024-05-30T16:22:56.045 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.043+0000 7fb3e764d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:56.045 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.047+0000 7fb3e764d980 20 rados_obj.operate() r=0 bl.length=200 2024-05-30T16:22:56.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.047+0000 7fb3e764d980 20 rados->read ofs=0 len=0 2024-05-30T16:22:56.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.047+0000 7fb3e764d980 20 rados_obj.operate() r=0 bl.length=414 2024-05-30T16:22:56.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.047+0000 7fb3e764d980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:22:56.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.047+0000 7fb3e764d980 20 INFO: main: max_entries=1000, index=gen=3, max_shards=1 2024-05-30T16:22:56.047 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.047+0000 7fb3e764d980 20 INFO: main: starting shard=0 2024-05-30T16:22:56.047 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.047+0000 7fb3e764d980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.3.0 2024-05-30T16:22:56.047 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.047+0000 7fb3e764d980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18.3.0 2024-05-30T16:22:56.047 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.047+0000 7fb3e764d980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:22:56.047 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.047+0000 7fb3e764d980 20 INFO: main: done 2024-05-30T16:22:56.048 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.047+0000 7fb29b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:56.053 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.055+0000 7fb3e764d980 20 remove_watcher() i=0 2024-05-30T16:22:56.054 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.055+0000 7fb3e764d980 2 removed watcher, disabling cache 2024-05-30T16:22:56.056 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.055+0000 7fb3e764d980 20 remove_watcher() i=1 2024-05-30T16:22:56.058 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.059+0000 7fb3e764d980 20 remove_watcher() i=2 2024-05-30T16:22:56.060 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.059+0000 7fb3e764d980 20 remove_watcher() i=3 2024-05-30T16:22:56.062 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.063+0000 7fb3e764d980 20 remove_watcher() i=4 2024-05-30T16:22:56.064 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.063+0000 7fb3e764d980 20 remove_watcher() i=5 2024-05-30T16:22:56.065 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.063+0000 7fb3e764d980 20 remove_watcher() i=6 2024-05-30T16:22:56.066 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.067+0000 7fb3e764d980 20 remove_watcher() i=7 2024-05-30T16:22:56.080 INFO:tasks.workunit.client.0.smithi006.stderr: test: PUT succeeds during reshard 2024-05-30T16:22:56.081 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket myver 2024-05-30T16:22:56.109 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:22:56.109 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:22:56.153 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.151+0000 7f1efc961980 20 rados->read ofs=0 len=0 2024-05-30T16:22:56.155 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.155+0000 7f1efc961980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:56.155 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.155+0000 7f1efc961980 20 realm 2024-05-30T16:22:56.155 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.155+0000 7f1efc961980 20 rados->read ofs=0 len=0 2024-05-30T16:22:56.155 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.155+0000 7f1efc961980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:56.155 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.155+0000 7f1efc961980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:22:56.155 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.155+0000 7f1efc961980 20 rados->read ofs=0 len=0 2024-05-30T16:22:56.155 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.155+0000 7f1efc961980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:56.156 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.155+0000 7f1efc961980 20 rados->read ofs=0 len=0 2024-05-30T16:22:56.157 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.155+0000 7f1efc961980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:56.157 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.155+0000 7f1efc961980 20 rados->read ofs=0 len=0 2024-05-30T16:22:56.158 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.159+0000 7f1efc961980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:22:56.158 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.159+0000 7f1efc961980 20 searching for the correct realm 2024-05-30T16:22:56.166 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.167+0000 7f1efac73640 0 -- 172.21.15.6:0/1810742922 >> [v2:172.21.15.204:6800/3687190393,v1:172.21.15.204:6801/3687190393] conn(0x7f1ef003ab70 msgr2=0x7f1ef005f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:56.383 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.383+0000 7f1efa472640 0 -- 172.21.15.6:0/1810742922 >> [v2:172.21.15.204:6816/2677861390,v1:172.21.15.204:6817/2677861390] conn(0x5609475ae4c0 msgr2=0x5609475b0980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:56.391 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.391+0000 7f1efc961980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:22:56.391 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.391+0000 7f1efc961980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:22:56.391 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.391+0000 7f1efc961980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:22:56.391 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.391+0000 7f1efc961980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:22:56.391 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.391+0000 7f1efc961980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:22:56.391 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.391+0000 7f1efc961980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:22:56.391 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.391+0000 7f1efc961980 20 rados->read ofs=0 len=0 2024-05-30T16:22:56.392 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.391+0000 7f1efc961980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:56.392 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.391+0000 7f1efc961980 20 rados->read ofs=0 len=0 2024-05-30T16:22:56.392 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.391+0000 7f1efc961980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:56.392 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.391+0000 7f1efc961980 20 rados->read ofs=0 len=0 2024-05-30T16:22:56.393 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.391+0000 7f1efc961980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:22:56.393 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.391+0000 7f1efc961980 20 zone default found 2024-05-30T16:22:56.393 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.391+0000 7f1efc961980 4 Realm: () 2024-05-30T16:22:56.393 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.391+0000 7f1efc961980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:22:56.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.391+0000 7f1efc961980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:22:56.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.391+0000 7f1efc961980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:22:56.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.391+0000 7f1efc961980 20 zonegroup default 2024-05-30T16:22:56.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.391+0000 7f1efc961980 20 rados->read ofs=0 len=0 2024-05-30T16:22:56.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.395+0000 7f1efc961980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:56.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.395+0000 7f1efc961980 20 rados->read ofs=0 len=0 2024-05-30T16:22:56.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.395+0000 7f1efc961980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:56.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.395+0000 7f1efc961980 20 rados->read ofs=0 len=0 2024-05-30T16:22:56.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.395+0000 7f1efc961980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:56.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.395+0000 7f1efc961980 20 started sync module instance, tier type = 2024-05-30T16:22:56.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.395+0000 7f1efc961980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:22:56.419 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.419+0000 7f1efc961980 20 add_watcher() i=0 2024-05-30T16:22:56.419 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.419+0000 7f1efc961980 20 add_watcher() i=1 2024-05-30T16:22:56.420 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.419+0000 7f1efc961980 20 add_watcher() i=2 2024-05-30T16:22:56.420 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.419+0000 7f1efc961980 20 add_watcher() i=3 2024-05-30T16:22:56.420 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.419+0000 7f1efc961980 20 add_watcher() i=4 2024-05-30T16:22:56.420 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.419+0000 7f1efc961980 20 add_watcher() i=5 2024-05-30T16:22:56.420 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.419+0000 7f1efc961980 20 add_watcher() i=6 2024-05-30T16:22:56.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.423+0000 7f1efc961980 20 add_watcher() i=7 2024-05-30T16:22:56.422 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.423+0000 7f1efc961980 2 all 8 watchers are set, enabling cache 2024-05-30T16:22:56.426 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.427+0000 7f1db17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:56.427 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.427+0000 7f1efc961980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:22:56.427 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.427+0000 7f1efc961980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:22:56.427 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.427+0000 7f1efc961980 5 note: GC not initialized 2024-05-30T16:22:56.428 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.427+0000 7f1dab7fe640 20 reqs_thread_entry: start 2024-05-30T16:22:56.428 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.427+0000 7f1efc961980 20 init_complete bucket index max shards: 11 2024-05-30T16:22:56.428 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.427+0000 7f1efc961980 20 Filter name: none 2024-05-30T16:22:56.429 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.427+0000 7f1da97fa640 20 reqs_thread_entry: start 2024-05-30T16:22:56.429 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.427+0000 7f1efc961980 20 rados->read ofs=0 len=0 2024-05-30T16:22:56.429 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.427+0000 7f1efc961980 20 rados_obj.operate() r=0 bl.length=197 2024-05-30T16:22:56.429 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.427+0000 7f1efc961980 20 rados->read ofs=0 len=0 2024-05-30T16:22:56.430 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.431+0000 7f1efc961980 20 rados_obj.operate() r=0 bl.length=411 2024-05-30T16:22:56.430 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.431+0000 7f1efc961980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:22:56.430 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.431+0000 7f1efc961980 20 rados->read ofs=0 len=0 2024-05-30T16:22:56.431 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.431+0000 7f1efc961980 20 rados_obj.operate() r=0 bl.length=197 2024-05-30T16:22:56.431 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.431+0000 7f1efc961980 20 rados->read ofs=0 len=0 2024-05-30T16:22:56.432 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.431+0000 7f1efc961980 20 rados_obj.operate() r=0 bl.length=411 2024-05-30T16:22:56.432 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.431+0000 7f1efc961980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:22:56.436 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.435+0000 7f1db17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:56.441 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.439+0000 7f1efc961980 20 remove_watcher() i=0 2024-05-30T16:22:56.441 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.439+0000 7f1efc961980 2 removed watcher, disabling cache 2024-05-30T16:22:56.443 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.443+0000 7f1efc961980 20 remove_watcher() i=1 2024-05-30T16:22:56.445 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.443+0000 7f1efc961980 20 remove_watcher() i=2 2024-05-30T16:22:56.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.447+0000 7f1efc961980 20 remove_watcher() i=3 2024-05-30T16:22:56.450 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.451+0000 7f1efc961980 20 remove_watcher() i=4 2024-05-30T16:22:56.451 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.451+0000 7f1efc961980 20 remove_watcher() i=5 2024-05-30T16:22:56.453 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.451+0000 7f1efc961980 20 remove_watcher() i=6 2024-05-30T16:22:56.455 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.455+0000 7f1efc961980 20 remove_watcher() i=7 2024-05-30T16:22:56.473 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:56.473 INFO:tasks.workunit.client.0.smithi006.stderr:BUCKET_STATS: 2024-05-30T16:22:56.473 INFO:tasks.workunit.client.0.smithi006.stderr:bucket: myver id: 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 num_objects: 0 size_kb: 0 num_shards: 12 2024-05-30T16:22:56.473 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:22:56.473 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin --inject-delay-at=do_reshard --inject-delay-ms=5000 bucket reshard --bucket myver --num-shards 13 2024-05-30T16:22:56.510 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:22:56.510 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:22:56.555 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.555+0000 7f4d106be980 20 rados->read ofs=0 len=0 2024-05-30T16:22:56.556 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.555+0000 7f4d106be980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:56.556 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.555+0000 7f4d106be980 20 realm 2024-05-30T16:22:56.556 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.555+0000 7f4d106be980 20 rados->read ofs=0 len=0 2024-05-30T16:22:56.557 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.555+0000 7f4d106be980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:56.557 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.555+0000 7f4d106be980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:22:56.557 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.555+0000 7f4d106be980 20 rados->read ofs=0 len=0 2024-05-30T16:22:56.557 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.555+0000 7f4d106be980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:56.557 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.555+0000 7f4d106be980 20 rados->read ofs=0 len=0 2024-05-30T16:22:56.558 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.559+0000 7f4d106be980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:56.558 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.559+0000 7f4d106be980 20 rados->read ofs=0 len=0 2024-05-30T16:22:56.559 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.559+0000 7f4d106be980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:22:56.559 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.559+0000 7f4d106be980 20 searching for the correct realm 2024-05-30T16:22:56.563 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.563+0000 7f4d0d9ce640 0 -- 172.21.15.6:0/3802966696 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f4d00042bf0 msgr2=0x7f4d00038400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:56.779 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.779+0000 7f4d0e1cf640 0 -- 172.21.15.6:0/3802966696 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f4d0407c2a0 msgr2=0x7f4d0408e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:22:56.996 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.995+0000 7f4d106be980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:22:56.997 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.995+0000 7f4d106be980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:22:56.997 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.995+0000 7f4d106be980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:22:56.997 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.995+0000 7f4d106be980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:22:56.997 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.995+0000 7f4d106be980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:22:56.997 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.995+0000 7f4d106be980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:22:56.997 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.995+0000 7f4d106be980 20 rados->read ofs=0 len=0 2024-05-30T16:22:56.997 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.995+0000 7f4d106be980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:56.997 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.995+0000 7f4d106be980 20 rados->read ofs=0 len=0 2024-05-30T16:22:56.998 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.999+0000 7f4d106be980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:22:56.998 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.999+0000 7f4d106be980 20 rados->read ofs=0 len=0 2024-05-30T16:22:56.999 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.999+0000 7f4d106be980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:22:56.999 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.999+0000 7f4d106be980 20 zone default found 2024-05-30T16:22:56.999 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.999+0000 7f4d106be980 4 Realm: () 2024-05-30T16:22:57.000 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.999+0000 7f4d106be980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:22:57.000 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.999+0000 7f4d106be980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:22:57.000 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.999+0000 7f4d106be980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:22:57.000 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.999+0000 7f4d106be980 20 zonegroup default 2024-05-30T16:22:57.000 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.999+0000 7f4d106be980 20 rados->read ofs=0 len=0 2024-05-30T16:22:57.000 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.999+0000 7f4d106be980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:57.000 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.999+0000 7f4d106be980 20 rados->read ofs=0 len=0 2024-05-30T16:22:57.001 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.999+0000 7f4d106be980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:57.001 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:56.999+0000 7f4d106be980 20 rados->read ofs=0 len=0 2024-05-30T16:22:57.002 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.003+0000 7f4d106be980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:22:57.002 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.003+0000 7f4d106be980 20 started sync module instance, tier type = 2024-05-30T16:22:57.002 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.003+0000 7f4d106be980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:22:57.025 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.023+0000 7f4d106be980 20 add_watcher() i=0 2024-05-30T16:22:57.025 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.023+0000 7f4d106be980 20 add_watcher() i=1 2024-05-30T16:22:57.025 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.023+0000 7f4d106be980 20 add_watcher() i=2 2024-05-30T16:22:57.025 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.023+0000 7f4d106be980 20 add_watcher() i=3 2024-05-30T16:22:57.025 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.023+0000 7f4d106be980 20 add_watcher() i=4 2024-05-30T16:22:57.025 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.023+0000 7f4d106be980 20 add_watcher() i=5 2024-05-30T16:22:57.026 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.027+0000 7f4d106be980 20 add_watcher() i=6 2024-05-30T16:22:57.027 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.027+0000 7f4d106be980 20 add_watcher() i=7 2024-05-30T16:22:57.027 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.027+0000 7f4d106be980 2 all 8 watchers are set, enabling cache 2024-05-30T16:22:57.031 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.031+0000 7f4bc4fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:22:57.032 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.031+0000 7f4d106be980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:22:57.032 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.031+0000 7f4d106be980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:22:57.032 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.031+0000 7f4d106be980 5 note: GC not initialized 2024-05-30T16:22:57.033 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.031+0000 7f4b3f7fe640 20 reqs_thread_entry: start 2024-05-30T16:22:57.033 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.031+0000 7f4d106be980 20 init_complete bucket index max shards: 11 2024-05-30T16:22:57.033 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.031+0000 7f4d106be980 20 Filter name: none 2024-05-30T16:22:57.033 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.031+0000 7f4b3d7fa640 20 reqs_thread_entry: start 2024-05-30T16:22:57.034 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.035+0000 7f4d106be980 10 cache get: name=default.rgw.meta+root+myver : miss 2024-05-30T16:22:57.034 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.035+0000 7f4d106be980 20 rados->read ofs=0 len=0 2024-05-30T16:22:57.034 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.035+0000 7f4d106be980 20 rados_obj.operate() r=0 bl.length=197 2024-05-30T16:22:57.034 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.035+0000 7f4d106be980 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2024-05-30T16:22:57.034 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.035+0000 7f4d106be980 10 adding default.rgw.meta+root+myver to cache LRU end 2024-05-30T16:22:57.034 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.035+0000 7f4d106be980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 : miss 2024-05-30T16:22:57.034 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.035+0000 7f4d106be980 20 rados->read ofs=0 len=0 2024-05-30T16:22:57.036 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.035+0000 7f4d106be980 20 rados_obj.operate() r=0 bl.length=411 2024-05-30T16:22:57.036 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.035+0000 7f4d106be980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 info.flags=0x17 2024-05-30T16:22:57.036 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.035+0000 7f4d106be980 10 adding default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 to cache LRU end 2024-05-30T16:22:57.036 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.035+0000 7f4d106be980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:22:57.036 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.035+0000 7f4d106be980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:22:57.036 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.035+0000 7f4d106be980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 2024-05-30T16:22:57.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.047+0000 7f4d106be980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 info.flags=0x17 2024-05-30T16:22:57.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.047+0000 7f4d106be980 10 moving default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 to cache LRU end 2024-05-30T16:22:57.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.047+0000 7f4d106be980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:22:57.046 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.047+0000 7f4d106be980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19, ofs0, ns] 2024-05-30T16:22:57.047 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.047+0000 7f4bf1ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431592 cookie 94323143594016 notifier 7212 bl.length()=897 2024-05-30T16:22:57.047 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.047+0000 7f4bf1ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 info.flags=0x17 2024-05-30T16:22:57.047 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.047+0000 7f4bf1ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 to cache LRU end 2024-05-30T16:22:57.047 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.047+0000 7f4bf1ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:22:57.050 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.051+0000 7f4d106be980 20 handle_bi_update(): bucket=:myver[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19]): orig_sources= new_sources= 2024-05-30T16:22:57.050 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.051+0000 7f4d106be980 20 handle_bi_update(): bucket=:myver[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19]): potential sources added=[] removed=[] 2024-05-30T16:22:57.050 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.051+0000 7f4d106be980 20 handle_bi_update(): bucket=:myver[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19]): orig_dests= new_dests= 2024-05-30T16:22:57.050 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.051+0000 7f4d106be980 20 handle_bi_update(): bucket=:myver[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19]): potential dests added=[] removed=[] 2024-05-30T16:22:57.056 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:22:57.055+0000 7f4d106be980 -1 Injecting delay=5.000000000s at location=do_reshard 2024-05-30T16:23:02.057 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.055+0000 7f4d106be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.0 2024-05-30T16:23:02.057 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.055+0000 7f4d106be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.0 2024-05-30T16:23:02.057 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.055+0000 7f4d106be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.1 2024-05-30T16:23:02.057 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.055+0000 7f4d106be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.1 2024-05-30T16:23:02.057 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.055+0000 7f4d106be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.2 2024-05-30T16:23:02.057 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.055+0000 7f4d106be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.2 2024-05-30T16:23:02.057 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.055+0000 7f4d106be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.3 2024-05-30T16:23:02.057 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.055+0000 7f4d106be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.3 2024-05-30T16:23:02.057 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.055+0000 7f4d106be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.4 2024-05-30T16:23:02.057 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.055+0000 7f4d106be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.4 2024-05-30T16:23:02.058 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.055+0000 7f4d106be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.5 2024-05-30T16:23:02.058 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.055+0000 7f4d106be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.5 2024-05-30T16:23:02.058 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.055+0000 7f4d106be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.6 2024-05-30T16:23:02.058 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.055+0000 7f4d106be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.6 2024-05-30T16:23:02.058 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.055+0000 7f4d106be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.7 2024-05-30T16:23:02.058 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.055+0000 7f4d106be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.7 2024-05-30T16:23:02.058 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.055+0000 7f4d106be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.8 2024-05-30T16:23:02.058 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.055+0000 7f4d106be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.8 2024-05-30T16:23:02.058 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.055+0000 7f4d106be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.9 2024-05-30T16:23:02.058 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.055+0000 7f4d106be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.9 2024-05-30T16:23:02.058 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.055+0000 7f4d106be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.10 2024-05-30T16:23:02.058 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.055+0000 7f4d106be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.10 2024-05-30T16:23:02.059 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.055+0000 7f4d106be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.11 2024-05-30T16:23:02.059 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.055+0000 7f4d106be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.11 2024-05-30T16:23:02.059 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.055+0000 7f4d106be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.12 2024-05-30T16:23:02.059 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.055+0000 7f4d106be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.12 2024-05-30T16:23:02.059 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.055+0000 7f4d106be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.0 2024-05-30T16:23:02.059 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.055+0000 7f4d106be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.0 2024-05-30T16:23:02.059 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.059+0000 7f4d106be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.1 2024-05-30T16:23:02.059 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.059+0000 7f4d106be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.1 2024-05-30T16:23:02.060 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.059+0000 7f4d106be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.2 2024-05-30T16:23:02.060 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.059+0000 7f4d106be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.2 2024-05-30T16:23:02.061 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.059+0000 7f4d106be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.3 2024-05-30T16:23:02.061 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.059+0000 7f4d106be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.3 2024-05-30T16:23:02.063 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.063+0000 7f4d106be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.4 2024-05-30T16:23:02.063 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.063+0000 7f4d106be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.4 2024-05-30T16:23:02.064 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.063+0000 7f4d106be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.5 2024-05-30T16:23:02.064 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.063+0000 7f4d106be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.5 2024-05-30T16:23:02.066 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.067+0000 7f4d106be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.6 2024-05-30T16:23:02.066 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.067+0000 7f4d106be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.6 2024-05-30T16:23:02.067 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.067+0000 7f4d106be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.7 2024-05-30T16:23:02.068 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.067+0000 7f4d106be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.7 2024-05-30T16:23:02.069 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.067+0000 7f4d106be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.8 2024-05-30T16:23:02.069 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.067+0000 7f4d106be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.8 2024-05-30T16:23:02.070 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.071+0000 7f4d106be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.9 2024-05-30T16:23:02.070 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.071+0000 7f4d106be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.9 2024-05-30T16:23:02.071 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.071+0000 7f4d106be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.10 2024-05-30T16:23:02.071 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.071+0000 7f4d106be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.10 2024-05-30T16:23:02.072 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.071+0000 7f4d106be980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.11 2024-05-30T16:23:02.073 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.071+0000 7f4d106be980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.1.11 2024-05-30T16:23:02.074 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.075+0000 7f4d106be980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 : hit (requested=0x17, cached=0x17) 2024-05-30T16:23:02.074 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.075+0000 7f4d106be980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 2024-05-30T16:23:02.078 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.079+0000 7f4d106be980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 info.flags=0x17 2024-05-30T16:23:02.078 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.079+0000 7f4d106be980 10 moving default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 to cache LRU end 2024-05-30T16:23:02.078 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.079+0000 7f4d106be980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:23:02.078 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.079+0000 7f4d106be980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19, ofs0, ns] 2024-05-30T16:23:02.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.079+0000 7f4bf1ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431593 cookie 94323143594016 notifier 7212 bl.length()=865 2024-05-30T16:23:02.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.079+0000 7f4bf1ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 info.flags=0x17 2024-05-30T16:23:02.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.079+0000 7f4bf1ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 to cache LRU end 2024-05-30T16:23:02.079 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.079+0000 7f4bf1ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:23:02.084 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.083+0000 7f4d106be980 20 handle_bi_update(): bucket=:myver[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19]): orig_sources= new_sources= 2024-05-30T16:23:02.084 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.083+0000 7f4d106be980 20 handle_bi_update(): bucket=:myver[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19]): potential sources added=[] removed=[] 2024-05-30T16:23:02.084 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.083+0000 7f4d106be980 20 handle_bi_update(): bucket=:myver[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19]): orig_dests= new_dests= 2024-05-30T16:23:02.084 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.083+0000 7f4d106be980 20 handle_bi_update(): bucket=:myver[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19]): potential dests added=[] removed=[] 2024-05-30T16:23:02.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.091+0000 7f4d106be980 1 execute INFO: reshard of bucket "myver" completed successfully 2024-05-30T16:23:02.095 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.095+0000 7f4bc4fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:23:02.101 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.103+0000 7f4d106be980 20 remove_watcher() i=0 2024-05-30T16:23:02.101 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.103+0000 7f4d106be980 2 removed watcher, disabling cache 2024-05-30T16:23:02.105 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.103+0000 7f4d106be980 20 remove_watcher() i=1 2024-05-30T16:23:02.108 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.107+0000 7f4d106be980 20 remove_watcher() i=2 2024-05-30T16:23:02.110 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.111+0000 7f4d106be980 20 remove_watcher() i=3 2024-05-30T16:23:02.113 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.111+0000 7f4d106be980 20 remove_watcher() i=4 2024-05-30T16:23:02.115 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.115+0000 7f4d106be980 20 remove_watcher() i=5 2024-05-30T16:23:02.117 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.115+0000 7f4d106be980 20 remove_watcher() i=6 2024-05-30T16:23:02.120 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:02.119+0000 7f4d106be980 20 remove_watcher() i=7 2024-05-30T16:23:02.700 INFO:tasks.workunit.client.0.smithi006.stderr:put object successful 2024-05-30T16:23:02.700 INFO:tasks.workunit.client.0.smithi006.stderr:TEST: check that bucket stats are correct after reshard with unlinked entries 2024-05-30T16:23:02.700 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-05-30T16:23:03.640 INFO:tasks.workunit.client.0.smithi006.stderr:ceph config set client rgw_debug_inject_set_olh_err 2 2024-05-30T16:23:04.080 INFO:tasks.workunit.client.0.smithi006.stderr:ceph config set client rgw_debug_inject_olh_cancel_modification_err true 2024-05-30T16:23:05.667 INFO:tasks.workunit.client.0.smithi006.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-05-30T16:23:05.667 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bi list --bucket myver --object x 2024-05-30T16:23:05.703 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:23:05.703 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:23:05.749 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.747+0000 7f808408b980 20 rados->read ofs=0 len=0 2024-05-30T16:23:05.752 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.751+0000 7f808408b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:05.752 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.751+0000 7f808408b980 20 realm 2024-05-30T16:23:05.752 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.751+0000 7f808408b980 20 rados->read ofs=0 len=0 2024-05-30T16:23:05.752 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.751+0000 7f808408b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:05.752 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.751+0000 7f808408b980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:23:05.752 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.751+0000 7f808408b980 20 rados->read ofs=0 len=0 2024-05-30T16:23:05.752 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.751+0000 7f808408b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:05.752 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.751+0000 7f808408b980 20 rados->read ofs=0 len=0 2024-05-30T16:23:05.753 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.751+0000 7f808408b980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:23:05.753 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.751+0000 7f808408b980 20 rados->read ofs=0 len=0 2024-05-30T16:23:05.754 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.755+0000 7f808408b980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:23:05.754 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.755+0000 7f808408b980 20 searching for the correct realm 2024-05-30T16:23:05.760 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.759+0000 7f808139b640 0 -- 172.21.15.6:0/4141217626 >> [v2:172.21.15.6:6819/1141406589,v1:172.21.15.6:6821/1141406589] conn(0x561fa407a370 msgr2=0x561fa407c760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:23:05.766 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.767+0000 7f8081b9c640 0 -- 172.21.15.6:0/4141217626 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f80780fc290 msgr2=0x7f80781201e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:23:05.775 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.775+0000 7f808408b980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:23:05.776 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.775+0000 7f808408b980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:23:05.776 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.775+0000 7f808408b980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:23:05.776 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.775+0000 7f808408b980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:23:05.776 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.775+0000 7f808408b980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:23:05.776 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.775+0000 7f808408b980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:23:05.776 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.775+0000 7f808408b980 20 rados->read ofs=0 len=0 2024-05-30T16:23:05.776 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.775+0000 7f808408b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:05.776 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.775+0000 7f808408b980 20 rados->read ofs=0 len=0 2024-05-30T16:23:05.776 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.775+0000 7f808408b980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:23:05.777 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.775+0000 7f808408b980 20 rados->read ofs=0 len=0 2024-05-30T16:23:05.777 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.775+0000 7f808408b980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:23:05.777 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.775+0000 7f808408b980 20 zone default found 2024-05-30T16:23:05.777 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.775+0000 7f808408b980 4 Realm: () 2024-05-30T16:23:05.778 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.775+0000 7f808408b980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:23:05.778 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.775+0000 7f808408b980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:23:05.778 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.775+0000 7f808408b980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:23:05.778 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.775+0000 7f808408b980 20 zonegroup default 2024-05-30T16:23:05.778 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.775+0000 7f808408b980 20 rados->read ofs=0 len=0 2024-05-30T16:23:05.778 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.775+0000 7f808408b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:05.778 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.775+0000 7f808408b980 20 rados->read ofs=0 len=0 2024-05-30T16:23:05.779 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.779+0000 7f808408b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:05.780 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.779+0000 7f808408b980 20 rados->read ofs=0 len=0 2024-05-30T16:23:05.780 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.779+0000 7f808408b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:05.780 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.779+0000 7f808408b980 20 started sync module instance, tier type = 2024-05-30T16:23:05.780 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.779+0000 7f808408b980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:23:05.795 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.795+0000 7f808408b980 20 add_watcher() i=0 2024-05-30T16:23:05.795 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.795+0000 7f808408b980 20 add_watcher() i=1 2024-05-30T16:23:05.795 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.795+0000 7f808408b980 20 add_watcher() i=2 2024-05-30T16:23:05.795 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.795+0000 7f808408b980 20 add_watcher() i=3 2024-05-30T16:23:05.795 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.795+0000 7f808408b980 20 add_watcher() i=4 2024-05-30T16:23:05.795 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.795+0000 7f808408b980 20 add_watcher() i=5 2024-05-30T16:23:05.795 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.795+0000 7f808408b980 20 add_watcher() i=6 2024-05-30T16:23:05.796 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.795+0000 7f808408b980 20 add_watcher() i=7 2024-05-30T16:23:05.796 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.795+0000 7f808408b980 2 all 8 watchers are set, enabling cache 2024-05-30T16:23:05.798 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.799+0000 7f7f337fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:23:05.799 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.799+0000 7f808408b980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:23:05.800 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.799+0000 7f808408b980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:23:05.800 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.799+0000 7f808408b980 5 note: GC not initialized 2024-05-30T16:23:05.800 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.799+0000 7f7f327fc640 20 reqs_thread_entry: start 2024-05-30T16:23:05.800 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.799+0000 7f808408b980 20 init_complete bucket index max shards: 11 2024-05-30T16:23:05.800 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.799+0000 7f808408b980 20 Filter name: none 2024-05-30T16:23:05.800 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.799+0000 7f7ea3fff640 20 reqs_thread_entry: start 2024-05-30T16:23:05.800 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.799+0000 7f808408b980 20 rados->read ofs=0 len=0 2024-05-30T16:23:05.801 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.799+0000 7f808408b980 20 rados_obj.operate() r=0 bl.length=197 2024-05-30T16:23:05.801 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.799+0000 7f808408b980 20 rados->read ofs=0 len=0 2024-05-30T16:23:05.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.799+0000 7f808408b980 20 rados_obj.operate() r=0 bl.length=411 2024-05-30T16:23:05.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.799+0000 7f808408b980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:23:05.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.799+0000 7f808408b980 20 INFO: main: max_entries=1000, index=gen=2, max_shards=13 2024-05-30T16:23:05.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.799+0000 7f808408b980 20 INFO: main: starting shard=0 2024-05-30T16:23:05.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.799+0000 7f808408b980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.0 2024-05-30T16:23:05.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.803+0000 7f808408b980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.0 2024-05-30T16:23:05.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.803+0000 7f808408b980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:05.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.803+0000 7f808408b980 20 INFO: main: starting shard=1 2024-05-30T16:23:05.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.803+0000 7f808408b980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.1 2024-05-30T16:23:05.803 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.803+0000 7f808408b980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.1 2024-05-30T16:23:05.804 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.803+0000 7f808408b980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:05.804 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.803+0000 7f808408b980 20 INFO: main: starting shard=2 2024-05-30T16:23:05.804 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.803+0000 7f808408b980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.2 2024-05-30T16:23:05.804 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.803+0000 7f808408b980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.2 2024-05-30T16:23:05.804 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.803+0000 7f808408b980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:05.804 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.803+0000 7f808408b980 20 INFO: main: starting shard=3 2024-05-30T16:23:05.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.803+0000 7f808408b980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.3 2024-05-30T16:23:05.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.803+0000 7f808408b980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.3 2024-05-30T16:23:05.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.807+0000 7f808408b980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:05.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.807+0000 7f808408b980 20 INFO: main: starting shard=4 2024-05-30T16:23:05.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.807+0000 7f808408b980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.4 2024-05-30T16:23:05.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.807+0000 7f808408b980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.4 2024-05-30T16:23:05.807 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.807+0000 7f808408b980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:05.807 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.807+0000 7f808408b980 20 INFO: main: starting shard=5 2024-05-30T16:23:05.807 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.807+0000 7f808408b980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.5 2024-05-30T16:23:05.807 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.807+0000 7f808408b980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.5 2024-05-30T16:23:05.808 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.807+0000 7f808408b980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-05-30T16:23:05.808 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.807+0000 7f808408b980 20 INFO: main: starting shard=6 2024-05-30T16:23:05.808 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.807+0000 7f808408b980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.6 2024-05-30T16:23:05.808 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.807+0000 7f808408b980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.6 2024-05-30T16:23:05.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.807+0000 7f808408b980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:05.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.807+0000 7f808408b980 20 INFO: main: starting shard=7 2024-05-30T16:23:05.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.807+0000 7f808408b980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.7 2024-05-30T16:23:05.809 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.807+0000 7f808408b980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.7 2024-05-30T16:23:05.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.811+0000 7f808408b980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:05.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.811+0000 7f808408b980 20 INFO: main: starting shard=8 2024-05-30T16:23:05.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.811+0000 7f808408b980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.8 2024-05-30T16:23:05.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.811+0000 7f808408b980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.8 2024-05-30T16:23:05.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.811+0000 7f808408b980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:05.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.811+0000 7f808408b980 20 INFO: main: starting shard=9 2024-05-30T16:23:05.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.811+0000 7f808408b980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.9 2024-05-30T16:23:05.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.811+0000 7f808408b980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.9 2024-05-30T16:23:05.812 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.811+0000 7f808408b980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:05.812 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.811+0000 7f808408b980 20 INFO: main: starting shard=10 2024-05-30T16:23:05.812 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.811+0000 7f808408b980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.10 2024-05-30T16:23:05.812 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.811+0000 7f808408b980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.10 2024-05-30T16:23:05.813 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.811+0000 7f808408b980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:05.813 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.811+0000 7f808408b980 20 INFO: main: starting shard=11 2024-05-30T16:23:05.813 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.811+0000 7f808408b980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.11 2024-05-30T16:23:05.813 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.811+0000 7f808408b980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.11 2024-05-30T16:23:05.814 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.815+0000 7f808408b980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:05.814 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.815+0000 7f808408b980 20 INFO: main: starting shard=12 2024-05-30T16:23:05.814 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.815+0000 7f808408b980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.12 2024-05-30T16:23:05.814 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.815+0000 7f808408b980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.12 2024-05-30T16:23:05.815 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.815+0000 7f808408b980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:05.815 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.815+0000 7f808408b980 20 INFO: main: done 2024-05-30T16:23:05.815 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.815+0000 7f7f337fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:23:05.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.819+0000 7f808408b980 20 remove_watcher() i=0 2024-05-30T16:23:05.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.819+0000 7f808408b980 2 removed watcher, disabling cache 2024-05-30T16:23:05.823 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.823+0000 7f808408b980 20 remove_watcher() i=1 2024-05-30T16:23:05.825 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.823+0000 7f808408b980 20 remove_watcher() i=2 2024-05-30T16:23:05.827 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.827+0000 7f808408b980 20 remove_watcher() i=3 2024-05-30T16:23:05.829 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.827+0000 7f808408b980 20 remove_watcher() i=4 2024-05-30T16:23:05.831 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.831+0000 7f808408b980 20 remove_watcher() i=5 2024-05-30T16:23:05.832 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.831+0000 7f808408b980 20 remove_watcher() i=6 2024-05-30T16:23:05.833 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:05.835+0000 7f808408b980 20 remove_watcher() i=7 2024-05-30T16:23:05.979 INFO:tasks.workunit.client.0.smithi006.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-05-30T16:23:05.979 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bi list --bucket myver --object y 2024-05-30T16:23:06.017 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:23:06.017 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:23:06.063 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.063+0000 7f59af613980 20 rados->read ofs=0 len=0 2024-05-30T16:23:06.065 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.063+0000 7f59af613980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:06.065 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.063+0000 7f59af613980 20 realm 2024-05-30T16:23:06.065 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.063+0000 7f59af613980 20 rados->read ofs=0 len=0 2024-05-30T16:23:06.066 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.067+0000 7f59af613980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:06.066 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.067+0000 7f59af613980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:23:06.066 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.067+0000 7f59af613980 20 rados->read ofs=0 len=0 2024-05-30T16:23:06.066 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.067+0000 7f59af613980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:06.066 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.067+0000 7f59af613980 20 rados->read ofs=0 len=0 2024-05-30T16:23:06.067 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.067+0000 7f59af613980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:23:06.067 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.067+0000 7f59af613980 20 rados->read ofs=0 len=0 2024-05-30T16:23:06.068 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.067+0000 7f59af613980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:23:06.068 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.067+0000 7f59af613980 20 searching for the correct realm 2024-05-30T16:23:06.074 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.075+0000 7f59ac923640 0 -- 172.21.15.6:0/2982556038 >> [v2:172.21.15.6:6819/1141406589,v1:172.21.15.6:6821/1141406589] conn(0x55869702f5e0 msgr2=0x5586970319f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:23:06.080 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.079+0000 7f59ad124640 0 -- 172.21.15.6:0/2982556038 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f5998103790 msgr2=0x7f59980e01b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:23:06.088 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.087+0000 7f59af613980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:23:06.089 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.087+0000 7f59af613980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:23:06.089 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.087+0000 7f59af613980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:23:06.089 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.087+0000 7f59af613980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:23:06.089 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.087+0000 7f59af613980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:23:06.089 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.087+0000 7f59af613980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:23:06.089 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.087+0000 7f59af613980 20 rados->read ofs=0 len=0 2024-05-30T16:23:06.089 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.087+0000 7f59af613980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:06.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.087+0000 7f59af613980 20 rados->read ofs=0 len=0 2024-05-30T16:23:06.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.091+0000 7f59af613980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:23:06.090 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.091+0000 7f59af613980 20 rados->read ofs=0 len=0 2024-05-30T16:23:06.091 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.091+0000 7f59af613980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:23:06.091 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.091+0000 7f59af613980 20 zone default found 2024-05-30T16:23:06.091 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.091+0000 7f59af613980 4 Realm: () 2024-05-30T16:23:06.091 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.091+0000 7f59af613980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:23:06.091 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.091+0000 7f59af613980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:23:06.091 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.091+0000 7f59af613980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:23:06.091 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.091+0000 7f59af613980 20 zonegroup default 2024-05-30T16:23:06.091 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.091+0000 7f59af613980 20 rados->read ofs=0 len=0 2024-05-30T16:23:06.091 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.091+0000 7f59af613980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:06.091 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.091+0000 7f59af613980 20 rados->read ofs=0 len=0 2024-05-30T16:23:06.092 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.091+0000 7f59af613980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:06.092 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.091+0000 7f59af613980 20 rados->read ofs=0 len=0 2024-05-30T16:23:06.092 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.091+0000 7f59af613980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:06.092 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.091+0000 7f59af613980 20 started sync module instance, tier type = 2024-05-30T16:23:06.092 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.091+0000 7f59af613980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:23:06.109 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.107+0000 7f59af613980 20 add_watcher() i=0 2024-05-30T16:23:06.109 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.107+0000 7f59af613980 20 add_watcher() i=1 2024-05-30T16:23:06.109 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.107+0000 7f59af613980 20 add_watcher() i=2 2024-05-30T16:23:06.109 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.107+0000 7f59af613980 20 add_watcher() i=3 2024-05-30T16:23:06.109 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.107+0000 7f59af613980 20 add_watcher() i=4 2024-05-30T16:23:06.109 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.107+0000 7f59af613980 20 add_watcher() i=5 2024-05-30T16:23:06.109 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.107+0000 7f59af613980 20 add_watcher() i=6 2024-05-30T16:23:06.110 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.111+0000 7f59af613980 20 add_watcher() i=7 2024-05-30T16:23:06.110 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.111+0000 7f59af613980 2 all 8 watchers are set, enabling cache 2024-05-30T16:23:06.113 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.111+0000 7f58637e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:23:06.114 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.111+0000 7f59af613980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:23:06.114 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.111+0000 7f59af613980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:23:06.114 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.111+0000 7f59af613980 5 note: GC not initialized 2024-05-30T16:23:06.114 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.111+0000 7f58627e4640 20 reqs_thread_entry: start 2024-05-30T16:23:06.114 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.115+0000 7f59af613980 20 init_complete bucket index max shards: 11 2024-05-30T16:23:06.114 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.115+0000 7f59af613980 20 Filter name: none 2024-05-30T16:23:06.114 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.115+0000 7f57d3fff640 20 reqs_thread_entry: start 2024-05-30T16:23:06.115 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.115+0000 7f59af613980 20 rados->read ofs=0 len=0 2024-05-30T16:23:06.115 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.115+0000 7f59af613980 20 rados_obj.operate() r=0 bl.length=197 2024-05-30T16:23:06.115 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.115+0000 7f59af613980 20 rados->read ofs=0 len=0 2024-05-30T16:23:06.116 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.115+0000 7f59af613980 20 rados_obj.operate() r=0 bl.length=411 2024-05-30T16:23:06.116 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.115+0000 7f59af613980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:23:06.116 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.115+0000 7f59af613980 20 INFO: main: max_entries=1000, index=gen=2, max_shards=13 2024-05-30T16:23:06.116 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.115+0000 7f59af613980 20 INFO: main: starting shard=0 2024-05-30T16:23:06.116 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.115+0000 7f59af613980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.0 2024-05-30T16:23:06.116 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.115+0000 7f59af613980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.0 2024-05-30T16:23:06.116 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.115+0000 7f59af613980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:06.117 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.115+0000 7f59af613980 20 INFO: main: starting shard=1 2024-05-30T16:23:06.117 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.115+0000 7f59af613980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.1 2024-05-30T16:23:06.117 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.115+0000 7f59af613980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.1 2024-05-30T16:23:06.117 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.115+0000 7f59af613980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:06.117 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.115+0000 7f59af613980 20 INFO: main: starting shard=2 2024-05-30T16:23:06.117 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.115+0000 7f59af613980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.2 2024-05-30T16:23:06.117 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.115+0000 7f59af613980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.2 2024-05-30T16:23:06.117 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.119+0000 7f59af613980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:06.118 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.119+0000 7f59af613980 20 INFO: main: starting shard=3 2024-05-30T16:23:06.118 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.119+0000 7f59af613980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.3 2024-05-30T16:23:06.118 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.119+0000 7f59af613980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.3 2024-05-30T16:23:06.119 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.119+0000 7f59af613980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:06.119 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.119+0000 7f59af613980 20 INFO: main: starting shard=4 2024-05-30T16:23:06.119 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.119+0000 7f59af613980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.4 2024-05-30T16:23:06.119 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.119+0000 7f59af613980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.4 2024-05-30T16:23:06.120 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.119+0000 7f59af613980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:06.120 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.119+0000 7f59af613980 20 INFO: main: starting shard=5 2024-05-30T16:23:06.120 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.119+0000 7f59af613980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.5 2024-05-30T16:23:06.120 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.119+0000 7f59af613980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.5 2024-05-30T16:23:06.121 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.119+0000 7f59af613980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:06.121 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.119+0000 7f59af613980 20 INFO: main: starting shard=6 2024-05-30T16:23:06.121 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.119+0000 7f59af613980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.6 2024-05-30T16:23:06.121 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.119+0000 7f59af613980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.6 2024-05-30T16:23:06.122 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.123+0000 7f59af613980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:06.122 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.123+0000 7f59af613980 20 INFO: main: starting shard=7 2024-05-30T16:23:06.122 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.123+0000 7f59af613980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.7 2024-05-30T16:23:06.122 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.123+0000 7f59af613980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.7 2024-05-30T16:23:06.123 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.123+0000 7f59af613980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:06.123 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.123+0000 7f59af613980 20 INFO: main: starting shard=8 2024-05-30T16:23:06.123 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.123+0000 7f59af613980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.8 2024-05-30T16:23:06.123 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.123+0000 7f59af613980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.8 2024-05-30T16:23:06.124 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.123+0000 7f59af613980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:06.124 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.123+0000 7f59af613980 20 INFO: main: starting shard=9 2024-05-30T16:23:06.124 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.123+0000 7f59af613980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.9 2024-05-30T16:23:06.124 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.123+0000 7f59af613980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.9 2024-05-30T16:23:06.125 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.127+0000 7f59af613980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-05-30T16:23:06.126 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.127+0000 7f59af613980 20 INFO: main: starting shard=10 2024-05-30T16:23:06.126 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.127+0000 7f59af613980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.10 2024-05-30T16:23:06.126 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.127+0000 7f59af613980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.10 2024-05-30T16:23:06.127 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.127+0000 7f59af613980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:06.127 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.127+0000 7f59af613980 20 INFO: main: starting shard=11 2024-05-30T16:23:06.127 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.127+0000 7f59af613980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.11 2024-05-30T16:23:06.127 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.127+0000 7f59af613980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.11 2024-05-30T16:23:06.127 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.127+0000 7f59af613980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:06.127 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.127+0000 7f59af613980 20 INFO: main: starting shard=12 2024-05-30T16:23:06.127 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.127+0000 7f59af613980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.12 2024-05-30T16:23:06.128 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.127+0000 7f59af613980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.12 2024-05-30T16:23:06.128 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.127+0000 7f59af613980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:06.128 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.127+0000 7f59af613980 20 INFO: main: done 2024-05-30T16:23:06.129 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.127+0000 7f58637e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:23:06.134 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.135+0000 7f59af613980 20 remove_watcher() i=0 2024-05-30T16:23:06.134 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.135+0000 7f59af613980 2 removed watcher, disabling cache 2024-05-30T16:23:06.136 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.135+0000 7f59af613980 20 remove_watcher() i=1 2024-05-30T16:23:06.139 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.139+0000 7f59af613980 20 remove_watcher() i=2 2024-05-30T16:23:06.141 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.139+0000 7f59af613980 20 remove_watcher() i=3 2024-05-30T16:23:06.143 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.143+0000 7f59af613980 20 remove_watcher() i=4 2024-05-30T16:23:06.144 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.143+0000 7f59af613980 20 remove_watcher() i=5 2024-05-30T16:23:06.146 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.147+0000 7f59af613980 20 remove_watcher() i=6 2024-05-30T16:23:06.147 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.147+0000 7f59af613980 20 remove_watcher() i=7 2024-05-30T16:23:06.291 INFO:tasks.workunit.client.0.smithi006.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-05-30T16:23:06.291 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bi list --bucket myver --object z 2024-05-30T16:23:06.327 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:23:06.327 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:23:06.374 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.375+0000 7fd9ef4c8980 20 rados->read ofs=0 len=0 2024-05-30T16:23:06.376 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.375+0000 7fd9ef4c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:06.376 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.375+0000 7fd9ef4c8980 20 realm 2024-05-30T16:23:06.376 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.375+0000 7fd9ef4c8980 20 rados->read ofs=0 len=0 2024-05-30T16:23:06.376 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.375+0000 7fd9ef4c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:06.376 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.375+0000 7fd9ef4c8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:23:06.376 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.375+0000 7fd9ef4c8980 20 rados->read ofs=0 len=0 2024-05-30T16:23:06.377 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.375+0000 7fd9ef4c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:06.377 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.375+0000 7fd9ef4c8980 20 rados->read ofs=0 len=0 2024-05-30T16:23:06.378 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.379+0000 7fd9ef4c8980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:23:06.378 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.379+0000 7fd9ef4c8980 20 rados->read ofs=0 len=0 2024-05-30T16:23:06.378 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.379+0000 7fd9ef4c8980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:23:06.379 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.379+0000 7fd9ef4c8980 20 searching for the correct realm 2024-05-30T16:23:06.382 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.383+0000 7fd9ec7d8640 0 -- 172.21.15.6:0/1567122574 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7fd9d4002350 msgr2=0x7fd9dc063af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:23:06.599 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.599+0000 7fd9ecfd9640 0 -- 172.21.15.6:0/1567122574 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7fd9e00de4f0 msgr2=0x7fd9e00fb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:23:06.817 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.815+0000 7fd9ef4c8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:23:06.817 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.815+0000 7fd9ef4c8980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:23:06.817 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.815+0000 7fd9ef4c8980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:23:06.817 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.815+0000 7fd9ef4c8980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:23:06.817 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.815+0000 7fd9ef4c8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:23:06.817 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.815+0000 7fd9ef4c8980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:23:06.817 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.815+0000 7fd9ef4c8980 20 rados->read ofs=0 len=0 2024-05-30T16:23:06.817 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.815+0000 7fd9ef4c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:06.818 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.815+0000 7fd9ef4c8980 20 rados->read ofs=0 len=0 2024-05-30T16:23:06.818 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.819+0000 7fd9ef4c8980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:23:06.818 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.819+0000 7fd9ef4c8980 20 rados->read ofs=0 len=0 2024-05-30T16:23:06.819 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.819+0000 7fd9ef4c8980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:23:06.819 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.819+0000 7fd9ef4c8980 20 zone default found 2024-05-30T16:23:06.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.819+0000 7fd9ef4c8980 4 Realm: () 2024-05-30T16:23:06.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.819+0000 7fd9ef4c8980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:23:06.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.819+0000 7fd9ef4c8980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:23:06.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.819+0000 7fd9ef4c8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:23:06.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.819+0000 7fd9ef4c8980 20 zonegroup default 2024-05-30T16:23:06.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.819+0000 7fd9ef4c8980 20 rados->read ofs=0 len=0 2024-05-30T16:23:06.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.819+0000 7fd9ef4c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:06.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.819+0000 7fd9ef4c8980 20 rados->read ofs=0 len=0 2024-05-30T16:23:06.821 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.819+0000 7fd9ef4c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:06.821 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.819+0000 7fd9ef4c8980 20 rados->read ofs=0 len=0 2024-05-30T16:23:06.822 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.823+0000 7fd9ef4c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:06.822 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.823+0000 7fd9ef4c8980 20 started sync module instance, tier type = 2024-05-30T16:23:06.822 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.823+0000 7fd9ef4c8980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:23:06.846 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.847+0000 7fd9ef4c8980 20 add_watcher() i=0 2024-05-30T16:23:06.846 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.847+0000 7fd9ef4c8980 20 add_watcher() i=1 2024-05-30T16:23:06.846 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.847+0000 7fd9ef4c8980 20 add_watcher() i=2 2024-05-30T16:23:06.847 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.847+0000 7fd9ef4c8980 20 add_watcher() i=3 2024-05-30T16:23:06.847 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.847+0000 7fd9ef4c8980 20 add_watcher() i=4 2024-05-30T16:23:06.847 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.847+0000 7fd9ef4c8980 20 add_watcher() i=5 2024-05-30T16:23:06.847 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.847+0000 7fd9ef4c8980 20 add_watcher() i=6 2024-05-30T16:23:06.849 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.847+0000 7fd9ef4c8980 20 add_watcher() i=7 2024-05-30T16:23:06.849 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.847+0000 7fd9ef4c8980 2 all 8 watchers are set, enabling cache 2024-05-30T16:23:06.853 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.851+0000 7fd8a37e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:23:06.854 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.855+0000 7fd9ef4c8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:23:06.854 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.855+0000 7fd9ef4c8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:23:06.854 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.855+0000 7fd9ef4c8980 5 note: GC not initialized 2024-05-30T16:23:06.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.855+0000 7fd8a27e4640 20 reqs_thread_entry: start 2024-05-30T16:23:06.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.855+0000 7fd9ef4c8980 20 init_complete bucket index max shards: 11 2024-05-30T16:23:06.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.855+0000 7fd9ef4c8980 20 Filter name: none 2024-05-30T16:23:06.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.855+0000 7fd89b7fe640 20 reqs_thread_entry: start 2024-05-30T16:23:06.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.855+0000 7fd9ef4c8980 20 rados->read ofs=0 len=0 2024-05-30T16:23:06.855 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.855+0000 7fd9ef4c8980 20 rados_obj.operate() r=0 bl.length=197 2024-05-30T16:23:06.856 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.855+0000 7fd9ef4c8980 20 rados->read ofs=0 len=0 2024-05-30T16:23:06.857 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.855+0000 7fd9ef4c8980 20 rados_obj.operate() r=0 bl.length=411 2024-05-30T16:23:06.857 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.855+0000 7fd9ef4c8980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:23:06.857 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.855+0000 7fd9ef4c8980 20 INFO: main: max_entries=1000, index=gen=2, max_shards=13 2024-05-30T16:23:06.857 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.855+0000 7fd9ef4c8980 20 INFO: main: starting shard=0 2024-05-30T16:23:06.857 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.855+0000 7fd9ef4c8980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.0 2024-05-30T16:23:06.857 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.855+0000 7fd9ef4c8980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.0 2024-05-30T16:23:06.859 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.859+0000 7fd9ef4c8980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-05-30T16:23:06.859 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.859+0000 7fd9ef4c8980 20 INFO: main: starting shard=1 2024-05-30T16:23:06.859 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.859+0000 7fd9ef4c8980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.1 2024-05-30T16:23:06.859 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.859+0000 7fd9ef4c8980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.1 2024-05-30T16:23:06.859 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.859+0000 7fd9ef4c8980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:06.859 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.859+0000 7fd9ef4c8980 20 INFO: main: starting shard=2 2024-05-30T16:23:06.859 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.859+0000 7fd9ef4c8980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.2 2024-05-30T16:23:06.860 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.859+0000 7fd9ef4c8980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.2 2024-05-30T16:23:06.860 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.859+0000 7fd9ef4c8980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:06.860 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.859+0000 7fd9ef4c8980 20 INFO: main: starting shard=3 2024-05-30T16:23:06.860 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.859+0000 7fd9ef4c8980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.3 2024-05-30T16:23:06.860 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.859+0000 7fd9ef4c8980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.3 2024-05-30T16:23:06.862 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.863+0000 7fd9ef4c8980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:06.862 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.863+0000 7fd9ef4c8980 20 INFO: main: starting shard=4 2024-05-30T16:23:06.862 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.863+0000 7fd9ef4c8980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.4 2024-05-30T16:23:06.862 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.863+0000 7fd9ef4c8980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.4 2024-05-30T16:23:06.863 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.863+0000 7fd9ef4c8980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:06.864 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.863+0000 7fd9ef4c8980 20 INFO: main: starting shard=5 2024-05-30T16:23:06.864 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.863+0000 7fd9ef4c8980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.5 2024-05-30T16:23:06.864 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.863+0000 7fd9ef4c8980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.5 2024-05-30T16:23:06.864 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.863+0000 7fd9ef4c8980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:06.864 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.863+0000 7fd9ef4c8980 20 INFO: main: starting shard=6 2024-05-30T16:23:06.864 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.863+0000 7fd9ef4c8980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.6 2024-05-30T16:23:06.865 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.863+0000 7fd9ef4c8980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.6 2024-05-30T16:23:06.866 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.867+0000 7fd9ef4c8980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:06.866 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.867+0000 7fd9ef4c8980 20 INFO: main: starting shard=7 2024-05-30T16:23:06.866 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.867+0000 7fd9ef4c8980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.7 2024-05-30T16:23:06.866 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.867+0000 7fd9ef4c8980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.7 2024-05-30T16:23:06.868 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.867+0000 7fd9ef4c8980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:06.868 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.867+0000 7fd9ef4c8980 20 INFO: main: starting shard=8 2024-05-30T16:23:06.868 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.867+0000 7fd9ef4c8980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.8 2024-05-30T16:23:06.868 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.867+0000 7fd9ef4c8980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.8 2024-05-30T16:23:06.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.867+0000 7fd9ef4c8980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:06.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.867+0000 7fd9ef4c8980 20 INFO: main: starting shard=9 2024-05-30T16:23:06.869 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.867+0000 7fd9ef4c8980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.9 2024-05-30T16:23:06.870 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.867+0000 7fd9ef4c8980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.9 2024-05-30T16:23:06.871 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.871+0000 7fd9ef4c8980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:06.871 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.871+0000 7fd9ef4c8980 20 INFO: main: starting shard=10 2024-05-30T16:23:06.871 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.871+0000 7fd9ef4c8980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.10 2024-05-30T16:23:06.871 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.871+0000 7fd9ef4c8980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.10 2024-05-30T16:23:06.873 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.871+0000 7fd9ef4c8980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:06.873 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.871+0000 7fd9ef4c8980 20 INFO: main: starting shard=11 2024-05-30T16:23:06.873 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.871+0000 7fd9ef4c8980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.11 2024-05-30T16:23:06.873 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.871+0000 7fd9ef4c8980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.11 2024-05-30T16:23:06.873 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.875+0000 7fd9ef4c8980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:06.874 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.875+0000 7fd9ef4c8980 20 INFO: main: starting shard=12 2024-05-30T16:23:06.874 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.875+0000 7fd9ef4c8980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.12 2024-05-30T16:23:06.874 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.875+0000 7fd9ef4c8980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.12 2024-05-30T16:23:06.874 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.875+0000 7fd9ef4c8980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-30T16:23:06.874 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.875+0000 7fd9ef4c8980 20 INFO: main: done 2024-05-30T16:23:06.875 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.875+0000 7fd8a37e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:23:06.880 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.879+0000 7fd9ef4c8980 20 remove_watcher() i=0 2024-05-30T16:23:06.881 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.879+0000 7fd9ef4c8980 2 removed watcher, disabling cache 2024-05-30T16:23:06.883 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.883+0000 7fd9ef4c8980 20 remove_watcher() i=1 2024-05-30T16:23:06.885 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.883+0000 7fd9ef4c8980 20 remove_watcher() i=2 2024-05-30T16:23:06.888 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.887+0000 7fd9ef4c8980 20 remove_watcher() i=3 2024-05-30T16:23:06.890 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.891+0000 7fd9ef4c8980 20 remove_watcher() i=4 2024-05-30T16:23:06.892 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.891+0000 7fd9ef4c8980 20 remove_watcher() i=5 2024-05-30T16:23:06.893 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.891+0000 7fd9ef4c8980 20 remove_watcher() i=6 2024-05-30T16:23:06.895 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:06.895+0000 7fd9ef4c8980 20 remove_watcher() i=7 2024-05-30T16:23:06.950 INFO:tasks.workunit.client.0.smithi006.stderr:ceph config rm client rgw_debug_inject_set_olh_err 2024-05-30T16:23:07.377 INFO:tasks.workunit.client.0.smithi006.stderr:ceph config rm client rgw_debug_inject_olh_cancel_modification_err 2024-05-30T16:23:07.806 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket reshard --bucket myver --num-shards 17 --yes-i-really-mean-it 2024-05-30T16:23:07.839 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:23:07.840 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:23:07.883 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:07.883+0000 7f049c2fb980 20 rados->read ofs=0 len=0 2024-05-30T16:23:07.885 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:07.883+0000 7f049c2fb980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:07.885 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:07.883+0000 7f049c2fb980 20 realm 2024-05-30T16:23:07.885 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:07.883+0000 7f049c2fb980 20 rados->read ofs=0 len=0 2024-05-30T16:23:07.885 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:07.883+0000 7f049c2fb980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:07.885 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:07.883+0000 7f049c2fb980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:23:07.886 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:07.883+0000 7f049c2fb980 20 rados->read ofs=0 len=0 2024-05-30T16:23:07.886 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:07.883+0000 7f049c2fb980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:07.886 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:07.883+0000 7f049c2fb980 20 rados->read ofs=0 len=0 2024-05-30T16:23:07.887 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:07.887+0000 7f049c2fb980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:23:07.887 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:07.887+0000 7f049c2fb980 20 rados->read ofs=0 len=0 2024-05-30T16:23:07.888 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:07.887+0000 7f049c2fb980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:23:07.888 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:07.887+0000 7f049c2fb980 20 searching for the correct realm 2024-05-30T16:23:07.894 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:07.895+0000 7f049a60d640 0 -- 172.21.15.6:0/357670294 >> [v2:172.21.15.204:6800/3687190393,v1:172.21.15.204:6801/3687190393] conn(0x7f0490008070 msgr2=0x7f049004a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:23:08.111 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.111+0000 7f0499e0c640 0 -- 172.21.15.6:0/357670294 >> [v2:172.21.15.204:6816/2677861390,v1:172.21.15.204:6817/2677861390] conn(0x559f1dd3e3e0 msgr2=0x559f1dd40810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:23:08.120 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.119+0000 7f049c2fb980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:23:08.121 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.119+0000 7f049c2fb980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:23:08.121 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.119+0000 7f049c2fb980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:23:08.121 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.119+0000 7f049c2fb980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:23:08.121 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.119+0000 7f049c2fb980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:23:08.121 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.119+0000 7f049c2fb980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:23:08.121 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.119+0000 7f049c2fb980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.122 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.119+0000 7f049c2fb980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:08.122 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.119+0000 7f049c2fb980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.123 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.123+0000 7f049c2fb980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:23:08.123 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.123+0000 7f049c2fb980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.124 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.123+0000 7f049c2fb980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:23:08.124 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.123+0000 7f049c2fb980 20 zone default found 2024-05-30T16:23:08.124 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.123+0000 7f049c2fb980 4 Realm: () 2024-05-30T16:23:08.124 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.123+0000 7f049c2fb980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:23:08.125 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.123+0000 7f049c2fb980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:23:08.125 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.123+0000 7f049c2fb980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:23:08.125 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.123+0000 7f049c2fb980 20 zonegroup default 2024-05-30T16:23:08.125 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.123+0000 7f049c2fb980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.125 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.123+0000 7f049c2fb980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:08.126 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.123+0000 7f049c2fb980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.126 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.123+0000 7f049c2fb980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:08.127 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.123+0000 7f049c2fb980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.127 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.123+0000 7f049c2fb980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:08.127 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.123+0000 7f049c2fb980 20 started sync module instance, tier type = 2024-05-30T16:23:08.127 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.123+0000 7f049c2fb980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:23:08.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.147+0000 7f049c2fb980 20 add_watcher() i=0 2024-05-30T16:23:08.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.147+0000 7f049c2fb980 20 add_watcher() i=1 2024-05-30T16:23:08.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.147+0000 7f049c2fb980 20 add_watcher() i=2 2024-05-30T16:23:08.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.147+0000 7f049c2fb980 20 add_watcher() i=3 2024-05-30T16:23:08.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.147+0000 7f049c2fb980 20 add_watcher() i=4 2024-05-30T16:23:08.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.147+0000 7f049c2fb980 20 add_watcher() i=5 2024-05-30T16:23:08.149 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.147+0000 7f049c2fb980 20 add_watcher() i=6 2024-05-30T16:23:08.151 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.151+0000 7f049c2fb980 20 add_watcher() i=7 2024-05-30T16:23:08.151 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.151+0000 7f049c2fb980 2 all 8 watchers are set, enabling cache 2024-05-30T16:23:08.155 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.155+0000 7f034b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:23:08.156 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.155+0000 7f049c2fb980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:23:08.156 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.155+0000 7f049c2fb980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:23:08.156 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.155+0000 7f049c2fb980 5 note: GC not initialized 2024-05-30T16:23:08.157 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.155+0000 7f034a7fc640 20 reqs_thread_entry: start 2024-05-30T16:23:08.157 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.155+0000 7f049c2fb980 20 init_complete bucket index max shards: 11 2024-05-30T16:23:08.157 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.155+0000 7f049c2fb980 20 Filter name: none 2024-05-30T16:23:08.157 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.155+0000 7f02bbfff640 20 reqs_thread_entry: start 2024-05-30T16:23:08.158 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.159+0000 7f049c2fb980 10 cache get: name=default.rgw.meta+root+myver : miss 2024-05-30T16:23:08.158 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.159+0000 7f049c2fb980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.158 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.159+0000 7f049c2fb980 20 rados_obj.operate() r=0 bl.length=197 2024-05-30T16:23:08.159 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.159+0000 7f049c2fb980 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2024-05-30T16:23:08.159 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.159+0000 7f049c2fb980 10 adding default.rgw.meta+root+myver to cache LRU end 2024-05-30T16:23:08.159 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.159+0000 7f049c2fb980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 : miss 2024-05-30T16:23:08.159 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.159+0000 7f049c2fb980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.160 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.159+0000 7f049c2fb980 20 rados_obj.operate() r=0 bl.length=411 2024-05-30T16:23:08.160 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.159+0000 7f049c2fb980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 info.flags=0x17 2024-05-30T16:23:08.160 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.159+0000 7f049c2fb980 10 adding default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 to cache LRU end 2024-05-30T16:23:08.160 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.159+0000 7f049c2fb980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:23:08.160 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.159+0000 7f049c2fb980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:23:08.160 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.159+0000 7f049c2fb980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 2024-05-30T16:23:08.174 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.171+0000 7f049c2fb980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 info.flags=0x17 2024-05-30T16:23:08.174 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.171+0000 7f049c2fb980 10 moving default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 to cache LRU end 2024-05-30T16:23:08.174 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.171+0000 7f049c2fb980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:23:08.175 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.171+0000 7f049c2fb980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19, ofs0, ns] 2024-05-30T16:23:08.175 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.171+0000 7f037cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431594 cookie 94141888620304 notifier 6917 bl.length()=897 2024-05-30T16:23:08.175 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.171+0000 7f037cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 info.flags=0x17 2024-05-30T16:23:08.175 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.171+0000 7f037cff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 to cache LRU end 2024-05-30T16:23:08.175 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.171+0000 7f037cff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:23:08.176 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.175+0000 7f049c2fb980 20 handle_bi_update(): bucket=:myver[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19]): orig_sources= new_sources= 2024-05-30T16:23:08.176 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.175+0000 7f049c2fb980 20 handle_bi_update(): bucket=:myver[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19]): potential sources added=[] removed=[] 2024-05-30T16:23:08.176 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.175+0000 7f049c2fb980 20 handle_bi_update(): bucket=:myver[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19]): orig_dests= new_dests= 2024-05-30T16:23:08.176 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.175+0000 7f049c2fb980 20 handle_bi_update(): bucket=:myver[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19]): potential dests added=[] removed=[] 2024-05-30T16:23:08.182 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.0 2024-05-30T16:23:08.182 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.0 2024-05-30T16:23:08.182 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.1 2024-05-30T16:23:08.183 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.1 2024-05-30T16:23:08.183 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.2 2024-05-30T16:23:08.183 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.2 2024-05-30T16:23:08.183 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.3 2024-05-30T16:23:08.183 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.3 2024-05-30T16:23:08.183 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.4 2024-05-30T16:23:08.183 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.4 2024-05-30T16:23:08.183 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.5 2024-05-30T16:23:08.183 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.5 2024-05-30T16:23:08.183 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.6 2024-05-30T16:23:08.183 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.6 2024-05-30T16:23:08.183 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.7 2024-05-30T16:23:08.183 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.7 2024-05-30T16:23:08.183 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.8 2024-05-30T16:23:08.184 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.8 2024-05-30T16:23:08.184 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.9 2024-05-30T16:23:08.184 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.9 2024-05-30T16:23:08.184 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.10 2024-05-30T16:23:08.184 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.10 2024-05-30T16:23:08.184 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.11 2024-05-30T16:23:08.184 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.11 2024-05-30T16:23:08.184 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.12 2024-05-30T16:23:08.184 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.12 2024-05-30T16:23:08.184 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.13 2024-05-30T16:23:08.184 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.13 2024-05-30T16:23:08.184 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.14 2024-05-30T16:23:08.184 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.14 2024-05-30T16:23:08.185 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.15 2024-05-30T16:23:08.185 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.15 2024-05-30T16:23:08.185 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.16 2024-05-30T16:23:08.185 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.16 2024-05-30T16:23:08.185 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.0 2024-05-30T16:23:08.185 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.0 2024-05-30T16:23:08.186 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.1 2024-05-30T16:23:08.186 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.1 2024-05-30T16:23:08.186 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.2 2024-05-30T16:23:08.186 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.2 2024-05-30T16:23:08.186 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.3 2024-05-30T16:23:08.186 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.183+0000 7f049c2fb980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.3 2024-05-30T16:23:08.186 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.187+0000 7f049c2fb980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.4 2024-05-30T16:23:08.186 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.187+0000 7f049c2fb980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.4 2024-05-30T16:23:08.187 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.187+0000 7f049c2fb980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.5 2024-05-30T16:23:08.187 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.187+0000 7f049c2fb980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.5 2024-05-30T16:23:08.188 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.187+0000 7f049c2fb980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.6 2024-05-30T16:23:08.188 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.187+0000 7f049c2fb980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.6 2024-05-30T16:23:08.189 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.187+0000 7f049c2fb980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.7 2024-05-30T16:23:08.189 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.187+0000 7f049c2fb980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.7 2024-05-30T16:23:08.190 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.191+0000 7f049c2fb980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.8 2024-05-30T16:23:08.190 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.191+0000 7f049c2fb980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.8 2024-05-30T16:23:08.192 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.191+0000 7f049c2fb980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.9 2024-05-30T16:23:08.192 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.191+0000 7f049c2fb980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.9 2024-05-30T16:23:08.193 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.191+0000 7f049c2fb980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.10 2024-05-30T16:23:08.193 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.191+0000 7f049c2fb980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.10 2024-05-30T16:23:08.194 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.195+0000 7f049c2fb980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.11 2024-05-30T16:23:08.194 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.195+0000 7f049c2fb980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.11 2024-05-30T16:23:08.195 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.195+0000 7f049c2fb980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.12 2024-05-30T16:23:08.195 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.195+0000 7f049c2fb980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.2.12 2024-05-30T16:23:08.201 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.199+0000 7f049c2fb980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 : hit (requested=0x17, cached=0x17) 2024-05-30T16:23:08.201 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.199+0000 7f049c2fb980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 2024-05-30T16:23:08.204 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.203+0000 7f049c2fb980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 info.flags=0x17 2024-05-30T16:23:08.204 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.203+0000 7f049c2fb980 10 moving default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 to cache LRU end 2024-05-30T16:23:08.204 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.203+0000 7f049c2fb980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:23:08.204 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.203+0000 7f049c2fb980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19, ofs0, ns] 2024-05-30T16:23:08.205 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.203+0000 7f037cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431595 cookie 94141888620304 notifier 6917 bl.length()=865 2024-05-30T16:23:08.205 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.203+0000 7f037cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 info.flags=0x17 2024-05-30T16:23:08.205 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.203+0000 7f037cff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 to cache LRU end 2024-05-30T16:23:08.205 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.203+0000 7f037cff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:23:08.208 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.207+0000 7f049c2fb980 20 handle_bi_update(): bucket=:myver[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19]): orig_sources= new_sources= 2024-05-30T16:23:08.208 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.207+0000 7f049c2fb980 20 handle_bi_update(): bucket=:myver[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19]): potential sources added=[] removed=[] 2024-05-30T16:23:08.208 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.207+0000 7f049c2fb980 20 handle_bi_update(): bucket=:myver[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19]): orig_dests= new_dests= 2024-05-30T16:23:08.208 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.207+0000 7f049c2fb980 20 handle_bi_update(): bucket=:myver[601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19]): potential dests added=[] removed=[] 2024-05-30T16:23:08.213 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.211+0000 7f049c2fb980 1 execute INFO: reshard of bucket "myver" completed successfully 2024-05-30T16:23:08.216 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.215+0000 7f034b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:23:08.221 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.219+0000 7f049c2fb980 20 remove_watcher() i=0 2024-05-30T16:23:08.221 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.219+0000 7f049c2fb980 2 removed watcher, disabling cache 2024-05-30T16:23:08.224 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.223+0000 7f049c2fb980 20 remove_watcher() i=1 2024-05-30T16:23:08.226 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.227+0000 7f049c2fb980 20 remove_watcher() i=2 2024-05-30T16:23:08.229 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.227+0000 7f049c2fb980 20 remove_watcher() i=3 2024-05-30T16:23:08.231 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.231+0000 7f049c2fb980 20 remove_watcher() i=4 2024-05-30T16:23:08.233 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.231+0000 7f049c2fb980 20 remove_watcher() i=5 2024-05-30T16:23:08.234 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.235+0000 7f049c2fb980 20 remove_watcher() i=6 2024-05-30T16:23:08.236 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.235+0000 7f049c2fb980 20 remove_watcher() i=7 2024-05-30T16:23:08.251 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket check unlinked --bucket myver --fix --min-age-hours 0 --rgw-olh-pending-timeout-sec 0 --dump-keys 2024-05-30T16:23:08.289 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:23:08.289 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:23:08.338 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.339+0000 7f59fcdad980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.339 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.339+0000 7f59fcdad980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:08.339 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.339+0000 7f59fcdad980 20 realm 2024-05-30T16:23:08.339 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.339+0000 7f59fcdad980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.339 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.339+0000 7f59fcdad980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:08.339 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.339+0000 7f59fcdad980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:23:08.339 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.339+0000 7f59fcdad980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.340 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.339+0000 7f59fcdad980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:08.340 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.339+0000 7f59fcdad980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.341 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.339+0000 7f59fcdad980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:23:08.341 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.339+0000 7f59fcdad980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.342 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.343+0000 7f59fcdad980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:23:08.342 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.343+0000 7f59fcdad980 20 searching for the correct realm 2024-05-30T16:23:08.345 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.343+0000 7f59fa8be640 0 -- 172.21.15.6:0/527494165 >> [v2:172.21.15.6:6816/2704410641,v1:172.21.15.6:6817/2704410641] conn(0x7f59e4003cc0 msgr2=0x7f59f4011450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:23:08.351 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.351+0000 7f59fb0bf640 0 -- 172.21.15.6:0/527494165 >> [v2:172.21.15.204:6823/3156311527,v1:172.21.15.204:6825/3156311527] conn(0x7f59f00aeb10 msgr2=0x7f59f00c4780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:23:08.361 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.359+0000 7f59fcdad980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:23:08.361 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.359+0000 7f59fcdad980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:23:08.361 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.359+0000 7f59fcdad980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:23:08.361 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.359+0000 7f59fcdad980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:23:08.361 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.359+0000 7f59fcdad980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:23:08.361 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.359+0000 7f59fcdad980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:23:08.361 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.359+0000 7f59fcdad980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.361 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.359+0000 7f59fcdad980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:08.361 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.359+0000 7f59fcdad980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.362 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.363+0000 7f59fcdad980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:23:08.362 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.363+0000 7f59fcdad980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.362 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.363+0000 7f59fcdad980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:23:08.363 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.363+0000 7f59fcdad980 20 zone default found 2024-05-30T16:23:08.363 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.363+0000 7f59fcdad980 4 Realm: () 2024-05-30T16:23:08.363 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.363+0000 7f59fcdad980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:23:08.363 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.363+0000 7f59fcdad980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:23:08.363 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.363+0000 7f59fcdad980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:23:08.363 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.363+0000 7f59fcdad980 20 zonegroup default 2024-05-30T16:23:08.363 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.363+0000 7f59fcdad980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.363 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.363+0000 7f59fcdad980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:08.363 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.363+0000 7f59fcdad980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.364 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.363+0000 7f59fcdad980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:08.364 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.363+0000 7f59fcdad980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.364 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.363+0000 7f59fcdad980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:08.364 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.363+0000 7f59fcdad980 20 started sync module instance, tier type = 2024-05-30T16:23:08.364 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.363+0000 7f59fcdad980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:23:08.378 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.379+0000 7f59fcdad980 20 add_watcher() i=0 2024-05-30T16:23:08.379 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.379+0000 7f59fcdad980 20 add_watcher() i=1 2024-05-30T16:23:08.379 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.379+0000 7f59fcdad980 20 add_watcher() i=2 2024-05-30T16:23:08.379 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.379+0000 7f59fcdad980 20 add_watcher() i=3 2024-05-30T16:23:08.379 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.379+0000 7f59fcdad980 20 add_watcher() i=4 2024-05-30T16:23:08.379 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.379+0000 7f59fcdad980 20 add_watcher() i=5 2024-05-30T16:23:08.379 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.379+0000 7f59fcdad980 20 add_watcher() i=6 2024-05-30T16:23:08.379 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.379+0000 7f59fcdad980 20 add_watcher() i=7 2024-05-30T16:23:08.380 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.379+0000 7f59fcdad980 2 all 8 watchers are set, enabling cache 2024-05-30T16:23:08.382 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.383+0000 7f58b1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:23:08.383 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.383+0000 7f59fcdad980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:23:08.383 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.383+0000 7f59fcdad980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:23:08.383 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.383+0000 7f59fcdad980 5 note: GC not initialized 2024-05-30T16:23:08.383 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.383+0000 7f58b0fe1640 20 reqs_thread_entry: start 2024-05-30T16:23:08.383 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.383+0000 7f59fcdad980 20 init_complete bucket index max shards: 11 2024-05-30T16:23:08.383 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.383+0000 7f59fcdad980 20 Filter name: none 2024-05-30T16:23:08.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.383+0000 7f58a9ffb640 20 reqs_thread_entry: start 2024-05-30T16:23:08.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.383+0000 7f59fcdad980 10 cache get: name=default.rgw.meta+root+myver : miss 2024-05-30T16:23:08.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.383+0000 7f59fcdad980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.383+0000 7f59fcdad980 20 rados_obj.operate() r=0 bl.length=197 2024-05-30T16:23:08.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.383+0000 7f59fcdad980 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2024-05-30T16:23:08.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.383+0000 7f59fcdad980 10 adding default.rgw.meta+root+myver to cache LRU end 2024-05-30T16:23:08.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.383+0000 7f59fcdad980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 : miss 2024-05-30T16:23:08.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.383+0000 7f59fcdad980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.385 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.383+0000 7f59fcdad980 20 rados_obj.operate() r=0 bl.length=411 2024-05-30T16:23:08.385 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.383+0000 7f59fcdad980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 info.flags=0x17 2024-05-30T16:23:08.385 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.383+0000 7f59fcdad980 10 adding default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 to cache LRU end 2024-05-30T16:23:08.385 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.383+0000 7f59fcdad980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:23:08.385 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.383+0000 7f59fcdad980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-05-30T16:23:08.385 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.383+0000 7f59fcdad980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19 2024-05-30T16:23:08.385 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.383+0000 7f59fcdad980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.0 2024-05-30T16:23:08.385 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.383+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.0 2024-05-30T16:23:08.386 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.387+0000 7f59fcdad980 1 NOTICE: finished shard 0 (0 entries removed) 2024-05-30T16:23:08.386 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.387+0000 7f59fcdad980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.1 2024-05-30T16:23:08.386 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.387+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.1 2024-05-30T16:23:08.387 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.387+0000 7f59fcdad980 1 NOTICE: finished shard 1 (0 entries removed) 2024-05-30T16:23:08.387 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.387+0000 7f59fcdad980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.2 2024-05-30T16:23:08.387 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.387+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.2 2024-05-30T16:23:08.387 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.387+0000 7f59fcdad980 1 NOTICE: finished shard 2 (0 entries removed) 2024-05-30T16:23:08.387 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.387+0000 7f59fcdad980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.3 2024-05-30T16:23:08.387 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.387+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.3 2024-05-30T16:23:08.388 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.387+0000 7f59fcdad980 1 NOTICE: finished shard 3 (0 entries removed) 2024-05-30T16:23:08.388 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.387+0000 7f59fcdad980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.4 2024-05-30T16:23:08.388 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.387+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.4 2024-05-30T16:23:08.388 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.387+0000 7f59fcdad980 1 NOTICE: finished shard 4 (0 entries removed) 2024-05-30T16:23:08.388 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.387+0000 7f59fcdad980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.5 2024-05-30T16:23:08.388 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.387+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.5 2024-05-30T16:23:08.389 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.387+0000 7f59fcdad980 1 NOTICE: finished shard 5 (0 entries removed) 2024-05-30T16:23:08.389 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.387+0000 7f59fcdad980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.6 2024-05-30T16:23:08.389 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.387+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.6 2024-05-30T16:23:08.390 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.391+0000 7f59fcdad980 1 NOTICE: finished shard 6 (0 entries removed) 2024-05-30T16:23:08.390 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.391+0000 7f59fcdad980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.7 2024-05-30T16:23:08.390 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.391+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.7 2024-05-30T16:23:08.391 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.391+0000 7f59fcdad980 1 NOTICE: finished shard 7 (0 entries removed) 2024-05-30T16:23:08.392 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.391+0000 7f59fcdad980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.8 2024-05-30T16:23:08.392 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.391+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.8 2024-05-30T16:23:08.393 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.391+0000 7f59fcdad980 1 NOTICE: finished shard 8 (0 entries removed) 2024-05-30T16:23:08.393 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.391+0000 7f59fcdad980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.9 2024-05-30T16:23:08.393 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.391+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.9 2024-05-30T16:23:08.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.395+0000 7f59fcdad980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.10 2024-05-30T16:23:08.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.395+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.10 2024-05-30T16:23:08.394 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.395+0000 7f59fcdad980 1 NOTICE: finished shard 10 (0 entries removed) 2024-05-30T16:23:08.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.395+0000 7f59fcdad980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.11 2024-05-30T16:23:08.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.395+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.11 2024-05-30T16:23:08.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.395+0000 7f59fcdad980 1 NOTICE: finished shard 11 (0 entries removed) 2024-05-30T16:23:08.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.395+0000 7f59fcdad980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.12 2024-05-30T16:23:08.395 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.395+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.12 2024-05-30T16:23:08.396 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.395+0000 7f59fcdad980 1 NOTICE: finished shard 12 (0 entries removed) 2024-05-30T16:23:08.396 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.395+0000 7f59fcdad980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.13 2024-05-30T16:23:08.396 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.395+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.13 2024-05-30T16:23:08.397 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.395+0000 7f59fcdad980 1 NOTICE: finished shard 13 (0 entries removed) 2024-05-30T16:23:08.397 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.395+0000 7f59fcdad980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.14 2024-05-30T16:23:08.397 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.395+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.14 2024-05-30T16:23:08.397 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.395+0000 7f59fcdad980 1 NOTICE: finished shard 14 (0 entries removed) 2024-05-30T16:23:08.397 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.395+0000 7f59fcdad980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.15 2024-05-30T16:23:08.397 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.395+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.15 2024-05-30T16:23:08.398 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.399+0000 7f59fcdad980 1 NOTICE: finished shard 15 (0 entries removed) 2024-05-30T16:23:08.398 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.399+0000 7f59fcdad980 10 bucket_obj is .dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.16 2024-05-30T16:23:08.398 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.399+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.16 2024-05-30T16:23:08.399 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.399+0000 7f59fcdad980 1 NOTICE: finished shard 16 (0 entries removed) 2024-05-30T16:23:08.399 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.399+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.9 2024-05-30T16:23:08.400 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.399+0000 7f59fcdad980 20 get_obj_state: octx=0x5628c3670a30 obj=myver:x state=0x5628c3923368 s->prefetch_data=0 2024-05-30T16:23:08.400 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.399+0000 7f59fcdad980 20 get_obj_state: setting s->obj_tag to or4p8xtfiqhx17zjn13r6sepaau40oxk 2024-05-30T16:23:08.400 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.399+0000 7f59fcdad980 20 get_obj_state_impl: setting s->olh_tag to fw3pit16hji4cfmguzkxq69041j4zo5u 2024-05-30T16:23:08.400 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.399+0000 7f59fcdad980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=fw3pit16hji4cfmguzkxq69041j4zo5u 2024-05-30T16:23:08.402 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.403+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.9 2024-05-30T16:23:08.405 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.403+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.9 2024-05-30T16:23:08.406 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.407+0000 7f59fcdad980 20 olh_log_entry: epoch=2 op=2 key=x[] 2024-05-30T16:23:08.406 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.407+0000 7f59fcdad980 20 olh_log_entry: epoch=2 op=3 key=x[up7GMMJshZRGaw.UI8yDGjKkimUmh.O] 2024-05-30T16:23:08.406 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.407+0000 7f59fcdad980 20 get_obj_state: octx=0x5628c3670a30 obj=myver:_:up7GMMJshZRGaw.UI8yDGjKkimUmh.O_x state=0x5628c394bee8 s->prefetch_data=0 2024-05-30T16:23:08.407 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.407+0000 7f59fcdad980 10 manifest: total_size = 9 2024-05-30T16:23:08.407 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.407+0000 7f59fcdad980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.132071969362746358 2024-05-30T16:23:08.407 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.407+0000 7f59fcdad980 20 get_obj_state: octx=0x5628c3670a30 obj=myver:_:up7GMMJshZRGaw.UI8yDGjKkimUmh.O_x state=0x5628c394bee8 s->prefetch_data=0 2024-05-30T16:23:08.407 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.407+0000 7f59fcdad980 20 prepare_atomic_modification: state is not atomic. state=0x5628c394bee8 2024-05-30T16:23:08.407 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.407+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.9 2024-05-30T16:23:08.414 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.415+0000 7f59fcdad980 20 get_obj_state: octx=0x5628c3670a30 obj=myver:x state=0x5628c3923368 s->prefetch_data=0 2024-05-30T16:23:08.414 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.415+0000 7f59fcdad980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=fw3pit16hji4cfmguzkxq69041j4zo5u 2024-05-30T16:23:08.414 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.415+0000 7f58ddffb640 20 handle_completion(): completion ok for obj=x[up7GMMJshZRGaw.UI8yDGjKkimUmh.O] 2024-05-30T16:23:08.418 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.419+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.9 2024-05-30T16:23:08.420 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.419+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.9 2024-05-30T16:23:08.421 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.423+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.9 2024-05-30T16:23:08.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.423+0000 7f59fcdad980 20 get_obj_state: octx=0x5628c3670a30 obj=myver:y state=0x5628c394bee8 s->prefetch_data=0 2024-05-30T16:23:08.423 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.423+0000 7f59fcdad980 20 get_obj_state: setting s->obj_tag to wb1d3u2747sjinfv7gzz24fwjlf065o9 2024-05-30T16:23:08.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.423+0000 7f59fcdad980 20 get_obj_state_impl: setting s->olh_tag to cydasaz23j71j5pysmn16l29k2ysx76m 2024-05-30T16:23:08.424 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.423+0000 7f59fcdad980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=cydasaz23j71j5pysmn16l29k2ysx76m 2024-05-30T16:23:08.427 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.427+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.9 2024-05-30T16:23:08.429 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.427+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.9 2024-05-30T16:23:08.430 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.431+0000 7f59fcdad980 20 olh_log_entry: epoch=2 op=2 key=y[] 2024-05-30T16:23:08.430 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.431+0000 7f59fcdad980 20 olh_log_entry: epoch=2 op=3 key=y[Sqq-yr0Sko.r6nH3olwdTtEVczBxBAp] 2024-05-30T16:23:08.430 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.431+0000 7f59fcdad980 20 get_obj_state: octx=0x5628c3670a30 obj=myver:_:Sqq-yr0Sko.r6nH3olwdTtEVczBxBAp_y state=0x5628c39206e8 s->prefetch_data=0 2024-05-30T16:23:08.431 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.431+0000 7f59fcdad980 10 manifest: total_size = 11 2024-05-30T16:23:08.431 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.431+0000 7f59fcdad980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.7403743549738220274 2024-05-30T16:23:08.431 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.431+0000 7f59fcdad980 20 get_obj_state: octx=0x5628c3670a30 obj=myver:_:Sqq-yr0Sko.r6nH3olwdTtEVczBxBAp_y state=0x5628c39206e8 s->prefetch_data=0 2024-05-30T16:23:08.431 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.431+0000 7f59fcdad980 20 prepare_atomic_modification: state is not atomic. state=0x5628c39206e8 2024-05-30T16:23:08.431 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.431+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.9 2024-05-30T16:23:08.439 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.439+0000 7f58ddffb640 20 handle_completion(): completion ok for obj=y[Sqq-yr0Sko.r6nH3olwdTtEVczBxBAp] 2024-05-30T16:23:08.440 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.439+0000 7f59fcdad980 20 get_obj_state: octx=0x5628c3670a30 obj=myver:y state=0x5628c394bee8 s->prefetch_data=0 2024-05-30T16:23:08.441 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.439+0000 7f59fcdad980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=cydasaz23j71j5pysmn16l29k2ysx76m 2024-05-30T16:23:08.448 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.447+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.9 2024-05-30T16:23:08.451 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.451+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.9 2024-05-30T16:23:08.453 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.451+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.9 2024-05-30T16:23:08.453 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.451+0000 7f59fcdad980 20 get_obj_state: octx=0x5628c3670a30 obj=myver:z state=0x5628c394bee8 s->prefetch_data=0 2024-05-30T16:23:08.455 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.455+0000 7f59fcdad980 20 get_obj_state: setting s->obj_tag to siey7x8gka2oo0naicbtg68q63vfu8np 2024-05-30T16:23:08.455 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.455+0000 7f59fcdad980 20 get_obj_state_impl: setting s->olh_tag to gv52cbgayeamcnai9ixa00haxl86jr6w 2024-05-30T16:23:08.455 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.455+0000 7f59fcdad980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=gv52cbgayeamcnai9ixa00haxl86jr6w 2024-05-30T16:23:08.457 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.459+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.9 2024-05-30T16:23:08.460 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.459+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.9 2024-05-30T16:23:08.461 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.459+0000 7f59fcdad980 20 olh_log_entry: epoch=2 op=2 key=z[] 2024-05-30T16:23:08.461 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.459+0000 7f59fcdad980 20 olh_log_entry: epoch=2 op=3 key=z[eo80pavGMr8Re67jKgdwIjtbdeQ6n3X] 2024-05-30T16:23:08.462 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.459+0000 7f59fcdad980 20 get_obj_state: octx=0x5628c3670a30 obj=myver:_:eo80pavGMr8Re67jKgdwIjtbdeQ6n3X_z state=0x5628c3953448 s->prefetch_data=0 2024-05-30T16:23:08.463 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.463+0000 7f59fcdad980 10 manifest: total_size = 22 2024-05-30T16:23:08.463 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.463+0000 7f59fcdad980 20 get_obj_state: setting s->obj_tag to 601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.6792364275514651810 2024-05-30T16:23:08.463 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.463+0000 7f59fcdad980 20 get_obj_state: octx=0x5628c3670a30 obj=myver:_:eo80pavGMr8Re67jKgdwIjtbdeQ6n3X_z state=0x5628c3953448 s->prefetch_data=0 2024-05-30T16:23:08.463 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.463+0000 7f59fcdad980 20 prepare_atomic_modification: state is not atomic. state=0x5628c3953448 2024-05-30T16:23:08.463 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.463+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.9 2024-05-30T16:23:08.471 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.471+0000 7f58de7fc640 20 handle_completion(): completion ok for obj=z[eo80pavGMr8Re67jKgdwIjtbdeQ6n3X] 2024-05-30T16:23:08.471 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.471+0000 7f59fcdad980 20 get_obj_state: octx=0x5628c3670a30 obj=myver:z state=0x5628c394bee8 s->prefetch_data=0 2024-05-30T16:23:08.471 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.471+0000 7f59fcdad980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=gv52cbgayeamcnai9ixa00haxl86jr6w 2024-05-30T16:23:08.478 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.479+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.9 2024-05-30T16:23:08.481 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.483+0000 7f59fcdad980 20 bucket index object: :.dir.601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.19.3.9 2024-05-30T16:23:08.482 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.483+0000 7f59fcdad980 1 NOTICE: finished shard 9 (3 entries removed) 2024-05-30T16:23:08.482 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.483+0000 7f59fcdad980 1 NOTICE: finished all shards (3 entries removed) 2024-05-30T16:23:08.483 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.483+0000 7f58b1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:23:08.487 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.487+0000 7f59fcdad980 20 remove_watcher() i=0 2024-05-30T16:23:08.488 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.487+0000 7f59fcdad980 2 removed watcher, disabling cache 2024-05-30T16:23:08.490 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.491+0000 7f59fcdad980 20 remove_watcher() i=1 2024-05-30T16:23:08.492 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.491+0000 7f59fcdad980 20 remove_watcher() i=2 2024-05-30T16:23:08.494 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.495+0000 7f59fcdad980 20 remove_watcher() i=3 2024-05-30T16:23:08.497 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.495+0000 7f59fcdad980 20 remove_watcher() i=4 2024-05-30T16:23:08.498 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.499+0000 7f59fcdad980 20 remove_watcher() i=5 2024-05-30T16:23:08.500 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.499+0000 7f59fcdad980 20 remove_watcher() i=6 2024-05-30T16:23:08.501 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.499+0000 7f59fcdad980 20 remove_watcher() i=7 2024-05-30T16:23:08.769 INFO:tasks.workunit.client.0.smithi006.stderr:radosgw-admin bucket stats --bucket myver 2024-05-30T16:23:08.803 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:23:08.803 INFO:tasks.workunit.client.0.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:23:08.849 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.847+0000 7f37e3656980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.851 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.851+0000 7f37e3656980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:08.851 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.851+0000 7f37e3656980 20 realm 2024-05-30T16:23:08.851 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.851+0000 7f37e3656980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.851 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.851+0000 7f37e3656980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:08.851 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.851+0000 7f37e3656980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:23:08.851 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.851+0000 7f37e3656980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.851 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.851+0000 7f37e3656980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:08.851 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.851+0000 7f37e3656980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.852 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.851+0000 7f37e3656980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:23:08.853 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.851+0000 7f37e3656980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.853 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.851+0000 7f37e3656980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:23:08.853 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.851+0000 7f37e3656980 20 searching for the correct realm 2024-05-30T16:23:08.859 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.859+0000 7f37e1968640 0 -- 172.21.15.6:0/1552471395 >> [v2:172.21.15.6:6808/1897362715,v1:172.21.15.6:6809/1897362715] conn(0x563f81b95380 msgr2=0x563f81b97840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:23:08.866 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.867+0000 7f37e1167640 0 -- 172.21.15.6:0/1552471395 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f37d809d2a0 msgr2=0x7f37d808d190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:23:08.873 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.871+0000 7f37e3656980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:23:08.873 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.871+0000 7f37e3656980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:23:08.873 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.871+0000 7f37e3656980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:23:08.873 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.871+0000 7f37e3656980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:23:08.874 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.871+0000 7f37e3656980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:23:08.874 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.871+0000 7f37e3656980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:23:08.874 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.871+0000 7f37e3656980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.874 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.871+0000 7f37e3656980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:08.874 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.871+0000 7f37e3656980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.874 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.875+0000 7f37e3656980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:23:08.874 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.875+0000 7f37e3656980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.875 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.875+0000 7f37e3656980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:23:08.876 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.875+0000 7f37e3656980 20 zone default found 2024-05-30T16:23:08.876 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.875+0000 7f37e3656980 4 Realm: () 2024-05-30T16:23:08.876 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.875+0000 7f37e3656980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:23:08.876 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.875+0000 7f37e3656980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:23:08.876 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.875+0000 7f37e3656980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:23:08.876 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.875+0000 7f37e3656980 20 zonegroup default 2024-05-30T16:23:08.876 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.875+0000 7f37e3656980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.876 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.875+0000 7f37e3656980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:08.876 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.875+0000 7f37e3656980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.875+0000 7f37e3656980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:08.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.875+0000 7f37e3656980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.875+0000 7f37e3656980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:08.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.875+0000 7f37e3656980 20 started sync module instance, tier type = 2024-05-30T16:23:08.877 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.875+0000 7f37e3656980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:23:08.893 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.891+0000 7f37e3656980 20 add_watcher() i=0 2024-05-30T16:23:08.893 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.891+0000 7f37e3656980 20 add_watcher() i=1 2024-05-30T16:23:08.893 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.891+0000 7f37e3656980 20 add_watcher() i=2 2024-05-30T16:23:08.893 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.891+0000 7f37e3656980 20 add_watcher() i=3 2024-05-30T16:23:08.893 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.891+0000 7f37e3656980 20 add_watcher() i=4 2024-05-30T16:23:08.893 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.891+0000 7f37e3656980 20 add_watcher() i=5 2024-05-30T16:23:08.894 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.891+0000 7f37e3656980 20 add_watcher() i=6 2024-05-30T16:23:08.894 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.895+0000 7f37e3656980 20 add_watcher() i=7 2024-05-30T16:23:08.894 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.895+0000 7f37e3656980 2 all 8 watchers are set, enabling cache 2024-05-30T16:23:08.897 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.895+0000 7f3697fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:23:08.898 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.895+0000 7f37e3656980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:23:08.898 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.895+0000 7f37e3656980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:23:08.898 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.895+0000 7f37e3656980 5 note: GC not initialized 2024-05-30T16:23:08.898 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.899+0000 7f3696fe5640 20 reqs_thread_entry: start 2024-05-30T16:23:08.898 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.899+0000 7f37e3656980 20 init_complete bucket index max shards: 11 2024-05-30T16:23:08.898 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.899+0000 7f37e3656980 20 Filter name: none 2024-05-30T16:23:08.898 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.899+0000 7f3694fe1640 20 reqs_thread_entry: start 2024-05-30T16:23:08.899 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.899+0000 7f37e3656980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.899 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.899+0000 7f37e3656980 20 rados_obj.operate() r=0 bl.length=197 2024-05-30T16:23:08.899 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.899+0000 7f37e3656980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.900 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.899+0000 7f37e3656980 20 rados_obj.operate() r=0 bl.length=411 2024-05-30T16:23:08.900 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.899+0000 7f37e3656980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:23:08.900 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.899+0000 7f37e3656980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.900 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.899+0000 7f37e3656980 20 rados_obj.operate() r=0 bl.length=197 2024-05-30T16:23:08.901 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.899+0000 7f37e3656980 20 rados->read ofs=0 len=0 2024-05-30T16:23:08.901 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.899+0000 7f37e3656980 20 rados_obj.operate() r=0 bl.length=411 2024-05-30T16:23:08.901 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.899+0000 7f37e3656980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-30T16:23:08.904 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.903+0000 7f3697fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:23:08.910 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.911+0000 7f37e3656980 20 remove_watcher() i=0 2024-05-30T16:23:08.910 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.911+0000 7f37e3656980 2 removed watcher, disabling cache 2024-05-30T16:23:08.912 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.911+0000 7f37e3656980 20 remove_watcher() i=1 2024-05-30T16:23:08.915 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.915+0000 7f37e3656980 20 remove_watcher() i=2 2024-05-30T16:23:08.917 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.915+0000 7f37e3656980 20 remove_watcher() i=3 2024-05-30T16:23:08.919 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.919+0000 7f37e3656980 20 remove_watcher() i=4 2024-05-30T16:23:08.921 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.919+0000 7f37e3656980 20 remove_watcher() i=5 2024-05-30T16:23:08.922 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.923+0000 7f37e3656980 20 remove_watcher() i=6 2024-05-30T16:23:08.923 INFO:tasks.workunit.client.0.smithi006.stderr:2024-05-30T16:23:08.923+0000 7f37e3656980 20 remove_watcher() i=7 2024-05-30T16:23:08.937 INFO:tasks.workunit.client.0.smithi006.stderr:{'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}} 2024-05-30T16:23:08.937 INFO:tasks.workunit.client.0.smithi006.stderr:Deleting bucket a-bucket 2024-05-30T16:23:09.074 INFO:tasks.workunit.client.0.smithi006.stderr:Deleting bucket myver 2024-05-30T16:23:09.690 INFO:tasks.workunit.client.0.smithi006.stderr:Completed resharding tests 2024-05-30T16:23:09.691 INFO:tasks.workunit.client.0.smithi006.stdout:connected to http://localhost:80 2024-05-30T16:23:09.800 INFO:tasks.workunit.client.0.smithi006.stderr:+ deactivate 2024-05-30T16:23:09.800 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -n /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin ']' 2024-05-30T16:23:09.801 INFO:tasks.workunit.client.0.smithi006.stderr:+ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-05-30T16:23:09.801 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-05-30T16:23:09.801 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset _OLD_VIRTUAL_PATH 2024-05-30T16:23:09.801 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -n '' ']' 2024-05-30T16:23:09.801 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -n /usr/bin/bash -o -n '' ']' 2024-05-30T16:23:09.801 INFO:tasks.workunit.client.0.smithi006.stderr:+ hash -r 2024-05-30T16:23:09.801 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -n '' ']' 2024-05-30T16:23:09.801 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset VIRTUAL_ENV 2024-05-30T16:23:09.801 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset VIRTUAL_ENV_PROMPT 2024-05-30T16:23:09.801 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' '!' '' = nondestructive ']' 2024-05-30T16:23:09.801 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset -f deactivate 2024-05-30T16:23:09.801 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo OK. 2024-05-30T16:23:09.802 INFO:tasks.workunit.client.0.smithi006.stdout:OK. 2024-05-30T16:23:09.803 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-30T16:23:09.803 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-30T16:23:09.862 INFO:tasks.workunit:Stopping ['rgw/run-reshard.sh'] on client.0... 2024-05-30T16:23:09.863 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-30T16:23:10.294 DEBUG:teuthology.parallel:result is None 2024-05-30T16:23:10.295 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-30T16:23:10.307 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-30T16:23:10.308 DEBUG:teuthology.orchestra.run.smithi006:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-05-30T16:23:10.354 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-05-30T16:23:10.354 INFO:teuthology.run_tasks:Running task s3tests-java... 2024-05-30T16:23:10.368 DEBUG:tasks.s3tests_java:S3 Tests Java: __INIT__ 2024-05-30T16:23:10.369 DEBUG:tasks.s3tests_java:S3 Tests Java: SETUP 2024-05-30T16:23:10.369 INFO:tasks.s3tests_java:S3 Tests Java: Downloading test suite... 2024-05-30T16:23:10.369 DEBUG:teuthology.orchestra.run.smithi006:> git clone -b ceph-master https://github.com/ceph/java_s3tests.git /home/ubuntu/cephtest/s3-tests-java 2024-05-30T16:23:10.410 INFO:teuthology.orchestra.run.smithi006.stderr:Cloning into '/home/ubuntu/cephtest/s3-tests-java'... 2024-05-30T16:23:14.923 INFO:tasks.s3tests_java:S3 Tests Java: Installing required packages... 2024-05-30T16:23:14.923 DEBUG:teuthology.orchestra.run.smithi006:> /home/ubuntu/cephtest/s3-tests-java/bootstrap.sh 2024-05-30T16:23:14.973 INFO:teuthology.orchestra.run.smithi006.stderr:/home/ubuntu/cephtest/s3-tests-java/bootstrap.sh: missing required DEB packages. Installing via sudo. 2024-05-30T16:23:15.036 INFO:teuthology.orchestra.run.smithi006.stdout:Reading package lists... 2024-05-30T16:23:15.245 INFO:teuthology.orchestra.run.smithi006.stdout:Building dependency tree... 2024-05-30T16:23:15.246 INFO:teuthology.orchestra.run.smithi006.stdout:Reading state information... 2024-05-30T16:23:15.448 INFO:teuthology.orchestra.run.smithi006.stdout:The following packages were automatically installed and are no longer required: 2024-05-30T16:23:15.448 INFO:teuthology.orchestra.run.smithi006.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-30T16:23:15.448 INFO:teuthology.orchestra.run.smithi006.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-30T16:23:15.448 INFO:teuthology.orchestra.run.smithi006.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-30T16:23:15.449 INFO:teuthology.orchestra.run.smithi006.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-30T16:23:15.449 INFO:teuthology.orchestra.run.smithi006.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-30T16:23:15.449 INFO:teuthology.orchestra.run.smithi006.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-30T16:23:15.450 INFO:teuthology.orchestra.run.smithi006.stdout:The following additional packages will be installed: 2024-05-30T16:23:15.451 INFO:teuthology.orchestra.run.smithi006.stdout: libgail-common libgail18 libgtk2.0-0 libgtk2.0-bin libgtk2.0-common 2024-05-30T16:23:15.451 INFO:teuthology.orchestra.run.smithi006.stdout: openjdk-8-jdk-headless openjdk-8-jre openjdk-8-jre-headless 2024-05-30T16:23:15.452 INFO:teuthology.orchestra.run.smithi006.stdout:Suggested packages: 2024-05-30T16:23:15.453 INFO:teuthology.orchestra.run.smithi006.stdout: gvfs openjdk-8-demo openjdk-8-source visualvm libnss-mdns fonts-nanum 2024-05-30T16:23:15.453 INFO:teuthology.orchestra.run.smithi006.stdout: fonts-ipafont-gothic fonts-ipafont-mincho fonts-wqy-microhei 2024-05-30T16:23:15.453 INFO:teuthology.orchestra.run.smithi006.stdout: fonts-wqy-zenhei fonts-indic 2024-05-30T16:23:15.486 INFO:teuthology.orchestra.run.smithi006.stdout:The following NEW packages will be installed: 2024-05-30T16:23:15.487 INFO:teuthology.orchestra.run.smithi006.stdout: libgail-common libgail18 libgtk2.0-0 libgtk2.0-bin libgtk2.0-common 2024-05-30T16:23:15.487 INFO:teuthology.orchestra.run.smithi006.stdout: openjdk-8-jdk openjdk-8-jdk-headless openjdk-8-jre openjdk-8-jre-headless 2024-05-30T16:23:15.546 INFO:teuthology.orchestra.run.smithi006.stdout:0 upgraded, 9 newly installed, 0 to remove and 307 not upgraded. 2024-05-30T16:23:15.546 INFO:teuthology.orchestra.run.smithi006.stdout:Need to get 46.1 MB of archives. 2024-05-30T16:23:15.546 INFO:teuthology.orchestra.run.smithi006.stdout:After this operation, 155 MB of additional disk space will be used. 2024-05-30T16:23:15.546 INFO:teuthology.orchestra.run.smithi006.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgtk2.0-common all 2.24.33-2ubuntu2 [125 kB] 2024-05-30T16:23:15.635 INFO:teuthology.orchestra.run.smithi006.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgtk2.0-0 amd64 2.24.33-2ubuntu2 [2,037 kB] 2024-05-30T16:23:15.735 INFO:teuthology.orchestra.run.smithi006.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgail18 amd64 2.24.33-2ubuntu2 [15.9 kB] 2024-05-30T16:23:15.736 INFO:teuthology.orchestra.run.smithi006.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgail-common amd64 2.24.33-2ubuntu2 [132 kB] 2024-05-30T16:23:15.739 INFO:teuthology.orchestra.run.smithi006.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgtk2.0-bin amd64 2.24.33-2ubuntu2 [7,932 B] 2024-05-30T16:23:15.739 INFO:teuthology.orchestra.run.smithi006.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 openjdk-8-jre-headless amd64 8u402-ga-2ubuntu1~22.04 [30.8 MB] 2024-05-30T16:23:16.193 INFO:teuthology.orchestra.run.smithi006.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 openjdk-8-jre amd64 8u402-ga-2ubuntu1~22.04 [75.5 kB] 2024-05-30T16:23:16.194 INFO:teuthology.orchestra.run.smithi006.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 openjdk-8-jdk-headless amd64 8u402-ga-2ubuntu1~22.04 [8,873 kB] 2024-05-30T16:23:16.333 INFO:teuthology.orchestra.run.smithi006.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 openjdk-8-jdk amd64 8u402-ga-2ubuntu1~22.04 [4,068 kB] 2024-05-30T16:23:16.685 INFO:teuthology.orchestra.run.smithi006.stderr:debconf: unable to initialize frontend: Dialog 2024-05-30T16:23:16.686 INFO:teuthology.orchestra.run.smithi006.stderr:debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.) 2024-05-30T16:23:16.686 INFO:teuthology.orchestra.run.smithi006.stderr:debconf: falling back to frontend: Readline 2024-05-30T16:23:16.700 INFO:teuthology.orchestra.run.smithi006.stderr:debconf: unable to initialize frontend: Readline 2024-05-30T16:23:16.700 INFO:teuthology.orchestra.run.smithi006.stderr:debconf: (This frontend requires a controlling tty.) 2024-05-30T16:23:16.700 INFO:teuthology.orchestra.run.smithi006.stderr:debconf: falling back to frontend: Teletype 2024-05-30T16:23:16.704 INFO:teuthology.orchestra.run.smithi006.stderr:dpkg-preconfigure: unable to re-open stdin: 2024-05-30T16:23:16.739 INFO:teuthology.orchestra.run.smithi006.stdout:Fetched 46.1 MB in 1s (52.7 MB/s) 2024-05-30T16:23:16.854 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libgtk2.0-common. 2024-05-30T16:23:16.919 INFO:teuthology.orchestra.run.smithi006.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174901 files and directories currently installed.) 2024-05-30T16:23:16.924 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../0-libgtk2.0-common_2.24.33-2ubuntu2_all.deb ... 2024-05-30T16:23:16.969 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libgtk2.0-common (2.24.33-2ubuntu2) ... 2024-05-30T16:23:17.449 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libgtk2.0-0:amd64. 2024-05-30T16:23:17.477 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../1-libgtk2.0-0_2.24.33-2ubuntu2_amd64.deb ... 2024-05-30T16:23:17.557 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libgtk2.0-0:amd64 (2.24.33-2ubuntu2) ... 2024-05-30T16:23:17.995 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libgail18:amd64. 2024-05-30T16:23:18.024 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../2-libgail18_2.24.33-2ubuntu2_amd64.deb ... 2024-05-30T16:23:18.061 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libgail18:amd64 (2.24.33-2ubuntu2) ... 2024-05-30T16:23:18.314 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libgail-common:amd64. 2024-05-30T16:23:18.342 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../3-libgail-common_2.24.33-2ubuntu2_amd64.deb ... 2024-05-30T16:23:18.380 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libgail-common:amd64 (2.24.33-2ubuntu2) ... 2024-05-30T16:23:18.633 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package libgtk2.0-bin. 2024-05-30T16:23:18.661 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../4-libgtk2.0-bin_2.24.33-2ubuntu2_amd64.deb ... 2024-05-30T16:23:18.699 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking libgtk2.0-bin (2.24.33-2ubuntu2) ... 2024-05-30T16:23:19.103 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package openjdk-8-jre-headless:amd64. 2024-05-30T16:23:19.131 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../5-openjdk-8-jre-headless_8u402-ga-2ubuntu1~22.04_amd64.deb ... 2024-05-30T16:23:19.198 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking openjdk-8-jre-headless:amd64 (8u402-ga-2ubuntu1~22.04) ... 2024-05-30T16:23:20.817 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package openjdk-8-jre:amd64. 2024-05-30T16:23:20.846 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../6-openjdk-8-jre_8u402-ga-2ubuntu1~22.04_amd64.deb ... 2024-05-30T16:23:20.887 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking openjdk-8-jre:amd64 (8u402-ga-2ubuntu1~22.04) ... 2024-05-30T16:23:21.338 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package openjdk-8-jdk-headless:amd64. 2024-05-30T16:23:21.367 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../7-openjdk-8-jdk-headless_8u402-ga-2ubuntu1~22.04_amd64.deb ... 2024-05-30T16:23:21.408 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking openjdk-8-jdk-headless:amd64 (8u402-ga-2ubuntu1~22.04) ... 2024-05-30T16:23:22.100 INFO:teuthology.orchestra.run.smithi006.stdout:Selecting previously unselected package openjdk-8-jdk:amd64. 2024-05-30T16:23:22.117 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing to unpack .../8-openjdk-8-jdk_8u402-ga-2ubuntu1~22.04_amd64.deb ... 2024-05-30T16:23:22.161 INFO:teuthology.orchestra.run.smithi006.stdout:Unpacking openjdk-8-jdk:amd64 (8u402-ga-2ubuntu1~22.04) ... 2024-05-30T16:23:22.475 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up openjdk-8-jre-headless:amd64 (8u402-ga-2ubuntu1~22.04) ... 2024-05-30T16:23:23.616 INFO:teuthology.orchestra.run.smithi006.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/orbd to provide /usr/bin/orbd (orbd) in auto mode 2024-05-30T16:23:23.643 INFO:teuthology.orchestra.run.smithi006.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/servertool to provide /usr/bin/servertool (servertool) in auto mode 2024-05-30T16:23:23.668 INFO:teuthology.orchestra.run.smithi006.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/tnameserv to provide /usr/bin/tnameserv (tnameserv) in auto mode 2024-05-30T16:23:24.067 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libgtk2.0-common (2.24.33-2ubuntu2) ... 2024-05-30T16:23:24.226 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libgtk2.0-0:amd64 (2.24.33-2ubuntu2) ... 2024-05-30T16:23:24.578 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up openjdk-8-jre:amd64 (8u402-ga-2ubuntu1~22.04) ... 2024-05-30T16:23:24.661 INFO:teuthology.orchestra.run.smithi006.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/policytool to provide /usr/bin/policytool (policytool) in auto mode 2024-05-30T16:23:24.730 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up openjdk-8-jdk-headless:amd64 (8u402-ga-2ubuntu1~22.04) ... 2024-05-30T16:23:24.821 INFO:teuthology.orchestra.run.smithi006.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/clhsdb to provide /usr/bin/clhsdb (clhsdb) in auto mode 2024-05-30T16:23:24.851 INFO:teuthology.orchestra.run.smithi006.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/extcheck to provide /usr/bin/extcheck (extcheck) in auto mode 2024-05-30T16:23:24.876 INFO:teuthology.orchestra.run.smithi006.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/hsdb to provide /usr/bin/hsdb (hsdb) in auto mode 2024-05-30T16:23:24.901 INFO:teuthology.orchestra.run.smithi006.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/idlj to provide /usr/bin/idlj (idlj) in auto mode 2024-05-30T16:23:25.020 INFO:teuthology.orchestra.run.smithi006.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/javah to provide /usr/bin/javah (javah) in auto mode 2024-05-30T16:23:25.164 INFO:teuthology.orchestra.run.smithi006.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/jhat to provide /usr/bin/jhat (jhat) in auto mode 2024-05-30T16:23:25.283 INFO:teuthology.orchestra.run.smithi006.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/jsadebugd to provide /usr/bin/jsadebugd (jsadebugd) in auto mode 2024-05-30T16:23:25.384 INFO:teuthology.orchestra.run.smithi006.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/native2ascii to provide /usr/bin/native2ascii (native2ascii) in auto mode 2024-05-30T16:23:25.435 INFO:teuthology.orchestra.run.smithi006.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/schemagen to provide /usr/bin/schemagen (schemagen) in auto mode 2024-05-30T16:23:25.488 INFO:teuthology.orchestra.run.smithi006.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/wsgen to provide /usr/bin/wsgen (wsgen) in auto mode 2024-05-30T16:23:25.503 INFO:teuthology.orchestra.run.smithi006.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/wsimport to provide /usr/bin/wsimport (wsimport) in auto mode 2024-05-30T16:23:25.530 INFO:teuthology.orchestra.run.smithi006.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/xjc to provide /usr/bin/xjc (xjc) in auto mode 2024-05-30T16:23:25.593 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libgail18:amd64 (2.24.33-2ubuntu2) ... 2024-05-30T16:23:25.694 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libgtk2.0-bin (2.24.33-2ubuntu2) ... 2024-05-30T16:23:25.795 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up openjdk-8-jdk:amd64 (8u402-ga-2ubuntu1~22.04) ... 2024-05-30T16:23:25.880 INFO:teuthology.orchestra.run.smithi006.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/appletviewer to provide /usr/bin/appletviewer (appletviewer) in auto mode 2024-05-30T16:23:25.981 INFO:teuthology.orchestra.run.smithi006.stdout:Setting up libgail-common:amd64 (2.24.33-2ubuntu2) ... 2024-05-30T16:23:26.125 INFO:teuthology.orchestra.run.smithi006.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-30T16:23:27.192 INFO:teuthology.orchestra.run.smithi006.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-05-30T16:23:27.292 INFO:teuthology.orchestra.run.smithi006.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-05-30T16:23:27.544 INFO:teuthology.orchestra.run.smithi006.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-05-30T16:23:28.007 INFO:teuthology.orchestra.run.smithi006.stdout: Gradle will be installed under /opt/gradle 2024-05-30T16:23:28.033 INFO:teuthology.orchestra.run.smithi006.stderr:--2024-05-30 16:23:28-- https://services.gradle.org/distributions/gradle-6.0.1-bin.zip 2024-05-30T16:23:28.036 INFO:teuthology.orchestra.run.smithi006.stderr:Resolving services.gradle.org (services.gradle.org)... 104.16.73.101, 104.16.72.101, 2606:4700::6810:4865, ... 2024-05-30T16:23:28.046 INFO:teuthology.orchestra.run.smithi006.stderr:Connecting to services.gradle.org (services.gradle.org)|104.16.73.101|:443... connected. 2024-05-30T16:23:28.086 INFO:teuthology.orchestra.run.smithi006.stderr:HTTP request sent, awaiting response... 301 Moved Permanently 2024-05-30T16:23:28.086 INFO:teuthology.orchestra.run.smithi006.stderr:Location: https://github.com/gradle/gradle-distributions/releases/download/v6.0.1/gradle-6.0.1-bin.zip [following] 2024-05-30T16:23:28.087 INFO:teuthology.orchestra.run.smithi006.stderr:--2024-05-30 16:23:28-- https://github.com/gradle/gradle-distributions/releases/download/v6.0.1/gradle-6.0.1-bin.zip 2024-05-30T16:23:28.088 INFO:teuthology.orchestra.run.smithi006.stderr:Resolving github.com (github.com)... 140.82.112.4 2024-05-30T16:23:28.099 INFO:teuthology.orchestra.run.smithi006.stderr:Connecting to github.com (github.com)|140.82.112.4|:443... connected. 2024-05-30T16:23:28.214 INFO:teuthology.orchestra.run.smithi006.stderr:HTTP request sent, awaiting response... 302 Found 2024-05-30T16:23:28.214 INFO:teuthology.orchestra.run.smithi006.stderr:Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/696192900/8f418ba1-c336-448c-95c6-5180382113cb?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20240530%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240530T162328Z&X-Amz-Expires=300&X-Amz-Signature=ab827b669b01fb5c465030e0f1da4c6e5c5e272fe77330f0d10217bce17c0075&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=696192900&response-content-disposition=attachment%3B%20filename%3Dgradle-6.0.1-bin.zip&response-content-type=application%2Foctet-stream [following] 2024-05-30T16:23:28.215 INFO:teuthology.orchestra.run.smithi006.stderr:--2024-05-30 16:23:28-- https://objects.githubusercontent.com/github-production-release-asset-2e65be/696192900/8f418ba1-c336-448c-95c6-5180382113cb?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20240530%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240530T162328Z&X-Amz-Expires=300&X-Amz-Signature=ab827b669b01fb5c465030e0f1da4c6e5c5e272fe77330f0d10217bce17c0075&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=696192900&response-content-disposition=attachment%3B%20filename%3Dgradle-6.0.1-bin.zip&response-content-type=application%2Foctet-stream 2024-05-30T16:23:28.216 INFO:teuthology.orchestra.run.smithi006.stderr:Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.111.133, 185.199.109.133, 185.199.108.133, ... 2024-05-30T16:23:28.232 INFO:teuthology.orchestra.run.smithi006.stderr:Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.111.133|:443... connected. 2024-05-30T16:23:28.324 INFO:teuthology.orchestra.run.smithi006.stderr:HTTP request sent, awaiting response... 200 OK 2024-05-30T16:23:28.324 INFO:teuthology.orchestra.run.smithi006.stderr:Length: 95129672 (91M) [application/octet-stream] 2024-05-30T16:23:28.324 INFO:teuthology.orchestra.run.smithi006.stderr:Saving to: 'gradle-6.0.1-bin.zip' 2024-05-30T16:23:28.324 INFO:teuthology.orchestra.run.smithi006.stderr: 2024-05-30T16:23:28.338 INFO:teuthology.orchestra.run.smithi006.stderr: 0K .......... .......... .......... .......... .......... 0% 3.28M 28s 2024-05-30T16:23:28.352 INFO:teuthology.orchestra.run.smithi006.stderr: 50K .......... .......... .......... .......... .......... 0% 3.35M 27s 2024-05-30T16:23:28.355 INFO:teuthology.orchestra.run.smithi006.stderr: 100K .......... .......... .......... .......... .......... 0% 16.8M 20s 2024-05-30T16:23:28.358 INFO:teuthology.orchestra.run.smithi006.stderr: 150K .......... .......... .......... .......... .......... 0% 15.8M 16s 2024-05-30T16:23:28.367 INFO:teuthology.orchestra.run.smithi006.stderr: 200K .......... .......... .......... .......... .......... 0% 5.61M 16s 2024-05-30T16:23:28.369 INFO:teuthology.orchestra.run.smithi006.stderr: 250K .......... .......... .......... .......... .......... 0% 29.5M 14s 2024-05-30T16:23:28.371 INFO:teuthology.orchestra.run.smithi006.stderr: 300K .......... .......... .......... .......... .......... 0% 21.4M 13s 2024-05-30T16:23:28.372 INFO:teuthology.orchestra.run.smithi006.stderr: 350K .......... .......... .......... .......... .......... 0% 37.6M 11s 2024-05-30T16:23:28.375 INFO:teuthology.orchestra.run.smithi006.stderr: 400K .......... .......... .......... .......... .......... 0% 26.0M 11s 2024-05-30T16:23:28.375 INFO:teuthology.orchestra.run.smithi006.stderr: 450K .......... .......... .......... .......... .......... 0% 43.4M 10s 2024-05-30T16:23:28.382 INFO:teuthology.orchestra.run.smithi006.stderr: 500K .......... .......... .......... .......... .......... 0% 7.34M 10s 2024-05-30T16:23:28.384 INFO:teuthology.orchestra.run.smithi006.stderr: 550K .......... .......... .......... .......... .......... 0% 34.1M 9s 2024-05-30T16:23:28.387 INFO:teuthology.orchestra.run.smithi006.stderr: 600K .......... .......... .......... .......... .......... 0% 27.9M 9s 2024-05-30T16:23:28.387 INFO:teuthology.orchestra.run.smithi006.stderr: 650K .......... .......... .......... .......... .......... 0% 68.6M 8s 2024-05-30T16:23:28.387 INFO:teuthology.orchestra.run.smithi006.stderr: 700K .......... .......... .......... .......... .......... 0% 89.4M 8s 2024-05-30T16:23:28.388 INFO:teuthology.orchestra.run.smithi006.stderr: 750K .......... .......... .......... .......... .......... 0% 59.9M 7s 2024-05-30T16:23:28.391 INFO:teuthology.orchestra.run.smithi006.stderr: 800K .......... .......... .......... .......... .......... 0% 50.3M 7s 2024-05-30T16:23:28.391 INFO:teuthology.orchestra.run.smithi006.stderr: 850K .......... .......... .......... .......... .......... 0% 46.5M 7s 2024-05-30T16:23:28.391 INFO:teuthology.orchestra.run.smithi006.stderr: 900K .......... .......... .......... .......... .......... 1% 74.6M 6s 2024-05-30T16:23:28.391 INFO:teuthology.orchestra.run.smithi006.stderr: 950K .......... .......... .......... .......... .......... 1% 100M 6s 2024-05-30T16:23:29.558 INFO:teuthology.orchestra.run.smithi006.stderr: 1000K .......... .......... .......... .......... .......... 1% 8.37M 6s 2024-05-30T16:23:29.559 INFO:teuthology.orchestra.run.smithi006.stderr: 1050K .......... .......... .......... .......... .......... 1% 73.4M 6s 2024-05-30T16:23:29.559 INFO:teuthology.orchestra.run.smithi006.stderr: 1100K .......... .......... .......... .......... .......... 1% 44.1M 6s 2024-05-30T16:23:29.559 INFO:teuthology.orchestra.run.smithi006.stderr: 1150K .......... .......... .......... .......... .......... 1% 85.7M 6s 2024-05-30T16:23:29.559 INFO:teuthology.orchestra.run.smithi006.stderr: 1200K .......... .......... .......... .......... .......... 1% 125M 6s 2024-05-30T16:23:29.559 INFO:teuthology.orchestra.run.smithi006.stderr: 1250K .......... .......... .......... .......... .......... 1% 51.2M 5s 2024-05-30T16:23:29.559 INFO:teuthology.orchestra.run.smithi006.stderr: 1300K .......... .......... .......... .......... .......... 1% 104M 5s 2024-05-30T16:23:29.559 INFO:teuthology.orchestra.run.smithi006.stderr: 1350K .......... .......... .......... .......... .......... 1% 122M 5s 2024-05-30T16:23:29.559 INFO:teuthology.orchestra.run.smithi006.stderr: 1400K .......... .......... .......... .......... .......... 1% 108M 5s 2024-05-30T16:23:29.559 INFO:teuthology.orchestra.run.smithi006.stderr: 1450K .......... .......... .......... .......... .......... 1% 30.0M 5s 2024-05-30T16:23:29.559 INFO:teuthology.orchestra.run.smithi006.stderr: 1500K .......... .......... .......... .......... .......... 1% 58.3M 5s 2024-05-30T16:23:29.559 INFO:teuthology.orchestra.run.smithi006.stderr: 1550K .......... .......... .......... .......... .......... 1% 106M 5s 2024-05-30T16:23:29.560 INFO:teuthology.orchestra.run.smithi006.stderr: 1600K .......... .......... .......... .......... .......... 1% 114M 5s 2024-05-30T16:23:29.560 INFO:teuthology.orchestra.run.smithi006.stderr: 1650K .......... .......... .......... .......... .......... 1% 95.8M 4s 2024-05-30T16:23:29.560 INFO:teuthology.orchestra.run.smithi006.stderr: 1700K .......... .......... .......... .......... .......... 1% 110M 4s 2024-05-30T16:23:29.560 INFO:teuthology.orchestra.run.smithi006.stderr: 1750K .......... .......... .......... .......... .......... 1% 108M 4s 2024-05-30T16:23:29.560 INFO:teuthology.orchestra.run.smithi006.stderr: 1800K .......... .......... .......... .......... .......... 1% 108M 4s 2024-05-30T16:23:29.560 INFO:teuthology.orchestra.run.smithi006.stderr: 1850K .......... .......... .......... .......... .......... 2% 97.5M 4s 2024-05-30T16:23:29.560 INFO:teuthology.orchestra.run.smithi006.stderr: 1900K .......... .......... .......... .......... .......... 2% 116M 4s 2024-05-30T16:23:29.560 INFO:teuthology.orchestra.run.smithi006.stderr: 1950K .......... .......... .......... .......... .......... 2% 121M 4s 2024-05-30T16:23:29.560 INFO:teuthology.orchestra.run.smithi006.stderr: 2000K .......... .......... .......... .......... .......... 2% 17.8M 4s 2024-05-30T16:23:29.560 INFO:teuthology.orchestra.run.smithi006.stderr: 2050K .......... .......... .......... .......... .......... 2% 72.0M 4s 2024-05-30T16:23:29.560 INFO:teuthology.orchestra.run.smithi006.stderr: 2100K .......... .......... .......... .......... .......... 2% 108M 4s 2024-05-30T16:23:29.560 INFO:teuthology.orchestra.run.smithi006.stderr: 2150K .......... .......... .......... .......... .......... 2% 106M 4s 2024-05-30T16:23:29.560 INFO:teuthology.orchestra.run.smithi006.stderr: 2200K .......... .......... .......... .......... .......... 2% 104M 4s 2024-05-30T16:23:29.560 INFO:teuthology.orchestra.run.smithi006.stderr: 2250K .......... .......... .......... .......... .......... 2% 95.6M 4s 2024-05-30T16:23:29.561 INFO:teuthology.orchestra.run.smithi006.stderr: 2300K .......... .......... .......... .......... .......... 2% 109M 3s 2024-05-30T16:23:29.561 INFO:teuthology.orchestra.run.smithi006.stderr: 2350K .......... .......... .......... .......... .......... 2% 96.5M 3s 2024-05-30T16:23:29.561 INFO:teuthology.orchestra.run.smithi006.stderr: 2400K .......... .......... .......... .......... .......... 2% 109M 3s 2024-05-30T16:23:29.561 INFO:teuthology.orchestra.run.smithi006.stderr: 2450K .......... .......... .......... .......... .......... 2% 90.9M 3s 2024-05-30T16:23:29.561 INFO:teuthology.orchestra.run.smithi006.stderr: 2500K .......... .......... .......... .......... .......... 2% 107M 3s 2024-05-30T16:23:29.561 INFO:teuthology.orchestra.run.smithi006.stderr: 2550K .......... .......... .......... .......... .......... 2% 106M 3s 2024-05-30T16:23:29.561 INFO:teuthology.orchestra.run.smithi006.stderr: 2600K .......... .......... .......... .......... .......... 2% 111M 3s 2024-05-30T16:23:29.561 INFO:teuthology.orchestra.run.smithi006.stderr: 2650K .......... .......... .......... .......... .......... 2% 96.2M 3s 2024-05-30T16:23:29.561 INFO:teuthology.orchestra.run.smithi006.stderr: 2700K .......... .......... .......... .......... .......... 2% 108M 3s 2024-05-30T16:23:29.561 INFO:teuthology.orchestra.run.smithi006.stderr: 2750K .......... .......... .......... .......... .......... 3% 103M 3s 2024-05-30T16:23:29.561 INFO:teuthology.orchestra.run.smithi006.stderr: 2800K .......... .......... .......... .......... .......... 3% 107M 3s 2024-05-30T16:23:29.561 INFO:teuthology.orchestra.run.smithi006.stderr: 2850K .......... .......... .......... .......... .......... 3% 88.5M 3s 2024-05-30T16:23:29.561 INFO:teuthology.orchestra.run.smithi006.stderr: 2900K .......... .......... .......... .......... .......... 3% 106M 3s 2024-05-30T16:23:29.562 INFO:teuthology.orchestra.run.smithi006.stderr: 2950K .......... .......... .......... .......... .......... 3% 105M 3s 2024-05-30T16:23:29.562 INFO:teuthology.orchestra.run.smithi006.stderr: 3000K .......... .......... .......... .......... .......... 3% 109M 3s 2024-05-30T16:23:29.562 INFO:teuthology.orchestra.run.smithi006.stderr: 3050K .......... .......... .......... .......... .......... 3% 104M 3s 2024-05-30T16:23:29.562 INFO:teuthology.orchestra.run.smithi006.stderr: 3100K .......... .......... .......... .......... .......... 3% 100M 3s 2024-05-30T16:23:29.562 INFO:teuthology.orchestra.run.smithi006.stderr: 3150K .......... .......... .......... .......... .......... 3% 112M 3s 2024-05-30T16:23:29.562 INFO:teuthology.orchestra.run.smithi006.stderr: 3200K .......... .......... .......... .......... .......... 3% 98.5M 3s 2024-05-30T16:23:29.562 INFO:teuthology.orchestra.run.smithi006.stderr: 3250K .......... .......... .......... .......... .......... 3% 89.1M 3s 2024-05-30T16:23:29.562 INFO:teuthology.orchestra.run.smithi006.stderr: 3300K .......... .......... .......... .......... .......... 3% 108M 3s 2024-05-30T16:23:29.562 INFO:teuthology.orchestra.run.smithi006.stderr: 3350K .......... .......... .......... .......... .......... 3% 102M 3s 2024-05-30T16:23:29.562 INFO:teuthology.orchestra.run.smithi006.stderr: 3400K .......... .......... .......... .......... .......... 3% 110M 3s 2024-05-30T16:23:29.562 INFO:teuthology.orchestra.run.smithi006.stderr: 3450K .......... .......... .......... .......... .......... 3% 109M 3s 2024-05-30T16:23:29.562 INFO:teuthology.orchestra.run.smithi006.stderr: 3500K .......... .......... .......... .......... .......... 3% 113M 3s 2024-05-30T16:23:29.562 INFO:teuthology.orchestra.run.smithi006.stderr: 3550K .......... .......... .......... .......... .......... 3% 92.4M 3s 2024-05-30T16:23:29.563 INFO:teuthology.orchestra.run.smithi006.stderr: 3600K .......... .......... .......... .......... .......... 3% 103M 3s 2024-05-30T16:23:29.563 INFO:teuthology.orchestra.run.smithi006.stderr: 3650K .......... .......... .......... .......... .......... 3% 109M 2s 2024-05-30T16:23:29.563 INFO:teuthology.orchestra.run.smithi006.stderr: 3700K .......... .......... .......... .......... .......... 4% 112M 2s 2024-05-30T16:23:29.563 INFO:teuthology.orchestra.run.smithi006.stderr: 3750K .......... .......... .......... .......... .......... 4% 97.1M 2s 2024-05-30T16:23:29.563 INFO:teuthology.orchestra.run.smithi006.stderr: 3800K .......... .......... .......... .......... .......... 4% 112M 2s 2024-05-30T16:23:29.563 INFO:teuthology.orchestra.run.smithi006.stderr: 3850K .......... .......... .......... .......... .......... 4% 122M 2s 2024-05-30T16:23:29.563 INFO:teuthology.orchestra.run.smithi006.stderr: 3900K .......... .......... .......... .......... .......... 4% 106M 2s 2024-05-30T16:23:29.563 INFO:teuthology.orchestra.run.smithi006.stderr: 3950K .......... .......... .......... .......... .......... 4% 92.1M 2s 2024-05-30T16:23:29.563 INFO:teuthology.orchestra.run.smithi006.stderr: 4000K .......... .......... .......... .......... .......... 4% 107M 2s 2024-05-30T16:23:29.563 INFO:teuthology.orchestra.run.smithi006.stderr: 4050K .......... .......... .......... .......... .......... 4% 103M 2s 2024-05-30T16:23:29.563 INFO:teuthology.orchestra.run.smithi006.stderr: 4100K .......... .......... .......... .......... .......... 4% 108M 2s 2024-05-30T16:23:29.563 INFO:teuthology.orchestra.run.smithi006.stderr: 4150K .......... .......... .......... .......... .......... 4% 96.7M 2s 2024-05-30T16:23:29.563 INFO:teuthology.orchestra.run.smithi006.stderr: 4200K .......... .......... .......... .......... .......... 4% 96.5M 2s 2024-05-30T16:23:29.563 INFO:teuthology.orchestra.run.smithi006.stderr: 4250K .......... .......... .......... .......... .......... 4% 111M 2s 2024-05-30T16:23:29.564 INFO:teuthology.orchestra.run.smithi006.stderr: 4300K .......... .......... .......... .......... .......... 4% 112M 2s 2024-05-30T16:23:29.564 INFO:teuthology.orchestra.run.smithi006.stderr: 4350K .......... .......... .......... .......... .......... 4% 89.9M 2s 2024-05-30T16:23:29.564 INFO:teuthology.orchestra.run.smithi006.stderr: 4400K .......... .......... .......... .......... .......... 4% 110M 2s 2024-05-30T16:23:29.564 INFO:teuthology.orchestra.run.smithi006.stderr: 4450K .......... .......... .......... .......... .......... 4% 88.3M 2s 2024-05-30T16:23:29.564 INFO:teuthology.orchestra.run.smithi006.stderr: 4500K .......... .......... .......... .......... .......... 4% 109M 2s 2024-05-30T16:23:29.564 INFO:teuthology.orchestra.run.smithi006.stderr: 4550K .......... .......... .......... .......... .......... 4% 106M 2s 2024-05-30T16:23:29.564 INFO:teuthology.orchestra.run.smithi006.stderr: 4600K .......... .......... .......... .......... .......... 5% 110M 2s 2024-05-30T16:23:29.564 INFO:teuthology.orchestra.run.smithi006.stderr: 4650K .......... .......... .......... .......... .......... 5% 111M 2s 2024-05-30T16:23:29.564 INFO:teuthology.orchestra.run.smithi006.stderr: 4700K .......... .......... .......... .......... .......... 5% 110M 2s 2024-05-30T16:23:29.564 INFO:teuthology.orchestra.run.smithi006.stderr: 4750K .......... .......... .......... .......... .......... 5% 91.6M 2s 2024-05-30T16:23:29.564 INFO:teuthology.orchestra.run.smithi006.stderr: 4800K .......... .......... .......... .......... .......... 5% 115M 2s 2024-05-30T16:23:29.564 INFO:teuthology.orchestra.run.smithi006.stderr: 4850K .......... .......... .......... .......... .......... 5% 107M 2s 2024-05-30T16:23:29.564 INFO:teuthology.orchestra.run.smithi006.stderr: 4900K .......... .......... .......... .......... .......... 5% 105M 2s 2024-05-30T16:23:29.565 INFO:teuthology.orchestra.run.smithi006.stderr: 4950K .......... .......... .......... .......... .......... 5% 101M 2s 2024-05-30T16:23:29.565 INFO:teuthology.orchestra.run.smithi006.stderr: 5000K .......... .......... .......... .......... .......... 5% 107M 2s 2024-05-30T16:23:29.565 INFO:teuthology.orchestra.run.smithi006.stderr: 5050K .......... .......... .......... .......... .......... 5% 115M 2s 2024-05-30T16:23:29.565 INFO:teuthology.orchestra.run.smithi006.stderr: 5100K .......... .......... .......... .......... .......... 5% 112M 2s 2024-05-30T16:23:29.565 INFO:teuthology.orchestra.run.smithi006.stderr: 5150K .......... .......... .......... .......... .......... 5% 91.5M 2s 2024-05-30T16:23:29.565 INFO:teuthology.orchestra.run.smithi006.stderr: 5200K .......... .......... .......... .......... .......... 5% 110M 2s 2024-05-30T16:23:29.565 INFO:teuthology.orchestra.run.smithi006.stderr: 5250K .......... .......... .......... .......... .......... 5% 111M 2s 2024-05-30T16:23:29.565 INFO:teuthology.orchestra.run.smithi006.stderr: 5300K .......... .......... .......... .......... .......... 5% 105M 2s 2024-05-30T16:23:29.565 INFO:teuthology.orchestra.run.smithi006.stderr: 5350K .......... .......... .......... .......... .......... 5% 101M 2s 2024-05-30T16:23:29.565 INFO:teuthology.orchestra.run.smithi006.stderr: 5400K .......... .......... .......... .......... .......... 5% 116M 2s 2024-05-30T16:23:29.565 INFO:teuthology.orchestra.run.smithi006.stderr: 5450K .......... .......... .......... .......... .......... 5% 114M 2s 2024-05-30T16:23:29.565 INFO:teuthology.orchestra.run.smithi006.stderr: 5500K .......... .......... .......... .......... .......... 5% 109M 2s 2024-05-30T16:23:29.565 INFO:teuthology.orchestra.run.smithi006.stderr: 5550K .......... .......... .......... .......... .......... 6% 100M 2s 2024-05-30T16:23:29.565 INFO:teuthology.orchestra.run.smithi006.stderr: 5600K .......... .......... .......... .......... .......... 6% 121M 2s 2024-05-30T16:23:29.566 INFO:teuthology.orchestra.run.smithi006.stderr: 5650K .......... .......... .......... .......... .......... 6% 121M 2s 2024-05-30T16:23:29.566 INFO:teuthology.orchestra.run.smithi006.stderr: 5700K .......... .......... .......... .......... .......... 6% 105M 2s 2024-05-30T16:23:29.566 INFO:teuthology.orchestra.run.smithi006.stderr: 5750K .......... .......... .......... .......... .......... 6% 113M 2s 2024-05-30T16:23:29.566 INFO:teuthology.orchestra.run.smithi006.stderr: 5800K .......... .......... .......... .......... .......... 6% 120M 2s 2024-05-30T16:23:29.566 INFO:teuthology.orchestra.run.smithi006.stderr: 5850K .......... .......... .......... .......... .......... 6% 134M 2s 2024-05-30T16:23:29.566 INFO:teuthology.orchestra.run.smithi006.stderr: 5900K .......... .......... .......... .......... .......... 6% 65.1M 2s 2024-05-30T16:23:29.566 INFO:teuthology.orchestra.run.smithi006.stderr: 5950K .......... .......... .......... .......... .......... 6% 98.0M 2s 2024-05-30T16:23:29.566 INFO:teuthology.orchestra.run.smithi006.stderr: 6000K .......... .......... .......... .......... .......... 6% 116M 2s 2024-05-30T16:23:29.566 INFO:teuthology.orchestra.run.smithi006.stderr: 6050K .......... .......... .......... .......... .......... 6% 102M 2s 2024-05-30T16:23:29.566 INFO:teuthology.orchestra.run.smithi006.stderr: 6100K .......... .......... .......... .......... .......... 6% 121M 2s 2024-05-30T16:23:29.566 INFO:teuthology.orchestra.run.smithi006.stderr: 6150K .......... .......... .......... .......... .......... 6% 102M 2s 2024-05-30T16:23:29.566 INFO:teuthology.orchestra.run.smithi006.stderr: 6200K .......... .......... .......... .......... .......... 6% 105M 2s 2024-05-30T16:23:29.566 INFO:teuthology.orchestra.run.smithi006.stderr: 6250K .......... .......... .......... .......... .......... 6% 84.2M 2s 2024-05-30T16:23:29.567 INFO:teuthology.orchestra.run.smithi006.stderr: 6300K .......... .......... .......... .......... .......... 6% 85.7M 2s 2024-05-30T16:23:29.567 INFO:teuthology.orchestra.run.smithi006.stderr: 6350K .......... .......... .......... .......... .......... 6% 96.3M 2s 2024-05-30T16:23:29.567 INFO:teuthology.orchestra.run.smithi006.stderr: 6400K .......... .......... .......... .......... .......... 6% 90.4M 2s 2024-05-30T16:23:29.567 INFO:teuthology.orchestra.run.smithi006.stderr: 6450K .......... .......... .......... .......... .......... 6% 106M 2s 2024-05-30T16:23:29.567 INFO:teuthology.orchestra.run.smithi006.stderr: 6500K .......... .......... .......... .......... .......... 7% 116M 2s 2024-05-30T16:23:29.567 INFO:teuthology.orchestra.run.smithi006.stderr: 6550K .......... .......... .......... .......... .......... 7% 119M 2s 2024-05-30T16:23:29.567 INFO:teuthology.orchestra.run.smithi006.stderr: 6600K .......... .......... .......... .......... .......... 7% 66.4M 2s 2024-05-30T16:23:29.567 INFO:teuthology.orchestra.run.smithi006.stderr: 6650K .......... .......... .......... .......... .......... 7% 115M 2s 2024-05-30T16:23:29.567 INFO:teuthology.orchestra.run.smithi006.stderr: 6700K .......... .......... .......... .......... .......... 7% 116M 2s 2024-05-30T16:23:29.567 INFO:teuthology.orchestra.run.smithi006.stderr: 6750K .......... .......... .......... .......... .......... 7% 121M 2s 2024-05-30T16:23:29.567 INFO:teuthology.orchestra.run.smithi006.stderr: 6800K .......... .......... .......... .......... .......... 7% 105M 2s 2024-05-30T16:23:29.567 INFO:teuthology.orchestra.run.smithi006.stderr: 6850K .......... .......... .......... .......... .......... 7% 105M 2s 2024-05-30T16:23:29.567 INFO:teuthology.orchestra.run.smithi006.stderr: 6900K .......... .......... .......... .......... .......... 7% 117M 2s 2024-05-30T16:23:29.568 INFO:teuthology.orchestra.run.smithi006.stderr: 6950K .......... .......... .......... .......... .......... 7% 119M 2s 2024-05-30T16:23:29.568 INFO:teuthology.orchestra.run.smithi006.stderr: 7000K .......... .......... .......... .......... .......... 7% 85.2M 2s 2024-05-30T16:23:29.568 INFO:teuthology.orchestra.run.smithi006.stderr: 7050K .......... .......... .......... .......... .......... 7% 117M 2s 2024-05-30T16:23:29.568 INFO:teuthology.orchestra.run.smithi006.stderr: 7100K .......... .......... .......... .......... .......... 7% 112M 2s 2024-05-30T16:23:29.568 INFO:teuthology.orchestra.run.smithi006.stderr: 7150K .......... .......... .......... .......... .......... 7% 121M 2s 2024-05-30T16:23:29.568 INFO:teuthology.orchestra.run.smithi006.stderr: 7200K .......... .......... .......... .......... .......... 7% 98.8M 2s 2024-05-30T16:23:29.568 INFO:teuthology.orchestra.run.smithi006.stderr: 7250K .......... .......... .......... .......... .......... 7% 114M 2s 2024-05-30T16:23:29.568 INFO:teuthology.orchestra.run.smithi006.stderr: 7300K .......... .......... .......... .......... .......... 7% 118M 2s 2024-05-30T16:23:29.568 INFO:teuthology.orchestra.run.smithi006.stderr: 7350K .......... .......... .......... .......... .......... 7% 108M 2s 2024-05-30T16:23:29.568 INFO:teuthology.orchestra.run.smithi006.stderr: 7400K .......... .......... .......... .......... .......... 8% 90.5M 2s 2024-05-30T16:23:29.568 INFO:teuthology.orchestra.run.smithi006.stderr: 7450K .......... .......... .......... .......... .......... 8% 53.6M 2s 2024-05-30T16:23:29.568 INFO:teuthology.orchestra.run.smithi006.stderr: 7500K .......... .......... .......... .......... .......... 8% 103M 2s 2024-05-30T16:23:29.568 INFO:teuthology.orchestra.run.smithi006.stderr: 7550K .......... .......... .......... .......... .......... 8% 117M 2s 2024-05-30T16:23:29.569 INFO:teuthology.orchestra.run.smithi006.stderr: 7600K .......... .......... .......... .......... .......... 8% 109M 2s 2024-05-30T16:23:29.569 INFO:teuthology.orchestra.run.smithi006.stderr: 7650K .......... .......... .......... .......... .......... 8% 109M 2s 2024-05-30T16:23:29.569 INFO:teuthology.orchestra.run.smithi006.stderr: 7700K .......... .......... .......... .......... .......... 8% 97.9M 2s 2024-05-30T16:23:29.569 INFO:teuthology.orchestra.run.smithi006.stderr: 7750K .......... .......... .......... .......... .......... 8% 122M 2s 2024-05-30T16:23:29.569 INFO:teuthology.orchestra.run.smithi006.stderr: 7800K .......... .......... .......... .......... .......... 8% 95.3M 2s 2024-05-30T16:23:29.569 INFO:teuthology.orchestra.run.smithi006.stderr: 7850K .......... .......... .......... .......... .......... 8% 90.1M 2s 2024-05-30T16:23:29.569 INFO:teuthology.orchestra.run.smithi006.stderr: 7900K .......... .......... .......... .......... .......... 8% 118M 2s 2024-05-30T16:23:29.569 INFO:teuthology.orchestra.run.smithi006.stderr: 7950K .......... .......... .......... .......... .......... 8% 123M 2s 2024-05-30T16:23:29.569 INFO:teuthology.orchestra.run.smithi006.stderr: 8000K .......... .......... .......... .......... .......... 8% 94.6M 2s 2024-05-30T16:23:29.569 INFO:teuthology.orchestra.run.smithi006.stderr: 8050K .......... .......... .......... .......... .......... 8% 110M 2s 2024-05-30T16:23:29.569 INFO:teuthology.orchestra.run.smithi006.stderr: 8100K .......... .......... .......... .......... .......... 8% 102M 2s 2024-05-30T16:23:29.569 INFO:teuthology.orchestra.run.smithi006.stderr: 8150K .......... .......... .......... .......... .......... 8% 114M 2s 2024-05-30T16:23:29.569 INFO:teuthology.orchestra.run.smithi006.stderr: 8200K .......... .......... .......... .......... .......... 8% 82.4M 2s 2024-05-30T16:23:29.569 INFO:teuthology.orchestra.run.smithi006.stderr: 8250K .......... .......... .......... .......... .......... 8% 111M 1s 2024-05-30T16:23:29.570 INFO:teuthology.orchestra.run.smithi006.stderr: 8300K .......... .......... .......... .......... .......... 8% 106M 1s 2024-05-30T16:23:29.570 INFO:teuthology.orchestra.run.smithi006.stderr: 8350K .......... .......... .......... .......... .......... 9% 124M 1s 2024-05-30T16:23:29.570 INFO:teuthology.orchestra.run.smithi006.stderr: 8400K .......... .......... .......... .......... .......... 9% 112M 1s 2024-05-30T16:23:29.570 INFO:teuthology.orchestra.run.smithi006.stderr: 8450K .......... .......... .......... .......... .......... 9% 107M 1s 2024-05-30T16:23:29.570 INFO:teuthology.orchestra.run.smithi006.stderr: 8500K .......... .......... .......... .......... .......... 9% 108M 1s 2024-05-30T16:23:29.570 INFO:teuthology.orchestra.run.smithi006.stderr: 8550K .......... .......... .......... .......... .......... 9% 123M 1s 2024-05-30T16:23:29.570 INFO:teuthology.orchestra.run.smithi006.stderr: 8600K .......... .......... .......... .......... .......... 9% 95.6M 1s 2024-05-30T16:23:29.570 INFO:teuthology.orchestra.run.smithi006.stderr: 8650K .......... .......... .......... .......... .......... 9% 106M 1s 2024-05-30T16:23:29.570 INFO:teuthology.orchestra.run.smithi006.stderr: 8700K .......... .......... .......... .......... .......... 9% 120M 1s 2024-05-30T16:23:29.570 INFO:teuthology.orchestra.run.smithi006.stderr: 8750K .......... .......... .......... .......... .......... 9% 98.5M 1s 2024-05-30T16:23:29.570 INFO:teuthology.orchestra.run.smithi006.stderr: 8800K .......... .......... .......... .......... .......... 9% 116M 1s 2024-05-30T16:23:29.570 INFO:teuthology.orchestra.run.smithi006.stderr: 8850K .......... .......... .......... .......... .......... 9% 117M 1s 2024-05-30T16:23:29.570 INFO:teuthology.orchestra.run.smithi006.stderr: 8900K .......... .......... .......... .......... .......... 9% 108M 1s 2024-05-30T16:23:29.570 INFO:teuthology.orchestra.run.smithi006.stderr: 8950K .......... .......... .......... .......... .......... 9% 51.7M 1s 2024-05-30T16:23:29.570 INFO:teuthology.orchestra.run.smithi006.stderr: 9000K .......... .......... .......... .......... .......... 9% 23.0M 1s 2024-05-30T16:23:29.571 INFO:teuthology.orchestra.run.smithi006.stderr: 9050K .......... .......... .......... .......... .......... 9% 37.1M 1s 2024-05-30T16:23:29.571 INFO:teuthology.orchestra.run.smithi006.stderr: 9100K .......... .......... .......... .......... .......... 9% 83.0M 1s 2024-05-30T16:23:29.571 INFO:teuthology.orchestra.run.smithi006.stderr: 9150K .......... .......... .......... .......... .......... 9% 90.5M 1s 2024-05-30T16:23:29.571 INFO:teuthology.orchestra.run.smithi006.stderr: 9200K .......... .......... .......... .......... .......... 9% 101M 1s 2024-05-30T16:23:29.571 INFO:teuthology.orchestra.run.smithi006.stderr: 9250K .......... .......... .......... .......... .......... 10% 107M 1s 2024-05-30T16:23:29.571 INFO:teuthology.orchestra.run.smithi006.stderr: 9300K .......... .......... .......... .......... .......... 10% 108M 1s 2024-05-30T16:23:29.571 INFO:teuthology.orchestra.run.smithi006.stderr: 9350K .......... .......... .......... .......... .......... 10% 101M 1s 2024-05-30T16:23:29.571 INFO:teuthology.orchestra.run.smithi006.stderr: 9400K .......... .......... .......... .......... .......... 10% 77.2M 1s 2024-05-30T16:23:29.571 INFO:teuthology.orchestra.run.smithi006.stderr: 9450K .......... .......... .......... .......... .......... 10% 116M 1s 2024-05-30T16:23:29.571 INFO:teuthology.orchestra.run.smithi006.stderr: 9500K .......... .......... .......... .......... .......... 10% 113M 1s 2024-05-30T16:23:29.571 INFO:teuthology.orchestra.run.smithi006.stderr: 9550K .......... .......... .......... .......... .......... 10% 108M 1s 2024-05-30T16:23:29.571 INFO:teuthology.orchestra.run.smithi006.stderr: 9600K .......... .......... .......... .......... .......... 10% 93.6M 1s 2024-05-30T16:23:29.571 INFO:teuthology.orchestra.run.smithi006.stderr: 9650K .......... .......... .......... .......... .......... 10% 90.7M 1s 2024-05-30T16:23:29.571 INFO:teuthology.orchestra.run.smithi006.stderr: 9700K .......... .......... .......... .......... .......... 10% 104M 1s 2024-05-30T16:23:29.571 INFO:teuthology.orchestra.run.smithi006.stderr: 9750K .......... .......... .......... .......... .......... 10% 134M 1s 2024-05-30T16:23:29.572 INFO:teuthology.orchestra.run.smithi006.stderr: 9800K .......... .......... .......... .......... .......... 10% 103M 1s 2024-05-30T16:23:29.572 INFO:teuthology.orchestra.run.smithi006.stderr: 9850K .......... .......... .......... .......... .......... 10% 87.3M 1s 2024-05-30T16:23:29.572 INFO:teuthology.orchestra.run.smithi006.stderr: 9900K .......... .......... .......... .......... .......... 10% 86.6M 1s 2024-05-30T16:23:29.572 INFO:teuthology.orchestra.run.smithi006.stderr: 9950K .......... .......... .......... .......... .......... 10% 113M 1s 2024-05-30T16:23:29.572 INFO:teuthology.orchestra.run.smithi006.stderr: 10000K .......... .......... .......... .......... .......... 10% 118M 1s 2024-05-30T16:23:29.572 INFO:teuthology.orchestra.run.smithi006.stderr: 10050K .......... .......... .......... .......... .......... 10% 131M 1s 2024-05-30T16:23:29.572 INFO:teuthology.orchestra.run.smithi006.stderr: 10100K .......... .......... .......... .......... .......... 10% 59.9M 1s 2024-05-30T16:23:29.572 INFO:teuthology.orchestra.run.smithi006.stderr: 10150K .......... .......... .......... .......... .......... 10% 87.9M 1s 2024-05-30T16:23:29.572 INFO:teuthology.orchestra.run.smithi006.stderr: 10200K .......... .......... .......... .......... .......... 11% 100M 1s 2024-05-30T16:23:29.572 INFO:teuthology.orchestra.run.smithi006.stderr: 10250K .......... .......... .......... .......... .......... 11% 125M 1s 2024-05-30T16:23:29.572 INFO:teuthology.orchestra.run.smithi006.stderr: 10300K .......... .......... .......... .......... .......... 11% 116M 1s 2024-05-30T16:23:29.572 INFO:teuthology.orchestra.run.smithi006.stderr: 10350K .......... .......... .......... .......... .......... 11% 94.2M 1s 2024-05-30T16:23:29.572 INFO:teuthology.orchestra.run.smithi006.stderr: 10400K .......... .......... .......... .......... .......... 11% 120M 1s 2024-05-30T16:23:29.572 INFO:teuthology.orchestra.run.smithi006.stderr: 10450K .......... .......... .......... .......... .......... 11% 123M 1s 2024-05-30T16:23:29.572 INFO:teuthology.orchestra.run.smithi006.stderr: 10500K .......... .......... .......... .......... .......... 11% 103M 1s 2024-05-30T16:23:29.573 INFO:teuthology.orchestra.run.smithi006.stderr: 10550K .......... .......... .......... .......... .......... 11% 98.7M 1s 2024-05-30T16:23:29.573 INFO:teuthology.orchestra.run.smithi006.stderr: 10600K .......... .......... .......... .......... .......... 11% 90.7M 1s 2024-05-30T16:23:29.573 INFO:teuthology.orchestra.run.smithi006.stderr: 10650K .......... .......... .......... .......... .......... 11% 125M 1s 2024-05-30T16:23:29.573 INFO:teuthology.orchestra.run.smithi006.stderr: 10700K .......... .......... .......... .......... .......... 11% 116M 1s 2024-05-30T16:23:29.573 INFO:teuthology.orchestra.run.smithi006.stderr: 10750K .......... .......... .......... .......... .......... 11% 115M 1s 2024-05-30T16:23:29.573 INFO:teuthology.orchestra.run.smithi006.stderr: 10800K .......... .......... .......... .......... .......... 11% 101M 1s 2024-05-30T16:23:29.573 INFO:teuthology.orchestra.run.smithi006.stderr: 10850K .......... .......... .......... .......... .......... 11% 122M 1s 2024-05-30T16:23:29.573 INFO:teuthology.orchestra.run.smithi006.stderr: 10900K .......... .......... .......... .......... .......... 11% 88.1M 1s 2024-05-30T16:23:29.573 INFO:teuthology.orchestra.run.smithi006.stderr: 10950K .......... .......... .......... .......... .......... 11% 115M 1s 2024-05-30T16:23:29.573 INFO:teuthology.orchestra.run.smithi006.stderr: 11000K .......... .......... .......... .......... .......... 11% 110M 1s 2024-05-30T16:23:29.573 INFO:teuthology.orchestra.run.smithi006.stderr: 11050K .......... .......... .......... .......... .......... 11% 107M 1s 2024-05-30T16:23:29.573 INFO:teuthology.orchestra.run.smithi006.stderr: 11100K .......... .......... .......... .......... .......... 12% 110M 1s 2024-05-30T16:23:29.573 INFO:teuthology.orchestra.run.smithi006.stderr: 11150K .......... .......... .......... .......... .......... 12% 115M 1s 2024-05-30T16:23:29.573 INFO:teuthology.orchestra.run.smithi006.stderr: 11200K .......... .......... .......... .......... .......... 12% 112M 1s 2024-05-30T16:23:29.574 INFO:teuthology.orchestra.run.smithi006.stderr: 11250K .......... .......... .......... .......... .......... 12% 126M 1s 2024-05-30T16:23:29.574 INFO:teuthology.orchestra.run.smithi006.stderr: 11300K .......... .......... .......... .......... .......... 12% 90.5M 1s 2024-05-30T16:23:29.574 INFO:teuthology.orchestra.run.smithi006.stderr: 11350K .......... .......... .......... .......... .......... 12% 51.3M 1s 2024-05-30T16:23:29.574 INFO:teuthology.orchestra.run.smithi006.stderr: 11400K .......... .......... .......... .......... .......... 12% 64.0M 1s 2024-05-30T16:23:29.574 INFO:teuthology.orchestra.run.smithi006.stderr: 11450K .......... .......... .......... .......... .......... 12% 111M 1s 2024-05-30T16:23:29.574 INFO:teuthology.orchestra.run.smithi006.stderr: 11500K .......... .......... .......... .......... .......... 12% 39.0M 1s 2024-05-30T16:23:29.574 INFO:teuthology.orchestra.run.smithi006.stderr: 11550K .......... .......... .......... .......... .......... 12% 39.0M 1s 2024-05-30T16:23:29.574 INFO:teuthology.orchestra.run.smithi006.stderr: 11600K .......... .......... .......... .......... .......... 12% 46.9M 1s 2024-05-30T16:23:29.574 INFO:teuthology.orchestra.run.smithi006.stderr: 11650K .......... .......... .......... .......... .......... 12% 116M 1s 2024-05-30T16:23:29.574 INFO:teuthology.orchestra.run.smithi006.stderr: 11700K .......... .......... .......... .......... .......... 12% 64.8M 1s 2024-05-30T16:23:29.574 INFO:teuthology.orchestra.run.smithi006.stderr: 11750K .......... .......... .......... .......... .......... 12% 54.5M 1s 2024-05-30T16:23:29.574 INFO:teuthology.orchestra.run.smithi006.stderr: 11800K .......... .......... .......... .......... .......... 12% 54.6M 1s 2024-05-30T16:23:29.574 INFO:teuthology.orchestra.run.smithi006.stderr: 11850K .......... .......... .......... .......... .......... 12% 68.4M 1s 2024-05-30T16:23:29.575 INFO:teuthology.orchestra.run.smithi006.stderr: 11900K .......... .......... .......... .......... .......... 12% 102M 1s 2024-05-30T16:23:29.575 INFO:teuthology.orchestra.run.smithi006.stderr: 11950K .......... .......... .......... .......... .......... 12% 102M 1s 2024-05-30T16:23:29.575 INFO:teuthology.orchestra.run.smithi006.stderr: 12000K .......... .......... .......... .......... .......... 12% 119M 1s 2024-05-30T16:23:29.575 INFO:teuthology.orchestra.run.smithi006.stderr: 12050K .......... .......... .......... .......... .......... 13% 109M 1s 2024-05-30T16:23:29.575 INFO:teuthology.orchestra.run.smithi006.stderr: 12100K .......... .......... .......... .......... .......... 13% 87.2M 1s 2024-05-30T16:23:29.575 INFO:teuthology.orchestra.run.smithi006.stderr: 12150K .......... .......... .......... .......... .......... 13% 104M 1s 2024-05-30T16:23:29.575 INFO:teuthology.orchestra.run.smithi006.stderr: 12200K .......... .......... .......... .......... .......... 13% 122M 1s 2024-05-30T16:23:29.575 INFO:teuthology.orchestra.run.smithi006.stderr: 12250K .......... .......... .......... .......... .......... 13% 119M 1s 2024-05-30T16:23:29.575 INFO:teuthology.orchestra.run.smithi006.stderr: 12300K .......... .......... .......... .......... .......... 13% 100M 1s 2024-05-30T16:23:29.575 INFO:teuthology.orchestra.run.smithi006.stderr: 12350K .......... .......... .......... .......... .......... 13% 123M 1s 2024-05-30T16:23:29.575 INFO:teuthology.orchestra.run.smithi006.stderr: 12400K .......... .......... .......... .......... .......... 13% 106M 1s 2024-05-30T16:23:29.575 INFO:teuthology.orchestra.run.smithi006.stderr: 12450K .......... .......... .......... .......... .......... 13% 103M 1s 2024-05-30T16:23:29.575 INFO:teuthology.orchestra.run.smithi006.stderr: 12500K .......... .......... .......... .......... .......... 13% 101M 1s 2024-05-30T16:23:29.575 INFO:teuthology.orchestra.run.smithi006.stderr: 12550K .......... .......... .......... .......... .......... 13% 59.6M 1s 2024-05-30T16:23:29.576 INFO:teuthology.orchestra.run.smithi006.stderr: 12600K .......... .......... .......... .......... .......... 13% 88.2M 1s 2024-05-30T16:23:29.576 INFO:teuthology.orchestra.run.smithi006.stderr: 12650K .......... .......... .......... .......... .......... 13% 28.5M 1s 2024-05-30T16:23:29.576 INFO:teuthology.orchestra.run.smithi006.stderr: 12700K .......... .......... .......... .......... .......... 13% 78.6M 1s 2024-05-30T16:23:29.576 INFO:teuthology.orchestra.run.smithi006.stderr: 12750K .......... .......... .......... .......... .......... 13% 101M 1s 2024-05-30T16:23:29.576 INFO:teuthology.orchestra.run.smithi006.stderr: 12800K .......... .......... .......... .......... .......... 13% 129M 1s 2024-05-30T16:23:29.576 INFO:teuthology.orchestra.run.smithi006.stderr: 12850K .......... .......... .......... .......... .......... 13% 124M 1s 2024-05-30T16:23:29.576 INFO:teuthology.orchestra.run.smithi006.stderr: 12900K .......... .......... .......... .......... .......... 13% 81.9M 1s 2024-05-30T16:23:29.576 INFO:teuthology.orchestra.run.smithi006.stderr: 12950K .......... .......... .......... .......... .......... 13% 111M 1s 2024-05-30T16:23:29.576 INFO:teuthology.orchestra.run.smithi006.stderr: 13000K .......... .......... .......... .......... .......... 14% 112M 1s 2024-05-30T16:23:29.576 INFO:teuthology.orchestra.run.smithi006.stderr: 13050K .......... .......... .......... .......... .......... 14% 119M 1s 2024-05-30T16:23:29.576 INFO:teuthology.orchestra.run.smithi006.stderr: 13100K .......... .......... .......... .......... .......... 14% 117M 1s 2024-05-30T16:23:29.576 INFO:teuthology.orchestra.run.smithi006.stderr: 13150K .......... .......... .......... .......... .......... 14% 114M 1s 2024-05-30T16:23:29.576 INFO:teuthology.orchestra.run.smithi006.stderr: 13200K .......... .......... .......... .......... .......... 14% 74.7M 1s 2024-05-30T16:23:29.576 INFO:teuthology.orchestra.run.smithi006.stderr: 13250K .......... .......... .......... .......... .......... 14% 126M 1s 2024-05-30T16:23:29.577 INFO:teuthology.orchestra.run.smithi006.stderr: 13300K .......... .......... .......... .......... .......... 14% 119M 1s 2024-05-30T16:23:29.577 INFO:teuthology.orchestra.run.smithi006.stderr: 13350K .......... .......... .......... .......... .......... 14% 104M 1s 2024-05-30T16:23:29.577 INFO:teuthology.orchestra.run.smithi006.stderr: 13400K .......... .......... .......... .......... .......... 14% 113M 1s 2024-05-30T16:23:29.577 INFO:teuthology.orchestra.run.smithi006.stderr: 13450K .......... .......... .......... .......... .......... 14% 130M 1s 2024-05-30T16:23:29.577 INFO:teuthology.orchestra.run.smithi006.stderr: 13500K .......... .......... .......... .......... .......... 14% 121M 1s 2024-05-30T16:23:29.577 INFO:teuthology.orchestra.run.smithi006.stderr: 13550K .......... .......... .......... .......... .......... 14% 104M 1s 2024-05-30T16:23:29.577 INFO:teuthology.orchestra.run.smithi006.stderr: 13600K .......... .......... .......... .......... .......... 14% 66.4M 1s 2024-05-30T16:23:29.577 INFO:teuthology.orchestra.run.smithi006.stderr: 13650K .......... .......... .......... .......... .......... 14% 122M 1s 2024-05-30T16:23:29.577 INFO:teuthology.orchestra.run.smithi006.stderr: 13700K .......... .......... .......... .......... .......... 14% 108M 1s 2024-05-30T16:23:29.577 INFO:teuthology.orchestra.run.smithi006.stderr: 13750K .......... .......... .......... .......... .......... 14% 121M 1s 2024-05-30T16:23:29.577 INFO:teuthology.orchestra.run.smithi006.stderr: 13800K .......... .......... .......... .......... .......... 14% 54.9M 1s 2024-05-30T16:23:29.577 INFO:teuthology.orchestra.run.smithi006.stderr: 13850K .......... .......... .......... .......... .......... 14% 79.7M 1s 2024-05-30T16:23:29.577 INFO:teuthology.orchestra.run.smithi006.stderr: 13900K .......... .......... .......... .......... .......... 15% 87.7M 1s 2024-05-30T16:23:29.577 INFO:teuthology.orchestra.run.smithi006.stderr: 13950K .......... .......... .......... .......... .......... 15% 75.8M 1s 2024-05-30T16:23:29.578 INFO:teuthology.orchestra.run.smithi006.stderr: 14000K .......... .......... .......... .......... .......... 15% 46.4M 1s 2024-05-30T16:23:29.578 INFO:teuthology.orchestra.run.smithi006.stderr: 14050K .......... .......... .......... .......... .......... 15% 84.2M 1s 2024-05-30T16:23:29.578 INFO:teuthology.orchestra.run.smithi006.stderr: 14100K .......... .......... .......... .......... .......... 15% 75.9M 1s 2024-05-30T16:23:29.578 INFO:teuthology.orchestra.run.smithi006.stderr: 14150K .......... .......... .......... .......... .......... 15% 101M 1s 2024-05-30T16:23:29.578 INFO:teuthology.orchestra.run.smithi006.stderr: 14200K .......... .......... .......... .......... .......... 15% 102M 1s 2024-05-30T16:23:29.578 INFO:teuthology.orchestra.run.smithi006.stderr: 14250K .......... .......... .......... .......... .......... 15% 115M 1s 2024-05-30T16:23:29.578 INFO:teuthology.orchestra.run.smithi006.stderr: 14300K .......... .......... .......... .......... .......... 15% 111M 1s 2024-05-30T16:23:29.578 INFO:teuthology.orchestra.run.smithi006.stderr: 14350K .......... .......... .......... .......... .......... 15% 118M 1s 2024-05-30T16:23:29.578 INFO:teuthology.orchestra.run.smithi006.stderr: 14400K .......... .......... .......... .......... .......... 15% 88.7M 1s 2024-05-30T16:23:29.578 INFO:teuthology.orchestra.run.smithi006.stderr: 14450K .......... .......... .......... .......... .......... 15% 114M 1s 2024-05-30T16:23:29.578 INFO:teuthology.orchestra.run.smithi006.stderr: 14500K .......... .......... .......... .......... .......... 15% 104M 1s 2024-05-30T16:23:29.578 INFO:teuthology.orchestra.run.smithi006.stderr: 14550K .......... .......... .......... .......... .......... 15% 130M 1s 2024-05-30T16:23:29.578 INFO:teuthology.orchestra.run.smithi006.stderr: 14600K .......... .......... .......... .......... .......... 15% 88.9M 1s 2024-05-30T16:23:29.578 INFO:teuthology.orchestra.run.smithi006.stderr: 14650K .......... .......... .......... .......... .......... 15% 117M 1s 2024-05-30T16:23:29.579 INFO:teuthology.orchestra.run.smithi006.stderr: 14700K .......... .......... .......... .......... .......... 15% 120M 1s 2024-05-30T16:23:29.579 INFO:teuthology.orchestra.run.smithi006.stderr: 14750K .......... .......... .......... .......... .......... 15% 122M 1s 2024-05-30T16:23:29.579 INFO:teuthology.orchestra.run.smithi006.stderr: 14800K .......... .......... .......... .......... .......... 15% 89.7M 1s 2024-05-30T16:23:29.579 INFO:teuthology.orchestra.run.smithi006.stderr: 14850K .......... .......... .......... .......... .......... 16% 113M 1s 2024-05-30T16:23:29.579 INFO:teuthology.orchestra.run.smithi006.stderr: 14900K .......... .......... .......... .......... .......... 16% 117M 1s 2024-05-30T16:23:29.579 INFO:teuthology.orchestra.run.smithi006.stderr: 14950K .......... .......... .......... .......... .......... 16% 68.2M 1s 2024-05-30T16:23:29.579 INFO:teuthology.orchestra.run.smithi006.stderr: 15000K .......... .......... .......... .......... .......... 16% 111M 1s 2024-05-30T16:23:29.579 INFO:teuthology.orchestra.run.smithi006.stderr: 15050K .......... .......... .......... .......... .......... 16% 124M 1s 2024-05-30T16:23:29.579 INFO:teuthology.orchestra.run.smithi006.stderr: 15100K .......... .......... .......... .......... .......... 16% 128M 1s 2024-05-30T16:23:29.579 INFO:teuthology.orchestra.run.smithi006.stderr: 15150K .......... .......... .......... .......... .......... 16% 125M 1s 2024-05-30T16:23:29.579 INFO:teuthology.orchestra.run.smithi006.stderr: 15200K .......... .......... .......... .......... .......... 16% 105M 1s 2024-05-30T16:23:29.579 INFO:teuthology.orchestra.run.smithi006.stderr: 15250K .......... .......... .......... .......... .......... 16% 111M 1s 2024-05-30T16:23:29.579 INFO:teuthology.orchestra.run.smithi006.stderr: 15300K .......... .......... .......... .......... .......... 16% 130M 1s 2024-05-30T16:23:29.579 INFO:teuthology.orchestra.run.smithi006.stderr: 15350K .......... .......... .......... .......... .......... 16% 112M 1s 2024-05-30T16:23:29.580 INFO:teuthology.orchestra.run.smithi006.stderr: 15400K .......... .......... .......... .......... .......... 16% 112M 1s 2024-05-30T16:23:29.580 INFO:teuthology.orchestra.run.smithi006.stderr: 15450K .......... .......... .......... .......... .......... 16% 111M 1s 2024-05-30T16:23:29.580 INFO:teuthology.orchestra.run.smithi006.stderr: 15500K .......... .......... .......... .......... .......... 16% 124M 1s 2024-05-30T16:23:29.580 INFO:teuthology.orchestra.run.smithi006.stderr: 15550K .......... .......... .......... .......... .......... 16% 109M 1s 2024-05-30T16:23:29.580 INFO:teuthology.orchestra.run.smithi006.stderr: 15600K .......... .......... .......... .......... .......... 16% 73.4M 1s 2024-05-30T16:23:29.580 INFO:teuthology.orchestra.run.smithi006.stderr: 15650K .......... .......... .......... .......... .......... 16% 141M 1s 2024-05-30T16:23:29.580 INFO:teuthology.orchestra.run.smithi006.stderr: 15700K .......... .......... .......... .......... .......... 16% 107M 1s 2024-05-30T16:23:29.580 INFO:teuthology.orchestra.run.smithi006.stderr: 15750K .......... .......... .......... .......... .......... 17% 123M 1s 2024-05-30T16:23:29.580 INFO:teuthology.orchestra.run.smithi006.stderr: 15800K .......... .......... .......... .......... .......... 17% 117M 1s 2024-05-30T16:23:29.580 INFO:teuthology.orchestra.run.smithi006.stderr: 15850K .......... .......... .......... .......... .......... 17% 116M 1s 2024-05-30T16:23:29.580 INFO:teuthology.orchestra.run.smithi006.stderr: 15900K .......... .......... .......... .......... .......... 17% 107M 1s 2024-05-30T16:23:29.580 INFO:teuthology.orchestra.run.smithi006.stderr: 15950K .......... .......... .......... .......... .......... 17% 113M 1s 2024-05-30T16:23:29.580 INFO:teuthology.orchestra.run.smithi006.stderr: 16000K .......... .......... .......... .......... .......... 17% 97.9M 1s 2024-05-30T16:23:29.580 INFO:teuthology.orchestra.run.smithi006.stderr: 16050K .......... .......... .......... .......... .......... 17% 112M 1s 2024-05-30T16:23:29.581 INFO:teuthology.orchestra.run.smithi006.stderr: 16100K .......... .......... .......... .......... .......... 17% 114M 1s 2024-05-30T16:23:29.581 INFO:teuthology.orchestra.run.smithi006.stderr: 16150K .......... .......... .......... .......... .......... 17% 88.7M 1s 2024-05-30T16:23:29.581 INFO:teuthology.orchestra.run.smithi006.stderr: 16200K .......... .......... .......... .......... .......... 17% 102M 1s 2024-05-30T16:23:29.581 INFO:teuthology.orchestra.run.smithi006.stderr: 16250K .......... .......... .......... .......... .......... 17% 140M 1s 2024-05-30T16:23:29.581 INFO:teuthology.orchestra.run.smithi006.stderr: 16300K .......... .......... .......... .......... .......... 17% 123M 1s 2024-05-30T16:23:29.581 INFO:teuthology.orchestra.run.smithi006.stderr: 16350K .......... .......... .......... .......... .......... 17% 77.8M 1s 2024-05-30T16:23:29.581 INFO:teuthology.orchestra.run.smithi006.stderr: 16400K .......... .......... .......... .......... .......... 17% 101M 1s 2024-05-30T16:23:29.581 INFO:teuthology.orchestra.run.smithi006.stderr: 16450K .......... .......... .......... .......... .......... 17% 118M 1s 2024-05-30T16:23:29.581 INFO:teuthology.orchestra.run.smithi006.stderr: 16500K .......... .......... .......... .......... .......... 17% 112M 1s 2024-05-30T16:23:29.581 INFO:teuthology.orchestra.run.smithi006.stderr: 16550K .......... .......... .......... .......... .......... 17% 111M 1s 2024-05-30T16:23:29.581 INFO:teuthology.orchestra.run.smithi006.stderr: 16600K .......... .......... .......... .......... .......... 17% 105M 1s 2024-05-30T16:23:29.581 INFO:teuthology.orchestra.run.smithi006.stderr: 16650K .......... .......... .......... .......... .......... 17% 111M 1s 2024-05-30T16:23:29.581 INFO:teuthology.orchestra.run.smithi006.stderr: 16700K .......... .......... .......... .......... .......... 18% 114M 1s 2024-05-30T16:23:29.581 INFO:teuthology.orchestra.run.smithi006.stderr: 16750K .......... .......... .......... .......... .......... 18% 117M 1s 2024-05-30T16:23:29.582 INFO:teuthology.orchestra.run.smithi006.stderr: 16800K .......... .......... .......... .......... .......... 18% 95.4M 1s 2024-05-30T16:23:29.582 INFO:teuthology.orchestra.run.smithi006.stderr: 16850K .......... .......... .......... .......... .......... 18% 101M 1s 2024-05-30T16:23:29.582 INFO:teuthology.orchestra.run.smithi006.stderr: 16900K .......... .......... .......... .......... .......... 18% 115M 1s 2024-05-30T16:23:29.582 INFO:teuthology.orchestra.run.smithi006.stderr: 16950K .......... .......... .......... .......... .......... 18% 106M 1s 2024-05-30T16:23:29.582 INFO:teuthology.orchestra.run.smithi006.stderr: 17000K .......... .......... .......... .......... .......... 18% 116M 1s 2024-05-30T16:23:29.582 INFO:teuthology.orchestra.run.smithi006.stderr: 17050K .......... .......... .......... .......... .......... 18% 109M 1s 2024-05-30T16:23:29.582 INFO:teuthology.orchestra.run.smithi006.stderr: 17100K .......... .......... .......... .......... .......... 18% 105M 1s 2024-05-30T16:23:29.582 INFO:teuthology.orchestra.run.smithi006.stderr: 17150K .......... .......... .......... .......... .......... 18% 119M 1s 2024-05-30T16:23:29.582 INFO:teuthology.orchestra.run.smithi006.stderr: 17200K .......... .......... .......... .......... .......... 18% 104M 1s 2024-05-30T16:23:29.582 INFO:teuthology.orchestra.run.smithi006.stderr: 17250K .......... .......... .......... .......... .......... 18% 96.5M 1s 2024-05-30T16:23:29.582 INFO:teuthology.orchestra.run.smithi006.stderr: 17300K .......... .......... .......... .......... .......... 18% 117M 1s 2024-05-30T16:23:29.582 INFO:teuthology.orchestra.run.smithi006.stderr: 17350K .......... .......... .......... .......... .......... 18% 117M 1s 2024-05-30T16:23:29.583 INFO:teuthology.orchestra.run.smithi006.stderr: 17400K .......... .......... .......... .......... .......... 18% 120M 1s 2024-05-30T16:23:29.583 INFO:teuthology.orchestra.run.smithi006.stderr: 17450K .......... .......... .......... .......... .......... 18% 101M 1s 2024-05-30T16:23:29.583 INFO:teuthology.orchestra.run.smithi006.stderr: 17500K .......... .......... .......... .......... .......... 18% 117M 1s 2024-05-30T16:23:29.583 INFO:teuthology.orchestra.run.smithi006.stderr: 17550K .......... .......... .......... .......... .......... 18% 106M 1s 2024-05-30T16:23:29.583 INFO:teuthology.orchestra.run.smithi006.stderr: 17600K .......... .......... .......... .......... .......... 18% 98.5M 1s 2024-05-30T16:23:29.583 INFO:teuthology.orchestra.run.smithi006.stderr: 17650K .......... .......... .......... .......... .......... 19% 92.3M 1s 2024-05-30T16:23:29.583 INFO:teuthology.orchestra.run.smithi006.stderr: 17700K .......... .......... .......... .......... .......... 19% 118M 1s 2024-05-30T16:23:29.583 INFO:teuthology.orchestra.run.smithi006.stderr: 17750K .......... .......... .......... .......... .......... 19% 123M 1s 2024-05-30T16:23:29.583 INFO:teuthology.orchestra.run.smithi006.stderr: 17800K .......... .......... .......... .......... .......... 19% 121M 1s 2024-05-30T16:23:29.583 INFO:teuthology.orchestra.run.smithi006.stderr: 17850K .......... .......... .......... .......... .......... 19% 75.7M 1s 2024-05-30T16:23:29.583 INFO:teuthology.orchestra.run.smithi006.stderr: 17900K .......... .......... .......... .......... .......... 19% 94.4M 1s 2024-05-30T16:23:29.583 INFO:teuthology.orchestra.run.smithi006.stderr: 17950K .......... .......... .......... .......... .......... 19% 124M 1s 2024-05-30T16:23:29.583 INFO:teuthology.orchestra.run.smithi006.stderr: 18000K .......... .......... .......... .......... .......... 19% 110M 1s 2024-05-30T16:23:29.584 INFO:teuthology.orchestra.run.smithi006.stderr: 18050K .......... .......... .......... .......... .......... 19% 91.7M 1s 2024-05-30T16:23:29.584 INFO:teuthology.orchestra.run.smithi006.stderr: 18100K .......... .......... .......... .......... .......... 19% 123M 1s 2024-05-30T16:23:29.584 INFO:teuthology.orchestra.run.smithi006.stderr: 18150K .......... .......... .......... .......... .......... 19% 105M 1s 2024-05-30T16:23:29.584 INFO:teuthology.orchestra.run.smithi006.stderr: 18200K .......... .......... .......... .......... .......... 19% 110M 1s 2024-05-30T16:23:29.584 INFO:teuthology.orchestra.run.smithi006.stderr: 18250K .......... .......... .......... .......... .......... 19% 109M 1s 2024-05-30T16:23:29.584 INFO:teuthology.orchestra.run.smithi006.stderr: 18300K .......... .......... .......... .......... .......... 19% 122M 1s 2024-05-30T16:23:29.584 INFO:teuthology.orchestra.run.smithi006.stderr: 18350K .......... .......... .......... .......... .......... 19% 107M 1s 2024-05-30T16:23:29.584 INFO:teuthology.orchestra.run.smithi006.stderr: 18400K .......... .......... .......... .......... .......... 19% 113M 1s 2024-05-30T16:23:29.584 INFO:teuthology.orchestra.run.smithi006.stderr: 18450K .......... .......... .......... .......... .......... 19% 93.7M 1s 2024-05-30T16:23:29.584 INFO:teuthology.orchestra.run.smithi006.stderr: 18500K .......... .......... .......... .......... .......... 19% 114M 1s 2024-05-30T16:23:29.584 INFO:teuthology.orchestra.run.smithi006.stderr: 18550K .......... .......... .......... .......... .......... 20% 113M 1s 2024-05-30T16:23:29.584 INFO:teuthology.orchestra.run.smithi006.stderr: 18600K .......... .......... .......... .......... .......... 20% 111M 1s 2024-05-30T16:23:29.584 INFO:teuthology.orchestra.run.smithi006.stderr: 18650K .......... .......... .......... .......... .......... 20% 103M 1s 2024-05-30T16:23:29.584 INFO:teuthology.orchestra.run.smithi006.stderr: 18700K .......... .......... .......... .......... .......... 20% 114M 1s 2024-05-30T16:23:29.585 INFO:teuthology.orchestra.run.smithi006.stderr: 18750K .......... .......... .......... .......... .......... 20% 111M 1s 2024-05-30T16:23:29.585 INFO:teuthology.orchestra.run.smithi006.stderr: 18800K .......... .......... .......... .......... .......... 20% 96.7M 1s 2024-05-30T16:23:29.585 INFO:teuthology.orchestra.run.smithi006.stderr: 18850K .......... .......... .......... .......... .......... 20% 96.5M 1s 2024-05-30T16:23:29.585 INFO:teuthology.orchestra.run.smithi006.stderr: 18900K .......... .......... .......... .......... .......... 20% 112M 1s 2024-05-30T16:23:29.585 INFO:teuthology.orchestra.run.smithi006.stderr: 18950K .......... .......... .......... .......... .......... 20% 95.3M 1s 2024-05-30T16:23:29.585 INFO:teuthology.orchestra.run.smithi006.stderr: 19000K .......... .......... .......... .......... .......... 20% 116M 1s 2024-05-30T16:23:29.585 INFO:teuthology.orchestra.run.smithi006.stderr: 19050K .......... .......... .......... .......... .......... 20% 133M 1s 2024-05-30T16:23:29.585 INFO:teuthology.orchestra.run.smithi006.stderr: 19100K .......... .......... .......... .......... .......... 20% 131M 1s 2024-05-30T16:23:29.585 INFO:teuthology.orchestra.run.smithi006.stderr: 19150K .......... .......... .......... .......... .......... 20% 90.2M 1s 2024-05-30T16:23:29.585 INFO:teuthology.orchestra.run.smithi006.stderr: 19200K .......... .......... .......... .......... .......... 20% 118M 1s 2024-05-30T16:23:29.585 INFO:teuthology.orchestra.run.smithi006.stderr: 19250K .......... .......... .......... .......... .......... 20% 107M 1s 2024-05-30T16:23:29.585 INFO:teuthology.orchestra.run.smithi006.stderr: 19300K .......... .......... .......... .......... .......... 20% 58.2M 1s 2024-05-30T16:23:29.585 INFO:teuthology.orchestra.run.smithi006.stderr: 19350K .......... .......... .......... .......... .......... 20% 91.6M 1s 2024-05-30T16:23:29.585 INFO:teuthology.orchestra.run.smithi006.stderr: 19400K .......... .......... .......... .......... .......... 20% 127M 1s 2024-05-30T16:23:29.586 INFO:teuthology.orchestra.run.smithi006.stderr: 19450K .......... .......... .......... .......... .......... 20% 122M 1s 2024-05-30T16:23:29.586 INFO:teuthology.orchestra.run.smithi006.stderr: 19500K .......... .......... .......... .......... .......... 21% 101M 1s 2024-05-30T16:23:29.586 INFO:teuthology.orchestra.run.smithi006.stderr: 19550K .......... .......... .......... .......... .......... 21% 96.6M 1s 2024-05-30T16:23:29.586 INFO:teuthology.orchestra.run.smithi006.stderr: 19600K .......... .......... .......... .......... .......... 21% 116M 1s 2024-05-30T16:23:29.586 INFO:teuthology.orchestra.run.smithi006.stderr: 19650K .......... .......... .......... .......... .......... 21% 96.5M 1s 2024-05-30T16:23:29.586 INFO:teuthology.orchestra.run.smithi006.stderr: 19700K .......... .......... .......... .......... .......... 21% 135M 1s 2024-05-30T16:23:29.586 INFO:teuthology.orchestra.run.smithi006.stderr: 19750K .......... .......... .......... .......... .......... 21% 105M 1s 2024-05-30T16:23:29.586 INFO:teuthology.orchestra.run.smithi006.stderr: 19800K .......... .......... .......... .......... .......... 21% 122M 1s 2024-05-30T16:23:29.586 INFO:teuthology.orchestra.run.smithi006.stderr: 19850K .......... .......... .......... .......... .......... 21% 103M 1s 2024-05-30T16:23:29.586 INFO:teuthology.orchestra.run.smithi006.stderr: 19900K .......... .......... .......... .......... .......... 21% 101M 1s 2024-05-30T16:23:29.586 INFO:teuthology.orchestra.run.smithi006.stderr: 19950K .......... .......... .......... .......... .......... 21% 108M 1s 2024-05-30T16:23:29.586 INFO:teuthology.orchestra.run.smithi006.stderr: 20000K .......... .......... .......... .......... .......... 21% 104M 1s 2024-05-30T16:23:29.586 INFO:teuthology.orchestra.run.smithi006.stderr: 20050K .......... .......... .......... .......... .......... 21% 130M 1s 2024-05-30T16:23:29.586 INFO:teuthology.orchestra.run.smithi006.stderr: 20100K .......... .......... .......... .......... .......... 21% 109M 1s 2024-05-30T16:23:29.587 INFO:teuthology.orchestra.run.smithi006.stderr: 20150K .......... .......... .......... .......... .......... 21% 106M 1s 2024-05-30T16:23:29.587 INFO:teuthology.orchestra.run.smithi006.stderr: 20200K .......... .......... .......... .......... .......... 21% 103M 1s 2024-05-30T16:23:29.587 INFO:teuthology.orchestra.run.smithi006.stderr: 20250K .......... .......... .......... .......... .......... 21% 111M 1s 2024-05-30T16:23:29.587 INFO:teuthology.orchestra.run.smithi006.stderr: 20300K .......... .......... .......... .......... .......... 21% 102M 1s 2024-05-30T16:23:29.587 INFO:teuthology.orchestra.run.smithi006.stderr: 20350K .......... .......... .......... .......... .......... 21% 87.3M 1s 2024-05-30T16:23:29.587 INFO:teuthology.orchestra.run.smithi006.stderr: 20400K .......... .......... .......... .......... .......... 22% 148M 1s 2024-05-30T16:23:29.587 INFO:teuthology.orchestra.run.smithi006.stderr: 20450K .......... .......... .......... .......... .......... 22% 131M 1s 2024-05-30T16:23:29.587 INFO:teuthology.orchestra.run.smithi006.stderr: 20500K .......... .......... .......... .......... .......... 22% 114M 1s 2024-05-30T16:23:29.587 INFO:teuthology.orchestra.run.smithi006.stderr: 20550K .......... .......... .......... .......... .......... 22% 105M 1s 2024-05-30T16:23:29.587 INFO:teuthology.orchestra.run.smithi006.stderr: 20600K .......... .......... .......... .......... .......... 22% 94.5M 1s 2024-05-30T16:23:29.587 INFO:teuthology.orchestra.run.smithi006.stderr: 20650K .......... .......... .......... .......... .......... 22% 129M 1s 2024-05-30T16:23:29.587 INFO:teuthology.orchestra.run.smithi006.stderr: 20700K .......... .......... .......... .......... .......... 22% 120M 1s 2024-05-30T16:23:29.587 INFO:teuthology.orchestra.run.smithi006.stderr: 20750K .......... .......... .......... .......... .......... 22% 88.8M 1s 2024-05-30T16:23:29.588 INFO:teuthology.orchestra.run.smithi006.stderr: 20800K .......... .......... .......... .......... .......... 22% 111M 1s 2024-05-30T16:23:29.588 INFO:teuthology.orchestra.run.smithi006.stderr: 20850K .......... .......... .......... .......... .......... 22% 128M 1s 2024-05-30T16:23:29.588 INFO:teuthology.orchestra.run.smithi006.stderr: 20900K .......... .......... .......... .......... .......... 22% 97.3M 1s 2024-05-30T16:23:29.588 INFO:teuthology.orchestra.run.smithi006.stderr: 20950K .......... .......... .......... .......... .......... 22% 110M 1s 2024-05-30T16:23:29.588 INFO:teuthology.orchestra.run.smithi006.stderr: 21000K .......... .......... .......... .......... .......... 22% 130M 1s 2024-05-30T16:23:29.588 INFO:teuthology.orchestra.run.smithi006.stderr: 21050K .......... .......... .......... .......... .......... 22% 114M 1s 2024-05-30T16:23:29.588 INFO:teuthology.orchestra.run.smithi006.stderr: 21100K .......... .......... .......... .......... .......... 22% 106M 1s 2024-05-30T16:23:29.588 INFO:teuthology.orchestra.run.smithi006.stderr: 21150K .......... .......... .......... .......... .......... 22% 104M 1s 2024-05-30T16:23:29.588 INFO:teuthology.orchestra.run.smithi006.stderr: 21200K .......... .......... .......... .......... .......... 22% 106M 1s 2024-05-30T16:23:29.588 INFO:teuthology.orchestra.run.smithi006.stderr: 21250K .......... .......... .......... .......... .......... 22% 118M 1s 2024-05-30T16:23:29.588 INFO:teuthology.orchestra.run.smithi006.stderr: 21300K .......... .......... .......... .......... .......... 22% 104M 1s 2024-05-30T16:23:29.588 INFO:teuthology.orchestra.run.smithi006.stderr: 21350K .......... .......... .......... .......... .......... 23% 97.9M 1s 2024-05-30T16:23:29.588 INFO:teuthology.orchestra.run.smithi006.stderr: 21400K .......... .......... .......... .......... .......... 23% 51.0M 1s 2024-05-30T16:23:29.588 INFO:teuthology.orchestra.run.smithi006.stderr: 21450K .......... .......... .......... .......... .......... 23% 25.8M 1s 2024-05-30T16:23:29.589 INFO:teuthology.orchestra.run.smithi006.stderr: 21500K .......... .......... .......... .......... .......... 23% 120M 1s 2024-05-30T16:23:29.589 INFO:teuthology.orchestra.run.smithi006.stderr: 21550K .......... .......... .......... .......... .......... 23% 105M 1s 2024-05-30T16:23:29.589 INFO:teuthology.orchestra.run.smithi006.stderr: 21600K .......... .......... .......... .......... .......... 23% 127M 1s 2024-05-30T16:23:29.589 INFO:teuthology.orchestra.run.smithi006.stderr: 21650K .......... .......... .......... .......... .......... 23% 61.4M 1s 2024-05-30T16:23:29.589 INFO:teuthology.orchestra.run.smithi006.stderr: 21700K .......... .......... .......... .......... .......... 23% 98.2M 1s 2024-05-30T16:23:29.589 INFO:teuthology.orchestra.run.smithi006.stderr: 21750K .......... .......... .......... .......... .......... 23% 92.7M 1s 2024-05-30T16:23:29.589 INFO:teuthology.orchestra.run.smithi006.stderr: 21800K .......... .......... .......... .......... .......... 23% 113M 1s 2024-05-30T16:23:29.590 INFO:teuthology.orchestra.run.smithi006.stderr: 21850K .......... .......... .......... .......... .......... 23% 89.4M 1s 2024-05-30T16:23:29.590 INFO:teuthology.orchestra.run.smithi006.stderr: 21900K .......... .......... .......... .......... .......... 23% 121M 1s 2024-05-30T16:23:29.590 INFO:teuthology.orchestra.run.smithi006.stderr: 21950K .......... .......... .......... .......... .......... 23% 114M 1s 2024-05-30T16:23:29.590 INFO:teuthology.orchestra.run.smithi006.stderr: 22000K .......... .......... .......... .......... .......... 23% 113M 1s 2024-05-30T16:23:29.590 INFO:teuthology.orchestra.run.smithi006.stderr: 22050K .......... .......... .......... .......... .......... 23% 110M 1s 2024-05-30T16:23:29.590 INFO:teuthology.orchestra.run.smithi006.stderr: 22100K .......... .......... .......... .......... .......... 23% 80.8M 1s 2024-05-30T16:23:29.590 INFO:teuthology.orchestra.run.smithi006.stderr: 22150K .......... .......... .......... .......... .......... 23% 121M 1s 2024-05-30T16:23:29.590 INFO:teuthology.orchestra.run.smithi006.stderr: 22200K .......... .......... .......... .......... .......... 23% 135M 1s 2024-05-30T16:23:29.590 INFO:teuthology.orchestra.run.smithi006.stderr: 22250K .......... .......... .......... .......... .......... 24% 95.1M 1s 2024-05-30T16:23:29.590 INFO:teuthology.orchestra.run.smithi006.stderr: 22300K .......... .......... .......... .......... .......... 24% 123M 1s 2024-05-30T16:23:29.590 INFO:teuthology.orchestra.run.smithi006.stderr: 22350K .......... .......... .......... .......... .......... 24% 110M 1s 2024-05-30T16:23:29.590 INFO:teuthology.orchestra.run.smithi006.stderr: 22400K .......... .......... .......... .......... .......... 24% 105M 1s 2024-05-30T16:23:29.590 INFO:teuthology.orchestra.run.smithi006.stderr: 22450K .......... .......... .......... .......... .......... 24% 110M 1s 2024-05-30T16:23:29.590 INFO:teuthology.orchestra.run.smithi006.stderr: 22500K .......... .......... .......... .......... .......... 24% 112M 1s 2024-05-30T16:23:29.591 INFO:teuthology.orchestra.run.smithi006.stderr: 22550K .......... .......... .......... .......... .......... 24% 121M 1s 2024-05-30T16:23:29.591 INFO:teuthology.orchestra.run.smithi006.stderr: 22600K .......... .......... .......... .......... .......... 24% 125M 1s 2024-05-30T16:23:29.591 INFO:teuthology.orchestra.run.smithi006.stderr: 22650K .......... .......... .......... .......... .......... 24% 89.8M 1s 2024-05-30T16:23:29.591 INFO:teuthology.orchestra.run.smithi006.stderr: 22700K .......... .......... .......... .......... .......... 24% 114M 1s 2024-05-30T16:23:29.591 INFO:teuthology.orchestra.run.smithi006.stderr: 22750K .......... .......... .......... .......... .......... 24% 96.4M 1s 2024-05-30T16:23:29.591 INFO:teuthology.orchestra.run.smithi006.stderr: 22800K .......... .......... .......... .......... .......... 24% 112M 1s 2024-05-30T16:23:29.591 INFO:teuthology.orchestra.run.smithi006.stderr: 22850K .......... .......... .......... .......... .......... 24% 109M 1s 2024-05-30T16:23:29.591 INFO:teuthology.orchestra.run.smithi006.stderr: 22900K .......... .......... .......... .......... .......... 24% 40.4M 1s 2024-05-30T16:23:29.591 INFO:teuthology.orchestra.run.smithi006.stderr: 22950K .......... .......... .......... .......... .......... 24% 42.7M 1s 2024-05-30T16:23:29.591 INFO:teuthology.orchestra.run.smithi006.stderr: 23000K .......... .......... .......... .......... .......... 24% 113M 1s 2024-05-30T16:23:29.591 INFO:teuthology.orchestra.run.smithi006.stderr: 23050K .......... .......... .......... .......... .......... 24% 80.0M 1s 2024-05-30T16:23:29.591 INFO:teuthology.orchestra.run.smithi006.stderr: 23100K .......... .......... .......... .......... .......... 24% 111M 1s 2024-05-30T16:23:29.591 INFO:teuthology.orchestra.run.smithi006.stderr: 23150K .......... .......... .......... .......... .......... 24% 102M 1s 2024-05-30T16:23:29.592 INFO:teuthology.orchestra.run.smithi006.stderr: 23200K .......... .......... .......... .......... .......... 25% 16.3M 1s 2024-05-30T16:23:29.592 INFO:teuthology.orchestra.run.smithi006.stderr: 23250K .......... .......... .......... .......... .......... 25% 102M 1s 2024-05-30T16:23:29.592 INFO:teuthology.orchestra.run.smithi006.stderr: 23300K .......... .......... .......... .......... .......... 25% 117M 1s 2024-05-30T16:23:29.592 INFO:teuthology.orchestra.run.smithi006.stderr: 23350K .......... .......... .......... .......... .......... 25% 120M 1s 2024-05-30T16:23:29.592 INFO:teuthology.orchestra.run.smithi006.stderr: 23400K .......... .......... .......... .......... .......... 25% 110M 1s 2024-05-30T16:23:29.592 INFO:teuthology.orchestra.run.smithi006.stderr: 23450K .......... .......... .......... .......... .......... 25% 98.8M 1s 2024-05-30T16:23:29.592 INFO:teuthology.orchestra.run.smithi006.stderr: 23500K .......... .......... .......... .......... .......... 25% 108M 1s 2024-05-30T16:23:29.592 INFO:teuthology.orchestra.run.smithi006.stderr: 23550K .......... .......... .......... .......... .......... 25% 114M 1s 2024-05-30T16:23:29.592 INFO:teuthology.orchestra.run.smithi006.stderr: 23600K .......... .......... .......... .......... .......... 25% 69.7M 1s 2024-05-30T16:23:29.592 INFO:teuthology.orchestra.run.smithi006.stderr: 23650K .......... .......... .......... .......... .......... 25% 11.0M 1s 2024-05-30T16:23:29.592 INFO:teuthology.orchestra.run.smithi006.stderr: 23700K .......... .......... .......... .......... .......... 25% 83.6M 1s 2024-05-30T16:23:29.592 INFO:teuthology.orchestra.run.smithi006.stderr: 23750K .......... .......... .......... .......... .......... 25% 87.3M 1s 2024-05-30T16:23:29.592 INFO:teuthology.orchestra.run.smithi006.stderr: 23800K .......... .......... .......... .......... .......... 25% 117M 1s 2024-05-30T16:23:29.592 INFO:teuthology.orchestra.run.smithi006.stderr: 23850K .......... .......... .......... .......... .......... 25% 115M 1s 2024-05-30T16:23:29.593 INFO:teuthology.orchestra.run.smithi006.stderr: 23900K .......... .......... .......... .......... .......... 25% 110M 1s 2024-05-30T16:23:29.593 INFO:teuthology.orchestra.run.smithi006.stderr: 23950K .......... .......... .......... .......... .......... 25% 101M 1s 2024-05-30T16:23:29.593 INFO:teuthology.orchestra.run.smithi006.stderr: 24000K .......... .......... .......... .......... .......... 25% 124M 1s 2024-05-30T16:23:29.593 INFO:teuthology.orchestra.run.smithi006.stderr: 24050K .......... .......... .......... .......... .......... 25% 114M 1s 2024-05-30T16:23:29.593 INFO:teuthology.orchestra.run.smithi006.stderr: 24100K .......... .......... .......... .......... .......... 25% 113M 1s 2024-05-30T16:23:29.593 INFO:teuthology.orchestra.run.smithi006.stderr: 24150K .......... .......... .......... .......... .......... 26% 89.9M 1s 2024-05-30T16:23:29.593 INFO:teuthology.orchestra.run.smithi006.stderr: 24200K .......... .......... .......... .......... .......... 26% 114M 1s 2024-05-30T16:23:29.593 INFO:teuthology.orchestra.run.smithi006.stderr: 24250K .......... .......... .......... .......... .......... 26% 94.4M 1s 2024-05-30T16:23:29.593 INFO:teuthology.orchestra.run.smithi006.stderr: 24300K .......... .......... .......... .......... .......... 26% 131M 1s 2024-05-30T16:23:29.593 INFO:teuthology.orchestra.run.smithi006.stderr: 24350K .......... .......... .......... .......... .......... 26% 114M 1s 2024-05-30T16:23:29.593 INFO:teuthology.orchestra.run.smithi006.stderr: 24400K .......... .......... .......... .......... .......... 26% 112M 1s 2024-05-30T16:23:29.593 INFO:teuthology.orchestra.run.smithi006.stderr: 24450K .......... .......... .......... .......... .......... 26% 106M 1s 2024-05-30T16:23:29.593 INFO:teuthology.orchestra.run.smithi006.stderr: 24500K .......... .......... .......... .......... .......... 26% 121M 1s 2024-05-30T16:23:29.593 INFO:teuthology.orchestra.run.smithi006.stderr: 24550K .......... .......... .......... .......... .......... 26% 86.6M 1s 2024-05-30T16:23:29.594 INFO:teuthology.orchestra.run.smithi006.stderr: 24600K .......... .......... .......... .......... .......... 26% 117M 1s 2024-05-30T16:23:29.594 INFO:teuthology.orchestra.run.smithi006.stderr: 24650K .......... .......... .......... .......... .......... 26% 107M 1s 2024-05-30T16:23:29.594 INFO:teuthology.orchestra.run.smithi006.stderr: 24700K .......... .......... .......... .......... .......... 26% 118M 1s 2024-05-30T16:23:29.594 INFO:teuthology.orchestra.run.smithi006.stderr: 24750K .......... .......... .......... .......... .......... 26% 111M 1s 2024-05-30T16:23:29.594 INFO:teuthology.orchestra.run.smithi006.stderr: 24800K .......... .......... .......... .......... .......... 26% 108M 1s 2024-05-30T16:23:29.594 INFO:teuthology.orchestra.run.smithi006.stderr: 24850K .......... .......... .......... .......... .......... 26% 120M 1s 2024-05-30T16:23:29.594 INFO:teuthology.orchestra.run.smithi006.stderr: 24900K .......... .......... .......... .......... .......... 26% 115M 1s 2024-05-30T16:23:29.594 INFO:teuthology.orchestra.run.smithi006.stderr: 24950K .......... .......... .......... .......... .......... 26% 93.6M 1s 2024-05-30T16:23:29.594 INFO:teuthology.orchestra.run.smithi006.stderr: 25000K .......... .......... .......... .......... .......... 26% 111M 1s 2024-05-30T16:23:29.594 INFO:teuthology.orchestra.run.smithi006.stderr: 25050K .......... .......... .......... .......... .......... 27% 109M 1s 2024-05-30T16:23:29.594 INFO:teuthology.orchestra.run.smithi006.stderr: 25100K .......... .......... .......... .......... .......... 27% 115M 1s 2024-05-30T16:23:29.594 INFO:teuthology.orchestra.run.smithi006.stderr: 25150K .......... .......... .......... .......... .......... 27% 101M 1s 2024-05-30T16:23:29.595 INFO:teuthology.orchestra.run.smithi006.stderr: 25200K .......... .......... .......... .......... .......... 27% 118M 1s 2024-05-30T16:23:29.595 INFO:teuthology.orchestra.run.smithi006.stderr: 25250K .......... .......... .......... .......... .......... 27% 64.9M 1s 2024-05-30T16:23:29.595 INFO:teuthology.orchestra.run.smithi006.stderr: 25300K .......... .......... .......... .......... .......... 27% 114M 1s 2024-05-30T16:23:29.595 INFO:teuthology.orchestra.run.smithi006.stderr: 25350K .......... .......... .......... .......... .......... 27% 89.7M 1s 2024-05-30T16:23:29.595 INFO:teuthology.orchestra.run.smithi006.stderr: 25400K .......... .......... .......... .......... .......... 27% 96.3M 1s 2024-05-30T16:23:29.595 INFO:teuthology.orchestra.run.smithi006.stderr: 25450K .......... .......... .......... .......... .......... 27% 151M 1s 2024-05-30T16:23:29.595 INFO:teuthology.orchestra.run.smithi006.stderr: 25500K .......... .......... .......... .......... .......... 27% 127M 1s 2024-05-30T16:23:29.595 INFO:teuthology.orchestra.run.smithi006.stderr: 25550K .......... .......... .......... .......... .......... 27% 103M 1s 2024-05-30T16:23:29.595 INFO:teuthology.orchestra.run.smithi006.stderr: 25600K .......... .......... .......... .......... .......... 27% 114M 1s 2024-05-30T16:23:29.595 INFO:teuthology.orchestra.run.smithi006.stderr: 25650K .......... .......... .......... .......... .......... 27% 116M 1s 2024-05-30T16:23:29.595 INFO:teuthology.orchestra.run.smithi006.stderr: 25700K .......... .......... .......... .......... .......... 27% 115M 1s 2024-05-30T16:23:29.595 INFO:teuthology.orchestra.run.smithi006.stderr: 25750K .......... .......... .......... .......... .......... 27% 91.5M 1s 2024-05-30T16:23:29.595 INFO:teuthology.orchestra.run.smithi006.stderr: 25800K .......... .......... .......... .......... .......... 27% 129M 1s 2024-05-30T16:23:29.595 INFO:teuthology.orchestra.run.smithi006.stderr: 25850K .......... .......... .......... .......... .......... 27% 103M 1s 2024-05-30T16:23:29.596 INFO:teuthology.orchestra.run.smithi006.stderr: 25900K .......... .......... .......... .......... .......... 27% 114M 1s 2024-05-30T16:23:29.596 INFO:teuthology.orchestra.run.smithi006.stderr: 25950K .......... .......... .......... .......... .......... 27% 121M 1s 2024-05-30T16:23:29.596 INFO:teuthology.orchestra.run.smithi006.stderr: 26000K .......... .......... .......... .......... .......... 28% 99.4M 1s 2024-05-30T16:23:29.596 INFO:teuthology.orchestra.run.smithi006.stderr: 26050K .......... .......... .......... .......... .......... 28% 97.8M 1s 2024-05-30T16:23:29.596 INFO:teuthology.orchestra.run.smithi006.stderr: 26100K .......... .......... .......... .......... .......... 28% 116M 1s 2024-05-30T16:23:29.596 INFO:teuthology.orchestra.run.smithi006.stderr: 26150K .......... .......... .......... .......... .......... 28% 115M 1s 2024-05-30T16:23:29.596 INFO:teuthology.orchestra.run.smithi006.stderr: 26200K .......... .......... .......... .......... .......... 28% 92.7M 1s 2024-05-30T16:23:29.596 INFO:teuthology.orchestra.run.smithi006.stderr: 26250K .......... .......... .......... .......... .......... 28% 113M 1s 2024-05-30T16:23:29.596 INFO:teuthology.orchestra.run.smithi006.stderr: 26300K .......... .......... .......... .......... .......... 28% 79.8M 1s 2024-05-30T16:23:29.596 INFO:teuthology.orchestra.run.smithi006.stderr: 26350K .......... .......... .......... .......... .......... 28% 96.4M 1s 2024-05-30T16:23:29.596 INFO:teuthology.orchestra.run.smithi006.stderr: 26400K .......... .......... .......... .......... .......... 28% 113M 1s 2024-05-30T16:23:29.596 INFO:teuthology.orchestra.run.smithi006.stderr: 26450K .......... .......... .......... .......... .......... 28% 64.1M 1s 2024-05-30T16:23:29.596 INFO:teuthology.orchestra.run.smithi006.stderr: 26500K .......... .......... .......... .......... .......... 28% 61.5M 1s 2024-05-30T16:23:29.596 INFO:teuthology.orchestra.run.smithi006.stderr: 26550K .......... .......... .......... .......... .......... 28% 31.3M 1s 2024-05-30T16:23:29.597 INFO:teuthology.orchestra.run.smithi006.stderr: 26600K .......... .......... .......... .......... .......... 28% 120M 1s 2024-05-30T16:23:29.597 INFO:teuthology.orchestra.run.smithi006.stderr: 26650K .......... .......... .......... .......... .......... 28% 74.0M 1s 2024-05-30T16:23:29.597 INFO:teuthology.orchestra.run.smithi006.stderr: 26700K .......... .......... .......... .......... .......... 28% 25.4M 1s 2024-05-30T16:23:29.597 INFO:teuthology.orchestra.run.smithi006.stderr: 26750K .......... .......... .......... .......... .......... 28% 40.9M 1s 2024-05-30T16:23:29.597 INFO:teuthology.orchestra.run.smithi006.stderr: 26800K .......... .......... .......... .......... .......... 28% 103M 1s 2024-05-30T16:23:29.597 INFO:teuthology.orchestra.run.smithi006.stderr: 26850K .......... .......... .......... .......... .......... 28% 37.6M 1s 2024-05-30T16:23:29.597 INFO:teuthology.orchestra.run.smithi006.stderr: 26900K .......... .......... .......... .......... .......... 29% 46.9M 1s 2024-05-30T16:23:29.597 INFO:teuthology.orchestra.run.smithi006.stderr: 26950K .......... .......... .......... .......... .......... 29% 39.0M 1s 2024-05-30T16:23:29.597 INFO:teuthology.orchestra.run.smithi006.stderr: 27000K .......... .......... .......... .......... .......... 29% 82.4M 1s 2024-05-30T16:23:29.597 INFO:teuthology.orchestra.run.smithi006.stderr: 27050K .......... .......... .......... .......... .......... 29% 83.4M 1s 2024-05-30T16:23:29.597 INFO:teuthology.orchestra.run.smithi006.stderr: 27100K .......... .......... .......... .......... .......... 29% 31.6M 1s 2024-05-30T16:23:29.597 INFO:teuthology.orchestra.run.smithi006.stderr: 27150K .......... .......... .......... .......... .......... 29% 67.1M 1s 2024-05-30T16:23:29.597 INFO:teuthology.orchestra.run.smithi006.stderr: 27200K .......... .......... .......... .......... .......... 29% 83.0M 1s 2024-05-30T16:23:29.598 INFO:teuthology.orchestra.run.smithi006.stderr: 27250K .......... .......... .......... .......... .......... 29% 56.9M 1s 2024-05-30T16:23:29.598 INFO:teuthology.orchestra.run.smithi006.stderr: 27300K .......... .......... .......... .......... .......... 29% 76.5M 1s 2024-05-30T16:23:29.598 INFO:teuthology.orchestra.run.smithi006.stderr: 27350K .......... .......... .......... .......... .......... 29% 110M 1s 2024-05-30T16:23:29.598 INFO:teuthology.orchestra.run.smithi006.stderr: 27400K .......... .......... .......... .......... .......... 29% 67.7M 1s 2024-05-30T16:23:29.598 INFO:teuthology.orchestra.run.smithi006.stderr: 27450K .......... .......... .......... .......... .......... 29% 84.1M 1s 2024-05-30T16:23:29.598 INFO:teuthology.orchestra.run.smithi006.stderr: 27500K .......... .......... .......... .......... .......... 29% 49.9M 1s 2024-05-30T16:23:29.598 INFO:teuthology.orchestra.run.smithi006.stderr: 27550K .......... .......... .......... .......... .......... 29% 108M 1s 2024-05-30T16:23:29.598 INFO:teuthology.orchestra.run.smithi006.stderr: 27600K .......... .......... .......... .......... .......... 29% 115M 1s 2024-05-30T16:23:29.598 INFO:teuthology.orchestra.run.smithi006.stderr: 27650K .......... .......... .......... .......... .......... 29% 87.9M 1s 2024-05-30T16:23:29.598 INFO:teuthology.orchestra.run.smithi006.stderr: 27700K .......... .......... .......... .......... .......... 29% 114M 1s 2024-05-30T16:23:29.598 INFO:teuthology.orchestra.run.smithi006.stderr: 27750K .......... .......... .......... .......... .......... 29% 111M 1s 2024-05-30T16:23:29.598 INFO:teuthology.orchestra.run.smithi006.stderr: 27800K .......... .......... .......... .......... .......... 29% 125M 1s 2024-05-30T16:23:29.598 INFO:teuthology.orchestra.run.smithi006.stderr: 27850K .......... .......... .......... .......... .......... 30% 113M 1s 2024-05-30T16:23:29.598 INFO:teuthology.orchestra.run.smithi006.stderr: 27900K .......... .......... .......... .......... .......... 30% 114M 1s 2024-05-30T16:23:29.599 INFO:teuthology.orchestra.run.smithi006.stderr: 27950K .......... .......... .......... .......... .......... 30% 106M 1s 2024-05-30T16:23:29.599 INFO:teuthology.orchestra.run.smithi006.stderr: 28000K .......... .......... .......... .......... .......... 30% 123M 1s 2024-05-30T16:23:29.599 INFO:teuthology.orchestra.run.smithi006.stderr: 28050K .......... .......... .......... .......... .......... 30% 86.8M 1s 2024-05-30T16:23:29.599 INFO:teuthology.orchestra.run.smithi006.stderr: 28100K .......... .......... .......... .......... .......... 30% 114M 1s 2024-05-30T16:23:29.599 INFO:teuthology.orchestra.run.smithi006.stderr: 28150K .......... .......... .......... .......... .......... 30% 112M 1s 2024-05-30T16:23:29.599 INFO:teuthology.orchestra.run.smithi006.stderr: 28200K .......... .......... .......... .......... .......... 30% 128M 1s 2024-05-30T16:23:29.599 INFO:teuthology.orchestra.run.smithi006.stderr: 28250K .......... .......... .......... .......... .......... 30% 100M 1s 2024-05-30T16:23:29.599 INFO:teuthology.orchestra.run.smithi006.stderr: 28300K .......... .......... .......... .......... .......... 30% 125M 1s 2024-05-30T16:23:29.599 INFO:teuthology.orchestra.run.smithi006.stderr: 28350K .......... .......... .......... .......... .......... 30% 44.1M 1s 2024-05-30T16:23:29.599 INFO:teuthology.orchestra.run.smithi006.stderr: 28400K .......... .......... .......... .......... .......... 30% 115M 1s 2024-05-30T16:23:29.599 INFO:teuthology.orchestra.run.smithi006.stderr: 28450K .......... .......... .......... .......... .......... 30% 116M 1s 2024-05-30T16:23:29.599 INFO:teuthology.orchestra.run.smithi006.stderr: 28500K .......... .......... .......... .......... .......... 30% 109M 1s 2024-05-30T16:23:29.599 INFO:teuthology.orchestra.run.smithi006.stderr: 28550K .......... .......... .......... .......... .......... 30% 103M 1s 2024-05-30T16:23:29.600 INFO:teuthology.orchestra.run.smithi006.stderr: 28600K .......... .......... .......... .......... .......... 30% 117M 1s 2024-05-30T16:23:29.600 INFO:teuthology.orchestra.run.smithi006.stderr: 28650K .......... .......... .......... .......... .......... 30% 105M 1s 2024-05-30T16:23:29.600 INFO:teuthology.orchestra.run.smithi006.stderr: 28700K .......... .......... .......... .......... .......... 30% 130M 1s 2024-05-30T16:23:29.600 INFO:teuthology.orchestra.run.smithi006.stderr: 28750K .......... .......... .......... .......... .......... 31% 73.4M 1s 2024-05-30T16:23:29.600 INFO:teuthology.orchestra.run.smithi006.stderr: 28800K .......... .......... .......... .......... .......... 31% 138M 1s 2024-05-30T16:23:29.600 INFO:teuthology.orchestra.run.smithi006.stderr: 28850K .......... .......... .......... .......... .......... 31% 134M 1s 2024-05-30T16:23:29.600 INFO:teuthology.orchestra.run.smithi006.stderr: 28900K .......... .......... .......... .......... .......... 31% 122M 1s 2024-05-30T16:23:29.600 INFO:teuthology.orchestra.run.smithi006.stderr: 28950K .......... .......... .......... .......... .......... 31% 90.6M 1s 2024-05-30T16:23:29.600 INFO:teuthology.orchestra.run.smithi006.stderr: 29000K .......... .......... .......... .......... .......... 31% 129M 1s 2024-05-30T16:23:29.600 INFO:teuthology.orchestra.run.smithi006.stderr: 29050K .......... .......... .......... .......... .......... 31% 123M 1s 2024-05-30T16:23:29.600 INFO:teuthology.orchestra.run.smithi006.stderr: 29100K .......... .......... .......... .......... .......... 31% 97.6M 1s 2024-05-30T16:23:29.600 INFO:teuthology.orchestra.run.smithi006.stderr: 29150K .......... .......... .......... .......... .......... 31% 101M 1s 2024-05-30T16:23:29.600 INFO:teuthology.orchestra.run.smithi006.stderr: 29200K .......... .......... .......... .......... .......... 31% 121M 1s 2024-05-30T16:23:29.600 INFO:teuthology.orchestra.run.smithi006.stderr: 29250K .......... .......... .......... .......... .......... 31% 96.7M 1s 2024-05-30T16:23:29.601 INFO:teuthology.orchestra.run.smithi006.stderr: 29300K .......... .......... .......... .......... .......... 31% 118M 1s 2024-05-30T16:23:29.601 INFO:teuthology.orchestra.run.smithi006.stderr: 29350K .......... .......... .......... .......... .......... 31% 90.6M 1s 2024-05-30T16:23:29.601 INFO:teuthology.orchestra.run.smithi006.stderr: 29400K .......... .......... .......... .......... .......... 31% 122M 1s 2024-05-30T16:23:29.601 INFO:teuthology.orchestra.run.smithi006.stderr: 29450K .......... .......... .......... .......... .......... 31% 93.3M 1s 2024-05-30T16:23:29.601 INFO:teuthology.orchestra.run.smithi006.stderr: 29500K .......... .......... .......... .......... .......... 31% 103M 1s 2024-05-30T16:23:29.601 INFO:teuthology.orchestra.run.smithi006.stderr: 29550K .......... .......... .......... .......... .......... 31% 73.4M 1s 2024-05-30T16:23:29.601 INFO:teuthology.orchestra.run.smithi006.stderr: 29600K .......... .......... .......... .......... .......... 31% 56.0M 1s 2024-05-30T16:23:29.601 INFO:teuthology.orchestra.run.smithi006.stderr: 29650K .......... .......... .......... .......... .......... 31% 63.4M 1s 2024-05-30T16:23:29.601 INFO:teuthology.orchestra.run.smithi006.stderr: 29700K .......... .......... .......... .......... .......... 32% 111M 1s 2024-05-30T16:23:29.601 INFO:teuthology.orchestra.run.smithi006.stderr: 29750K .......... .......... .......... .......... .......... 32% 25.9M 1s 2024-05-30T16:23:29.601 INFO:teuthology.orchestra.run.smithi006.stderr: 29800K .......... .......... .......... .......... .......... 32% 48.1M 1s 2024-05-30T16:23:29.601 INFO:teuthology.orchestra.run.smithi006.stderr: 29850K .......... .......... .......... .......... .......... 32% 127M 1s 2024-05-30T16:23:29.601 INFO:teuthology.orchestra.run.smithi006.stderr: 29900K .......... .......... .......... .......... .......... 32% 96.9M 1s 2024-05-30T16:23:29.602 INFO:teuthology.orchestra.run.smithi006.stderr: 29950K .......... .......... .......... .......... .......... 32% 64.0M 1s 2024-05-30T16:23:29.602 INFO:teuthology.orchestra.run.smithi006.stderr: 30000K .......... .......... .......... .......... .......... 32% 86.3M 1s 2024-05-30T16:23:29.602 INFO:teuthology.orchestra.run.smithi006.stderr: 30050K .......... .......... .......... .......... .......... 32% 112M 1s 2024-05-30T16:23:29.602 INFO:teuthology.orchestra.run.smithi006.stderr: 30100K .......... .......... .......... .......... .......... 32% 120M 1s 2024-05-30T16:23:29.602 INFO:teuthology.orchestra.run.smithi006.stderr: 30150K .......... .......... .......... .......... .......... 32% 92.9M 1s 2024-05-30T16:23:29.602 INFO:teuthology.orchestra.run.smithi006.stderr: 30200K .......... .......... .......... .......... .......... 32% 117M 1s 2024-05-30T16:23:29.602 INFO:teuthology.orchestra.run.smithi006.stderr: 30250K .......... .......... .......... .......... .......... 32% 105M 1s 2024-05-30T16:23:29.602 INFO:teuthology.orchestra.run.smithi006.stderr: 30300K .......... .......... .......... .......... .......... 32% 131M 1s 2024-05-30T16:23:29.602 INFO:teuthology.orchestra.run.smithi006.stderr: 30350K .......... .......... .......... .......... .......... 32% 73.0M 1s 2024-05-30T16:23:29.602 INFO:teuthology.orchestra.run.smithi006.stderr: 30400K .......... .......... .......... .......... .......... 32% 78.8M 1s 2024-05-30T16:23:29.602 INFO:teuthology.orchestra.run.smithi006.stderr: 30450K .......... .......... .......... .......... .......... 32% 45.3M 1s 2024-05-30T16:23:29.602 INFO:teuthology.orchestra.run.smithi006.stderr: 30500K .......... .......... .......... .......... .......... 32% 16.8M 1s 2024-05-30T16:23:29.602 INFO:teuthology.orchestra.run.smithi006.stderr: 30550K .......... .......... .......... .......... .......... 32% 117M 1s 2024-05-30T16:23:29.603 INFO:teuthology.orchestra.run.smithi006.stderr: 30600K .......... .......... .......... .......... .......... 32% 45.3M 1s 2024-05-30T16:23:29.603 INFO:teuthology.orchestra.run.smithi006.stderr: 30650K .......... .......... .......... .......... .......... 33% 82.4M 1s 2024-05-30T16:23:29.603 INFO:teuthology.orchestra.run.smithi006.stderr: 30700K .......... .......... .......... .......... .......... 33% 102M 1s 2024-05-30T16:23:29.603 INFO:teuthology.orchestra.run.smithi006.stderr: 30750K .......... .......... .......... .......... .......... 33% 121M 1s 2024-05-30T16:23:29.603 INFO:teuthology.orchestra.run.smithi006.stderr: 30800K .......... .......... .......... .......... .......... 33% 125M 1s 2024-05-30T16:23:29.603 INFO:teuthology.orchestra.run.smithi006.stderr: 30850K .......... .......... .......... .......... .......... 33% 111M 1s 2024-05-30T16:23:29.603 INFO:teuthology.orchestra.run.smithi006.stderr: 30900K .......... .......... .......... .......... .......... 33% 115M 1s 2024-05-30T16:23:29.603 INFO:teuthology.orchestra.run.smithi006.stderr: 30950K .......... .......... .......... .......... .......... 33% 109M 1s 2024-05-30T16:23:29.603 INFO:teuthology.orchestra.run.smithi006.stderr: 31000K .......... .......... .......... .......... .......... 33% 116M 1s 2024-05-30T16:23:29.603 INFO:teuthology.orchestra.run.smithi006.stderr: 31050K .......... .......... .......... .......... .......... 33% 82.1M 1s 2024-05-30T16:23:29.603 INFO:teuthology.orchestra.run.smithi006.stderr: 31100K .......... .......... .......... .......... .......... 33% 137M 1s 2024-05-30T16:23:29.603 INFO:teuthology.orchestra.run.smithi006.stderr: 31150K .......... .......... .......... .......... .......... 33% 113M 1s 2024-05-30T16:23:29.603 INFO:teuthology.orchestra.run.smithi006.stderr: 31200K .......... .......... .......... .......... .......... 33% 115M 1s 2024-05-30T16:23:29.604 INFO:teuthology.orchestra.run.smithi006.stderr: 31250K .......... .......... .......... .......... .......... 33% 94.5M 1s 2024-05-30T16:23:29.604 INFO:teuthology.orchestra.run.smithi006.stderr: 31300K .......... .......... .......... .......... .......... 33% 109M 1s 2024-05-30T16:23:29.604 INFO:teuthology.orchestra.run.smithi006.stderr: 31350K .......... .......... .......... .......... .......... 33% 130M 1s 2024-05-30T16:23:29.604 INFO:teuthology.orchestra.run.smithi006.stderr: 31400K .......... .......... .......... .......... .......... 33% 114M 1s 2024-05-30T16:23:29.604 INFO:teuthology.orchestra.run.smithi006.stderr: 31450K .......... .......... .......... .......... .......... 33% 90.2M 1s 2024-05-30T16:23:29.604 INFO:teuthology.orchestra.run.smithi006.stderr: 31500K .......... .......... .......... .......... .......... 33% 119M 1s 2024-05-30T16:23:29.604 INFO:teuthology.orchestra.run.smithi006.stderr: 31550K .......... .......... .......... .......... .......... 34% 119M 1s 2024-05-30T16:23:29.604 INFO:teuthology.orchestra.run.smithi006.stderr: 31600K .......... .......... .......... .......... .......... 34% 112M 1s 2024-05-30T16:23:29.604 INFO:teuthology.orchestra.run.smithi006.stderr: 31650K .......... .......... .......... .......... .......... 34% 110M 1s 2024-05-30T16:23:29.604 INFO:teuthology.orchestra.run.smithi006.stderr: 31700K .......... .......... .......... .......... .......... 34% 115M 1s 2024-05-30T16:23:29.604 INFO:teuthology.orchestra.run.smithi006.stderr: 31750K .......... .......... .......... .......... .......... 34% 109M 1s 2024-05-30T16:23:29.604 INFO:teuthology.orchestra.run.smithi006.stderr: 31800K .......... .......... .......... .......... .......... 34% 118M 1s 2024-05-30T16:23:29.604 INFO:teuthology.orchestra.run.smithi006.stderr: 31850K .......... .......... .......... .......... .......... 34% 85.1M 1s 2024-05-30T16:23:29.604 INFO:teuthology.orchestra.run.smithi006.stderr: 31900K .......... .......... .......... .......... .......... 34% 119M 1s 2024-05-30T16:23:29.605 INFO:teuthology.orchestra.run.smithi006.stderr: 31950K .......... .......... .......... .......... .......... 34% 111M 1s 2024-05-30T16:23:29.605 INFO:teuthology.orchestra.run.smithi006.stderr: 32000K .......... .......... .......... .......... .......... 34% 120M 1s 2024-05-30T16:23:29.605 INFO:teuthology.orchestra.run.smithi006.stderr: 32050K .......... .......... .......... .......... .......... 34% 94.5M 1s 2024-05-30T16:23:29.605 INFO:teuthology.orchestra.run.smithi006.stderr: 32100K .......... .......... .......... .......... .......... 34% 127M 1s 2024-05-30T16:23:29.605 INFO:teuthology.orchestra.run.smithi006.stderr: 32150K .......... .......... .......... .......... .......... 34% 92.4M 1s 2024-05-30T16:23:29.605 INFO:teuthology.orchestra.run.smithi006.stderr: 32200K .......... .......... .......... .......... .......... 34% 161M 1s 2024-05-30T16:23:29.605 INFO:teuthology.orchestra.run.smithi006.stderr: 32250K .......... .......... .......... .......... .......... 34% 89.8M 1s 2024-05-30T16:23:29.605 INFO:teuthology.orchestra.run.smithi006.stderr: 32300K .......... .......... .......... .......... .......... 34% 117M 1s 2024-05-30T16:23:29.605 INFO:teuthology.orchestra.run.smithi006.stderr: 32350K .......... .......... .......... .......... .......... 34% 116M 1s 2024-05-30T16:23:29.605 INFO:teuthology.orchestra.run.smithi006.stderr: 32400K .......... .......... .......... .......... .......... 34% 102M 1s 2024-05-30T16:23:29.605 INFO:teuthology.orchestra.run.smithi006.stderr: 32450K .......... .......... .......... .......... .......... 34% 106M 1s 2024-05-30T16:23:29.605 INFO:teuthology.orchestra.run.smithi006.stderr: 32500K .......... .......... .......... .......... .......... 35% 51.9M 1s 2024-05-30T16:23:29.605 INFO:teuthology.orchestra.run.smithi006.stderr: 32550K .......... .......... .......... .......... .......... 35% 314M 1s 2024-05-30T16:23:29.605 INFO:teuthology.orchestra.run.smithi006.stderr: 32600K .......... .......... .......... .......... .......... 35% 196M 1s 2024-05-30T16:23:29.606 INFO:teuthology.orchestra.run.smithi006.stderr: 32650K .......... .......... .......... .......... .......... 35% 96.2M 1s 2024-05-30T16:23:29.606 INFO:teuthology.orchestra.run.smithi006.stderr: 32700K .......... .......... .......... .......... .......... 35% 107M 1s 2024-05-30T16:23:29.606 INFO:teuthology.orchestra.run.smithi006.stderr: 32750K .......... .......... .......... .......... .......... 35% 115M 1s 2024-05-30T16:23:29.606 INFO:teuthology.orchestra.run.smithi006.stderr: 32800K .......... .......... .......... .......... .......... 35% 108M 1s 2024-05-30T16:23:29.606 INFO:teuthology.orchestra.run.smithi006.stderr: 32850K .......... .......... .......... .......... .......... 35% 98.2M 1s 2024-05-30T16:23:29.606 INFO:teuthology.orchestra.run.smithi006.stderr: 32900K .......... .......... .......... .......... .......... 35% 141M 1s 2024-05-30T16:23:29.606 INFO:teuthology.orchestra.run.smithi006.stderr: 32950K .......... .......... .......... .......... .......... 35% 89.8M 1s 2024-05-30T16:23:29.606 INFO:teuthology.orchestra.run.smithi006.stderr: 33000K .......... .......... .......... .......... .......... 35% 102M 1s 2024-05-30T16:23:29.606 INFO:teuthology.orchestra.run.smithi006.stderr: 33050K .......... .......... .......... .......... .......... 35% 84.6M 1s 2024-05-30T16:23:29.606 INFO:teuthology.orchestra.run.smithi006.stderr: 33100K .......... .......... .......... .......... .......... 35% 98.9M 1s 2024-05-30T16:23:29.606 INFO:teuthology.orchestra.run.smithi006.stderr: 33150K .......... .......... .......... .......... .......... 35% 122M 1s 2024-05-30T16:23:29.606 INFO:teuthology.orchestra.run.smithi006.stderr: 33200K .......... .......... .......... .......... .......... 35% 111M 1s 2024-05-30T16:23:29.606 INFO:teuthology.orchestra.run.smithi006.stderr: 33250K .......... .......... .......... .......... .......... 35% 108M 1s 2024-05-30T16:23:29.607 INFO:teuthology.orchestra.run.smithi006.stderr: 33300K .......... .......... .......... .......... .......... 35% 118M 1s 2024-05-30T16:23:29.607 INFO:teuthology.orchestra.run.smithi006.stderr: 33350K .......... .......... .......... .......... .......... 35% 88.8M 1s 2024-05-30T16:23:29.607 INFO:teuthology.orchestra.run.smithi006.stderr: 33400K .......... .......... .......... .......... .......... 36% 119M 1s 2024-05-30T16:23:29.607 INFO:teuthology.orchestra.run.smithi006.stderr: 33450K .......... .......... .......... .......... .......... 36% 112M 1s 2024-05-30T16:23:29.607 INFO:teuthology.orchestra.run.smithi006.stderr: 33500K .......... .......... .......... .......... .......... 36% 98.6M 1s 2024-05-30T16:23:29.607 INFO:teuthology.orchestra.run.smithi006.stderr: 33550K .......... .......... .......... .......... .......... 36% 87.5M 1s 2024-05-30T16:23:29.607 INFO:teuthology.orchestra.run.smithi006.stderr: 33600K .......... .......... .......... .......... .......... 36% 124M 1s 2024-05-30T16:23:29.607 INFO:teuthology.orchestra.run.smithi006.stderr: 33650K .......... .......... .......... .......... .......... 36% 84.5M 1s 2024-05-30T16:23:29.607 INFO:teuthology.orchestra.run.smithi006.stderr: 33700K .......... .......... .......... .......... .......... 36% 116M 1s 2024-05-30T16:23:29.607 INFO:teuthology.orchestra.run.smithi006.stderr: 33750K .......... .......... .......... .......... .......... 36% 74.5M 1s 2024-05-30T16:23:29.607 INFO:teuthology.orchestra.run.smithi006.stderr: 33800K .......... .......... .......... .......... .......... 36% 133M 1s 2024-05-30T16:23:29.607 INFO:teuthology.orchestra.run.smithi006.stderr: 33850K .......... .......... .......... .......... .......... 36% 120M 1s 2024-05-30T16:23:29.607 INFO:teuthology.orchestra.run.smithi006.stderr: 33900K .......... .......... .......... .......... .......... 36% 119M 1s 2024-05-30T16:23:29.608 INFO:teuthology.orchestra.run.smithi006.stderr: 33950K .......... .......... .......... .......... .......... 36% 111M 1s 2024-05-30T16:23:29.608 INFO:teuthology.orchestra.run.smithi006.stderr: 34000K .......... .......... .......... .......... .......... 36% 112M 1s 2024-05-30T16:23:29.608 INFO:teuthology.orchestra.run.smithi006.stderr: 34050K .......... .......... .......... .......... .......... 36% 108M 1s 2024-05-30T16:23:29.608 INFO:teuthology.orchestra.run.smithi006.stderr: 34100K .......... .......... .......... .......... .......... 36% 120M 1s 2024-05-30T16:23:29.608 INFO:teuthology.orchestra.run.smithi006.stderr: 34150K .......... .......... .......... .......... .......... 36% 90.5M 1s 2024-05-30T16:23:29.608 INFO:teuthology.orchestra.run.smithi006.stderr: 34200K .......... .......... .......... .......... .......... 36% 99.9M 1s 2024-05-30T16:23:29.608 INFO:teuthology.orchestra.run.smithi006.stderr: 34250K .......... .......... .......... .......... .......... 36% 135M 1s 2024-05-30T16:23:29.608 INFO:teuthology.orchestra.run.smithi006.stderr: 34300K .......... .......... .......... .......... .......... 36% 119M 1s 2024-05-30T16:23:29.608 INFO:teuthology.orchestra.run.smithi006.stderr: 34350K .......... .......... .......... .......... .......... 37% 101M 1s 2024-05-30T16:23:29.608 INFO:teuthology.orchestra.run.smithi006.stderr: 34400K .......... .......... .......... .......... .......... 37% 120M 1s 2024-05-30T16:23:29.608 INFO:teuthology.orchestra.run.smithi006.stderr: 34450K .......... .......... .......... .......... .......... 37% 110M 1s 2024-05-30T16:23:29.608 INFO:teuthology.orchestra.run.smithi006.stderr: 34500K .......... .......... .......... .......... .......... 37% 121M 1s 2024-05-30T16:23:29.608 INFO:teuthology.orchestra.run.smithi006.stderr: 34550K .......... .......... .......... .......... .......... 37% 76.7M 1s 2024-05-30T16:23:29.608 INFO:teuthology.orchestra.run.smithi006.stderr: 34600K .......... .......... .......... .......... .......... 37% 144M 1s 2024-05-30T16:23:29.609 INFO:teuthology.orchestra.run.smithi006.stderr: 34650K .......... .......... .......... .......... .......... 37% 113M 1s 2024-05-30T16:23:29.609 INFO:teuthology.orchestra.run.smithi006.stderr: 34700K .......... .......... .......... .......... .......... 37% 79.9M 1s 2024-05-30T16:23:29.609 INFO:teuthology.orchestra.run.smithi006.stderr: 34750K .......... .......... .......... .......... .......... 37% 109M 1s 2024-05-30T16:23:29.609 INFO:teuthology.orchestra.run.smithi006.stderr: 34800K .......... .......... .......... .......... .......... 37% 105M 1s 2024-05-30T16:23:29.609 INFO:teuthology.orchestra.run.smithi006.stderr: 34850K .......... .......... .......... .......... .......... 37% 129M 1s 2024-05-30T16:23:29.609 INFO:teuthology.orchestra.run.smithi006.stderr: 34900K .......... .......... .......... .......... .......... 37% 114M 1s 2024-05-30T16:23:29.609 INFO:teuthology.orchestra.run.smithi006.stderr: 34950K .......... .......... .......... .......... .......... 37% 94.0M 1s 2024-05-30T16:23:29.609 INFO:teuthology.orchestra.run.smithi006.stderr: 35000K .......... .......... .......... .......... .......... 37% 118M 1s 2024-05-30T16:23:29.609 INFO:teuthology.orchestra.run.smithi006.stderr: 35050K .......... .......... .......... .......... .......... 37% 109M 1s 2024-05-30T16:23:29.609 INFO:teuthology.orchestra.run.smithi006.stderr: 35100K .......... .......... .......... .......... .......... 37% 116M 1s 2024-05-30T16:23:29.609 INFO:teuthology.orchestra.run.smithi006.stderr: 35150K .......... .......... .......... .......... .......... 37% 112M 1s 2024-05-30T16:23:29.609 INFO:teuthology.orchestra.run.smithi006.stderr: 35200K .......... .......... .......... .......... .......... 37% 117M 1s 2024-05-30T16:23:29.609 INFO:teuthology.orchestra.run.smithi006.stderr: 35250K .......... .......... .......... .......... .......... 37% 78.2M 1s 2024-05-30T16:23:29.609 INFO:teuthology.orchestra.run.smithi006.stderr: 35300K .......... .......... .......... .......... .......... 38% 135M 1s 2024-05-30T16:23:29.610 INFO:teuthology.orchestra.run.smithi006.stderr: 35350K .......... .......... .......... .......... .......... 38% 115M 1s 2024-05-30T16:23:29.610 INFO:teuthology.orchestra.run.smithi006.stderr: 35400K .......... .......... .......... .......... .......... 38% 109M 1s 2024-05-30T16:23:29.610 INFO:teuthology.orchestra.run.smithi006.stderr: 35450K .......... .......... .......... .......... .......... 38% 89.2M 1s 2024-05-30T16:23:29.610 INFO:teuthology.orchestra.run.smithi006.stderr: 35500K .......... .......... .......... .......... .......... 38% 104M 1s 2024-05-30T16:23:29.610 INFO:teuthology.orchestra.run.smithi006.stderr: 35550K .......... .......... .......... .......... .......... 38% 126M 1s 2024-05-30T16:23:29.610 INFO:teuthology.orchestra.run.smithi006.stderr: 35600K .......... .......... .......... .......... .......... 38% 109M 1s 2024-05-30T16:23:29.610 INFO:teuthology.orchestra.run.smithi006.stderr: 35650K .......... .......... .......... .......... .......... 38% 85.6M 1s 2024-05-30T16:23:29.610 INFO:teuthology.orchestra.run.smithi006.stderr: 35700K .......... .......... .......... .......... .......... 38% 129M 1s 2024-05-30T16:23:29.610 INFO:teuthology.orchestra.run.smithi006.stderr: 35750K .......... .......... .......... .......... .......... 38% 112M 1s 2024-05-30T16:23:29.610 INFO:teuthology.orchestra.run.smithi006.stderr: 35800K .......... .......... .......... .......... .......... 38% 120M 1s 2024-05-30T16:23:29.610 INFO:teuthology.orchestra.run.smithi006.stderr: 35850K .......... .......... .......... .......... .......... 38% 109M 1s 2024-05-30T16:23:29.610 INFO:teuthology.orchestra.run.smithi006.stderr: 35900K .......... .......... .......... .......... .......... 38% 106M 1s 2024-05-30T16:23:29.610 INFO:teuthology.orchestra.run.smithi006.stderr: 35950K .......... .......... .......... .......... .......... 38% 116M 1s 2024-05-30T16:23:29.611 INFO:teuthology.orchestra.run.smithi006.stderr: 36000K .......... .......... .......... .......... .......... 38% 80.3M 1s 2024-05-30T16:23:29.611 INFO:teuthology.orchestra.run.smithi006.stderr: 36050K .......... .......... .......... .......... .......... 38% 92.7M 1s 2024-05-30T16:23:29.611 INFO:teuthology.orchestra.run.smithi006.stderr: 36100K .......... .......... .......... .......... .......... 38% 81.5M 1s 2024-05-30T16:23:29.611 INFO:teuthology.orchestra.run.smithi006.stderr: 36150K .......... .......... .......... .......... .......... 38% 95.7M 1s 2024-05-30T16:23:29.611 INFO:teuthology.orchestra.run.smithi006.stderr: 36200K .......... .......... .......... .......... .......... 39% 125M 1s 2024-05-30T16:23:29.611 INFO:teuthology.orchestra.run.smithi006.stderr: 36250K .......... .......... .......... .......... .......... 39% 114M 1s 2024-05-30T16:23:29.611 INFO:teuthology.orchestra.run.smithi006.stderr: 36300K .......... .......... .......... .......... .......... 39% 109M 1s 2024-05-30T16:23:29.611 INFO:teuthology.orchestra.run.smithi006.stderr: 36350K .......... .......... .......... .......... .......... 39% 115M 1s 2024-05-30T16:23:29.611 INFO:teuthology.orchestra.run.smithi006.stderr: 36400K .......... .......... .......... .......... .......... 39% 117M 1s 2024-05-30T16:23:29.611 INFO:teuthology.orchestra.run.smithi006.stderr: 36450K .......... .......... .......... .......... .......... 39% 90.1M 1s 2024-05-30T16:23:29.611 INFO:teuthology.orchestra.run.smithi006.stderr: 36500K .......... .......... .......... .......... .......... 39% 116M 1s 2024-05-30T16:23:29.611 INFO:teuthology.orchestra.run.smithi006.stderr: 36550K .......... .......... .......... .......... .......... 39% 104M 1s 2024-05-30T16:23:29.611 INFO:teuthology.orchestra.run.smithi006.stderr: 36600K .......... .......... .......... .......... .......... 39% 120M 1s 2024-05-30T16:23:29.611 INFO:teuthology.orchestra.run.smithi006.stderr: 36650K .......... .......... .......... .......... .......... 39% 110M 1s 2024-05-30T16:23:29.612 INFO:teuthology.orchestra.run.smithi006.stderr: 36700K .......... .......... .......... .......... .......... 39% 124M 1s 2024-05-30T16:23:29.612 INFO:teuthology.orchestra.run.smithi006.stderr: 36750K .......... .......... .......... .......... .......... 39% 114M 1s 2024-05-30T16:23:29.612 INFO:teuthology.orchestra.run.smithi006.stderr: 36800K .......... .......... .......... .......... .......... 39% 113M 1s 2024-05-30T16:23:29.612 INFO:teuthology.orchestra.run.smithi006.stderr: 36850K .......... .......... .......... .......... .......... 39% 85.5M 1s 2024-05-30T16:23:29.612 INFO:teuthology.orchestra.run.smithi006.stderr: 36900K .......... .......... .......... .......... .......... 39% 116M 1s 2024-05-30T16:23:29.612 INFO:teuthology.orchestra.run.smithi006.stderr: 36950K .......... .......... .......... .......... .......... 39% 115M 1s 2024-05-30T16:23:29.612 INFO:teuthology.orchestra.run.smithi006.stderr: 37000K .......... .......... .......... .......... .......... 39% 123M 1s 2024-05-30T16:23:29.612 INFO:teuthology.orchestra.run.smithi006.stderr: 37050K .......... .......... .......... .......... .......... 39% 105M 1s 2024-05-30T16:23:29.612 INFO:teuthology.orchestra.run.smithi006.stderr: 37100K .......... .......... .......... .......... .......... 39% 102M 1s 2024-05-30T16:23:29.612 INFO:teuthology.orchestra.run.smithi006.stderr: 37150K .......... .......... .......... .......... .......... 40% 121M 1s 2024-05-30T16:23:29.612 INFO:teuthology.orchestra.run.smithi006.stderr: 37200K .......... .......... .......... .......... .......... 40% 120M 1s 2024-05-30T16:23:29.612 INFO:teuthology.orchestra.run.smithi006.stderr: 37250K .......... .......... .......... .......... .......... 40% 96.7M 1s 2024-05-30T16:23:29.612 INFO:teuthology.orchestra.run.smithi006.stderr: 37300K .......... .......... .......... .......... .......... 40% 115M 1s 2024-05-30T16:23:29.613 INFO:teuthology.orchestra.run.smithi006.stderr: 37350K .......... .......... .......... .......... .......... 40% 109M 1s 2024-05-30T16:23:29.613 INFO:teuthology.orchestra.run.smithi006.stderr: 37400K .......... .......... .......... .......... .......... 40% 116M 1s 2024-05-30T16:23:29.613 INFO:teuthology.orchestra.run.smithi006.stderr: 37450K .......... .......... .......... .......... .......... 40% 98.5M 1s 2024-05-30T16:23:29.613 INFO:teuthology.orchestra.run.smithi006.stderr: 37500K .......... .......... .......... .......... .......... 40% 131M 1s 2024-05-30T16:23:29.613 INFO:teuthology.orchestra.run.smithi006.stderr: 37550K .......... .......... .......... .......... .......... 40% 42.3M 1s 2024-05-30T16:23:29.613 INFO:teuthology.orchestra.run.smithi006.stderr: 37600K .......... .......... .......... .......... .......... 40% 118M 1s 2024-05-30T16:23:29.613 INFO:teuthology.orchestra.run.smithi006.stderr: 37650K .......... .......... .......... .......... .......... 40% 94.3M 1s 2024-05-30T16:23:29.613 INFO:teuthology.orchestra.run.smithi006.stderr: 37700K .......... .......... .......... .......... .......... 40% 110M 1s 2024-05-30T16:23:29.613 INFO:teuthology.orchestra.run.smithi006.stderr: 37750K .......... .......... .......... .......... .......... 40% 113M 1s 2024-05-30T16:23:29.613 INFO:teuthology.orchestra.run.smithi006.stderr: 37800K .......... .......... .......... .......... .......... 40% 112M 1s 2024-05-30T16:23:29.613 INFO:teuthology.orchestra.run.smithi006.stderr: 37850K .......... .......... .......... .......... .......... 40% 115M 1s 2024-05-30T16:23:29.613 INFO:teuthology.orchestra.run.smithi006.stderr: 37900K .......... .......... .......... .......... .......... 40% 116M 1s 2024-05-30T16:23:29.613 INFO:teuthology.orchestra.run.smithi006.stderr: 37950K .......... .......... .......... .......... .......... 40% 90.7M 1s 2024-05-30T16:23:29.613 INFO:teuthology.orchestra.run.smithi006.stderr: 38000K .......... .......... .......... .......... .......... 40% 102M 1s 2024-05-30T16:23:29.614 INFO:teuthology.orchestra.run.smithi006.stderr: 38050K .......... .......... .......... .......... .......... 41% 123M 1s 2024-05-30T16:23:29.614 INFO:teuthology.orchestra.run.smithi006.stderr: 38100K .......... .......... .......... .......... .......... 41% 118M 1s 2024-05-30T16:23:29.614 INFO:teuthology.orchestra.run.smithi006.stderr: 38150K .......... .......... .......... .......... .......... 41% 10.1M 1s 2024-05-30T16:23:29.614 INFO:teuthology.orchestra.run.smithi006.stderr: 38200K .......... .......... .......... .......... .......... 41% 326M 1s 2024-05-30T16:23:29.614 INFO:teuthology.orchestra.run.smithi006.stderr: 38250K .......... .......... .......... .......... .......... 41% 343M 1s 2024-05-30T16:23:29.614 INFO:teuthology.orchestra.run.smithi006.stderr: 38300K .......... .......... .......... .......... .......... 41% 324M 1s 2024-05-30T16:23:29.614 INFO:teuthology.orchestra.run.smithi006.stderr: 38350K .......... .......... .......... .......... .......... 41% 274M 1s 2024-05-30T16:23:29.614 INFO:teuthology.orchestra.run.smithi006.stderr: 38400K .......... .......... .......... .......... .......... 41% 347M 1s 2024-05-30T16:23:29.614 INFO:teuthology.orchestra.run.smithi006.stderr: 38450K .......... .......... .......... .......... .......... 41% 322M 1s 2024-05-30T16:23:29.614 INFO:teuthology.orchestra.run.smithi006.stderr: 38500K .......... .......... .......... .......... .......... 41% 359M 1s 2024-05-30T16:23:29.614 INFO:teuthology.orchestra.run.smithi006.stderr: 38550K .......... .......... .......... .......... .......... 41% 315M 1s 2024-05-30T16:23:29.614 INFO:teuthology.orchestra.run.smithi006.stderr: 38600K .......... .......... .......... .......... .......... 41% 340M 1s 2024-05-30T16:23:29.614 INFO:teuthology.orchestra.run.smithi006.stderr: 38650K .......... .......... .......... .......... .......... 41% 357M 1s 2024-05-30T16:23:29.615 INFO:teuthology.orchestra.run.smithi006.stderr: 38700K .......... .......... .......... .......... .......... 41% 86.2M 1s 2024-05-30T16:23:29.615 INFO:teuthology.orchestra.run.smithi006.stderr: 38750K .......... .......... .......... .......... .......... 41% 63.6M 1s 2024-05-30T16:23:29.615 INFO:teuthology.orchestra.run.smithi006.stderr: 38800K .......... .......... .......... .......... .......... 41% 121M 1s 2024-05-30T16:23:29.615 INFO:teuthology.orchestra.run.smithi006.stderr: 38850K .......... .......... .......... .......... .......... 41% 116M 1s 2024-05-30T16:23:29.615 INFO:teuthology.orchestra.run.smithi006.stderr: 38900K .......... .......... .......... .......... .......... 41% 115M 1s 2024-05-30T16:23:29.615 INFO:teuthology.orchestra.run.smithi006.stderr: 38950K .......... .......... .......... .......... .......... 41% 97.0M 1s 2024-05-30T16:23:29.615 INFO:teuthology.orchestra.run.smithi006.stderr: 39000K .......... .......... .......... .......... .......... 42% 134M 1s 2024-05-30T16:23:29.615 INFO:teuthology.orchestra.run.smithi006.stderr: 39050K .......... .......... .......... .......... .......... 42% 98.5M 1s 2024-05-30T16:23:29.615 INFO:teuthology.orchestra.run.smithi006.stderr: 39100K .......... .......... .......... .......... .......... 42% 85.0M 1s 2024-05-30T16:23:29.615 INFO:teuthology.orchestra.run.smithi006.stderr: 39150K .......... .......... .......... .......... .......... 42% 82.4M 1s 2024-05-30T16:23:29.615 INFO:teuthology.orchestra.run.smithi006.stderr: 39200K .......... .......... .......... .......... .......... 42% 111M 1s 2024-05-30T16:23:29.615 INFO:teuthology.orchestra.run.smithi006.stderr: 39250K .......... .......... .......... .......... .......... 42% 118M 1s 2024-05-30T16:23:29.615 INFO:teuthology.orchestra.run.smithi006.stderr: 39300K .......... .......... .......... .......... .......... 42% 103M 1s 2024-05-30T16:23:29.616 INFO:teuthology.orchestra.run.smithi006.stderr: 39350K .......... .......... .......... .......... .......... 42% 93.3M 1s 2024-05-30T16:23:29.616 INFO:teuthology.orchestra.run.smithi006.stderr: 39400K .......... .......... .......... .......... .......... 42% 126M 1s 2024-05-30T16:23:29.616 INFO:teuthology.orchestra.run.smithi006.stderr: 39450K .......... .......... .......... .......... .......... 42% 122M 1s 2024-05-30T16:23:29.616 INFO:teuthology.orchestra.run.smithi006.stderr: 39500K .......... .......... .......... .......... .......... 42% 56.2M 1s 2024-05-30T16:23:29.616 INFO:teuthology.orchestra.run.smithi006.stderr: 39550K .......... .......... .......... .......... .......... 42% 43.6M 1s 2024-05-30T16:23:29.616 INFO:teuthology.orchestra.run.smithi006.stderr: 39600K .......... .......... .......... .......... .......... 42% 106M 1s 2024-05-30T16:23:29.616 INFO:teuthology.orchestra.run.smithi006.stderr: 39650K .......... .......... .......... .......... .......... 42% 117M 1s 2024-05-30T16:23:29.616 INFO:teuthology.orchestra.run.smithi006.stderr: 39700K .......... .......... .......... .......... .......... 42% 115M 1s 2024-05-30T16:23:29.616 INFO:teuthology.orchestra.run.smithi006.stderr: 39750K .......... .......... .......... .......... .......... 42% 111M 1s 2024-05-30T16:23:29.616 INFO:teuthology.orchestra.run.smithi006.stderr: 39800K .......... .......... .......... .......... .......... 42% 113M 1s 2024-05-30T16:23:29.616 INFO:teuthology.orchestra.run.smithi006.stderr: 39850K .......... .......... .......... .......... .......... 42% 89.2M 1s 2024-05-30T16:23:29.616 INFO:teuthology.orchestra.run.smithi006.stderr: 39900K .......... .......... .......... .......... .......... 43% 116M 1s 2024-05-30T16:23:29.616 INFO:teuthology.orchestra.run.smithi006.stderr: 39950K .......... .......... .......... .......... .......... 43% 116M 1s 2024-05-30T16:23:29.616 INFO:teuthology.orchestra.run.smithi006.stderr: 40000K .......... .......... .......... .......... .......... 43% 102M 1s 2024-05-30T16:23:29.617 INFO:teuthology.orchestra.run.smithi006.stderr: 40050K .......... .......... .......... .......... .......... 43% 120M 1s 2024-05-30T16:23:29.617 INFO:teuthology.orchestra.run.smithi006.stderr: 40100K .......... .......... .......... .......... .......... 43% 79.4M 1s 2024-05-30T16:23:29.617 INFO:teuthology.orchestra.run.smithi006.stderr: 40150K .......... .......... .......... .......... .......... 43% 108M 1s 2024-05-30T16:23:29.617 INFO:teuthology.orchestra.run.smithi006.stderr: 40200K .......... .......... .......... .......... .......... 43% 113M 1s 2024-05-30T16:23:29.617 INFO:teuthology.orchestra.run.smithi006.stderr: 40250K .......... .......... .......... .......... .......... 43% 86.7M 1s 2024-05-30T16:23:29.617 INFO:teuthology.orchestra.run.smithi006.stderr: 40300K .......... .......... .......... .......... .......... 43% 118M 1s 2024-05-30T16:23:29.617 INFO:teuthology.orchestra.run.smithi006.stderr: 40350K .......... .......... .......... .......... .......... 43% 112M 1s 2024-05-30T16:23:29.617 INFO:teuthology.orchestra.run.smithi006.stderr: 40400K .......... .......... .......... .......... .......... 43% 118M 1s 2024-05-30T16:23:29.617 INFO:teuthology.orchestra.run.smithi006.stderr: 40450K .......... .......... .......... .......... .......... 43% 108M 1s 2024-05-30T16:23:29.617 INFO:teuthology.orchestra.run.smithi006.stderr: 40500K .......... .......... .......... .......... .......... 43% 117M 1s 2024-05-30T16:23:29.617 INFO:teuthology.orchestra.run.smithi006.stderr: 40550K .......... .......... .......... .......... .......... 43% 110M 1s 2024-05-30T16:23:29.617 INFO:teuthology.orchestra.run.smithi006.stderr: 40600K .......... .......... .......... .......... .......... 43% 114M 1s 2024-05-30T16:23:29.617 INFO:teuthology.orchestra.run.smithi006.stderr: 40650K .......... .......... .......... .......... .......... 43% 81.7M 1s 2024-05-30T16:23:29.617 INFO:teuthology.orchestra.run.smithi006.stderr: 40700K .......... .......... .......... .......... .......... 43% 124M 1s 2024-05-30T16:23:29.618 INFO:teuthology.orchestra.run.smithi006.stderr: 40750K .......... .......... .......... .......... .......... 43% 114M 1s 2024-05-30T16:23:29.618 INFO:teuthology.orchestra.run.smithi006.stderr: 40800K .......... .......... .......... .......... .......... 43% 118M 1s 2024-05-30T16:23:29.618 INFO:teuthology.orchestra.run.smithi006.stderr: 40850K .......... .......... .......... .......... .......... 44% 104M 1s 2024-05-30T16:23:29.618 INFO:teuthology.orchestra.run.smithi006.stderr: 40900K .......... .......... .......... .......... .......... 44% 97.8M 1s 2024-05-30T16:23:29.618 INFO:teuthology.orchestra.run.smithi006.stderr: 40950K .......... .......... .......... .......... .......... 44% 115M 1s 2024-05-30T16:23:29.618 INFO:teuthology.orchestra.run.smithi006.stderr: 41000K .......... .......... .......... .......... .......... 44% 106M 1s 2024-05-30T16:23:29.618 INFO:teuthology.orchestra.run.smithi006.stderr: 41050K .......... .......... .......... .......... .......... 44% 102M 1s 2024-05-30T16:23:29.618 INFO:teuthology.orchestra.run.smithi006.stderr: 41100K .......... .......... .......... .......... .......... 44% 128M 1s 2024-05-30T16:23:29.618 INFO:teuthology.orchestra.run.smithi006.stderr: 41150K .......... .......... .......... .......... .......... 44% 114M 1s 2024-05-30T16:23:29.618 INFO:teuthology.orchestra.run.smithi006.stderr: 41200K .......... .......... .......... .......... .......... 44% 58.4M 1s 2024-05-30T16:23:29.618 INFO:teuthology.orchestra.run.smithi006.stderr: 41250K .......... .......... .......... .......... .......... 44% 105M 1s 2024-05-30T16:23:29.618 INFO:teuthology.orchestra.run.smithi006.stderr: 41300K .......... .......... .......... .......... .......... 44% 122M 1s 2024-05-30T16:23:29.618 INFO:teuthology.orchestra.run.smithi006.stderr: 41350K .......... .......... .......... .......... .......... 44% 72.2M 1s 2024-05-30T16:23:29.619 INFO:teuthology.orchestra.run.smithi006.stderr: 41400K .......... .......... .......... .......... .......... 44% 38.1M 1s 2024-05-30T16:23:29.619 INFO:teuthology.orchestra.run.smithi006.stderr: 41450K .......... .......... .......... .......... .......... 44% 37.0M 1s 2024-05-30T16:23:29.619 INFO:teuthology.orchestra.run.smithi006.stderr: 41500K .......... .......... .......... .......... .......... 44% 84.4M 1s 2024-05-30T16:23:29.619 INFO:teuthology.orchestra.run.smithi006.stderr: 41550K .......... .......... .......... .......... .......... 44% 106M 1s 2024-05-30T16:23:29.619 INFO:teuthology.orchestra.run.smithi006.stderr: 41600K .......... .......... .......... .......... .......... 44% 128M 1s 2024-05-30T16:23:29.619 INFO:teuthology.orchestra.run.smithi006.stderr: 41650K .......... .......... .......... .......... .......... 44% 109M 1s 2024-05-30T16:23:29.619 INFO:teuthology.orchestra.run.smithi006.stderr: 41700K .......... .......... .......... .......... .......... 44% 120M 1s 2024-05-30T16:23:29.619 INFO:teuthology.orchestra.run.smithi006.stderr: 41750K .......... .......... .......... .......... .......... 44% 118M 1s 2024-05-30T16:23:29.619 INFO:teuthology.orchestra.run.smithi006.stderr: 41800K .......... .......... .......... .......... .......... 45% 110M 1s 2024-05-30T16:23:29.619 INFO:teuthology.orchestra.run.smithi006.stderr: 41850K .......... .......... .......... .......... .......... 45% 96.3M 1s 2024-05-30T16:23:29.619 INFO:teuthology.orchestra.run.smithi006.stderr: 41900K .......... .......... .......... .......... .......... 45% 101M 1s 2024-05-30T16:23:29.619 INFO:teuthology.orchestra.run.smithi006.stderr: 41950K .......... .......... .......... .......... .......... 45% 106M 1s 2024-05-30T16:23:29.619 INFO:teuthology.orchestra.run.smithi006.stderr: 42000K .......... .......... .......... .......... .......... 45% 121M 1s 2024-05-30T16:23:29.620 INFO:teuthology.orchestra.run.smithi006.stderr: 42050K .......... .......... .......... .......... .......... 45% 88.9M 1s 2024-05-30T16:23:29.620 INFO:teuthology.orchestra.run.smithi006.stderr: 42100K .......... .......... .......... .......... .......... 45% 109M 1s 2024-05-30T16:23:29.620 INFO:teuthology.orchestra.run.smithi006.stderr: 42150K .......... .......... .......... .......... .......... 45% 89.7M 1s 2024-05-30T16:23:29.620 INFO:teuthology.orchestra.run.smithi006.stderr: 42200K .......... .......... .......... .......... .......... 45% 112M 1s 2024-05-30T16:23:29.620 INFO:teuthology.orchestra.run.smithi006.stderr: 42250K .......... .......... .......... .......... .......... 45% 115M 1s 2024-05-30T16:23:29.620 INFO:teuthology.orchestra.run.smithi006.stderr: 42300K .......... .......... .......... .......... .......... 45% 115M 1s 2024-05-30T16:23:29.620 INFO:teuthology.orchestra.run.smithi006.stderr: 42350K .......... .......... .......... .......... .......... 45% 112M 1s 2024-05-30T16:23:29.620 INFO:teuthology.orchestra.run.smithi006.stderr: 42400K .......... .......... .......... .......... .......... 45% 98.5M 1s 2024-05-30T16:23:29.620 INFO:teuthology.orchestra.run.smithi006.stderr: 42450K .......... .......... .......... .......... .......... 45% 117M 1s 2024-05-30T16:23:29.620 INFO:teuthology.orchestra.run.smithi006.stderr: 42500K .......... .......... .......... .......... .......... 45% 118M 1s 2024-05-30T16:23:29.620 INFO:teuthology.orchestra.run.smithi006.stderr: 42550K .......... .......... .......... .......... .......... 45% 37.3M 1s 2024-05-30T16:23:29.620 INFO:teuthology.orchestra.run.smithi006.stderr: 42600K .......... .......... .......... .......... .......... 45% 50.9M 1s 2024-05-30T16:23:29.620 INFO:teuthology.orchestra.run.smithi006.stderr: 42650K .......... .......... .......... .......... .......... 45% 88.5M 1s 2024-05-30T16:23:29.620 INFO:teuthology.orchestra.run.smithi006.stderr: 42700K .......... .......... .......... .......... .......... 46% 115M 1s 2024-05-30T16:23:29.621 INFO:teuthology.orchestra.run.smithi006.stderr: 42750K .......... .......... .......... .......... .......... 46% 115M 1s 2024-05-30T16:23:29.621 INFO:teuthology.orchestra.run.smithi006.stderr: 42800K .......... .......... .......... .......... .......... 46% 107M 1s 2024-05-30T16:23:29.621 INFO:teuthology.orchestra.run.smithi006.stderr: 42850K .......... .......... .......... .......... .......... 46% 117M 1s 2024-05-30T16:23:29.621 INFO:teuthology.orchestra.run.smithi006.stderr: 42900K .......... .......... .......... .......... .......... 46% 112M 1s 2024-05-30T16:23:29.621 INFO:teuthology.orchestra.run.smithi006.stderr: 42950K .......... .......... .......... .......... .......... 46% 85.9M 1s 2024-05-30T16:23:29.621 INFO:teuthology.orchestra.run.smithi006.stderr: 43000K .......... .......... .......... .......... .......... 46% 119M 1s 2024-05-30T16:23:29.621 INFO:teuthology.orchestra.run.smithi006.stderr: 43050K .......... .......... .......... .......... .......... 46% 118M 1s 2024-05-30T16:23:29.621 INFO:teuthology.orchestra.run.smithi006.stderr: 43100K .......... .......... .......... .......... .......... 46% 119M 1s 2024-05-30T16:23:29.621 INFO:teuthology.orchestra.run.smithi006.stderr: 43150K .......... .......... .......... .......... .......... 46% 104M 1s 2024-05-30T16:23:29.621 INFO:teuthology.orchestra.run.smithi006.stderr: 43200K .......... .......... .......... .......... .......... 46% 118M 1s 2024-05-30T16:23:29.621 INFO:teuthology.orchestra.run.smithi006.stderr: 43250K .......... .......... .......... .......... .......... 46% 108M 1s 2024-05-30T16:23:29.621 INFO:teuthology.orchestra.run.smithi006.stderr: 43300K .......... .......... .......... .......... .......... 46% 108M 1s 2024-05-30T16:23:29.621 INFO:teuthology.orchestra.run.smithi006.stderr: 43350K .......... .......... .......... .......... .......... 46% 98.2M 1s 2024-05-30T16:23:29.621 INFO:teuthology.orchestra.run.smithi006.stderr: 43400K .......... .......... .......... .......... .......... 46% 117M 1s 2024-05-30T16:23:29.621 INFO:teuthology.orchestra.run.smithi006.stderr: 43450K .......... .......... .......... .......... .......... 46% 117M 1s 2024-05-30T16:23:29.622 INFO:teuthology.orchestra.run.smithi006.stderr: 43500K .......... .......... .......... .......... .......... 46% 111M 1s 2024-05-30T16:23:29.622 INFO:teuthology.orchestra.run.smithi006.stderr: 43550K .......... .......... .......... .......... .......... 46% 111M 1s 2024-05-30T16:23:29.622 INFO:teuthology.orchestra.run.smithi006.stderr: 43600K .......... .......... .......... .......... .......... 46% 106M 1s 2024-05-30T16:23:29.622 INFO:teuthology.orchestra.run.smithi006.stderr: 43650K .......... .......... .......... .......... .......... 47% 126M 1s 2024-05-30T16:23:29.622 INFO:teuthology.orchestra.run.smithi006.stderr: 43700K .......... .......... .......... .......... .......... 47% 62.0M 1s 2024-05-30T16:23:29.622 INFO:teuthology.orchestra.run.smithi006.stderr: 43750K .......... .......... .......... .......... .......... 47% 68.4M 1s 2024-05-30T16:23:29.622 INFO:teuthology.orchestra.run.smithi006.stderr: 43800K .......... .......... .......... .......... .......... 47% 118M 1s 2024-05-30T16:23:29.622 INFO:teuthology.orchestra.run.smithi006.stderr: 43850K .......... .......... .......... .......... .......... 47% 105M 1s 2024-05-30T16:23:29.622 INFO:teuthology.orchestra.run.smithi006.stderr: 43900K .......... .......... .......... .......... .......... 47% 105M 1s 2024-05-30T16:23:29.622 INFO:teuthology.orchestra.run.smithi006.stderr: 43950K .......... .......... .......... .......... .......... 47% 124M 1s 2024-05-30T16:23:29.622 INFO:teuthology.orchestra.run.smithi006.stderr: 44000K .......... .......... .......... .......... .......... 47% 111M 1s 2024-05-30T16:23:29.622 INFO:teuthology.orchestra.run.smithi006.stderr: 44050K .......... .......... .......... .......... .......... 47% 119M 1s 2024-05-30T16:23:29.622 INFO:teuthology.orchestra.run.smithi006.stderr: 44100K .......... .......... .......... .......... .......... 47% 108M 1s 2024-05-30T16:23:29.623 INFO:teuthology.orchestra.run.smithi006.stderr: 44150K .......... .......... .......... .......... .......... 47% 89.9M 1s 2024-05-30T16:23:29.623 INFO:teuthology.orchestra.run.smithi006.stderr: 44200K .......... .......... .......... .......... .......... 47% 111M 1s 2024-05-30T16:23:29.623 INFO:teuthology.orchestra.run.smithi006.stderr: 44250K .......... .......... .......... .......... .......... 47% 62.3M 1s 2024-05-30T16:23:29.623 INFO:teuthology.orchestra.run.smithi006.stderr: 44300K .......... .......... .......... .......... .......... 47% 84.4M 1s 2024-05-30T16:23:29.623 INFO:teuthology.orchestra.run.smithi006.stderr: 44350K .......... .......... .......... .......... .......... 47% 103M 1s 2024-05-30T16:23:29.623 INFO:teuthology.orchestra.run.smithi006.stderr: 44400K .......... .......... .......... .......... .......... 47% 80.4M 1s 2024-05-30T16:23:29.623 INFO:teuthology.orchestra.run.smithi006.stderr: 44450K .......... .......... .......... .......... .......... 47% 20.9M 1s 2024-05-30T16:23:29.623 INFO:teuthology.orchestra.run.smithi006.stderr: 44500K .......... .......... .......... .......... .......... 47% 26.8M 1s 2024-05-30T16:23:29.623 INFO:teuthology.orchestra.run.smithi006.stderr: 44550K .......... .......... .......... .......... .......... 48% 48.6M 1s 2024-05-30T16:23:29.623 INFO:teuthology.orchestra.run.smithi006.stderr: 44600K .......... .......... .......... .......... .......... 48% 94.7M 1s 2024-05-30T16:23:29.623 INFO:teuthology.orchestra.run.smithi006.stderr: 44650K .......... .......... .......... .......... .......... 48% 110M 1s 2024-05-30T16:23:29.623 INFO:teuthology.orchestra.run.smithi006.stderr: 44700K .......... .......... .......... .......... .......... 48% 114M 1s 2024-05-30T16:23:29.623 INFO:teuthology.orchestra.run.smithi006.stderr: 44750K .......... .......... .......... .......... .......... 48% 110M 1s 2024-05-30T16:23:29.623 INFO:teuthology.orchestra.run.smithi006.stderr: 44800K .......... .......... .......... .......... .......... 48% 111M 1s 2024-05-30T16:23:29.624 INFO:teuthology.orchestra.run.smithi006.stderr: 44850K .......... .......... .......... .......... .......... 48% 114M 1s 2024-05-30T16:23:29.624 INFO:teuthology.orchestra.run.smithi006.stderr: 44900K .......... .......... .......... .......... .......... 48% 116M 1s 2024-05-30T16:23:29.624 INFO:teuthology.orchestra.run.smithi006.stderr: 44950K .......... .......... .......... .......... .......... 48% 84.0M 1s 2024-05-30T16:23:29.624 INFO:teuthology.orchestra.run.smithi006.stderr: 45000K .......... .......... .......... .......... .......... 48% 109M 1s 2024-05-30T16:23:29.624 INFO:teuthology.orchestra.run.smithi006.stderr: 45050K .......... .......... .......... .......... .......... 48% 117M 1s 2024-05-30T16:23:29.624 INFO:teuthology.orchestra.run.smithi006.stderr: 45100K .......... .......... .......... .......... .......... 48% 50.0M 1s 2024-05-30T16:23:29.624 INFO:teuthology.orchestra.run.smithi006.stderr: 45150K .......... .......... .......... .......... .......... 48% 93.9M 1s 2024-05-30T16:23:29.624 INFO:teuthology.orchestra.run.smithi006.stderr: 45200K .......... .......... .......... .......... .......... 48% 84.5M 1s 2024-05-30T16:23:29.624 INFO:teuthology.orchestra.run.smithi006.stderr: 45250K .......... .......... .......... .......... .......... 48% 64.7M 1s 2024-05-30T16:23:29.624 INFO:teuthology.orchestra.run.smithi006.stderr: 45300K .......... .......... .......... .......... .......... 48% 113M 1s 2024-05-30T16:23:29.624 INFO:teuthology.orchestra.run.smithi006.stderr: 45350K .......... .......... .......... .......... .......... 48% 31.3M 1s 2024-05-30T16:23:29.624 INFO:teuthology.orchestra.run.smithi006.stderr: 45400K .......... .......... .......... .......... .......... 48% 99.6M 1s 2024-05-30T16:23:29.624 INFO:teuthology.orchestra.run.smithi006.stderr: 45450K .......... .......... .......... .......... .......... 48% 130M 1s 2024-05-30T16:23:29.624 INFO:teuthology.orchestra.run.smithi006.stderr: 45500K .......... .......... .......... .......... .......... 49% 109M 1s 2024-05-30T16:23:29.625 INFO:teuthology.orchestra.run.smithi006.stderr: 45550K .......... .......... .......... .......... .......... 49% 115M 1s 2024-05-30T16:23:29.625 INFO:teuthology.orchestra.run.smithi006.stderr: 45600K .......... .......... .......... .......... .......... 49% 109M 1s 2024-05-30T16:23:29.625 INFO:teuthology.orchestra.run.smithi006.stderr: 45650K .......... .......... .......... .......... .......... 49% 112M 1s 2024-05-30T16:23:29.625 INFO:teuthology.orchestra.run.smithi006.stderr: 45700K .......... .......... .......... .......... .......... 49% 119M 1s 2024-05-30T16:23:29.625 INFO:teuthology.orchestra.run.smithi006.stderr: 45750K .......... .......... .......... .......... .......... 49% 87.7M 1s 2024-05-30T16:23:29.625 INFO:teuthology.orchestra.run.smithi006.stderr: 45800K .......... .......... .......... .......... .......... 49% 114M 1s 2024-05-30T16:23:29.625 INFO:teuthology.orchestra.run.smithi006.stderr: 45850K .......... .......... .......... .......... .......... 49% 111M 1s 2024-05-30T16:23:29.625 INFO:teuthology.orchestra.run.smithi006.stderr: 45900K .......... .......... .......... .......... .......... 49% 114M 1s 2024-05-30T16:23:29.625 INFO:teuthology.orchestra.run.smithi006.stderr: 45950K .......... .......... .......... .......... .......... 49% 106M 1s 2024-05-30T16:23:29.625 INFO:teuthology.orchestra.run.smithi006.stderr: 46000K .......... .......... .......... .......... .......... 49% 133M 1s 2024-05-30T16:23:29.625 INFO:teuthology.orchestra.run.smithi006.stderr: 46050K .......... .......... .......... .......... .......... 49% 79.3M 1s 2024-05-30T16:23:29.625 INFO:teuthology.orchestra.run.smithi006.stderr: 46100K .......... .......... .......... .......... .......... 49% 147M 1s 2024-05-30T16:23:29.625 INFO:teuthology.orchestra.run.smithi006.stderr: 46150K .......... .......... .......... .......... .......... 49% 112M 1s 2024-05-30T16:23:29.626 INFO:teuthology.orchestra.run.smithi006.stderr: 46200K .......... .......... .......... .......... .......... 49% 114M 1s 2024-05-30T16:23:29.626 INFO:teuthology.orchestra.run.smithi006.stderr: 46250K .......... .......... .......... .......... .......... 49% 115M 1s 2024-05-30T16:23:29.626 INFO:teuthology.orchestra.run.smithi006.stderr: 46300K .......... .......... .......... .......... .......... 49% 102M 1s 2024-05-30T16:23:29.626 INFO:teuthology.orchestra.run.smithi006.stderr: 46350K .......... .......... .......... .......... .......... 49% 119M 1s 2024-05-30T16:23:29.626 INFO:teuthology.orchestra.run.smithi006.stderr: 46400K .......... .......... .......... .......... .......... 49% 116M 1s 2024-05-30T16:23:29.626 INFO:teuthology.orchestra.run.smithi006.stderr: 46450K .......... .......... .......... .......... .......... 50% 86.8M 1s 2024-05-30T16:23:29.626 INFO:teuthology.orchestra.run.smithi006.stderr: 46500K .......... .......... .......... .......... .......... 50% 126M 1s 2024-05-30T16:23:29.626 INFO:teuthology.orchestra.run.smithi006.stderr: 46550K .......... .......... .......... .......... .......... 50% 105M 1s 2024-05-30T16:23:29.626 INFO:teuthology.orchestra.run.smithi006.stderr: 46600K .......... .......... .......... .......... .......... 50% 127M 1s 2024-05-30T16:23:29.626 INFO:teuthology.orchestra.run.smithi006.stderr: 46650K .......... .......... .......... .......... .......... 50% 109M 1s 2024-05-30T16:23:29.626 INFO:teuthology.orchestra.run.smithi006.stderr: 46700K .......... .......... .......... .......... .......... 50% 114M 1s 2024-05-30T16:23:29.626 INFO:teuthology.orchestra.run.smithi006.stderr: 46750K .......... .......... .......... .......... .......... 50% 92.1M 1s 2024-05-30T16:23:29.626 INFO:teuthology.orchestra.run.smithi006.stderr: 46800K .......... .......... .......... .......... .......... 50% 111M 1s 2024-05-30T16:23:29.626 INFO:teuthology.orchestra.run.smithi006.stderr: 46850K .......... .......... .......... .......... .......... 50% 42.8M 1s 2024-05-30T16:23:29.627 INFO:teuthology.orchestra.run.smithi006.stderr: 46900K .......... .......... .......... .......... .......... 50% 90.5M 1s 2024-05-30T16:23:29.627 INFO:teuthology.orchestra.run.smithi006.stderr: 46950K .......... .......... .......... .......... .......... 50% 115M 1s 2024-05-30T16:23:29.627 INFO:teuthology.orchestra.run.smithi006.stderr: 47000K .......... .......... .......... .......... .......... 50% 101M 1s 2024-05-30T16:23:29.627 INFO:teuthology.orchestra.run.smithi006.stderr: 47050K .......... .......... .......... .......... .......... 50% 111M 1s 2024-05-30T16:23:29.627 INFO:teuthology.orchestra.run.smithi006.stderr: 47100K .......... .......... .......... .......... .......... 50% 114M 1s 2024-05-30T16:23:29.627 INFO:teuthology.orchestra.run.smithi006.stderr: 47150K .......... .......... .......... .......... .......... 50% 110M 1s 2024-05-30T16:23:29.627 INFO:teuthology.orchestra.run.smithi006.stderr: 47200K .......... .......... .......... .......... .......... 50% 107M 1s 2024-05-30T16:23:29.627 INFO:teuthology.orchestra.run.smithi006.stderr: 47250K .......... .......... .......... .......... .......... 50% 82.8M 1s 2024-05-30T16:23:29.627 INFO:teuthology.orchestra.run.smithi006.stderr: 47300K .......... .......... .......... .......... .......... 50% 69.1M 1s 2024-05-30T16:23:29.627 INFO:teuthology.orchestra.run.smithi006.stderr: 47350K .......... .......... .......... .......... .......... 51% 111M 1s 2024-05-30T16:23:29.627 INFO:teuthology.orchestra.run.smithi006.stderr: 47400K .......... .......... .......... .......... .......... 51% 53.7M 1s 2024-05-30T16:23:29.627 INFO:teuthology.orchestra.run.smithi006.stderr: 47450K .......... .......... .......... .......... .......... 51% 50.0M 1s 2024-05-30T16:23:29.627 INFO:teuthology.orchestra.run.smithi006.stderr: 47500K .......... .......... .......... .......... .......... 51% 17.9M 1s 2024-05-30T16:23:29.627 INFO:teuthology.orchestra.run.smithi006.stderr: 47550K .......... .......... .......... .......... .......... 51% 23.3M 1s 2024-05-30T16:23:29.628 INFO:teuthology.orchestra.run.smithi006.stderr: 47600K .......... .......... .......... .......... .......... 51% 49.6M 1s 2024-05-30T16:23:29.628 INFO:teuthology.orchestra.run.smithi006.stderr: 47650K .......... .......... .......... .......... .......... 51% 12.9M 1s 2024-05-30T16:23:29.628 INFO:teuthology.orchestra.run.smithi006.stderr: 47700K .......... .......... .......... .......... .......... 51% 94.8M 1s 2024-05-30T16:23:29.628 INFO:teuthology.orchestra.run.smithi006.stderr: 47750K .......... .......... .......... .......... .......... 51% 117M 1s 2024-05-30T16:23:29.628 INFO:teuthology.orchestra.run.smithi006.stderr: 47800K .......... .......... .......... .......... .......... 51% 111M 1s 2024-05-30T16:23:29.628 INFO:teuthology.orchestra.run.smithi006.stderr: 47850K .......... .......... .......... .......... .......... 51% 116M 1s 2024-05-30T16:23:29.628 INFO:teuthology.orchestra.run.smithi006.stderr: 47900K .......... .......... .......... .......... .......... 51% 109M 1s 2024-05-30T16:23:29.628 INFO:teuthology.orchestra.run.smithi006.stderr: 47950K .......... .......... .......... .......... .......... 51% 112M 1s 2024-05-30T16:23:29.628 INFO:teuthology.orchestra.run.smithi006.stderr: 48000K .......... .......... .......... .......... .......... 51% 113M 1s 2024-05-30T16:23:29.628 INFO:teuthology.orchestra.run.smithi006.stderr: 48050K .......... .......... .......... .......... .......... 51% 55.3M 1s 2024-05-30T16:23:29.628 INFO:teuthology.orchestra.run.smithi006.stderr: 48100K .......... .......... .......... .......... .......... 51% 113M 1s 2024-05-30T16:23:29.628 INFO:teuthology.orchestra.run.smithi006.stderr: 48150K .......... .......... .......... .......... .......... 51% 86.5M 1s 2024-05-30T16:23:29.628 INFO:teuthology.orchestra.run.smithi006.stderr: 48200K .......... .......... .......... .......... .......... 51% 116M 1s 2024-05-30T16:23:29.628 INFO:teuthology.orchestra.run.smithi006.stderr: 48250K .......... .......... .......... .......... .......... 51% 113M 1s 2024-05-30T16:23:29.629 INFO:teuthology.orchestra.run.smithi006.stderr: 48300K .......... .......... .......... .......... .......... 52% 112M 1s 2024-05-30T16:23:29.629 INFO:teuthology.orchestra.run.smithi006.stderr: 48350K .......... .......... .......... .......... .......... 52% 85.0M 1s 2024-05-30T16:23:29.629 INFO:teuthology.orchestra.run.smithi006.stderr: 48400K .......... .......... .......... .......... .......... 52% 90.8M 1s 2024-05-30T16:23:29.629 INFO:teuthology.orchestra.run.smithi006.stderr: 48450K .......... .......... .......... .......... .......... 52% 129M 1s 2024-05-30T16:23:29.629 INFO:teuthology.orchestra.run.smithi006.stderr: 48500K .......... .......... .......... .......... .......... 52% 118M 1s 2024-05-30T16:23:29.629 INFO:teuthology.orchestra.run.smithi006.stderr: 48550K .......... .......... .......... .......... .......... 52% 124M 1s 2024-05-30T16:23:29.629 INFO:teuthology.orchestra.run.smithi006.stderr: 48600K .......... .......... .......... .......... .......... 52% 110M 1s 2024-05-30T16:23:29.629 INFO:teuthology.orchestra.run.smithi006.stderr: 48650K .......... .......... .......... .......... .......... 52% 90.6M 1s 2024-05-30T16:23:29.629 INFO:teuthology.orchestra.run.smithi006.stderr: 48700K .......... .......... .......... .......... .......... 52% 112M 1s 2024-05-30T16:23:29.629 INFO:teuthology.orchestra.run.smithi006.stderr: 48750K .......... .......... .......... .......... .......... 52% 107M 1s 2024-05-30T16:23:29.629 INFO:teuthology.orchestra.run.smithi006.stderr: 48800K .......... .......... .......... .......... .......... 52% 127M 1s 2024-05-30T16:23:29.629 INFO:teuthology.orchestra.run.smithi006.stderr: 48850K .......... .......... .......... .......... .......... 52% 111M 1s 2024-05-30T16:23:29.629 INFO:teuthology.orchestra.run.smithi006.stderr: 48900K .......... .......... .......... .......... .......... 52% 108M 1s 2024-05-30T16:23:29.629 INFO:teuthology.orchestra.run.smithi006.stderr: 48950K .......... .......... .......... .......... .......... 52% 112M 1s 2024-05-30T16:23:29.629 INFO:teuthology.orchestra.run.smithi006.stderr: 49000K .......... .......... .......... .......... .......... 52% 130M 1s 2024-05-30T16:23:29.630 INFO:teuthology.orchestra.run.smithi006.stderr: 49050K .......... .......... .......... .......... .......... 52% 89.0M 1s 2024-05-30T16:23:29.630 INFO:teuthology.orchestra.run.smithi006.stderr: 49100K .......... .......... .......... .......... .......... 52% 117M 1s 2024-05-30T16:23:29.630 INFO:teuthology.orchestra.run.smithi006.stderr: 49150K .......... .......... .......... .......... .......... 52% 112M 1s 2024-05-30T16:23:29.630 INFO:teuthology.orchestra.run.smithi006.stderr: 49200K .......... .......... .......... .......... .......... 53% 110M 1s 2024-05-30T16:23:29.630 INFO:teuthology.orchestra.run.smithi006.stderr: 49250K .......... .......... .......... .......... .......... 53% 121M 1s 2024-05-30T16:23:29.630 INFO:teuthology.orchestra.run.smithi006.stderr: 49300K .......... .......... .......... .......... .......... 53% 83.6M 1s 2024-05-30T16:23:29.630 INFO:teuthology.orchestra.run.smithi006.stderr: 49350K .......... .......... .......... .......... .......... 53% 122M 1s 2024-05-30T16:23:29.630 INFO:teuthology.orchestra.run.smithi006.stderr: 49400K .......... .......... .......... .......... .......... 53% 114M 1s 2024-05-30T16:23:29.630 INFO:teuthology.orchestra.run.smithi006.stderr: 49450K .......... .......... .......... .......... .......... 53% 106M 1s 2024-05-30T16:23:29.630 INFO:teuthology.orchestra.run.smithi006.stderr: 49500K .......... .......... .......... .......... .......... 53% 116M 1s 2024-05-30T16:23:29.630 INFO:teuthology.orchestra.run.smithi006.stderr: 49550K .......... .......... .......... .......... .......... 53% 105M 1s 2024-05-30T16:23:29.630 INFO:teuthology.orchestra.run.smithi006.stderr: 49600K .......... .......... .......... .......... .......... 53% 119M 1s 2024-05-30T16:23:29.630 INFO:teuthology.orchestra.run.smithi006.stderr: 49650K .......... .......... .......... .......... .......... 53% 123M 1s 2024-05-30T16:23:29.631 INFO:teuthology.orchestra.run.smithi006.stderr: 49700K .......... .......... .......... .......... .......... 53% 97.1M 1s 2024-05-30T16:23:29.631 INFO:teuthology.orchestra.run.smithi006.stderr: 49750K .......... .......... .......... .......... .......... 53% 116M 1s 2024-05-30T16:23:29.631 INFO:teuthology.orchestra.run.smithi006.stderr: 49800K .......... .......... .......... .......... .......... 53% 106M 1s 2024-05-30T16:23:29.631 INFO:teuthology.orchestra.run.smithi006.stderr: 49850K .......... .......... .......... .......... .......... 53% 111M 1s 2024-05-30T16:23:29.631 INFO:teuthology.orchestra.run.smithi006.stderr: 49900K .......... .......... .......... .......... .......... 53% 99.4M 1s 2024-05-30T16:23:29.631 INFO:teuthology.orchestra.run.smithi006.stderr: 49950K .......... .......... .......... .......... .......... 53% 79.3M 1s 2024-05-30T16:23:29.631 INFO:teuthology.orchestra.run.smithi006.stderr: 50000K .......... .......... .......... .......... .......... 53% 142M 1s 2024-05-30T16:23:29.631 INFO:teuthology.orchestra.run.smithi006.stderr: 50050K .......... .......... .......... .......... .......... 53% 120M 1s 2024-05-30T16:23:29.631 INFO:teuthology.orchestra.run.smithi006.stderr: 50100K .......... .......... .......... .......... .......... 53% 111M 1s 2024-05-30T16:23:29.631 INFO:teuthology.orchestra.run.smithi006.stderr: 50150K .......... .......... .......... .......... .......... 54% 116M 1s 2024-05-30T16:23:29.631 INFO:teuthology.orchestra.run.smithi006.stderr: 50200K .......... .......... .......... .......... .......... 54% 108M 1s 2024-05-30T16:23:29.631 INFO:teuthology.orchestra.run.smithi006.stderr: 50250K .......... .......... .......... .......... .......... 54% 116M 1s 2024-05-30T16:23:29.632 INFO:teuthology.orchestra.run.smithi006.stderr: 50300K .......... .......... .......... .......... .......... 54% 112M 1s 2024-05-30T16:23:29.632 INFO:teuthology.orchestra.run.smithi006.stderr: 50350K .......... .......... .......... .......... .......... 54% 84.6M 1s 2024-05-30T16:23:29.632 INFO:teuthology.orchestra.run.smithi006.stderr: 50400K .......... .......... .......... .......... .......... 54% 27.0M 1s 2024-05-30T16:23:29.632 INFO:teuthology.orchestra.run.smithi006.stderr: 50450K .......... .......... .......... .......... .......... 54% 121M 0s 2024-05-30T16:23:29.632 INFO:teuthology.orchestra.run.smithi006.stderr: 50500K .......... .......... .......... .......... .......... 54% 32.2M 1s 2024-05-30T16:23:29.632 INFO:teuthology.orchestra.run.smithi006.stderr: 50550K .......... .......... .......... .......... .......... 54% 17.7M 1s 2024-05-30T16:23:29.632 INFO:teuthology.orchestra.run.smithi006.stderr: 50600K .......... .......... .......... .......... .......... 54% 10.1M 1s 2024-05-30T16:23:29.632 INFO:teuthology.orchestra.run.smithi006.stderr: 50650K .......... .......... .......... .......... .......... 54% 31.5M 1s 2024-05-30T16:23:29.632 INFO:teuthology.orchestra.run.smithi006.stderr: 50700K .......... .......... .......... .......... .......... 54% 9.94M 1s 2024-05-30T16:23:29.632 INFO:teuthology.orchestra.run.smithi006.stderr: 50750K .......... .......... .......... .......... .......... 54% 31.8M 1s 2024-05-30T16:23:29.632 INFO:teuthology.orchestra.run.smithi006.stderr: 50800K .......... .......... .......... .......... .......... 54% 22.2M 1s 2024-05-30T16:23:29.632 INFO:teuthology.orchestra.run.smithi006.stderr: 50850K .......... .......... .......... .......... .......... 54% 49.5M 1s 2024-05-30T16:23:29.632 INFO:teuthology.orchestra.run.smithi006.stderr: 50900K .......... .......... .......... .......... .......... 54% 119M 1s 2024-05-30T16:23:29.632 INFO:teuthology.orchestra.run.smithi006.stderr: 50950K .......... .......... .......... .......... .......... 54% 108M 1s 2024-05-30T16:23:29.633 INFO:teuthology.orchestra.run.smithi006.stderr: 51000K .......... .......... .......... .......... .......... 54% 101M 1s 2024-05-30T16:23:29.633 INFO:teuthology.orchestra.run.smithi006.stderr: 51050K .......... .......... .......... .......... .......... 55% 123M 1s 2024-05-30T16:23:29.633 INFO:teuthology.orchestra.run.smithi006.stderr: 51100K .......... .......... .......... .......... .......... 55% 116M 1s 2024-05-30T16:23:29.633 INFO:teuthology.orchestra.run.smithi006.stderr: 51150K .......... .......... .......... .......... .......... 55% 92.4M 1s 2024-05-30T16:23:29.633 INFO:teuthology.orchestra.run.smithi006.stderr: 51200K .......... .......... .......... .......... .......... 55% 75.9M 1s 2024-05-30T16:23:29.633 INFO:teuthology.orchestra.run.smithi006.stderr: 51250K .......... .......... .......... .......... .......... 55% 107M 1s 2024-05-30T16:23:29.633 INFO:teuthology.orchestra.run.smithi006.stderr: 51300K .......... .......... .......... .......... .......... 55% 115M 1s 2024-05-30T16:23:29.633 INFO:teuthology.orchestra.run.smithi006.stderr: 51350K .......... .......... .......... .......... .......... 55% 123M 1s 2024-05-30T16:23:29.633 INFO:teuthology.orchestra.run.smithi006.stderr: 51400K .......... .......... .......... .......... .......... 55% 107M 1s 2024-05-30T16:23:29.633 INFO:teuthology.orchestra.run.smithi006.stderr: 51450K .......... .......... .......... .......... .......... 55% 113M 0s 2024-05-30T16:23:29.633 INFO:teuthology.orchestra.run.smithi006.stderr: 51500K .......... .......... .......... .......... .......... 55% 116M 0s 2024-05-30T16:23:29.633 INFO:teuthology.orchestra.run.smithi006.stderr: 51550K .......... .......... .......... .......... .......... 55% 118M 0s 2024-05-30T16:23:29.633 INFO:teuthology.orchestra.run.smithi006.stderr: 51600K .......... .......... .......... .......... .......... 55% 88.5M 0s 2024-05-30T16:23:29.634 INFO:teuthology.orchestra.run.smithi006.stderr: 51650K .......... .......... .......... .......... .......... 55% 114M 0s 2024-05-30T16:23:29.634 INFO:teuthology.orchestra.run.smithi006.stderr: 51700K .......... .......... .......... .......... .......... 55% 109M 0s 2024-05-30T16:23:29.634 INFO:teuthology.orchestra.run.smithi006.stderr: 51750K .......... .......... .......... .......... .......... 55% 122M 0s 2024-05-30T16:23:29.634 INFO:teuthology.orchestra.run.smithi006.stderr: 51800K .......... .......... .......... .......... .......... 55% 108M 0s 2024-05-30T16:23:29.634 INFO:teuthology.orchestra.run.smithi006.stderr: 51850K .......... .......... .......... .......... .......... 55% 115M 0s 2024-05-30T16:23:29.634 INFO:teuthology.orchestra.run.smithi006.stderr: 51900K .......... .......... .......... .......... .......... 55% 113M 0s 2024-05-30T16:23:29.634 INFO:teuthology.orchestra.run.smithi006.stderr: 51950K .......... .......... .......... .......... .......... 55% 115M 0s 2024-05-30T16:23:29.634 INFO:teuthology.orchestra.run.smithi006.stderr: 52000K .......... .......... .......... .......... .......... 56% 87.5M 0s 2024-05-30T16:23:29.634 INFO:teuthology.orchestra.run.smithi006.stderr: 52050K .......... .......... .......... .......... .......... 56% 122M 0s 2024-05-30T16:23:29.634 INFO:teuthology.orchestra.run.smithi006.stderr: 52100K .......... .......... .......... .......... .......... 56% 106M 0s 2024-05-30T16:23:29.634 INFO:teuthology.orchestra.run.smithi006.stderr: 52150K .......... .......... .......... .......... .......... 56% 55.1M 0s 2024-05-30T16:23:29.634 INFO:teuthology.orchestra.run.smithi006.stderr: 52200K .......... .......... .......... .......... .......... 56% 92.4M 0s 2024-05-30T16:23:29.634 INFO:teuthology.orchestra.run.smithi006.stderr: 52250K .......... .......... .......... .......... .......... 56% 109M 0s 2024-05-30T16:23:29.635 INFO:teuthology.orchestra.run.smithi006.stderr: 52300K .......... .......... .......... .......... .......... 56% 88.9M 0s 2024-05-30T16:23:29.635 INFO:teuthology.orchestra.run.smithi006.stderr: 52350K .......... .......... .......... .......... .......... 56% 115M 0s 2024-05-30T16:23:29.635 INFO:teuthology.orchestra.run.smithi006.stderr: 52400K .......... .......... .......... .......... .......... 56% 109M 0s 2024-05-30T16:23:29.635 INFO:teuthology.orchestra.run.smithi006.stderr: 52450K .......... .......... .......... .......... .......... 56% 118M 0s 2024-05-30T16:23:29.635 INFO:teuthology.orchestra.run.smithi006.stderr: 52500K .......... .......... .......... .......... .......... 56% 110M 0s 2024-05-30T16:23:29.635 INFO:teuthology.orchestra.run.smithi006.stderr: 52550K .......... .......... .......... .......... .......... 56% 111M 0s 2024-05-30T16:23:29.635 INFO:teuthology.orchestra.run.smithi006.stderr: 52600K .......... .......... .......... .......... .......... 56% 125M 0s 2024-05-30T16:23:29.635 INFO:teuthology.orchestra.run.smithi006.stderr: 52650K .......... .......... .......... .......... .......... 56% 112M 0s 2024-05-30T16:23:29.635 INFO:teuthology.orchestra.run.smithi006.stderr: 52700K .......... .......... .......... .......... .......... 56% 79.2M 0s 2024-05-30T16:23:29.635 INFO:teuthology.orchestra.run.smithi006.stderr: 52750K .......... .......... .......... .......... .......... 56% 109M 0s 2024-05-30T16:23:29.635 INFO:teuthology.orchestra.run.smithi006.stderr: 52800K .......... .......... .......... .......... .......... 56% 129M 0s 2024-05-30T16:23:29.635 INFO:teuthology.orchestra.run.smithi006.stderr: 52850K .......... .......... .......... .......... .......... 56% 122M 0s 2024-05-30T16:23:29.635 INFO:teuthology.orchestra.run.smithi006.stderr: 52900K .......... .......... .......... .......... .......... 56% 100M 0s 2024-05-30T16:23:29.636 INFO:teuthology.orchestra.run.smithi006.stderr: 52950K .......... .......... .......... .......... .......... 57% 121M 0s 2024-05-30T16:23:29.636 INFO:teuthology.orchestra.run.smithi006.stderr: 53000K .......... .......... .......... .......... .......... 57% 112M 0s 2024-05-30T16:23:29.636 INFO:teuthology.orchestra.run.smithi006.stderr: 53050K .......... .......... .......... .......... .......... 57% 113M 0s 2024-05-30T16:23:29.636 INFO:teuthology.orchestra.run.smithi006.stderr: 53100K .......... .......... .......... .......... .......... 57% 71.9M 0s 2024-05-30T16:23:29.636 INFO:teuthology.orchestra.run.smithi006.stderr: 53150K .......... .......... .......... .......... .......... 57% 104M 0s 2024-05-30T16:23:29.636 INFO:teuthology.orchestra.run.smithi006.stderr: 53200K .......... .......... .......... .......... .......... 57% 62.1M 0s 2024-05-30T16:23:29.636 INFO:teuthology.orchestra.run.smithi006.stderr: 53250K .......... .......... .......... .......... .......... 57% 137M 0s 2024-05-30T16:23:29.636 INFO:teuthology.orchestra.run.smithi006.stderr: 53300K .......... .......... .......... .......... .......... 57% 95.8M 0s 2024-05-30T16:23:29.636 INFO:teuthology.orchestra.run.smithi006.stderr: 53350K .......... .......... .......... .......... .......... 57% 124M 0s 2024-05-30T16:23:29.636 INFO:teuthology.orchestra.run.smithi006.stderr: 53400K .......... .......... .......... .......... .......... 57% 84.4M 0s 2024-05-30T16:23:29.636 INFO:teuthology.orchestra.run.smithi006.stderr: 53450K .......... .......... .......... .......... .......... 57% 123M 0s 2024-05-30T16:23:29.636 INFO:teuthology.orchestra.run.smithi006.stderr: 53500K .......... .......... .......... .......... .......... 57% 17.4M 0s 2024-05-30T16:23:29.636 INFO:teuthology.orchestra.run.smithi006.stderr: 53550K .......... .......... .......... .......... .......... 57% 23.3M 0s 2024-05-30T16:23:29.636 INFO:teuthology.orchestra.run.smithi006.stderr: 53600K .......... .......... .......... .......... .......... 57% 7.32M 0s 2024-05-30T16:23:29.637 INFO:teuthology.orchestra.run.smithi006.stderr: 53650K .......... .......... .......... .......... .......... 57% 11.0M 0s 2024-05-30T16:23:29.637 INFO:teuthology.orchestra.run.smithi006.stderr: 53700K .......... .......... .......... .......... .......... 57% 15.3M 0s 2024-05-30T16:23:29.637 INFO:teuthology.orchestra.run.smithi006.stderr: 53750K .......... .......... .......... .......... .......... 57% 6.33M 0s 2024-05-30T16:23:29.637 INFO:teuthology.orchestra.run.smithi006.stderr: 53800K .......... .......... .......... .......... .......... 57% 18.1M 0s 2024-05-30T16:23:29.637 INFO:teuthology.orchestra.run.smithi006.stderr: 53850K .......... .......... .......... .......... .......... 58% 31.9M 0s 2024-05-30T16:23:29.637 INFO:teuthology.orchestra.run.smithi006.stderr: 53900K .......... .......... .......... .......... .......... 58% 14.7M 0s 2024-05-30T16:23:29.637 INFO:teuthology.orchestra.run.smithi006.stderr: 53950K .......... .......... .......... .......... .......... 58% 109M 0s 2024-05-30T16:23:29.637 INFO:teuthology.orchestra.run.smithi006.stderr: 54000K .......... .......... .......... .......... .......... 58% 1.21M 1s 2024-05-30T16:23:29.637 INFO:teuthology.orchestra.run.smithi006.stderr: 54050K .......... .......... .......... .......... .......... 58% 92.8M 1s 2024-05-30T16:23:29.637 INFO:teuthology.orchestra.run.smithi006.stderr: 54100K .......... .......... .......... .......... .......... 58% 104M 1s 2024-05-30T16:23:29.637 INFO:teuthology.orchestra.run.smithi006.stderr: 54150K .......... .......... .......... .......... .......... 58% 122M 1s 2024-05-30T16:23:29.637 INFO:teuthology.orchestra.run.smithi006.stderr: 54200K .......... .......... .......... .......... .......... 58% 142M 1s 2024-05-30T16:23:29.637 INFO:teuthology.orchestra.run.smithi006.stderr: 54250K .......... .......... .......... .......... .......... 58% 105M 1s 2024-05-30T16:23:29.637 INFO:teuthology.orchestra.run.smithi006.stderr: 54300K .......... .......... .......... .......... .......... 58% 126M 1s 2024-05-30T16:23:29.638 INFO:teuthology.orchestra.run.smithi006.stderr: 54350K .......... .......... .......... .......... .......... 58% 179M 1s 2024-05-30T16:23:29.638 INFO:teuthology.orchestra.run.smithi006.stderr: 54400K .......... .......... .......... .......... .......... 58% 181M 1s 2024-05-30T16:23:29.638 INFO:teuthology.orchestra.run.smithi006.stderr: 54450K .......... .......... .......... .......... .......... 58% 156M 1s 2024-05-30T16:23:29.638 INFO:teuthology.orchestra.run.smithi006.stderr: 54500K .......... .......... .......... .......... .......... 58% 160M 1s 2024-05-30T16:23:29.638 INFO:teuthology.orchestra.run.smithi006.stderr: 54550K .......... .......... .......... .......... .......... 58% 182M 1s 2024-05-30T16:23:29.638 INFO:teuthology.orchestra.run.smithi006.stderr: 54600K .......... .......... .......... .......... .......... 58% 162M 1s 2024-05-30T16:23:29.638 INFO:teuthology.orchestra.run.smithi006.stderr: 54650K .......... .......... .......... .......... .......... 58% 182M 1s 2024-05-30T16:23:29.638 INFO:teuthology.orchestra.run.smithi006.stderr: 54700K .......... .......... .......... .......... .......... 58% 150M 1s 2024-05-30T16:23:29.638 INFO:teuthology.orchestra.run.smithi006.stderr: 54750K .......... .......... .......... .......... .......... 58% 159M 1s 2024-05-30T16:23:29.638 INFO:teuthology.orchestra.run.smithi006.stderr: 54800K .......... .......... .......... .......... .......... 59% 175M 1s 2024-05-30T16:23:29.638 INFO:teuthology.orchestra.run.smithi006.stderr: 54850K .......... .......... .......... .......... .......... 59% 179M 0s 2024-05-30T16:23:29.638 INFO:teuthology.orchestra.run.smithi006.stderr: 54900K .......... .......... .......... .......... .......... 59% 146M 0s 2024-05-30T16:23:29.638 INFO:teuthology.orchestra.run.smithi006.stderr: 54950K .......... .......... .......... .......... .......... 59% 177M 0s 2024-05-30T16:23:29.639 INFO:teuthology.orchestra.run.smithi006.stderr: 55000K .......... .......... .......... .......... .......... 59% 161M 0s 2024-05-30T16:23:29.639 INFO:teuthology.orchestra.run.smithi006.stderr: 55050K .......... .......... .......... .......... .......... 59% 171M 0s 2024-05-30T16:23:29.639 INFO:teuthology.orchestra.run.smithi006.stderr: 55100K .......... .......... .......... .......... .......... 59% 147M 0s 2024-05-30T16:23:29.639 INFO:teuthology.orchestra.run.smithi006.stderr: 55150K .......... .......... .......... .......... .......... 59% 146M 0s 2024-05-30T16:23:29.639 INFO:teuthology.orchestra.run.smithi006.stderr: 55200K .......... .......... .......... .......... .......... 59% 171M 0s 2024-05-30T16:23:29.639 INFO:teuthology.orchestra.run.smithi006.stderr: 55250K .......... .......... .......... .......... .......... 59% 181M 0s 2024-05-30T16:23:29.639 INFO:teuthology.orchestra.run.smithi006.stderr: 55300K .......... .......... .......... .......... .......... 59% 146M 0s 2024-05-30T16:23:29.639 INFO:teuthology.orchestra.run.smithi006.stderr: 55350K .......... .......... .......... .......... .......... 59% 173M 0s 2024-05-30T16:23:29.639 INFO:teuthology.orchestra.run.smithi006.stderr: 55400K .......... .......... .......... .......... .......... 59% 181M 0s 2024-05-30T16:23:29.639 INFO:teuthology.orchestra.run.smithi006.stderr: 55450K .......... .......... .......... .......... .......... 59% 163M 0s 2024-05-30T16:23:29.639 INFO:teuthology.orchestra.run.smithi006.stderr: 55500K .......... .......... .......... .......... .......... 59% 148M 0s 2024-05-30T16:23:29.639 INFO:teuthology.orchestra.run.smithi006.stderr: 55550K .......... .......... .......... .......... .......... 59% 161M 0s 2024-05-30T16:23:29.639 INFO:teuthology.orchestra.run.smithi006.stderr: 55600K .......... .......... .......... .......... .......... 59% 164M 0s 2024-05-30T16:23:29.640 INFO:teuthology.orchestra.run.smithi006.stderr: 55650K .......... .......... .......... .......... .......... 59% 179M 0s 2024-05-30T16:23:29.640 INFO:teuthology.orchestra.run.smithi006.stderr: 55700K .......... .......... .......... .......... .......... 60% 152M 0s 2024-05-30T16:23:29.640 INFO:teuthology.orchestra.run.smithi006.stderr: 55750K .......... .......... .......... .......... .......... 60% 180M 0s 2024-05-30T16:23:29.640 INFO:teuthology.orchestra.run.smithi006.stderr: 55800K .......... .......... .......... .......... .......... 60% 171M 0s 2024-05-30T16:23:29.640 INFO:teuthology.orchestra.run.smithi006.stderr: 55850K .......... .......... .......... .......... .......... 60% 159M 0s 2024-05-30T16:23:29.640 INFO:teuthology.orchestra.run.smithi006.stderr: 55900K .......... .......... .......... .......... .......... 60% 148M 0s 2024-05-30T16:23:29.640 INFO:teuthology.orchestra.run.smithi006.stderr: 55950K .......... .......... .......... .......... .......... 60% 173M 0s 2024-05-30T16:23:29.640 INFO:teuthology.orchestra.run.smithi006.stderr: 56000K .......... .......... .......... .......... .......... 60% 173M 0s 2024-05-30T16:23:29.640 INFO:teuthology.orchestra.run.smithi006.stderr: 56050K .......... .......... .......... .......... .......... 60% 175M 0s 2024-05-30T16:23:29.640 INFO:teuthology.orchestra.run.smithi006.stderr: 56100K .......... .......... .......... .......... .......... 60% 143M 0s 2024-05-30T16:23:29.640 INFO:teuthology.orchestra.run.smithi006.stderr: 56150K .......... .......... .......... .......... .......... 60% 176M 0s 2024-05-30T16:23:29.640 INFO:teuthology.orchestra.run.smithi006.stderr: 56200K .......... .......... .......... .......... .......... 60% 178M 0s 2024-05-30T16:23:29.640 INFO:teuthology.orchestra.run.smithi006.stderr: 56250K .......... .......... .......... .......... .......... 60% 145M 0s 2024-05-30T16:23:29.641 INFO:teuthology.orchestra.run.smithi006.stderr: 56300K .......... .......... .......... .......... .......... 60% 159M 0s 2024-05-30T16:23:29.641 INFO:teuthology.orchestra.run.smithi006.stderr: 56350K .......... .......... .......... .......... .......... 60% 179M 0s 2024-05-30T16:23:29.641 INFO:teuthology.orchestra.run.smithi006.stderr: 56400K .......... .......... .......... .......... .......... 60% 137M 0s 2024-05-30T16:23:29.641 INFO:teuthology.orchestra.run.smithi006.stderr: 56450K .......... .......... .......... .......... .......... 60% 157M 0s 2024-05-30T16:23:29.641 INFO:teuthology.orchestra.run.smithi006.stderr: 56500K .......... .......... .......... .......... .......... 60% 180M 0s 2024-05-30T16:23:29.641 INFO:teuthology.orchestra.run.smithi006.stderr: 56550K .......... .......... .......... .......... .......... 60% 170M 0s 2024-05-30T16:23:29.641 INFO:teuthology.orchestra.run.smithi006.stderr: 56600K .......... .......... .......... .......... .......... 60% 134M 0s 2024-05-30T16:23:29.641 INFO:teuthology.orchestra.run.smithi006.stderr: 56650K .......... .......... .......... .......... .......... 61% 159M 0s 2024-05-30T16:23:29.641 INFO:teuthology.orchestra.run.smithi006.stderr: 56700K .......... .......... .......... .......... .......... 61% 168M 0s 2024-05-30T16:23:29.641 INFO:teuthology.orchestra.run.smithi006.stderr: 56750K .......... .......... .......... .......... .......... 61% 108M 0s 2024-05-30T16:23:29.641 INFO:teuthology.orchestra.run.smithi006.stderr: 56800K .......... .......... .......... .......... .......... 61% 97.7M 0s 2024-05-30T16:23:29.641 INFO:teuthology.orchestra.run.smithi006.stderr: 56850K .......... .......... .......... .......... .......... 61% 162M 0s 2024-05-30T16:23:29.641 INFO:teuthology.orchestra.run.smithi006.stderr: 56900K .......... .......... .......... .......... .......... 61% 152M 0s 2024-05-30T16:23:29.642 INFO:teuthology.orchestra.run.smithi006.stderr: 56950K .......... .......... .......... .......... .......... 61% 140M 0s 2024-05-30T16:23:29.642 INFO:teuthology.orchestra.run.smithi006.stderr: 57000K .......... .......... .......... .......... .......... 61% 149M 0s 2024-05-30T16:23:29.642 INFO:teuthology.orchestra.run.smithi006.stderr: 57050K .......... .......... .......... .......... .......... 61% 168M 0s 2024-05-30T16:23:29.642 INFO:teuthology.orchestra.run.smithi006.stderr: 57100K .......... .......... .......... .......... .......... 61% 141M 0s 2024-05-30T16:23:29.642 INFO:teuthology.orchestra.run.smithi006.stderr: 57150K .......... .......... .......... .......... .......... 61% 171M 0s 2024-05-30T16:23:29.642 INFO:teuthology.orchestra.run.smithi006.stderr: 57200K .......... .......... .......... .......... .......... 61% 155M 0s 2024-05-30T16:23:29.642 INFO:teuthology.orchestra.run.smithi006.stderr: 57250K .......... .......... .......... .......... .......... 61% 145M 0s 2024-05-30T16:23:29.642 INFO:teuthology.orchestra.run.smithi006.stderr: 57300K .......... .......... .......... .......... .......... 61% 167M 0s 2024-05-30T16:23:29.642 INFO:teuthology.orchestra.run.smithi006.stderr: 57350K .......... .......... .......... .......... .......... 61% 27.7M 0s 2024-05-30T16:23:29.642 INFO:teuthology.orchestra.run.smithi006.stderr: 57400K .......... .......... .......... .......... .......... 61% 14.4M 0s 2024-05-30T16:23:29.642 INFO:teuthology.orchestra.run.smithi006.stderr: 57450K .......... .......... .......... .......... .......... 61% 60.2M 0s 2024-05-30T16:23:29.642 INFO:teuthology.orchestra.run.smithi006.stderr: 57500K .......... .......... .......... .......... .......... 61% 26.3M 0s 2024-05-30T16:23:29.642 INFO:teuthology.orchestra.run.smithi006.stderr: 57550K .......... .......... .......... .......... .......... 62% 55.8M 0s 2024-05-30T16:23:29.643 INFO:teuthology.orchestra.run.smithi006.stderr: 57600K .......... .......... .......... .......... .......... 62% 29.3M 0s 2024-05-30T16:23:29.643 INFO:teuthology.orchestra.run.smithi006.stderr: 57650K .......... .......... .......... .......... .......... 62% 71.9M 0s 2024-05-30T16:23:29.643 INFO:teuthology.orchestra.run.smithi006.stderr: 57700K .......... .......... .......... .......... .......... 62% 41.4M 0s 2024-05-30T16:23:29.643 INFO:teuthology.orchestra.run.smithi006.stderr: 57750K .......... .......... .......... .......... .......... 62% 21.4M 0s 2024-05-30T16:23:29.643 INFO:teuthology.orchestra.run.smithi006.stderr: 57800K .......... .......... .......... .......... .......... 62% 42.4M 0s 2024-05-30T16:23:29.643 INFO:teuthology.orchestra.run.smithi006.stderr: 57850K .......... .......... .......... .......... .......... 62% 41.7M 0s 2024-05-30T16:23:29.643 INFO:teuthology.orchestra.run.smithi006.stderr: 57900K .......... .......... .......... .......... .......... 62% 113M 0s 2024-05-30T16:23:29.643 INFO:teuthology.orchestra.run.smithi006.stderr: 57950K .......... .......... .......... .......... .......... 62% 75.5M 0s 2024-05-30T16:23:29.643 INFO:teuthology.orchestra.run.smithi006.stderr: 58000K .......... .......... .......... .......... .......... 62% 76.2M 0s 2024-05-30T16:23:29.643 INFO:teuthology.orchestra.run.smithi006.stderr: 58050K .......... .......... .......... .......... .......... 62% 112M 0s 2024-05-30T16:23:29.643 INFO:teuthology.orchestra.run.smithi006.stderr: 58100K .......... .......... .......... .......... .......... 62% 110M 0s 2024-05-30T16:23:29.643 INFO:teuthology.orchestra.run.smithi006.stderr: 58150K .......... .......... .......... .......... .......... 62% 112M 0s 2024-05-30T16:23:29.643 INFO:teuthology.orchestra.run.smithi006.stderr: 58200K .......... .......... .......... .......... .......... 62% 99.9M 0s 2024-05-30T16:23:29.644 INFO:teuthology.orchestra.run.smithi006.stderr: 58250K .......... .......... .......... .......... .......... 62% 103M 0s 2024-05-30T16:23:29.644 INFO:teuthology.orchestra.run.smithi006.stderr: 58300K .......... .......... .......... .......... .......... 62% 108M 0s 2024-05-30T16:23:29.644 INFO:teuthology.orchestra.run.smithi006.stderr: 58350K .......... .......... .......... .......... .......... 62% 116M 0s 2024-05-30T16:23:29.644 INFO:teuthology.orchestra.run.smithi006.stderr: 58400K .......... .......... .......... .......... .......... 62% 87.8M 0s 2024-05-30T16:23:29.644 INFO:teuthology.orchestra.run.smithi006.stderr: 58450K .......... .......... .......... .......... .......... 62% 85.2M 0s 2024-05-30T16:23:29.644 INFO:teuthology.orchestra.run.smithi006.stderr: 58500K .......... .......... .......... .......... .......... 63% 108M 0s 2024-05-30T16:23:29.644 INFO:teuthology.orchestra.run.smithi006.stderr: 58550K .......... .......... .......... .......... .......... 63% 98.4M 0s 2024-05-30T16:23:29.644 INFO:teuthology.orchestra.run.smithi006.stderr: 58600K .......... .......... .......... .......... .......... 63% 115M 0s 2024-05-30T16:23:29.644 INFO:teuthology.orchestra.run.smithi006.stderr: 58650K .......... .......... .......... .......... .......... 63% 114M 0s 2024-05-30T16:23:29.644 INFO:teuthology.orchestra.run.smithi006.stderr: 58700K .......... .......... .......... .......... .......... 63% 99.8M 0s 2024-05-30T16:23:29.644 INFO:teuthology.orchestra.run.smithi006.stderr: 58750K .......... .......... .......... .......... .......... 63% 116M 0s 2024-05-30T16:23:29.644 INFO:teuthology.orchestra.run.smithi006.stderr: 58800K .......... .......... .......... .......... .......... 63% 90.4M 0s 2024-05-30T16:23:29.644 INFO:teuthology.orchestra.run.smithi006.stderr: 58850K .......... .......... .......... .......... .......... 63% 84.1M 0s 2024-05-30T16:23:29.644 INFO:teuthology.orchestra.run.smithi006.stderr: 58900K .......... .......... .......... .......... .......... 63% 82.7M 0s 2024-05-30T16:23:29.645 INFO:teuthology.orchestra.run.smithi006.stderr: 58950K .......... .......... .......... .......... .......... 63% 103M 0s 2024-05-30T16:23:29.645 INFO:teuthology.orchestra.run.smithi006.stderr: 59000K .......... .......... .......... .......... .......... 63% 90.3M 0s 2024-05-30T16:23:29.645 INFO:teuthology.orchestra.run.smithi006.stderr: 59050K .......... .......... .......... .......... .......... 63% 91.7M 0s 2024-05-30T16:23:29.645 INFO:teuthology.orchestra.run.smithi006.stderr: 59100K .......... .......... .......... .......... .......... 63% 104M 0s 2024-05-30T16:23:29.645 INFO:teuthology.orchestra.run.smithi006.stderr: 59150K .......... .......... .......... .......... .......... 63% 115M 0s 2024-05-30T16:23:29.645 INFO:teuthology.orchestra.run.smithi006.stderr: 59200K .......... .......... .......... .......... .......... 63% 116M 0s 2024-05-30T16:23:29.645 INFO:teuthology.orchestra.run.smithi006.stderr: 59250K .......... .......... .......... .......... .......... 63% 99.6M 0s 2024-05-30T16:23:29.645 INFO:teuthology.orchestra.run.smithi006.stderr: 59300K .......... .......... .......... .......... .......... 63% 94.5M 0s 2024-05-30T16:23:29.645 INFO:teuthology.orchestra.run.smithi006.stderr: 59350K .......... .......... .......... .......... .......... 63% 114M 0s 2024-05-30T16:23:29.645 INFO:teuthology.orchestra.run.smithi006.stderr: 59400K .......... .......... .......... .......... .......... 63% 110M 0s 2024-05-30T16:23:29.645 INFO:teuthology.orchestra.run.smithi006.stderr: 59450K .......... .......... .......... .......... .......... 64% 115M 0s 2024-05-30T16:23:29.645 INFO:teuthology.orchestra.run.smithi006.stderr: 59500K .......... .......... .......... .......... .......... 64% 100M 0s 2024-05-30T16:23:29.645 INFO:teuthology.orchestra.run.smithi006.stderr: 59550K .......... .......... .......... .......... .......... 64% 112M 0s 2024-05-30T16:23:29.646 INFO:teuthology.orchestra.run.smithi006.stderr: 59600K .......... .......... .......... .......... .......... 64% 110M 0s 2024-05-30T16:23:29.646 INFO:teuthology.orchestra.run.smithi006.stderr: 59650K .......... .......... .......... .......... .......... 64% 110M 0s 2024-05-30T16:23:29.646 INFO:teuthology.orchestra.run.smithi006.stderr: 59700K .......... .......... .......... .......... .......... 64% 66.4M 0s 2024-05-30T16:23:29.646 INFO:teuthology.orchestra.run.smithi006.stderr: 59750K .......... .......... .......... .......... .......... 64% 70.3M 0s 2024-05-30T16:23:29.646 INFO:teuthology.orchestra.run.smithi006.stderr: 59800K .......... .......... .......... .......... .......... 64% 59.6M 0s 2024-05-30T16:23:29.646 INFO:teuthology.orchestra.run.smithi006.stderr: 59850K .......... .......... .......... .......... .......... 64% 73.0M 0s 2024-05-30T16:23:29.646 INFO:teuthology.orchestra.run.smithi006.stderr: 59900K .......... .......... .......... .......... .......... 64% 57.6M 0s 2024-05-30T16:23:29.646 INFO:teuthology.orchestra.run.smithi006.stderr: 59950K .......... .......... .......... .......... .......... 64% 59.8M 0s 2024-05-30T16:23:29.646 INFO:teuthology.orchestra.run.smithi006.stderr: 60000K .......... .......... .......... .......... .......... 64% 69.9M 0s 2024-05-30T16:23:29.646 INFO:teuthology.orchestra.run.smithi006.stderr: 60050K .......... .......... .......... .......... .......... 64% 70.2M 0s 2024-05-30T16:23:29.646 INFO:teuthology.orchestra.run.smithi006.stderr: 60100K .......... .......... .......... .......... .......... 64% 75.5M 0s 2024-05-30T16:23:29.646 INFO:teuthology.orchestra.run.smithi006.stderr: 60150K .......... .......... .......... .......... .......... 64% 65.3M 0s 2024-05-30T16:23:29.646 INFO:teuthology.orchestra.run.smithi006.stderr: 60200K .......... .......... .......... .......... .......... 64% 190M 0s 2024-05-30T16:23:29.647 INFO:teuthology.orchestra.run.smithi006.stderr: 60250K .......... .......... .......... .......... .......... 64% 105M 0s 2024-05-30T16:23:29.647 INFO:teuthology.orchestra.run.smithi006.stderr: 60300K .......... .......... .......... .......... .......... 64% 117M 0s 2024-05-30T16:23:29.647 INFO:teuthology.orchestra.run.smithi006.stderr: 60350K .......... .......... .......... .......... .......... 65% 108M 0s 2024-05-30T16:23:29.647 INFO:teuthology.orchestra.run.smithi006.stderr: 60400K .......... .......... .......... .......... .......... 65% 160M 0s 2024-05-30T16:23:29.647 INFO:teuthology.orchestra.run.smithi006.stderr: 60450K .......... .......... .......... .......... .......... 65% 110M 0s 2024-05-30T16:23:29.647 INFO:teuthology.orchestra.run.smithi006.stderr: 60500K .......... .......... .......... .......... .......... 65% 95.6M 0s 2024-05-30T16:23:29.647 INFO:teuthology.orchestra.run.smithi006.stderr: 60550K .......... .......... .......... .......... .......... 65% 115M 0s 2024-05-30T16:23:29.647 INFO:teuthology.orchestra.run.smithi006.stderr: 60600K .......... .......... .......... .......... .......... 65% 112M 0s 2024-05-30T16:23:29.647 INFO:teuthology.orchestra.run.smithi006.stderr: 60650K .......... .......... .......... .......... .......... 65% 98.4M 0s 2024-05-30T16:23:29.647 INFO:teuthology.orchestra.run.smithi006.stderr: 60700K .......... .......... .......... .......... .......... 65% 96.9M 0s 2024-05-30T16:23:29.647 INFO:teuthology.orchestra.run.smithi006.stderr: 60750K .......... .......... .......... .......... .......... 65% 116M 0s 2024-05-30T16:23:29.647 INFO:teuthology.orchestra.run.smithi006.stderr: 60800K .......... .......... .......... .......... .......... 65% 94.3M 0s 2024-05-30T16:23:29.647 INFO:teuthology.orchestra.run.smithi006.stderr: 60850K .......... .......... .......... .......... .......... 65% 118M 0s 2024-05-30T16:23:29.648 INFO:teuthology.orchestra.run.smithi006.stderr: 60900K .......... .......... .......... .......... .......... 65% 118M 0s 2024-05-30T16:23:29.648 INFO:teuthology.orchestra.run.smithi006.stderr: 60950K .......... .......... .......... .......... .......... 65% 119M 0s 2024-05-30T16:23:29.648 INFO:teuthology.orchestra.run.smithi006.stderr: 61000K .......... .......... .......... .......... .......... 65% 103M 0s 2024-05-30T16:23:29.648 INFO:teuthology.orchestra.run.smithi006.stderr: 61050K .......... .......... .......... .......... .......... 65% 121M 0s 2024-05-30T16:23:29.648 INFO:teuthology.orchestra.run.smithi006.stderr: 61100K .......... .......... .......... .......... .......... 65% 117M 0s 2024-05-30T16:23:29.648 INFO:teuthology.orchestra.run.smithi006.stderr: 61150K .......... .......... .......... .......... .......... 65% 115M 0s 2024-05-30T16:23:29.648 INFO:teuthology.orchestra.run.smithi006.stderr: 61200K .......... .......... .......... .......... .......... 65% 96.9M 0s 2024-05-30T16:23:29.648 INFO:teuthology.orchestra.run.smithi006.stderr: 61250K .......... .......... .......... .......... .......... 65% 128M 0s 2024-05-30T16:23:29.648 INFO:teuthology.orchestra.run.smithi006.stderr: 61300K .......... .......... .......... .......... .......... 66% 123M 0s 2024-05-30T16:23:29.648 INFO:teuthology.orchestra.run.smithi006.stderr: 61350K .......... .......... .......... .......... .......... 66% 124M 0s 2024-05-30T16:23:29.648 INFO:teuthology.orchestra.run.smithi006.stderr: 61400K .......... .......... .......... .......... .......... 66% 114M 0s 2024-05-30T16:23:29.648 INFO:teuthology.orchestra.run.smithi006.stderr: 61450K .......... .......... .......... .......... .......... 66% 127M 0s 2024-05-30T16:23:29.648 INFO:teuthology.orchestra.run.smithi006.stderr: 61500K .......... .......... .......... .......... .......... 66% 127M 0s 2024-05-30T16:23:29.648 INFO:teuthology.orchestra.run.smithi006.stderr: 61550K .......... .......... .......... .......... .......... 66% 107M 0s 2024-05-30T16:23:29.649 INFO:teuthology.orchestra.run.smithi006.stderr: 61600K .......... .......... .......... .......... .......... 66% 103M 0s 2024-05-30T16:23:29.649 INFO:teuthology.orchestra.run.smithi006.stderr: 61650K .......... .......... .......... .......... .......... 66% 123M 0s 2024-05-30T16:23:29.649 INFO:teuthology.orchestra.run.smithi006.stderr: 61700K .......... .......... .......... .......... .......... 66% 133M 0s 2024-05-30T16:23:29.649 INFO:teuthology.orchestra.run.smithi006.stderr: 61750K .......... .......... .......... .......... .......... 66% 142M 0s 2024-05-30T16:23:29.649 INFO:teuthology.orchestra.run.smithi006.stderr: 61800K .......... .......... .......... .......... .......... 66% 112M 0s 2024-05-30T16:23:29.649 INFO:teuthology.orchestra.run.smithi006.stderr: 61850K .......... .......... .......... .......... .......... 66% 129M 0s 2024-05-30T16:23:29.649 INFO:teuthology.orchestra.run.smithi006.stderr: 61900K .......... .......... .......... .......... .......... 66% 130M 0s 2024-05-30T16:23:29.649 INFO:teuthology.orchestra.run.smithi006.stderr: 61950K .......... .......... .......... .......... .......... 66% 133M 0s 2024-05-30T16:23:29.649 INFO:teuthology.orchestra.run.smithi006.stderr: 62000K .......... .......... .......... .......... .......... 66% 110M 0s 2024-05-30T16:23:29.649 INFO:teuthology.orchestra.run.smithi006.stderr: 62050K .......... .......... .......... .......... .......... 66% 135M 0s 2024-05-30T16:23:29.649 INFO:teuthology.orchestra.run.smithi006.stderr: 62100K .......... .......... .......... .......... .......... 66% 125M 0s 2024-05-30T16:23:29.649 INFO:teuthology.orchestra.run.smithi006.stderr: 62150K .......... .......... .......... .......... .......... 66% 223M 0s 2024-05-30T16:23:29.649 INFO:teuthology.orchestra.run.smithi006.stderr: 62200K .......... .......... .......... .......... .......... 67% 121M 0s 2024-05-30T16:23:29.650 INFO:teuthology.orchestra.run.smithi006.stderr: 62250K .......... .......... .......... .......... .......... 67% 111M 0s 2024-05-30T16:23:29.650 INFO:teuthology.orchestra.run.smithi006.stderr: 62300K .......... .......... .......... .......... .......... 67% 115M 0s 2024-05-30T16:23:29.650 INFO:teuthology.orchestra.run.smithi006.stderr: 62350K .......... .......... .......... .......... .......... 67% 96.1M 0s 2024-05-30T16:23:29.650 INFO:teuthology.orchestra.run.smithi006.stderr: 62400K .......... .......... .......... .......... .......... 67% 109M 0s 2024-05-30T16:23:29.650 INFO:teuthology.orchestra.run.smithi006.stderr: 62450K .......... .......... .......... .......... .......... 67% 113M 0s 2024-05-30T16:23:29.650 INFO:teuthology.orchestra.run.smithi006.stderr: 62500K .......... .......... .......... .......... .......... 67% 107M 0s 2024-05-30T16:23:29.650 INFO:teuthology.orchestra.run.smithi006.stderr: 62550K .......... .......... .......... .......... .......... 67% 77.7M 0s 2024-05-30T16:23:29.650 INFO:teuthology.orchestra.run.smithi006.stderr: 62600K .......... .......... .......... .......... .......... 67% 107M 0s 2024-05-30T16:23:29.650 INFO:teuthology.orchestra.run.smithi006.stderr: 62650K .......... .......... .......... .......... .......... 67% 47.0M 0s 2024-05-30T16:23:29.650 INFO:teuthology.orchestra.run.smithi006.stderr: 62700K .......... .......... .......... .......... .......... 67% 94.5M 0s 2024-05-30T16:23:29.650 INFO:teuthology.orchestra.run.smithi006.stderr: 62750K .......... .......... .......... .......... .......... 67% 104M 0s 2024-05-30T16:23:29.651 INFO:teuthology.orchestra.run.smithi006.stderr: 62800K .......... .......... .......... .......... .......... 67% 98.9M 0s 2024-05-30T16:23:29.651 INFO:teuthology.orchestra.run.smithi006.stderr: 62850K .......... .......... .......... .......... .......... 67% 106M 0s 2024-05-30T16:23:29.651 INFO:teuthology.orchestra.run.smithi006.stderr: 62900K .......... .......... .......... .......... .......... 67% 114M 0s 2024-05-30T16:23:29.651 INFO:teuthology.orchestra.run.smithi006.stderr: 62950K .......... .......... .......... .......... .......... 67% 118M 0s 2024-05-30T16:23:29.651 INFO:teuthology.orchestra.run.smithi006.stderr: 63000K .......... .......... .......... .......... .......... 67% 102M 0s 2024-05-30T16:23:29.651 INFO:teuthology.orchestra.run.smithi006.stderr: 63050K .......... .......... .......... .......... .......... 67% 120M 0s 2024-05-30T16:23:29.651 INFO:teuthology.orchestra.run.smithi006.stderr: 63100K .......... .......... .......... .......... .......... 67% 85.2M 0s 2024-05-30T16:23:29.651 INFO:teuthology.orchestra.run.smithi006.stderr: 63150K .......... .......... .......... .......... .......... 68% 114M 0s 2024-05-30T16:23:29.651 INFO:teuthology.orchestra.run.smithi006.stderr: 63200K .......... .......... .......... .......... .......... 68% 95.4M 0s 2024-05-30T16:23:29.651 INFO:teuthology.orchestra.run.smithi006.stderr: 63250K .......... .......... .......... .......... .......... 68% 73.4M 0s 2024-05-30T16:23:29.651 INFO:teuthology.orchestra.run.smithi006.stderr: 63300K .......... .......... .......... .......... .......... 68% 108M 0s 2024-05-30T16:23:29.651 INFO:teuthology.orchestra.run.smithi006.stderr: 63350K .......... .......... .......... .......... .......... 68% 71.1M 0s 2024-05-30T16:23:29.651 INFO:teuthology.orchestra.run.smithi006.stderr: 63400K .......... .......... .......... .......... .......... 68% 116M 0s 2024-05-30T16:23:29.651 INFO:teuthology.orchestra.run.smithi006.stderr: 63450K .......... .......... .......... .......... .......... 68% 122M 0s 2024-05-30T16:23:29.652 INFO:teuthology.orchestra.run.smithi006.stderr: 63500K .......... .......... .......... .......... .......... 68% 101M 0s 2024-05-30T16:23:29.652 INFO:teuthology.orchestra.run.smithi006.stderr: 63550K .......... .......... .......... .......... .......... 68% 105M 0s 2024-05-30T16:23:29.652 INFO:teuthology.orchestra.run.smithi006.stderr: 63600K .......... .......... .......... .......... .......... 68% 105M 0s 2024-05-30T16:23:29.652 INFO:teuthology.orchestra.run.smithi006.stderr: 63650K .......... .......... .......... .......... .......... 68% 120M 0s 2024-05-30T16:23:29.652 INFO:teuthology.orchestra.run.smithi006.stderr: 63700K .......... .......... .......... .......... .......... 68% 119M 0s 2024-05-30T16:23:29.652 INFO:teuthology.orchestra.run.smithi006.stderr: 63750K .......... .......... .......... .......... .......... 68% 96.4M 0s 2024-05-30T16:23:29.652 INFO:teuthology.orchestra.run.smithi006.stderr: 63800K .......... .......... .......... .......... .......... 68% 108M 0s 2024-05-30T16:23:29.652 INFO:teuthology.orchestra.run.smithi006.stderr: 63850K .......... .......... .......... .......... .......... 68% 116M 0s 2024-05-30T16:23:29.652 INFO:teuthology.orchestra.run.smithi006.stderr: 63900K .......... .......... .......... .......... .......... 68% 109M 0s 2024-05-30T16:23:29.652 INFO:teuthology.orchestra.run.smithi006.stderr: 63950K .......... .......... .......... .......... .......... 68% 112M 0s 2024-05-30T16:23:29.652 INFO:teuthology.orchestra.run.smithi006.stderr: 64000K .......... .......... .......... .......... .......... 68% 114M 0s 2024-05-30T16:23:29.652 INFO:teuthology.orchestra.run.smithi006.stderr: 64050K .......... .......... .......... .......... .......... 68% 110M 0s 2024-05-30T16:23:29.652 INFO:teuthology.orchestra.run.smithi006.stderr: 64100K .......... .......... .......... .......... .......... 69% 105M 0s 2024-05-30T16:23:29.653 INFO:teuthology.orchestra.run.smithi006.stderr: 64150K .......... .......... .......... .......... .......... 69% 121M 0s 2024-05-30T16:23:29.653 INFO:teuthology.orchestra.run.smithi006.stderr: 64200K .......... .......... .......... .......... .......... 69% 115M 0s 2024-05-30T16:23:29.653 INFO:teuthology.orchestra.run.smithi006.stderr: 64250K .......... .......... .......... .......... .......... 69% 106M 0s 2024-05-30T16:23:29.653 INFO:teuthology.orchestra.run.smithi006.stderr: 64300K .......... .......... .......... .......... .......... 69% 93.9M 0s 2024-05-30T16:23:29.653 INFO:teuthology.orchestra.run.smithi006.stderr: 64350K .......... .......... .......... .......... .......... 69% 53.6M 0s 2024-05-30T16:23:29.653 INFO:teuthology.orchestra.run.smithi006.stderr: 64400K .......... .......... .......... .......... .......... 69% 99.4M 0s 2024-05-30T16:23:29.653 INFO:teuthology.orchestra.run.smithi006.stderr: 64450K .......... .......... .......... .......... .......... 69% 103M 0s 2024-05-30T16:23:29.653 INFO:teuthology.orchestra.run.smithi006.stderr: 64500K .......... .......... .......... .......... .......... 69% 129M 0s 2024-05-30T16:23:29.653 INFO:teuthology.orchestra.run.smithi006.stderr: 64550K .......... .......... .......... .......... .......... 69% 91.9M 0s 2024-05-30T16:23:29.653 INFO:teuthology.orchestra.run.smithi006.stderr: 64600K .......... .......... .......... .......... .......... 69% 83.4M 0s 2024-05-30T16:23:29.653 INFO:teuthology.orchestra.run.smithi006.stderr: 64650K .......... .......... .......... .......... .......... 69% 105M 0s 2024-05-30T16:23:29.653 INFO:teuthology.orchestra.run.smithi006.stderr: 64700K .......... .......... .......... .......... .......... 69% 113M 0s 2024-05-30T16:23:29.653 INFO:teuthology.orchestra.run.smithi006.stderr: 64750K .......... .......... .......... .......... .......... 69% 123M 0s 2024-05-30T16:23:29.653 INFO:teuthology.orchestra.run.smithi006.stderr: 64800K .......... .......... .......... .......... .......... 69% 104M 0s 2024-05-30T16:23:29.653 INFO:teuthology.orchestra.run.smithi006.stderr: 64850K .......... .......... .......... .......... .......... 69% 85.5M 0s 2024-05-30T16:23:29.654 INFO:teuthology.orchestra.run.smithi006.stderr: 64900K .......... .......... .......... .......... .......... 69% 112M 0s 2024-05-30T16:23:29.654 INFO:teuthology.orchestra.run.smithi006.stderr: 64950K .......... .......... .......... .......... .......... 69% 126M 0s 2024-05-30T16:23:29.654 INFO:teuthology.orchestra.run.smithi006.stderr: 65000K .......... .......... .......... .......... .......... 70% 125M 0s 2024-05-30T16:23:29.654 INFO:teuthology.orchestra.run.smithi006.stderr: 65050K .......... .......... .......... .......... .......... 70% 99.6M 0s 2024-05-30T16:23:29.654 INFO:teuthology.orchestra.run.smithi006.stderr: 65100K .......... .......... .......... .......... .......... 70% 116M 0s 2024-05-30T16:23:29.654 INFO:teuthology.orchestra.run.smithi006.stderr: 65150K .......... .......... .......... .......... .......... 70% 116M 0s 2024-05-30T16:23:29.654 INFO:teuthology.orchestra.run.smithi006.stderr: 65200K .......... .......... .......... .......... .......... 70% 101M 0s 2024-05-30T16:23:29.654 INFO:teuthology.orchestra.run.smithi006.stderr: 65250K .......... .......... .......... .......... .......... 70% 72.6M 0s 2024-05-30T16:23:29.654 INFO:teuthology.orchestra.run.smithi006.stderr: 65300K .......... .......... .......... .......... .......... 70% 126M 0s 2024-05-30T16:23:29.654 INFO:teuthology.orchestra.run.smithi006.stderr: 65350K .......... .......... .......... .......... .......... 70% 130M 0s 2024-05-30T16:23:29.654 INFO:teuthology.orchestra.run.smithi006.stderr: 65400K .......... .......... .......... .......... .......... 70% 105M 0s 2024-05-30T16:23:29.654 INFO:teuthology.orchestra.run.smithi006.stderr: 65450K .......... .......... .......... .......... .......... 70% 114M 0s 2024-05-30T16:23:29.655 INFO:teuthology.orchestra.run.smithi006.stderr: 65500K .......... .......... .......... .......... .......... 70% 120M 0s 2024-05-30T16:23:29.655 INFO:teuthology.orchestra.run.smithi006.stderr: 65550K .......... .......... .......... .......... .......... 70% 63.5M 0s 2024-05-30T16:23:29.655 INFO:teuthology.orchestra.run.smithi006.stderr: 65600K .......... .......... .......... .......... .......... 70% 59.0M 0s 2024-05-30T16:23:29.655 INFO:teuthology.orchestra.run.smithi006.stderr: 65650K .......... .......... .......... .......... .......... 70% 63.2M 0s 2024-05-30T16:23:29.655 INFO:teuthology.orchestra.run.smithi006.stderr: 65700K .......... .......... .......... .......... .......... 70% 62.9M 0s 2024-05-30T16:23:29.655 INFO:teuthology.orchestra.run.smithi006.stderr: 65750K .......... .......... .......... .......... .......... 70% 69.4M 0s 2024-05-30T16:23:29.655 INFO:teuthology.orchestra.run.smithi006.stderr: 65800K .......... .......... .......... .......... .......... 70% 63.4M 0s 2024-05-30T16:23:29.655 INFO:teuthology.orchestra.run.smithi006.stderr: 65850K .......... .......... .......... .......... .......... 70% 70.3M 0s 2024-05-30T16:23:29.655 INFO:teuthology.orchestra.run.smithi006.stderr: 65900K .......... .......... .......... .......... .......... 70% 78.9M 0s 2024-05-30T16:23:29.655 INFO:teuthology.orchestra.run.smithi006.stderr: 65950K .......... .......... .......... .......... .......... 71% 70.2M 0s 2024-05-30T16:23:29.655 INFO:teuthology.orchestra.run.smithi006.stderr: 66000K .......... .......... .......... .......... .......... 71% 61.1M 0s 2024-05-30T16:23:29.655 INFO:teuthology.orchestra.run.smithi006.stderr: 66050K .......... .......... .......... .......... .......... 71% 84.6M 0s 2024-05-30T16:23:29.655 INFO:teuthology.orchestra.run.smithi006.stderr: 66100K .......... .......... .......... .......... .......... 71% 74.2M 0s 2024-05-30T16:23:29.656 INFO:teuthology.orchestra.run.smithi006.stderr: 66150K .......... .......... .......... .......... .......... 71% 94.1M 0s 2024-05-30T16:23:29.656 INFO:teuthology.orchestra.run.smithi006.stderr: 66200K .......... .......... .......... .......... .......... 71% 81.8M 0s 2024-05-30T16:23:29.656 INFO:teuthology.orchestra.run.smithi006.stderr: 66250K .......... .......... .......... .......... .......... 71% 77.0M 0s 2024-05-30T16:23:29.656 INFO:teuthology.orchestra.run.smithi006.stderr: 66300K .......... .......... .......... .......... .......... 71% 78.9M 0s 2024-05-30T16:23:29.656 INFO:teuthology.orchestra.run.smithi006.stderr: 66350K .......... .......... .......... .......... .......... 71% 69.2M 0s 2024-05-30T16:23:29.656 INFO:teuthology.orchestra.run.smithi006.stderr: 66400K .......... .......... .......... .......... .......... 71% 65.1M 0s 2024-05-30T16:23:29.656 INFO:teuthology.orchestra.run.smithi006.stderr: 66450K .......... .......... .......... .......... .......... 71% 81.2M 0s 2024-05-30T16:23:29.656 INFO:teuthology.orchestra.run.smithi006.stderr: 66500K .......... .......... .......... .......... .......... 71% 81.2M 0s 2024-05-30T16:23:29.656 INFO:teuthology.orchestra.run.smithi006.stderr: 66550K .......... .......... .......... .......... .......... 71% 76.4M 0s 2024-05-30T16:23:29.656 INFO:teuthology.orchestra.run.smithi006.stderr: 66600K .......... .......... .......... .......... .......... 71% 63.4M 0s 2024-05-30T16:23:29.656 INFO:teuthology.orchestra.run.smithi006.stderr: 66650K .......... .......... .......... .......... .......... 71% 81.4M 0s 2024-05-30T16:23:29.656 INFO:teuthology.orchestra.run.smithi006.stderr: 66700K .......... .......... .......... .......... .......... 71% 75.9M 0s 2024-05-30T16:23:29.656 INFO:teuthology.orchestra.run.smithi006.stderr: 66750K .......... .......... .......... .......... .......... 71% 92.3M 0s 2024-05-30T16:23:29.656 INFO:teuthology.orchestra.run.smithi006.stderr: 66800K .......... .......... .......... .......... .......... 71% 76.0M 0s 2024-05-30T16:23:29.657 INFO:teuthology.orchestra.run.smithi006.stderr: 66850K .......... .......... .......... .......... .......... 72% 83.4M 0s 2024-05-30T16:23:29.657 INFO:teuthology.orchestra.run.smithi006.stderr: 66900K .......... .......... .......... .......... .......... 72% 86.2M 0s 2024-05-30T16:23:29.657 INFO:teuthology.orchestra.run.smithi006.stderr: 66950K .......... .......... .......... .......... .......... 72% 84.2M 0s 2024-05-30T16:23:29.657 INFO:teuthology.orchestra.run.smithi006.stderr: 67000K .......... .......... .......... .......... .......... 72% 114M 0s 2024-05-30T16:23:29.657 INFO:teuthology.orchestra.run.smithi006.stderr: 67050K .......... .......... .......... .......... .......... 72% 78.8M 0s 2024-05-30T16:23:29.657 INFO:teuthology.orchestra.run.smithi006.stderr: 67100K .......... .......... .......... .......... .......... 72% 81.9M 0s 2024-05-30T16:23:29.657 INFO:teuthology.orchestra.run.smithi006.stderr: 67150K .......... .......... .......... .......... .......... 72% 84.5M 0s 2024-05-30T16:23:29.657 INFO:teuthology.orchestra.run.smithi006.stderr: 67200K .......... .......... .......... .......... .......... 72% 77.8M 0s 2024-05-30T16:23:29.657 INFO:teuthology.orchestra.run.smithi006.stderr: 67250K .......... .......... .......... .......... .......... 72% 88.9M 0s 2024-05-30T16:23:29.657 INFO:teuthology.orchestra.run.smithi006.stderr: 67300K .......... .......... .......... .......... .......... 72% 105M 0s 2024-05-30T16:23:29.657 INFO:teuthology.orchestra.run.smithi006.stderr: 67350K .......... .......... .......... .......... .......... 72% 162M 0s 2024-05-30T16:23:29.657 INFO:teuthology.orchestra.run.smithi006.stderr: 67400K .......... .......... .......... .......... .......... 72% 71.7M 0s 2024-05-30T16:23:29.657 INFO:teuthology.orchestra.run.smithi006.stderr: 67450K .......... .......... .......... .......... .......... 72% 150M 0s 2024-05-30T16:23:29.658 INFO:teuthology.orchestra.run.smithi006.stderr: 67500K .......... .......... .......... .......... .......... 72% 82.6M 0s 2024-05-30T16:23:29.658 INFO:teuthology.orchestra.run.smithi006.stderr: 67550K .......... .......... .......... .......... .......... 72% 84.4M 0s 2024-05-30T16:23:29.658 INFO:teuthology.orchestra.run.smithi006.stderr: 67600K .......... .......... .......... .......... .......... 72% 82.1M 0s 2024-05-30T16:23:29.658 INFO:teuthology.orchestra.run.smithi006.stderr: 67650K .......... .......... .......... .......... .......... 72% 89.1M 0s 2024-05-30T16:23:29.658 INFO:teuthology.orchestra.run.smithi006.stderr: 67700K .......... .......... .......... .......... .......... 72% 79.1M 0s 2024-05-30T16:23:29.658 INFO:teuthology.orchestra.run.smithi006.stderr: 67750K .......... .......... .......... .......... .......... 72% 94.9M 0s 2024-05-30T16:23:29.658 INFO:teuthology.orchestra.run.smithi006.stderr: 67800K .......... .......... .......... .......... .......... 73% 124M 0s 2024-05-30T16:23:29.658 INFO:teuthology.orchestra.run.smithi006.stderr: 67850K .......... .......... .......... .......... .......... 73% 91.0M 0s 2024-05-30T16:23:29.658 INFO:teuthology.orchestra.run.smithi006.stderr: 67900K .......... .......... .......... .......... .......... 73% 146M 0s 2024-05-30T16:23:29.658 INFO:teuthology.orchestra.run.smithi006.stderr: 67950K .......... .......... .......... .......... .......... 73% 122M 0s 2024-05-30T16:23:29.658 INFO:teuthology.orchestra.run.smithi006.stderr: 68000K .......... .......... .......... .......... .......... 73% 104M 0s 2024-05-30T16:23:29.658 INFO:teuthology.orchestra.run.smithi006.stderr: 68050K .......... .......... .......... .......... .......... 73% 92.6M 0s 2024-05-30T16:23:29.658 INFO:teuthology.orchestra.run.smithi006.stderr: 68100K .......... .......... .......... .......... .......... 73% 85.9M 0s 2024-05-30T16:23:29.658 INFO:teuthology.orchestra.run.smithi006.stderr: 68150K .......... .......... .......... .......... .......... 73% 87.9M 0s 2024-05-30T16:23:29.659 INFO:teuthology.orchestra.run.smithi006.stderr: 68200K .......... .......... .......... .......... .......... 73% 109M 0s 2024-05-30T16:23:29.659 INFO:teuthology.orchestra.run.smithi006.stderr: 68250K .......... .......... .......... .......... .......... 73% 90.7M 0s 2024-05-30T16:23:29.659 INFO:teuthology.orchestra.run.smithi006.stderr: 68300K .......... .......... .......... .......... .......... 73% 94.9M 0s 2024-05-30T16:23:29.659 INFO:teuthology.orchestra.run.smithi006.stderr: 68350K .......... .......... .......... .......... .......... 73% 98.1M 0s 2024-05-30T16:23:29.659 INFO:teuthology.orchestra.run.smithi006.stderr: 68400K .......... .......... .......... .......... .......... 73% 93.5M 0s 2024-05-30T16:23:29.659 INFO:teuthology.orchestra.run.smithi006.stderr: 68450K .......... .......... .......... .......... .......... 73% 96.4M 0s 2024-05-30T16:23:29.659 INFO:teuthology.orchestra.run.smithi006.stderr: 68500K .......... .......... .......... .......... .......... 73% 79.3M 0s 2024-05-30T16:23:29.659 INFO:teuthology.orchestra.run.smithi006.stderr: 68550K .......... .......... .......... .......... .......... 73% 92.4M 0s 2024-05-30T16:23:29.659 INFO:teuthology.orchestra.run.smithi006.stderr: 68600K .......... .......... .......... .......... .......... 73% 93.0M 0s 2024-05-30T16:23:29.659 INFO:teuthology.orchestra.run.smithi006.stderr: 68650K .......... .......... .......... .......... .......... 73% 104M 0s 2024-05-30T16:23:29.659 INFO:teuthology.orchestra.run.smithi006.stderr: 68700K .......... .......... .......... .......... .......... 74% 90.1M 0s 2024-05-30T16:23:29.659 INFO:teuthology.orchestra.run.smithi006.stderr: 68750K .......... .......... .......... .......... .......... 74% 105M 0s 2024-05-30T16:23:29.659 INFO:teuthology.orchestra.run.smithi006.stderr: 68800K .......... .......... .......... .......... .......... 74% 194M 0s 2024-05-30T16:23:29.659 INFO:teuthology.orchestra.run.smithi006.stderr: 68850K .......... .......... .......... .......... .......... 74% 182M 0s 2024-05-30T16:23:29.659 INFO:teuthology.orchestra.run.smithi006.stderr: 68900K .......... .......... .......... .......... .......... 74% 158M 0s 2024-05-30T16:23:29.660 INFO:teuthology.orchestra.run.smithi006.stderr: 68950K .......... .......... .......... .......... .......... 74% 183M 0s 2024-05-30T16:23:29.660 INFO:teuthology.orchestra.run.smithi006.stderr: 69000K .......... .......... .......... .......... .......... 74% 189M 0s 2024-05-30T16:23:29.660 INFO:teuthology.orchestra.run.smithi006.stderr: 69050K .......... .......... .......... .......... .......... 74% 210M 0s 2024-05-30T16:23:29.660 INFO:teuthology.orchestra.run.smithi006.stderr: 69100K .......... .......... .......... .......... .......... 74% 175M 0s 2024-05-30T16:23:29.660 INFO:teuthology.orchestra.run.smithi006.stderr: 69150K .......... .......... .......... .......... .......... 74% 191M 0s 2024-05-30T16:23:29.660 INFO:teuthology.orchestra.run.smithi006.stderr: 69200K .......... .......... .......... .......... .......... 74% 169M 0s 2024-05-30T16:23:29.660 INFO:teuthology.orchestra.run.smithi006.stderr: 69250K .......... .......... .......... .......... .......... 74% 195M 0s 2024-05-30T16:23:29.660 INFO:teuthology.orchestra.run.smithi006.stderr: 69300K .......... .......... .......... .......... .......... 74% 155M 0s 2024-05-30T16:23:29.660 INFO:teuthology.orchestra.run.smithi006.stderr: 69350K .......... .......... .......... .......... .......... 74% 203M 0s 2024-05-30T16:23:29.660 INFO:teuthology.orchestra.run.smithi006.stderr: 69400K .......... .......... .......... .......... .......... 74% 288M 0s 2024-05-30T16:23:29.660 INFO:teuthology.orchestra.run.smithi006.stderr: 69450K .......... .......... .......... .......... .......... 74% 339M 0s 2024-05-30T16:23:29.660 INFO:teuthology.orchestra.run.smithi006.stderr: 69500K .......... .......... .......... .......... .......... 74% 308M 0s 2024-05-30T16:23:29.660 INFO:teuthology.orchestra.run.smithi006.stderr: 69550K .......... .......... .......... .......... .......... 74% 341M 0s 2024-05-30T16:23:29.660 INFO:teuthology.orchestra.run.smithi006.stderr: 69600K .......... .......... .......... .......... .......... 74% 341M 0s 2024-05-30T16:23:29.660 INFO:teuthology.orchestra.run.smithi006.stderr: 69650K .......... .......... .......... .......... .......... 75% 236M 0s 2024-05-30T16:23:29.661 INFO:teuthology.orchestra.run.smithi006.stderr: 69700K .......... .......... .......... .......... .......... 75% 167M 0s 2024-05-30T16:23:29.661 INFO:teuthology.orchestra.run.smithi006.stderr: 69750K .......... .......... .......... .......... .......... 75% 161M 0s 2024-05-30T16:23:29.661 INFO:teuthology.orchestra.run.smithi006.stderr: 69800K .......... .......... .......... .......... .......... 75% 122M 0s 2024-05-30T16:23:29.661 INFO:teuthology.orchestra.run.smithi006.stderr: 69850K .......... .......... .......... .......... .......... 75% 108M 0s 2024-05-30T16:23:29.661 INFO:teuthology.orchestra.run.smithi006.stderr: 69900K .......... .......... .......... .......... .......... 75% 89.4M 0s 2024-05-30T16:23:29.661 INFO:teuthology.orchestra.run.smithi006.stderr: 69950K .......... .......... .......... .......... .......... 75% 66.9M 0s 2024-05-30T16:23:29.661 INFO:teuthology.orchestra.run.smithi006.stderr: 70000K .......... .......... .......... .......... .......... 75% 49.4M 0s 2024-05-30T16:23:29.661 INFO:teuthology.orchestra.run.smithi006.stderr: 70050K .......... .......... .......... .......... .......... 75% 101M 0s 2024-05-30T16:23:29.661 INFO:teuthology.orchestra.run.smithi006.stderr: 70100K .......... .......... .......... .......... .......... 75% 69.5M 0s 2024-05-30T16:23:29.661 INFO:teuthology.orchestra.run.smithi006.stderr: 70150K .......... .......... .......... .......... .......... 75% 108M 0s 2024-05-30T16:23:29.661 INFO:teuthology.orchestra.run.smithi006.stderr: 70200K .......... .......... .......... .......... .......... 75% 115M 0s 2024-05-30T16:23:29.661 INFO:teuthology.orchestra.run.smithi006.stderr: 70250K .......... .......... .......... .......... .......... 75% 114M 0s 2024-05-30T16:23:29.661 INFO:teuthology.orchestra.run.smithi006.stderr: 70300K .......... .......... .......... .......... .......... 75% 110M 0s 2024-05-30T16:23:29.661 INFO:teuthology.orchestra.run.smithi006.stderr: 70350K .......... .......... .......... .......... .......... 75% 111M 0s 2024-05-30T16:23:29.661 INFO:teuthology.orchestra.run.smithi006.stderr: 70400K .......... .......... .......... .......... .......... 75% 79.8M 0s 2024-05-30T16:23:29.662 INFO:teuthology.orchestra.run.smithi006.stderr: 70450K .......... .......... .......... .......... .......... 75% 80.0M 0s 2024-05-30T16:23:29.662 INFO:teuthology.orchestra.run.smithi006.stderr: 70500K .......... .......... .......... .......... .......... 75% 106M 0s 2024-05-30T16:23:29.662 INFO:teuthology.orchestra.run.smithi006.stderr: 70550K .......... .......... .......... .......... .......... 75% 128M 0s 2024-05-30T16:23:29.662 INFO:teuthology.orchestra.run.smithi006.stderr: 70600K .......... .......... .......... .......... .......... 76% 48.4M 0s 2024-05-30T16:23:29.662 INFO:teuthology.orchestra.run.smithi006.stderr: 70650K .......... .......... .......... .......... .......... 76% 115M 0s 2024-05-30T16:23:29.662 INFO:teuthology.orchestra.run.smithi006.stderr: 70700K .......... .......... .......... .......... .......... 76% 89.8M 0s 2024-05-30T16:23:29.662 INFO:teuthology.orchestra.run.smithi006.stderr: 70750K .......... .......... .......... .......... .......... 76% 151M 0s 2024-05-30T16:23:29.662 INFO:teuthology.orchestra.run.smithi006.stderr: 70800K .......... .......... .......... .......... .......... 76% 79.5M 0s 2024-05-30T16:23:29.662 INFO:teuthology.orchestra.run.smithi006.stderr: 70850K .......... .......... .......... .......... .......... 76% 109M 0s 2024-05-30T16:23:29.662 INFO:teuthology.orchestra.run.smithi006.stderr: 70900K .......... .......... .......... .......... .......... 76% 101M 0s 2024-05-30T16:23:29.662 INFO:teuthology.orchestra.run.smithi006.stderr: 70950K .......... .......... .......... .......... .......... 76% 123M 0s 2024-05-30T16:23:29.662 INFO:teuthology.orchestra.run.smithi006.stderr: 71000K .......... .......... .......... .......... .......... 76% 121M 0s 2024-05-30T16:23:29.662 INFO:teuthology.orchestra.run.smithi006.stderr: 71050K .......... .......... .......... .......... .......... 76% 123M 0s 2024-05-30T16:23:29.662 INFO:teuthology.orchestra.run.smithi006.stderr: 71100K .......... .......... .......... .......... .......... 76% 101M 0s 2024-05-30T16:23:29.663 INFO:teuthology.orchestra.run.smithi006.stderr: 71150K .......... .......... .......... .......... .......... 76% 89.8M 0s 2024-05-30T16:23:29.663 INFO:teuthology.orchestra.run.smithi006.stderr: 71200K .......... .......... .......... .......... .......... 76% 88.8M 0s 2024-05-30T16:23:29.663 INFO:teuthology.orchestra.run.smithi006.stderr: 71250K .......... .......... .......... .......... .......... 76% 115M 0s 2024-05-30T16:23:29.663 INFO:teuthology.orchestra.run.smithi006.stderr: 71300K .......... .......... .......... .......... .......... 76% 81.9M 0s 2024-05-30T16:23:29.663 INFO:teuthology.orchestra.run.smithi006.stderr: 71350K .......... .......... .......... .......... .......... 76% 103M 0s 2024-05-30T16:23:29.663 INFO:teuthology.orchestra.run.smithi006.stderr: 71400K .......... .......... .......... .......... .......... 76% 100M 0s 2024-05-30T16:23:29.663 INFO:teuthology.orchestra.run.smithi006.stderr: 71450K .......... .......... .......... .......... .......... 76% 115M 0s 2024-05-30T16:23:29.663 INFO:teuthology.orchestra.run.smithi006.stderr: 71500K .......... .......... .......... .......... .......... 77% 85.2M 0s 2024-05-30T16:23:29.663 INFO:teuthology.orchestra.run.smithi006.stderr: 71550K .......... .......... .......... .......... .......... 77% 89.2M 0s 2024-05-30T16:23:29.663 INFO:teuthology.orchestra.run.smithi006.stderr: 71600K .......... .......... .......... .......... .......... 77% 123M 0s 2024-05-30T16:23:29.663 INFO:teuthology.orchestra.run.smithi006.stderr: 71650K .......... .......... .......... .......... .......... 77% 108M 0s 2024-05-30T16:23:29.663 INFO:teuthology.orchestra.run.smithi006.stderr: 71700K .......... .......... .......... .......... .......... 77% 105M 0s 2024-05-30T16:23:29.663 INFO:teuthology.orchestra.run.smithi006.stderr: 71750K .......... .......... .......... .......... .......... 77% 59.9M 0s 2024-05-30T16:23:29.663 INFO:teuthology.orchestra.run.smithi006.stderr: 71800K .......... .......... .......... .......... .......... 77% 61.0M 0s 2024-05-30T16:23:29.663 INFO:teuthology.orchestra.run.smithi006.stderr: 71850K .......... .......... .......... .......... .......... 77% 128M 0s 2024-05-30T16:23:29.664 INFO:teuthology.orchestra.run.smithi006.stderr: 71900K .......... .......... .......... .......... .......... 77% 84.5M 0s 2024-05-30T16:23:29.664 INFO:teuthology.orchestra.run.smithi006.stderr: 71950K .......... .......... .......... .......... .......... 77% 121M 0s 2024-05-30T16:23:29.664 INFO:teuthology.orchestra.run.smithi006.stderr: 72000K .......... .......... .......... .......... .......... 77% 102M 0s 2024-05-30T16:23:29.664 INFO:teuthology.orchestra.run.smithi006.stderr: 72050K .......... .......... .......... .......... .......... 77% 97.9M 0s 2024-05-30T16:23:29.664 INFO:teuthology.orchestra.run.smithi006.stderr: 72100K .......... .......... .......... .......... .......... 77% 78.5M 0s 2024-05-30T16:23:29.664 INFO:teuthology.orchestra.run.smithi006.stderr: 72150K .......... .......... .......... .......... .......... 77% 114M 0s 2024-05-30T16:23:29.664 INFO:teuthology.orchestra.run.smithi006.stderr: 72200K .......... .......... .......... .......... .......... 77% 73.6M 0s 2024-05-30T16:23:29.664 INFO:teuthology.orchestra.run.smithi006.stderr: 72250K .......... .......... .......... .......... .......... 77% 82.4M 0s 2024-05-30T16:23:29.664 INFO:teuthology.orchestra.run.smithi006.stderr: 72300K .......... .......... .......... .......... .......... 77% 59.7M 0s 2024-05-30T16:23:29.664 INFO:teuthology.orchestra.run.smithi006.stderr: 72350K .......... .......... .......... .......... .......... 77% 92.2M 0s 2024-05-30T16:23:29.664 INFO:teuthology.orchestra.run.smithi006.stderr: 72400K .......... .......... .......... .......... .......... 77% 122M 0s 2024-05-30T16:23:29.664 INFO:teuthology.orchestra.run.smithi006.stderr: 72450K .......... .......... .......... .......... .......... 78% 114M 0s 2024-05-30T16:23:29.664 INFO:teuthology.orchestra.run.smithi006.stderr: 72500K .......... .......... .......... .......... .......... 78% 106M 0s 2024-05-30T16:23:29.665 INFO:teuthology.orchestra.run.smithi006.stderr: 72550K .......... .......... .......... .......... .......... 78% 113M 0s 2024-05-30T16:23:29.665 INFO:teuthology.orchestra.run.smithi006.stderr: 72600K .......... .......... .......... .......... .......... 78% 114M 0s 2024-05-30T16:23:29.665 INFO:teuthology.orchestra.run.smithi006.stderr: 72650K .......... .......... .......... .......... .......... 78% 80.5M 0s 2024-05-30T16:23:29.665 INFO:teuthology.orchestra.run.smithi006.stderr: 72700K .......... .......... .......... .......... .......... 78% 75.6M 0s 2024-05-30T16:23:29.665 INFO:teuthology.orchestra.run.smithi006.stderr: 72750K .......... .......... .......... .......... .......... 78% 103M 0s 2024-05-30T16:23:29.665 INFO:teuthology.orchestra.run.smithi006.stderr: 72800K .......... .......... .......... .......... .......... 78% 46.3M 0s 2024-05-30T16:23:29.665 INFO:teuthology.orchestra.run.smithi006.stderr: 72850K .......... .......... .......... .......... .......... 78% 120M 0s 2024-05-30T16:23:29.665 INFO:teuthology.orchestra.run.smithi006.stderr: 72900K .......... .......... .......... .......... .......... 78% 114M 0s 2024-05-30T16:23:29.665 INFO:teuthology.orchestra.run.smithi006.stderr: 72950K .......... .......... .......... .......... .......... 78% 109M 0s 2024-05-30T16:23:29.665 INFO:teuthology.orchestra.run.smithi006.stderr: 73000K .......... .......... .......... .......... .......... 78% 87.3M 0s 2024-05-30T16:23:29.665 INFO:teuthology.orchestra.run.smithi006.stderr: 73050K .......... .......... .......... .......... .......... 78% 167M 0s 2024-05-30T16:23:29.665 INFO:teuthology.orchestra.run.smithi006.stderr: 73100K .......... .......... .......... .......... .......... 78% 91.8M 0s 2024-05-30T16:23:29.665 INFO:teuthology.orchestra.run.smithi006.stderr: 73150K .......... .......... .......... .......... .......... 78% 125M 0s 2024-05-30T16:23:29.666 INFO:teuthology.orchestra.run.smithi006.stderr: 73200K .......... .......... .......... .......... .......... 78% 103M 0s 2024-05-30T16:23:29.666 INFO:teuthology.orchestra.run.smithi006.stderr: 73250K .......... .......... .......... .......... .......... 78% 122M 0s 2024-05-30T16:23:29.666 INFO:teuthology.orchestra.run.smithi006.stderr: 73300K .......... .......... .......... .......... .......... 78% 107M 0s 2024-05-30T16:23:29.666 INFO:teuthology.orchestra.run.smithi006.stderr: 73350K .......... .......... .......... .......... .......... 79% 99.5M 0s 2024-05-30T16:23:29.666 INFO:teuthology.orchestra.run.smithi006.stderr: 73400K .......... .......... .......... .......... .......... 79% 103M 0s 2024-05-30T16:23:29.666 INFO:teuthology.orchestra.run.smithi006.stderr: 73450K .......... .......... .......... .......... .......... 79% 113M 0s 2024-05-30T16:23:29.666 INFO:teuthology.orchestra.run.smithi006.stderr: 73500K .......... .......... .......... .......... .......... 79% 118M 0s 2024-05-30T16:23:29.666 INFO:teuthology.orchestra.run.smithi006.stderr: 73550K .......... .......... .......... .......... .......... 79% 125M 0s 2024-05-30T16:23:29.666 INFO:teuthology.orchestra.run.smithi006.stderr: 73600K .......... .......... .......... .......... .......... 79% 107M 0s 2024-05-30T16:23:29.666 INFO:teuthology.orchestra.run.smithi006.stderr: 73650K .......... .......... .......... .......... .......... 79% 82.9M 0s 2024-05-30T16:23:29.666 INFO:teuthology.orchestra.run.smithi006.stderr: 73700K .......... .......... .......... .......... .......... 79% 86.7M 0s 2024-05-30T16:23:29.666 INFO:teuthology.orchestra.run.smithi006.stderr: 73750K .......... .......... .......... .......... .......... 79% 149M 0s 2024-05-30T16:23:29.666 INFO:teuthology.orchestra.run.smithi006.stderr: 73800K .......... .......... .......... .......... .......... 79% 105M 0s 2024-05-30T16:23:29.667 INFO:teuthology.orchestra.run.smithi006.stderr: 73850K .......... .......... .......... .......... .......... 79% 95.5M 0s 2024-05-30T16:23:29.667 INFO:teuthology.orchestra.run.smithi006.stderr: 73900K .......... .......... .......... .......... .......... 79% 75.8M 0s 2024-05-30T16:23:29.667 INFO:teuthology.orchestra.run.smithi006.stderr: 73950K .......... .......... .......... .......... .......... 79% 26.9M 0s 2024-05-30T16:23:29.667 INFO:teuthology.orchestra.run.smithi006.stderr: 74000K .......... .......... .......... .......... .......... 79% 113M 0s 2024-05-30T16:23:29.667 INFO:teuthology.orchestra.run.smithi006.stderr: 74050K .......... .......... .......... .......... .......... 79% 93.7M 0s 2024-05-30T16:23:29.667 INFO:teuthology.orchestra.run.smithi006.stderr: 74100K .......... .......... .......... .......... .......... 79% 77.5M 0s 2024-05-30T16:23:29.667 INFO:teuthology.orchestra.run.smithi006.stderr: 74150K .......... .......... .......... .......... .......... 79% 125M 0s 2024-05-30T16:23:29.667 INFO:teuthology.orchestra.run.smithi006.stderr: 74200K .......... .......... .......... .......... .......... 79% 86.3M 0s 2024-05-30T16:23:29.667 INFO:teuthology.orchestra.run.smithi006.stderr: 74250K .......... .......... .......... .......... .......... 79% 114M 0s 2024-05-30T16:23:29.667 INFO:teuthology.orchestra.run.smithi006.stderr: 74300K .......... .......... .......... .......... .......... 80% 65.3M 0s 2024-05-30T16:23:29.667 INFO:teuthology.orchestra.run.smithi006.stderr: 74350K .......... .......... .......... .......... .......... 80% 63.9M 0s 2024-05-30T16:23:29.667 INFO:teuthology.orchestra.run.smithi006.stderr: 74400K .......... .......... .......... .......... .......... 80% 105M 0s 2024-05-30T16:23:29.667 INFO:teuthology.orchestra.run.smithi006.stderr: 74450K .......... .......... .......... .......... .......... 80% 64.9M 0s 2024-05-30T16:23:29.668 INFO:teuthology.orchestra.run.smithi006.stderr: 74500K .......... .......... .......... .......... .......... 80% 120M 0s 2024-05-30T16:23:29.668 INFO:teuthology.orchestra.run.smithi006.stderr: 74550K .......... .......... .......... .......... .......... 80% 67.8M 0s 2024-05-30T16:23:29.668 INFO:teuthology.orchestra.run.smithi006.stderr: 74600K .......... .......... .......... .......... .......... 80% 82.7M 0s 2024-05-30T16:23:29.668 INFO:teuthology.orchestra.run.smithi006.stderr: 74650K .......... .......... .......... .......... .......... 80% 117M 0s 2024-05-30T16:23:29.668 INFO:teuthology.orchestra.run.smithi006.stderr: 74700K .......... .......... .......... .......... .......... 80% 93.4M 0s 2024-05-30T16:23:29.668 INFO:teuthology.orchestra.run.smithi006.stderr: 74750K .......... .......... .......... .......... .......... 80% 114M 0s 2024-05-30T16:23:29.668 INFO:teuthology.orchestra.run.smithi006.stderr: 74800K .......... .......... .......... .......... .......... 80% 138M 0s 2024-05-30T16:23:29.668 INFO:teuthology.orchestra.run.smithi006.stderr: 74850K .......... .......... .......... .......... .......... 80% 84.6M 0s 2024-05-30T16:23:29.668 INFO:teuthology.orchestra.run.smithi006.stderr: 74900K .......... .......... .......... .......... .......... 80% 118M 0s 2024-05-30T16:23:29.668 INFO:teuthology.orchestra.run.smithi006.stderr: 74950K .......... .......... .......... .......... .......... 80% 114M 0s 2024-05-30T16:23:29.668 INFO:teuthology.orchestra.run.smithi006.stderr: 75000K .......... .......... .......... .......... .......... 80% 98.5M 0s 2024-05-30T16:23:29.668 INFO:teuthology.orchestra.run.smithi006.stderr: 75050K .......... .......... .......... .......... .......... 80% 47.1M 0s 2024-05-30T16:23:29.668 INFO:teuthology.orchestra.run.smithi006.stderr: 75100K .......... .......... .......... .......... .......... 80% 112M 0s 2024-05-30T16:23:29.669 INFO:teuthology.orchestra.run.smithi006.stderr: 75150K .......... .......... .......... .......... .......... 80% 119M 0s 2024-05-30T16:23:29.669 INFO:teuthology.orchestra.run.smithi006.stderr: 75200K .......... .......... .......... .......... .......... 81% 92.4M 0s 2024-05-30T16:23:29.669 INFO:teuthology.orchestra.run.smithi006.stderr: 75250K .......... .......... .......... .......... .......... 81% 98.9M 0s 2024-05-30T16:23:29.669 INFO:teuthology.orchestra.run.smithi006.stderr: 75300K .......... .......... .......... .......... .......... 81% 99.7M 0s 2024-05-30T16:23:29.669 INFO:teuthology.orchestra.run.smithi006.stderr: 75350K .......... .......... .......... .......... .......... 81% 98.4M 0s 2024-05-30T16:23:29.669 INFO:teuthology.orchestra.run.smithi006.stderr: 75400K .......... .......... .......... .......... .......... 81% 121M 0s 2024-05-30T16:23:29.669 INFO:teuthology.orchestra.run.smithi006.stderr: 75450K .......... .......... .......... .......... .......... 81% 93.6M 0s 2024-05-30T16:23:29.669 INFO:teuthology.orchestra.run.smithi006.stderr: 75500K .......... .......... .......... .......... .......... 81% 116M 0s 2024-05-30T16:23:29.669 INFO:teuthology.orchestra.run.smithi006.stderr: 75550K .......... .......... .......... .......... .......... 81% 106M 0s 2024-05-30T16:23:29.669 INFO:teuthology.orchestra.run.smithi006.stderr: 75600K .......... .......... .......... .......... .......... 81% 117M 0s 2024-05-30T16:23:29.669 INFO:teuthology.orchestra.run.smithi006.stderr: 75650K .......... .......... .......... .......... .......... 81% 85.4M 0s 2024-05-30T16:23:29.669 INFO:teuthology.orchestra.run.smithi006.stderr: 75700K .......... .......... .......... .......... .......... 81% 121M 0s 2024-05-30T16:23:29.669 INFO:teuthology.orchestra.run.smithi006.stderr: 75750K .......... .......... .......... .......... .......... 81% 112M 0s 2024-05-30T16:23:29.670 INFO:teuthology.orchestra.run.smithi006.stderr: 75800K .......... .......... .......... .......... .......... 81% 112M 0s 2024-05-30T16:23:29.670 INFO:teuthology.orchestra.run.smithi006.stderr: 75850K .......... .......... .......... .......... .......... 81% 109M 0s 2024-05-30T16:23:29.670 INFO:teuthology.orchestra.run.smithi006.stderr: 75900K .......... .......... .......... .......... .......... 81% 114M 0s 2024-05-30T16:23:29.670 INFO:teuthology.orchestra.run.smithi006.stderr: 75950K .......... .......... .......... .......... .......... 81% 116M 0s 2024-05-30T16:23:29.670 INFO:teuthology.orchestra.run.smithi006.stderr: 76000K .......... .......... .......... .......... .......... 81% 110M 0s 2024-05-30T16:23:29.670 INFO:teuthology.orchestra.run.smithi006.stderr: 76050K .......... .......... .......... .......... .......... 81% 41.9M 0s 2024-05-30T16:23:29.670 INFO:teuthology.orchestra.run.smithi006.stderr: 76100K .......... .......... .......... .......... .......... 81% 47.3M 0s 2024-05-30T16:23:29.670 INFO:teuthology.orchestra.run.smithi006.stderr: 76150K .......... .......... .......... .......... .......... 82% 168M 0s 2024-05-30T16:23:29.670 INFO:teuthology.orchestra.run.smithi006.stderr: 76200K .......... .......... .......... .......... .......... 82% 50.6M 0s 2024-05-30T16:23:29.670 INFO:teuthology.orchestra.run.smithi006.stderr: 76250K .......... .......... .......... .......... .......... 82% 61.2M 0s 2024-05-30T16:23:29.670 INFO:teuthology.orchestra.run.smithi006.stderr: 76300K .......... .......... .......... .......... .......... 82% 118M 0s 2024-05-30T16:23:29.670 INFO:teuthology.orchestra.run.smithi006.stderr: 76350K .......... .......... .......... .......... .......... 82% 107M 0s 2024-05-30T16:23:29.671 INFO:teuthology.orchestra.run.smithi006.stderr: 76400K .......... .......... .......... .......... .......... 82% 83.3M 0s 2024-05-30T16:23:29.671 INFO:teuthology.orchestra.run.smithi006.stderr: 76450K .......... .......... .......... .......... .......... 82% 114M 0s 2024-05-30T16:23:29.671 INFO:teuthology.orchestra.run.smithi006.stderr: 76500K .......... .......... .......... .......... .......... 82% 114M 0s 2024-05-30T16:23:29.671 INFO:teuthology.orchestra.run.smithi006.stderr: 76550K .......... .......... .......... .......... .......... 82% 95.6M 0s 2024-05-30T16:23:29.671 INFO:teuthology.orchestra.run.smithi006.stderr: 76600K .......... .......... .......... .......... .......... 82% 111M 0s 2024-05-30T16:23:29.671 INFO:teuthology.orchestra.run.smithi006.stderr: 76650K .......... .......... .......... .......... .......... 82% 72.9M 0s 2024-05-30T16:23:29.671 INFO:teuthology.orchestra.run.smithi006.stderr: 76700K .......... .......... .......... .......... .......... 82% 51.2M 0s 2024-05-30T16:23:29.671 INFO:teuthology.orchestra.run.smithi006.stderr: 76750K .......... .......... .......... .......... .......... 82% 72.4M 0s 2024-05-30T16:23:29.671 INFO:teuthology.orchestra.run.smithi006.stderr: 76800K .......... .......... .......... .......... .......... 82% 54.6M 0s 2024-05-30T16:23:29.671 INFO:teuthology.orchestra.run.smithi006.stderr: 76850K .......... .......... .......... .......... .......... 82% 112M 0s 2024-05-30T16:23:29.671 INFO:teuthology.orchestra.run.smithi006.stderr: 76900K .......... .......... .......... .......... .......... 82% 98.7M 0s 2024-05-30T16:23:29.671 INFO:teuthology.orchestra.run.smithi006.stderr: 76950K .......... .......... .......... .......... .......... 82% 105M 0s 2024-05-30T16:23:29.671 INFO:teuthology.orchestra.run.smithi006.stderr: 77000K .......... .......... .......... .......... .......... 82% 74.8M 0s 2024-05-30T16:23:29.671 INFO:teuthology.orchestra.run.smithi006.stderr: 77050K .......... .......... .......... .......... .......... 82% 115M 0s 2024-05-30T16:23:29.672 INFO:teuthology.orchestra.run.smithi006.stderr: 77100K .......... .......... .......... .......... .......... 83% 101M 0s 2024-05-30T16:23:29.672 INFO:teuthology.orchestra.run.smithi006.stderr: 77150K .......... .......... .......... .......... .......... 83% 110M 0s 2024-05-30T16:23:29.672 INFO:teuthology.orchestra.run.smithi006.stderr: 77200K .......... .......... .......... .......... .......... 83% 85.2M 0s 2024-05-30T16:23:29.672 INFO:teuthology.orchestra.run.smithi006.stderr: 77250K .......... .......... .......... .......... .......... 83% 45.2M 0s 2024-05-30T16:23:29.672 INFO:teuthology.orchestra.run.smithi006.stderr: 77300K .......... .......... .......... .......... .......... 83% 106M 0s 2024-05-30T16:23:29.672 INFO:teuthology.orchestra.run.smithi006.stderr: 77350K .......... .......... .......... .......... .......... 83% 116M 0s 2024-05-30T16:23:29.672 INFO:teuthology.orchestra.run.smithi006.stderr: 77400K .......... .......... .......... .......... .......... 83% 100M 0s 2024-05-30T16:23:29.672 INFO:teuthology.orchestra.run.smithi006.stderr: 77450K .......... .......... .......... .......... .......... 83% 118M 0s 2024-05-30T16:23:29.672 INFO:teuthology.orchestra.run.smithi006.stderr: 77500K .......... .......... .......... .......... .......... 83% 126M 0s 2024-05-30T16:23:29.672 INFO:teuthology.orchestra.run.smithi006.stderr: 77550K .......... .......... .......... .......... .......... 83% 107M 0s 2024-05-30T16:23:29.672 INFO:teuthology.orchestra.run.smithi006.stderr: 77600K .......... .......... .......... .......... .......... 83% 90.4M 0s 2024-05-30T16:23:29.672 INFO:teuthology.orchestra.run.smithi006.stderr: 77650K .......... .......... .......... .......... .......... 83% 113M 0s 2024-05-30T16:23:29.672 INFO:teuthology.orchestra.run.smithi006.stderr: 77700K .......... .......... .......... .......... .......... 83% 128M 0s 2024-05-30T16:23:29.672 INFO:teuthology.orchestra.run.smithi006.stderr: 77750K .......... .......... .......... .......... .......... 83% 111M 0s 2024-05-30T16:23:29.673 INFO:teuthology.orchestra.run.smithi006.stderr: 77800K .......... .......... .......... .......... .......... 83% 104M 0s 2024-05-30T16:23:29.673 INFO:teuthology.orchestra.run.smithi006.stderr: 77850K .......... .......... .......... .......... .......... 83% 114M 0s 2024-05-30T16:23:29.673 INFO:teuthology.orchestra.run.smithi006.stderr: 77900K .......... .......... .......... .......... .......... 83% 92.8M 0s 2024-05-30T16:23:29.673 INFO:teuthology.orchestra.run.smithi006.stderr: 77950K .......... .......... .......... .......... .......... 83% 137M 0s 2024-05-30T16:23:29.673 INFO:teuthology.orchestra.run.smithi006.stderr: 78000K .......... .......... .......... .......... .......... 84% 88.2M 0s 2024-05-30T16:23:29.673 INFO:teuthology.orchestra.run.smithi006.stderr: 78050K .......... .......... .......... .......... .......... 84% 116M 0s 2024-05-30T16:23:29.673 INFO:teuthology.orchestra.run.smithi006.stderr: 78100K .......... .......... .......... .......... .......... 84% 104M 0s 2024-05-30T16:23:29.673 INFO:teuthology.orchestra.run.smithi006.stderr: 78150K .......... .......... .......... .......... .......... 84% 104M 0s 2024-05-30T16:23:29.673 INFO:teuthology.orchestra.run.smithi006.stderr: 78200K .......... .......... .......... .......... .......... 84% 84.6M 0s 2024-05-30T16:23:29.673 INFO:teuthology.orchestra.run.smithi006.stderr: 78250K .......... .......... .......... .......... .......... 84% 44.0M 0s 2024-05-30T16:23:29.673 INFO:teuthology.orchestra.run.smithi006.stderr: 78300K .......... .......... .......... .......... .......... 84% 79.1M 0s 2024-05-30T16:23:29.673 INFO:teuthology.orchestra.run.smithi006.stderr: 78350K .......... .......... .......... .......... .......... 84% 68.6M 0s 2024-05-30T16:23:29.673 INFO:teuthology.orchestra.run.smithi006.stderr: 78400K .......... .......... .......... .......... .......... 84% 56.8M 0s 2024-05-30T16:23:29.673 INFO:teuthology.orchestra.run.smithi006.stderr: 78450K .......... .......... .......... .......... .......... 84% 87.2M 0s 2024-05-30T16:23:29.674 INFO:teuthology.orchestra.run.smithi006.stderr: 78500K .......... .......... .......... .......... .......... 84% 95.3M 0s 2024-05-30T16:23:29.674 INFO:teuthology.orchestra.run.smithi006.stderr: 78550K .......... .......... .......... .......... .......... 84% 35.1M 0s 2024-05-30T16:23:29.674 INFO:teuthology.orchestra.run.smithi006.stderr: 78600K .......... .......... .......... .......... .......... 84% 102M 0s 2024-05-30T16:23:29.674 INFO:teuthology.orchestra.run.smithi006.stderr: 78650K .......... .......... .......... .......... .......... 84% 107M 0s 2024-05-30T16:23:29.674 INFO:teuthology.orchestra.run.smithi006.stderr: 78700K .......... .......... .......... .......... .......... 84% 88.2M 0s 2024-05-30T16:23:29.674 INFO:teuthology.orchestra.run.smithi006.stderr: 78750K .......... .......... .......... .......... .......... 84% 133M 0s 2024-05-30T16:23:29.674 INFO:teuthology.orchestra.run.smithi006.stderr: 78800K .......... .......... .......... .......... .......... 84% 99.8M 0s 2024-05-30T16:23:29.674 INFO:teuthology.orchestra.run.smithi006.stderr: 78850K .......... .......... .......... .......... .......... 84% 74.2M 0s 2024-05-30T16:23:29.674 INFO:teuthology.orchestra.run.smithi006.stderr: 78900K .......... .......... .......... .......... .......... 84% 105M 0s 2024-05-30T16:23:29.674 INFO:teuthology.orchestra.run.smithi006.stderr: 78950K .......... .......... .......... .......... .......... 85% 51.0M 0s 2024-05-30T16:23:29.674 INFO:teuthology.orchestra.run.smithi006.stderr: 79000K .......... .......... .......... .......... .......... 85% 71.9M 0s 2024-05-30T16:23:29.675 INFO:teuthology.orchestra.run.smithi006.stderr: 79050K .......... .......... .......... .......... .......... 85% 64.8M 0s 2024-05-30T16:23:29.675 INFO:teuthology.orchestra.run.smithi006.stderr: 79100K .......... .......... .......... .......... .......... 85% 83.0M 0s 2024-05-30T16:23:29.675 INFO:teuthology.orchestra.run.smithi006.stderr: 79150K .......... .......... .......... .......... .......... 85% 92.2M 0s 2024-05-30T16:23:29.675 INFO:teuthology.orchestra.run.smithi006.stderr: 79200K .......... .......... .......... .......... .......... 85% 88.7M 0s 2024-05-30T16:23:29.675 INFO:teuthology.orchestra.run.smithi006.stderr: 79250K .......... .......... .......... .......... .......... 85% 67.7M 0s 2024-05-30T16:23:29.675 INFO:teuthology.orchestra.run.smithi006.stderr: 79300K .......... .......... .......... .......... .......... 85% 63.6M 0s 2024-05-30T16:23:29.675 INFO:teuthology.orchestra.run.smithi006.stderr: 79350K .......... .......... .......... .......... .......... 85% 69.1M 0s 2024-05-30T16:23:29.675 INFO:teuthology.orchestra.run.smithi006.stderr: 79400K .......... .......... .......... .......... .......... 85% 98.3M 0s 2024-05-30T16:23:29.675 INFO:teuthology.orchestra.run.smithi006.stderr: 79450K .......... .......... .......... .......... .......... 85% 128M 0s 2024-05-30T16:23:29.675 INFO:teuthology.orchestra.run.smithi006.stderr: 79500K .......... .......... .......... .......... .......... 85% 53.7M 0s 2024-05-30T16:23:29.675 INFO:teuthology.orchestra.run.smithi006.stderr: 79550K .......... .......... .......... .......... .......... 85% 69.8M 0s 2024-05-30T16:23:29.675 INFO:teuthology.orchestra.run.smithi006.stderr: 79600K .......... .......... .......... .......... .......... 85% 66.9M 0s 2024-05-30T16:23:29.675 INFO:teuthology.orchestra.run.smithi006.stderr: 79650K .......... .......... .......... .......... .......... 85% 63.3M 0s 2024-05-30T16:23:29.676 INFO:teuthology.orchestra.run.smithi006.stderr: 79700K .......... .......... .......... .......... .......... 85% 65.1M 0s 2024-05-30T16:23:29.676 INFO:teuthology.orchestra.run.smithi006.stderr: 79750K .......... .......... .......... .......... .......... 85% 69.3M 0s 2024-05-30T16:23:29.676 INFO:teuthology.orchestra.run.smithi006.stderr: 79800K .......... .......... .......... .......... .......... 85% 72.0M 0s 2024-05-30T16:23:29.676 INFO:teuthology.orchestra.run.smithi006.stderr: 79850K .......... .......... .......... .......... .......... 86% 72.6M 0s 2024-05-30T16:23:29.676 INFO:teuthology.orchestra.run.smithi006.stderr: 79900K .......... .......... .......... .......... .......... 86% 57.0M 0s 2024-05-30T16:23:29.676 INFO:teuthology.orchestra.run.smithi006.stderr: 79950K .......... .......... .......... .......... .......... 86% 63.0M 0s 2024-05-30T16:23:29.676 INFO:teuthology.orchestra.run.smithi006.stderr: 80000K .......... .......... .......... .......... .......... 86% 90.6M 0s 2024-05-30T16:23:29.676 INFO:teuthology.orchestra.run.smithi006.stderr: 80050K .......... .......... .......... .......... .......... 86% 94.0M 0s 2024-05-30T16:23:29.676 INFO:teuthology.orchestra.run.smithi006.stderr: 80100K .......... .......... .......... .......... .......... 86% 90.1M 0s 2024-05-30T16:23:29.676 INFO:teuthology.orchestra.run.smithi006.stderr: 80150K .......... .......... .......... .......... .......... 86% 73.7M 0s 2024-05-30T16:23:29.676 INFO:teuthology.orchestra.run.smithi006.stderr: 80200K .......... .......... .......... .......... .......... 86% 79.4M 0s 2024-05-30T16:23:29.676 INFO:teuthology.orchestra.run.smithi006.stderr: 80250K .......... .......... .......... .......... .......... 86% 78.0M 0s 2024-05-30T16:23:29.676 INFO:teuthology.orchestra.run.smithi006.stderr: 80300K .......... .......... .......... .......... .......... 86% 65.2M 0s 2024-05-30T16:23:29.677 INFO:teuthology.orchestra.run.smithi006.stderr: 80350K .......... .......... .......... .......... .......... 86% 78.0M 0s 2024-05-30T16:23:29.677 INFO:teuthology.orchestra.run.smithi006.stderr: 80400K .......... .......... .......... .......... .......... 86% 144M 0s 2024-05-30T16:23:29.677 INFO:teuthology.orchestra.run.smithi006.stderr: 80450K .......... .......... .......... .......... .......... 86% 75.9M 0s 2024-05-30T16:23:29.677 INFO:teuthology.orchestra.run.smithi006.stderr: 80500K .......... .......... .......... .......... .......... 86% 75.0M 0s 2024-05-30T16:23:29.677 INFO:teuthology.orchestra.run.smithi006.stderr: 80550K .......... .......... .......... .......... .......... 86% 82.7M 0s 2024-05-30T16:23:29.677 INFO:teuthology.orchestra.run.smithi006.stderr: 80600K .......... .......... .......... .......... .......... 86% 139M 0s 2024-05-30T16:23:29.677 INFO:teuthology.orchestra.run.smithi006.stderr: 80650K .......... .......... .......... .......... .......... 86% 81.2M 0s 2024-05-30T16:23:29.677 INFO:teuthology.orchestra.run.smithi006.stderr: 80700K .......... .......... .......... .......... .......... 86% 103M 0s 2024-05-30T16:23:29.677 INFO:teuthology.orchestra.run.smithi006.stderr: 80750K .......... .......... .......... .......... .......... 86% 133M 0s 2024-05-30T16:23:29.677 INFO:teuthology.orchestra.run.smithi006.stderr: 80800K .......... .......... .......... .......... .......... 87% 123M 0s 2024-05-30T16:23:29.677 INFO:teuthology.orchestra.run.smithi006.stderr: 80850K .......... .......... .......... .......... .......... 87% 129M 0s 2024-05-30T16:23:29.677 INFO:teuthology.orchestra.run.smithi006.stderr: 80900K .......... .......... .......... .......... .......... 87% 127M 0s 2024-05-30T16:23:29.677 INFO:teuthology.orchestra.run.smithi006.stderr: 80950K .......... .......... .......... .......... .......... 87% 138M 0s 2024-05-30T16:23:29.678 INFO:teuthology.orchestra.run.smithi006.stderr: 81000K .......... .......... .......... .......... .......... 87% 110M 0s 2024-05-30T16:23:29.678 INFO:teuthology.orchestra.run.smithi006.stderr: 81050K .......... .......... .......... .......... .......... 87% 140M 0s 2024-05-30T16:23:29.678 INFO:teuthology.orchestra.run.smithi006.stderr: 81100K .......... .......... .......... .......... .......... 87% 144M 0s 2024-05-30T16:23:29.678 INFO:teuthology.orchestra.run.smithi006.stderr: 81150K .......... .......... .......... .......... .......... 87% 118M 0s 2024-05-30T16:23:29.678 INFO:teuthology.orchestra.run.smithi006.stderr: 81200K .......... .......... .......... .......... .......... 87% 225M 0s 2024-05-30T16:23:29.678 INFO:teuthology.orchestra.run.smithi006.stderr: 81250K .......... .......... .......... .......... .......... 87% 259M 0s 2024-05-30T16:23:29.678 INFO:teuthology.orchestra.run.smithi006.stderr: 81300K .......... .......... .......... .......... .......... 87% 87.4M 0s 2024-05-30T16:23:29.678 INFO:teuthology.orchestra.run.smithi006.stderr: 81350K .......... .......... .......... .......... .......... 87% 101M 0s 2024-05-30T16:23:29.678 INFO:teuthology.orchestra.run.smithi006.stderr: 81400K .......... .......... .......... .......... .......... 87% 89.3M 0s 2024-05-30T16:23:29.678 INFO:teuthology.orchestra.run.smithi006.stderr: 81450K .......... .......... .......... .......... .......... 87% 109M 0s 2024-05-30T16:23:29.678 INFO:teuthology.orchestra.run.smithi006.stderr: 81500K .......... .......... .......... .......... .......... 87% 119M 0s 2024-05-30T16:23:29.678 INFO:teuthology.orchestra.run.smithi006.stderr: 81550K .......... .......... .......... .......... .......... 87% 106M 0s 2024-05-30T16:23:29.678 INFO:teuthology.orchestra.run.smithi006.stderr: 81600K .......... .......... .......... .......... .......... 87% 99.9M 0s 2024-05-30T16:23:29.679 INFO:teuthology.orchestra.run.smithi006.stderr: 81650K .......... .......... .......... .......... .......... 87% 87.3M 0s 2024-05-30T16:23:29.679 INFO:teuthology.orchestra.run.smithi006.stderr: 81700K .......... .......... .......... .......... .......... 87% 121M 0s 2024-05-30T16:23:29.679 INFO:teuthology.orchestra.run.smithi006.stderr: 81750K .......... .......... .......... .......... .......... 88% 108M 0s 2024-05-30T16:23:29.679 INFO:teuthology.orchestra.run.smithi006.stderr: 81800K .......... .......... .......... .......... .......... 88% 40.3M 0s 2024-05-30T16:23:29.679 INFO:teuthology.orchestra.run.smithi006.stderr: 81850K .......... .......... .......... .......... .......... 88% 99.3M 0s 2024-05-30T16:23:29.679 INFO:teuthology.orchestra.run.smithi006.stderr: 81900K .......... .......... .......... .......... .......... 88% 105M 0s 2024-05-30T16:23:29.679 INFO:teuthology.orchestra.run.smithi006.stderr: 81950K .......... .......... .......... .......... .......... 88% 111M 0s 2024-05-30T16:23:29.679 INFO:teuthology.orchestra.run.smithi006.stderr: 82000K .......... .......... .......... .......... .......... 88% 103M 0s 2024-05-30T16:23:29.679 INFO:teuthology.orchestra.run.smithi006.stderr: 82050K .......... .......... .......... .......... .......... 88% 129M 0s 2024-05-30T16:23:29.679 INFO:teuthology.orchestra.run.smithi006.stderr: 82100K .......... .......... .......... .......... .......... 88% 107M 0s 2024-05-30T16:23:29.679 INFO:teuthology.orchestra.run.smithi006.stderr: 82150K .......... .......... .......... .......... .......... 88% 124M 0s 2024-05-30T16:23:29.679 INFO:teuthology.orchestra.run.smithi006.stderr: 82200K .......... .......... .......... .......... .......... 88% 82.0M 0s 2024-05-30T16:23:29.680 INFO:teuthology.orchestra.run.smithi006.stderr: 82250K .......... .......... .......... .......... .......... 88% 84.7M 0s 2024-05-30T16:23:29.680 INFO:teuthology.orchestra.run.smithi006.stderr: 82300K .......... .......... .......... .......... .......... 88% 116M 0s 2024-05-30T16:23:29.680 INFO:teuthology.orchestra.run.smithi006.stderr: 82350K .......... .......... .......... .......... .......... 88% 110M 0s 2024-05-30T16:23:29.680 INFO:teuthology.orchestra.run.smithi006.stderr: 82400K .......... .......... .......... .......... .......... 88% 100M 0s 2024-05-30T16:23:29.680 INFO:teuthology.orchestra.run.smithi006.stderr: 82450K .......... .......... .......... .......... .......... 88% 116M 0s 2024-05-30T16:23:29.680 INFO:teuthology.orchestra.run.smithi006.stderr: 82500K .......... .......... .......... .......... .......... 88% 85.1M 0s 2024-05-30T16:23:29.680 INFO:teuthology.orchestra.run.smithi006.stderr: 82550K .......... .......... .......... .......... .......... 88% 55.0M 0s 2024-05-30T16:23:29.680 INFO:teuthology.orchestra.run.smithi006.stderr: 82600K .......... .......... .......... .......... .......... 88% 48.8M 0s 2024-05-30T16:23:29.680 INFO:teuthology.orchestra.run.smithi006.stderr: 82650K .......... .......... .......... .......... .......... 89% 123M 0s 2024-05-30T16:23:29.680 INFO:teuthology.orchestra.run.smithi006.stderr: 82700K .......... .......... .......... .......... .......... 89% 118M 0s 2024-05-30T16:23:29.680 INFO:teuthology.orchestra.run.smithi006.stderr: 82750K .......... .......... .......... .......... .......... 89% 117M 0s 2024-05-30T16:23:29.680 INFO:teuthology.orchestra.run.smithi006.stderr: 82800K .......... .......... .......... .......... .......... 89% 104M 0s 2024-05-30T16:23:29.681 INFO:teuthology.orchestra.run.smithi006.stderr: 82850K .......... .......... .......... .......... .......... 89% 123M 0s 2024-05-30T16:23:29.681 INFO:teuthology.orchestra.run.smithi006.stderr: 82900K .......... .......... .......... .......... .......... 89% 118M 0s 2024-05-30T16:23:29.681 INFO:teuthology.orchestra.run.smithi006.stderr: 82950K .......... .......... .......... .......... .......... 89% 111M 0s 2024-05-30T16:23:29.681 INFO:teuthology.orchestra.run.smithi006.stderr: 83000K .......... .......... .......... .......... .......... 89% 91.3M 0s 2024-05-30T16:23:29.681 INFO:teuthology.orchestra.run.smithi006.stderr: 83050K .......... .......... .......... .......... .......... 89% 87.0M 0s 2024-05-30T16:23:29.681 INFO:teuthology.orchestra.run.smithi006.stderr: 83100K .......... .......... .......... .......... .......... 89% 117M 0s 2024-05-30T16:23:29.681 INFO:teuthology.orchestra.run.smithi006.stderr: 83150K .......... .......... .......... .......... .......... 89% 121M 0s 2024-05-30T16:23:29.681 INFO:teuthology.orchestra.run.smithi006.stderr: 83200K .......... .......... .......... .......... .......... 89% 113M 0s 2024-05-30T16:23:29.681 INFO:teuthology.orchestra.run.smithi006.stderr: 83250K .......... .......... .......... .......... .......... 89% 113M 0s 2024-05-30T16:23:29.681 INFO:teuthology.orchestra.run.smithi006.stderr: 83300K .......... .......... .......... .......... .......... 89% 88.6M 0s 2024-05-30T16:23:29.681 INFO:teuthology.orchestra.run.smithi006.stderr: 83350K .......... .......... .......... .......... .......... 89% 103M 0s 2024-05-30T16:23:29.682 INFO:teuthology.orchestra.run.smithi006.stderr: 83400K .......... .......... .......... .......... .......... 89% 136M 0s 2024-05-30T16:23:29.682 INFO:teuthology.orchestra.run.smithi006.stderr: 83450K .......... .......... .......... .......... .......... 89% 109M 0s 2024-05-30T16:23:29.682 INFO:teuthology.orchestra.run.smithi006.stderr: 83500K .......... .......... .......... .......... .......... 89% 42.7M 0s 2024-05-30T16:23:29.682 INFO:teuthology.orchestra.run.smithi006.stderr: 83550K .......... .......... .......... .......... .......... 89% 21.4M 0s 2024-05-30T16:23:29.682 INFO:teuthology.orchestra.run.smithi006.stderr: 83600K .......... .......... .......... .......... .......... 90% 96.0M 0s 2024-05-30T16:23:29.682 INFO:teuthology.orchestra.run.smithi006.stderr: 83650K .......... .......... .......... .......... .......... 90% 58.2M 0s 2024-05-30T16:23:29.682 INFO:teuthology.orchestra.run.smithi006.stderr: 83700K .......... .......... .......... .......... .......... 90% 76.7M 0s 2024-05-30T16:23:29.682 INFO:teuthology.orchestra.run.smithi006.stderr: 83750K .......... .......... .......... .......... .......... 90% 113M 0s 2024-05-30T16:23:29.682 INFO:teuthology.orchestra.run.smithi006.stderr: 83800K .......... .......... .......... .......... .......... 90% 100M 0s 2024-05-30T16:23:29.682 INFO:teuthology.orchestra.run.smithi006.stderr: 83850K .......... .......... .......... .......... .......... 90% 121M 0s 2024-05-30T16:23:29.682 INFO:teuthology.orchestra.run.smithi006.stderr: 83900K .......... .......... .......... .......... .......... 90% 81.4M 0s 2024-05-30T16:23:29.682 INFO:teuthology.orchestra.run.smithi006.stderr: 83950K .......... .......... .......... .......... .......... 90% 102M 0s 2024-05-30T16:23:29.683 INFO:teuthology.orchestra.run.smithi006.stderr: 84000K .......... .......... .......... .......... .......... 90% 43.5M 0s 2024-05-30T16:23:29.683 INFO:teuthology.orchestra.run.smithi006.stderr: 84050K .......... .......... .......... .......... .......... 90% 94.7M 0s 2024-05-30T16:23:29.683 INFO:teuthology.orchestra.run.smithi006.stderr: 84100K .......... .......... .......... .......... .......... 90% 107M 0s 2024-05-30T16:23:29.683 INFO:teuthology.orchestra.run.smithi006.stderr: 84150K .......... .......... .......... .......... .......... 90% 109M 0s 2024-05-30T16:23:29.683 INFO:teuthology.orchestra.run.smithi006.stderr: 84200K .......... .......... .......... .......... .......... 90% 105M 0s 2024-05-30T16:23:29.683 INFO:teuthology.orchestra.run.smithi006.stderr: 84250K .......... .......... .......... .......... .......... 90% 118M 0s 2024-05-30T16:23:29.683 INFO:teuthology.orchestra.run.smithi006.stderr: 84300K .......... .......... .......... .......... .......... 90% 105M 0s 2024-05-30T16:23:29.683 INFO:teuthology.orchestra.run.smithi006.stderr: 84350K .......... .......... .......... .......... .......... 90% 117M 0s 2024-05-30T16:23:29.683 INFO:teuthology.orchestra.run.smithi006.stderr: 84400K .......... .......... .......... .......... .......... 90% 110M 0s 2024-05-30T16:23:29.683 INFO:teuthology.orchestra.run.smithi006.stderr: 84450K .......... .......... .......... .......... .......... 90% 123M 0s 2024-05-30T16:23:29.683 INFO:teuthology.orchestra.run.smithi006.stderr: 84500K .......... .......... .......... .......... .......... 91% 76.9M 0s 2024-05-30T16:23:29.683 INFO:teuthology.orchestra.run.smithi006.stderr: 84550K .......... .......... .......... .......... .......... 91% 137M 0s 2024-05-30T16:23:29.684 INFO:teuthology.orchestra.run.smithi006.stderr: 84600K .......... .......... .......... .......... .......... 91% 100M 0s 2024-05-30T16:23:29.684 INFO:teuthology.orchestra.run.smithi006.stderr: 84650K .......... .......... .......... .......... .......... 91% 116M 0s 2024-05-30T16:23:29.684 INFO:teuthology.orchestra.run.smithi006.stderr: 84700K .......... .......... .......... .......... .......... 91% 83.2M 0s 2024-05-30T16:23:29.684 INFO:teuthology.orchestra.run.smithi006.stderr: 84750K .......... .......... .......... .......... .......... 91% 122M 0s 2024-05-30T16:23:29.684 INFO:teuthology.orchestra.run.smithi006.stderr: 84800K .......... .......... .......... .......... .......... 91% 71.7M 0s 2024-05-30T16:23:29.684 INFO:teuthology.orchestra.run.smithi006.stderr: 84850K .......... .......... .......... .......... .......... 91% 63.6M 0s 2024-05-30T16:23:29.684 INFO:teuthology.orchestra.run.smithi006.stderr: 84900K .......... .......... .......... .......... .......... 91% 92.1M 0s 2024-05-30T16:23:29.684 INFO:teuthology.orchestra.run.smithi006.stderr: 84950K .......... .......... .......... .......... .......... 91% 87.1M 0s 2024-05-30T16:23:29.684 INFO:teuthology.orchestra.run.smithi006.stderr: 85000K .......... .......... .......... .......... .......... 91% 111M 0s 2024-05-30T16:23:29.684 INFO:teuthology.orchestra.run.smithi006.stderr: 85050K .......... .......... .......... .......... .......... 91% 117M 0s 2024-05-30T16:23:29.684 INFO:teuthology.orchestra.run.smithi006.stderr: 85100K .......... .......... .......... .......... .......... 91% 101M 0s 2024-05-30T16:23:29.684 INFO:teuthology.orchestra.run.smithi006.stderr: 85150K .......... .......... .......... .......... .......... 91% 126M 0s 2024-05-30T16:23:29.684 INFO:teuthology.orchestra.run.smithi006.stderr: 85200K .......... .......... .......... .......... .......... 91% 115M 0s 2024-05-30T16:23:29.684 INFO:teuthology.orchestra.run.smithi006.stderr: 85250K .......... .......... .......... .......... .......... 91% 93.5M 0s 2024-05-30T16:23:29.685 INFO:teuthology.orchestra.run.smithi006.stderr: 85300K .......... .......... .......... .......... .......... 91% 47.1M 0s 2024-05-30T16:23:29.685 INFO:teuthology.orchestra.run.smithi006.stderr: 85350K .......... .......... .......... .......... .......... 91% 202M 0s 2024-05-30T16:23:29.685 INFO:teuthology.orchestra.run.smithi006.stderr: 85400K .......... .......... .......... .......... .......... 91% 111M 0s 2024-05-30T16:23:29.685 INFO:teuthology.orchestra.run.smithi006.stderr: 85450K .......... .......... .......... .......... .......... 92% 118M 0s 2024-05-30T16:23:29.685 INFO:teuthology.orchestra.run.smithi006.stderr: 85500K .......... .......... .......... .......... .......... 92% 106M 0s 2024-05-30T16:23:29.685 INFO:teuthology.orchestra.run.smithi006.stderr: 85550K .......... .......... .......... .......... .......... 92% 117M 0s 2024-05-30T16:23:29.685 INFO:teuthology.orchestra.run.smithi006.stderr: 85600K .......... .......... .......... .......... .......... 92% 93.0M 0s 2024-05-30T16:23:29.685 INFO:teuthology.orchestra.run.smithi006.stderr: 85650K .......... .......... .......... .......... .......... 92% 94.9M 0s 2024-05-30T16:23:29.685 INFO:teuthology.orchestra.run.smithi006.stderr: 85700K .......... .......... .......... .......... .......... 92% 128M 0s 2024-05-30T16:23:29.685 INFO:teuthology.orchestra.run.smithi006.stderr: 85750K .......... .......... .......... .......... .......... 92% 40.7M 0s 2024-05-30T16:23:29.685 INFO:teuthology.orchestra.run.smithi006.stderr: 85800K .......... .......... .......... .......... .......... 92% 107M 0s 2024-05-30T16:23:29.685 INFO:teuthology.orchestra.run.smithi006.stderr: 85850K .......... .......... .......... .......... .......... 92% 34.5M 0s 2024-05-30T16:23:29.685 INFO:teuthology.orchestra.run.smithi006.stderr: 85900K .......... .......... .......... .......... .......... 92% 119M 0s 2024-05-30T16:23:29.685 INFO:teuthology.orchestra.run.smithi006.stderr: 85950K .......... .......... .......... .......... .......... 92% 48.7M 0s 2024-05-30T16:23:29.685 INFO:teuthology.orchestra.run.smithi006.stderr: 86000K .......... .......... .......... .......... .......... 92% 56.1M 0s 2024-05-30T16:23:29.686 INFO:teuthology.orchestra.run.smithi006.stderr: 86050K .......... .......... .......... .......... .......... 92% 79.2M 0s 2024-05-30T16:23:29.686 INFO:teuthology.orchestra.run.smithi006.stderr: 86100K .......... .......... .......... .......... .......... 92% 85.7M 0s 2024-05-30T16:23:29.686 INFO:teuthology.orchestra.run.smithi006.stderr: 86150K .......... .......... .......... .......... .......... 92% 122M 0s 2024-05-30T16:23:29.686 INFO:teuthology.orchestra.run.smithi006.stderr: 86200K .......... .......... .......... .......... .......... 92% 108M 0s 2024-05-30T16:23:29.686 INFO:teuthology.orchestra.run.smithi006.stderr: 86250K .......... .......... .......... .......... .......... 92% 86.4M 0s 2024-05-30T16:23:29.686 INFO:teuthology.orchestra.run.smithi006.stderr: 86300K .......... .......... .......... .......... .......... 92% 86.0M 0s 2024-05-30T16:23:29.686 INFO:teuthology.orchestra.run.smithi006.stderr: 86350K .......... .......... .......... .......... .......... 93% 73.5M 0s 2024-05-30T16:23:29.686 INFO:teuthology.orchestra.run.smithi006.stderr: 86400K .......... .......... .......... .......... .......... 93% 83.1M 0s 2024-05-30T16:23:29.686 INFO:teuthology.orchestra.run.smithi006.stderr: 86450K .......... .......... .......... .......... .......... 93% 117M 0s 2024-05-30T16:23:29.686 INFO:teuthology.orchestra.run.smithi006.stderr: 86500K .......... .......... .......... .......... .......... 93% 96.7M 0s 2024-05-30T16:23:29.686 INFO:teuthology.orchestra.run.smithi006.stderr: 86550K .......... .......... .......... .......... .......... 93% 103M 0s 2024-05-30T16:23:29.686 INFO:teuthology.orchestra.run.smithi006.stderr: 86600K .......... .......... .......... .......... .......... 93% 101M 0s 2024-05-30T16:23:29.687 INFO:teuthology.orchestra.run.smithi006.stderr: 86650K .......... .......... .......... .......... .......... 93% 116M 0s 2024-05-30T16:23:29.687 INFO:teuthology.orchestra.run.smithi006.stderr: 86700K .......... .......... .......... .......... .......... 93% 97.7M 0s 2024-05-30T16:23:29.687 INFO:teuthology.orchestra.run.smithi006.stderr: 86750K .......... .......... .......... .......... .......... 93% 126M 0s 2024-05-30T16:23:29.687 INFO:teuthology.orchestra.run.smithi006.stderr: 86800K .......... .......... .......... .......... .......... 93% 64.4M 0s 2024-05-30T16:23:29.687 INFO:teuthology.orchestra.run.smithi006.stderr: 86850K .......... .......... .......... .......... .......... 93% 123M 0s 2024-05-30T16:23:29.687 INFO:teuthology.orchestra.run.smithi006.stderr: 86900K .......... .......... .......... .......... .......... 93% 120M 0s 2024-05-30T16:23:29.687 INFO:teuthology.orchestra.run.smithi006.stderr: 86950K .......... .......... .......... .......... .......... 93% 70.0M 0s 2024-05-30T16:23:29.687 INFO:teuthology.orchestra.run.smithi006.stderr: 87000K .......... .......... .......... .......... .......... 93% 46.3M 0s 2024-05-30T16:23:29.687 INFO:teuthology.orchestra.run.smithi006.stderr: 87050K .......... .......... .......... .......... .......... 93% 65.4M 0s 2024-05-30T16:23:29.687 INFO:teuthology.orchestra.run.smithi006.stderr: 87100K .......... .......... .......... .......... .......... 93% 113M 0s 2024-05-30T16:23:29.687 INFO:teuthology.orchestra.run.smithi006.stderr: 87150K .......... .......... .......... .......... .......... 93% 122M 0s 2024-05-30T16:23:29.687 INFO:teuthology.orchestra.run.smithi006.stderr: 87200K .......... .......... .......... .......... .......... 93% 83.6M 0s 2024-05-30T16:23:29.687 INFO:teuthology.orchestra.run.smithi006.stderr: 87250K .......... .......... .......... .......... .......... 93% 126M 0s 2024-05-30T16:23:29.687 INFO:teuthology.orchestra.run.smithi006.stderr: 87300K .......... .......... .......... .......... .......... 94% 113M 0s 2024-05-30T16:23:29.688 INFO:teuthology.orchestra.run.smithi006.stderr: 87350K .......... .......... .......... .......... .......... 94% 110M 0s 2024-05-30T16:23:29.688 INFO:teuthology.orchestra.run.smithi006.stderr: 87400K .......... .......... .......... .......... .......... 94% 98.5M 0s 2024-05-30T16:23:29.688 INFO:teuthology.orchestra.run.smithi006.stderr: 87450K .......... .......... .......... .......... .......... 94% 114M 0s 2024-05-30T16:23:29.688 INFO:teuthology.orchestra.run.smithi006.stderr: 87500K .......... .......... .......... .......... .......... 94% 114M 0s 2024-05-30T16:23:29.688 INFO:teuthology.orchestra.run.smithi006.stderr: 87550K .......... .......... .......... .......... .......... 94% 109M 0s 2024-05-30T16:23:29.688 INFO:teuthology.orchestra.run.smithi006.stderr: 87600K .......... .......... .......... .......... .......... 94% 94.5M 0s 2024-05-30T16:23:29.688 INFO:teuthology.orchestra.run.smithi006.stderr: 87650K .......... .......... .......... .......... .......... 94% 114M 0s 2024-05-30T16:23:29.688 INFO:teuthology.orchestra.run.smithi006.stderr: 87700K .......... .......... .......... .......... .......... 94% 106M 0s 2024-05-30T16:23:29.688 INFO:teuthology.orchestra.run.smithi006.stderr: 87750K .......... .......... .......... .......... .......... 94% 77.9M 0s 2024-05-30T16:23:29.688 INFO:teuthology.orchestra.run.smithi006.stderr: 87800K .......... .......... .......... .......... .......... 94% 125M 0s 2024-05-30T16:23:29.688 INFO:teuthology.orchestra.run.smithi006.stderr: 87850K .......... .......... .......... .......... .......... 94% 117M 0s 2024-05-30T16:23:29.688 INFO:teuthology.orchestra.run.smithi006.stderr: 87900K .......... .......... .......... .......... .......... 94% 89.6M 0s 2024-05-30T16:23:29.688 INFO:teuthology.orchestra.run.smithi006.stderr: 87950K .......... .......... .......... .......... .......... 94% 115M 0s 2024-05-30T16:23:29.688 INFO:teuthology.orchestra.run.smithi006.stderr: 88000K .......... .......... .......... .......... .......... 94% 37.5M 0s 2024-05-30T16:23:29.688 INFO:teuthology.orchestra.run.smithi006.stderr: 88050K .......... .......... .......... .......... .......... 94% 125M 0s 2024-05-30T16:23:29.689 INFO:teuthology.orchestra.run.smithi006.stderr: 88100K .......... .......... .......... .......... .......... 94% 64.9M 0s 2024-05-30T16:23:29.689 INFO:teuthology.orchestra.run.smithi006.stderr: 88150K .......... .......... .......... .......... .......... 94% 36.5M 0s 2024-05-30T16:23:29.689 INFO:teuthology.orchestra.run.smithi006.stderr: 88200K .......... .......... .......... .......... .......... 94% 80.7M 0s 2024-05-30T16:23:29.689 INFO:teuthology.orchestra.run.smithi006.stderr: 88250K .......... .......... .......... .......... .......... 95% 97.7M 0s 2024-05-30T16:23:29.689 INFO:teuthology.orchestra.run.smithi006.stderr: 88300K .......... .......... .......... .......... .......... 95% 53.3M 0s 2024-05-30T16:23:29.689 INFO:teuthology.orchestra.run.smithi006.stderr: 88350K .......... .......... .......... .......... .......... 95% 75.2M 0s 2024-05-30T16:23:29.689 INFO:teuthology.orchestra.run.smithi006.stderr: 88400K .......... .......... .......... .......... .......... 95% 62.0M 0s 2024-05-30T16:23:29.689 INFO:teuthology.orchestra.run.smithi006.stderr: 88450K .......... .......... .......... .......... .......... 95% 82.9M 0s 2024-05-30T16:23:29.689 INFO:teuthology.orchestra.run.smithi006.stderr: 88500K .......... .......... .......... .......... .......... 95% 100M 0s 2024-05-30T16:23:29.689 INFO:teuthology.orchestra.run.smithi006.stderr: 88550K .......... .......... .......... .......... .......... 95% 87.7M 0s 2024-05-30T16:23:29.689 INFO:teuthology.orchestra.run.smithi006.stderr: 88600K .......... .......... .......... .......... .......... 95% 97.6M 0s 2024-05-30T16:23:29.689 INFO:teuthology.orchestra.run.smithi006.stderr: 88650K .......... .......... .......... .......... .......... 95% 95.3M 0s 2024-05-30T16:23:29.689 INFO:teuthology.orchestra.run.smithi006.stderr: 88700K .......... .......... .......... .......... .......... 95% 91.9M 0s 2024-05-30T16:23:29.689 INFO:teuthology.orchestra.run.smithi006.stderr: 88750K .......... .......... .......... .......... .......... 95% 101M 0s 2024-05-30T16:23:29.689 INFO:teuthology.orchestra.run.smithi006.stderr: 88800K .......... .......... .......... .......... .......... 95% 119M 0s 2024-05-30T16:23:29.690 INFO:teuthology.orchestra.run.smithi006.stderr: 88850K .......... .......... .......... .......... .......... 95% 91.3M 0s 2024-05-30T16:23:29.690 INFO:teuthology.orchestra.run.smithi006.stderr: 88900K .......... .......... .......... .......... .......... 95% 112M 0s 2024-05-30T16:23:29.690 INFO:teuthology.orchestra.run.smithi006.stderr: 88950K .......... .......... .......... .......... .......... 95% 120M 0s 2024-05-30T16:23:29.690 INFO:teuthology.orchestra.run.smithi006.stderr: 89000K .......... .......... .......... .......... .......... 95% 113M 0s 2024-05-30T16:23:29.690 INFO:teuthology.orchestra.run.smithi006.stderr: 89050K .......... .......... .......... .......... .......... 95% 107M 0s 2024-05-30T16:23:29.690 INFO:teuthology.orchestra.run.smithi006.stderr: 89100K .......... .......... .......... .......... .......... 95% 67.9M 0s 2024-05-30T16:23:29.690 INFO:teuthology.orchestra.run.smithi006.stderr: 89150K .......... .......... .......... .......... .......... 96% 41.4M 0s 2024-05-30T16:23:29.690 INFO:teuthology.orchestra.run.smithi006.stderr: 89200K .......... .......... .......... .......... .......... 96% 92.4M 0s 2024-05-30T16:23:29.690 INFO:teuthology.orchestra.run.smithi006.stderr: 89250K .......... .......... .......... .......... .......... 96% 78.6M 0s 2024-05-30T16:23:29.690 INFO:teuthology.orchestra.run.smithi006.stderr: 89300K .......... .......... .......... .......... .......... 96% 83.8M 0s 2024-05-30T16:23:29.690 INFO:teuthology.orchestra.run.smithi006.stderr: 89350K .......... .......... .......... .......... .......... 96% 69.6M 0s 2024-05-30T16:23:29.690 INFO:teuthology.orchestra.run.smithi006.stderr: 89400K .......... .......... .......... .......... .......... 96% 116M 0s 2024-05-30T16:23:29.690 INFO:teuthology.orchestra.run.smithi006.stderr: 89450K .......... .......... .......... .......... .......... 96% 119M 0s 2024-05-30T16:23:29.691 INFO:teuthology.orchestra.run.smithi006.stderr: 89500K .......... .......... .......... .......... .......... 96% 90.2M 0s 2024-05-30T16:23:29.691 INFO:teuthology.orchestra.run.smithi006.stderr: 89550K .......... .......... .......... .......... .......... 96% 113M 0s 2024-05-30T16:23:29.691 INFO:teuthology.orchestra.run.smithi006.stderr: 89600K .......... .......... .......... .......... .......... 96% 115M 0s 2024-05-30T16:23:29.691 INFO:teuthology.orchestra.run.smithi006.stderr: 89650K .......... .......... .......... .......... .......... 96% 109M 0s 2024-05-30T16:23:29.691 INFO:teuthology.orchestra.run.smithi006.stderr: 89700K .......... .......... .......... .......... .......... 96% 96.0M 0s 2024-05-30T16:23:29.691 INFO:teuthology.orchestra.run.smithi006.stderr: 89750K .......... .......... .......... .......... .......... 96% 111M 0s 2024-05-30T16:23:29.691 INFO:teuthology.orchestra.run.smithi006.stderr: 89800K .......... .......... .......... .......... .......... 96% 140M 0s 2024-05-30T16:23:29.691 INFO:teuthology.orchestra.run.smithi006.stderr: 89850K .......... .......... .......... .......... .......... 96% 115M 0s 2024-05-30T16:23:29.691 INFO:teuthology.orchestra.run.smithi006.stderr: 89900K .......... .......... .......... .......... .......... 96% 65.3M 0s 2024-05-30T16:23:29.691 INFO:teuthology.orchestra.run.smithi006.stderr: 89950K .......... .......... .......... .......... .......... 96% 89.3M 0s 2024-05-30T16:23:29.691 INFO:teuthology.orchestra.run.smithi006.stderr: 90000K .......... .......... .......... .......... .......... 96% 89.0M 0s 2024-05-30T16:23:29.691 INFO:teuthology.orchestra.run.smithi006.stderr: 90050K .......... .......... .......... .......... .......... 96% 130M 0s 2024-05-30T16:23:29.691 INFO:teuthology.orchestra.run.smithi006.stderr: 90100K .......... .......... .......... .......... .......... 97% 123M 0s 2024-05-30T16:23:29.691 INFO:teuthology.orchestra.run.smithi006.stderr: 90150K .......... .......... .......... .......... .......... 97% 119M 0s 2024-05-30T16:23:29.691 INFO:teuthology.orchestra.run.smithi006.stderr: 90200K .......... .......... .......... .......... .......... 97% 82.8M 0s 2024-05-30T16:23:29.692 INFO:teuthology.orchestra.run.smithi006.stderr: 90250K .......... .......... .......... .......... .......... 97% 68.4M 0s 2024-05-30T16:23:29.692 INFO:teuthology.orchestra.run.smithi006.stderr: 90300K .......... .......... .......... .......... .......... 97% 54.7M 0s 2024-05-30T16:23:29.692 INFO:teuthology.orchestra.run.smithi006.stderr: 90350K .......... .......... .......... .......... .......... 97% 114M 0s 2024-05-30T16:23:29.692 INFO:teuthology.orchestra.run.smithi006.stderr: 90400K .......... .......... .......... .......... .......... 97% 96.3M 0s 2024-05-30T16:23:29.692 INFO:teuthology.orchestra.run.smithi006.stderr: 90450K .......... .......... .......... .......... .......... 97% 84.4M 0s 2024-05-30T16:23:29.692 INFO:teuthology.orchestra.run.smithi006.stderr: 90500K .......... .......... .......... .......... .......... 97% 39.0M 0s 2024-05-30T16:23:29.692 INFO:teuthology.orchestra.run.smithi006.stderr: 90550K .......... .......... .......... .......... .......... 97% 94.0M 0s 2024-05-30T16:23:29.692 INFO:teuthology.orchestra.run.smithi006.stderr: 90600K .......... .......... .......... .......... .......... 97% 73.5M 0s 2024-05-30T16:23:29.692 INFO:teuthology.orchestra.run.smithi006.stderr: 90650K .......... .......... .......... .......... .......... 97% 78.1M 0s 2024-05-30T16:23:29.692 INFO:teuthology.orchestra.run.smithi006.stderr: 90700K .......... .......... .......... .......... .......... 97% 71.4M 0s 2024-05-30T16:23:29.692 INFO:teuthology.orchestra.run.smithi006.stderr: 90750K .......... .......... .......... .......... .......... 97% 100M 0s 2024-05-30T16:23:29.692 INFO:teuthology.orchestra.run.smithi006.stderr: 90800K .......... .......... .......... .......... .......... 97% 45.4M 0s 2024-05-30T16:23:29.692 INFO:teuthology.orchestra.run.smithi006.stderr: 90850K .......... .......... .......... .......... .......... 97% 119M 0s 2024-05-30T16:23:29.692 INFO:teuthology.orchestra.run.smithi006.stderr: 90900K .......... .......... .......... .......... .......... 97% 59.3M 0s 2024-05-30T16:23:29.692 INFO:teuthology.orchestra.run.smithi006.stderr: 90950K .......... .......... .......... .......... .......... 97% 60.9M 0s 2024-05-30T16:23:29.693 INFO:teuthology.orchestra.run.smithi006.stderr: 91000K .......... .......... .......... .......... .......... 98% 53.5M 0s 2024-05-30T16:23:29.693 INFO:teuthology.orchestra.run.smithi006.stderr: 91050K .......... .......... .......... .......... .......... 98% 112M 0s 2024-05-30T16:23:29.693 INFO:teuthology.orchestra.run.smithi006.stderr: 91100K .......... .......... .......... .......... .......... 98% 124M 0s 2024-05-30T16:23:29.693 INFO:teuthology.orchestra.run.smithi006.stderr: 91150K .......... .......... .......... .......... .......... 98% 85.2M 0s 2024-05-30T16:23:29.693 INFO:teuthology.orchestra.run.smithi006.stderr: 91200K .......... .......... .......... .......... .......... 98% 86.1M 0s 2024-05-30T16:23:29.693 INFO:teuthology.orchestra.run.smithi006.stderr: 91250K .......... .......... .......... .......... .......... 98% 98.3M 0s 2024-05-30T16:23:29.693 INFO:teuthology.orchestra.run.smithi006.stderr: 91300K .......... .......... .......... .......... .......... 98% 119M 0s 2024-05-30T16:23:29.693 INFO:teuthology.orchestra.run.smithi006.stderr: 91350K .......... .......... .......... .......... .......... 98% 101M 0s 2024-05-30T16:23:29.693 INFO:teuthology.orchestra.run.smithi006.stderr: 91400K .......... .......... .......... .......... .......... 98% 92.3M 0s 2024-05-30T16:23:29.693 INFO:teuthology.orchestra.run.smithi006.stderr: 91450K .......... .......... .......... .......... .......... 98% 88.1M 0s 2024-05-30T16:23:29.693 INFO:teuthology.orchestra.run.smithi006.stderr: 91500K .......... .......... .......... .......... .......... 98% 73.0M 0s 2024-05-30T16:23:29.693 INFO:teuthology.orchestra.run.smithi006.stderr: 91550K .......... .......... .......... .......... .......... 98% 117M 0s 2024-05-30T16:23:29.693 INFO:teuthology.orchestra.run.smithi006.stderr: 91600K .......... .......... .......... .......... .......... 98% 94.5M 0s 2024-05-30T16:23:29.694 INFO:teuthology.orchestra.run.smithi006.stderr: 91650K .......... .......... .......... .......... .......... 98% 95.8M 0s 2024-05-30T16:23:29.694 INFO:teuthology.orchestra.run.smithi006.stderr: 91700K .......... .......... .......... .......... .......... 98% 78.3M 0s 2024-05-30T16:23:29.694 INFO:teuthology.orchestra.run.smithi006.stderr: 91750K .......... .......... .......... .......... .......... 98% 114M 0s 2024-05-30T16:23:29.694 INFO:teuthology.orchestra.run.smithi006.stderr: 91800K .......... .......... .......... .......... .......... 98% 90.7M 0s 2024-05-30T16:23:29.694 INFO:teuthology.orchestra.run.smithi006.stderr: 91850K .......... .......... .......... .......... .......... 98% 113M 0s 2024-05-30T16:23:29.694 INFO:teuthology.orchestra.run.smithi006.stderr: 91900K .......... .......... .......... .......... .......... 98% 107M 0s 2024-05-30T16:23:29.694 INFO:teuthology.orchestra.run.smithi006.stderr: 91950K .......... .......... .......... .......... .......... 99% 87.9M 0s 2024-05-30T16:23:29.694 INFO:teuthology.orchestra.run.smithi006.stderr: 92000K .......... .......... .......... .......... .......... 99% 107M 0s 2024-05-30T16:23:29.694 INFO:teuthology.orchestra.run.smithi006.stderr: 92050K .......... .......... .......... .......... .......... 99% 125M 0s 2024-05-30T16:23:29.694 INFO:teuthology.orchestra.run.smithi006.stderr: 92100K .......... .......... .......... .......... .......... 99% 97.1M 0s 2024-05-30T16:23:29.694 INFO:teuthology.orchestra.run.smithi006.stderr: 92150K .......... .......... .......... .......... .......... 99% 53.1M 0s 2024-05-30T16:23:29.694 INFO:teuthology.orchestra.run.smithi006.stderr: 92200K .......... .......... .......... .......... .......... 99% 92.0M 0s 2024-05-30T16:23:29.694 INFO:teuthology.orchestra.run.smithi006.stderr: 92250K .......... .......... .......... .......... .......... 99% 104M 0s 2024-05-30T16:23:29.695 INFO:teuthology.orchestra.run.smithi006.stderr: 92300K .......... .......... .......... .......... .......... 99% 136M 0s 2024-05-30T16:23:29.695 INFO:teuthology.orchestra.run.smithi006.stderr: 92350K .......... .......... .......... .......... .......... 99% 118M 0s 2024-05-30T16:23:29.695 INFO:teuthology.orchestra.run.smithi006.stderr: 92400K .......... .......... .......... .......... .......... 99% 102M 0s 2024-05-30T16:23:29.695 INFO:teuthology.orchestra.run.smithi006.stderr: 92450K .......... .......... .......... .......... .......... 99% 117M 0s 2024-05-30T16:23:29.695 INFO:teuthology.orchestra.run.smithi006.stderr: 92500K .......... .......... .......... .......... .......... 99% 50.8M 0s 2024-05-30T16:23:29.695 INFO:teuthology.orchestra.run.smithi006.stderr: 92550K .......... .......... .......... .......... .......... 99% 69.2M 0s 2024-05-30T16:23:29.695 INFO:teuthology.orchestra.run.smithi006.stderr: 92600K .......... .......... .......... .......... .......... 99% 76.7M 0s 2024-05-30T16:23:29.695 INFO:teuthology.orchestra.run.smithi006.stderr: 92650K .......... .......... .......... .......... .......... 99% 101M 0s 2024-05-30T16:23:29.695 INFO:teuthology.orchestra.run.smithi006.stderr: 92700K .......... .......... .......... .......... .......... 99% 109M 0s 2024-05-30T16:23:29.695 INFO:teuthology.orchestra.run.smithi006.stderr: 92750K .......... .......... .......... .......... .......... 99% 108M 0s 2024-05-30T16:23:29.695 INFO:teuthology.orchestra.run.smithi006.stderr: 92800K .......... .......... .......... .......... .......... 99% 85.8M 0s 2024-05-30T16:23:29.695 INFO:teuthology.orchestra.run.smithi006.stderr: 92850K .......... .......... .......... .......... .......... 99% 40.3M 0s 2024-05-30T16:23:29.695 INFO:teuthology.orchestra.run.smithi006.stderr: 92900K 100% 134G=1.1s 2024-05-30T16:23:29.695 INFO:teuthology.orchestra.run.smithi006.stderr: 2024-05-30T16:23:29.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30 16:23:29 (80.6 MB/s) - 'gradle-6.0.1-bin.zip' saved [95129672/95129672] 2024-05-30T16:23:29.696 INFO:teuthology.orchestra.run.smithi006.stderr: 2024-05-30T16:23:29.703 INFO:teuthology.orchestra.run.smithi006.stdout:Archive: gradle-6.0.1-bin.zip 2024-05-30T16:23:29.703 INFO:teuthology.orchestra.run.smithi006.stdout: creating: /opt/gradle/gradle-6.0.1/ 2024-05-30T16:23:29.703 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/getting-started.html 2024-05-30T16:23:29.703 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/LICENSE 2024-05-30T16:23:29.703 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/NOTICE 2024-05-30T16:23:29.703 INFO:teuthology.orchestra.run.smithi006.stdout: creating: /opt/gradle/gradle-6.0.1/init.d/ 2024-05-30T16:23:29.703 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/init.d/readme.txt 2024-05-30T16:23:29.703 INFO:teuthology.orchestra.run.smithi006.stdout: creating: /opt/gradle/gradle-6.0.1/bin/ 2024-05-30T16:23:29.703 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/bin/gradle 2024-05-30T16:23:29.703 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/bin/gradle.bat 2024-05-30T16:23:29.703 INFO:teuthology.orchestra.run.smithi006.stdout: creating: /opt/gradle/gradle-6.0.1/lib/ 2024-05-30T16:23:29.703 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-api-metadata-6.0.1.jar 2024-05-30T16:23:29.704 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-launcher-6.0.1.jar 2024-05-30T16:23:29.704 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-runtime-api-info-6.0.1.jar 2024-05-30T16:23:29.704 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-kotlin-dsl-6.0.1.jar 2024-05-30T16:23:29.704 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-tooling-api-6.0.1.jar 2024-05-30T16:23:29.704 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-wrapper-6.0.1.jar 2024-05-30T16:23:29.704 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-installation-beacon-6.0.1.jar 2024-05-30T16:23:29.704 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-bootstrap-6.0.1.jar 2024-05-30T16:23:29.704 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-core-6.0.1.jar 2024-05-30T16:23:29.704 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-file-collections-6.0.1.jar 2024-05-30T16:23:29.704 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-model-groovy-6.0.1.jar 2024-05-30T16:23:29.704 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-execution-6.0.1.jar 2024-05-30T16:23:29.704 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-model-core-6.0.1.jar 2024-05-30T16:23:29.704 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-build-cache-packaging-6.0.1.jar 2024-05-30T16:23:29.704 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-build-cache-6.0.1.jar 2024-05-30T16:23:29.705 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-core-api-6.0.1.jar 2024-05-30T16:23:29.705 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-persistent-cache-6.0.1.jar 2024-05-30T16:23:29.705 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-worker-processes-6.0.1.jar 2024-05-30T16:23:29.705 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-logging-6.0.1.jar 2024-05-30T16:23:29.705 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-jvm-services-6.0.1.jar 2024-05-30T16:23:29.705 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-process-services-6.0.1.jar 2024-05-30T16:23:29.705 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-resources-6.0.1.jar 2024-05-30T16:23:29.705 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-messaging-6.0.1.jar 2024-05-30T16:23:29.705 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-native-6.0.1.jar 2024-05-30T16:23:29.705 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-base-services-groovy-6.0.1.jar 2024-05-30T16:23:29.705 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-base-services-6.0.1.jar 2024-05-30T16:23:29.705 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-build-option-6.0.1.jar 2024-05-30T16:23:29.705 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-cli-6.0.1.jar 2024-05-30T16:23:29.706 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-snapshots-6.0.1.jar 2024-05-30T16:23:29.706 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-files-6.0.1.jar 2024-05-30T16:23:29.706 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-kotlin-dsl-tooling-models-6.0.1.jar 2024-05-30T16:23:29.768 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-compiler-embeddable-1.3.50-patched-for-gradle-6.0.1.jar 2024-05-30T16:23:29.769 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-hashing-6.0.1.jar 2024-05-30T16:23:29.769 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-pineapple-6.0.1.jar 2024-05-30T16:23:29.769 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-docs-6.0.1.jar 2024-05-30T16:23:29.825 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/groovy-all-1.3-2.5.8.jar 2024-05-30T16:23:29.825 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/jul-to-slf4j-1.7.28.jar 2024-05-30T16:23:29.825 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/log4j-over-slf4j-1.7.28.jar 2024-05-30T16:23:29.825 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/jcl-over-slf4j-1.7.28.jar 2024-05-30T16:23:29.825 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/slf4j-api-1.7.28.jar 2024-05-30T16:23:29.842 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/guava-27.1-android.jar 2024-05-30T16:23:29.844 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/commons-io-2.6.jar 2024-05-30T16:23:29.846 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/commons-lang-2.6.jar 2024-05-30T16:23:29.846 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/asm-commons-7.1.jar 2024-05-30T16:23:29.847 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/asm-analysis-7.1.jar 2024-05-30T16:23:29.847 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/asm-tree-7.1.jar 2024-05-30T16:23:29.848 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/asm-7.1.jar 2024-05-30T16:23:29.863 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/ant-1.10.7.jar 2024-05-30T16:23:29.863 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-stdlib-jdk8-1.3.50.jar 2024-05-30T16:23:29.863 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/javax.inject-1.jar 2024-05-30T16:23:29.864 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-common-1.3.50.jar 2024-05-30T16:23:29.865 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-jvm-1.3.50.jar 2024-05-30T16:23:29.866 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-jvm-host-embeddable-1.3.50.jar 2024-05-30T16:23:29.868 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-compiler-embeddable-1.3.50.jar 2024-05-30T16:23:29.870 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-compiler-impl-embeddable-1.3.50.jar 2024-05-30T16:23:29.871 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-sam-with-receiver-compiler-plugin-1.3.50.jar 2024-05-30T16:23:29.886 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlinx-metadata-jvm-0.1.0.jar 2024-05-30T16:23:29.890 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/commons-compress-1.19.jar 2024-05-30T16:23:29.893 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/fastutil-8.3.0-min.jar 2024-05-30T16:23:29.896 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/jansi-1.18.jar 2024-05-30T16:23:29.896 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/jsr305-3.0.2.jar 2024-05-30T16:23:29.898 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kryo-2.24.0.jar 2024-05-30T16:23:29.899 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-0.18.jar 2024-05-30T16:23:29.899 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/objenesis-2.6.jar 2024-05-30T16:23:29.903 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/trove4j-1.0.20181211.jar 2024-05-30T16:23:29.905 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/xml-apis-1.4.01.jar 2024-05-30T16:23:29.924 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-reflect-1.3.50.jar 2024-05-30T16:23:29.924 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-stdlib-jdk7-1.3.50.jar 2024-05-30T16:23:29.933 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-stdlib-1.3.50.jar 2024-05-30T16:23:29.934 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/annotations-13.0.jar 2024-05-30T16:23:29.934 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/ant-launcher-1.10.7.jar 2024-05-30T16:23:29.934 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/minlog-1.2.jar 2024-05-30T16:23:29.934 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-script-runtime-1.3.50.jar 2024-05-30T16:23:29.941 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-daemon-embeddable-1.3.50.jar 2024-05-30T16:23:29.942 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/failureaccess-1.0.1.jar 2024-05-30T16:23:29.944 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-stdlib-common-1.3.50.jar 2024-05-30T16:23:29.944 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-freebsd-amd64-libcpp-0.18.jar 2024-05-30T16:23:29.944 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-freebsd-amd64-libstdcpp-0.18.jar 2024-05-30T16:23:29.944 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-freebsd-i386-libcpp-0.18.jar 2024-05-30T16:23:29.944 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-freebsd-i386-libstdcpp-0.18.jar 2024-05-30T16:23:29.944 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-aarch64-0.18.jar 2024-05-30T16:23:29.944 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-amd64-0.18.jar 2024-05-30T16:23:29.944 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-i386-0.18.jar 2024-05-30T16:23:29.945 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-osx-amd64-0.18.jar 2024-05-30T16:23:29.946 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-windows-amd64-0.18.jar 2024-05-30T16:23:29.947 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-windows-amd64-min-0.18.jar 2024-05-30T16:23:29.947 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-windows-i386-0.18.jar 2024-05-30T16:23:29.947 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-windows-i386-min-0.18.jar 2024-05-30T16:23:29.947 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-aarch64-ncurses5-0.18.jar 2024-05-30T16:23:29.947 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-amd64-ncurses5-0.18.jar 2024-05-30T16:23:29.947 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-amd64-ncurses6-0.18.jar 2024-05-30T16:23:29.947 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-i386-ncurses5-0.18.jar 2024-05-30T16:23:29.947 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-i386-ncurses6-0.18.jar 2024-05-30T16:23:29.947 INFO:teuthology.orchestra.run.smithi006.stdout: creating: /opt/gradle/gradle-6.0.1/lib/plugins/ 2024-05-30T16:23:29.947 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-antlr-6.0.1.jar 2024-05-30T16:23:29.948 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-code-quality-6.0.1.jar 2024-05-30T16:23:29.948 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-signing-6.0.1.jar 2024-05-30T16:23:29.948 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-tooling-native-6.0.1.jar 2024-05-30T16:23:29.950 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ide-native-6.0.1.jar 2024-05-30T16:23:29.950 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-testing-native-6.0.1.jar 2024-05-30T16:23:29.953 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-native-6.0.1.jar 2024-05-30T16:23:29.954 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-kotlin-dsl-provider-plugins-6.0.1.jar 2024-05-30T16:23:29.955 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-plugin-development-6.0.1.jar 2024-05-30T16:23:29.957 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-maven-6.0.1.jar 2024-05-30T16:23:29.957 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ide-play-6.0.1.jar 2024-05-30T16:23:29.960 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ide-6.0.1.jar 2024-05-30T16:23:29.960 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-scala-6.0.1.jar 2024-05-30T16:23:29.960 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ear-6.0.1.jar 2024-05-30T16:23:29.962 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-platform-play-6.0.1.jar 2024-05-30T16:23:29.962 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-javascript-6.0.1.jar 2024-05-30T16:23:29.963 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ivy-6.0.1.jar 2024-05-30T16:23:29.964 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-jacoco-6.0.1.jar 2024-05-30T16:23:29.965 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-build-init-6.0.1.jar 2024-05-30T16:23:29.966 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-kotlin-dsl-tooling-builders-6.0.1.jar 2024-05-30T16:23:29.968 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-plugins-6.0.1.jar 2024-05-30T16:23:29.969 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-test-kit-6.0.1.jar 2024-05-30T16:23:29.969 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-groovy-6.0.1.jar 2024-05-30T16:23:29.970 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-scala-6.0.1.jar 2024-05-30T16:23:29.970 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-testing-junit-platform-6.0.1.jar 2024-05-30T16:23:29.970 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-tooling-api-builders-6.0.1.jar 2024-05-30T16:23:29.971 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-testing-jvm-6.0.1.jar 2024-05-30T16:23:29.974 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-java-6.0.1.jar 2024-05-30T16:23:29.974 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-jvm-6.0.1.jar 2024-05-30T16:23:29.975 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-platform-jvm-6.0.1.jar 2024-05-30T16:23:29.978 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-platform-native-6.0.1.jar 2024-05-30T16:23:29.980 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-diagnostics-6.0.1.jar 2024-05-30T16:23:29.981 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-publish-6.0.1.jar 2024-05-30T16:23:29.982 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-testing-base-6.0.1.jar 2024-05-30T16:23:29.984 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-platform-base-6.0.1.jar 2024-05-30T16:23:29.984 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-plugin-use-6.0.1.jar 2024-05-30T16:23:29.990 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-instant-execution-6.0.1.jar 2024-05-30T16:23:29.991 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-version-control-6.0.1.jar 2024-05-30T16:23:30.005 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-dependency-management-6.0.1.jar 2024-05-30T16:23:30.006 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-composite-builds-6.0.1.jar 2024-05-30T16:23:30.007 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-reporting-6.0.1.jar 2024-05-30T16:23:30.007 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-resources-s3-6.0.1.jar 2024-05-30T16:23:30.007 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-resources-gcs-6.0.1.jar 2024-05-30T16:23:30.007 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-build-cache-http-6.0.1.jar 2024-05-30T16:23:30.008 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-resources-http-6.0.1.jar 2024-05-30T16:23:30.008 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-resources-sftp-6.0.1.jar 2024-05-30T16:23:30.008 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-build-profile-6.0.1.jar 2024-05-30T16:23:30.009 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-workers-6.0.1.jar 2024-05-30T16:23:30.028 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/org.eclipse.jgit-5.5.0.201909110433-r.jar 2024-05-30T16:23:30.028 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/wagon-http-3.0.0.jar 2024-05-30T16:23:30.028 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/wagon-http-shared-3.0.0.jar 2024-05-30T16:23:30.029 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-api-services-storage-v1-rev136-1.25.0.jar 2024-05-30T16:23:30.030 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-api-client-1.25.0.jar 2024-05-30T16:23:30.031 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/asm-util-7.1.jar 2024-05-30T16:23:30.039 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/ivy-2.3.0.jar 2024-05-30T16:23:30.041 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-compat-3.0.4.jar 2024-05-30T16:23:30.045 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-core-3.0.4.jar 2024-05-30T16:23:30.045 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/pmaven-common-0.8-20100325.jar 2024-05-30T16:23:30.046 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/pmaven-groovy-0.8-20100325.jar 2024-05-30T16:23:30.046 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/wagon-file-3.0.0.jar 2024-05-30T16:23:30.047 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-container-default-1.7.1.jar 2024-05-30T16:23:30.047 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-connector-wagon-1.13.1.jar 2024-05-30T16:23:30.049 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/bcpg-jdk15on-1.63.jar 2024-05-30T16:23:30.058 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/rhino-1.7.10.jar 2024-05-30T16:23:30.060 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gson-2.8.5.jar 2024-05-30T16:23:30.061 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/simple-4.1.21.jar 2024-05-30T16:23:30.062 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jatl-0.2.3.jar 2024-05-30T16:23:30.062 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-plugin-api-3.0.4.jar 2024-05-30T16:23:30.062 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-oauth-client-1.25.0.jar 2024-05-30T16:23:30.063 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-http-client-jackson2-1.25.0.jar 2024-05-30T16:23:30.065 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-http-client-1.25.0.jar 2024-05-30T16:23:30.069 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/testng-6.3.1.jar 2024-05-30T16:23:30.071 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/snakeyaml-1.17.jar 2024-05-30T16:23:30.072 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/dd-plist-1.21.jar 2024-05-30T16:23:30.074 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/junit-4.12.jar 2024-05-30T16:23:30.077 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/bsh-2.0b6.jar 2024-05-30T16:23:30.077 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/junit-platform-launcher-1.3.1.jar 2024-05-30T16:23:30.084 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aws-java-sdk-s3-1.11.633.jar 2024-05-30T16:23:30.088 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aws-java-sdk-kms-1.11.633.jar 2024-05-30T16:23:30.094 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aws-java-sdk-core-1.11.633.jar 2024-05-30T16:23:30.100 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/httpclient-4.5.10.jar 2024-05-30T16:23:30.102 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jcifs-1.3.17.jar 2024-05-30T16:23:30.103 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/nekohtml-1.9.22.jar 2024-05-30T16:23:30.113 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/xercesImpl-2.12.0.jar 2024-05-30T16:23:30.114 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jsch-0.1.55.jar 2024-05-30T16:23:30.121 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jaxb-impl-2.3.2.jar 2024-05-30T16:23:30.122 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jmespath-java-1.11.633.jar 2024-05-30T16:23:30.131 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jackson-databind-2.9.10.jar 2024-05-30T16:23:30.133 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jackson-core-2.9.10.jar 2024-05-30T16:23:30.133 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jackson-annotations-2.9.10.jar 2024-05-30T16:23:30.138 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/joda-time-2.10.4.jar 2024-05-30T16:23:30.144 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/bcpkix-jdk15on-1.61.jar 2024-05-30T16:23:30.174 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/bcprov-jdk15on-1.63.jar 2024-05-30T16:23:30.177 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/commons-codec-1.13.jar 2024-05-30T16:23:30.177 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-aether-provider-3.0.4.jar 2024-05-30T16:23:30.178 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-impl-1.13.1.jar 2024-05-30T16:23:30.179 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-util-1.13.1.jar 2024-05-30T16:23:30.179 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-spi-1.13.1.jar 2024-05-30T16:23:30.180 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-api-1.13.1.jar 2024-05-30T16:23:30.180 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/hamcrest-core-1.3.jar 2024-05-30T16:23:30.183 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/httpcore-4.4.12.jar 2024-05-30T16:23:30.183 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jcommander-1.72.jar 2024-05-30T16:23:30.184 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jzlib-1.1.3.jar 2024-05-30T16:23:30.184 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-artifact-3.0.4.jar 2024-05-30T16:23:30.185 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-model-builder-3.0.4.jar 2024-05-30T16:23:30.187 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-model-3.0.4.jar 2024-05-30T16:23:30.187 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-repository-metadata-3.0.4.jar 2024-05-30T16:23:30.188 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-settings-builder-3.0.4.jar 2024-05-30T16:23:30.188 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-settings-3.0.4.jar 2024-05-30T16:23:30.188 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-sec-dispatcher-1.3.jar 2024-05-30T16:23:30.188 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-cipher-1.7.jar 2024-05-30T16:23:30.189 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-classworlds-2.5.1.jar 2024-05-30T16:23:30.189 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-component-annotations-1.5.5.jar 2024-05-30T16:23:30.189 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-interpolation-1.14.jar 2024-05-30T16:23:30.189 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/wagon-provider-api-3.0.0.jar 2024-05-30T16:23:30.191 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-utils-3.1.0.jar 2024-05-30T16:23:30.192 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/xbean-reflect-3.7.jar 2024-05-30T16:23:30.196 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/ion-java-1.0.2.jar 2024-05-30T16:23:30.197 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/junit-platform-engine-1.3.1.jar 2024-05-30T16:23:30.198 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/junit-platform-commons-1.3.1.jar 2024-05-30T16:23:30.198 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/apiguardian-api-1.0.0.jar 2024-05-30T16:23:30.198 INFO:teuthology.orchestra.run.smithi006.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/opentest4j-1.1.1.jar 2024-05-30T16:23:30.235 INFO:teuthology.orchestra.run.smithi006.stdout:export PATH=/opt/gradle/gradle-6.0.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin 2024-05-30T16:23:30.739 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:23:30.740 INFO:teuthology.orchestra.run.smithi006.stdout:Welcome to Gradle 6.0.1! 2024-05-30T16:23:30.740 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:23:30.740 INFO:teuthology.orchestra.run.smithi006.stdout:Here are the highlights of this release: 2024-05-30T16:23:30.740 INFO:teuthology.orchestra.run.smithi006.stdout: - Substantial improvements in dependency management, including 2024-05-30T16:23:30.740 INFO:teuthology.orchestra.run.smithi006.stdout: - Publishing Gradle Module Metadata in addition to pom.xml 2024-05-30T16:23:30.740 INFO:teuthology.orchestra.run.smithi006.stdout: - Advanced control of transitive versions 2024-05-30T16:23:30.740 INFO:teuthology.orchestra.run.smithi006.stdout: - Support for optional features and dependencies 2024-05-30T16:23:30.740 INFO:teuthology.orchestra.run.smithi006.stdout: - Rules to tweak published metadata 2024-05-30T16:23:30.740 INFO:teuthology.orchestra.run.smithi006.stdout: - Support for Java 13 2024-05-30T16:23:30.740 INFO:teuthology.orchestra.run.smithi006.stdout: - Faster incremental Java and Groovy compilation 2024-05-30T16:23:30.740 INFO:teuthology.orchestra.run.smithi006.stdout: - New Zinc compiler for Scala 2024-05-30T16:23:30.740 INFO:teuthology.orchestra.run.smithi006.stdout: - VS2019 support 2024-05-30T16:23:30.741 INFO:teuthology.orchestra.run.smithi006.stdout: - Support for Gradle Enterprise plugin 3.0 2024-05-30T16:23:30.743 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:23:30.743 INFO:teuthology.orchestra.run.smithi006.stdout:For more details see https://docs.gradle.org/6.0.1/release-notes.html 2024-05-30T16:23:30.743 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:23:30.743 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:23:30.744 INFO:teuthology.orchestra.run.smithi006.stdout:------------------------------------------------------------ 2024-05-30T16:23:30.744 INFO:teuthology.orchestra.run.smithi006.stdout:Gradle 6.0.1 2024-05-30T16:23:30.744 INFO:teuthology.orchestra.run.smithi006.stdout:------------------------------------------------------------ 2024-05-30T16:23:30.744 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:23:30.744 INFO:teuthology.orchestra.run.smithi006.stdout:Build time: 2019-11-18 20:25:01 UTC 2024-05-30T16:23:30.744 INFO:teuthology.orchestra.run.smithi006.stdout:Revision: fad121066a68c4701acd362daf4287a7c309a0f5 2024-05-30T16:23:30.744 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:23:30.744 INFO:teuthology.orchestra.run.smithi006.stdout:Kotlin: 1.3.50 2024-05-30T16:23:30.744 INFO:teuthology.orchestra.run.smithi006.stdout:Groovy: 2.5.8 2024-05-30T16:23:30.744 INFO:teuthology.orchestra.run.smithi006.stdout:Ant: Apache Ant(TM) version 1.10.7 compiled on September 1 2019 2024-05-30T16:23:30.744 INFO:teuthology.orchestra.run.smithi006.stdout:JVM: 11.0.17 (Ubuntu 11.0.17+8-post-Ubuntu-1ubuntu222.04) 2024-05-30T16:23:30.744 INFO:teuthology.orchestra.run.smithi006.stdout:OS: Linux 5.15.0-107-generic amd64 2024-05-30T16:23:30.745 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:23:30.750 DEBUG:tasks.s3tests_java:S3 Tests Java: BEGIN 2024-05-30T16:23:30.751 DEBUG:tasks.s3tests_java:S3 Tests Java: Cluster config is: ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-05-30T16:23:30.751 DEBUG:tasks.s3tests_java:S3 Tests Java: Host is: ubuntu@smithi006.front.sepia.ceph.com 2024-05-30T16:23:30.751 DEBUG:tasks.s3tests_java:S3 Tests Java: Cluster config is: ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-05-30T16:23:30.751 DEBUG:tasks.s3tests_java:S3 Tests Java: Host is: ubuntu@smithi204.front.sepia.ceph.com 2024-05-30T16:23:30.751 INFO:tasks.s3tests_java:S3 Tests Java: Creating S3 users... 2024-05-30T16:23:30.944 DEBUG:tasks.s3tests_java:S3 Tests Java: s3tests_conf is {'bucket_prefix': 'test-', 's3main': {'access_key': None, 'access_secret': None, 'region': 'mexico', 'endpoint': None, 'port': None, 'display_name': None, 'email': None, 'is_secure': None, 'SSE': 'AES256', 's3dir': 'folder', 'dir': '../data'}} 2024-05-30T16:23:30.944 DEBUG:tasks.s3tests_java:S3 Tests Java: Creating user tester.client.0 2024-05-30T16:23:30.945 DEBUG:tasks.s3tests_java:S3 Tests Java: s3tests_conf[s3main] is {'access_key': 'WMTCEIRCXEZTRMVCZBLU', 'access_secret': 'iNbbzXvZSrec+9bJRXtd001wDvdeX/bUUiQZw6yx9+ku4rbmQqemjg==', 'region': 'mexico', 'endpoint': 'smithi006.front.sepia.ceph.com:80', 'port': 80, 'display_name': 'Ms. tester.client.0', 'email': 'tester.client.0_test@test.test', 'is_secure': False, 'SSE': 'AES256', 's3dir': 'folder', 'dir': '../data', 'user_id': 'tester.client.0', 'host': 'smithi006.front.sepia.ceph.com'} 2024-05-30T16:23:30.945 DEBUG:tasks.s3tests_java:S3 Tests Java: Setion, User = s3main, tester.client.0 2024-05-30T16:23:30.945 INFO:tasks.s3tests_java:['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '-n', 'client.0', 'user', 'create', '--uid', 'tester.client.0', '--display-name', 'Ms. tester.client.0', '--access-key', 'WMTCEIRCXEZTRMVCZBLU', '--secret', 'iNbbzXvZSrec+9bJRXtd001wDvdeX/bUUiQZw6yx9+ku4rbmQqemjg==', '--email', 'tester.client.0_test@test.test', '--cluster', 'ceph'] 2024-05-30T16:23:30.945 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid tester.client.0 --display-name 'Ms. tester.client.0' --access-key WMTCEIRCXEZTRMVCZBLU --secret iNbbzXvZSrec+9bJRXtd001wDvdeX/bUUiQZw6yx9+ku4rbmQqemjg== --email tester.client.0_test@test.test --cluster ceph 2024-05-30T16:23:30.990 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:23:30.990 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:23:31.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.031+0000 7f6c7df4d980 20 rados->read ofs=0 len=0 2024-05-30T16:23:31.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.035+0000 7f6c7df4d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:31.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.035+0000 7f6c7df4d980 20 realm 2024-05-30T16:23:31.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.035+0000 7f6c7df4d980 20 rados->read ofs=0 len=0 2024-05-30T16:23:31.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.035+0000 7f6c7df4d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:31.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.035+0000 7f6c7df4d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:23:31.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.035+0000 7f6c7df4d980 20 rados->read ofs=0 len=0 2024-05-30T16:23:31.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.035+0000 7f6c7df4d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:31.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.035+0000 7f6c7df4d980 20 rados->read ofs=0 len=0 2024-05-30T16:23:31.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.035+0000 7f6c7df4d980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:23:31.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.035+0000 7f6c7df4d980 20 rados->read ofs=0 len=0 2024-05-30T16:23:31.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.035+0000 7f6c7df4d980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:23:31.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.035+0000 7f6c7df4d980 20 searching for the correct realm 2024-05-30T16:23:31.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.043+0000 7f6c7b25d640 0 -- 172.21.15.6:0/2296195781 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f6c74004020 msgr2=0x7f6c74085a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:23:31.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.051+0000 7f6c7b25d640 0 -- 172.21.15.6:0/2296195781 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f6b4407e510 msgr2=0x7f6b44080940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:23:31.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.055+0000 7f6c7df4d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:23:31.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.055+0000 7f6c7df4d980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:23:31.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.055+0000 7f6c7df4d980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:23:31.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.055+0000 7f6c7df4d980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:23:31.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.055+0000 7f6c7df4d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:23:31.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.055+0000 7f6c7df4d980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:23:31.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.055+0000 7f6c7df4d980 20 rados->read ofs=0 len=0 2024-05-30T16:23:31.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.055+0000 7f6c7df4d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:31.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.055+0000 7f6c7df4d980 20 rados->read ofs=0 len=0 2024-05-30T16:23:31.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.059+0000 7f6c7df4d980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:23:31.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.059+0000 7f6c7df4d980 20 rados->read ofs=0 len=0 2024-05-30T16:23:31.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.059+0000 7f6c7df4d980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:23:31.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.059+0000 7f6c7df4d980 20 zone default found 2024-05-30T16:23:31.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.059+0000 7f6c7df4d980 4 Realm: () 2024-05-30T16:23:31.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.059+0000 7f6c7df4d980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:23:31.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.059+0000 7f6c7df4d980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:23:31.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.059+0000 7f6c7df4d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:23:31.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.059+0000 7f6c7df4d980 20 zonegroup default 2024-05-30T16:23:31.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.059+0000 7f6c7df4d980 20 rados->read ofs=0 len=0 2024-05-30T16:23:31.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.059+0000 7f6c7df4d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:31.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.059+0000 7f6c7df4d980 20 rados->read ofs=0 len=0 2024-05-30T16:23:31.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.059+0000 7f6c7df4d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:31.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.059+0000 7f6c7df4d980 20 rados->read ofs=0 len=0 2024-05-30T16:23:31.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.059+0000 7f6c7df4d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:31.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.059+0000 7f6c7df4d980 20 started sync module instance, tier type = 2024-05-30T16:23:31.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.059+0000 7f6c7df4d980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:23:31.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.075+0000 7f6c7df4d980 20 add_watcher() i=0 2024-05-30T16:23:31.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.075+0000 7f6c7df4d980 20 add_watcher() i=1 2024-05-30T16:23:31.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.075+0000 7f6c7df4d980 20 add_watcher() i=2 2024-05-30T16:23:31.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.075+0000 7f6c7df4d980 20 add_watcher() i=3 2024-05-30T16:23:31.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.075+0000 7f6c7df4d980 20 add_watcher() i=4 2024-05-30T16:23:31.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.075+0000 7f6c7df4d980 20 add_watcher() i=5 2024-05-30T16:23:31.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.075+0000 7f6c7df4d980 20 add_watcher() i=6 2024-05-30T16:23:31.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.079+0000 7f6c7df4d980 20 add_watcher() i=7 2024-05-30T16:23:31.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.079+0000 7f6c7df4d980 2 all 8 watchers are set, enabling cache 2024-05-30T16:23:31.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.079+0000 7f6b31fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:23:31.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.079+0000 7f6c7df4d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:23:31.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.079+0000 7f6c7df4d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:23:31.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.079+0000 7f6c7df4d980 5 note: GC not initialized 2024-05-30T16:23:31.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.079+0000 7f6b30fe1640 20 reqs_thread_entry: start 2024-05-30T16:23:31.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.079+0000 7f6c7df4d980 20 init_complete bucket index max shards: 11 2024-05-30T16:23:31.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.079+0000 7f6c7df4d980 20 Filter name: none 2024-05-30T16:23:31.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.079+0000 7f6aaa7fc640 20 reqs_thread_entry: start 2024-05-30T16:23:31.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.079+0000 7f6c7df4d980 10 cache get: name=default.rgw.meta+users.uid+tester.client.0 : miss 2024-05-30T16:23:31.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.079+0000 7f6c7df4d980 20 rados->read ofs=0 len=0 2024-05-30T16:23:31.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.083+0000 7f6c7df4d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:31.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.083+0000 7f6c7df4d980 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x0 2024-05-30T16:23:31.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.083+0000 7f6c7df4d980 10 adding default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2024-05-30T16:23:31.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.083+0000 7f6c7df4d980 10 cache get: name=default.rgw.meta+users.email+tester.client.0_test@test.test : miss 2024-05-30T16:23:31.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.083+0000 7f6c7df4d980 20 rados->read ofs=0 len=0 2024-05-30T16:23:31.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.083+0000 7f6c7df4d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:31.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.083+0000 7f6c7df4d980 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x0 2024-05-30T16:23:31.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.083+0000 7f6c7df4d980 10 adding default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2024-05-30T16:23:31.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.083+0000 7f6c7df4d980 10 cache get: name=default.rgw.meta+users.keys+WMTCEIRCXEZTRMVCZBLU : miss 2024-05-30T16:23:31.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.083+0000 7f6c7df4d980 20 rados->read ofs=0 len=0 2024-05-30T16:23:31.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.083+0000 7f6c7df4d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:31.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.083+0000 7f6c7df4d980 10 cache put: name=default.rgw.meta+users.keys+WMTCEIRCXEZTRMVCZBLU info.flags=0x0 2024-05-30T16:23:31.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.083+0000 7f6c7df4d980 10 adding default.rgw.meta+users.keys+WMTCEIRCXEZTRMVCZBLU to cache LRU end 2024-05-30T16:23:31.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.083+0000 7f6c7df4d980 10 cache get: name=default.rgw.meta+users.keys+WMTCEIRCXEZTRMVCZBLU : hit (negative entry) 2024-05-30T16:23:31.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.083+0000 7f6c7df4d980 10 cache get: name=default.rgw.meta+users.keys+WMTCEIRCXEZTRMVCZBLU : hit (negative entry) 2024-05-30T16:23:31.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.087+0000 7f6c7df4d980 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x17 2024-05-30T16:23:31.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.087+0000 7f6c7df4d980 10 moving default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2024-05-30T16:23:31.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.087+0000 7f6c7df4d980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:users.uid:tester.client.0, ofs0, ns] 2024-05-30T16:23:31.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.087+0000 7f6b5affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431643 cookie 94840371196400 notifier 7371 bl.length()=670 2024-05-30T16:23:31.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.087+0000 7f6b5affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x17 2024-05-30T16:23:31.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.087+0000 7f6b5affd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2024-05-30T16:23:31.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.091+0000 7f6c7df4d980 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x7 2024-05-30T16:23:31.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.091+0000 7f6c7df4d980 10 moving default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2024-05-30T16:23:31.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.091+0000 7f6c7df4d980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:tester.client.0_test@test.test, ofs0, ns] 2024-05-30T16:23:31.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.091+0000 7f6b5affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431581 cookie 94840371192528 notifier 7371 bl.length()=200 2024-05-30T16:23:31.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.091+0000 7f6b5affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x7 2024-05-30T16:23:31.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.091+0000 7f6b5affd640 10 rgw watcher librados: moving default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2024-05-30T16:23:31.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.091+0000 7f6c7df4d980 10 cache put: name=default.rgw.meta+users.keys+WMTCEIRCXEZTRMVCZBLU info.flags=0x7 2024-05-30T16:23:31.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.091+0000 7f6c7df4d980 10 moving default.rgw.meta+users.keys+WMTCEIRCXEZTRMVCZBLU to cache LRU end 2024-05-30T16:23:31.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.091+0000 7f6c7df4d980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.keys:WMTCEIRCXEZTRMVCZBLU, ofs0, ns] 2024-05-30T16:23:31.093 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.091+0000 7f6b5affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431582 cookie 94840371192528 notifier 7371 bl.length()=189 2024-05-30T16:23:31.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.091+0000 7f6b5affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+WMTCEIRCXEZTRMVCZBLU info.flags=0x7 2024-05-30T16:23:31.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.091+0000 7f6b5affd640 10 rgw watcher librados: moving default.rgw.meta+users.keys+WMTCEIRCXEZTRMVCZBLU to cache LRU end 2024-05-30T16:23:31.095 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-30T16:23:31.095 INFO:teuthology.orchestra.run.smithi006.stdout: "user_id": "tester.client.0", 2024-05-30T16:23:31.095 INFO:teuthology.orchestra.run.smithi006.stdout: "display_name": "Ms. tester.client.0", 2024-05-30T16:23:31.095 INFO:teuthology.orchestra.run.smithi006.stdout: "email": "tester.client.0_test@test.test", 2024-05-30T16:23:31.095 INFO:teuthology.orchestra.run.smithi006.stdout: "suspended": 0, 2024-05-30T16:23:31.095 INFO:teuthology.orchestra.run.smithi006.stdout: "max_buckets": 1000, 2024-05-30T16:23:31.095 INFO:teuthology.orchestra.run.smithi006.stdout: "subusers": [], 2024-05-30T16:23:31.095 INFO:teuthology.orchestra.run.smithi006.stdout: "keys": [ 2024-05-30T16:23:31.096 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T16:23:31.096 INFO:teuthology.orchestra.run.smithi006.stdout: "user": "tester.client.0", 2024-05-30T16:23:31.096 INFO:teuthology.orchestra.run.smithi006.stdout: "access_key": "WMTCEIRCXEZTRMVCZBLU", 2024-05-30T16:23:31.096 INFO:teuthology.orchestra.run.smithi006.stdout: "secret_key": "iNbbzXvZSrec+9bJRXtd001wDvdeX/bUUiQZw6yx9+ku4rbmQqemjg==", 2024-05-30T16:23:31.096 INFO:teuthology.orchestra.run.smithi006.stdout: "active": true, 2024-05-30T16:23:31.096 INFO:teuthology.orchestra.run.smithi006.stdout: "create_date": "2024-05-30T16:23:31.085683Z" 2024-05-30T16:23:31.096 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T16:23:31.096 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-30T16:23:31.096 INFO:teuthology.orchestra.run.smithi006.stdout: "swift_keys": [], 2024-05-30T16:23:31.096 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": [], 2024-05-30T16:23:31.096 INFO:teuthology.orchestra.run.smithi006.stdout: "op_mask": "read, write, delete", 2024-05-30T16:23:31.096 INFO:teuthology.orchestra.run.smithi006.stdout: "default_placement": "", 2024-05-30T16:23:31.096 INFO:teuthology.orchestra.run.smithi006.stdout: "default_storage_class": "", 2024-05-30T16:23:31.097 INFO:teuthology.orchestra.run.smithi006.stdout: "placement_tags": [], 2024-05-30T16:23:31.097 INFO:teuthology.orchestra.run.smithi006.stdout: "bucket_quota": { 2024-05-30T16:23:31.097 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": false, 2024-05-30T16:23:31.097 INFO:teuthology.orchestra.run.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:23:31.097 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size": -1, 2024-05-30T16:23:31.097 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:23:31.097 INFO:teuthology.orchestra.run.smithi006.stdout: "max_objects": -1 2024-05-30T16:23:31.097 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T16:23:31.097 INFO:teuthology.orchestra.run.smithi006.stdout: "user_quota": { 2024-05-30T16:23:31.097 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": false, 2024-05-30T16:23:31.097 INFO:teuthology.orchestra.run.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:23:31.097 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size": -1, 2024-05-30T16:23:31.097 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:23:31.098 INFO:teuthology.orchestra.run.smithi006.stdout: "max_objects": -1 2024-05-30T16:23:31.098 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T16:23:31.098 INFO:teuthology.orchestra.run.smithi006.stdout: "temp_url_keys": [], 2024-05-30T16:23:31.098 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "rgw", 2024-05-30T16:23:31.098 INFO:teuthology.orchestra.run.smithi006.stdout: "mfa_ids": [], 2024-05-30T16:23:31.098 INFO:teuthology.orchestra.run.smithi006.stdout: "account_id": "", 2024-05-30T16:23:31.098 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2024-05-30T16:23:31.098 INFO:teuthology.orchestra.run.smithi006.stdout: "create_date": "2024-05-30T16:23:31.085664Z", 2024-05-30T16:23:31.098 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-30T16:23:31.098 INFO:teuthology.orchestra.run.smithi006.stdout: "group_ids": [] 2024-05-30T16:23:31.098 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-30T16:23:31.098 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:23:31.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.095+0000 7f6b31fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:23:31.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.099+0000 7f6c7df4d980 20 remove_watcher() i=0 2024-05-30T16:23:31.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.099+0000 7f6c7df4d980 2 removed watcher, disabling cache 2024-05-30T16:23:31.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.103+0000 7f6c7df4d980 20 remove_watcher() i=1 2024-05-30T16:23:31.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.107+0000 7f6c7df4d980 20 remove_watcher() i=2 2024-05-30T16:23:31.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.107+0000 7f6c7df4d980 20 remove_watcher() i=3 2024-05-30T16:23:31.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.111+0000 7f6c7df4d980 20 remove_watcher() i=4 2024-05-30T16:23:31.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.111+0000 7f6c7df4d980 20 remove_watcher() i=5 2024-05-30T16:23:31.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.111+0000 7f6c7df4d980 20 remove_watcher() i=6 2024-05-30T16:23:31.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.115+0000 7f6c7df4d980 20 remove_watcher() i=7 2024-05-30T16:23:31.131 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T16:23:31.131 DEBUG:teuthology.orchestra.run.smithi006:> dd of=/home/ubuntu/cephtest/archive/s3-tests-java.client.0.conf 2024-05-30T16:23:31.138 INFO:tasks.s3tests_java:S3 Tests Java: Running tests... 2024-05-30T16:23:31.138 DEBUG:teuthology.orchestra.run.smithi006:> cp /home/ubuntu/cephtest/archive/s3-tests-java.client.0.conf /home/ubuntu/cephtest/s3-tests-java/config.properties 2024-05-30T16:23:31.182 DEBUG:teuthology.orchestra.run.smithi006:> radosgw-admin gc process --include-all 2024-05-30T16:23:31.257 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:23:31.257 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:23:31.305 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.303+0000 7f08993d3980 20 rados->read ofs=0 len=0 2024-05-30T16:23:31.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.307+0000 7f08993d3980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:31.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.307+0000 7f08993d3980 20 realm 2024-05-30T16:23:31.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.307+0000 7f08993d3980 20 rados->read ofs=0 len=0 2024-05-30T16:23:31.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.307+0000 7f08993d3980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:31.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.307+0000 7f08993d3980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:23:31.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.307+0000 7f08993d3980 20 rados->read ofs=0 len=0 2024-05-30T16:23:31.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.307+0000 7f08993d3980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:31.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.307+0000 7f08993d3980 20 rados->read ofs=0 len=0 2024-05-30T16:23:31.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.307+0000 7f08993d3980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:23:31.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.307+0000 7f08993d3980 20 rados->read ofs=0 len=0 2024-05-30T16:23:31.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.307+0000 7f08993d3980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:23:31.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.307+0000 7f08993d3980 20 searching for the correct realm 2024-05-30T16:23:31.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.315+0000 7f08976e5640 0 -- 172.21.15.6:0/2500683632 >> [v2:172.21.15.204:6823/3156311527,v1:172.21.15.204:6825/3156311527] conn(0x7f088003f070 msgr2=0x7f0880040460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:23:31.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.319+0000 7f0896ee4640 0 -- 172.21.15.6:0/2500683632 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f0880050410 msgr2=0x7f0880050890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:23:31.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.535+0000 7f08993d3980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:23:31.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.535+0000 7f08993d3980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:23:31.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.535+0000 7f08993d3980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:23:31.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.535+0000 7f08993d3980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:23:31.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.535+0000 7f08993d3980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:23:31.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.535+0000 7f08993d3980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:23:31.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.535+0000 7f08993d3980 20 rados->read ofs=0 len=0 2024-05-30T16:23:31.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.535+0000 7f08993d3980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:31.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.535+0000 7f08993d3980 20 rados->read ofs=0 len=0 2024-05-30T16:23:31.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.539+0000 7f08993d3980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:23:31.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.539+0000 7f08993d3980 20 rados->read ofs=0 len=0 2024-05-30T16:23:31.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.539+0000 7f08993d3980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:23:31.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.539+0000 7f08993d3980 20 zone default found 2024-05-30T16:23:31.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.539+0000 7f08993d3980 4 Realm: () 2024-05-30T16:23:31.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.539+0000 7f08993d3980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:23:31.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.539+0000 7f08993d3980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:23:31.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.539+0000 7f08993d3980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:23:31.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.539+0000 7f08993d3980 20 zonegroup default 2024-05-30T16:23:31.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.539+0000 7f08993d3980 20 rados->read ofs=0 len=0 2024-05-30T16:23:31.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.539+0000 7f08993d3980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:31.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.539+0000 7f08993d3980 20 rados->read ofs=0 len=0 2024-05-30T16:23:31.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.539+0000 7f08993d3980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:31.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.539+0000 7f08993d3980 20 rados->read ofs=0 len=0 2024-05-30T16:23:31.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.543+0000 7f08993d3980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:31.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.543+0000 7f08993d3980 20 started sync module instance, tier type = 2024-05-30T16:23:31.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.543+0000 7f08993d3980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:23:31.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.567+0000 7f08993d3980 20 add_watcher() i=0 2024-05-30T16:23:31.566 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.567+0000 7f08993d3980 20 add_watcher() i=1 2024-05-30T16:23:31.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.567+0000 7f08993d3980 20 add_watcher() i=2 2024-05-30T16:23:31.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.567+0000 7f08993d3980 20 add_watcher() i=3 2024-05-30T16:23:31.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.567+0000 7f08993d3980 20 add_watcher() i=4 2024-05-30T16:23:31.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.567+0000 7f08993d3980 20 add_watcher() i=5 2024-05-30T16:23:31.567 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.567+0000 7f08993d3980 20 add_watcher() i=6 2024-05-30T16:23:31.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.567+0000 7f08993d3980 20 add_watcher() i=7 2024-05-30T16:23:31.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.567+0000 7f08993d3980 2 all 8 watchers are set, enabling cache 2024-05-30T16:23:31.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.571+0000 7f074d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:23:31.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.575+0000 7f08993d3980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:23:31.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.575+0000 7f08993d3980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:23:31.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.671+0000 7f06c7fff640 20 reqs_thread_entry: start 2024-05-30T16:23:31.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.671+0000 7f08993d3980 20 init_complete bucket index max shards: 11 2024-05-30T16:23:31.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.671+0000 7f08993d3980 20 Filter name: none 2024-05-30T16:23:31.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.671+0000 7f06c5ffb640 20 reqs_thread_entry: start 2024-05-30T16:23:31.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.671+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-05-30T16:23:31.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.675+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.675+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:31.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.679+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:23:31.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.679+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.7737070351589065933', time=2024-05-30T18:18:19.023303+0000, chain.objs.size()=2 2024-05-30T16:23:31.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.679+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.14__shadow_.AXOmusT1Gw2HgSoCl5q-gSncRGm6ftL_0 2024-05-30T16:23:31.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.679+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.14__shadow_.AXOmusT1Gw2HgSoCl5q-gSncRGm6ftL_1 2024-05-30T16:23:31.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.687+0000 7f08993d3980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:23:31.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.691+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-05-30T16:23:31.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.695+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.695+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:31.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.695+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-30T16:23:31.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.695+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='_qv0iLHZEhEywlkjgazaBnZp19fc0ofw', time=2024-05-30T18:18:18.612276+0000, chain.objs.size()=1 2024-05-30T16:23:31.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.695+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.M0ITihsC-YD2zOMpJDeMeYhuYGgabJb_0 2024-05-30T16:23:31.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.695+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.1621245897829638086', time=2024-05-30T18:18:18.833998+0000, chain.objs.size()=3 2024-05-30T16:23:31.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.695+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-LUKEWARM.2~wajqBmpC82qXXhM9doS3FFBuijhwm7k.1 2024-05-30T16:23:31.697 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.695+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-LUKEWARM.2~wajqBmpC82qXXhM9doS3FFBuijhwm7k.2 2024-05-30T16:23:31.697 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.695+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-LUKEWARM.2~wajqBmpC82qXXhM9doS3FFBuijhwm7k.3 2024-05-30T16:23:31.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.699+0000 7f08993d3980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:23:31.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.703+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-05-30T16:23:31.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.707+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.707+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:31.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.707+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.711+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-05-30T16:23:31.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.711+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.715+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:31.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.715+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:23:31.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.715+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11488756881640554302', time=2024-05-30T18:18:18.963695+0000, chain.objs.size()=3 2024-05-30T16:23:31.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.715+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15__multipart_foo-LUKEWARM.2~Hsrr_GObGFXEobyJ1aV_Kc0udgQ82aw.1 2024-05-30T16:23:31.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.715+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15__multipart_foo-LUKEWARM.2~Hsrr_GObGFXEobyJ1aV_Kc0udgQ82aw.2 2024-05-30T16:23:31.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.715+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15__multipart_foo-LUKEWARM.2~Hsrr_GObGFXEobyJ1aV_Kc0udgQ82aw.3 2024-05-30T16:23:31.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.719+0000 7f08993d3980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:23:31.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.723+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-05-30T16:23:31.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.727+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.731+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:31.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.731+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.735+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-05-30T16:23:31.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.739+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.739+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:31.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.739+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.743+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-05-30T16:23:31.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.747+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.747+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:31.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.747+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-30T16:23:31.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.747+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12523056908549656105', time=2024-05-30T18:18:18.503386+0000, chain.objs.size()=4 2024-05-30T16:23:31.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.747+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.10__multipart_foo.2~SQVlhgfeYBCxqaPLnUGzte0SNojFJDp.1 2024-05-30T16:23:31.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.747+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.10__multipart_foo.2~SQVlhgfeYBCxqaPLnUGzte0SNojFJDp.2 2024-05-30T16:23:31.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.747+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.10__multipart_foo.2~SQVlhgfeYBCxqaPLnUGzte0SNojFJDp.3 2024-05-30T16:23:31.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.747+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.10__multipart_foo.2~SQVlhgfeYBCxqaPLnUGzte0SNojFJDp.4 2024-05-30T16:23:31.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.747+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='_UV2RhYiS5-EwRIGn2WlcbIOWCjkW94d', time=2024-05-30T18:18:18.600552+0000, chain.objs.size()=1 2024-05-30T16:23:31.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.747+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.RbGtPjlEyaaxk8wiUnUqqJUTuuCpI5__0 2024-05-30T16:23:31.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.755+0000 7f08993d3980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:23:31.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.759+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-05-30T16:23:31.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.763+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.763+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:31.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.763+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.767+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-05-30T16:23:31.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.771+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.771+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:31.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.771+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.775+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-05-30T16:23:31.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.779+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.779+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:31.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.783+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:23:31.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.783+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='_b_-sVqGh-yH058cG7edPlrV51cWwRdw', time=2024-05-30T18:18:18.869854+0000, chain.objs.size()=1 2024-05-30T16:23:31.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.783+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__shadow_.abmgsR_PDSXz9BE5Oo36g4vJ_Sd7CbP_0 2024-05-30T16:23:31.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.783+0000 7f08993d3980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:23:31.790 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.791+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-05-30T16:23:31.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.795+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.795+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:31.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.795+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-30T16:23:31.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.795+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.sVnrVYW9-ta8wIRvjaw10ZLJEkUm-k__1_a735fe677fd16778bb509a6b8fb3a9a7', time=2024-05-30T18:18:18.671914+0000, chain.objs.size()=1 2024-05-30T16:23:31.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.795+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.sVnrVYW9-ta8wIRvjaw10ZLJEkUm-k__0 2024-05-30T16:23:31.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.795+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.hTEeNl7avS8xfv478tR1-RVidOqTxuj_1_ddf4806003dde406047b90bf47d75905', time=2024-05-30T18:18:18.695921+0000, chain.objs.size()=1 2024-05-30T16:23:31.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.795+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.hTEeNl7avS8xfv478tR1-RVidOqTxuj_0 2024-05-30T16:23:31.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.799+0000 7f08993d3980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:23:31.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.803+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-05-30T16:23:31.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.807+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.811+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:31.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.811+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:23:31.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.811+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12873641608161991758', time=2024-05-30T18:18:18.585697+0000, chain.objs.size()=1 2024-05-30T16:23:31.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.811+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.hTEeNl7avS8xfv478tR1-RVidOqTxuj_0 2024-05-30T16:23:31.814 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.815+0000 7f08993d3980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:23:31.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.819+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-05-30T16:23:31.821 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.823+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.823+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:31.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.823+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.827+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-05-30T16:23:31.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.831+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.831+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:31.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.835+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.835+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-05-30T16:23:31.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.839+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.839+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:31.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.843+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:23:31.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.843+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='_8RV0OTyicl_HXdetg1Vl8oWv0mwM22P', time=2024-05-30T18:18:18.635401+0000, chain.objs.size()=1 2024-05-30T16:23:31.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.843+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.WDNKaVYSkoRupdUyl31sobeBcYzQ-1F_1 2024-05-30T16:23:31.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.847+0000 7f08993d3980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:23:31.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.851+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-05-30T16:23:31.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.855+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.855+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:31.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.859+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.859+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-05-30T16:23:31.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.863+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.867+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:31.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.867+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:23:31.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.867+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11952383582725206237', time=2024-05-30T18:18:18.378250+0000, chain.objs.size()=4 2024-05-30T16:23:31.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.867+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.13__multipart_foo.2~5teFu6HIxTDVSEf9D0cZ9ZwIH5aOMFH.1 2024-05-30T16:23:31.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.867+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.13__multipart_foo.2~5teFu6HIxTDVSEf9D0cZ9ZwIH5aOMFH.2 2024-05-30T16:23:31.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.867+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.13__multipart_foo.2~5teFu6HIxTDVSEf9D0cZ9ZwIH5aOMFH.3 2024-05-30T16:23:31.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.867+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.13__multipart_foo.2~5teFu6HIxTDVSEf9D0cZ9ZwIH5aOMFH.4 2024-05-30T16:23:31.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.875+0000 7f08993d3980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:23:31.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.879+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-05-30T16:23:31.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.883+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.887+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:31.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.887+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-05-30T16:23:31.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.887+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='_kC7FzPcvJAXSqn36qCEfy1ooHRUncgZ', time=2024-05-30T18:18:18.572456+0000, chain.objs.size()=1 2024-05-30T16:23:31.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.887+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.sAiQcD3_iUscIyjpheJeMPPpllHhzO9_1 2024-05-30T16:23:31.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.887+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='_Xa0MqTgVK3PtPq9sEAKgmwldHs5USY6', time=2024-05-30T18:18:18.755006+0000, chain.objs.size()=1 2024-05-30T16:23:31.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.887+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__shadow_.3x-gEVKbJAF2tDW4EwV63UEa1aUisGd_0 2024-05-30T16:23:31.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.887+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='_vnHmXe10Qv_Vs0QKfs4_8YrGQcVcSdx', time=2024-05-30T18:18:18.766662+0000, chain.objs.size()=2 2024-05-30T16:23:31.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.887+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__shadow_.2IR4yfvjh4iUsvYhjJUHaDAsLeVi04__1 2024-05-30T16:23:31.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.887+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__shadow_.2IR4yfvjh4iUsvYhjJUHaDAsLeVi04__2 2024-05-30T16:23:31.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.887+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17310732107394054150', time=2024-05-30T18:18:18.777448+0000, chain.objs.size()=3 2024-05-30T16:23:31.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.887+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-FROZEN.2~BFRcb5Q8ovi-IgTqlOD8ZC7hmBrYMTv.1 2024-05-30T16:23:31.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.887+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-FROZEN.2~BFRcb5Q8ovi-IgTqlOD8ZC7hmBrYMTv.2 2024-05-30T16:23:31.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.887+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-FROZEN.2~BFRcb5Q8ovi-IgTqlOD8ZC7hmBrYMTv.3 2024-05-30T16:23:31.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.891+0000 7f08993d3980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:23:31.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.903+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-05-30T16:23:31.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.907+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.907+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:31.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.907+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.911+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-05-30T16:23:31.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.915+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.915+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:31.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.915+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:23:31.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.915+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.3911489254370521540', time=2024-05-30T18:18:18.437568+0000, chain.objs.size()=4 2024-05-30T16:23:31.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.915+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11__multipart_foo.2~rsfKWR_VmhHpQAGvhcCSpjk4c7U0f4N.1 2024-05-30T16:23:31.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.919+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11__multipart_foo.2~rsfKWR_VmhHpQAGvhcCSpjk4c7U0f4N.2 2024-05-30T16:23:31.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.919+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11__multipart_foo.2~rsfKWR_VmhHpQAGvhcCSpjk4c7U0f4N.3 2024-05-30T16:23:31.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.919+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11__multipart_foo.2~rsfKWR_VmhHpQAGvhcCSpjk4c7U0f4N.4 2024-05-30T16:23:31.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.923+0000 7f08993d3980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:23:31.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.931+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-05-30T16:23:31.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.935+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.935+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.939+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:23:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.939+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.FsErSdszcSbW8ptNqjfN4gica2HNJIg_1_98c122ff0375e35c43d882e682fe36ec', time=2024-05-30T18:18:18.659769+0000, chain.objs.size()=1 2024-05-30T16:23:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.939+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.FsErSdszcSbW8ptNqjfN4gica2HNJIg_1 2024-05-30T16:23:31.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.943+0000 7f08993d3980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:23:31.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.947+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-05-30T16:23:31.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.955+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.955+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:31.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.955+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.959+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-05-30T16:23:31.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.963+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.963+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:31.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.963+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-05-30T16:23:31.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.963+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17418197569884719744', time=2024-05-30T18:18:18.624135+0000, chain.objs.size()=1 2024-05-30T16:23:31.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.963+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.FsErSdszcSbW8ptNqjfN4gica2HNJIg_1 2024-05-30T16:23:31.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.963+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='_V8edkAThlk7MAOlvAqltCrZ2wKS-eS_', time=2024-05-30T18:18:18.822497+0000, chain.objs.size()=2 2024-05-30T16:23:31.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.963+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__shadow_.w0_wBDs8bF63Tv757aR0tb8_2pXTQyw_1 2024-05-30T16:23:31.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.963+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__shadow_.w0_wBDs8bF63Tv757aR0tb8_2pXTQyw_2 2024-05-30T16:23:31.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.963+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-FROZEN.2~BFRcb5Q8ovi-IgTqlOD8ZC7hmBrYMTv.2_aedc64ca1185a391387b70943af4161d', time=2024-05-30T18:18:18.880210+0000, chain.objs.size()=3 2024-05-30T16:23:31.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.963+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-FROZEN.2~BFRcb5Q8ovi-IgTqlOD8ZC7hmBrYMTv.1 2024-05-30T16:23:31.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.963+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-FROZEN.2~BFRcb5Q8ovi-IgTqlOD8ZC7hmBrYMTv.2 2024-05-30T16:23:31.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.963+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-FROZEN.2~BFRcb5Q8ovi-IgTqlOD8ZC7hmBrYMTv.3 2024-05-30T16:23:31.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.971+0000 7f08993d3980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:23:31.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.975+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-05-30T16:23:31.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.979+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:31.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.983+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:31.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.983+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:23:31.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.983+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11509641366497265618', time=2024-05-30T18:18:18.941896+0000, chain.objs.size()=3 2024-05-30T16:23:31.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.983+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15__multipart_foo-FROZEN.2~CW-sTJfcAqjohKEMX8QliHZkV3OM6hr.1 2024-05-30T16:23:31.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.983+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15__multipart_foo-FROZEN.2~CW-sTJfcAqjohKEMX8QliHZkV3OM6hr.2 2024-05-30T16:23:31.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.983+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15__multipart_foo-FROZEN.2~CW-sTJfcAqjohKEMX8QliHZkV3OM6hr.3 2024-05-30T16:23:31.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.991+0000 7f08993d3980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:23:31.998 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:31.999+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-05-30T16:23:32.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.003+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.003+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.003+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.007+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-05-30T16:23:32.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.011+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.011+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.015+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-30T16:23:32.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.015+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='_OcE_IYB-B5quL9-wrpppgcdhg2HuA_T', time=2024-05-30T18:18:18.560509+0000, chain.objs.size()=1 2024-05-30T16:23:32.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.015+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.Znfc6ITYhEqooNv7e4G7MaY3U5V403h_0 2024-05-30T16:23:32.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.015+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-STANDARD.2~9zF-INvfUU3CV2mHkJEI4N-gDYI1hCM.2_8bf2befbb85c41f573c5363aefdbcdaa', time=2024-05-30T18:18:18.845942+0000, chain.objs.size()=3 2024-05-30T16:23:32.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.015+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-STANDARD.2~9zF-INvfUU3CV2mHkJEI4N-gDYI1hCM.1 2024-05-30T16:23:32.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.015+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-STANDARD.2~9zF-INvfUU3CV2mHkJEI4N-gDYI1hCM.2 2024-05-30T16:23:32.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.015+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-STANDARD.2~9zF-INvfUU3CV2mHkJEI4N-gDYI1hCM.3 2024-05-30T16:23:32.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.023+0000 7f08993d3980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:23:32.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.027+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-05-30T16:23:32.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.031+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.031+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.031+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-05-30T16:23:32.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.031+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12830297622961229936', time=2024-05-30T18:18:18.322506+0000, chain.objs.size()=4 2024-05-30T16:23:32.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.031+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12__multipart_foo.2~oEALAIvQxkhRs-gz1eBFd-L8SHqQ_ZD.1 2024-05-30T16:23:32.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.031+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12__multipart_foo.2~oEALAIvQxkhRs-gz1eBFd-L8SHqQ_ZD.2 2024-05-30T16:23:32.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.031+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12__multipart_foo.2~oEALAIvQxkhRs-gz1eBFd-L8SHqQ_ZD.3 2024-05-30T16:23:32.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.031+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12__multipart_foo.2~oEALAIvQxkhRs-gz1eBFd-L8SHqQ_ZD.4 2024-05-30T16:23:32.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.031+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16322308426137548912', time=2024-05-30T18:18:18.810886+0000, chain.objs.size()=3 2024-05-30T16:23:32.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.031+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-STANDARD.2~9zF-INvfUU3CV2mHkJEI4N-gDYI1hCM.1 2024-05-30T16:23:32.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.031+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-STANDARD.2~9zF-INvfUU3CV2mHkJEI4N-gDYI1hCM.2 2024-05-30T16:23:32.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.031+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-STANDARD.2~9zF-INvfUU3CV2mHkJEI4N-gDYI1hCM.3 2024-05-30T16:23:32.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.031+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-LUKEWARM.2~wajqBmpC82qXXhM9doS3FFBuijhwm7k.2_dc86120fc4cbe247e2ec35caf3b7d080', time=2024-05-30T18:18:18.857338+0000, chain.objs.size()=3 2024-05-30T16:23:32.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.031+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-LUKEWARM.2~wajqBmpC82qXXhM9doS3FFBuijhwm7k.1 2024-05-30T16:23:32.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.031+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-LUKEWARM.2~wajqBmpC82qXXhM9doS3FFBuijhwm7k.2 2024-05-30T16:23:32.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.031+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__multipart_foo-LUKEWARM.2~wajqBmpC82qXXhM9doS3FFBuijhwm7k.3 2024-05-30T16:23:32.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.039+0000 7f08993d3980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:23:32.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.047+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-05-30T16:23:32.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.051+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.051+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.055+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.055+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-05-30T16:23:32.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.059+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.063+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.063+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:23:32.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.063+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.310794086773851024', time=2024-05-30T18:18:18.647325+0000, chain.objs.size()=1 2024-05-30T16:23:32.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.063+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.sVnrVYW9-ta8wIRvjaw10ZLJEkUm-k__0 2024-05-30T16:23:32.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.067+0000 7f08993d3980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:23:32.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.075+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-05-30T16:23:32.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.075+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.079+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.079+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-30T16:23:32.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.079+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='_VLaDFQwaCiFDwqOPngb13FJ0EgbZw11', time=2024-05-30T18:18:18.684521+0000, chain.objs.size()=1 2024-05-30T16:23:32.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.079+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16__shadow_.SVdi7TJiQFTQeBdO0Z3XFH3Rz7vn1A7_0 2024-05-30T16:23:32.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.079+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11005071422508405881', time=2024-05-30T18:18:18.953679+0000, chain.objs.size()=3 2024-05-30T16:23:32.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.079+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15__multipart_foo-STANDARD.2~gIaO10Qh7XOEWbUAn_wo19sWSzd6og2.1 2024-05-30T16:23:32.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.079+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15__multipart_foo-STANDARD.2~gIaO10Qh7XOEWbUAn_wo19sWSzd6og2.2 2024-05-30T16:23:32.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.079+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15__multipart_foo-STANDARD.2~gIaO10Qh7XOEWbUAn_wo19sWSzd6og2.3 2024-05-30T16:23:32.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.087+0000 7f08993d3980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:23:32.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.095+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-05-30T16:23:32.097 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.099+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.099+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.099+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-30T16:23:32.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.099+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='_qIFw_2_h-URjgdqm0UZ2AgdHK4WxqD6', time=2024-05-30T18:18:18.788690+0000, chain.objs.size()=2 2024-05-30T16:23:32.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.099+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__shadow_.T0ycfx6PC5jYOSHRzYF6kP3mqQoeAAT_0 2024-05-30T16:23:32.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.099+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__shadow_.T0ycfx6PC5jYOSHRzYF6kP3mqQoeAAT_1 2024-05-30T16:23:32.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.099+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='_c32z-THpIOEp_EqO3zGgwcYx5kdlsBi', time=2024-05-30T18:18:18.799396+0000, chain.objs.size()=2 2024-05-30T16:23:32.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.099+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__shadow_.OJOkxH1CyfmmDDMChYlCZActQFQ8GS2_0 2024-05-30T16:23:32.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.099+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17__shadow_.OJOkxH1CyfmmDDMChYlCZActQFQ8GS2_1 2024-05-30T16:23:32.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.107+0000 7f08993d3980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:23:32.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.115+0000 7f08993d3980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-05-30T16:23:32.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.119+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.119+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.119+0000 7f08993d3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:23:32.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.119+0000 7f08993d3980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.6717838609504419540', time=2024-05-30T18:18:19.043011+0000, chain.objs.size()=2 2024-05-30T16:23:32.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.119+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.14__shadow_.gK8bQIHPaijsLps-ink8QZ-02ky_Yjd_0 2024-05-30T16:23:32.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.119+0000 7f08993d3980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.14__shadow_.gK8bQIHPaijsLps-ink8QZ-02ky_Yjd_1 2024-05-30T16:23:32.128 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.127+0000 7f08993d3980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:23:32.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.135+0000 7f074d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:23:32.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.139+0000 7f08993d3980 20 remove_watcher() i=0 2024-05-30T16:23:32.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.139+0000 7f08993d3980 2 removed watcher, disabling cache 2024-05-30T16:23:32.144 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.143+0000 7f08993d3980 20 remove_watcher() i=1 2024-05-30T16:23:32.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.147+0000 7f08993d3980 20 remove_watcher() i=2 2024-05-30T16:23:32.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.151+0000 7f08993d3980 20 remove_watcher() i=3 2024-05-30T16:23:32.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.151+0000 7f08993d3980 20 remove_watcher() i=4 2024-05-30T16:23:32.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.155+0000 7f08993d3980 20 remove_watcher() i=5 2024-05-30T16:23:32.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.159+0000 7f08993d3980 20 remove_watcher() i=6 2024-05-30T16:23:32.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.159+0000 7f08993d3980 20 remove_watcher() i=7 2024-05-30T16:23:32.182 DEBUG:teuthology.orchestra.run.smithi006:> radosgw-admin gc process --include-all 2024-05-30T16:23:32.258 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:23:32.259 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:23:32.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.307+0000 7f6a5c717980 20 rados->read ofs=0 len=0 2024-05-30T16:23:32.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.307+0000 7f6a5c717980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:32.307 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.307+0000 7f6a5c717980 20 realm 2024-05-30T16:23:32.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.307+0000 7f6a5c717980 20 rados->read ofs=0 len=0 2024-05-30T16:23:32.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.307+0000 7f6a5c717980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:32.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.307+0000 7f6a5c717980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:23:32.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.307+0000 7f6a5c717980 20 rados->read ofs=0 len=0 2024-05-30T16:23:32.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.307+0000 7f6a5c717980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:32.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.307+0000 7f6a5c717980 20 rados->read ofs=0 len=0 2024-05-30T16:23:32.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.311+0000 7f6a5c717980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:23:32.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.311+0000 7f6a5c717980 20 rados->read ofs=0 len=0 2024-05-30T16:23:32.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.311+0000 7f6a5c717980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:23:32.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.311+0000 7f6a5c717980 20 searching for the correct realm 2024-05-30T16:23:32.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.315+0000 7f6a59a27640 0 -- 172.21.15.6:0/2555183112 >> [v2:172.21.15.6:6819/1141406589,v1:172.21.15.6:6821/1141406589] conn(0x558e348a1aa0 msgr2=0x558e348a3f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:23:32.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.323+0000 7f6a5a228640 0 -- 172.21.15.6:0/2555183112 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f6a50050410 msgr2=0x7f6a50050870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:23:32.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.331+0000 7f6a5c717980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:23:32.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.331+0000 7f6a5c717980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:23:32.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.331+0000 7f6a5c717980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:23:32.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.331+0000 7f6a5c717980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:23:32.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.331+0000 7f6a5c717980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:23:32.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.331+0000 7f6a5c717980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:23:32.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.331+0000 7f6a5c717980 20 rados->read ofs=0 len=0 2024-05-30T16:23:32.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.331+0000 7f6a5c717980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:32.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.331+0000 7f6a5c717980 20 rados->read ofs=0 len=0 2024-05-30T16:23:32.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.331+0000 7f6a5c717980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:23:32.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.331+0000 7f6a5c717980 20 rados->read ofs=0 len=0 2024-05-30T16:23:32.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.331+0000 7f6a5c717980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:23:32.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.331+0000 7f6a5c717980 20 zone default found 2024-05-30T16:23:32.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.331+0000 7f6a5c717980 4 Realm: () 2024-05-30T16:23:32.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.331+0000 7f6a5c717980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:23:32.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.331+0000 7f6a5c717980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:23:32.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.331+0000 7f6a5c717980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:23:32.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.331+0000 7f6a5c717980 20 zonegroup default 2024-05-30T16:23:32.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.331+0000 7f6a5c717980 20 rados->read ofs=0 len=0 2024-05-30T16:23:32.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.335+0000 7f6a5c717980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:32.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.335+0000 7f6a5c717980 20 rados->read ofs=0 len=0 2024-05-30T16:23:32.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.335+0000 7f6a5c717980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:32.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.335+0000 7f6a5c717980 20 rados->read ofs=0 len=0 2024-05-30T16:23:32.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.335+0000 7f6a5c717980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:23:32.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.335+0000 7f6a5c717980 20 started sync module instance, tier type = 2024-05-30T16:23:32.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.335+0000 7f6a5c717980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:23:32.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.351+0000 7f6a5c717980 20 add_watcher() i=0 2024-05-30T16:23:32.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.351+0000 7f6a5c717980 20 add_watcher() i=1 2024-05-30T16:23:32.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.351+0000 7f6a5c717980 20 add_watcher() i=2 2024-05-30T16:23:32.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.351+0000 7f6a5c717980 20 add_watcher() i=3 2024-05-30T16:23:32.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.351+0000 7f6a5c717980 20 add_watcher() i=4 2024-05-30T16:23:32.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.351+0000 7f6a5c717980 20 add_watcher() i=5 2024-05-30T16:23:32.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.351+0000 7f6a5c717980 20 add_watcher() i=6 2024-05-30T16:23:32.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.351+0000 7f6a5c717980 20 add_watcher() i=7 2024-05-30T16:23:32.354 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.351+0000 7f6a5c717980 2 all 8 watchers are set, enabling cache 2024-05-30T16:23:32.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.355+0000 7f6910fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:23:32.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.355+0000 7f6a5c717980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:23:32.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.355+0000 7f6a5c717980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:23:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.435+0000 7f688b7fe640 20 reqs_thread_entry: start 2024-05-30T16:23:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.435+0000 7f6a5c717980 20 init_complete bucket index max shards: 11 2024-05-30T16:23:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.435+0000 7f6a5c717980 20 Filter name: none 2024-05-30T16:23:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.435+0000 7f68897fa640 20 reqs_thread_entry: start 2024-05-30T16:23:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.435+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-05-30T16:23:32.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.439+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.439+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.439+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.443+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-05-30T16:23:32.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.447+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.447+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.447+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.447+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-05-30T16:23:32.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.451+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.455+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.455+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.459+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-05-30T16:23:32.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.459+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.459+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.459+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.463+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-05-30T16:23:32.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.463+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.467+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.467+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.467+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-05-30T16:23:32.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.471+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.475+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.475+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.479+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-05-30T16:23:32.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.483+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.483+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.483+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.487+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-05-30T16:23:32.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.491+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.491+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.491+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.491+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-05-30T16:23:32.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.495+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.498 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.499+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.499+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.501 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.499+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-05-30T16:23:32.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.503+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.503+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.503+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.507+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-05-30T16:23:32.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.511+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.511+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.511+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.515+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-05-30T16:23:32.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.519+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.520 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.519+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.523+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.523+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-05-30T16:23:32.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.527+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.527 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.527+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.528 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.527+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.527+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-05-30T16:23:32.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.531+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.535+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.535+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.535+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-05-30T16:23:32.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.539+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.539+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.539+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.543+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-05-30T16:23:32.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.547+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.547+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.547+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.551+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-05-30T16:23:32.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.555+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.555+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.555+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.559+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-05-30T16:23:32.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.563+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.563+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.563+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.567+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-05-30T16:23:32.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.571+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.571+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.571+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.575+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-05-30T16:23:32.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.579+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.579+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.579+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.584 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.583+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-05-30T16:23:32.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.587+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.587+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.587+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.591+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-05-30T16:23:32.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.595+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.595+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.595+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.599+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-05-30T16:23:32.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.599+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.603+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.603+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.607+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-05-30T16:23:32.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.611+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.611+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.611+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.615+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-05-30T16:23:32.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.619+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.619+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.619+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.623 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.623+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-05-30T16:23:32.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.627+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.627+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.627+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.631 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.631+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-05-30T16:23:32.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.635+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.635+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.639+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.639+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-05-30T16:23:32.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.643+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.647+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.647+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.647+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-05-30T16:23:32.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.651+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.655+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.656 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.655+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.659+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-05-30T16:23:32.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.663+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.663+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.663+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.667+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-05-30T16:23:32.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.671+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.671+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.671+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.675+0000 7f6a5c717980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-05-30T16:23:32.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.679+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.681 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.679+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:23:32.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.683+0000 7f6a5c717980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:23:32.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.687+0000 7f6910fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:23:32.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.691+0000 7f6a5c717980 20 remove_watcher() i=0 2024-05-30T16:23:32.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.691+0000 7f6a5c717980 2 removed watcher, disabling cache 2024-05-30T16:23:32.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.695+0000 7f6a5c717980 20 remove_watcher() i=1 2024-05-30T16:23:32.697 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.695+0000 7f6a5c717980 20 remove_watcher() i=2 2024-05-30T16:23:32.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.699+0000 7f6a5c717980 20 remove_watcher() i=3 2024-05-30T16:23:32.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.703+0000 7f6a5c717980 20 remove_watcher() i=4 2024-05-30T16:23:32.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.703+0000 7f6a5c717980 20 remove_watcher() i=5 2024-05-30T16:23:32.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.707+0000 7f6a5c717980 20 remove_watcher() i=6 2024-05-30T16:23:32.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:23:32.707+0000 7f6a5c717980 20 remove_watcher() i=7 2024-05-30T16:23:32.737 DEBUG:teuthology.orchestra.run.smithi006:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean test --rerun-tasks --no-build-cache --tests AWS4Test 2024-05-30T16:23:33.356 INFO:teuthology.orchestra.run.smithi006.stdout:Starting a Gradle Daemon (subsequent builds will be faster) 2024-05-30T16:23:38.257 INFO:teuthology.orchestra.run.smithi006.stdout:> Task :clean UP-TO-DATE 2024-05-30T16:23:57.456 INFO:teuthology.orchestra.run.smithi006.stdout:> Task :compileJava 2024-05-30T16:23:57.456 INFO:teuthology.orchestra.run.smithi006.stdout:> Task :processResources 2024-05-30T16:23:57.456 INFO:teuthology.orchestra.run.smithi006.stdout:> Task :classes 2024-05-30T16:23:58.356 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:23:58.356 INFO:teuthology.orchestra.run.smithi006.stdout:> Task :compileTestJava 2024-05-30T16:23:58.357 INFO:teuthology.orchestra.run.smithi006.stderr:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2024-05-30T16:23:58.357 INFO:teuthology.orchestra.run.smithi006.stderr:Note: Recompile with -Xlint:deprecation for details. 2024-05-30T16:23:58.358 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:23:58.358 INFO:teuthology.orchestra.run.smithi006.stdout:> Task :processTestResources NO-SOURCE 2024-05-30T16:23:58.358 INFO:teuthology.orchestra.run.smithi006.stdout:> Task :testClasses 2024-05-30T16:24:04.656 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:24:04.656 INFO:teuthology.orchestra.run.smithi006.stdout:> Task :test 2024-05-30T16:24:04.656 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:24:04.656 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadLLAPIAWS4 STARTED 2024-05-30T16:24:22.556 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:24:22.556 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadLLAPIAWS4 PASSED 2024-05-30T16:24:23.355 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:24:23.356 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadNotFoundLLAPIAWS4 STARTED 2024-05-30T16:24:23.655 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:24:23.655 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadNotFoundLLAPIAWS4 PASSED 2024-05-30T16:24:24.455 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:24:24.456 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testDownloadHLAPIAWS4 STARTED 2024-05-30T16:24:35.456 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:24:35.456 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testDownloadHLAPIAWS4 PASSED 2024-05-30T16:24:36.355 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:24:36.356 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoBucketHLAPIAWS4 STARTED 2024-05-30T16:24:36.455 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:24:36.456 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoBucketHLAPIAWS4 PASSED 2024-05-30T16:24:36.755 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:24:36.756 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoKeyHLAPIAWS4 STARTED 2024-05-30T16:24:36.955 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:24:36.956 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoKeyHLAPIAWS4 PASSED 2024-05-30T16:24:37.755 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:24:37.756 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyHLAPIAWS4 STARTED 2024-05-30T16:24:46.556 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:24:46.556 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyHLAPIAWS4 PASSED 2024-05-30T16:24:48.255 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:24:48.256 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 STARTED 2024-05-30T16:24:48.756 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:24:48.756 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 STANDARD_OUT 2024-05-30T16:24:48.756 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30 16:24:48,662 [Test worker] WARN com.amazonaws.util.Base64 - JAXB is unavailable. Will fallback to SDK implementation which may be less performant 2024-05-30T16:25:54.257 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:25:54.257 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 PASSED 2024-05-30T16:25:55.856 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:25:55.856 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCBucketHLAPIAWS4 STARTED 2024-05-30T16:25:56.056 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:25:56.056 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCBucketHLAPIAWS4 PASSED 2024-05-30T16:25:56.856 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:25:56.856 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCKeyHLAPIAWS4 STARTED 2024-05-30T16:25:57.156 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:25:57.156 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCKeyHLAPIAWS4 PASSED 2024-05-30T16:25:58.555 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:25:58.556 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadHLAPIAWS4 STARTED 2024-05-30T16:26:06.856 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:26:06.856 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadHLAPIAWS4 PASSED 2024-05-30T16:26:07.756 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:26:07.756 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadNoBucketHLAPIAWS4 STARTED 2024-05-30T16:26:07.855 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:26:07.856 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadNoBucketHLAPIAWS4 PASSED 2024-05-30T16:26:08.055 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:26:08.056 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadWithPauseHLAPIAWS4 STARTED 2024-05-30T16:26:18.456 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:26:18.456 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadWithPauseHLAPIAWS4 PASSED 2024-05-30T16:26:19.355 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:26:19.356 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadHLAPIAWS4 STARTED 2024-05-30T16:26:55.257 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:26:55.257 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadHLAPIAWS4 PASSED 2024-05-30T16:26:56.456 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:26:56.456 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadIncorrectMissingPartLLAPIAWS4 STARTED 2024-05-30T16:27:05.356 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:27:05.356 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadIncorrectMissingPartLLAPIAWS4 PASSED 2024-05-30T16:27:06.155 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:27:06.156 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadMultipleSizesLLAPIAWS4 STARTED 2024-05-30T16:29:11.556 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:29:11.557 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadMultipleSizesLLAPIAWS4 PASSED 2024-05-30T16:29:12.456 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:29:12.456 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadOverwriteExistingObjectLLAPIAWS4 STARTED 2024-05-30T16:29:22.456 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:29:22.456 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadOverwriteExistingObjectLLAPIAWS4 PASSED 2024-05-30T16:29:23.256 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:29:23.256 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadSmallLLAPIAWS4 STARTED 2024-05-30T16:29:31.455 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:29:31.456 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadSmallLLAPIAWS4 PASSED 2024-05-30T16:29:32.455 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:29:32.456 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadWithPauseAWS4 STARTED 2024-05-30T16:29:38.155 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:29:38.156 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadWithPauseAWS4 PASSED 2024-05-30T16:30:04.456 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:04.456 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateAfterToday4AWS4 STARTED 2024-05-30T16:30:04.756 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:04.756 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateAfterToday4AWS4 PASSED 2024-05-30T16:30:05.555 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:05.556 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeEpochAWS4 STARTED 2024-05-30T16:30:05.755 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:05.756 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeEpochAWS4 PASSED 2024-05-30T16:30:06.756 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:06.756 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeToday4AWS4 STARTED 2024-05-30T16:30:06.955 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:06.956 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeToday4AWS4 PASSED 2024-05-30T16:30:07.855 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:07.856 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateEmptyAWS4 STARTED 2024-05-30T16:30:08.155 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:08.156 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateEmptyAWS4 PASSED 2024-05-30T16:30:09.155 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:09.156 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateInvalidAWS4 STARTED 2024-05-30T16:30:09.456 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:09.456 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateInvalidAWS4 PASSED 2024-05-30T16:30:10.356 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:10.356 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateNoneAWS4 STARTED 2024-05-30T16:30:10.555 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:10.556 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateNoneAWS4 PASSED 2024-05-30T16:30:11.455 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:11.456 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadMd5InvalidGarbageAWS4 STARTED 2024-05-30T16:30:11.755 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:11.756 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadMd5InvalidGarbageAWS4 PASSED 2024-05-30T16:30:12.555 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:12.556 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testUploadFileHLAPIBigFileAWS4 STARTED 2024-05-30T16:30:30.856 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:30.856 INFO:teuthology.orchestra.run.smithi006.stdout:suite > AWS4 tests > AWS4Test.testUploadFileHLAPIBigFileAWS4 PASSED 2024-05-30T16:30:31.856 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:31.857 INFO:teuthology.orchestra.run.smithi006.stdout:---------------------------------------------------------------------- 2024-05-30T16:30:31.857 INFO:teuthology.orchestra.run.smithi006.stdout:| Results: SUCCESS (26 tests, 26 successes, 0 failures, 0 skipped) | 2024-05-30T16:30:31.857 INFO:teuthology.orchestra.run.smithi006.stdout:---------------------------------------------------------------------- 2024-05-30T16:30:31.857 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:31.857 INFO:teuthology.orchestra.run.smithi006.stdout:Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. 2024-05-30T16:30:31.857 INFO:teuthology.orchestra.run.smithi006.stdout:Use '--warning-mode all' to show the individual deprecation warnings. 2024-05-30T16:30:31.857 INFO:teuthology.orchestra.run.smithi006.stdout:See https://docs.gradle.org/6.0.1/userguide/command_line_interface.html#sec:command_line_warnings 2024-05-30T16:30:31.857 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:31.857 INFO:teuthology.orchestra.run.smithi006.stdout:BUILD SUCCESSFUL in 6m 58s 2024-05-30T16:30:31.857 INFO:teuthology.orchestra.run.smithi006.stdout:5 actionable tasks: 4 executed, 1 up-to-date 2024-05-30T16:30:32.196 DEBUG:teuthology.orchestra.run.smithi006:> radosgw-admin gc process --include-all 2024-05-30T16:30:32.229 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:30:32.229 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:30:32.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.274+0000 7fa8b1d34980 20 rados->read ofs=0 len=0 2024-05-30T16:30:32.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.278+0000 7fa8b1d34980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:32.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.278+0000 7fa8b1d34980 20 realm 2024-05-30T16:30:32.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.278+0000 7fa8b1d34980 20 rados->read ofs=0 len=0 2024-05-30T16:30:32.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.278+0000 7fa8b1d34980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:32.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.278+0000 7fa8b1d34980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:30:32.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.278+0000 7fa8b1d34980 20 rados->read ofs=0 len=0 2024-05-30T16:30:32.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.278+0000 7fa8b1d34980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:32.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.278+0000 7fa8b1d34980 20 rados->read ofs=0 len=0 2024-05-30T16:30:32.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.278+0000 7fa8b1d34980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:30:32.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.278+0000 7fa8b1d34980 20 rados->read ofs=0 len=0 2024-05-30T16:30:32.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.278+0000 7fa8b1d34980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:30:32.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.282+0000 7fa8b1d34980 20 searching for the correct realm 2024-05-30T16:30:32.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.286+0000 7fa8b0046640 0 -- 172.21.15.6:0/3254696764 >> [v2:172.21.15.204:6823/3156311527,v1:172.21.15.204:6825/3156311527] conn(0x7fa8a40de920 msgr2=0x7fa8a40b63f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:30:32.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.294+0000 7fa8af845640 0 -- 172.21.15.6:0/3254696764 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7fa8a40eb900 msgr2=0x7fa8a40bf3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:30:32.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.510+0000 7fa8b1d34980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:30:32.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.510+0000 7fa8b1d34980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:30:32.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.510+0000 7fa8b1d34980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:30:32.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.510+0000 7fa8b1d34980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:30:32.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.510+0000 7fa8b1d34980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:30:32.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.510+0000 7fa8b1d34980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:30:32.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.510+0000 7fa8b1d34980 20 rados->read ofs=0 len=0 2024-05-30T16:30:32.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.514+0000 7fa8b1d34980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:32.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.514+0000 7fa8b1d34980 20 rados->read ofs=0 len=0 2024-05-30T16:30:32.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.514+0000 7fa8b1d34980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:30:32.515 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.514+0000 7fa8b1d34980 20 rados->read ofs=0 len=0 2024-05-30T16:30:32.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.514+0000 7fa8b1d34980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:30:32.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.514+0000 7fa8b1d34980 20 zone default found 2024-05-30T16:30:32.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.514+0000 7fa8b1d34980 4 Realm: () 2024-05-30T16:30:32.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.514+0000 7fa8b1d34980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:30:32.516 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.514+0000 7fa8b1d34980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:30:32.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.514+0000 7fa8b1d34980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:30:32.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.514+0000 7fa8b1d34980 20 zonegroup default 2024-05-30T16:30:32.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.514+0000 7fa8b1d34980 20 rados->read ofs=0 len=0 2024-05-30T16:30:32.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.514+0000 7fa8b1d34980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:32.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.514+0000 7fa8b1d34980 20 rados->read ofs=0 len=0 2024-05-30T16:30:32.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.518+0000 7fa8b1d34980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:32.518 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.518+0000 7fa8b1d34980 20 rados->read ofs=0 len=0 2024-05-30T16:30:32.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.518+0000 7fa8b1d34980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:32.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.518+0000 7fa8b1d34980 20 started sync module instance, tier type = 2024-05-30T16:30:32.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.518+0000 7fa8b1d34980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:30:32.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.542+0000 7fa8b1d34980 20 add_watcher() i=0 2024-05-30T16:30:32.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.542+0000 7fa8b1d34980 20 add_watcher() i=1 2024-05-30T16:30:32.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.542+0000 7fa8b1d34980 20 add_watcher() i=2 2024-05-30T16:30:32.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.542+0000 7fa8b1d34980 20 add_watcher() i=3 2024-05-30T16:30:32.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.542+0000 7fa8b1d34980 20 add_watcher() i=4 2024-05-30T16:30:32.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.542+0000 7fa8b1d34980 20 add_watcher() i=5 2024-05-30T16:30:32.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.542+0000 7fa8b1d34980 20 add_watcher() i=6 2024-05-30T16:30:32.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.542+0000 7fa8b1d34980 20 add_watcher() i=7 2024-05-30T16:30:32.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.542+0000 7fa8b1d34980 2 all 8 watchers are set, enabling cache 2024-05-30T16:30:32.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.546+0000 7fa765fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:30:32.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.546+0000 7fa8b1d34980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:30:32.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.546+0000 7fa8b1d34980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:30:32.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.634+0000 7fa764fe1640 20 reqs_thread_entry: start 2024-05-30T16:30:32.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.634+0000 7fa8b1d34980 20 init_complete bucket index max shards: 11 2024-05-30T16:30:32.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.634+0000 7fa8b1d34980 20 Filter name: none 2024-05-30T16:30:32.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.634+0000 7fa6de7fc640 20 reqs_thread_entry: start 2024-05-30T16:30:32.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.634+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-05-30T16:30:32.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.638+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.642+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:32.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.642+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.646+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-05-30T16:30:32.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.646+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.646+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:32.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.650+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:30:32.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.650+0000 7fa8b1d34980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.34__multipart_key1.2~M5ep7aTUHWL6BbgHFBaXJAsUYnhweED.2_53e7a588dc695db28ded6fe6c3ad92aa', time=2024-05-30T18:24:46.812818+0000, chain.objs.size()=5 2024-05-30T16:30:32.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.650+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.34__multipart_key1.2~M5ep7aTUHWL6BbgHFBaXJAsUYnhweED.1 2024-05-30T16:30:32.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.650+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.34__multipart_key1.2~M5ep7aTUHWL6BbgHFBaXJAsUYnhweED.2 2024-05-30T16:30:32.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.650+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.34__multipart_key1.2~M5ep7aTUHWL6BbgHFBaXJAsUYnhweED.3 2024-05-30T16:30:32.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.650+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.34__multipart_key1.2~M5ep7aTUHWL6BbgHFBaXJAsUYnhweED.4 2024-05-30T16:30:32.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.650+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.34__multipart_key1.2~M5ep7aTUHWL6BbgHFBaXJAsUYnhweED.5 2024-05-30T16:30:32.655 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.654+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:30:32.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.658+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-05-30T16:30:32.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.662+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.662+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:32.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.666+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:30:32.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.666+0000 7fa8b1d34980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11506812848552924939', time=2024-05-30T18:25:27.395465+0000, chain.objs.size()=5 2024-05-30T16:30:32.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.666+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.37__multipart_key1.2~JeyDO8h7jfujzckn7GhewB4BXg1yE9u.1 2024-05-30T16:30:32.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.666+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.37__multipart_key1.2~JeyDO8h7jfujzckn7GhewB4BXg1yE9u.2 2024-05-30T16:30:32.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.666+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.37__multipart_key1.2~JeyDO8h7jfujzckn7GhewB4BXg1yE9u.3 2024-05-30T16:30:32.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.666+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.37__multipart_key1.2~JeyDO8h7jfujzckn7GhewB4BXg1yE9u.4 2024-05-30T16:30:32.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.666+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.37__multipart_key1.2~JeyDO8h7jfujzckn7GhewB4BXg1yE9u.5 2024-05-30T16:30:32.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.670+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:30:32.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.674+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-05-30T16:30:32.679 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.678+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.678+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:32.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.678+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.682+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-05-30T16:30:32.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.686+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.686+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:32.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.690+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:30:32.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.690+0000 7fa8b1d34980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.4889494792448771493', time=2024-05-30T18:27:58.148051+0000, chain.objs.size()=11 2024-05-30T16:30:32.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.690+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~ZWTDOqEMrHDEgw5CTKW1D26jwcPcOaw.1 2024-05-30T16:30:32.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.690+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~ZWTDOqEMrHDEgw5CTKW1D26jwcPcOaw.2 2024-05-30T16:30:32.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.690+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~ZWTDOqEMrHDEgw5CTKW1D26jwcPcOaw.3 2024-05-30T16:30:32.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.690+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~ZWTDOqEMrHDEgw5CTKW1D26jwcPcOaw.4 2024-05-30T16:30:32.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.690+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~ZWTDOqEMrHDEgw5CTKW1D26jwcPcOaw.5 2024-05-30T16:30:32.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.690+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~ZWTDOqEMrHDEgw5CTKW1D26jwcPcOaw.6 2024-05-30T16:30:32.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.690+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~ZWTDOqEMrHDEgw5CTKW1D26jwcPcOaw.7 2024-05-30T16:30:32.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.690+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~ZWTDOqEMrHDEgw5CTKW1D26jwcPcOaw.8 2024-05-30T16:30:32.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.690+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~ZWTDOqEMrHDEgw5CTKW1D26jwcPcOaw.9 2024-05-30T16:30:32.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.690+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~ZWTDOqEMrHDEgw5CTKW1D26jwcPcOaw.10 2024-05-30T16:30:32.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.690+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~ZWTDOqEMrHDEgw5CTKW1D26jwcPcOaw.11 2024-05-30T16:30:32.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.702+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:30:32.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.710+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-05-30T16:30:32.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.714+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.718+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:32.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.718+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.718+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-05-30T16:30:32.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.722+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.726+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:32.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.726+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:30:32.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.726+0000 7fa8b1d34980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12170881538171695698', time=2024-05-30T18:26:07.041321+0000, chain.objs.size()=5 2024-05-30T16:30:32.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.726+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.41__multipart_key1.2~Mmr5KI2Hmyq8s2qCGVqGLY7x0ddSvzN.1 2024-05-30T16:30:32.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.726+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.41__multipart_key1.2~Mmr5KI2Hmyq8s2qCGVqGLY7x0ddSvzN.2 2024-05-30T16:30:32.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.726+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.41__multipart_key1.2~Mmr5KI2Hmyq8s2qCGVqGLY7x0ddSvzN.3 2024-05-30T16:30:32.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.726+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.41__multipart_key1.2~Mmr5KI2Hmyq8s2qCGVqGLY7x0ddSvzN.4 2024-05-30T16:30:32.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.726+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.41__multipart_key1.2~Mmr5KI2Hmyq8s2qCGVqGLY7x0ddSvzN.5 2024-05-30T16:30:32.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.734+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:30:32.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.738+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-05-30T16:30:32.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.742+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.742+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:32.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.742+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.746+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-05-30T16:30:32.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.750+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.750+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:32.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.750+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-30T16:30:32.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.750+0000 7fa8b1d34980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.1093381973475649779', time=2024-05-30T18:25:48.257315+0000, chain.objs.size()=5 2024-05-30T16:30:32.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.750+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.37__multipart_key1.2~y-a5GtIk_i_j8jKIsfbYNWMLFEh9Y2E.1 2024-05-30T16:30:32.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.750+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.37__shadow_key1.2~y-a5GtIk_i_j8jKIsfbYNWMLFEh9Y2E.1_1 2024-05-30T16:30:32.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.750+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.37__multipart_key1.2~y-a5GtIk_i_j8jKIsfbYNWMLFEh9Y2E.2 2024-05-30T16:30:32.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.750+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.37__shadow_key1.2~y-a5GtIk_i_j8jKIsfbYNWMLFEh9Y2E.2_1 2024-05-30T16:30:32.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.750+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.37__multipart_key1.2~y-a5GtIk_i_j8jKIsfbYNWMLFEh9Y2E.3 2024-05-30T16:30:32.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.750+0000 7fa8b1d34980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.7906571291034473865', time=2024-05-30T18:26:55.455555+0000, chain.objs.size()=5 2024-05-30T16:30:32.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.750+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.43__multipart_file.mpg.2~xpcMYTkQ8JGG3XwhhMvBcnAeBYoOL4n.1 2024-05-30T16:30:32.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.750+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.43__multipart_file.mpg.2~xpcMYTkQ8JGG3XwhhMvBcnAeBYoOL4n.2 2024-05-30T16:30:32.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.750+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.43__multipart_file.mpg.2~xpcMYTkQ8JGG3XwhhMvBcnAeBYoOL4n.3 2024-05-30T16:30:32.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.750+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.43__multipart_file.mpg.2~xpcMYTkQ8JGG3XwhhMvBcnAeBYoOL4n.4 2024-05-30T16:30:32.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.750+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.43__multipart_file.mpg.2~xpcMYTkQ8JGG3XwhhMvBcnAeBYoOL4n.5 2024-05-30T16:30:32.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.762+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:30:32.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.766+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-05-30T16:30:32.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.770+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.770+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:32.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.770+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.770+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-05-30T16:30:32.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.774+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.774+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:32.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.774+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.778+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-05-30T16:30:32.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.778+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.782+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:32.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.782+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:30:32.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.782+0000 7fa8b1d34980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.13054747635761116636', time=2024-05-30T18:25:54.144087+0000, chain.objs.size()=5 2024-05-30T16:30:32.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.782+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.37__multipart_key1.2~nCdvZGjOM8W12rKjI8oE4lZbedX95mV.1 2024-05-30T16:30:32.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.782+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.37__shadow_key1.2~nCdvZGjOM8W12rKjI8oE4lZbedX95mV.1_1 2024-05-30T16:30:32.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.782+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.37__multipart_key1.2~nCdvZGjOM8W12rKjI8oE4lZbedX95mV.2 2024-05-30T16:30:32.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.782+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.37__shadow_key1.2~nCdvZGjOM8W12rKjI8oE4lZbedX95mV.2_1 2024-05-30T16:30:32.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.782+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.37__multipart_key1.2~nCdvZGjOM8W12rKjI8oE4lZbedX95mV.3 2024-05-30T16:30:32.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.786+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:30:32.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.790+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-05-30T16:30:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.794+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.794+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:32.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.798+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-05-30T16:30:32.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.798+0000 7fa8b1d34980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.5167496796413348345', time=2024-05-30T18:26:55.636763+0000, chain.objs.size()=5 2024-05-30T16:30:32.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.798+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.43__multipart_file2.mpg.2~ub-4ORIYdV9LjTsn-_xzlED8XQQza9N.1 2024-05-30T16:30:32.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.798+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.43__multipart_file2.mpg.2~ub-4ORIYdV9LjTsn-_xzlED8XQQza9N.2 2024-05-30T16:30:32.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.798+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.43__multipart_file2.mpg.2~ub-4ORIYdV9LjTsn-_xzlED8XQQza9N.3 2024-05-30T16:30:32.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.798+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.43__multipart_file2.mpg.2~ub-4ORIYdV9LjTsn-_xzlED8XQQza9N.4 2024-05-30T16:30:32.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.798+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.43__multipart_file2.mpg.2~ub-4ORIYdV9LjTsn-_xzlED8XQQza9N.5 2024-05-30T16:30:32.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.798+0000 7fa8b1d34980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.3509709340113275486', time=2024-05-30T18:29:31.777997+0000, chain.objs.size()=5 2024-05-30T16:30:32.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.798+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.47__multipart_key1.2~fHq5EswtIzXlzBVEmr4cBEAQPob0snV.1 2024-05-30T16:30:32.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.798+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.47__multipart_key1.2~fHq5EswtIzXlzBVEmr4cBEAQPob0snV.2 2024-05-30T16:30:32.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.798+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.47__multipart_key1.2~fHq5EswtIzXlzBVEmr4cBEAQPob0snV.3 2024-05-30T16:30:32.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.798+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.47__multipart_key1.2~fHq5EswtIzXlzBVEmr4cBEAQPob0snV.4 2024-05-30T16:30:32.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.798+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.47__multipart_key1.2~fHq5EswtIzXlzBVEmr4cBEAQPob0snV.5 2024-05-30T16:30:32.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.798+0000 7fa8b1d34980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.7513511218614654436', time=2024-05-30T18:30:31.073333+0000, chain.objs.size()=11 2024-05-30T16:30:32.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.798+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.56__multipart_key1.2~L0z3o5uLwt5dGWN3BTNQE4c88OF82OS.1 2024-05-30T16:30:32.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.798+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.56__multipart_key1.2~L0z3o5uLwt5dGWN3BTNQE4c88OF82OS.2 2024-05-30T16:30:32.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.802+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.56__multipart_key1.2~L0z3o5uLwt5dGWN3BTNQE4c88OF82OS.3 2024-05-30T16:30:32.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.806+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.56__multipart_key1.2~L0z3o5uLwt5dGWN3BTNQE4c88OF82OS.4 2024-05-30T16:30:32.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.806+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.56__multipart_key1.2~L0z3o5uLwt5dGWN3BTNQE4c88OF82OS.5 2024-05-30T16:30:32.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.810+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.56__multipart_key1.2~L0z3o5uLwt5dGWN3BTNQE4c88OF82OS.6 2024-05-30T16:30:32.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.810+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.56__multipart_key1.2~L0z3o5uLwt5dGWN3BTNQE4c88OF82OS.7 2024-05-30T16:30:32.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.810+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.56__multipart_key1.2~L0z3o5uLwt5dGWN3BTNQE4c88OF82OS.8 2024-05-30T16:30:32.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.810+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.56__multipart_key1.2~L0z3o5uLwt5dGWN3BTNQE4c88OF82OS.9 2024-05-30T16:30:32.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.810+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.56__multipart_key1.2~L0z3o5uLwt5dGWN3BTNQE4c88OF82OS.10 2024-05-30T16:30:32.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.810+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.56__multipart_key1.2~L0z3o5uLwt5dGWN3BTNQE4c88OF82OS.11 2024-05-30T16:30:32.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.822+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:30:32.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.826+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-05-30T16:30:32.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.826+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.830+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:32.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.830+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.830+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-05-30T16:30:32.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.834+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.834+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:32.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.834+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.834+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-05-30T16:30:32.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.838+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.838+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:32.841 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.838+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.842+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-05-30T16:30:32.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.846+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.846+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:32.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.846+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=6, truncated=0, next_marker='' 2024-05-30T16:30:32.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.846+0000 7fa8b1d34980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.14767793927323527148', time=2024-05-30T18:25:19.816870+0000, chain.objs.size()=5 2024-05-30T16:30:32.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.846+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.37__multipart_key1.2~-6Qnd1fK4Aa50yb2fhPqtmapEKhTi3r.1 2024-05-30T16:30:32.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.846+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.37__multipart_key1.2~-6Qnd1fK4Aa50yb2fhPqtmapEKhTi3r.2 2024-05-30T16:30:32.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.846+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.37__multipart_key1.2~-6Qnd1fK4Aa50yb2fhPqtmapEKhTi3r.3 2024-05-30T16:30:32.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.846+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.37__multipart_key1.2~-6Qnd1fK4Aa50yb2fhPqtmapEKhTi3r.4 2024-05-30T16:30:32.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.846+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.37__multipart_key1.2~-6Qnd1fK4Aa50yb2fhPqtmapEKhTi3r.5 2024-05-30T16:30:32.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.846+0000 7fa8b1d34980 20 garbage collection: RGWGC::process iterating over entry tag='2~aJag-a5et3cuARdhC1HcsttPCcGktfN', time=2024-05-30T18:30:00.585090+0000, chain.objs.size()=1 2024-05-30T16:30:32.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.846+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.16 2024-05-30T16:30:32.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.846+0000 7fa8b1d34980 20 garbage collection: RGWGC::process iterating over entry tag='2~aJag-a5et3cuARdhC1HcsttPCcGktfN', time=2024-05-30T18:30:00.790276+0000, chain.objs.size()=1 2024-05-30T16:30:32.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.846+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.18 2024-05-30T16:30:32.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.846+0000 7fa8b1d34980 20 garbage collection: RGWGC::process iterating over entry tag='2~aJag-a5et3cuARdhC1HcsttPCcGktfN', time=2024-05-30T18:30:01.089414+0000, chain.objs.size()=1 2024-05-30T16:30:32.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.846+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.19 2024-05-30T16:30:32.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.846+0000 7fa8b1d34980 20 garbage collection: RGWGC::process iterating over entry tag='2~aJag-a5et3cuARdhC1HcsttPCcGktfN', time=2024-05-30T18:30:01.359231+0000, chain.objs.size()=1 2024-05-30T16:30:32.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.846+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.21 2024-05-30T16:30:32.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.846+0000 7fa8b1d34980 20 garbage collection: RGWGC::process iterating over entry tag='2~aJag-a5et3cuARdhC1HcsttPCcGktfN', time=2024-05-30T18:30:02.063935+0000, chain.objs.size()=73 2024-05-30T16:30:32.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.846+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.1 2024-05-30T16:30:32.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.846+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.2 2024-05-30T16:30:32.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.846+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.3 2024-05-30T16:30:32.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.854+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.4 2024-05-30T16:30:32.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.854+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.5 2024-05-30T16:30:32.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.854+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.6 2024-05-30T16:30:32.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.854+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.7 2024-05-30T16:30:32.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.854+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.8 2024-05-30T16:30:32.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.854+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.9 2024-05-30T16:30:32.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.858+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.10 2024-05-30T16:30:32.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.858+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.11 2024-05-30T16:30:32.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.858+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.12 2024-05-30T16:30:32.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.858+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.13 2024-05-30T16:30:32.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.858+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.14 2024-05-30T16:30:32.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.862+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.15 2024-05-30T16:30:32.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.862+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.W5bHPm9PVfBi6M6bNXOyleUoT0CBvb15.16 2024-05-30T16:30:32.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.862+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.17 2024-05-30T16:30:32.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.862+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.cc_IWyZWl0HUmrGfbUU6nQxcIimc_sBv.18 2024-05-30T16:30:32.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.862+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.W4oCrhobgcVb8xsVjRCy5zCxf5Ntlbak.19 2024-05-30T16:30:32.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.862+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.20 2024-05-30T16:30:32.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.862+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1._awjpDNza8telmWjUDwDd1JmaLRUv0iM.21 2024-05-30T16:30:32.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.862+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.22 2024-05-30T16:30:32.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.862+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.23 2024-05-30T16:30:32.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.862+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.24 2024-05-30T16:30:32.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.862+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.25 2024-05-30T16:30:32.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.866+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.26 2024-05-30T16:30:32.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.866+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.27 2024-05-30T16:30:32.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.866+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.28 2024-05-30T16:30:32.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.866+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.29 2024-05-30T16:30:32.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.870+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.30 2024-05-30T16:30:32.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.870+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.31 2024-05-30T16:30:32.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.870+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.32 2024-05-30T16:30:32.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.870+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.33 2024-05-30T16:30:32.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.870+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.34 2024-05-30T16:30:32.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.874+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.35 2024-05-30T16:30:32.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.874+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.36 2024-05-30T16:30:32.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.874+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.37 2024-05-30T16:30:32.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.874+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.38 2024-05-30T16:30:32.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.874+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.39 2024-05-30T16:30:32.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.874+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.40 2024-05-30T16:30:32.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.874+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.41 2024-05-30T16:30:32.878 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.878+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.42 2024-05-30T16:30:32.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.878+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.43 2024-05-30T16:30:32.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.878+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.44 2024-05-30T16:30:32.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.878+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.45 2024-05-30T16:30:32.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.878+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.46 2024-05-30T16:30:32.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.878+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.47 2024-05-30T16:30:32.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.882+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.48 2024-05-30T16:30:32.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.882+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.49 2024-05-30T16:30:32.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.882+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.50 2024-05-30T16:30:32.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.882+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.51 2024-05-30T16:30:32.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.882+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.52 2024-05-30T16:30:32.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.882+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.53 2024-05-30T16:30:32.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.882+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.54 2024-05-30T16:30:32.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.886+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.55 2024-05-30T16:30:32.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.886+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.56 2024-05-30T16:30:32.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.886+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.57 2024-05-30T16:30:32.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.886+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.58 2024-05-30T16:30:32.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.886+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.59 2024-05-30T16:30:32.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.886+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.60 2024-05-30T16:30:32.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.886+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.61 2024-05-30T16:30:32.889 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.886+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.62 2024-05-30T16:30:32.889 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.886+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.63 2024-05-30T16:30:32.889 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.886+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.64 2024-05-30T16:30:32.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.890+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.65 2024-05-30T16:30:32.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.890+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.66 2024-05-30T16:30:32.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.890+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.67 2024-05-30T16:30:32.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.890+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.68 2024-05-30T16:30:32.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.890+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.69 2024-05-30T16:30:32.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.890+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.70 2024-05-30T16:30:32.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.894+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.71 2024-05-30T16:30:32.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.894+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.72 2024-05-30T16:30:32.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.894+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.48__multipart_key1.2~aJag-a5et3cuARdhC1HcsttPCcGktfN.73 2024-05-30T16:30:32.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.898+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:30:32.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.898+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-05-30T16:30:32.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.902+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.902+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:32.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.902+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.902+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-05-30T16:30:32.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.906+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.906+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:32.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.906+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.906+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-05-30T16:30:32.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.910+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.910+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:32.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.910+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:30:32.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.910+0000 7fa8b1d34980 20 garbage collection: RGWGC::process iterating over entry tag='2~D270_4iY00AbPc2DwDBZL8yPGLU25k9', time=2024-05-30T18:24:22.458457+0000, chain.objs.size()=5 2024-05-30T16:30:32.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.910+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.30__multipart_key1.2~D270_4iY00AbPc2DwDBZL8yPGLU25k9.1 2024-05-30T16:30:32.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.910+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.30__multipart_key1.2~D270_4iY00AbPc2DwDBZL8yPGLU25k9.2 2024-05-30T16:30:32.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.910+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.30__multipart_key1.2~D270_4iY00AbPc2DwDBZL8yPGLU25k9.3 2024-05-30T16:30:32.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.910+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.30__multipart_key1.2~D270_4iY00AbPc2DwDBZL8yPGLU25k9.4 2024-05-30T16:30:32.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.910+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.30__multipart_key1.2~D270_4iY00AbPc2DwDBZL8yPGLU25k9.5 2024-05-30T16:30:32.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.914+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:30:32.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.918+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-05-30T16:30:32.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.918+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.918+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:32.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.918+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:30:32.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.918+0000 7fa8b1d34980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18438259295124495619', time=2024-05-30T18:29:11.812621+0000, chain.objs.size()=11 2024-05-30T16:30:32.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.918+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~pBk0XEHVo-t5jep_Kf525_IpBFaUgCq.1 2024-05-30T16:30:32.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.918+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__shadow_key1.2~pBk0XEHVo-t5jep_Kf525_IpBFaUgCq.1_1 2024-05-30T16:30:32.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.918+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~pBk0XEHVo-t5jep_Kf525_IpBFaUgCq.2 2024-05-30T16:30:32.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.918+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__shadow_key1.2~pBk0XEHVo-t5jep_Kf525_IpBFaUgCq.2_1 2024-05-30T16:30:32.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.918+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~pBk0XEHVo-t5jep_Kf525_IpBFaUgCq.3 2024-05-30T16:30:32.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.918+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__shadow_key1.2~pBk0XEHVo-t5jep_Kf525_IpBFaUgCq.3_1 2024-05-30T16:30:32.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.918+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~pBk0XEHVo-t5jep_Kf525_IpBFaUgCq.4 2024-05-30T16:30:32.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.918+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__shadow_key1.2~pBk0XEHVo-t5jep_Kf525_IpBFaUgCq.4_1 2024-05-30T16:30:32.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.918+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~pBk0XEHVo-t5jep_Kf525_IpBFaUgCq.5 2024-05-30T16:30:32.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.918+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__shadow_key1.2~pBk0XEHVo-t5jep_Kf525_IpBFaUgCq.5_1 2024-05-30T16:30:32.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.918+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~pBk0XEHVo-t5jep_Kf525_IpBFaUgCq.6 2024-05-30T16:30:32.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.926+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:30:32.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.930+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-05-30T16:30:32.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.930+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.930+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:32.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.930+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:30:32.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.930+0000 7fa8b1d34980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.420275214664824590', time=2024-05-30T18:24:47.563607+0000, chain.objs.size()=5 2024-05-30T16:30:32.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.930+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.34__multipart_key1.2~M5ep7aTUHWL6BbgHFBaXJAsUYnhweED.1 2024-05-30T16:30:32.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.930+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.34__multipart_key1.2~M5ep7aTUHWL6BbgHFBaXJAsUYnhweED.2 2024-05-30T16:30:32.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.930+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.34__multipart_key1.2~M5ep7aTUHWL6BbgHFBaXJAsUYnhweED.3 2024-05-30T16:30:32.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.930+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.34__multipart_key1.2~M5ep7aTUHWL6BbgHFBaXJAsUYnhweED.4 2024-05-30T16:30:32.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.930+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.34__multipart_key1.2~M5ep7aTUHWL6BbgHFBaXJAsUYnhweED.5 2024-05-30T16:30:32.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.934+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:30:32.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.934+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-05-30T16:30:32.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.938+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.938+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:32.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.938+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:30:32.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.938+0000 7fa8b1d34980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.10067742742788154993', time=2024-05-30T18:25:42.132215+0000, chain.objs.size()=5 2024-05-30T16:30:32.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.938+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.37__multipart_key1.2~Q-qGPYfbWIf8x2J8j626RiZE8274mjU.1 2024-05-30T16:30:32.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.938+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.37__multipart_key1.2~Q-qGPYfbWIf8x2J8j626RiZE8274mjU.2 2024-05-30T16:30:32.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.938+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.37__multipart_key1.2~Q-qGPYfbWIf8x2J8j626RiZE8274mjU.3 2024-05-30T16:30:32.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.938+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.37__multipart_key1.2~Q-qGPYfbWIf8x2J8j626RiZE8274mjU.4 2024-05-30T16:30:32.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.938+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.37__multipart_key1.2~Q-qGPYfbWIf8x2J8j626RiZE8274mjU.5 2024-05-30T16:30:32.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.942+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:30:32.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.946+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-05-30T16:30:32.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.946+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.946+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:32.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.946+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.950+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-05-30T16:30:32.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.950+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.950+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:32.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.950+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-30T16:30:32.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.950+0000 7fa8b1d34980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11907595369109710637', time=2024-05-30T18:25:55.235595+0000, chain.objs.size()=4 2024-05-30T16:30:32.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.950+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.36__shadow_.aiAPwD7q4sBusoqrAL1SB6PqK1t4iOO_1 2024-05-30T16:30:32.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.950+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.36__shadow_.aiAPwD7q4sBusoqrAL1SB6PqK1t4iOO_2 2024-05-30T16:30:32.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.950+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.36__shadow_.aiAPwD7q4sBusoqrAL1SB6PqK1t4iOO_3 2024-05-30T16:30:32.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.950+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.36__shadow_.aiAPwD7q4sBusoqrAL1SB6PqK1t4iOO_4 2024-05-30T16:30:32.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.950+0000 7fa8b1d34980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.8767535245486438325', time=2024-05-30T18:26:18.641037+0000, chain.objs.size()=5 2024-05-30T16:30:32.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.950+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.42__multipart_key1.2~Cu4ZCG_bZsASpvW7jof922mYScjy23-.1 2024-05-30T16:30:32.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.950+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.42__multipart_key1.2~Cu4ZCG_bZsASpvW7jof922mYScjy23-.2 2024-05-30T16:30:32.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.950+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.42__multipart_key1.2~Cu4ZCG_bZsASpvW7jof922mYScjy23-.3 2024-05-30T16:30:32.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.950+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.42__multipart_key1.2~Cu4ZCG_bZsASpvW7jof922mYScjy23-.4 2024-05-30T16:30:32.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.950+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.42__multipart_key1.2~Cu4ZCG_bZsASpvW7jof922mYScjy23-.5 2024-05-30T16:30:32.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.962+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:30:32.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.962+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-05-30T16:30:32.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.966+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.966+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:32.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.966+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-30T16:30:32.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.966+0000 7fa8b1d34980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.4038390306784663497', time=2024-05-30T18:27:40.800385+0000, chain.objs.size()=11 2024-05-30T16:30:32.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.966+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~kLSLL1s4n6EnxVQnXkXakZ_5rl8T_Ik.1 2024-05-30T16:30:32.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.966+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~kLSLL1s4n6EnxVQnXkXakZ_5rl8T_Ik.2 2024-05-30T16:30:32.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.966+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~kLSLL1s4n6EnxVQnXkXakZ_5rl8T_Ik.3 2024-05-30T16:30:32.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.966+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~kLSLL1s4n6EnxVQnXkXakZ_5rl8T_Ik.4 2024-05-30T16:30:32.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.966+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~kLSLL1s4n6EnxVQnXkXakZ_5rl8T_Ik.5 2024-05-30T16:30:32.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.966+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~kLSLL1s4n6EnxVQnXkXakZ_5rl8T_Ik.6 2024-05-30T16:30:32.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.966+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~kLSLL1s4n6EnxVQnXkXakZ_5rl8T_Ik.7 2024-05-30T16:30:32.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.966+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~kLSLL1s4n6EnxVQnXkXakZ_5rl8T_Ik.8 2024-05-30T16:30:32.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.966+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~kLSLL1s4n6EnxVQnXkXakZ_5rl8T_Ik.9 2024-05-30T16:30:32.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.966+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~kLSLL1s4n6EnxVQnXkXakZ_5rl8T_Ik.10 2024-05-30T16:30:32.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.966+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~kLSLL1s4n6EnxVQnXkXakZ_5rl8T_Ik.11 2024-05-30T16:30:32.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.966+0000 7fa8b1d34980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18344468711789289161', time=2024-05-30T18:28:54.775346+0000, chain.objs.size()=11 2024-05-30T16:30:32.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.966+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~KVUm07rMOubYP8PV-W4PrZl-r-uWaTY.1 2024-05-30T16:30:32.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.970+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__shadow_key1.2~KVUm07rMOubYP8PV-W4PrZl-r-uWaTY.1_1 2024-05-30T16:30:32.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.970+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~KVUm07rMOubYP8PV-W4PrZl-r-uWaTY.2 2024-05-30T16:30:32.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.970+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__shadow_key1.2~KVUm07rMOubYP8PV-W4PrZl-r-uWaTY.2_1 2024-05-30T16:30:32.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.970+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~KVUm07rMOubYP8PV-W4PrZl-r-uWaTY.3 2024-05-30T16:30:32.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.970+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__shadow_key1.2~KVUm07rMOubYP8PV-W4PrZl-r-uWaTY.3_1 2024-05-30T16:30:32.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.970+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~KVUm07rMOubYP8PV-W4PrZl-r-uWaTY.4 2024-05-30T16:30:32.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.970+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__shadow_key1.2~KVUm07rMOubYP8PV-W4PrZl-r-uWaTY.4_1 2024-05-30T16:30:32.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.970+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~KVUm07rMOubYP8PV-W4PrZl-r-uWaTY.5 2024-05-30T16:30:32.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.970+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__shadow_key1.2~KVUm07rMOubYP8PV-W4PrZl-r-uWaTY.5_1 2024-05-30T16:30:32.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.974+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~KVUm07rMOubYP8PV-W4PrZl-r-uWaTY.6 2024-05-30T16:30:32.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.982+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:30:32.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.982+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-05-30T16:30:32.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.986+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.986+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:32.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.986+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:30:32.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.986+0000 7fa8b1d34980 20 garbage collection: RGWGC::process iterating over entry tag='2~RHwNayf-C3esRfeGvUqKjI9mhjES4dx', time=2024-05-30T18:27:05.854744+0000, chain.objs.size()=3 2024-05-30T16:30:32.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.986+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.44__multipart_key1.2~RHwNayf-C3esRfeGvUqKjI9mhjES4dx.7 2024-05-30T16:30:32.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.986+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.44__multipart_key1.2~RHwNayf-C3esRfeGvUqKjI9mhjES4dx.10 2024-05-30T16:30:32.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.986+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.44__multipart_key1.2~RHwNayf-C3esRfeGvUqKjI9mhjES4dx.13 2024-05-30T16:30:32.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.990+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:30:32.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.994+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-05-30T16:30:32.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.994+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.994+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:32.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.994+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.994+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-05-30T16:30:32.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.998+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:32.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.998+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.998+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-30T16:30:33.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.998+0000 7fa8b1d34980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.3657420064910864404', time=2024-05-30T18:25:54.520956+0000, chain.objs.size()=5 2024-05-30T16:30:33.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.998+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.37__multipart_key1.2~1VmDiT6nW3hMCAPOhhYqxihimh5JWtn.1 2024-05-30T16:30:33.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.998+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.37__shadow_key1.2~1VmDiT6nW3hMCAPOhhYqxihimh5JWtn.1_1 2024-05-30T16:30:33.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.998+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.37__multipart_key1.2~1VmDiT6nW3hMCAPOhhYqxihimh5JWtn.2 2024-05-30T16:30:33.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.998+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.37__shadow_key1.2~1VmDiT6nW3hMCAPOhhYqxihimh5JWtn.2_1 2024-05-30T16:30:33.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.998+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.37__multipart_key1.2~1VmDiT6nW3hMCAPOhhYqxihimh5JWtn.3 2024-05-30T16:30:33.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.998+0000 7fa8b1d34980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.6516804909432983361', time=2024-05-30T18:28:15.462494+0000, chain.objs.size()=10 2024-05-30T16:30:33.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.998+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~fdXFkltp-VPsaCpCW7a_SoZEn2zhZeJ.1 2024-05-30T16:30:33.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.998+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~fdXFkltp-VPsaCpCW7a_SoZEn2zhZeJ.2 2024-05-30T16:30:33.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.998+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~fdXFkltp-VPsaCpCW7a_SoZEn2zhZeJ.3 2024-05-30T16:30:33.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.998+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~fdXFkltp-VPsaCpCW7a_SoZEn2zhZeJ.4 2024-05-30T16:30:33.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.998+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~fdXFkltp-VPsaCpCW7a_SoZEn2zhZeJ.5 2024-05-30T16:30:33.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.998+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~fdXFkltp-VPsaCpCW7a_SoZEn2zhZeJ.6 2024-05-30T16:30:33.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:32.998+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~fdXFkltp-VPsaCpCW7a_SoZEn2zhZeJ.7 2024-05-30T16:30:33.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.002+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~fdXFkltp-VPsaCpCW7a_SoZEn2zhZeJ.8 2024-05-30T16:30:33.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.002+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~fdXFkltp-VPsaCpCW7a_SoZEn2zhZeJ.9 2024-05-30T16:30:33.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.006+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~fdXFkltp-VPsaCpCW7a_SoZEn2zhZeJ.10 2024-05-30T16:30:33.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.010+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:30:33.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.014+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-05-30T16:30:33.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.014+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.018+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.018+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.018+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-05-30T16:30:33.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.018+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.018+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.022+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-05-30T16:30:33.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.022+0000 7fa8b1d34980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.8243674962062241786', time=2024-05-30T18:24:35.700225+0000, chain.objs.size()=5 2024-05-30T16:30:33.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.022+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.32__multipart_key1.2~jpvoqiHKsy4JnovisgWYd8_lRBChkdE.1 2024-05-30T16:30:33.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.022+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.32__multipart_key1.2~jpvoqiHKsy4JnovisgWYd8_lRBChkdE.2 2024-05-30T16:30:33.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.022+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.32__multipart_key1.2~jpvoqiHKsy4JnovisgWYd8_lRBChkdE.3 2024-05-30T16:30:33.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.022+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.32__multipart_key1.2~jpvoqiHKsy4JnovisgWYd8_lRBChkdE.4 2024-05-30T16:30:33.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.022+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.32__multipart_key1.2~jpvoqiHKsy4JnovisgWYd8_lRBChkdE.5 2024-05-30T16:30:33.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.022+0000 7fa8b1d34980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.3239939832823289639', time=2024-05-30T18:29:11.453498+0000, chain.objs.size()=11 2024-05-30T16:30:33.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.022+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~mpTV2ifkyLvYws7KRnQlqUohzOWl39X.1 2024-05-30T16:30:33.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.022+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__shadow_key1.2~mpTV2ifkyLvYws7KRnQlqUohzOWl39X.1_1 2024-05-30T16:30:33.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.022+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~mpTV2ifkyLvYws7KRnQlqUohzOWl39X.2 2024-05-30T16:30:33.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.022+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__shadow_key1.2~mpTV2ifkyLvYws7KRnQlqUohzOWl39X.2_1 2024-05-30T16:30:33.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.022+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~mpTV2ifkyLvYws7KRnQlqUohzOWl39X.3 2024-05-30T16:30:33.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.022+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__shadow_key1.2~mpTV2ifkyLvYws7KRnQlqUohzOWl39X.3_1 2024-05-30T16:30:33.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.022+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~mpTV2ifkyLvYws7KRnQlqUohzOWl39X.4 2024-05-30T16:30:33.026 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.026+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__shadow_key1.2~mpTV2ifkyLvYws7KRnQlqUohzOWl39X.4_1 2024-05-30T16:30:33.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.026+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~mpTV2ifkyLvYws7KRnQlqUohzOWl39X.5 2024-05-30T16:30:33.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.026+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__shadow_key1.2~mpTV2ifkyLvYws7KRnQlqUohzOWl39X.5_1 2024-05-30T16:30:33.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.026+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.45__multipart_key1.2~mpTV2ifkyLvYws7KRnQlqUohzOWl39X.6 2024-05-30T16:30:33.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.026+0000 7fa8b1d34980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.10095110735525402882', time=2024-05-30T18:29:22.599378+0000, chain.objs.size()=5 2024-05-30T16:30:33.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.026+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.46__multipart_key1.2~Y0td1N584mjstOSGMyxEKTXwn_ERgV4.1 2024-05-30T16:30:33.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.026+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.46__multipart_key1.2~Y0td1N584mjstOSGMyxEKTXwn_ERgV4.2 2024-05-30T16:30:33.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.026+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.46__multipart_key1.2~Y0td1N584mjstOSGMyxEKTXwn_ERgV4.3 2024-05-30T16:30:33.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.026+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.46__multipart_key1.2~Y0td1N584mjstOSGMyxEKTXwn_ERgV4.4 2024-05-30T16:30:33.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.026+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.46__multipart_key1.2~Y0td1N584mjstOSGMyxEKTXwn_ERgV4.5 2024-05-30T16:30:33.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.034+0000 7fa8b1d34980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:30:33.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.038+0000 7fa8b1d34980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-05-30T16:30:33.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.042+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.042+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.042+0000 7fa8b1d34980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.046+0000 7fa765fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:30:33.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.050+0000 7fa8b1d34980 20 remove_watcher() i=0 2024-05-30T16:30:33.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.050+0000 7fa8b1d34980 2 removed watcher, disabling cache 2024-05-30T16:30:33.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.050+0000 7fa8b1d34980 20 remove_watcher() i=1 2024-05-30T16:30:33.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.050+0000 7fa8b1d34980 20 remove_watcher() i=2 2024-05-30T16:30:33.055 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.054+0000 7fa8b1d34980 20 remove_watcher() i=3 2024-05-30T16:30:33.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.054+0000 7fa8b1d34980 20 remove_watcher() i=4 2024-05-30T16:30:33.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.054+0000 7fa8b1d34980 20 remove_watcher() i=5 2024-05-30T16:30:33.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.058+0000 7fa8b1d34980 20 remove_watcher() i=6 2024-05-30T16:30:33.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.058+0000 7fa8b1d34980 20 remove_watcher() i=7 2024-05-30T16:30:33.075 DEBUG:teuthology.orchestra.run.smithi006:> radosgw-admin gc process --include-all 2024-05-30T16:30:33.142 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:30:33.143 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:30:33.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.186+0000 7f8323217980 20 rados->read ofs=0 len=0 2024-05-30T16:30:33.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.190+0000 7f8323217980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:33.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.190+0000 7f8323217980 20 realm 2024-05-30T16:30:33.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.190+0000 7f8323217980 20 rados->read ofs=0 len=0 2024-05-30T16:30:33.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.190+0000 7f8323217980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:33.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.190+0000 7f8323217980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:30:33.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.190+0000 7f8323217980 20 rados->read ofs=0 len=0 2024-05-30T16:30:33.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.190+0000 7f8323217980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:33.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.190+0000 7f8323217980 20 rados->read ofs=0 len=0 2024-05-30T16:30:33.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.190+0000 7f8323217980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:30:33.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.190+0000 7f8323217980 20 rados->read ofs=0 len=0 2024-05-30T16:30:33.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.190+0000 7f8323217980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:30:33.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.190+0000 7f8323217980 20 searching for the correct realm 2024-05-30T16:30:33.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.198+0000 7f8320527640 0 -- 172.21.15.6:0/3392686645 >> [v2:172.21.15.6:6819/1141406589,v1:172.21.15.6:6821/1141406589] conn(0x5574e83c3130 msgr2=0x5574e83c55f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:30:33.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.206+0000 7f8320d28640 0 -- 172.21.15.6:0/3392686645 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f83080b7b60 msgr2=0x7f83080fe330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:30:33.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.214+0000 7f8323217980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:30:33.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.214+0000 7f8323217980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:30:33.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.214+0000 7f8323217980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:30:33.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.214+0000 7f8323217980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:30:33.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.214+0000 7f8323217980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:30:33.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.214+0000 7f8323217980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:30:33.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.214+0000 7f8323217980 20 rados->read ofs=0 len=0 2024-05-30T16:30:33.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.214+0000 7f8323217980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:33.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.214+0000 7f8323217980 20 rados->read ofs=0 len=0 2024-05-30T16:30:33.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.214+0000 7f8323217980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:30:33.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.214+0000 7f8323217980 20 rados->read ofs=0 len=0 2024-05-30T16:30:33.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.214+0000 7f8323217980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:30:33.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.214+0000 7f8323217980 20 zone default found 2024-05-30T16:30:33.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.214+0000 7f8323217980 4 Realm: () 2024-05-30T16:30:33.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.214+0000 7f8323217980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:30:33.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.214+0000 7f8323217980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:30:33.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.214+0000 7f8323217980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:30:33.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.214+0000 7f8323217980 20 zonegroup default 2024-05-30T16:30:33.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.214+0000 7f8323217980 20 rados->read ofs=0 len=0 2024-05-30T16:30:33.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.214+0000 7f8323217980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:33.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.214+0000 7f8323217980 20 rados->read ofs=0 len=0 2024-05-30T16:30:33.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.218+0000 7f8323217980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:33.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.218+0000 7f8323217980 20 rados->read ofs=0 len=0 2024-05-30T16:30:33.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.218+0000 7f8323217980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:33.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.218+0000 7f8323217980 20 started sync module instance, tier type = 2024-05-30T16:30:33.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.218+0000 7f8323217980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:30:33.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.234+0000 7f8323217980 20 add_watcher() i=0 2024-05-30T16:30:33.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.234+0000 7f8323217980 20 add_watcher() i=1 2024-05-30T16:30:33.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.234+0000 7f8323217980 20 add_watcher() i=2 2024-05-30T16:30:33.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.234+0000 7f8323217980 20 add_watcher() i=3 2024-05-30T16:30:33.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.234+0000 7f8323217980 20 add_watcher() i=4 2024-05-30T16:30:33.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.234+0000 7f8323217980 20 add_watcher() i=5 2024-05-30T16:30:33.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.234+0000 7f8323217980 20 add_watcher() i=6 2024-05-30T16:30:33.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.234+0000 7f8323217980 20 add_watcher() i=7 2024-05-30T16:30:33.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.234+0000 7f8323217980 2 all 8 watchers are set, enabling cache 2024-05-30T16:30:33.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.238+0000 7f81d6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:30:33.239 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.238+0000 7f8323217980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:30:33.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.238+0000 7f8323217980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:30:33.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.318+0000 7f81d5fe3640 20 reqs_thread_entry: start 2024-05-30T16:30:33.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.318+0000 7f8323217980 20 init_complete bucket index max shards: 11 2024-05-30T16:30:33.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.318+0000 7f8323217980 20 Filter name: none 2024-05-30T16:30:33.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.318+0000 7f81477fe640 20 reqs_thread_entry: start 2024-05-30T16:30:33.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.318+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-05-30T16:30:33.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.322+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.322+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.322+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.322+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-05-30T16:30:33.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.326+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.326+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.326+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.330+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-05-30T16:30:33.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.334+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.336 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.334+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.334+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.338+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-05-30T16:30:33.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.342+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.342+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.342+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.342+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-05-30T16:30:33.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.346+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.346+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.346+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.350+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-05-30T16:30:33.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.354+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.354+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.354+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.360 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.358+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-05-30T16:30:33.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.362+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.362+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.366+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.366+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-05-30T16:30:33.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.370+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.370+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.370+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.374+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-05-30T16:30:33.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.378+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.380 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.378+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.378+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.382+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-05-30T16:30:33.386 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.386+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.386+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.386+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.390+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-05-30T16:30:33.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.390+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.394+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.394+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.398+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-05-30T16:30:33.403 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.402+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.402+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.406+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.406+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-05-30T16:30:33.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.410+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.414+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.414+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.418+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-05-30T16:30:33.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.422+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.422+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.422+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.426+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-05-30T16:30:33.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.430+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.430+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.430+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.434+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-05-30T16:30:33.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.438+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.438+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.438+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.442+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-05-30T16:30:33.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.446+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.450+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.450+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.450+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-05-30T16:30:33.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.458+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.458+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.458+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.462+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-05-30T16:30:33.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.466+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.466+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.470+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.470+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-05-30T16:30:33.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.474+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.478 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.478+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.478+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.478+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-05-30T16:30:33.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.486+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.486+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.486+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.490+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-05-30T16:30:33.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.494+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.498+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.498+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.502+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-05-30T16:30:33.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.506+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.508 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.506+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.510+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.510+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-05-30T16:30:33.517 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.514+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.518+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.519 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.518+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.522+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-05-30T16:30:33.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.522+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.526+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.526+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.526+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-05-30T16:30:33.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.530+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.534+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.534+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.538+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-05-30T16:30:33.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.542+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.542+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.542+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.546+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-05-30T16:30:33.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.550+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.550+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.550+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.554+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-05-30T16:30:33.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.558+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.558+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.558+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.562+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-05-30T16:30:33.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.566+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.570 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.570+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.571 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.570+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.574+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-05-30T16:30:33.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.574+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.578+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.578+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.582+0000 7f8323217980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-05-30T16:30:33.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.586+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.586+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.586+0000 7f8323217980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.594+0000 7f81d6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:30:33.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.598+0000 7f8323217980 20 remove_watcher() i=0 2024-05-30T16:30:33.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.598+0000 7f8323217980 2 removed watcher, disabling cache 2024-05-30T16:30:33.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.602+0000 7f8323217980 20 remove_watcher() i=1 2024-05-30T16:30:33.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.606+0000 7f8323217980 20 remove_watcher() i=2 2024-05-30T16:30:33.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.606+0000 7f8323217980 20 remove_watcher() i=3 2024-05-30T16:30:33.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.610+0000 7f8323217980 20 remove_watcher() i=4 2024-05-30T16:30:33.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.614+0000 7f8323217980 20 remove_watcher() i=5 2024-05-30T16:30:33.617 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.614+0000 7f8323217980 20 remove_watcher() i=6 2024-05-30T16:30:33.619 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.618+0000 7f8323217980 20 remove_watcher() i=7 2024-05-30T16:30:33.642 DEBUG:teuthology.orchestra.run.smithi006:> radosgw-admin gc process --include-all 2024-05-30T16:30:33.717 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:30:33.718 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:30:33.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.762+0000 7f9901810980 20 rados->read ofs=0 len=0 2024-05-30T16:30:33.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.766+0000 7f9901810980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:33.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.766+0000 7f9901810980 20 realm 2024-05-30T16:30:33.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.766+0000 7f9901810980 20 rados->read ofs=0 len=0 2024-05-30T16:30:33.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.766+0000 7f9901810980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:33.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.766+0000 7f9901810980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:30:33.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.766+0000 7f9901810980 20 rados->read ofs=0 len=0 2024-05-30T16:30:33.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.766+0000 7f9901810980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:33.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.766+0000 7f9901810980 20 rados->read ofs=0 len=0 2024-05-30T16:30:33.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.766+0000 7f9901810980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:30:33.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.766+0000 7f9901810980 20 rados->read ofs=0 len=0 2024-05-30T16:30:33.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.766+0000 7f9901810980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:30:33.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.766+0000 7f9901810980 20 searching for the correct realm 2024-05-30T16:30:33.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.774+0000 7f98feb20640 0 -- 172.21.15.6:0/2443317123 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f98f0018070 msgr2=0x7f98f00670e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:30:33.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.782+0000 7f98feb20640 0 -- 172.21.15.6:0/2443317123 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f97c807e510 msgr2=0x7f97c8080940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:30:33.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.790+0000 7f9901810980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:30:33.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.790+0000 7f9901810980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:30:33.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.790+0000 7f9901810980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:30:33.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.790+0000 7f9901810980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:30:33.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.790+0000 7f9901810980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:30:33.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.790+0000 7f9901810980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:30:33.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.790+0000 7f9901810980 20 rados->read ofs=0 len=0 2024-05-30T16:30:33.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.790+0000 7f9901810980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:33.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.790+0000 7f9901810980 20 rados->read ofs=0 len=0 2024-05-30T16:30:33.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.790+0000 7f9901810980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:30:33.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.790+0000 7f9901810980 20 rados->read ofs=0 len=0 2024-05-30T16:30:33.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.790+0000 7f9901810980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:30:33.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.790+0000 7f9901810980 20 zone default found 2024-05-30T16:30:33.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.790+0000 7f9901810980 4 Realm: () 2024-05-30T16:30:33.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.790+0000 7f9901810980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:30:33.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.790+0000 7f9901810980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:30:33.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.790+0000 7f9901810980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:30:33.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.790+0000 7f9901810980 20 zonegroup default 2024-05-30T16:30:33.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.790+0000 7f9901810980 20 rados->read ofs=0 len=0 2024-05-30T16:30:33.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.790+0000 7f9901810980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:33.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.790+0000 7f9901810980 20 rados->read ofs=0 len=0 2024-05-30T16:30:33.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.794+0000 7f9901810980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:33.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.794+0000 7f9901810980 20 rados->read ofs=0 len=0 2024-05-30T16:30:33.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.794+0000 7f9901810980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:33.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.794+0000 7f9901810980 20 started sync module instance, tier type = 2024-05-30T16:30:33.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.794+0000 7f9901810980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:30:33.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.810+0000 7f9901810980 20 add_watcher() i=0 2024-05-30T16:30:33.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.810+0000 7f9901810980 20 add_watcher() i=1 2024-05-30T16:30:33.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.810+0000 7f9901810980 20 add_watcher() i=2 2024-05-30T16:30:33.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.810+0000 7f9901810980 20 add_watcher() i=3 2024-05-30T16:30:33.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.810+0000 7f9901810980 20 add_watcher() i=4 2024-05-30T16:30:33.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.810+0000 7f9901810980 20 add_watcher() i=5 2024-05-30T16:30:33.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.810+0000 7f9901810980 20 add_watcher() i=6 2024-05-30T16:30:33.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.810+0000 7f9901810980 20 add_watcher() i=7 2024-05-30T16:30:33.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.810+0000 7f9901810980 2 all 8 watchers are set, enabling cache 2024-05-30T16:30:33.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.814+0000 7f97b5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:30:33.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.814+0000 7f9901810980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:30:33.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.814+0000 7f9901810980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:30:33.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.894+0000 7f97b4fe1640 20 reqs_thread_entry: start 2024-05-30T16:30:33.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.894+0000 7f9901810980 20 init_complete bucket index max shards: 11 2024-05-30T16:30:33.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.894+0000 7f9901810980 20 Filter name: none 2024-05-30T16:30:33.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.894+0000 7f972e7fc640 20 reqs_thread_entry: start 2024-05-30T16:30:33.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.894+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-05-30T16:30:33.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.898+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.898+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.900 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.898+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.902+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-05-30T16:30:33.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.902+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.906+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.906+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.906+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-05-30T16:30:33.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.910+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.914+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.914+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.918+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-05-30T16:30:33.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.918+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.922+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.922+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.926+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-05-30T16:30:33.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.926+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.930+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.930+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.930+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-05-30T16:30:33.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.934+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.938+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.939 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.938+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.942+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-05-30T16:30:33.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.946+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.946+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.946+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.950+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-05-30T16:30:33.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.954+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.954+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.954+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.954+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-05-30T16:30:33.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.958+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.958+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.958+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.962+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-05-30T16:30:33.967 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.966+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.966+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.966+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.970+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-05-30T16:30:33.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.974+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.974+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.974+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.974+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-05-30T16:30:33.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.978+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.982+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.982+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.986+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-05-30T16:30:33.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.986+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.986+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.989 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.986+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.990+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-05-30T16:30:33.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.990+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.994+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:33.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.994+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:33.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.994+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-05-30T16:30:34.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:33.998+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.002+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.002+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.006+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-05-30T16:30:34.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.006+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.010+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.010+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.010+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-05-30T16:30:34.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.014+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.016 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.014+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.014+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.018+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-05-30T16:30:34.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.018+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.022+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.022+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.022+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-05-30T16:30:34.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.026+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.030+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.030+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.034+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-05-30T16:30:34.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.034+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.038+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.038+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.038+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-05-30T16:30:34.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.042+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.046+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.046+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.050 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.050+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-05-30T16:30:34.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.050+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.054+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.054 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.054+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.054+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-05-30T16:30:34.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.058+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.058+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.062+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.066+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-05-30T16:30:34.070 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.070+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.070+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.073 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.070+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.074+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-05-30T16:30:34.079 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.078+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.082+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.082+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.082+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-05-30T16:30:34.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.086+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.090+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.090+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.094+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-05-30T16:30:34.099 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.098+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.098+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.098+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.102+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-05-30T16:30:34.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.106+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.110+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.110+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.110+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-05-30T16:30:34.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.114+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.118+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.118+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.123 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.122+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-05-30T16:30:34.127 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.126+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.126+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.130 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.130+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.130+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-05-30T16:30:34.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.134+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.138+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.138+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.138+0000 7f9901810980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-05-30T16:30:34.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.142+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.146+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.146+0000 7f9901810980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.150+0000 7f97b5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:30:34.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.154+0000 7f9901810980 20 remove_watcher() i=0 2024-05-30T16:30:34.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.154+0000 7f9901810980 2 removed watcher, disabling cache 2024-05-30T16:30:34.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.158+0000 7f9901810980 20 remove_watcher() i=1 2024-05-30T16:30:34.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.162+0000 7f9901810980 20 remove_watcher() i=2 2024-05-30T16:30:34.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.162+0000 7f9901810980 20 remove_watcher() i=3 2024-05-30T16:30:34.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.166+0000 7f9901810980 20 remove_watcher() i=4 2024-05-30T16:30:34.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.170+0000 7f9901810980 20 remove_watcher() i=5 2024-05-30T16:30:34.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.170+0000 7f9901810980 20 remove_watcher() i=6 2024-05-30T16:30:34.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.174+0000 7f9901810980 20 remove_watcher() i=7 2024-05-30T16:30:34.191 DEBUG:teuthology.orchestra.run.smithi006:> radosgw-admin gc process --include-all 2024-05-30T16:30:34.265 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:30:34.266 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:30:34.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.310+0000 7f1029736980 20 rados->read ofs=0 len=0 2024-05-30T16:30:34.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.314+0000 7f1029736980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:34.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.314+0000 7f1029736980 20 realm 2024-05-30T16:30:34.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.314+0000 7f1029736980 20 rados->read ofs=0 len=0 2024-05-30T16:30:34.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.314+0000 7f1029736980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:34.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.314+0000 7f1029736980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:30:34.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.314+0000 7f1029736980 20 rados->read ofs=0 len=0 2024-05-30T16:30:34.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.314+0000 7f1029736980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:34.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.314+0000 7f1029736980 20 rados->read ofs=0 len=0 2024-05-30T16:30:34.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.314+0000 7f1029736980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:30:34.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.314+0000 7f1029736980 20 rados->read ofs=0 len=0 2024-05-30T16:30:34.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.314+0000 7f1029736980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:30:34.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.314+0000 7f1029736980 20 searching for the correct realm 2024-05-30T16:30:34.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.322+0000 7f101ffff640 0 -- 172.21.15.6:0/769948680 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f101800e070 msgr2=0x7f1018065ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:30:34.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.330+0000 7f1027247640 0 -- 172.21.15.6:0/769948680 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f1010106960 msgr2=0x7f10100e73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:30:34.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.546+0000 7f1029736980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:30:34.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.546+0000 7f1029736980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:30:34.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.546+0000 7f1029736980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:30:34.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.546+0000 7f1029736980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:30:34.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.546+0000 7f1029736980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:30:34.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.546+0000 7f1029736980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:30:34.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.546+0000 7f1029736980 20 rados->read ofs=0 len=0 2024-05-30T16:30:34.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.546+0000 7f1029736980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:34.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.546+0000 7f1029736980 20 rados->read ofs=0 len=0 2024-05-30T16:30:34.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.550+0000 7f1029736980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:30:34.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.550+0000 7f1029736980 20 rados->read ofs=0 len=0 2024-05-30T16:30:34.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.550+0000 7f1029736980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:30:34.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.550+0000 7f1029736980 20 zone default found 2024-05-30T16:30:34.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.550+0000 7f1029736980 4 Realm: () 2024-05-30T16:30:34.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.550+0000 7f1029736980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:30:34.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.550+0000 7f1029736980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:30:34.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.550+0000 7f1029736980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:30:34.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.550+0000 7f1029736980 20 zonegroup default 2024-05-30T16:30:34.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.550+0000 7f1029736980 20 rados->read ofs=0 len=0 2024-05-30T16:30:34.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.550+0000 7f1029736980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:34.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.550+0000 7f1029736980 20 rados->read ofs=0 len=0 2024-05-30T16:30:34.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.550+0000 7f1029736980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:34.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.550+0000 7f1029736980 20 rados->read ofs=0 len=0 2024-05-30T16:30:34.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.550+0000 7f1029736980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:34.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.554+0000 7f1029736980 20 started sync module instance, tier type = 2024-05-30T16:30:34.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.554+0000 7f1029736980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:30:34.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.574+0000 7f1029736980 20 add_watcher() i=0 2024-05-30T16:30:34.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.574+0000 7f1029736980 20 add_watcher() i=1 2024-05-30T16:30:34.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.574+0000 7f1029736980 20 add_watcher() i=2 2024-05-30T16:30:34.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.574+0000 7f1029736980 20 add_watcher() i=3 2024-05-30T16:30:34.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.574+0000 7f1029736980 20 add_watcher() i=4 2024-05-30T16:30:34.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.574+0000 7f1029736980 20 add_watcher() i=5 2024-05-30T16:30:34.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.574+0000 7f1029736980 20 add_watcher() i=6 2024-05-30T16:30:34.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.574+0000 7f1029736980 20 add_watcher() i=7 2024-05-30T16:30:34.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.574+0000 7f1029736980 2 all 8 watchers are set, enabling cache 2024-05-30T16:30:34.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.582+0000 7f0ede7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:30:34.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.582+0000 7f1029736980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:30:34.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.582+0000 7f1029736980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:30:34.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.682+0000 7f0edd7e2640 20 reqs_thread_entry: start 2024-05-30T16:30:34.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.682+0000 7f1029736980 20 init_complete bucket index max shards: 11 2024-05-30T16:30:34.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.682+0000 7f1029736980 20 Filter name: none 2024-05-30T16:30:34.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.682+0000 7f0ed67fc640 20 reqs_thread_entry: start 2024-05-30T16:30:34.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.682+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-05-30T16:30:34.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.686+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.686+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.686+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.690+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-05-30T16:30:34.694 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.694+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.694+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.697 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.694+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.698+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-05-30T16:30:34.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.702+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.702+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.702+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.706+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-05-30T16:30:34.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.710+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.710+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.710+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.710+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-05-30T16:30:34.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.714+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.718+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.718+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.718+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-05-30T16:30:34.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.722+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.726+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.726+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.726+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-05-30T16:30:34.732 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.730+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.730+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.730+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.734+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-05-30T16:30:34.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.738+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.738+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.738+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.742+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-05-30T16:30:34.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.746+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.746+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.746+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.750+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-05-30T16:30:34.754 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.754+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.754+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.754+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.758+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-05-30T16:30:34.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.762+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.762+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.762+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.766+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-05-30T16:30:34.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.770+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.770+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.772 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.770+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.774 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.774+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-05-30T16:30:34.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.774+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.774+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.778+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.778+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-05-30T16:30:34.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.782+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.782+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.782+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.786+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-05-30T16:30:34.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.790+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.790+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.793 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.790+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.795 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.794+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-05-30T16:30:34.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.798+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.798+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.798+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.798+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-05-30T16:30:34.805 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.802+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.806+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.806+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.810+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-05-30T16:30:34.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.814+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.814+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.818+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.818+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-05-30T16:30:34.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.826+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.826+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.826+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.830+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-05-30T16:30:34.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.834+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.834+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.834+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.838+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-05-30T16:30:34.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.842+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.842+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.846+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.846+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-05-30T16:30:34.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.850+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.854+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.854+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.854+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-05-30T16:30:34.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.858+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.862+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.862+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.866+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-05-30T16:30:34.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.870+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.870+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.874+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.874+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-05-30T16:30:34.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.878+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.878+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.878+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.882+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-05-30T16:30:34.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.886+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.889 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.886+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.890+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.890+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-05-30T16:30:34.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.894+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.894+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.898+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.898+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-05-30T16:30:34.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.902+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.902+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.902+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.906+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-05-30T16:30:34.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.910+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.914+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.914+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.918+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-05-30T16:30:34.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.922+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.922+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.926+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.926+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-05-30T16:30:34.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.930+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.934+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.934+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.934+0000 7f1029736980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-05-30T16:30:34.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.938+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.942+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:34.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.942+0000 7f1029736980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:34.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.946+0000 7f0ede7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:30:34.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.954+0000 7f1029736980 20 remove_watcher() i=0 2024-05-30T16:30:34.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.954+0000 7f1029736980 2 removed watcher, disabling cache 2024-05-30T16:30:34.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.954+0000 7f1029736980 20 remove_watcher() i=1 2024-05-30T16:30:34.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.958+0000 7f1029736980 20 remove_watcher() i=2 2024-05-30T16:30:34.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.962+0000 7f1029736980 20 remove_watcher() i=3 2024-05-30T16:30:34.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.962+0000 7f1029736980 20 remove_watcher() i=4 2024-05-30T16:30:34.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.966+0000 7f1029736980 20 remove_watcher() i=5 2024-05-30T16:30:34.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.970+0000 7f1029736980 20 remove_watcher() i=6 2024-05-30T16:30:34.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:34.970+0000 7f1029736980 20 remove_watcher() i=7 2024-05-30T16:30:34.995 DEBUG:teuthology.orchestra.run.smithi006:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean test --rerun-tasks --no-build-cache --tests BucketTest 2024-05-30T16:30:35.709 INFO:teuthology.orchestra.run.smithi006.stdout:> Task :clean 2024-05-30T16:30:36.606 INFO:teuthology.orchestra.run.smithi006.stdout:> Task :compileJava 2024-05-30T16:30:36.606 INFO:teuthology.orchestra.run.smithi006.stdout:> Task :processResources 2024-05-30T16:30:36.606 INFO:teuthology.orchestra.run.smithi006.stdout:> Task :classes 2024-05-30T16:30:37.106 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:37.106 INFO:teuthology.orchestra.run.smithi006.stdout:> Task :compileTestJava 2024-05-30T16:30:37.107 INFO:teuthology.orchestra.run.smithi006.stderr:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2024-05-30T16:30:37.107 INFO:teuthology.orchestra.run.smithi006.stderr:Note: Recompile with -Xlint:deprecation for details. 2024-05-30T16:30:37.108 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:37.108 INFO:teuthology.orchestra.run.smithi006.stdout:> Task :processTestResources NO-SOURCE 2024-05-30T16:30:37.108 INFO:teuthology.orchestra.run.smithi006.stdout:> Task :testClasses 2024-05-30T16:30:39.106 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:39.106 INFO:teuthology.orchestra.run.smithi006.stdout:> Task :test 2024-05-30T16:30:39.107 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:39.107 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Bucket tests STANDARD_OUT 2024-05-30T16:30:39.108 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30 16:30:38,411 [Test worker] WARN com.amazonaws.util.Base64 - JAXB is unavailable. Will fallback to SDK implementation which may be less performant 2024-05-30T16:30:39.108 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:39.108 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectEmpty STARTED 2024-05-30T16:30:39.108 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:39.108 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectEmpty PASSED 2024-05-30T16:30:39.306 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:39.306 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectMismatch STARTED 2024-05-30T16:30:39.406 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:39.406 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectMismatch PASSED 2024-05-30T16:30:40.206 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:40.206 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectUnreadable STARTED 2024-05-30T16:30:40.306 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:40.306 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectUnreadable PASSED 2024-05-30T16:30:41.006 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:41.006 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Bucket tests > BucketTest.testBucketCreateReadDelete STARTED 2024-05-30T16:30:41.606 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:41.606 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Bucket tests > BucketTest.testBucketCreateReadDelete PASSED 2024-05-30T16:30:41.806 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:41.806 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNonEmpty STARTED 2024-05-30T16:30:42.106 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:42.106 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNonEmpty PASSED 2024-05-30T16:30:42.906 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:42.906 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNotExist STARTED 2024-05-30T16:30:43.006 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:43.006 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNotExist PASSED 2024-05-30T16:30:43.206 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:43.206 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Bucket tests > BucketTest.testBucketListDistinct STARTED 2024-05-30T16:30:43.606 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:43.606 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Bucket tests > BucketTest.testBucketListDistinct PASSED 2024-05-30T16:30:44.906 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:44.906 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Bucket tests > BucketTest.testBucketListEmpty STARTED 2024-05-30T16:30:45.106 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:45.106 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Bucket tests > BucketTest.testBucketListEmpty PASSED 2024-05-30T16:30:45.806 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:45.806 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Bucket tests > BucketTest.testBucketNotExist STARTED 2024-05-30T16:30:45.906 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:45.906 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Bucket tests > BucketTest.testBucketNotExist PASSED 2024-05-30T16:30:46.072 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:46.072 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------- 2024-05-30T16:30:46.072 INFO:teuthology.orchestra.run.smithi006.stdout:| Results: SUCCESS (9 tests, 9 successes, 0 failures, 0 skipped) | 2024-05-30T16:30:46.072 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------- 2024-05-30T16:30:46.072 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:46.072 INFO:teuthology.orchestra.run.smithi006.stdout:Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. 2024-05-30T16:30:46.073 INFO:teuthology.orchestra.run.smithi006.stdout:Use '--warning-mode all' to show the individual deprecation warnings. 2024-05-30T16:30:46.073 INFO:teuthology.orchestra.run.smithi006.stdout:See https://docs.gradle.org/6.0.1/userguide/command_line_interface.html#sec:command_line_warnings 2024-05-30T16:30:46.073 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:46.073 INFO:teuthology.orchestra.run.smithi006.stdout:BUILD SUCCESSFUL in 10s 2024-05-30T16:30:46.073 INFO:teuthology.orchestra.run.smithi006.stdout:5 actionable tasks: 5 executed 2024-05-30T16:30:46.080 DEBUG:teuthology.orchestra.run.smithi006:> radosgw-admin gc process --include-all 2024-05-30T16:30:46.106 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:30:46.106 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:30:46.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.150+0000 7f5b0b763980 20 rados->read ofs=0 len=0 2024-05-30T16:30:46.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.150+0000 7f5b0b763980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:46.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.150+0000 7f5b0b763980 20 realm 2024-05-30T16:30:46.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.150+0000 7f5b0b763980 20 rados->read ofs=0 len=0 2024-05-30T16:30:46.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.150+0000 7f5b0b763980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:46.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.150+0000 7f5b0b763980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:30:46.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.150+0000 7f5b0b763980 20 rados->read ofs=0 len=0 2024-05-30T16:30:46.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.150+0000 7f5b0b763980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:46.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.150+0000 7f5b0b763980 20 rados->read ofs=0 len=0 2024-05-30T16:30:46.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.154+0000 7f5b0b763980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:30:46.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.154+0000 7f5b0b763980 20 rados->read ofs=0 len=0 2024-05-30T16:30:46.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.154+0000 7f5b0b763980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:30:46.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.154+0000 7f5b0b763980 20 searching for the correct realm 2024-05-30T16:30:46.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.158+0000 7f5b09a75640 0 -- 172.21.15.6:0/2493923102 >> [v2:172.21.15.204:6823/3156311527,v1:172.21.15.204:6825/3156311527] conn(0x7f5b00036060 msgr2=0x7f5b00061100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:30:46.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.166+0000 7f5b09274640 0 -- 172.21.15.6:0/2493923102 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f5b000660b0 msgr2=0x7f5b000678c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:30:46.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.382+0000 7f5b0b763980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:30:46.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.382+0000 7f5b0b763980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:30:46.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.382+0000 7f5b0b763980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:30:46.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.382+0000 7f5b0b763980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:30:46.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.382+0000 7f5b0b763980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:30:46.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.382+0000 7f5b0b763980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:30:46.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.382+0000 7f5b0b763980 20 rados->read ofs=0 len=0 2024-05-30T16:30:46.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.382+0000 7f5b0b763980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:46.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.382+0000 7f5b0b763980 20 rados->read ofs=0 len=0 2024-05-30T16:30:46.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.386+0000 7f5b0b763980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:30:46.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.386+0000 7f5b0b763980 20 rados->read ofs=0 len=0 2024-05-30T16:30:46.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.386+0000 7f5b0b763980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:30:46.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.386+0000 7f5b0b763980 20 zone default found 2024-05-30T16:30:46.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.386+0000 7f5b0b763980 4 Realm: () 2024-05-30T16:30:46.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.386+0000 7f5b0b763980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:30:46.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.386+0000 7f5b0b763980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:30:46.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.386+0000 7f5b0b763980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:30:46.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.386+0000 7f5b0b763980 20 zonegroup default 2024-05-30T16:30:46.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.386+0000 7f5b0b763980 20 rados->read ofs=0 len=0 2024-05-30T16:30:46.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.390+0000 7f5b0b763980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:46.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.390+0000 7f5b0b763980 20 rados->read ofs=0 len=0 2024-05-30T16:30:46.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.390+0000 7f5b0b763980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:46.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.390+0000 7f5b0b763980 20 rados->read ofs=0 len=0 2024-05-30T16:30:46.392 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.390+0000 7f5b0b763980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:46.392 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.390+0000 7f5b0b763980 20 started sync module instance, tier type = 2024-05-30T16:30:46.392 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.390+0000 7f5b0b763980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:30:46.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.418+0000 7f5b0b763980 20 add_watcher() i=0 2024-05-30T16:30:46.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.418+0000 7f5b0b763980 20 add_watcher() i=1 2024-05-30T16:30:46.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.418+0000 7f5b0b763980 20 add_watcher() i=2 2024-05-30T16:30:46.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.418+0000 7f5b0b763980 20 add_watcher() i=3 2024-05-30T16:30:46.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.418+0000 7f5b0b763980 20 add_watcher() i=4 2024-05-30T16:30:46.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.418+0000 7f5b0b763980 20 add_watcher() i=5 2024-05-30T16:30:46.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.418+0000 7f5b0b763980 20 add_watcher() i=6 2024-05-30T16:30:46.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.418+0000 7f5b0b763980 20 add_watcher() i=7 2024-05-30T16:30:46.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.418+0000 7f5b0b763980 2 all 8 watchers are set, enabling cache 2024-05-30T16:30:46.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.422+0000 7f59bf7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:30:46.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.426+0000 7f5b0b763980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:30:46.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.426+0000 7f5b0b763980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:30:46.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.530+0000 7f59be7e4640 20 reqs_thread_entry: start 2024-05-30T16:30:46.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.530+0000 7f5b0b763980 20 init_complete bucket index max shards: 11 2024-05-30T16:30:46.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.530+0000 7f5b0b763980 20 Filter name: none 2024-05-30T16:30:46.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.530+0000 7f592ffff640 20 reqs_thread_entry: start 2024-05-30T16:30:46.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.534+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-05-30T16:30:46.536 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.534+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.534+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.537 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.534+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.538+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-05-30T16:30:46.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.542+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.542+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.542+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.546+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-05-30T16:30:46.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.550+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.550+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.550+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.550+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-05-30T16:30:46.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.554+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.554+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.554+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.558+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-05-30T16:30:46.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.562+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.562+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.562+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.566+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-05-30T16:30:46.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.570+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.574 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.574+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.574+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.574+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-05-30T16:30:46.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.578+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.578+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.582+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.582+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-05-30T16:30:46.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.586+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.591 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.590+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.592 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.590+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.594+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-05-30T16:30:46.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.598+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.598+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.598+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.602+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-05-30T16:30:46.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.606+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.610 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.610+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.610+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.614+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-05-30T16:30:46.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.618+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.620 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.618+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.621 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.618+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.622+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-05-30T16:30:46.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.626+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.629 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.626+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.630 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.630+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.633 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.630+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-05-30T16:30:46.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.634+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.638+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.638+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.642+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-05-30T16:30:46.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.646+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.646+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.650+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.650+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-05-30T16:30:46.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.654+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.658+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.658+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.662+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-05-30T16:30:46.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.666+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.666+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.666+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.670+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-05-30T16:30:46.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.674+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.676 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.674+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.677 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.674+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.680 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.678+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-05-30T16:30:46.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.682+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.685 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.682+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.686+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.686+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-05-30T16:30:46.693 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.690+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.694+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.694+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.698+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-05-30T16:30:46.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.702+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.702+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.702+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.708 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.706+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-05-30T16:30:46.712 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.710+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.710+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.714+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.714+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-05-30T16:30:46.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.722+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.722+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.722+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.726+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-05-30T16:30:46.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.730+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.730+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.734+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.736 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.734+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-05-30T16:30:46.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.738+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.738+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.742+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.742+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-05-30T16:30:46.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.746+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.750+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.750+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.750+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-05-30T16:30:46.757 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.754+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.758+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.758+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.762 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.762+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-05-30T16:30:46.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.766+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.766+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.768 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.766+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.770+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-05-30T16:30:46.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.774+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.778+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.778+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.782+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-05-30T16:30:46.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.786+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.788 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.786+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.786+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.791 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.790+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-05-30T16:30:46.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.794+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.798+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.798+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.802 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.802+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-05-30T16:30:46.806 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.806+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.806+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.806+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.813 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.810+0000 7f5b0b763980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-05-30T16:30:46.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.814+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.818+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:46.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.818+0000 7f5b0b763980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:46.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.822+0000 7f59bf7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:30:46.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.826+0000 7f5b0b763980 20 remove_watcher() i=0 2024-05-30T16:30:46.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.826+0000 7f5b0b763980 2 removed watcher, disabling cache 2024-05-30T16:30:46.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.830+0000 7f5b0b763980 20 remove_watcher() i=1 2024-05-30T16:30:46.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.834+0000 7f5b0b763980 20 remove_watcher() i=2 2024-05-30T16:30:46.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.838+0000 7f5b0b763980 20 remove_watcher() i=3 2024-05-30T16:30:46.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.842+0000 7f5b0b763980 20 remove_watcher() i=4 2024-05-30T16:30:46.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.842+0000 7f5b0b763980 20 remove_watcher() i=5 2024-05-30T16:30:46.847 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.846+0000 7f5b0b763980 20 remove_watcher() i=6 2024-05-30T16:30:46.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.846+0000 7f5b0b763980 20 remove_watcher() i=7 2024-05-30T16:30:46.871 DEBUG:teuthology.orchestra.run.smithi006:> radosgw-admin gc process --include-all 2024-05-30T16:30:46.946 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:30:46.946 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:30:46.992 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.990+0000 7f09515fb980 20 rados->read ofs=0 len=0 2024-05-30T16:30:46.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.994+0000 7f09515fb980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:46.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.994+0000 7f09515fb980 20 realm 2024-05-30T16:30:46.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.994+0000 7f09515fb980 20 rados->read ofs=0 len=0 2024-05-30T16:30:46.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.994+0000 7f09515fb980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:46.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.994+0000 7f09515fb980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:30:46.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.994+0000 7f09515fb980 20 rados->read ofs=0 len=0 2024-05-30T16:30:46.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.994+0000 7f09515fb980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:46.995 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.994+0000 7f09515fb980 20 rados->read ofs=0 len=0 2024-05-30T16:30:46.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.994+0000 7f09515fb980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:30:46.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.994+0000 7f09515fb980 20 rados->read ofs=0 len=0 2024-05-30T16:30:46.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.994+0000 7f09515fb980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:30:46.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:46.994+0000 7f09515fb980 20 searching for the correct realm 2024-05-30T16:30:47.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.002+0000 7f094e90b640 0 -- 172.21.15.6:0/1039463098 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f09400c2070 msgr2=0x7f09400f69e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:30:47.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.010+0000 7f094e90b640 0 -- 172.21.15.6:0/1039463098 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f081807e4c0 msgr2=0x7f08180808f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:30:47.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.014+0000 7f09515fb980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:30:47.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.014+0000 7f09515fb980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:30:47.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.014+0000 7f09515fb980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:30:47.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.014+0000 7f09515fb980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:30:47.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.014+0000 7f09515fb980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:30:47.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.014+0000 7f09515fb980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:30:47.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.014+0000 7f09515fb980 20 rados->read ofs=0 len=0 2024-05-30T16:30:47.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.018+0000 7f09515fb980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:47.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.018+0000 7f09515fb980 20 rados->read ofs=0 len=0 2024-05-30T16:30:47.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.018+0000 7f09515fb980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:30:47.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.018+0000 7f09515fb980 20 rados->read ofs=0 len=0 2024-05-30T16:30:47.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.018+0000 7f09515fb980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:30:47.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.018+0000 7f09515fb980 20 zone default found 2024-05-30T16:30:47.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.018+0000 7f09515fb980 4 Realm: () 2024-05-30T16:30:47.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.018+0000 7f09515fb980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:30:47.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.018+0000 7f09515fb980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:30:47.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.018+0000 7f09515fb980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:30:47.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.018+0000 7f09515fb980 20 zonegroup default 2024-05-30T16:30:47.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.018+0000 7f09515fb980 20 rados->read ofs=0 len=0 2024-05-30T16:30:47.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.018+0000 7f09515fb980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:47.020 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.018+0000 7f09515fb980 20 rados->read ofs=0 len=0 2024-05-30T16:30:47.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.018+0000 7f09515fb980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:47.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.018+0000 7f09515fb980 20 rados->read ofs=0 len=0 2024-05-30T16:30:47.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.018+0000 7f09515fb980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:47.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.018+0000 7f09515fb980 20 started sync module instance, tier type = 2024-05-30T16:30:47.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.018+0000 7f09515fb980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:30:47.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.042+0000 7f09515fb980 20 add_watcher() i=0 2024-05-30T16:30:47.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.042+0000 7f09515fb980 20 add_watcher() i=1 2024-05-30T16:30:47.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.042+0000 7f09515fb980 20 add_watcher() i=2 2024-05-30T16:30:47.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.042+0000 7f09515fb980 20 add_watcher() i=3 2024-05-30T16:30:47.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.042+0000 7f09515fb980 20 add_watcher() i=4 2024-05-30T16:30:47.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.042+0000 7f09515fb980 20 add_watcher() i=5 2024-05-30T16:30:47.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.042+0000 7f09515fb980 20 add_watcher() i=6 2024-05-30T16:30:47.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.042+0000 7f09515fb980 20 add_watcher() i=7 2024-05-30T16:30:47.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.042+0000 7f09515fb980 2 all 8 watchers are set, enabling cache 2024-05-30T16:30:47.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.042+0000 7f0805fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:30:47.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.046+0000 7f09515fb980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:30:47.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.046+0000 7f09515fb980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:30:47.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.126+0000 7f0804fe1640 20 reqs_thread_entry: start 2024-05-30T16:30:47.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.126+0000 7f09515fb980 20 init_complete bucket index max shards: 11 2024-05-30T16:30:47.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.126+0000 7f09515fb980 20 Filter name: none 2024-05-30T16:30:47.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.126+0000 7f077e7fc640 20 reqs_thread_entry: start 2024-05-30T16:30:47.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.126+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-05-30T16:30:47.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.130+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.130+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.134+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.134+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-05-30T16:30:47.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.138+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.138+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.142+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.142+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-05-30T16:30:47.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.146+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.150+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.150+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.154+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-05-30T16:30:47.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.158+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.158+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.158+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.158+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-05-30T16:30:47.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.162+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.166+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.166+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.170+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-05-30T16:30:47.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.170+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.170+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.170+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.174+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-05-30T16:30:47.178 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.178+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.178+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.178+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.178+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-05-30T16:30:47.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.182+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.186+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.186+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.190+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-05-30T16:30:47.193 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.190+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.194+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.194+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.198+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-05-30T16:30:47.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.202+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.202+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.202+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.206+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-05-30T16:30:47.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.206+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.210+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.210+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.214+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-05-30T16:30:47.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.214+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.218+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.218+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.222+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-05-30T16:30:47.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.226+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.226+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.226+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.230+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-05-30T16:30:47.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.234+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.238+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.238+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.243 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.242+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-05-30T16:30:47.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.246+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.248 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.246+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.246+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.250+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-05-30T16:30:47.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.254+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.254+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.254+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.258+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-05-30T16:30:47.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.262+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.262+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.266+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.266+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-05-30T16:30:47.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.270+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.274+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.274+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.278 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.278+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-05-30T16:30:47.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.282+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.282+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.282+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.286+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-05-30T16:30:47.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.290+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.290+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.290+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.294+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-05-30T16:30:47.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.298+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.298+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.302+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.302+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-05-30T16:30:47.308 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.306+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.310+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.311 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.310+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.314+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-05-30T16:30:47.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.318+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.318+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.318+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.322+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-05-30T16:30:47.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.326+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.326+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.326+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.330+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-05-30T16:30:47.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.334+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.334+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.338+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.338+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-05-30T16:30:47.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.342+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.346+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.347 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.346+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.350+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-05-30T16:30:47.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.350+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.354+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.354+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.358+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-05-30T16:30:47.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.362+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.362+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.362+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.366+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-05-30T16:30:47.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.370+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.370+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.374+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.374+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-05-30T16:30:47.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.378+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.382+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.382+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.386+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-05-30T16:30:47.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.390+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.392 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.390+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.390+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.394+0000 7f09515fb980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-05-30T16:30:47.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.398+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.398+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:47.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.402+0000 7f09515fb980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:47.407 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.406+0000 7f0805fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:30:47.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.410+0000 7f09515fb980 20 remove_watcher() i=0 2024-05-30T16:30:47.413 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.410+0000 7f09515fb980 2 removed watcher, disabling cache 2024-05-30T16:30:47.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.414+0000 7f09515fb980 20 remove_watcher() i=1 2024-05-30T16:30:47.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.418+0000 7f09515fb980 20 remove_watcher() i=2 2024-05-30T16:30:47.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.418+0000 7f09515fb980 20 remove_watcher() i=3 2024-05-30T16:30:47.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.422+0000 7f09515fb980 20 remove_watcher() i=4 2024-05-30T16:30:47.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.426+0000 7f09515fb980 20 remove_watcher() i=5 2024-05-30T16:30:47.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.426+0000 7f09515fb980 20 remove_watcher() i=6 2024-05-30T16:30:47.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.430+0000 7f09515fb980 20 remove_watcher() i=7 2024-05-30T16:30:47.452 DEBUG:teuthology.orchestra.run.smithi006:> radosgw-admin gc process --include-all 2024-05-30T16:30:47.525 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:30:47.525 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:30:47.573 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.570+0000 7f8a2016c980 20 rados->read ofs=0 len=0 2024-05-30T16:30:47.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.574+0000 7f8a2016c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:47.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.574+0000 7f8a2016c980 20 realm 2024-05-30T16:30:47.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.574+0000 7f8a2016c980 20 rados->read ofs=0 len=0 2024-05-30T16:30:47.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.574+0000 7f8a2016c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:47.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.574+0000 7f8a2016c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:30:47.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.574+0000 7f8a2016c980 20 rados->read ofs=0 len=0 2024-05-30T16:30:47.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.574+0000 7f8a2016c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:47.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.574+0000 7f8a2016c980 20 rados->read ofs=0 len=0 2024-05-30T16:30:47.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.574+0000 7f8a2016c980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:30:47.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.574+0000 7f8a2016c980 20 rados->read ofs=0 len=0 2024-05-30T16:30:47.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.574+0000 7f8a2016c980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:30:47.578 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.574+0000 7f8a2016c980 20 searching for the correct realm 2024-05-30T16:30:47.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.782+0000 7f8a1d47c640 0 -- 172.21.15.6:0/1042240730 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f8a1002a040 msgr2=0x7f8a1006b890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:30:47.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:47.794+0000 7f8a1dc7d640 0 -- 172.21.15.6:0/1042240730 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f8a140ff910 msgr2=0x7f8a140e83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:30:48.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.010+0000 7f8a2016c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:30:48.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.010+0000 7f8a2016c980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:30:48.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.010+0000 7f8a2016c980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:30:48.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.010+0000 7f8a2016c980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:30:48.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.010+0000 7f8a2016c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:30:48.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.010+0000 7f8a2016c980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:30:48.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.010+0000 7f8a2016c980 20 rados->read ofs=0 len=0 2024-05-30T16:30:48.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.014+0000 7f8a2016c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:48.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.014+0000 7f8a2016c980 20 rados->read ofs=0 len=0 2024-05-30T16:30:48.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.014+0000 7f8a2016c980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:30:48.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.014+0000 7f8a2016c980 20 rados->read ofs=0 len=0 2024-05-30T16:30:48.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.014+0000 7f8a2016c980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:30:48.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.014+0000 7f8a2016c980 20 zone default found 2024-05-30T16:30:48.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.014+0000 7f8a2016c980 4 Realm: () 2024-05-30T16:30:48.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.014+0000 7f8a2016c980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:30:48.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.014+0000 7f8a2016c980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:30:48.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.014+0000 7f8a2016c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:30:48.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.014+0000 7f8a2016c980 20 zonegroup default 2024-05-30T16:30:48.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.014+0000 7f8a2016c980 20 rados->read ofs=0 len=0 2024-05-30T16:30:48.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.014+0000 7f8a2016c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:48.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.014+0000 7f8a2016c980 20 rados->read ofs=0 len=0 2024-05-30T16:30:48.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.018+0000 7f8a2016c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:48.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.018+0000 7f8a2016c980 20 rados->read ofs=0 len=0 2024-05-30T16:30:48.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.018+0000 7f8a2016c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:48.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.018+0000 7f8a2016c980 20 started sync module instance, tier type = 2024-05-30T16:30:48.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.018+0000 7f8a2016c980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:30:48.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.038+0000 7f8a2016c980 20 add_watcher() i=0 2024-05-30T16:30:48.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.038+0000 7f8a2016c980 20 add_watcher() i=1 2024-05-30T16:30:48.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.038+0000 7f8a2016c980 20 add_watcher() i=2 2024-05-30T16:30:48.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.038+0000 7f8a2016c980 20 add_watcher() i=3 2024-05-30T16:30:48.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.038+0000 7f8a2016c980 20 add_watcher() i=4 2024-05-30T16:30:48.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.038+0000 7f8a2016c980 20 add_watcher() i=5 2024-05-30T16:30:48.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.038+0000 7f8a2016c980 20 add_watcher() i=6 2024-05-30T16:30:48.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.042+0000 7f8a2016c980 20 add_watcher() i=7 2024-05-30T16:30:48.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.042+0000 7f8a2016c980 2 all 8 watchers are set, enabling cache 2024-05-30T16:30:48.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.046+0000 7f88cf7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:30:48.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.046+0000 7f8a2016c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:30:48.047 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.046+0000 7f8a2016c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:30:48.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.134+0000 7f88ce7fc640 20 reqs_thread_entry: start 2024-05-30T16:30:48.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.134+0000 7f8a2016c980 20 init_complete bucket index max shards: 11 2024-05-30T16:30:48.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.134+0000 7f8a2016c980 20 Filter name: none 2024-05-30T16:30:48.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.134+0000 7f883ffff640 20 reqs_thread_entry: start 2024-05-30T16:30:48.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.134+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-05-30T16:30:48.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.138+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.142+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.142+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.142+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-05-30T16:30:48.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.146+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.146+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.146+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.150 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.150+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-05-30T16:30:48.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.150+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.154+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.154+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.154+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-05-30T16:30:48.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.158+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.162+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.163 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.162+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.162+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-05-30T16:30:48.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.166+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.166+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.166+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.170+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-05-30T16:30:48.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.170+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.174+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.174+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.174+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-05-30T16:30:48.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.178+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.178+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.178+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.182+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-05-30T16:30:48.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.186+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.186+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.190+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.192 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.190+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-05-30T16:30:48.195 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.194+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.194+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.194+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.198+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-05-30T16:30:48.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.202+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.202+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.202+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.206+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-05-30T16:30:48.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.206+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.210+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.210+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.210+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-05-30T16:30:48.217 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.214+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.218+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.218+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.222+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-05-30T16:30:48.225 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.222+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.226+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.226+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.230+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-05-30T16:30:48.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.234+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.234+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.234+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.238+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-05-30T16:30:48.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.242+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.242+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.246+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.248 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.246+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-05-30T16:30:48.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.250+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.254+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.254+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.258+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-05-30T16:30:48.263 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.262+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.262+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.262+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.268 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.266+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-05-30T16:30:48.271 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.270+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.270+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.273 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.270+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.274+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-05-30T16:30:48.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.278+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.283 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.282+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.282+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.286+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-05-30T16:30:48.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.290+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.290+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.290+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.294+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-05-30T16:30:48.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.298+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.298+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.302 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.302+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.305 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.302+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-05-30T16:30:48.309 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.306+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.312 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.310+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.310+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.314+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-05-30T16:30:48.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.318+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.318+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.322+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.322+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-05-30T16:30:48.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.326+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.330+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.330+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.334+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-05-30T16:30:48.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.338+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.338+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.338+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.342+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-05-30T16:30:48.348 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.346+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.350 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.350+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.351 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.350+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.350+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-05-30T16:30:48.357 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.354+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.358+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.360 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.358+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.362+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-05-30T16:30:48.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.366+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.366+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.366+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.370+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-05-30T16:30:48.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.374+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.377 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.374+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.378+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.378+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-05-30T16:30:48.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.382+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.386 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.386+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.386 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.386+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.386+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-05-30T16:30:48.393 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.390+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.395 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.394+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.394+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.398+0000 7f8a2016c980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-05-30T16:30:48.403 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.402+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.402+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.402+0000 7f8a2016c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.410+0000 7f88cf7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:30:48.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.414+0000 7f8a2016c980 20 remove_watcher() i=0 2024-05-30T16:30:48.416 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.414+0000 7f8a2016c980 2 removed watcher, disabling cache 2024-05-30T16:30:48.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.418+0000 7f8a2016c980 20 remove_watcher() i=1 2024-05-30T16:30:48.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.418+0000 7f8a2016c980 20 remove_watcher() i=2 2024-05-30T16:30:48.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.422+0000 7f8a2016c980 20 remove_watcher() i=3 2024-05-30T16:30:48.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.426+0000 7f8a2016c980 20 remove_watcher() i=4 2024-05-30T16:30:48.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.426+0000 7f8a2016c980 20 remove_watcher() i=5 2024-05-30T16:30:48.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.430+0000 7f8a2016c980 20 remove_watcher() i=6 2024-05-30T16:30:48.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.434+0000 7f8a2016c980 20 remove_watcher() i=7 2024-05-30T16:30:48.456 DEBUG:teuthology.orchestra.run.smithi006:> radosgw-admin gc process --include-all 2024-05-30T16:30:48.530 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:30:48.530 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:30:48.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.574+0000 7f00ed04e980 20 rados->read ofs=0 len=0 2024-05-30T16:30:48.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.578+0000 7f00ed04e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:48.579 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.578+0000 7f00ed04e980 20 realm 2024-05-30T16:30:48.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.578+0000 7f00ed04e980 20 rados->read ofs=0 len=0 2024-05-30T16:30:48.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.578+0000 7f00ed04e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:48.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.578+0000 7f00ed04e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:30:48.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.578+0000 7f00ed04e980 20 rados->read ofs=0 len=0 2024-05-30T16:30:48.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.578+0000 7f00ed04e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:48.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.578+0000 7f00ed04e980 20 rados->read ofs=0 len=0 2024-05-30T16:30:48.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.582+0000 7f00ed04e980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:30:48.582 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.582+0000 7f00ed04e980 20 rados->read ofs=0 len=0 2024-05-30T16:30:48.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.582+0000 7f00ed04e980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:30:48.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.582+0000 7f00ed04e980 20 searching for the correct realm 2024-05-30T16:30:48.589 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.586+0000 7f00ea35e640 0 -- 172.21.15.6:0/1931085962 >> [v2:172.21.15.6:6819/1141406589,v1:172.21.15.6:6821/1141406589] conn(0x5586076de4d0 msgr2=0x5586076e0990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:30:48.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.594+0000 7f00eab5f640 0 -- 172.21.15.6:0/1931085962 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f00e001f400 msgr2=0x7f00e001f880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:30:48.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.602+0000 7f00ed04e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:30:48.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.602+0000 7f00ed04e980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:30:48.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.602+0000 7f00ed04e980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:30:48.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.602+0000 7f00ed04e980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:30:48.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.602+0000 7f00ed04e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:30:48.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.602+0000 7f00ed04e980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:30:48.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.602+0000 7f00ed04e980 20 rados->read ofs=0 len=0 2024-05-30T16:30:48.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.602+0000 7f00ed04e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:48.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.602+0000 7f00ed04e980 20 rados->read ofs=0 len=0 2024-05-30T16:30:48.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.606+0000 7f00ed04e980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:30:48.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.606+0000 7f00ed04e980 20 rados->read ofs=0 len=0 2024-05-30T16:30:48.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.606+0000 7f00ed04e980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:30:48.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.606+0000 7f00ed04e980 20 zone default found 2024-05-30T16:30:48.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.606+0000 7f00ed04e980 4 Realm: () 2024-05-30T16:30:48.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.606+0000 7f00ed04e980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:30:48.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.606+0000 7f00ed04e980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:30:48.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.606+0000 7f00ed04e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:30:48.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.606+0000 7f00ed04e980 20 zonegroup default 2024-05-30T16:30:48.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.606+0000 7f00ed04e980 20 rados->read ofs=0 len=0 2024-05-30T16:30:48.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.606+0000 7f00ed04e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:48.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.606+0000 7f00ed04e980 20 rados->read ofs=0 len=0 2024-05-30T16:30:48.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.606+0000 7f00ed04e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:48.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.606+0000 7f00ed04e980 20 rados->read ofs=0 len=0 2024-05-30T16:30:48.608 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.606+0000 7f00ed04e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:30:48.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.606+0000 7f00ed04e980 20 started sync module instance, tier type = 2024-05-30T16:30:48.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.606+0000 7f00ed04e980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:30:48.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.622+0000 7f00ed04e980 20 add_watcher() i=0 2024-05-30T16:30:48.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.622+0000 7f00ed04e980 20 add_watcher() i=1 2024-05-30T16:30:48.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.622+0000 7f00ed04e980 20 add_watcher() i=2 2024-05-30T16:30:48.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.622+0000 7f00ed04e980 20 add_watcher() i=3 2024-05-30T16:30:48.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.622+0000 7f00ed04e980 20 add_watcher() i=4 2024-05-30T16:30:48.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.622+0000 7f00ed04e980 20 add_watcher() i=5 2024-05-30T16:30:48.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.622+0000 7f00ed04e980 20 add_watcher() i=6 2024-05-30T16:30:48.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.622+0000 7f00ed04e980 20 add_watcher() i=7 2024-05-30T16:30:48.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.622+0000 7f00ed04e980 2 all 8 watchers are set, enabling cache 2024-05-30T16:30:48.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.626+0000 7effa17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:30:48.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.626+0000 7f00ed04e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:30:48.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.626+0000 7f00ed04e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:30:48.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.710+0000 7eff1bfff640 20 reqs_thread_entry: start 2024-05-30T16:30:48.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.714+0000 7f00ed04e980 20 init_complete bucket index max shards: 11 2024-05-30T16:30:48.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.714+0000 7f00ed04e980 20 Filter name: none 2024-05-30T16:30:48.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.714+0000 7eff19ffb640 20 reqs_thread_entry: start 2024-05-30T16:30:48.714 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.714+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-05-30T16:30:48.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.718+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.718+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.718+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.722+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-05-30T16:30:48.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.722+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.726+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.726+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.726+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-05-30T16:30:48.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.730+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.730+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.734+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.734+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-05-30T16:30:48.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.738+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.738+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.738+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.742 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.742+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-05-30T16:30:48.743 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.742+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.742+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.742+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.746+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-05-30T16:30:48.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.750+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.750+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.753 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.750+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.754+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-05-30T16:30:48.758 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.758+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.758+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.758+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.758+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-05-30T16:30:48.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.762+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.762+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.762+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.766+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-05-30T16:30:48.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.770+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.770+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.770+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.770+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-05-30T16:30:48.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.774+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.779 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.778+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.780 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.778+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.782+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-05-30T16:30:48.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.782+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.786 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.786+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.787 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.786+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.789 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.786+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-05-30T16:30:48.792 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.790+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.794+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.794 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.794+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.794+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-05-30T16:30:48.800 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.798+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.798+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.801 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.798+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.803 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.802+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-05-30T16:30:48.807 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.806+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.806+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.810 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.810+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.812 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.810+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-05-30T16:30:48.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.814+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.818+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.819 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.818+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.822+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-05-30T16:30:48.826 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.826+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.826+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.828 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.826+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.830+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-05-30T16:30:48.835 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.834+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.836 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.834+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.834+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.840 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.838+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-05-30T16:30:48.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.842+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.842+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.846+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.846+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-05-30T16:30:48.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.850+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.854+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.855 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.854+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.858 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.858+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-05-30T16:30:48.862 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.862+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.862+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.862+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.866+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-05-30T16:30:48.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.870+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.872 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.870+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.870+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.874+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-05-30T16:30:48.880 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.878+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.882+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.882+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.886+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-05-30T16:30:48.890 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.890+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.890+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.890+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.894+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-05-30T16:30:48.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.898+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.898+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.902+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.905 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.902+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-05-30T16:30:48.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.906+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.910+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.910+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.914 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.914+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-05-30T16:30:48.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.918+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.918+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.918+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.922+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-05-30T16:30:48.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.926+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.926+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.930+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.930+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-05-30T16:30:48.936 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.934+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.934+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.938+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.938+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-05-30T16:30:48.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.942+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.946+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.946+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.950+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-05-30T16:30:48.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.954+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.954+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.954+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.958+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-05-30T16:30:48.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.962+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.965 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.962+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.966 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.966+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.966+0000 7f00ed04e980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-05-30T16:30:48.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.970+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.974+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:30:48.975 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.974+0000 7f00ed04e980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:30:48.980 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.978+0000 7effa17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:30:48.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.982+0000 7f00ed04e980 20 remove_watcher() i=0 2024-05-30T16:30:48.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.982+0000 7f00ed04e980 2 removed watcher, disabling cache 2024-05-30T16:30:48.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.986+0000 7f00ed04e980 20 remove_watcher() i=1 2024-05-30T16:30:48.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.990+0000 7f00ed04e980 20 remove_watcher() i=2 2024-05-30T16:30:48.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.994+0000 7f00ed04e980 20 remove_watcher() i=3 2024-05-30T16:30:48.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.994+0000 7f00ed04e980 20 remove_watcher() i=4 2024-05-30T16:30:48.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.998+0000 7f00ed04e980 20 remove_watcher() i=5 2024-05-30T16:30:49.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:48.998+0000 7f00ed04e980 20 remove_watcher() i=6 2024-05-30T16:30:49.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:30:49.002+0000 7f00ed04e980 20 remove_watcher() i=7 2024-05-30T16:30:49.023 DEBUG:teuthology.orchestra.run.smithi006:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean test --rerun-tasks --no-build-cache --tests ObjectTest 2024-05-30T16:30:49.735 INFO:teuthology.orchestra.run.smithi006.stdout:> Task :clean 2024-05-30T16:30:50.532 INFO:teuthology.orchestra.run.smithi006.stdout:> Task :compileJava 2024-05-30T16:30:50.532 INFO:teuthology.orchestra.run.smithi006.stdout:> Task :processResources 2024-05-30T16:30:50.532 INFO:teuthology.orchestra.run.smithi006.stdout:> Task :classes 2024-05-30T16:30:51.132 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:51.132 INFO:teuthology.orchestra.run.smithi006.stdout:> Task :compileTestJava 2024-05-30T16:30:51.133 INFO:teuthology.orchestra.run.smithi006.stderr:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2024-05-30T16:30:51.133 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:51.133 INFO:teuthology.orchestra.run.smithi006.stdout:> Task :processTestResources NO-SOURCE 2024-05-30T16:30:51.134 INFO:teuthology.orchestra.run.smithi006.stdout:> Task :testClasses 2024-05-30T16:30:51.134 INFO:teuthology.orchestra.run.smithi006.stderr:Note: Recompile with -Xlint:deprecation for details. 2024-05-30T16:30:53.132 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:53.133 INFO:teuthology.orchestra.run.smithi006.stdout:> Task :test 2024-05-30T16:30:53.133 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:53.133 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests STANDARD_OUT 2024-05-30T16:30:53.134 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30 16:30:52,623 [Test worker] WARN com.amazonaws.util.Base64 - JAXB is unavailable. Will fallback to SDK implementation which may be less performant 2024-05-30T16:30:53.134 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:53.134 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.TestObjectCopyKeyNotFound STARTED 2024-05-30T16:30:53.134 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:53.134 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.TestObjectCopyKeyNotFound PASSED 2024-05-30T16:30:54.332 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:54.332 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadExpectNone STARTED 2024-05-30T16:30:54.532 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:54.532 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadExpectNone PASSED 2024-05-30T16:30:55.332 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:55.332 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadMd5Empty STARTED 2024-05-30T16:30:55.532 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:55.532 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadMd5Empty PASSED 2024-05-30T16:30:56.232 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:30:56.232 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadLLAPI STARTED 2024-05-30T16:31:00.532 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:31:00.532 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadLLAPI PASSED 2024-05-30T16:31:01.332 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:31:01.332 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadNotFoundLLAPI STARTED 2024-05-30T16:31:01.532 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:31:01.532 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadNotFoundLLAPI PASSED 2024-05-30T16:31:02.232 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:31:02.232 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testDownloadHLAPI STARTED 2024-05-30T16:31:02.632 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:31:02.632 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testDownloadHLAPI PASSED 2024-05-30T16:31:03.432 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:31:03.433 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testDownloadNoBucketHLAPI STARTED 2024-05-30T16:31:03.433 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:31:03.433 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testDownloadNoBucketHLAPI PASSED 2024-05-30T16:31:03.632 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:31:03.632 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testDownloadNoKeyHLAPI STARTED 2024-05-30T16:31:03.732 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:31:03.732 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testDownloadNoKeyHLAPI PASSED 2024-05-30T16:31:04.432 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:31:04.432 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1MB STARTED 2024-05-30T16:31:06.732 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:31:06.733 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1MB PASSED 2024-05-30T16:31:07.532 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:31:07.532 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1b STARTED 2024-05-30T16:31:07.732 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:31:07.732 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1b PASSED 2024-05-30T16:31:08.532 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:31:08.532 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1kb STARTED 2024-05-30T16:31:08.732 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:31:08.732 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1kb PASSED 2024-05-30T16:31:09.432 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:31:09.432 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testEncryptionKeyNoSSEC STARTED 2024-05-30T16:31:09.632 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:31:09.632 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testEncryptionKeyNoSSEC PASSED 2024-05-30T16:31:10.332 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:31:10.332 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECInvalidMd5 STARTED 2024-05-30T16:31:10.532 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:31:10.532 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECInvalidMd5 PASSED 2024-05-30T16:31:11.232 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:31:11.232 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoKey STARTED 2024-05-30T16:31:11.432 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:31:11.432 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoKey PASSED 2024-05-30T16:31:12.132 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:31:12.133 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoMd5 STARTED 2024-05-30T16:31:12.432 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:31:12.433 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoMd5 PASSED 2024-05-30T16:31:13.132 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:31:13.132 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testMultiObjectDelete STARTED 2024-05-30T16:31:13.532 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:31:13.532 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testMultiObjectDelete PASSED 2024-05-30T16:31:14.132 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:31:14.132 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testMultipartCopyHLAPIA STARTED 2024-05-30T16:31:18.632 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:31:18.632 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testMultipartCopyHLAPIA PASSED 2024-05-30T16:31:20.032 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:31:20.032 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testMultipartCopyMultipleSizesLLAPI STARTED 2024-05-30T16:32:00.532 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:32:00.533 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testMultipartCopyMultipleSizesLLAPI PASSED 2024-05-30T16:32:01.932 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:32:01.932 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCBucketHLAPI STARTED 2024-05-30T16:32:02.032 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:32:02.032 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCBucketHLAPI PASSED 2024-05-30T16:32:02.732 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:32:02.732 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCKeyHLAPI STARTED 2024-05-30T16:32:03.032 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:32:03.032 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCKeyHLAPI PASSED 2024-05-30T16:32:04.232 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:32:04.232 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testMultipartDownloadHLAPI STARTED 2024-05-30T16:32:08.332 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:32:08.332 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testMultipartDownloadHLAPI PASSED 2024-05-30T16:32:09.132 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:32:09.132 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testMultipartDownloadNoBucketHLAPI STARTED 2024-05-30T16:32:09.232 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:32:09.232 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testMultipartDownloadNoBucketHLAPI PASSED 2024-05-30T16:32:09.332 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:32:09.332 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testMultipartDownloadWithPauseHLAPI STARTED 2024-05-30T16:32:14.832 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:32:14.832 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testMultipartDownloadWithPauseHLAPI PASSED 2024-05-30T16:32:16.332 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:32:16.333 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests STANDARD_OUT 2024-05-30T16:32:16.333 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30 16:32:16,279 [s3-transfer-manager-worker-10] WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. 2024-05-30T16:32:16.333 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30 16:32:16,283 [s3-transfer-manager-worker-9] WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. 2024-05-30T16:32:17.032 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:32:17.032 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testMultipartUploadHLAPIAWS4 STARTED 2024-05-30T16:32:29.132 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:32:29.132 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testMultipartUploadHLAPIAWS4 PASSED 2024-05-30T16:32:30.232 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:32:30.232 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testMultipartUploadIncorrectMissingPartLLAPI STARTED 2024-05-30T16:32:42.632 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:32:42.632 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testMultipartUploadIncorrectMissingPartLLAPI PASSED 2024-05-30T16:32:43.432 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:32:43.432 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testMultipartUploadMultipleSizesLLAPI STARTED 2024-05-30T16:33:13.432 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:13.433 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testMultipartUploadMultipleSizesLLAPI PASSED 2024-05-30T16:33:14.132 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:14.132 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testMultipartUploadOverwriteExistingObjectLLAPI STARTED 2024-05-30T16:33:19.532 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:19.532 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testMultipartUploadOverwriteExistingObjectLLAPI PASSED 2024-05-30T16:33:20.232 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:20.232 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testMultipartUploadSmallLLAPI STARTED 2024-05-30T16:33:23.932 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:23.932 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testMultipartUploadSmallLLAPI PASSED 2024-05-30T16:33:24.732 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:24.732 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testMultipartUploadWithPause STARTED 2024-05-30T16:33:27.532 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:27.532 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testMultipartUploadWithPause PASSED 2024-05-30T16:33:41.332 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:41.332 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectCopyBucketNotFound STARTED 2024-05-30T16:33:41.532 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:41.532 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectCopyBucketNotFound PASSED 2024-05-30T16:33:41.832 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:41.832 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectCreateBadExpectMismatch STARTED 2024-05-30T16:33:42.332 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:42.333 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectCreateBadExpectMismatch PASSED 2024-05-30T16:33:46.232 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:46.232 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5InvalidShort STARTED 2024-05-30T16:33:46.432 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:46.432 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5InvalidShort PASSED 2024-05-30T16:33:47.132 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:47.132 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Ivalid STARTED 2024-05-30T16:33:47.332 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:47.332 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Ivalid PASSED 2024-05-30T16:33:48.032 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:48.032 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5None STARTED 2024-05-30T16:33:48.232 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:48.232 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5None PASSED 2024-05-30T16:33:49.032 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:49.032 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Unreadable STARTED 2024-05-30T16:33:49.232 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:49.232 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Unreadable PASSED 2024-05-30T16:33:49.932 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:49.932 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectHeadZeroBytes STARTED 2024-05-30T16:33:50.132 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:50.132 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectHeadZeroBytes PASSED 2024-05-30T16:33:50.832 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:50.832 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterDot STARTED 2024-05-30T16:33:51.432 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:51.432 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterDot PASSED 2024-05-30T16:33:52.432 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:52.432 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterNotExist STARTED 2024-05-30T16:33:52.932 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:52.932 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterNotExist PASSED 2024-05-30T16:33:53.932 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:53.932 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterPercentage STARTED 2024-05-30T16:33:54.432 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:54.432 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterPercentage PASSED 2024-05-30T16:33:55.432 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:55.432 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListMarkerAfterList STARTED 2024-05-30T16:33:55.932 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:55.932 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListMarkerAfterList PASSED 2024-05-30T16:33:56.932 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:56.932 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListMarkerBeforeList STARTED 2024-05-30T16:33:57.432 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:57.432 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListMarkerBeforeList PASSED 2024-05-30T16:33:58.432 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:58.432 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListMarkerNotInList STARTED 2024-05-30T16:33:58.932 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:58.932 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListMarkerNotInList PASSED 2024-05-30T16:33:59.832 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:33:59.832 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListMarkerUnreadable STARTED 2024-05-30T16:34:00.332 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:00.332 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListMarkerUnreadable PASSED 2024-05-30T16:34:01.332 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:01.332 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNegative STARTED 2024-05-30T16:34:01.832 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:01.832 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNegative PASSED 2024-05-30T16:34:02.832 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:02.832 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNone STARTED 2024-05-30T16:34:03.432 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:03.432 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNone PASSED 2024-05-30T16:34:04.332 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:04.332 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysOne STARTED 2024-05-30T16:34:04.932 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:04.932 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysOne PASSED 2024-05-30T16:34:05.832 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:05.832 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysZero STARTED 2024-05-30T16:34:06.332 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:06.332 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysZero PASSED 2024-05-30T16:34:07.332 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:07.332 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListPrefixAlt STARTED 2024-05-30T16:34:07.832 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:07.832 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListPrefixAlt PASSED 2024-05-30T16:34:08.732 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:08.732 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListPrefixBasic STARTED 2024-05-30T16:34:09.132 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:09.132 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListPrefixBasic PASSED 2024-05-30T16:34:10.033 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:10.033 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterAlt STARTED 2024-05-30T16:34:10.532 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:10.532 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterAlt PASSED 2024-05-30T16:34:11.533 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:11.533 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterBasic STARTED 2024-05-30T16:34:12.032 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:12.032 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterBasic PASSED 2024-05-30T16:34:13.032 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:13.032 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterDelimiterNotExist STARTED 2024-05-30T16:34:13.532 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:13.532 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterDelimiterNotExist PASSED 2024-05-30T16:34:14.532 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:14.532 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixDelimiterNotExist STARTED 2024-05-30T16:34:15.032 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:15.032 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixDelimiterNotExist PASSED 2024-05-30T16:34:16.032 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:16.032 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixNotExist STARTED 2024-05-30T16:34:16.532 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:16.532 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixNotExist PASSED 2024-05-30T16:34:17.532 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:17.532 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListPrefixEmpty STARTED 2024-05-30T16:34:17.932 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:17.932 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListPrefixEmpty PASSED 2024-05-30T16:34:18.932 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:18.932 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNone STARTED 2024-05-30T16:34:19.332 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:19.332 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNone PASSED 2024-05-30T16:34:20.232 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:20.232 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNotExist STARTED 2024-05-30T16:34:20.632 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:20.632 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNotExist PASSED 2024-05-30T16:34:21.632 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:21.632 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListPrefixUnreadable STARTED 2024-05-30T16:34:22.032 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:22.032 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectListPrefixUnreadable PASSED 2024-05-30T16:34:22.932 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:22.932 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectReadFromNonExistantBucket STARTED 2024-05-30T16:34:23.132 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:23.132 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectReadFromNonExistantBucket PASSED 2024-05-30T16:34:23.832 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:23.832 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectReadNotExist STARTED 2024-05-30T16:34:24.032 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:24.032 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectReadNotExist PASSED 2024-05-30T16:34:24.732 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:24.732 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectWriteCacheControl STARTED 2024-05-30T16:34:24.934 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:24.934 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectWriteCacheControl PASSED 2024-05-30T16:34:25.732 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:25.732 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectWriteCheckEtag STARTED 2024-05-30T16:34:25.932 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:25.932 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectWriteCheckEtag PASSED 2024-05-30T16:34:26.732 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:26.732 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectWriteReadUpdateReadDelete STARTED 2024-05-30T16:34:27.332 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:27.332 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectWriteReadUpdateReadDelete PASSED 2024-05-30T16:34:28.032 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:28.032 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectWriteToNonExistantBucket STARTED 2024-05-30T16:34:28.132 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:28.132 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testObjectWriteToNonExistantBucket PASSED 2024-05-30T16:34:28.232 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:28.232 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testRangedSkipLeadingBytesResponseCode STARTED 2024-05-30T16:34:28.532 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:28.532 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testRangedSkipLeadingBytesResponseCode PASSED 2024-05-30T16:34:29.332 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:29.332 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testRangedrequestResponseCode STARTED 2024-05-30T16:34:29.532 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:29.532 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testRangedrequestResponseCode PASSED 2024-05-30T16:34:30.332 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:30.332 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testSSEKMSNoKey STARTED 2024-05-30T16:34:30.632 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:30.632 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testSSEKMSNoKey PASSED 2024-05-30T16:34:31.332 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:31.332 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testSSEKMSNotDeclared STARTED 2024-05-30T16:34:31.532 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:31.532 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testSSEKMSNotDeclared PASSED 2024-05-30T16:34:32.232 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:32.232 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testUploadFileHLAPIBigFile STARTED 2024-05-30T16:34:35.832 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:35.833 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testUploadFileHLAPIBigFile PASSED 2024-05-30T16:34:36.632 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:36.632 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testUploadFileList STARTED 2024-05-30T16:34:49.932 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:49.932 INFO:teuthology.orchestra.run.smithi006.stdout:suite > Object tests > ObjectTest.testUploadFileList PASSED 2024-05-30T16:34:50.798 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:50.798 INFO:teuthology.orchestra.run.smithi006.stdout:---------------------------------------------------------------------- 2024-05-30T16:34:50.798 INFO:teuthology.orchestra.run.smithi006.stdout:| Results: SUCCESS (70 tests, 70 successes, 0 failures, 0 skipped) | 2024-05-30T16:34:50.798 INFO:teuthology.orchestra.run.smithi006.stdout:---------------------------------------------------------------------- 2024-05-30T16:34:50.798 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:50.798 INFO:teuthology.orchestra.run.smithi006.stdout:Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. 2024-05-30T16:34:50.799 INFO:teuthology.orchestra.run.smithi006.stdout:Use '--warning-mode all' to show the individual deprecation warnings. 2024-05-30T16:34:50.799 INFO:teuthology.orchestra.run.smithi006.stdout:See https://docs.gradle.org/6.0.1/userguide/command_line_interface.html#sec:command_line_warnings 2024-05-30T16:34:50.799 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:50.799 INFO:teuthology.orchestra.run.smithi006.stdout:BUILD SUCCESSFUL in 4m 1s 2024-05-30T16:34:50.799 INFO:teuthology.orchestra.run.smithi006.stdout:5 actionable tasks: 5 executed 2024-05-30T16:34:50.818 DEBUG:teuthology.orchestra.run.smithi006:> radosgw-admin gc process --include-all 2024-05-30T16:34:50.850 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:34:50.851 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:34:51.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.076+0000 7fefc8422640 0 --2- 172.21.15.6:0/2007540113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5648c06e19e0 0x5648c0709240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-05-30T16:34:51.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.104+0000 7fefcb112980 20 rados->read ofs=0 len=0 2024-05-30T16:34:51.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.108+0000 7fefcb112980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:51.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.108+0000 7fefcb112980 20 realm 2024-05-30T16:34:51.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.108+0000 7fefcb112980 20 rados->read ofs=0 len=0 2024-05-30T16:34:51.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.108+0000 7fefcb112980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:51.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.108+0000 7fefcb112980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:34:51.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.108+0000 7fefcb112980 20 rados->read ofs=0 len=0 2024-05-30T16:34:51.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.108+0000 7fefcb112980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:51.109 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.108+0000 7fefcb112980 20 rados->read ofs=0 len=0 2024-05-30T16:34:51.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.112+0000 7fefcb112980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:51.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.112+0000 7fefcb112980 20 rados->read ofs=0 len=0 2024-05-30T16:34:51.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.112+0000 7fefcb112980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:34:51.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.112+0000 7fefcb112980 20 searching for the correct realm 2024-05-30T16:34:51.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.116+0000 7fefc8c23640 0 -- 172.21.15.6:0/3520664034 >> [v2:172.21.15.6:6816/2704410641,v1:172.21.15.6:6817/2704410641] conn(0x7fefc0029070 msgr2=0x7fefc0020420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:34:51.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.124+0000 7fefc8c23640 0 -- 172.21.15.6:0/3520664034 >> [v2:172.21.15.6:6816/2704410641,v1:172.21.15.6:6817/2704410641] conn(0x7fee9407e540 msgr2=0x7fee94080970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:34:51.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.132+0000 7fefcb112980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:34:51.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.132+0000 7fefcb112980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:34:51.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.132+0000 7fefcb112980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:34:51.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.132+0000 7fefcb112980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:34:51.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.132+0000 7fefcb112980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:34:51.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.132+0000 7fefcb112980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:34:51.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.136+0000 7fefcb112980 20 rados->read ofs=0 len=0 2024-05-30T16:34:51.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.136+0000 7fefcb112980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:51.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.136+0000 7fefcb112980 20 rados->read ofs=0 len=0 2024-05-30T16:34:51.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.136+0000 7fefcb112980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:51.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.136+0000 7fefcb112980 20 rados->read ofs=0 len=0 2024-05-30T16:34:51.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.136+0000 7fefcb112980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:34:51.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.136+0000 7fefcb112980 20 zone default found 2024-05-30T16:34:51.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.136+0000 7fefcb112980 4 Realm: () 2024-05-30T16:34:51.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.136+0000 7fefcb112980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:34:51.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.136+0000 7fefcb112980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:34:51.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.136+0000 7fefcb112980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:34:51.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.136+0000 7fefcb112980 20 zonegroup default 2024-05-30T16:34:51.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.136+0000 7fefcb112980 20 rados->read ofs=0 len=0 2024-05-30T16:34:51.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.136+0000 7fefcb112980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:51.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.136+0000 7fefcb112980 20 rados->read ofs=0 len=0 2024-05-30T16:34:51.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.136+0000 7fefcb112980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:51.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.136+0000 7fefcb112980 20 rados->read ofs=0 len=0 2024-05-30T16:34:51.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.136+0000 7fefcb112980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:51.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.136+0000 7fefcb112980 20 started sync module instance, tier type = 2024-05-30T16:34:51.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.136+0000 7fefcb112980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:34:51.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.156+0000 7fefcb112980 20 add_watcher() i=0 2024-05-30T16:34:51.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.156+0000 7fefcb112980 20 add_watcher() i=1 2024-05-30T16:34:51.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.156+0000 7fefcb112980 20 add_watcher() i=2 2024-05-30T16:34:51.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.156+0000 7fefcb112980 20 add_watcher() i=3 2024-05-30T16:34:51.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.156+0000 7fefcb112980 20 add_watcher() i=4 2024-05-30T16:34:51.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.156+0000 7fefcb112980 20 add_watcher() i=5 2024-05-30T16:34:51.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.156+0000 7fefcb112980 20 add_watcher() i=6 2024-05-30T16:34:51.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.156+0000 7fefcb112980 20 add_watcher() i=7 2024-05-30T16:34:51.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.156+0000 7fefcb112980 2 all 8 watchers are set, enabling cache 2024-05-30T16:34:51.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.160+0000 7fee7f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:51.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.160+0000 7fefcb112980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:34:51.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.160+0000 7fefcb112980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:34:51.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.244+0000 7fee7e7e4640 20 reqs_thread_entry: start 2024-05-30T16:34:51.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.244+0000 7fefcb112980 20 init_complete bucket index max shards: 11 2024-05-30T16:34:51.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.244+0000 7fefcb112980 20 Filter name: none 2024-05-30T16:34:51.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.244+0000 7fee77fff640 20 reqs_thread_entry: start 2024-05-30T16:34:51.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.244+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-05-30T16:34:51.248 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.248+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.248+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.252+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:34:51.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.252+0000 7fefcb112980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12123705067827263014', time=2024-05-30T18:32:29.497919+0000, chain.objs.size()=5 2024-05-30T16:34:51.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.252+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.90__multipart_file2.mpg.2~uEzPccA8wmU56ZlU8G6crRJdLqdNaHP.1 2024-05-30T16:34:51.251 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.252+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.90__multipart_file2.mpg.2~uEzPccA8wmU56ZlU8G6crRJdLqdNaHP.2 2024-05-30T16:34:51.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.252+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.90__multipart_file2.mpg.2~uEzPccA8wmU56ZlU8G6crRJdLqdNaHP.3 2024-05-30T16:34:51.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.252+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.90__multipart_file2.mpg.2~uEzPccA8wmU56ZlU8G6crRJdLqdNaHP.4 2024-05-30T16:34:51.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.252+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.90__multipart_file2.mpg.2~uEzPccA8wmU56ZlU8G6crRJdLqdNaHP.5 2024-05-30T16:34:51.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.256+0000 7fefcb112980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:34:51.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.264+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-05-30T16:34:51.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.268+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.267 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.268+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.268+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:34:51.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.268+0000 7fefcb112980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.5891527587806863195', time=2024-05-30T18:32:29.353030+0000, chain.objs.size()=5 2024-05-30T16:34:51.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.268+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.90__multipart_file.mpg.2~ZtiXb7JlsDlDM4ofGwDL4guG7v2ZSVf.1 2024-05-30T16:34:51.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.268+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.90__multipart_file.mpg.2~ZtiXb7JlsDlDM4ofGwDL4guG7v2ZSVf.2 2024-05-30T16:34:51.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.268+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.90__multipart_file.mpg.2~ZtiXb7JlsDlDM4ofGwDL4guG7v2ZSVf.3 2024-05-30T16:34:51.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.268+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.90__multipart_file.mpg.2~ZtiXb7JlsDlDM4ofGwDL4guG7v2ZSVf.4 2024-05-30T16:34:51.269 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.268+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.90__multipart_file.mpg.2~ZtiXb7JlsDlDM4ofGwDL4guG7v2ZSVf.5 2024-05-30T16:34:51.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.276+0000 7fefcb112980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:34:51.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.284+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-05-30T16:34:51.284 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.284+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.284+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.284+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.287 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.288+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-05-30T16:34:51.290 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.292+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.292+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.292+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.296+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-05-30T16:34:51.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.300+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.300+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.300+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-30T16:34:51.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.300+0000 7fefcb112980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.9101367856709014498', time=2024-05-30T18:32:00.718150+0000, chain.objs.size()=4 2024-05-30T16:34:51.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.300+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.83__shadow_.YQAVVa1QYookLQctHRMUNT-5_FuADoa_1 2024-05-30T16:34:51.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.300+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.83__shadow_.YQAVVa1QYookLQctHRMUNT-5_FuADoa_2 2024-05-30T16:34:51.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.300+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.83__shadow_.YQAVVa1QYookLQctHRMUNT-5_FuADoa_3 2024-05-30T16:34:51.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.300+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.83__shadow_.YQAVVa1QYookLQctHRMUNT-5_FuADoa_4 2024-05-30T16:34:51.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.300+0000 7fefcb112980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12254908688939270239', time=2024-05-30T18:33:13.264915+0000, chain.objs.size()=5 2024-05-30T16:34:51.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.300+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.92__multipart_key1.2~rAGbhEISfgOsE3HMvqyfqwqraUd5W22.1 2024-05-30T16:34:51.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.300+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.92__shadow_key1.2~rAGbhEISfgOsE3HMvqyfqwqraUd5W22.1_1 2024-05-30T16:34:51.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.300+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.92__multipart_key1.2~rAGbhEISfgOsE3HMvqyfqwqraUd5W22.2 2024-05-30T16:34:51.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.300+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.92__shadow_key1.2~rAGbhEISfgOsE3HMvqyfqwqraUd5W22.2_1 2024-05-30T16:34:51.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.300+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.92__multipart_key1.2~rAGbhEISfgOsE3HMvqyfqwqraUd5W22.3 2024-05-30T16:34:51.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.308+0000 7fefcb112980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:34:51.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.316+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-05-30T16:34:51.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.320+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.320+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.320+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:34:51.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.320+0000 7fefcb112980 20 garbage collection: RGWGC::process iterating over entry tag='2~IODqS7uItHZODBs8tCTxjRGLUJI6d8w', time=2024-05-30T18:31:00.486121+0000, chain.objs.size()=5 2024-05-30T16:34:51.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.324+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.69__multipart_key1.2~IODqS7uItHZODBs8tCTxjRGLUJI6d8w.1 2024-05-30T16:34:51.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.324+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.69__multipart_key1.2~IODqS7uItHZODBs8tCTxjRGLUJI6d8w.2 2024-05-30T16:34:51.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.324+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.69__multipart_key1.2~IODqS7uItHZODBs8tCTxjRGLUJI6d8w.3 2024-05-30T16:34:51.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.324+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.69__multipart_key1.2~IODqS7uItHZODBs8tCTxjRGLUJI6d8w.4 2024-05-30T16:34:51.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.324+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.69__multipart_key1.2~IODqS7uItHZODBs8tCTxjRGLUJI6d8w.5 2024-05-30T16:34:51.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.328+0000 7fefcb112980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:34:51.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.332+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-05-30T16:34:51.337 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.336+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.340+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.340+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-30T16:34:51.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.340+0000 7fefcb112980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.4192574141265595562', time=2024-05-30T18:31:46.607617+0000, chain.objs.size()=5 2024-05-30T16:34:51.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.340+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.84__multipart_key1.2~D0LwMw3rUbWUT2jDQ537CLqx1KiMeve.1 2024-05-30T16:34:51.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.340+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.84__shadow_key1.2~D0LwMw3rUbWUT2jDQ537CLqx1KiMeve.1_1 2024-05-30T16:34:51.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.340+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.84__multipart_key1.2~D0LwMw3rUbWUT2jDQ537CLqx1KiMeve.2 2024-05-30T16:34:51.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.340+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.84__shadow_key1.2~D0LwMw3rUbWUT2jDQ537CLqx1KiMeve.2_1 2024-05-30T16:34:51.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.340+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.84__multipart_key1.2~D0LwMw3rUbWUT2jDQ537CLqx1KiMeve.3 2024-05-30T16:34:51.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.340+0000 7fefcb112980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16786105666675691742', time=2024-05-30T18:33:24.126680+0000, chain.objs.size()=5 2024-05-30T16:34:51.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.340+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.94__multipart_key1.2~I7YwmUqgOwjrJCUi9cy3tCWBKr6Ot6f.1 2024-05-30T16:34:51.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.340+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.94__multipart_key1.2~I7YwmUqgOwjrJCUi9cy3tCWBKr6Ot6f.2 2024-05-30T16:34:51.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.340+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.94__multipart_key1.2~I7YwmUqgOwjrJCUi9cy3tCWBKr6Ot6f.3 2024-05-30T16:34:51.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.340+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.94__multipart_key1.2~I7YwmUqgOwjrJCUi9cy3tCWBKr6Ot6f.4 2024-05-30T16:34:51.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.340+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.94__multipart_key1.2~I7YwmUqgOwjrJCUi9cy3tCWBKr6Ot6f.5 2024-05-30T16:34:51.353 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.352+0000 7fefcb112980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:34:51.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.360+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-05-30T16:34:51.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.364+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.364+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.364+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.364+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-05-30T16:34:51.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.368+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.370 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.372+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.370 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.372+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.373 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.372+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-05-30T16:34:51.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.376+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.376+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.376+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.380+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-05-30T16:34:51.381 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.380+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.382 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.384+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.384+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-30T16:34:51.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.384+0000 7fefcb112980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12842541659072262037', time=2024-05-30T18:31:06.941002+0000, chain.objs.size()=2 2024-05-30T16:34:51.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.384+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.73__shadow_.mk3si_50uHt7parKE7TAAbeCg9PzegI_1 2024-05-30T16:34:51.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.384+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.73__shadow_.mk3si_50uHt7parKE7TAAbeCg9PzegI_2 2024-05-30T16:34:51.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.384+0000 7fefcb112980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16240370028800716551', time=2024-05-30T18:31:32.293194+0000, chain.objs.size()=5 2024-05-30T16:34:51.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.384+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.84__multipart_key1.2~GM2NJQiDwVNSzB3zvGorqdinaftfuRW.1 2024-05-30T16:34:51.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.384+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.84__multipart_key1.2~GM2NJQiDwVNSzB3zvGorqdinaftfuRW.2 2024-05-30T16:34:51.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.384+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.84__multipart_key1.2~GM2NJQiDwVNSzB3zvGorqdinaftfuRW.3 2024-05-30T16:34:51.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.384+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.84__multipart_key1.2~GM2NJQiDwVNSzB3zvGorqdinaftfuRW.4 2024-05-30T16:34:51.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.384+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.84__multipart_key1.2~GM2NJQiDwVNSzB3zvGorqdinaftfuRW.5 2024-05-30T16:34:51.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.392+0000 7fefcb112980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:34:51.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.396+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-05-30T16:34:51.398 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.400+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.400+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.400+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-30T16:34:51.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.400+0000 7fefcb112980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.9900708503968231329', time=2024-05-30T18:31:37.673082+0000, chain.objs.size()=5 2024-05-30T16:34:51.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.400+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.84__multipart_key1.2~8A85IpnQGhw6kBN12qV7dOO5c8nnihb.1 2024-05-30T16:34:51.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.400+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.84__multipart_key1.2~8A85IpnQGhw6kBN12qV7dOO5c8nnihb.2 2024-05-30T16:34:51.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.400+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.84__multipart_key1.2~8A85IpnQGhw6kBN12qV7dOO5c8nnihb.3 2024-05-30T16:34:51.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.400+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.84__multipart_key1.2~8A85IpnQGhw6kBN12qV7dOO5c8nnihb.4 2024-05-30T16:34:51.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.400+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.84__multipart_key1.2~8A85IpnQGhw6kBN12qV7dOO5c8nnihb.5 2024-05-30T16:34:51.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.400+0000 7fefcb112980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.5389732858653583418', time=2024-05-30T18:33:09.562901+0000, chain.objs.size()=5 2024-05-30T16:34:51.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.400+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.92__multipart_key1.2~fIzD8n-Lpa17d0qwqvbUVw-pDoCw82F.1 2024-05-30T16:34:51.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.400+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.92__shadow_key1.2~fIzD8n-Lpa17d0qwqvbUVw-pDoCw82F.1_1 2024-05-30T16:34:51.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.400+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.92__multipart_key1.2~fIzD8n-Lpa17d0qwqvbUVw-pDoCw82F.2 2024-05-30T16:34:51.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.400+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.92__shadow_key1.2~fIzD8n-Lpa17d0qwqvbUVw-pDoCw82F.2_1 2024-05-30T16:34:51.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.400+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.92__multipart_key1.2~fIzD8n-Lpa17d0qwqvbUVw-pDoCw82F.3 2024-05-30T16:34:51.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.408+0000 7fefcb112980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:34:51.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.416+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-05-30T16:34:51.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.420+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.420+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.420+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.424+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-05-30T16:34:51.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.424+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.425 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.424+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.428+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.428+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-05-30T16:34:51.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.432+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.432+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.436+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:34:51.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.436+0000 7fefcb112980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2837688741934135408', time=2024-05-30T18:32:01.332738+0000, chain.objs.size()=5 2024-05-30T16:34:51.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.436+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.84__multipart_key1.2~FbUXYw4G_H6NTDsDGfGQJsMss8tBucY.1 2024-05-30T16:34:51.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.436+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.84__shadow_key1.2~FbUXYw4G_H6NTDsDGfGQJsMss8tBucY.1_1 2024-05-30T16:34:51.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.436+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.84__multipart_key1.2~FbUXYw4G_H6NTDsDGfGQJsMss8tBucY.2 2024-05-30T16:34:51.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.436+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.84__shadow_key1.2~FbUXYw4G_H6NTDsDGfGQJsMss8tBucY.2_1 2024-05-30T16:34:51.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.436+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.84__multipart_key1.2~FbUXYw4G_H6NTDsDGfGQJsMss8tBucY.3 2024-05-30T16:34:51.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.440+0000 7fefcb112980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:34:51.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.448+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-05-30T16:34:51.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.448+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.452+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.451 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.452+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.452+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-05-30T16:34:51.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.456+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.456+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.460+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:34:51.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.460+0000 7fefcb112980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.4540623370845440528', time=2024-05-30T18:33:00.427715+0000, chain.objs.size()=5 2024-05-30T16:34:51.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.460+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.92__multipart_key1.2~N3r_yWLy334yLFKtZqFQwQeq4tGnbA_.1 2024-05-30T16:34:51.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.460+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.92__multipart_key1.2~N3r_yWLy334yLFKtZqFQwQeq4tGnbA_.2 2024-05-30T16:34:51.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.460+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.92__multipart_key1.2~N3r_yWLy334yLFKtZqFQwQeq4tGnbA_.3 2024-05-30T16:34:51.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.460+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.92__multipart_key1.2~N3r_yWLy334yLFKtZqFQwQeq4tGnbA_.4 2024-05-30T16:34:51.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.460+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.92__multipart_key1.2~N3r_yWLy334yLFKtZqFQwQeq4tGnbA_.5 2024-05-30T16:34:51.464 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.464+0000 7fefcb112980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:34:51.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.468+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-05-30T16:34:51.472 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.472+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.472+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.472+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:34:51.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.472+0000 7fefcb112980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.14660749664834244321', time=2024-05-30T18:32:08.557402+0000, chain.objs.size()=5 2024-05-30T16:34:51.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.472+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.88__multipart_key1.2~K5ty2t7xFjPdQzHNiq7XtmL8fSb8Ogs.1 2024-05-30T16:34:51.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.472+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.88__multipart_key1.2~K5ty2t7xFjPdQzHNiq7XtmL8fSb8Ogs.2 2024-05-30T16:34:51.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.472+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.88__multipart_key1.2~K5ty2t7xFjPdQzHNiq7XtmL8fSb8Ogs.3 2024-05-30T16:34:51.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.476+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.88__multipart_key1.2~K5ty2t7xFjPdQzHNiq7XtmL8fSb8Ogs.4 2024-05-30T16:34:51.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.476+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.88__multipart_key1.2~K5ty2t7xFjPdQzHNiq7XtmL8fSb8Ogs.5 2024-05-30T16:34:51.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.484+0000 7fefcb112980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:34:51.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.488+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-05-30T16:34:51.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.492+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.492 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.492+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.492+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:34:51.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.492+0000 7fefcb112980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.81__multipart_key1.2~ZnqURvy6t6Oo9TfflYFZ_OYFeNK_xVX.2_536a913faf00e0325f3def8a572867c6', time=2024-05-30T18:31:19.428876+0000, chain.objs.size()=5 2024-05-30T16:34:51.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.492+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.81__multipart_key1.2~ZnqURvy6t6Oo9TfflYFZ_OYFeNK_xVX.1 2024-05-30T16:34:51.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.492+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.81__multipart_key1.2~ZnqURvy6t6Oo9TfflYFZ_OYFeNK_xVX.2 2024-05-30T16:34:51.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.492+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.81__multipart_key1.2~ZnqURvy6t6Oo9TfflYFZ_OYFeNK_xVX.3 2024-05-30T16:34:51.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.492+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.81__multipart_key1.2~ZnqURvy6t6Oo9TfflYFZ_OYFeNK_xVX.4 2024-05-30T16:34:51.494 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.492+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.81__multipart_key1.2~ZnqURvy6t6Oo9TfflYFZ_OYFeNK_xVX.5 2024-05-30T16:34:51.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.496+0000 7fefcb112980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:34:51.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.504+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-05-30T16:34:51.504 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.504+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.505 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.504+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.508+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-30T16:34:51.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.508+0000 7fefcb112980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.870100198774778656', time=2024-05-30T18:31:18.777762+0000, chain.objs.size()=5 2024-05-30T16:34:51.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.508+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.81__multipart_key1.2~ZnqURvy6t6Oo9TfflYFZ_OYFeNK_xVX.1 2024-05-30T16:34:51.506 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.508+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.81__multipart_key1.2~ZnqURvy6t6Oo9TfflYFZ_OYFeNK_xVX.2 2024-05-30T16:34:51.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.508+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.81__multipart_key1.2~ZnqURvy6t6Oo9TfflYFZ_OYFeNK_xVX.3 2024-05-30T16:34:51.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.508+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.81__multipart_key1.2~ZnqURvy6t6Oo9TfflYFZ_OYFeNK_xVX.4 2024-05-30T16:34:51.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.508+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.81__multipart_key1.2~ZnqURvy6t6Oo9TfflYFZ_OYFeNK_xVX.5 2024-05-30T16:34:51.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.508+0000 7fefcb112980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.3573737598703515434', time=2024-05-30T18:32:00.400266+0000, chain.objs.size()=5 2024-05-30T16:34:51.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.508+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.84__multipart_key1.2~_5Nw_wxunYWkTy-6lSAazu3yvGh2MlZ.1 2024-05-30T16:34:51.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.508+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.84__shadow_key1.2~_5Nw_wxunYWkTy-6lSAazu3yvGh2MlZ.1_1 2024-05-30T16:34:51.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.508+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.84__multipart_key1.2~_5Nw_wxunYWkTy-6lSAazu3yvGh2MlZ.2 2024-05-30T16:34:51.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.508+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.84__shadow_key1.2~_5Nw_wxunYWkTy-6lSAazu3yvGh2MlZ.2_1 2024-05-30T16:34:51.507 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.508+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.84__multipart_key1.2~_5Nw_wxunYWkTy-6lSAazu3yvGh2MlZ.3 2024-05-30T16:34:51.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.512+0000 7fefcb112980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:34:51.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.520+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-05-30T16:34:51.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.524+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.524+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.524+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:34:51.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.524+0000 7fefcb112980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.4498939390609945824', time=2024-05-30T18:34:36.037305+0000, chain.objs.size()=5 2024-05-30T16:34:51.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.524+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.133__multipart_key1.2~tNb9lbZxeiogGUPQimpbQkKzi6mJzCC.1 2024-05-30T16:34:51.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.524+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.133__multipart_key1.2~tNb9lbZxeiogGUPQimpbQkKzi6mJzCC.2 2024-05-30T16:34:51.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.524+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.133__multipart_key1.2~tNb9lbZxeiogGUPQimpbQkKzi6mJzCC.3 2024-05-30T16:34:51.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.524+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.133__multipart_key1.2~tNb9lbZxeiogGUPQimpbQkKzi6mJzCC.4 2024-05-30T16:34:51.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.524+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.133__multipart_key1.2~tNb9lbZxeiogGUPQimpbQkKzi6mJzCC.5 2024-05-30T16:34:51.529 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.528+0000 7fefcb112980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:34:51.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.536+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-05-30T16:34:51.538 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.540+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.540+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.540+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.544+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-05-30T16:34:51.545 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.544+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.546 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.548+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.548+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.549 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.548+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-05-30T16:34:51.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.552+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.552+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.556+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:34:51.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.556+0000 7fefcb112980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.1127798118535109208', time=2024-05-30T18:33:19.645117+0000, chain.objs.size()=5 2024-05-30T16:34:51.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.556+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.93__multipart_key1.2~XbCy81h_0F6JOKdvmVPDuxTNTIO55ye.1 2024-05-30T16:34:51.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.556+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.93__multipart_key1.2~XbCy81h_0F6JOKdvmVPDuxTNTIO55ye.2 2024-05-30T16:34:51.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.556+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.93__multipart_key1.2~XbCy81h_0F6JOKdvmVPDuxTNTIO55ye.3 2024-05-30T16:34:51.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.556+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.93__multipart_key1.2~XbCy81h_0F6JOKdvmVPDuxTNTIO55ye.4 2024-05-30T16:34:51.555 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.556+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.93__multipart_key1.2~XbCy81h_0F6JOKdvmVPDuxTNTIO55ye.5 2024-05-30T16:34:51.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.560+0000 7fefcb112980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:34:51.565 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.564+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-05-30T16:34:51.568 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.568+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.568+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.568+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.572 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.572+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-05-30T16:34:51.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.576+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.576 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.576+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.577 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.576+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.580 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.580+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-05-30T16:34:51.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.584+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.585 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.584+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.588+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-05-30T16:34:51.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.588+0000 7fefcb112980 20 garbage collection: RGWGC::process iterating over entry tag='2~iSHyfc8peOBmOyBUkravTsb8_--iW5_', time=2024-05-30T18:32:43.063092+0000, chain.objs.size()=5 2024-05-30T16:34:51.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.588+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.91__multipart_key1.2~iSHyfc8peOBmOyBUkravTsb8_--iW5_.7 2024-05-30T16:34:51.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.588+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.91__multipart_key1.2~iSHyfc8peOBmOyBUkravTsb8_--iW5_.8 2024-05-30T16:34:51.587 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.588+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.91__multipart_key1.2~iSHyfc8peOBmOyBUkravTsb8_--iW5_.9 2024-05-30T16:34:51.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.588+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.91__multipart_key1.2~iSHyfc8peOBmOyBUkravTsb8_--iW5_.10 2024-05-30T16:34:51.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.588+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.91__multipart_key1.2~iSHyfc8peOBmOyBUkravTsb8_--iW5_.11 2024-05-30T16:34:51.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.588+0000 7fefcb112980 20 garbage collection: RGWGC::process iterating over entry tag='2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-', time=2024-05-30T18:33:38.716716+0000, chain.objs.size()=1 2024-05-30T16:34:51.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.588+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.20 2024-05-30T16:34:51.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.588+0000 7fefcb112980 20 garbage collection: RGWGC::process iterating over entry tag='2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-', time=2024-05-30T18:33:38.871756+0000, chain.objs.size()=75 2024-05-30T16:34:51.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.588+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.1 2024-05-30T16:34:51.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.588+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.2 2024-05-30T16:34:51.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.588+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.3 2024-05-30T16:34:51.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.588+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.4 2024-05-30T16:34:51.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.588+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.5 2024-05-30T16:34:51.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.588+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.6 2024-05-30T16:34:51.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.592+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.7 2024-05-30T16:34:51.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.592+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.8 2024-05-30T16:34:51.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.596+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.9 2024-05-30T16:34:51.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.596+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.10 2024-05-30T16:34:51.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.596+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.11 2024-05-30T16:34:51.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.596+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.12 2024-05-30T16:34:51.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.596+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.13 2024-05-30T16:34:51.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.596+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.14 2024-05-30T16:34:51.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.596+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.15 2024-05-30T16:34:51.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.604+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.16 2024-05-30T16:34:51.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.604+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.17 2024-05-30T16:34:51.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.604+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.18 2024-05-30T16:34:51.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.608+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.19 2024-05-30T16:34:51.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.608+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.Q8hjIAIf885z-8v_8bmtF1gsIxI2fgeg.20 2024-05-30T16:34:51.606 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.608+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.21 2024-05-30T16:34:51.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.608+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.22 2024-05-30T16:34:51.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.608+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.23 2024-05-30T16:34:51.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.608+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.24 2024-05-30T16:34:51.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.608+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.25 2024-05-30T16:34:51.609 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.608+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.26 2024-05-30T16:34:51.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.612+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.27 2024-05-30T16:34:51.611 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.612+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.28 2024-05-30T16:34:51.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.616+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.29 2024-05-30T16:34:51.615 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.616+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.30 2024-05-30T16:34:51.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.616+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.31 2024-05-30T16:34:51.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.616+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.32 2024-05-30T16:34:51.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.616+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.33 2024-05-30T16:34:51.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.616+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.34 2024-05-30T16:34:51.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.616+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.35 2024-05-30T16:34:51.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.616+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.36 2024-05-30T16:34:51.616 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.616+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.37 2024-05-30T16:34:51.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.620+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.38 2024-05-30T16:34:51.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.620+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.39 2024-05-30T16:34:51.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.624+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.40 2024-05-30T16:34:51.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.624+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.41 2024-05-30T16:34:51.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.624+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.42 2024-05-30T16:34:51.624 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.624+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.43 2024-05-30T16:34:51.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.624+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.44 2024-05-30T16:34:51.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.624+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.45 2024-05-30T16:34:51.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.628+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.46 2024-05-30T16:34:51.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.628+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.47 2024-05-30T16:34:51.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.628+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.48 2024-05-30T16:34:51.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.628+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.49 2024-05-30T16:34:51.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.628+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.50 2024-05-30T16:34:51.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.636+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.51 2024-05-30T16:34:51.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.636+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.52 2024-05-30T16:34:51.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.636+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.53 2024-05-30T16:34:51.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.636+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.54 2024-05-30T16:34:51.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.636+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.55 2024-05-30T16:34:51.635 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.636+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.56 2024-05-30T16:34:51.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.636+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.57 2024-05-30T16:34:51.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.636+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.58 2024-05-30T16:34:51.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.636+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.59 2024-05-30T16:34:51.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.636+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.60 2024-05-30T16:34:51.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.636+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.61 2024-05-30T16:34:51.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.640+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.62 2024-05-30T16:34:51.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.640+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.63 2024-05-30T16:34:51.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.640+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.64 2024-05-30T16:34:51.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.640+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.65 2024-05-30T16:34:51.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.640+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.66 2024-05-30T16:34:51.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.640+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.67 2024-05-30T16:34:51.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.640+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.68 2024-05-30T16:34:51.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.640+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.69 2024-05-30T16:34:51.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.640+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.70 2024-05-30T16:34:51.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.640+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.71 2024-05-30T16:34:51.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.640+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.72 2024-05-30T16:34:51.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.644+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.73 2024-05-30T16:34:51.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.648+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.74 2024-05-30T16:34:51.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.648+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.95__multipart_key1.2~uzZbVvrbkjL92ghxIKFvqzdWXwZoEx-.77 2024-05-30T16:34:51.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.656+0000 7fefcb112980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:34:51.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.656+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-05-30T16:34:51.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.660+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.660+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.660+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:34:51.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.660+0000 7fefcb112980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.13039402703485770795', time=2024-05-30T18:31:42.410154+0000, chain.objs.size()=5 2024-05-30T16:34:51.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.660+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.84__multipart_key1.2~-g2SSSjYFakFaYmawUe1fH7PwGu51W-.1 2024-05-30T16:34:51.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.660+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.84__multipart_key1.2~-g2SSSjYFakFaYmawUe1fH7PwGu51W-.2 2024-05-30T16:34:51.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.660+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.84__multipart_key1.2~-g2SSSjYFakFaYmawUe1fH7PwGu51W-.3 2024-05-30T16:34:51.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.660+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.84__multipart_key1.2~-g2SSSjYFakFaYmawUe1fH7PwGu51W-.4 2024-05-30T16:34:51.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.660+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.84__multipart_key1.2~-g2SSSjYFakFaYmawUe1fH7PwGu51W-.5 2024-05-30T16:34:51.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.664+0000 7fefcb112980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:34:51.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.668+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-05-30T16:34:51.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.672+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.672+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.672 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.672+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-30T16:34:51.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.672+0000 7fefcb112980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.3459388679995413388', time=2024-05-30T18:32:16.263177+0000, chain.objs.size()=5 2024-05-30T16:34:51.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.672+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.89__multipart_key1.2~5mIKPXSi7m7RmhJLCWESK0CLz09w1IZ.1 2024-05-30T16:34:51.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.672+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.89__multipart_key1.2~5mIKPXSi7m7RmhJLCWESK0CLz09w1IZ.2 2024-05-30T16:34:51.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.672+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.89__multipart_key1.2~5mIKPXSi7m7RmhJLCWESK0CLz09w1IZ.3 2024-05-30T16:34:51.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.672+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.89__multipart_key1.2~5mIKPXSi7m7RmhJLCWESK0CLz09w1IZ.4 2024-05-30T16:34:51.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.672+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.89__multipart_key1.2~5mIKPXSi7m7RmhJLCWESK0CLz09w1IZ.5 2024-05-30T16:34:51.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.672+0000 7fefcb112980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.3481205809266426657', time=2024-05-30T18:33:03.737353+0000, chain.objs.size()=5 2024-05-30T16:34:51.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.672+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.92__multipart_key1.2~wMZgUCxgvVmJnHtG7E8iogCN-AQi75M.1 2024-05-30T16:34:51.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.672+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.92__multipart_key1.2~wMZgUCxgvVmJnHtG7E8iogCN-AQi75M.2 2024-05-30T16:34:51.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.672+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.92__multipart_key1.2~wMZgUCxgvVmJnHtG7E8iogCN-AQi75M.3 2024-05-30T16:34:51.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.672+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.92__multipart_key1.2~wMZgUCxgvVmJnHtG7E8iogCN-AQi75M.4 2024-05-30T16:34:51.675 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.672+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.92__multipart_key1.2~wMZgUCxgvVmJnHtG7E8iogCN-AQi75M.5 2024-05-30T16:34:51.678 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.680+0000 7fefcb112980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:34:51.684 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.684+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-05-30T16:34:51.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.684+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.686 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.688+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.688+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:34:51.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.688+0000 7fefcb112980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18167817478457640456', time=2024-05-30T18:33:06.701230+0000, chain.objs.size()=5 2024-05-30T16:34:51.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.688+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.92__multipart_key1.2~RQ5O5i6GHz9aWE4gcouIRqZodg8Jo3l.1 2024-05-30T16:34:51.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.688+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.92__multipart_key1.2~RQ5O5i6GHz9aWE4gcouIRqZodg8Jo3l.2 2024-05-30T16:34:51.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.688+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.92__multipart_key1.2~RQ5O5i6GHz9aWE4gcouIRqZodg8Jo3l.3 2024-05-30T16:34:51.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.688+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.92__multipart_key1.2~RQ5O5i6GHz9aWE4gcouIRqZodg8Jo3l.4 2024-05-30T16:34:51.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.688+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.92__multipart_key1.2~RQ5O5i6GHz9aWE4gcouIRqZodg8Jo3l.5 2024-05-30T16:34:51.692 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.692+0000 7fefcb112980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:34:51.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.696+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-05-30T16:34:51.698 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.700+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.700+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.700+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:34:51.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.700+0000 7fefcb112980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.10902703247765796324', time=2024-05-30T18:33:13.569583+0000, chain.objs.size()=5 2024-05-30T16:34:51.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.700+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.92__multipart_key1.2~aget0Z1kXnZN6S5gGT54KDlYlEeFdMr.1 2024-05-30T16:34:51.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.700+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.92__shadow_key1.2~aget0Z1kXnZN6S5gGT54KDlYlEeFdMr.1_1 2024-05-30T16:34:51.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.700+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.92__multipart_key1.2~aget0Z1kXnZN6S5gGT54KDlYlEeFdMr.2 2024-05-30T16:34:51.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.700+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.92__shadow_key1.2~aget0Z1kXnZN6S5gGT54KDlYlEeFdMr.2_1 2024-05-30T16:34:51.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.700+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.92__multipart_key1.2~aget0Z1kXnZN6S5gGT54KDlYlEeFdMr.3 2024-05-30T16:34:51.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.704+0000 7fefcb112980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:34:51.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.708+0000 7fefcb112980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-05-30T16:34:51.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.708+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:51.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.708+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:51.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.712+0000 7fefcb112980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T16:34:51.710 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.712+0000 7fefcb112980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16105189200341993535', time=2024-05-30T18:34:50.151016+0000, chain.objs.size()=5 2024-05-30T16:34:51.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.712+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.134__multipart_key1/data/file.mpg.2~Rhz9gJBLNtpeJqncp0KOHw0DIRDSQTJ.1 2024-05-30T16:34:51.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.712+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.134__multipart_key1/data/file.mpg.2~Rhz9gJBLNtpeJqncp0KOHw0DIRDSQTJ.2 2024-05-30T16:34:51.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.712+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.134__multipart_key1/data/file.mpg.2~Rhz9gJBLNtpeJqncp0KOHw0DIRDSQTJ.3 2024-05-30T16:34:51.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.712+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.134__multipart_key1/data/file.mpg.2~Rhz9gJBLNtpeJqncp0KOHw0DIRDSQTJ.4 2024-05-30T16:34:51.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.712+0000 7fefcb112980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.134__multipart_key1/data/file.mpg.2~Rhz9gJBLNtpeJqncp0KOHw0DIRDSQTJ.5 2024-05-30T16:34:51.715 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.716+0000 7fefcb112980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T16:34:51.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.720+0000 7fee7f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:51.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.724+0000 7fefcb112980 20 remove_watcher() i=0 2024-05-30T16:34:51.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.724+0000 7fefcb112980 2 removed watcher, disabling cache 2024-05-30T16:34:51.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.724+0000 7fefcb112980 20 remove_watcher() i=1 2024-05-30T16:34:51.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.728+0000 7fefcb112980 20 remove_watcher() i=2 2024-05-30T16:34:51.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.728+0000 7fefcb112980 20 remove_watcher() i=3 2024-05-30T16:34:51.730 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.732+0000 7fefcb112980 20 remove_watcher() i=4 2024-05-30T16:34:51.731 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.732+0000 7fefcb112980 20 remove_watcher() i=5 2024-05-30T16:34:51.733 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.732+0000 7fefcb112980 20 remove_watcher() i=6 2024-05-30T16:34:51.734 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.736+0000 7fefcb112980 20 remove_watcher() i=7 2024-05-30T16:34:51.750 DEBUG:teuthology.orchestra.run.smithi006:> radosgw-admin gc process --include-all 2024-05-30T16:34:51.826 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:34:51.902 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:34:51.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.872+0000 7f8105a21980 20 rados->read ofs=0 len=0 2024-05-30T16:34:51.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.876+0000 7f8105a21980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:51.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.876+0000 7f8105a21980 20 realm 2024-05-30T16:34:51.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.876+0000 7f8105a21980 20 rados->read ofs=0 len=0 2024-05-30T16:34:51.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.876+0000 7f8105a21980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:51.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.876+0000 7f8105a21980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:34:51.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.876+0000 7f8105a21980 20 rados->read ofs=0 len=0 2024-05-30T16:34:51.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.876+0000 7f8105a21980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:51.906 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.876+0000 7f8105a21980 20 rados->read ofs=0 len=0 2024-05-30T16:34:51.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.876+0000 7f8105a21980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:51.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.876+0000 7f8105a21980 20 rados->read ofs=0 len=0 2024-05-30T16:34:51.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.876+0000 7f8105a21980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:34:51.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.876+0000 7f8105a21980 20 searching for the correct realm 2024-05-30T16:34:51.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.884+0000 7f80fad31640 0 -- 172.21.15.6:0/1109917883 >> [v2:172.21.15.6:6819/1141406589,v1:172.21.15.6:6821/1141406589] conn(0x555648c4b8b0 msgr2=0x555648c4dd70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:34:51.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.888+0000 7f8103532640 0 -- 172.21.15.6:0/1109917883 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f80ec056150 msgr2=0x7f80ec06b040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:34:51.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.900+0000 7f8105a21980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:34:51.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.900+0000 7f8105a21980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:34:51.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.900+0000 7f8105a21980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:34:51.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.900+0000 7f8105a21980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:34:51.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.900+0000 7f8105a21980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:34:51.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.900+0000 7f8105a21980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:34:51.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.900+0000 7f8105a21980 20 rados->read ofs=0 len=0 2024-05-30T16:34:51.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.900+0000 7f8105a21980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:51.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.900+0000 7f8105a21980 20 rados->read ofs=0 len=0 2024-05-30T16:34:51.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.900+0000 7f8105a21980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:51.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.900+0000 7f8105a21980 20 rados->read ofs=0 len=0 2024-05-30T16:34:51.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.900+0000 7f8105a21980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:34:51.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.900+0000 7f8105a21980 20 zone default found 2024-05-30T16:34:51.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.900+0000 7f8105a21980 4 Realm: () 2024-05-30T16:34:51.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.900+0000 7f8105a21980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:34:51.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.900+0000 7f8105a21980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:34:51.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.900+0000 7f8105a21980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:34:51.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.900+0000 7f8105a21980 20 zonegroup default 2024-05-30T16:34:51.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.900+0000 7f8105a21980 20 rados->read ofs=0 len=0 2024-05-30T16:34:51.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.900+0000 7f8105a21980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:51.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.900+0000 7f8105a21980 20 rados->read ofs=0 len=0 2024-05-30T16:34:51.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.900+0000 7f8105a21980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:51.909 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.900+0000 7f8105a21980 20 rados->read ofs=0 len=0 2024-05-30T16:34:51.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.900+0000 7f8105a21980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:51.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.900+0000 7f8105a21980 20 started sync module instance, tier type = 2024-05-30T16:34:51.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.900+0000 7f8105a21980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:34:51.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.916+0000 7f8105a21980 20 add_watcher() i=0 2024-05-30T16:34:51.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.916+0000 7f8105a21980 20 add_watcher() i=1 2024-05-30T16:34:51.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.916+0000 7f8105a21980 20 add_watcher() i=2 2024-05-30T16:34:51.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.916+0000 7f8105a21980 20 add_watcher() i=3 2024-05-30T16:34:51.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.916+0000 7f8105a21980 20 add_watcher() i=4 2024-05-30T16:34:51.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.916+0000 7f8105a21980 20 add_watcher() i=5 2024-05-30T16:34:51.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.916+0000 7f8105a21980 20 add_watcher() i=6 2024-05-30T16:34:51.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.920+0000 7f8105a21980 20 add_watcher() i=7 2024-05-30T16:34:51.918 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.920+0000 7f8105a21980 2 all 8 watchers are set, enabling cache 2024-05-30T16:34:51.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.920+0000 7f7fb9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:51.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.920+0000 7f8105a21980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:34:51.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.920+0000 7f8105a21980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:34:51.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.996+0000 7f7fb8fe1640 20 reqs_thread_entry: start 2024-05-30T16:34:51.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.996+0000 7f8105a21980 20 init_complete bucket index max shards: 11 2024-05-30T16:34:51.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.996+0000 7f8105a21980 20 Filter name: none 2024-05-30T16:34:51.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.996+0000 7f7f327fc640 20 reqs_thread_entry: start 2024-05-30T16:34:51.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:51.996+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-05-30T16:34:51.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.000+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.000+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.000+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.004+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-05-30T16:34:52.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.004+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.006 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.008+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.008+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.008+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-05-30T16:34:52.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.012+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.012+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.016+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.017 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.016+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-05-30T16:34:52.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.020+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.023 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.024+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.024+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.027 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.028+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-05-30T16:34:52.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.028+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.030 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.032+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.032+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.032+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-05-30T16:34:52.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.036+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.038 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.040+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.040+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.044+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-05-30T16:34:52.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.044+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.044+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.044+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.048+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-05-30T16:34:52.051 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.052+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.052+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.053 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.052+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.056+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-05-30T16:34:52.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.060+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.060+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.060+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.064 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.064+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-05-30T16:34:52.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.068+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.068+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.069 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.068+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.072+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-05-30T16:34:52.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.076+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.076+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.080+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.084+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-05-30T16:34:52.087 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.088+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.088+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.088+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.092+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-05-30T16:34:52.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.096+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.096+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.098 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.100+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.100+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-05-30T16:34:52.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.104+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.108+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.108+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.112+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-05-30T16:34:52.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.116+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.116+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.116+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.120+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-05-30T16:34:52.124 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.124+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.124+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.128+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.129 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.128+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-05-30T16:34:52.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.132+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.136+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.136+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.140+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-05-30T16:34:52.143 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.144+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.144+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.144+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.148+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-05-30T16:34:52.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.152+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.156+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.156+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.160+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-05-30T16:34:52.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.164+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.164+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.165 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.164+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.168+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-05-30T16:34:52.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.172+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.172+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.174 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.176+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.176+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-05-30T16:34:52.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.180+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.180+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.184+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.184+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-05-30T16:34:52.189 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.188+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.190 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.192+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.192+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.192+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-05-30T16:34:52.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.196+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.200+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.200+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.204+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-05-30T16:34:52.206 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.208+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.208+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.208+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.212+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-05-30T16:34:52.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.216+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.218 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.216+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.220+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.224+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-05-30T16:34:52.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.224+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.228+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.228+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.232+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-05-30T16:34:52.235 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.236+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.236 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.236+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.237 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.236+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.240+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-05-30T16:34:52.244 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.244+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.244+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.246 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.248+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.248+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-05-30T16:34:52.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.252+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.256+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.256+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.260+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-05-30T16:34:52.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.260+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.263 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.264+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.264+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.267 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.268+0000 7f8105a21980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-05-30T16:34:52.272 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.272+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.276+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T16:34:52.275 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.276+0000 7f8105a21980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T16:34:52.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.280+0000 7f7fb9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:52.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.288+0000 7f8105a21980 20 remove_watcher() i=0 2024-05-30T16:34:52.286 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.288+0000 7f8105a21980 2 removed watcher, disabling cache 2024-05-30T16:34:52.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.288+0000 7f8105a21980 20 remove_watcher() i=1 2024-05-30T16:34:52.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.292+0000 7f8105a21980 20 remove_watcher() i=2 2024-05-30T16:34:52.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.296+0000 7f8105a21980 20 remove_watcher() i=3 2024-05-30T16:34:52.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.296+0000 7f8105a21980 20 remove_watcher() i=4 2024-05-30T16:34:52.299 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.300+0000 7f8105a21980 20 remove_watcher() i=5 2024-05-30T16:34:52.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.300+0000 7f8105a21980 20 remove_watcher() i=6 2024-05-30T16:34:52.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:52.304+0000 7f8105a21980 20 remove_watcher() i=7 2024-05-30T16:34:52.328 INFO:teuthology.run_tasks:Running task s3tests... 2024-05-30T16:34:52.354 DEBUG:tasks.s3tests:s3tests config is {'client.0': {'rgw_server': 'client.0', 'accounts': {'iam alt root': 'RGW99999999999999999', 'iam root': 'RGW88888888888888888', 's3 main': 'RGW11111111111111111', 's3 tenant': 'RGW33333333333333333'}, 'force-branch': 'ceph-master', 'slow_backend': True, 'storage classes': 'LUKEWARM, FROZEN'}} 2024-05-30T16:34:52.354 INFO:tasks.s3tests:Downloading s3-tests... 2024-05-30T16:34:52.354 INFO:tasks.s3tests:Using branch 'ceph-master' for s3tests 2024-05-30T16:34:52.355 DEBUG:teuthology.orchestra.run.smithi006:> git clone -b ceph-master https://git.ceph.com/s3-tests.git /home/ubuntu/cephtest/s3-tests-client.0 2024-05-30T16:34:52.375 INFO:teuthology.orchestra.run.smithi006.stderr:Cloning into '/home/ubuntu/cephtest/s3-tests-client.0'... 2024-05-30T16:34:53.281 INFO:tasks.s3tests:Creating rgw users... 2024-05-30T16:34:53.281 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph account create --account-id RGW11111111111111111 --account-name Mr.RGW11111111111111111 --email RGW11111111111111111+test@test.test 2024-05-30T16:34:53.363 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:34:53.363 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:34:53.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.404+0000 7fdd489ee980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.408+0000 7fdd489ee980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:53.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.408+0000 7fdd489ee980 20 realm 2024-05-30T16:34:53.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.408+0000 7fdd489ee980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.408+0000 7fdd489ee980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:53.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.408+0000 7fdd489ee980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:34:53.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.408+0000 7fdd489ee980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.408+0000 7fdd489ee980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:53.409 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.408+0000 7fdd489ee980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.412+0000 7fdd489ee980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:53.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.412+0000 7fdd489ee980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.412+0000 7fdd489ee980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:34:53.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.412+0000 7fdd489ee980 20 searching for the correct realm 2024-05-30T16:34:53.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.416+0000 7fdd46d00640 0 -- 172.21.15.6:0/3339609408 >> [v2:172.21.15.6:6808/1897362715,v1:172.21.15.6:6809/1897362715] conn(0x5631c4112110 msgr2=0x5631c41145a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:34:53.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.424+0000 7fdd464ff640 0 -- 172.21.15.6:0/3339609408 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7fdd3c092230 msgr2=0x7fdd3c0095e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:34:53.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.432+0000 7fdd489ee980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:34:53.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.432+0000 7fdd489ee980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:34:53.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.432+0000 7fdd489ee980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:34:53.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.432+0000 7fdd489ee980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:34:53.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.432+0000 7fdd489ee980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:34:53.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.432+0000 7fdd489ee980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:34:53.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.432+0000 7fdd489ee980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.432+0000 7fdd489ee980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:53.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.432+0000 7fdd489ee980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.432+0000 7fdd489ee980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:53.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.432+0000 7fdd489ee980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.436+0000 7fdd489ee980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:34:53.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.436+0000 7fdd489ee980 20 zone default found 2024-05-30T16:34:53.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.436+0000 7fdd489ee980 4 Realm: () 2024-05-30T16:34:53.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.436+0000 7fdd489ee980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:34:53.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.436+0000 7fdd489ee980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:34:53.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.436+0000 7fdd489ee980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:34:53.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.436+0000 7fdd489ee980 20 zonegroup default 2024-05-30T16:34:53.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.436+0000 7fdd489ee980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.436+0000 7fdd489ee980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:53.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.436+0000 7fdd489ee980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.436+0000 7fdd489ee980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:53.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.436+0000 7fdd489ee980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.436+0000 7fdd489ee980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:53.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.436+0000 7fdd489ee980 20 started sync module instance, tier type = 2024-05-30T16:34:53.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.436+0000 7fdd489ee980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:34:53.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.452+0000 7fdd489ee980 20 add_watcher() i=0 2024-05-30T16:34:53.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.452+0000 7fdd489ee980 20 add_watcher() i=1 2024-05-30T16:34:53.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.452+0000 7fdd489ee980 20 add_watcher() i=2 2024-05-30T16:34:53.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.452+0000 7fdd489ee980 20 add_watcher() i=3 2024-05-30T16:34:53.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.452+0000 7fdd489ee980 20 add_watcher() i=4 2024-05-30T16:34:53.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.452+0000 7fdd489ee980 20 add_watcher() i=5 2024-05-30T16:34:53.453 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.452+0000 7fdd489ee980 20 add_watcher() i=6 2024-05-30T16:34:53.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.456+0000 7fdd489ee980 20 add_watcher() i=7 2024-05-30T16:34:53.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.456+0000 7fdd489ee980 2 all 8 watchers are set, enabling cache 2024-05-30T16:34:53.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.456+0000 7fdbfcfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:53.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.456+0000 7fdd489ee980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:34:53.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.456+0000 7fdd489ee980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:34:53.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.456+0000 7fdd489ee980 5 note: GC not initialized 2024-05-30T16:34:53.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.456+0000 7fdb777fe640 20 reqs_thread_entry: start 2024-05-30T16:34:53.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.456+0000 7fdd489ee980 20 init_complete bucket index max shards: 11 2024-05-30T16:34:53.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.456+0000 7fdd489ee980 20 Filter name: none 2024-05-30T16:34:53.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.456+0000 7fdb757fa640 20 reqs_thread_entry: start 2024-05-30T16:34:53.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.456+0000 7fdd489ee980 10 cache get: name=default.rgw.meta+accounts+name.$Mr.RGW11111111111111111 : miss 2024-05-30T16:34:53.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.456+0000 7fdd489ee980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.460+0000 7fdd489ee980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:53.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.460+0000 7fdd489ee980 10 cache put: name=default.rgw.meta+accounts+name.$Mr.RGW11111111111111111 info.flags=0x0 2024-05-30T16:34:53.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.460+0000 7fdd489ee980 10 adding default.rgw.meta+accounts+name.$Mr.RGW11111111111111111 to cache LRU end 2024-05-30T16:34:53.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.460+0000 7fdd489ee980 20 failed to read name.$Mr.RGW11111111111111111 with: (2) No such file or directory 2024-05-30T16:34:53.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.460+0000 7fdd489ee980 10 cache get: name=default.rgw.meta+users.email+rgw11111111111111111+test@test.test : miss 2024-05-30T16:34:53.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.460+0000 7fdd489ee980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.460+0000 7fdd489ee980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:53.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.460+0000 7fdd489ee980 10 cache put: name=default.rgw.meta+users.email+rgw11111111111111111+test@test.test info.flags=0x0 2024-05-30T16:34:53.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.460+0000 7fdd489ee980 10 adding default.rgw.meta+users.email+rgw11111111111111111+test@test.test to cache LRU end 2024-05-30T16:34:53.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.460+0000 7fdd489ee980 20 failed to read rgw11111111111111111+test@test.test with: (2) No such file or directory 2024-05-30T16:34:53.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.460+0000 7fdd489ee980 10 cache put: name=default.rgw.meta+accounts+account.RGW11111111111111111 info.flags=0x17 2024-05-30T16:34:53.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.460+0000 7fdd489ee980 10 adding default.rgw.meta+accounts+account.RGW11111111111111111 to cache LRU end 2024-05-30T16:34:53.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.460+0000 7fdd489ee980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:accounts:account.RGW11111111111111111, ofs0, ns] 2024-05-30T16:34:53.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.460+0000 7fdc29ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431705 cookie 94771742895488 notifier 7175 bl.length()=384 2024-05-30T16:34:53.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.460+0000 7fdc29ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+account.RGW11111111111111111 info.flags=0x17 2024-05-30T16:34:53.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.460+0000 7fdc29ffb640 10 rgw watcher librados: moving default.rgw.meta+accounts+account.RGW11111111111111111 to cache LRU end 2024-05-30T16:34:53.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.468+0000 7fdd489ee980 10 cache put: name=default.rgw.meta+accounts+name.$Mr.RGW11111111111111111 info.flags=0x17 2024-05-30T16:34:53.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.468+0000 7fdd489ee980 10 moving default.rgw.meta+accounts+name.$Mr.RGW11111111111111111 to cache LRU end 2024-05-30T16:34:53.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.468+0000 7fdd489ee980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:accounts:name.$Mr.RGW11111111111111111, ofs0, ns] 2024-05-30T16:34:53.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.468+0000 7fdc297fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431700 cookie 94771742848896 notifier 7175 bl.length()=225 2024-05-30T16:34:53.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.468+0000 7fdc297fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+name.$Mr.RGW11111111111111111 info.flags=0x17 2024-05-30T16:34:53.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.468+0000 7fdc297fa640 10 rgw watcher librados: moving default.rgw.meta+accounts+name.$Mr.RGW11111111111111111 to cache LRU end 2024-05-30T16:34:53.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.472+0000 7fdd489ee980 10 cache put: name=default.rgw.meta+users.email+rgw11111111111111111+test@test.test info.flags=0x17 2024-05-30T16:34:53.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.472+0000 7fdd489ee980 10 moving default.rgw.meta+users.email+rgw11111111111111111+test@test.test to cache LRU end 2024-05-30T16:34:53.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.472+0000 7fdd489ee980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:rgw11111111111111111+test@test.test, ofs0, ns] 2024-05-30T16:34:53.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.472+0000 7fdc297fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431641 cookie 94771742901840 notifier 7175 bl.length()=234 2024-05-30T16:34:53.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.472+0000 7fdc297fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+rgw11111111111111111+test@test.test info.flags=0x17 2024-05-30T16:34:53.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.472+0000 7fdc297fa640 10 rgw watcher librados: moving default.rgw.meta+users.email+rgw11111111111111111+test@test.test to cache LRU end 2024-05-30T16:34:53.474 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.476+0000 7fdbfcfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:53.478 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.480+0000 7fdd489ee980 20 remove_watcher() i=0 2024-05-30T16:34:53.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.480+0000 7fdd489ee980 2 removed watcher, disabling cache 2024-05-30T16:34:53.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.480+0000 7fdd489ee980 20 remove_watcher() i=1 2024-05-30T16:34:53.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.484+0000 7fdd489ee980 20 remove_watcher() i=2 2024-05-30T16:34:53.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.484+0000 7fdd489ee980 20 remove_watcher() i=3 2024-05-30T16:34:53.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.488+0000 7fdd489ee980 20 remove_watcher() i=4 2024-05-30T16:34:53.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.488+0000 7fdd489ee980 20 remove_watcher() i=5 2024-05-30T16:34:53.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.488+0000 7fdd489ee980 20 remove_watcher() i=6 2024-05-30T16:34:53.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.492+0000 7fdd489ee980 20 remove_watcher() i=7 2024-05-30T16:34:53.503 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-30T16:34:53.504 INFO:teuthology.orchestra.run.smithi006.stdout: "id": "RGW11111111111111111", 2024-05-30T16:34:53.504 INFO:teuthology.orchestra.run.smithi006.stdout: "tenant": "", 2024-05-30T16:34:53.504 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "Mr.RGW11111111111111111", 2024-05-30T16:34:53.504 INFO:teuthology.orchestra.run.smithi006.stdout: "email": "RGW11111111111111111+test@test.test", 2024-05-30T16:34:53.504 INFO:teuthology.orchestra.run.smithi006.stdout: "quota": { 2024-05-30T16:34:53.504 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": false, 2024-05-30T16:34:53.504 INFO:teuthology.orchestra.run.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:34:53.504 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size": -1, 2024-05-30T16:34:53.504 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:34:53.504 INFO:teuthology.orchestra.run.smithi006.stdout: "max_objects": -1 2024-05-30T16:34:53.504 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T16:34:53.504 INFO:teuthology.orchestra.run.smithi006.stdout: "bucket_quota": { 2024-05-30T16:34:53.504 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": false, 2024-05-30T16:34:53.504 INFO:teuthology.orchestra.run.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:34:53.505 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size": -1, 2024-05-30T16:34:53.505 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:34:53.505 INFO:teuthology.orchestra.run.smithi006.stdout: "max_objects": -1 2024-05-30T16:34:53.505 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T16:34:53.505 INFO:teuthology.orchestra.run.smithi006.stdout: "max_users": 1000, 2024-05-30T16:34:53.505 INFO:teuthology.orchestra.run.smithi006.stdout: "max_roles": 1000, 2024-05-30T16:34:53.505 INFO:teuthology.orchestra.run.smithi006.stdout: "max_groups": 1000, 2024-05-30T16:34:53.505 INFO:teuthology.orchestra.run.smithi006.stdout: "max_buckets": 1000, 2024-05-30T16:34:53.505 INFO:teuthology.orchestra.run.smithi006.stdout: "max_access_keys": 4 2024-05-30T16:34:53.505 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-30T16:34:53.507 DEBUG:tasks.s3tests:Creating user RGW11111111111111111 on client.0 2024-05-30T16:34:53.507 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid foo.client.0 --display-name Mr.RGW11111111111111111 --email foo+test@test.test --access-key JDTXYSCQPBCLVKUVUOWS --secret Gmw5wtP0YlqURs0YLn1qs2k2G1ACJBwG0tMe1g6CJFvbzNDOHIk4Yw== --cluster ceph --account-id RGW11111111111111111 --account-root 2024-05-30T16:34:53.593 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:34:53.593 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:34:53.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.636+0000 7f0d88720980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.636+0000 7f0d88720980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:53.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.636+0000 7f0d88720980 20 realm 2024-05-30T16:34:53.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.636+0000 7f0d88720980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.636+0000 7f0d88720980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:53.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.636+0000 7f0d88720980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:34:53.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.636+0000 7f0d88720980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.638 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.640+0000 7f0d88720980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:53.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.640+0000 7f0d88720980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.640+0000 7f0d88720980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:53.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.640+0000 7f0d88720980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.640+0000 7f0d88720980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:34:53.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.640+0000 7f0d88720980 20 searching for the correct realm 2024-05-30T16:34:53.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.648+0000 7f0d85a30640 0 -- 172.21.15.6:0/1384650131 >> [v2:172.21.15.6:6819/1141406589,v1:172.21.15.6:6821/1141406589] conn(0x55ab386d9df0 msgr2=0x55ab386dc2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:34:53.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.656+0000 7f0d85a30640 0 -- 172.21.15.6:0/1384650131 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x55ab386d9df0 msgr2=0x7f0c500832a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:34:53.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.664+0000 7f0d88720980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:34:53.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.664+0000 7f0d88720980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:34:53.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.664+0000 7f0d88720980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:34:53.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.664+0000 7f0d88720980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:34:53.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.664+0000 7f0d88720980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:34:53.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.664+0000 7f0d88720980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:34:53.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.664+0000 7f0d88720980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.663 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.664+0000 7f0d88720980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:53.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.664+0000 7f0d88720980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.664+0000 7f0d88720980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:53.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.664+0000 7f0d88720980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.664+0000 7f0d88720980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:34:53.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.664+0000 7f0d88720980 20 zone default found 2024-05-30T16:34:53.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.664+0000 7f0d88720980 4 Realm: () 2024-05-30T16:34:53.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.664+0000 7f0d88720980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:34:53.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.664+0000 7f0d88720980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:34:53.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.664+0000 7f0d88720980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:34:53.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.664+0000 7f0d88720980 20 zonegroup default 2024-05-30T16:34:53.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.664+0000 7f0d88720980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.664+0000 7f0d88720980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:53.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.664+0000 7f0d88720980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.668+0000 7f0d88720980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:53.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.668+0000 7f0d88720980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.668+0000 7f0d88720980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:53.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.668+0000 7f0d88720980 20 started sync module instance, tier type = 2024-05-30T16:34:53.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.668+0000 7f0d88720980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:34:53.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.684+0000 7f0d88720980 20 add_watcher() i=0 2024-05-30T16:34:53.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.684+0000 7f0d88720980 20 add_watcher() i=1 2024-05-30T16:34:53.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.684+0000 7f0d88720980 20 add_watcher() i=2 2024-05-30T16:34:53.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.684+0000 7f0d88720980 20 add_watcher() i=3 2024-05-30T16:34:53.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.684+0000 7f0d88720980 20 add_watcher() i=4 2024-05-30T16:34:53.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.684+0000 7f0d88720980 20 add_watcher() i=5 2024-05-30T16:34:53.682 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.684+0000 7f0d88720980 20 add_watcher() i=6 2024-05-30T16:34:53.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.684+0000 7f0d88720980 20 add_watcher() i=7 2024-05-30T16:34:53.683 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.684+0000 7f0d88720980 2 all 8 watchers are set, enabling cache 2024-05-30T16:34:53.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.684+0000 7f0c3cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:53.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.688+0000 7f0d88720980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:34:53.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.688+0000 7f0d88720980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:34:53.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.688+0000 7f0d88720980 5 note: GC not initialized 2024-05-30T16:34:53.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.688+0000 7f0bb77fe640 20 reqs_thread_entry: start 2024-05-30T16:34:53.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.688+0000 7f0d88720980 20 init_complete bucket index max shards: 11 2024-05-30T16:34:53.687 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.688+0000 7f0d88720980 20 Filter name: none 2024-05-30T16:34:53.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.688+0000 7f0bb57fa640 20 reqs_thread_entry: start 2024-05-30T16:34:53.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.688+0000 7f0d88720980 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2024-05-30T16:34:53.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.688+0000 7f0d88720980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.688+0000 7f0d88720980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:53.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.688+0000 7f0d88720980 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x0 2024-05-30T16:34:53.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.688+0000 7f0d88720980 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-05-30T16:34:53.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.688+0000 7f0d88720980 10 cache get: name=default.rgw.meta+users.email+foo+test@test.test : miss 2024-05-30T16:34:53.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.688+0000 7f0d88720980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.688+0000 7f0d88720980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:53.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.688+0000 7f0d88720980 10 cache put: name=default.rgw.meta+users.email+foo+test@test.test info.flags=0x0 2024-05-30T16:34:53.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.688+0000 7f0d88720980 10 adding default.rgw.meta+users.email+foo+test@test.test to cache LRU end 2024-05-30T16:34:53.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.688+0000 7f0d88720980 10 cache get: name=default.rgw.meta+users.keys+JDTXYSCQPBCLVKUVUOWS : miss 2024-05-30T16:34:53.688 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.688+0000 7f0d88720980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.688+0000 7f0d88720980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:53.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.688+0000 7f0d88720980 10 cache put: name=default.rgw.meta+users.keys+JDTXYSCQPBCLVKUVUOWS info.flags=0x0 2024-05-30T16:34:53.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.688+0000 7f0d88720980 10 adding default.rgw.meta+users.keys+JDTXYSCQPBCLVKUVUOWS to cache LRU end 2024-05-30T16:34:53.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.688+0000 7f0d88720980 10 cache get: name=default.rgw.meta+accounts+account.RGW11111111111111111 : miss 2024-05-30T16:34:53.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.688+0000 7f0d88720980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.688+0000 7f0d88720980 20 rados_obj.operate() r=0 bl.length=184 2024-05-30T16:34:53.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.688+0000 7f0d88720980 10 cache put: name=default.rgw.meta+accounts+account.RGW11111111111111111 info.flags=0x17 2024-05-30T16:34:53.689 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.688+0000 7f0d88720980 10 adding default.rgw.meta+accounts+account.RGW11111111111111111 to cache LRU end 2024-05-30T16:34:53.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.688+0000 7f0d88720980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:34:53.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.688+0000 7f0d88720980 10 cache get: name=default.rgw.meta+users.keys+JDTXYSCQPBCLVKUVUOWS : hit (negative entry) 2024-05-30T16:34:53.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.688+0000 7f0d88720980 10 cache get: name=default.rgw.meta+users.keys+JDTXYSCQPBCLVKUVUOWS : hit (negative entry) 2024-05-30T16:34:53.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.692+0000 7f0d88720980 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2024-05-30T16:34:53.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.692+0000 7f0d88720980 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-05-30T16:34:53.690 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.692+0000 7f0d88720980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.uid:foo.client.0, ofs0, ns] 2024-05-30T16:34:53.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.692+0000 7f0c69ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431701 cookie 94193874482480 notifier 7190 bl.length()=673 2024-05-30T16:34:53.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.692+0000 7f0c69ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2024-05-30T16:34:53.691 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.692+0000 7f0c69ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-05-30T16:34:53.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.696+0000 7f0d88720980 10 cache put: name=default.rgw.meta+users.email+foo+test@test.test info.flags=0x7 2024-05-30T16:34:53.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.696+0000 7f0d88720980 10 moving default.rgw.meta+users.email+foo+test@test.test to cache LRU end 2024-05-30T16:34:53.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.696+0000 7f0d88720980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.email:foo+test@test.test, ofs0, ns] 2024-05-30T16:34:53.695 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.696+0000 7f0c69ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431702 cookie 94193874500992 notifier 7190 bl.length()=185 2024-05-30T16:34:53.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.696+0000 7f0c69ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+foo+test@test.test info.flags=0x7 2024-05-30T16:34:53.696 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.696+0000 7f0c69ffb640 10 rgw watcher librados: moving default.rgw.meta+users.email+foo+test@test.test to cache LRU end 2024-05-30T16:34:53.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.700+0000 7f0d88720980 10 cache put: name=default.rgw.meta+users.keys+JDTXYSCQPBCLVKUVUOWS info.flags=0x7 2024-05-30T16:34:53.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.700+0000 7f0d88720980 10 moving default.rgw.meta+users.keys+JDTXYSCQPBCLVKUVUOWS to cache LRU end 2024-05-30T16:34:53.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.700+0000 7f0d88720980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.keys:JDTXYSCQPBCLVKUVUOWS, ofs0, ns] 2024-05-30T16:34:53.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.700+0000 7f0c69ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431642 cookie 94193874496720 notifier 7190 bl.length()=186 2024-05-30T16:34:53.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.700+0000 7f0c69ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+JDTXYSCQPBCLVKUVUOWS info.flags=0x7 2024-05-30T16:34:53.700 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.700+0000 7f0c69ffb640 10 rgw watcher librados: moving default.rgw.meta+users.keys+JDTXYSCQPBCLVKUVUOWS to cache LRU end 2024-05-30T16:34:53.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.704+0000 7f0d88720980 20 linked user foo.client.0 to account RGW11111111111111111 2024-05-30T16:34:53.703 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-30T16:34:53.703 INFO:teuthology.orchestra.run.smithi006.stdout: "user_id": "foo.client.0", 2024-05-30T16:34:53.703 INFO:teuthology.orchestra.run.smithi006.stdout: "display_name": "Mr.RGW11111111111111111", 2024-05-30T16:34:53.703 INFO:teuthology.orchestra.run.smithi006.stdout: "email": "foo+test@test.test", 2024-05-30T16:34:53.703 INFO:teuthology.orchestra.run.smithi006.stdout: "suspended": 0, 2024-05-30T16:34:53.703 INFO:teuthology.orchestra.run.smithi006.stdout: "max_buckets": 1000, 2024-05-30T16:34:53.703 INFO:teuthology.orchestra.run.smithi006.stdout: "subusers": [], 2024-05-30T16:34:53.703 INFO:teuthology.orchestra.run.smithi006.stdout: "keys": [ 2024-05-30T16:34:53.703 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T16:34:53.703 INFO:teuthology.orchestra.run.smithi006.stdout: "user": "foo.client.0", 2024-05-30T16:34:53.703 INFO:teuthology.orchestra.run.smithi006.stdout: "access_key": "JDTXYSCQPBCLVKUVUOWS", 2024-05-30T16:34:53.704 INFO:teuthology.orchestra.run.smithi006.stdout: "secret_key": "Gmw5wtP0YlqURs0YLn1qs2k2G1ACJBwG0tMe1g6CJFvbzNDOHIk4Yw==", 2024-05-30T16:34:53.704 INFO:teuthology.orchestra.run.smithi006.stdout: "active": true, 2024-05-30T16:34:53.704 INFO:teuthology.orchestra.run.smithi006.stdout: "create_date": "2024-05-30T16:34:53.690998Z" 2024-05-30T16:34:53.704 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T16:34:53.704 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-30T16:34:53.704 INFO:teuthology.orchestra.run.smithi006.stdout: "swift_keys": [], 2024-05-30T16:34:53.704 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": [], 2024-05-30T16:34:53.704 INFO:teuthology.orchestra.run.smithi006.stdout: "op_mask": "read, write, delete", 2024-05-30T16:34:53.704 INFO:teuthology.orchestra.run.smithi006.stdout: "default_placement": "", 2024-05-30T16:34:53.704 INFO:teuthology.orchestra.run.smithi006.stdout: "default_storage_class": "", 2024-05-30T16:34:53.704 INFO:teuthology.orchestra.run.smithi006.stdout: "placement_tags": [], 2024-05-30T16:34:53.704 INFO:teuthology.orchestra.run.smithi006.stdout: "bucket_quota": { 2024-05-30T16:34:53.704 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": false, 2024-05-30T16:34:53.704 INFO:teuthology.orchestra.run.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:34:53.705 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size": -1, 2024-05-30T16:34:53.705 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:34:53.705 INFO:teuthology.orchestra.run.smithi006.stdout: "max_objects": -1 2024-05-30T16:34:53.705 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T16:34:53.705 INFO:teuthology.orchestra.run.smithi006.stdout: "user_quota": { 2024-05-30T16:34:53.705 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": false, 2024-05-30T16:34:53.705 INFO:teuthology.orchestra.run.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:34:53.705 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size": -1, 2024-05-30T16:34:53.705 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:34:53.705 INFO:teuthology.orchestra.run.smithi006.stdout: "max_objects": -1 2024-05-30T16:34:53.705 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T16:34:53.705 INFO:teuthology.orchestra.run.smithi006.stdout: "temp_url_keys": [], 2024-05-30T16:34:53.705 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "root", 2024-05-30T16:34:53.705 INFO:teuthology.orchestra.run.smithi006.stdout: "mfa_ids": [], 2024-05-30T16:34:53.705 INFO:teuthology.orchestra.run.smithi006.stdout: "account_id": "RGW11111111111111111", 2024-05-30T16:34:53.705 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2024-05-30T16:34:53.706 INFO:teuthology.orchestra.run.smithi006.stdout: "create_date": "2024-05-30T16:34:53.690980Z", 2024-05-30T16:34:53.706 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-30T16:34:53.706 INFO:teuthology.orchestra.run.smithi006.stdout: "group_ids": [] 2024-05-30T16:34:53.706 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-30T16:34:53.706 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:53.706 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.704+0000 7f0c3cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:53.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.708+0000 7f0d88720980 20 remove_watcher() i=0 2024-05-30T16:34:53.709 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.708+0000 7f0d88720980 2 removed watcher, disabling cache 2024-05-30T16:34:53.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.712+0000 7f0d88720980 20 remove_watcher() i=1 2024-05-30T16:34:53.713 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.712+0000 7f0d88720980 20 remove_watcher() i=2 2024-05-30T16:34:53.716 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.716+0000 7f0d88720980 20 remove_watcher() i=3 2024-05-30T16:34:53.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.716+0000 7f0d88720980 20 remove_watcher() i=4 2024-05-30T16:34:53.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.720+0000 7f0d88720980 20 remove_watcher() i=5 2024-05-30T16:34:53.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.720+0000 7f0d88720980 20 remove_watcher() i=6 2024-05-30T16:34:53.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.720+0000 7f0d88720980 20 remove_watcher() i=7 2024-05-30T16:34:53.737 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid foo.client.0 --totp-serial 5815199363 --totp-seed QFD3FDS6J3TSWW46SMJYVXN4UCM3ON5WFRY4WINAL6HR7NLAZC7VRFDETCBBWIYC --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-05-30T16:34:53.822 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:34:53.822 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:34:53.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.868+0000 7fc52b2ca980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.868+0000 7fc52b2ca980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:53.868 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.868+0000 7fc52b2ca980 20 realm 2024-05-30T16:34:53.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.868+0000 7fc52b2ca980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.868+0000 7fc52b2ca980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:53.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.868+0000 7fc52b2ca980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:34:53.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.868+0000 7fc52b2ca980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.868+0000 7fc52b2ca980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:53.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.868+0000 7fc52b2ca980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.872+0000 7fc52b2ca980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:53.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.872+0000 7fc52b2ca980 20 rados->read ofs=0 len=0 2024-05-30T16:34:53.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.872+0000 7fc52b2ca980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:34:53.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.872+0000 7fc52b2ca980 20 searching for the correct realm 2024-05-30T16:34:53.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.876+0000 7fc5285da640 0 -- 172.21.15.6:0/2142707386 >> [v2:172.21.15.6:6819/1141406589,v1:172.21.15.6:6821/1141406589] conn(0x562902cde340 msgr2=0x562902ce0800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:34:53.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:53.884+0000 7fc528ddb640 0 -- 172.21.15.6:0/2142707386 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7fc51806d150 msgr2=0x7fc5180651a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:34:54.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.100+0000 7fc52b2ca980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:34:54.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.100+0000 7fc52b2ca980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:34:54.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.100+0000 7fc52b2ca980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:34:54.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.100+0000 7fc52b2ca980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:34:54.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.100+0000 7fc52b2ca980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:34:54.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.100+0000 7fc52b2ca980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:34:54.102 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.100+0000 7fc52b2ca980 20 rados->read ofs=0 len=0 2024-05-30T16:34:54.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.104+0000 7fc52b2ca980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:54.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.104+0000 7fc52b2ca980 20 rados->read ofs=0 len=0 2024-05-30T16:34:54.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.104+0000 7fc52b2ca980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:54.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.104+0000 7fc52b2ca980 20 rados->read ofs=0 len=0 2024-05-30T16:34:54.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.104+0000 7fc52b2ca980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:34:54.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.104+0000 7fc52b2ca980 20 zone default found 2024-05-30T16:34:54.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.104+0000 7fc52b2ca980 4 Realm: () 2024-05-30T16:34:54.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.104+0000 7fc52b2ca980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:34:54.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.104+0000 7fc52b2ca980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:34:54.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.104+0000 7fc52b2ca980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:34:54.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.104+0000 7fc52b2ca980 20 zonegroup default 2024-05-30T16:34:54.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.104+0000 7fc52b2ca980 20 rados->read ofs=0 len=0 2024-05-30T16:34:54.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.104+0000 7fc52b2ca980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:54.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.104+0000 7fc52b2ca980 20 rados->read ofs=0 len=0 2024-05-30T16:34:54.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.108+0000 7fc52b2ca980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:54.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.108+0000 7fc52b2ca980 20 rados->read ofs=0 len=0 2024-05-30T16:34:54.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.108+0000 7fc52b2ca980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:54.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.108+0000 7fc52b2ca980 20 started sync module instance, tier type = 2024-05-30T16:34:54.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.108+0000 7fc52b2ca980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:34:54.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.132+0000 7fc52b2ca980 20 add_watcher() i=0 2024-05-30T16:34:54.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.132+0000 7fc52b2ca980 20 add_watcher() i=1 2024-05-30T16:34:54.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.132+0000 7fc52b2ca980 20 add_watcher() i=2 2024-05-30T16:34:54.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.132+0000 7fc52b2ca980 20 add_watcher() i=3 2024-05-30T16:34:54.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.132+0000 7fc52b2ca980 20 add_watcher() i=4 2024-05-30T16:34:54.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.132+0000 7fc52b2ca980 20 add_watcher() i=5 2024-05-30T16:34:54.131 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.132+0000 7fc52b2ca980 20 add_watcher() i=6 2024-05-30T16:34:54.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.132+0000 7fc52b2ca980 20 add_watcher() i=7 2024-05-30T16:34:54.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.132+0000 7fc52b2ca980 2 all 8 watchers are set, enabling cache 2024-05-30T16:34:54.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.136+0000 7fc3defe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:54.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.136+0000 7fc52b2ca980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:34:54.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.136+0000 7fc52b2ca980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:34:54.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.136+0000 7fc52b2ca980 5 note: GC not initialized 2024-05-30T16:34:54.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.140+0000 7fc3ddfe3640 20 reqs_thread_entry: start 2024-05-30T16:34:54.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.140+0000 7fc52b2ca980 20 init_complete bucket index max shards: 11 2024-05-30T16:34:54.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.140+0000 7fc52b2ca980 20 Filter name: none 2024-05-30T16:34:54.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.140+0000 7fc34f7fe640 20 reqs_thread_entry: start 2024-05-30T16:34:54.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.140+0000 7fc52b2ca980 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2024-05-30T16:34:54.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.140+0000 7fc52b2ca980 20 rados->read ofs=0 len=0 2024-05-30T16:34:54.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.140+0000 7fc52b2ca980 20 rados_obj.operate() r=0 bl.length=488 2024-05-30T16:34:54.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.140+0000 7fc52b2ca980 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x13 2024-05-30T16:34:54.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.140+0000 7fc52b2ca980 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-05-30T16:34:54.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:54.140+0000 7fc52b2ca980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:34:55.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:55.896+0000 7fc52b2ca980 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2024-05-30T16:34:55.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:55.896+0000 7fc52b2ca980 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-05-30T16:34:55.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:55.896+0000 7fc52b2ca980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.uid:foo.client.0, ofs0, ns] 2024-05-30T16:34:55.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:55.896+0000 7fc3ff7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366295 cookie 94734140701312 notifier 7653 bl.length()=687 2024-05-30T16:34:55.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:55.896+0000 7fc3ff7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2024-05-30T16:34:55.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:55.896+0000 7fc3ff7fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-05-30T16:34:55.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:55.904+0000 7fc3defe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:55.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:55.908+0000 7fc52b2ca980 20 remove_watcher() i=0 2024-05-30T16:34:55.908 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:55.908+0000 7fc52b2ca980 2 removed watcher, disabling cache 2024-05-30T16:34:55.911 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:55.912+0000 7fc52b2ca980 20 remove_watcher() i=1 2024-05-30T16:34:55.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:55.912+0000 7fc52b2ca980 20 remove_watcher() i=2 2024-05-30T16:34:55.916 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:55.916+0000 7fc52b2ca980 20 remove_watcher() i=3 2024-05-30T16:34:55.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:55.920+0000 7fc52b2ca980 20 remove_watcher() i=4 2024-05-30T16:34:55.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:55.920+0000 7fc52b2ca980 20 remove_watcher() i=5 2024-05-30T16:34:55.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:55.924+0000 7fc52b2ca980 20 remove_watcher() i=6 2024-05-30T16:34:55.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:55.924+0000 7fc52b2ca980 20 remove_watcher() i=7 2024-05-30T16:34:55.947 DEBUG:tasks.s3tests:Creating user bar.client.0 on client.0 2024-05-30T16:34:55.948 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid bar.client.0 --display-name Mr.bar.client.0 --email bar+test@test.test --access-key CLKXCRNCVHZTWVSJGNGN --secret 03Lg9M3lx9RzN3jCj57+dPo+BvfS0pQ+pEUgaWkG8x31okHqW0uaNw== --cluster ceph 2024-05-30T16:34:56.036 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:34:56.036 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:34:56.078 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.080+0000 7fcba78b4980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.080 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.080+0000 7fcba78b4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:56.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.080+0000 7fcba78b4980 20 realm 2024-05-30T16:34:56.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.080+0000 7fcba78b4980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.080+0000 7fcba78b4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:56.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.080+0000 7fcba78b4980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:34:56.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.080+0000 7fcba78b4980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.080+0000 7fcba78b4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:56.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.080+0000 7fcba78b4980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.084+0000 7fcba78b4980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:56.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.084+0000 7fcba78b4980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.084+0000 7fcba78b4980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:34:56.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.084+0000 7fcba78b4980 20 searching for the correct realm 2024-05-30T16:34:56.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.296+0000 7fcba5bc6640 0 -- 172.21.15.6:0/2595497388 >> [v2:172.21.15.204:6823/3156311527,v1:172.21.15.204:6825/3156311527] conn(0x7fcb9001a050 msgr2=0x7fcb900961f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:34:56.304 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.304+0000 7fcba53c5640 0 -- 172.21.15.6:0/2595497388 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7fcb9007b790 msgr2=0x7fcb9006a050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:34:56.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.520+0000 7fcba78b4980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:34:56.521 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.520+0000 7fcba78b4980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:34:56.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.520+0000 7fcba78b4980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:34:56.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.520+0000 7fcba78b4980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:34:56.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.520+0000 7fcba78b4980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:34:56.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.520+0000 7fcba78b4980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:34:56.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.520+0000 7fcba78b4980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.520+0000 7fcba78b4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:56.522 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.520+0000 7fcba78b4980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.524+0000 7fcba78b4980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:56.523 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.524+0000 7fcba78b4980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.524+0000 7fcba78b4980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:34:56.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.524+0000 7fcba78b4980 20 zone default found 2024-05-30T16:34:56.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.524+0000 7fcba78b4980 4 Realm: () 2024-05-30T16:34:56.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.524+0000 7fcba78b4980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:34:56.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.524+0000 7fcba78b4980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:34:56.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.524+0000 7fcba78b4980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:34:56.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.524+0000 7fcba78b4980 20 zonegroup default 2024-05-30T16:34:56.524 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.524+0000 7fcba78b4980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.524+0000 7fcba78b4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:56.525 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.524+0000 7fcba78b4980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.524+0000 7fcba78b4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:56.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.524+0000 7fcba78b4980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.528+0000 7fcba78b4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:56.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.528+0000 7fcba78b4980 20 started sync module instance, tier type = 2024-05-30T16:34:56.526 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.528+0000 7fcba78b4980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:34:56.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.552+0000 7fcba78b4980 20 add_watcher() i=0 2024-05-30T16:34:56.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.552+0000 7fcba78b4980 20 add_watcher() i=1 2024-05-30T16:34:56.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.552+0000 7fcba78b4980 20 add_watcher() i=2 2024-05-30T16:34:56.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.552+0000 7fcba78b4980 20 add_watcher() i=3 2024-05-30T16:34:56.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.552+0000 7fcba78b4980 20 add_watcher() i=4 2024-05-30T16:34:56.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.552+0000 7fcba78b4980 20 add_watcher() i=5 2024-05-30T16:34:56.551 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.552+0000 7fcba78b4980 20 add_watcher() i=6 2024-05-30T16:34:56.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.552+0000 7fcba78b4980 20 add_watcher() i=7 2024-05-30T16:34:56.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.552+0000 7fcba78b4980 2 all 8 watchers are set, enabling cache 2024-05-30T16:34:56.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.556+0000 7fca5b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:56.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.556+0000 7fcba78b4980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:34:56.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.556+0000 7fcba78b4980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:34:56.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.556+0000 7fcba78b4980 5 note: GC not initialized 2024-05-30T16:34:56.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.556+0000 7fca5a7e4640 20 reqs_thread_entry: start 2024-05-30T16:34:56.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.556+0000 7fcba78b4980 20 init_complete bucket index max shards: 11 2024-05-30T16:34:56.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.556+0000 7fcba78b4980 20 Filter name: none 2024-05-30T16:34:56.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.556+0000 7fc9cbfff640 20 reqs_thread_entry: start 2024-05-30T16:34:56.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.556+0000 7fcba78b4980 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2024-05-30T16:34:56.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.556+0000 7fcba78b4980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.560+0000 7fcba78b4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:56.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.560+0000 7fcba78b4980 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x0 2024-05-30T16:34:56.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.560+0000 7fcba78b4980 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-05-30T16:34:56.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.560+0000 7fcba78b4980 10 cache get: name=default.rgw.meta+users.email+bar+test@test.test : miss 2024-05-30T16:34:56.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.560+0000 7fcba78b4980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.560+0000 7fcba78b4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:56.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.560+0000 7fcba78b4980 10 cache put: name=default.rgw.meta+users.email+bar+test@test.test info.flags=0x0 2024-05-30T16:34:56.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.560+0000 7fcba78b4980 10 adding default.rgw.meta+users.email+bar+test@test.test to cache LRU end 2024-05-30T16:34:56.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.560+0000 7fcba78b4980 10 cache get: name=default.rgw.meta+users.keys+CLKXCRNCVHZTWVSJGNGN : miss 2024-05-30T16:34:56.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.560+0000 7fcba78b4980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.560+0000 7fcba78b4980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:56.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.560+0000 7fcba78b4980 10 cache put: name=default.rgw.meta+users.keys+CLKXCRNCVHZTWVSJGNGN info.flags=0x0 2024-05-30T16:34:56.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.560+0000 7fcba78b4980 10 adding default.rgw.meta+users.keys+CLKXCRNCVHZTWVSJGNGN to cache LRU end 2024-05-30T16:34:56.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.560+0000 7fcba78b4980 10 cache get: name=default.rgw.meta+users.keys+CLKXCRNCVHZTWVSJGNGN : hit (negative entry) 2024-05-30T16:34:56.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.560+0000 7fcba78b4980 10 cache get: name=default.rgw.meta+users.keys+CLKXCRNCVHZTWVSJGNGN : hit (negative entry) 2024-05-30T16:34:56.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.564+0000 7fcba78b4980 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2024-05-30T16:34:56.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.564+0000 7fcba78b4980 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-05-30T16:34:56.563 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.564+0000 7fcba78b4980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:bar.client.0, ofs0, ns] 2024-05-30T16:34:56.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.564+0000 7fca88ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366298 cookie 94387206099904 notifier 7671 bl.length()=645 2024-05-30T16:34:56.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.564+0000 7fca88ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2024-05-30T16:34:56.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.564+0000 7fca88ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-05-30T16:34:56.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.568+0000 7fcba78b4980 10 cache put: name=default.rgw.meta+users.email+bar+test@test.test info.flags=0x7 2024-05-30T16:34:56.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.568+0000 7fcba78b4980 10 moving default.rgw.meta+users.email+bar+test@test.test to cache LRU end 2024-05-30T16:34:56.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.568+0000 7fcba78b4980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.email:bar+test@test.test, ofs0, ns] 2024-05-30T16:34:56.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.568+0000 7fca88ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366328 cookie 94387206107584 notifier 7671 bl.length()=185 2024-05-30T16:34:56.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.568+0000 7fca88ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+bar+test@test.test info.flags=0x7 2024-05-30T16:34:56.569 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.568+0000 7fca88ff9640 10 rgw watcher librados: moving default.rgw.meta+users.email+bar+test@test.test to cache LRU end 2024-05-30T16:34:56.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.576+0000 7fcba78b4980 10 cache put: name=default.rgw.meta+users.keys+CLKXCRNCVHZTWVSJGNGN info.flags=0x7 2024-05-30T16:34:56.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.576+0000 7fcba78b4980 10 moving default.rgw.meta+users.keys+CLKXCRNCVHZTWVSJGNGN to cache LRU end 2024-05-30T16:34:56.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.576+0000 7fcba78b4980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:CLKXCRNCVHZTWVSJGNGN, ofs0, ns] 2024-05-30T16:34:56.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.576+0000 7fca77fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366329 cookie 94387206107584 notifier 7671 bl.length()=186 2024-05-30T16:34:56.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.576+0000 7fca77fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+CLKXCRNCVHZTWVSJGNGN info.flags=0x7 2024-05-30T16:34:56.575 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.576+0000 7fca77fff640 10 rgw watcher librados: moving default.rgw.meta+users.keys+CLKXCRNCVHZTWVSJGNGN to cache LRU end 2024-05-30T16:34:56.577 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-30T16:34:56.577 INFO:teuthology.orchestra.run.smithi006.stdout: "user_id": "bar.client.0", 2024-05-30T16:34:56.578 INFO:teuthology.orchestra.run.smithi006.stdout: "display_name": "Mr.bar.client.0", 2024-05-30T16:34:56.578 INFO:teuthology.orchestra.run.smithi006.stdout: "email": "bar+test@test.test", 2024-05-30T16:34:56.578 INFO:teuthology.orchestra.run.smithi006.stdout: "suspended": 0, 2024-05-30T16:34:56.578 INFO:teuthology.orchestra.run.smithi006.stdout: "max_buckets": 1000, 2024-05-30T16:34:56.578 INFO:teuthology.orchestra.run.smithi006.stdout: "subusers": [], 2024-05-30T16:34:56.578 INFO:teuthology.orchestra.run.smithi006.stdout: "keys": [ 2024-05-30T16:34:56.578 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T16:34:56.578 INFO:teuthology.orchestra.run.smithi006.stdout: "user": "bar.client.0", 2024-05-30T16:34:56.578 INFO:teuthology.orchestra.run.smithi006.stdout: "access_key": "CLKXCRNCVHZTWVSJGNGN", 2024-05-30T16:34:56.578 INFO:teuthology.orchestra.run.smithi006.stdout: "secret_key": "03Lg9M3lx9RzN3jCj57+dPo+BvfS0pQ+pEUgaWkG8x31okHqW0uaNw==", 2024-05-30T16:34:56.578 INFO:teuthology.orchestra.run.smithi006.stdout: "active": true, 2024-05-30T16:34:56.578 INFO:teuthology.orchestra.run.smithi006.stdout: "create_date": "2024-05-30T16:34:56.562008Z" 2024-05-30T16:34:56.578 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T16:34:56.578 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-30T16:34:56.579 INFO:teuthology.orchestra.run.smithi006.stdout: "swift_keys": [], 2024-05-30T16:34:56.579 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": [], 2024-05-30T16:34:56.579 INFO:teuthology.orchestra.run.smithi006.stdout: "op_mask": "read, write, delete", 2024-05-30T16:34:56.579 INFO:teuthology.orchestra.run.smithi006.stdout: "default_placement": "", 2024-05-30T16:34:56.579 INFO:teuthology.orchestra.run.smithi006.stdout: "default_storage_class": "", 2024-05-30T16:34:56.579 INFO:teuthology.orchestra.run.smithi006.stdout: "placement_tags": [], 2024-05-30T16:34:56.579 INFO:teuthology.orchestra.run.smithi006.stdout: "bucket_quota": { 2024-05-30T16:34:56.579 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": false, 2024-05-30T16:34:56.579 INFO:teuthology.orchestra.run.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:34:56.579 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size": -1, 2024-05-30T16:34:56.579 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:34:56.579 INFO:teuthology.orchestra.run.smithi006.stdout: "max_objects": -1 2024-05-30T16:34:56.579 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T16:34:56.579 INFO:teuthology.orchestra.run.smithi006.stdout: "user_quota": { 2024-05-30T16:34:56.580 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": false, 2024-05-30T16:34:56.580 INFO:teuthology.orchestra.run.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:34:56.580 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size": -1, 2024-05-30T16:34:56.580 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:34:56.580 INFO:teuthology.orchestra.run.smithi006.stdout: "max_objects": -1 2024-05-30T16:34:56.580 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T16:34:56.580 INFO:teuthology.orchestra.run.smithi006.stdout: "temp_url_keys": [], 2024-05-30T16:34:56.580 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "rgw", 2024-05-30T16:34:56.580 INFO:teuthology.orchestra.run.smithi006.stdout: "mfa_ids": [], 2024-05-30T16:34:56.580 INFO:teuthology.orchestra.run.smithi006.stdout: "account_id": "", 2024-05-30T16:34:56.580 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2024-05-30T16:34:56.580 INFO:teuthology.orchestra.run.smithi006.stdout: "create_date": "2024-05-30T16:34:56.561986Z", 2024-05-30T16:34:56.581 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-30T16:34:56.581 INFO:teuthology.orchestra.run.smithi006.stdout: "group_ids": [] 2024-05-30T16:34:56.581 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-30T16:34:56.581 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:56.581 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.580+0000 7fca5b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:56.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.584+0000 7fcba78b4980 20 remove_watcher() i=0 2024-05-30T16:34:56.583 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.584+0000 7fcba78b4980 2 removed watcher, disabling cache 2024-05-30T16:34:56.586 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.584+0000 7fcba78b4980 20 remove_watcher() i=1 2024-05-30T16:34:56.588 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.588+0000 7fcba78b4980 20 remove_watcher() i=2 2024-05-30T16:34:56.590 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.592+0000 7fcba78b4980 20 remove_watcher() i=3 2024-05-30T16:34:56.593 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.592+0000 7fcba78b4980 20 remove_watcher() i=4 2024-05-30T16:34:56.594 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.596+0000 7fcba78b4980 20 remove_watcher() i=5 2024-05-30T16:34:56.595 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.596+0000 7fcba78b4980 20 remove_watcher() i=6 2024-05-30T16:34:56.597 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.596+0000 7fcba78b4980 20 remove_watcher() i=7 2024-05-30T16:34:56.615 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid bar.client.0 --totp-serial 1386153437 --totp-seed LE6UCMDETVQQT6KWENYLA6O6YK4EVVYQCSSJ7L74ROJQVM5QDHKKTINWBDZDWFB5 --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-05-30T16:34:56.656 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:34:56.656 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:34:56.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.700+0000 7fcbe6a3c980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.700+0000 7fcbe6a3c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:56.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.700+0000 7fcbe6a3c980 20 realm 2024-05-30T16:34:56.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.700+0000 7fcbe6a3c980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.700+0000 7fcbe6a3c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:56.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.700+0000 7fcbe6a3c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:34:56.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.700+0000 7fcbe6a3c980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.700+0000 7fcbe6a3c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:56.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.700+0000 7fcbe6a3c980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.704+0000 7fcbe6a3c980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:56.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.704+0000 7fcbe6a3c980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.704+0000 7fcbe6a3c980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:34:56.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.704+0000 7fcbe6a3c980 20 searching for the correct realm 2024-05-30T16:34:56.711 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.712+0000 7fcbe3d4c640 0 -- 172.21.15.6:0/2500145155 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7fcbd4096070 msgr2=0x7fcbd4082ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:34:56.717 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.716+0000 7fcbe3d4c640 0 -- 172.21.15.6:0/2500145155 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7fcaac07f270 msgr2=0x7fcaac0816a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:34:56.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.724+0000 7fcbe6a3c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:34:56.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.724+0000 7fcbe6a3c980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:34:56.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.724+0000 7fcbe6a3c980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:34:56.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.724+0000 7fcbe6a3c980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:34:56.725 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.724+0000 7fcbe6a3c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:34:56.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.724+0000 7fcbe6a3c980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:34:56.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.724+0000 7fcbe6a3c980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.724+0000 7fcbe6a3c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:56.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.724+0000 7fcbe6a3c980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.728+0000 7fcbe6a3c980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:56.726 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.728+0000 7fcbe6a3c980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.728+0000 7fcbe6a3c980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:34:56.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.728+0000 7fcbe6a3c980 20 zone default found 2024-05-30T16:34:56.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.728+0000 7fcbe6a3c980 4 Realm: () 2024-05-30T16:34:56.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.728+0000 7fcbe6a3c980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:34:56.727 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.728+0000 7fcbe6a3c980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:34:56.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.728+0000 7fcbe6a3c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:34:56.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.728+0000 7fcbe6a3c980 20 zonegroup default 2024-05-30T16:34:56.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.728+0000 7fcbe6a3c980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.728+0000 7fcbe6a3c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:56.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.728+0000 7fcbe6a3c980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.728+0000 7fcbe6a3c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:56.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.728+0000 7fcbe6a3c980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.728+0000 7fcbe6a3c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:56.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.728+0000 7fcbe6a3c980 20 started sync module instance, tier type = 2024-05-30T16:34:56.729 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.728+0000 7fcbe6a3c980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:34:56.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.744+0000 7fcbe6a3c980 20 add_watcher() i=0 2024-05-30T16:34:56.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.744+0000 7fcbe6a3c980 20 add_watcher() i=1 2024-05-30T16:34:56.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.744+0000 7fcbe6a3c980 20 add_watcher() i=2 2024-05-30T16:34:56.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.744+0000 7fcbe6a3c980 20 add_watcher() i=3 2024-05-30T16:34:56.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.744+0000 7fcbe6a3c980 20 add_watcher() i=4 2024-05-30T16:34:56.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.744+0000 7fcbe6a3c980 20 add_watcher() i=5 2024-05-30T16:34:56.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.744+0000 7fcbe6a3c980 20 add_watcher() i=6 2024-05-30T16:34:56.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.744+0000 7fcbe6a3c980 20 add_watcher() i=7 2024-05-30T16:34:56.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.744+0000 7fcbe6a3c980 2 all 8 watchers are set, enabling cache 2024-05-30T16:34:56.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.748+0000 7fca9afdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:56.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.748+0000 7fcbe6a3c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:34:56.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.748+0000 7fcbe6a3c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:34:56.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.748+0000 7fcbe6a3c980 5 note: GC not initialized 2024-05-30T16:34:56.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.748+0000 7fca99fdb640 20 reqs_thread_entry: start 2024-05-30T16:34:56.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.748+0000 7fcbe6a3c980 20 init_complete bucket index max shards: 11 2024-05-30T16:34:56.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.748+0000 7fcbe6a3c980 20 Filter name: none 2024-05-30T16:34:56.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.748+0000 7fca0b7fe640 20 reqs_thread_entry: start 2024-05-30T16:34:56.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.748+0000 7fcbe6a3c980 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2024-05-30T16:34:56.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.748+0000 7fcbe6a3c980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.748+0000 7fcbe6a3c980 20 rados_obj.operate() r=0 bl.length=460 2024-05-30T16:34:56.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.748+0000 7fcbe6a3c980 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x13 2024-05-30T16:34:56.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.748+0000 7fcbe6a3c980 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-05-30T16:34:56.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.748+0000 7fcbe6a3c980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:34:56.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.756+0000 7fcbe6a3c980 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2024-05-30T16:34:56.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.756+0000 7fcbe6a3c980 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-05-30T16:34:56.755 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.756+0000 7fcbe6a3c980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:bar.client.0, ofs0, ns] 2024-05-30T16:34:56.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.756+0000 7fcabb7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366299 cookie 93826427211792 notifier 7689 bl.length()=659 2024-05-30T16:34:56.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.756+0000 7fcabb7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2024-05-30T16:34:56.756 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.756+0000 7fcabb7fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-05-30T16:34:56.759 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.760+0000 7fca9afdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:56.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.764+0000 7fcbe6a3c980 20 remove_watcher() i=0 2024-05-30T16:34:56.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.764+0000 7fcbe6a3c980 2 removed watcher, disabling cache 2024-05-30T16:34:56.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.768+0000 7fcbe6a3c980 20 remove_watcher() i=1 2024-05-30T16:34:56.769 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.768+0000 7fcbe6a3c980 20 remove_watcher() i=2 2024-05-30T16:34:56.771 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.772+0000 7fcbe6a3c980 20 remove_watcher() i=3 2024-05-30T16:34:56.773 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.772+0000 7fcbe6a3c980 20 remove_watcher() i=4 2024-05-30T16:34:56.775 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.776+0000 7fcbe6a3c980 20 remove_watcher() i=5 2024-05-30T16:34:56.776 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.776+0000 7fcbe6a3c980 20 remove_watcher() i=6 2024-05-30T16:34:56.777 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.776+0000 7fcbe6a3c980 20 remove_watcher() i=7 2024-05-30T16:34:56.792 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph account create --account-id RGW33333333333333333 --account-name Mr.RGW33333333333333333 --email RGW33333333333333333+test@test.test --tenant testx 2024-05-30T16:34:56.871 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:34:56.871 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:34:56.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.924+0000 7f39b0b17980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.924+0000 7f39b0b17980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:56.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.924+0000 7f39b0b17980 20 realm 2024-05-30T16:34:56.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.924+0000 7f39b0b17980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.924+0000 7f39b0b17980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:56.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.924+0000 7f39b0b17980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:34:56.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.928+0000 7f39b0b17980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.928+0000 7f39b0b17980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:56.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.928+0000 7f39b0b17980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.932+0000 7f39b0b17980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:56.930 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.932+0000 7f39b0b17980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.932+0000 7f39b0b17980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:34:56.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.932+0000 7f39b0b17980 20 searching for the correct realm 2024-05-30T16:34:56.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.940+0000 7f39aee29640 0 -- 172.21.15.6:0/1929143032 >> [v2:172.21.15.6:6808/1897362715,v1:172.21.15.6:6809/1897362715] conn(0x55e77c0c6c50 msgr2=0x55e77c0c90d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:34:56.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.948+0000 7f39ae628640 0 -- 172.21.15.6:0/1929143032 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f39a402e3e0 msgr2=0x7f39a402e860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:34:56.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.952+0000 7f39b0b17980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:34:56.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.952+0000 7f39b0b17980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:34:56.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.952+0000 7f39b0b17980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:34:56.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.952+0000 7f39b0b17980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:34:56.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.952+0000 7f39b0b17980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:34:56.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.952+0000 7f39b0b17980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:34:56.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.952+0000 7f39b0b17980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.952+0000 7f39b0b17980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:56.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.952+0000 7f39b0b17980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.956+0000 7f39b0b17980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:56.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.956+0000 7f39b0b17980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.956+0000 7f39b0b17980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:34:56.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.956+0000 7f39b0b17980 20 zone default found 2024-05-30T16:34:56.955 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.956+0000 7f39b0b17980 4 Realm: () 2024-05-30T16:34:56.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.956+0000 7f39b0b17980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:34:56.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.956+0000 7f39b0b17980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:34:56.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.956+0000 7f39b0b17980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:34:56.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.956+0000 7f39b0b17980 20 zonegroup default 2024-05-30T16:34:56.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.956+0000 7f39b0b17980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.956+0000 7f39b0b17980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:56.956 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.956+0000 7f39b0b17980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.956+0000 7f39b0b17980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:56.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.956+0000 7f39b0b17980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.956+0000 7f39b0b17980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:56.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.956+0000 7f39b0b17980 20 started sync module instance, tier type = 2024-05-30T16:34:56.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.956+0000 7f39b0b17980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:34:56.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.972+0000 7f39b0b17980 20 add_watcher() i=0 2024-05-30T16:34:56.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.972+0000 7f39b0b17980 20 add_watcher() i=1 2024-05-30T16:34:56.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.972+0000 7f39b0b17980 20 add_watcher() i=2 2024-05-30T16:34:56.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.972+0000 7f39b0b17980 20 add_watcher() i=3 2024-05-30T16:34:56.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.972+0000 7f39b0b17980 20 add_watcher() i=4 2024-05-30T16:34:56.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.972+0000 7f39b0b17980 20 add_watcher() i=5 2024-05-30T16:34:56.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.972+0000 7f39b0b17980 20 add_watcher() i=6 2024-05-30T16:34:56.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.976+0000 7f39b0b17980 20 add_watcher() i=7 2024-05-30T16:34:56.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.976+0000 7f39b0b17980 2 all 8 watchers are set, enabling cache 2024-05-30T16:34:56.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.976+0000 7f38657e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:56.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.976+0000 7f39b0b17980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:34:56.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.976+0000 7f39b0b17980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:34:56.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.976+0000 7f39b0b17980 5 note: GC not initialized 2024-05-30T16:34:56.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.976+0000 7f385f7fe640 20 reqs_thread_entry: start 2024-05-30T16:34:56.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.976+0000 7f39b0b17980 20 init_complete bucket index max shards: 11 2024-05-30T16:34:56.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.976+0000 7f39b0b17980 20 Filter name: none 2024-05-30T16:34:56.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.976+0000 7f385d7fa640 20 reqs_thread_entry: start 2024-05-30T16:34:56.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.976+0000 7f39b0b17980 10 cache get: name=default.rgw.meta+accounts+name.testx$Mr.RGW33333333333333333 : miss 2024-05-30T16:34:56.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.976+0000 7f39b0b17980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.980+0000 7f39b0b17980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:56.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.980+0000 7f39b0b17980 10 cache put: name=default.rgw.meta+accounts+name.testx$Mr.RGW33333333333333333 info.flags=0x0 2024-05-30T16:34:56.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.980+0000 7f39b0b17980 10 adding default.rgw.meta+accounts+name.testx$Mr.RGW33333333333333333 to cache LRU end 2024-05-30T16:34:56.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.980+0000 7f39b0b17980 20 failed to read name.testx$Mr.RGW33333333333333333 with: (2) No such file or directory 2024-05-30T16:34:56.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.980+0000 7f39b0b17980 10 cache get: name=default.rgw.meta+users.email+rgw33333333333333333+test@test.test : miss 2024-05-30T16:34:56.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.980+0000 7f39b0b17980 20 rados->read ofs=0 len=0 2024-05-30T16:34:56.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.980+0000 7f39b0b17980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:56.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.980+0000 7f39b0b17980 10 cache put: name=default.rgw.meta+users.email+rgw33333333333333333+test@test.test info.flags=0x0 2024-05-30T16:34:56.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.980+0000 7f39b0b17980 10 adding default.rgw.meta+users.email+rgw33333333333333333+test@test.test to cache LRU end 2024-05-30T16:34:56.979 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.980+0000 7f39b0b17980 20 failed to read rgw33333333333333333+test@test.test with: (2) No such file or directory 2024-05-30T16:34:56.981 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.980+0000 7f39b0b17980 10 cache put: name=default.rgw.meta+accounts+account.RGW33333333333333333 info.flags=0x17 2024-05-30T16:34:56.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.980+0000 7f39b0b17980 10 adding default.rgw.meta+accounts+account.RGW33333333333333333 to cache LRU end 2024-05-30T16:34:56.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.980+0000 7f39b0b17980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:accounts:account.RGW33333333333333333, ofs0, ns] 2024-05-30T16:34:56.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.984+0000 7f3891ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333596 cookie 94452707045952 notifier 7707 bl.length()=389 2024-05-30T16:34:56.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.984+0000 7f3891ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+account.RGW33333333333333333 info.flags=0x17 2024-05-30T16:34:56.982 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.984+0000 7f3891ffb640 10 rgw watcher librados: moving default.rgw.meta+accounts+account.RGW33333333333333333 to cache LRU end 2024-05-30T16:34:56.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.988+0000 7f39b0b17980 10 cache put: name=default.rgw.meta+accounts+name.testx$Mr.RGW33333333333333333 info.flags=0x17 2024-05-30T16:34:56.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.988+0000 7f39b0b17980 10 moving default.rgw.meta+accounts+name.testx$Mr.RGW33333333333333333 to cache LRU end 2024-05-30T16:34:56.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.988+0000 7f39b0b17980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:accounts:name.testx$Mr.RGW33333333333333333, ofs0, ns] 2024-05-30T16:34:56.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.988+0000 7f3891ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333592 cookie 94452707052432 notifier 7707 bl.length()=230 2024-05-30T16:34:56.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.988+0000 7f3891ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+name.testx$Mr.RGW33333333333333333 info.flags=0x17 2024-05-30T16:34:56.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.988+0000 7f3891ffb640 10 rgw watcher librados: moving default.rgw.meta+accounts+name.testx$Mr.RGW33333333333333333 to cache LRU end 2024-05-30T16:34:56.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.992+0000 7f39b0b17980 10 cache put: name=default.rgw.meta+users.email+rgw33333333333333333+test@test.test info.flags=0x17 2024-05-30T16:34:56.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.992+0000 7f39b0b17980 10 moving default.rgw.meta+users.email+rgw33333333333333333+test@test.test to cache LRU end 2024-05-30T16:34:56.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.992+0000 7f39b0b17980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:rgw33333333333333333+test@test.test, ofs0, ns] 2024-05-30T16:34:56.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.996+0000 7f38917fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333531 cookie 94452707049792 notifier 7707 bl.length()=234 2024-05-30T16:34:56.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.996+0000 7f38917fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+rgw33333333333333333+test@test.test info.flags=0x17 2024-05-30T16:34:56.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.996+0000 7f38917fa640 10 rgw watcher librados: moving default.rgw.meta+users.email+rgw33333333333333333+test@test.test to cache LRU end 2024-05-30T16:34:56.996 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:56.996+0000 7f38657e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:57.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.000+0000 7f39b0b17980 20 remove_watcher() i=0 2024-05-30T16:34:57.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.000+0000 7f39b0b17980 2 removed watcher, disabling cache 2024-05-30T16:34:57.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.004+0000 7f39b0b17980 20 remove_watcher() i=1 2024-05-30T16:34:57.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.008+0000 7f39b0b17980 20 remove_watcher() i=2 2024-05-30T16:34:57.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.008+0000 7f39b0b17980 20 remove_watcher() i=3 2024-05-30T16:34:57.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.012+0000 7f39b0b17980 20 remove_watcher() i=4 2024-05-30T16:34:57.013 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.012+0000 7f39b0b17980 20 remove_watcher() i=5 2024-05-30T16:34:57.014 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.016+0000 7f39b0b17980 20 remove_watcher() i=6 2024-05-30T16:34:57.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.016+0000 7f39b0b17980 20 remove_watcher() i=7 2024-05-30T16:34:57.027 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-30T16:34:57.027 INFO:teuthology.orchestra.run.smithi006.stdout: "id": "RGW33333333333333333", 2024-05-30T16:34:57.027 INFO:teuthology.orchestra.run.smithi006.stdout: "tenant": "testx", 2024-05-30T16:34:57.027 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "Mr.RGW33333333333333333", 2024-05-30T16:34:57.027 INFO:teuthology.orchestra.run.smithi006.stdout: "email": "RGW33333333333333333+test@test.test", 2024-05-30T16:34:57.027 INFO:teuthology.orchestra.run.smithi006.stdout: "quota": { 2024-05-30T16:34:57.028 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": false, 2024-05-30T16:34:57.028 INFO:teuthology.orchestra.run.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:34:57.028 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size": -1, 2024-05-30T16:34:57.028 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:34:57.028 INFO:teuthology.orchestra.run.smithi006.stdout: "max_objects": -1 2024-05-30T16:34:57.028 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T16:34:57.028 INFO:teuthology.orchestra.run.smithi006.stdout: "bucket_quota": { 2024-05-30T16:34:57.028 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": false, 2024-05-30T16:34:57.028 INFO:teuthology.orchestra.run.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:34:57.028 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size": -1, 2024-05-30T16:34:57.028 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:34:57.028 INFO:teuthology.orchestra.run.smithi006.stdout: "max_objects": -1 2024-05-30T16:34:57.028 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T16:34:57.028 INFO:teuthology.orchestra.run.smithi006.stdout: "max_users": 1000, 2024-05-30T16:34:57.029 INFO:teuthology.orchestra.run.smithi006.stdout: "max_roles": 1000, 2024-05-30T16:34:57.029 INFO:teuthology.orchestra.run.smithi006.stdout: "max_groups": 1000, 2024-05-30T16:34:57.029 INFO:teuthology.orchestra.run.smithi006.stdout: "max_buckets": 1000, 2024-05-30T16:34:57.029 INFO:teuthology.orchestra.run.smithi006.stdout: "max_access_keys": 4 2024-05-30T16:34:57.029 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-30T16:34:57.030 DEBUG:tasks.s3tests:Creating user RGW33333333333333333 on client.0 2024-05-30T16:34:57.030 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid 'testx$tenanteduser.client.0' --display-name Mr.RGW33333333333333333 --email 'testx$tenanteduser+test@test.test' --access-key RIIEWYRBZTMNWUDATHTI --secret e2y2/wqkr66ajB6j9CKeo3wb+LLCbdCyB/eFVWMrniLWW6Lmcz2DXQ== --cluster ceph --account-id RGW33333333333333333 --account-root 2024-05-30T16:34:57.123 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:34:57.123 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:34:57.167 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.168+0000 7fed139b1980 20 rados->read ofs=0 len=0 2024-05-30T16:34:57.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.168+0000 7fed139b1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:57.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.168+0000 7fed139b1980 20 realm 2024-05-30T16:34:57.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.168+0000 7fed139b1980 20 rados->read ofs=0 len=0 2024-05-30T16:34:57.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.168+0000 7fed139b1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:57.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.168+0000 7fed139b1980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:34:57.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.168+0000 7fed139b1980 20 rados->read ofs=0 len=0 2024-05-30T16:34:57.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.168+0000 7fed139b1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:57.169 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.168+0000 7fed139b1980 20 rados->read ofs=0 len=0 2024-05-30T16:34:57.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.172+0000 7fed139b1980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:57.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.172+0000 7fed139b1980 20 rados->read ofs=0 len=0 2024-05-30T16:34:57.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.172+0000 7fed139b1980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:34:57.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.172+0000 7fed139b1980 20 searching for the correct realm 2024-05-30T16:34:57.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.176+0000 7fed11cc3640 0 -- 172.21.15.6:0/3597680799 >> [v2:172.21.15.204:6823/3156311527,v1:172.21.15.204:6825/3156311527] conn(0x7fed080c0020 msgr2=0x7fed080ac170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:34:57.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.184+0000 7fed114c2640 0 -- 172.21.15.6:0/3597680799 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7fed0811e490 msgr2=0x7fed081311b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:34:57.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.400+0000 7fed139b1980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:34:57.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.400+0000 7fed139b1980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:34:57.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.400+0000 7fed139b1980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:34:57.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.400+0000 7fed139b1980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:34:57.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.400+0000 7fed139b1980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:34:57.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.400+0000 7fed139b1980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:34:57.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.400+0000 7fed139b1980 20 rados->read ofs=0 len=0 2024-05-30T16:34:57.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.400+0000 7fed139b1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:57.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.400+0000 7fed139b1980 20 rados->read ofs=0 len=0 2024-05-30T16:34:57.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.404+0000 7fed139b1980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:57.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.404+0000 7fed139b1980 20 rados->read ofs=0 len=0 2024-05-30T16:34:57.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.404+0000 7fed139b1980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:34:57.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.404+0000 7fed139b1980 20 zone default found 2024-05-30T16:34:57.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.404+0000 7fed139b1980 4 Realm: () 2024-05-30T16:34:57.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.404+0000 7fed139b1980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:34:57.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.404+0000 7fed139b1980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:34:57.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.404+0000 7fed139b1980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:34:57.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.404+0000 7fed139b1980 20 zonegroup default 2024-05-30T16:34:57.404 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.404+0000 7fed139b1980 20 rados->read ofs=0 len=0 2024-05-30T16:34:57.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.404+0000 7fed139b1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:57.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.404+0000 7fed139b1980 20 rados->read ofs=0 len=0 2024-05-30T16:34:57.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.404+0000 7fed139b1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:57.405 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.404+0000 7fed139b1980 20 rados->read ofs=0 len=0 2024-05-30T16:34:57.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.408+0000 7fed139b1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:57.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.408+0000 7fed139b1980 20 started sync module instance, tier type = 2024-05-30T16:34:57.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.408+0000 7fed139b1980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:34:57.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.432+0000 7fed139b1980 20 add_watcher() i=0 2024-05-30T16:34:57.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.432+0000 7fed139b1980 20 add_watcher() i=1 2024-05-30T16:34:57.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.432+0000 7fed139b1980 20 add_watcher() i=2 2024-05-30T16:34:57.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.432+0000 7fed139b1980 20 add_watcher() i=3 2024-05-30T16:34:57.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.432+0000 7fed139b1980 20 add_watcher() i=4 2024-05-30T16:34:57.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.432+0000 7fed139b1980 20 add_watcher() i=5 2024-05-30T16:34:57.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.432+0000 7fed139b1980 20 add_watcher() i=6 2024-05-30T16:34:57.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.432+0000 7fed139b1980 20 add_watcher() i=7 2024-05-30T16:34:57.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.432+0000 7fed139b1980 2 all 8 watchers are set, enabling cache 2024-05-30T16:34:57.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.436+0000 7febc7fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:57.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.440+0000 7fed139b1980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:34:57.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.440+0000 7fed139b1980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:34:57.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.440+0000 7fed139b1980 5 note: GC not initialized 2024-05-30T16:34:57.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.440+0000 7febc6fe5640 20 reqs_thread_entry: start 2024-05-30T16:34:57.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.440+0000 7fed139b1980 20 init_complete bucket index max shards: 11 2024-05-30T16:34:57.439 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.440+0000 7fed139b1980 20 Filter name: none 2024-05-30T16:34:57.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.440+0000 7febc4fe1640 20 reqs_thread_entry: start 2024-05-30T16:34:57.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.440+0000 7fed139b1980 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2024-05-30T16:34:57.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.440+0000 7fed139b1980 20 rados->read ofs=0 len=0 2024-05-30T16:34:57.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.440+0000 7fed139b1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:57.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.440+0000 7fed139b1980 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x0 2024-05-30T16:34:57.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.440+0000 7fed139b1980 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-05-30T16:34:57.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.440+0000 7fed139b1980 10 cache get: name=default.rgw.meta+users.email+testx$tenanteduser+test@test.test : miss 2024-05-30T16:34:57.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.440+0000 7fed139b1980 20 rados->read ofs=0 len=0 2024-05-30T16:34:57.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.440+0000 7fed139b1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:57.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.440+0000 7fed139b1980 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser+test@test.test info.flags=0x0 2024-05-30T16:34:57.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.440+0000 7fed139b1980 10 adding default.rgw.meta+users.email+testx$tenanteduser+test@test.test to cache LRU end 2024-05-30T16:34:57.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.440+0000 7fed139b1980 10 cache get: name=default.rgw.meta+users.keys+RIIEWYRBZTMNWUDATHTI : miss 2024-05-30T16:34:57.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.440+0000 7fed139b1980 20 rados->read ofs=0 len=0 2024-05-30T16:34:57.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.444+0000 7fed139b1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:57.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.444+0000 7fed139b1980 10 cache put: name=default.rgw.meta+users.keys+RIIEWYRBZTMNWUDATHTI info.flags=0x0 2024-05-30T16:34:57.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.444+0000 7fed139b1980 10 adding default.rgw.meta+users.keys+RIIEWYRBZTMNWUDATHTI to cache LRU end 2024-05-30T16:34:57.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.444+0000 7fed139b1980 10 cache get: name=default.rgw.meta+accounts+account.RGW33333333333333333 : miss 2024-05-30T16:34:57.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.444+0000 7fed139b1980 20 rados->read ofs=0 len=0 2024-05-30T16:34:57.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.444+0000 7fed139b1980 20 rados_obj.operate() r=0 bl.length=189 2024-05-30T16:34:57.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.444+0000 7fed139b1980 10 cache put: name=default.rgw.meta+accounts+account.RGW33333333333333333 info.flags=0x17 2024-05-30T16:34:57.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.444+0000 7fed139b1980 10 adding default.rgw.meta+accounts+account.RGW33333333333333333 to cache LRU end 2024-05-30T16:34:57.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.444+0000 7fed139b1980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:34:57.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.444+0000 7fed139b1980 10 cache get: name=default.rgw.meta+users.keys+RIIEWYRBZTMNWUDATHTI : hit (negative entry) 2024-05-30T16:34:57.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.444+0000 7fed139b1980 10 cache get: name=default.rgw.meta+users.keys+RIIEWYRBZTMNWUDATHTI : hit (negative entry) 2024-05-30T16:34:57.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.448+0000 7fed139b1980 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2024-05-30T16:34:57.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.448+0000 7fed139b1980 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-05-30T16:34:57.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.448+0000 7fed139b1980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:testx$tenanteduser.client.0, ofs0, ns] 2024-05-30T16:34:57.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.448+0000 7febf4ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333626 cookie 94058439766992 notifier 7277 bl.length()=732 2024-05-30T16:34:57.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.448+0000 7febf4ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2024-05-30T16:34:57.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.448+0000 7febf4ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-05-30T16:34:57.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.456+0000 7fed139b1980 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser+test@test.test info.flags=0x7 2024-05-30T16:34:57.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.456+0000 7fed139b1980 10 moving default.rgw.meta+users.email+testx$tenanteduser+test@test.test to cache LRU end 2024-05-30T16:34:57.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.456+0000 7fed139b1980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.email:testx$tenanteduser+test@test.test, ofs0, ns] 2024-05-30T16:34:57.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.456+0000 7febf4ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333525 cookie 94058439771744 notifier 7277 bl.length()=215 2024-05-30T16:34:57.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.456+0000 7febf4ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+testx$tenanteduser+test@test.test info.flags=0x7 2024-05-30T16:34:57.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.456+0000 7febf4ff9640 10 rgw watcher librados: moving default.rgw.meta+users.email+testx$tenanteduser+test@test.test to cache LRU end 2024-05-30T16:34:57.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.460+0000 7fed139b1980 10 cache put: name=default.rgw.meta+users.keys+RIIEWYRBZTMNWUDATHTI info.flags=0x7 2024-05-30T16:34:57.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.460+0000 7fed139b1980 10 moving default.rgw.meta+users.keys+RIIEWYRBZTMNWUDATHTI to cache LRU end 2024-05-30T16:34:57.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.460+0000 7fed139b1980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:RIIEWYRBZTMNWUDATHTI, ofs0, ns] 2024-05-30T16:34:57.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.460+0000 7febe3fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333627 cookie 94058439766992 notifier 7277 bl.length()=201 2024-05-30T16:34:57.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.460+0000 7febe3fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+RIIEWYRBZTMNWUDATHTI info.flags=0x7 2024-05-30T16:34:57.462 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.460+0000 7febe3fff640 10 rgw watcher librados: moving default.rgw.meta+users.keys+RIIEWYRBZTMNWUDATHTI to cache LRU end 2024-05-30T16:34:57.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.468+0000 7fed139b1980 20 linked user testx$tenanteduser.client.0 to account RGW33333333333333333 2024-05-30T16:34:57.467 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-30T16:34:57.467 INFO:teuthology.orchestra.run.smithi006.stdout: "user_id": "testx$tenanteduser.client.0", 2024-05-30T16:34:57.467 INFO:teuthology.orchestra.run.smithi006.stdout: "display_name": "Mr.RGW33333333333333333", 2024-05-30T16:34:57.467 INFO:teuthology.orchestra.run.smithi006.stdout: "email": "testx$tenanteduser+test@test.test", 2024-05-30T16:34:57.467 INFO:teuthology.orchestra.run.smithi006.stdout: "suspended": 0, 2024-05-30T16:34:57.467 INFO:teuthology.orchestra.run.smithi006.stdout: "max_buckets": 1000, 2024-05-30T16:34:57.468 INFO:teuthology.orchestra.run.smithi006.stdout: "subusers": [], 2024-05-30T16:34:57.468 INFO:teuthology.orchestra.run.smithi006.stdout: "keys": [ 2024-05-30T16:34:57.468 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T16:34:57.468 INFO:teuthology.orchestra.run.smithi006.stdout: "user": "testx$tenanteduser.client.0", 2024-05-30T16:34:57.468 INFO:teuthology.orchestra.run.smithi006.stdout: "access_key": "RIIEWYRBZTMNWUDATHTI", 2024-05-30T16:34:57.468 INFO:teuthology.orchestra.run.smithi006.stdout: "secret_key": "e2y2/wqkr66ajB6j9CKeo3wb+LLCbdCyB/eFVWMrniLWW6Lmcz2DXQ==", 2024-05-30T16:34:57.468 INFO:teuthology.orchestra.run.smithi006.stdout: "active": true, 2024-05-30T16:34:57.468 INFO:teuthology.orchestra.run.smithi006.stdout: "create_date": "2024-05-30T16:34:57.446333Z" 2024-05-30T16:34:57.468 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T16:34:57.468 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-30T16:34:57.468 INFO:teuthology.orchestra.run.smithi006.stdout: "swift_keys": [], 2024-05-30T16:34:57.468 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": [], 2024-05-30T16:34:57.469 INFO:teuthology.orchestra.run.smithi006.stdout: "op_mask": "read, write, delete", 2024-05-30T16:34:57.469 INFO:teuthology.orchestra.run.smithi006.stdout: "default_placement": "", 2024-05-30T16:34:57.469 INFO:teuthology.orchestra.run.smithi006.stdout: "default_storage_class": "", 2024-05-30T16:34:57.469 INFO:teuthology.orchestra.run.smithi006.stdout: "placement_tags": [], 2024-05-30T16:34:57.469 INFO:teuthology.orchestra.run.smithi006.stdout: "bucket_quota": { 2024-05-30T16:34:57.469 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": false, 2024-05-30T16:34:57.469 INFO:teuthology.orchestra.run.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:34:57.469 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size": -1, 2024-05-30T16:34:57.469 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:34:57.469 INFO:teuthology.orchestra.run.smithi006.stdout: "max_objects": -1 2024-05-30T16:34:57.469 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T16:34:57.469 INFO:teuthology.orchestra.run.smithi006.stdout: "user_quota": { 2024-05-30T16:34:57.469 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": false, 2024-05-30T16:34:57.470 INFO:teuthology.orchestra.run.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:34:57.470 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size": -1, 2024-05-30T16:34:57.470 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:34:57.470 INFO:teuthology.orchestra.run.smithi006.stdout: "max_objects": -1 2024-05-30T16:34:57.470 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T16:34:57.470 INFO:teuthology.orchestra.run.smithi006.stdout: "temp_url_keys": [], 2024-05-30T16:34:57.470 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "root", 2024-05-30T16:34:57.470 INFO:teuthology.orchestra.run.smithi006.stdout: "mfa_ids": [], 2024-05-30T16:34:57.470 INFO:teuthology.orchestra.run.smithi006.stdout: "account_id": "RGW33333333333333333", 2024-05-30T16:34:57.470 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2024-05-30T16:34:57.470 INFO:teuthology.orchestra.run.smithi006.stdout: "create_date": "2024-05-30T16:34:57.446276Z", 2024-05-30T16:34:57.470 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-30T16:34:57.471 INFO:teuthology.orchestra.run.smithi006.stdout: "group_ids": [] 2024-05-30T16:34:57.471 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-30T16:34:57.471 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:57.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.468+0000 7febc7fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:57.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.472+0000 7fed139b1980 20 remove_watcher() i=0 2024-05-30T16:34:57.473 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.472+0000 7fed139b1980 2 removed watcher, disabling cache 2024-05-30T16:34:57.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.476+0000 7fed139b1980 20 remove_watcher() i=1 2024-05-30T16:34:57.478 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.476+0000 7fed139b1980 20 remove_watcher() i=2 2024-05-30T16:34:57.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.480+0000 7fed139b1980 20 remove_watcher() i=3 2024-05-30T16:34:57.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.484+0000 7fed139b1980 20 remove_watcher() i=4 2024-05-30T16:34:57.483 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.484+0000 7fed139b1980 20 remove_watcher() i=5 2024-05-30T16:34:57.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.484+0000 7fed139b1980 20 remove_watcher() i=6 2024-05-30T16:34:57.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.488+0000 7fed139b1980 20 remove_watcher() i=7 2024-05-30T16:34:57.504 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid 'testx$tenanteduser.client.0' --totp-serial 7221396009 --totp-seed GAVINDVBQTS64YBTZDFLWQWWW7NIZD7F6OQWIGMGFJKZD4B746Q5WR7VEOB3T5O5 --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-05-30T16:34:57.547 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:34:57.547 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:34:57.596 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.596+0000 7f84e93ea980 20 rados->read ofs=0 len=0 2024-05-30T16:34:57.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.596+0000 7f84e93ea980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:57.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.596+0000 7f84e93ea980 20 realm 2024-05-30T16:34:57.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.596+0000 7f84e93ea980 20 rados->read ofs=0 len=0 2024-05-30T16:34:57.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.596+0000 7f84e93ea980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:57.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.596+0000 7f84e93ea980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:34:57.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.596+0000 7f84e93ea980 20 rados->read ofs=0 len=0 2024-05-30T16:34:57.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.600+0000 7f84e93ea980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:57.598 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.600+0000 7f84e93ea980 20 rados->read ofs=0 len=0 2024-05-30T16:34:57.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.600+0000 7f84e93ea980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:57.600 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.600+0000 7f84e93ea980 20 rados->read ofs=0 len=0 2024-05-30T16:34:57.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.600+0000 7f84e93ea980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:34:57.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.600+0000 7f84e93ea980 20 searching for the correct realm 2024-05-30T16:34:57.607 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.608+0000 7f84e76fc640 0 -- 172.21.15.6:0/2104682359 >> [v2:172.21.15.204:6823/3156311527,v1:172.21.15.204:6825/3156311527] conn(0x7f84dc08f040 msgr2=0x7f84dc05b050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:34:57.614 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.612+0000 7f84e6efb640 0 -- 172.21.15.6:0/2104682359 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f84dc071130 msgr2=0x7f84dc023730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:34:57.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.828+0000 7f84e93ea980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:34:57.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.828+0000 7f84e93ea980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:34:57.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.828+0000 7f84e93ea980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:34:57.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.828+0000 7f84e93ea980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:34:57.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.828+0000 7f84e93ea980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:34:57.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.828+0000 7f84e93ea980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:34:57.829 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.828+0000 7f84e93ea980 20 rados->read ofs=0 len=0 2024-05-30T16:34:57.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.828+0000 7f84e93ea980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:57.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.828+0000 7f84e93ea980 20 rados->read ofs=0 len=0 2024-05-30T16:34:57.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.832+0000 7f84e93ea980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:57.830 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.832+0000 7f84e93ea980 20 rados->read ofs=0 len=0 2024-05-30T16:34:57.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.832+0000 7f84e93ea980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:34:57.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.832+0000 7f84e93ea980 20 zone default found 2024-05-30T16:34:57.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.832+0000 7f84e93ea980 4 Realm: () 2024-05-30T16:34:57.831 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.832+0000 7f84e93ea980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:34:57.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.832+0000 7f84e93ea980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:34:57.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.832+0000 7f84e93ea980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:34:57.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.832+0000 7f84e93ea980 20 zonegroup default 2024-05-30T16:34:57.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.832+0000 7f84e93ea980 20 rados->read ofs=0 len=0 2024-05-30T16:34:57.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.832+0000 7f84e93ea980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:57.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.832+0000 7f84e93ea980 20 rados->read ofs=0 len=0 2024-05-30T16:34:57.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.832+0000 7f84e93ea980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:57.833 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.832+0000 7f84e93ea980 20 rados->read ofs=0 len=0 2024-05-30T16:34:57.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.832+0000 7f84e93ea980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:57.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.832+0000 7f84e93ea980 20 started sync module instance, tier type = 2024-05-30T16:34:57.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.832+0000 7f84e93ea980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:34:57.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.856+0000 7f84e93ea980 20 add_watcher() i=0 2024-05-30T16:34:57.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.856+0000 7f84e93ea980 20 add_watcher() i=1 2024-05-30T16:34:57.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.856+0000 7f84e93ea980 20 add_watcher() i=2 2024-05-30T16:34:57.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.856+0000 7f84e93ea980 20 add_watcher() i=3 2024-05-30T16:34:57.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.856+0000 7f84e93ea980 20 add_watcher() i=4 2024-05-30T16:34:57.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.856+0000 7f84e93ea980 20 add_watcher() i=5 2024-05-30T16:34:57.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.856+0000 7f84e93ea980 20 add_watcher() i=6 2024-05-30T16:34:57.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.860+0000 7f84e93ea980 20 add_watcher() i=7 2024-05-30T16:34:57.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.860+0000 7f84e93ea980 2 all 8 watchers are set, enabling cache 2024-05-30T16:34:57.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.864+0000 7f839d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:57.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.864+0000 7f84e93ea980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:34:57.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.864+0000 7f84e93ea980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:34:57.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.864+0000 7f84e93ea980 5 note: GC not initialized 2024-05-30T16:34:57.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.864+0000 7f8317fff640 20 reqs_thread_entry: start 2024-05-30T16:34:57.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.864+0000 7f84e93ea980 20 init_complete bucket index max shards: 11 2024-05-30T16:34:57.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.864+0000 7f84e93ea980 20 Filter name: none 2024-05-30T16:34:57.864 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.864+0000 7f8315ffb640 20 reqs_thread_entry: start 2024-05-30T16:34:57.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.864+0000 7f84e93ea980 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2024-05-30T16:34:57.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.864+0000 7f84e93ea980 20 rados->read ofs=0 len=0 2024-05-30T16:34:57.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.864+0000 7f84e93ea980 20 rados_obj.operate() r=0 bl.length=532 2024-05-30T16:34:57.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.864+0000 7f84e93ea980 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x13 2024-05-30T16:34:57.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.864+0000 7f84e93ea980 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-05-30T16:34:57.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.864+0000 7f84e93ea980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:34:57.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.872+0000 7f84e93ea980 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2024-05-30T16:34:57.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.872+0000 7f84e93ea980 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-05-30T16:34:57.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.872+0000 7f84e93ea980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:testx$tenanteduser.client.0, ofs0, ns] 2024-05-30T16:34:57.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.872+0000 7f83ca7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333628 cookie 94601351020768 notifier 7295 bl.length()=746 2024-05-30T16:34:57.872 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.872+0000 7f83ca7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2024-05-30T16:34:57.872 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.872+0000 7f83ca7fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-05-30T16:34:57.877 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.876+0000 7f839d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:57.882 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.884+0000 7f84e93ea980 20 remove_watcher() i=0 2024-05-30T16:34:57.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.884+0000 7f84e93ea980 2 removed watcher, disabling cache 2024-05-30T16:34:57.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.884+0000 7f84e93ea980 20 remove_watcher() i=1 2024-05-30T16:34:57.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.888+0000 7f84e93ea980 20 remove_watcher() i=2 2024-05-30T16:34:57.889 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.888+0000 7f84e93ea980 20 remove_watcher() i=3 2024-05-30T16:34:57.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.892+0000 7f84e93ea980 20 remove_watcher() i=4 2024-05-30T16:34:57.893 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.892+0000 7f84e93ea980 20 remove_watcher() i=5 2024-05-30T16:34:57.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.896+0000 7f84e93ea980 20 remove_watcher() i=6 2024-05-30T16:34:57.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:57.896+0000 7f84e93ea980 20 remove_watcher() i=7 2024-05-30T16:34:57.916 DEBUG:tasks.s3tests:Creating user foobar.client.0 on client.0 2024-05-30T16:34:57.916 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid foobar.client.0 --display-name Mr.foobar.client.0 --email foobar+test@test.test --access-key HEEFWLJSJSYCGLOPZCKY --secret 0yXhMhLofXja1MrcKx1IuvlFprMvcbgo5IcthpWeaLTFdnNJeyMOBw== --cluster ceph 2024-05-30T16:34:58.009 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:34:58.009 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:34:58.056 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.056+0000 7f77b3692980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.056+0000 7f77b3692980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:58.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.056+0000 7f77b3692980 20 realm 2024-05-30T16:34:58.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.056+0000 7f77b3692980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.058 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.060+0000 7f77b3692980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:58.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.060+0000 7f77b3692980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:34:58.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.060+0000 7f77b3692980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.060+0000 7f77b3692980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:58.059 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.060+0000 7f77b3692980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.060+0000 7f77b3692980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:58.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.060+0000 7f77b3692980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.060+0000 7f77b3692980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:34:58.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.060+0000 7f77b3692980 20 searching for the correct realm 2024-05-30T16:34:58.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.068+0000 7f77b19a4640 0 -- 172.21.15.6:0/2392041386 >> [v2:172.21.15.204:6823/3156311527,v1:172.21.15.204:6825/3156311527] conn(0x7f779c075390 msgr2=0x7f779c05b710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:34:58.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.076+0000 7f77b11a3640 0 -- 172.21.15.6:0/2392041386 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f779c08ea00 msgr2=0x7f779c084a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:34:58.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.292+0000 7f77b3692980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:34:58.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.292+0000 7f77b3692980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:34:58.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.292+0000 7f77b3692980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:34:58.293 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.292+0000 7f77b3692980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:34:58.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.292+0000 7f77b3692980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:34:58.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.292+0000 7f77b3692980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:34:58.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.292+0000 7f77b3692980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.292+0000 7f77b3692980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:58.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.292+0000 7f77b3692980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.296+0000 7f77b3692980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:58.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.296+0000 7f77b3692980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.296+0000 7f77b3692980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:34:58.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.296+0000 7f77b3692980 20 zone default found 2024-05-30T16:34:58.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.296+0000 7f77b3692980 4 Realm: () 2024-05-30T16:34:58.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.296+0000 7f77b3692980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:34:58.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.296+0000 7f77b3692980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:34:58.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.296+0000 7f77b3692980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:34:58.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.296+0000 7f77b3692980 20 zonegroup default 2024-05-30T16:34:58.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.296+0000 7f77b3692980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.296+0000 7f77b3692980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:58.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.296+0000 7f77b3692980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.296+0000 7f77b3692980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:58.297 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.296+0000 7f77b3692980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.300+0000 7f77b3692980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:58.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.300+0000 7f77b3692980 20 started sync module instance, tier type = 2024-05-30T16:34:58.298 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.300+0000 7f77b3692980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:34:58.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.320+0000 7f77b3692980 20 add_watcher() i=0 2024-05-30T16:34:58.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.320+0000 7f77b3692980 20 add_watcher() i=1 2024-05-30T16:34:58.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.320+0000 7f77b3692980 20 add_watcher() i=2 2024-05-30T16:34:58.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.320+0000 7f77b3692980 20 add_watcher() i=3 2024-05-30T16:34:58.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.320+0000 7f77b3692980 20 add_watcher() i=4 2024-05-30T16:34:58.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.320+0000 7f77b3692980 20 add_watcher() i=5 2024-05-30T16:34:58.321 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.320+0000 7f77b3692980 20 add_watcher() i=6 2024-05-30T16:34:58.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.324+0000 7f77b3692980 20 add_watcher() i=7 2024-05-30T16:34:58.323 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.324+0000 7f77b3692980 2 all 8 watchers are set, enabling cache 2024-05-30T16:34:58.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.328+0000 7f7667fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:58.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.328+0000 7f77b3692980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:34:58.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.328+0000 7f77b3692980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:34:58.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.328+0000 7f77b3692980 5 note: GC not initialized 2024-05-30T16:34:58.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.328+0000 7f7666fe5640 20 reqs_thread_entry: start 2024-05-30T16:34:58.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.328+0000 7f77b3692980 20 init_complete bucket index max shards: 11 2024-05-30T16:34:58.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.328+0000 7f77b3692980 20 Filter name: none 2024-05-30T16:34:58.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.328+0000 7f7664fe1640 20 reqs_thread_entry: start 2024-05-30T16:34:58.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.328+0000 7f77b3692980 10 cache get: name=default.rgw.meta+users.uid+foobar.client.0 : miss 2024-05-30T16:34:58.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.328+0000 7f77b3692980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.328+0000 7f77b3692980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:58.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.328+0000 7f77b3692980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x0 2024-05-30T16:34:58.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.328+0000 7f77b3692980 10 adding default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-05-30T16:34:58.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.328+0000 7f77b3692980 10 cache get: name=default.rgw.meta+users.email+foobar+test@test.test : miss 2024-05-30T16:34:58.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.328+0000 7f77b3692980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.332+0000 7f77b3692980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:58.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.332+0000 7f77b3692980 10 cache put: name=default.rgw.meta+users.email+foobar+test@test.test info.flags=0x0 2024-05-30T16:34:58.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.332+0000 7f77b3692980 10 adding default.rgw.meta+users.email+foobar+test@test.test to cache LRU end 2024-05-30T16:34:58.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.332+0000 7f77b3692980 10 cache get: name=default.rgw.meta+users.keys+HEEFWLJSJSYCGLOPZCKY : miss 2024-05-30T16:34:58.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.332+0000 7f77b3692980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.332+0000 7f77b3692980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:58.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.332+0000 7f77b3692980 10 cache put: name=default.rgw.meta+users.keys+HEEFWLJSJSYCGLOPZCKY info.flags=0x0 2024-05-30T16:34:58.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.332+0000 7f77b3692980 10 adding default.rgw.meta+users.keys+HEEFWLJSJSYCGLOPZCKY to cache LRU end 2024-05-30T16:34:58.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.332+0000 7f77b3692980 10 cache get: name=default.rgw.meta+users.keys+HEEFWLJSJSYCGLOPZCKY : hit (negative entry) 2024-05-30T16:34:58.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.332+0000 7f77b3692980 10 cache get: name=default.rgw.meta+users.keys+HEEFWLJSJSYCGLOPZCKY : hit (negative entry) 2024-05-30T16:34:58.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.332+0000 7f77b3692980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-05-30T16:34:58.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.332+0000 7f77b3692980 10 moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-05-30T16:34:58.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.332+0000 7f77b3692980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:foobar.client.0, ofs0, ns] 2024-05-30T16:34:58.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.332+0000 7f7694ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333526 cookie 94534062275504 notifier 7313 bl.length()=660 2024-05-30T16:34:58.333 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.332+0000 7f7694ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-05-30T16:34:58.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.332+0000 7f7694ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-05-30T16:34:58.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.340+0000 7f77b3692980 10 cache put: name=default.rgw.meta+users.email+foobar+test@test.test info.flags=0x7 2024-05-30T16:34:58.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.340+0000 7f77b3692980 10 moving default.rgw.meta+users.email+foobar+test@test.test to cache LRU end 2024-05-30T16:34:58.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.340+0000 7f77b3692980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.email:foobar+test@test.test, ofs0, ns] 2024-05-30T16:34:58.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.340+0000 7f7683fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333597 cookie 94534062257952 notifier 7313 bl.length()=191 2024-05-30T16:34:58.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.340+0000 7f7683fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+foobar+test@test.test info.flags=0x7 2024-05-30T16:34:58.339 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.340+0000 7f7683fff640 10 rgw watcher librados: moving default.rgw.meta+users.email+foobar+test@test.test to cache LRU end 2024-05-30T16:34:58.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.344+0000 7f77b3692980 10 cache put: name=default.rgw.meta+users.keys+HEEFWLJSJSYCGLOPZCKY info.flags=0x7 2024-05-30T16:34:58.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.344+0000 7f77b3692980 10 moving default.rgw.meta+users.keys+HEEFWLJSJSYCGLOPZCKY to cache LRU end 2024-05-30T16:34:58.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.344+0000 7f77b3692980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.keys:HEEFWLJSJSYCGLOPZCKY, ofs0, ns] 2024-05-30T16:34:58.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.344+0000 7f7683fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333593 cookie 94534062264272 notifier 7313 bl.length()=189 2024-05-30T16:34:58.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.344+0000 7f7683fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+HEEFWLJSJSYCGLOPZCKY info.flags=0x7 2024-05-30T16:34:58.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.344+0000 7f7683fff640 10 rgw watcher librados: moving default.rgw.meta+users.keys+HEEFWLJSJSYCGLOPZCKY to cache LRU end 2024-05-30T16:34:58.349 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-30T16:34:58.349 INFO:teuthology.orchestra.run.smithi006.stdout: "user_id": "foobar.client.0", 2024-05-30T16:34:58.349 INFO:teuthology.orchestra.run.smithi006.stdout: "display_name": "Mr.foobar.client.0", 2024-05-30T16:34:58.349 INFO:teuthology.orchestra.run.smithi006.stdout: "email": "foobar+test@test.test", 2024-05-30T16:34:58.349 INFO:teuthology.orchestra.run.smithi006.stdout: "suspended": 0, 2024-05-30T16:34:58.349 INFO:teuthology.orchestra.run.smithi006.stdout: "max_buckets": 1000, 2024-05-30T16:34:58.349 INFO:teuthology.orchestra.run.smithi006.stdout: "subusers": [], 2024-05-30T16:34:58.349 INFO:teuthology.orchestra.run.smithi006.stdout: "keys": [ 2024-05-30T16:34:58.349 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T16:34:58.350 INFO:teuthology.orchestra.run.smithi006.stdout: "user": "foobar.client.0", 2024-05-30T16:34:58.350 INFO:teuthology.orchestra.run.smithi006.stdout: "access_key": "HEEFWLJSJSYCGLOPZCKY", 2024-05-30T16:34:58.350 INFO:teuthology.orchestra.run.smithi006.stdout: "secret_key": "0yXhMhLofXja1MrcKx1IuvlFprMvcbgo5IcthpWeaLTFdnNJeyMOBw==", 2024-05-30T16:34:58.350 INFO:teuthology.orchestra.run.smithi006.stdout: "active": true, 2024-05-30T16:34:58.350 INFO:teuthology.orchestra.run.smithi006.stdout: "create_date": "2024-05-30T16:34:58.333707Z" 2024-05-30T16:34:58.350 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T16:34:58.350 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-30T16:34:58.350 INFO:teuthology.orchestra.run.smithi006.stdout: "swift_keys": [], 2024-05-30T16:34:58.350 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": [], 2024-05-30T16:34:58.350 INFO:teuthology.orchestra.run.smithi006.stdout: "op_mask": "read, write, delete", 2024-05-30T16:34:58.350 INFO:teuthology.orchestra.run.smithi006.stdout: "default_placement": "", 2024-05-30T16:34:58.350 INFO:teuthology.orchestra.run.smithi006.stdout: "default_storage_class": "", 2024-05-30T16:34:58.350 INFO:teuthology.orchestra.run.smithi006.stdout: "placement_tags": [], 2024-05-30T16:34:58.350 INFO:teuthology.orchestra.run.smithi006.stdout: "bucket_quota": { 2024-05-30T16:34:58.350 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": false, 2024-05-30T16:34:58.351 INFO:teuthology.orchestra.run.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:34:58.351 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size": -1, 2024-05-30T16:34:58.351 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:34:58.351 INFO:teuthology.orchestra.run.smithi006.stdout: "max_objects": -1 2024-05-30T16:34:58.351 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T16:34:58.351 INFO:teuthology.orchestra.run.smithi006.stdout: "user_quota": { 2024-05-30T16:34:58.351 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": false, 2024-05-30T16:34:58.351 INFO:teuthology.orchestra.run.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:34:58.351 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size": -1, 2024-05-30T16:34:58.351 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:34:58.351 INFO:teuthology.orchestra.run.smithi006.stdout: "max_objects": -1 2024-05-30T16:34:58.351 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T16:34:58.351 INFO:teuthology.orchestra.run.smithi006.stdout: "temp_url_keys": [], 2024-05-30T16:34:58.351 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "rgw", 2024-05-30T16:34:58.351 INFO:teuthology.orchestra.run.smithi006.stdout: "mfa_ids": [], 2024-05-30T16:34:58.352 INFO:teuthology.orchestra.run.smithi006.stdout: "account_id": "", 2024-05-30T16:34:58.352 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2024-05-30T16:34:58.352 INFO:teuthology.orchestra.run.smithi006.stdout: "create_date": "2024-05-30T16:34:58.333649Z", 2024-05-30T16:34:58.352 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-30T16:34:58.352 INFO:teuthology.orchestra.run.smithi006.stdout: "group_ids": [] 2024-05-30T16:34:58.352 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-30T16:34:58.352 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:58.352 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.352+0000 7f7667fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:58.355 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.356+0000 7f77b3692980 20 remove_watcher() i=0 2024-05-30T16:34:58.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.356+0000 7f77b3692980 2 removed watcher, disabling cache 2024-05-30T16:34:58.358 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.356+0000 7f77b3692980 20 remove_watcher() i=1 2024-05-30T16:34:58.360 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.360+0000 7f77b3692980 20 remove_watcher() i=2 2024-05-30T16:34:58.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.364+0000 7f77b3692980 20 remove_watcher() i=3 2024-05-30T16:34:58.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.364+0000 7f77b3692980 20 remove_watcher() i=4 2024-05-30T16:34:58.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.368+0000 7f77b3692980 20 remove_watcher() i=5 2024-05-30T16:34:58.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.368+0000 7f77b3692980 20 remove_watcher() i=6 2024-05-30T16:34:58.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.368+0000 7f77b3692980 20 remove_watcher() i=7 2024-05-30T16:34:58.387 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid foobar.client.0 --totp-serial 7619364576 --totp-seed U3CZBGFEZFQB37PZN6M7EA2K2T566EWGEN5KQVDBIOGX6VBVIEBP6XUHS5VPS42Z --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-05-30T16:34:58.441 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:34:58.441 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:34:58.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.484+0000 7fab850e6980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.486 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.488+0000 7fab850e6980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:58.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.488+0000 7fab850e6980 20 realm 2024-05-30T16:34:58.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.488+0000 7fab850e6980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.488+0000 7fab850e6980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:58.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.488+0000 7fab850e6980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:34:58.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.488+0000 7fab850e6980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.488+0000 7fab850e6980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:58.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.488+0000 7fab850e6980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.488+0000 7fab850e6980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:58.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.488+0000 7fab850e6980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.488+0000 7fab850e6980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:34:58.490 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.488+0000 7fab850e6980 20 searching for the correct realm 2024-05-30T16:34:58.496 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.496+0000 7fab823f6640 0 -- 172.21.15.6:0/1783865884 >> [v2:172.21.15.6:6819/1141406589,v1:172.21.15.6:6821/1141406589] conn(0x555d3c6253d0 msgr2=0x555d3c627890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:34:58.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.504+0000 7fab82bf7640 0 -- 172.21.15.6:0/1783865884 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7fab78070880 msgr2=0x7fab7801bfd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:34:58.510 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.512+0000 7fab850e6980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:34:58.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.512+0000 7fab850e6980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:34:58.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.512+0000 7fab850e6980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:34:58.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.512+0000 7fab850e6980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:34:58.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.512+0000 7fab850e6980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:34:58.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.512+0000 7fab850e6980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:34:58.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.512+0000 7fab850e6980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.512+0000 7fab850e6980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:58.511 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.512+0000 7fab850e6980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.512+0000 7fab850e6980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:58.512 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.512+0000 7fab850e6980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.512+0000 7fab850e6980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:34:58.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.512+0000 7fab850e6980 20 zone default found 2024-05-30T16:34:58.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.512+0000 7fab850e6980 4 Realm: () 2024-05-30T16:34:58.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.512+0000 7fab850e6980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:34:58.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.512+0000 7fab850e6980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:34:58.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.512+0000 7fab850e6980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:34:58.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.512+0000 7fab850e6980 20 zonegroup default 2024-05-30T16:34:58.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.512+0000 7fab850e6980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.512+0000 7fab850e6980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:58.513 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.512+0000 7fab850e6980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.512+0000 7fab850e6980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:58.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.512+0000 7fab850e6980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.512+0000 7fab850e6980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:58.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.512+0000 7fab850e6980 20 started sync module instance, tier type = 2024-05-30T16:34:58.514 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.512+0000 7fab850e6980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:34:58.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.528+0000 7fab850e6980 20 add_watcher() i=0 2024-05-30T16:34:58.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.528+0000 7fab850e6980 20 add_watcher() i=1 2024-05-30T16:34:58.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.528+0000 7fab850e6980 20 add_watcher() i=2 2024-05-30T16:34:58.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.528+0000 7fab850e6980 20 add_watcher() i=3 2024-05-30T16:34:58.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.528+0000 7fab850e6980 20 add_watcher() i=4 2024-05-30T16:34:58.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.532+0000 7fab850e6980 20 add_watcher() i=5 2024-05-30T16:34:58.530 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.532+0000 7fab850e6980 20 add_watcher() i=6 2024-05-30T16:34:58.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.532+0000 7fab850e6980 20 add_watcher() i=7 2024-05-30T16:34:58.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.532+0000 7fab850e6980 2 all 8 watchers are set, enabling cache 2024-05-30T16:34:58.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.532+0000 7faa397e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:58.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.532+0000 7fab850e6980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:34:58.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.532+0000 7fab850e6980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:34:58.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.532+0000 7fab850e6980 5 note: GC not initialized 2024-05-30T16:34:58.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.536+0000 7fa9b3fff640 20 reqs_thread_entry: start 2024-05-30T16:34:58.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.536+0000 7fab850e6980 20 init_complete bucket index max shards: 11 2024-05-30T16:34:58.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.536+0000 7fab850e6980 20 Filter name: none 2024-05-30T16:34:58.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.536+0000 7fa9b1ffb640 20 reqs_thread_entry: start 2024-05-30T16:34:58.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.536+0000 7fab850e6980 10 cache get: name=default.rgw.meta+users.uid+foobar.client.0 : miss 2024-05-30T16:34:58.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.536+0000 7fab850e6980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.536+0000 7fab850e6980 20 rados_obj.operate() r=0 bl.length=472 2024-05-30T16:34:58.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.536+0000 7fab850e6980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x13 2024-05-30T16:34:58.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.536+0000 7fab850e6980 10 adding default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-05-30T16:34:58.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.536+0000 7fab850e6980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:34:58.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.540+0000 7fab850e6980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-05-30T16:34:58.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.540+0000 7fab850e6980 10 moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-05-30T16:34:58.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.540+0000 7fab850e6980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:foobar.client.0, ofs0, ns] 2024-05-30T16:34:58.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.540+0000 7faa667fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333527 cookie 93858933413296 notifier 7328 bl.length()=674 2024-05-30T16:34:58.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.540+0000 7faa667fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-05-30T16:34:58.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.540+0000 7faa667fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-05-30T16:34:58.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.544+0000 7faa397e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:58.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.548+0000 7fab850e6980 20 remove_watcher() i=0 2024-05-30T16:34:58.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.548+0000 7fab850e6980 2 removed watcher, disabling cache 2024-05-30T16:34:58.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.552+0000 7fab850e6980 20 remove_watcher() i=1 2024-05-30T16:34:58.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.552+0000 7fab850e6980 20 remove_watcher() i=2 2024-05-30T16:34:58.554 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.556+0000 7fab850e6980 20 remove_watcher() i=3 2024-05-30T16:34:58.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.556+0000 7fab850e6980 20 remove_watcher() i=4 2024-05-30T16:34:58.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.556+0000 7fab850e6980 20 remove_watcher() i=5 2024-05-30T16:34:58.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.560+0000 7fab850e6980 20 remove_watcher() i=6 2024-05-30T16:34:58.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.560+0000 7fab850e6980 20 remove_watcher() i=7 2024-05-30T16:34:58.575 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 caps add --uid foobar.client.0 --caps 'oidc-provider=*;roles=*;user-policy=*' --cluster ceph 2024-05-30T16:34:58.606 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:34:58.606 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:34:58.647 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.648+0000 7f075362e980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.648+0000 7f075362e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:58.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.648+0000 7f075362e980 20 realm 2024-05-30T16:34:58.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.648+0000 7f075362e980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.648+0000 7f075362e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:58.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.648+0000 7f075362e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:34:58.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.648+0000 7f075362e980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.648+0000 7f075362e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:58.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.648+0000 7f075362e980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.652+0000 7f075362e980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:58.651 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.652+0000 7f075362e980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.652+0000 7f075362e980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:34:58.652 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.652+0000 7f075362e980 20 searching for the correct realm 2024-05-30T16:34:58.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.660+0000 7f0751940640 0 -- 172.21.15.6:0/4065481714 >> [v2:172.21.15.204:6800/3687190393,v1:172.21.15.204:6801/3687190393] conn(0x7f0748063030 msgr2=0x7f0748083230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:34:58.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.876+0000 7f075113f640 0 -- 172.21.15.6:0/4065481714 >> [v2:172.21.15.204:6816/2677861390,v1:172.21.15.204:6817/2677861390] conn(0x564b4e6105c0 msgr2=0x564b4e6129f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:34:58.883 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.884+0000 7f075362e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:34:58.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.884+0000 7f075362e980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:34:58.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.884+0000 7f075362e980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:34:58.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.884+0000 7f075362e980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:34:58.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.884+0000 7f075362e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:34:58.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.884+0000 7f075362e980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:34:58.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.884+0000 7f075362e980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.884+0000 7f075362e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:58.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.884+0000 7f075362e980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.884+0000 7f075362e980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:58.885 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.884+0000 7f075362e980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.884+0000 7f075362e980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:34:58.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.884+0000 7f075362e980 20 zone default found 2024-05-30T16:34:58.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.884+0000 7f075362e980 4 Realm: () 2024-05-30T16:34:58.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.884+0000 7f075362e980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:34:58.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.884+0000 7f075362e980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:34:58.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.884+0000 7f075362e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:34:58.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.884+0000 7f075362e980 20 zonegroup default 2024-05-30T16:34:58.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.884+0000 7f075362e980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.888+0000 7f075362e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:58.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.888+0000 7f075362e980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.888+0000 7f075362e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:58.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.888+0000 7f075362e980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.888+0000 7f075362e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:58.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.888+0000 7f075362e980 20 started sync module instance, tier type = 2024-05-30T16:34:58.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.888+0000 7f075362e980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:34:58.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.912+0000 7f075362e980 20 add_watcher() i=0 2024-05-30T16:34:58.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.912+0000 7f075362e980 20 add_watcher() i=1 2024-05-30T16:34:58.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.912+0000 7f075362e980 20 add_watcher() i=2 2024-05-30T16:34:58.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.912+0000 7f075362e980 20 add_watcher() i=3 2024-05-30T16:34:58.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.912+0000 7f075362e980 20 add_watcher() i=4 2024-05-30T16:34:58.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.912+0000 7f075362e980 20 add_watcher() i=5 2024-05-30T16:34:58.913 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.912+0000 7f075362e980 20 add_watcher() i=6 2024-05-30T16:34:58.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.916+0000 7f075362e980 20 add_watcher() i=7 2024-05-30T16:34:58.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.916+0000 7f075362e980 2 all 8 watchers are set, enabling cache 2024-05-30T16:34:58.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.920+0000 7f0607fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:58.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.920+0000 7f075362e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:34:58.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.920+0000 7f075362e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:34:58.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.920+0000 7f075362e980 5 note: GC not initialized 2024-05-30T16:34:58.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.920+0000 7f0606fe5640 20 reqs_thread_entry: start 2024-05-30T16:34:58.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.920+0000 7f075362e980 20 init_complete bucket index max shards: 11 2024-05-30T16:34:58.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.920+0000 7f075362e980 20 Filter name: none 2024-05-30T16:34:58.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.920+0000 7f0604fe1640 20 reqs_thread_entry: start 2024-05-30T16:34:58.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.924+0000 7f075362e980 10 cache get: name=default.rgw.meta+users.uid+foobar.client.0 : miss 2024-05-30T16:34:58.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.924+0000 7f075362e980 20 rados->read ofs=0 len=0 2024-05-30T16:34:58.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.924+0000 7f075362e980 20 rados_obj.operate() r=0 bl.length=486 2024-05-30T16:34:58.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.924+0000 7f075362e980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x13 2024-05-30T16:34:58.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.924+0000 7f075362e980 10 adding default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-05-30T16:34:58.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.924+0000 7f075362e980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:34:58.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.924+0000 7f075362e980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-05-30T16:34:58.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.924+0000 7f075362e980 10 moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-05-30T16:34:58.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.924+0000 7f075362e980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:foobar.client.0, ofs0, ns] 2024-05-30T16:34:58.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.928+0000 7f0634ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333528 cookie 94881437569344 notifier 7800 bl.length()=727 2024-05-30T16:34:58.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.928+0000 7f0634ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-05-30T16:34:58.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.928+0000 7f0634ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-05-30T16:34:58.929 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-30T16:34:58.929 INFO:teuthology.orchestra.run.smithi006.stdout: "user_id": "foobar.client.0", 2024-05-30T16:34:58.929 INFO:teuthology.orchestra.run.smithi006.stdout: "display_name": "Mr.foobar.client.0", 2024-05-30T16:34:58.929 INFO:teuthology.orchestra.run.smithi006.stdout: "email": "foobar+test@test.test", 2024-05-30T16:34:58.929 INFO:teuthology.orchestra.run.smithi006.stdout: "suspended": 0, 2024-05-30T16:34:58.929 INFO:teuthology.orchestra.run.smithi006.stdout: "max_buckets": 1000, 2024-05-30T16:34:58.929 INFO:teuthology.orchestra.run.smithi006.stdout: "subusers": [], 2024-05-30T16:34:58.929 INFO:teuthology.orchestra.run.smithi006.stdout: "keys": [ 2024-05-30T16:34:58.929 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T16:34:58.929 INFO:teuthology.orchestra.run.smithi006.stdout: "user": "foobar.client.0", 2024-05-30T16:34:58.929 INFO:teuthology.orchestra.run.smithi006.stdout: "access_key": "HEEFWLJSJSYCGLOPZCKY", 2024-05-30T16:34:58.930 INFO:teuthology.orchestra.run.smithi006.stdout: "secret_key": "0yXhMhLofXja1MrcKx1IuvlFprMvcbgo5IcthpWeaLTFdnNJeyMOBw==", 2024-05-30T16:34:58.930 INFO:teuthology.orchestra.run.smithi006.stdout: "active": true, 2024-05-30T16:34:58.930 INFO:teuthology.orchestra.run.smithi006.stdout: "create_date": "2024-05-30T16:34:58.333707Z" 2024-05-30T16:34:58.930 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T16:34:58.930 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-30T16:34:58.930 INFO:teuthology.orchestra.run.smithi006.stdout: "swift_keys": [], 2024-05-30T16:34:58.930 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": [ 2024-05-30T16:34:58.930 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T16:34:58.930 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "oidc-provider", 2024-05-30T16:34:58.930 INFO:teuthology.orchestra.run.smithi006.stdout: "perm": "*" 2024-05-30T16:34:58.930 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T16:34:58.930 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T16:34:58.931 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "roles", 2024-05-30T16:34:58.931 INFO:teuthology.orchestra.run.smithi006.stdout: "perm": "*" 2024-05-30T16:34:58.931 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T16:34:58.931 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T16:34:58.931 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "user-policy", 2024-05-30T16:34:58.931 INFO:teuthology.orchestra.run.smithi006.stdout: "perm": "*" 2024-05-30T16:34:58.931 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T16:34:58.931 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-30T16:34:58.931 INFO:teuthology.orchestra.run.smithi006.stdout: "op_mask": "read, write, delete", 2024-05-30T16:34:58.931 INFO:teuthology.orchestra.run.smithi006.stdout: "default_placement": "", 2024-05-30T16:34:58.931 INFO:teuthology.orchestra.run.smithi006.stdout: "default_storage_class": "", 2024-05-30T16:34:58.931 INFO:teuthology.orchestra.run.smithi006.stdout: "placement_tags": [], 2024-05-30T16:34:58.931 INFO:teuthology.orchestra.run.smithi006.stdout: "bucket_quota": { 2024-05-30T16:34:58.931 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": false, 2024-05-30T16:34:58.931 INFO:teuthology.orchestra.run.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:34:58.932 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size": -1, 2024-05-30T16:34:58.932 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:34:58.932 INFO:teuthology.orchestra.run.smithi006.stdout: "max_objects": -1 2024-05-30T16:34:58.932 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T16:34:58.932 INFO:teuthology.orchestra.run.smithi006.stdout: "user_quota": { 2024-05-30T16:34:58.932 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": false, 2024-05-30T16:34:58.932 INFO:teuthology.orchestra.run.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:34:58.932 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size": -1, 2024-05-30T16:34:58.932 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:34:58.932 INFO:teuthology.orchestra.run.smithi006.stdout: "max_objects": -1 2024-05-30T16:34:58.932 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T16:34:58.932 INFO:teuthology.orchestra.run.smithi006.stdout: "temp_url_keys": [], 2024-05-30T16:34:58.932 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "rgw", 2024-05-30T16:34:58.932 INFO:teuthology.orchestra.run.smithi006.stdout: "mfa_ids": [ 2024-05-30T16:34:58.932 INFO:teuthology.orchestra.run.smithi006.stdout: "7619364576" 2024-05-30T16:34:58.932 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-30T16:34:58.933 INFO:teuthology.orchestra.run.smithi006.stdout: "account_id": "", 2024-05-30T16:34:58.933 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2024-05-30T16:34:58.933 INFO:teuthology.orchestra.run.smithi006.stdout: "create_date": "2024-05-30T16:34:58.333649Z", 2024-05-30T16:34:58.933 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-30T16:34:58.933 INFO:teuthology.orchestra.run.smithi006.stdout: "group_ids": [] 2024-05-30T16:34:58.933 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-30T16:34:58.933 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:58.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.932+0000 7f0607fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:58.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.936+0000 7f075362e980 20 remove_watcher() i=0 2024-05-30T16:34:58.935 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.936+0000 7f075362e980 2 removed watcher, disabling cache 2024-05-30T16:34:58.937 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.936+0000 7f075362e980 20 remove_watcher() i=1 2024-05-30T16:34:58.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.940+0000 7f075362e980 20 remove_watcher() i=2 2024-05-30T16:34:58.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.944+0000 7f075362e980 20 remove_watcher() i=3 2024-05-30T16:34:58.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.944+0000 7f075362e980 20 remove_watcher() i=4 2024-05-30T16:34:58.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.948+0000 7f075362e980 20 remove_watcher() i=5 2024-05-30T16:34:58.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.948+0000 7f075362e980 20 remove_watcher() i=6 2024-05-30T16:34:58.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:58.948+0000 7f075362e980 20 remove_watcher() i=7 2024-05-30T16:34:58.965 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph account create --account-id RGW88888888888888888 --account-name Mr.RGW88888888888888888 --email RGW88888888888888888+test@test.test 2024-05-30T16:34:59.047 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:34:59.047 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:34:59.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.088+0000 7f72fda6b980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.092+0000 7f72fda6b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:59.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.092+0000 7f72fda6b980 20 realm 2024-05-30T16:34:59.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.092+0000 7f72fda6b980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.092+0000 7f72fda6b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:59.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.092+0000 7f72fda6b980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:34:59.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.092+0000 7f72fda6b980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.092+0000 7f72fda6b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:59.092 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.092+0000 7f72fda6b980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.096+0000 7f72fda6b980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:59.094 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.096+0000 7f72fda6b980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.096+0000 7f72fda6b980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:34:59.095 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.096+0000 7f72fda6b980 20 searching for the correct realm 2024-05-30T16:34:59.100 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.100+0000 7f72fad7b640 0 -- 172.21.15.6:0/1982340939 >> [v2:172.21.15.6:6819/1141406589,v1:172.21.15.6:6821/1141406589] conn(0x557458d8f410 msgr2=0x557458d918d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:34:59.107 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.108+0000 7f72fb57c640 0 -- 172.21.15.6:0/1982340939 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f72f00e13f0 msgr2=0x7f72f00cd700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:34:59.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.116+0000 7f72fda6b980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:34:59.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.116+0000 7f72fda6b980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:34:59.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.116+0000 7f72fda6b980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:34:59.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.116+0000 7f72fda6b980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:34:59.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.116+0000 7f72fda6b980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:34:59.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.116+0000 7f72fda6b980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:34:59.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.116+0000 7f72fda6b980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.116+0000 7f72fda6b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:59.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.116+0000 7f72fda6b980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.116+0000 7f72fda6b980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:59.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.116+0000 7f72fda6b980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.116+0000 7f72fda6b980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:34:59.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.116+0000 7f72fda6b980 20 zone default found 2024-05-30T16:34:59.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.116+0000 7f72fda6b980 4 Realm: () 2024-05-30T16:34:59.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.116+0000 7f72fda6b980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:34:59.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.116+0000 7f72fda6b980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:34:59.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.116+0000 7f72fda6b980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:34:59.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.116+0000 7f72fda6b980 20 zonegroup default 2024-05-30T16:34:59.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.116+0000 7f72fda6b980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.116+0000 7f72fda6b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:59.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.116+0000 7f72fda6b980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.120+0000 7f72fda6b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:59.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.120+0000 7f72fda6b980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.118 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.120+0000 7f72fda6b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:59.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.120+0000 7f72fda6b980 20 started sync module instance, tier type = 2024-05-30T16:34:59.119 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.120+0000 7f72fda6b980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:34:59.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.132+0000 7f72fda6b980 20 add_watcher() i=0 2024-05-30T16:34:59.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.132+0000 7f72fda6b980 20 add_watcher() i=1 2024-05-30T16:34:59.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.132+0000 7f72fda6b980 20 add_watcher() i=2 2024-05-30T16:34:59.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.132+0000 7f72fda6b980 20 add_watcher() i=3 2024-05-30T16:34:59.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.132+0000 7f72fda6b980 20 add_watcher() i=4 2024-05-30T16:34:59.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.132+0000 7f72fda6b980 20 add_watcher() i=5 2024-05-30T16:34:59.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.132+0000 7f72fda6b980 20 add_watcher() i=6 2024-05-30T16:34:59.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.136+0000 7f72fda6b980 20 add_watcher() i=7 2024-05-30T16:34:59.135 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.136+0000 7f72fda6b980 2 all 8 watchers are set, enabling cache 2024-05-30T16:34:59.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.136+0000 7f71b27e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:59.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.136+0000 7f72fda6b980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:34:59.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.136+0000 7f72fda6b980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:34:59.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.136+0000 7f72fda6b980 5 note: GC not initialized 2024-05-30T16:34:59.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.140+0000 7f71b17e2640 20 reqs_thread_entry: start 2024-05-30T16:34:59.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.140+0000 7f72fda6b980 20 init_complete bucket index max shards: 11 2024-05-30T16:34:59.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.140+0000 7f72fda6b980 20 Filter name: none 2024-05-30T16:34:59.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.140+0000 7f71aa7fc640 20 reqs_thread_entry: start 2024-05-30T16:34:59.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.140+0000 7f72fda6b980 10 cache get: name=default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 : miss 2024-05-30T16:34:59.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.140+0000 7f72fda6b980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.140+0000 7f72fda6b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:59.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.140+0000 7f72fda6b980 10 cache put: name=default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 info.flags=0x0 2024-05-30T16:34:59.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.140+0000 7f72fda6b980 10 adding default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 to cache LRU end 2024-05-30T16:34:59.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.140+0000 7f72fda6b980 20 failed to read name.$Mr.RGW88888888888888888 with: (2) No such file or directory 2024-05-30T16:34:59.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.140+0000 7f72fda6b980 10 cache get: name=default.rgw.meta+users.email+rgw88888888888888888+test@test.test : miss 2024-05-30T16:34:59.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.140+0000 7f72fda6b980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.140+0000 7f72fda6b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:59.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.140+0000 7f72fda6b980 10 cache put: name=default.rgw.meta+users.email+rgw88888888888888888+test@test.test info.flags=0x0 2024-05-30T16:34:59.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.140+0000 7f72fda6b980 10 adding default.rgw.meta+users.email+rgw88888888888888888+test@test.test to cache LRU end 2024-05-30T16:34:59.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.140+0000 7f72fda6b980 20 failed to read rgw88888888888888888+test@test.test with: (2) No such file or directory 2024-05-30T16:34:59.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.140+0000 7f72fda6b980 10 cache put: name=default.rgw.meta+accounts+account.RGW88888888888888888 info.flags=0x17 2024-05-30T16:34:59.141 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.140+0000 7f72fda6b980 10 adding default.rgw.meta+accounts+account.RGW88888888888888888 to cache LRU end 2024-05-30T16:34:59.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.140+0000 7f72fda6b980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:accounts:account.RGW88888888888888888, ofs0, ns] 2024-05-30T16:34:59.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.144+0000 7f71deffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333598 cookie 93958195195776 notifier 7361 bl.length()=384 2024-05-30T16:34:59.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.144+0000 7f71deffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+account.RGW88888888888888888 info.flags=0x17 2024-05-30T16:34:59.142 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.144+0000 7f71deffd640 10 rgw watcher librados: moving default.rgw.meta+accounts+account.RGW88888888888888888 to cache LRU end 2024-05-30T16:34:59.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.148+0000 7f72fda6b980 10 cache put: name=default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 info.flags=0x17 2024-05-30T16:34:59.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.148+0000 7f72fda6b980 10 moving default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 to cache LRU end 2024-05-30T16:34:59.147 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.148+0000 7f72fda6b980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:accounts:name.$Mr.RGW88888888888888888, ofs0, ns] 2024-05-30T16:34:59.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.148+0000 7f71de7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333594 cookie 93958195202048 notifier 7361 bl.length()=225 2024-05-30T16:34:59.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.148+0000 7f71de7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 info.flags=0x17 2024-05-30T16:34:59.148 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.148+0000 7f71de7fc640 10 rgw watcher librados: moving default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 to cache LRU end 2024-05-30T16:34:59.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.152+0000 7f72fda6b980 10 cache put: name=default.rgw.meta+users.email+rgw88888888888888888+test@test.test info.flags=0x17 2024-05-30T16:34:59.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.152+0000 7f72fda6b980 10 moving default.rgw.meta+users.email+rgw88888888888888888+test@test.test to cache LRU end 2024-05-30T16:34:59.152 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.152+0000 7f72fda6b980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:rgw88888888888888888+test@test.test, ofs0, ns] 2024-05-30T16:34:59.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.152+0000 7f71de7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333532 cookie 93958195196832 notifier 7361 bl.length()=234 2024-05-30T16:34:59.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.152+0000 7f71de7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+rgw88888888888888888+test@test.test info.flags=0x17 2024-05-30T16:34:59.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.152+0000 7f71de7fc640 10 rgw watcher librados: moving default.rgw.meta+users.email+rgw88888888888888888+test@test.test to cache LRU end 2024-05-30T16:34:59.155 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.156+0000 7f71b27e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:59.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.160+0000 7f72fda6b980 20 remove_watcher() i=0 2024-05-30T16:34:59.160 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.160+0000 7f72fda6b980 2 removed watcher, disabling cache 2024-05-30T16:34:59.162 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.164+0000 7f72fda6b980 20 remove_watcher() i=1 2024-05-30T16:34:59.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.164+0000 7f72fda6b980 20 remove_watcher() i=2 2024-05-30T16:34:59.166 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.168+0000 7f72fda6b980 20 remove_watcher() i=3 2024-05-30T16:34:59.168 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.168+0000 7f72fda6b980 20 remove_watcher() i=4 2024-05-30T16:34:59.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.172+0000 7f72fda6b980 20 remove_watcher() i=5 2024-05-30T16:34:59.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.172+0000 7f72fda6b980 20 remove_watcher() i=6 2024-05-30T16:34:59.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.172+0000 7f72fda6b980 20 remove_watcher() i=7 2024-05-30T16:34:59.185 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-30T16:34:59.185 INFO:teuthology.orchestra.run.smithi006.stdout: "id": "RGW88888888888888888", 2024-05-30T16:34:59.185 INFO:teuthology.orchestra.run.smithi006.stdout: "tenant": "", 2024-05-30T16:34:59.185 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "Mr.RGW88888888888888888", 2024-05-30T16:34:59.185 INFO:teuthology.orchestra.run.smithi006.stdout: "email": "RGW88888888888888888+test@test.test", 2024-05-30T16:34:59.185 INFO:teuthology.orchestra.run.smithi006.stdout: "quota": { 2024-05-30T16:34:59.185 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": false, 2024-05-30T16:34:59.185 INFO:teuthology.orchestra.run.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:34:59.185 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size": -1, 2024-05-30T16:34:59.185 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:34:59.185 INFO:teuthology.orchestra.run.smithi006.stdout: "max_objects": -1 2024-05-30T16:34:59.186 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T16:34:59.186 INFO:teuthology.orchestra.run.smithi006.stdout: "bucket_quota": { 2024-05-30T16:34:59.186 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": false, 2024-05-30T16:34:59.186 INFO:teuthology.orchestra.run.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:34:59.186 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size": -1, 2024-05-30T16:34:59.186 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:34:59.186 INFO:teuthology.orchestra.run.smithi006.stdout: "max_objects": -1 2024-05-30T16:34:59.186 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T16:34:59.186 INFO:teuthology.orchestra.run.smithi006.stdout: "max_users": 1000, 2024-05-30T16:34:59.186 INFO:teuthology.orchestra.run.smithi006.stdout: "max_roles": 1000, 2024-05-30T16:34:59.186 INFO:teuthology.orchestra.run.smithi006.stdout: "max_groups": 1000, 2024-05-30T16:34:59.186 INFO:teuthology.orchestra.run.smithi006.stdout: "max_buckets": 1000, 2024-05-30T16:34:59.186 INFO:teuthology.orchestra.run.smithi006.stdout: "max_access_keys": 4 2024-05-30T16:34:59.186 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-30T16:34:59.187 DEBUG:tasks.s3tests:Creating user RGW88888888888888888 on client.0 2024-05-30T16:34:59.188 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid root1.client.0 --display-name Mr.RGW88888888888888888 --email root1+test@test.test --access-key DWRZQWFGPXOSEJPMOQJN --secret JZCbFq5VyJaouXBb3WCuSk/PXTimviMhr/F3spFhdsmjbj+uXM3ICQ== --cluster ceph --account-id RGW88888888888888888 --account-root 2024-05-30T16:34:59.272 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:34:59.272 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:34:59.315 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.316+0000 7fb134d2c980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.316+0000 7fb134d2c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:59.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.316+0000 7fb134d2c980 20 realm 2024-05-30T16:34:59.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.316+0000 7fb134d2c980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.316+0000 7fb134d2c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:59.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.316+0000 7fb134d2c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:34:59.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.316+0000 7fb134d2c980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.316+0000 7fb134d2c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:59.317 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.316+0000 7fb134d2c980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.320+0000 7fb134d2c980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:59.318 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.320+0000 7fb134d2c980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.320+0000 7fb134d2c980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:34:59.319 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.320+0000 7fb134d2c980 20 searching for the correct realm 2024-05-30T16:34:59.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.324+0000 7fb13303e640 0 -- 172.21.15.6:0/3251419729 >> [v2:172.21.15.6:6808/1897362715,v1:172.21.15.6:6809/1897362715] conn(0x5628cede6e50 msgr2=0x5628cede9310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:34:59.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.332+0000 7fb13203c640 0 -- 172.21.15.6:0/3251419729 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7fb12c0701d0 msgr2=0x7fb12c010800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:34:59.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.340+0000 7fb134d2c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:34:59.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.340+0000 7fb134d2c980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:34:59.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.340+0000 7fb134d2c980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:34:59.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.340+0000 7fb134d2c980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:34:59.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.340+0000 7fb134d2c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:34:59.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.340+0000 7fb134d2c980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:34:59.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.340+0000 7fb134d2c980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.340+0000 7fb134d2c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:59.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.340+0000 7fb134d2c980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.344+0000 7fb134d2c980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:59.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.344+0000 7fb134d2c980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.344+0000 7fb134d2c980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:34:59.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.344+0000 7fb134d2c980 20 zone default found 2024-05-30T16:34:59.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.344+0000 7fb134d2c980 4 Realm: () 2024-05-30T16:34:59.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.344+0000 7fb134d2c980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:34:59.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.344+0000 7fb134d2c980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:34:59.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.344+0000 7fb134d2c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:34:59.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.344+0000 7fb134d2c980 20 zonegroup default 2024-05-30T16:34:59.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.344+0000 7fb134d2c980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.344+0000 7fb134d2c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:59.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.344+0000 7fb134d2c980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.344+0000 7fb134d2c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:59.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.344+0000 7fb134d2c980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.344+0000 7fb134d2c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:59.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.344+0000 7fb134d2c980 20 started sync module instance, tier type = 2024-05-30T16:34:59.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.344+0000 7fb134d2c980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:34:59.360 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.360+0000 7fb134d2c980 20 add_watcher() i=0 2024-05-30T16:34:59.360 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.360+0000 7fb134d2c980 20 add_watcher() i=1 2024-05-30T16:34:59.360 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.360+0000 7fb134d2c980 20 add_watcher() i=2 2024-05-30T16:34:59.360 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.360+0000 7fb134d2c980 20 add_watcher() i=3 2024-05-30T16:34:59.360 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.360+0000 7fb134d2c980 20 add_watcher() i=4 2024-05-30T16:34:59.360 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.360+0000 7fb134d2c980 20 add_watcher() i=5 2024-05-30T16:34:59.360 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.360+0000 7fb134d2c980 20 add_watcher() i=6 2024-05-30T16:34:59.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.360+0000 7fb134d2c980 20 add_watcher() i=7 2024-05-30T16:34:59.361 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.360+0000 7fb134d2c980 2 all 8 watchers are set, enabling cache 2024-05-30T16:34:59.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.364+0000 7fafe97e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:59.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.364+0000 7fb134d2c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:34:59.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.364+0000 7fb134d2c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:34:59.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.364+0000 7fb134d2c980 5 note: GC not initialized 2024-05-30T16:34:59.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.364+0000 7faf63fff640 20 reqs_thread_entry: start 2024-05-30T16:34:59.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.364+0000 7fb134d2c980 20 init_complete bucket index max shards: 11 2024-05-30T16:34:59.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.364+0000 7fb134d2c980 20 Filter name: none 2024-05-30T16:34:59.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.364+0000 7faf61ffb640 20 reqs_thread_entry: start 2024-05-30T16:34:59.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.364+0000 7fb134d2c980 10 cache get: name=default.rgw.meta+users.uid+root1.client.0 : miss 2024-05-30T16:34:59.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.364+0000 7fb134d2c980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.364+0000 7fb134d2c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:59.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.364+0000 7fb134d2c980 10 cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x0 2024-05-30T16:34:59.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.364+0000 7fb134d2c980 10 adding default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2024-05-30T16:34:59.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.364+0000 7fb134d2c980 10 cache get: name=default.rgw.meta+users.email+root1+test@test.test : miss 2024-05-30T16:34:59.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.364+0000 7fb134d2c980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.368+0000 7fb134d2c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:59.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.368+0000 7fb134d2c980 10 cache put: name=default.rgw.meta+users.email+root1+test@test.test info.flags=0x0 2024-05-30T16:34:59.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.368+0000 7fb134d2c980 10 adding default.rgw.meta+users.email+root1+test@test.test to cache LRU end 2024-05-30T16:34:59.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.368+0000 7fb134d2c980 10 cache get: name=default.rgw.meta+users.keys+DWRZQWFGPXOSEJPMOQJN : miss 2024-05-30T16:34:59.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.368+0000 7fb134d2c980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.368+0000 7fb134d2c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:59.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.368+0000 7fb134d2c980 10 cache put: name=default.rgw.meta+users.keys+DWRZQWFGPXOSEJPMOQJN info.flags=0x0 2024-05-30T16:34:59.367 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.368+0000 7fb134d2c980 10 adding default.rgw.meta+users.keys+DWRZQWFGPXOSEJPMOQJN to cache LRU end 2024-05-30T16:34:59.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.368+0000 7fb134d2c980 10 cache get: name=default.rgw.meta+accounts+account.RGW88888888888888888 : miss 2024-05-30T16:34:59.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.368+0000 7fb134d2c980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.368+0000 7fb134d2c980 20 rados_obj.operate() r=0 bl.length=184 2024-05-30T16:34:59.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.368+0000 7fb134d2c980 10 cache put: name=default.rgw.meta+accounts+account.RGW88888888888888888 info.flags=0x17 2024-05-30T16:34:59.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.368+0000 7fb134d2c980 10 adding default.rgw.meta+accounts+account.RGW88888888888888888 to cache LRU end 2024-05-30T16:34:59.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.368+0000 7fb134d2c980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:34:59.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.368+0000 7fb134d2c980 10 cache get: name=default.rgw.meta+users.keys+DWRZQWFGPXOSEJPMOQJN : hit (negative entry) 2024-05-30T16:34:59.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.368+0000 7fb134d2c980 10 cache get: name=default.rgw.meta+users.keys+DWRZQWFGPXOSEJPMOQJN : hit (negative entry) 2024-05-30T16:34:59.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.372+0000 7fb134d2c980 10 cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x17 2024-05-30T16:34:59.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.372+0000 7fb134d2c980 10 moving default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2024-05-30T16:34:59.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.372+0000 7fb134d2c980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:root1.client.0, ofs0, ns] 2024-05-30T16:34:59.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.372+0000 7fb0167fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333599 cookie 94733269415040 notifier 7379 bl.length()=681 2024-05-30T16:34:59.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.372+0000 7fb0167fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x17 2024-05-30T16:34:59.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.372+0000 7fb0167fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2024-05-30T16:34:59.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.376+0000 7fb134d2c980 10 cache put: name=default.rgw.meta+users.email+root1+test@test.test info.flags=0x7 2024-05-30T16:34:59.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.376+0000 7fb134d2c980 10 moving default.rgw.meta+users.email+root1+test@test.test to cache LRU end 2024-05-30T16:34:59.375 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.376+0000 7fb134d2c980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.email:root1+test@test.test, ofs0, ns] 2024-05-30T16:34:59.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.376+0000 7fb0167fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333629 cookie 94733269425168 notifier 7379 bl.length()=189 2024-05-30T16:34:59.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.376+0000 7fb0167fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+root1+test@test.test info.flags=0x7 2024-05-30T16:34:59.376 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.376+0000 7fb0167fc640 10 rgw watcher librados: moving default.rgw.meta+users.email+root1+test@test.test to cache LRU end 2024-05-30T16:34:59.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.380+0000 7fb134d2c980 10 cache put: name=default.rgw.meta+users.keys+DWRZQWFGPXOSEJPMOQJN info.flags=0x7 2024-05-30T16:34:59.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.380+0000 7fb134d2c980 10 moving default.rgw.meta+users.keys+DWRZQWFGPXOSEJPMOQJN to cache LRU end 2024-05-30T16:34:59.378 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.380+0000 7fb134d2c980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.keys:DWRZQWFGPXOSEJPMOQJN, ofs0, ns] 2024-05-30T16:34:59.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.380+0000 7fb015ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333533 cookie 94733269416096 notifier 7379 bl.length()=188 2024-05-30T16:34:59.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.380+0000 7fb015ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+DWRZQWFGPXOSEJPMOQJN info.flags=0x7 2024-05-30T16:34:59.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.380+0000 7fb015ffb640 10 rgw watcher librados: moving default.rgw.meta+users.keys+DWRZQWFGPXOSEJPMOQJN to cache LRU end 2024-05-30T16:34:59.383 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.384+0000 7fb134d2c980 20 linked user root1.client.0 to account RGW88888888888888888 2024-05-30T16:34:59.384 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-30T16:34:59.384 INFO:teuthology.orchestra.run.smithi006.stdout: "user_id": "root1.client.0", 2024-05-30T16:34:59.384 INFO:teuthology.orchestra.run.smithi006.stdout: "display_name": "Mr.RGW88888888888888888", 2024-05-30T16:34:59.384 INFO:teuthology.orchestra.run.smithi006.stdout: "email": "root1+test@test.test", 2024-05-30T16:34:59.384 INFO:teuthology.orchestra.run.smithi006.stdout: "suspended": 0, 2024-05-30T16:34:59.384 INFO:teuthology.orchestra.run.smithi006.stdout: "max_buckets": 1000, 2024-05-30T16:34:59.384 INFO:teuthology.orchestra.run.smithi006.stdout: "subusers": [], 2024-05-30T16:34:59.384 INFO:teuthology.orchestra.run.smithi006.stdout: "keys": [ 2024-05-30T16:34:59.384 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T16:34:59.384 INFO:teuthology.orchestra.run.smithi006.stdout: "user": "root1.client.0", 2024-05-30T16:34:59.384 INFO:teuthology.orchestra.run.smithi006.stdout: "access_key": "DWRZQWFGPXOSEJPMOQJN", 2024-05-30T16:34:59.384 INFO:teuthology.orchestra.run.smithi006.stdout: "secret_key": "JZCbFq5VyJaouXBb3WCuSk/PXTimviMhr/F3spFhdsmjbj+uXM3ICQ==", 2024-05-30T16:34:59.385 INFO:teuthology.orchestra.run.smithi006.stdout: "active": true, 2024-05-30T16:34:59.385 INFO:teuthology.orchestra.run.smithi006.stdout: "create_date": "2024-05-30T16:34:59.370676Z" 2024-05-30T16:34:59.385 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T16:34:59.385 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-30T16:34:59.385 INFO:teuthology.orchestra.run.smithi006.stdout: "swift_keys": [], 2024-05-30T16:34:59.385 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": [], 2024-05-30T16:34:59.385 INFO:teuthology.orchestra.run.smithi006.stdout: "op_mask": "read, write, delete", 2024-05-30T16:34:59.385 INFO:teuthology.orchestra.run.smithi006.stdout: "default_placement": "", 2024-05-30T16:34:59.385 INFO:teuthology.orchestra.run.smithi006.stdout: "default_storage_class": "", 2024-05-30T16:34:59.385 INFO:teuthology.orchestra.run.smithi006.stdout: "placement_tags": [], 2024-05-30T16:34:59.385 INFO:teuthology.orchestra.run.smithi006.stdout: "bucket_quota": { 2024-05-30T16:34:59.385 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": false, 2024-05-30T16:34:59.385 INFO:teuthology.orchestra.run.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:34:59.385 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size": -1, 2024-05-30T16:34:59.385 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:34:59.386 INFO:teuthology.orchestra.run.smithi006.stdout: "max_objects": -1 2024-05-30T16:34:59.386 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T16:34:59.386 INFO:teuthology.orchestra.run.smithi006.stdout: "user_quota": { 2024-05-30T16:34:59.386 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": false, 2024-05-30T16:34:59.386 INFO:teuthology.orchestra.run.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:34:59.386 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size": -1, 2024-05-30T16:34:59.386 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:34:59.386 INFO:teuthology.orchestra.run.smithi006.stdout: "max_objects": -1 2024-05-30T16:34:59.386 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T16:34:59.386 INFO:teuthology.orchestra.run.smithi006.stdout: "temp_url_keys": [], 2024-05-30T16:34:59.386 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "root", 2024-05-30T16:34:59.386 INFO:teuthology.orchestra.run.smithi006.stdout: "mfa_ids": [], 2024-05-30T16:34:59.386 INFO:teuthology.orchestra.run.smithi006.stdout: "account_id": "RGW88888888888888888", 2024-05-30T16:34:59.386 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2024-05-30T16:34:59.387 INFO:teuthology.orchestra.run.smithi006.stdout: "create_date": "2024-05-30T16:34:59.370655Z", 2024-05-30T16:34:59.387 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-30T16:34:59.387 INFO:teuthology.orchestra.run.smithi006.stdout: "group_ids": [] 2024-05-30T16:34:59.387 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-30T16:34:59.387 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:34:59.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.384+0000 7fafe97e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:59.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.388+0000 7fb134d2c980 20 remove_watcher() i=0 2024-05-30T16:34:59.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.388+0000 7fb134d2c980 2 removed watcher, disabling cache 2024-05-30T16:34:59.392 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.392+0000 7fb134d2c980 20 remove_watcher() i=1 2024-05-30T16:34:59.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.396+0000 7fb134d2c980 20 remove_watcher() i=2 2024-05-30T16:34:59.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.396+0000 7fb134d2c980 20 remove_watcher() i=3 2024-05-30T16:34:59.399 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.400+0000 7fb134d2c980 20 remove_watcher() i=4 2024-05-30T16:34:59.400 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.400+0000 7fb134d2c980 20 remove_watcher() i=5 2024-05-30T16:34:59.401 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.400+0000 7fb134d2c980 20 remove_watcher() i=6 2024-05-30T16:34:59.403 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.404+0000 7fb134d2c980 20 remove_watcher() i=7 2024-05-30T16:34:59.417 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid root1.client.0 --totp-serial 8359548345 --totp-seed PLH73WJQQVOGV7VCF7LGJDLDWXV6SO5IVYAHOD4V6OWFY3DXJ7R7WYZLXLM3VBRN --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-05-30T16:34:59.505 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:34:59.505 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:34:59.548 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.548+0000 7f2cebcb5980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.548+0000 7f2cebcb5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:59.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.548+0000 7f2cebcb5980 20 realm 2024-05-30T16:34:59.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.548+0000 7f2cebcb5980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.548+0000 7f2cebcb5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:59.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.548+0000 7f2cebcb5980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:34:59.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.552+0000 7f2cebcb5980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.552+0000 7f2cebcb5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:59.550 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.552+0000 7f2cebcb5980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.552+0000 7f2cebcb5980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:59.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.552+0000 7f2cebcb5980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.552 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.552+0000 7f2cebcb5980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:34:59.553 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.552+0000 7f2cebcb5980 20 searching for the correct realm 2024-05-30T16:34:59.558 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.560+0000 7f2ce9fc7640 0 -- 172.21.15.6:0/4185088025 >> [v2:172.21.15.204:6823/3156311527,v1:172.21.15.204:6825/3156311527] conn(0x7f2ce002b060 msgr2=0x7f2ce005b030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:34:59.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.564+0000 7f2ce97c6640 0 -- 172.21.15.6:0/4185088025 >> [v2:172.21.15.6:6800/177258275,v1:172.21.15.6:6801/177258275] conn(0x7f2ce0008050 msgr2=0x7f2ce0075a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:34:59.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.780+0000 7f2cebcb5980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:34:59.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.780+0000 7f2cebcb5980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:34:59.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.780+0000 7f2cebcb5980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:34:59.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.780+0000 7f2cebcb5980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:34:59.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.780+0000 7f2cebcb5980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:34:59.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.780+0000 7f2cebcb5980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:34:59.781 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.780+0000 7f2cebcb5980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.780+0000 7f2cebcb5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:59.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.780+0000 7f2cebcb5980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.784+0000 7f2cebcb5980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:59.782 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.784+0000 7f2cebcb5980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.784+0000 7f2cebcb5980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:34:59.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.784+0000 7f2cebcb5980 20 zone default found 2024-05-30T16:34:59.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.784+0000 7f2cebcb5980 4 Realm: () 2024-05-30T16:34:59.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.784+0000 7f2cebcb5980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:34:59.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.784+0000 7f2cebcb5980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:34:59.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.784+0000 7f2cebcb5980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:34:59.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.784+0000 7f2cebcb5980 20 zonegroup default 2024-05-30T16:34:59.783 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.784+0000 7f2cebcb5980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.784+0000 7f2cebcb5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:59.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.784+0000 7f2cebcb5980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.784 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.784+0000 7f2cebcb5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:59.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.784+0000 7f2cebcb5980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.784+0000 7f2cebcb5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:59.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.784+0000 7f2cebcb5980 20 started sync module instance, tier type = 2024-05-30T16:34:59.785 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.784+0000 7f2cebcb5980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:34:59.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.808+0000 7f2cebcb5980 20 add_watcher() i=0 2024-05-30T16:34:59.808 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.808+0000 7f2cebcb5980 20 add_watcher() i=1 2024-05-30T16:34:59.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.808+0000 7f2cebcb5980 20 add_watcher() i=2 2024-05-30T16:34:59.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.808+0000 7f2cebcb5980 20 add_watcher() i=3 2024-05-30T16:34:59.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.808+0000 7f2cebcb5980 20 add_watcher() i=4 2024-05-30T16:34:59.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.808+0000 7f2cebcb5980 20 add_watcher() i=5 2024-05-30T16:34:59.809 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.808+0000 7f2cebcb5980 20 add_watcher() i=6 2024-05-30T16:34:59.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.812+0000 7f2cebcb5980 20 add_watcher() i=7 2024-05-30T16:34:59.811 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.812+0000 7f2cebcb5980 2 all 8 watchers are set, enabling cache 2024-05-30T16:34:59.815 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.816+0000 7f2b9f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:59.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.816+0000 7f2cebcb5980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:34:59.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.816+0000 7f2cebcb5980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:34:59.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.816+0000 7f2cebcb5980 5 note: GC not initialized 2024-05-30T16:34:59.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.816+0000 7f2b9e7e4640 20 reqs_thread_entry: start 2024-05-30T16:34:59.816 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.816+0000 7f2cebcb5980 20 init_complete bucket index max shards: 11 2024-05-30T16:34:59.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.816+0000 7f2cebcb5980 20 Filter name: none 2024-05-30T16:34:59.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.816+0000 7f2b0ffff640 20 reqs_thread_entry: start 2024-05-30T16:34:59.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.816+0000 7f2cebcb5980 10 cache get: name=default.rgw.meta+users.uid+root1.client.0 : miss 2024-05-30T16:34:59.817 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.816+0000 7f2cebcb5980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.820+0000 7f2cebcb5980 20 rados_obj.operate() r=0 bl.length=494 2024-05-30T16:34:59.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.820+0000 7f2cebcb5980 10 cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x13 2024-05-30T16:34:59.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.820+0000 7f2cebcb5980 10 adding default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2024-05-30T16:34:59.818 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.820+0000 7f2cebcb5980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:34:59.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.824+0000 7f2cebcb5980 10 cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x17 2024-05-30T16:34:59.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.824+0000 7f2cebcb5980 10 moving default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2024-05-30T16:34:59.822 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.824+0000 7f2cebcb5980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:root1.client.0, ofs0, ns] 2024-05-30T16:34:59.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.824+0000 7f2bccff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333600 cookie 94570487863936 notifier 7397 bl.length()=695 2024-05-30T16:34:59.823 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.824+0000 7f2bccff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x17 2024-05-30T16:34:59.824 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.824+0000 7f2bccff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2024-05-30T16:34:59.827 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.828+0000 7f2b9f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:34:59.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.832+0000 7f2cebcb5980 20 remove_watcher() i=0 2024-05-30T16:34:59.832 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.832+0000 7f2cebcb5980 2 removed watcher, disabling cache 2024-05-30T16:34:59.834 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.836+0000 7f2cebcb5980 20 remove_watcher() i=1 2024-05-30T16:34:59.837 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.836+0000 7f2cebcb5980 20 remove_watcher() i=2 2024-05-30T16:34:59.839 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.840+0000 7f2cebcb5980 20 remove_watcher() i=3 2024-05-30T16:34:59.842 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.840+0000 7f2cebcb5980 20 remove_watcher() i=4 2024-05-30T16:34:59.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.844+0000 7f2cebcb5980 20 remove_watcher() i=5 2024-05-30T16:34:59.844 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.844+0000 7f2cebcb5980 20 remove_watcher() i=6 2024-05-30T16:34:59.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.848+0000 7f2cebcb5980 20 remove_watcher() i=7 2024-05-30T16:34:59.864 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph account create --account-id RGW99999999999999999 --account-name Mr.RGW99999999999999999 --email RGW99999999999999999+test@test.test 2024-05-30T16:34:59.905 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:34:59.905 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:34:59.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.948+0000 7f15aa115980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.948+0000 7f15aa115980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:59.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.948+0000 7f15aa115980 20 realm 2024-05-30T16:34:59.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.948+0000 7f15aa115980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.952+0000 7f15aa115980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:59.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.952+0000 7f15aa115980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:34:59.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.952+0000 7f15aa115980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.952+0000 7f15aa115980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:34:59.951 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.952+0000 7f15aa115980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.952+0000 7f15aa115980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:34:59.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.952+0000 7f15aa115980 20 rados->read ofs=0 len=0 2024-05-30T16:34:59.952 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.952+0000 7f15aa115980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:34:59.953 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.952+0000 7f15aa115980 20 searching for the correct realm 2024-05-30T16:34:59.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:34:59.960+0000 7f15a8427640 0 -- 172.21.15.6:0/812529416 >> [v2:172.21.15.204:6800/3687190393,v1:172.21.15.204:6801/3687190393] conn(0x7f159c0c5040 msgr2=0x7f159c0d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T16:35:00.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.176+0000 7f15a7c26640 0 -- 172.21.15.6:0/812529416 >> [v2:172.21.15.204:6816/2677861390,v1:172.21.15.204:6817/2677861390] conn(0x55d453a34cb0 msgr2=0x55d453a37100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:35:00.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.184+0000 7f15aa115980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:35:00.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.184+0000 7f15aa115980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:35:00.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.184+0000 7f15aa115980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:35:00.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.184+0000 7f15aa115980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:35:00.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.184+0000 7f15aa115980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:35:00.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.184+0000 7f15aa115980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:35:00.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.184+0000 7f15aa115980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.184+0000 7f15aa115980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:35:00.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.184+0000 7f15aa115980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.184+0000 7f15aa115980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:35:00.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.184+0000 7f15aa115980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.184+0000 7f15aa115980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:35:00.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.184+0000 7f15aa115980 20 zone default found 2024-05-30T16:35:00.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.184+0000 7f15aa115980 4 Realm: () 2024-05-30T16:35:00.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.184+0000 7f15aa115980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:35:00.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.184+0000 7f15aa115980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:35:00.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.184+0000 7f15aa115980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:35:00.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.184+0000 7f15aa115980 20 zonegroup default 2024-05-30T16:35:00.186 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.184+0000 7f15aa115980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.188+0000 7f15aa115980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:35:00.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.188+0000 7f15aa115980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.188+0000 7f15aa115980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:35:00.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.188+0000 7f15aa115980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.188+0000 7f15aa115980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:35:00.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.188+0000 7f15aa115980 20 started sync module instance, tier type = 2024-05-30T16:35:00.188 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.188+0000 7f15aa115980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:35:00.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.212+0000 7f15aa115980 20 add_watcher() i=0 2024-05-30T16:35:00.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.212+0000 7f15aa115980 20 add_watcher() i=1 2024-05-30T16:35:00.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.212+0000 7f15aa115980 20 add_watcher() i=2 2024-05-30T16:35:00.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.212+0000 7f15aa115980 20 add_watcher() i=3 2024-05-30T16:35:00.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.212+0000 7f15aa115980 20 add_watcher() i=4 2024-05-30T16:35:00.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.212+0000 7f15aa115980 20 add_watcher() i=5 2024-05-30T16:35:00.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.212+0000 7f15aa115980 20 add_watcher() i=6 2024-05-30T16:35:00.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.216+0000 7f15aa115980 20 add_watcher() i=7 2024-05-30T16:35:00.216 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.216+0000 7f15aa115980 2 all 8 watchers are set, enabling cache 2024-05-30T16:35:00.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.220+0000 7f145dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:35:00.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.220+0000 7f15aa115980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:35:00.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.220+0000 7f15aa115980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:35:00.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.220+0000 7f15aa115980 5 note: GC not initialized 2024-05-30T16:35:00.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.220+0000 7f145cfe1640 20 reqs_thread_entry: start 2024-05-30T16:35:00.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.220+0000 7f15aa115980 20 init_complete bucket index max shards: 11 2024-05-30T16:35:00.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.220+0000 7f15aa115980 20 Filter name: none 2024-05-30T16:35:00.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.220+0000 7f13d67fc640 20 reqs_thread_entry: start 2024-05-30T16:35:00.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.224+0000 7f15aa115980 10 cache get: name=default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 : miss 2024-05-30T16:35:00.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.224+0000 7f15aa115980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.224+0000 7f15aa115980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:35:00.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.224+0000 7f15aa115980 10 cache put: name=default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 info.flags=0x0 2024-05-30T16:35:00.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.224+0000 7f15aa115980 10 adding default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 to cache LRU end 2024-05-30T16:35:00.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.224+0000 7f15aa115980 20 failed to read name.$Mr.RGW99999999999999999 with: (2) No such file or directory 2024-05-30T16:35:00.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.224+0000 7f15aa115980 10 cache get: name=default.rgw.meta+users.email+rgw99999999999999999+test@test.test : miss 2024-05-30T16:35:00.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.224+0000 7f15aa115980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.223 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.224+0000 7f15aa115980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:35:00.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.224+0000 7f15aa115980 10 cache put: name=default.rgw.meta+users.email+rgw99999999999999999+test@test.test info.flags=0x0 2024-05-30T16:35:00.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.224+0000 7f15aa115980 10 adding default.rgw.meta+users.email+rgw99999999999999999+test@test.test to cache LRU end 2024-05-30T16:35:00.224 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.224+0000 7f15aa115980 20 failed to read rgw99999999999999999+test@test.test with: (2) No such file or directory 2024-05-30T16:35:00.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.228+0000 7f15aa115980 10 cache put: name=default.rgw.meta+accounts+account.RGW99999999999999999 info.flags=0x17 2024-05-30T16:35:00.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.228+0000 7f15aa115980 10 adding default.rgw.meta+accounts+account.RGW99999999999999999 to cache LRU end 2024-05-30T16:35:00.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.228+0000 7f15aa115980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:accounts:account.RGW99999999999999999, ofs0, ns] 2024-05-30T16:35:00.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.228+0000 7f1486ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333601 cookie 94370424674032 notifier 7412 bl.length()=384 2024-05-30T16:35:00.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.228+0000 7f1486ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+account.RGW99999999999999999 info.flags=0x17 2024-05-30T16:35:00.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.228+0000 7f1486ffd640 10 rgw watcher librados: moving default.rgw.meta+accounts+account.RGW99999999999999999 to cache LRU end 2024-05-30T16:35:00.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.232+0000 7f15aa115980 10 cache put: name=default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 info.flags=0x17 2024-05-30T16:35:00.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.232+0000 7f15aa115980 10 moving default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 to cache LRU end 2024-05-30T16:35:00.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.232+0000 7f15aa115980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:accounts:name.$Mr.RGW99999999999999999, ofs0, ns] 2024-05-30T16:35:00.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.236+0000 7f1486ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333595 cookie 94370424682768 notifier 7412 bl.length()=225 2024-05-30T16:35:00.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.236+0000 7f1486ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 info.flags=0x17 2024-05-30T16:35:00.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.236+0000 7f1486ffd640 10 rgw watcher librados: moving default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 to cache LRU end 2024-05-30T16:35:00.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.240+0000 7f15aa115980 10 cache put: name=default.rgw.meta+users.email+rgw99999999999999999+test@test.test info.flags=0x17 2024-05-30T16:35:00.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.240+0000 7f15aa115980 10 moving default.rgw.meta+users.email+rgw99999999999999999+test@test.test to cache LRU end 2024-05-30T16:35:00.240 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.240+0000 7f15aa115980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:rgw99999999999999999+test@test.test, ofs0, ns] 2024-05-30T16:35:00.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.240+0000 7f14867fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333534 cookie 94370424679168 notifier 7412 bl.length()=234 2024-05-30T16:35:00.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.240+0000 7f14867fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+rgw99999999999999999+test@test.test info.flags=0x17 2024-05-30T16:35:00.241 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.240+0000 7f14867fc640 10 rgw watcher librados: moving default.rgw.meta+users.email+rgw99999999999999999+test@test.test to cache LRU end 2024-05-30T16:35:00.245 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.244+0000 7f145dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:35:00.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.252+0000 7f15aa115980 20 remove_watcher() i=0 2024-05-30T16:35:00.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.252+0000 7f15aa115980 2 removed watcher, disabling cache 2024-05-30T16:35:00.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.252+0000 7f15aa115980 20 remove_watcher() i=1 2024-05-30T16:35:00.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.256+0000 7f15aa115980 20 remove_watcher() i=2 2024-05-30T16:35:00.258 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.260+0000 7f15aa115980 20 remove_watcher() i=3 2024-05-30T16:35:00.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.260+0000 7f15aa115980 20 remove_watcher() i=4 2024-05-30T16:35:00.262 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.264+0000 7f15aa115980 20 remove_watcher() i=5 2024-05-30T16:35:00.264 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.264+0000 7f15aa115980 20 remove_watcher() i=6 2024-05-30T16:35:00.265 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.264+0000 7f15aa115980 20 remove_watcher() i=7 2024-05-30T16:35:00.278 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-30T16:35:00.278 INFO:teuthology.orchestra.run.smithi006.stdout: "id": "RGW99999999999999999", 2024-05-30T16:35:00.278 INFO:teuthology.orchestra.run.smithi006.stdout: "tenant": "", 2024-05-30T16:35:00.279 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "Mr.RGW99999999999999999", 2024-05-30T16:35:00.279 INFO:teuthology.orchestra.run.smithi006.stdout: "email": "RGW99999999999999999+test@test.test", 2024-05-30T16:35:00.279 INFO:teuthology.orchestra.run.smithi006.stdout: "quota": { 2024-05-30T16:35:00.279 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": false, 2024-05-30T16:35:00.279 INFO:teuthology.orchestra.run.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:35:00.279 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size": -1, 2024-05-30T16:35:00.279 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:35:00.279 INFO:teuthology.orchestra.run.smithi006.stdout: "max_objects": -1 2024-05-30T16:35:00.279 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T16:35:00.279 INFO:teuthology.orchestra.run.smithi006.stdout: "bucket_quota": { 2024-05-30T16:35:00.279 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": false, 2024-05-30T16:35:00.279 INFO:teuthology.orchestra.run.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:35:00.279 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size": -1, 2024-05-30T16:35:00.279 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:35:00.279 INFO:teuthology.orchestra.run.smithi006.stdout: "max_objects": -1 2024-05-30T16:35:00.280 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T16:35:00.280 INFO:teuthology.orchestra.run.smithi006.stdout: "max_users": 1000, 2024-05-30T16:35:00.280 INFO:teuthology.orchestra.run.smithi006.stdout: "max_roles": 1000, 2024-05-30T16:35:00.280 INFO:teuthology.orchestra.run.smithi006.stdout: "max_groups": 1000, 2024-05-30T16:35:00.280 INFO:teuthology.orchestra.run.smithi006.stdout: "max_buckets": 1000, 2024-05-30T16:35:00.280 INFO:teuthology.orchestra.run.smithi006.stdout: "max_access_keys": 4 2024-05-30T16:35:00.280 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-30T16:35:00.281 DEBUG:tasks.s3tests:Creating user RGW99999999999999999 on client.0 2024-05-30T16:35:00.281 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid root2.client.0 --display-name Mr.RGW99999999999999999 --email root2+test@test.test --access-key LQQKSDSIJPOLWLHQAPVT --secret x61bULegzZXPtIXjNfG76hPZRLraGkIVVTgGauhkR8b1/qeoQbhDbw== --cluster ceph --account-id RGW99999999999999999 --account-root 2024-05-30T16:35:00.372 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:35:00.372 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:35:00.415 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.416+0000 7f95297ad980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.416+0000 7f95297ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:35:00.417 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.416+0000 7f95297ad980 20 realm 2024-05-30T16:35:00.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.416+0000 7f95297ad980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.416+0000 7f95297ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:35:00.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.416+0000 7f95297ad980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:35:00.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.416+0000 7f95297ad980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.416+0000 7f95297ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:35:00.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.416+0000 7f95297ad980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.420+0000 7f95297ad980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:35:00.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.420+0000 7f95297ad980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.420+0000 7f95297ad980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:35:00.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.420+0000 7f95297ad980 20 searching for the correct realm 2024-05-30T16:35:00.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.428+0000 7f9527abf640 0 -- 172.21.15.6:0/2379754959 >> [v2:172.21.15.6:6808/1897362715,v1:172.21.15.6:6809/1897362715] conn(0x55e5ac9f74b0 msgr2=0x55e5ac9f9950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:35:00.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.432+0000 7f9526abd640 0 -- 172.21.15.6:0/2379754959 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f9518124040 msgr2=0x7f95180f2ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:35:00.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.440+0000 7f95297ad980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:35:00.440 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.440+0000 7f95297ad980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:35:00.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.440+0000 7f95297ad980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:35:00.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.440+0000 7f95297ad980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:35:00.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.440+0000 7f95297ad980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:35:00.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.440+0000 7f95297ad980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:35:00.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.440+0000 7f95297ad980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.440+0000 7f95297ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:35:00.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.440+0000 7f95297ad980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.441 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.440+0000 7f95297ad980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:35:00.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.440+0000 7f95297ad980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.444+0000 7f95297ad980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:35:00.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.444+0000 7f95297ad980 20 zone default found 2024-05-30T16:35:00.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.444+0000 7f95297ad980 4 Realm: () 2024-05-30T16:35:00.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.444+0000 7f95297ad980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:35:00.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.444+0000 7f95297ad980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:35:00.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.444+0000 7f95297ad980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:35:00.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.444+0000 7f95297ad980 20 zonegroup default 2024-05-30T16:35:00.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.444+0000 7f95297ad980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.444+0000 7f95297ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:35:00.443 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.444+0000 7f95297ad980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.444+0000 7f95297ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:35:00.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.444+0000 7f95297ad980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.444+0000 7f95297ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:35:00.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.444+0000 7f95297ad980 20 started sync module instance, tier type = 2024-05-30T16:35:00.444 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.444+0000 7f95297ad980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:35:00.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.460+0000 7f95297ad980 20 add_watcher() i=0 2024-05-30T16:35:00.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.460+0000 7f95297ad980 20 add_watcher() i=1 2024-05-30T16:35:00.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.460+0000 7f95297ad980 20 add_watcher() i=2 2024-05-30T16:35:00.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.460+0000 7f95297ad980 20 add_watcher() i=3 2024-05-30T16:35:00.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.460+0000 7f95297ad980 20 add_watcher() i=4 2024-05-30T16:35:00.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.460+0000 7f95297ad980 20 add_watcher() i=5 2024-05-30T16:35:00.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.460+0000 7f95297ad980 20 add_watcher() i=6 2024-05-30T16:35:00.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.460+0000 7f95297ad980 20 add_watcher() i=7 2024-05-30T16:35:00.461 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.460+0000 7f95297ad980 2 all 8 watchers are set, enabling cache 2024-05-30T16:35:00.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.464+0000 7f93ddfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:35:00.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.464+0000 7f95297ad980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:35:00.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.464+0000 7f95297ad980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:35:00.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.464+0000 7f95297ad980 5 note: GC not initialized 2024-05-30T16:35:00.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.464+0000 7f93dcfe1640 20 reqs_thread_entry: start 2024-05-30T16:35:00.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.464+0000 7f95297ad980 20 init_complete bucket index max shards: 11 2024-05-30T16:35:00.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.464+0000 7f95297ad980 20 Filter name: none 2024-05-30T16:35:00.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.464+0000 7f93567fc640 20 reqs_thread_entry: start 2024-05-30T16:35:00.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.464+0000 7f95297ad980 10 cache get: name=default.rgw.meta+users.uid+root2.client.0 : miss 2024-05-30T16:35:00.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.464+0000 7f95297ad980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.464+0000 7f95297ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:35:00.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.464+0000 7f95297ad980 10 cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x0 2024-05-30T16:35:00.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.464+0000 7f95297ad980 10 adding default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2024-05-30T16:35:00.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.464+0000 7f95297ad980 10 cache get: name=default.rgw.meta+users.email+root2+test@test.test : miss 2024-05-30T16:35:00.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.464+0000 7f95297ad980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.464+0000 7f95297ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:35:00.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.464+0000 7f95297ad980 10 cache put: name=default.rgw.meta+users.email+root2+test@test.test info.flags=0x0 2024-05-30T16:35:00.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.464+0000 7f95297ad980 10 adding default.rgw.meta+users.email+root2+test@test.test to cache LRU end 2024-05-30T16:35:00.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.464+0000 7f95297ad980 10 cache get: name=default.rgw.meta+users.keys+LQQKSDSIJPOLWLHQAPVT : miss 2024-05-30T16:35:00.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.464+0000 7f95297ad980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.467 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.468+0000 7f95297ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:35:00.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.468+0000 7f95297ad980 10 cache put: name=default.rgw.meta+users.keys+LQQKSDSIJPOLWLHQAPVT info.flags=0x0 2024-05-30T16:35:00.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.468+0000 7f95297ad980 10 adding default.rgw.meta+users.keys+LQQKSDSIJPOLWLHQAPVT to cache LRU end 2024-05-30T16:35:00.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.468+0000 7f95297ad980 10 cache get: name=default.rgw.meta+accounts+account.RGW99999999999999999 : miss 2024-05-30T16:35:00.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.468+0000 7f95297ad980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.468+0000 7f95297ad980 20 rados_obj.operate() r=0 bl.length=184 2024-05-30T16:35:00.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.468+0000 7f95297ad980 10 cache put: name=default.rgw.meta+accounts+account.RGW99999999999999999 info.flags=0x17 2024-05-30T16:35:00.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.468+0000 7f95297ad980 10 adding default.rgw.meta+accounts+account.RGW99999999999999999 to cache LRU end 2024-05-30T16:35:00.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.468+0000 7f95297ad980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:35:00.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.468+0000 7f95297ad980 10 cache get: name=default.rgw.meta+users.keys+LQQKSDSIJPOLWLHQAPVT : hit (negative entry) 2024-05-30T16:35:00.468 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.468+0000 7f95297ad980 10 cache get: name=default.rgw.meta+users.keys+LQQKSDSIJPOLWLHQAPVT : hit (negative entry) 2024-05-30T16:35:00.469 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.468+0000 7f95297ad980 10 cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x17 2024-05-30T16:35:00.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.468+0000 7f95297ad980 10 moving default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2024-05-30T16:35:00.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.468+0000 7f95297ad980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:root2.client.0, ofs0, ns] 2024-05-30T16:35:00.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.472+0000 7f940affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333602 cookie 94444932055648 notifier 7890 bl.length()=681 2024-05-30T16:35:00.470 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.472+0000 7f940affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x17 2024-05-30T16:35:00.471 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.472+0000 7f940affd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2024-05-30T16:35:00.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.476+0000 7f95297ad980 10 cache put: name=default.rgw.meta+users.email+root2+test@test.test info.flags=0x7 2024-05-30T16:35:00.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.476+0000 7f95297ad980 10 moving default.rgw.meta+users.email+root2+test@test.test to cache LRU end 2024-05-30T16:35:00.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.476+0000 7f95297ad980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.email:root2+test@test.test, ofs0, ns] 2024-05-30T16:35:00.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.476+0000 7f940affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333630 cookie 94444932062864 notifier 7890 bl.length()=189 2024-05-30T16:35:00.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.476+0000 7f940affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+root2+test@test.test info.flags=0x7 2024-05-30T16:35:00.475 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.476+0000 7f940affd640 10 rgw watcher librados: moving default.rgw.meta+users.email+root2+test@test.test to cache LRU end 2024-05-30T16:35:00.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.480+0000 7f95297ad980 10 cache put: name=default.rgw.meta+users.keys+LQQKSDSIJPOLWLHQAPVT info.flags=0x7 2024-05-30T16:35:00.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.480+0000 7f95297ad980 10 moving default.rgw.meta+users.keys+LQQKSDSIJPOLWLHQAPVT to cache LRU end 2024-05-30T16:35:00.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.480+0000 7f95297ad980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.keys:LQQKSDSIJPOLWLHQAPVT, ofs0, ns] 2024-05-30T16:35:00.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.480+0000 7f940affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333603 cookie 94444932055648 notifier 7890 bl.length()=188 2024-05-30T16:35:00.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.480+0000 7f940affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+LQQKSDSIJPOLWLHQAPVT info.flags=0x7 2024-05-30T16:35:00.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.480+0000 7f940affd640 10 rgw watcher librados: moving default.rgw.meta+users.keys+LQQKSDSIJPOLWLHQAPVT to cache LRU end 2024-05-30T16:35:00.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.484+0000 7f95297ad980 20 linked user root2.client.0 to account RGW99999999999999999 2024-05-30T16:35:00.485 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-30T16:35:00.485 INFO:teuthology.orchestra.run.smithi006.stdout: "user_id": "root2.client.0", 2024-05-30T16:35:00.486 INFO:teuthology.orchestra.run.smithi006.stdout: "display_name": "Mr.RGW99999999999999999", 2024-05-30T16:35:00.486 INFO:teuthology.orchestra.run.smithi006.stdout: "email": "root2+test@test.test", 2024-05-30T16:35:00.486 INFO:teuthology.orchestra.run.smithi006.stdout: "suspended": 0, 2024-05-30T16:35:00.486 INFO:teuthology.orchestra.run.smithi006.stdout: "max_buckets": 1000, 2024-05-30T16:35:00.486 INFO:teuthology.orchestra.run.smithi006.stdout: "subusers": [], 2024-05-30T16:35:00.486 INFO:teuthology.orchestra.run.smithi006.stdout: "keys": [ 2024-05-30T16:35:00.486 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T16:35:00.486 INFO:teuthology.orchestra.run.smithi006.stdout: "user": "root2.client.0", 2024-05-30T16:35:00.486 INFO:teuthology.orchestra.run.smithi006.stdout: "access_key": "LQQKSDSIJPOLWLHQAPVT", 2024-05-30T16:35:00.486 INFO:teuthology.orchestra.run.smithi006.stdout: "secret_key": "x61bULegzZXPtIXjNfG76hPZRLraGkIVVTgGauhkR8b1/qeoQbhDbw==", 2024-05-30T16:35:00.486 INFO:teuthology.orchestra.run.smithi006.stdout: "active": true, 2024-05-30T16:35:00.486 INFO:teuthology.orchestra.run.smithi006.stdout: "create_date": "2024-05-30T16:35:00.470170Z" 2024-05-30T16:35:00.486 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T16:35:00.487 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2024-05-30T16:35:00.487 INFO:teuthology.orchestra.run.smithi006.stdout: "swift_keys": [], 2024-05-30T16:35:00.487 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": [], 2024-05-30T16:35:00.487 INFO:teuthology.orchestra.run.smithi006.stdout: "op_mask": "read, write, delete", 2024-05-30T16:35:00.487 INFO:teuthology.orchestra.run.smithi006.stdout: "default_placement": "", 2024-05-30T16:35:00.487 INFO:teuthology.orchestra.run.smithi006.stdout: "default_storage_class": "", 2024-05-30T16:35:00.487 INFO:teuthology.orchestra.run.smithi006.stdout: "placement_tags": [], 2024-05-30T16:35:00.487 INFO:teuthology.orchestra.run.smithi006.stdout: "bucket_quota": { 2024-05-30T16:35:00.487 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": false, 2024-05-30T16:35:00.487 INFO:teuthology.orchestra.run.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:35:00.487 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size": -1, 2024-05-30T16:35:00.487 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:35:00.487 INFO:teuthology.orchestra.run.smithi006.stdout: "max_objects": -1 2024-05-30T16:35:00.488 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T16:35:00.488 INFO:teuthology.orchestra.run.smithi006.stdout: "user_quota": { 2024-05-30T16:35:00.488 INFO:teuthology.orchestra.run.smithi006.stdout: "enabled": false, 2024-05-30T16:35:00.488 INFO:teuthology.orchestra.run.smithi006.stdout: "check_on_raw": false, 2024-05-30T16:35:00.488 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size": -1, 2024-05-30T16:35:00.488 INFO:teuthology.orchestra.run.smithi006.stdout: "max_size_kb": 0, 2024-05-30T16:35:00.488 INFO:teuthology.orchestra.run.smithi006.stdout: "max_objects": -1 2024-05-30T16:35:00.488 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T16:35:00.488 INFO:teuthology.orchestra.run.smithi006.stdout: "temp_url_keys": [], 2024-05-30T16:35:00.488 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "root", 2024-05-30T16:35:00.488 INFO:teuthology.orchestra.run.smithi006.stdout: "mfa_ids": [], 2024-05-30T16:35:00.488 INFO:teuthology.orchestra.run.smithi006.stdout: "account_id": "RGW99999999999999999", 2024-05-30T16:35:00.488 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2024-05-30T16:35:00.488 INFO:teuthology.orchestra.run.smithi006.stdout: "create_date": "2024-05-30T16:35:00.470152Z", 2024-05-30T16:35:00.488 INFO:teuthology.orchestra.run.smithi006.stdout: "tags": [], 2024-05-30T16:35:00.489 INFO:teuthology.orchestra.run.smithi006.stdout: "group_ids": [] 2024-05-30T16:35:00.489 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-30T16:35:00.489 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:35:00.489 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.488+0000 7f93ddfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:35:00.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.492+0000 7f95297ad980 20 remove_watcher() i=0 2024-05-30T16:35:00.491 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.492+0000 7f95297ad980 2 removed watcher, disabling cache 2024-05-30T16:35:00.493 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.492+0000 7f95297ad980 20 remove_watcher() i=1 2024-05-30T16:35:00.495 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.496+0000 7f95297ad980 20 remove_watcher() i=2 2024-05-30T16:35:00.497 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.496+0000 7f95297ad980 20 remove_watcher() i=3 2024-05-30T16:35:00.499 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.500+0000 7f95297ad980 20 remove_watcher() i=4 2024-05-30T16:35:00.500 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.500+0000 7f95297ad980 20 remove_watcher() i=5 2024-05-30T16:35:00.502 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.500+0000 7f95297ad980 20 remove_watcher() i=6 2024-05-30T16:35:00.503 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.504+0000 7f95297ad980 20 remove_watcher() i=7 2024-05-30T16:35:00.518 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid root2.client.0 --totp-serial 8795409066 --totp-seed 67WYE43DMEQ4JJCRSZ7YUL7SUKHS5XLKLGTYYMOHBPYLG4SWNNVRLON6ZIZHBWBI --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-05-30T16:35:00.558 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T16:35:00.559 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T16:35:00.601 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.600+0000 7f8819178980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.604+0000 7f8819178980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:35:00.602 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.604+0000 7f8819178980 20 realm 2024-05-30T16:35:00.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.604+0000 7f8819178980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.604+0000 7f8819178980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:35:00.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.604+0000 7f8819178980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T16:35:00.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.604+0000 7f8819178980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.603 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.604+0000 7f8819178980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:35:00.604 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.604+0000 7f8819178980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.604+0000 7f8819178980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:35:00.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.604+0000 7f8819178980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.604+0000 7f8819178980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T16:35:00.605 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.604+0000 7f8819178980 20 searching for the correct realm 2024-05-30T16:35:00.612 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.612+0000 7f8816488640 0 -- 172.21.15.6:0/4016057686 >> [v2:172.21.15.204:6806/986196749,v1:172.21.15.204:6808/986196749] conn(0x7f88080f6060 msgr2=0x7f88080ad9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:35:00.618 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.620+0000 7f881748a640 0 -- 172.21.15.6:0/4016057686 >> [v2:172.21.15.204:6800/3687190393,v1:172.21.15.204:6801/3687190393] conn(0x7f880c017070 msgr2=0x7f880c05e0c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T16:35:00.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.624+0000 7f8819178980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T16:35:00.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.624+0000 7f8819178980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T16:35:00.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.624+0000 7f8819178980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T16:35:00.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.624+0000 7f8819178980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T16:35:00.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.624+0000 7f8819178980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T16:35:00.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.624+0000 7f8819178980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T16:35:00.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.624+0000 7f8819178980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.624+0000 7f8819178980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:35:00.625 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.624+0000 7f8819178980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.624+0000 7f8819178980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T16:35:00.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.624+0000 7f8819178980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.628+0000 7f8819178980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T16:35:00.626 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.628+0000 7f8819178980 20 zone default found 2024-05-30T16:35:00.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.628+0000 7f8819178980 4 Realm: () 2024-05-30T16:35:00.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.628+0000 7f8819178980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T16:35:00.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.628+0000 7f8819178980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T16:35:00.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.628+0000 7f8819178980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T16:35:00.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.628+0000 7f8819178980 20 zonegroup default 2024-05-30T16:35:00.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.628+0000 7f8819178980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.628+0000 7f8819178980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:35:00.627 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.628+0000 7f8819178980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.628+0000 7f8819178980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:35:00.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.628+0000 7f8819178980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.628+0000 7f8819178980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T16:35:00.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.628+0000 7f8819178980 20 started sync module instance, tier type = 2024-05-30T16:35:00.628 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.628+0000 7f8819178980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T16:35:00.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.644+0000 7f8819178980 20 add_watcher() i=0 2024-05-30T16:35:00.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.644+0000 7f8819178980 20 add_watcher() i=1 2024-05-30T16:35:00.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.644+0000 7f8819178980 20 add_watcher() i=2 2024-05-30T16:35:00.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.644+0000 7f8819178980 20 add_watcher() i=3 2024-05-30T16:35:00.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.644+0000 7f8819178980 20 add_watcher() i=4 2024-05-30T16:35:00.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.644+0000 7f8819178980 20 add_watcher() i=5 2024-05-30T16:35:00.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.644+0000 7f8819178980 20 add_watcher() i=6 2024-05-30T16:35:00.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.644+0000 7f8819178980 20 add_watcher() i=7 2024-05-30T16:35:00.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.644+0000 7f8819178980 2 all 8 watchers are set, enabling cache 2024-05-30T16:35:00.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.648+0000 7f86cdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:35:00.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.648+0000 7f8819178980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T16:35:00.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.648+0000 7f8819178980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T16:35:00.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.648+0000 7f8819178980 5 note: GC not initialized 2024-05-30T16:35:00.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.648+0000 7f86ccfe1640 20 reqs_thread_entry: start 2024-05-30T16:35:00.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.648+0000 7f8819178980 20 init_complete bucket index max shards: 11 2024-05-30T16:35:00.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.648+0000 7f8819178980 20 Filter name: none 2024-05-30T16:35:00.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.648+0000 7f86c5ffb640 20 reqs_thread_entry: start 2024-05-30T16:35:00.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.648+0000 7f8819178980 10 cache get: name=default.rgw.meta+users.uid+root2.client.0 : miss 2024-05-30T16:35:00.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.648+0000 7f8819178980 20 rados->read ofs=0 len=0 2024-05-30T16:35:00.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.648+0000 7f8819178980 20 rados_obj.operate() r=0 bl.length=494 2024-05-30T16:35:00.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.648+0000 7f8819178980 10 cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x13 2024-05-30T16:35:00.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.648+0000 7f8819178980 10 adding default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2024-05-30T16:35:00.649 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.648+0000 7f8819178980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T16:35:00.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.652+0000 7f8819178980 10 cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x17 2024-05-30T16:35:00.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.652+0000 7f8819178980 10 moving default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2024-05-30T16:35:00.653 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.652+0000 7f8819178980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:root2.client.0, ofs0, ns] 2024-05-30T16:35:00.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.656+0000 7f86fa7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333604 cookie 94711019197728 notifier 7908 bl.length()=695 2024-05-30T16:35:00.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.656+0000 7f86fa7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x17 2024-05-30T16:35:00.654 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.656+0000 7f86fa7fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2024-05-30T16:35:00.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.656+0000 7f86cdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T16:35:00.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.660+0000 7f8819178980 20 remove_watcher() i=0 2024-05-30T16:35:00.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.660+0000 7f8819178980 2 removed watcher, disabling cache 2024-05-30T16:35:00.664 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.664+0000 7f8819178980 20 remove_watcher() i=1 2024-05-30T16:35:00.666 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.668+0000 7f8819178980 20 remove_watcher() i=2 2024-05-30T16:35:00.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.668+0000 7f8819178980 20 remove_watcher() i=3 2024-05-30T16:35:00.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.672+0000 7f8819178980 20 remove_watcher() i=4 2024-05-30T16:35:00.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.672+0000 7f8819178980 20 remove_watcher() i=5 2024-05-30T16:35:00.673 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.672+0000 7f8819178980 20 remove_watcher() i=6 2024-05-30T16:35:00.674 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T16:35:00.676+0000 7f8819178980 20 remove_watcher() i=7 2024-05-30T16:35:00.688 INFO:tasks.s3tests:Configuring s3-tests... 2024-05-30T16:35:00.689 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T16:35:00.689 DEBUG:teuthology.orchestra.run.smithi006:> dd of=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf 2024-05-30T16:35:00.735 INFO:tasks.s3tests:Configuring boto... 2024-05-30T16:35:00.735 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-05-30T16:35:00.735 DEBUG:teuthology.orchestra.run.smithi006:> dd of=/home/ubuntu/cephtest/boto-client.0.cfg 2024-05-30T16:35:00.783 DEBUG:teuthology.orchestra.run.smithi006:s3 tests against rgw> source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3' 2024-05-30T16:35:00.949 INFO:teuthology.orchestra.run.smithi006.stdout:GLOB sdist-make: /home/ubuntu/cephtest/s3-tests-client.0/setup.py 2024-05-30T16:35:01.150 INFO:teuthology.orchestra.run.smithi006.stdout:py create: /home/ubuntu/cephtest/s3-tests-client.0/.tox/py 2024-05-30T16:35:01.441 INFO:teuthology.orchestra.run.smithi006.stdout:py installdeps: -rrequirements.txt 2024-05-30T16:36:53.765 INFO:teuthology.orchestra.run.smithi006.stdout:py inst: /home/ubuntu/cephtest/s3-tests-client.0/.tox/.tmp/package/1/s3tests-0.0.1.zip 2024-05-30T16:36:55.833 INFO:teuthology.orchestra.run.smithi006.stdout:py installed: boto==2.49.0,boto3==1.24.96,botocore==1.27.96,cachetools==5.3.3,certifi==2024.2.2,chardet==5.2.0,charset-normalizer==3.3.2,colorama==0.4.6,distlib==0.3.8,exceptiongroup==1.2.1,filelock==3.14.0,gevent==24.2.1,greenlet==3.0.3,httplib2==0.22.0,idna==3.7,iniconfig==2.0.0,isodate==0.6.1,jmespath==1.0.1,lxml==5.2.2,munch==4.0.0,packaging==24.0,platformdirs==4.2.2,pluggy==1.5.0,pyparsing==3.1.2,pyproject-api==1.6.1,pytest==8.2.1,python-dateutil==2.9.0.post0,pytz==2024.1,PyYAML==6.0.1,requests==2.32.3,s3tests @ file:///home/ubuntu/cephtest/s3-tests-client.0/.tox/.tmp/package/1/s3tests-0.0.1.zip#sha256=a3965d2e9684ae95ed9c7ef38b10265cadf42eafeaa43f1e780cb3282693aeef,s3transfer==0.6.2,six==1.16.0,tomli==2.0.1,tox==4.15.0,urllib3==1.26.18,virtualenv==20.26.2,zope.event==5.0,zope.interface==6.4.post2 2024-05-30T16:36:55.833 INFO:teuthology.orchestra.run.smithi006.stdout:py run-test-pre: PYTHONHASHSEED='2029042077' 2024-05-30T16:36:55.833 INFO:teuthology.orchestra.run.smithi006.stdout:py run-test: commands[0] | pytest -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3' 2024-05-30T16:36:56.027 INFO:teuthology.orchestra.run.smithi006.stdout:============================= test session starts ============================== 2024-05-30T16:36:56.027 INFO:teuthology.orchestra.run.smithi006.stdout:platform linux -- Python 3.10.6, pytest-8.2.1, pluggy-1.5.0 -- /home/ubuntu/cephtest/s3-tests-client.0/.tox/py/bin/python 2024-05-30T16:36:56.027 INFO:teuthology.orchestra.run.smithi006.stdout:cachedir: .tox/py/.pytest_cache 2024-05-30T16:36:56.027 INFO:teuthology.orchestra.run.smithi006.stdout:rootdir: /home/ubuntu/cephtest/s3-tests-client.0 2024-05-30T16:36:56.027 INFO:teuthology.orchestra.run.smithi006.stdout:configfile: pytest.ini 2024-05-30T16:36:57.864 INFO:teuthology.orchestra.run.smithi006.stdout:collecting ... collected 859 items / 127 deselected / 732 selected 2024-05-30T16:36:57.864 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T16:36:58.843 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_create_bad_contentlength_none PASSED [ 0%] 2024-05-30T16:36:59.608 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_empty PASSED [ 0%] 2024-05-30T16:37:00.462 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_create_date_and_amz_date PASSED [ 0%] 2024-05-30T16:37:01.318 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_create_amz_date_and_no_date PASSED [ 0%] 2024-05-30T16:37:02.045 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_none PASSED [ 0%] 2024-05-30T16:37:02.742 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_bucket_create_contentlength_none PASSED [ 0%] 2024-05-30T16:37:03.751 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_acl_create_contentlength_none PASSED [ 0%] 2024-05-30T16:37:04.490 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_contentlength_none PASSED [ 1%] 2024-05-30T16:37:04.660 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_authorization_empty PASSED [ 1%] 2024-05-30T16:37:04.826 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_authorization_none PASSED [ 1%] 2024-05-30T16:37:05.554 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_create_bad_contentlength_mismatch_below_aws2 PASSED [ 1%] 2024-05-30T16:37:06.298 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_incorrect_aws2 PASSED [ 1%] 2024-05-30T16:37:07.063 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_invalid_aws2 PASSED [ 1%] 2024-05-30T16:37:07.810 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_none_aws2 PASSED [ 1%] 2024-05-30T16:37:07.987 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_authorization_invalid_aws2 PASSED [ 2%] 2024-05-30T16:37:08.164 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_none_aws2 PASSED [ 2%] 2024-05-30T16:37:08.278 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_create_bad_md5_invalid_garbage_aws4 SKIPPED [ 2%] 2024-05-30T16:37:08.393 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_create_bad_contentlength_mismatch_below_aws4 SKIPPED [ 2%] 2024-05-30T16:37:08.509 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_incorrect_aws4 SKIPPED [ 2%] 2024-05-30T16:37:08.624 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_invalid_aws4 SKIPPED [ 2%] 2024-05-30T16:37:08.740 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_create_bad_ua_empty_aws4 SKIPPED [ 2%] 2024-05-30T16:37:08.855 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_create_bad_ua_none_aws4 SKIPPED [ 3%] 2024-05-30T16:37:08.971 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_invalid_aws4 SKIPPED [ 3%] 2024-05-30T16:37:09.101 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_invalid_aws4 SKIPPED [ 3%] 2024-05-30T16:37:09.242 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_empty_aws4 SKIPPED [ 3%] 2024-05-30T16:37:09.360 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_empty_aws4 SKIPPED [ 3%] 2024-05-30T16:37:09.478 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_none_aws4 SKIPPED [ 3%] 2024-05-30T16:37:09.593 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_none_aws4 SKIPPED [ 3%] 2024-05-30T16:37:09.710 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_before_today_aws4 SKIPPED [ 3%] 2024-05-30T16:37:09.826 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_before_today_aws4 SKIPPED [ 4%] 2024-05-30T16:37:09.945 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_after_today_aws4 SKIPPED [ 4%] 2024-05-30T16:37:10.076 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_after_today_aws4 SKIPPED [ 4%] 2024-05-30T16:37:10.193 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_before_epoch_aws4 SKIPPED [ 4%] 2024-05-30T16:37:10.312 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_before_epoch_aws4 SKIPPED [ 4%] 2024-05-30T16:37:10.426 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_after_end_aws4 SKIPPED [ 4%] 2024-05-30T16:37:10.541 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_after_end_aws4 SKIPPED [ 4%] 2024-05-30T16:37:10.654 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_create_missing_signed_custom_header_aws4 SKIPPED [ 5%] 2024-05-30T16:37:10.768 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_object_create_missing_signed_header_aws4 SKIPPED [ 5%] 2024-05-30T16:37:10.884 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_authorization_invalid_aws4 SKIPPED [ 5%] 2024-05-30T16:37:10.999 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_ua_empty_aws4 SKIPPED [ 5%] 2024-05-30T16:37:11.115 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_ua_none_aws4 SKIPPED [ 5%] 2024-05-30T16:37:11.232 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_invalid_aws4 SKIPPED [ 5%] 2024-05-30T16:37:11.347 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_invalid_aws4 SKIPPED [ 5%] 2024-05-30T16:37:11.461 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_empty_aws4 SKIPPED [ 6%] 2024-05-30T16:37:11.579 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_empty_aws4 SKIPPED [ 6%] 2024-05-30T16:37:11.694 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_none_aws4 SKIPPED [ 6%] 2024-05-30T16:37:11.810 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_none_aws4 SKIPPED [ 6%] 2024-05-30T16:37:11.925 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_before_today_aws4 SKIPPED [ 6%] 2024-05-30T16:37:12.042 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_before_today_aws4 SKIPPED [ 6%] 2024-05-30T16:37:12.158 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_after_today_aws4 SKIPPED [ 6%] 2024-05-30T16:37:12.276 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_after_today_aws4 SKIPPED [ 6%] 2024-05-30T16:37:12.391 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_before_epoch_aws4 SKIPPED [ 7%] 2024-05-30T16:37:12.505 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_before_epoch_aws4 SKIPPED [ 7%] 2024-05-30T16:37:13.801 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3.py::test_versioning_obj_read_not_exist_null PASSED [ 7%] 2024-05-30T16:37:19.883 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3.py::test_object_storage_class PASSED [ 7%] 2024-05-30T16:37:26.902 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3.py::test_object_storage_class_multipart PASSED [ 7%] 2024-05-30T16:37:36.253 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3.py::test_object_modify_storage_class PASSED [ 7%] 2024-05-30T16:37:49.395 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3.py::test_object_modify_storage_class_multipart PASSED [ 7%] 2024-05-30T16:37:53.189 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3.py::test_object_storage_class_copy PASSED [ 8%] 2024-05-30T16:37:58.065 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3.py::test_object_storage_class_copy_multipart PASSED [ 8%] 2024-05-30T16:37:59.730 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3.py::test_atomic_dual_conditional_write_1mb PASSED [ 8%] 2024-05-30T16:38:00.411 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3.py::test_atomic_write_bucket_gone PASSED [ 8%] 2024-05-30T16:38:01.950 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3.py::test_encryption_sse_c_multipart_invalid_chunks_1 PASSED [ 8%] 2024-05-30T16:38:03.459 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3.py::test_encryption_sse_c_multipart_invalid_chunks_2 PASSED [ 8%] 2024-05-30T16:38:05.599 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3.py::test_bucket_policy_set_condition_operator_end_with_IfExists PASSED [ 8%] 2024-05-30T16:38:07.081 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3.py::test_bucket_policy_put_obj_request_obj_tag PASSED [ 9%] 2024-05-30T16:38:08.130 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py::test_website_nonexistant_bucket_rgw SKIPPED [ 9%] 2024-05-30T16:38:08.135 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_public_index SKIPPED [ 9%] 2024-05-30T16:38:08.138 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_public_index SKIPPED [ 9%] 2024-05-30T16:38:08.140 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_empty SKIPPED [ 9%] 2024-05-30T16:38:08.142 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_empty SKIPPED [ 9%] 2024-05-30T16:38:08.145 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_private_index SKIPPED [ 9%] 2024-05-30T16:38:08.148 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_private_index SKIPPED [ 9%] 2024-05-30T16:38:08.151 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_empty_missingerrordoc SKIPPED [ 10%] 2024-05-30T16:38:08.154 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_empty_missingerrordoc SKIPPED [ 10%] 2024-05-30T16:38:08.157 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_private_index_missingerrordoc SKIPPED [ 10%] 2024-05-30T16:38:08.159 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_private_index_missingerrordoc SKIPPED [ 10%] 2024-05-30T16:38:08.162 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_empty_blockederrordoc SKIPPED [ 10%] 2024-05-30T16:38:08.164 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_pubilc_errordoc SKIPPED [ 10%] 2024-05-30T16:38:08.167 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_empty_blockederrordoc SKIPPED [ 10%] 2024-05-30T16:38:08.169 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_private_index_blockederrordoc SKIPPED [ 11%] 2024-05-30T16:38:08.172 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_private_index_blockederrordoc SKIPPED [ 11%] 2024-05-30T16:38:08.174 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_empty_gooderrordoc SKIPPED [ 11%] 2024-05-30T16:38:08.177 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_empty_gooderrordoc SKIPPED [ 11%] 2024-05-30T16:38:08.179 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_private_index_gooderrordoc SKIPPED [ 11%] 2024-05-30T16:38:08.183 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_private_index_gooderrordoc SKIPPED [ 11%] 2024-05-30T16:38:08.221 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py::test_website_bucket_private_redirectall_base SKIPPED [ 11%] 2024-05-30T16:38:08.222 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py::test_website_bucket_private_redirectall_path SKIPPED [ 12%] 2024-05-30T16:38:08.224 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py::test_website_bucket_private_redirectall_path_upgrade SKIPPED [ 12%] 2024-05-30T16:38:08.225 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_nonwebsite SKIPPED [ 12%] 2024-05-30T16:38:08.227 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_public_relative SKIPPED [ 12%] 2024-05-30T16:38:08.228 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_public_abs SKIPPED [ 12%] 2024-05-30T16:38:08.230 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_private_relative SKIPPED [ 12%] 2024-05-30T16:38:08.231 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_private_abs SKIPPED [ 12%] 2024-05-30T16:38:08.332 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py::test_routing_generator XFAIL [ 12%] 2024-05-30T16:38:08.340 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_utils.py::test_generate PASSED [ 13%] 2024-05-30T16:38:10.578 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_invalid_short PASSED [ 13%] 2024-05-30T16:38:16.170 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_bad PASSED [ 13%] 2024-05-30T16:38:17.494 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_empty PASSED [ 13%] 2024-05-30T16:38:18.970 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_none PASSED [ 13%] 2024-05-30T16:38:20.502 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_expect_mismatch PASSED [ 13%] 2024-05-30T16:38:21.986 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_expect_empty PASSED [ 13%] 2024-05-30T16:38:23.466 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_expect_none PASSED [ 14%] 2024-05-30T16:38:24.666 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_contentlength_negative PASSED [ 14%] 2024-05-30T16:38:26.198 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_contenttype_invalid PASSED [ 14%] 2024-05-30T16:38:27.562 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_contenttype_empty PASSED [ 14%] 2024-05-30T16:38:28.962 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_contenttype_none PASSED [ 14%] 2024-05-30T16:38:30.198 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_put_bad_canned_acl PASSED [ 14%] 2024-05-30T16:38:31.402 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_expect_mismatch PASSED [ 14%] 2024-05-30T16:38:32.570 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_expect_empty PASSED [ 15%] 2024-05-30T16:38:33.130 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_contentlength_negative PASSED [ 15%] 2024-05-30T16:38:34.406 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_invalid_garbage_aws2 PASSED [ 15%] 2024-05-30T16:38:35.874 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_ua_empty_aws2 PASSED [ 15%] 2024-05-30T16:38:37.310 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_ua_none_aws2 PASSED [ 15%] 2024-05-30T16:38:38.522 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_invalid_aws2 PASSED [ 15%] 2024-05-30T16:38:39.778 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_empty_aws2 PASSED [ 15%] 2024-05-30T16:38:41.058 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_before_today_aws2 PASSED [ 15%] 2024-05-30T16:38:42.274 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_before_epoch_aws2 PASSED [ 16%] 2024-05-30T16:38:43.518 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_after_end_aws2 PASSED [ 16%] 2024-05-30T16:38:44.662 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_ua_empty_aws2 PASSED [ 16%] 2024-05-30T16:38:45.834 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_ua_none_aws2 PASSED [ 16%] 2024-05-30T16:38:46.426 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_invalid_aws2 PASSED [ 16%] 2024-05-30T16:38:46.999 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_empty_aws2 PASSED [ 16%] 2024-05-30T16:38:47.570 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_before_today_aws2 PASSED [ 16%] 2024-05-30T16:38:48.198 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_after_today_aws2 PASSED [ 17%] 2024-05-30T16:38:48.778 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_before_epoch_aws2 PASSED [ 17%] 2024-05-30T16:38:49.882 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_put_user_policy PASSED [ 17%] 2024-05-30T16:38:50.530 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_put_user_policy_invalid_user PASSED [ 17%] 2024-05-30T16:38:51.230 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_put_user_policy_parameter_limit PASSED [ 17%] 2024-05-30T16:38:52.206 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_put_existing_user_policy PASSED [ 17%] 2024-05-30T16:38:53.126 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_list_user_policy PASSED [ 17%] 2024-05-30T16:38:53.726 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_list_user_policy_invalid_user PASSED [ 18%] 2024-05-30T16:38:54.658 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_get_user_policy PASSED [ 18%] 2024-05-30T16:38:55.654 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_get_user_policy_invalid_user PASSED [ 18%] 2024-05-30T16:38:56.910 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_get_user_policy_from_multiple_policies PASSED [ 18%] 2024-05-30T16:38:57.754 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_delete_user_policy PASSED [ 18%] 2024-05-30T16:38:58.658 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_delete_user_policy_invalid_user PASSED [ 18%] 2024-05-30T16:38:59.554 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_delete_user_policy_invalid_policy_name PASSED [ 18%] 2024-05-30T16:39:01.110 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_delete_user_policy_from_multiple_policies PASSED [ 18%] 2024-05-30T16:39:02.782 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_allow_bucket_actions_in_user_policy PASSED [ 19%] 2024-05-30T16:39:04.230 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_deny_bucket_actions_in_user_policy PASSED [ 19%] 2024-05-30T16:39:05.958 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_allow_object_actions_in_user_policy PASSED [ 19%] 2024-05-30T16:39:07.950 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_deny_object_actions_in_user_policy PASSED [ 19%] 2024-05-30T16:39:10.414 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_allow_multipart_actions_in_user_policy PASSED [ 19%] 2024-05-30T16:39:13.426 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_deny_multipart_actions_in_user_policy PASSED [ 19%] 2024-05-30T16:39:15.438 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_allow_tagging_actions_in_user_policy PASSED [ 19%] 2024-05-30T16:39:17.402 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_deny_tagging_actions_in_user_policy PASSED [ 20%] 2024-05-30T16:39:18.886 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_verify_conflicting_user_policy_statements PASSED [ 20%] 2024-05-30T16:39:20.682 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_verify_conflicting_user_policies PASSED [ 20%] 2024-05-30T16:39:21.742 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_verify_allow_iam_actions PASSED [ 20%] 2024-05-30T16:39:24.375 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_create PASSED [ 20%] 2024-05-30T16:39:26.011 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_case_insensitive_name PASSED [ 20%] 2024-05-30T16:39:27.970 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_delete PASSED [ 20%] 2024-05-30T16:39:31.894 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_list PASSED [ 21%] 2024-05-30T16:39:37.098 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_list_path_prefix PASSED [ 21%] 2024-05-30T16:39:40.010 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_update_name PASSED [ 21%] 2024-05-30T16:39:42.707 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_update_path PASSED [ 21%] 2024-05-30T16:39:44.778 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_access_key_create PASSED [ 21%] 2024-05-30T16:39:46.203 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_current_user_access_key_create PASSED [ 21%] 2024-05-30T16:39:48.634 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_access_key_update PASSED [ 21%] 2024-05-30T16:39:50.322 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_current_user_access_key_update PASSED [ 21%] 2024-05-30T16:39:52.651 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_access_key_delete PASSED [ 22%] 2024-05-30T16:39:54.238 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_current_user_access_key_delete PASSED [ 22%] 2024-05-30T16:39:57.283 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_access_key_list PASSED [ 22%] 2024-05-30T16:39:59.966 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_bucket_policy_allow PASSED [ 22%] 2024-05-30T16:40:03.226 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_policy PASSED [ 22%] 2024-05-30T16:40:06.628 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_policy_managed PASSED [ 22%] 2024-05-30T16:40:08.898 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_policy_allow PASSED [ 22%] 2024-05-30T16:40:10.987 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_group_create PASSED [ 23%] 2024-05-30T16:40:12.694 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_group_case_insensitive_name PASSED [ 23%] 2024-05-30T16:40:16.666 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_group_list PASSED [ 23%] 2024-05-30T16:40:19.579 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_group_update PASSED [ 23%] 2024-05-30T16:40:22.879 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_inline_group_policy PASSED [ 23%] 2024-05-30T16:40:25.970 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_managed_group_policy PASSED [ 23%] 2024-05-30T16:40:29.030 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_inline_group_policy_allow PASSED [ 23%] 2024-05-30T16:40:32.019 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_managed_group_policy_allow PASSED [ 24%] 2024-05-30T16:40:34.611 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_create PASSED [ 24%] 2024-05-30T16:40:36.422 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_case_insensitive_name PASSED [ 24%] 2024-05-30T16:40:38.394 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_delete PASSED [ 24%] 2024-05-30T16:40:42.083 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_list PASSED [ 24%] 2024-05-30T16:40:46.822 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_list_path_prefix PASSED [ 24%] 2024-05-30T16:40:48.803 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_update PASSED [ 24%] 2024-05-30T16:40:51.938 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_policy PASSED [ 25%] 2024-05-30T16:40:54.922 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_policy_managed PASSED [ 25%] 2024-05-30T16:40:57.879 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_policy_allow PASSED [ 25%] 2024-05-30T16:41:01.007 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_same_account_role_policy_allow PASSED [ 25%] 2024-05-30T16:41:04.378 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_role_policy_allow PASSED [ 25%] 2024-05-30T16:41:07.530 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_policy_allow_create_bucket PASSED [ 25%] 2024-05-30T16:41:10.547 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_policy_allow_get_role PASSED [ 25%] 2024-05-30T16:41:12.578 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_account_oidc_provider PASSED [ 25%] 2024-05-30T16:41:15.723 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_bucket_user_policy_allow_user_arn PASSED [ 26%] 2024-05-30T16:41:18.911 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_user_bucket_policy_allow_user_arn PASSED [ 26%] 2024-05-30T16:41:22.083 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_user_bucket_policy_allow_account_arn PASSED [ 26%] 2024-05-30T16:41:25.215 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_bucket_user_policy_allow_account_arn PASSED [ 26%] 2024-05-30T16:41:28.470 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_user_bucket_policy_allow_account_id PASSED [ 26%] 2024-05-30T16:41:31.607 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_bucket_user_policy_allow_account_id PASSED [ 26%] 2024-05-30T16:41:34.741 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_bucket_acl_user_policy_grant_user_id PASSED [ 26%] 2024-05-30T16:41:37.899 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_user_policy_bucket_acl_grant_user_id PASSED [ 27%] 2024-05-30T16:41:41.023 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_bucket_acl_user_policy_grant_canonical_id PASSED [ 27%] 2024-05-30T16:41:44.219 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_user_policy_bucket_acl_grant_canonical_id PASSED [ 27%] 2024-05-30T16:41:47.395 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_bucket_acl_user_policy_grant_account_email PASSED [ 27%] 2024-05-30T16:41:50.527 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_user_policy_bucket_acl_grant_account_email PASSED [ 27%] 2024-05-30T16:41:52.659 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_root_bucket_policy_allow_account_arn PASSED [ 27%] 2024-05-30T16:41:54.783 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_root_bucket_policy_allow_account_id PASSED [ 27%] 2024-05-30T16:41:56.823 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_root_bucket_acl_grant_canonical_id PASSED [ 28%] 2024-05-30T16:41:58.839 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_root_bucket_acl_grant_account_email PASSED [ 28%] 2024-05-30T16:42:00.386 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_empty PASSED [ 28%] 2024-05-30T16:42:02.554 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_distinct PASSED [ 28%] 2024-05-30T16:42:04.350 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_many PASSED [ 28%] 2024-05-30T16:42:06.154 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_many PASSED [ 28%] 2024-05-30T16:42:08.114 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_basic_key_count PASSED [ 28%] 2024-05-30T16:42:09.982 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_basic PASSED [ 28%] 2024-05-30T16:42:11.810 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_basic PASSED [ 29%] 2024-05-30T16:42:13.646 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_encoding_basic PASSED [ 29%] 2024-05-30T16:42:15.462 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_encoding_basic PASSED [ 29%] 2024-05-30T16:42:18.158 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_prefix PASSED [ 29%] 2024-05-30T16:42:20.882 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_prefix PASSED [ 29%] 2024-05-30T16:42:22.394 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_prefix_ends_with_delimiter PASSED [ 29%] 2024-05-30T16:42:23.858 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_prefix_ends_with_delimiter PASSED [ 29%] 2024-05-30T16:42:25.670 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_alt PASSED [ 30%] 2024-05-30T16:42:27.522 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_alt PASSED [ 30%] 2024-05-30T16:42:30.162 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_prefix_underscore PASSED [ 30%] 2024-05-30T16:42:32.902 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_prefix_underscore PASSED [ 30%] 2024-05-30T16:42:34.726 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_percentage PASSED [ 30%] 2024-05-30T16:42:36.514 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_percentage PASSED [ 30%] 2024-05-30T16:42:38.311 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_whitespace PASSED [ 30%] 2024-05-30T16:42:40.126 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_whitespace PASSED [ 31%] 2024-05-30T16:42:41.966 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_dot PASSED [ 31%] 2024-05-30T16:42:43.794 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_dot PASSED [ 31%] 2024-05-30T16:42:45.598 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_unreadable PASSED [ 31%] 2024-05-30T16:42:47.419 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_unreadable PASSED [ 31%] 2024-05-30T16:42:49.226 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_empty PASSED [ 31%] 2024-05-30T16:42:51.042 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_empty PASSED [ 31%] 2024-05-30T16:42:52.906 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_none PASSED [ 31%] 2024-05-30T16:42:54.782 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_none PASSED [ 32%] 2024-05-30T16:42:56.490 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_fetchowner_notempty PASSED [ 32%] 2024-05-30T16:42:58.382 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_fetchowner_defaultempty PASSED [ 32%] 2024-05-30T16:43:00.090 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_fetchowner_empty PASSED [ 32%] 2024-05-30T16:43:01.902 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_not_exist PASSED [ 32%] 2024-05-30T16:43:03.738 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_not_exist PASSED [ 32%] 2024-05-30T16:45:03.815 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_not_skip_special PASSED [ 32%] 2024-05-30T16:45:05.782 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_basic PASSED [ 33%] 2024-05-30T16:45:07.566 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_basic PASSED [ 33%] 2024-05-30T16:45:09.330 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_alt PASSED [ 33%] 2024-05-30T16:45:11.026 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_alt PASSED [ 33%] 2024-05-30T16:45:12.754 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_empty PASSED [ 33%] 2024-05-30T16:45:14.478 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_empty PASSED [ 33%] 2024-05-30T16:45:16.159 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_none PASSED [ 33%] 2024-05-30T16:45:17.890 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_none PASSED [ 34%] 2024-05-30T16:45:19.626 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_not_exist PASSED [ 34%] 2024-05-30T16:45:21.366 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_not_exist PASSED [ 34%] 2024-05-30T16:45:23.086 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_unreadable PASSED [ 34%] 2024-05-30T16:45:24.834 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_unreadable PASSED [ 34%] 2024-05-30T16:45:26.646 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_basic PASSED [ 34%] 2024-05-30T16:45:28.502 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_basic PASSED [ 34%] 2024-05-30T16:45:30.354 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_alt PASSED [ 34%] 2024-05-30T16:45:32.166 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_alt PASSED [ 35%] 2024-05-30T16:45:34.070 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_prefix_not_exist PASSED [ 35%] 2024-05-30T16:45:35.874 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_prefix_not_exist PASSED [ 35%] 2024-05-30T16:45:37.682 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_delimiter_not_exist PASSED [ 35%] 2024-05-30T16:45:39.546 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_delimiter_not_exist PASSED [ 35%] 2024-05-30T16:45:41.358 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_prefix_delimiter_not_exist PASSED [ 35%] 2024-05-30T16:45:43.178 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_prefix_delimiter_not_exist PASSED [ 35%] 2024-05-30T16:45:45.218 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_maxkeys_one PASSED [ 36%] 2024-05-30T16:45:47.138 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_maxkeys_one PASSED [ 36%] 2024-05-30T16:45:48.970 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_maxkeys_zero PASSED [ 36%] 2024-05-30T16:45:50.831 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_maxkeys_zero PASSED [ 36%] 2024-05-30T16:45:52.654 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_maxkeys_none PASSED [ 36%] 2024-05-30T16:45:54.514 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_maxkeys_none PASSED [ 36%] 2024-05-30T16:45:55.202 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_account_usage PASSED [ 36%] 2024-05-30T16:45:56.650 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_head_bucket_usage PASSED [ 37%] 2024-05-30T16:46:02.018 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_unordered PASSED [ 37%] 2024-05-30T16:46:06.938 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_unordered PASSED [ 37%] 2024-05-30T16:46:08.694 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_maxkeys_invalid PASSED [ 37%] 2024-05-30T16:46:10.522 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_none PASSED [ 37%] 2024-05-30T16:46:12.394 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_empty PASSED [ 37%] 2024-05-30T16:46:14.211 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_continuationtoken_empty PASSED [ 37%] 2024-05-30T16:46:16.182 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_continuationtoken PASSED [ 37%] 2024-05-30T16:46:18.134 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_both_continuationtoken_startafter PASSED [ 38%] 2024-05-30T16:46:20.002 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_unreadable PASSED [ 38%] 2024-05-30T16:46:21.810 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_startafter_unreadable PASSED [ 38%] 2024-05-30T16:46:23.642 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_not_in_list PASSED [ 38%] 2024-05-30T16:46:25.566 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_startafter_not_in_list PASSED [ 38%] 2024-05-30T16:46:27.366 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_after_list PASSED [ 38%] 2024-05-30T16:46:29.190 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_startafter_after_list PASSED [ 38%] 2024-05-30T16:46:31.486 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_return_data PASSED [ 39%] 2024-05-30T16:46:34.410 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_return_data_versioning PASSED [ 39%] 2024-05-30T16:46:35.798 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_objects_anonymous PASSED [ 39%] 2024-05-30T16:46:37.214 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_objects_anonymous PASSED [ 39%] 2024-05-30T16:46:38.458 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_objects_anonymous_fail PASSED [ 39%] 2024-05-30T16:46:39.662 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_objects_anonymous_fail PASSED [ 39%] 2024-05-30T16:46:40.338 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_notexist PASSED [ 39%] 2024-05-30T16:46:40.946 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_notexist PASSED [ 40%] 2024-05-30T16:46:41.546 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_delete_notexist PASSED [ 40%] 2024-05-30T16:46:43.094 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_delete_nonempty PASSED [ 40%] 2024-05-30T16:46:47.125 INFO:tasks.rgw.client.0.smithi006.stdout:2024-05-30T16:46:47.120+0000 6449b640 -1 req 5367923511924228627 0.299996883s s3:delete_bucket ERROR: could not remove bucket test-zrhryoy189ajjeo9txwotj0u-156 2024-05-30T16:46:47.570 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_concurrent_set_canned_acl PASSED [ 40%] 2024-05-30T16:46:47.978 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_to_nonexist_bucket PASSED [ 40%] 2024-05-30T16:46:47.978 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_to_nonexist_bucket ERROR [ 40%] 2024-05-30T16:46:49.029 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_with_chunked_transfer_encoding PASSED [ 40%] 2024-05-30T16:46:49.029 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_with_chunked_transfer_encoding ERROR [ 40%] 2024-05-30T16:46:49.851 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_delete PASSED [ 40%] 2024-05-30T16:46:49.851 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_delete ERROR [ 40%] 2024-05-30T16:46:50.760 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_read_not_exist PASSED [ 40%] 2024-05-30T16:46:50.761 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_read_not_exist ERROR [ 40%] 2024-05-30T16:46:51.736 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_requestid_matches_header_on_error PASSED [ 41%] 2024-05-30T16:46:51.737 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_requestid_matches_header_on_error ERROR [ 41%] 2024-05-30T16:46:56.679 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_concurrent_multi_object_delete PASSED [ 41%] 2024-05-30T16:46:56.679 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_concurrent_multi_object_delete ERROR [ 41%] 2024-05-30T16:46:58.432 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multi_object_delete PASSED [ 41%] 2024-05-30T16:46:58.433 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multi_object_delete ERROR [ 41%] 2024-05-30T16:47:00.225 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multi_objectv2_delete PASSED [ 41%] 2024-05-30T16:47:00.226 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multi_objectv2_delete ERROR [ 41%] 2024-05-30T16:49:01.154 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multi_object_delete_key_limit PASSED [ 41%] 2024-05-30T16:49:01.154 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multi_object_delete_key_limit ERROR [ 41%] 2024-05-30T16:51:02.172 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multi_objectv2_delete_key_limit PASSED [ 41%] 2024-05-30T16:51:02.172 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multi_objectv2_delete_key_limit ERROR [ 41%] 2024-05-30T16:51:03.334 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_head_zero_bytes PASSED [ 41%] 2024-05-30T16:51:03.334 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_head_zero_bytes ERROR [ 41%] 2024-05-30T16:51:04.373 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_check_etag PASSED [ 42%] 2024-05-30T16:51:04.373 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_check_etag ERROR [ 42%] 2024-05-30T16:51:05.516 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_cache_control PASSED [ 42%] 2024-05-30T16:51:05.516 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_cache_control ERROR [ 42%] 2024-05-30T16:51:06.592 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_expires PASSED [ 42%] 2024-05-30T16:51:06.592 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_expires ERROR [ 42%] 2024-05-30T16:51:07.984 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_read_update_read_delete PASSED [ 42%] 2024-05-30T16:51:07.984 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_read_update_read_delete ERROR [ 42%] 2024-05-30T16:51:09.137 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_none_to_good PASSED [ 42%] 2024-05-30T16:51:09.137 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_none_to_good ERROR [ 42%] 2024-05-30T16:51:10.243 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_none_to_empty PASSED [ 42%] 2024-05-30T16:51:10.243 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_none_to_empty ERROR [ 42%] 2024-05-30T16:51:11.642 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_overwrite_to_empty PASSED [ 42%] 2024-05-30T16:51:11.642 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_overwrite_to_empty ERROR [ 42%] 2024-05-30T16:51:12.833 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_metadata_replaced_on_put PASSED [ 43%] 2024-05-30T16:51:12.833 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_metadata_replaced_on_put ERROR [ 43%] 2024-05-30T16:51:14.031 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_file PASSED [ 43%] 2024-05-30T16:51:14.031 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_file ERROR [ 43%] 2024-05-30T16:51:15.254 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_anonymous_request PASSED [ 43%] 2024-05-30T16:51:15.254 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_anonymous_request ERROR [ 43%] 2024-05-30T16:51:16.536 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_request PASSED [ 43%] 2024-05-30T16:51:16.536 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_request ERROR [ 43%] 2024-05-30T16:51:17.666 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_no_content_type PASSED [ 43%] 2024-05-30T16:51:17.667 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_no_content_type ERROR [ 43%] 2024-05-30T16:51:18.642 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_request_bad_access_key PASSED [ 43%] 2024-05-30T16:51:18.643 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_request_bad_access_key ERROR [ 43%] 2024-05-30T16:51:19.692 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_success_code PASSED [ 43%] 2024-05-30T16:51:19.693 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_success_code ERROR [ 43%] 2024-05-30T16:51:20.760 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_invalid_success_code PASSED [ 43%] 2024-05-30T16:51:20.760 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_invalid_success_code ERROR [ 43%] 2024-05-30T16:51:22.258 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_larger_than_chunk PASSED [ 44%] 2024-05-30T16:51:22.258 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_larger_than_chunk ERROR [ 44%] 2024-05-30T16:51:23.469 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_key_from_filename PASSED [ 44%] 2024-05-30T16:51:23.469 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_key_from_filename ERROR [ 44%] 2024-05-30T16:51:24.535 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_ignored_header PASSED [ 44%] 2024-05-30T16:51:24.535 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_ignored_header ERROR [ 44%] 2024-05-30T16:51:25.590 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_case_insensitive_condition_fields PASSED [ 44%] 2024-05-30T16:51:25.590 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_case_insensitive_condition_fields ERROR [ 44%] 2024-05-30T16:51:26.788 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_escaped_field_values PASSED [ 44%] 2024-05-30T16:51:26.788 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_escaped_field_values ERROR [ 44%] 2024-05-30T16:51:28.045 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_success_redirect_action PASSED [ 44%] 2024-05-30T16:51:28.045 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_success_redirect_action ERROR [ 44%] 2024-05-30T16:51:28.969 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_signature PASSED [ 44%] 2024-05-30T16:51:28.969 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_signature ERROR [ 44%] 2024-05-30T16:51:29.910 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_access_key PASSED [ 45%] 2024-05-30T16:51:29.910 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_access_key ERROR [ 45%] 2024-05-30T16:51:30.780 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_date_format PASSED [ 45%] 2024-05-30T16:51:30.780 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_date_format ERROR [ 45%] 2024-05-30T16:51:31.644 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_no_key_specified PASSED [ 45%] 2024-05-30T16:51:31.644 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_no_key_specified ERROR [ 45%] 2024-05-30T16:51:32.523 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_signature PASSED [ 45%] 2024-05-30T16:51:32.524 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_signature ERROR [ 45%] 2024-05-30T16:51:33.497 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_policy_condition PASSED [ 45%] 2024-05-30T16:51:33.497 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_policy_condition ERROR [ 45%] 2024-05-30T16:51:34.708 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_user_specified_header PASSED [ 45%] 2024-05-30T16:51:34.708 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_user_specified_header ERROR [ 45%] 2024-05-30T16:51:35.640 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_request_missing_policy_specified_field PASSED [ 45%] 2024-05-30T16:51:35.640 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_request_missing_policy_specified_field ERROR [ 45%] 2024-05-30T16:51:36.544 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_condition_is_case_sensitive PASSED [ 46%] 2024-05-30T16:51:36.545 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_condition_is_case_sensitive ERROR [ 46%] 2024-05-30T16:51:37.470 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_expires_is_case_sensitive PASSED [ 46%] 2024-05-30T16:51:37.470 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_expires_is_case_sensitive ERROR [ 46%] 2024-05-30T16:51:38.456 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_expired_policy PASSED [ 46%] 2024-05-30T16:51:38.457 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_expired_policy ERROR [ 46%] 2024-05-30T16:51:40.034 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_wrong_bucket PASSED [ 46%] 2024-05-30T16:51:40.034 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_wrong_bucket ERROR [ 46%] 2024-05-30T16:51:40.949 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_request_field_value PASSED [ 46%] 2024-05-30T16:51:40.949 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_request_field_value ERROR [ 46%] 2024-05-30T16:51:41.910 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_expires_condition PASSED [ 46%] 2024-05-30T16:51:41.910 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_expires_condition ERROR [ 46%] 2024-05-30T16:51:42.870 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_conditions_list PASSED [ 46%] 2024-05-30T16:51:42.871 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_conditions_list ERROR [ 46%] 2024-05-30T16:51:43.850 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_limit_exceeded PASSED [ 46%] 2024-05-30T16:51:43.850 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_limit_exceeded ERROR [ 46%] 2024-05-30T16:51:44.791 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_content_length_argument PASSED [ 47%] 2024-05-30T16:51:44.791 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_content_length_argument ERROR [ 47%] 2024-05-30T16:51:45.762 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_content_length_argument PASSED [ 47%] 2024-05-30T16:51:45.762 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_content_length_argument ERROR [ 47%] 2024-05-30T16:51:46.686 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_below_minimum PASSED [ 47%] 2024-05-30T16:51:46.686 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_below_minimum ERROR [ 47%] 2024-05-30T16:51:48.769 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_rgw_chunk_size_bug PASSED [ 47%] 2024-05-30T16:51:48.769 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_rgw_chunk_size_bug ERROR [ 47%] 2024-05-30T16:51:49.632 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_empty_conditions PASSED [ 47%] 2024-05-30T16:51:49.633 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_empty_conditions ERROR [ 47%] 2024-05-30T16:51:50.791 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmatch_good PASSED [ 47%] 2024-05-30T16:51:50.791 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmatch_good ERROR [ 47%] 2024-05-30T16:51:51.993 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmatch_failed PASSED [ 47%] 2024-05-30T16:51:51.993 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmatch_failed ERROR [ 47%] 2024-05-30T16:51:53.147 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifnonematch_good PASSED [ 48%] 2024-05-30T16:51:53.147 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifnonematch_good ERROR [ 48%] 2024-05-30T16:51:54.352 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifnonematch_failed PASSED [ 48%] 2024-05-30T16:51:54.352 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifnonematch_failed ERROR [ 48%] 2024-05-30T16:51:55.484 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmodifiedsince_good PASSED [ 48%] 2024-05-30T16:51:55.484 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmodifiedsince_good ERROR [ 48%] 2024-05-30T16:51:57.702 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmodifiedsince_failed PASSED [ 48%] 2024-05-30T16:51:57.702 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmodifiedsince_failed ERROR [ 48%] 2024-05-30T16:51:58.966 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifunmodifiedsince_good PASSED [ 48%] 2024-05-30T16:51:58.966 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifunmodifiedsince_good ERROR [ 48%] 2024-05-30T16:52:00.368 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifunmodifiedsince_failed PASSED [ 48%] 2024-05-30T16:52:00.369 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifunmodifiedsince_failed ERROR [ 48%] 2024-05-30T16:52:01.736 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_good PASSED [ 48%] 2024-05-30T16:52:01.737 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_good ERROR [ 48%] 2024-05-30T16:52:03.195 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_failed PASSED [ 49%] 2024-05-30T16:52:03.196 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_failed ERROR [ 49%] 2024-05-30T16:52:04.577 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_overwrite_existed_good PASSED [ 49%] 2024-05-30T16:52:04.578 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_overwrite_existed_good ERROR [ 49%] 2024-05-30T16:52:05.663 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_nonexisted_failed PASSED [ 49%] 2024-05-30T16:52:05.663 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_nonexisted_failed ERROR [ 49%] 2024-05-30T16:52:07.046 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_good PASSED [ 49%] 2024-05-30T16:52:07.046 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_good ERROR [ 49%] 2024-05-30T16:52:08.500 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_failed PASSED [ 49%] 2024-05-30T16:52:08.501 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_failed ERROR [ 49%] 2024-05-30T16:52:09.701 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_nonexisted_good PASSED [ 49%] 2024-05-30T16:52:09.701 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_nonexisted_good ERROR [ 49%] 2024-05-30T16:52:11.076 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_overwrite_existed_failed PASSED [ 49%] 2024-05-30T16:52:11.076 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_overwrite_existed_failed ERROR [ 49%] 2024-05-30T16:52:12.185 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get PASSED [ 50%] 2024-05-30T16:52:12.185 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get ERROR [ 50%] 2024-05-30T16:52:13.176 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_bucket_gone PASSED [ 50%] 2024-05-30T16:52:13.177 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_bucket_gone ERROR [ 50%] 2024-05-30T16:52:14.196 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_delete_key_bucket_gone PASSED [ 50%] 2024-05-30T16:52:14.196 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_delete_key_bucket_gone ERROR [ 50%] 2024-05-30T16:52:15.291 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_object_gone PASSED [ 50%] 2024-05-30T16:52:15.291 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_object_gone ERROR [ 50%] 2024-05-30T16:52:16.224 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head PASSED [ 50%] 2024-05-30T16:52:16.225 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head ERROR [ 50%] 2024-05-30T16:52:16.592 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head_notexist PASSED [ 50%] 2024-05-30T16:52:16.592 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head_notexist ERROR [ 50%] 2024-05-30T16:52:18.210 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head_extended PASSED [ 50%] 2024-05-30T16:52:18.211 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head_extended ERROR [ 50%] 2024-05-30T16:52:19.357 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_bucket_acl PASSED [ 50%] 2024-05-30T16:52:19.357 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_bucket_acl ERROR [ 50%] 2024-05-30T16:52:20.475 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_object_acl PASSED [ 51%] 2024-05-30T16:52:20.475 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_object_acl ERROR [ 51%] 2024-05-30T16:52:24.665 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_put_acl_mtime PASSED [ 51%] 2024-05-30T16:52:24.665 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_put_acl_mtime ERROR [ 51%] 2024-05-30T16:52:25.747 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated PASSED [ 51%] 2024-05-30T16:52:25.747 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated ERROR [ 51%] 2024-05-30T16:52:26.922 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_response_headers PASSED [ 51%] 2024-05-30T16:52:26.922 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_response_headers ERROR [ 51%] 2024-05-30T16:52:28.100 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_bucket_acl PASSED [ 51%] 2024-05-30T16:52:28.101 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_bucket_acl ERROR [ 51%] 2024-05-30T16:52:29.231 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_object_acl PASSED [ 51%] 2024-05-30T16:52:29.231 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_object_acl ERROR [ 51%] 2024-05-30T16:52:30.279 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_bucket_gone PASSED [ 51%] 2024-05-30T16:52:30.279 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_bucket_gone ERROR [ 51%] 2024-05-30T16:52:31.442 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_object_gone PASSED [ 52%] 2024-05-30T16:52:31.442 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_object_gone ERROR [ 52%] 2024-05-30T16:52:32.727 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired PASSED [ 52%] 2024-05-30T16:52:32.727 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired ERROR [ 52%] 2024-05-30T16:52:33.366 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired_tenant PASSED [ 52%] 2024-05-30T16:52:33.366 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired_tenant ERROR [ 52%] 2024-05-30T16:52:34.530 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_range_zero PASSED [ 52%] 2024-05-30T16:52:34.531 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_range_zero ERROR [ 52%] 2024-05-30T16:52:35.599 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_max_range PASSED [ 52%] 2024-05-30T16:52:35.599 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_max_range ERROR [ 52%] 2024-05-30T16:52:36.673 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_positive_range PASSED [ 52%] 2024-05-30T16:52:36.674 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_positive_range ERROR [ 52%] 2024-05-30T16:52:37.802 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_anon_put PASSED [ 52%] 2024-05-30T16:52:37.802 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_anon_put ERROR [ 52%] 2024-05-30T16:52:38.956 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_anon_put_write_access PASSED [ 53%] 2024-05-30T16:52:38.956 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_anon_put_write_access ERROR [ 53%] 2024-05-30T16:52:40.004 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_put_authenticated PASSED [ 53%] 2024-05-30T16:52:40.004 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_put_authenticated ERROR [ 53%] 2024-05-30T16:52:41.195 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_presigned_put_object_with_acl PASSED [ 53%] 2024-05-30T16:52:41.196 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_presigned_put_object_with_acl ERROR [ 53%] 2024-05-30T16:52:41.698 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_presigned_put_object_with_acl_tenant PASSED [ 53%] 2024-05-30T16:52:41.698 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_presigned_put_object_with_acl_tenant ERROR [ 53%] 2024-05-30T16:52:42.806 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_put_authenticated_expired PASSED [ 53%] 2024-05-30T16:52:42.807 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_put_authenticated_expired ERROR [ 53%] 2024-05-30T16:52:43.083 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_starts_nonalpha PASSED [ 53%] 2024-05-30T16:52:43.083 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_starts_nonalpha ERROR [ 53%] 2024-05-30T16:52:43.360 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_short_one PASSED [ 53%] 2024-05-30T16:52:43.361 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_short_one ERROR [ 53%] 2024-05-30T16:52:43.676 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_short_two PASSED [ 53%] 2024-05-30T16:52:43.676 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_short_two ERROR [ 53%] 2024-05-30T16:52:44.604 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_60 PASSED [ 54%] 2024-05-30T16:52:44.604 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_60 ERROR [ 54%] 2024-05-30T16:52:45.459 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_61 PASSED [ 54%] 2024-05-30T16:52:45.459 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_61 ERROR [ 54%] 2024-05-30T16:52:46.293 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_62 PASSED [ 54%] 2024-05-30T16:52:46.293 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_62 ERROR [ 54%] 2024-05-30T16:52:47.200 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_63 PASSED [ 54%] 2024-05-30T16:52:47.200 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_63 ERROR [ 54%] 2024-05-30T16:52:48.203 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_long_name PASSED [ 54%] 2024-05-30T16:52:48.203 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_long_name ERROR [ 54%] 2024-05-30T16:52:48.501 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_ip PASSED [ 54%] 2024-05-30T16:52:48.501 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_ip ERROR [ 54%] 2024-05-30T16:52:48.795 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_underscore PASSED [ 54%] 2024-05-30T16:52:48.795 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_underscore ERROR [ 54%] 2024-05-30T16:52:49.730 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_long PASSED [ 55%] 2024-05-30T16:52:49.730 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_long ERROR [ 55%] 2024-05-30T16:52:50.015 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dash_at_end PASSED [ 55%] 2024-05-30T16:52:50.016 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dash_at_end ERROR [ 55%] 2024-05-30T16:52:50.306 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dot_dot PASSED [ 55%] 2024-05-30T16:52:50.306 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dot_dot ERROR [ 55%] 2024-05-30T16:52:50.608 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dot_dash PASSED [ 55%] 2024-05-30T16:52:50.608 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dot_dash ERROR [ 55%] 2024-05-30T16:52:50.913 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dash_dot PASSED [ 55%] 2024-05-30T16:52:50.913 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dash_dot ERROR [ 55%] 2024-05-30T16:52:52.018 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_exists PASSED [ 55%] 2024-05-30T16:52:52.018 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_exists ERROR [ 55%] 2024-05-30T16:52:52.972 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_get_location PASSED [ 55%] 2024-05-30T16:52:52.972 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_get_location ERROR [ 55%] 2024-05-30T16:52:53.908 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_exists_nonowner PASSED [ 56%] 2024-05-30T16:52:53.908 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_exists_nonowner ERROR [ 56%] 2024-05-30T16:52:54.893 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_overwrite_acl PASSED [ 56%] 2024-05-30T16:52:54.893 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_overwrite_acl ERROR [ 56%] 2024-05-30T16:52:55.881 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_new_acl PASSED [ 56%] 2024-05-30T16:52:55.881 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_new_acl ERROR [ 56%] 2024-05-30T16:52:56.838 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_default PASSED [ 56%] 2024-05-30T16:52:56.838 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_default ERROR [ 56%] 2024-05-30T16:52:57.805 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_during_create PASSED [ 56%] 2024-05-30T16:52:57.806 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_during_create ERROR [ 56%] 2024-05-30T16:52:58.925 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned PASSED [ 56%] 2024-05-30T16:52:58.925 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned ERROR [ 56%] 2024-05-30T16:52:59.980 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_publicreadwrite PASSED [ 56%] 2024-05-30T16:52:59.980 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_publicreadwrite ERROR [ 56%] 2024-05-30T16:53:00.929 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_authenticatedread PASSED [ 56%] 2024-05-30T16:53:00.929 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_authenticatedread ERROR [ 56%] 2024-05-30T16:53:02.110 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_default PASSED [ 57%] 2024-05-30T16:53:02.111 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_default ERROR [ 57%] 2024-05-30T16:53:03.329 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_during_create PASSED [ 57%] 2024-05-30T16:53:03.329 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_during_create ERROR [ 57%] 2024-05-30T16:53:04.859 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned PASSED [ 57%] 2024-05-30T16:53:04.859 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned ERROR [ 57%] 2024-05-30T16:53:06.019 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_publicreadwrite PASSED [ 57%] 2024-05-30T16:53:06.019 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_publicreadwrite ERROR [ 57%] 2024-05-30T16:53:07.188 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_authenticatedread PASSED [ 57%] 2024-05-30T16:53:07.188 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_authenticatedread ERROR [ 57%] 2024-05-30T16:53:08.563 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_bucketownerread PASSED [ 57%] 2024-05-30T16:53:08.563 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_bucketownerread ERROR [ 57%] 2024-05-30T16:53:09.981 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_bucketownerfullcontrol PASSED [ 57%] 2024-05-30T16:53:09.981 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_bucketownerfullcontrol ERROR [ 57%] 2024-05-30T16:53:11.423 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_full_control_verify_owner PASSED [ 58%] 2024-05-30T16:53:11.423 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_full_control_verify_owner ERROR [ 58%] 2024-05-30T16:53:12.780 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_full_control_verify_attributes PASSED [ 58%] 2024-05-30T16:53:12.780 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_full_control_verify_attributes ERROR [ 58%] 2024-05-30T16:53:13.694 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_private_to_private PASSED [ 58%] 2024-05-30T16:53:13.694 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_private_to_private ERROR [ 58%] 2024-05-30T16:53:15.204 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl PASSED [ 58%] 2024-05-30T16:53:15.204 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl ERROR [ 58%] 2024-05-30T16:53:16.545 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_write PASSED [ 58%] 2024-05-30T16:53:16.545 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_write ERROR [ 58%] 2024-05-30T16:53:17.921 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_writeacp PASSED [ 58%] 2024-05-30T16:53:17.921 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_writeacp ERROR [ 58%] 2024-05-30T16:53:19.349 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_read PASSED [ 58%] 2024-05-30T16:53:19.349 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_read ERROR [ 58%] 2024-05-30T16:53:20.798 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_readacp PASSED [ 59%] 2024-05-30T16:53:20.798 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_readacp ERROR [ 59%] 2024-05-30T16:53:22.426 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_fullcontrol PASSED [ 59%] 2024-05-30T16:53:22.427 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_fullcontrol ERROR [ 59%] 2024-05-30T16:53:23.861 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_read PASSED [ 59%] 2024-05-30T16:53:23.861 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_read ERROR [ 59%] 2024-05-30T16:53:25.329 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_readacp PASSED [ 59%] 2024-05-30T16:53:25.330 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_readacp ERROR [ 59%] 2024-05-30T16:53:26.849 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_write PASSED [ 59%] 2024-05-30T16:53:26.849 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_write ERROR [ 59%] 2024-05-30T16:53:28.253 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_writeacp PASSED [ 59%] 2024-05-30T16:53:28.253 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_writeacp ERROR [ 59%] 2024-05-30T16:53:29.345 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_nonexist_user PASSED [ 59%] 2024-05-30T16:53:29.346 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_nonexist_user ERROR [ 59%] 2024-05-30T16:53:30.522 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_header_acl_grants PASSED [ 59%] 2024-05-30T16:53:30.522 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_header_acl_grants ERROR [ 59%] 2024-05-30T16:53:31.804 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_header_acl_grants PASSED [ 60%] 2024-05-30T16:53:31.804 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_header_acl_grants ERROR [ 60%] 2024-05-30T16:53:32.950 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_email PASSED [ 60%] 2024-05-30T16:53:32.951 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_email ERROR [ 60%] 2024-05-30T16:53:34.018 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_email_not_exist PASSED [ 60%] 2024-05-30T16:53:34.018 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_email_not_exist ERROR [ 60%] 2024-05-30T16:53:35.453 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_revoke_all PASSED [ 60%] 2024-05-30T16:53:35.453 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_revoke_all ERROR [ 60%] 2024-05-30T16:53:37.302 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_private PASSED [ 60%] 2024-05-30T16:53:37.302 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_private ERROR [ 60%] 2024-05-30T16:53:39.194 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_private PASSED [ 60%] 2024-05-30T16:53:39.194 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_private ERROR [ 60%] 2024-05-30T16:53:41.238 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_publicread PASSED [ 60%] 2024-05-30T16:53:41.238 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_publicread ERROR [ 60%] 2024-05-30T16:53:43.129 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_publicread PASSED [ 61%] 2024-05-30T16:53:43.129 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_publicread ERROR [ 61%] 2024-05-30T16:53:45.099 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_publicreadwrite PASSED [ 61%] 2024-05-30T16:53:45.099 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_publicreadwrite ERROR [ 61%] 2024-05-30T16:53:47.106 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_publicreadwrite PASSED [ 61%] 2024-05-30T16:53:47.107 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_publicreadwrite ERROR [ 61%] 2024-05-30T16:53:49.043 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_private PASSED [ 61%] 2024-05-30T16:53:49.044 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_private ERROR [ 61%] 2024-05-30T16:53:51.037 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_publicread PASSED [ 61%] 2024-05-30T16:53:51.037 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_publicread ERROR [ 61%] 2024-05-30T16:53:52.987 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_publicreadwrite PASSED [ 61%] 2024-05-30T16:53:52.988 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_publicreadwrite ERROR [ 61%] 2024-05-30T16:53:55.090 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_private PASSED [ 61%] 2024-05-30T16:53:55.090 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_private ERROR [ 61%] 2024-05-30T16:53:57.110 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicread PASSED [ 62%] 2024-05-30T16:53:57.110 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicread ERROR [ 62%] 2024-05-30T16:53:59.216 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicreadwrite PASSED [ 62%] 2024-05-30T16:53:59.216 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicreadwrite ERROR [ 62%] 2024-05-30T16:54:02.815 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_buckets_create_then_list PASSED [ 62%] 2024-05-30T16:54:02.815 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_buckets_create_then_list ERROR [ 62%] 2024-05-30T16:54:06.328 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_buckets_list_ctime PASSED [ 62%] 2024-05-30T16:54:06.328 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_buckets_list_ctime ERROR [ 62%] 2024-05-30T16:54:06.604 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_anonymous PASSED [ 62%] 2024-05-30T16:54:06.604 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_anonymous ERROR [ 62%] 2024-05-30T16:54:06.927 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_invalid_auth PASSED [ 62%] 2024-05-30T16:54:06.927 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_invalid_auth ERROR [ 62%] 2024-05-30T16:54:07.191 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_bad_auth PASSED [ 62%] 2024-05-30T16:54:07.191 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_bad_auth ERROR [ 62%] 2024-05-30T16:54:08.210 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_starts_alpha PASSED [ 62%] 2024-05-30T16:54:08.210 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_starts_alpha ERROR [ 62%] 2024-05-30T16:54:09.245 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_starts_digit PASSED [ 63%] 2024-05-30T16:54:09.245 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_starts_digit ERROR [ 63%] 2024-05-30T16:54:10.167 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_contains_period PASSED [ 63%] 2024-05-30T16:54:10.167 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_contains_period ERROR [ 63%] 2024-05-30T16:54:11.096 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_contains_hyphen PASSED [ 63%] 2024-05-30T16:54:11.097 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_contains_hyphen ERROR [ 63%] 2024-05-30T16:54:12.673 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_not_overriding PASSED [ 63%] 2024-05-30T16:54:12.673 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_not_overriding ERROR [ 63%] 2024-05-30T16:54:17.571 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_special_key_names PASSED [ 63%] 2024-05-30T16:54:17.571 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_special_key_names ERROR [ 63%] 2024-05-30T16:54:19.381 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_special_prefix PASSED [ 63%] 2024-05-30T16:54:19.381 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_special_prefix ERROR [ 63%] 2024-05-30T16:54:20.880 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_zero_size PASSED [ 63%] 2024-05-30T16:54:20.880 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_zero_size ERROR [ 63%] 2024-05-30T16:54:24.659 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_16m PASSED [ 64%] 2024-05-30T16:54:24.659 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_16m ERROR [ 64%] 2024-05-30T16:54:26.101 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_same_bucket PASSED [ 64%] 2024-05-30T16:54:26.105 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_same_bucket ERROR [ 64%] 2024-05-30T16:54:27.532 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_verify_contenttype PASSED [ 64%] 2024-05-30T16:54:27.532 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_verify_contenttype ERROR [ 64%] 2024-05-30T16:54:28.780 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_to_itself PASSED [ 64%] 2024-05-30T16:54:28.781 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_to_itself ERROR [ 64%] 2024-05-30T16:54:30.097 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_to_itself_with_metadata PASSED [ 64%] 2024-05-30T16:54:30.097 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_to_itself_with_metadata ERROR [ 64%] 2024-05-30T16:54:32.289 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_diff_bucket PASSED [ 64%] 2024-05-30T16:54:32.290 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_diff_bucket ERROR [ 64%] 2024-05-30T16:54:33.577 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_not_owned_bucket PASSED [ 64%] 2024-05-30T16:54:33.577 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_not_owned_bucket ERROR [ 64%] 2024-05-30T16:54:35.372 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_not_owned_object_bucket PASSED [ 65%] 2024-05-30T16:54:35.372 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_not_owned_object_bucket ERROR [ 65%] 2024-05-30T16:54:36.920 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_canned_acl PASSED [ 65%] 2024-05-30T16:54:36.920 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_canned_acl ERROR [ 65%] 2024-05-30T16:54:39.766 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_retaining_metadata PASSED [ 65%] 2024-05-30T16:54:39.766 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_retaining_metadata ERROR [ 65%] 2024-05-30T16:54:42.413 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_replacing_metadata PASSED [ 65%] 2024-05-30T16:54:42.413 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_replacing_metadata ERROR [ 65%] 2024-05-30T16:54:43.326 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_bucket_not_found PASSED [ 65%] 2024-05-30T16:54:43.326 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_bucket_not_found ERROR [ 65%] 2024-05-30T16:54:44.256 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_key_not_found PASSED [ 65%] 2024-05-30T16:54:44.256 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_key_not_found ERROR [ 65%] 2024-05-30T16:54:48.991 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioned_bucket PASSED [ 65%] 2024-05-30T16:54:48.992 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioned_bucket ERROR [ 65%] 2024-05-30T16:54:50.776 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioned_url_encoding PASSED [ 65%] 2024-05-30T16:54:50.776 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioned_url_encoding ERROR [ 65%] 2024-05-30T16:55:09.704 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioning_multipart_upload PASSED [ 66%] 2024-05-30T16:55:09.704 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioning_multipart_upload ERROR [ 66%] 2024-05-30T16:55:10.779 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_empty PASSED [ 66%] 2024-05-30T16:55:10.780 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_empty ERROR [ 66%] 2024-05-30T16:55:12.242 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_small PASSED [ 66%] 2024-05-30T16:55:12.242 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_small ERROR [ 66%] 2024-05-30T16:55:16.267 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_small PASSED [ 66%] 2024-05-30T16:55:16.268 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_small ERROR [ 66%] 2024-05-30T16:55:17.562 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_invalid_range PASSED [ 66%] 2024-05-30T16:55:17.562 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_invalid_range ERROR [ 66%] 2024-05-30T16:55:20.169 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_without_range PASSED [ 66%] 2024-05-30T16:55:20.169 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_without_range ERROR [ 66%] 2024-05-30T16:55:31.792 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_special_names PASSED [ 66%] 2024-05-30T16:55:31.793 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_special_names ERROR [ 66%] 2024-05-30T16:55:46.892 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload PASSED [ 67%] 2024-05-30T16:55:46.892 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload ERROR [ 67%] 2024-05-30T16:56:07.204 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_versioned PASSED [ 67%] 2024-05-30T16:56:07.204 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_versioned ERROR [ 67%] 2024-05-30T16:57:42.889 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_resend_part PASSED [ 67%] 2024-05-30T16:57:42.890 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_resend_part ERROR [ 67%] 2024-05-30T16:57:54.323 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_multiple_sizes PASSED [ 67%] 2024-05-30T16:57:54.323 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_multiple_sizes ERROR [ 67%] 2024-05-30T16:58:10.876 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_multiple_sizes PASSED [ 67%] 2024-05-30T16:58:10.876 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_multiple_sizes ERROR [ 67%] 2024-05-30T16:58:13.428 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_size_too_small PASSED [ 67%] 2024-05-30T16:58:13.428 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_size_too_small ERROR [ 67%] 2024-05-30T16:58:19.145 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_contents PASSED [ 67%] 2024-05-30T16:58:19.145 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_contents ERROR [ 67%] 2024-05-30T16:58:23.918 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_overwrite_existing_object PASSED [ 68%] 2024-05-30T16:58:23.919 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_overwrite_existing_object ERROR [ 68%] 2024-05-30T16:58:26.818 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_abort_multipart_upload PASSED [ 68%] 2024-05-30T16:58:26.818 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_abort_multipart_upload ERROR [ 68%] 2024-05-30T16:58:27.969 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_abort_multipart_upload_not_found PASSED [ 68%] 2024-05-30T16:58:27.970 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_abort_multipart_upload_not_found ERROR [ 68%] 2024-05-30T16:58:33.029 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_list_multipart_upload PASSED [ 68%] 2024-05-30T16:58:33.029 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_list_multipart_upload ERROR [ 68%] 2024-05-30T16:58:34.602 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_list_multipart_upload_owner PASSED [ 68%] 2024-05-30T16:58:34.602 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_list_multipart_upload_owner ERROR [ 68%] 2024-05-30T16:58:35.877 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_missing_part PASSED [ 68%] 2024-05-30T16:58:35.877 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_missing_part ERROR [ 68%] 2024-05-30T16:58:37.234 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_incorrect_etag PASSED [ 68%] 2024-05-30T16:58:37.235 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_incorrect_etag ERROR [ 68%] 2024-05-30T16:58:57.682 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_get_part PASSED [ 68%] 2024-05-30T16:58:57.682 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_get_part ERROR [ 68%] 2024-05-30T16:58:58.975 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_non_multipart_get_part PASSED [ 69%] 2024-05-30T16:58:58.976 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_non_multipart_get_part ERROR [ 69%] 2024-05-30T16:58:59.990 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_100_continue PASSED [ 69%] 2024-05-30T16:58:59.990 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_100_continue ERROR [ 69%] 2024-05-30T16:59:01.288 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_set_cors PASSED [ 69%] 2024-05-30T16:59:01.288 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_set_cors ERROR [ 69%] 2024-05-30T16:59:06.847 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_cors_origin_response PASSED [ 69%] 2024-05-30T16:59:06.847 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_cors_origin_response ERROR [ 69%] 2024-05-30T16:59:11.113 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_cors_origin_wildcard PASSED [ 69%] 2024-05-30T16:59:11.114 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_cors_origin_wildcard ERROR [ 69%] 2024-05-30T16:59:15.171 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_cors_header_option PASSED [ 69%] 2024-05-30T16:59:15.171 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_cors_header_option ERROR [ 69%] 2024-05-30T16:59:16.423 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_get_object PASSED [ 69%] 2024-05-30T16:59:16.424 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_get_object ERROR [ 69%] 2024-05-30T16:59:17.106 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_get_object_tenant PASSED [ 70%] 2024-05-30T16:59:17.106 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_get_object_tenant ERROR [ 70%] 2024-05-30T16:59:18.451 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object PASSED [ 70%] 2024-05-30T16:59:18.451 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object ERROR [ 70%] 2024-05-30T16:59:19.709 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_with_acl PASSED [ 70%] 2024-05-30T16:59:19.709 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_with_acl ERROR [ 70%] 2024-05-30T16:59:20.333 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_tenant PASSED [ 70%] 2024-05-30T16:59:20.334 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_tenant ERROR [ 70%] 2024-05-30T16:59:21.007 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_tenant_with_acl PASSED [ 70%] 2024-05-30T16:59:21.007 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_tenant_with_acl ERROR [ 70%] 2024-05-30T16:59:22.344 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_set_bucket_tagging PASSED [ 70%] 2024-05-30T16:59:22.344 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_set_bucket_tagging ERROR [ 70%] 2024-05-30T16:59:24.263 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_read_1mb PASSED [ 70%] 2024-05-30T16:59:24.264 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_read_1mb ERROR [ 70%] 2024-05-30T16:59:27.535 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_read_4mb PASSED [ 71%] 2024-05-30T16:59:27.535 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_read_4mb ERROR [ 71%] 2024-05-30T16:59:31.664 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_read_8mb PASSED [ 71%] 2024-05-30T16:59:31.665 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_read_8mb ERROR [ 71%] 2024-05-30T16:59:34.056 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_write_1mb PASSED [ 71%] 2024-05-30T16:59:34.056 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_write_1mb ERROR [ 71%] 2024-05-30T16:59:37.761 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_write_4mb PASSED [ 71%] 2024-05-30T16:59:37.761 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_write_4mb ERROR [ 71%] 2024-05-30T16:59:43.483 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_write_8mb PASSED [ 71%] 2024-05-30T16:59:43.484 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_write_8mb ERROR [ 71%] 2024-05-30T16:59:45.829 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_1mb PASSED [ 71%] 2024-05-30T16:59:45.829 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_1mb ERROR [ 71%] 2024-05-30T16:59:50.388 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_4mb PASSED [ 71%] 2024-05-30T16:59:50.388 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_4mb ERROR [ 71%] 2024-05-30T16:59:57.242 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_8mb PASSED [ 71%] 2024-05-30T16:59:57.243 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_8mb ERROR [ 71%] 2024-05-30T16:59:59.405 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_conditional_write_1mb PASSED [ 72%] 2024-05-30T16:59:59.405 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_conditional_write_1mb ERROR [ 72%] 2024-05-30T17:00:00.820 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_multipart_upload_write PASSED [ 72%] 2024-05-30T17:00:00.820 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_multipart_upload_write ERROR [ 72%] 2024-05-30T17:00:03.062 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_resend_first_finishes_last PASSED [ 72%] 2024-05-30T17:00:03.062 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_resend_first_finishes_last ERROR [ 72%] 2024-05-30T17:00:04.290 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_response_code PASSED [ 72%] 2024-05-30T17:00:04.291 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_response_code ERROR [ 72%] 2024-05-30T17:00:11.147 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_big_request_response_code PASSED [ 72%] 2024-05-30T17:00:11.147 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_big_request_response_code ERROR [ 72%] 2024-05-30T17:00:12.345 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_skip_leading_bytes_response_code PASSED [ 72%] 2024-05-30T17:00:12.345 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_skip_leading_bytes_response_code ERROR [ 72%] 2024-05-30T17:00:13.598 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_return_trailing_bytes_response_code PASSED [ 72%] 2024-05-30T17:00:13.598 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_return_trailing_bytes_response_code ERROR [ 72%] 2024-05-30T17:00:14.837 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_invalid_range PASSED [ 73%] 2024-05-30T17:00:14.837 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_invalid_range ERROR [ 73%] 2024-05-30T17:00:16.030 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_empty_object PASSED [ 73%] 2024-05-30T17:00:16.030 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_empty_object ERROR [ 73%] 2024-05-30T17:00:17.715 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_bucket_create_suspend PASSED [ 73%] 2024-05-30T17:00:17.715 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_bucket_create_suspend ERROR [ 73%] 2024-05-30T17:00:40.590 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_read_remove PASSED [ 73%] 2024-05-30T17:00:40.590 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_read_remove ERROR [ 73%] 2024-05-30T17:00:43.388 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_read_remove_head PASSED [ 73%] 2024-05-30T17:00:43.388 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_read_remove_head ERROR [ 73%] 2024-05-30T17:00:45.055 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_removal PASSED [ 73%] 2024-05-30T17:00:45.055 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_removal ERROR [ 73%] 2024-05-30T17:00:47.106 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_overwrite PASSED [ 73%] 2024-05-30T17:00:47.106 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_overwrite ERROR [ 73%] 2024-05-30T17:00:49.306 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_overwrite_suspended PASSED [ 74%] 2024-05-30T17:00:49.306 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_overwrite_suspended ERROR [ 74%] 2024-05-30T17:00:58.845 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_suspend_versions PASSED [ 74%] 2024-05-30T17:00:58.845 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_suspend_versions ERROR [ 74%] 2024-05-30T17:01:01.118 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_suspended_copy PASSED [ 74%] 2024-05-30T17:01:01.119 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_suspended_copy ERROR [ 74%] 2024-05-30T17:01:12.199 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_versions_remove_all PASSED [ 74%] 2024-05-30T17:01:12.199 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_versions_remove_all ERROR [ 74%] 2024-05-30T17:01:53.364 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_versions_remove_special_names PASSED [ 74%] 2024-05-30T17:01:53.364 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_versions_remove_special_names ERROR [ 74%] 2024-05-30T17:02:11.175 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_overwrite_multipart PASSED [ 74%] 2024-05-30T17:02:11.176 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_overwrite_multipart ERROR [ 74%] 2024-05-30T17:02:16.072 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_list_marker PASSED [ 74%] 2024-05-30T17:02:16.072 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_list_marker ERROR [ 74%] 2024-05-30T17:02:20.965 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_copy_obj_version PASSED [ 75%] 2024-05-30T17:02:20.965 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_copy_obj_version ERROR [ 75%] 2024-05-30T17:02:22.896 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete PASSED [ 75%] 2024-05-30T17:02:22.896 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete ERROR [ 75%] 2024-05-30T17:02:24.964 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete_with_marker PASSED [ 75%] 2024-05-30T17:02:24.965 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete_with_marker ERROR [ 75%] 2024-05-30T17:02:26.442 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete_with_marker_create PASSED [ 75%] 2024-05-30T17:02:26.443 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete_with_marker_create ERROR [ 75%] 2024-05-30T17:02:29.061 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioned_object_acl PASSED [ 75%] 2024-05-30T17:02:29.062 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioned_object_acl ERROR [ 75%] 2024-05-30T17:02:31.641 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioned_object_acl_no_version_specified PASSED [ 75%] 2024-05-30T17:02:31.641 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioned_object_acl_no_version_specified ERROR [ 75%] 2024-05-30T17:02:35.964 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioned_concurrent_object_create_and_remove PASSED [ 75%] 2024-05-30T17:02:35.964 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioned_concurrent_object_create_and_remove ERROR [ 75%] 2024-05-30T17:02:37.026 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set PASSED [ 75%] 2024-05-30T17:02:37.026 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set ERROR [ 75%] 2024-05-30T17:02:38.098 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_get PASSED [ 76%] 2024-05-30T17:02:38.099 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_get ERROR [ 76%] 2024-05-30T17:02:39.191 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_get_no_id PASSED [ 76%] 2024-05-30T17:02:39.192 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_get_no_id ERROR [ 76%] 2024-05-30T17:02:40.094 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_id_too_long PASSED [ 76%] 2024-05-30T17:02:40.094 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_id_too_long ERROR [ 76%] 2024-05-30T17:02:41.153 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_same_id PASSED [ 76%] 2024-05-30T17:02:41.153 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_same_id ERROR [ 76%] 2024-05-30T17:02:42.301 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_invalid_status PASSED [ 76%] 2024-05-30T17:02:42.301 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_invalid_status ERROR [ 76%] 2024-05-30T17:02:43.277 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_date PASSED [ 76%] 2024-05-30T17:02:43.278 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_date ERROR [ 76%] 2024-05-30T17:02:44.177 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_invalid_date PASSED [ 76%] 2024-05-30T17:02:44.177 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_invalid_date ERROR [ 76%] 2024-05-30T17:02:45.510 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_noncurrent PASSED [ 77%] 2024-05-30T17:02:45.510 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_noncurrent ERROR [ 77%] 2024-05-30T17:02:46.541 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_deletemarker PASSED [ 77%] 2024-05-30T17:02:46.541 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_deletemarker ERROR [ 77%] 2024-05-30T17:02:47.478 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_filter PASSED [ 77%] 2024-05-30T17:02:47.478 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_filter ERROR [ 77%] 2024-05-30T17:02:48.439 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_empty_filter PASSED [ 77%] 2024-05-30T17:02:48.439 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_empty_filter ERROR [ 77%] 2024-05-30T17:02:49.382 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_multipart PASSED [ 77%] 2024-05-30T17:02:49.382 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_multipart ERROR [ 77%] 2024-05-30T17:02:50.399 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_transition_set_invalid_date PASSED [ 77%] 2024-05-30T17:02:50.399 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_transition_set_invalid_date ERROR [ 77%] 2024-05-30T17:02:51.794 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1b PASSED [ 77%] 2024-05-30T17:02:51.794 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1b ERROR [ 77%] 2024-05-30T17:02:53.009 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1kb PASSED [ 78%] 2024-05-30T17:02:53.009 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1kb ERROR [ 78%] 2024-05-30T17:02:54.658 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1MB PASSED [ 78%] 2024-05-30T17:02:54.659 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1MB ERROR [ 78%] 2024-05-30T17:02:55.994 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_13b PASSED [ 78%] 2024-05-30T17:02:55.994 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_13b ERROR [ 78%] 2024-05-30T17:02:57.326 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_method_head PASSED [ 78%] 2024-05-30T17:02:57.327 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_method_head ERROR [ 78%] 2024-05-30T17:02:58.442 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_present PASSED [ 78%] 2024-05-30T17:02:58.443 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_present ERROR [ 78%] 2024-05-30T17:02:59.689 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_other_key PASSED [ 78%] 2024-05-30T17:02:59.689 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_other_key ERROR [ 78%] 2024-05-30T17:03:00.702 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_invalid_md5 PASSED [ 78%] 2024-05-30T17:03:00.702 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_invalid_md5 ERROR [ 78%] 2024-05-30T17:03:01.719 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_no_md5 PASSED [ 78%] 2024-05-30T17:03:01.720 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_no_md5 ERROR [ 78%] 2024-05-30T17:03:02.707 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_no_key PASSED [ 79%] 2024-05-30T17:03:02.707 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_no_key ERROR [ 79%] 2024-05-30T17:03:03.784 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_key_no_sse_c PASSED [ 79%] 2024-05-30T17:03:03.784 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_key_no_sse_c ERROR [ 79%] 2024-05-30T17:03:45.746 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_multipart_upload PASSED [ 79%] 2024-05-30T17:03:45.747 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_multipart_upload ERROR [ 79%] 2024-05-30T17:04:27.087 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_unaligned_multipart_upload PASSED [ 79%] 2024-05-30T17:04:27.088 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_unaligned_multipart_upload ERROR [ 79%] 2024-05-30T17:04:37.908 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_multipart_bad_download PASSED [ 79%] 2024-05-30T17:04:37.909 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_multipart_bad_download ERROR [ 79%] 2024-05-30T17:04:39.126 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_post_object_authenticated_request PASSED [ 79%] 2024-05-30T17:04:39.126 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_post_object_authenticated_request ERROR [ 79%] 2024-05-30T17:04:40.457 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_method_head PASSED [ 79%] 2024-05-30T17:04:40.457 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_method_head ERROR [ 79%] 2024-05-30T17:04:41.675 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_present PASSED [ 80%] 2024-05-30T17:04:41.675 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_present ERROR [ 80%] 2024-05-30T17:04:42.687 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_no_key PASSED [ 80%] 2024-05-30T17:04:42.687 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_no_key ERROR [ 80%] 2024-05-30T17:04:43.590 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_not_declared PASSED [ 80%] 2024-05-30T17:04:43.590 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_not_declared ERROR [ 80%] 2024-05-30T17:05:14.849 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_upload PASSED [ 80%] 2024-05-30T17:05:14.849 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_upload ERROR [ 80%] 2024-05-30T17:05:24.221 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_1 PASSED [ 80%] 2024-05-30T17:05:24.222 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_1 ERROR [ 80%] 2024-05-30T17:05:33.570 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_2 PASSED [ 80%] 2024-05-30T17:05:33.570 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_2 ERROR [ 80%] 2024-05-30T17:05:34.696 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_post_object_authenticated_request PASSED [ 80%] 2024-05-30T17:05:34.696 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_post_object_authenticated_request ERROR [ 80%] 2024-05-30T17:05:35.878 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1b PASSED [ 81%] 2024-05-30T17:05:35.878 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1b ERROR [ 81%] 2024-05-30T17:05:37.030 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1kb PASSED [ 81%] 2024-05-30T17:05:37.030 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1kb ERROR [ 81%] 2024-05-30T17:05:38.638 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1MB PASSED [ 81%] 2024-05-30T17:05:38.638 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1MB ERROR [ 81%] 2024-05-30T17:05:39.927 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_13b PASSED [ 81%] 2024-05-30T17:05:39.927 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_13b ERROR [ 81%] 2024-05-30T17:05:41.133 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_read_declare PASSED [ 81%] 2024-05-30T17:05:41.134 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_read_declare ERROR [ 81%] 2024-05-30T17:05:42.355 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy PASSED [ 81%] 2024-05-30T17:05:42.356 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy ERROR [ 81%] 2024-05-30T17:05:43.638 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_policy PASSED [ 81%] 2024-05-30T17:05:43.639 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_policy ERROR [ 81%] 2024-05-30T17:05:45.221 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_acl PASSED [ 81%] 2024-05-30T17:05:45.222 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_acl ERROR [ 81%] 2024-05-30T17:05:46.679 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_policy_acl PASSED [ 82%] 2024-05-30T17:05:46.679 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_policy_acl ERROR [ 82%] 2024-05-30T17:05:49.146 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_another_bucket PASSED [ 82%] 2024-05-30T17:05:49.146 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_another_bucket ERROR [ 82%] 2024-05-30T17:05:51.617 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_policy_another_bucket PASSED [ 82%] 2024-05-30T17:05:51.617 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_policy_another_bucket ERROR [ 82%] 2024-05-30T17:05:53.989 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_obj_tagging PASSED [ 82%] 2024-05-30T17:05:53.989 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_obj_tagging ERROR [ 82%] 2024-05-30T17:05:56.166 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_obj_head_tagging PASSED [ 82%] 2024-05-30T17:05:56.166 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_obj_head_tagging ERROR [ 82%] 2024-05-30T17:05:58.551 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_max_tags PASSED [ 82%] 2024-05-30T17:05:58.551 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_max_tags ERROR [ 82%] 2024-05-30T17:06:00.843 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_excess_tags PASSED [ 82%] 2024-05-30T17:06:00.843 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_excess_tags ERROR [ 82%] 2024-05-30T17:06:03.170 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_max_kvsize_tags PASSED [ 83%] 2024-05-30T17:06:03.170 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_max_kvsize_tags ERROR [ 83%] 2024-05-30T17:06:05.510 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_excess_key_tags PASSED [ 83%] 2024-05-30T17:06:05.510 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_excess_key_tags ERROR [ 83%] 2024-05-30T17:06:07.877 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_excess_val_tags PASSED [ 83%] 2024-05-30T17:06:07.877 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_excess_val_tags ERROR [ 83%] 2024-05-30T17:06:10.563 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_modify_tags PASSED [ 83%] 2024-05-30T17:06:10.563 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_modify_tags ERROR [ 83%] 2024-05-30T17:06:13.292 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_delete_tags PASSED [ 83%] 2024-05-30T17:06:13.293 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_delete_tags ERROR [ 83%] 2024-05-30T17:06:14.566 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_tags_anonymous_request PASSED [ 83%] 2024-05-30T17:06:14.566 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_tags_anonymous_request ERROR [ 83%] 2024-05-30T17:06:15.672 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_tags_authenticated_request PASSED [ 83%] 2024-05-30T17:06:15.672 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_tags_authenticated_request ERROR [ 83%] 2024-05-30T17:06:17.021 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_with_tags PASSED [ 84%] 2024-05-30T17:06:17.021 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_with_tags ERROR [ 84%] 2024-05-30T17:06:19.379 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_tags_acl_public PASSED [ 84%] 2024-05-30T17:06:19.380 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_tags_acl_public ERROR [ 84%] 2024-05-30T17:06:21.975 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_tags_acl_public PASSED [ 84%] 2024-05-30T17:06:21.975 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_tags_acl_public ERROR [ 84%] 2024-05-30T17:06:25.085 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_delete_tags_obj_public PASSED [ 84%] 2024-05-30T17:06:25.085 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_delete_tags_obj_public ERROR [ 84%] 2024-05-30T17:06:28.384 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_bucket_atomic_upload_return_version_id PASSED [ 84%] 2024-05-30T17:06:28.385 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_bucket_atomic_upload_return_version_id ERROR [ 84%] 2024-05-30T17:06:50.550 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_bucket_multipart_upload_return_version_id PASSED [ 84%] 2024-05-30T17:06:50.551 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_bucket_multipart_upload_return_version_id ERROR [ 84%] 2024-05-30T17:06:52.577 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_existing_tag PASSED [ 84%] 2024-05-30T17:06:52.577 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_existing_tag ERROR [ 84%] 2024-05-30T17:06:54.855 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_tagging_existing_tag PASSED [ 84%] 2024-05-30T17:06:54.855 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_tagging_existing_tag ERROR [ 84%] 2024-05-30T17:06:56.962 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_tagging_existing_tag PASSED [ 85%] 2024-05-30T17:06:56.962 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_tagging_existing_tag ERROR [ 85%] 2024-05-30T17:07:00.002 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_copy_source PASSED [ 85%] 2024-05-30T17:07:00.002 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_copy_source ERROR [ 85%] 2024-05-30T17:07:02.615 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_copy_source_meta PASSED [ 85%] 2024-05-30T17:07:02.615 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_copy_source_meta ERROR [ 85%] 2024-05-30T17:07:03.905 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_acl PASSED [ 85%] 2024-05-30T17:07:03.905 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_acl ERROR [ 85%] 2024-05-30T17:07:06.585 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_grant PASSED [ 85%] 2024-05-30T17:07:06.585 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_grant ERROR [ 85%] 2024-05-30T17:07:07.550 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_c_s3 PASSED [ 85%] 2024-05-30T17:07:07.550 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_c_s3 ERROR [ 85%] 2024-05-30T17:07:08.463 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_c_kms PASSED [ 85%] 2024-05-30T17:07:08.463 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_c_kms ERROR [ 85%] 2024-05-30T17:07:09.328 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_s3_kms PASSED [ 86%] 2024-05-30T17:07:09.328 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_s3_kms ERROR [ 86%] 2024-05-30T17:07:10.279 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_bad_enc_kms PASSED [ 86%] 2024-05-30T17:07:10.279 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_bad_enc_kms ERROR [ 86%] 2024-05-30T17:07:11.588 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_kms_noenc PASSED [ 86%] 2024-05-30T17:07:11.588 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_kms_noenc ERROR [ 86%] 2024-05-30T17:07:12.651 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_kms_s3 PASSED [ 86%] 2024-05-30T17:07:12.651 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_kms_s3 ERROR [ 86%] 2024-05-30T17:07:14.848 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_acl_existing_tag PASSED [ 86%] 2024-05-30T17:07:14.848 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_acl_existing_tag ERROR [ 86%] 2024-05-30T17:07:16.034 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock PASSED [ 86%] 2024-05-30T17:07:16.034 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock ERROR [ 86%] 2024-05-30T17:07:17.112 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_bucket PASSED [ 86%] 2024-05-30T17:07:17.112 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_bucket ERROR [ 86%] 2024-05-30T17:07:18.096 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_with_days_and_years PASSED [ 87%] 2024-05-30T17:07:18.097 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_with_days_and_years ERROR [ 87%] 2024-05-30T17:07:19.044 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_days PASSED [ 87%] 2024-05-30T17:07:19.044 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_days ERROR [ 87%] 2024-05-30T17:07:20.055 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_years PASSED [ 87%] 2024-05-30T17:07:20.055 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_years ERROR [ 87%] 2024-05-30T17:07:21.206 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_mode PASSED [ 87%] 2024-05-30T17:07:21.207 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_mode ERROR [ 87%] 2024-05-30T17:07:22.179 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_status PASSED [ 87%] 2024-05-30T17:07:22.179 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_status ERROR [ 87%] 2024-05-30T17:07:23.105 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_suspend_versioning PASSED [ 87%] 2024-05-30T17:07:23.106 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_suspend_versioning ERROR [ 87%] 2024-05-30T17:07:24.175 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_lock PASSED [ 87%] 2024-05-30T17:07:24.175 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_lock ERROR [ 87%] 2024-05-30T17:07:25.227 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_lock_invalid_bucket PASSED [ 87%] 2024-05-30T17:07:25.227 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_lock_invalid_bucket ERROR [ 87%] 2024-05-30T17:07:26.715 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention PASSED [ 88%] 2024-05-30T17:07:26.715 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention ERROR [ 88%] 2024-05-30T17:07:27.894 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_invalid_bucket PASSED [ 88%] 2024-05-30T17:07:27.894 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_invalid_bucket ERROR [ 88%] 2024-05-30T17:07:29.276 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_invalid_mode PASSED [ 88%] 2024-05-30T17:07:29.276 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_invalid_mode ERROR [ 88%] 2024-05-30T17:07:30.697 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention PASSED [ 88%] 2024-05-30T17:07:30.697 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention ERROR [ 88%] 2024-05-30T17:07:32.092 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention_iso8601 PASSED [ 88%] 2024-05-30T17:07:32.092 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention_iso8601 ERROR [ 88%] 2024-05-30T17:07:33.284 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention_invalid_bucket PASSED [ 88%] 2024-05-30T17:07:33.284 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention_invalid_bucket ERROR [ 88%] 2024-05-30T17:07:34.956 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_versionid PASSED [ 88%] 2024-05-30T17:07:34.957 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_versionid ERROR [ 88%] 2024-05-30T17:07:36.527 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_override_default_retention PASSED [ 89%] 2024-05-30T17:07:36.527 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_override_default_retention ERROR [ 89%] 2024-05-30T17:07:38.095 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_increase_period PASSED [ 89%] 2024-05-30T17:07:38.095 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_increase_period ERROR [ 89%] 2024-05-30T17:07:39.530 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_shorten_period PASSED [ 89%] 2024-05-30T17:07:39.530 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_shorten_period ERROR [ 89%] 2024-05-30T17:07:41.085 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_shorten_period_bypass PASSED [ 89%] 2024-05-30T17:07:41.085 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_shorten_period_bypass ERROR [ 89%] 2024-05-30T17:07:42.448 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_retention PASSED [ 89%] 2024-05-30T17:07:42.448 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_retention ERROR [ 89%] 2024-05-30T17:07:44.053 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_multipart_object_with_retention PASSED [ 89%] 2024-05-30T17:07:44.053 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_multipart_object_with_retention ERROR [ 89%] 2024-05-30T17:07:45.881 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_retention_and_marker PASSED [ 89%] 2024-05-30T17:07:45.882 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_retention_and_marker ERROR [ 89%] 2024-05-30T17:07:47.484 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_multi_delete_object_with_retention PASSED [ 90%] 2024-05-30T17:07:47.484 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_multi_delete_object_with_retention ERROR [ 90%] 2024-05-30T17:07:48.915 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold PASSED [ 90%] 2024-05-30T17:07:48.915 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold ERROR [ 90%] 2024-05-30T17:07:50.095 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold_invalid_bucket PASSED [ 90%] 2024-05-30T17:07:50.096 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold_invalid_bucket ERROR [ 90%] 2024-05-30T17:07:51.530 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold_invalid_status PASSED [ 90%] 2024-05-30T17:07:51.531 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold_invalid_status ERROR [ 90%] 2024-05-30T17:07:53.161 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_legal_hold PASSED [ 90%] 2024-05-30T17:07:53.161 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_legal_hold ERROR [ 90%] 2024-05-30T17:07:54.347 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_legal_hold_invalid_bucket PASSED [ 90%] 2024-05-30T17:07:54.347 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_legal_hold_invalid_bucket ERROR [ 90%] 2024-05-30T17:07:55.901 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_legal_hold_on PASSED [ 90%] 2024-05-30T17:07:55.901 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_legal_hold_on ERROR [ 90%] 2024-05-30T17:07:57.670 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_multipart_object_with_legal_hold_on PASSED [ 90%] 2024-05-30T17:07:57.670 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_multipart_object_with_legal_hold_on ERROR [ 90%] 2024-05-30T17:07:58.912 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_legal_hold_off PASSED [ 91%] 2024-05-30T17:07:58.912 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_legal_hold_off ERROR [ 91%] 2024-05-30T17:08:00.513 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_metadata PASSED [ 91%] 2024-05-30T17:08:00.513 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_metadata ERROR [ 91%] 2024-05-30T17:08:01.861 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_uploading_obj PASSED [ 91%] 2024-05-30T17:08:01.861 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_uploading_obj ERROR [ 91%] 2024-05-30T17:08:12.790 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_governance_with_bypass PASSED [ 91%] 2024-05-30T17:08:12.790 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_governance_with_bypass ERROR [ 91%] 2024-05-30T17:08:14.118 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_governance_without_bypass PASSED [ 91%] 2024-05-30T17:08:14.118 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_governance_without_bypass ERROR [ 91%] 2024-05-30T17:08:24.717 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_compliance PASSED [ 91%] 2024-05-30T17:08:24.717 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_compliance ERROR [ 91%] 2024-05-30T17:08:26.138 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_copy_object_ifmatch_good PASSED [ 91%] 2024-05-30T17:08:26.139 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_copy_object_ifmatch_good ERROR [ 91%] 2024-05-30T17:08:27.605 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_copy_object_ifnonematch_failed PASSED [ 92%] 2024-05-30T17:08:27.605 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_copy_object_ifnonematch_failed ERROR [ 92%] 2024-05-30T17:08:28.480 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_bucket_policy_status PASSED [ 92%] 2024-05-30T17:08:28.480 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_bucket_policy_status ERROR [ 92%] 2024-05-30T17:08:29.455 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_public_acl_bucket_policy_status PASSED [ 92%] 2024-05-30T17:08:29.456 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_public_acl_bucket_policy_status ERROR [ 92%] 2024-05-30T17:08:30.495 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_authpublic_acl_bucket_policy_status PASSED [ 92%] 2024-05-30T17:08:30.496 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_authpublic_acl_bucket_policy_status ERROR [ 92%] 2024-05-30T17:08:31.661 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_publicpolicy_acl_bucket_policy_status PASSED [ 92%] 2024-05-30T17:08:31.662 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_publicpolicy_acl_bucket_policy_status ERROR [ 92%] 2024-05-30T17:08:32.870 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_nonpublicpolicy_acl_bucket_policy_status PASSED [ 92%] 2024-05-30T17:08:32.870 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_nonpublicpolicy_acl_bucket_policy_status ERROR [ 92%] 2024-05-30T17:08:33.995 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_nonpublicpolicy_deny_bucket_policy_status PASSED [ 92%] 2024-05-30T17:08:33.996 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_nonpublicpolicy_deny_bucket_policy_status ERROR [ 92%] 2024-05-30T17:08:35.047 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_undefined_public_block PASSED [ 93%] 2024-05-30T17:08:35.047 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_undefined_public_block ERROR [ 93%] 2024-05-30T17:08:36.375 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_public_block_deny_bucket_policy PASSED [ 93%] 2024-05-30T17:08:36.375 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_public_block_deny_bucket_policy ERROR [ 93%] 2024-05-30T17:08:37.488 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_public_block PASSED [ 93%] 2024-05-30T17:08:37.488 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_public_block ERROR [ 93%] 2024-05-30T17:08:38.834 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_block_public_put_bucket_acls PASSED [ 93%] 2024-05-30T17:08:38.834 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_block_public_put_bucket_acls ERROR [ 93%] 2024-05-30T17:08:40.065 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_block_public_object_canned_acls PASSED [ 93%] 2024-05-30T17:08:40.066 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_block_public_object_canned_acls ERROR [ 93%] 2024-05-30T17:08:41.218 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_block_public_policy PASSED [ 93%] 2024-05-30T17:08:41.218 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_block_public_policy ERROR [ 93%] 2024-05-30T17:08:42.924 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_ignore_public_acls PASSED [ 93%] 2024-05-30T17:08:42.925 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_ignore_public_acls ERROR [ 93%] 2024-05-30T17:08:56.851 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_on_a_bucket_with_policy PASSED [ 93%] 2024-05-30T17:08:56.851 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_on_a_bucket_with_policy ERROR [ 93%] 2024-05-30T17:08:57.754 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_bucket_encryption_kms PASSED [ 94%] 2024-05-30T17:08:57.754 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_put_bucket_encryption_kms ERROR [ 94%] 2024-05-30T17:08:58.836 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_bucket_encryption_kms PASSED [ 94%] 2024-05-30T17:08:58.836 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_bucket_encryption_kms ERROR [ 94%] 2024-05-30T17:09:00.035 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_delete_bucket_encryption_kms PASSED [ 94%] 2024-05-30T17:09:00.036 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_delete_bucket_encryption_kms ERROR [ 94%] 2024-05-30T17:09:01.257 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_default_post_object_authenticated_request PASSED [ 94%] 2024-05-30T17:09:01.257 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_default_post_object_authenticated_request ERROR [ 94%] 2024-05-30T17:09:03.844 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_torrent PASSED [ 94%] 2024-05-30T17:09:03.844 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_torrent ERROR [ 94%] 2024-05-30T17:09:16.811 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_generate_where_clause PASSED [ 94%] 2024-05-30T17:09:27.863 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_generate_projection PASSED [ 94%] 2024-05-30T17:09:29.102 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_count_operation PASSED [ 95%] 2024-05-30T17:09:30.711 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_count_json_operation PASSED [ 95%] 2024-05-30T17:09:47.759 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_json_column_sum_min_max PASSED [ 95%] 2024-05-30T17:10:08.215 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_json_nullif_expressions PASSED [ 95%] 2024-05-30T17:10:13.038 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_column_sum_min_max PASSED [ 95%] 2024-05-30T17:10:16.083 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_nullif_expressions PASSED [ 95%] 2024-05-30T17:10:23.691 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_nulliftrue_expressions PASSED [ 95%] 2024-05-30T17:10:26.358 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_is_not_null_expressions PASSED [ 96%] 2024-05-30T17:10:27.462 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_lowerupper_expressions PASSED [ 96%] 2024-05-30T17:10:42.039 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_in_expressions PASSED [ 96%] 2024-05-30T17:10:48.398 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_true_false_in_expressions PASSED [ 96%] 2024-05-30T17:11:07.706 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_like_expressions PASSED [ 96%] 2024-05-30T17:11:12.075 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_truefalselike_expressions PASSED [ 96%] 2024-05-30T17:11:14.854 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_complex_expressions PASSED [ 96%] 2024-05-30T17:11:20.074 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_alias PASSED [ 96%] 2024-05-30T17:11:21.427 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_alias_cyclic_refernce PASSED [ 97%] 2024-05-30T17:11:29.547 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_datetime PASSED [ 97%] 2024-05-30T17:11:33.831 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_true_false_datetime PASSED [ 97%] 2024-05-30T17:11:35.387 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_csv_parser PASSED [ 97%] 2024-05-30T17:11:37.334 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_csv_definition PASSED [ 97%] 2024-05-30T17:11:44.955 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_schema_definition PASSED [ 97%] 2024-05-30T17:11:49.531 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_when_then_else_expressions PASSED [ 97%] 2024-05-30T17:11:55.727 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_coalesce_expressions PASSED [ 98%] 2024-05-30T17:11:58.298 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_cast_expressions PASSED [ 98%] 2024-05-30T17:11:58.839 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_version PASSED [ 98%] 2024-05-30T17:12:14.942 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_trim_expressions PASSED [ 98%] 2024-05-30T17:12:20.171 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_truefalse_trim_expressions PASSED [ 98%] 2024-05-30T17:12:28.063 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_escape_expressions PASSED [ 98%] 2024-05-30T17:12:33.667 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_case_value_expressions PASSED [ 98%] 2024-05-30T17:12:35.670 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_bool_cast_expressions PASSED [ 99%] 2024-05-30T17:13:13.654 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_progress_expressions PASSED [ 99%] 2024-05-30T17:13:14.295 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3select.py::test_output_serial_expressions PASSED [ 99%] 2024-05-30T17:13:16.238 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_sns.py::test_account_topic PASSED [ 99%] 2024-05-30T17:13:17.659 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_sns.py::test_cross_account_topic PASSED [ 99%] 2024-05-30T17:13:19.467 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_sns.py::test_account_topic_publish PASSED [ 99%] 2024-05-30T17:13:21.507 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_sns.py::test_cross_account_topic_publish PASSED [ 99%] 2024-05-30T17:13:21.550 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_utils.py::test_generate PASSED [100%] 2024-05-30T17:13:21.550 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.550 INFO:teuthology.orchestra.run.smithi006.stdout:==================================== ERRORS ==================================== 2024-05-30T17:13:21.550 INFO:teuthology.orchestra.run.smithi006.stdout:__________ ERROR at teardown of test_object_write_to_nonexist_bucket ___________ 2024-05-30T17:13:21.550 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.551 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.551 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.551 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.551 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.551 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.551 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.551 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.551 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.551 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.551 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.551 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.551 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.552 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.552 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.552 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.552 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.552 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.552 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.552 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.552 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.552 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.552 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.552 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.553 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.553 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.553 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.553 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.553 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.553 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.553 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.553 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.553 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.553 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.553 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.553 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.554 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.554 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.554 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.554 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.554 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.554 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.554 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.554 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.554 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.554 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.554 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.554 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.555 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.555 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.555 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.555 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.555 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.555 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.555 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.555 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.555 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.555 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.555 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.555 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.556 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.556 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.556 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.556 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.556 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.556 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.556 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.556 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.556 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.556 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.556 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.556 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.557 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.557 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.557 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.557 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.557 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.557 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.557 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.557 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.557 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.557 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.557 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.557 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.557 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.558 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.558 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.558 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.558 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.558 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.558 INFO:teuthology.orchestra.run.smithi006.stdout:____ ERROR at teardown of test_object_write_with_chunked_transfer_encoding _____ 2024-05-30T17:13:21.558 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.558 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.558 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.558 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.558 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.558 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.559 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.559 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.559 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.559 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.559 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.559 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.559 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.559 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.559 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.559 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.559 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.559 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.559 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.560 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.560 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.560 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.560 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.560 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.560 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.560 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.560 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.560 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.560 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.560 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.560 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.561 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.561 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.561 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.561 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.561 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.561 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.561 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.561 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.561 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.561 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.561 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.561 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.561 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.561 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.561 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.562 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.562 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.562 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.562 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.562 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.562 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.562 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.562 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.562 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.562 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.562 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.562 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.563 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.563 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.563 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.563 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.563 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.563 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.563 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.563 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.563 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.563 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.563 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.563 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.563 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.563 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.564 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.564 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.564 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.564 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.564 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.564 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.564 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.564 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.564 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.564 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.564 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.564 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.564 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.565 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.565 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.565 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.565 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.565 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.565 INFO:teuthology.orchestra.run.smithi006.stdout:________________ ERROR at teardown of test_bucket_create_delete ________________ 2024-05-30T17:13:21.565 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.565 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.565 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.565 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.565 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.565 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.565 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.566 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.566 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.566 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.566 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.566 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.568 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.568 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.568 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.568 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.568 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.569 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.569 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.569 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.569 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.569 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.569 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.569 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.569 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.569 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.569 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.569 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.569 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.569 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.570 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.570 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.570 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.570 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.570 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.570 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.570 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.570 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.570 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.570 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.570 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.570 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.571 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.571 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.571 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.571 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.571 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.571 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.571 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.571 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.571 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.571 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.571 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.571 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.571 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.572 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.572 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.572 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.572 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.572 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.572 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.572 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.572 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.572 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.572 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.572 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.572 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.572 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.573 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.573 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.573 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.573 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.573 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.573 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.573 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.573 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.573 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.573 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.573 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.573 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.573 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.573 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.574 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.574 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.574 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.574 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.574 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.574 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.574 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.574 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.574 INFO:teuthology.orchestra.run.smithi006.stdout:_______________ ERROR at teardown of test_object_read_not_exist ________________ 2024-05-30T17:13:21.574 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.574 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.574 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.575 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.575 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.575 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.575 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.575 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.575 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.575 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.575 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.575 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.575 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.575 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.575 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.575 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.576 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.576 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.576 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.576 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.576 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.576 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.576 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.576 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.576 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.576 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.576 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.576 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.576 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.577 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.577 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.577 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.577 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.577 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.577 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.577 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.577 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.577 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.577 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.577 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.577 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.577 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.578 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.578 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.578 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.578 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.578 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.578 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.578 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.578 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.578 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.578 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.578 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.578 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.578 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.578 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.579 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.579 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.579 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.579 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.579 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.579 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.579 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.579 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.579 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.579 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.579 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.579 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.579 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.579 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.580 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.580 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.580 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.580 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.580 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.580 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.580 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.580 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.580 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.580 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.580 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.580 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.580 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.580 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.581 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.581 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.581 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.581 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.581 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.581 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.581 INFO:teuthology.orchestra.run.smithi006.stdout:______ ERROR at teardown of test_object_requestid_matches_header_on_error ______ 2024-05-30T17:13:21.581 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.581 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.581 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.581 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.581 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.581 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.582 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.582 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.582 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.582 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.582 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.582 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.582 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.582 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.582 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.582 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.582 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.582 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.582 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.583 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.583 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.583 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.583 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.583 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.583 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.583 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.583 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.583 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.583 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.583 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.583 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.583 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.583 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.584 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.584 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.584 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.584 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.584 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.584 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.584 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.584 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.584 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.584 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.584 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.584 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.584 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.585 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.585 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.585 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.585 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.585 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.585 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.585 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.585 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.585 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.585 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.585 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.585 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.585 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.586 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.586 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.586 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.586 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.586 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.586 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.586 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.586 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.586 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.586 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.586 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.586 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.586 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.587 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.587 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.587 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.587 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.587 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.587 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.587 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.587 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.587 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.587 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.587 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.587 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.587 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.588 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.588 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.588 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.588 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.588 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.588 INFO:teuthology.orchestra.run.smithi006.stdout:_____ ERROR at teardown of test_versioning_concurrent_multi_object_delete ______ 2024-05-30T17:13:21.588 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.588 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.588 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.588 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.588 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.588 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.589 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.589 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.589 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.589 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.589 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.589 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.589 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.589 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.589 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.589 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.589 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.589 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.589 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.590 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.590 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.590 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.590 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.590 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.590 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.590 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.590 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.590 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.590 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.590 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.591 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.591 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.591 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.591 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.591 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.591 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.591 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.591 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.591 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.591 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.591 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.591 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.591 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.591 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.592 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.592 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.592 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.592 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.592 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.592 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.592 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.592 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.592 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.592 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.592 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.592 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.592 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.593 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.593 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.593 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.593 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.593 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.593 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.593 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.593 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.593 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.593 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.593 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.593 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.593 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.594 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.594 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.594 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.594 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.594 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.594 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.594 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.594 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.594 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.594 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.594 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.594 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.594 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.595 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.595 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.595 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.595 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.595 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.595 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.595 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.595 INFO:teuthology.orchestra.run.smithi006.stdout:________________ ERROR at teardown of test_multi_object_delete _________________ 2024-05-30T17:13:21.595 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.595 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.595 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.595 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.595 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.596 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.596 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.596 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.596 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.596 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.596 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.596 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.596 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.596 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.596 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.596 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.596 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.596 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.597 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.597 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.597 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.597 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.597 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.597 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.597 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.597 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.597 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.597 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.597 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.597 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.597 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.598 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.598 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.598 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.598 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.598 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.598 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.598 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.598 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.598 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.598 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.598 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.598 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.599 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.599 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.599 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.599 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.599 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.599 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.599 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.599 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.599 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.599 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.599 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.600 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.600 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.600 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.600 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.600 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.600 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.600 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.600 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.600 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.600 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.600 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.600 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.600 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.601 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.601 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.601 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.601 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.601 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.601 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.601 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.601 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.601 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.601 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.601 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.601 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.601 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.602 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.602 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.602 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.602 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.602 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.602 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.602 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.602 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.602 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.602 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.602 INFO:teuthology.orchestra.run.smithi006.stdout:_______________ ERROR at teardown of test_multi_objectv2_delete ________________ 2024-05-30T17:13:21.602 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.603 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.603 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.603 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.603 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.603 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.603 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.603 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.603 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.603 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.603 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.603 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.603 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.603 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.604 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.604 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.604 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.604 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.604 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.604 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.604 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.604 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.604 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.604 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.604 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.604 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.604 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.605 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.605 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.605 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.605 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.605 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.605 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.605 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.605 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.605 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.605 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.605 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.605 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.606 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.606 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.606 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.606 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.606 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.606 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.606 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.606 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.606 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.606 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.606 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.606 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.606 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.607 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.607 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.607 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.607 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.607 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.607 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.607 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.607 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.607 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.607 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.607 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.607 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.607 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.608 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.608 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.608 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.608 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.608 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.608 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.608 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.608 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.608 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.608 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.608 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.608 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.608 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.608 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.608 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.609 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.609 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.609 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.609 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.609 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.609 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.609 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.609 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.609 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.609 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.609 INFO:teuthology.orchestra.run.smithi006.stdout:___________ ERROR at teardown of test_multi_object_delete_key_limit ____________ 2024-05-30T17:13:21.609 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.609 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.609 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.610 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.610 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.610 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.610 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.610 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.610 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.610 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.610 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.610 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.610 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.610 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.610 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.610 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.610 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.611 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.611 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.611 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.611 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.611 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.611 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.611 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.611 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.611 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.611 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.611 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.611 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.611 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.611 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.612 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.612 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.612 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.612 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.612 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.612 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.612 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.612 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.612 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.612 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.612 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.612 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.612 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.612 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.612 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.613 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.613 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.613 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.613 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.613 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.613 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.613 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.613 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.613 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.613 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.613 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.613 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.614 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.614 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.614 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.614 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.614 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.614 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.614 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.614 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.614 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.614 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.614 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.614 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.614 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.614 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.615 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.615 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.615 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.615 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.615 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.615 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.615 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.615 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.615 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.615 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.615 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.615 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.615 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.615 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.616 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.616 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.616 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.616 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.616 INFO:teuthology.orchestra.run.smithi006.stdout:__________ ERROR at teardown of test_multi_objectv2_delete_key_limit ___________ 2024-05-30T17:13:21.616 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.616 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.616 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.616 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.616 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.616 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.616 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.616 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.616 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.617 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.617 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.617 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.617 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.617 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.617 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.617 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.617 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.617 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.617 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.617 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.617 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.617 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.617 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.618 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.618 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.618 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.618 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.618 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.618 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.618 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.618 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.618 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.618 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.619 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.619 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.619 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.619 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.619 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.619 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.619 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.619 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.619 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.619 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.619 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.619 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.619 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.619 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.620 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.620 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.620 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.620 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.620 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.620 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.620 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.620 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.620 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.620 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.620 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.620 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.620 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.621 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.621 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.621 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.621 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.621 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.621 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.621 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.621 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.621 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.621 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.621 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.621 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.621 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.621 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.622 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.622 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.622 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.622 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.622 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.622 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.622 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.622 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.622 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.622 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.622 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.622 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.622 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.622 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.623 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.623 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.623 INFO:teuthology.orchestra.run.smithi006.stdout:_______________ ERROR at teardown of test_object_head_zero_bytes _______________ 2024-05-30T17:13:21.623 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.623 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.623 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.623 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.623 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.623 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.623 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.623 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.623 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.623 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.623 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.624 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.624 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.624 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.624 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.624 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.624 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.624 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.624 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.624 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.624 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.624 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.624 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.624 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.625 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.625 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.625 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.625 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.625 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.625 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.625 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.625 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.625 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.625 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.625 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.625 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.625 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.626 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.626 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.626 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.626 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.626 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.626 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.626 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.626 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.626 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.626 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.626 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.626 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.626 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.626 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.627 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.627 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.627 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.627 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.627 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.627 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.627 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.627 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.627 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.627 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.627 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.627 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.627 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.627 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.627 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.628 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.628 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.628 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.628 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.628 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.628 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.628 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.628 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.628 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.628 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.628 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.628 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.628 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.628 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.629 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.629 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.629 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.629 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.629 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.629 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.629 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.629 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.629 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.629 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.629 INFO:teuthology.orchestra.run.smithi006.stdout:______________ ERROR at teardown of test_object_write_check_etag _______________ 2024-05-30T17:13:21.629 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.629 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.629 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.629 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.630 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.630 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.630 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.630 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.630 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.630 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.630 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.630 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.630 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.630 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.630 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.630 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.630 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.630 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.631 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.631 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.631 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.631 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.631 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.631 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.631 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.631 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.631 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.631 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.631 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.631 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.631 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.631 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.631 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.632 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.632 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.632 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.632 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.632 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.632 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.632 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.632 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.632 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.632 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.632 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.632 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.632 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.632 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.633 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.633 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.633 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.633 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.633 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.633 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.633 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.633 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.633 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.633 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.633 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.633 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.633 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.633 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.634 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.634 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.634 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.634 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.634 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.634 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.634 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.634 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.634 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.634 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.634 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.634 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.634 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.635 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.635 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.635 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.635 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.635 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.635 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.635 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.635 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.635 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.635 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.635 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.635 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.635 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.635 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.636 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.636 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.636 INFO:teuthology.orchestra.run.smithi006.stdout:_____________ ERROR at teardown of test_object_write_cache_control _____________ 2024-05-30T17:13:21.636 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.636 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.636 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.636 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.636 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.636 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.636 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.636 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.636 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.636 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.636 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.636 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.637 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.637 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.637 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.637 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.637 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.637 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.637 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.637 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.637 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.637 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.637 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.637 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.637 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.638 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.638 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.638 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.638 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.638 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.638 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.638 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.638 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.638 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.638 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.638 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.638 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.638 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.638 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.639 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.639 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.639 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.639 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.639 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.639 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.639 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.639 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.639 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.639 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.639 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.639 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.639 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.640 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.640 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.640 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.640 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.640 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.640 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.640 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.640 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.640 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.640 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.640 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.640 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.640 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.640 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.640 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.641 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.641 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.641 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.641 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.641 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.641 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.641 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.641 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.641 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.641 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.641 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.641 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.641 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.641 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.642 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.642 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.642 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.642 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.642 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.642 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.642 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.642 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.642 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.642 INFO:teuthology.orchestra.run.smithi006.stdout:________________ ERROR at teardown of test_object_write_expires ________________ 2024-05-30T17:13:21.642 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.642 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.642 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.642 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.643 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.643 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.643 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.643 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.643 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.643 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.643 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.643 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.643 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.643 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.643 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.643 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.643 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.643 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.643 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.644 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.644 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.644 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.644 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.644 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.644 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.644 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.644 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.644 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.644 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.644 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.644 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.644 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.644 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.645 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.645 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.645 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.645 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.645 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.645 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.645 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.645 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.645 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.645 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.645 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.645 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.645 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.645 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.646 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.646 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.646 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.646 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.646 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.646 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.646 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.646 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.646 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.646 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.646 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.646 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.646 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.646 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.647 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.647 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.647 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.647 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.647 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.647 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.647 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.647 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.647 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.647 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.647 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.647 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.647 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.647 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.647 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.648 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.648 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.648 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.648 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.648 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.648 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.648 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.648 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.648 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.648 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.648 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.648 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.648 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.649 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.649 INFO:teuthology.orchestra.run.smithi006.stdout:________ ERROR at teardown of test_object_write_read_update_read_delete ________ 2024-05-30T17:13:21.649 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.649 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.649 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.649 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.649 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.649 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.649 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.649 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.649 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.649 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.649 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.649 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.650 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.650 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.650 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.650 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.650 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.650 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.650 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.650 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.650 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.650 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.650 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.650 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.650 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.651 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.651 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.651 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.651 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.651 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.651 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.651 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.651 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.651 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.651 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.651 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.651 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.651 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.652 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.652 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.652 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.652 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.652 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.652 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.652 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.652 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.652 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.652 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.652 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.652 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.652 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.653 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.653 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.653 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.653 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.653 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.653 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.653 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.653 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.653 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.653 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.653 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.653 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.653 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.653 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.654 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.654 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.654 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.654 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.654 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.654 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.654 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.654 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.654 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.654 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.654 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.654 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.654 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.655 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.655 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.655 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.655 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.655 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.655 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.655 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.655 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.655 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.655 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.655 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.655 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.655 INFO:teuthology.orchestra.run.smithi006.stdout:________ ERROR at teardown of test_object_set_get_metadata_none_to_good ________ 2024-05-30T17:13:21.656 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.656 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.656 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.656 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.656 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.656 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.656 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.656 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.656 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.656 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.656 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.656 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.656 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.656 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.657 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.657 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.657 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.657 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.657 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.657 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.657 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.657 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.657 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.657 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.657 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.657 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.657 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.657 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.657 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.658 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.658 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.658 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.658 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.658 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.658 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.658 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.658 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.658 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.658 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.658 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.658 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.658 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.659 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.659 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.659 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.659 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.659 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.659 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.659 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.659 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.659 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.659 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.659 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.659 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.659 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.659 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.660 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.660 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.660 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.660 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.660 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.660 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.660 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.660 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.660 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.660 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.660 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.660 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.660 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.660 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.661 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.661 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.661 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.661 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.661 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.661 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.661 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.661 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.661 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.661 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.661 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.661 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.661 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.661 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.662 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.662 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.662 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.662 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.662 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.662 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.662 INFO:teuthology.orchestra.run.smithi006.stdout:_______ ERROR at teardown of test_object_set_get_metadata_none_to_empty ________ 2024-05-30T17:13:21.662 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.662 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.662 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.662 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.662 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.662 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.663 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.663 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.663 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.663 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.663 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.663 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.663 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.663 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.663 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.663 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.663 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.663 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.663 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.663 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.664 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.664 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.664 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.664 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.664 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.664 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.664 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.664 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.664 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.664 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.664 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.664 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.664 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.664 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.665 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.665 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.665 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.665 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.665 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.665 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.665 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.665 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.665 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.665 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.665 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.665 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.665 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.666 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.666 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.666 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.666 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.666 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.666 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.666 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.666 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.666 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.666 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.666 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.666 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.666 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.666 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.667 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.667 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.667 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.667 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.667 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.667 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.667 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.667 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.667 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.667 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.667 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.667 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.667 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.667 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.668 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.668 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.668 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.668 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.668 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.668 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.668 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.668 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.668 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.668 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.668 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.668 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.668 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.668 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.669 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.669 INFO:teuthology.orchestra.run.smithi006.stdout:_____ ERROR at teardown of test_object_set_get_metadata_overwrite_to_empty _____ 2024-05-30T17:13:21.669 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.669 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.669 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.669 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.669 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.669 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.669 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.669 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.669 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.669 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.669 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.670 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.670 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.670 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.670 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.670 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.670 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.670 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.670 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.670 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.670 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.670 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.670 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.670 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.671 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.671 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.671 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.671 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.671 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.671 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.671 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.671 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.671 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.671 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.671 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.671 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.671 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.671 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.671 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.672 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.672 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.672 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.672 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.672 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.672 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.672 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.672 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.672 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.672 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.672 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.672 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.672 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.672 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.672 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.673 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.673 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.673 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.673 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.673 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.673 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.673 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.673 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.673 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.673 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.673 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.673 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.673 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.673 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.673 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.674 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.674 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.674 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.674 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.674 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.674 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.674 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.674 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.674 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.674 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.674 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.674 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.674 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.674 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.675 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.675 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.675 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.675 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.675 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.675 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.675 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.675 INFO:teuthology.orchestra.run.smithi006.stdout:__________ ERROR at teardown of test_object_metadata_replaced_on_put ___________ 2024-05-30T17:13:21.675 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.675 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.675 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.675 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.675 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.675 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.675 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.676 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.676 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.676 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.676 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.676 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.676 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.676 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.676 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.676 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.676 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.676 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.676 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.676 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.676 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.676 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.677 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.677 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.677 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.677 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.677 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.677 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.677 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.677 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.677 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.677 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.677 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.677 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.677 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.677 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.677 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.678 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.678 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.678 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.678 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.678 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.678 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.678 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.678 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.678 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.678 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.678 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.678 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.678 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.678 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.679 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.679 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.679 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.679 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.679 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.679 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.679 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.679 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.679 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.679 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.679 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.679 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.679 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.679 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.679 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.679 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.680 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.680 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.680 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.680 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.680 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.680 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.680 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.680 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.680 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.680 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.680 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.680 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.680 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.680 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.680 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.681 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.681 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.681 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.681 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.681 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.681 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.681 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.681 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.681 INFO:teuthology.orchestra.run.smithi006.stdout:_________________ ERROR at teardown of test_object_write_file __________________ 2024-05-30T17:13:21.681 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.681 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.681 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.681 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.681 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.681 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.682 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.682 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.682 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.682 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.682 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.682 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.682 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.682 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.682 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.682 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.682 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.682 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.682 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.682 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.683 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.683 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.683 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.683 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.683 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.683 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.683 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.683 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.683 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.683 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.683 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.683 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.683 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.684 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.684 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.684 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.684 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.684 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.684 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.684 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.684 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.684 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.684 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.684 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.684 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.684 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.684 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.684 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.684 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.685 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.685 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.685 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.685 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.685 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.685 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.685 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.685 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.685 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.685 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.685 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.685 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.685 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.685 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.685 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.686 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.686 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.686 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.686 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.686 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.686 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.686 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.686 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.686 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.686 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.686 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.686 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.686 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.686 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.687 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.687 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.687 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.687 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.687 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.687 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.687 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.687 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.687 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.687 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.687 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.687 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.687 INFO:teuthology.orchestra.run.smithi006.stdout:___________ ERROR at teardown of test_post_object_anonymous_request ____________ 2024-05-30T17:13:21.687 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.687 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.688 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.688 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.688 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.688 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.688 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.688 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.688 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.688 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.688 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.688 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.688 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.688 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.688 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.688 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.688 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.689 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.689 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.689 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.689 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.689 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.689 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.689 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.689 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.689 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.689 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.689 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.689 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.689 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.689 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.689 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.689 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.690 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.690 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.690 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.690 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.690 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.690 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.690 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.690 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.690 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.690 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.690 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.690 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.690 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.691 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.691 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.691 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.691 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.691 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.691 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.691 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.691 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.691 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.691 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.691 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.691 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.691 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.691 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.691 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.692 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.692 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.692 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.692 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.692 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.692 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.692 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.692 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.692 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.692 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.692 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.692 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.692 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.692 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.692 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.692 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.693 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.693 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.693 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.693 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.693 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.693 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.693 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.693 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.693 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.693 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.693 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.693 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.693 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.693 INFO:teuthology.orchestra.run.smithi006.stdout:_________ ERROR at teardown of test_post_object_authenticated_request __________ 2024-05-30T17:13:21.693 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.694 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.694 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.694 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.694 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.694 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.694 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.694 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.694 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.694 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.694 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.694 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.694 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.694 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.694 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.694 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.695 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.695 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.695 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.695 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.695 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.695 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.695 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.695 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.695 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.695 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.695 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.695 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.695 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.695 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.695 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.696 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.696 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.696 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.696 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.696 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.696 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.696 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.696 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.696 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.696 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.696 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.696 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.696 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.696 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.697 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.697 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.697 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.697 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.697 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.697 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.697 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.697 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.697 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.697 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.697 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.697 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.697 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.697 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.697 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.697 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.698 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.698 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.698 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.698 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.698 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.698 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.698 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.698 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.698 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.698 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.698 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.698 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.698 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.699 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.699 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.699 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.699 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.699 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.699 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.699 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.699 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.699 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.699 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.699 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.699 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.699 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.699 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.699 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.700 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.700 INFO:teuthology.orchestra.run.smithi006.stdout:_____ ERROR at teardown of test_post_object_authenticated_no_content_type ______ 2024-05-30T17:13:21.700 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.700 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.700 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.700 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.700 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.700 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.700 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.700 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.700 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.700 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.700 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.700 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.700 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.700 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.701 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.701 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.701 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.701 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.701 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.701 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.701 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.701 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.701 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.701 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.701 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.701 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.701 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.701 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.701 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.702 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.702 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.702 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.702 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.702 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.702 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.702 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.702 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.702 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.702 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.702 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.702 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.702 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.702 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.703 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.703 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.703 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.703 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.703 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.703 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.703 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.703 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.703 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.703 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.703 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.703 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.703 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.703 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.704 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.704 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.704 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.704 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.704 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.704 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.704 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.704 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.704 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.704 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.704 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.704 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.704 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.704 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.705 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.705 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.705 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.705 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.705 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.705 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.705 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.705 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.705 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.705 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.705 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.705 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.705 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.705 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.706 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.706 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.706 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.706 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.706 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.706 INFO:teuthology.orchestra.run.smithi006.stdout:__ ERROR at teardown of test_post_object_authenticated_request_bad_access_key __ 2024-05-30T17:13:21.706 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.706 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.706 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.706 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.706 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.706 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.706 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.706 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.707 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.707 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.707 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.707 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.707 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.707 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.707 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.707 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.707 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.707 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.707 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.707 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.707 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.707 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.708 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.708 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.708 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.708 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.708 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.708 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.708 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.708 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.708 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.708 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.708 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.708 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.708 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.708 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.709 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.709 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.709 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.709 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.709 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.709 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.709 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.709 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.709 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.709 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.709 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.709 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.709 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.709 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.710 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.710 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.710 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.710 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.710 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.710 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.710 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.710 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.710 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.710 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.710 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.710 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.710 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.710 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.711 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.711 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.711 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.711 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.711 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.711 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.711 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.711 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.711 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.711 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.711 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.711 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.711 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.711 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.712 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.712 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.712 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.712 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.712 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.712 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.712 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.712 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.712 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.712 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.712 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.712 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.712 INFO:teuthology.orchestra.run.smithi006.stdout:____________ ERROR at teardown of test_post_object_set_success_code ____________ 2024-05-30T17:13:21.712 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.713 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.713 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.713 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.713 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.713 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.713 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.713 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.713 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.713 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.713 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.713 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.713 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.713 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.713 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.714 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.714 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.714 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.714 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.714 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.714 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.714 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.714 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.714 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.714 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.714 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.714 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.714 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.715 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.715 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.715 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.715 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.715 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.715 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.715 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.715 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.715 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.715 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.715 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.715 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.715 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.715 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.715 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.716 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.716 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.716 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.716 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.716 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.716 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.716 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.716 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.716 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.716 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.716 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.716 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.716 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.716 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.716 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.717 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.717 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.717 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.717 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.717 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.717 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.717 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.717 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.717 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.717 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.717 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.717 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.717 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.717 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.717 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.718 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.718 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.718 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.718 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.718 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.718 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.718 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.718 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.718 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.718 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.719 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.719 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.719 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.719 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.719 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.719 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.719 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.719 INFO:teuthology.orchestra.run.smithi006.stdout:________ ERROR at teardown of test_post_object_set_invalid_success_code ________ 2024-05-30T17:13:21.719 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.719 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.719 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.719 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.719 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.719 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.719 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.720 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.720 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.720 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.720 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.720 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.720 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.720 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.720 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.720 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.720 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.720 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.720 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.720 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.721 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.721 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.721 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.721 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.721 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.721 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.721 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.721 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.721 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.721 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.721 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.721 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.721 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.721 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.722 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.722 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.722 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.722 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.722 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.722 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.722 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.722 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.722 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.722 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.722 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.722 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.722 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.722 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.723 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.723 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.723 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.723 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.723 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.723 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.723 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.723 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.723 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.723 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.723 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.723 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.723 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.723 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.723 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.723 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.724 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.724 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.724 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.724 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.724 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.724 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.724 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.724 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.724 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.724 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.724 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.724 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.724 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.724 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.725 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.725 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.725 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.725 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.725 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.725 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.725 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.725 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.725 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.725 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.725 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.725 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.725 INFO:teuthology.orchestra.run.smithi006.stdout:________ ERROR at teardown of test_post_object_upload_larger_than_chunk ________ 2024-05-30T17:13:21.725 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.725 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.725 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.726 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.726 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.726 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.726 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.726 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.726 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.726 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.726 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.726 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.726 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.726 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.726 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.726 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.727 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.727 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.727 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.727 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.727 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.727 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.727 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.727 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.727 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.727 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.727 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.727 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.727 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.727 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.727 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.728 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.728 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.728 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.728 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.728 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.728 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.728 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.728 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.728 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.728 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.728 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.728 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.728 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.728 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.728 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.729 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.729 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.729 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.729 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.729 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.729 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.729 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.729 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.729 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.729 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.729 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.729 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.729 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.729 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.729 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.730 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.730 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.730 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.730 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.730 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.730 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.730 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.730 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.730 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.730 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.730 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.730 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.730 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.730 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.731 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.731 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.731 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.731 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.731 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.731 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.731 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.731 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.731 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.731 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.731 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.731 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.731 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.731 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.732 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.732 INFO:teuthology.orchestra.run.smithi006.stdout:_________ ERROR at teardown of test_post_object_set_key_from_filename __________ 2024-05-30T17:13:21.732 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.732 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.732 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.732 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.732 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.732 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.732 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.732 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.732 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.732 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.732 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.732 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.732 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.733 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.733 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.733 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.733 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.733 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.733 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.733 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.733 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.733 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.733 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.733 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.733 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.733 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.733 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.734 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.734 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.734 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.734 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.734 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.734 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.734 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.734 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.734 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.734 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.734 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.734 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.734 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.734 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.735 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.735 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.735 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.735 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.735 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.735 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.735 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.735 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.735 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.735 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.735 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.735 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.735 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.735 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.736 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.736 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.736 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.736 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.736 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.736 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.736 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.736 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.736 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.736 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.736 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.736 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.736 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.736 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.737 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.754 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.754 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.754 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.754 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.754 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.755 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.755 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.755 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.755 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.755 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.755 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.755 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.755 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.755 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.755 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.755 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.755 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.755 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.755 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.756 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.756 INFO:teuthology.orchestra.run.smithi006.stdout:_____________ ERROR at teardown of test_post_object_ignored_header _____________ 2024-05-30T17:13:21.756 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.756 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.756 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.756 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.756 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.756 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.756 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.756 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.756 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.756 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.757 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.757 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.757 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.757 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.757 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.757 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.757 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.757 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.757 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.757 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.757 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.757 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.757 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.757 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.757 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.758 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.758 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.758 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.758 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.758 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.758 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.758 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.758 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.758 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.758 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.758 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.758 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.758 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.758 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.759 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.759 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.759 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.759 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.759 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.759 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.759 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.759 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.759 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.759 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.759 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.759 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.759 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.759 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.759 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.760 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.760 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.760 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.760 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.760 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.760 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.760 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.760 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.760 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.760 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.760 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.760 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.760 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.760 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.760 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.761 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.761 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.761 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.761 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.761 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.761 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.761 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.761 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.761 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.761 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.761 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.761 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.761 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.761 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.762 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.762 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.762 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.762 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.762 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.762 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.762 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.762 INFO:teuthology.orchestra.run.smithi006.stdout:___ ERROR at teardown of test_post_object_case_insensitive_condition_fields ____ 2024-05-30T17:13:21.762 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.762 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.762 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.762 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.762 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.762 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.763 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.763 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.763 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.763 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.763 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.763 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.763 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.763 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.763 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.763 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.763 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.763 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.763 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.763 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.763 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.764 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.764 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.764 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.764 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.764 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.764 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.764 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.764 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.764 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.764 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.764 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.764 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.764 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.764 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.764 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.765 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.765 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.765 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.765 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.765 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.765 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.765 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.765 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.765 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.765 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.765 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.765 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.765 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.765 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.765 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.766 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.766 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.766 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.766 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.766 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.766 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.766 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.766 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.766 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.766 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.766 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.766 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.766 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.767 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.767 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.767 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.767 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.767 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.767 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.767 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.767 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.767 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.767 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.767 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.767 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.767 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.767 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.767 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.767 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.768 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.768 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.768 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.768 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.768 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.768 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.768 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.768 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.768 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.768 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.768 INFO:teuthology.orchestra.run.smithi006.stdout:__________ ERROR at teardown of test_post_object_escaped_field_values __________ 2024-05-30T17:13:21.768 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.769 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.769 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.769 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.769 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.769 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.769 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.769 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.769 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.769 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.769 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.769 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.769 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.769 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.769 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.769 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.770 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.770 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.770 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.770 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.770 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.770 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.770 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.770 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.770 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.770 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.770 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.770 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.770 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.771 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.771 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.771 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.771 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.771 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.771 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.771 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.771 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.771 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.771 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.771 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.771 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.771 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.771 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.771 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.772 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.772 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.772 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.772 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.772 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.772 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.772 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.772 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.772 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.772 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.772 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.772 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.772 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.772 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.772 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.772 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.773 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.773 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.773 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.773 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.773 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.773 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.773 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.773 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.773 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.773 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.773 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.773 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.773 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.773 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.773 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.774 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.774 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.774 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.774 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.774 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.774 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.774 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.774 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.774 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.774 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.774 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.774 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.774 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.774 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.775 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.775 INFO:teuthology.orchestra.run.smithi006.stdout:________ ERROR at teardown of test_post_object_success_redirect_action _________ 2024-05-30T17:13:21.775 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.775 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.775 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.775 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.775 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.775 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.775 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.775 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.775 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.775 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.775 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.775 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.775 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.775 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.776 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.776 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.776 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.776 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.776 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.776 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.776 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.776 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.776 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.776 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.776 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.776 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.776 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.776 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.776 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.777 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.777 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.777 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.777 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.777 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.777 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.777 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.777 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.777 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.777 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.777 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.777 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.777 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.777 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.777 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.778 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.778 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.778 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.778 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.778 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.778 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.778 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.778 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.778 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.778 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.778 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.778 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.778 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.778 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.779 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.779 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.779 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.779 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.779 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.779 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.779 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.779 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.779 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.779 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.779 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.779 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.779 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.779 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.779 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.780 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.780 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.780 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.780 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.780 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.780 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.780 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.780 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.780 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.780 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.780 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.780 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.780 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.780 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.780 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.780 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.781 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.781 INFO:teuthology.orchestra.run.smithi006.stdout:___________ ERROR at teardown of test_post_object_invalid_signature ____________ 2024-05-30T17:13:21.781 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.781 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.781 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.781 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.781 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.781 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.781 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.781 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.781 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.781 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.781 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.782 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.782 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.782 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.782 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.782 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.782 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.782 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.782 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.782 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.782 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.782 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.782 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.782 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.783 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.783 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.783 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.783 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.783 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.783 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.783 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.783 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.783 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.783 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.783 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.783 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.783 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.783 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.783 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.784 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.784 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.784 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.784 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.784 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.784 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.784 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.784 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.784 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.784 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.784 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.784 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.784 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.785 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.785 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.785 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.785 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.785 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.785 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.785 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.785 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.785 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.785 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.785 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.785 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.785 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.785 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.785 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.786 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.786 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.786 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.786 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.786 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.786 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.786 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.786 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.786 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.786 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.786 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.786 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.786 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.786 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.787 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.787 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.787 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.787 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.787 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.787 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.787 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.787 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.787 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.787 INFO:teuthology.orchestra.run.smithi006.stdout:___________ ERROR at teardown of test_post_object_invalid_access_key ___________ 2024-05-30T17:13:21.787 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.787 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.787 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.787 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.787 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.788 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.788 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.788 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.788 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.788 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.788 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.788 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.788 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.788 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.788 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.788 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.788 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.788 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.788 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.788 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.789 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.789 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.789 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.789 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.789 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.789 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.789 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.789 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.789 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.789 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.789 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.789 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.789 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.789 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.790 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.790 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.790 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.790 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.790 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.790 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.790 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.790 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.790 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.790 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.790 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.790 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.790 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.790 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.790 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.790 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.791 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.791 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.791 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.791 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.791 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.791 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.791 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.791 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.791 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.791 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.791 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.791 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.791 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.791 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.791 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.792 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.792 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.792 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.792 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.792 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.792 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.792 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.792 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.792 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.792 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.792 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.792 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.792 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.792 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.793 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.793 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.793 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.793 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.793 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.793 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.793 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.793 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.793 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.793 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.793 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.793 INFO:teuthology.orchestra.run.smithi006.stdout:__________ ERROR at teardown of test_post_object_invalid_date_format ___________ 2024-05-30T17:13:21.793 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.793 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.793 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.793 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.794 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.794 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.794 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.794 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.794 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.794 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.794 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.794 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.794 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.794 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.794 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.794 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.794 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.794 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.795 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.795 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.795 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.795 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.795 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.795 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.795 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.795 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.795 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.795 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.795 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.795 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.795 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.796 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.796 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.796 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.796 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.796 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.796 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.796 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.796 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.796 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.796 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.796 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.796 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.796 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.796 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.796 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.797 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.797 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.797 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.797 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.797 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.797 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.797 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.797 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.797 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.797 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.797 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.797 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.797 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.797 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.797 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.798 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.798 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.798 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.798 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.798 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.798 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.798 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.798 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.798 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.798 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.798 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.798 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.798 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.799 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.799 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.799 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.799 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.799 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.799 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.799 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.799 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.799 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.799 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.799 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.799 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.799 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.799 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.799 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.800 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.800 INFO:teuthology.orchestra.run.smithi006.stdout:____________ ERROR at teardown of test_post_object_no_key_specified ____________ 2024-05-30T17:13:21.800 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.800 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.800 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.800 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.800 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.800 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.800 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.800 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.800 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.800 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.801 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.801 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.801 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.801 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.801 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.801 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.801 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.801 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.801 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.801 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.801 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.801 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.801 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.802 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.802 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.802 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.802 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.802 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.802 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.802 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.802 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.802 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.802 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.802 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.802 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.802 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.802 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.802 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.803 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.803 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.803 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.803 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.803 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.803 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.803 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.803 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.803 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.803 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.803 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.803 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.804 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.804 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.804 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.804 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.804 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.804 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.804 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.804 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.804 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.804 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.804 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.804 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.804 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.804 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.804 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.805 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.805 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.805 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.805 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.805 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.805 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.805 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.805 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.805 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.805 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.805 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.805 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.805 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.805 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.805 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.806 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.806 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.806 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.806 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.806 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.806 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.806 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.806 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.806 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.806 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.806 INFO:teuthology.orchestra.run.smithi006.stdout:___________ ERROR at teardown of test_post_object_missing_signature ____________ 2024-05-30T17:13:21.806 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.806 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.806 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.806 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.807 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.807 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.807 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.807 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.807 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.807 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.807 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.807 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.807 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.807 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.807 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.807 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.807 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.807 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.807 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.807 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.808 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.808 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.808 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.808 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.808 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.808 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.808 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.808 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.808 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.808 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.808 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.808 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.809 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.809 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.809 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.809 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.809 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.809 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.809 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.809 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.809 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.809 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.809 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.809 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.809 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.809 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.810 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.810 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.810 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.810 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.810 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.810 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.810 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.810 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.810 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.810 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.810 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.810 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.810 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.810 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.811 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.811 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.811 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.811 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.811 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.811 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.811 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.811 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.811 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.811 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.811 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.811 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.811 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.811 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.811 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.812 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.812 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.812 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.812 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.812 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.812 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.812 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.812 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.812 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.812 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.812 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.812 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.812 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.813 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.813 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.813 INFO:teuthology.orchestra.run.smithi006.stdout:________ ERROR at teardown of test_post_object_missing_policy_condition ________ 2024-05-30T17:13:21.813 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.813 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.813 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.813 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.813 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.813 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.813 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.813 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.813 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.813 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.813 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.813 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.814 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.814 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.814 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.814 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.814 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.814 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.814 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.814 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.814 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.814 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.814 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.814 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.815 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.815 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.815 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.815 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.815 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.815 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.815 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.815 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.815 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.815 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.815 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.815 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.815 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.816 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.816 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.816 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.816 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.816 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.816 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.816 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.816 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.816 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.816 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.816 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.816 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.816 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.817 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.817 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.817 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.817 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.817 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.817 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.817 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.817 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.817 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.817 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.817 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.817 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.817 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.817 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.818 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.818 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.818 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.818 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.818 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.818 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.818 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.818 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.818 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.818 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.818 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.819 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.819 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.819 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.819 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.819 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.819 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.819 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.819 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.819 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.819 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.819 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.819 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.819 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.819 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.820 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.820 INFO:teuthology.orchestra.run.smithi006.stdout:_________ ERROR at teardown of test_post_object_user_specified_header __________ 2024-05-30T17:13:21.820 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.820 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.820 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.820 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.820 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.820 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.820 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.820 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.820 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.820 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.820 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.820 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.821 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.821 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.821 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.821 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.821 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.821 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.821 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.821 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.821 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.821 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.821 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.821 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.821 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.822 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.822 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.822 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.822 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.822 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.822 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.822 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.822 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.822 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.822 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.822 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.822 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.822 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.822 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.822 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.823 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.823 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.823 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.823 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.823 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.823 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.823 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.823 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.823 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.823 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.823 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.823 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.823 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.824 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.824 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.824 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.824 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.824 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.824 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.824 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.824 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.824 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.824 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.824 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.824 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.824 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.824 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.824 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.825 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.825 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.825 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.825 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.825 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.825 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.825 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.825 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.825 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.825 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.825 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.825 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.825 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.825 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.825 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.826 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.826 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.826 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.826 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.826 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.826 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.826 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.826 INFO:teuthology.orchestra.run.smithi006.stdout:_ ERROR at teardown of test_post_object_request_missing_policy_specified_field _ 2024-05-30T17:13:21.826 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.826 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.826 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.826 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.826 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.826 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.827 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.827 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.827 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.827 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.827 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.827 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.827 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.827 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.827 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.827 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.827 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.827 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.827 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.827 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.827 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.827 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.828 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.828 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.828 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.828 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.828 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.828 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.828 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.828 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.828 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.828 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.828 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.828 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.828 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.829 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.829 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.829 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.829 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.829 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.829 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.829 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.829 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.829 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.829 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.829 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.829 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.829 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.829 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.830 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.830 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.830 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.830 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.830 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.830 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.830 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.830 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.830 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.830 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.830 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.830 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.830 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.830 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.830 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.831 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.831 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.831 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.831 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.831 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.831 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.831 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.831 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.831 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.831 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.831 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.831 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.831 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.831 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.831 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.832 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.832 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.832 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.832 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.832 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.832 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.832 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.832 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.832 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.832 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.832 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.832 INFO:teuthology.orchestra.run.smithi006.stdout:______ ERROR at teardown of test_post_object_condition_is_case_sensitive _______ 2024-05-30T17:13:21.832 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.832 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.833 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.833 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.833 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.833 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.833 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.833 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.833 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.833 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.833 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.833 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.833 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.833 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.833 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.833 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.833 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.834 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.834 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.834 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.834 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.834 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.834 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.834 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.834 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.834 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.834 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.834 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.834 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.834 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.834 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.835 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.835 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.835 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.835 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.835 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.835 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.835 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.835 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.835 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.835 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.835 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.835 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.835 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.835 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.835 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.836 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.836 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.836 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.836 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.836 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.836 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.836 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.836 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.836 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.836 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.836 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.836 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.836 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.836 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.837 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.837 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.837 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.837 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.837 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.837 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.837 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.837 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.837 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.837 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.837 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.837 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.837 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.837 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.838 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.838 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.838 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.838 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.838 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.838 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.838 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.838 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.838 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.838 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.838 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.838 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.838 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.838 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.839 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.839 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.839 INFO:teuthology.orchestra.run.smithi006.stdout:_______ ERROR at teardown of test_post_object_expires_is_case_sensitive ________ 2024-05-30T17:13:21.839 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.839 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.839 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.839 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.839 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.839 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.839 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.839 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.839 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.839 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.839 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.840 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.840 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.840 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.840 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.840 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.840 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.840 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.840 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.840 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.840 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.840 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.840 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.840 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.841 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.841 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.841 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.841 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.841 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.841 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.841 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.841 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.841 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.841 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.841 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.841 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.841 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.841 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.841 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.841 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.842 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.842 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.842 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.842 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.842 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.842 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.842 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.842 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.842 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.842 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.842 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.842 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.842 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.842 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.843 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.843 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.843 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.843 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.843 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.843 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.843 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.843 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.843 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.843 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.843 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.843 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.843 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.843 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.843 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.844 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.844 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.844 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.844 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.844 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.844 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.844 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.844 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.844 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.844 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.844 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.844 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.844 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.844 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.844 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.845 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.845 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.845 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.845 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.845 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.845 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.845 INFO:teuthology.orchestra.run.smithi006.stdout:_____________ ERROR at teardown of test_post_object_expired_policy _____________ 2024-05-30T17:13:21.845 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.845 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.845 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.845 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.845 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.845 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.845 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.845 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.845 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.846 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.846 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.846 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.846 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.846 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.846 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.846 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.846 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.846 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.846 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.846 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.846 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.846 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.846 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.847 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.847 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.847 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.847 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.847 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.847 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.847 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.847 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.847 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.847 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.847 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.847 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.847 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.847 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.848 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.848 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.848 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.848 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.848 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.848 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.848 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.848 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.848 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.848 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.848 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.848 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.848 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.848 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.849 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.849 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.849 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.849 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.849 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.849 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.849 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.849 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.849 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.849 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.849 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.849 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.849 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.849 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.849 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.850 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.850 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.850 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.850 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.850 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.850 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.850 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.850 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.850 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.850 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.850 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.850 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.851 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.851 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.851 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.851 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.851 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.851 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.851 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.851 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.851 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.851 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.851 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.851 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.851 INFO:teuthology.orchestra.run.smithi006.stdout:______________ ERROR at teardown of test_post_object_wrong_bucket ______________ 2024-05-30T17:13:21.852 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.852 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.852 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.852 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.852 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.852 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.852 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.852 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.852 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.852 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.852 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.852 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.852 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.852 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.852 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.852 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.853 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.853 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.853 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.853 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.853 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.853 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.853 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.853 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.853 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.853 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.853 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.853 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.853 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.853 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.853 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.854 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.854 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.854 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.854 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.854 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.854 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.854 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.854 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.854 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.854 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.854 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.854 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.854 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.854 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.855 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.855 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.855 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.855 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.855 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.855 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.855 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.855 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.855 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.855 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.855 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.855 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.855 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.855 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.856 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.856 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.856 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.856 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.856 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.856 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.856 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.856 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.856 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.856 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.856 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.856 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.856 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.856 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.856 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.857 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.857 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.857 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.857 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.857 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.857 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.857 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.857 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.857 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.857 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.857 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.857 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.857 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.857 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.858 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.858 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.858 INFO:teuthology.orchestra.run.smithi006.stdout:______ ERROR at teardown of test_post_object_invalid_request_field_value _______ 2024-05-30T17:13:21.858 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.858 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.858 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.858 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.858 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.858 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.858 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.858 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.858 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.858 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.859 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.859 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.859 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.859 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.859 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.859 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.859 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.859 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.859 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.859 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.859 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.859 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.859 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.859 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.859 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.860 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.860 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.860 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.860 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.860 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.860 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.860 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.860 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.860 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.860 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.860 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.860 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.860 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.860 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.860 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.861 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.861 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.861 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.861 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.861 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.861 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.861 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.861 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.861 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.861 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.861 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.861 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.861 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.861 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.861 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.862 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.862 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.862 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.862 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.862 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.862 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.862 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.862 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.862 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.862 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.862 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.862 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.862 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.862 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.863 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.863 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.863 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.863 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.863 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.863 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.863 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.863 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.863 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.863 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.863 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.863 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.863 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.863 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.863 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.864 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.864 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.864 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.864 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.864 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.864 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.864 INFO:teuthology.orchestra.run.smithi006.stdout:_______ ERROR at teardown of test_post_object_missing_expires_condition ________ 2024-05-30T17:13:21.864 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.864 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.864 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.864 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.864 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.864 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.864 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.865 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.865 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.865 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.865 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.865 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.865 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.865 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.865 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.865 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.865 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.865 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.865 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.865 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.865 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.865 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.866 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.866 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.866 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.866 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.866 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.866 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.866 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.866 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.866 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.866 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.866 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.866 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.866 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.867 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.867 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.867 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.867 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.867 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.867 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.867 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.867 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.867 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.867 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.867 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.867 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.867 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.867 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.867 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.867 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.868 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.868 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.868 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.868 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.868 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.868 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.868 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.868 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.868 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.868 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.868 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.868 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.868 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.868 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.868 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.869 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.869 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.869 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.869 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.869 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.869 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.869 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.869 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.869 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.869 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.869 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.869 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.869 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.869 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.869 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.869 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.870 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.870 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.870 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.870 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.870 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.870 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.870 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.870 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.870 INFO:teuthology.orchestra.run.smithi006.stdout:________ ERROR at teardown of test_post_object_missing_conditions_list _________ 2024-05-30T17:13:21.870 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.870 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.870 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.870 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.870 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.871 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.871 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.871 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.871 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.871 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.871 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.871 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.871 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.871 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.871 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.871 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.871 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.871 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.871 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.872 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.872 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.872 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.872 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.872 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.872 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.872 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.872 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.872 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.872 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.872 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.872 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.872 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.872 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.872 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.873 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.873 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.873 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.873 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.873 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.873 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.873 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.873 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.873 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.873 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.873 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.873 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.873 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.873 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.873 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.874 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.874 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.874 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.874 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.874 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.874 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.874 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.874 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.874 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.874 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.874 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.874 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.874 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.874 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.874 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.875 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.875 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.875 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.875 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.875 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.875 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.875 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.875 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.875 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.875 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.875 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.875 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.875 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.875 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.875 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.876 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.876 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.876 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.876 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.876 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.876 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.876 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.876 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.876 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.876 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.876 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.876 INFO:teuthology.orchestra.run.smithi006.stdout:_______ ERROR at teardown of test_post_object_upload_size_limit_exceeded _______ 2024-05-30T17:13:21.876 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.876 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.876 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.877 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.877 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.877 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.877 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.877 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.877 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.877 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.877 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.877 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.877 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.877 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.877 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.877 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.878 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.878 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.878 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.878 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.878 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.878 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.878 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.878 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.878 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.878 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.878 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.878 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.878 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.878 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.878 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.879 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.879 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.879 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.879 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.879 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.879 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.879 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.879 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.879 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.879 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.879 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.879 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.879 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.879 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.879 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.880 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.880 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.880 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.880 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.880 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.880 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.880 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.880 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.880 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.880 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.880 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.880 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.880 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.880 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.880 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.881 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.881 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.881 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.881 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.881 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.881 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.881 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.881 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.881 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.881 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.881 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.881 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.881 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.881 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.881 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.881 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.882 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.882 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.882 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.882 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.882 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.882 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.882 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.882 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.882 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.882 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.882 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.882 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.882 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.882 INFO:teuthology.orchestra.run.smithi006.stdout:____ ERROR at teardown of test_post_object_missing_content_length_argument _____ 2024-05-30T17:13:21.883 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.883 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.883 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.883 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.883 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.883 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.883 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.883 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.883 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.883 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.883 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.883 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.883 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.883 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.884 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.884 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.884 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.884 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.884 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.884 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.884 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.884 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.884 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.884 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.884 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.884 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.884 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.884 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.884 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.884 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.885 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.885 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.885 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.885 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.885 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.885 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.885 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.885 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.885 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.885 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.885 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.885 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.885 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.885 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.885 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.886 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.886 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.886 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.886 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.886 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.886 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.886 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.886 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.886 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.886 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.886 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.886 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.886 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.886 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.886 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.887 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.887 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.887 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.887 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.887 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.887 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.887 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.887 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.887 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.887 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.887 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.887 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.887 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.887 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.888 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.888 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.888 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.888 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.888 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.888 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.888 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.888 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.888 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.888 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.888 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.888 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.888 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.888 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.888 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.889 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.889 INFO:teuthology.orchestra.run.smithi006.stdout:____ ERROR at teardown of test_post_object_invalid_content_length_argument _____ 2024-05-30T17:13:21.889 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.889 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.889 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.889 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.889 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.889 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.889 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.889 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.889 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.889 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.889 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.889 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.889 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.889 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.890 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.890 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.890 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.890 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.890 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.890 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.890 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.890 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.890 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.890 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.890 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.890 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.890 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.891 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.891 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.891 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.891 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.891 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.891 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.891 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.891 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.891 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.891 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.891 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.891 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.891 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.891 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.891 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.891 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.892 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.892 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.892 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.892 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.892 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.892 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.892 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.892 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.892 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.892 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.892 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.892 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.892 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.892 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.893 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.893 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.893 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.893 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.893 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.893 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.893 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.893 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.893 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.893 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.893 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.893 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.893 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.893 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.893 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.894 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.894 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.894 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.894 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.894 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.894 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.894 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.894 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.894 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.894 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.894 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.894 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.894 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.894 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.894 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.895 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.895 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.895 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.895 INFO:teuthology.orchestra.run.smithi006.stdout:_______ ERROR at teardown of test_post_object_upload_size_below_minimum ________ 2024-05-30T17:13:21.895 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.895 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.895 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.895 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.895 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.895 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.895 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.895 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.895 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.895 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.895 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.896 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.896 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.896 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.896 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.896 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.896 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.896 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.896 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.896 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.896 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.896 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.896 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.896 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.896 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.896 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.896 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.897 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.897 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.897 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.897 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.897 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.897 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.897 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.897 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.897 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.897 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.897 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.897 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.897 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.897 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.897 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.898 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.898 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.898 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.898 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.898 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.898 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.898 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.898 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.898 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.898 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.898 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.898 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.898 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.898 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.899 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.899 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.899 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.899 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.899 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.899 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.899 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.899 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.899 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.899 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.899 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.899 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.899 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.899 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.899 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.899 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.900 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.900 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.900 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.900 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.900 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.900 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.900 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.900 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.900 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.900 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.900 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.900 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.900 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.900 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.900 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.901 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.901 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.901 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.901 INFO:teuthology.orchestra.run.smithi006.stdout:_____ ERROR at teardown of test_post_object_upload_size_rgw_chunk_size_bug _____ 2024-05-30T17:13:21.901 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.901 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.901 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.901 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.901 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.901 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.901 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.901 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.901 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.901 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.901 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.901 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.902 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.902 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.902 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.902 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.902 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.902 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.902 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.902 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.902 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.902 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.902 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.902 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.902 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.902 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.903 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.903 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.903 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.903 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.903 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.903 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.903 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.903 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.903 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.903 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.903 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.903 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.903 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.903 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.904 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.904 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.904 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.904 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.904 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.904 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.904 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.904 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.904 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.904 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.904 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.904 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.904 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.904 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.905 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.905 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.905 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.905 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.905 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.905 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.905 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.905 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.905 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.905 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.905 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.905 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.905 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.905 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.905 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.906 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.906 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.906 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.906 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.906 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.906 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.906 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.906 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.906 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.906 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.906 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.906 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.906 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.906 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.907 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.907 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.907 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.907 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.907 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.907 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.907 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.907 INFO:teuthology.orchestra.run.smithi006.stdout:____________ ERROR at teardown of test_post_object_empty_conditions ____________ 2024-05-30T17:13:21.907 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.907 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.907 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.907 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.907 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.907 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.907 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.908 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.908 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.908 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.908 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.908 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.908 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.908 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.908 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.908 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.908 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.908 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.908 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.908 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.908 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.908 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.908 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.909 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.909 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.909 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.909 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.909 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.909 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.909 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.909 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.909 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.909 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.909 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.909 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.909 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.909 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.910 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.910 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.910 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.910 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.910 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.910 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.910 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.910 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.910 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.910 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.910 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.910 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.910 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.910 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.911 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.911 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.911 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.911 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.911 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.911 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.911 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.911 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.911 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.911 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.911 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.911 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.911 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.911 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.911 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.912 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.912 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.912 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.912 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.912 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.912 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.912 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.912 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.912 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.912 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.912 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.912 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.912 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.912 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.912 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.913 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.913 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.913 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.913 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.913 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.913 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.913 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.913 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.913 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.913 INFO:teuthology.orchestra.run.smithi006.stdout:______________ ERROR at teardown of test_get_object_ifmatch_good _______________ 2024-05-30T17:13:21.913 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.913 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.913 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.913 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.913 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.913 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.914 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.914 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.914 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.914 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.914 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.914 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.914 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.914 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.914 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.914 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.914 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.914 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.914 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.915 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.915 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.915 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.915 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.915 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.915 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.915 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.915 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.915 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.915 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.915 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.915 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.915 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.915 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.915 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.915 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.916 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.916 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.916 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.916 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.916 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.916 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.916 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.916 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.916 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.916 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.916 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.916 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.916 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.917 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.917 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.917 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.917 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.917 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.917 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.917 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.917 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.917 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.917 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.917 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.917 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.917 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.917 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.917 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.918 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.918 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.918 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.918 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.918 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.918 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.918 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.918 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.918 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.918 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.918 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.918 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.919 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.919 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.919 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.919 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.919 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.919 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.919 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.919 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.919 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.919 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.919 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.919 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.919 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.919 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.920 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.920 INFO:teuthology.orchestra.run.smithi006.stdout:_____________ ERROR at teardown of test_get_object_ifmatch_failed ______________ 2024-05-30T17:13:21.920 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.920 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.920 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.920 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.920 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.920 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.920 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.920 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.920 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.920 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.920 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.921 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.921 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.921 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.921 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.921 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.921 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.921 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.921 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.921 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.921 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.921 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.921 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.921 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.921 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.921 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.922 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.922 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.922 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.922 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.922 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.922 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.922 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.922 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.922 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.922 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.922 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.922 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.922 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.922 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.922 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.923 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.923 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.923 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.923 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.923 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.923 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.923 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.923 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.923 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.923 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.923 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.923 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.923 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.923 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.923 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.924 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.924 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.924 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.924 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.924 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.924 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.924 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.924 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.924 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.924 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.924 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.924 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.924 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.924 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.925 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.925 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.925 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.925 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.925 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.925 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.925 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.925 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.925 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.925 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.925 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.925 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.925 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.926 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.926 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.926 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.926 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.926 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.926 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.926 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.926 INFO:teuthology.orchestra.run.smithi006.stdout:____________ ERROR at teardown of test_get_object_ifnonematch_good _____________ 2024-05-30T17:13:21.926 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.926 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.926 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.926 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.926 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.926 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.926 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.927 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.927 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.927 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.927 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.927 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.927 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.927 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.927 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.927 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.927 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.927 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.927 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.927 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.927 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.927 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.928 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.928 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.928 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.928 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.928 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.928 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.928 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.928 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.928 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.928 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.928 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.928 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.928 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.928 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.928 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.929 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.929 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.929 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.929 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.929 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.929 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.929 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.929 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.929 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.929 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.929 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.929 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.929 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.929 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.929 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.929 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.930 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.930 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.930 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.930 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.930 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.930 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.930 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.930 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.930 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.930 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.930 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.930 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.930 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.931 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.931 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.931 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.931 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.931 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.931 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.931 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.931 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.931 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.931 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.931 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.931 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.931 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.931 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.931 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.931 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.932 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.932 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.932 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.932 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.932 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.932 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.932 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.932 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.932 INFO:teuthology.orchestra.run.smithi006.stdout:___________ ERROR at teardown of test_get_object_ifnonematch_failed ____________ 2024-05-30T17:13:21.932 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.932 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.932 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.932 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.932 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.932 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.933 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.933 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.933 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.933 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.933 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.933 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.933 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.933 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.933 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.933 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.933 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.933 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.933 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.933 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.933 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.934 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.934 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.934 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.934 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.934 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.934 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.934 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.934 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.934 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.934 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.934 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.934 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.934 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.935 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.935 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.935 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.935 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.935 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.935 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.935 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.935 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.935 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.935 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.935 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.935 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.935 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.935 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.936 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.936 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.936 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.936 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.936 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.936 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.936 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.936 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.936 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.936 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.936 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.936 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.936 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.936 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.936 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.937 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.937 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.937 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.937 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.937 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.937 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.937 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.937 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.937 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.937 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.937 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.937 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.937 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.937 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.937 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.938 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.938 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.938 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.938 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.938 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.938 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.938 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.938 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.938 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.938 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.938 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.938 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.938 INFO:teuthology.orchestra.run.smithi006.stdout:__________ ERROR at teardown of test_get_object_ifmodifiedsince_good ___________ 2024-05-30T17:13:21.939 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.939 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.939 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.939 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.939 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.939 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.939 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.939 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.939 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.939 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.939 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.939 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.939 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.939 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.939 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.940 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.940 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.940 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.940 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.940 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.940 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.940 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.940 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.940 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.940 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.940 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.940 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.940 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.941 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.941 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.941 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.941 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.941 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.941 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.941 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.941 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.941 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.941 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.941 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.941 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.941 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.941 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.942 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.942 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.942 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.942 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.942 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.942 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.942 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.942 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.942 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.942 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.942 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.942 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.942 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.942 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.942 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.943 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.943 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.943 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.943 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.943 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.943 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.943 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.943 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.943 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.943 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.943 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.943 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.943 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.943 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.943 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.944 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.944 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.944 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.944 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.944 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.944 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.944 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.944 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.944 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.944 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.944 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.944 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.944 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.944 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.945 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.945 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.945 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.945 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.945 INFO:teuthology.orchestra.run.smithi006.stdout:_________ ERROR at teardown of test_get_object_ifmodifiedsince_failed __________ 2024-05-30T17:13:21.945 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.945 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.945 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.945 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.945 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.945 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.945 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.945 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.945 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.945 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.946 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.946 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.946 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.946 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.946 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.946 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.946 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.946 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.946 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.946 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.946 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.946 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.946 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.946 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.946 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.947 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.947 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.947 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.947 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.947 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.947 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.947 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.947 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.947 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.947 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.947 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.947 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.947 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.947 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.947 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.947 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.948 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.948 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.948 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.948 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.948 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.948 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.948 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.948 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.948 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.948 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.948 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.948 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.948 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.948 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.948 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.949 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.949 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.949 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.949 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.949 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.949 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.949 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.949 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.949 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.949 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.949 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.949 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.949 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.949 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.949 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.950 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.950 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.950 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.950 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.950 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.950 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.950 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.950 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.950 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.950 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.950 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.950 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.950 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.951 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.951 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.951 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.951 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.951 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.951 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.951 INFO:teuthology.orchestra.run.smithi006.stdout:_________ ERROR at teardown of test_get_object_ifunmodifiedsince_good __________ 2024-05-30T17:13:21.951 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.951 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.951 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.951 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.951 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.951 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.951 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.952 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.952 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.952 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.952 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.952 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.952 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.952 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.952 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.952 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.952 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.952 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.952 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.952 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.952 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.952 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.952 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.953 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.953 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.953 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.953 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.953 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.953 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.953 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.953 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.953 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.953 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.953 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.953 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.953 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.953 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.953 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.954 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.954 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.954 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.954 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.954 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.954 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.954 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.954 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.954 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.954 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.954 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.954 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.954 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.954 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.955 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.955 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.955 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.955 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.955 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.955 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.955 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.955 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.955 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.955 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.955 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.955 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.955 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.955 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.955 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.956 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.956 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.956 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.956 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.956 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.956 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.956 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.956 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.956 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.956 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.956 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.956 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.956 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.956 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.956 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.957 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.957 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.957 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.957 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.957 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.957 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.957 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.957 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.957 INFO:teuthology.orchestra.run.smithi006.stdout:________ ERROR at teardown of test_get_object_ifunmodifiedsince_failed _________ 2024-05-30T17:13:21.957 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.957 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.957 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.957 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.958 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.958 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.958 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.958 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.958 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.958 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.958 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.958 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.958 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.958 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.958 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.958 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.958 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.959 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.959 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.959 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.959 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.959 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.959 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.959 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.959 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.959 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.959 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.959 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.959 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.959 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.959 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.959 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.960 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.960 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.960 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.960 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.960 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.960 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.960 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.960 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.960 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.960 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.960 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.960 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.960 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.960 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.960 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.960 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.961 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.961 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.961 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.961 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.961 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.961 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.961 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.961 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.961 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.961 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.961 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.961 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.961 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.961 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.961 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.962 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.962 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.962 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.962 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.962 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.962 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.962 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.962 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.962 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.962 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.962 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.962 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.962 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.963 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.963 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.963 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.963 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.963 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.963 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.963 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.963 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.963 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.963 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.963 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.963 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.963 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.963 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.963 INFO:teuthology.orchestra.run.smithi006.stdout:______________ ERROR at teardown of test_put_object_ifmatch_good _______________ 2024-05-30T17:13:21.964 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.964 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.964 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.964 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.964 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.964 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.964 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.964 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.964 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.964 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.964 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.964 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.964 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.964 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.965 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.965 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.965 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.965 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.965 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.965 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.965 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.965 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.965 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.965 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.965 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.965 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.965 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.966 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.966 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.966 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.966 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.966 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.966 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.966 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.966 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.966 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.966 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.966 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.966 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.966 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.967 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.967 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.967 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.967 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.967 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.967 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.967 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.967 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.967 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.967 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.967 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.967 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.967 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.967 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.967 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.968 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.968 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.968 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.968 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.968 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.968 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.968 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.968 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.968 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.968 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.968 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.968 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.968 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.969 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.969 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.969 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.969 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.969 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.969 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.969 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.969 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.969 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.969 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.969 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.969 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.969 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.969 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.970 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.970 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.970 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.970 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.970 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.970 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.970 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.970 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.970 INFO:teuthology.orchestra.run.smithi006.stdout:_____________ ERROR at teardown of test_put_object_ifmatch_failed ______________ 2024-05-30T17:13:21.970 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.970 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.970 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.970 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.970 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.970 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.971 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.971 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.971 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.971 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.971 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.971 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.971 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.971 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.971 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.971 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.971 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.971 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.971 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.971 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.972 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.972 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.972 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.972 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.972 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.972 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.972 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.972 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.972 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.972 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.972 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.972 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.972 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.972 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.972 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.973 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.973 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.973 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.973 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.973 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.973 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.973 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.973 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.973 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.973 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.973 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.973 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.973 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.973 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.973 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.973 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.974 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.974 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.974 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.974 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.974 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.974 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.974 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.974 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.974 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.974 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.974 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.974 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.974 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.974 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.975 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.975 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.975 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.975 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.975 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.975 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.975 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.975 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.975 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.975 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.975 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.975 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.975 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.975 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.976 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.976 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.976 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.976 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.976 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.976 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.976 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.976 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.976 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.976 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.976 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.976 INFO:teuthology.orchestra.run.smithi006.stdout:_____ ERROR at teardown of test_put_object_ifmatch_overwrite_existed_good ______ 2024-05-30T17:13:21.976 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.976 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.976 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.977 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.977 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.977 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.977 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.977 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.977 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.977 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.977 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.977 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.977 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.977 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.977 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.977 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.977 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.977 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.977 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.978 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.978 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.978 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.978 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.978 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.978 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.978 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.978 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.978 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.978 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.978 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.978 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.978 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.979 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.979 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.979 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.979 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.979 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.979 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.979 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.979 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.979 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.979 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.979 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.979 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.979 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.979 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.979 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.980 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.980 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.980 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.980 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.980 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.980 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.980 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.980 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.980 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.980 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.980 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.980 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.980 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.980 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.980 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.981 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.981 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.981 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.981 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.981 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.981 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.981 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.981 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.981 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.981 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.981 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.981 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.981 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.981 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.981 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.982 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.982 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.982 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.982 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.982 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.982 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.982 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.982 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.982 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.982 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.982 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.982 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.982 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.982 INFO:teuthology.orchestra.run.smithi006.stdout:________ ERROR at teardown of test_put_object_ifmatch_nonexisted_failed ________ 2024-05-30T17:13:21.983 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.983 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.983 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.983 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.983 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.983 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.983 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.983 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.983 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.983 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.983 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.983 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.983 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.983 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.983 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.984 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.984 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.984 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.984 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.984 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.984 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.984 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.984 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.984 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.984 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.984 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.984 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.984 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.985 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.985 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.985 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.985 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.985 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.985 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.985 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.985 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.985 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.985 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.985 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.985 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.985 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.985 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.986 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.986 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.986 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.986 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.986 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.986 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.986 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.986 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.986 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.986 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.986 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.986 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.986 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.987 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.987 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.987 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.987 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.987 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.987 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.987 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.987 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.987 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.987 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.987 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.987 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.987 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.987 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.987 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.988 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.988 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.988 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.989 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.989 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.989 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.989 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.989 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.989 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.989 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.989 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.990 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.990 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.990 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.990 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.990 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.990 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.990 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.990 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.990 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.990 INFO:teuthology.orchestra.run.smithi006.stdout:_____________ ERROR at teardown of test_put_object_ifnonmatch_good _____________ 2024-05-30T17:13:21.990 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.990 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.990 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.990 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.991 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.991 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.991 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.991 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.991 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.991 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.991 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.991 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.991 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.991 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.991 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.991 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.991 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.991 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.991 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.991 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.992 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.992 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.992 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.992 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.992 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.992 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.992 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.992 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.992 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.992 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.992 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.992 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.992 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.992 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.992 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.993 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.993 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.993 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.993 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.993 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.993 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.993 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.993 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.993 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.993 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.993 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.993 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.993 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:21.993 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:21.993 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:21.994 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:21.994 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.994 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:21.994 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.994 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:21.994 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:21.994 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.994 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.994 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.994 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.994 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:21.994 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:21.994 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.994 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.994 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.995 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:21.995 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:21.995 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:21.995 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:21.995 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:21.995 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:21.995 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.995 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.995 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:21.995 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:21.995 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:21.995 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:21.995 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:21.995 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:21.995 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:21.995 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:21.996 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.996 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.996 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:21.996 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:21.996 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:21.996 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:21.996 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:21.996 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.996 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:21.996 INFO:teuthology.orchestra.run.smithi006.stdout:____________ ERROR at teardown of test_put_object_ifnonmatch_failed ____________ 2024-05-30T17:13:21.996 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.996 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:21.996 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.996 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:21.996 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:21.997 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:21.997 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:21.997 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.997 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:21.997 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.997 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:21.997 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:21.997 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:21.997 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:21.997 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:21.997 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:21.997 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:21.997 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:21.997 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:21.997 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:21.997 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:21.998 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:21.998 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:21.998 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.998 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:21.998 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:21.998 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:21.998 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:21.998 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:21.998 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:21.998 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:21.998 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:21.998 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:21.998 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:21.998 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:21.999 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:21.999 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:21.999 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:21.999 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.999 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:21.999 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:21.999 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:21.999 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:21.999 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:21.999 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:21.999 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:21.999 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:21.999 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.000 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.000 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.000 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.000 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.000 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.000 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.000 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.000 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.000 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.000 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.000 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.000 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.000 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.000 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.000 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.000 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.001 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.001 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.001 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.001 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.001 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.001 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.001 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.001 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.001 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.001 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.001 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.001 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.001 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.001 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.001 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.002 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.002 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.002 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.002 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.002 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.002 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.002 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.002 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.002 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.002 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.002 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.002 INFO:teuthology.orchestra.run.smithi006.stdout:_______ ERROR at teardown of test_put_object_ifnonmatch_nonexisted_good ________ 2024-05-30T17:13:22.002 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.002 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.002 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.003 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.003 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.003 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.003 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.003 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.003 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.003 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.003 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.003 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.003 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.003 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.003 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.003 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.003 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.003 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.004 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.004 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.004 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.004 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.004 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.004 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.004 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.004 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.004 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.004 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.004 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.004 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.004 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.004 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.004 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.005 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.005 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.005 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.005 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.005 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.005 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.005 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.005 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.005 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.005 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.005 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.005 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.005 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.005 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.005 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.006 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.006 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.006 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.006 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.006 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.006 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.006 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.006 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.006 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.006 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.006 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.006 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.006 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.006 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.007 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.007 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.007 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.007 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.007 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.007 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.007 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.007 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.007 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.007 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.007 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.007 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.007 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.007 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.007 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.008 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.008 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.008 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.008 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.008 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.008 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.008 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.008 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.008 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.008 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.008 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.008 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.008 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.008 INFO:teuthology.orchestra.run.smithi006.stdout:___ ERROR at teardown of test_put_object_ifnonmatch_overwrite_existed_failed ___ 2024-05-30T17:13:22.008 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.009 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.009 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.009 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.009 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.009 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.009 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.009 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.009 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.009 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.009 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.009 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.009 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.009 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.009 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.009 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.010 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.010 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.010 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.010 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.010 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.010 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.010 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.010 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.010 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.010 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.010 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.010 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.010 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.010 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.011 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.011 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.011 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.011 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.011 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.011 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.011 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.011 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.011 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.011 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.011 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.011 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.011 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.011 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.011 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.011 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.012 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.012 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.012 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.012 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.012 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.012 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.012 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.012 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.012 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.012 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.012 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.012 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.012 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.013 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.013 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.013 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.013 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.013 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.013 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.013 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.013 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.013 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.013 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.013 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.013 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.013 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.013 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.013 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.013 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.014 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.014 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.014 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.014 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.014 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.014 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.014 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.014 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.014 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.014 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.014 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.014 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.014 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.015 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.015 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.015 INFO:teuthology.orchestra.run.smithi006.stdout:___________________ ERROR at teardown of test_object_raw_get ___________________ 2024-05-30T17:13:22.015 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.015 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.015 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.015 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.015 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.015 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.015 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.015 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.015 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.015 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.015 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.015 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.016 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.016 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.016 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.016 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.016 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.016 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.016 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.016 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.016 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.016 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.016 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.016 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.016 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.016 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.016 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.017 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.017 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.017 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.017 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.017 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.017 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.017 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.017 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.017 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.017 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.017 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.017 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.017 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.017 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.017 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.018 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.018 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.018 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.018 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.018 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.018 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.018 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.018 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.018 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.018 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.018 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.018 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.018 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.019 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.019 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.019 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.019 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.019 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.019 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.019 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.019 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.019 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.019 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.019 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.019 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.019 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.019 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.020 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.020 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.020 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.020 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.020 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.020 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.020 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.020 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.020 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.020 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.020 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.020 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.020 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.020 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.020 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.020 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.021 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.021 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.021 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.021 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.021 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.021 INFO:teuthology.orchestra.run.smithi006.stdout:_____________ ERROR at teardown of test_object_raw_get_bucket_gone _____________ 2024-05-30T17:13:22.021 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.021 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.021 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.021 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.021 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.021 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.021 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.021 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.021 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.021 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.022 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.022 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.022 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.022 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.022 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.022 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.022 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.022 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.022 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.022 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.022 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.022 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.022 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.022 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.023 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.023 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.023 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.023 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.023 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.023 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.023 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.023 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.023 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.023 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.023 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.023 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.023 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.023 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.023 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.024 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.024 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.024 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.024 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.024 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.024 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.024 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.024 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.024 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.024 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.024 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.024 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.024 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.024 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.024 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.025 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.025 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.025 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.025 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.025 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.025 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.025 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.025 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.025 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.025 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.025 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.025 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.025 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.025 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.026 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.026 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.026 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.026 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.026 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.026 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.026 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.026 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.026 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.026 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.026 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.026 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.026 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.027 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.027 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.027 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.027 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.027 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.027 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.027 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.027 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.027 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.027 INFO:teuthology.orchestra.run.smithi006.stdout:___________ ERROR at teardown of test_object_delete_key_bucket_gone ____________ 2024-05-30T17:13:22.027 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.027 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.027 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.027 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.028 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.028 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.028 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.028 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.028 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.028 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.028 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.028 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.028 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.028 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.028 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.028 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.028 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.028 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.028 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.029 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.029 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.029 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.029 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.029 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.029 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.029 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.029 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.029 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.029 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.029 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.029 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.029 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.029 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.029 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.030 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.030 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.030 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.030 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.030 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.030 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.030 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.030 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.030 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.030 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.030 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.030 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.030 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.030 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.031 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.031 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.031 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.031 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.031 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.031 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.031 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.031 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.031 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.031 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.031 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.031 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.031 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.032 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.032 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.032 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.032 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.032 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.032 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.032 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.032 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.032 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.032 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.032 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.032 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.032 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.032 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.033 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.033 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.033 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.033 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.033 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.033 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.033 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.033 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.033 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.033 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.033 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.033 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.033 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.033 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.034 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.034 INFO:teuthology.orchestra.run.smithi006.stdout:_____________ ERROR at teardown of test_object_raw_get_object_gone _____________ 2024-05-30T17:13:22.034 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.034 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.034 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.034 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.034 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.034 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.034 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.034 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.034 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.034 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.034 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.034 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.034 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.035 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.035 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.035 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.035 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.035 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.035 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.035 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.035 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.035 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.035 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.035 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.035 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.035 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.035 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.035 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.035 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.036 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.036 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.036 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.036 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.036 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.036 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.036 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.036 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.036 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.036 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.036 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.036 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.036 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.037 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.037 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.037 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.037 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.037 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.037 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.037 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.037 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.037 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.037 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.037 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.037 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.037 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.037 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.037 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.038 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.038 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.038 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.038 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.038 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.038 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.038 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.038 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.038 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.038 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.038 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.038 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.038 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.038 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.039 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.039 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.039 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.039 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.039 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.039 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.039 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.039 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.039 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.039 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.039 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.039 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.039 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.040 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.040 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.040 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.040 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.040 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.040 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.040 INFO:teuthology.orchestra.run.smithi006.stdout:____________________ ERROR at teardown of test_bucket_head _____________________ 2024-05-30T17:13:22.040 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.040 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.040 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.040 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.040 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.040 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.040 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.041 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.041 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.041 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.041 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.041 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.041 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.041 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.041 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.041 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.041 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.041 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.041 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.041 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.041 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.042 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.042 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.042 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.042 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.042 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.042 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.042 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.042 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.042 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.042 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.042 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.042 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.042 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.042 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.043 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.043 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.043 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.043 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.043 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.043 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.043 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.043 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.043 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.043 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.043 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.043 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.043 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.043 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.043 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.044 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.044 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.044 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.044 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.044 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.044 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.044 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.044 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.044 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.044 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.044 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.044 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.044 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.044 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.044 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.045 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.045 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.045 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.045 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.045 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.045 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.045 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.045 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.045 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.045 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.045 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.045 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.045 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.045 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.045 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.045 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.046 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.046 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.046 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.046 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.046 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.046 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.046 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.046 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.046 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.046 INFO:teuthology.orchestra.run.smithi006.stdout:________________ ERROR at teardown of test_bucket_head_notexist ________________ 2024-05-30T17:13:22.046 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.046 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.046 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.046 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.047 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.047 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.047 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.047 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.047 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.047 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.047 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.047 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.047 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.047 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.047 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.047 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.047 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.047 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.047 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.047 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.048 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.048 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.048 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.048 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.048 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.048 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.048 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.048 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.048 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.048 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.048 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.048 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.048 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.048 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.048 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.049 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.049 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.049 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.049 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.049 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.049 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.049 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.049 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.049 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.049 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.049 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.049 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.049 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.049 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.049 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.049 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.050 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.050 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.050 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.050 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.050 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.050 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.050 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.050 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.050 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.050 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.050 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.050 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.050 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.050 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.050 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.051 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.051 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.051 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.051 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.051 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.051 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.051 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.051 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.051 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.051 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.051 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.051 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.051 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.051 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.051 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.052 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.052 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.052 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.052 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.052 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.052 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.052 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.052 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.052 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.052 INFO:teuthology.orchestra.run.smithi006.stdout:________________ ERROR at teardown of test_bucket_head_extended ________________ 2024-05-30T17:13:22.052 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.052 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.052 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.053 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.053 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.053 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.053 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.053 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.053 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.053 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.053 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.053 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.053 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.053 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.053 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.053 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.053 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.053 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.054 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.054 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.054 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.054 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.054 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.054 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.054 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.054 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.054 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.054 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.054 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.054 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.054 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.054 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.055 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.055 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.055 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.055 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.055 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.055 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.055 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.055 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.055 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.055 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.055 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.055 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.055 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.055 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.056 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.056 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.056 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.056 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.056 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.056 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.056 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.056 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.056 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.056 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.056 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.056 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.056 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.056 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.057 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.057 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.057 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.057 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.057 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.057 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.057 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.057 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.057 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.057 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.057 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.057 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.057 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.057 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.058 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.058 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.058 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.058 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.058 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.058 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.058 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.058 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.058 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.058 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.058 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.058 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.058 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.059 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.059 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.059 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.059 INFO:teuthology.orchestra.run.smithi006.stdout:_____________ ERROR at teardown of test_object_raw_get_bucket_acl ______________ 2024-05-30T17:13:22.059 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.059 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.059 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.059 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.059 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.059 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.059 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.059 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.059 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.059 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.059 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.060 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.060 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.060 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.060 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.060 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.060 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.060 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.060 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.060 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.060 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.060 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.060 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.060 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.060 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.060 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.060 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.061 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.061 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.061 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.061 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.061 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.061 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.061 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.061 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.061 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.061 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.061 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.061 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.061 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.061 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.061 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.062 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.062 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.062 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.062 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.062 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.062 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.062 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.062 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.062 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.062 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.062 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.062 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.062 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.062 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.062 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.062 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.063 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.063 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.063 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.063 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.063 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.063 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.063 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.063 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.063 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.063 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.063 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.063 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.063 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.063 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.063 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.064 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.064 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.064 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.064 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.064 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.064 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.064 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.064 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.064 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.064 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.064 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.064 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.064 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.064 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.064 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.065 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.065 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.065 INFO:teuthology.orchestra.run.smithi006.stdout:_____________ ERROR at teardown of test_object_raw_get_object_acl ______________ 2024-05-30T17:13:22.065 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.065 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.065 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.065 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.065 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.065 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.065 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.065 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.065 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.065 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.065 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.066 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.066 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.066 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.066 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.066 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.066 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.066 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.066 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.066 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.066 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.066 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.066 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.066 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.067 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.067 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.067 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.067 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.067 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.067 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.067 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.067 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.067 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.067 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.067 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.067 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.067 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.067 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.067 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.068 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.068 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.068 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.068 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.068 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.068 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.068 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.068 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.068 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.068 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.068 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.068 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.068 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.068 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.069 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.069 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.069 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.069 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.069 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.069 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.069 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.069 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.069 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.069 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.069 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.069 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.069 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.069 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.069 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.070 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.070 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.070 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.070 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.070 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.070 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.070 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.070 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.070 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.070 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.070 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.070 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.070 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.070 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.070 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.071 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.071 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.071 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.071 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.071 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.071 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.071 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.071 INFO:teuthology.orchestra.run.smithi006.stdout:________________ ERROR at teardown of test_object_put_acl_mtime ________________ 2024-05-30T17:13:22.071 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.071 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.071 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.071 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.071 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.071 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.071 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.072 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.072 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.072 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.072 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.072 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.072 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.072 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.072 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.072 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.072 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.072 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.072 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.072 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.072 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.073 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.073 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.073 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.073 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.073 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.073 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.073 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.073 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.073 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.073 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.073 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.073 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.073 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.073 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.073 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.073 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.074 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.074 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.074 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.074 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.074 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.074 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.074 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.074 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.074 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.074 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.074 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.074 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.074 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.074 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.075 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.075 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.075 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.075 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.075 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.075 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.075 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.075 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.075 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.075 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.075 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.075 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.075 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.075 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.075 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.076 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.076 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.076 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.076 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.076 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.076 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.076 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.076 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.076 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.076 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.076 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.076 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.076 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.077 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.077 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.077 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.077 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.077 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.077 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.077 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.077 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.077 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.077 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.077 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.077 INFO:teuthology.orchestra.run.smithi006.stdout:______________ ERROR at teardown of test_object_raw_authenticated ______________ 2024-05-30T17:13:22.077 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.077 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.077 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.078 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.078 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.078 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.078 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.078 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.078 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.078 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.078 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.078 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.078 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.078 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.078 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.078 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.078 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.079 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.079 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.079 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.079 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.079 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.079 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.079 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.079 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.079 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.079 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.079 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.079 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.079 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.080 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.080 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.080 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.080 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.080 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.080 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.080 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.080 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.080 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.080 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.080 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.080 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.080 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.080 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.080 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.080 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.081 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.081 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.081 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.081 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.081 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.081 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.081 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.081 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.081 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.081 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.081 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.081 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.081 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.081 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.081 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.082 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.082 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.082 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.082 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.082 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.082 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.082 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.082 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.082 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.082 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.082 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.082 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.082 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.082 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.082 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.083 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.083 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.083 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.083 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.083 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.083 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.083 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.083 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.083 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.083 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.083 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.083 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.083 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.083 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.083 INFO:teuthology.orchestra.run.smithi006.stdout:____________ ERROR at teardown of test_object_raw_response_headers _____________ 2024-05-30T17:13:22.084 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.084 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.084 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.084 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.084 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.084 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.084 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.084 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.084 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.084 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.084 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.084 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.084 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.084 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.084 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.085 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.085 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.085 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.085 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.085 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.085 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.085 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.085 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.085 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.085 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.085 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.085 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.085 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.085 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.085 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.085 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.086 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.086 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.086 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.086 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.086 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.086 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.086 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.086 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.086 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.086 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.086 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.086 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.086 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.087 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.087 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.087 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.087 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.087 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.087 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.087 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.087 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.087 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.087 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.087 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.087 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.087 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.087 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.087 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.087 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.088 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.088 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.088 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.088 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.088 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.088 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.088 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.088 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.088 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.088 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.088 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.088 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.088 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.088 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.088 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.089 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.089 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.089 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.089 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.089 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.089 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.089 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.089 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.089 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.089 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.089 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.089 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.089 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.089 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.089 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.089 INFO:teuthology.orchestra.run.smithi006.stdout:________ ERROR at teardown of test_object_raw_authenticated_bucket_acl _________ 2024-05-30T17:13:22.090 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.090 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.090 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.090 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.090 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.090 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.090 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.090 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.090 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.090 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.090 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.090 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.090 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.090 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.091 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.091 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.091 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.091 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.091 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.091 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.091 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.091 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.091 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.091 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.091 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.091 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.091 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.091 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.091 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.092 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.092 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.092 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.092 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.092 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.092 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.092 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.092 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.092 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.092 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.092 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.092 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.092 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.092 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.092 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.093 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.093 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.093 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.093 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.093 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.093 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.093 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.093 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.093 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.093 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.093 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.093 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.093 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.093 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.094 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.094 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.094 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.094 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.094 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.094 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.094 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.094 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.094 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.094 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.094 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.094 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.094 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.094 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.094 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.095 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.095 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.095 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.095 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.095 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.095 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.095 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.095 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.095 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.095 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.095 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.095 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.095 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.095 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.095 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.095 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.096 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.096 INFO:teuthology.orchestra.run.smithi006.stdout:________ ERROR at teardown of test_object_raw_authenticated_object_acl _________ 2024-05-30T17:13:22.096 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.096 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.096 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.096 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.096 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.096 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.096 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.096 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.096 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.096 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.096 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.096 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.096 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.097 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.097 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.097 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.097 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.097 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.097 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.097 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.097 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.097 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.097 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.097 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.097 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.097 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.097 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.097 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.098 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.098 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.098 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.098 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.098 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.098 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.098 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.098 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.098 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.098 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.098 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.098 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.098 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.098 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.099 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.099 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.099 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.099 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.099 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.099 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.099 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.099 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.099 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.099 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.099 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.099 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.099 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.099 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.100 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.100 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.100 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.100 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.100 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.100 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.100 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.100 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.100 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.100 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.100 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.100 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.100 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.100 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.100 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.100 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.101 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.101 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.101 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.101 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.101 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.101 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.101 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.101 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.101 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.101 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.101 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.101 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.101 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.101 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.101 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.102 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.102 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.102 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.102 INFO:teuthology.orchestra.run.smithi006.stdout:________ ERROR at teardown of test_object_raw_authenticated_bucket_gone ________ 2024-05-30T17:13:22.102 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.102 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.102 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.102 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.102 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.102 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.102 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.102 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.102 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.102 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.102 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.102 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.103 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.103 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.103 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.103 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.103 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.103 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.103 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.103 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.103 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.103 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.103 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.103 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.103 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.103 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.103 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.104 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.104 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.104 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.104 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.104 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.104 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.104 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.104 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.104 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.104 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.104 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.104 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.104 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.104 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.104 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.104 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.105 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.105 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.105 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.105 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.105 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.105 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.105 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.105 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.105 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.105 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.105 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.106 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.106 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.106 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.106 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.106 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.106 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.106 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.106 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.106 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.106 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.106 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.106 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.106 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.106 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.106 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.107 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.107 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.107 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.107 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.107 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.107 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.107 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.107 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.107 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.107 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.107 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.107 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.107 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.107 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.108 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.108 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.108 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.108 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.108 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.108 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.108 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.108 INFO:teuthology.orchestra.run.smithi006.stdout:________ ERROR at teardown of test_object_raw_authenticated_object_gone ________ 2024-05-30T17:13:22.108 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.108 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.108 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.108 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.108 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.108 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.108 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.109 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.109 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.109 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.109 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.109 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.109 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.109 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.109 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.109 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.109 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.109 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.109 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.109 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.109 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.110 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.110 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.110 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.110 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.110 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.110 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.110 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.110 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.110 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.110 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.110 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.110 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.110 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.110 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.110 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.111 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.111 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.111 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.111 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.111 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.111 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.111 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.111 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.111 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.111 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.111 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.111 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.111 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.111 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.111 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.112 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.112 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.112 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.112 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.112 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.112 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.112 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.112 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.112 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.112 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.112 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.112 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.112 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.112 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.112 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.113 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.113 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.113 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.113 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.113 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.113 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.113 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.113 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.113 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.113 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.113 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.113 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.113 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.113 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.113 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.113 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.114 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.114 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.114 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.114 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.114 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.114 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.114 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.114 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.114 INFO:teuthology.orchestra.run.smithi006.stdout:______ ERROR at teardown of test_object_raw_get_x_amz_expires_not_expired ______ 2024-05-30T17:13:22.114 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.114 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.114 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.114 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.114 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.115 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.115 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.115 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.115 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.115 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.115 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.115 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.115 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.115 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.115 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.115 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.115 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.115 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.115 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.115 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.116 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.116 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.116 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.116 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.116 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.116 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.116 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.116 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.116 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.116 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.116 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.116 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.116 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.116 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.116 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.117 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.117 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.117 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.117 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.117 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.117 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.117 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.117 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.117 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.117 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.117 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.117 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.117 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.117 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.117 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.117 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.118 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.118 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.118 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.118 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.118 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.118 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.118 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.118 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.118 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.118 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.118 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.118 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.118 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.118 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.119 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.119 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.119 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.119 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.119 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.119 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.119 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.119 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.119 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.119 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.119 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.119 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.120 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.120 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.120 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.120 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.120 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.120 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.120 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.120 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.120 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.120 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.120 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.120 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.120 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.120 INFO:teuthology.orchestra.run.smithi006.stdout:__ ERROR at teardown of test_object_raw_get_x_amz_expires_not_expired_tenant ___ 2024-05-30T17:13:22.120 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.121 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.121 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.121 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.121 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.121 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.121 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.121 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.121 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.121 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.121 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.121 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.121 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.121 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.121 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.122 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.122 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.122 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.122 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.122 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.122 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.122 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.122 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.122 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.122 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.122 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.122 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.122 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.123 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.123 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.123 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.123 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.123 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.123 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.123 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.123 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.123 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.123 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.123 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.123 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.123 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.123 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.123 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.124 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.124 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.124 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.124 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.124 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.124 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.124 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.124 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.124 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.124 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.124 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.124 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.124 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.124 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.124 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.125 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.125 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.125 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.125 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.125 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.125 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.125 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.125 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.125 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.125 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.125 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.125 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.125 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.125 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.125 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.126 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.126 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.126 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.126 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.126 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.126 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.126 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.126 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.126 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.126 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.126 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.126 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.126 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.127 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.127 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.127 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.127 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.127 INFO:teuthology.orchestra.run.smithi006.stdout:____ ERROR at teardown of test_object_raw_get_x_amz_expires_out_range_zero _____ 2024-05-30T17:13:22.127 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.127 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.127 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.127 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.127 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.127 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.127 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.127 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.127 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.127 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.128 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.128 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.128 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.128 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.128 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.128 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.128 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.128 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.128 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.128 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.128 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.128 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.128 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.128 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.128 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.129 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.129 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.129 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.129 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.129 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.129 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.129 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.129 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.129 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.129 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.129 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.129 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.129 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.129 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.129 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.130 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.130 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.130 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.130 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.130 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.130 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.130 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.130 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.130 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.130 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.130 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.130 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.130 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.130 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.131 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.131 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.131 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.131 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.131 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.131 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.131 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.131 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.131 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.131 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.131 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.131 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.131 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.131 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.131 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.131 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.132 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.132 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.132 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.132 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.132 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.132 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.132 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.132 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.132 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.132 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.132 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.132 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.132 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.132 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.133 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.133 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.133 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.133 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.133 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.133 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.133 INFO:teuthology.orchestra.run.smithi006.stdout:_____ ERROR at teardown of test_object_raw_get_x_amz_expires_out_max_range _____ 2024-05-30T17:13:22.133 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.133 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.133 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.133 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.133 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.133 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.133 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.134 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.134 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.134 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.134 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.134 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.134 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.134 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.134 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.134 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.134 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.134 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.134 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.134 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.134 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.135 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.135 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.135 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.135 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.135 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.135 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.135 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.135 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.135 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.135 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.135 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.135 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.135 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.135 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.135 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.136 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.136 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.136 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.136 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.136 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.136 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.136 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.136 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.136 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.136 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.136 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.136 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.136 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.136 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.136 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.136 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.137 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.137 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.137 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.137 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.137 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.137 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.137 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.137 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.137 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.137 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.137 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.137 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.137 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.137 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.138 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.138 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.138 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.138 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.138 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.138 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.138 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.138 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.138 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.138 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.138 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.138 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.138 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.138 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.139 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.139 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.139 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.139 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.139 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.139 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.139 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.139 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.139 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.139 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.139 INFO:teuthology.orchestra.run.smithi006.stdout:__ ERROR at teardown of test_object_raw_get_x_amz_expires_out_positive_range ___ 2024-05-30T17:13:22.139 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.139 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.139 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.139 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.140 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.140 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.140 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.140 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.140 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.140 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.140 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.140 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.140 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.140 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.140 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.140 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.140 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.140 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.141 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.141 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.141 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.141 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.141 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.141 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.141 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.141 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.141 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.141 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.141 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.141 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.141 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.141 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.141 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.141 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.142 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.142 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.142 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.142 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.142 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.142 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.142 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.142 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.142 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.142 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.142 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.142 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.142 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.142 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.142 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.143 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.143 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.143 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.143 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.143 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.143 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.143 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.143 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.143 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.143 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.143 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.143 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.143 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.143 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.143 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.144 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.144 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.144 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.144 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.144 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.144 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.144 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.144 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.144 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.144 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.144 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.144 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.144 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.144 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.144 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.145 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.145 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.145 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.145 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.145 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.145 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.145 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.145 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.145 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.145 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.145 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.145 INFO:teuthology.orchestra.run.smithi006.stdout:__________________ ERROR at teardown of test_object_anon_put ___________________ 2024-05-30T17:13:22.145 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.145 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.146 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.146 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.146 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.146 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.146 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.146 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.146 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.146 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.146 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.146 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.146 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.146 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.146 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.146 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.147 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.147 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.147 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.147 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.147 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.147 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.147 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.147 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.147 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.147 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.147 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.147 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.147 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.147 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.147 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.147 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.148 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.148 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.148 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.148 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.148 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.148 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.148 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.148 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.148 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.148 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.148 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.148 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.148 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.148 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.148 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.149 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.149 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.149 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.149 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.149 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.149 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.149 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.149 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.149 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.149 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.149 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.149 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.149 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.149 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.149 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.149 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.150 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.150 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.150 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.150 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.150 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.150 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.150 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.150 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.150 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.150 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.150 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.150 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.150 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.150 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.150 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.151 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.151 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.151 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.151 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.151 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.151 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.151 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.151 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.151 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.151 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.151 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.151 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.151 INFO:teuthology.orchestra.run.smithi006.stdout:____________ ERROR at teardown of test_object_anon_put_write_access ____________ 2024-05-30T17:13:22.151 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.152 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.152 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.152 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.152 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.152 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.152 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.152 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.152 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.152 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.152 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.152 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.152 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.152 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.152 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.152 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.153 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.153 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.153 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.153 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.153 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.153 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.153 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.153 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.153 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.153 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.153 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.153 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.153 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.153 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.153 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.153 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.154 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.154 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.154 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.154 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.154 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.154 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.154 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.154 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.154 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.154 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.154 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.154 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.154 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.154 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.154 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.155 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.155 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.155 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.155 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.155 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.155 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.155 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.155 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.155 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.155 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.155 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.155 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.155 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.155 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.155 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.156 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.156 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.156 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.156 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.156 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.156 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.156 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.156 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.156 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.156 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.156 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.156 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.156 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.156 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.156 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.156 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.157 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.157 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.157 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.157 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.157 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.157 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.157 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.157 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.157 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.157 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.157 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.157 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.157 INFO:teuthology.orchestra.run.smithi006.stdout:______________ ERROR at teardown of test_object_put_authenticated ______________ 2024-05-30T17:13:22.157 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.157 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.157 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.158 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.158 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.158 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.158 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.158 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.158 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.158 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.158 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.158 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.158 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.158 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.158 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.158 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.159 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.159 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.159 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.159 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.159 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.159 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.159 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.159 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.159 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.159 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.159 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.159 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.159 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.159 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.159 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.160 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.160 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.160 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.160 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.160 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.160 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.160 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.160 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.160 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.160 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.160 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.160 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.160 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.160 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.160 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.161 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.161 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.161 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.161 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.161 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.161 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.161 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.161 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.161 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.161 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.161 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.161 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.161 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.161 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.161 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.162 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.162 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.162 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.162 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.162 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.162 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.162 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.162 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.162 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.162 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.162 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.162 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.162 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.162 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.163 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.163 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.163 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.163 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.163 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.163 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.163 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.163 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.163 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.163 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.163 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.163 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.163 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.163 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.163 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.163 INFO:teuthology.orchestra.run.smithi006.stdout:________ ERROR at teardown of test_object_presigned_put_object_with_acl ________ 2024-05-30T17:13:22.164 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.164 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.164 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.164 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.164 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.164 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.164 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.164 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.164 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.164 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.164 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.164 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.164 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.164 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.164 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.165 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.165 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.165 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.165 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.165 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.165 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.165 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.165 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.165 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.165 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.165 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.165 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.165 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.165 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.166 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.166 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.166 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.166 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.166 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.166 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.166 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.166 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.166 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.166 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.166 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.166 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.166 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.166 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.167 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.167 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.167 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.167 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.167 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.167 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.167 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.167 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.167 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.167 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.167 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.167 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.167 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.167 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.167 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.167 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.168 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.168 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.168 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.168 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.168 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.168 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.168 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.168 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.168 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.168 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.168 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.168 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.168 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.168 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.168 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.169 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.169 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.169 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.169 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.169 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.169 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.169 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.169 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.169 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.169 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.169 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.169 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.169 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.169 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.169 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.170 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.170 INFO:teuthology.orchestra.run.smithi006.stdout:____ ERROR at teardown of test_object_presigned_put_object_with_acl_tenant _____ 2024-05-30T17:13:22.170 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.170 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.170 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.170 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.170 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.170 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.170 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.170 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.170 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.170 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.170 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.170 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.170 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.171 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.171 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.171 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.171 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.171 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.171 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.171 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.171 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.171 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.171 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.171 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.171 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.171 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.171 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.172 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.172 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.172 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.172 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.172 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.172 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.172 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.172 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.172 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.172 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.172 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.172 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.172 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.172 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.173 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.173 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.173 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.173 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.173 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.173 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.173 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.173 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.173 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.173 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.173 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.173 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.173 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.173 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.174 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.174 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.174 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.174 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.174 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.174 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.174 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.174 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.174 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.174 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.174 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.174 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.174 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.174 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.175 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.175 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.175 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.175 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.175 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.175 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.175 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.175 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.175 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.175 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.175 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.175 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.175 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.175 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.175 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.176 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.176 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.176 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.176 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.176 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.176 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.176 INFO:teuthology.orchestra.run.smithi006.stdout:________ ERROR at teardown of test_object_raw_put_authenticated_expired ________ 2024-05-30T17:13:22.176 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.176 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.176 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.176 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.176 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.176 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.176 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.177 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.177 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.177 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.177 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.177 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.177 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.177 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.177 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.177 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.177 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.177 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.177 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.177 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.177 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.177 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.178 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.178 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.178 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.178 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.178 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.178 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.178 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.178 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.178 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.178 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.178 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.178 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.178 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.178 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.178 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.179 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.179 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.179 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.179 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.179 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.179 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.179 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.179 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.179 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.179 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.179 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.179 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.179 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.179 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.179 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.180 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.180 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.180 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.180 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.180 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.180 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.180 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.180 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.180 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.180 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.180 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.180 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.180 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.180 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.181 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.181 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.181 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.181 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.181 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.181 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.181 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.181 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.181 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.181 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.181 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.181 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.181 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.181 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.182 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.182 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.182 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.182 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.182 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.182 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.182 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.182 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.182 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.182 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.182 INFO:teuthology.orchestra.run.smithi006.stdout:______ ERROR at teardown of test_bucket_create_naming_bad_starts_nonalpha ______ 2024-05-30T17:13:22.182 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.182 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.182 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.183 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.183 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.183 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.183 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.183 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.183 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.183 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.183 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.183 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.183 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.183 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.183 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.183 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.183 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.183 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.184 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.184 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.184 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.184 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.184 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.184 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.184 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.184 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.184 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.184 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.184 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.184 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.184 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.184 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.184 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.185 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.185 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.185 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.185 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.185 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.185 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.185 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.185 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.185 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.185 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.185 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.185 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.185 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.185 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.186 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.186 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.186 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.186 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.186 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.186 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.186 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.186 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.186 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.186 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.186 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.186 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.186 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.187 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.187 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.187 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.187 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.187 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.187 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.187 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.187 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.187 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.187 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.187 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.187 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.187 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.187 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.187 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.187 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.188 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.188 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.188 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.188 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.188 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.188 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.188 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.188 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.188 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.188 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.188 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.188 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.188 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.188 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.188 INFO:teuthology.orchestra.run.smithi006.stdout:_________ ERROR at teardown of test_bucket_create_naming_bad_short_one _________ 2024-05-30T17:13:22.189 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.189 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.189 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.189 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.189 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.189 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.189 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.189 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.189 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.189 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.189 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.189 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.189 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.189 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.189 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.189 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.190 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.190 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.190 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.190 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.190 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.190 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.190 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.190 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.190 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.190 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.190 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.190 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.190 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.191 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.191 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.191 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.193 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.193 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.193 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.193 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.193 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.193 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.193 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.193 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.193 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.193 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.194 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.194 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.194 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.194 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.194 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.194 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.194 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.194 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.194 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.194 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.194 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.194 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.194 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.194 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.194 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.195 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.195 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.195 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.195 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.195 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.195 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.195 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.195 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.195 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.195 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.195 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.195 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.195 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.195 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.195 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.196 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.196 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.196 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.196 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.196 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.196 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.196 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.196 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.196 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.196 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.196 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.196 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.196 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.196 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.196 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.196 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.197 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.197 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.197 INFO:teuthology.orchestra.run.smithi006.stdout:_________ ERROR at teardown of test_bucket_create_naming_bad_short_two _________ 2024-05-30T17:13:22.197 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.197 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.197 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.197 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.197 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.197 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.197 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.197 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.197 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.197 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.197 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.197 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.197 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.198 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.198 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.198 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.198 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.198 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.198 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.198 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.198 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.198 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.198 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.198 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.198 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.198 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.198 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.199 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.199 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.199 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.199 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.199 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.199 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.199 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.199 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.199 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.199 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.199 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.199 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.199 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.199 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.199 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.199 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.200 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.200 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.200 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.200 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.200 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.200 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.200 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.200 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.200 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.200 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.200 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.200 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.200 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.201 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.201 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.201 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.201 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.201 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.201 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.201 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.201 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.201 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.201 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.201 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.201 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.201 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.201 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.201 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.202 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.202 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.202 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.202 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.202 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.202 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.202 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.202 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.202 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.202 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.202 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.202 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.202 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.202 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.202 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.203 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.203 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.203 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.203 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.203 INFO:teuthology.orchestra.run.smithi006.stdout:_________ ERROR at teardown of test_bucket_create_naming_good_long_60 __________ 2024-05-30T17:13:22.203 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.203 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.203 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.203 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.203 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.203 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.203 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.203 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.203 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.203 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.203 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.204 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.204 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.204 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.204 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.204 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.204 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.204 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.204 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.204 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.204 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.204 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.204 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.204 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.204 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.204 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.205 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.205 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.205 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.205 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.205 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.205 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.205 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.205 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.205 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.205 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.205 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.205 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.205 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.205 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.205 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.205 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.206 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.206 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.206 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.206 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.206 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.206 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.206 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.206 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.206 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.206 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.206 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.206 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.206 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.207 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.207 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.207 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.207 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.207 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.207 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.207 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.207 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.207 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.207 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.207 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.207 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.207 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.207 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.208 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.208 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.208 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.208 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.208 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.208 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.208 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.208 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.208 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.208 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.208 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.208 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.208 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.208 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.208 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.209 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.209 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.209 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.209 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.209 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.209 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.209 INFO:teuthology.orchestra.run.smithi006.stdout:_________ ERROR at teardown of test_bucket_create_naming_good_long_61 __________ 2024-05-30T17:13:22.209 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.209 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.209 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.209 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.209 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.209 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.209 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.209 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.209 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.210 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.210 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.210 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.210 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.210 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.210 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.210 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.210 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.210 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.210 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.210 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.210 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.210 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.210 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.211 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.211 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.211 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.211 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.211 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.211 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.211 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.211 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.211 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.211 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.211 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.211 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.211 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.211 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.211 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.212 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.212 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.212 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.212 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.212 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.212 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.212 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.212 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.212 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.212 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.212 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.212 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.212 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.212 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.212 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.212 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.213 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.213 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.213 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.213 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.213 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.213 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.213 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.213 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.213 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.213 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.213 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.213 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.213 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.213 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.214 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.214 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.214 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.214 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.214 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.214 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.214 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.214 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.214 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.214 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.214 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.214 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.214 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.214 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.214 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.215 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.215 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.215 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.215 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.215 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.215 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.215 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.215 INFO:teuthology.orchestra.run.smithi006.stdout:_________ ERROR at teardown of test_bucket_create_naming_good_long_62 __________ 2024-05-30T17:13:22.215 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.215 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.215 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.215 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.215 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.216 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.216 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.216 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.216 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.216 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.216 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.216 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.216 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.216 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.216 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.216 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.216 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.216 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.216 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.216 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.217 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.217 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.217 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.217 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.217 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.217 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.217 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.217 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.217 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.217 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.217 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.217 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.217 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.217 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.217 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.217 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.218 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.218 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.218 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.218 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.218 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.218 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.218 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.218 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.218 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.218 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.218 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.218 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.218 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.219 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.219 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.219 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.219 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.219 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.219 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.219 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.219 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.219 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.219 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.219 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.219 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.219 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.219 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.220 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.220 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.220 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.220 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.220 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.220 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.220 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.220 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.220 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.220 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.220 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.220 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.220 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.220 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.220 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.221 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.221 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.221 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.221 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.221 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.221 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.221 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.221 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.221 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.221 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.221 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.221 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.221 INFO:teuthology.orchestra.run.smithi006.stdout:_________ ERROR at teardown of test_bucket_create_naming_good_long_63 __________ 2024-05-30T17:13:22.221 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.221 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.221 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.222 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.222 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.222 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.222 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.222 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.222 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.222 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.222 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.222 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.222 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.222 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.222 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.222 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.222 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.222 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.223 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.223 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.223 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.223 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.223 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.223 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.223 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.223 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.223 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.223 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.223 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.223 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.223 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.223 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.223 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.224 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.224 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.224 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.224 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.224 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.224 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.224 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.224 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.224 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.224 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.224 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.224 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.224 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.224 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.224 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.225 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.225 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.225 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.225 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.225 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.225 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.225 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.225 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.225 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.225 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.225 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.225 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.225 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.225 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.225 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.225 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.226 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.226 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.226 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.226 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.226 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.226 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.226 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.226 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.226 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.226 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.226 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.226 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.226 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.226 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.227 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.227 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.227 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.227 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.227 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.227 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.227 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.227 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.227 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.227 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.227 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.227 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.227 INFO:teuthology.orchestra.run.smithi006.stdout:_______________ ERROR at teardown of test_bucket_list_long_name ________________ 2024-05-30T17:13:22.227 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.228 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.228 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.228 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.228 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.228 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.228 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.228 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.228 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.228 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.228 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.228 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.228 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.228 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.228 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.228 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.229 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.229 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.229 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.229 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.229 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.229 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.229 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.229 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.229 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.229 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.229 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.229 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.229 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.229 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.229 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.230 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.230 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.230 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.230 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.230 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.230 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.230 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.230 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.230 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.230 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.230 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.230 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.230 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.230 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.231 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.231 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.231 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.231 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.231 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.231 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.231 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.231 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.231 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.231 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.231 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.231 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.231 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.231 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.231 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.232 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.232 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.232 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.232 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.232 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.232 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.232 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.232 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.232 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.232 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.232 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.232 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.232 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.232 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.232 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.232 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.233 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.233 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.233 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.233 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.233 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.233 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.233 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.233 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.233 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.233 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.233 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.233 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.233 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.233 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.233 INFO:teuthology.orchestra.run.smithi006.stdout:____________ ERROR at teardown of test_bucket_create_naming_bad_ip _____________ 2024-05-30T17:13:22.234 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.234 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.234 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.234 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.234 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.234 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.234 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.234 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.234 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.234 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.234 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.234 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.234 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.234 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.235 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.235 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.235 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.235 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.235 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.235 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.235 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.235 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.235 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.235 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.235 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.235 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.235 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.235 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.235 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.235 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.236 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.236 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.236 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.236 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.236 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.236 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.236 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.236 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.236 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.236 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.236 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.236 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.236 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.236 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.236 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.237 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.237 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.237 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.237 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.237 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.237 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.237 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.237 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.237 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.237 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.237 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.237 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.237 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.237 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.237 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.237 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.238 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.238 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.238 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.238 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.238 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.238 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.238 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.238 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.238 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.238 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.238 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.238 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.238 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.238 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.239 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.239 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.239 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.239 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.239 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.239 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.239 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.239 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.239 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.239 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.239 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.239 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.239 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.239 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.239 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.239 INFO:teuthology.orchestra.run.smithi006.stdout:________ ERROR at teardown of test_bucket_create_naming_dns_underscore _________ 2024-05-30T17:13:22.240 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.240 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.240 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.240 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.240 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.240 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.240 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.240 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.240 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.240 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.240 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.240 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.240 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.240 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.241 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.241 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.241 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.241 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.241 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.241 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.241 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.241 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.241 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.241 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.241 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.241 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.241 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.241 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.241 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.242 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.242 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.242 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.242 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.242 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.242 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.242 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.242 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.242 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.242 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.242 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.242 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.242 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.242 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.242 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.243 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.243 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.243 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.243 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.243 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.243 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.243 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.243 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.243 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.243 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.243 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.243 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.243 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.243 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.243 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.244 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.244 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.244 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.244 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.244 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.244 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.244 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.244 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.244 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.244 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.244 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.244 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.244 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.244 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.244 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.245 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.245 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.245 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.245 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.245 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.245 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.245 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.245 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.245 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.245 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.245 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.245 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.245 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.245 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.245 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.246 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.246 INFO:teuthology.orchestra.run.smithi006.stdout:___________ ERROR at teardown of test_bucket_create_naming_dns_long ____________ 2024-05-30T17:13:22.246 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.246 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.246 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.246 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.246 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.246 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.246 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.246 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.246 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.246 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.246 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.246 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.246 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.247 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.247 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.247 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.247 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.247 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.247 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.247 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.247 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.247 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.247 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.247 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.247 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.247 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.247 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.247 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.248 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.248 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.248 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.248 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.248 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.248 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.248 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.248 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.248 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.248 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.248 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.248 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.248 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.248 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.248 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.249 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.249 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.249 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.249 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.249 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.249 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.249 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.249 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.249 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.249 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.249 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.249 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.249 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.249 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.249 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.249 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.250 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.250 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.250 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.250 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.250 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.250 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.250 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.250 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.250 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.250 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.250 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.250 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.250 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.250 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.250 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.251 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.251 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.251 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.251 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.251 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.251 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.251 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.251 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.251 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.251 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.251 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.251 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.251 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.251 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.252 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.252 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.252 INFO:teuthology.orchestra.run.smithi006.stdout:________ ERROR at teardown of test_bucket_create_naming_dns_dash_at_end ________ 2024-05-30T17:13:22.252 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.252 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.252 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.252 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.252 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.252 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.252 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.252 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.252 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.252 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.252 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.253 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.253 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.253 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.253 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.253 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.253 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.253 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.253 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.253 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.253 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.253 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.253 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.253 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.253 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.254 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.254 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.254 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.254 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.254 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.254 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.254 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.254 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.254 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.254 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.254 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.254 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.254 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.254 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.254 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.255 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.255 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.255 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.255 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.255 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.255 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.255 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.255 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.255 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.255 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.255 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.255 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.255 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.255 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.255 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.256 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.256 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.256 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.256 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.256 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.256 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.256 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.256 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.256 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.256 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.256 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.256 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.256 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.256 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.257 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.257 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.257 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.257 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.257 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.257 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.257 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.257 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.257 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.257 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.257 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.257 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.257 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.257 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.257 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.258 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.258 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.258 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.258 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.258 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.258 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.258 INFO:teuthology.orchestra.run.smithi006.stdout:__________ ERROR at teardown of test_bucket_create_naming_dns_dot_dot __________ 2024-05-30T17:13:22.258 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.258 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.258 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.258 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.258 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.258 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.258 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.258 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.259 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.259 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.259 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.259 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.259 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.259 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.259 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.259 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.259 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.259 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.259 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.259 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.259 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.259 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.259 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.260 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.260 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.260 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.260 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.260 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.260 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.260 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.260 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.260 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.260 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.260 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.260 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.260 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.260 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.260 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.261 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.261 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.261 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.261 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.261 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.261 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.261 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.261 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.261 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.261 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.261 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.261 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.261 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.261 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.261 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.261 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.262 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.262 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.262 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.262 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.262 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.262 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.262 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.262 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.262 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.262 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.262 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.262 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.262 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.262 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.262 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.263 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.263 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.263 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.263 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.263 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.263 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.263 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.263 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.263 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.263 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.263 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.263 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.263 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.263 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.263 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.264 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.264 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.264 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.264 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.264 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.264 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.264 INFO:teuthology.orchestra.run.smithi006.stdout:_________ ERROR at teardown of test_bucket_create_naming_dns_dot_dash __________ 2024-05-30T17:13:22.264 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.264 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.264 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.264 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.264 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.264 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.264 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.264 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.265 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.265 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.265 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.265 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.265 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.265 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.265 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.265 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.265 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.265 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.265 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.265 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.265 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.265 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.265 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.266 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.266 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.266 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.266 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.266 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.266 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.266 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.266 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.266 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.266 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.266 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.266 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.266 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.267 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.267 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.267 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.267 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.267 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.267 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.267 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.267 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.267 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.267 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.267 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.267 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.267 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.267 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.267 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.268 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.268 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.268 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.268 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.268 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.268 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.268 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.268 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.268 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.268 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.268 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.268 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.268 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.268 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.268 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.269 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.269 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.269 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.269 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.269 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.269 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.269 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.269 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.269 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.269 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.269 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.269 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.269 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.269 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.269 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.269 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.270 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.270 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.270 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.270 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.270 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.270 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.270 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.270 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.270 INFO:teuthology.orchestra.run.smithi006.stdout:_________ ERROR at teardown of test_bucket_create_naming_dns_dash_dot __________ 2024-05-30T17:13:22.270 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.270 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.270 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.270 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.270 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.270 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.271 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.271 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.271 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.271 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.271 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.271 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.271 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.271 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.271 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.271 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.271 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.271 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.271 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.271 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.271 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.272 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.272 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.272 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.272 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.272 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.272 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.272 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.272 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.272 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.272 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.272 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.272 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.272 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.272 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.272 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.273 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.273 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.273 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.273 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.273 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.273 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.273 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.273 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.273 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.273 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.273 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.273 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.273 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.273 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.273 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.274 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.274 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.274 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.274 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.274 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.274 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.274 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.274 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.274 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.274 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.274 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.274 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.274 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.274 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.274 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.275 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.275 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.275 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.275 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.275 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.275 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.275 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.275 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.275 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.275 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.275 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.275 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.275 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.275 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.275 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.276 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.276 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.276 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.276 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.276 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.276 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.276 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.276 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.276 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.276 INFO:teuthology.orchestra.run.smithi006.stdout:________________ ERROR at teardown of test_bucket_create_exists ________________ 2024-05-30T17:13:22.276 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.276 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.276 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.276 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.276 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.276 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.277 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.277 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.277 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.277 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.277 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.277 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.277 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.277 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.277 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.277 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.277 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.277 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.277 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.277 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.277 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.278 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.278 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.278 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.278 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.278 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.278 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.278 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.278 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.278 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.278 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.278 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.278 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.278 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.278 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.278 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.279 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.279 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.279 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.279 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.279 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.279 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.279 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.279 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.279 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.279 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.279 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.279 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.279 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.280 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.280 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.280 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.280 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.280 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.280 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.280 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.280 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.280 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.280 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.280 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.280 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.280 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.280 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.280 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.281 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.281 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.281 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.281 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.281 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.281 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.281 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.281 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.281 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.281 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.281 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.281 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.281 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.281 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.281 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.281 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.282 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.282 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.282 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.282 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.282 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.282 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.282 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.282 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.282 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.282 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.282 INFO:teuthology.orchestra.run.smithi006.stdout:________________ ERROR at teardown of test_bucket_get_location _________________ 2024-05-30T17:13:22.282 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.282 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.282 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.283 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.283 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.283 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.283 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.283 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.283 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.283 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.283 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.283 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.283 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.283 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.283 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.283 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.283 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.283 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.284 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.284 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.284 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.284 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.284 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.284 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.284 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.284 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.284 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.284 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.284 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.284 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.284 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.284 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.284 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.285 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.285 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.285 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.285 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.285 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.285 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.285 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.285 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.285 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.285 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.285 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.285 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.285 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.285 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.285 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.286 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.286 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.286 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.286 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.286 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.286 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.286 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.286 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.286 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.286 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.286 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.286 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.286 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.286 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.286 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.287 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.287 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.287 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.287 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.287 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.287 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.287 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.287 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.287 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.287 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.287 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.287 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.287 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.287 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.287 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.288 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.288 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.288 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.288 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.288 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.288 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.288 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.288 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.288 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.288 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.288 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.288 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.288 INFO:teuthology.orchestra.run.smithi006.stdout:___________ ERROR at teardown of test_bucket_create_exists_nonowner ____________ 2024-05-30T17:13:22.288 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.288 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.289 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.289 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.289 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.289 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.289 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.289 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.289 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.289 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.289 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.289 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.289 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.289 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.289 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.289 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.290 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.290 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.290 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.290 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.290 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.290 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.290 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.290 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.290 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.290 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.290 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.290 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.290 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.290 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.290 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.291 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.291 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.291 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.291 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.291 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.291 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.291 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.291 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.291 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.291 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.291 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.291 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.291 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.291 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.291 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.292 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.292 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.292 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.292 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.292 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.292 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.292 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.292 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.292 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.292 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.292 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.292 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.293 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.293 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.293 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.293 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.293 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.293 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.293 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.293 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.293 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.293 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.293 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.293 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.293 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.294 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.294 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.294 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.294 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.294 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.294 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.294 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.294 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.294 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.294 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.294 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.294 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.294 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.294 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.295 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.295 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.295 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.295 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.295 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.295 INFO:teuthology.orchestra.run.smithi006.stdout:___________ ERROR at teardown of test_bucket_recreate_overwrite_acl ____________ 2024-05-30T17:13:22.295 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.295 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.295 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.295 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.295 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.295 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.295 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.295 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.296 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.296 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.296 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.296 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.296 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.296 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.296 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.296 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.296 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.296 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.296 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.296 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.296 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.296 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.296 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.297 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.297 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.297 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.297 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.297 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.297 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.297 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.297 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.297 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.297 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.297 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.297 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.297 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.297 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.297 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.297 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.298 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.298 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.298 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.298 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.298 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.298 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.298 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.298 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.298 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.298 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.298 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.298 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.298 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.298 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.299 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.299 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.299 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.299 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.299 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.299 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.299 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.299 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.299 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.299 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.299 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.299 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.299 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.299 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.299 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.300 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.300 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.300 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.300 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.300 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.300 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.300 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.300 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.300 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.300 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.300 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.300 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.300 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.300 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.300 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.300 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.301 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.301 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.301 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.301 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.301 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.301 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.301 INFO:teuthology.orchestra.run.smithi006.stdout:______________ ERROR at teardown of test_bucket_recreate_new_acl _______________ 2024-05-30T17:13:22.301 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.301 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.301 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.301 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.301 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.301 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.301 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.301 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.301 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.302 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.302 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.302 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.302 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.302 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.302 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.302 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.302 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.302 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.302 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.302 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.302 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.302 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.302 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.303 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.303 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.303 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.303 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.303 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.303 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.303 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.303 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.303 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.303 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.303 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.303 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.303 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.303 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.303 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.304 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.304 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.304 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.304 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.304 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.304 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.304 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.304 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.304 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.304 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.304 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.304 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.304 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.304 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.304 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.305 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.305 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.305 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.305 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.305 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.305 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.305 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.305 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.305 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.305 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.305 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.305 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.305 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.305 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.305 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.306 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.306 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.306 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.306 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.306 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.306 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.306 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.306 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.306 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.306 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.306 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.306 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.306 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.306 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.307 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.307 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.307 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.307 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.307 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.307 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.307 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.307 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.307 INFO:teuthology.orchestra.run.smithi006.stdout:_________________ ERROR at teardown of test_bucket_acl_default _________________ 2024-05-30T17:13:22.307 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.307 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.307 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.307 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.307 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.308 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.308 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.308 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.308 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.308 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.308 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.308 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.308 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.308 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.308 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.308 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.308 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.308 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.308 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.309 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.309 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.309 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.309 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.309 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.309 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.309 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.309 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.309 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.309 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.309 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.309 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.309 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.309 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.309 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.309 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.310 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.310 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.310 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.310 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.310 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.310 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.310 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.310 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.310 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.310 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.310 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.310 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.310 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.310 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.310 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.311 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.311 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.311 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.311 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.311 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.311 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.311 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.311 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.311 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.311 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.311 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.311 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.311 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.311 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.311 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.312 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.312 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.312 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.312 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.312 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.312 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.312 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.312 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.312 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.312 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.312 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.312 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.312 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.312 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.312 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.313 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.313 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.313 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.313 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.313 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.313 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.313 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.313 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.313 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.313 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.313 INFO:teuthology.orchestra.run.smithi006.stdout:__________ ERROR at teardown of test_bucket_acl_canned_during_create ___________ 2024-05-30T17:13:22.313 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.313 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.313 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.313 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.314 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.314 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.314 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.314 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.314 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.314 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.314 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.314 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.314 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.314 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.314 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.314 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.314 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.315 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.315 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.315 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.315 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.315 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.315 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.315 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.315 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.315 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.315 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.315 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.315 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.315 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.315 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.315 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.316 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.316 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.316 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.316 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.316 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.316 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.316 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.316 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.316 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.316 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.316 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.316 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.316 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.316 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.317 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.317 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.317 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.317 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.317 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.317 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.317 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.317 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.317 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.317 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.317 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.317 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.317 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.317 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.318 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.318 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.318 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.318 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.318 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.318 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.318 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.318 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.318 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.318 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.318 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.318 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.318 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.318 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.318 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.319 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.319 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.319 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.319 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.319 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.319 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.319 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.319 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.319 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.319 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.319 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.319 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.319 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.320 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.320 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.320 INFO:teuthology.orchestra.run.smithi006.stdout:_________________ ERROR at teardown of test_bucket_acl_canned __________________ 2024-05-30T17:13:22.320 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.320 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.320 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.320 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.320 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.320 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.320 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.320 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.320 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.320 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.320 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.321 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.321 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.321 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.321 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.321 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.321 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.321 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.321 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.321 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.321 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.321 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.321 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.321 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.321 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.321 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.321 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.322 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.322 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.322 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.322 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.322 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.322 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.322 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.322 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.322 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.322 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.322 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.322 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.322 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.322 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.323 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.323 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.323 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.323 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.323 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.323 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.323 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.323 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.323 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.323 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.323 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.323 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.323 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.323 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.323 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.323 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.324 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.324 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.324 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.324 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.324 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.324 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.324 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.324 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.324 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.324 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.324 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.324 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.324 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.324 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.324 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.325 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.325 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.325 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.325 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.325 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.325 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.325 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.325 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.325 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.325 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.325 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.325 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.325 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.325 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.325 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.325 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.326 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.326 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.326 INFO:teuthology.orchestra.run.smithi006.stdout:_________ ERROR at teardown of test_bucket_acl_canned_publicreadwrite __________ 2024-05-30T17:13:22.326 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.326 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.326 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.326 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.326 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.326 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.326 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.326 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.326 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.326 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.326 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.326 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.327 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.327 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.327 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.327 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.327 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.327 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.327 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.327 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.327 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.327 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.327 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.327 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.327 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.327 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.327 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.328 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.328 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.328 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.328 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.328 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.328 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.328 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.328 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.328 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.328 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.328 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.328 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.328 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.328 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.329 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.329 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.329 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.329 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.329 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.329 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.329 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.329 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.329 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.329 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.329 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.329 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.329 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.329 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.329 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.329 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.330 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.330 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.330 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.330 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.330 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.330 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.330 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.330 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.330 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.330 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.330 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.330 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.330 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.330 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.330 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.331 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.331 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.331 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.331 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.331 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.331 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.331 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.331 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.331 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.331 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.331 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.331 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.331 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.331 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.331 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.332 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.332 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.332 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.332 INFO:teuthology.orchestra.run.smithi006.stdout:________ ERROR at teardown of test_bucket_acl_canned_authenticatedread _________ 2024-05-30T17:13:22.332 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.332 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.332 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.332 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.332 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.332 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.332 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.332 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.332 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.332 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.332 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.333 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.333 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.333 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.333 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.333 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.333 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.333 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.333 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.333 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.333 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.333 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.333 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.333 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.333 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.333 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.334 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.334 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.334 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.334 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.334 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.334 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.334 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.334 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.334 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.334 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.334 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.334 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.334 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.335 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.335 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.335 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.335 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.335 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.335 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.335 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.335 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.335 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.335 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.335 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.335 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.335 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.335 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.335 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.336 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.336 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.336 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.336 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.336 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.336 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.336 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.336 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.336 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.336 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.336 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.336 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.336 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.336 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.336 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.337 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.337 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.337 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.337 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.337 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.337 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.337 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.337 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.337 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.337 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.337 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.337 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.337 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.337 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.337 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.337 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.338 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.338 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.338 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.338 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.338 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.338 INFO:teuthology.orchestra.run.smithi006.stdout:_________________ ERROR at teardown of test_object_acl_default _________________ 2024-05-30T17:13:22.338 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.338 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.338 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.338 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.338 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.338 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.338 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.339 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.339 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.339 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.339 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.339 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.339 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.339 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.339 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.339 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.339 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.339 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.339 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.339 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.339 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.339 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.340 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.340 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.340 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.340 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.340 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.340 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.340 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.340 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.340 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.340 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.340 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.340 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.340 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.340 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.340 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.341 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.341 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.341 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.341 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.341 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.341 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.341 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.341 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.341 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.341 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.341 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.341 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.341 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.341 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.341 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.341 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.342 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.342 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.342 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.342 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.342 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.342 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.342 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.342 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.342 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.342 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.342 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.342 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.342 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.342 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.343 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.343 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.343 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.343 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.343 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.343 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.343 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.343 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.343 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.343 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.343 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.343 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.343 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.343 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.343 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.343 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.344 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.344 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.344 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.344 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.344 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.344 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.344 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.344 INFO:teuthology.orchestra.run.smithi006.stdout:__________ ERROR at teardown of test_object_acl_canned_during_create ___________ 2024-05-30T17:13:22.344 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.344 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.344 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.344 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.344 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.344 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.345 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.345 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.345 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.345 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.345 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.345 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.345 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.345 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.345 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.345 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.345 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.345 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.345 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.345 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.346 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.346 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.346 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.346 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.346 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.346 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.346 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.346 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.346 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.346 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.346 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.346 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.346 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.346 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.347 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.347 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.347 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.347 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.347 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.347 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.347 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.347 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.347 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.347 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.347 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.347 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.347 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.347 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.347 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.348 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.348 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.348 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.348 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.348 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.348 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.348 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.348 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.348 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.348 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.348 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.348 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.348 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.348 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.349 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.349 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.349 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.349 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.349 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.349 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.349 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.349 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.349 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.349 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.349 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.349 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.349 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.349 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.349 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.349 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.350 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.350 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.350 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.350 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.350 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.350 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.350 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.350 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.350 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.350 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.350 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.350 INFO:teuthology.orchestra.run.smithi006.stdout:_________________ ERROR at teardown of test_object_acl_canned __________________ 2024-05-30T17:13:22.350 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.350 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.351 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.351 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.351 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.351 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.351 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.351 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.351 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.351 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.351 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.351 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.351 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.351 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.351 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.351 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.351 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.352 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.352 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.352 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.352 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.352 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.352 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.352 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.352 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.352 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.352 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.352 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.352 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.352 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.352 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.352 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.353 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.353 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.353 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.353 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.353 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.353 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.353 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.353 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.353 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.353 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.353 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.353 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.353 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.353 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.353 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.353 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.354 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.354 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.354 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.354 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.354 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.354 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.354 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.354 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.354 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.354 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.354 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.354 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.354 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.355 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.355 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.355 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.355 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.355 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.355 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.355 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.355 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.355 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.355 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.355 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.355 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.355 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.355 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.355 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.356 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.356 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.356 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.356 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.356 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.356 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.356 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.356 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.356 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.356 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.356 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.356 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.356 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.356 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.356 INFO:teuthology.orchestra.run.smithi006.stdout:_________ ERROR at teardown of test_object_acl_canned_publicreadwrite __________ 2024-05-30T17:13:22.357 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.357 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.357 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.357 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.357 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.357 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.357 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.357 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.357 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.357 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.357 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.357 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.357 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.357 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.357 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.358 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.358 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.358 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.358 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.358 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.358 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.358 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.358 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.358 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.358 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.358 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.358 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.359 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.359 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.359 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.359 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.359 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.359 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.359 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.359 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.359 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.359 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.359 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.359 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.359 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.360 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.360 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.360 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.360 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.360 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.360 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.360 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.360 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.360 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.360 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.360 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.360 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.360 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.360 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.361 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.361 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.361 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.361 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.361 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.361 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.361 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.361 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.361 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.361 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.361 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.361 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.361 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.361 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.361 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.362 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.362 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.362 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.362 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.362 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.362 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.362 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.362 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.362 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.362 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.362 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.362 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.362 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.363 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.363 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.363 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.363 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.363 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.363 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.363 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.363 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.363 INFO:teuthology.orchestra.run.smithi006.stdout:________ ERROR at teardown of test_object_acl_canned_authenticatedread _________ 2024-05-30T17:13:22.363 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.363 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.363 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.363 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.363 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.363 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.364 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.364 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.364 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.364 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.364 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.364 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.364 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.364 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.364 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.364 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.364 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.364 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.364 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.364 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.364 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.364 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.365 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.365 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.365 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.365 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.365 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.365 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.365 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.365 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.365 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.365 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.365 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.365 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.365 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.365 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.365 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.366 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.366 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.366 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.366 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.366 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.366 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.366 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.366 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.366 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.366 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.366 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.366 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.366 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.366 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.367 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.367 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.367 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.367 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.367 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.367 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.367 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.367 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.367 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.367 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.367 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.367 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.367 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.367 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.367 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.367 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.368 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.368 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.368 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.368 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.368 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.368 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.368 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.368 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.368 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.368 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.368 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.368 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.368 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.368 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.369 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.369 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.369 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.369 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.369 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.369 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.369 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.369 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.369 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.369 INFO:teuthology.orchestra.run.smithi006.stdout:_________ ERROR at teardown of test_object_acl_canned_bucketownerread __________ 2024-05-30T17:13:22.369 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.369 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.369 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.369 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.369 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.369 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.370 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.370 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.370 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.370 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.370 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.370 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.370 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.370 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.370 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.370 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.370 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.370 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.370 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.370 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.370 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.371 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.371 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.371 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.371 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.371 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.371 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.371 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.371 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.371 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.371 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.371 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.371 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.371 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.371 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.371 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.372 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.372 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.372 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.372 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.372 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.372 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.372 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.372 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.372 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.372 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.372 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.372 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.372 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.372 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.373 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.373 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.373 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.373 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.373 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.373 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.373 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.373 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.373 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.373 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.373 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.373 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.373 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.373 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.373 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.374 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.374 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.374 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.374 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.374 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.374 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.374 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.374 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.374 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.374 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.374 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.374 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.374 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.374 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.375 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.375 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.375 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.375 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.375 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.375 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.375 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.375 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.375 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.375 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.375 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.375 INFO:teuthology.orchestra.run.smithi006.stdout:______ ERROR at teardown of test_object_acl_canned_bucketownerfullcontrol ______ 2024-05-30T17:13:22.375 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.375 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.375 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.376 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.376 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.376 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.376 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.376 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.376 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.376 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.376 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.376 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.376 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.376 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.376 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.376 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.376 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.376 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.377 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.377 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.377 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.377 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.377 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.377 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.377 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.377 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.377 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.377 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.377 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.377 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.377 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.377 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.377 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.377 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.378 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.378 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.378 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.378 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.378 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.378 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.378 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.378 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.378 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.378 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.378 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.378 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.378 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.379 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.379 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.379 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.379 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.379 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.379 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.379 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.379 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.379 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.379 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.379 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.379 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.379 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.379 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.379 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.379 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.380 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.380 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.380 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.380 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.380 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.380 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.380 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.380 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.380 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.380 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.380 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.380 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.380 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.380 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.380 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.380 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.381 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.381 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.381 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.381 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.381 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.381 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.381 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.381 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.381 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.381 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.381 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.381 INFO:teuthology.orchestra.run.smithi006.stdout:________ ERROR at teardown of test_object_acl_full_control_verify_owner ________ 2024-05-30T17:13:22.381 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.381 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.381 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.382 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.382 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.382 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.382 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.382 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.382 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.382 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.382 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.382 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.382 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.382 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.382 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.382 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.382 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.383 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.383 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.383 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.383 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.383 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.383 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.383 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.383 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.383 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.383 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.383 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.383 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.383 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.383 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.383 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.384 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.384 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.384 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.384 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.384 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.384 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.384 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.384 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.384 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.384 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.384 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.384 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.384 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.384 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.384 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.385 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.385 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.385 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.385 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.385 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.385 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.385 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.385 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.385 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.385 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.385 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.385 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.385 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.386 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.386 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.386 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.386 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.386 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.386 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.386 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.386 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.386 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.386 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.386 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.386 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.386 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.386 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.387 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.387 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.387 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.387 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.387 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.387 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.387 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.387 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.387 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.387 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.387 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.387 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.387 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.387 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.388 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.388 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.388 INFO:teuthology.orchestra.run.smithi006.stdout:_____ ERROR at teardown of test_object_acl_full_control_verify_attributes ______ 2024-05-30T17:13:22.388 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.388 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.388 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.388 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.388 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.388 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.388 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.388 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.388 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.388 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.388 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.388 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.388 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.389 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.389 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.389 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.389 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.389 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.389 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.389 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.389 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.389 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.389 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.389 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.389 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.389 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.389 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.389 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.390 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.390 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.390 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.390 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.390 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.390 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.390 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.390 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.390 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.390 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.390 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.390 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.390 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.390 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.391 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.391 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.391 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.391 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.391 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.391 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.391 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.391 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.391 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.391 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.391 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.391 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.391 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.391 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.391 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.392 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.392 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.392 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.392 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.392 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.392 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.392 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.392 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.392 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.392 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.392 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.392 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.392 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.392 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.392 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.392 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.393 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.393 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.393 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.393 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.393 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.393 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.393 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.393 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.393 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.393 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.393 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.393 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.393 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.393 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.393 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.394 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.394 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.394 INFO:teuthology.orchestra.run.smithi006.stdout:________ ERROR at teardown of test_bucket_acl_canned_private_to_private ________ 2024-05-30T17:13:22.394 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.394 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.394 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.394 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.394 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.394 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.394 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.394 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.394 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.394 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.395 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.395 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.395 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.395 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.395 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.395 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.395 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.395 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.395 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.395 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.395 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.395 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.395 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.395 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.395 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.395 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.396 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.396 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.396 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.396 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.396 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.396 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.396 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.396 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.396 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.396 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.396 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.396 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.396 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.396 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.396 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.397 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.397 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.397 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.397 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.397 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.397 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.397 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.397 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.397 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.397 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.397 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.397 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.397 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.397 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.397 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.397 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.398 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.398 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.398 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.398 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.398 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.398 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.398 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.398 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.398 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.398 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.398 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.398 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.398 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.399 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.399 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.399 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.399 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.399 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.399 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.399 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.399 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.399 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.399 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.399 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.399 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.399 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.400 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.400 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.400 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.400 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.400 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.400 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.400 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.400 INFO:teuthology.orchestra.run.smithi006.stdout:_____________________ ERROR at teardown of test_object_acl _____________________ 2024-05-30T17:13:22.400 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.400 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.400 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.400 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.400 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.401 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.401 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.401 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.401 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.401 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.401 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.401 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.401 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.401 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.401 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.401 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.401 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.401 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.401 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.402 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.402 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.402 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.402 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.402 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.402 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.402 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.402 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.402 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.402 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.402 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.402 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.402 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.402 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.403 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.403 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.403 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.403 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.403 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.403 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.403 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.403 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.403 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.403 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.403 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.403 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.403 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.403 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.403 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.404 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.404 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.404 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.404 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.404 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.404 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.404 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.404 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.404 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.404 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.404 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.404 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.404 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.404 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.405 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.405 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.405 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.405 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.405 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.405 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.405 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.405 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.405 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.405 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.405 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.405 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.405 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.405 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.405 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.406 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.406 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.406 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.406 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.406 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.406 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.406 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.406 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.406 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.406 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.406 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.406 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.406 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.407 INFO:teuthology.orchestra.run.smithi006.stdout:__________________ ERROR at teardown of test_object_acl_write __________________ 2024-05-30T17:13:22.407 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.407 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.407 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.407 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.407 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.407 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.407 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.407 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.407 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.407 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.407 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.407 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.407 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.408 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.408 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.408 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.408 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.408 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.408 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.408 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.408 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.408 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.408 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.408 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.408 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.408 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.408 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.409 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.409 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.409 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.409 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.409 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.409 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.409 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.409 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.409 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.409 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.409 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.409 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.409 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.409 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.409 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.410 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.410 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.410 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.410 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.410 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.410 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.410 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.410 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.410 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.410 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.410 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.410 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.410 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.410 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.410 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.411 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.411 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.411 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.411 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.411 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.411 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.411 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.411 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.411 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.411 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.411 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.411 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.411 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.411 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.411 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.412 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.412 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.412 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.412 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.412 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.412 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.412 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.412 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.412 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.412 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.412 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.412 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.412 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.412 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.412 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.413 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.413 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.413 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.413 INFO:teuthology.orchestra.run.smithi006.stdout:________________ ERROR at teardown of test_object_acl_writeacp _________________ 2024-05-30T17:13:22.413 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.413 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.413 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.413 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.413 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.413 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.413 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.413 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.413 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.413 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.413 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.414 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.414 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.414 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.414 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.414 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.414 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.414 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.414 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.414 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.414 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.414 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.414 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.414 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.415 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.415 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.415 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.415 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.415 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.415 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.415 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.415 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.415 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.415 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.415 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.415 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.415 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.415 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.415 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.415 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.416 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.416 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.416 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.416 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.416 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.416 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.416 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.416 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.416 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.416 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.416 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.416 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.416 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.416 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.416 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.417 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.417 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.417 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.417 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.417 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.417 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.417 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.417 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.417 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.417 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.417 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.417 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.417 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.417 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.417 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.418 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.418 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.418 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.418 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.418 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.418 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.418 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.418 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.418 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.418 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.418 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.418 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.418 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.418 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.419 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.419 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.419 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.419 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.419 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.419 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.419 INFO:teuthology.orchestra.run.smithi006.stdout:__________________ ERROR at teardown of test_object_acl_read ___________________ 2024-05-30T17:13:22.419 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.419 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.419 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.419 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.419 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.419 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.419 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.420 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.420 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.420 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.420 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.420 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.420 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.420 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.420 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.420 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.420 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.420 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.420 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.420 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.420 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.421 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.421 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.421 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.421 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.421 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.421 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.421 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.421 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.421 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.421 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.421 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.421 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.421 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.421 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.421 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.422 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.422 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.422 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.422 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.422 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.422 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.422 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.422 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.422 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.422 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.422 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.422 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.422 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.422 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.423 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.423 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.423 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.423 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.423 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.423 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.423 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.423 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.423 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.423 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.423 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.423 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.423 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.423 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.424 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.424 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.424 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.424 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.424 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.424 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.424 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.424 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.424 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.424 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.424 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.424 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.424 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.425 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.425 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.425 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.425 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.425 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.425 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.425 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.425 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.425 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.425 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.425 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.425 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.425 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.425 INFO:teuthology.orchestra.run.smithi006.stdout:_________________ ERROR at teardown of test_object_acl_readacp _________________ 2024-05-30T17:13:22.426 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.426 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.426 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.426 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.426 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.426 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.426 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.426 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.426 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.426 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.426 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.426 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.426 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.427 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.427 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.427 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.427 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.427 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.427 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.427 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.427 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.427 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.427 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.427 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.427 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.427 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.427 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.427 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.428 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.428 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.428 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.428 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.428 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.428 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.428 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.428 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.428 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.428 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.428 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.428 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.428 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.428 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.428 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.429 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.429 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.429 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.429 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.429 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.429 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.429 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.429 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.429 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.429 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.429 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.429 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.429 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.429 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.429 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.430 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.430 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.430 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.430 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.430 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.430 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.430 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.430 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.430 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.430 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.430 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.430 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.430 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.430 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.431 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.431 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.431 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.431 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.431 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.431 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.431 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.431 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.431 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.431 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.431 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.431 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.431 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.431 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.431 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.432 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.432 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.432 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.432 INFO:teuthology.orchestra.run.smithi006.stdout:________ ERROR at teardown of test_bucket_acl_grant_userid_fullcontrol _________ 2024-05-30T17:13:22.432 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.432 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.432 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.432 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.432 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.432 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.432 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.432 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.432 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.433 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.433 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.433 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.433 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.433 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.433 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.433 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.433 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.433 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.433 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.433 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.433 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.433 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.433 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.434 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.434 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.434 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.434 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.434 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.434 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.434 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.434 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.434 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.434 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.434 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.434 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.434 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.435 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.435 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.435 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.435 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.435 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.435 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.435 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.435 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.435 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.435 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.435 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.435 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.435 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.435 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.435 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.436 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.436 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.436 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.436 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.436 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.436 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.436 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.436 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.436 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.436 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.436 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.436 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.436 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.436 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.437 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.437 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.437 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.437 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.437 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.437 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.437 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.437 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.437 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.437 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.437 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.437 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.437 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.438 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.438 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.438 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.438 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.438 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.438 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.438 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.438 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.438 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.438 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.438 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.438 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.438 INFO:teuthology.orchestra.run.smithi006.stdout:____________ ERROR at teardown of test_bucket_acl_grant_userid_read ____________ 2024-05-30T17:13:22.438 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.439 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.439 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.439 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.439 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.439 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.439 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.439 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.439 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.439 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.439 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.439 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.439 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.439 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.439 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.440 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.440 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.440 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.440 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.440 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.440 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.440 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.440 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.440 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.440 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.440 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.441 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.441 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.441 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.441 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.441 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.441 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.441 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.441 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.441 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.441 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.441 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.441 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.441 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.441 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.442 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.442 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.442 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.442 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.442 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.442 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.442 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.442 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.442 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.442 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.442 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.442 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.442 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.442 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.443 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.443 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.443 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.443 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.443 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.443 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.443 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.443 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.443 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.443 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.443 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.443 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.443 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.443 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.443 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.444 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.444 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.444 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.444 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.444 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.444 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.444 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.444 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.444 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.444 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.444 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.444 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.444 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.444 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.445 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.445 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.445 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.445 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.445 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.445 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.445 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.445 INFO:teuthology.orchestra.run.smithi006.stdout:__________ ERROR at teardown of test_bucket_acl_grant_userid_readacp ___________ 2024-05-30T17:13:22.445 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.445 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.445 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.445 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.445 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.445 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.446 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.446 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.446 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.446 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.446 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.446 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.446 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.446 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.446 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.446 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.446 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.446 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.446 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.446 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.447 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.447 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.447 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.447 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.447 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.447 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.447 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.447 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.447 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.447 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.447 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.447 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.447 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.447 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.447 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.448 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.448 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.448 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.448 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.448 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.448 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.448 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.448 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.448 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.448 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.448 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.448 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.448 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.448 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.448 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.449 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.449 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.449 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.449 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.449 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.449 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.449 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.449 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.449 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.449 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.449 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.449 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.449 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.449 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.449 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.449 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.450 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.450 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.450 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.450 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.450 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.450 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.450 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.450 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.450 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.450 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.450 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.450 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.450 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.450 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.451 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.451 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.451 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.451 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.451 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.451 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.451 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.451 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.451 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.451 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.451 INFO:teuthology.orchestra.run.smithi006.stdout:___________ ERROR at teardown of test_bucket_acl_grant_userid_write ____________ 2024-05-30T17:13:22.451 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.451 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.451 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.451 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.451 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.452 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.452 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.452 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.452 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.452 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.452 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.452 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.452 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.452 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.452 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.452 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.452 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.452 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.452 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.452 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.453 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.453 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.453 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.453 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.453 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.453 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.453 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.453 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.453 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.453 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.453 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.453 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.454 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.454 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.454 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.454 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.454 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.454 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.454 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.454 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.454 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.454 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.454 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.454 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.454 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.455 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.455 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.455 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.455 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.455 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.455 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.455 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.455 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.455 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.455 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.455 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.455 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.455 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.455 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.456 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.456 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.456 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.456 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.456 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.456 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.456 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.456 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.456 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.456 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.456 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.456 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.456 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.456 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.456 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.457 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.457 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.457 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.457 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.457 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.457 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.457 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.457 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.457 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.457 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.457 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.457 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.457 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.457 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.457 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.458 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.458 INFO:teuthology.orchestra.run.smithi006.stdout:__________ ERROR at teardown of test_bucket_acl_grant_userid_writeacp __________ 2024-05-30T17:13:22.458 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.458 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.458 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.458 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.458 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.458 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.458 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.458 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.458 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.458 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.458 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.458 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.459 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.459 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.459 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.459 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.459 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.459 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.459 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.459 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.459 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.459 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.459 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.459 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.459 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.459 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.459 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.460 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.460 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.460 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.460 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.460 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.460 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.460 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.460 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.460 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.460 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.460 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.460 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.460 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.460 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.460 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.461 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.461 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.461 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.461 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.461 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.461 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.461 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.461 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.461 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.461 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.461 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.461 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.461 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.461 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.461 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.461 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.462 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.462 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.462 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.462 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.462 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.462 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.462 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.462 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.462 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.462 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.462 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.462 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.462 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.462 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.463 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.463 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.463 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.463 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.463 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.463 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.463 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.463 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.463 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.463 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.463 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.463 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.463 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.463 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.463 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.464 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.464 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.464 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.464 INFO:teuthology.orchestra.run.smithi006.stdout:___________ ERROR at teardown of test_bucket_acl_grant_nonexist_user ___________ 2024-05-30T17:13:22.464 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.464 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.464 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.464 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.464 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.464 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.464 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.464 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.464 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.464 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.464 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.465 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.465 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.465 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.465 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.465 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.465 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.465 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.465 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.465 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.465 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.465 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.465 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.465 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.465 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.465 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.466 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.466 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.466 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.466 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.466 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.466 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.466 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.466 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.466 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.466 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.466 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.466 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.466 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.466 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.467 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.467 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.467 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.467 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.467 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.467 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.467 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.467 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.467 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.467 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.467 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.467 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.467 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.468 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.468 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.468 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.468 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.468 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.468 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.468 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.468 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.468 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.468 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.468 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.468 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.468 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.468 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.468 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.469 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.469 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.469 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.469 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.469 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.469 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.469 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.469 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.469 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.469 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.469 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.469 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.469 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.469 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.469 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.470 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.470 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.470 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.470 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.470 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.470 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.470 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.470 INFO:teuthology.orchestra.run.smithi006.stdout:______________ ERROR at teardown of test_object_header_acl_grants ______________ 2024-05-30T17:13:22.470 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.470 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.470 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.470 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.470 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.470 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.471 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.471 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.471 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.471 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.471 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.471 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.471 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.471 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.471 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.471 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.471 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.471 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.471 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.471 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.471 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.472 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.472 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.472 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.472 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.472 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.472 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.472 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.472 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.472 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.472 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.472 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.472 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.472 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.472 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.472 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.473 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.473 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.473 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.473 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.473 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.473 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.473 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.473 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.473 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.473 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.473 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.473 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.473 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.473 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.473 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.474 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.474 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.474 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.474 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.474 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.474 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.474 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.474 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.474 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.474 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.474 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.474 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.474 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.474 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.475 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.475 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.475 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.475 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.475 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.475 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.475 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.475 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.475 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.475 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.475 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.475 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.475 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.475 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.475 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.475 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.476 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.476 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.476 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.476 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.476 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.476 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.476 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.476 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.476 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.476 INFO:teuthology.orchestra.run.smithi006.stdout:______________ ERROR at teardown of test_bucket_header_acl_grants ______________ 2024-05-30T17:13:22.476 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.476 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.476 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.476 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.477 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.477 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.477 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.477 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.477 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.477 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.477 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.477 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.477 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.477 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.477 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.477 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.477 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.477 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.477 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.478 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.478 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.478 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.478 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.478 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.478 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.478 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.478 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.478 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.478 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.478 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.478 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.478 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.478 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.478 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.479 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.479 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.479 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.479 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.479 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.479 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.479 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.479 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.479 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.479 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.479 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.479 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.479 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.479 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.479 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.480 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.480 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.480 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.480 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.480 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.480 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.480 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.480 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.480 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.480 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.480 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.480 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.480 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.481 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.481 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.481 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.481 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.481 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.481 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.481 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.481 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.481 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.481 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.481 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.481 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.481 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.481 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.481 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.482 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.482 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.482 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.482 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.482 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.482 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.482 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.482 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.482 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.482 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.482 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.482 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.482 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.482 INFO:teuthology.orchestra.run.smithi006.stdout:_______________ ERROR at teardown of test_bucket_acl_grant_email _______________ 2024-05-30T17:13:22.483 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.483 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.483 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.483 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.483 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.483 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.483 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.483 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.483 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.483 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.483 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.483 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.483 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.483 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.483 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.484 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.484 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.484 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.484 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.484 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.484 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.484 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.484 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.484 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.484 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.484 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.484 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.484 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.484 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.484 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.484 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.485 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.485 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.485 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.485 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.485 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.485 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.485 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.485 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.485 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.485 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.485 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.485 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.485 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.485 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.485 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.486 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.486 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.486 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.486 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.486 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.486 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.486 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.486 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.486 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.486 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.486 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.486 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.486 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.486 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.486 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.487 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.487 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.487 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.487 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.487 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.487 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.487 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.487 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.487 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.487 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.487 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.487 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.487 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.487 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.487 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.488 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.488 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.488 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.488 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.488 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.488 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.488 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.488 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.488 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.488 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.488 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.488 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.488 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.488 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.488 INFO:teuthology.orchestra.run.smithi006.stdout:__________ ERROR at teardown of test_bucket_acl_grant_email_not_exist __________ 2024-05-30T17:13:22.489 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.489 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.489 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.489 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.489 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.489 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.489 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.489 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.489 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.489 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.489 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.489 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.489 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.489 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.489 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.489 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.490 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.490 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.490 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.490 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.490 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.490 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.490 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.490 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.490 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.490 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.490 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.490 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.490 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.490 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.490 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.491 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.491 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.491 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.491 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.491 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.491 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.491 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.491 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.491 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.491 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.491 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.491 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.491 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.491 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.491 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.492 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.492 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.492 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.492 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.492 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.492 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.492 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.492 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.492 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.492 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.492 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.492 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.492 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.492 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.492 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.492 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.493 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.493 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.493 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.493 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.493 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.493 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.493 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.493 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.493 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.493 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.493 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.493 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.493 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.493 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.494 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.494 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.494 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.494 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.494 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.494 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.494 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.494 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.494 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.494 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.494 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.494 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.494 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.494 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.494 INFO:teuthology.orchestra.run.smithi006.stdout:_______________ ERROR at teardown of test_bucket_acl_revoke_all ________________ 2024-05-30T17:13:22.495 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.495 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.495 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.495 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.495 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.495 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.495 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.495 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.495 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.495 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.495 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.495 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.495 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.495 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.495 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.495 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.496 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.496 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.496 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.496 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.496 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.496 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.496 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.496 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.496 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.496 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.496 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.496 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.496 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.496 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.496 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.497 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.497 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.497 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.497 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.497 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.497 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.497 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.497 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.497 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.497 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.497 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.497 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.497 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.497 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.497 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.498 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.498 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.498 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.498 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.498 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.498 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.498 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.498 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.498 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.498 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.498 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.498 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.498 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.498 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.498 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.499 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.499 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.499 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.499 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.499 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.499 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.499 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.499 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.499 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.499 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.499 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.499 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.499 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.499 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.499 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.500 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.500 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.500 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.500 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.500 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.500 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.500 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.500 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.500 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.500 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.500 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.500 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.500 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.500 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.500 INFO:teuthology.orchestra.run.smithi006.stdout:________ ERROR at teardown of test_access_bucket_private_object_private ________ 2024-05-30T17:13:22.501 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.501 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.501 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.501 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.501 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.501 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.501 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.501 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.501 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.501 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.501 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.501 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.501 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.501 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.501 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.501 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.502 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.502 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.502 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.502 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.502 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.502 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.502 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.502 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.502 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.502 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.502 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.502 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.502 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.502 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.502 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.503 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.503 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.503 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.503 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.503 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.503 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.503 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.503 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.503 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.503 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.503 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.503 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.503 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.503 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.503 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.503 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.504 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.504 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.504 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.504 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.504 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.504 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.504 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.504 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.504 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.504 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.504 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.504 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.504 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.504 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.504 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.505 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.505 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.505 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.505 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.505 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.505 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.505 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.505 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.505 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.505 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.505 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.505 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.505 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.505 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.505 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.505 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.506 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.506 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.506 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.506 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.506 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.506 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.506 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.506 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.506 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.506 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.506 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.506 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.506 INFO:teuthology.orchestra.run.smithi006.stdout:_______ ERROR at teardown of test_access_bucket_private_objectv2_private _______ 2024-05-30T17:13:22.506 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.507 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.507 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.507 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.507 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.507 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.507 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.507 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.507 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.507 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.507 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.507 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.507 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.507 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.507 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.508 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.508 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.508 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.508 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.508 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.508 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.508 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.508 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.508 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.508 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.508 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.508 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.508 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.508 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.509 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.509 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.509 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.509 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.509 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.509 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.509 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.509 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.509 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.509 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.509 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.509 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.509 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.510 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.510 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.510 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.510 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.510 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.510 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.510 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.510 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.510 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.510 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.510 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.510 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.510 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.511 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.511 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.511 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.511 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.511 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.511 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.511 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.511 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.511 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.511 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.511 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.511 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.511 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.512 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.512 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.512 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.512 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.512 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.512 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.512 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.512 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.512 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.512 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.512 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.512 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.512 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.512 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.513 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.513 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.513 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.513 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.513 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.513 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.513 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.513 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.513 INFO:teuthology.orchestra.run.smithi006.stdout:______ ERROR at teardown of test_access_bucket_private_object_publicread _______ 2024-05-30T17:13:22.513 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.513 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.513 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.513 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.513 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.513 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.514 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.514 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.514 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.514 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.514 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.514 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.514 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.514 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.514 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.514 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.514 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.514 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.514 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.514 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.515 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.515 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.515 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.515 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.515 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.515 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.515 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.515 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.515 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.515 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.515 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.515 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.515 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.516 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.516 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.516 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.516 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.516 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.516 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.516 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.516 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.516 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.516 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.516 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.516 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.516 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.516 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.516 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.517 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.517 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.517 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.517 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.517 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.517 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.517 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.517 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.517 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.517 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.517 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.517 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.517 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.517 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.517 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.518 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.518 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.518 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.518 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.518 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.518 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.518 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.518 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.518 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.518 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.518 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.518 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.518 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.518 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.518 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.519 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.519 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.519 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.519 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.519 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.519 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.519 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.519 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.519 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.519 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.519 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.519 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.519 INFO:teuthology.orchestra.run.smithi006.stdout:_____ ERROR at teardown of test_access_bucket_private_objectv2_publicread ______ 2024-05-30T17:13:22.519 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.520 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.520 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.520 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.520 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.520 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.520 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.520 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.520 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.520 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.520 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.520 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.520 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.521 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.521 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.521 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.521 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.521 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.521 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.521 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.521 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.521 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.521 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.521 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.521 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.521 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.521 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.521 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.522 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.522 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.522 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.522 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.522 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.522 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.522 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.522 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.522 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.522 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.522 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.522 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.522 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.522 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.523 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.523 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.523 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.523 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.523 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.523 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.523 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.523 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.523 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.523 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.523 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.523 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.523 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.523 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.523 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.523 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.524 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.524 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.524 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.524 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.524 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.524 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.524 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.524 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.524 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.524 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.524 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.524 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.524 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.524 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.524 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.525 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.525 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.525 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.525 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.525 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.525 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.525 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.525 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.525 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.525 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.525 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.525 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.525 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.525 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.525 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.525 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.526 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.526 INFO:teuthology.orchestra.run.smithi006.stdout:____ ERROR at teardown of test_access_bucket_private_object_publicreadwrite ____ 2024-05-30T17:13:22.526 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.526 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.526 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.526 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.526 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.526 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.526 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.526 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.526 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.526 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.526 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.526 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.526 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.527 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.527 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.527 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.527 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.527 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.527 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.527 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.527 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.527 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.527 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.527 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.527 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.527 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.527 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.527 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.528 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.528 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.528 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.528 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.528 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.528 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.528 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.528 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.528 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.528 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.528 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.528 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.528 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.528 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.528 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.528 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.529 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.529 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.529 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.529 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.529 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.529 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.529 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.529 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.529 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.529 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.529 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.529 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.529 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.529 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.529 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.530 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.530 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.530 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.530 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.530 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.530 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.530 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.530 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.530 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.530 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.530 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.530 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.530 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.530 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.531 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.531 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.531 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.531 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.531 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.531 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.531 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.531 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.531 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.531 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.531 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.531 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.531 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.531 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.531 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.531 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.532 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.532 INFO:teuthology.orchestra.run.smithi006.stdout:___ ERROR at teardown of test_access_bucket_private_objectv2_publicreadwrite ___ 2024-05-30T17:13:22.532 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.532 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.532 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.532 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.532 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.532 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.532 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.532 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.532 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.532 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.532 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.532 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.532 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.533 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.533 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.533 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.533 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.533 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.533 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.533 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.533 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.533 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.533 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.533 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.533 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.533 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.533 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.534 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.534 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.534 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.534 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.534 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.534 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.534 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.534 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.534 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.534 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.534 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.534 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.534 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.534 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.535 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.535 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.535 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.535 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.535 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.535 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.535 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.535 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.535 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.535 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.535 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.535 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.535 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.535 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.535 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.535 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.536 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.536 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.536 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.536 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.536 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.536 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.536 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.536 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.536 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.536 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.536 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.536 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.536 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.536 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.536 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.537 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.537 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.537 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.537 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.537 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.537 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.537 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.537 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.537 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.537 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.537 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.537 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.537 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.537 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.537 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.538 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.538 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.538 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.538 INFO:teuthology.orchestra.run.smithi006.stdout:______ ERROR at teardown of test_access_bucket_publicread_object_private _______ 2024-05-30T17:13:22.538 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.538 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.538 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.538 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.538 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.538 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.538 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.538 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.538 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.538 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.539 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.539 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.539 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.539 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.539 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.539 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.539 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.539 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.539 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.539 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.539 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.539 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.539 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.539 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.539 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.539 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.540 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.540 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.540 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.540 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.540 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.540 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.540 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.540 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.540 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.540 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.540 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.540 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.540 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.540 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.541 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.541 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.541 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.541 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.541 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.541 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.541 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.541 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.541 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.541 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.541 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.541 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.541 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.541 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.541 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.542 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.542 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.542 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.542 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.542 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.542 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.542 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.542 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.542 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.542 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.542 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.542 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.542 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.542 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.542 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.542 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.543 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.543 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.543 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.543 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.543 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.543 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.543 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.543 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.543 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.543 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.543 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.543 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.543 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.543 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.543 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.544 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.544 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.544 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.544 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.544 INFO:teuthology.orchestra.run.smithi006.stdout:_____ ERROR at teardown of test_access_bucket_publicread_object_publicread _____ 2024-05-30T17:13:22.544 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.544 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.544 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.544 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.544 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.544 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.544 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.544 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.544 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.544 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.545 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.545 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.545 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.545 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.545 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.545 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.545 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.545 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.545 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.545 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.545 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.545 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.545 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.545 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.545 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.545 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.546 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.546 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.546 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.546 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.546 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.546 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.546 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.546 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.546 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.546 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.546 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.546 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.546 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.547 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.547 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.547 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.547 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.547 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.547 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.547 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.547 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.547 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.547 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.547 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.547 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.547 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.547 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.547 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.548 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.548 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.548 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.548 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.548 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.548 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.548 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.548 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.548 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.548 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.548 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.548 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.548 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.548 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.548 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.549 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.549 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.549 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.549 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.549 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.549 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.549 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.549 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.549 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.549 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.549 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.549 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.549 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.549 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.549 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.549 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.550 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.550 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.550 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.550 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.550 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.550 INFO:teuthology.orchestra.run.smithi006.stdout:__ ERROR at teardown of test_access_bucket_publicread_object_publicreadwrite ___ 2024-05-30T17:13:22.550 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.550 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.550 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.550 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.550 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.550 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.550 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.550 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.551 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.551 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.551 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.551 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.551 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.551 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.551 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.551 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.551 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.551 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.551 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.551 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.551 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.551 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.551 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.552 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.552 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.552 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.552 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.552 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.552 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.552 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.552 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.552 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.552 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.552 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.552 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.552 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.552 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.552 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.553 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.553 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.553 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.553 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.553 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.553 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.553 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.553 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.553 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.553 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.553 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.553 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.553 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.553 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.553 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.554 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.554 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.554 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.554 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.554 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.554 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.554 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.554 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.554 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.554 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.554 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.554 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.554 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.554 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.555 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.555 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.555 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.555 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.555 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.555 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.555 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.555 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.555 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.555 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.555 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.555 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.555 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.555 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.555 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.556 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.556 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.556 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.556 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.556 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.556 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.556 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.556 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.556 INFO:teuthology.orchestra.run.smithi006.stdout:____ ERROR at teardown of test_access_bucket_publicreadwrite_object_private ____ 2024-05-30T17:13:22.556 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.556 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.556 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.556 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.556 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.556 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.557 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.557 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.557 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.557 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.557 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.557 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.557 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.557 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.557 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.557 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.557 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.557 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.557 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.557 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.557 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.557 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.558 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.558 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.558 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.558 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.558 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.558 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.558 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.558 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.558 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.558 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.558 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.558 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.558 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.559 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.559 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.559 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.559 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.559 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.559 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.559 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.559 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.559 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.559 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.559 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.559 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.559 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.559 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.560 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.560 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.560 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.560 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.560 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.560 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.560 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.560 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.560 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.560 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.560 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.560 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.560 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.560 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.560 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.561 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.561 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.561 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.561 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.561 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.561 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.561 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.561 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.561 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.561 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.561 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.561 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.561 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.561 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.561 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.561 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.562 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.562 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.562 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.562 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.562 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.562 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.562 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.562 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.562 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.562 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.562 INFO:teuthology.orchestra.run.smithi006.stdout:__ ERROR at teardown of test_access_bucket_publicreadwrite_object_publicread ___ 2024-05-30T17:13:22.562 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.562 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.562 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.562 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.563 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.563 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.563 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.563 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.563 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.563 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.563 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.563 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.563 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.563 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.563 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.563 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.563 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.563 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.564 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.564 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.564 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.564 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.564 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.564 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.564 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.564 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.564 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.564 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.564 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.564 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.564 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.564 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.564 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.564 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.565 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.565 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.565 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.565 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.565 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.565 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.565 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.565 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.565 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.565 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.565 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.565 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.565 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.565 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.566 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.566 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.566 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.566 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.566 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.566 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.566 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.566 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.566 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.566 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.566 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.566 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.566 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.566 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.567 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.567 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.567 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.567 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.567 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.567 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.567 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.567 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.567 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.567 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.567 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.567 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.567 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.567 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.567 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.567 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.568 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.568 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.568 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.568 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.568 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.568 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.568 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.568 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.568 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.568 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.568 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.568 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.568 INFO:teuthology.orchestra.run.smithi006.stdout:_ ERROR at teardown of test_access_bucket_publicreadwrite_object_publicreadwrite _ 2024-05-30T17:13:22.568 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.568 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.569 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.569 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.569 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.569 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.569 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.569 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.569 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.569 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.569 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.569 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.569 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.569 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.569 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.569 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.569 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.569 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.570 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.570 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.570 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.570 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.570 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.570 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.570 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.570 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.570 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.570 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.570 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.570 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.570 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.570 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.570 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.571 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.571 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.571 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.571 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.571 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.571 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.571 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.571 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.571 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.571 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.571 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.571 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.571 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.571 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.571 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.572 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.572 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.572 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.572 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.572 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.572 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.572 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.572 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.572 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.572 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.572 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.572 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.572 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.572 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.572 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.573 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.573 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.573 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.573 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.573 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.573 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.573 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.573 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.573 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.573 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.573 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.573 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.573 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.573 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.573 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.574 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.574 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.574 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.574 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.574 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.574 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.574 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.574 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.574 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.574 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.574 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.574 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.574 INFO:teuthology.orchestra.run.smithi006.stdout:______________ ERROR at teardown of test_buckets_create_then_list ______________ 2024-05-30T17:13:22.574 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.575 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.575 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.575 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.575 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.575 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.575 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.575 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.575 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.575 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.575 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.575 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.575 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.575 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.576 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.576 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.576 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.576 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.576 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.576 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.576 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.576 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.576 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.576 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.576 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.576 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.576 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.576 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.576 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.576 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.577 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.577 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.577 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.577 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.577 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.577 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.577 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.577 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.577 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.577 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.577 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.577 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.577 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.577 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.577 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.578 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.578 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.578 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.578 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.578 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.578 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.578 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.578 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.578 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.578 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.578 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.578 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.578 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.578 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.578 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.579 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.579 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.579 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.579 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.579 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.579 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.579 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.579 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.579 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.579 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.579 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.579 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.579 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.579 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.579 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.580 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.580 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.580 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.580 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.580 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.580 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.580 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.580 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.580 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.580 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.580 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.580 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.580 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.580 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.580 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.581 INFO:teuthology.orchestra.run.smithi006.stdout:_________________ ERROR at teardown of test_buckets_list_ctime _________________ 2024-05-30T17:13:22.581 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.581 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.581 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.581 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.581 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.581 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.581 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.581 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.581 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.581 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.581 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.581 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.581 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.581 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.581 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.582 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.582 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.582 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.582 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.582 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.582 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.582 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.582 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.582 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.582 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.582 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.582 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.582 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.582 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.583 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.583 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.583 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.583 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.583 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.583 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.583 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.583 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.583 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.583 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.583 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.583 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.583 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.583 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.583 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.583 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.584 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.584 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.584 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.584 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.584 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.584 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.584 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.584 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.584 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.584 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.584 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.584 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.584 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.584 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.584 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.585 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.585 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.585 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.585 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.585 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.585 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.585 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.585 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.585 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.585 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.585 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.585 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.585 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.585 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.586 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.586 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.586 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.586 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.586 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.586 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.586 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.586 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.586 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.586 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.586 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.586 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.586 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.587 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.587 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.587 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.587 INFO:teuthology.orchestra.run.smithi006.stdout:_______________ ERROR at teardown of test_list_buckets_anonymous _______________ 2024-05-30T17:13:22.587 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.587 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.587 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.587 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.587 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.587 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.587 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.587 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.587 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.587 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.587 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.588 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.588 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.588 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.588 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.588 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.588 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.588 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.588 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.588 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.588 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.588 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.588 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.588 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.588 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.588 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.589 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.589 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.589 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.589 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.589 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.589 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.589 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.589 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.589 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.589 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.589 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.589 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.589 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.589 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.589 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.590 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.590 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.590 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.590 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.590 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.590 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.590 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.590 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.590 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.590 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.590 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.590 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.590 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.590 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.590 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.591 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.591 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.591 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.591 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.591 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.591 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.591 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.591 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.591 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.591 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.591 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.591 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.591 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.591 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.591 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.591 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.592 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.592 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.592 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.592 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.592 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.592 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.592 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.592 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.592 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.592 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.592 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.592 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.592 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.592 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.592 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.593 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.593 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.593 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.593 INFO:teuthology.orchestra.run.smithi006.stdout:_____________ ERROR at teardown of test_list_buckets_invalid_auth ______________ 2024-05-30T17:13:22.593 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.593 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.593 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.593 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.593 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.593 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.593 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.593 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.593 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.593 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.593 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.594 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.594 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.594 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.594 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.594 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.594 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.594 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.594 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.594 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.594 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.594 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.594 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.594 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.594 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.594 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.595 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.595 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.595 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.595 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.595 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.595 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.595 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.595 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.595 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.595 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.595 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.595 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.595 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.595 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.596 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.596 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.596 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.596 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.596 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.596 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.596 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.596 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.596 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.596 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.596 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.596 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.596 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.596 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.596 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.597 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.597 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.597 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.597 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.597 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.597 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.597 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.597 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.597 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.597 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.597 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.597 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.597 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.597 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.597 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.597 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.598 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.598 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.598 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.598 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.598 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.598 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.598 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.598 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.598 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.598 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.598 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.598 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.598 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.598 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.599 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.599 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.599 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.599 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.599 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.599 INFO:teuthology.orchestra.run.smithi006.stdout:_______________ ERROR at teardown of test_list_buckets_bad_auth ________________ 2024-05-30T17:13:22.599 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.599 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.599 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.599 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.599 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.599 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.599 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.600 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.600 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.600 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.600 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.600 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.600 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.600 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.600 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.600 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.600 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.600 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.600 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.600 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.600 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.600 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.600 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.601 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.601 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.601 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.601 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.601 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.601 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.601 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.601 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.601 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.601 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.601 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.601 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.601 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.601 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.601 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.602 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.602 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.602 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.602 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.602 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.602 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.602 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.602 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.602 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.602 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.602 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.602 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.602 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.602 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.603 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.603 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.603 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.603 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.603 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.603 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.603 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.603 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.603 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.603 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.603 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.603 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.603 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.603 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.603 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.604 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.604 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.604 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.604 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.604 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.604 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.604 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.604 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.604 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.604 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.604 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.604 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.604 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.604 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.604 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.605 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.605 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.605 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.605 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.605 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.605 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.605 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.605 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.605 INFO:teuthology.orchestra.run.smithi006.stdout:_______ ERROR at teardown of test_bucket_create_naming_good_starts_alpha _______ 2024-05-30T17:13:22.605 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.605 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.605 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.605 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.605 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.605 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.605 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.606 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.606 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.606 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.606 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.606 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.606 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.606 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.606 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.606 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.606 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.606 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.606 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.606 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.606 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.607 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.607 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.607 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.607 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.607 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.607 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.607 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.607 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.607 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.607 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.607 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.607 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.607 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.607 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.607 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.607 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.608 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.608 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.608 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.608 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.608 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.608 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.608 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.608 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.608 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.608 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.608 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.608 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.608 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.608 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.608 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.609 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.609 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.609 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.609 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.609 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.609 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.609 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.609 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.609 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.609 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.609 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.609 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.609 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.609 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.609 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.610 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.610 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.610 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.610 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.610 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.610 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.610 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.610 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.610 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.610 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.610 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.610 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.610 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.610 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.611 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.611 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.611 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.611 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.611 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.611 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.611 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.611 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.611 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.611 INFO:teuthology.orchestra.run.smithi006.stdout:---------------------------- Captured stdout setup ----------------------------- 2024-05-30T17:13:22.611 INFO:teuthology.orchestra.run.smithi006.stdout:Done with cleanup of buckets in tests. 2024-05-30T17:13:22.611 INFO:teuthology.orchestra.run.smithi006.stdout:--------------------------- Captured stdout teardown --------------------------- 2024-05-30T17:13:22.611 INFO:teuthology.orchestra.run.smithi006.stdout:Done with cleanup of buckets in tests. 2024-05-30T17:13:22.611 INFO:teuthology.orchestra.run.smithi006.stdout:_______ ERROR at teardown of test_bucket_create_naming_good_starts_digit _______ 2024-05-30T17:13:22.611 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.612 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.612 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.612 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.612 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.612 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.612 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.612 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.612 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.612 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.612 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.612 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.612 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.612 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.612 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.613 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.613 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.613 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.613 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.613 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.613 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.613 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.613 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.613 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.613 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.613 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.613 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.613 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.613 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.613 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.613 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.614 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.614 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.614 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.614 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.614 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.614 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.614 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.614 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.614 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.614 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.614 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.614 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.614 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.614 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.615 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.615 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.615 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.615 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.615 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.615 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.615 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.615 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.615 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.615 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.615 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.615 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.615 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.615 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.615 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.616 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.616 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.616 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.616 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.616 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.616 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.616 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.616 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.616 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.616 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.616 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.616 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.616 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.616 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.617 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.617 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.617 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.617 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.617 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.617 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.617 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.617 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.617 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.617 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.617 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.617 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.617 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.617 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.617 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.618 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.618 INFO:teuthology.orchestra.run.smithi006.stdout:---------------------------- Captured stdout setup ----------------------------- 2024-05-30T17:13:22.618 INFO:teuthology.orchestra.run.smithi006.stdout:Done with cleanup of buckets in tests. 2024-05-30T17:13:22.618 INFO:teuthology.orchestra.run.smithi006.stdout:--------------------------- Captured stdout teardown --------------------------- 2024-05-30T17:13:22.618 INFO:teuthology.orchestra.run.smithi006.stdout:Done with cleanup of buckets in tests. 2024-05-30T17:13:22.618 INFO:teuthology.orchestra.run.smithi006.stdout:_____ ERROR at teardown of test_bucket_create_naming_good_contains_period ______ 2024-05-30T17:13:22.618 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.618 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.618 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.618 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.618 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.618 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.618 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.619 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.619 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.619 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.619 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.619 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.619 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.619 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.619 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.619 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.619 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.619 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.619 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.619 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.619 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.620 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.620 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.620 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.620 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.620 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.620 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.620 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.620 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.620 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.620 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.620 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.620 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.620 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.621 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.621 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.621 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.621 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.621 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.621 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.621 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.621 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.621 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.621 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.621 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.621 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.621 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.621 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.622 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.622 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.622 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.622 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.622 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.622 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.622 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.622 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.622 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.622 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.622 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.622 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.622 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.623 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.623 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.623 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.623 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.623 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.623 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.623 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.623 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.623 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.623 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.623 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.623 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.623 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.623 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.624 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.624 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.624 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.624 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.624 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.624 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.624 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.624 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.624 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.624 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.624 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.624 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.624 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.624 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.624 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.625 INFO:teuthology.orchestra.run.smithi006.stdout:_____ ERROR at teardown of test_bucket_create_naming_good_contains_hyphen ______ 2024-05-30T17:13:22.625 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.625 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.625 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.625 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.625 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.625 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.625 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.625 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.625 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.625 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.625 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.625 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.625 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.625 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.626 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.626 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.626 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.626 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.626 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.626 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.626 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.626 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.626 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.626 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.626 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.626 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.627 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.627 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.627 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.627 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.627 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.627 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.627 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.627 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.627 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.627 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.627 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.627 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.627 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.627 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.627 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.628 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.628 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.628 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.628 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.628 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.628 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.628 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.628 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.628 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.628 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.628 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.628 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.628 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.628 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.628 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.629 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.629 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.629 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.629 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.629 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.629 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.629 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.629 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.629 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.629 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.629 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.629 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.629 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.629 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.629 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.629 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.630 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.630 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.630 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.630 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.630 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.630 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.630 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.630 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.630 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.630 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.630 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.630 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.630 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.630 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.631 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.631 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.631 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.631 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.631 INFO:teuthology.orchestra.run.smithi006.stdout:___________ ERROR at teardown of test_bucket_recreate_not_overriding ___________ 2024-05-30T17:13:22.631 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.631 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.631 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.631 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.631 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.631 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.631 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.631 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.631 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.631 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.631 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.632 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.632 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.632 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.632 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.632 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.632 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.632 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.632 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.632 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.632 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.632 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.632 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.632 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.632 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.632 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.633 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.633 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.633 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.633 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.633 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.633 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.633 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.633 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.633 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.633 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.633 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.633 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.633 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.633 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.633 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.634 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.634 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.634 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.634 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.634 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.634 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.634 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.634 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.634 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.634 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.634 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.634 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.634 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.634 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.634 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.635 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.635 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.635 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.635 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.635 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.635 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.635 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.635 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.635 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.635 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.635 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.635 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.635 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.635 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.635 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.636 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.636 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.636 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.636 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.636 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.636 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.636 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.636 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.636 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.636 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.636 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.636 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.636 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.636 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.636 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.636 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.637 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.637 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.637 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.637 INFO:teuthology.orchestra.run.smithi006.stdout:__________ ERROR at teardown of test_bucket_create_special_key_names ___________ 2024-05-30T17:13:22.637 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.637 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.637 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.637 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.637 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.637 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.637 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.637 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.637 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.637 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.637 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.638 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.638 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.638 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.638 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.638 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.638 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.638 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.638 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.638 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.638 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.638 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.638 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.638 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.638 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.639 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.639 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.639 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.639 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.639 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.639 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.639 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.639 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.639 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.639 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.639 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.639 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.639 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.639 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.640 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.640 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.640 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.640 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.640 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.640 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.640 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.640 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.640 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.640 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.640 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.640 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.640 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.640 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.640 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.641 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.641 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.641 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.641 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.641 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.641 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.641 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.641 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.641 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.641 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.641 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.641 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.641 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.641 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.641 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.642 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.642 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.642 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.642 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.642 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.642 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.642 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.642 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.642 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.642 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.642 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.642 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.642 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.642 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.642 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.643 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.643 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.643 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.643 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.643 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.643 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.643 INFO:teuthology.orchestra.run.smithi006.stdout:_____________ ERROR at teardown of test_bucket_list_special_prefix _____________ 2024-05-30T17:13:22.643 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.643 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.643 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.643 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.643 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.643 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.643 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.643 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.644 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.644 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.644 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.644 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.644 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.644 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.644 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.644 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.644 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.644 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.644 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.644 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.644 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.644 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.644 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.644 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.645 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.645 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.645 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.645 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.645 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.645 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.645 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.645 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.645 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.645 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.645 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.645 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.645 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.645 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.645 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.646 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.646 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.646 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.646 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.646 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.646 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.646 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.646 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.646 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.646 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.646 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.646 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.646 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.646 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.646 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.647 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.647 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.647 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.647 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.647 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.647 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.647 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.647 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.647 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.647 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.647 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.647 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.647 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.647 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.648 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.648 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.648 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.648 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.648 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.648 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.648 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.648 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.648 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.648 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.648 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.648 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.648 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.648 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.648 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.649 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.649 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.649 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.649 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.649 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.649 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.649 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.649 INFO:teuthology.orchestra.run.smithi006.stdout:_______________ ERROR at teardown of test_object_copy_zero_size ________________ 2024-05-30T17:13:22.649 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.649 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.649 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.649 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.649 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.649 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.649 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.650 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.650 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.650 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.650 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.650 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.650 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.650 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.650 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.650 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.650 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.650 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.650 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.650 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.650 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.650 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.650 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.651 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.651 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.651 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.651 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.651 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.651 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.651 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.651 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.651 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.651 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.651 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.651 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.651 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.651 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.652 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.652 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.652 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.652 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.652 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.652 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.652 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.652 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.652 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.652 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.652 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.652 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.652 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.653 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.653 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.653 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.653 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.653 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.653 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.653 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.653 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.653 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.653 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.653 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.653 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.653 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.653 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.654 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.654 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.654 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.654 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.654 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.654 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.654 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.654 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.654 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.654 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.654 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.654 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.654 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.655 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.655 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.655 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.655 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.655 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.655 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.655 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.655 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.655 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.655 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.655 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.655 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.655 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.656 INFO:teuthology.orchestra.run.smithi006.stdout:__________________ ERROR at teardown of test_object_copy_16m ___________________ 2024-05-30T17:13:22.656 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.656 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.656 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.656 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.656 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.656 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.656 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.656 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.656 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.656 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.656 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.656 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.657 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.657 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.657 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.657 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.657 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.657 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.657 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.657 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.657 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.657 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.657 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.657 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.657 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.657 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.657 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.658 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.658 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.658 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.658 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.658 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.658 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.658 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.658 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.658 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.658 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.658 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.658 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.658 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.658 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.659 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.659 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.659 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.659 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.659 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.659 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.659 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.659 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.659 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.659 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.659 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.659 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.659 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.660 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.660 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.660 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.660 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.660 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.660 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.660 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.660 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.660 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.660 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.660 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.660 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.660 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.660 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.660 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.661 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.661 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.661 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.661 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.661 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.661 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.661 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.661 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.661 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.661 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.661 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.661 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.661 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.661 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.661 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.662 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.662 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.662 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.662 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.662 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.662 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.662 INFO:teuthology.orchestra.run.smithi006.stdout:______________ ERROR at teardown of test_object_copy_same_bucket _______________ 2024-05-30T17:13:22.662 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.662 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.662 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.662 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.662 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.662 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.663 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.663 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.663 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.663 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.663 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.663 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.663 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.663 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.663 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.663 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.663 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.663 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.663 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.663 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.664 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.664 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.664 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.664 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.664 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.664 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.664 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.664 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.664 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.664 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.664 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.664 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.664 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.665 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.665 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.665 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.665 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.665 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.665 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.665 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.665 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.665 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.665 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.665 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.665 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.665 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.665 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.665 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.666 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.666 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.666 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.666 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.666 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.666 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.666 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.666 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.666 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.666 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.666 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.666 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.666 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.667 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.667 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.667 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.667 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.667 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.667 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.667 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.667 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.667 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.667 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.667 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.667 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.667 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.667 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.667 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.668 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.668 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.668 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.668 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.668 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.668 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.668 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.668 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.668 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.668 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.668 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.668 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.668 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.668 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.668 INFO:teuthology.orchestra.run.smithi006.stdout:___________ ERROR at teardown of test_object_copy_verify_contenttype ___________ 2024-05-30T17:13:22.669 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.669 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.669 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.669 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.669 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.669 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.669 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.669 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.669 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.669 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.669 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.669 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.669 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.669 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.669 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.670 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.670 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.670 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.670 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.670 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.670 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.670 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.670 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.670 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.670 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.670 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.670 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.670 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.670 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.671 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.671 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.671 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.671 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.671 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.671 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.671 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.671 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.671 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.671 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.671 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.671 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.671 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.671 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.671 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.672 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.672 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.672 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.672 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.672 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.672 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.672 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.672 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.672 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.672 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.672 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.672 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.672 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.672 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.672 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.673 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.673 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.673 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.673 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.673 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.673 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.673 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.673 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.673 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.673 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.673 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.673 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.673 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.673 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.673 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.674 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.674 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.674 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.674 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.674 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.674 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.674 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.674 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.674 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.674 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.674 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.674 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.674 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.674 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.675 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.675 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.675 INFO:teuthology.orchestra.run.smithi006.stdout:_______________ ERROR at teardown of test_object_copy_to_itself ________________ 2024-05-30T17:13:22.675 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.675 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.675 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.675 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.675 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.675 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.675 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.675 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.675 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.675 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.675 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.675 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.676 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.676 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.676 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.676 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.676 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.676 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.676 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.676 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.676 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.676 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.676 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.676 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.676 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.676 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.676 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.676 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.677 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.677 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.677 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.677 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.677 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.677 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.677 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.677 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.677 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.677 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.677 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.677 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.677 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.677 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.677 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.678 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.678 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.678 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.678 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.678 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.678 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.678 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.678 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.678 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.678 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.678 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.678 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.678 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.678 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.679 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.679 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.679 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.679 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.679 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.679 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.679 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.679 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.679 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.679 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.679 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.679 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.679 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.679 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.680 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.680 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.680 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.680 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.680 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.680 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.680 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.680 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.680 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.680 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.680 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.680 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.680 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.680 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.681 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.681 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.681 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.681 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.681 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.681 INFO:teuthology.orchestra.run.smithi006.stdout:________ ERROR at teardown of test_object_copy_to_itself_with_metadata _________ 2024-05-30T17:13:22.681 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.681 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.681 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.681 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.681 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.681 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.681 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.681 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.681 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.682 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.682 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.682 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.682 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.682 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.682 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.682 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.682 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.682 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.682 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.682 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.682 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.682 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.682 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.683 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.683 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.683 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.683 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.683 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.683 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.683 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.683 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.683 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.683 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.683 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.683 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.683 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.683 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.683 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.684 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.684 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.684 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.684 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.684 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.684 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.684 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.684 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.684 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.684 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.684 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.684 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.684 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.684 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.684 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.685 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.685 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.685 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.685 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.685 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.685 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.685 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.685 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.685 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.685 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.685 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.685 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.685 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.685 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.685 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.686 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.686 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.686 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.686 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.686 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.686 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.686 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.686 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.686 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.686 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.686 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.686 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.686 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.686 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.687 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.687 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.687 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.687 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.687 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.687 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.687 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.687 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.687 INFO:teuthology.orchestra.run.smithi006.stdout:______________ ERROR at teardown of test_object_copy_diff_bucket _______________ 2024-05-30T17:13:22.687 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.687 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.687 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.687 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.687 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.687 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.687 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.688 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.688 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.688 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.688 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.688 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.688 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.688 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.688 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.688 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.688 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.688 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.688 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.688 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.689 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.689 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.689 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.689 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.689 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.689 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.689 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.689 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.689 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.689 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.689 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.689 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.689 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.689 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.689 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.689 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.690 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.690 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.690 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.690 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.690 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.690 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.690 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.690 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.690 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.690 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.690 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.690 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.690 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.690 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.691 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.691 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.691 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.691 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.691 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.691 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.691 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.691 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.691 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.691 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.691 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.691 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.691 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.691 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.691 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.691 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.692 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.692 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.692 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.692 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.692 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.692 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.692 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.692 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.692 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.692 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.692 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.692 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.692 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.692 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.692 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.693 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.693 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.693 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.693 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.693 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.693 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.693 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.693 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.693 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.693 INFO:teuthology.orchestra.run.smithi006.stdout:____________ ERROR at teardown of test_object_copy_not_owned_bucket ____________ 2024-05-30T17:13:22.693 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.693 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.693 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.693 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.694 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.694 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.694 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.694 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.694 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.694 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.694 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.694 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.694 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.694 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.694 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.694 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.694 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.694 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.695 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.695 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.695 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.695 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.695 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.695 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.695 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.695 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.695 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.695 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.695 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.695 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.695 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.695 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.695 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.696 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.696 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.696 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.696 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.696 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.696 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.696 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.696 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.696 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.696 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.696 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.696 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.696 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.696 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.696 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.697 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.697 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.697 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.697 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.697 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.697 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.697 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.697 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.697 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.697 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.697 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.697 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.697 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.697 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.697 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.698 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.698 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.698 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.698 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.698 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.698 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.698 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.698 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.698 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.698 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.698 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.698 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.698 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.698 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.699 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.699 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.699 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.699 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.699 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.699 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.699 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.699 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.699 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.699 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.699 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.699 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.699 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.699 INFO:teuthology.orchestra.run.smithi006.stdout:________ ERROR at teardown of test_object_copy_not_owned_object_bucket _________ 2024-05-30T17:13:22.700 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.700 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.700 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.700 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.700 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.700 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.700 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.700 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.700 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.700 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.700 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.700 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.700 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.700 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.700 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.700 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.701 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.701 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.701 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.701 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.701 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.701 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.701 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.701 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.701 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.701 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.701 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.701 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.701 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.701 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.701 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.702 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.702 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.702 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.702 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.702 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.702 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.702 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.702 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.702 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.702 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.702 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.702 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.702 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.702 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.703 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.703 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.703 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.703 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.703 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.703 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.703 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.703 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.703 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.703 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.703 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.703 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.703 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.703 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.703 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.703 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.704 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.704 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.704 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.704 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.704 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.704 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.704 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.704 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.704 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.704 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.704 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.704 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.704 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.704 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.704 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.705 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.705 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.705 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.705 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.705 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.705 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.705 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.705 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.705 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.705 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.705 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.705 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.705 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.705 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.705 INFO:teuthology.orchestra.run.smithi006.stdout:_______________ ERROR at teardown of test_object_copy_canned_acl _______________ 2024-05-30T17:13:22.706 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.706 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.706 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.706 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.706 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.706 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.706 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.706 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.706 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.706 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.706 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.706 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.707 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.707 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.707 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.707 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.707 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.707 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.707 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.707 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.707 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.707 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.707 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.707 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.707 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.707 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.707 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.707 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.708 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.708 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.708 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.708 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.708 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.708 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.708 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.708 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.708 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.708 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.708 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.708 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.708 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.708 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.708 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.709 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.709 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.709 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.709 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.709 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.709 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.709 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.709 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.709 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.709 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.709 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.709 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.709 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.709 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.709 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.710 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.710 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.710 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.710 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.710 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.710 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.710 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.710 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.710 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.710 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.710 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.710 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.710 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.711 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.711 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.711 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.711 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.711 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.711 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.711 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.711 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.711 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.711 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.711 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.711 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.711 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.711 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.711 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.712 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.712 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.712 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.712 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.712 INFO:teuthology.orchestra.run.smithi006.stdout:___________ ERROR at teardown of test_object_copy_retaining_metadata ___________ 2024-05-30T17:13:22.712 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.712 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.712 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.712 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.712 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.712 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.712 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.712 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.712 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.712 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.713 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.713 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.713 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.713 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.713 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.713 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.713 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.713 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.713 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.713 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.713 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.713 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.713 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.713 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.713 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.714 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.714 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.714 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.714 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.714 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.714 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.714 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.714 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.714 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.714 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.714 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.714 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.714 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.714 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.715 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.715 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.715 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.715 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.715 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.715 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.715 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.715 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.715 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.715 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.715 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.715 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.715 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.715 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.715 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.716 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.716 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.716 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.716 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.716 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.716 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.716 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.716 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.716 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.716 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.716 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.716 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.716 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.716 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.716 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.717 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.717 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.717 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.717 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.717 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.717 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.717 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.717 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.717 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.717 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.717 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.717 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.717 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.717 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.717 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.717 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.718 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.718 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.718 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.718 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.718 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.718 INFO:teuthology.orchestra.run.smithi006.stdout:___________ ERROR at teardown of test_object_copy_replacing_metadata ___________ 2024-05-30T17:13:22.718 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.718 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.718 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.718 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.718 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.718 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.718 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.719 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.719 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.719 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.719 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.719 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.719 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.719 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.719 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.719 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.719 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.719 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.719 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.719 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.720 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.720 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.720 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.720 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.720 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.720 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.720 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.720 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.720 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.720 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.720 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.720 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.720 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.720 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.721 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.721 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.721 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.721 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.721 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.721 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.721 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.721 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.721 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.721 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.721 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.721 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.721 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.721 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.722 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.722 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.722 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.722 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.722 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.722 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.722 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.722 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.722 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.722 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.722 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.722 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.722 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.722 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.723 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.723 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.723 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.723 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.723 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.723 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.723 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.723 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.723 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.723 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.723 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.723 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.723 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.723 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.723 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.723 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.724 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.724 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.724 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.724 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.724 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.724 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.724 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.724 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.724 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.724 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.724 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.724 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.724 INFO:teuthology.orchestra.run.smithi006.stdout:____________ ERROR at teardown of test_object_copy_bucket_not_found ____________ 2024-05-30T17:13:22.724 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.724 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.725 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.725 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.725 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.725 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.725 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.725 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.725 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.725 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.725 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.725 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.725 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.725 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.725 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.725 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.725 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.726 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.726 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.726 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.726 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.726 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.726 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.726 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.726 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.726 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.726 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.726 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.726 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.726 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.726 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.726 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.727 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.727 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.727 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.727 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.727 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.727 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.727 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.727 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.727 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.727 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.727 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.727 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.727 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.727 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.728 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.728 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.728 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.728 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.728 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.728 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.728 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.728 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.728 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.728 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.728 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.728 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.728 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.728 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.728 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.729 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.729 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.729 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.729 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.729 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.729 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.729 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.729 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.729 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.729 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.729 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.729 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.729 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.729 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.730 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.730 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.730 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.730 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.730 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.730 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.730 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.730 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.730 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.730 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.730 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.730 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.730 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.730 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.730 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.731 INFO:teuthology.orchestra.run.smithi006.stdout:_____________ ERROR at teardown of test_object_copy_key_not_found ______________ 2024-05-30T17:13:22.731 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.731 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.731 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.731 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.731 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.731 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.731 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.731 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.731 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.731 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.731 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.731 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.731 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.731 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.732 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.732 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.732 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.732 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.732 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.732 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.732 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.732 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.732 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.732 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.732 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.732 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.732 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.732 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.733 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.733 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.733 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.733 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.733 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.733 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.733 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.733 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.733 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.733 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.733 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.733 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.733 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.733 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.734 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.734 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.734 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.734 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.734 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.734 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.734 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.734 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.734 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.734 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.734 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.734 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.734 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.735 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.735 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.735 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.735 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.735 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.735 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.735 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.735 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.735 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.735 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.735 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.735 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.735 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.735 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.735 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.736 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.736 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.736 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.736 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.736 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.736 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.736 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.736 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.736 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.736 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.736 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.736 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.736 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.737 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.737 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.737 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.737 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.737 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.737 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.737 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.737 INFO:teuthology.orchestra.run.smithi006.stdout:____________ ERROR at teardown of test_object_copy_versioned_bucket ____________ 2024-05-30T17:13:22.737 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.737 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.737 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.737 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.737 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.737 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.737 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.738 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.738 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.738 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.738 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.738 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.738 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.738 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.738 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.738 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.738 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.738 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.738 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.738 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.739 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.739 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.739 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.739 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.739 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.739 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.739 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.739 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.739 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.739 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.739 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.739 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.739 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.739 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.739 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.740 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.740 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.740 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.740 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.740 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.740 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.740 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.740 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.740 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.740 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.740 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.740 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.740 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.740 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.740 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.740 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.741 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.741 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.741 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.741 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.741 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.741 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.741 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.741 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.741 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.741 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.741 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.741 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.741 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.741 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.741 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.742 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.742 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.742 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.742 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.742 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.742 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.742 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.742 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.742 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.742 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.742 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.742 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.742 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.742 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.742 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.743 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.743 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.743 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.743 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.743 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.743 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.743 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.743 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.743 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.743 INFO:teuthology.orchestra.run.smithi006.stdout:_________ ERROR at teardown of test_object_copy_versioned_url_encoding _________ 2024-05-30T17:13:22.743 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.743 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.743 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.743 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.743 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.743 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.744 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.744 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.744 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.744 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.744 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.744 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.744 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.744 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.744 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.744 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.744 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.744 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.744 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.744 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.744 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.745 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.745 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.745 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.745 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.745 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.745 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.745 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.745 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.745 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.745 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.745 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.745 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.745 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.745 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.745 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.745 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.746 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.746 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.746 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.746 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.746 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.746 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.746 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.746 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.746 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.746 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.746 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.746 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.746 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.747 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.747 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.747 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.747 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.747 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.747 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.747 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.747 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.747 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.747 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.747 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.747 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.747 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.747 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.747 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.748 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.748 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.748 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.748 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.748 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.748 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.748 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.748 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.748 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.748 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.748 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.748 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.748 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.748 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.749 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.749 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.749 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.749 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.749 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.749 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.749 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.749 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.749 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.749 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.749 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.749 INFO:teuthology.orchestra.run.smithi006.stdout:______ ERROR at teardown of test_object_copy_versioning_multipart_upload _______ 2024-05-30T17:13:22.749 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.749 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.749 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.749 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.750 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.750 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.750 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.750 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.750 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.750 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.750 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.750 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.750 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.750 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.750 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.750 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.750 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.750 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.751 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.751 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.751 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.751 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.751 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.751 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.751 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.751 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.751 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.751 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.751 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.751 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.751 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.751 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.751 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.752 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.752 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.752 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.752 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.752 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.752 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.752 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.752 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.752 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.752 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.752 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.752 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.752 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.752 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.752 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.753 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.753 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.753 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.753 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.753 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.753 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.753 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.753 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.753 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.753 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.753 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.753 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.753 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.753 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.753 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.754 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.754 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.754 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.754 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.754 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.754 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.754 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.754 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.754 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.754 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.754 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.754 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.754 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.754 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.754 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.755 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.755 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.755 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.755 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.755 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.755 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.755 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.755 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.755 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.755 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.755 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.755 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.755 INFO:teuthology.orchestra.run.smithi006.stdout:_______________ ERROR at teardown of test_multipart_upload_empty _______________ 2024-05-30T17:13:22.755 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.755 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.756 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.756 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.756 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.756 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.756 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.756 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.756 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.756 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.756 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.756 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.756 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.756 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.756 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.756 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.756 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.757 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.757 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.757 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.757 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.757 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.757 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.757 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.757 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.757 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.757 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.757 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.757 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.757 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.757 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.757 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.757 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.758 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.758 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.758 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.758 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.758 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.758 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.758 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.758 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.758 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.758 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.758 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.758 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.758 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.758 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.759 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.759 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.759 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.759 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.759 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.759 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.759 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.759 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.759 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.759 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.759 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.759 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.759 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.759 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.760 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.760 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.760 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.760 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.760 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.760 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.760 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.760 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.760 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.760 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.760 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.760 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.760 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.760 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.760 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.760 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.761 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.761 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.761 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.761 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.761 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.761 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.761 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.761 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.761 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.761 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.761 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.761 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.761 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.761 INFO:teuthology.orchestra.run.smithi006.stdout:_______________ ERROR at teardown of test_multipart_upload_small _______________ 2024-05-30T17:13:22.761 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.762 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.762 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.762 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.762 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.762 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.762 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.762 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.762 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.762 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.762 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.762 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.762 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.762 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.762 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.762 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.763 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.763 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.763 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.763 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.763 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.763 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.763 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.763 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.763 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.763 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.763 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.763 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.763 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.763 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.763 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.764 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.764 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.764 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.764 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.764 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.764 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.764 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.764 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.764 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.764 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.764 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.764 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.764 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.764 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.764 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.764 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.765 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.765 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.765 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.765 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.765 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.765 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.765 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.765 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.765 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.765 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.765 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.765 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.765 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.765 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.765 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.766 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.766 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.766 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.766 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.766 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.766 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.766 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.766 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.766 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.766 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.766 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.766 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.766 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.766 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.767 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.767 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.767 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.767 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.767 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.767 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.767 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.767 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.767 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.767 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.767 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.767 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.767 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.767 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.767 INFO:teuthology.orchestra.run.smithi006.stdout:________________ ERROR at teardown of test_multipart_copy_small ________________ 2024-05-30T17:13:22.767 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.768 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.768 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.768 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.768 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.768 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.768 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.768 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.768 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.768 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.768 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.768 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.768 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.768 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.768 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.768 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.769 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.769 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.769 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.769 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.769 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.769 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.769 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.769 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.769 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.769 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.769 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.769 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.769 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.769 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.770 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.770 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.770 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.770 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.770 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.770 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.770 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.770 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.770 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.770 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.770 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.770 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.770 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.770 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.770 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.771 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.771 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.771 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.771 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.771 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.771 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.771 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.771 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.771 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.771 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.771 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.771 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.771 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.771 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.771 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.772 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.772 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.772 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.772 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.772 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.772 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.772 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.772 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.772 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.772 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.772 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.772 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.772 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.772 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.772 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.773 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.773 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.773 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.773 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.773 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.773 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.773 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.773 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.773 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.773 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.773 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.773 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.773 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.773 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.773 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.774 INFO:teuthology.orchestra.run.smithi006.stdout:____________ ERROR at teardown of test_multipart_copy_invalid_range ____________ 2024-05-30T17:13:22.774 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.774 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.774 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.774 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.774 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.774 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.774 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.774 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.774 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.774 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.774 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.774 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.774 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.775 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.775 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.775 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.775 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.775 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.775 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.775 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.775 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.775 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.775 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.775 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.775 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.775 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.775 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.775 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.776 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.776 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.776 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.776 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.776 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.776 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.776 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.776 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.776 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.776 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.776 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.776 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.776 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.776 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.776 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.777 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.777 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.777 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.777 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.777 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.777 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.777 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.777 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.777 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.777 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.777 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.777 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.777 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.777 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.777 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.777 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.778 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.778 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.778 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.778 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.778 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.778 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.778 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.778 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.778 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.778 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.778 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.778 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.778 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.779 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.779 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.779 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.779 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.779 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.779 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.779 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.779 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.779 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.779 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.779 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.779 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.779 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.779 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.779 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.779 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.780 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.780 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.780 INFO:teuthology.orchestra.run.smithi006.stdout:____________ ERROR at teardown of test_multipart_copy_without_range ____________ 2024-05-30T17:13:22.780 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.780 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.780 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.780 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.780 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.780 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.780 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.780 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.780 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.780 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.780 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.780 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.781 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.781 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.781 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.781 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.781 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.781 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.781 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.781 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.781 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.781 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.781 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.781 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.781 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.781 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.781 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.782 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.782 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.782 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.782 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.782 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.782 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.782 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.782 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.782 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.782 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.782 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.782 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.782 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.782 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.782 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.783 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.783 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.783 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.783 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.783 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.783 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.783 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.783 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.783 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.783 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.783 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.783 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.783 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.783 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.783 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.783 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.784 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.784 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.784 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.784 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.784 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.784 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.784 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.784 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.784 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.784 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.784 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.784 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.784 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.784 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.785 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.785 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.785 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.785 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.785 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.785 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.785 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.785 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.785 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.785 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.785 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.785 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.785 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.785 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.785 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.786 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.786 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.786 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.786 INFO:teuthology.orchestra.run.smithi006.stdout:____________ ERROR at teardown of test_multipart_copy_special_names ____________ 2024-05-30T17:13:22.786 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.786 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.786 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.786 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.786 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.786 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.786 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.786 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.786 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.786 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.786 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.787 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.787 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.787 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.787 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.787 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.787 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.787 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.787 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.787 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.787 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.787 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.787 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.787 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.787 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.787 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.788 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.788 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.788 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.788 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.788 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.788 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.788 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.788 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.788 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.788 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.788 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.788 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.788 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.788 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.788 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.789 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.789 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.789 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.789 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.789 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.789 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.789 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.789 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.789 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.789 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.789 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.789 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.789 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.789 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.790 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.790 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.790 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.790 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.790 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.790 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.790 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.790 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.790 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.790 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.790 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.790 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.790 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.790 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.791 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.791 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.791 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.791 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.791 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.791 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.791 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.791 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.791 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.791 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.791 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.791 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.791 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.791 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.791 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.792 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.792 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.792 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.792 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.792 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.792 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.792 INFO:teuthology.orchestra.run.smithi006.stdout:__________________ ERROR at teardown of test_multipart_upload __________________ 2024-05-30T17:13:22.792 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.792 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.792 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.792 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.792 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.792 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.792 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.793 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.793 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.793 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.793 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.793 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.793 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.793 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.793 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.793 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.793 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.793 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.793 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.793 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.793 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.793 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.794 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.794 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.794 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.794 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.794 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.794 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.794 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.794 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.794 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.794 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.794 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.794 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.794 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.794 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.794 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.794 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.795 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.795 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.795 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.795 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.795 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.795 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.795 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.795 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.795 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.795 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.795 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.795 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.795 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.795 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.795 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.796 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.796 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.796 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.796 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.796 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.796 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.796 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.796 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.796 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.796 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.796 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.796 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.796 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.796 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.797 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.797 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.797 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.797 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.797 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.797 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.797 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.797 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.797 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.797 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.797 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.797 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.797 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.797 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.797 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.797 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.798 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.798 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.798 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.798 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.798 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.798 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.798 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.798 INFO:teuthology.orchestra.run.smithi006.stdout:______________ ERROR at teardown of test_multipart_copy_versioned ______________ 2024-05-30T17:13:22.798 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.798 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.798 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.798 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.798 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.799 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.799 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.799 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.799 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.799 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.799 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.799 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.799 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.799 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.799 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.799 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.799 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.800 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.800 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.800 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.800 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.800 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.800 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.800 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.800 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.800 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.800 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.800 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.800 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.800 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.800 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.800 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.801 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.801 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.801 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.801 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.801 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.801 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.801 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.801 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.801 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.801 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.801 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.801 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.801 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.801 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.801 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.801 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.802 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.802 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.802 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.802 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.802 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.802 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.802 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.802 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.802 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.802 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.802 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.802 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.802 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.803 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.803 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.803 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.803 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.803 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.803 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.803 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.803 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.803 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.803 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.803 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.803 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.803 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.803 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.803 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.804 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.804 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.804 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.804 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.804 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.804 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.805 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.805 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.805 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.805 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.805 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.805 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.805 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.805 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.806 INFO:teuthology.orchestra.run.smithi006.stdout:____________ ERROR at teardown of test_multipart_upload_resend_part ____________ 2024-05-30T17:13:22.806 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.806 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.806 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.806 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.806 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.806 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.806 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.806 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.806 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.806 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.806 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.806 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.806 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.807 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.807 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.807 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.807 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.807 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.807 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.807 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.807 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.807 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.807 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.807 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.807 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.807 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.807 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.808 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.808 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.808 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.808 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.808 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.808 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.808 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.808 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.808 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.808 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.808 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.808 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.808 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.808 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.808 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.808 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.809 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.809 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.809 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.809 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.809 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.809 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.809 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.809 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.809 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.809 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.809 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.809 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.809 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.809 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.809 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.809 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.810 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.810 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.810 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.810 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.810 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.810 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.810 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.810 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.810 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.810 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.810 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.810 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.810 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.810 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.810 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.811 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.811 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.811 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.811 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.811 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.811 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.811 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.811 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.811 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.811 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.811 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.811 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.811 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.811 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.811 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.812 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.812 INFO:teuthology.orchestra.run.smithi006.stdout:__________ ERROR at teardown of test_multipart_upload_multiple_sizes ___________ 2024-05-30T17:13:22.812 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.812 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.812 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.812 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.812 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.812 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.812 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.812 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.812 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.812 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.812 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.812 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.812 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.813 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.813 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.813 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.813 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.813 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.813 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.813 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.813 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.813 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.813 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.813 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.813 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.813 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.813 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.814 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.814 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.814 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.814 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.814 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.814 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.814 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.814 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.814 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.814 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.814 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.814 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.814 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.814 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.815 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.815 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.815 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.815 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.815 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.815 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.815 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.815 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.815 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.815 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.815 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.815 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.815 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.815 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.815 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.816 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.816 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.816 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.816 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.816 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.816 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.816 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.816 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.816 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.816 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.816 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.816 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.816 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.816 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.816 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.817 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.817 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.817 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.817 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.817 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.817 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.817 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.817 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.817 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.817 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.817 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.817 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.817 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.817 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.817 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.817 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.818 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.818 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.818 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.818 INFO:teuthology.orchestra.run.smithi006.stdout:___________ ERROR at teardown of test_multipart_copy_multiple_sizes ____________ 2024-05-30T17:13:22.818 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.818 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.818 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.818 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.818 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.818 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.818 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.818 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.818 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.818 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.819 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.819 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.819 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.819 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.819 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.819 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.819 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.819 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.819 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.819 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.819 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.819 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.819 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.819 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.819 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.819 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.820 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.820 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.820 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.820 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.820 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.820 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.820 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.820 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.820 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.820 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.820 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.820 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.820 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.820 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.821 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.821 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.821 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.821 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.821 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.821 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.821 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.821 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.821 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.821 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.821 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.821 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.821 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.821 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.822 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.822 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.822 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.822 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.822 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.822 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.822 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.822 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.822 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.822 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.822 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.822 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.822 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.822 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.823 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.823 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.823 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.823 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.823 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.823 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.823 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.823 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.823 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.823 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.823 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.823 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.823 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.823 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.823 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.823 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.824 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.824 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.824 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.824 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.824 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.824 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.824 INFO:teuthology.orchestra.run.smithi006.stdout:__________ ERROR at teardown of test_multipart_upload_size_too_small ___________ 2024-05-30T17:13:22.824 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.824 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.824 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.824 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.824 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.824 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.824 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.824 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.825 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.825 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.825 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.825 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.825 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.825 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.825 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.825 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.825 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.825 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.825 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.825 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.825 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.825 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.825 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.825 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.826 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.826 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.826 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.826 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.826 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.826 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.826 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.826 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.826 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.826 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.826 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.826 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.826 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.827 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.827 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.827 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.827 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.827 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.827 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.827 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.827 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.827 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.827 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.827 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.827 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.827 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.827 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.827 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.827 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.828 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.828 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.828 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.828 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.828 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.828 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.828 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.828 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.828 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.828 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.828 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.828 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.828 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.828 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.828 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.829 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.829 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.829 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.829 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.829 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.829 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.829 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.829 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.829 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.829 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.829 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.829 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.829 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.829 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.829 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.830 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.830 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.830 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.830 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.830 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.830 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.830 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.830 INFO:teuthology.orchestra.run.smithi006.stdout:_____________ ERROR at teardown of test_multipart_upload_contents ______________ 2024-05-30T17:13:22.830 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.830 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.830 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.830 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.830 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.830 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.831 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.831 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.831 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.831 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.831 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.831 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.831 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.831 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.831 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.831 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.831 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.831 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.831 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.831 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.831 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.831 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.832 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.832 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.832 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.832 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.832 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.832 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.832 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.832 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.832 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.832 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.832 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.832 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.832 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.832 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.832 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.833 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.833 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.833 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.833 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.833 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.833 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.833 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.833 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.833 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.833 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.833 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.833 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.833 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.833 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.833 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.834 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.834 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.834 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.834 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.834 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.834 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.834 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.834 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.834 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.834 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.834 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.834 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.834 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.834 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.834 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.835 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.835 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.835 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.835 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.835 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.835 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.835 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.835 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.835 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.835 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.835 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.835 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.835 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.835 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.835 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.836 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.836 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.836 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.836 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.836 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.836 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.836 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.836 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.836 INFO:teuthology.orchestra.run.smithi006.stdout:_____ ERROR at teardown of test_multipart_upload_overwrite_existing_object _____ 2024-05-30T17:13:22.836 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.836 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.836 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.836 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.836 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.837 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.837 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.837 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.837 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.837 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.837 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.837 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.837 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.837 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.837 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.837 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.837 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.837 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.837 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.837 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.838 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.838 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.838 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.838 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.838 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.838 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.838 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.838 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.838 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.838 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.838 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.838 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.838 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.838 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.839 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.839 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.839 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.839 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.839 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.839 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.839 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.839 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.839 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.839 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.839 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.839 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.839 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.839 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.839 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.840 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.840 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.840 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.840 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.840 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.840 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.840 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.840 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.840 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.840 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.840 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.840 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.840 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.840 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.841 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.841 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.841 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.841 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.841 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.841 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.841 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.841 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.841 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.841 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.841 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.841 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.841 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.841 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.842 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.842 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.842 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.842 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.842 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.842 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.842 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.842 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.842 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.842 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.842 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.842 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.842 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.843 INFO:teuthology.orchestra.run.smithi006.stdout:_______________ ERROR at teardown of test_abort_multipart_upload _______________ 2024-05-30T17:13:22.843 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.843 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.843 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.843 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.843 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.843 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.843 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.843 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.843 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.843 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.843 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.843 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.843 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.844 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.844 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.844 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.844 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.844 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.844 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.844 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.844 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.844 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.844 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.844 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.844 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.844 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.844 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.844 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.845 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.845 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.845 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.845 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.845 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.845 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.845 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.845 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.845 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.845 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.845 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.845 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.845 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.845 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.845 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.846 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.846 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.846 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.846 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.846 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.846 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.846 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.846 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.846 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.846 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.846 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.846 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.846 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.846 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.846 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.847 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.847 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.847 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.847 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.847 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.847 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.847 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.847 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.847 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.847 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.847 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.847 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.847 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.847 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.847 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.848 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.848 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.848 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.848 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.848 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.848 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.848 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.848 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.848 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.848 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.848 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.848 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.848 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.848 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.849 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.849 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.849 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.849 INFO:teuthology.orchestra.run.smithi006.stdout:__________ ERROR at teardown of test_abort_multipart_upload_not_found __________ 2024-05-30T17:13:22.849 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.849 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.849 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.849 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.849 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.849 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.849 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.849 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.849 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.849 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.849 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.850 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.850 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.850 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.850 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.850 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.850 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.850 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.850 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.850 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.850 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.850 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.850 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.850 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.850 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.850 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.851 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.851 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.851 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.851 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.851 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.851 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.851 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.851 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.851 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.851 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.851 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.851 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.851 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.851 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.851 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.851 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.852 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.852 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.852 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.852 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.852 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.852 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.852 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.852 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.852 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.852 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.852 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.852 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.852 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.852 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.852 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.853 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.853 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.853 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.853 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.853 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.853 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.853 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.853 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.853 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.853 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.853 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.853 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.853 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.853 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.854 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.854 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.854 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.854 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.854 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.854 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.854 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.854 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.854 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.854 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.854 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.854 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.854 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.854 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.855 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.855 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.855 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.855 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.855 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.855 INFO:teuthology.orchestra.run.smithi006.stdout:_______________ ERROR at teardown of test_list_multipart_upload ________________ 2024-05-30T17:13:22.855 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.855 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.855 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.855 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.855 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.855 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.855 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.855 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.855 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.856 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.856 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.856 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.856 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.856 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.856 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.856 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.856 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.856 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.856 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.856 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.856 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.856 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.856 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.856 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.857 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.857 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.857 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.857 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.857 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.857 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.857 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.857 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.857 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.857 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.857 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.857 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.857 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.857 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.857 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.857 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.858 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.858 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.858 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.858 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.858 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.858 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.858 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.858 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.858 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.858 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.858 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.858 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.858 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.858 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.859 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.859 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.859 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.859 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.859 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.859 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.859 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.859 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.859 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.859 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.859 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.859 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.859 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.859 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.859 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.860 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.860 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.860 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.860 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.860 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.860 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.860 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.860 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.860 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.860 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.860 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.860 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.860 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.860 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.860 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.861 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.861 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.861 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.861 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.861 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.861 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.861 INFO:teuthology.orchestra.run.smithi006.stdout:____________ ERROR at teardown of test_list_multipart_upload_owner _____________ 2024-05-30T17:13:22.861 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.861 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.861 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.861 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.861 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.861 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.861 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.861 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.861 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.862 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.862 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.862 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.862 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.862 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.862 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.862 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.862 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.862 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.862 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.862 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.862 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.862 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.862 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.863 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.863 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.863 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.863 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.863 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.863 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.863 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.863 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.863 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.863 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.863 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.863 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.863 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.863 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.863 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.863 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.864 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.864 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.864 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.864 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.864 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.864 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.864 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.864 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.864 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.864 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.864 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.864 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.864 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.864 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.864 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.865 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.865 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.865 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.865 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.865 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.865 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.865 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.865 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.865 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.865 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.865 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.865 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.865 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.865 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.865 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.865 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.866 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.866 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.866 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.866 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.866 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.866 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.866 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.866 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.866 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.866 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.866 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.866 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.867 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.867 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.867 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.867 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.867 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.867 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.867 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.867 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.867 INFO:teuthology.orchestra.run.smithi006.stdout:___________ ERROR at teardown of test_multipart_upload_missing_part ____________ 2024-05-30T17:13:22.867 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.867 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.867 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.867 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.867 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.867 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.867 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.868 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.868 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.868 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.868 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.868 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.868 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.868 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.868 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.868 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.868 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.868 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.868 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.868 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.868 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.868 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.869 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.869 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.869 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.869 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.869 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.869 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.869 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.869 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.869 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.869 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.869 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.869 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.869 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.869 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.869 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.869 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.870 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.870 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.870 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.870 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.870 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.870 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.870 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.870 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.870 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.870 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.870 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.870 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.870 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.871 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.871 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.871 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.871 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.871 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.871 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.871 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.871 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.871 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.871 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.871 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.871 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.871 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.871 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.871 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.872 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.872 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.872 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.872 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.872 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.872 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.872 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.872 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.872 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.872 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.872 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.872 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.872 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.872 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.872 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.872 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.873 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.873 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.873 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.873 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.873 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.873 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.873 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.873 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.873 INFO:teuthology.orchestra.run.smithi006.stdout:__________ ERROR at teardown of test_multipart_upload_incorrect_etag ___________ 2024-05-30T17:13:22.873 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.873 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.873 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.873 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.873 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.873 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.874 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.874 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.874 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.874 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.874 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.874 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.874 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.874 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.874 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.874 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.874 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.874 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.874 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.874 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.874 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.875 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.875 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.875 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.875 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.875 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.875 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.875 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.875 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.875 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.875 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.875 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.875 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.875 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.875 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.875 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.876 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.876 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.876 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.876 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.876 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.876 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.876 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.876 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.876 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.876 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.876 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.876 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.876 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.876 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.876 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.876 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.877 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.877 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.877 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.877 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.877 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.877 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.877 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.877 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.877 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.877 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.877 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.877 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.877 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.877 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.878 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.878 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.878 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.878 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.878 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.878 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.878 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.878 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.878 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.878 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.878 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.878 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.878 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.878 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.878 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.879 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.879 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.879 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.879 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.879 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.879 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.879 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.879 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.879 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.879 INFO:teuthology.orchestra.run.smithi006.stdout:_________________ ERROR at teardown of test_multipart_get_part _________________ 2024-05-30T17:13:22.879 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.879 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.879 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.879 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.879 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.880 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.880 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.880 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.880 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.880 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.880 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.880 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.880 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.880 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.880 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.880 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.880 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.880 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.880 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.880 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.881 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.881 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.881 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.881 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.881 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.881 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.881 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.881 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.881 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.881 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.881 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.881 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.881 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.881 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.881 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.882 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.882 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.882 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.882 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.882 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.882 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.882 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.882 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.882 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.882 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.882 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.882 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.882 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.882 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.882 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.883 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.883 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.883 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.883 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.883 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.883 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.883 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.883 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.883 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.883 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.883 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.883 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.883 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.883 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.883 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.884 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.884 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.884 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.884 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.884 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.884 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.884 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.884 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.884 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.884 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.884 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.884 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.884 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.884 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.884 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.884 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.885 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.885 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.885 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.885 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.885 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.885 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.885 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.885 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.885 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.885 INFO:teuthology.orchestra.run.smithi006.stdout:_______________ ERROR at teardown of test_non_multipart_get_part _______________ 2024-05-30T17:13:22.885 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.885 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.885 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.885 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.886 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.886 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.886 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.886 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.886 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.886 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.886 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.886 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.886 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.886 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.886 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.886 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.886 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.886 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.886 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.887 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.887 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.887 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.887 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.887 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.887 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.887 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.887 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.887 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.887 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.887 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.887 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.887 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.887 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.887 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.887 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.888 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.888 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.888 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.888 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.888 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.888 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.888 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.888 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.888 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.888 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.888 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.888 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.888 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.888 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.889 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.889 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.889 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.889 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.889 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.889 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.889 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.889 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.889 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.889 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.889 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.889 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.889 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.889 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.889 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.889 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.890 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.890 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.890 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.890 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.890 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.890 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.890 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.890 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.890 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.890 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.890 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.890 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.890 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.890 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.890 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.891 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.891 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.891 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.891 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.891 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.891 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.891 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.891 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.891 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.891 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.891 INFO:teuthology.orchestra.run.smithi006.stdout:____________________ ERROR at teardown of test_100_continue ____________________ 2024-05-30T17:13:22.891 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.891 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.891 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.891 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.892 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.892 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.892 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.892 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.892 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.892 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.892 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.892 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.892 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.892 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.892 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.892 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.892 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.892 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.892 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.893 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.893 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.893 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.893 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.893 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.893 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.893 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.893 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.893 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.893 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.893 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.893 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.893 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.893 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.893 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.894 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.894 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.894 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.894 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.894 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.894 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.894 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.894 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.894 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.894 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.894 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.894 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.894 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.894 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.894 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.895 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.895 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.895 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.895 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.895 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.895 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.895 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.895 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.895 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.895 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.895 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.895 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.895 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.895 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.895 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.896 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.896 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.896 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.896 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.896 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.896 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.896 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.896 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.896 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.896 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.896 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.896 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.896 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.896 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.897 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.897 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.897 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.897 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.897 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.897 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.897 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.897 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.897 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.897 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.897 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.897 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.897 INFO:teuthology.orchestra.run.smithi006.stdout:______________________ ERROR at teardown of test_set_cors ______________________ 2024-05-30T17:13:22.897 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.897 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.898 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.898 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.898 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.898 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.898 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.898 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.898 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.898 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.898 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.898 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.898 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.898 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.898 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.898 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.898 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.899 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.899 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.899 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.899 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.899 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.899 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.899 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.899 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.899 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.899 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.899 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.899 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.899 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.899 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.899 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.900 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.900 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.900 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.900 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.900 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.900 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.900 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.900 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.900 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.900 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.900 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.900 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.900 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.900 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.901 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.901 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.901 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.901 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.901 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.901 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.901 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.901 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.901 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.901 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.901 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.901 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.901 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.901 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.901 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.902 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.902 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.902 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.902 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.902 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.902 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.902 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.902 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.902 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.902 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.902 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.902 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.902 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.902 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.903 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.903 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.903 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.903 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.903 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.903 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.903 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.903 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.903 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.903 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.903 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.903 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.903 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.903 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.903 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.903 INFO:teuthology.orchestra.run.smithi006.stdout:________________ ERROR at teardown of test_cors_origin_response ________________ 2024-05-30T17:13:22.904 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.904 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.904 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.904 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.904 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.904 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.904 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.904 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.904 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.904 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.904 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.904 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.904 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.904 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.904 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.905 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.905 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.905 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.905 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.905 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.905 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.905 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.905 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.905 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.905 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.905 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.905 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.905 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.905 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.905 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.906 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.906 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.906 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.906 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.906 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.906 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.906 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.906 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.906 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.906 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.906 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.906 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.906 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.906 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.906 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.907 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.907 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.907 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.907 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.907 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.907 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.907 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.907 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.907 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.907 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.907 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.907 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.907 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.907 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.907 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.908 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.908 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.908 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.908 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.908 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.908 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.908 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.908 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.908 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.908 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.908 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.908 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.908 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.908 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.908 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.908 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.909 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.909 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.909 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.909 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.909 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.909 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.909 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.909 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.909 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.909 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.909 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.909 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.909 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.909 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.910 INFO:teuthology.orchestra.run.smithi006.stdout:________________ ERROR at teardown of test_cors_origin_wildcard ________________ 2024-05-30T17:13:22.910 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.910 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.910 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.910 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.910 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.910 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.910 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.910 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.910 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.910 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.910 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.910 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.910 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.910 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.911 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.911 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.911 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.911 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.911 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.911 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.911 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.911 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.911 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.911 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.911 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.911 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.911 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.911 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.911 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.912 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.912 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.912 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.912 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.912 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.912 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.912 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.912 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.912 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.912 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.912 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.912 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.912 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.912 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.912 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.913 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.913 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.913 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.913 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.913 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.913 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.913 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.913 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.913 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.913 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.913 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.913 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.913 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.913 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.913 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.914 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.914 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.914 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.914 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.914 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.914 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.914 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.914 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.914 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.914 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.914 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.914 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.914 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.914 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.915 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.915 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.915 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.915 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.915 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.915 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.915 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.915 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.915 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.915 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.915 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.915 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.915 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.915 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.916 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.916 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.916 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.916 INFO:teuthology.orchestra.run.smithi006.stdout:_________________ ERROR at teardown of test_cors_header_option _________________ 2024-05-30T17:13:22.916 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.916 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.916 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.916 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.916 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.916 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.916 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.916 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.916 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.916 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.916 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.917 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.917 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.917 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.917 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.917 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.917 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.917 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.917 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.917 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.917 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.917 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.917 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.917 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.917 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.917 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.918 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.918 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.918 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.918 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.918 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.918 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.918 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.918 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.918 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.918 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.918 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.918 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.918 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.919 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.919 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.919 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.919 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.919 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.919 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.919 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.919 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.919 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.919 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.919 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.919 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.919 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.919 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.919 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.919 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.920 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.920 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.920 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.920 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.920 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.920 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.920 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.920 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.920 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.920 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.920 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.920 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.920 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.920 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.921 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.921 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.921 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.921 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.921 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.921 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.921 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.921 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.921 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.921 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.921 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.921 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.921 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.922 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.922 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.922 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.922 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.922 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.922 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.922 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.922 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.922 INFO:teuthology.orchestra.run.smithi006.stdout:_____________ ERROR at teardown of test_cors_presigned_get_object ______________ 2024-05-30T17:13:22.922 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.922 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.922 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.922 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.922 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.922 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.923 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.923 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.923 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.923 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.923 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.923 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.923 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.923 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.923 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.923 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.923 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.923 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.923 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.923 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.923 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.923 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.924 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.924 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.924 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.924 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.924 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.924 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.924 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.924 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.924 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.924 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.924 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.924 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.924 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.924 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.924 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.925 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.925 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.925 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.925 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.925 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.925 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.925 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.925 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.925 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.925 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.925 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.925 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.925 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.925 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.925 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.926 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.926 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.926 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.926 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.926 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.926 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.926 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.926 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.926 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.926 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.926 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.926 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.926 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.926 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.926 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.927 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.927 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.927 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.927 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.927 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.927 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.927 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.927 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.927 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.927 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.927 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.927 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.927 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.927 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.927 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.928 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.928 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.928 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.928 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.928 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.928 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.928 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.928 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.928 INFO:teuthology.orchestra.run.smithi006.stdout:__________ ERROR at teardown of test_cors_presigned_get_object_tenant __________ 2024-05-30T17:13:22.928 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.928 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.928 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.928 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.928 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.928 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.928 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.929 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.929 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.929 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.929 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.929 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.929 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.929 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.929 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.929 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.929 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.929 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.929 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.929 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.929 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.929 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.930 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.930 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.930 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.930 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.930 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.930 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.930 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.930 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.930 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.930 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.930 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.930 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.930 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.930 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.930 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.931 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.931 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.931 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.931 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.931 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.931 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.931 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.931 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.931 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.931 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.931 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.931 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.931 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.931 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.932 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.932 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.932 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.932 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.932 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.932 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.932 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.932 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.932 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.932 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.932 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.932 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.932 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.932 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.932 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.932 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.933 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.933 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.933 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.933 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.933 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.933 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.933 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.933 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.933 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.933 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.933 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.933 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.933 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.933 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.933 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.934 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.934 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.934 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.934 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.934 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.934 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.934 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.934 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.934 INFO:teuthology.orchestra.run.smithi006.stdout:_____________ ERROR at teardown of test_cors_presigned_put_object ______________ 2024-05-30T17:13:22.934 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.934 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.934 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.934 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.934 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.934 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.935 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.935 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.935 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.935 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.935 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.935 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.935 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.935 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.935 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.935 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.935 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.935 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.935 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.935 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.935 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.935 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.936 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.936 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.936 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.936 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.936 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.936 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.936 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.936 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.936 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.936 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.936 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.936 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.936 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.936 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.936 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.937 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.937 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.937 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.937 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.937 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.937 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.937 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.937 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.937 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.937 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.937 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.937 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.937 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.937 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.937 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.937 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.938 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.938 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.938 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.938 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.938 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.938 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.938 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.938 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.938 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.938 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.938 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.938 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.938 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.938 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.939 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.939 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.939 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.939 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.939 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.939 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.939 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.939 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.939 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.939 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.939 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.939 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.939 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.939 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.939 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.939 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.940 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.940 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.940 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.940 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.940 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.940 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.940 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.940 INFO:teuthology.orchestra.run.smithi006.stdout:_________ ERROR at teardown of test_cors_presigned_put_object_with_acl _________ 2024-05-30T17:13:22.940 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.940 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.940 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.940 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.940 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.940 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.940 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.940 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.941 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.941 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.941 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.941 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.941 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.941 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.941 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.941 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.941 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.941 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.941 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.941 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.941 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.941 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.941 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.942 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.942 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.942 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.942 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.942 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.942 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.942 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.942 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.942 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.942 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.942 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.942 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.942 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.942 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.942 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.942 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.943 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.943 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.943 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.943 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.943 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.943 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.943 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.943 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.943 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.943 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.943 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.943 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.943 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.943 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.944 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.944 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.944 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.944 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.944 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.944 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.944 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.944 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.944 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.944 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.944 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.944 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.944 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.944 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.945 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.945 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.945 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.945 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.945 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.945 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.945 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.945 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.945 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.945 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.945 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.945 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.945 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.945 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.945 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.946 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.946 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.946 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.946 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.946 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.946 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.946 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.946 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.946 INFO:teuthology.orchestra.run.smithi006.stdout:__________ ERROR at teardown of test_cors_presigned_put_object_tenant __________ 2024-05-30T17:13:22.946 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.946 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.946 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.946 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.946 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.947 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.947 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.947 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.947 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.947 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.947 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.947 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.947 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.947 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.947 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.947 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.947 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.947 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.947 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.948 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.948 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.948 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.948 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.948 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.948 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.948 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.948 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.948 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.948 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.948 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.948 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.948 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.948 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.949 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.949 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.949 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.949 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.949 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.949 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.949 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.949 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.949 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.949 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.949 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.949 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.949 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.949 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.949 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.950 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.950 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.950 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.950 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.950 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.950 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.950 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.950 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.950 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.950 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.950 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.950 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.950 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.950 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.951 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.951 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.951 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.951 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.951 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.951 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.951 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.951 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.951 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.951 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.951 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.951 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.951 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.951 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.951 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.952 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.952 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.952 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.952 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.952 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.952 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.952 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.952 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.952 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.952 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.952 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.952 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.952 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.952 INFO:teuthology.orchestra.run.smithi006.stdout:_____ ERROR at teardown of test_cors_presigned_put_object_tenant_with_acl ______ 2024-05-30T17:13:22.952 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.952 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.953 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.953 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.953 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.953 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.953 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.953 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.953 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.953 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.953 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.953 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.953 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.953 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.953 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.953 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.953 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.954 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.954 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.954 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.954 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.954 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.954 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.954 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.954 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.954 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.954 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.954 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.954 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.954 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.955 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.955 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.955 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.955 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.955 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.955 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.955 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.955 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.955 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.955 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.955 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.955 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.955 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.955 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.955 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.956 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.956 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.956 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.956 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.956 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.956 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.956 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.956 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.956 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.956 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.956 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.956 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.956 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.956 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.956 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.957 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.957 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.957 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.957 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.957 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.957 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.957 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.957 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.957 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.957 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.957 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.957 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.957 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.957 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.958 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.958 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.958 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.958 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.958 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.958 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.958 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.958 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.958 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.958 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.958 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.958 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.958 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.959 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.959 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.959 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.959 INFO:teuthology.orchestra.run.smithi006.stdout:_________________ ERROR at teardown of test_set_bucket_tagging _________________ 2024-05-30T17:13:22.959 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.959 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.959 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.959 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.959 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.959 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.959 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.959 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.959 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.959 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.959 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.959 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.960 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.960 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.960 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.960 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.960 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.960 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.960 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.960 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.960 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.960 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.960 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.960 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.960 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.960 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.961 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.961 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.961 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.961 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.961 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.961 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.961 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.961 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.961 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.961 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.961 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.961 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.961 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.961 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.962 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.962 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.962 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.962 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.962 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.962 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.962 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.962 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.962 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.962 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.962 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.962 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.962 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.962 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.962 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.963 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.963 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.963 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.963 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.963 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.963 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.963 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.963 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.963 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.963 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.963 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.963 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.963 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.963 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.963 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.964 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.964 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.964 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.964 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.964 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.964 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.964 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.964 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.964 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.964 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.964 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.964 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.964 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.964 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.964 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.965 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.965 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.965 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.965 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.965 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.965 INFO:teuthology.orchestra.run.smithi006.stdout:__________________ ERROR at teardown of test_atomic_read_1mb ___________________ 2024-05-30T17:13:22.965 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.965 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.965 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.965 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.965 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.965 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.965 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.965 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.965 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.965 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.966 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.966 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.966 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.966 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.966 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.966 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.966 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.966 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.966 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.966 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.966 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.966 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.966 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.967 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.967 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.967 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.967 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.967 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.967 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.967 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.967 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.967 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.967 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.967 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.967 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.967 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.967 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.967 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.967 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.968 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.968 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.968 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.968 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.968 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.968 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.968 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.968 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.968 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.968 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.968 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.968 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.968 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.968 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.969 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.969 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.969 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.969 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.969 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.969 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.969 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.969 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.969 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.969 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.969 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.969 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.969 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.969 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.969 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.970 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.970 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.970 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.970 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.970 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.970 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.970 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.970 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.970 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.970 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.970 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.970 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.970 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.970 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.970 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.971 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.971 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.971 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.971 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.971 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.971 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.971 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.971 INFO:teuthology.orchestra.run.smithi006.stdout:__________________ ERROR at teardown of test_atomic_read_4mb ___________________ 2024-05-30T17:13:22.971 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.971 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.971 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.971 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.971 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.972 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.972 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.972 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.972 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.972 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.972 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.972 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.972 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.972 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.972 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.972 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.972 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.972 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.972 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.972 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.973 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.973 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.973 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.973 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.973 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.973 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.973 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.973 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.973 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.973 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.973 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.973 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.973 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.973 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.973 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.974 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.974 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.974 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.974 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.974 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.974 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.974 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.974 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.974 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.974 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.974 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.974 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.974 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.974 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.974 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.975 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.975 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.975 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.975 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.975 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.975 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.975 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.975 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.975 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.975 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.975 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.975 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.975 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.975 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.975 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.976 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.976 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.976 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.976 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.976 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.976 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.976 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.976 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.976 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.976 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.976 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.976 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.976 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.976 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.976 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.977 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.977 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.977 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.977 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.977 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.977 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.977 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.977 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.977 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.977 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.977 INFO:teuthology.orchestra.run.smithi006.stdout:__________________ ERROR at teardown of test_atomic_read_8mb ___________________ 2024-05-30T17:13:22.977 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.977 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.977 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.977 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.978 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.978 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.978 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.978 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.978 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.978 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.978 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.978 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.978 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.978 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.978 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.978 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.978 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.979 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.979 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.979 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.979 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.979 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.979 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.979 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.979 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.979 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.979 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.979 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.979 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.979 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.979 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.979 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.980 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.980 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.980 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.980 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.980 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.980 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.980 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.980 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.980 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.980 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.980 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.980 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.980 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.980 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.980 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.980 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.981 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.981 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.981 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.981 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.981 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.981 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.981 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.981 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.981 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.981 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.981 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.981 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.981 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.981 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.981 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.982 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.982 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.982 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.982 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.982 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.982 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.982 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.982 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.982 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.982 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.982 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.982 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.982 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.982 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.982 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.983 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.983 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.983 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.983 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.983 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.983 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.983 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.983 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.983 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.983 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.983 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.983 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.983 INFO:teuthology.orchestra.run.smithi006.stdout:__________________ ERROR at teardown of test_atomic_write_1mb __________________ 2024-05-30T17:13:22.983 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.983 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.984 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.984 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.984 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.984 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.984 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.984 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.984 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.984 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.984 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.984 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.984 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.984 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.984 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.984 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.984 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.985 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.985 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.985 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.985 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.985 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.985 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.985 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.985 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.985 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.985 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.985 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.985 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.985 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.985 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.985 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.986 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.986 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.986 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.986 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.986 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.986 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.986 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.986 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.986 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.986 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.986 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.986 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.986 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.986 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.986 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.987 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.987 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.987 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.987 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.987 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.987 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.987 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.987 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.987 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.987 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.987 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.987 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.987 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.987 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.987 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.988 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.988 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.988 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.988 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.988 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.988 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.988 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.988 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.988 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.988 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.988 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.988 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.988 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.988 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.989 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.989 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.989 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.989 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.989 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.989 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.989 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.989 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.989 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.989 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.989 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.989 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.989 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.989 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.989 INFO:teuthology.orchestra.run.smithi006.stdout:__________________ ERROR at teardown of test_atomic_write_4mb __________________ 2024-05-30T17:13:22.990 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.990 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.990 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.990 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.990 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.990 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.990 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.990 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.990 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.990 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.990 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.990 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.990 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.990 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.990 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.991 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.991 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.991 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.991 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.991 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.991 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.991 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.991 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.991 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.991 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.991 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.991 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.991 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.991 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.991 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.991 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.992 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.992 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.992 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.992 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.992 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.992 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.992 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.992 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.992 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.992 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.992 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.992 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.992 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.992 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.992 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.993 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.993 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.993 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.993 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.993 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.993 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.993 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.993 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.993 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.993 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.993 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.993 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.993 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.993 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.993 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:22.993 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:22.994 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.994 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.994 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.994 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:22.994 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:22.994 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:22.994 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:22.994 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:22.994 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:22.994 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.994 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.994 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:22.994 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.994 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.994 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.995 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:22.995 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:22.995 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:22.995 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:22.995 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.995 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.995 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:22.995 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:22.995 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:22.995 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:22.995 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:22.995 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.995 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:22.995 INFO:teuthology.orchestra.run.smithi006.stdout:__________________ ERROR at teardown of test_atomic_write_8mb __________________ 2024-05-30T17:13:22.995 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.996 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:22.996 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.996 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:22.996 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:22.996 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:22.996 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:22.996 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.996 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:22.996 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.996 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:22.996 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:22.996 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:22.996 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:22.996 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:22.996 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:22.996 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:22.997 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:22.997 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:22.997 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:22.997 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:22.997 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:22.997 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:22.997 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.997 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:22.997 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:22.997 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:22.997 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.997 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:22.997 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:22.998 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:22.998 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:22.998 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:22.998 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:22.998 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:22.998 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:22.998 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:22.998 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:22.998 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.998 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:22.998 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:22.998 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:22.998 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.998 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:22.998 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:22.999 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:22.999 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:22.999 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:22.999 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:22.999 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:22.999 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:22.999 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:22.999 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:22.999 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:22.999 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:22.999 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:22.999 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:22.999 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:22.999 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:22.999 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.000 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.000 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.000 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.000 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.000 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.000 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.000 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.000 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.000 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.000 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.000 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.000 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.000 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.000 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.000 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.000 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.001 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.001 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.001 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.001 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.001 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.001 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.001 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.001 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.001 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.001 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.001 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.001 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.001 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.001 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.001 INFO:teuthology.orchestra.run.smithi006.stdout:_______________ ERROR at teardown of test_atomic_dual_write_1mb ________________ 2024-05-30T17:13:23.002 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.002 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.002 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.002 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.002 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.002 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.002 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.002 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.002 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.002 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.002 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.002 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.002 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.002 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.003 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.003 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.003 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.003 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.003 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.003 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.003 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.003 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.003 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.003 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.003 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.003 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.003 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.003 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.003 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.003 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.004 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.004 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.004 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.004 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.004 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.004 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.004 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.004 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.004 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.004 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.004 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.004 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.004 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.004 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.004 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.005 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.005 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.005 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.005 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.005 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.005 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.005 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.005 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.005 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.005 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.005 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.005 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.005 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.005 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.005 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.006 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.006 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.006 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.006 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.006 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.006 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.006 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.006 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.006 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.006 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.006 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.006 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.006 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.006 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.007 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.007 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.007 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.007 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.007 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.007 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.007 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.007 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.007 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.007 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.007 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.007 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.007 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.007 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.007 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.007 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.008 INFO:teuthology.orchestra.run.smithi006.stdout:_______________ ERROR at teardown of test_atomic_dual_write_4mb ________________ 2024-05-30T17:13:23.008 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.008 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.008 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.008 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.008 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.008 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.008 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.008 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.008 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.008 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.008 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.008 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.008 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.009 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.009 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.009 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.009 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.009 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.009 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.009 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.009 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.009 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.009 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.009 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.009 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.009 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.009 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.009 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.010 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.010 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.010 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.010 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.010 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.010 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.010 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.010 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.010 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.010 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.010 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.010 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.010 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.011 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.011 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.011 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.011 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.011 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.011 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.011 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.011 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.011 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.011 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.011 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.011 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.011 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.011 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.011 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.012 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.012 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.012 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.012 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.012 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.012 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.012 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.012 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.012 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.012 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.012 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.012 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.012 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.012 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.012 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.013 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.013 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.013 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.013 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.013 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.013 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.013 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.013 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.013 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.013 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.013 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.013 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.013 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.013 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.013 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.013 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.014 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.014 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.014 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.014 INFO:teuthology.orchestra.run.smithi006.stdout:_______________ ERROR at teardown of test_atomic_dual_write_8mb ________________ 2024-05-30T17:13:23.014 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.014 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.014 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.014 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.014 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.014 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.014 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.014 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.014 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.014 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.015 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.015 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.015 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.015 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.015 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.015 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.015 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.015 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.015 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.015 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.015 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.015 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.015 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.015 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.015 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.015 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.016 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.016 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.016 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.016 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.016 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.016 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.016 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.016 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.016 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.016 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.016 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.016 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.016 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.016 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.016 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.017 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.017 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.017 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.017 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.017 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.017 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.017 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.017 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.017 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.017 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.017 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.017 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.017 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.017 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.017 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.018 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.018 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.018 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.018 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.018 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.018 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.018 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.018 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.018 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.018 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.018 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.018 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.018 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.018 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.018 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.019 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.019 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.019 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.019 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.019 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.019 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.019 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.019 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.019 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.019 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.019 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.019 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.019 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.019 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.019 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.020 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.020 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.020 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.020 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.020 INFO:teuthology.orchestra.run.smithi006.stdout:____________ ERROR at teardown of test_atomic_conditional_write_1mb ____________ 2024-05-30T17:13:23.020 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.020 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.020 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.020 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.020 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.020 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.020 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.020 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.020 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.021 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.021 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.021 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.021 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.021 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.021 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.021 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.021 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.021 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.021 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.021 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.021 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.021 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.021 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.022 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.022 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.022 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.022 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.022 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.022 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.022 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.022 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.022 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.022 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.022 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.022 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.022 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.022 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.022 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.022 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.023 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.023 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.023 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.023 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.023 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.023 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.023 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.023 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.023 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.023 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.023 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.023 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.023 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.023 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.024 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.024 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.024 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.024 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.024 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.024 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.024 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.024 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.024 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.024 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.024 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.024 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.024 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.024 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.024 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.025 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.025 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.025 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.025 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.025 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.025 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.025 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.025 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.025 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.025 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.025 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.025 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.025 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.025 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.025 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.025 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.026 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.026 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.026 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.026 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.026 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.026 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.026 INFO:teuthology.orchestra.run.smithi006.stdout:___________ ERROR at teardown of test_atomic_multipart_upload_write ____________ 2024-05-30T17:13:23.026 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.026 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.026 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.026 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.026 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.026 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.026 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.027 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.027 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.027 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.027 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.027 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.027 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.027 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.027 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.027 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.027 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.027 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.027 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.027 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.027 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.027 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.028 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.028 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.028 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.028 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.028 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.028 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.028 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.028 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.028 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.028 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.028 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.028 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.028 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.028 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.028 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.029 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.029 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.029 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.029 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.029 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.029 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.029 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.029 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.029 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.029 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.029 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.029 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.029 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.029 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.029 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.030 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.030 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.030 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.030 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.030 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.030 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.030 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.030 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.030 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.030 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.030 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.030 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.030 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.030 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.030 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.031 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.031 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.031 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.031 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.031 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.031 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.031 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.031 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.031 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.031 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.031 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.031 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.031 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.031 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.031 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.032 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.032 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.032 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.032 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.032 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.032 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.032 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.032 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.032 INFO:teuthology.orchestra.run.smithi006.stdout:________ ERROR at teardown of test_multipart_resend_first_finishes_last ________ 2024-05-30T17:13:23.032 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.032 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.032 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.032 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.032 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.032 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.032 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.033 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.033 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.033 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.033 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.033 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.033 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.033 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.033 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.033 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.033 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.033 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.033 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.033 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.033 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.033 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.034 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.034 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.034 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.034 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.034 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.034 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.034 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.034 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.034 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.034 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.034 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.034 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.034 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.035 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.035 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.035 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.035 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.035 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.035 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.035 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.035 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.035 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.035 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.035 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.035 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.035 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.035 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.035 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.036 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.036 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.036 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.036 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.036 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.036 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.036 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.036 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.036 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.036 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.036 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.036 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.036 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.036 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.037 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.037 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.037 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.037 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.037 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.037 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.037 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.037 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.037 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.037 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.037 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.037 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.037 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.037 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.037 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.037 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.038 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.038 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.038 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.038 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.038 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.038 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.038 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.038 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.038 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.038 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.038 INFO:teuthology.orchestra.run.smithi006.stdout:____________ ERROR at teardown of test_ranged_request_response_code ____________ 2024-05-30T17:13:23.038 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.038 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.038 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.039 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.039 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.039 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.039 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.039 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.039 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.039 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.039 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.039 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.039 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.039 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.039 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.039 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.039 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.039 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.040 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.040 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.040 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.040 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.040 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.040 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.040 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.040 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.040 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.040 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.040 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.040 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.040 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.040 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.040 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.041 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.041 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.041 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.041 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.041 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.041 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.041 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.041 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.041 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.041 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.041 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.041 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.041 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.041 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.041 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.042 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.042 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.042 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.042 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.042 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.042 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.042 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.042 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.042 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.042 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.042 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.042 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.042 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.042 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.043 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.043 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.043 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.043 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.043 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.043 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.043 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.043 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.043 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.043 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.043 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.043 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.043 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.043 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.043 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.044 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.044 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.044 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.044 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.044 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.044 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.044 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.044 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.044 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.044 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.044 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.044 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.044 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.044 INFO:teuthology.orchestra.run.smithi006.stdout:__________ ERROR at teardown of test_ranged_big_request_response_code __________ 2024-05-30T17:13:23.044 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.044 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.045 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.045 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.045 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.045 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.045 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.045 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.045 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.045 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.045 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.045 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.045 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.045 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.045 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.045 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.045 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.045 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.046 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.046 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.046 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.046 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.046 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.046 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.046 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.046 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.046 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.046 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.046 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.046 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.046 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.046 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.047 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.047 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.047 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.047 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.047 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.047 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.047 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.047 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.047 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.047 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.047 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.047 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.047 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.047 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.047 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.047 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.048 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.048 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.048 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.048 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.048 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.048 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.048 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.048 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.048 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.048 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.048 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.048 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.048 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.048 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.048 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.049 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.049 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.049 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.049 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.049 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.049 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.049 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.049 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.049 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.049 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.049 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.049 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.049 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.049 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.050 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.050 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.050 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.050 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.050 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.050 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.050 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.050 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.050 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.050 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.050 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.050 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.050 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.050 INFO:teuthology.orchestra.run.smithi006.stdout:__ ERROR at teardown of test_ranged_request_skip_leading_bytes_response_code ___ 2024-05-30T17:13:23.051 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.051 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.051 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.051 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.051 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.051 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.051 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.051 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.051 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.051 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.051 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.051 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.051 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.051 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.052 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.052 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.052 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.052 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.052 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.052 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.052 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.052 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.052 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.052 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.052 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.052 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.052 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.052 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.052 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.053 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.053 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.053 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.053 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.053 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.053 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.053 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.053 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.053 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.053 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.053 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.053 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.053 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.053 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.053 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.054 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.054 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.054 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.054 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.054 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.054 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.054 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.054 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.054 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.054 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.054 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.054 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.054 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.055 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.055 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.055 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.055 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.055 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.055 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.055 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.055 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.055 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.055 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.055 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.055 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.055 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.055 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.055 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.056 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.056 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.056 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.056 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.056 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.056 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.056 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.056 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.056 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.056 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.056 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.056 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.056 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.056 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.056 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.057 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.057 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.057 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.057 INFO:teuthology.orchestra.run.smithi006.stdout:_ ERROR at teardown of test_ranged_request_return_trailing_bytes_response_code _ 2024-05-30T17:13:23.057 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.057 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.057 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.057 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.057 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.057 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.057 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.057 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.057 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.057 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.057 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.058 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.058 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.058 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.058 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.058 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.058 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.058 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.058 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.058 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.058 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.058 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.058 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.058 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.058 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.059 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.059 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.059 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.059 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.059 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.059 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.059 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.059 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.059 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.059 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.059 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.059 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.059 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.059 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.060 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.060 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.060 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.060 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.060 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.060 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.060 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.060 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.060 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.060 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.060 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.060 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.060 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.060 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.060 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.061 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.061 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.061 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.061 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.061 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.061 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.061 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.061 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.061 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.061 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.061 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.061 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.061 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.061 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.062 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.062 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.062 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.062 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.062 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.062 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.062 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.062 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.062 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.062 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.062 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.062 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.062 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.062 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.063 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.063 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.063 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.063 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.063 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.063 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.063 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.063 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.063 INFO:teuthology.orchestra.run.smithi006.stdout:____________ ERROR at teardown of test_ranged_request_invalid_range ____________ 2024-05-30T17:13:23.063 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.063 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.063 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.063 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.063 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.063 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.064 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.064 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.064 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.064 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.064 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.064 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.064 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.064 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.064 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.064 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.064 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.064 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.064 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.064 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.064 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.065 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.065 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.065 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.065 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.065 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.065 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.065 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.065 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.065 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.065 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.065 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.065 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.065 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.065 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.065 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.066 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.066 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.066 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.066 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.066 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.066 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.066 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.066 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.066 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.066 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.066 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.066 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.066 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.066 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.066 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.067 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.067 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.067 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.067 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.067 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.067 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.067 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.067 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.067 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.067 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.067 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.067 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.067 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.067 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.067 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.068 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.068 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.068 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.068 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.068 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.068 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.068 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.068 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.068 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.068 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.068 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.068 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.068 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.068 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.068 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.069 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.069 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.069 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.069 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.069 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.069 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.069 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.069 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.069 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.069 INFO:teuthology.orchestra.run.smithi006.stdout:____________ ERROR at teardown of test_ranged_request_empty_object _____________ 2024-05-30T17:13:23.069 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.069 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.069 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.069 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.069 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.069 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.070 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.070 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.070 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.070 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.070 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.070 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.070 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.070 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.070 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.070 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.070 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.070 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.070 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.071 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.071 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.071 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.071 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.071 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.071 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.071 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.071 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.071 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.071 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.071 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.071 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.071 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.071 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.071 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.072 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.072 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.072 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.072 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.072 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.072 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.072 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.072 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.072 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.072 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.072 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.072 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.072 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.072 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.072 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.073 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.073 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.073 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.073 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.073 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.073 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.073 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.073 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.073 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.073 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.073 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.073 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.073 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.073 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.073 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.073 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.074 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.074 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.074 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.074 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.074 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.074 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.074 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.074 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.074 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.074 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.074 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.074 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.074 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.074 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.075 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.075 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.075 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.075 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.075 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.075 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.075 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.075 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.075 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.075 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.075 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.075 INFO:teuthology.orchestra.run.smithi006.stdout:__________ ERROR at teardown of test_versioning_bucket_create_suspend __________ 2024-05-30T17:13:23.075 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.075 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.075 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.076 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.076 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.076 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.076 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.076 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.076 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.076 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.076 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.076 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.076 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.076 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.076 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.076 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.076 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.076 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.077 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.077 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.077 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.077 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.077 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.077 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.077 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.077 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.077 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.077 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.077 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.077 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.077 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.077 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.078 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.078 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.078 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.078 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.078 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.078 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.078 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.078 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.078 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.078 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.078 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.078 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.078 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.078 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.078 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.079 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.079 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.079 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.079 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.079 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.079 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.079 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.079 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.079 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.079 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.079 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.079 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.079 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.079 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.079 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.080 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.080 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.080 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.080 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.080 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.080 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.080 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.080 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.080 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.080 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.080 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.080 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.080 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.080 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.080 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.080 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.081 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.081 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.081 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.081 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.081 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.081 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.081 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.081 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.081 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.081 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.081 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.081 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.081 INFO:teuthology.orchestra.run.smithi006.stdout:_________ ERROR at teardown of test_versioning_obj_create_read_remove __________ 2024-05-30T17:13:23.081 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.081 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.082 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.082 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.082 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.082 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.082 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.082 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.082 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.082 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.082 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.082 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.082 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.082 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.082 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.082 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.083 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.083 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.083 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.083 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.083 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.083 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.083 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.083 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.083 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.083 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.083 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.083 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.083 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.083 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.083 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.083 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.084 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.084 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.084 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.084 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.084 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.084 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.084 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.084 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.084 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.084 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.084 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.084 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.084 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.084 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.084 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.085 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.085 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.085 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.085 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.085 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.085 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.085 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.085 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.085 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.085 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.085 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.085 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.085 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.085 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.085 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.086 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.086 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.086 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.086 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.086 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.086 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.086 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.086 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.086 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.086 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.086 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.086 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.086 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.086 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.086 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.086 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.087 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.087 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.087 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.087 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.087 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.087 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.087 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.087 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.087 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.087 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.087 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.087 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.087 INFO:teuthology.orchestra.run.smithi006.stdout:_______ ERROR at teardown of test_versioning_obj_create_read_remove_head _______ 2024-05-30T17:13:23.087 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.087 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.088 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.088 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.088 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.088 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.088 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.088 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.088 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.088 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.088 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.088 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.088 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.088 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.088 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.088 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.088 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.088 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.089 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.089 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.089 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.089 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.089 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.089 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.089 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.089 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.089 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.089 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.089 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.089 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.089 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.089 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.089 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.090 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.090 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.090 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.090 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.090 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.090 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.090 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.090 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.090 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.090 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.090 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.090 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.090 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.090 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.090 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.091 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.091 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.091 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.091 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.091 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.091 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.091 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.091 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.091 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.091 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.091 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.091 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.091 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.091 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.092 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.092 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.092 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.092 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.092 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.092 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.092 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.092 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.092 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.092 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.092 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.092 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.092 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.092 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.092 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.093 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.093 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.093 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.093 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.093 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.093 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.093 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.093 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.093 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.093 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.093 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.093 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.093 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.093 INFO:teuthology.orchestra.run.smithi006.stdout:_____ ERROR at teardown of test_versioning_obj_plain_null_version_removal ______ 2024-05-30T17:13:23.093 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.093 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.094 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.094 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.094 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.094 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.094 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.094 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.094 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.094 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.094 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.094 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.094 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.094 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.094 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.094 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.095 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.095 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.095 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.095 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.095 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.095 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.095 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.095 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.095 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.095 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.095 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.095 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.095 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.095 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.095 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.095 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.096 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.096 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.096 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.096 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.096 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.096 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.096 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.096 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.096 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.096 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.096 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.096 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.096 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.096 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.096 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.097 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.097 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.097 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.097 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.097 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.097 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.097 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.097 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.097 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.097 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.097 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.097 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.097 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.097 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.097 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.098 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.098 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.098 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.098 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.098 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.098 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.098 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.098 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.098 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.098 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.098 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.098 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.098 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.098 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.098 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.099 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.099 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.099 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.099 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.099 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.099 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.099 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.099 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.099 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.099 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.099 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.099 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.099 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.099 INFO:teuthology.orchestra.run.smithi006.stdout:____ ERROR at teardown of test_versioning_obj_plain_null_version_overwrite _____ 2024-05-30T17:13:23.099 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.099 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.100 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.100 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.100 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.100 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.100 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.100 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.100 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.100 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.100 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.100 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.100 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.100 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.100 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.100 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.100 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.101 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.101 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.101 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.101 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.101 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.101 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.101 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.101 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.101 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.101 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.101 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.101 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.101 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.101 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.101 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.101 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.102 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.102 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.102 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.102 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.102 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.102 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.102 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.102 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.102 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.102 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.102 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.102 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.102 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.103 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.103 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.103 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.103 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.103 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.103 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.103 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.103 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.103 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.103 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.103 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.103 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.103 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.103 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.103 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.104 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.104 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.104 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.104 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.104 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.104 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.104 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.104 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.104 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.104 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.104 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.104 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.104 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.104 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.104 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.104 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.105 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.105 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.105 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.105 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.105 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.105 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.105 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.105 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.105 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.105 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.105 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.105 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.105 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.105 INFO:teuthology.orchestra.run.smithi006.stdout:_ ERROR at teardown of test_versioning_obj_plain_null_version_overwrite_suspended _ 2024-05-30T17:13:23.105 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.106 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.106 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.106 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.106 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.106 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.106 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.106 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.106 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.106 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.106 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.106 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.106 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.106 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.106 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.106 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.107 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.107 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.107 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.107 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.107 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.107 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.107 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.107 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.107 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.107 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.107 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.107 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.107 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.107 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.107 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.108 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.108 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.108 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.108 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.108 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.108 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.108 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.108 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.108 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.108 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.108 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.108 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.108 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.108 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.108 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.108 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.109 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.109 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.109 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.109 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.109 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.109 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.109 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.109 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.109 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.109 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.109 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.109 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.109 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.109 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.109 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.110 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.110 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.110 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.110 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.110 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.110 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.110 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.110 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.110 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.110 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.110 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.110 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.110 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.110 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.110 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.110 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.111 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.111 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.111 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.111 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.111 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.111 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.111 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.111 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.111 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.111 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.111 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.111 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.111 INFO:teuthology.orchestra.run.smithi006.stdout:__________ ERROR at teardown of test_versioning_obj_suspend_versions ___________ 2024-05-30T17:13:23.111 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.111 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.112 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.112 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.112 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.112 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.112 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.112 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.112 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.112 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.112 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.112 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.112 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.112 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.112 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.112 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.112 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.112 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.113 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.113 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.113 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.113 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.113 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.113 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.113 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.113 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.113 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.113 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.113 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.113 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.113 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.113 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.113 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.114 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.114 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.114 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.114 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.114 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.114 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.114 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.114 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.114 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.114 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.114 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.114 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.114 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.114 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.114 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.115 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.115 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.115 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.115 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.115 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.115 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.115 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.115 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.115 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.115 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.115 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.115 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.115 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.115 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.115 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.116 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.116 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.116 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.116 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.116 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.116 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.116 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.116 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.116 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.116 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.116 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.116 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.116 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.116 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.117 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.117 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.117 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.117 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.117 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.117 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.117 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.117 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.117 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.117 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.117 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.117 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.117 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.117 INFO:teuthology.orchestra.run.smithi006.stdout:___________ ERROR at teardown of test_versioning_obj_suspended_copy ____________ 2024-05-30T17:13:23.117 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.117 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.118 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.118 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.118 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.118 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.118 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.118 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.118 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.118 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.118 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.118 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.118 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.118 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.118 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.118 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.119 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.119 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.119 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.119 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.119 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.119 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.119 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.119 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.119 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.119 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.119 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.119 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.119 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.119 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.119 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.120 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.120 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.120 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.120 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.120 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.120 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.120 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.120 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.120 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.120 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.120 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.120 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.120 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.120 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.121 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.121 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.121 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.121 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.121 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.121 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.121 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.121 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.121 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.121 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.121 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.121 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.121 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.121 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.122 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.122 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.122 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.122 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.122 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.122 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.122 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.122 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.122 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.122 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.122 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.122 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.122 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.122 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.123 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.123 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.123 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.123 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.123 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.123 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.123 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.123 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.123 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.123 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.123 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.123 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.123 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.123 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.123 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.124 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.124 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.124 INFO:teuthology.orchestra.run.smithi006.stdout:_____ ERROR at teardown of test_versioning_obj_create_versions_remove_all ______ 2024-05-30T17:13:23.124 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.124 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.124 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.124 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.124 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.124 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.124 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.124 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.124 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.124 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.124 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.124 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.124 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.125 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.125 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.125 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.125 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.125 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.125 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.125 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.125 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.125 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.125 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.125 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.125 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.125 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.125 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.125 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.125 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.126 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.126 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.126 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.126 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.126 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.126 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.126 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.126 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.126 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.126 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.126 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.126 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.126 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.126 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.126 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.127 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.127 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.127 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.127 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.127 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.127 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.127 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.127 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.127 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.127 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.127 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.127 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.127 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.127 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.127 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.127 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.128 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.128 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.128 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.128 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.128 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.128 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.128 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.128 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.128 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.128 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.128 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.128 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.128 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.128 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.129 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.129 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.129 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.129 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.129 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.129 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.129 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.129 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.129 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.129 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.129 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.129 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.129 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.129 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.129 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.129 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.130 INFO:teuthology.orchestra.run.smithi006.stdout:_ ERROR at teardown of test_versioning_obj_create_versions_remove_special_names _ 2024-05-30T17:13:23.130 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.130 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.130 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.130 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.130 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.130 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.130 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.130 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.130 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.130 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.130 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.130 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.130 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.130 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.131 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.131 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.131 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.131 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.131 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.131 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.131 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.131 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.131 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.131 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.131 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.131 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.131 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.131 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.131 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.132 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.132 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.132 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.132 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.132 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.132 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.132 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.132 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.132 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.132 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.132 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.132 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.132 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.132 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.132 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.132 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.133 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.133 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.133 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.133 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.133 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.133 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.133 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.133 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.133 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.133 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.133 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.133 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.133 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.133 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.133 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.134 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.134 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.134 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.134 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.134 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.134 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.134 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.134 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.134 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.134 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.134 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.134 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.134 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.134 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.134 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.135 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.135 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.135 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.135 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.135 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.135 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.135 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.135 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.135 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.135 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.135 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.135 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.135 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.135 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.135 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.136 INFO:teuthology.orchestra.run.smithi006.stdout:_____ ERROR at teardown of test_versioning_obj_create_overwrite_multipart ______ 2024-05-30T17:13:23.136 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.136 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.136 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.136 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.136 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.136 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.136 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.136 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.136 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.136 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.136 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.136 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.136 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.136 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.136 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.137 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.137 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.137 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.137 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.137 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.137 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.137 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.137 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.137 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.137 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.137 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.137 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.137 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.137 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.137 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.138 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.138 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.138 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.138 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.138 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.138 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.138 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.138 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.138 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.138 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.138 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.138 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.138 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.138 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.138 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.139 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.139 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.139 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.139 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.139 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.139 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.139 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.139 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.139 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.139 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.139 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.139 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.139 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.139 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.139 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.139 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.140 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.140 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.140 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.140 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.140 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.140 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.140 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.140 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.140 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.140 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.140 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.140 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.140 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.140 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.140 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.140 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.141 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.141 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.141 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.141 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.141 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.141 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.141 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.141 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.141 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.141 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.141 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.142 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.142 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.142 INFO:teuthology.orchestra.run.smithi006.stdout:_____________ ERROR at teardown of test_versioning_obj_list_marker _____________ 2024-05-30T17:13:23.142 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.142 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.142 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.142 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.142 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.142 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.142 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.142 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.142 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.142 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.142 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.142 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.142 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.143 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.143 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.143 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.143 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.143 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.143 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.143 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.143 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.143 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.143 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.143 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.143 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.143 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.143 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.143 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.144 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.144 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.144 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.144 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.144 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.144 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.144 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.144 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.144 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.144 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.144 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.144 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.144 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.144 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.145 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.145 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.145 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.145 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.145 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.145 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.145 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.145 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.145 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.145 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.145 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.145 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.145 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.145 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.145 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.146 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.146 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.146 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.146 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.146 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.146 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.146 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.146 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.146 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.146 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.146 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.146 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.146 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.146 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.146 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.147 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.147 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.147 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.147 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.147 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.147 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.147 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.147 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.147 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.147 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.147 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.147 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.147 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.147 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.147 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.147 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.148 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.148 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.148 INFO:teuthology.orchestra.run.smithi006.stdout:____________ ERROR at teardown of test_versioning_copy_obj_version _____________ 2024-05-30T17:13:23.148 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.148 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.148 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.148 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.148 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.148 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.148 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.148 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.148 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.148 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.148 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.148 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.149 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.149 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.149 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.149 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.149 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.149 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.149 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.149 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.149 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.149 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.149 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.149 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.149 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.149 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.149 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.150 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.150 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.150 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.150 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.150 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.150 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.150 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.150 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.150 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.150 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.150 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.150 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.150 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.150 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.150 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.151 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.151 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.151 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.151 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.151 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.151 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.151 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.151 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.151 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.151 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.151 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.151 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.151 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.151 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.152 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.152 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.152 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.152 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.152 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.152 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.152 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.152 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.152 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.152 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.152 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.152 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.152 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.152 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.153 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.153 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.153 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.153 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.153 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.153 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.153 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.153 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.153 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.153 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.153 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.153 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.153 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.153 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.154 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.154 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.154 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.154 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.154 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.154 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.154 INFO:teuthology.orchestra.run.smithi006.stdout:___________ ERROR at teardown of test_versioning_multi_object_delete ___________ 2024-05-30T17:13:23.154 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.154 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.154 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.154 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.154 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.154 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.154 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.155 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.155 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.155 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.155 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.155 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.155 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.155 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.155 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.155 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.155 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.155 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.155 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.155 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.156 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.156 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.156 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.156 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.156 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.156 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.156 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.156 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.156 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.156 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.156 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.156 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.156 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.156 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.157 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.157 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.157 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.157 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.157 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.157 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.157 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.157 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.157 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.157 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.157 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.157 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.157 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.157 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.157 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.157 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.158 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.158 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.158 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.158 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.158 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.158 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.158 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.158 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.158 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.158 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.158 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.158 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.158 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.158 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.159 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.159 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.159 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.159 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.159 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.159 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.159 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.159 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.159 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.159 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.159 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.159 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.159 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.159 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.159 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.159 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.160 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.160 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.160 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.160 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.160 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.160 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.160 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.160 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.160 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.160 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.160 INFO:teuthology.orchestra.run.smithi006.stdout:_____ ERROR at teardown of test_versioning_multi_object_delete_with_marker _____ 2024-05-30T17:13:23.160 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.160 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.160 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.160 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.161 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.161 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.161 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.161 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.161 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.161 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.161 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.161 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.161 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.161 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.161 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.161 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.161 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.161 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.161 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.161 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.162 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.162 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.162 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.162 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.162 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.162 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.162 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.162 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.162 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.162 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.162 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.162 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.162 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.162 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.163 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.163 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.163 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.163 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.163 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.163 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.163 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.163 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.163 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.163 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.163 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.163 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.163 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.163 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.163 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.164 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.164 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.164 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.164 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.164 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.164 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.164 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.164 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.164 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.164 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.164 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.164 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.164 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.164 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.164 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.165 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.165 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.165 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.165 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.165 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.165 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.165 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.165 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.165 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.165 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.165 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.165 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.165 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.165 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.165 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.165 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.166 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.166 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.166 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.166 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.166 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.166 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.166 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.166 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.166 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.166 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.166 INFO:teuthology.orchestra.run.smithi006.stdout:_ ERROR at teardown of test_versioning_multi_object_delete_with_marker_create __ 2024-05-30T17:13:23.166 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.166 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.166 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.167 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.167 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.167 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.167 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.167 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.167 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.167 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.167 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.167 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.167 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.167 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.167 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.167 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.167 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.167 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.168 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.168 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.168 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.168 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.168 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.168 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.168 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.168 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.168 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.168 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.168 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.168 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.168 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.168 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.169 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.169 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.169 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.169 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.169 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.169 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.169 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.169 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.169 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.169 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.169 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.169 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.169 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.169 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.169 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.169 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.170 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.170 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.170 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.170 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.170 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.170 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.170 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.170 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.170 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.170 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.170 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.170 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.170 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.170 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.171 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.171 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.171 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.171 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.171 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.171 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.171 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.171 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.171 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.171 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.171 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.171 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.171 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.171 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.171 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.172 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.172 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.172 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.172 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.172 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.172 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.172 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.172 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.172 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.172 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.172 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.172 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.172 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.172 INFO:teuthology.orchestra.run.smithi006.stdout:________________ ERROR at teardown of test_versioned_object_acl ________________ 2024-05-30T17:13:23.172 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.172 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.173 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.173 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.173 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.173 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.173 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.173 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.173 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.173 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.173 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.173 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.173 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.173 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.173 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.173 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.173 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.174 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.174 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.174 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.174 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.174 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.174 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.174 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.174 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.174 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.174 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.174 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.174 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.174 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.174 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.174 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.175 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.175 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.175 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.175 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.175 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.175 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.175 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.175 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.175 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.175 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.175 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.175 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.175 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.175 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.175 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.176 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.176 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.176 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.176 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.176 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.176 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.176 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.176 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.176 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.176 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.176 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.176 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.176 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.176 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.176 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.176 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.177 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.177 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.177 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.177 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.177 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.177 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.177 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.177 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.177 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.177 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.177 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.177 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.177 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.177 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.177 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.177 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.178 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.178 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.178 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.178 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.178 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.178 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.178 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.178 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.178 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.178 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.178 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.178 INFO:teuthology.orchestra.run.smithi006.stdout:_____ ERROR at teardown of test_versioned_object_acl_no_version_specified ______ 2024-05-30T17:13:23.178 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.179 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.179 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.179 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.179 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.179 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.179 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.179 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.179 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.179 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.179 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.179 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.179 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.179 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.179 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.179 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.179 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.180 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.180 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.180 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.180 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.180 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.180 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.180 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.180 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.180 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.180 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.180 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.180 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.180 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.180 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.181 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.181 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.181 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.181 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.181 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.181 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.181 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.181 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.181 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.181 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.181 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.181 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.181 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.181 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.181 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.182 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.182 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.182 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.182 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.182 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.182 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.182 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.182 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.182 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.182 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.182 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.182 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.182 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.183 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.183 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.183 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.183 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.183 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.183 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.183 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.183 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.183 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.183 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.183 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.183 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.183 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.183 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.183 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.183 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.184 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.184 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.184 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.184 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.184 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.184 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.184 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.184 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.184 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.184 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.184 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.184 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.184 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.184 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.184 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.185 INFO:teuthology.orchestra.run.smithi006.stdout:___ ERROR at teardown of test_versioned_concurrent_object_create_and_remove ____ 2024-05-30T17:13:23.185 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.185 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.185 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.185 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.185 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.185 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.185 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.185 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.185 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.185 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.185 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.185 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.185 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.185 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.186 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.186 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.186 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.186 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.186 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.186 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.186 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.186 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.186 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.186 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.186 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.186 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.186 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.186 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.186 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.187 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.187 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.187 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.187 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.187 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.187 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.187 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.187 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.187 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.187 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.187 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.187 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.187 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.187 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.187 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.188 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.188 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.188 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.188 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.188 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.188 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.188 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.188 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.188 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.188 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.188 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.188 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.188 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.188 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.188 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.189 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.189 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.189 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.189 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.189 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.189 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.189 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.189 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.189 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.189 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.189 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.189 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.189 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.189 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.189 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.190 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.190 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.190 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.190 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.190 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.190 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.190 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.190 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.190 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.190 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.190 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.190 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.190 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.190 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.190 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.191 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.191 INFO:teuthology.orchestra.run.smithi006.stdout:___________________ ERROR at teardown of test_lifecycle_set ____________________ 2024-05-30T17:13:23.191 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.191 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.191 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.191 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.191 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.191 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.191 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.191 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.191 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.191 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.191 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.191 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.191 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.191 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.192 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.192 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.192 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.192 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.192 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.192 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.192 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.192 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.192 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.192 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.192 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.192 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.192 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.192 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.192 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.193 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.193 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.193 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.193 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.193 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.193 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.193 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.193 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.193 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.193 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.193 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.193 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.193 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.193 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.193 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.194 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.194 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.194 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.194 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.194 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.194 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.194 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.194 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.194 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.194 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.194 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.194 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.194 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.194 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.195 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.195 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.195 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.195 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.195 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.195 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.195 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.195 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.195 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.195 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.195 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.195 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.195 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.195 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.195 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.195 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.196 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.196 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.196 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.196 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.196 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.196 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.196 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.196 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.196 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.196 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.196 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.196 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.196 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.196 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.196 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.197 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.197 INFO:teuthology.orchestra.run.smithi006.stdout:___________________ ERROR at teardown of test_lifecycle_get ____________________ 2024-05-30T17:13:23.197 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.197 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.197 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.197 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.197 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.197 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.197 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.197 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.197 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.197 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.197 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.197 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.197 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.197 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.198 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.198 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.198 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.198 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.198 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.198 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.198 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.198 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.198 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.198 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.198 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.198 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.198 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.198 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.198 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.199 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.199 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.199 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.199 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.199 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.199 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.199 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.199 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.199 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.199 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.199 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.199 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.199 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.199 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.199 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.200 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.200 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.200 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.200 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.200 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.200 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.200 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.200 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.200 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.200 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.200 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.200 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.200 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.200 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.200 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.201 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.201 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.201 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.201 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.201 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.201 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.201 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.201 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.201 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.201 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.201 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.201 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.201 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.201 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.201 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.202 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.202 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.202 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.202 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.202 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.202 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.202 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.202 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.202 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.202 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.202 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.202 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.202 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.202 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.202 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.203 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.203 INFO:teuthology.orchestra.run.smithi006.stdout:________________ ERROR at teardown of test_lifecycle_get_no_id _________________ 2024-05-30T17:13:23.203 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.203 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.203 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.203 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.203 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.203 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.203 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.203 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.203 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.203 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.203 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.203 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.203 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.204 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.204 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.204 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.204 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.204 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.204 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.204 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.204 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.204 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.204 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.204 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.204 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.204 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.204 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.204 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.205 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.205 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.205 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.205 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.205 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.205 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.205 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.205 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.205 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.205 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.205 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.205 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.205 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.205 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.205 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.205 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.206 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.206 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.206 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.206 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.206 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.206 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.206 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.206 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.206 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.206 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.206 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.206 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.206 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.206 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.207 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.207 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.207 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.207 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.207 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.207 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.207 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.207 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.207 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.207 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.207 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.207 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.207 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.207 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.207 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.208 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.208 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.208 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.208 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.208 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.208 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.208 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.208 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.208 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.208 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.208 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.208 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.208 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.208 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.208 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.209 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.209 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.209 INFO:teuthology.orchestra.run.smithi006.stdout:_______________ ERROR at teardown of test_lifecycle_id_too_long ________________ 2024-05-30T17:13:23.209 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.209 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.209 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.209 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.209 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.209 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.209 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.209 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.209 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.209 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.209 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.209 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.209 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.210 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.210 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.210 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.210 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.210 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.210 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.210 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.210 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.210 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.210 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.210 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.210 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.210 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.210 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.211 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.211 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.211 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.211 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.211 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.211 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.211 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.211 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.211 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.211 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.211 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.211 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.211 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.211 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.211 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.211 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.212 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.212 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.212 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.212 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.212 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.212 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.212 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.212 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.212 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.212 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.212 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.212 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.212 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.212 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.212 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.213 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.213 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.213 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.213 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.213 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.213 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.213 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.213 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.213 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.213 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.213 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.213 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.213 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.213 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.213 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.213 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.214 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.214 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.214 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.214 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.214 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.214 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.214 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.214 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.214 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.214 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.214 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.214 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.214 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.214 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.214 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.215 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.215 INFO:teuthology.orchestra.run.smithi006.stdout:_________________ ERROR at teardown of test_lifecycle_same_id __________________ 2024-05-30T17:13:23.215 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.215 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.215 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.215 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.215 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.215 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.215 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.215 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.215 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.215 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.215 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.215 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.216 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.216 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.216 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.216 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.216 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.216 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.216 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.216 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.216 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.216 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.216 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.216 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.216 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.216 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.216 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.217 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.217 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.217 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.217 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.217 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.217 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.217 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.217 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.217 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.217 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.217 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.217 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.217 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.217 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.217 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.218 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.218 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.218 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.218 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.218 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.218 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.218 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.218 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.218 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.218 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.218 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.218 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.218 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.218 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.218 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.219 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.219 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.219 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.219 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.219 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.219 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.219 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.219 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.219 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.219 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.219 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.219 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.219 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.219 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.219 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.220 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.220 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.220 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.220 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.220 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.220 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.220 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.220 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.220 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.220 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.220 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.220 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.220 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.220 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.220 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.221 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.221 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.221 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.221 INFO:teuthology.orchestra.run.smithi006.stdout:______________ ERROR at teardown of test_lifecycle_invalid_status ______________ 2024-05-30T17:13:23.221 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.221 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.221 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.221 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.221 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.221 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.221 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.221 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.221 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.221 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.222 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.222 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.222 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.222 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.222 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.222 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.222 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.222 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.222 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.222 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.222 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.222 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.222 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.222 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.223 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.223 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.223 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.223 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.223 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.223 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.223 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.223 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.223 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.223 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.223 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.223 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.223 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.223 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.223 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.224 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.224 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.224 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.224 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.224 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.224 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.224 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.224 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.224 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.224 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.224 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.224 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.224 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.224 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.224 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.225 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.225 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.225 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.225 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.225 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.225 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.225 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.225 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.225 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.225 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.225 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.225 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.225 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.225 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.226 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.226 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.226 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.226 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.226 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.226 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.226 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.226 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.226 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.226 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.226 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.226 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.226 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.226 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.226 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.227 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.227 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.227 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.227 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.227 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.227 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.227 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.227 INFO:teuthology.orchestra.run.smithi006.stdout:_________________ ERROR at teardown of test_lifecycle_set_date _________________ 2024-05-30T17:13:23.227 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.227 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.227 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.227 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.227 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.227 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.227 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.228 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.228 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.228 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.228 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.228 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.228 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.228 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.228 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.228 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.228 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.228 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.228 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.228 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.228 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.228 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.229 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.229 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.229 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.229 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.229 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.229 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.229 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.229 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.229 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.229 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.229 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.229 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.229 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.229 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.229 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.229 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.230 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.230 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.230 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.230 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.230 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.230 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.230 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.230 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.230 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.230 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.230 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.230 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.230 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.230 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.231 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.231 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.231 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.231 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.231 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.231 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.231 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.231 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.231 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.231 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.231 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.231 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.231 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.231 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.231 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.232 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.232 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.232 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.232 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.232 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.232 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.232 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.232 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.232 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.232 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.232 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.232 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.232 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.232 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.232 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.233 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.233 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.233 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.233 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.233 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.233 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.233 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.233 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.233 INFO:teuthology.orchestra.run.smithi006.stdout:_____________ ERROR at teardown of test_lifecycle_set_invalid_date _____________ 2024-05-30T17:13:23.233 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.233 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.233 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.233 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.233 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.233 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.234 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.234 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.234 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.234 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.234 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.234 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.234 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.234 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.234 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.234 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.234 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.234 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.234 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.234 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.234 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.235 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.235 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.235 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.235 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.235 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.235 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.235 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.235 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.235 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.235 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.235 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.235 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.235 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.235 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.235 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.236 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.236 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.236 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.236 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.236 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.236 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.236 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.236 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.236 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.236 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.236 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.236 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.236 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.236 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.236 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.237 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.237 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.237 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.237 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.237 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.237 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.237 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.237 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.237 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.237 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.237 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.237 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.237 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.237 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.237 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.237 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.238 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.238 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.238 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.238 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.238 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.238 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.238 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.238 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.238 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.238 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.238 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.238 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.238 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.238 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.238 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.239 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.239 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.239 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.239 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.239 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.239 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.239 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.239 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.239 INFO:teuthology.orchestra.run.smithi006.stdout:______________ ERROR at teardown of test_lifecycle_set_noncurrent ______________ 2024-05-30T17:13:23.239 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.239 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.239 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.239 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.239 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.239 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.240 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.240 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.240 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.240 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.240 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.240 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.240 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.240 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.240 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.240 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.240 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.240 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.240 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.240 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.240 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.240 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.241 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.241 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.241 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.241 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.241 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.241 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.241 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.241 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.241 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.241 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.241 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.241 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.241 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.241 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.241 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.241 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.242 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.242 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.242 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.242 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.242 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.242 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.242 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.242 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.242 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.242 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.242 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.242 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.242 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.242 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.242 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.243 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.243 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.243 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.243 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.243 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.243 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.243 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.243 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.243 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.243 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.243 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.243 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.243 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.243 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.243 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.243 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.244 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.244 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.244 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.244 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.244 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.244 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.244 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.244 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.244 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.244 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.244 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.244 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.244 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.244 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.244 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.245 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.245 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.245 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.245 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.245 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.245 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.245 INFO:teuthology.orchestra.run.smithi006.stdout:_____________ ERROR at teardown of test_lifecycle_set_deletemarker _____________ 2024-05-30T17:13:23.245 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.245 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.245 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.245 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.245 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.245 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.245 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.246 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.246 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.246 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.246 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.246 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.246 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.246 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.246 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.246 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.246 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.246 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.246 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.246 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.246 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.247 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.247 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.247 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.247 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.247 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.247 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.247 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.247 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.247 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.247 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.247 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.247 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.247 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.247 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.247 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.247 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.248 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.248 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.248 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.248 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.248 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.248 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.248 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.248 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.248 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.248 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.248 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.248 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.248 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.248 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.248 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.249 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.249 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.249 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.249 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.249 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.249 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.249 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.249 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.249 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.249 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.249 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.249 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.249 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.249 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.249 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.250 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.250 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.250 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.250 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.250 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.250 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.250 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.250 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.250 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.250 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.250 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.250 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.250 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.250 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.250 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.251 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.251 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.251 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.251 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.251 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.251 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.251 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.251 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.251 INFO:teuthology.orchestra.run.smithi006.stdout:________________ ERROR at teardown of test_lifecycle_set_filter ________________ 2024-05-30T17:13:23.251 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.251 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.251 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.251 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.251 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.251 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.252 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.252 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.252 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.252 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.252 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.252 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.252 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.252 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.252 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.252 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.252 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.252 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.252 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.252 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.252 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.253 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.253 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.253 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.253 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.253 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.253 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.253 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.253 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.253 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.253 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.253 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.253 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.253 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.253 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.253 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.253 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.254 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.254 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.254 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.254 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.254 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.254 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.254 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.254 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.254 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.254 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.254 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.254 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.254 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.254 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.255 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.255 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.255 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.255 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.255 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.255 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.255 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.255 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.255 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.255 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.255 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.255 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.255 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.255 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.255 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.256 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.256 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.256 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.256 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.256 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.256 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.256 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.256 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.256 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.256 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.256 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.256 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.256 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.256 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.256 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.257 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.257 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.257 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.257 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.257 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.257 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.257 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.257 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.257 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.257 INFO:teuthology.orchestra.run.smithi006.stdout:_____________ ERROR at teardown of test_lifecycle_set_empty_filter _____________ 2024-05-30T17:13:23.257 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.257 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.257 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.257 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.258 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.258 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.258 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.258 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.258 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.258 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.258 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.258 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.258 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.258 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.258 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.258 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.258 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.259 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.259 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.259 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.259 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.259 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.259 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.259 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.259 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.259 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.259 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.259 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.259 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.259 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.259 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.259 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.260 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.260 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.260 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.260 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.260 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.260 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.260 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.260 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.260 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.260 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.260 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.260 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.260 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.260 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.260 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.261 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.261 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.261 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.261 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.261 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.261 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.261 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.261 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.261 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.261 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.261 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.261 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.261 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.261 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.261 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.262 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.262 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.262 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.262 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.262 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.262 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.262 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.262 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.262 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.262 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.262 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.262 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.262 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.262 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.263 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.263 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.263 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.263 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.263 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.263 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.263 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.263 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.263 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.263 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.263 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.263 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.263 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.263 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.263 INFO:teuthology.orchestra.run.smithi006.stdout:______________ ERROR at teardown of test_lifecycle_set_multipart _______________ 2024-05-30T17:13:23.264 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.264 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.264 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.264 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.264 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.264 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.264 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.264 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.264 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.264 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.264 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.264 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.264 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.264 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.265 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.265 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.265 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.265 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.265 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.265 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.265 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.265 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.265 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.265 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.265 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.265 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.265 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.265 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.265 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.265 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.266 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.266 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.266 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.266 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.266 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.266 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.266 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.266 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.266 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.266 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.266 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.266 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.266 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.266 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.266 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.267 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.267 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.267 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.267 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.267 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.267 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.267 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.267 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.267 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.267 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.267 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.267 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.267 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.267 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.267 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.268 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.268 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.268 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.268 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.268 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.268 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.268 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.268 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.268 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.268 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.268 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.268 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.268 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.268 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.268 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.269 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.269 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.269 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.269 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.269 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.269 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.269 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.269 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.269 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.269 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.269 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.269 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.269 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.269 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.269 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.269 INFO:teuthology.orchestra.run.smithi006.stdout:_______ ERROR at teardown of test_lifecycle_transition_set_invalid_date ________ 2024-05-30T17:13:23.270 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.270 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.270 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.270 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.270 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.270 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.270 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.270 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.270 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.270 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.270 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.270 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.270 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.270 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.271 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.271 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.271 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.271 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.271 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.271 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.271 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.271 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.271 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.271 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.271 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.271 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.271 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.272 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.272 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.272 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.272 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.272 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.272 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.272 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.272 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.272 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.272 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.272 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.272 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.272 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.272 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.272 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.273 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.273 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.273 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.273 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.273 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.273 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.273 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.273 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.273 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.273 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.273 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.273 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.273 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.273 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.273 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.274 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.274 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.274 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.274 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.274 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.274 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.274 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.274 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.274 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.274 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.274 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.274 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.274 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.275 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.275 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.275 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.275 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.275 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.275 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.275 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.275 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.275 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.275 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.275 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.275 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.275 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.275 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.275 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.276 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.276 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.276 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.276 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.276 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.276 INFO:teuthology.orchestra.run.smithi006.stdout:_______________ ERROR at teardown of test_encrypted_transfer_1b ________________ 2024-05-30T17:13:23.276 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.276 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.276 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.276 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.276 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.276 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.276 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.276 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.277 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.277 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.277 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.277 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.277 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.277 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.277 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.277 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.277 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.277 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.277 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.277 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.277 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.277 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.277 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.278 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.278 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.278 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.278 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.278 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.278 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.278 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.278 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.278 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.278 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.278 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.278 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.278 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.278 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.278 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.279 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.279 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.279 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.279 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.279 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.279 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.279 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.279 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.279 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.279 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.279 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.279 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.279 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.279 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.280 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.280 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.280 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.280 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.280 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.280 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.280 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.280 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.280 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.280 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.280 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.280 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.280 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.280 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.280 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.280 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.281 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.281 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.281 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.281 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.281 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.281 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.281 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.281 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.281 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.281 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.281 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.281 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.281 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.281 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.281 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.281 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.282 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.282 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.282 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.282 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.282 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.282 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.282 INFO:teuthology.orchestra.run.smithi006.stdout:_______________ ERROR at teardown of test_encrypted_transfer_1kb _______________ 2024-05-30T17:13:23.282 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.282 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.282 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.282 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.282 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.283 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.283 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.283 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.283 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.283 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.283 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.283 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.283 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.283 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.283 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.283 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.283 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.283 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.283 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.283 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.283 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.284 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.284 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.284 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.284 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.284 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.284 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.284 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.284 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.284 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.284 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.284 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.284 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.284 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.284 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.284 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.284 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.285 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.285 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.285 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.285 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.285 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.285 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.285 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.285 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.285 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.285 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.285 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.285 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.285 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.285 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.286 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.286 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.286 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.286 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.286 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.286 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.286 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.286 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.286 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.286 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.286 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.286 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.286 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.286 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.286 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.287 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.287 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.287 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.287 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.287 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.287 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.287 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.287 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.287 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.287 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.287 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.287 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.287 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.287 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.287 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.288 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.288 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.288 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.288 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.288 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.288 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.288 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.288 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.288 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.288 INFO:teuthology.orchestra.run.smithi006.stdout:_______________ ERROR at teardown of test_encrypted_transfer_1MB _______________ 2024-05-30T17:13:23.288 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.288 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.288 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.288 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.288 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.288 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.289 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.289 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.289 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.289 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.289 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.289 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.289 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.289 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.289 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.289 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.289 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.289 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.289 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.289 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.290 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.290 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.290 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.290 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.290 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.290 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.290 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.290 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.290 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.290 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.290 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.290 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.290 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.290 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.290 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.291 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.291 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.291 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.291 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.291 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.291 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.291 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.291 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.291 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.291 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.291 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.291 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.291 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.291 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.291 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.292 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.292 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.292 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.292 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.292 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.292 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.292 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.292 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.292 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.292 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.292 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.292 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.292 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.292 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.292 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.293 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.293 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.293 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.293 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.293 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.293 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.293 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.293 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.293 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.293 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.293 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.293 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.293 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.293 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.293 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.294 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.294 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.294 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.294 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.294 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.294 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.294 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.294 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.294 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.294 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.294 INFO:teuthology.orchestra.run.smithi006.stdout:_______________ ERROR at teardown of test_encrypted_transfer_13b _______________ 2024-05-30T17:13:23.294 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.294 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.294 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.294 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.295 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.295 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.295 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.295 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.295 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.295 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.295 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.295 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.295 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.295 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.295 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.295 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.295 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.295 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.295 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.296 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.296 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.296 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.296 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.296 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.296 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.296 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.296 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.296 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.296 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.296 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.296 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.296 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.296 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.297 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.297 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.297 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.297 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.297 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.297 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.297 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.297 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.297 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.297 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.297 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.297 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.297 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.297 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.298 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.298 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.298 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.298 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.298 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.298 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.298 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.298 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.298 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.298 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.298 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.298 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.298 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.299 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.299 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.299 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.299 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.299 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.299 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.299 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.299 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.299 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.299 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.299 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.299 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.299 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.299 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.299 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.300 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.300 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.300 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.300 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.300 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.300 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.300 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.300 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.300 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.300 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.300 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.300 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.300 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.300 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.300 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.301 INFO:teuthology.orchestra.run.smithi006.stdout:____________ ERROR at teardown of test_encryption_sse_c_method_head ____________ 2024-05-30T17:13:23.301 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.301 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.301 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.301 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.301 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.301 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.301 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.301 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.301 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.301 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.301 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.301 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.301 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.301 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.302 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.302 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.302 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.302 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.302 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.302 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.302 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.302 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.302 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.302 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.302 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.302 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.302 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.302 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.302 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.303 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.303 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.303 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.303 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.303 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.303 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.303 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.303 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.303 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.303 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.303 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.303 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.303 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.303 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.303 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.303 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.304 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.304 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.304 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.304 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.304 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.304 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.304 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.304 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.304 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.304 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.304 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.304 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.304 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.304 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.304 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.305 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.305 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.305 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.305 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.305 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.305 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.305 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.305 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.305 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.305 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.305 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.305 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.305 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.305 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.305 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.306 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.306 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.306 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.306 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.306 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.306 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.306 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.306 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.306 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.306 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.306 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.306 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.306 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.306 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.306 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.307 INFO:teuthology.orchestra.run.smithi006.stdout:______________ ERROR at teardown of test_encryption_sse_c_present ______________ 2024-05-30T17:13:23.307 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.307 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.307 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.307 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.307 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.307 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.307 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.307 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.307 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.307 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.307 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.307 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.307 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.307 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.308 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.308 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.308 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.308 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.308 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.308 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.308 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.308 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.308 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.308 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.308 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.308 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.308 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.308 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.308 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.309 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.309 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.309 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.309 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.309 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.309 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.309 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.309 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.309 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.309 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.309 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.309 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.309 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.309 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.309 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.310 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.310 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.310 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.310 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.310 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.310 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.310 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.310 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.310 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.310 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.310 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.310 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.310 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.310 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.310 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.310 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.311 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.311 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.311 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.311 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.311 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.311 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.311 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.311 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.311 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.311 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.311 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.311 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.311 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.311 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.312 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.312 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.312 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.312 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.312 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.312 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.312 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.312 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.312 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.312 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.312 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.312 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.312 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.312 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.312 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.313 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.313 INFO:teuthology.orchestra.run.smithi006.stdout:_____________ ERROR at teardown of test_encryption_sse_c_other_key _____________ 2024-05-30T17:13:23.313 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.313 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.313 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.313 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.313 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.313 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.313 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.313 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.313 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.313 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.313 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.313 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.313 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.314 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.314 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.314 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.314 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.314 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.314 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.314 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.314 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.314 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.314 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.314 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.314 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.314 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.314 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.314 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.315 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.315 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.315 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.315 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.315 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.315 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.315 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.315 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.315 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.315 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.315 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.315 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.315 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.315 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.315 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.315 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.316 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.316 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.316 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.316 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.316 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.316 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.316 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.316 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.316 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.316 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.316 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.316 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.316 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.316 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.317 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.317 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.317 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.317 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.317 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.317 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.317 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.317 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.317 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.317 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.317 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.317 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.317 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.317 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.318 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.318 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.318 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.318 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.318 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.318 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.318 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.318 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.318 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.318 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.318 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.318 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.318 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.318 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.319 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.319 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.319 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.319 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.319 INFO:teuthology.orchestra.run.smithi006.stdout:____________ ERROR at teardown of test_encryption_sse_c_invalid_md5 ____________ 2024-05-30T17:13:23.319 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.319 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.319 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.319 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.319 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.319 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.319 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.319 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.319 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.319 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.319 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.320 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.320 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.320 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.320 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.320 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.320 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.320 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.320 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.320 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.320 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.320 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.320 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.320 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.320 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.320 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.321 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.321 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.321 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.321 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.321 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.321 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.321 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.321 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.321 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.321 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.321 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.321 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.321 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.321 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.322 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.322 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.322 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.322 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.322 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.322 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.322 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.322 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.322 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.322 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.322 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.322 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.322 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.322 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.323 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.323 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.323 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.323 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.323 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.323 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.323 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.323 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.323 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.323 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.323 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.323 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.323 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.323 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.323 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.324 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.324 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.324 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.324 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.324 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.324 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.324 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.324 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.324 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.324 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.324 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.324 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.324 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.324 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.324 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.325 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.325 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.325 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.325 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.325 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.325 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.325 INFO:teuthology.orchestra.run.smithi006.stdout:______________ ERROR at teardown of test_encryption_sse_c_no_md5 _______________ 2024-05-30T17:13:23.325 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.325 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.325 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.325 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.325 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.325 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.325 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.325 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.326 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.326 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.326 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.326 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.326 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.326 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.326 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.326 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.326 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.326 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.326 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.326 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.326 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.326 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.326 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.327 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.327 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.327 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.327 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.327 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.327 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.327 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.327 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.327 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.327 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.327 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.327 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.327 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.327 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.327 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.327 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.328 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.328 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.328 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.328 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.328 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.328 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.328 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.328 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.328 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.328 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.328 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.328 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.328 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.328 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.328 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.329 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.329 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.329 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.329 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.329 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.329 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.329 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.329 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.329 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.329 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.329 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.329 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.329 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.329 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.330 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.330 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.330 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.330 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.330 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.330 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.330 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.330 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.330 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.330 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.330 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.330 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.330 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.330 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.331 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.331 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.331 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.331 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.331 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.331 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.331 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.331 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.331 INFO:teuthology.orchestra.run.smithi006.stdout:______________ ERROR at teardown of test_encryption_sse_c_no_key _______________ 2024-05-30T17:13:23.331 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.331 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.331 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.331 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.331 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.331 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.332 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.332 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.332 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.332 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.332 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.332 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.332 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.332 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.332 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.332 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.332 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.332 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.332 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.332 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.332 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.333 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.333 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.333 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.333 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.333 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.333 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.333 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.333 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.333 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.333 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.333 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.333 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.333 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.333 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.334 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.334 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.334 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.334 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.334 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.334 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.334 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.334 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.334 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.334 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.334 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.334 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.334 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.334 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.335 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.335 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.335 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.335 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.335 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.335 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.335 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.335 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.335 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.335 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.335 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.335 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.335 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.335 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.335 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.336 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.336 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.336 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.336 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.336 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.336 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.336 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.336 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.336 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.336 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.336 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.336 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.336 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.336 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.336 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.336 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.337 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.337 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.337 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.337 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.337 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.337 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.337 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.337 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.337 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.337 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.337 INFO:teuthology.orchestra.run.smithi006.stdout:______________ ERROR at teardown of test_encryption_key_no_sse_c _______________ 2024-05-30T17:13:23.337 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.337 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.337 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.337 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.338 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.338 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.338 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.338 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.338 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.338 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.338 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.338 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.338 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.338 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.338 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.338 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.338 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.339 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.339 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.339 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.339 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.339 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.339 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.339 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.339 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.339 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.339 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.339 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.339 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.339 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.339 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.339 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.339 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.340 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.340 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.340 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.340 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.340 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.340 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.340 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.340 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.340 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.340 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.340 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.340 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.340 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.340 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.340 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.341 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.341 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.341 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.341 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.341 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.341 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.341 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.341 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.341 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.341 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.341 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.341 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.341 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.341 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.341 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.341 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.342 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.342 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.342 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.342 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.342 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.342 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.342 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.342 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.342 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.342 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.342 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.342 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.342 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.342 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.343 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.343 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.343 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.343 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.343 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.343 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.343 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.343 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.343 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.343 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.343 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.343 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.343 INFO:teuthology.orchestra.run.smithi006.stdout:_________ ERROR at teardown of test_encryption_sse_c_multipart_upload __________ 2024-05-30T17:13:23.343 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.343 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.344 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.344 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.344 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.344 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.344 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.344 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.344 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.344 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.344 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.344 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.344 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.344 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.344 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.344 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.344 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.344 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.345 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.345 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.345 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.345 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.345 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.345 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.345 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.345 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.345 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.345 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.345 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.345 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.345 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.345 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.345 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.346 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.346 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.346 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.346 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.346 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.346 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.346 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.346 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.346 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.346 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.346 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.346 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.346 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.346 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.347 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.347 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.347 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.347 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.347 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.347 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.347 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.347 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.347 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.347 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.347 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.347 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.347 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.347 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.347 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.347 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.348 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.348 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.348 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.348 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.348 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.348 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.348 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.348 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.348 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.348 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.348 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.348 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.348 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.348 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.348 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.349 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.349 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.349 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.349 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.349 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.349 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.349 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.349 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.349 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.349 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.349 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.349 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.349 INFO:teuthology.orchestra.run.smithi006.stdout:____ ERROR at teardown of test_encryption_sse_c_unaligned_multipart_upload _____ 2024-05-30T17:13:23.349 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.349 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.349 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.350 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.350 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.350 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.350 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.350 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.350 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.350 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.350 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.350 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.350 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.350 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.350 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.350 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.350 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.351 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.351 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.351 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.351 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.351 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.351 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.351 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.351 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.351 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.351 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.351 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.351 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.351 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.351 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.352 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.352 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.352 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.352 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.352 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.352 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.352 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.352 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.352 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.352 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.352 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.352 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.352 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.352 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.352 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.352 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.353 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.353 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.353 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.353 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.353 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.353 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.353 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.353 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.353 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.353 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.353 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.353 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.353 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.353 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.353 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.354 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.354 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.354 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.354 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.354 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.354 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.354 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.354 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.354 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.354 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.354 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.354 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.354 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.354 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.354 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.355 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.355 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.355 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.355 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.355 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.355 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.355 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.355 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.355 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.355 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.355 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.355 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.355 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.355 INFO:teuthology.orchestra.run.smithi006.stdout:______ ERROR at teardown of test_encryption_sse_c_multipart_bad_download _______ 2024-05-30T17:13:23.355 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.356 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.356 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.356 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.356 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.356 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.356 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.356 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.356 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.356 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.356 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.356 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.356 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.356 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.357 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.357 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.357 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.357 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.357 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.357 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.357 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.357 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.357 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.357 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.357 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.357 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.357 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.357 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.357 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.358 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.358 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.358 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.358 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.358 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.358 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.358 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.358 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.358 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.358 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.358 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.358 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.358 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.359 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.359 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.359 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.359 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.359 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.359 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.359 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.359 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.359 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.359 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.359 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.359 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.359 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.359 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.359 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.359 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.360 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.360 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.360 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.360 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.360 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.360 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.360 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.360 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.360 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.360 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.360 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.360 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.360 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.360 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.360 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.360 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.361 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.361 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.361 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.361 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.361 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.361 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.361 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.361 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.361 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.361 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.361 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.361 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.361 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.361 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.361 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.362 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.362 INFO:teuthology.orchestra.run.smithi006.stdout:_ ERROR at teardown of test_encryption_sse_c_post_object_authenticated_request _ 2024-05-30T17:13:23.362 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.362 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.362 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.362 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.362 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.362 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.362 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.362 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.362 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.362 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.362 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.362 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.363 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.363 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.363 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.363 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.363 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.363 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.363 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.363 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.363 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.363 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.363 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.363 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.363 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.363 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.363 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.363 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.364 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.364 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.364 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.364 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.364 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.364 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.364 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.364 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.364 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.364 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.364 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.364 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.364 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.364 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.365 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.365 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.365 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.365 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.365 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.365 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.365 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.365 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.365 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.365 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.365 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.365 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.365 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.365 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.365 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.366 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.366 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.366 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.366 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.366 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.366 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.366 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.366 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.366 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.366 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.366 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.366 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.366 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.366 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.367 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.367 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.367 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.367 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.367 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.367 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.367 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.367 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.367 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.367 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.367 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.367 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.367 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.367 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.367 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.368 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.368 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.368 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.368 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.368 INFO:teuthology.orchestra.run.smithi006.stdout:________________ ERROR at teardown of test_sse_kms_method_head _________________ 2024-05-30T17:13:23.368 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.368 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.368 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.368 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.368 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.368 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.368 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.368 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.368 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.368 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.368 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.369 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.369 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.369 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.369 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.369 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.369 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.369 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.369 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.369 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.369 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.369 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.369 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.369 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.369 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.370 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.370 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.370 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.370 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.370 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.370 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.370 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.370 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.370 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.370 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.370 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.370 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.370 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.371 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.371 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.371 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.371 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.371 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.371 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.371 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.371 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.371 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.371 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.371 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.371 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.371 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.371 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.372 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.372 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.372 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.372 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.372 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.372 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.372 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.372 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.372 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.372 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.372 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.372 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.372 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.372 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.372 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.373 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.373 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.373 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.373 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.373 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.373 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.373 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.373 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.373 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.373 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.373 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.373 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.373 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.373 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.373 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.373 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.374 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.374 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.374 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.374 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.374 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.374 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.374 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.374 INFO:teuthology.orchestra.run.smithi006.stdout:__________________ ERROR at teardown of test_sse_kms_present ___________________ 2024-05-30T17:13:23.374 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.374 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.374 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.374 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.374 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.374 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.375 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.375 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.375 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.375 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.375 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.375 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.375 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.375 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.375 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.375 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.375 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.375 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.375 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.375 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.375 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.376 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.376 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.376 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.376 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.376 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.376 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.376 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.376 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.376 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.376 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.376 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.376 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.376 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.376 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.376 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.376 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.377 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.377 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.377 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.377 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.377 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.377 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.377 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.377 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.377 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.377 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.377 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.377 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.377 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.377 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.378 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.378 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.378 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.378 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.378 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.378 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.378 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.378 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.378 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.378 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.378 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.378 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.378 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.379 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.379 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.379 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.379 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.379 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.379 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.379 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.379 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.379 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.379 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.379 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.379 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.379 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.379 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.379 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.380 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.380 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.380 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.380 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.380 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.380 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.380 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.380 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.380 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.380 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.380 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.380 INFO:teuthology.orchestra.run.smithi006.stdout:___________________ ERROR at teardown of test_sse_kms_no_key ___________________ 2024-05-30T17:13:23.380 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.380 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.381 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.381 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.381 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.381 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.381 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.381 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.381 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.381 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.381 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.381 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.381 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.381 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.381 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.381 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.381 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.381 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.382 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.382 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.382 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.382 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.382 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.382 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.382 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.382 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.382 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.382 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.382 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.382 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.382 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.383 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.383 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.383 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.383 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.383 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.383 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.383 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.383 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.383 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.383 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.383 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.383 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.383 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.383 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.383 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.384 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.384 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.384 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.384 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.384 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.384 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.384 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.384 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.384 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.384 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.384 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.384 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.384 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.384 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.384 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.384 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.385 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.385 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.385 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.385 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.385 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.385 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.385 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.385 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.385 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.385 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.385 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.385 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.385 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.385 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.386 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.386 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.386 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.386 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.386 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.386 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.386 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.386 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.386 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.386 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.386 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.386 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.386 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.387 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.387 INFO:teuthology.orchestra.run.smithi006.stdout:________________ ERROR at teardown of test_sse_kms_not_declared ________________ 2024-05-30T17:13:23.387 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.387 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.387 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.387 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.387 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.387 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.387 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.387 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.387 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.387 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.387 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.387 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.388 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.388 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.388 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.388 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.388 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.388 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.388 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.388 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.388 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.388 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.388 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.388 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.388 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.388 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.389 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.389 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.389 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.389 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.389 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.389 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.389 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.389 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.389 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.389 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.389 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.389 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.389 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.389 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.389 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.389 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.390 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.390 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.390 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.390 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.390 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.390 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.390 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.390 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.390 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.390 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.390 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.390 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.390 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.391 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.391 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.391 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.391 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.391 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.391 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.391 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.391 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.391 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.391 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.391 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.391 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.391 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.391 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.391 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.391 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.392 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.392 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.392 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.392 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.392 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.392 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.392 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.392 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.392 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.392 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.392 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.392 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.392 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.392 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.393 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.393 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.393 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.393 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.393 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.393 INFO:teuthology.orchestra.run.smithi006.stdout:______________ ERROR at teardown of test_sse_kms_multipart_upload ______________ 2024-05-30T17:13:23.393 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.393 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.393 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.393 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.393 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.393 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.393 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.393 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.393 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.394 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.394 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.394 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.394 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.394 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.394 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.394 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.394 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.394 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.394 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.394 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.394 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.394 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.394 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.395 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.395 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.395 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.395 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.395 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.395 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.395 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.395 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.395 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.395 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.395 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.395 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.395 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.395 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.395 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.395 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.396 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.396 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.396 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.396 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.396 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.396 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.396 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.396 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.396 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.396 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.396 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.396 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.396 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.396 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.396 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.397 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.397 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.397 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.397 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.397 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.397 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.397 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.397 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.397 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.397 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.397 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.397 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.397 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.397 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.397 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.398 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.398 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.398 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.398 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.398 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.398 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.398 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.398 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.398 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.398 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.398 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.398 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.398 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.398 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.398 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.399 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.399 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.399 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.399 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.399 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.399 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.399 INFO:teuthology.orchestra.run.smithi006.stdout:_________ ERROR at teardown of test_sse_kms_multipart_invalid_chunks_1 _________ 2024-05-30T17:13:23.399 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.399 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.399 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.399 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.399 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.399 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.399 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.399 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.399 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.400 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.400 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.400 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.400 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.400 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.400 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.400 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.400 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.400 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.400 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.400 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.400 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.400 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.400 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.401 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.401 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.401 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.401 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.401 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.401 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.401 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.401 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.401 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.401 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.401 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.401 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.401 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.401 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.401 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.401 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.402 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.402 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.402 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.402 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.402 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.402 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.402 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.402 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.402 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.402 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.402 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.402 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.402 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.402 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.403 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.403 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.403 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.403 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.403 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.403 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.403 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.403 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.403 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.403 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.403 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.403 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.403 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.403 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.403 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.404 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.404 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.404 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.404 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.404 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.404 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.404 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.404 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.404 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.404 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.404 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.404 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.404 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.405 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.405 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.405 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.405 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.405 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.405 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.405 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.405 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.405 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.405 INFO:teuthology.orchestra.run.smithi006.stdout:_________ ERROR at teardown of test_sse_kms_multipart_invalid_chunks_2 _________ 2024-05-30T17:13:23.405 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.405 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.405 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.405 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.405 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.405 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.406 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.406 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.406 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.406 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.406 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.406 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.406 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.406 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.406 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.406 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.406 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.406 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.406 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.406 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.407 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.407 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.407 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.407 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.407 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.407 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.407 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.407 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.407 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.407 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.407 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.407 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.407 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.407 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.407 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.408 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.408 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.408 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.408 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.408 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.408 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.408 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.408 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.408 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.408 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.408 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.408 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.408 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.408 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.408 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.409 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.409 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.409 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.409 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.409 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.409 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.409 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.409 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.409 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.409 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.409 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.409 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.409 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.409 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.409 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.409 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.410 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.410 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.410 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.410 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.410 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.410 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.410 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.410 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.410 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.410 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.410 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.410 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.410 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.410 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.410 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.411 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.411 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.411 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.411 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.411 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.411 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.411 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.411 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.411 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.411 INFO:teuthology.orchestra.run.smithi006.stdout:_____ ERROR at teardown of test_sse_kms_post_object_authenticated_request ______ 2024-05-30T17:13:23.411 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.411 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.411 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.411 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.411 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.412 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.412 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.412 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.412 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.412 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.412 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.412 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.412 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.412 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.412 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.412 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.412 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.412 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.412 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.412 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.413 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.413 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.413 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.413 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.413 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.413 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.413 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.413 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.413 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.413 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.413 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.413 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.413 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.413 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.413 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.413 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.414 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.414 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.414 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.414 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.414 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.414 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.414 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.414 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.414 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.414 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.414 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.414 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.414 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.414 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.415 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.415 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.415 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.415 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.415 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.415 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.415 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.415 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.415 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.415 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.415 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.415 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.415 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.415 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.415 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.415 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.416 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.416 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.416 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.416 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.416 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.416 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.416 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.416 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.416 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.416 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.416 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.416 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.416 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.416 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.416 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.417 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.417 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.417 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.417 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.417 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.417 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.417 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.417 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.417 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.417 INFO:teuthology.orchestra.run.smithi006.stdout:________________ ERROR at teardown of test_sse_kms_transfer_1b _________________ 2024-05-30T17:13:23.417 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.417 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.417 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.418 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.418 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.418 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.418 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.418 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.418 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.418 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.418 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.418 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.418 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.418 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.418 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.418 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.418 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.418 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.419 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.419 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.419 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.419 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.419 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.419 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.419 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.419 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.419 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.419 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.419 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.419 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.419 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.419 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.419 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.419 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.420 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.420 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.420 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.420 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.420 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.420 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.420 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.420 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.420 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.420 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.420 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.420 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.420 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.420 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.420 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.421 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.421 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.421 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.421 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.421 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.421 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.421 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.421 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.421 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.421 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.421 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.421 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.421 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.421 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.422 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.422 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.422 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.422 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.422 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.422 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.422 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.422 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.422 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.422 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.422 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.422 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.422 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.422 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.423 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.423 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.423 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.423 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.423 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.423 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.423 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.423 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.423 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.423 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.423 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.423 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.423 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.423 INFO:teuthology.orchestra.run.smithi006.stdout:________________ ERROR at teardown of test_sse_kms_transfer_1kb ________________ 2024-05-30T17:13:23.423 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.424 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.424 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.424 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.424 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.424 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.424 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.424 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.424 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.424 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.424 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.424 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.424 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.424 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.424 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.424 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.425 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.425 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.425 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.425 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.425 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.425 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.425 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.425 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.425 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.425 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.425 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.425 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.425 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.425 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.425 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.425 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.426 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.426 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.426 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.426 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.426 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.426 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.426 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.426 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.426 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.426 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.426 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.426 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.426 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.426 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.426 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.427 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.427 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.427 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.427 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.427 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.427 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.427 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.427 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.427 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.427 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.427 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.427 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.427 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.427 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.427 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.428 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.428 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.428 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.428 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.428 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.428 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.428 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.428 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.428 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.428 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.428 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.428 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.428 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.428 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.428 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.428 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.429 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.429 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.429 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.429 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.429 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.429 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.429 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.429 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.429 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.429 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.429 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.429 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.429 INFO:teuthology.orchestra.run.smithi006.stdout:________________ ERROR at teardown of test_sse_kms_transfer_1MB ________________ 2024-05-30T17:13:23.429 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.429 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.429 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.430 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.430 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.430 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.430 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.430 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.430 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.430 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.430 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.430 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.430 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.430 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.430 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.430 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.430 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.431 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.431 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.431 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.431 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.431 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.431 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.431 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.431 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.431 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.431 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.431 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.431 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.431 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.431 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.431 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.432 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.432 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.432 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.432 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.432 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.432 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.432 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.432 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.432 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.432 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.432 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.432 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.432 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.432 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.432 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.433 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.433 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.433 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.433 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.433 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.433 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.433 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.433 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.433 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.433 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.433 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.433 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.433 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.433 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.433 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.433 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.434 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.434 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.434 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.434 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.434 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.434 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.434 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.434 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.434 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.434 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.434 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.434 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.434 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.434 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.435 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.435 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.435 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.435 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.435 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.435 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.435 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.435 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.435 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.435 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.435 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.435 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.435 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.435 INFO:teuthology.orchestra.run.smithi006.stdout:________________ ERROR at teardown of test_sse_kms_transfer_13b ________________ 2024-05-30T17:13:23.435 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.435 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.436 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.436 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.436 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.436 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.436 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.436 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.436 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.436 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.436 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.436 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.436 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.436 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.436 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.436 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.436 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.437 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.437 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.437 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.437 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.437 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.437 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.437 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.437 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.437 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.437 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.437 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.437 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.437 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.437 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.437 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.437 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.438 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.438 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.438 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.438 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.438 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.438 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.438 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.438 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.438 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.438 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.438 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.438 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.438 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.438 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.439 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.439 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.439 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.439 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.439 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.439 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.439 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.439 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.439 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.439 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.439 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.439 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.439 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.439 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.439 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.439 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.440 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.440 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.440 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.440 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.440 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.440 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.440 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.440 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.440 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.440 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.440 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.440 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.440 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.440 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.440 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.440 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.441 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.441 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.441 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.441 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.441 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.441 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.441 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.441 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.441 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.441 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.441 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.441 INFO:teuthology.orchestra.run.smithi006.stdout:________________ ERROR at teardown of test_sse_kms_read_declare ________________ 2024-05-30T17:13:23.441 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.441 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.441 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.442 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.442 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.442 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.442 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.442 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.442 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.442 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.442 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.442 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.442 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.442 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.442 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.442 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.442 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.442 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.443 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.443 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.443 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.443 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.443 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.443 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.443 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.443 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.443 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.443 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.443 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.443 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.443 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.443 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.443 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.444 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.444 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.444 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.444 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.444 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.444 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.444 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.444 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.444 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.444 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.444 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.444 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.444 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.444 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.444 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.445 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.445 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.445 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.445 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.445 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.445 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.445 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.445 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.445 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.445 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.445 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.445 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.445 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.445 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.445 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.446 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.446 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.446 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.446 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.446 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.446 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.446 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.446 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.446 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.446 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.446 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.446 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.446 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.446 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.446 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.447 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.447 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.447 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.447 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.447 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.447 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.447 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.447 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.447 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.447 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.447 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.447 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.447 INFO:teuthology.orchestra.run.smithi006.stdout:___________________ ERROR at teardown of test_bucket_policy ____________________ 2024-05-30T17:13:23.447 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.447 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.447 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.448 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.448 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.448 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.448 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.448 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.448 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.448 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.448 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.448 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.448 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.448 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.448 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.448 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.448 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.448 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.449 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.449 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.449 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.449 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.449 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.449 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.449 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.449 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.449 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.449 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.449 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.449 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.449 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.449 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.449 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.449 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.450 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.450 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.450 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.450 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.450 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.450 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.450 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.450 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.450 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.450 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.450 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.450 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.450 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.450 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.450 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.451 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.451 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.451 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.451 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.451 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.451 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.451 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.451 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.451 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.451 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.451 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.451 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.451 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.451 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.451 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.451 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.452 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.452 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.452 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.452 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.452 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.452 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.452 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.452 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.452 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.452 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.452 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.452 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.452 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.452 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.452 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.453 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.453 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.453 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.453 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.453 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.453 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.453 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.453 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.453 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.453 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.453 INFO:teuthology.orchestra.run.smithi006.stdout:__________________ ERROR at teardown of test_bucketv2_policy ___________________ 2024-05-30T17:13:23.453 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.453 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.453 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.453 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.453 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.454 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.454 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.454 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.454 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.454 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.454 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.454 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.454 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.454 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.454 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.454 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.454 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.454 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.454 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.454 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.455 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.455 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.455 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.455 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.455 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.455 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.455 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.455 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.455 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.455 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.455 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.455 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.455 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.455 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.456 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.456 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.456 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.456 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.456 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.456 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.456 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.456 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.456 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.456 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.456 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.456 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.456 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.456 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.457 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.457 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.457 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.457 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.457 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.457 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.457 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.457 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.457 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.457 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.457 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.457 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.457 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.458 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.458 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.458 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.458 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.458 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.458 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.458 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.458 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.458 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.458 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.458 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.458 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.458 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.459 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.459 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.459 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.459 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.459 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.459 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.459 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.459 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.459 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.459 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.459 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.459 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.459 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.459 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.459 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.460 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.460 INFO:teuthology.orchestra.run.smithi006.stdout:_________________ ERROR at teardown of test_bucket_policy_acl __________________ 2024-05-30T17:13:23.460 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.460 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.460 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.460 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.460 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.460 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.460 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.460 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.460 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.460 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.460 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.460 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.460 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.461 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.461 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.461 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.461 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.461 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.461 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.461 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.461 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.461 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.461 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.461 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.461 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.461 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.461 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.462 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.462 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.462 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.462 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.462 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.462 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.462 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.462 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.462 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.462 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.462 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.462 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.462 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.462 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.463 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.463 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.463 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.463 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.463 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.463 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.463 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.463 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.463 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.463 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.463 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.463 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.463 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.464 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.464 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.464 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.464 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.464 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.464 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.464 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.464 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.464 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.464 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.464 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.464 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.464 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.464 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.464 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.464 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.465 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.465 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.465 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.465 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.465 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.465 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.465 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.465 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.465 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.465 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.465 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.465 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.465 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.465 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.465 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.466 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.466 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.466 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.466 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.466 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.466 INFO:teuthology.orchestra.run.smithi006.stdout:________________ ERROR at teardown of test_bucketv2_policy_acl _________________ 2024-05-30T17:13:23.466 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.466 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.466 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.466 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.466 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.466 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.466 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.466 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.467 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.467 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.467 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.467 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.467 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.467 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.467 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.467 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.467 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.467 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.467 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.467 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.467 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.467 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.467 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.468 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.468 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.468 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.468 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.468 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.468 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.468 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.468 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.468 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.468 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.468 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.468 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.468 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.468 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.468 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.469 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.469 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.469 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.469 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.469 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.469 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.469 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.469 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.469 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.469 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.469 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.469 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.469 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.470 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.470 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.470 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.470 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.470 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.470 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.470 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.470 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.470 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.470 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.470 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.470 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.471 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.471 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.471 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.471 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.471 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.471 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.471 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.471 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.471 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.471 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.471 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.471 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.471 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.471 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.472 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.472 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.472 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.472 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.472 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.472 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.472 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.472 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.472 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.472 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.472 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.472 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.472 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.472 INFO:teuthology.orchestra.run.smithi006.stdout:____________ ERROR at teardown of test_bucket_policy_another_bucket ____________ 2024-05-30T17:13:23.473 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.473 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.473 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.473 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.473 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.473 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.473 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.473 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.473 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.473 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.473 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.473 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.473 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.473 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.474 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.474 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.474 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.474 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.474 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.474 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.474 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.474 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.474 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.474 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.474 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.474 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.474 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.474 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.475 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.475 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.475 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.475 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.475 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.475 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.475 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.475 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.475 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.475 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.475 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.475 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.475 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.475 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.475 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.476 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.476 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.476 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.476 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.476 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.476 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.476 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.476 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.476 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.476 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.476 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.476 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.476 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.476 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.477 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.477 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.477 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.477 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.477 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.477 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.477 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.477 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.477 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.477 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.477 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.477 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.477 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.477 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.478 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.478 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.478 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.478 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.478 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.478 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.478 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.478 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.478 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.478 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.478 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.478 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.478 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.478 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.479 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.479 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.479 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.479 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.479 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.479 INFO:teuthology.orchestra.run.smithi006.stdout:___________ ERROR at teardown of test_bucketv2_policy_another_bucket ___________ 2024-05-30T17:13:23.479 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.479 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.479 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.479 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.479 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.479 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.479 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.479 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.479 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.480 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.480 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.480 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.480 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.480 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.480 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.480 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.480 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.480 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.480 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.480 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.480 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.480 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.480 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.480 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.481 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.481 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.481 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.481 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.481 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.481 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.481 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.481 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.481 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.481 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.481 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.481 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.481 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.481 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.481 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.482 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.482 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.482 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.482 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.482 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.482 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.482 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.482 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.482 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.482 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.482 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.482 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.482 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.482 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.482 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.483 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.483 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.483 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.483 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.483 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.483 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.483 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.483 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.483 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.483 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.483 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.484 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.484 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.484 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.484 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.484 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.484 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.484 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.484 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.484 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.484 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.484 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.484 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.484 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.485 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.485 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.485 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.485 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.485 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.485 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.485 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.485 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.485 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.485 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.485 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.485 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.485 INFO:teuthology.orchestra.run.smithi006.stdout:__________________ ERROR at teardown of test_get_obj_tagging ___________________ 2024-05-30T17:13:23.485 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.485 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.486 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.486 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.486 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.486 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.486 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.486 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.486 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.486 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.486 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.486 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.486 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.486 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.486 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.487 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.487 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.487 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.487 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.487 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.487 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.487 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.487 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.487 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.487 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.487 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.487 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.487 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.487 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.488 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.488 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.488 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.488 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.488 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.488 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.488 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.488 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.488 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.488 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.488 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.488 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.488 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.488 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.488 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.489 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.489 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.489 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.489 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.489 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.489 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.489 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.489 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.489 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.489 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.489 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.489 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.489 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.489 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.489 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.490 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.490 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.490 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.490 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.490 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.490 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.490 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.490 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.490 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.490 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.490 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.490 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.490 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.490 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.490 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.491 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.491 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.491 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.491 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.491 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.491 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.491 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.491 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.491 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.491 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.491 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.491 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.491 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.491 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.492 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.492 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.492 INFO:teuthology.orchestra.run.smithi006.stdout:________________ ERROR at teardown of test_get_obj_head_tagging ________________ 2024-05-30T17:13:23.492 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.492 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.492 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.492 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.492 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.492 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.492 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.492 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.492 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.492 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.492 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.492 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.493 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.493 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.493 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.493 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.493 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.493 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.493 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.493 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.493 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.493 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.493 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.493 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.493 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.493 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.494 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.494 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.494 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.494 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.494 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.494 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.494 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.494 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.494 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.494 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.494 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.494 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.494 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.494 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.495 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.495 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.495 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.495 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.495 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.495 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.495 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.495 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.495 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.495 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.495 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.495 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.495 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.495 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.495 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.496 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.496 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.496 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.496 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.496 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.496 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.496 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.496 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.496 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.496 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.496 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.496 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.496 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.496 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.496 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.496 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.497 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.497 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.497 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.497 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.497 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.497 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.497 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.497 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.497 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.497 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.497 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.497 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.497 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.497 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.498 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.498 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.498 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.498 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.498 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.498 INFO:teuthology.orchestra.run.smithi006.stdout:____________________ ERROR at teardown of test_put_max_tags ____________________ 2024-05-30T17:13:23.498 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.498 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.498 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.498 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.498 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.498 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.498 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.498 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.498 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.499 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.499 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.499 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.499 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.499 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.499 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.499 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.499 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.499 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.499 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.499 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.499 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.499 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.499 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.499 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.500 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.500 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.500 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.500 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.500 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.500 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.500 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.500 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.500 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.500 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.500 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.500 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.500 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.500 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.500 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.501 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.501 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.501 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.501 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.501 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.501 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.501 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.501 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.501 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.501 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.501 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.501 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.501 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.501 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.501 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.502 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.502 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.502 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.502 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.502 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.502 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.502 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.502 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.502 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.502 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.502 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.502 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.502 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.502 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.502 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.503 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.503 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.503 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.503 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.503 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.503 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.503 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.503 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.503 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.503 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.503 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.503 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.503 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.503 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.503 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.504 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.504 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.504 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.504 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.504 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.504 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.504 INFO:teuthology.orchestra.run.smithi006.stdout:__________________ ERROR at teardown of test_put_excess_tags ___________________ 2024-05-30T17:13:23.504 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.504 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.504 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.504 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.504 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.504 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.504 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.504 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.505 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.505 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.505 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.505 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.505 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.505 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.505 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.505 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.505 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.505 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.505 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.505 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.505 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.505 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.505 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.505 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.506 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.506 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.506 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.506 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.506 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.506 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.506 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.506 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.506 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.506 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.506 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.506 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.506 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.506 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.507 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.507 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.507 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.507 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.507 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.507 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.507 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.507 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.507 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.507 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.507 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.507 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.507 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.507 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.507 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.508 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.508 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.508 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.508 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.508 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.508 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.508 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.508 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.508 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.508 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.508 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.508 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.508 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.508 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.508 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.508 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.509 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.509 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.509 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.509 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.509 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.509 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.509 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.509 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.509 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.509 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.509 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.509 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.509 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.509 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.509 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.510 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.510 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.510 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.510 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.510 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.510 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.510 INFO:teuthology.orchestra.run.smithi006.stdout:________________ ERROR at teardown of test_put_max_kvsize_tags _________________ 2024-05-30T17:13:23.510 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.510 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.510 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.510 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.510 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.510 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.510 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.511 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.511 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.511 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.511 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.511 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.511 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.511 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.511 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.511 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.511 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.511 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.511 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.511 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.511 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.512 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.512 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.512 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.512 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.512 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.512 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.512 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.512 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.512 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.512 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.512 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.512 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.512 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.512 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.512 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.512 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.513 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.513 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.513 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.513 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.513 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.513 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.513 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.513 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.513 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.513 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.513 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.513 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.513 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.513 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.514 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.514 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.514 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.514 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.514 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.514 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.514 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.514 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.514 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.514 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.514 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.514 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.514 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.514 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.515 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.515 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.515 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.515 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.515 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.515 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.515 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.515 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.515 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.515 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.515 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.515 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.515 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.515 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.515 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.515 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.516 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.516 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.516 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.516 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.516 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.516 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.516 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.516 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.516 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.516 INFO:teuthology.orchestra.run.smithi006.stdout:________________ ERROR at teardown of test_put_excess_key_tags _________________ 2024-05-30T17:13:23.516 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.516 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.516 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.516 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.516 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.517 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.517 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.517 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.517 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.517 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.517 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.517 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.517 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.517 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.517 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.517 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.517 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.517 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.517 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.518 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.518 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.518 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.518 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.518 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.518 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.518 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.518 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.518 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.518 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.518 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.518 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.518 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.519 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.519 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.519 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.519 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.519 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.519 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.519 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.519 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.519 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.519 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.519 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.519 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.519 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.519 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.519 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.519 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.520 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.520 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.520 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.520 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.520 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.520 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.520 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.520 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.520 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.520 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.520 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.520 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.520 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.520 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.520 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.521 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.521 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.521 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.521 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.521 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.521 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.521 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.521 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.521 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.521 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.521 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.521 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.521 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.521 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.521 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.522 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.522 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.522 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.522 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.522 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.522 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.522 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.522 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.522 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.522 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.522 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.522 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.522 INFO:teuthology.orchestra.run.smithi006.stdout:________________ ERROR at teardown of test_put_excess_val_tags _________________ 2024-05-30T17:13:23.523 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.523 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.523 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.523 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.523 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.523 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.523 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.523 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.523 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.523 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.523 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.523 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.523 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.523 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.524 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.524 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.524 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.524 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.524 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.524 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.524 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.524 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.524 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.524 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.524 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.524 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.524 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.524 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.524 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.524 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.525 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.525 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.525 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.525 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.525 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.525 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.525 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.525 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.525 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.525 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.525 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.525 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.525 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.525 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.525 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.526 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.526 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.526 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.526 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.526 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.526 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.526 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.526 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.526 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.526 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.526 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.526 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.526 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.526 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.526 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.527 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.527 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.527 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.527 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.527 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.527 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.527 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.527 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.527 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.527 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.527 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.527 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.527 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.527 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.527 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.527 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.528 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.528 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.528 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.528 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.528 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.528 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.528 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.528 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.528 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.528 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.528 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.528 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.528 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.528 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.528 INFO:teuthology.orchestra.run.smithi006.stdout:__________________ ERROR at teardown of test_put_modify_tags ___________________ 2024-05-30T17:13:23.529 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.529 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.529 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.529 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.529 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.529 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.529 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.529 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.529 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.529 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.529 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.529 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.529 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.529 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.529 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.529 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.530 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.530 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.530 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.530 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.530 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.530 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.530 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.530 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.530 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.530 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.530 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.530 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.530 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.530 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.531 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.531 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.531 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.531 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.531 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.531 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.531 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.531 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.531 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.531 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.531 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.531 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.531 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.531 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.531 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.531 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.532 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.532 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.532 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.532 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.532 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.532 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.532 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.532 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.532 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.532 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.532 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.532 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.532 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.532 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.532 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.533 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.533 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.533 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.533 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.533 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.533 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.533 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.533 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.533 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.533 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.533 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.533 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.533 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.533 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.533 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.533 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.534 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.534 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.534 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.534 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.534 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.534 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.534 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.534 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.534 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.534 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.534 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.534 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.534 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.534 INFO:teuthology.orchestra.run.smithi006.stdout:__________________ ERROR at teardown of test_put_delete_tags ___________________ 2024-05-30T17:13:23.534 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.535 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.535 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.535 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.535 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.535 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.535 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.535 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.535 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.535 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.535 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.535 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.535 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.535 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.535 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.535 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.536 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.536 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.536 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.536 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.536 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.536 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.536 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.536 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.536 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.536 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.536 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.536 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.536 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.536 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.536 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.537 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.537 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.537 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.537 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.537 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.537 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.537 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.537 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.537 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.537 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.537 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.537 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.537 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.537 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.537 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.537 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.538 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.538 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.538 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.538 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.538 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.538 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.538 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.538 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.538 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.538 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.538 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.538 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.538 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.538 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.539 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.539 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.539 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.539 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.539 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.539 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.539 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.539 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.539 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.539 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.539 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.539 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.539 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.539 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.539 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.540 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.540 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.540 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.540 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.540 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.540 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.540 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.540 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.540 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.540 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.540 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.540 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.540 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.540 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.540 INFO:teuthology.orchestra.run.smithi006.stdout:_________ ERROR at teardown of test_post_object_tags_anonymous_request _________ 2024-05-30T17:13:23.541 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.541 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.541 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.541 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.541 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.541 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.541 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.541 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.541 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.541 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.541 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.541 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.541 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.541 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.541 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.541 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.542 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.542 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.542 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.542 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.542 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.542 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.542 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.542 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.542 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.542 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.542 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.542 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.542 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.542 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.543 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.543 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.543 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.543 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.543 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.543 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.543 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.543 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.543 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.543 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.543 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.543 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.543 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.543 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.543 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.544 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.544 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.544 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.544 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.544 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.544 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.544 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.544 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.544 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.544 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.544 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.544 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.544 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.544 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.544 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.544 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.545 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.545 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.545 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.545 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.545 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.545 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.545 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.545 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.545 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.545 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.545 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.545 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.545 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.545 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.545 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.545 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.546 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.546 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.546 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.546 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.546 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.546 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.546 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.546 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.546 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.546 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.546 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.546 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.546 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.546 INFO:teuthology.orchestra.run.smithi006.stdout:_______ ERROR at teardown of test_post_object_tags_authenticated_request _______ 2024-05-30T17:13:23.546 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.547 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.547 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.547 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.547 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.547 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.547 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.547 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.547 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.547 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.547 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.547 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.547 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.547 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.547 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.547 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.548 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.548 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.548 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.548 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.548 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.548 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.548 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.548 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.548 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.548 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.548 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.548 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.548 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.548 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.548 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.549 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.549 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.549 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.549 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.549 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.549 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.549 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.549 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.549 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.549 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.549 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.549 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.549 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.549 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.549 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.550 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.550 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.550 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.550 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.550 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.550 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.550 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.550 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.550 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.550 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.550 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.550 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.550 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.550 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.550 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.551 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.551 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.551 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.551 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.551 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.551 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.551 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.551 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.551 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.551 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.551 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.551 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.551 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.551 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.552 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.552 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.552 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.552 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.552 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.552 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.552 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.552 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.552 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.552 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.552 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.552 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.552 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.552 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.552 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.553 INFO:teuthology.orchestra.run.smithi006.stdout:_________________ ERROR at teardown of test_put_obj_with_tags __________________ 2024-05-30T17:13:23.553 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.553 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.553 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.553 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.553 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.553 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.553 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.553 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.553 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.553 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.553 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.553 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.553 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.553 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.553 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.554 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.554 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.554 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.554 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.554 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.554 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.554 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.554 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.554 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.554 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.554 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.554 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.554 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.554 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.555 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.555 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.555 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.555 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.555 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.555 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.555 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.555 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.555 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.555 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.555 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.555 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.555 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.555 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.556 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.556 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.556 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.556 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.556 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.556 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.556 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.556 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.556 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.556 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.556 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.556 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.556 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.556 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.556 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.557 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.557 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.557 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.557 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.557 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.557 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.557 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.557 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.557 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.557 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.557 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.557 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.557 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.557 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.558 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.558 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.558 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.558 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.558 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.558 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.558 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.558 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.558 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.558 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.558 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.558 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.558 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.558 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.559 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.559 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.559 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.559 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.559 INFO:teuthology.orchestra.run.smithi006.stdout:________________ ERROR at teardown of test_get_tags_acl_public _________________ 2024-05-30T17:13:23.559 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.559 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.559 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.559 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.559 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.559 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.559 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.559 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.559 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.559 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.560 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.560 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.560 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.560 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.560 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.560 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.560 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.560 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.560 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.560 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.560 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.560 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.560 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.560 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.560 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.560 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.561 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.561 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.561 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.561 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.561 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.561 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.561 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.561 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.561 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.561 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.561 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.561 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.561 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.561 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.561 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.562 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.562 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.562 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.562 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.562 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.562 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.562 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.562 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.562 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.562 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.562 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.562 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.562 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.562 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.563 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.563 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.563 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.563 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.563 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.563 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.563 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.563 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.563 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.563 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.563 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.563 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.563 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.563 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.563 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.563 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.564 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.564 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.564 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.564 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.564 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.564 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.564 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.564 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.564 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.564 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.564 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.564 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.564 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.564 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.564 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.565 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.565 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.565 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.565 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.565 INFO:teuthology.orchestra.run.smithi006.stdout:________________ ERROR at teardown of test_put_tags_acl_public _________________ 2024-05-30T17:13:23.565 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.565 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.565 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.565 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.565 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.565 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.565 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.565 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.565 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.565 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.565 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.566 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.566 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.566 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.566 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.566 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.566 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.566 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.566 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.566 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.566 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.566 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.566 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.566 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.566 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.566 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.567 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.567 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.567 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.567 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.567 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.567 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.567 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.567 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.567 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.567 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.567 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.567 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.567 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.567 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.567 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.568 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.568 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.568 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.568 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.568 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.568 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.568 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.568 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.568 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.568 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.568 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.568 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.568 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.568 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.568 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.569 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.569 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.569 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.569 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.569 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.569 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.569 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.569 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.569 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.569 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.569 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.569 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.569 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.569 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.569 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.569 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.570 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.570 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.570 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.570 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.570 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.570 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.570 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.570 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.570 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.570 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.570 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.570 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.570 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.570 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.570 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.571 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.571 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.571 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.571 INFO:teuthology.orchestra.run.smithi006.stdout:_______________ ERROR at teardown of test_delete_tags_obj_public _______________ 2024-05-30T17:13:23.571 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.571 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.571 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.571 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.571 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.571 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.571 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.571 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.571 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.571 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.571 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.572 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.572 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.572 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.572 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.572 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.572 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.572 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.572 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.572 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.572 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.572 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.572 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.572 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.572 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.572 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.572 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.573 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.573 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.573 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.573 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.573 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.573 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.573 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.573 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.573 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.573 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.573 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.573 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.573 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.573 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.573 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.574 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.574 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.574 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.574 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.574 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.574 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.574 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.574 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.574 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.574 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.574 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.574 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.574 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.574 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.575 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.575 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.575 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.575 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.575 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.575 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.575 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.575 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.575 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.575 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.575 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.575 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.575 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.575 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.575 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.576 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.576 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.576 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.576 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.576 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.576 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.576 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.576 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.576 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.576 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.576 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.576 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.576 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.576 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.576 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.577 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.577 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.577 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.577 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.577 INFO:teuthology.orchestra.run.smithi006.stdout:_ ERROR at teardown of test_versioning_bucket_atomic_upload_return_version_id __ 2024-05-30T17:13:23.577 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.577 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.577 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.577 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.577 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.577 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.577 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.577 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.577 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.577 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.577 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.578 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.578 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.578 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.578 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.578 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.578 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.578 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.578 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.578 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.578 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.578 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.578 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.578 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.578 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.578 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.579 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.579 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.579 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.579 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.579 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.579 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.579 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.579 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.579 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.579 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.579 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.579 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.579 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.579 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.579 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.580 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.580 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.580 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.580 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.580 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.580 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.580 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.580 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.580 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.580 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.580 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.580 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.580 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.580 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.580 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.581 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.581 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.581 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.581 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.581 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.581 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.581 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.581 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.581 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.581 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.581 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.581 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.581 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.581 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.581 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.582 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.582 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.582 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.582 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.582 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.582 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.582 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.582 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.582 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.582 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.582 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.582 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.582 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.582 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.582 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.582 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.583 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.583 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.583 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.583 INFO:teuthology.orchestra.run.smithi006.stdout:_ ERROR at teardown of test_versioning_bucket_multipart_upload_return_version_id _ 2024-05-30T17:13:23.583 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.583 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.583 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.583 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.583 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.583 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.583 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.583 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.583 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.583 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.584 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.584 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.584 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.584 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.584 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.584 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.584 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.584 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.584 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.584 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.584 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.584 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.584 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.584 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.584 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.585 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.585 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.585 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.585 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.585 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.585 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.585 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.585 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.585 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.585 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.585 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.585 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.585 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.585 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.586 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.586 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.586 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.586 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.586 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.586 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.586 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.586 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.586 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.586 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.586 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.586 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.586 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.587 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.587 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.587 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.587 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.587 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.587 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.587 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.587 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.587 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.587 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.587 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.587 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.587 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.587 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.587 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.588 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.588 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.588 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.588 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.588 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.588 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.588 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.588 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.588 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.588 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.588 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.588 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.588 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.589 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.589 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.589 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.589 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.589 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.589 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.589 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.589 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.589 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.589 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.589 INFO:teuthology.orchestra.run.smithi006.stdout:_________ ERROR at teardown of test_bucket_policy_get_obj_existing_tag _________ 2024-05-30T17:13:23.589 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.589 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.589 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.589 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.589 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.590 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.590 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.590 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.590 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.590 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.590 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.590 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.590 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.590 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.590 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.590 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.590 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.590 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.590 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.590 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.591 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.591 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.591 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.591 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.591 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.591 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.591 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.591 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.591 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.591 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.591 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.591 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.591 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.591 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.591 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.592 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.592 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.592 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.592 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.592 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.592 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.592 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.592 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.592 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.592 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.592 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.592 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.592 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.592 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.592 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.592 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.593 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.593 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.593 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.593 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.593 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.593 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.593 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.593 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.593 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.593 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.593 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.593 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.593 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.593 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.593 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.594 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.594 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.594 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.594 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.594 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.594 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.594 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.594 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.594 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.594 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.594 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.594 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.594 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.594 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.594 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.595 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.595 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.595 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.595 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.595 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.595 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.595 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.595 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.595 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.595 INFO:teuthology.orchestra.run.smithi006.stdout:_____ ERROR at teardown of test_bucket_policy_get_obj_tagging_existing_tag _____ 2024-05-30T17:13:23.595 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.595 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.595 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.595 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.595 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.596 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.596 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.596 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.596 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.596 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.596 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.596 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.596 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.596 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.596 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.596 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.596 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.596 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.596 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.596 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.597 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.597 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.597 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.597 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.597 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.597 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.597 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.597 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.597 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.597 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.597 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.597 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.597 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.597 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.597 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.597 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.598 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.598 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.598 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.598 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.598 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.598 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.598 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.598 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.598 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.598 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.598 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.598 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.598 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.598 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.599 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.599 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.599 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.599 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.599 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.599 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.599 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.599 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.599 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.599 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.599 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.599 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.599 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.599 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.600 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.600 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.600 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.600 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.600 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.600 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.600 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.600 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.600 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.600 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.600 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.600 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.600 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.600 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.600 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.600 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.601 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.601 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.601 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.601 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.601 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.601 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.601 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.601 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.601 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.601 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.601 INFO:teuthology.orchestra.run.smithi006.stdout:_____ ERROR at teardown of test_bucket_policy_put_obj_tagging_existing_tag _____ 2024-05-30T17:13:23.601 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.601 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.601 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.601 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.602 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.602 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.602 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.602 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.602 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.602 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.602 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.602 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.602 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.602 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.602 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.602 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.602 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.602 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.603 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.603 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.603 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.603 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.603 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.603 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.603 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.603 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.603 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.603 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.603 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.603 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.603 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.603 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.604 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.604 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.604 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.604 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.604 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.604 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.604 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.604 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.604 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.604 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.604 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.604 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.604 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.604 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.605 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.605 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.605 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.605 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.605 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.605 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.605 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.605 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.605 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.605 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.605 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.605 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.605 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.605 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.605 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.606 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.606 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.606 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.606 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.606 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.606 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.606 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.606 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.606 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.606 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.606 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.606 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.606 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.607 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.607 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.607 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.607 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.607 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.607 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.607 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.607 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.607 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.607 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.607 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.607 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.607 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.607 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.608 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.608 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.608 INFO:teuthology.orchestra.run.smithi006.stdout:_________ ERROR at teardown of test_bucket_policy_put_obj_copy_source __________ 2024-05-30T17:13:23.608 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.608 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.608 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.608 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.608 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.608 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.608 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.608 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.608 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.608 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.608 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.608 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.609 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.609 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.609 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.609 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.609 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.609 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.609 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.609 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.609 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.609 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.609 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.609 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.609 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.609 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.609 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.610 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.610 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.610 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.610 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.610 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.610 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.610 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.610 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.610 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.610 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.610 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.610 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.610 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.610 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.611 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.611 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.611 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.611 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.611 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.611 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.611 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.611 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.611 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.611 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.611 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.611 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.611 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.611 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.611 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.612 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.612 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.612 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.612 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.612 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.612 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.612 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.612 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.612 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.612 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.612 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.612 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.612 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.612 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.612 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.612 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.613 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.613 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.613 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.613 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.613 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.613 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.613 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.613 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.613 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.613 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.613 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.613 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.613 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.613 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.613 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.613 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.614 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.614 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.614 INFO:teuthology.orchestra.run.smithi006.stdout:_______ ERROR at teardown of test_bucket_policy_put_obj_copy_source_meta _______ 2024-05-30T17:13:23.614 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.614 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.614 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.614 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.614 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.614 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.614 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.614 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.614 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.614 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.614 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.615 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.615 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.615 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.615 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.615 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.615 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.615 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.615 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.615 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.615 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.615 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.615 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.615 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.615 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.616 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.616 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.616 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.616 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.616 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.616 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.616 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.616 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.616 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.616 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.616 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.616 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.616 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.616 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.616 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.616 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.617 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.617 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.617 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.617 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.617 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.617 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.617 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.617 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.617 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.617 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.617 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.617 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.617 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.617 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.618 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.618 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.618 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.618 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.618 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.618 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.618 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.618 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.618 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.618 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.618 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.618 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.618 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.618 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.618 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.619 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.619 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.619 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.619 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.619 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.619 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.619 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.619 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.619 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.619 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.619 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.619 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.619 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.619 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.619 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.620 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.620 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.620 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.620 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.620 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.620 INFO:teuthology.orchestra.run.smithi006.stdout:_____________ ERROR at teardown of test_bucket_policy_put_obj_acl ______________ 2024-05-30T17:13:23.620 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.620 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.620 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.620 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.620 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.620 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.620 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.620 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.620 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.621 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.621 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.621 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.621 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.621 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.621 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.621 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.621 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.621 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.621 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.621 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.621 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.621 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.621 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.621 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.621 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.622 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.622 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.622 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.622 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.622 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.622 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.622 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.622 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.622 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.622 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.622 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.622 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.623 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.623 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.623 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.623 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.623 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.623 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.623 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.623 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.623 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.623 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.623 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.623 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.623 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.623 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.624 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.624 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.624 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.624 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.624 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.624 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.624 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.624 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.624 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.624 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.624 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.624 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.624 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.624 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.624 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.625 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.625 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.625 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.625 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.625 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.625 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.625 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.625 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.625 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.625 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.625 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.625 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.625 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.625 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.625 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.626 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.626 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.626 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.626 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.626 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.626 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.626 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.626 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.626 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.626 INFO:teuthology.orchestra.run.smithi006.stdout:____________ ERROR at teardown of test_bucket_policy_put_obj_grant _____________ 2024-05-30T17:13:23.626 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.626 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.626 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.626 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.626 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.627 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.627 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.627 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.627 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.627 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.627 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.627 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.627 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.627 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.627 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.627 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.627 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.627 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.627 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.627 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.627 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.628 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.628 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.628 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.628 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.628 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.628 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.628 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.628 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.628 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.628 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.628 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.628 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.628 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.628 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.629 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.629 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.629 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.629 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.629 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.629 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.629 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.629 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.629 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.629 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.629 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.629 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.629 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.629 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.629 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.629 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.630 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.630 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.630 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.630 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.630 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.630 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.630 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.630 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.630 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.630 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.630 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.630 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.630 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.630 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.631 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.631 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.631 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.631 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.631 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.631 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.631 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.631 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.631 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.631 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.631 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.631 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.631 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.631 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.631 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.632 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.632 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.632 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.632 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.632 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.632 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.632 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.632 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.632 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.632 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.632 INFO:teuthology.orchestra.run.smithi006.stdout:_____________ ERROR at teardown of test_put_obj_enc_conflict_c_s3 ______________ 2024-05-30T17:13:23.632 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.632 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.632 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.632 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.633 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.633 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.633 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.633 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.633 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.633 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.633 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.633 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.633 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.633 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.633 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.633 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.633 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.633 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.633 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.633 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.634 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.634 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.634 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.634 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.634 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.634 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.634 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.634 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.634 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.634 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.634 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.634 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.634 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.634 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.635 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.635 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.635 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.635 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.635 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.635 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.635 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.635 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.635 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.635 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.635 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.635 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.635 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.635 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.635 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.636 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.636 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.636 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.636 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.636 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.636 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.636 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.636 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.636 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.636 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.636 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.636 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.636 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.636 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.636 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.636 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.637 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.637 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.637 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.637 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.637 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.637 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.637 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.637 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.637 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.637 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.637 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.637 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.637 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.637 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.637 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.638 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.638 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.638 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.638 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.638 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.638 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.638 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.638 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.638 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.638 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.638 INFO:teuthology.orchestra.run.smithi006.stdout:_____________ ERROR at teardown of test_put_obj_enc_conflict_c_kms _____________ 2024-05-30T17:13:23.638 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.638 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.638 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.638 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.639 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.639 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.639 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.639 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.639 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.639 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.639 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.639 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.639 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.639 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.639 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.639 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.639 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.639 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.639 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.639 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.640 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.640 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.640 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.640 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.640 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.640 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.640 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.640 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.640 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.640 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.640 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.640 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.640 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.640 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.640 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.641 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.641 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.641 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.641 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.641 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.641 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.641 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.641 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.641 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.641 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.641 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.641 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.641 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.641 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.641 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.642 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.642 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.642 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.642 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.642 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.642 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.642 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.642 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.642 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.642 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.642 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.642 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.642 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.642 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.643 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.643 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.643 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.643 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.643 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.643 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.643 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.643 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.643 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.643 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.643 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.643 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.643 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.643 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.643 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.643 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.644 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.644 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.644 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.644 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.644 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.644 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.644 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.644 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.644 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.644 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.644 INFO:teuthology.orchestra.run.smithi006.stdout:____________ ERROR at teardown of test_put_obj_enc_conflict_s3_kms _____________ 2024-05-30T17:13:23.644 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.644 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.644 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.644 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.645 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.645 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.645 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.645 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.645 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.645 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.645 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.645 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.645 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.645 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.645 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.645 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.645 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.645 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.645 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.645 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.646 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.646 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.646 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.646 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.646 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.646 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.646 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.646 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.646 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.646 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.646 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.646 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.646 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.646 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.646 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.647 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.647 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.647 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.647 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.647 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.647 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.647 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.647 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.647 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.647 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.647 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.647 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.647 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.647 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.647 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.648 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.648 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.648 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.648 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.648 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.648 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.648 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.648 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.648 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.648 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.648 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.648 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.648 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.648 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.649 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.649 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.649 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.649 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.649 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.649 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.649 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.649 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.649 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.649 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.649 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.649 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.649 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.649 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.649 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.650 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.650 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.650 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.650 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.650 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.650 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.650 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.650 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.650 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.650 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.650 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.650 INFO:teuthology.orchestra.run.smithi006.stdout:__________ ERROR at teardown of test_put_obj_enc_conflict_bad_enc_kms __________ 2024-05-30T17:13:23.650 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.650 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.650 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.650 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.651 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.651 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.651 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.651 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.651 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.651 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.651 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.651 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.651 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.651 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.651 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.651 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.651 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.651 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.652 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.652 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.652 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.652 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.652 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.652 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.652 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.652 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.652 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.652 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.652 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.652 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.652 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.653 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.653 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.653 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.653 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.653 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.653 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.653 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.653 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.653 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.653 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.653 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.653 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.653 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.653 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.653 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.654 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.654 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.654 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.654 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.654 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.654 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.654 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.654 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.654 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.654 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.654 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.654 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.655 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.655 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.655 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.655 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.655 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.655 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.655 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.655 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.655 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.655 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.655 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.655 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.655 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.655 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.656 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.656 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.656 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.656 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.656 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.656 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.656 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.656 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.656 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.656 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.656 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.656 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.656 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.656 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.657 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.657 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.657 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.657 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.657 INFO:teuthology.orchestra.run.smithi006.stdout:__________ ERROR at teardown of test_bucket_policy_put_obj_kms_noenc ___________ 2024-05-30T17:13:23.657 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.657 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.657 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.657 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.657 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.657 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.657 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.657 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.658 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.658 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.658 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.658 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.658 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.658 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.658 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.658 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.658 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.658 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.658 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.658 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.658 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.659 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.659 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.659 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.659 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.659 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.659 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.659 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.659 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.659 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.659 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.659 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.659 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.659 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.659 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.659 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.660 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.660 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.660 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.660 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.660 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.660 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.660 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.660 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.660 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.660 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.660 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.660 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.660 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.660 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.661 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.661 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.661 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.661 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.661 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.661 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.661 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.661 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.661 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.661 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.661 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.661 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.661 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.661 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.662 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.662 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.662 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.662 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.662 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.662 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.662 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.662 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.662 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.662 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.662 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.662 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.662 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.662 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.663 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.663 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.663 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.663 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.663 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.663 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.663 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.663 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.663 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.663 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.663 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.663 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.663 INFO:teuthology.orchestra.run.smithi006.stdout:____________ ERROR at teardown of test_bucket_policy_put_obj_kms_s3 ____________ 2024-05-30T17:13:23.663 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.664 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.664 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.664 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.664 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.664 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.664 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.664 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.664 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.664 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.664 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.664 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.664 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.664 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.664 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.664 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.665 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.665 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.665 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.665 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.665 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.665 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.665 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.665 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.665 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.665 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.665 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.665 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.665 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.665 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.666 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.666 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.666 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.666 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.666 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.666 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.666 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.666 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.666 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.666 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.666 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.666 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.666 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.666 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.666 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.667 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.667 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.667 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.667 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.667 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.667 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.667 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.667 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.667 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.667 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.667 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.667 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.667 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.667 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.667 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.668 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.668 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.668 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.668 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.668 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.668 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.668 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.668 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.668 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.668 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.668 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.668 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.668 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.668 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.669 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.669 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.669 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.669 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.669 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.669 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.669 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.669 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.669 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.669 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.669 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.669 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.669 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.669 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.669 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.670 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.670 INFO:teuthology.orchestra.run.smithi006.stdout:_______ ERROR at teardown of test_bucket_policy_get_obj_acl_existing_tag _______ 2024-05-30T17:13:23.670 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.670 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.670 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.670 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.670 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.670 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.670 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.670 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.670 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.670 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.670 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.671 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.671 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.671 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.671 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.671 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.671 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.671 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.671 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.671 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.671 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.671 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.671 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.671 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.671 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.671 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.672 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.672 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.672 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.672 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.672 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.672 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.672 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.672 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.672 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.672 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.672 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.672 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.672 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.672 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.672 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.673 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.673 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.673 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.673 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.673 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.673 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.673 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.673 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.673 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.673 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.673 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.673 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.673 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.674 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.674 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.674 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.674 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.674 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.674 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.674 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.674 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.674 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.674 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.674 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.674 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.674 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.675 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.675 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.675 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.675 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.675 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.675 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.675 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.675 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.675 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.675 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.675 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.675 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.675 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.675 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.675 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.676 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.676 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.676 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.676 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.676 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.676 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.676 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.676 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.676 INFO:teuthology.orchestra.run.smithi006.stdout:______________ ERROR at teardown of test_object_lock_put_obj_lock ______________ 2024-05-30T17:13:23.676 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.676 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.676 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.676 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.676 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.676 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.677 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.677 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.677 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.677 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.677 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.677 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.677 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.677 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.677 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.677 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.677 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.677 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.677 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.677 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.677 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.678 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.678 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.678 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.678 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.678 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.678 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.678 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.678 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.678 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.678 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.678 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.678 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.678 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.678 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.679 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.679 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.679 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.679 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.679 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.679 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.679 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.679 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.679 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.679 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.679 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.679 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.679 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.679 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.679 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.680 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.680 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.680 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.680 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.680 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.680 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.680 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.680 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.680 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.680 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.680 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.680 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.680 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.680 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.680 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.681 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.681 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.681 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.681 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.681 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.681 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.681 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.681 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.681 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.681 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.681 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.681 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.681 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.682 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.682 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.682 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.682 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.682 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.682 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.682 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.682 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.682 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.682 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.682 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.682 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.682 INFO:teuthology.orchestra.run.smithi006.stdout:______ ERROR at teardown of test_object_lock_put_obj_lock_invalid_bucket _______ 2024-05-30T17:13:23.683 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.683 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.683 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.683 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.683 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.683 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.683 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.683 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.683 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.683 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.683 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.683 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.683 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.683 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.684 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.684 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.684 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.684 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.684 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.684 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.684 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.684 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.684 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.684 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.684 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.684 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.684 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.684 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.684 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.685 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.685 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.685 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.685 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.685 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.685 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.685 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.685 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.685 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.685 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.685 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.685 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.685 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.685 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.685 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.685 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.686 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.686 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.686 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.686 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.686 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.686 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.686 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.686 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.686 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.686 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.686 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.686 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.686 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.686 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.687 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.687 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.687 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.687 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.687 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.687 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.687 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.687 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.687 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.687 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.687 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.687 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.687 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.687 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.687 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.688 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.688 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.688 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.688 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.688 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.688 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.688 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.688 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.688 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.688 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.688 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.688 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.688 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.688 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.688 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.689 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.689 INFO:teuthology.orchestra.run.smithi006.stdout:____ ERROR at teardown of test_object_lock_put_obj_lock_with_days_and_years ____ 2024-05-30T17:13:23.689 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.689 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.689 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.689 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.689 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.689 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.689 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.689 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.689 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.689 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.689 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.689 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.690 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.690 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.690 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.690 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.690 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.690 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.690 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.690 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.690 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.690 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.690 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.690 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.690 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.690 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.691 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.691 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.691 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.691 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.691 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.691 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.691 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.691 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.691 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.691 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.691 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.691 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.691 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.691 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.691 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.691 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.692 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.692 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.692 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.692 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.692 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.692 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.692 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.692 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.692 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.692 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.692 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.692 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.692 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.692 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.693 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.693 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.693 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.693 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.693 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.693 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.693 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.693 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.693 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.693 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.693 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.693 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.693 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.693 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.693 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.694 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.694 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.694 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.694 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.694 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.694 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.694 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.694 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.694 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.694 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.694 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.694 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.694 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.694 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.695 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.695 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.695 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.695 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.695 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.695 INFO:teuthology.orchestra.run.smithi006.stdout:_______ ERROR at teardown of test_object_lock_put_obj_lock_invalid_days ________ 2024-05-30T17:13:23.695 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.695 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.695 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.695 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.695 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.695 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.695 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.695 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.695 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.696 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.696 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.696 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.696 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.696 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.696 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.696 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.696 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.696 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.696 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.696 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.696 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.696 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.696 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.696 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.697 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.697 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.697 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.697 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.697 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.697 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.697 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.697 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.697 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.697 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.697 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.697 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.697 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.697 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.697 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.697 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.698 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.698 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.698 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.698 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.698 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.698 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.698 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.698 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.698 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.698 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.698 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.698 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.698 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.698 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.699 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.699 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.699 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.699 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.699 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.699 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.699 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.699 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.699 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.699 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.699 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.699 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.699 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.699 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.699 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.700 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.700 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.700 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.700 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.700 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.700 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.700 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.700 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.700 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.700 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.700 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.700 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.700 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.700 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.700 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.701 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.701 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.701 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.701 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.701 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.701 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.701 INFO:teuthology.orchestra.run.smithi006.stdout:_______ ERROR at teardown of test_object_lock_put_obj_lock_invalid_years _______ 2024-05-30T17:13:23.701 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.701 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.701 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.701 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.701 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.701 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.701 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.701 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.702 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.702 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.702 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.702 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.702 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.702 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.702 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.702 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.702 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.702 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.702 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.702 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.702 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.702 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.703 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.703 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.703 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.703 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.703 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.703 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.703 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.703 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.703 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.703 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.703 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.703 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.703 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.703 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.703 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.703 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.704 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.704 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.704 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.704 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.704 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.704 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.704 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.704 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.704 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.704 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.704 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.704 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.704 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.704 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.704 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.705 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.705 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.705 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.705 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.705 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.705 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.705 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.705 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.705 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.705 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.705 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.705 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.705 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.705 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.706 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.706 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.706 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.706 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.706 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.706 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.706 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.706 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.706 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.706 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.706 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.706 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.706 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.706 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.707 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.707 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.707 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.707 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.707 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.707 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.707 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.707 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.707 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.707 INFO:teuthology.orchestra.run.smithi006.stdout:_______ ERROR at teardown of test_object_lock_put_obj_lock_invalid_mode ________ 2024-05-30T17:13:23.707 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.707 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.707 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.707 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.707 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.707 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.708 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.708 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.708 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.708 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.708 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.708 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.708 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.708 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.708 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.708 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.708 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.708 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.708 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.709 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.709 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.709 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.709 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.709 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.709 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.709 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.709 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.709 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.709 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.709 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.709 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.709 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.709 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.710 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.710 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.710 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.710 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.710 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.710 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.710 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.710 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.710 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.710 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.710 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.710 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.710 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.711 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.711 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.711 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.711 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.711 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.711 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.711 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.711 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.711 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.711 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.711 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.711 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.711 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.711 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.712 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.712 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.712 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.712 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.712 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.712 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.712 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.712 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.712 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.712 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.712 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.712 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.712 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.712 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.712 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.712 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.713 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.713 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.713 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.713 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.713 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.713 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.713 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.713 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.713 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.713 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.713 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.713 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.713 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.713 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.713 INFO:teuthology.orchestra.run.smithi006.stdout:______ ERROR at teardown of test_object_lock_put_obj_lock_invalid_status _______ 2024-05-30T17:13:23.714 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.714 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.714 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.714 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.714 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.714 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.714 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.714 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.714 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.714 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.714 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.714 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.714 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.715 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.715 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.715 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.715 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.715 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.715 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.715 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.715 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.715 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.715 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.715 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.715 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.715 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.715 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.715 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.716 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.716 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.716 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.716 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.716 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.716 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.716 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.716 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.716 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.716 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.716 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.716 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.716 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.716 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.716 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.716 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.717 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.717 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.717 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.717 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.717 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.717 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.717 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.717 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.717 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.717 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.717 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.717 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.717 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.717 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.717 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.718 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.718 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.718 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.718 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.718 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.718 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.718 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.718 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.718 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.718 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.718 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.718 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.718 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.718 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.719 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.719 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.719 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.719 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.719 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.719 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.719 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.719 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.719 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.719 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.719 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.719 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.719 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.719 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.719 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.720 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.720 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.720 INFO:teuthology.orchestra.run.smithi006.stdout:___________ ERROR at teardown of test_object_lock_suspend_versioning ___________ 2024-05-30T17:13:23.720 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.720 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.720 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.720 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.720 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.720 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.720 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.720 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.720 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.720 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.720 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.720 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.721 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.721 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.721 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.721 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.721 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.721 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.721 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.721 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.721 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.721 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.721 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.721 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.721 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.721 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.721 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.722 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.722 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.722 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.722 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.722 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.722 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.722 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.722 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.722 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.722 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.722 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.722 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.722 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.723 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.723 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.723 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.723 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.723 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.723 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.723 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.723 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.723 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.723 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.723 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.723 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.723 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.723 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.724 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.724 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.724 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.724 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.724 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.724 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.724 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.724 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.724 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.724 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.724 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.724 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.724 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.724 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.724 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.725 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.725 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.725 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.725 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.725 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.725 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.725 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.725 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.725 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.725 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.725 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.725 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.725 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.725 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.725 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.726 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.726 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.726 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.726 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.726 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.726 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.726 INFO:teuthology.orchestra.run.smithi006.stdout:______________ ERROR at teardown of test_object_lock_get_obj_lock ______________ 2024-05-30T17:13:23.726 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.726 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.726 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.726 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.726 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.726 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.726 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.726 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.727 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.727 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.727 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.727 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.727 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.727 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.727 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.727 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.727 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.727 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.727 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.727 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.727 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.727 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.727 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.728 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.728 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.728 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.728 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.728 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.728 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.728 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.728 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.728 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.728 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.728 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.728 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.728 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.728 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.728 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.729 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.729 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.729 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.729 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.729 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.729 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.729 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.729 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.729 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.729 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.729 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.729 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.729 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.729 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.729 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.730 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.730 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.730 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.730 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.730 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.730 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.730 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.730 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.730 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.730 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.730 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.730 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.730 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.730 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.731 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.731 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.731 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.731 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.731 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.731 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.731 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.731 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.731 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.731 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.731 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.731 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.731 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.731 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.731 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.732 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.732 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.732 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.732 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.732 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.732 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.732 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.732 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.732 INFO:teuthology.orchestra.run.smithi006.stdout:______ ERROR at teardown of test_object_lock_get_obj_lock_invalid_bucket _______ 2024-05-30T17:13:23.732 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.732 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.732 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.732 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.732 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.732 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.732 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.733 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.733 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.733 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.733 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.733 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.733 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.733 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.733 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.733 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.733 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.733 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.733 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.733 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.733 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.733 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.734 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.734 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.734 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.734 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.734 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.734 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.734 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.734 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.734 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.734 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.734 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.734 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.734 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.734 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.735 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.735 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.735 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.735 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.735 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.735 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.735 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.735 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.735 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.735 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.735 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.735 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.735 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.735 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.735 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.736 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.736 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.736 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.736 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.736 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.736 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.736 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.736 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.736 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.736 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.736 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.736 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.736 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.736 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.736 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.737 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.737 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.737 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.737 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.737 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.737 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.737 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.737 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.737 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.737 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.737 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.737 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.737 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.737 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.737 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.738 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.738 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.738 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.738 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.738 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.738 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.738 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.738 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.738 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.738 INFO:teuthology.orchestra.run.smithi006.stdout:___________ ERROR at teardown of test_object_lock_put_obj_retention ____________ 2024-05-30T17:13:23.738 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.738 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.738 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.738 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.739 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.739 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.739 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.739 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.739 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.739 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.739 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.739 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.739 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.739 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.739 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.739 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.739 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.739 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.739 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.740 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.740 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.740 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.740 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.740 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.740 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.740 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.740 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.740 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.740 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.740 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.740 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.740 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.740 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.740 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.741 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.741 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.741 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.741 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.741 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.741 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.741 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.741 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.741 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.741 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.741 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.741 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.741 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.741 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.741 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.742 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.742 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.742 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.742 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.742 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.742 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.742 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.742 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.742 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.742 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.742 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.742 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.742 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.742 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.743 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.743 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.743 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.743 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.743 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.743 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.743 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.743 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.743 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.743 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.743 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.743 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.743 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.743 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.743 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.744 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.744 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.744 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.744 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.744 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.744 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.744 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.744 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.744 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.744 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.744 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.744 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.744 INFO:teuthology.orchestra.run.smithi006.stdout:____ ERROR at teardown of test_object_lock_put_obj_retention_invalid_bucket ____ 2024-05-30T17:13:23.744 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.744 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.745 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.745 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.745 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.745 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.745 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.745 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.745 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.745 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.745 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.745 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.745 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.745 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.745 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.745 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.745 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.745 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.746 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.746 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.746 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.746 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.746 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.746 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.746 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.746 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.746 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.746 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.746 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.746 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.746 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.746 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.747 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.747 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.747 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.747 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.747 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.747 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.747 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.747 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.747 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.747 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.747 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.747 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.747 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.747 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.747 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.748 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.748 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.748 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.748 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.748 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.748 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.748 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.748 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.748 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.748 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.748 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.748 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.748 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.748 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.748 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.749 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.749 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.749 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.749 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.749 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.749 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.749 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.749 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.749 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.749 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.749 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.749 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.749 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.749 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.749 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.749 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.750 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.750 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.750 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.750 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.750 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.750 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.750 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.750 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.750 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.750 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.750 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.750 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.750 INFO:teuthology.orchestra.run.smithi006.stdout:_____ ERROR at teardown of test_object_lock_put_obj_retention_invalid_mode _____ 2024-05-30T17:13:23.751 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.751 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.751 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.751 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.751 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.751 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.751 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.751 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.751 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.751 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.751 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.751 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.751 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.751 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.751 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.751 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.752 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.752 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.752 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.752 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.752 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.752 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.752 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.752 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.752 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.752 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.752 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.752 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.752 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.752 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.753 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.753 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.753 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.753 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.753 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.753 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.753 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.753 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.753 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.753 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.753 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.753 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.753 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.753 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.753 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.754 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.754 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.754 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.754 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.754 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.754 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.754 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.754 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.754 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.754 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.754 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.754 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.754 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.754 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.755 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.755 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.755 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.755 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.755 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.755 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.755 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.755 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.755 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.755 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.755 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.755 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.755 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.755 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.755 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.756 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.756 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.756 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.756 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.756 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.756 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.756 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.756 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.756 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.756 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.756 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.756 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.756 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.756 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.756 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.757 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.757 INFO:teuthology.orchestra.run.smithi006.stdout:___________ ERROR at teardown of test_object_lock_get_obj_retention ____________ 2024-05-30T17:13:23.757 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.757 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.757 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.757 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.757 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.757 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.757 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.757 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.757 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.757 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.757 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.757 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.757 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.758 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.758 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.758 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.758 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.758 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.758 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.758 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.758 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.758 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.758 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.758 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.758 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.758 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.759 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.759 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.759 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.759 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.759 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.759 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.759 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.759 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.759 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.759 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.759 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.759 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.759 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.759 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.759 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.760 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.760 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.760 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.760 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.760 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.760 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.760 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.760 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.760 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.760 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.760 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.760 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.760 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.760 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.760 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.760 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.761 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.761 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.761 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.761 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.761 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.761 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.761 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.761 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.761 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.761 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.761 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.761 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.761 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.761 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.762 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.762 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.762 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.762 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.762 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.762 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.762 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.762 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.762 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.762 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.762 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.762 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.762 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.762 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.763 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.763 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.763 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.763 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.763 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.763 INFO:teuthology.orchestra.run.smithi006.stdout:_______ ERROR at teardown of test_object_lock_get_obj_retention_iso8601 ________ 2024-05-30T17:13:23.763 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.763 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.763 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.763 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.763 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.763 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.763 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.763 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.763 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.763 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.764 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.764 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.764 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.764 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.764 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.764 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.764 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.764 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.764 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.764 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.764 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.764 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.764 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.764 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.764 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.765 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.765 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.765 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.765 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.765 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.765 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.765 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.765 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.765 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.765 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.765 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.765 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.765 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.765 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.765 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.766 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.766 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.766 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.766 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.766 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.766 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.766 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.766 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.766 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.766 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.766 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.766 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.766 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.766 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.767 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.767 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.767 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.767 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.767 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.767 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.767 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.767 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.767 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.767 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.767 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.767 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.767 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.767 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.767 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.768 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.768 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.768 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.768 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.768 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.768 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.768 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.768 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.768 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.768 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.768 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.768 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.768 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.768 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.768 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.769 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.769 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.769 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.769 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.769 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.769 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.769 INFO:teuthology.orchestra.run.smithi006.stdout:____ ERROR at teardown of test_object_lock_get_obj_retention_invalid_bucket ____ 2024-05-30T17:13:23.769 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.769 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.769 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.769 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.769 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.769 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.769 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.769 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.770 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.770 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.770 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.770 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.770 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.770 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.770 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.770 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.770 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.770 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.770 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.770 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.770 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.770 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.770 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.771 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.771 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.771 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.771 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.771 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.771 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.771 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.771 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.771 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.771 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.771 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.771 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.771 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.771 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.771 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.772 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.772 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.772 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.772 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.772 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.772 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.772 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.772 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.772 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.772 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.772 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.772 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.772 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.772 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.772 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.773 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.773 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.773 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.773 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.773 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.773 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.773 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.773 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.773 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.773 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.773 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.773 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.773 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.773 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.773 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.774 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.774 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.774 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.774 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.774 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.774 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.774 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.774 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.774 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.774 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.774 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.774 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.774 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.774 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.774 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.775 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.775 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.775 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.775 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.775 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.775 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.775 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.775 INFO:teuthology.orchestra.run.smithi006.stdout:______ ERROR at teardown of test_object_lock_put_obj_retention_versionid _______ 2024-05-30T17:13:23.775 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.775 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.775 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.775 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.775 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.775 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.776 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.776 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.776 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.776 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.776 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.776 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.776 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.776 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.776 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.776 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.776 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.776 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.776 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.776 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.776 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.777 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.777 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.777 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.777 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.777 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.777 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.777 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.777 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.777 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.777 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.777 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.777 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.777 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.777 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.777 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.778 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.778 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.778 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.778 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.778 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.778 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.778 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.778 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.778 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.778 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.778 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.778 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.778 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.778 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.778 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.779 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.779 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.779 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.779 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.779 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.779 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.779 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.779 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.779 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.779 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.779 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.779 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.779 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.779 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.779 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.780 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.780 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.780 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.780 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.780 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.780 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.780 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.780 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.780 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.780 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.780 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.780 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.780 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.780 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.780 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.781 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.781 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.781 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.781 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.781 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.781 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.781 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.781 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.781 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.781 INFO:teuthology.orchestra.run.smithi006.stdout:_ ERROR at teardown of test_object_lock_put_obj_retention_override_default_retention _ 2024-05-30T17:13:23.781 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.781 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.781 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.781 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.781 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.782 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.782 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.782 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.782 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.782 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.782 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.782 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.782 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.782 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.782 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.782 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.782 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.782 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.782 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.782 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.783 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.783 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.783 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.783 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.783 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.783 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.783 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.783 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.783 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.783 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.783 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.783 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.783 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.783 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.783 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.784 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.784 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.784 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.784 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.784 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.784 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.784 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.784 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.784 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.784 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.784 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.784 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.784 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.784 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.784 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.785 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.785 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.785 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.785 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.785 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.785 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.785 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.785 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.785 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.785 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.785 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.785 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.785 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.785 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.785 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.785 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.786 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.786 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.786 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.786 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.786 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.786 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.786 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.786 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.786 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.786 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.786 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.786 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.786 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.786 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.786 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.787 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.787 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.787 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.787 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.787 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.787 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.787 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.787 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.787 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.787 INFO:teuthology.orchestra.run.smithi006.stdout:___ ERROR at teardown of test_object_lock_put_obj_retention_increase_period ____ 2024-05-30T17:13:23.787 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.787 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.787 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.787 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.787 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.788 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.788 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.788 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.788 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.788 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.788 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.788 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.788 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.788 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.788 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.788 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.788 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.788 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.788 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.788 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.789 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.789 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.789 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.789 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.789 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.789 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.789 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.789 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.789 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.789 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.789 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.789 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.789 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.789 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.789 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.790 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.790 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.790 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.790 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.790 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.790 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.790 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.790 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.790 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.790 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.790 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.790 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.790 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.790 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.791 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.791 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.791 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.791 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.791 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.791 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.791 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.791 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.791 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.791 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.791 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.791 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.791 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.791 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.791 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.792 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.792 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.792 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.792 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.792 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.792 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.792 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.792 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.792 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.792 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.792 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.792 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.792 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.792 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.792 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.793 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.793 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.793 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.793 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.793 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.793 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.793 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.793 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.793 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.793 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.793 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.793 INFO:teuthology.orchestra.run.smithi006.stdout:____ ERROR at teardown of test_object_lock_put_obj_retention_shorten_period ____ 2024-05-30T17:13:23.793 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.793 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.794 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.794 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.794 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.794 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.794 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.794 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.794 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.794 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.794 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.794 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.794 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.794 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.794 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.795 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.795 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.795 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.795 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.795 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.795 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.795 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.795 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.795 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.795 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.795 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.795 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.795 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.795 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.795 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.795 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.796 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.796 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.796 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.796 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.796 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.796 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.796 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.796 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.796 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.796 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.796 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.796 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.796 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.796 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.797 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.797 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.797 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.797 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.797 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.797 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.797 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.797 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.797 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.797 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.797 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.797 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.797 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.797 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.797 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.798 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.798 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.798 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.798 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.798 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.798 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.798 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.798 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.798 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.798 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.798 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.798 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.798 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.798 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.798 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.799 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.799 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.799 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.799 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.799 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.799 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.799 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.799 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.799 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.799 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.799 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.799 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.799 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.799 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.799 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.799 INFO:teuthology.orchestra.run.smithi006.stdout:_ ERROR at teardown of test_object_lock_put_obj_retention_shorten_period_bypass _ 2024-05-30T17:13:23.800 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.800 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.800 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.800 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.800 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.800 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.800 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.800 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.800 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.800 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.800 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.800 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.800 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.800 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.800 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.801 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.801 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.801 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.801 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.801 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.801 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.801 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.801 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.801 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.801 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.801 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.801 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.801 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.801 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.801 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.802 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.802 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.802 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.802 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.802 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.802 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.802 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.802 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.802 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.802 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.802 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.802 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.802 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.802 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.802 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.803 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.803 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.803 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.803 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.803 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.803 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.803 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.803 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.803 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.803 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.803 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.803 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.803 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.803 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.803 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.804 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.804 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.804 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.804 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.804 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.804 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.804 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.804 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.804 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.804 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.804 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.804 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.804 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.804 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.804 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.805 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.805 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.805 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.805 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.805 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.805 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.805 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.805 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.805 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.805 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.805 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.805 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.805 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.805 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.805 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.805 INFO:teuthology.orchestra.run.smithi006.stdout:______ ERROR at teardown of test_object_lock_delete_object_with_retention ______ 2024-05-30T17:13:23.806 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.806 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.806 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.806 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.806 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.806 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.806 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.806 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.806 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.806 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.806 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.806 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.806 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.806 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.806 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.807 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.807 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.807 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.807 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.807 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.807 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.807 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.807 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.807 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.807 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.807 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.807 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.807 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.807 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.807 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.808 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.808 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.808 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.808 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.808 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.808 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.808 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.808 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.808 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.808 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.808 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.808 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.808 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.808 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.808 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.809 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.809 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.809 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.809 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.809 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.809 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.809 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.809 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.809 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.809 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.809 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.809 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.809 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.809 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.809 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.809 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.810 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.810 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.810 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.810 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.810 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.810 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.810 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.810 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.810 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.810 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.810 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.810 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.810 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.810 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.810 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.811 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.811 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.811 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.811 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.811 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.811 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.811 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.811 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.811 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.811 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.811 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.811 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.811 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.811 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.811 INFO:teuthology.orchestra.run.smithi006.stdout:_ ERROR at teardown of test_object_lock_delete_multipart_object_with_retention _ 2024-05-30T17:13:23.811 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.812 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.812 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.812 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.812 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.812 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.812 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.812 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.812 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.812 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.812 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.812 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.812 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.812 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.812 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.812 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.813 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.813 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.813 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.813 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.813 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.813 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.813 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.813 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.813 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.813 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.813 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.813 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.813 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.813 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.813 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.814 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.814 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.814 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.814 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.814 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.814 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.814 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.814 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.814 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.814 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.814 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.814 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.814 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.814 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.815 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.815 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.815 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.815 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.815 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.815 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.815 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.815 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.815 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.815 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.815 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.815 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.815 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.815 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.815 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.815 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.816 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.816 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.816 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.816 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.816 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.816 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.816 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.816 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.816 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.816 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.816 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.816 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.816 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.816 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.816 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.817 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.817 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.817 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.817 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.817 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.817 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.817 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.817 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.817 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.817 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.817 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.817 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.817 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.817 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.817 INFO:teuthology.orchestra.run.smithi006.stdout:_ ERROR at teardown of test_object_lock_delete_object_with_retention_and_marker _ 2024-05-30T17:13:23.818 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.818 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.818 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.818 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.818 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.818 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.818 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.818 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.818 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.818 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.818 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.818 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.818 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.819 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.819 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.819 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.819 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.819 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.819 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.819 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.819 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.819 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.819 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.819 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.819 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.819 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.819 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.819 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.820 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.820 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.820 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.820 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.820 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.820 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.820 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.820 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.820 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.820 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.820 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.820 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.820 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.820 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.821 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.821 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.821 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.821 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.821 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.821 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.821 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.821 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.821 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.821 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.821 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.821 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.821 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.821 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.821 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.822 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.822 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.822 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.822 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.822 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.822 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.822 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.822 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.822 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.822 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.822 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.822 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.822 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.823 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.823 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.823 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.823 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.823 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.823 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.823 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.823 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.823 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.823 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.823 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.823 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.823 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.824 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.824 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.824 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.824 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.824 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.824 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.824 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.824 INFO:teuthology.orchestra.run.smithi006.stdout:___ ERROR at teardown of test_object_lock_multi_delete_object_with_retention ___ 2024-05-30T17:13:23.824 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.824 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.824 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.824 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.824 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.824 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.824 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.825 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.825 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.825 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.825 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.825 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.825 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.825 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.825 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.825 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.825 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.825 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.825 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.825 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.825 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.825 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.826 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.826 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.826 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.826 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.826 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.826 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.826 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.826 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.826 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.826 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.826 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.826 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.826 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.826 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.829 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.829 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.829 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.829 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.829 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.829 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.829 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.829 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.830 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.830 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.830 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.830 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.830 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.830 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.830 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.830 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.830 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.830 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.830 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.830 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.830 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.830 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.831 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.831 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.831 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.831 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.831 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.831 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.831 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.831 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.831 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.831 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.831 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.831 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.831 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.832 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.832 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.832 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.832 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.832 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.832 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.832 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.832 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.832 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.832 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.832 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.832 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.833 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.833 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.833 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.833 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.833 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.833 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.833 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.833 INFO:teuthology.orchestra.run.smithi006.stdout:_____________ ERROR at teardown of test_object_lock_put_legal_hold _____________ 2024-05-30T17:13:23.833 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.833 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.833 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.833 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.833 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.833 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.834 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.834 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.834 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.834 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.834 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.834 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.834 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.834 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.834 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.834 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.834 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.834 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.834 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.835 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.835 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.835 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.835 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.835 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.835 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.835 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.835 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.835 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.835 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.835 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.835 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.835 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.835 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.836 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.836 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.836 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.836 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.836 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.836 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.836 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.836 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.836 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.836 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.836 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.836 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.836 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.837 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.837 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.837 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.837 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.837 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.837 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.837 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.837 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.837 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.837 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.837 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.837 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.837 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.837 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.838 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.838 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.838 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.838 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.838 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.838 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.838 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.838 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.838 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.838 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.838 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.838 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.838 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.838 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.839 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.839 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.839 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.839 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.839 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.839 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.839 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.839 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.839 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.839 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.839 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.839 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.839 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.839 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.839 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.840 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.840 INFO:teuthology.orchestra.run.smithi006.stdout:_____ ERROR at teardown of test_object_lock_put_legal_hold_invalid_bucket ______ 2024-05-30T17:13:23.840 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.840 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.840 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.840 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.840 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.840 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.840 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.840 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.840 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.840 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.840 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.840 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.840 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.841 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.841 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.841 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.841 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.841 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.841 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.841 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.841 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.841 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.841 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.841 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.841 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.841 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.841 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.841 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.842 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.842 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.842 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.842 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.842 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.842 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.842 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.842 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.842 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.842 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.842 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.842 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.842 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.842 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.843 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.843 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.843 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.843 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.843 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.843 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.843 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.843 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.843 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.843 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.843 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.843 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.843 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.844 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.844 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.844 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.844 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.844 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.844 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.844 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.844 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.844 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.844 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.844 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.844 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.844 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.844 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.844 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.845 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.845 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.845 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.845 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.845 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.845 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.845 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.845 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.845 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.845 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.845 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.845 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.845 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.845 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.845 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.846 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.846 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.846 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.846 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.846 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.846 INFO:teuthology.orchestra.run.smithi006.stdout:_____ ERROR at teardown of test_object_lock_put_legal_hold_invalid_status ______ 2024-05-30T17:13:23.846 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.846 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.846 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.846 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.846 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.846 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.846 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.846 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.847 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.847 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.847 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.847 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.847 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.847 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.847 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.847 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.847 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.847 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.847 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.847 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.847 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.847 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.848 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.848 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.848 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.848 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.848 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.848 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.848 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.848 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.848 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.848 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.848 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.848 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.848 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.849 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.849 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.849 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.849 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.849 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.849 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.849 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.849 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.849 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.849 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.849 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.849 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.849 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.849 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.850 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.850 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.850 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.850 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.850 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.850 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.850 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.850 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.850 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.850 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.850 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.850 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.850 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.850 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.850 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.851 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.851 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.851 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.851 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.851 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.851 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.851 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.851 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.851 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.851 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.851 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.851 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.851 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.851 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.851 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.852 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.852 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.852 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.852 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.852 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.852 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.852 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.852 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.852 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.852 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.852 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.852 INFO:teuthology.orchestra.run.smithi006.stdout:_____________ ERROR at teardown of test_object_lock_get_legal_hold _____________ 2024-05-30T17:13:23.852 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.852 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.853 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.853 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.853 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.853 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.853 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.853 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.853 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.853 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.853 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.853 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.853 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.853 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.853 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.853 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.853 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.854 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.854 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.854 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.854 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.854 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.854 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.854 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.854 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.854 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.854 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.854 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.854 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.854 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.854 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.854 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.855 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.855 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.855 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.855 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.855 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.855 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.855 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.855 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.855 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.855 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.855 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.855 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.855 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.855 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.855 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.856 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.856 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.856 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.856 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.856 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.856 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.856 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.856 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.856 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.856 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.856 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.856 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.856 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.856 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.856 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.857 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.857 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.857 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.857 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.857 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.857 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.857 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.857 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.857 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.857 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.857 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.857 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.857 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.857 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.858 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.858 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.858 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.858 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.858 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.858 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.858 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.858 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.858 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.858 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.858 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.858 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.859 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.859 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.859 INFO:teuthology.orchestra.run.smithi006.stdout:_____ ERROR at teardown of test_object_lock_get_legal_hold_invalid_bucket ______ 2024-05-30T17:13:23.859 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.859 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.859 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.859 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.859 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.859 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.859 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.859 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.859 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.859 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.860 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.860 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.860 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.860 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.860 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.860 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.860 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.860 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.860 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.860 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.860 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.860 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.860 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.861 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.861 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.861 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.861 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.861 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.861 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.861 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.861 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.861 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.861 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.861 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.861 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.861 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.862 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.862 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.862 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.862 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.862 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.862 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.862 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.862 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.862 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.862 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.862 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.863 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.863 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.863 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.863 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.863 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.863 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.863 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.863 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.863 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.863 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.863 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.863 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.864 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.864 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.864 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.864 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.864 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.864 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.864 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.864 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.864 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.864 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.864 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.864 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.864 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.865 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.865 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.865 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.865 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.865 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.865 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.865 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.865 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.865 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.865 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.865 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.865 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.865 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.866 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.866 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.866 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.866 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.866 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.866 INFO:teuthology.orchestra.run.smithi006.stdout:____ ERROR at teardown of test_object_lock_delete_object_with_legal_hold_on ____ 2024-05-30T17:13:23.866 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.866 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.866 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.866 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.866 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.866 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.866 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.866 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.867 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.867 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.867 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.867 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.867 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.867 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.867 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.867 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.867 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.867 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.867 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.867 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.867 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.867 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.867 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.868 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.868 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.868 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.868 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.868 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.868 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.868 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.868 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.868 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.868 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.868 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.868 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.868 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.869 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.869 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.869 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.869 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.869 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.869 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.869 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.869 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.869 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.869 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.869 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.869 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.869 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.869 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.869 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.870 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.870 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.870 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.870 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.870 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.870 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.870 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.870 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.870 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.870 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.870 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.870 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.870 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.870 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.871 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.871 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.871 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.871 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.871 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.871 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.871 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.871 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.871 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.871 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.871 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.871 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.871 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.872 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.872 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.872 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.872 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.872 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.872 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.872 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.872 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.872 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.872 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.872 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.872 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.872 INFO:teuthology.orchestra.run.smithi006.stdout:_ ERROR at teardown of test_object_lock_delete_multipart_object_with_legal_hold_on _ 2024-05-30T17:13:23.872 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.873 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.873 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.873 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.873 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.873 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.873 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.873 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.873 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.873 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.873 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.873 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.873 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.874 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.874 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.874 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.874 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.874 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.874 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.874 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.874 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.874 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.874 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.874 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.874 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.875 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.875 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.875 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.875 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.875 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.875 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.875 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.875 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.875 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.875 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.875 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.875 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.875 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.875 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.876 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.876 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.876 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.876 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.876 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.876 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.876 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.876 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.876 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.876 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.876 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.876 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.877 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.877 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.877 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.877 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.877 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.877 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.877 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.877 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.877 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.877 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.877 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.877 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.877 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.877 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.877 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.878 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.878 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.878 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.878 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.878 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.878 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.878 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.878 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.878 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.878 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.878 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.878 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.878 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.879 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.879 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.879 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.879 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.879 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.879 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.879 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.879 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.879 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.879 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.879 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.879 INFO:teuthology.orchestra.run.smithi006.stdout:___ ERROR at teardown of test_object_lock_delete_object_with_legal_hold_off ____ 2024-05-30T17:13:23.880 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.880 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.880 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.880 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.880 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.880 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.880 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.880 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.880 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.880 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.880 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.880 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.880 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.881 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.881 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.881 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.881 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.881 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.881 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.881 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.881 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.881 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.881 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.881 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.881 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.882 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.882 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.882 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.882 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.882 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.882 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.882 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.882 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.882 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.882 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.882 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.882 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.883 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.883 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.883 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.883 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.883 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.883 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.883 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.883 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.883 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.883 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.883 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.883 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.883 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.884 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.884 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.884 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.884 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.884 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.884 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.884 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.884 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.884 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.884 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.884 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.884 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.884 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.885 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.885 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.885 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.885 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.885 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.885 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.885 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.885 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.885 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.885 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.885 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.885 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.885 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.886 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.886 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.886 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.886 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.886 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.886 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.886 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.886 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.886 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.886 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.886 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.886 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.886 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.887 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.887 INFO:teuthology.orchestra.run.smithi006.stdout:____________ ERROR at teardown of test_object_lock_get_obj_metadata ____________ 2024-05-30T17:13:23.887 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.887 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.887 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.887 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.887 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.887 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.887 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.887 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.887 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.887 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.887 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.887 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.888 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.888 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.888 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.888 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.888 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.888 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.888 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.888 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.888 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.888 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.888 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.888 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.888 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.888 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.889 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.889 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.889 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.889 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.889 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.889 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.889 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.889 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.889 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.889 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.889 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.889 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.889 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.889 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.889 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.890 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.890 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.890 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.890 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.890 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.890 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.890 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.890 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.890 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.890 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.890 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.890 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.890 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.890 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.890 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.891 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.891 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.891 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.891 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.891 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.891 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.891 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.891 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.891 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.891 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.891 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.891 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.891 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.891 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.892 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.892 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.892 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.892 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.892 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.892 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.892 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.892 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.892 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.892 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.892 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.892 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.892 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.892 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.893 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.893 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.893 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.893 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.893 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.893 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.893 INFO:teuthology.orchestra.run.smithi006.stdout:_____________ ERROR at teardown of test_object_lock_uploading_obj ______________ 2024-05-30T17:13:23.893 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.893 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.893 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.893 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.893 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.893 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.893 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.893 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.894 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.894 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.894 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.894 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.894 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.894 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.894 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.894 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.894 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.894 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.894 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.894 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.895 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.895 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.895 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.895 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.895 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.895 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.895 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.895 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.895 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.895 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.895 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.895 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.895 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.896 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.896 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.896 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.896 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.896 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.896 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.896 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.896 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.896 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.896 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.896 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.896 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.896 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.897 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.897 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.897 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.897 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.897 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.897 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.897 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.897 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.897 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.897 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.897 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.897 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.897 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.897 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.898 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.898 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.898 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.898 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.898 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.898 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.898 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.898 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.898 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.898 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.898 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.898 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.898 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.899 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.899 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.899 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.899 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.899 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.899 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.899 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.899 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.899 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.899 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.899 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.899 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.899 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.899 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.900 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.900 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.900 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.900 INFO:teuthology.orchestra.run.smithi006.stdout:_ ERROR at teardown of test_object_lock_changing_mode_from_governance_with_bypass _ 2024-05-30T17:13:23.900 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.900 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.900 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.900 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.900 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.900 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.900 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.900 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.900 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.901 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.901 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.901 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.901 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.901 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.901 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.901 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.901 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.901 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.901 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.901 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.901 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.901 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.902 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.902 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.902 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.902 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.902 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.902 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.902 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.902 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.902 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.902 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.902 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.902 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.903 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.903 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.903 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.903 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.903 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.903 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.903 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.903 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.903 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.903 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.903 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.903 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.903 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.903 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.904 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.904 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.904 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.904 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.904 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.904 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.904 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.904 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.904 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.904 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.904 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.904 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.904 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.905 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.905 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.905 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.905 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.905 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.905 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.905 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.905 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.905 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.905 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.905 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.905 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.905 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.906 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.906 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.906 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.906 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.906 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.906 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.906 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.906 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.906 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.906 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.906 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.906 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.906 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.906 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.907 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.907 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.907 INFO:teuthology.orchestra.run.smithi006.stdout:--------------------------- Captured stdout teardown --------------------------- 2024-05-30T17:13:23.907 INFO:teuthology.orchestra.run.smithi006.stdout:nuke_bucket test-zrhryoy189ajjeo9txwotj0u-550 waiting 9.253502 seconds for object locks to expire 2024-05-30T17:13:23.907 INFO:teuthology.orchestra.run.smithi006.stdout:_ ERROR at teardown of test_object_lock_changing_mode_from_governance_without_bypass _ 2024-05-30T17:13:23.907 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.907 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.907 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.907 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.907 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.907 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.907 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.907 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.907 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.908 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.908 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.908 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.908 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.908 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.908 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.908 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.908 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.908 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.908 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.908 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.908 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.908 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.908 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.909 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.909 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.909 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.909 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.909 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.909 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.909 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.909 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.909 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.909 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.909 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.909 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.909 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.909 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.909 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.909 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.910 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.910 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.910 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.910 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.910 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.910 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.910 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.910 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.910 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.910 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.910 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.910 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.910 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.910 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.911 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.911 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.911 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.911 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.911 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.911 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.911 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.911 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.911 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.911 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.911 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.911 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.911 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.911 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.911 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.912 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.912 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.912 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.912 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.912 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.912 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.912 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.912 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.912 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.912 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.912 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.912 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.912 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.912 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.912 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.913 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.913 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.913 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.913 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.913 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.913 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.913 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.913 INFO:teuthology.orchestra.run.smithi006.stdout:_____ ERROR at teardown of test_object_lock_changing_mode_from_compliance ______ 2024-05-30T17:13:23.913 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.913 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.913 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.913 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.913 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.913 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.913 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.914 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.914 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.914 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.914 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.914 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.914 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.914 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.914 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.914 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.914 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.914 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.914 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.914 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.914 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.915 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.915 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.915 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.915 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.915 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.915 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.915 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.915 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.915 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.915 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.915 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.915 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.915 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.915 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.916 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.916 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.916 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.916 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.916 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.916 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.916 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.916 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.916 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.916 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.916 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.916 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.916 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.916 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.916 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.917 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.917 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.917 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.917 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.917 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.917 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.917 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.917 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.917 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.917 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.917 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.917 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.917 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.917 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.918 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.918 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.918 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.918 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.918 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.918 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.918 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.918 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.918 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.918 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.918 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.918 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.918 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.918 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.919 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.919 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.919 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.919 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.919 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.919 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.919 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.919 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.919 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.919 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.919 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.919 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.919 INFO:teuthology.orchestra.run.smithi006.stdout:--------------------------- Captured stdout teardown --------------------------- 2024-05-30T17:13:23.919 INFO:teuthology.orchestra.run.smithi006.stdout:nuke_bucket test-zrhryoy189ajjeo9txwotj0u-552 waiting 8.965788 seconds for object locks to expire 2024-05-30T17:13:23.920 INFO:teuthology.orchestra.run.smithi006.stdout:______________ ERROR at teardown of test_copy_object_ifmatch_good ______________ 2024-05-30T17:13:23.920 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.920 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.920 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.920 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.920 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.920 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.920 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.920 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.920 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.920 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.920 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.920 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.920 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.921 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.921 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.921 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.921 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.921 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.921 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.921 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.921 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.921 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.921 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.921 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.921 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.921 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.921 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.921 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.922 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.922 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.922 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.922 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.922 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.922 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.922 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.922 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.922 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.922 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.922 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.922 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.922 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.922 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.923 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.923 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.923 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.923 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.923 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.923 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.923 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.923 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.923 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.923 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.923 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.923 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.923 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.924 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.924 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.924 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.924 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.924 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.924 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.924 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.924 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.924 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.924 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.924 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.924 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.924 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.924 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.925 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.925 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.925 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.925 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.925 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.925 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.925 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.925 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.925 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.925 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.925 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.925 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.925 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.925 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.925 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.926 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.926 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.926 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.926 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.926 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.926 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.926 INFO:teuthology.orchestra.run.smithi006.stdout:___________ ERROR at teardown of test_copy_object_ifnonematch_failed ___________ 2024-05-30T17:13:23.926 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.926 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.926 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.926 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.926 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.926 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.926 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.927 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.927 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.927 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.927 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.927 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.927 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.927 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.927 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.927 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.927 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.927 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.927 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.927 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.927 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.927 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.928 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.928 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.928 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.928 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.928 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.928 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.928 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.928 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.928 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.928 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.928 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.928 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.928 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.928 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.929 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.929 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.929 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.929 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.929 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.929 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.929 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.929 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.929 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.929 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.929 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.929 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.929 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.929 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.930 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.930 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.930 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.930 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.930 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.930 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.930 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.930 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.930 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.930 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.930 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.931 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.931 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.931 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.931 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.931 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.931 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.931 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.931 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.931 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.931 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.931 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.931 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.931 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.932 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.932 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.932 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.932 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.932 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.932 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.932 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.932 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.932 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.932 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.932 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.932 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.932 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.933 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.933 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.933 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.933 INFO:teuthology.orchestra.run.smithi006.stdout:______________ ERROR at teardown of test_get_bucket_policy_status ______________ 2024-05-30T17:13:23.933 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.933 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.933 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.933 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.933 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.933 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.933 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.933 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.933 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.933 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.934 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.934 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.934 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.934 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.934 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.934 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.934 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.934 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.934 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.934 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.934 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.934 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.934 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.935 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.935 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.935 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.935 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.935 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.935 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.935 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.935 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.935 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.935 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.935 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.935 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.935 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.935 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.935 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.936 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.936 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.936 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.936 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.936 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.936 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.936 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.936 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.936 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.936 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.936 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.936 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.936 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.936 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.937 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.937 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.937 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.937 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.937 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.937 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.937 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.937 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.937 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.937 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.937 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.937 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.937 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.937 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.937 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.938 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.938 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.938 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.938 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.938 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.938 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.938 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.938 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.938 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.938 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.938 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.938 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.938 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.938 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.938 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.939 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.939 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.939 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.939 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.939 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.939 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.939 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.939 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.939 INFO:teuthology.orchestra.run.smithi006.stdout:________ ERROR at teardown of test_get_public_acl_bucket_policy_status _________ 2024-05-30T17:13:23.939 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.939 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.939 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.939 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.939 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.940 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.940 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.940 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.940 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.940 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.940 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.940 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.940 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.940 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.940 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.940 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.940 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.940 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.940 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.940 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.941 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.941 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.941 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.941 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.941 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.941 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.941 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.941 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.941 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.941 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.941 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.941 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.941 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.941 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.941 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.942 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.942 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.942 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.942 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.942 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.942 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.942 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.942 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.942 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.942 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.942 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.942 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.942 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.942 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.942 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.943 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.943 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.943 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.943 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.943 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.943 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.943 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.943 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.943 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.943 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.943 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.943 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.943 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.943 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.943 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.944 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.944 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.944 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.944 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.944 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.944 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.944 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.944 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.944 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.944 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.944 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.944 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.944 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.944 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.944 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.945 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.945 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.945 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.945 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.945 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.945 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.945 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.945 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.945 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.945 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.945 INFO:teuthology.orchestra.run.smithi006.stdout:______ ERROR at teardown of test_get_authpublic_acl_bucket_policy_status _______ 2024-05-30T17:13:23.945 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.945 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.945 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.946 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.946 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.946 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.946 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.946 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.946 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.946 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.946 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.946 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.946 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.946 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.946 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.946 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.946 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.946 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.947 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.947 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.947 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.947 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.947 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.947 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.947 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.947 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.947 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.947 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.947 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.947 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.947 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.947 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.947 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.948 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.948 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.948 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.948 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.948 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.948 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.948 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.948 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.948 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.948 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.948 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.948 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.948 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.948 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.949 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.949 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.949 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.949 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.949 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.949 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.949 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.949 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.949 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.949 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.949 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.949 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.949 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.949 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.949 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.950 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.950 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.950 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.950 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.950 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.950 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.950 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.950 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.950 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.950 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.950 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.950 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.950 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.950 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.950 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.951 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.951 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.951 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.951 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.951 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.951 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.951 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.951 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.951 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.951 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.951 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.951 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.951 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.951 INFO:teuthology.orchestra.run.smithi006.stdout:_____ ERROR at teardown of test_get_publicpolicy_acl_bucket_policy_status ______ 2024-05-30T17:13:23.951 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.952 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.952 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.952 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.952 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.952 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.952 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.952 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.952 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.952 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.952 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.952 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.952 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.952 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.952 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.953 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.953 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.953 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.953 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.953 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.953 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.953 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.953 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.953 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.953 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.953 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.953 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.953 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.953 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.953 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.954 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.954 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.954 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.954 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.954 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.954 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.954 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.954 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.954 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.954 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.954 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.954 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.954 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.954 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.954 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.955 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.955 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.955 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.955 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.955 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.955 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.955 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.955 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.955 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.955 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.955 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.955 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.955 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.955 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.955 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.956 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.956 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.956 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.956 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.956 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.956 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.956 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.956 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.956 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.956 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.956 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.956 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.956 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.956 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.956 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.957 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.957 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.957 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.957 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.957 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.957 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.957 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.957 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.957 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.957 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.957 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.957 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.957 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.957 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.957 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.958 INFO:teuthology.orchestra.run.smithi006.stdout:____ ERROR at teardown of test_get_nonpublicpolicy_acl_bucket_policy_status ____ 2024-05-30T17:13:23.958 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.958 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.958 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.958 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.958 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.958 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.958 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.958 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.958 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.958 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.958 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.958 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.959 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.959 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.959 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.959 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.959 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.959 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.959 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.959 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.959 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.959 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.959 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.959 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.959 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.959 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.959 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.960 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.960 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.960 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.960 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.960 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.960 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.960 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.960 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.960 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.960 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.960 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.960 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.960 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.960 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.960 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.961 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.961 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.961 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.961 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.961 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.961 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.961 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.961 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.961 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.961 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.961 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.961 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.961 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.961 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.961 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.962 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.962 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.962 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.962 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.962 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.962 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.962 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.962 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.962 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.962 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.962 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.962 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.962 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.962 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.962 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.963 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.963 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.963 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.963 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.963 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.963 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.963 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.963 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.963 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.963 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.963 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.963 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.963 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.963 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.963 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.964 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.964 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.964 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.964 INFO:teuthology.orchestra.run.smithi006.stdout:___ ERROR at teardown of test_get_nonpublicpolicy_deny_bucket_policy_status ____ 2024-05-30T17:13:23.964 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.964 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.964 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.964 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.964 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.964 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.964 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.964 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.964 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.964 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.964 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.964 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.965 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.965 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.965 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.965 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.965 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.965 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.965 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.965 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.965 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.965 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.965 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.965 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.965 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.965 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.966 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.966 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.966 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.966 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.966 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.966 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.966 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.966 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.966 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.966 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.966 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.966 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.966 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.966 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.967 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.967 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.967 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.967 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.967 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.967 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.967 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.967 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.967 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.967 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.967 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.967 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.967 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.968 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.968 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.968 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.968 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.968 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.968 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.968 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.968 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.968 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.968 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.968 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.968 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.968 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.968 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.969 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.969 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.969 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.969 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.969 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.969 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.969 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.969 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.969 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.969 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.969 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.969 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.970 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.970 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.970 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.970 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.970 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.970 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.970 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.970 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.970 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.970 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.970 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.970 INFO:teuthology.orchestra.run.smithi006.stdout:_____________ ERROR at teardown of test_get_undefined_public_block _____________ 2024-05-30T17:13:23.970 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.971 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.971 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.971 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.971 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.971 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.971 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.971 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.971 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.971 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.971 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.971 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.971 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.971 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.971 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.972 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.972 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.972 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.972 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.972 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.972 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.972 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.972 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.972 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.972 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.972 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.972 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.972 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.972 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.972 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.973 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.973 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.973 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.973 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.973 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.973 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.973 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.973 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.973 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.973 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.973 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.973 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.974 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.974 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.974 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.974 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.974 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.974 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.974 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.974 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.974 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.974 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.974 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.974 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.974 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.975 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.975 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.975 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.975 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.975 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.975 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.975 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.975 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.975 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.975 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.975 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.975 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.975 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.975 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.976 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.976 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.976 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.976 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.976 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.976 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.976 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.976 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.976 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.976 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.976 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.976 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.976 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.977 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.977 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.977 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.977 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.977 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.977 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.977 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.977 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.977 INFO:teuthology.orchestra.run.smithi006.stdout:________ ERROR at teardown of test_get_public_block_deny_bucket_policy _________ 2024-05-30T17:13:23.977 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.977 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.977 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.977 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.978 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.978 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.978 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.978 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.978 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.978 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.978 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.978 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.978 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.978 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.978 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.978 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.978 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.978 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.979 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.979 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.979 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.979 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.979 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.979 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.979 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.979 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.979 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.979 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.979 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.979 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.979 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.979 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.980 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.980 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.980 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.980 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.980 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.980 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.980 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.980 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.980 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.980 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.980 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.980 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.980 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.980 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.981 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.981 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.981 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.981 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.981 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.981 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.981 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.981 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.981 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.981 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.981 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.981 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.981 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.982 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.982 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.982 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.982 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.982 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.982 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.982 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.982 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.982 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.982 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.982 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.982 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.982 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.982 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.982 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.983 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.983 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.983 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.983 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.983 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.983 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.983 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.983 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.983 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.983 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.983 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.983 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.983 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.984 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.984 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.984 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.984 INFO:teuthology.orchestra.run.smithi006.stdout:__________________ ERROR at teardown of test_put_public_block __________________ 2024-05-30T17:13:23.984 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.984 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.984 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.984 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.984 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.984 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.984 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.984 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.984 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.984 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.985 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.985 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.985 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.985 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.985 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.985 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.985 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.985 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.985 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.985 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.985 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.985 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.985 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.985 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.986 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.986 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.986 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.986 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.986 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.986 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.986 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.986 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.986 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.986 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.986 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.986 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.986 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.986 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.987 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.987 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.987 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.987 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.987 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.987 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.987 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.987 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.987 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.987 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.987 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.987 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.987 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.987 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.988 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.988 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.988 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.988 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.988 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.988 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.988 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.988 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.988 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.988 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.988 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.988 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.988 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.988 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.988 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.989 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.989 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.989 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.989 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.989 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.989 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.989 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.989 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.989 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.989 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.989 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.989 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.989 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.989 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.990 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.990 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.990 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.990 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.990 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.990 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.990 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.990 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.990 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.990 INFO:teuthology.orchestra.run.smithi006.stdout:____________ ERROR at teardown of test_block_public_put_bucket_acls ____________ 2024-05-30T17:13:23.990 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.990 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.991 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.991 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.991 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.991 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.991 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.991 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.991 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.991 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.991 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.991 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.991 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.991 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.991 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.991 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.992 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.992 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.992 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.992 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.992 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.992 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.992 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.992 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.992 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.992 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.992 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.992 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.992 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:23.992 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:23.993 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:23.993 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:23.993 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:23.993 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:23.993 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:23.993 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:23.993 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:23.993 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:23.993 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.993 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:23.993 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:23.993 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:23.994 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.994 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:23.994 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:23.994 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:23.994 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:23.994 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:23.994 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:23.994 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:23.994 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:23.994 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.994 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:23.994 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.995 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:23.995 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:23.995 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.995 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.995 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.995 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.995 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:23.995 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:23.995 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.995 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.995 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.995 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:23.996 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:23.996 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:23.996 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:23.996 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:23.996 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:23.996 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.996 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.996 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:23.996 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:23.996 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:23.996 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:23.996 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:23.996 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:23.997 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:23.997 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:23.997 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:23.997 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.997 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:23.997 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:23.997 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:23.997 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:23.997 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:23.997 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.997 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:23.997 INFO:teuthology.orchestra.run.smithi006.stdout:__________ ERROR at teardown of test_block_public_object_canned_acls ___________ 2024-05-30T17:13:23.997 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.997 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:23.998 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.998 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:23.998 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:23.998 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:23.998 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:23.998 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.998 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:23.998 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.998 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:23.998 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:23.998 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:23.998 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:23.998 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:23.999 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:23.999 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:23.999 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:23.999 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:23.999 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:23.999 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:23.999 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:23.999 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:23.999 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:23.999 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:23.999 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:23.999 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:23.999 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.000 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:24.000 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:24.000 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:24.000 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:24.000 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:24.000 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:24.000 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:24.000 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:24.000 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:24.000 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:24.000 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.000 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:24.000 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:24.000 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:24.001 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.001 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:24.001 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:24.001 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:24.001 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:24.001 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:24.001 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:24.001 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:24.001 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:24.001 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.001 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:24.001 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.001 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:24.001 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:24.001 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:24.001 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:24.002 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:24.002 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:24.002 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:24.002 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:24.002 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:24.002 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.002 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.002 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:24.002 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:24.002 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:24.002 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:24.002 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:24.002 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:24.003 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.003 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.003 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:24.003 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:24.003 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:24.003 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:24.003 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:24.003 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:24.003 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:24.003 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:24.003 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.003 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.004 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:24.004 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:24.004 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:24.004 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:24.004 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:24.004 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.004 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:24.004 INFO:teuthology.orchestra.run.smithi006.stdout:________________ ERROR at teardown of test_block_public_policy _________________ 2024-05-30T17:13:24.004 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.004 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:24.004 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.004 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:24.004 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:24.005 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:24.005 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:24.005 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.005 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:24.005 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:24.005 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:24.005 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:24.005 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:24.005 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:24.005 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:24.005 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:24.005 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:24.006 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:24.006 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:24.006 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:24.006 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:24.006 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:24.006 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:24.006 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.006 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:24.006 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:24.006 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:24.006 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.006 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:24.006 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:24.006 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:24.007 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:24.007 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:24.007 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:24.007 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:24.007 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:24.007 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:24.007 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:24.007 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.007 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:24.007 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:24.007 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:24.007 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.007 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:24.007 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:24.007 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:24.007 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:24.008 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:24.008 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:24.008 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:24.008 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:24.008 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.008 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:24.008 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.008 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:24.008 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:24.008 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:24.008 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:24.008 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:24.008 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:24.008 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:24.008 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:24.009 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:24.009 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.009 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.009 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:24.009 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:24.009 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:24.009 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:24.009 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:24.009 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:24.009 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.009 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.009 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:24.009 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:24.009 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:24.009 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:24.010 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:24.010 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:24.010 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:24.010 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:24.010 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.010 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.010 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:24.010 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:24.010 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:24.010 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:24.010 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:24.010 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.010 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:24.010 INFO:teuthology.orchestra.run.smithi006.stdout:_________________ ERROR at teardown of test_ignore_public_acls _________________ 2024-05-30T17:13:24.010 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.011 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:24.011 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.011 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:24.011 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:24.011 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:24.011 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:24.011 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.011 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:24.011 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:24.011 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:24.011 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:24.011 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:24.011 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:24.012 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:24.012 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:24.012 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:24.012 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:24.012 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:24.012 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:24.012 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:24.012 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:24.012 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:24.012 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.012 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:24.012 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:24.012 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:24.013 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.013 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:24.013 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:24.013 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:24.013 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:24.013 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:24.013 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:24.013 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:24.013 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:24.013 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:24.013 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:24.013 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.013 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:24.013 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:24.013 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:24.014 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.014 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:24.014 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:24.014 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:24.014 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:24.014 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:24.014 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:24.014 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:24.014 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:24.014 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.014 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:24.014 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.014 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:24.015 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:24.015 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:24.015 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:24.015 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:24.015 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:24.015 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:24.015 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:24.015 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:24.015 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.015 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.015 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:24.015 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:24.015 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:24.015 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:24.015 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:24.016 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:24.016 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.016 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.016 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:24.016 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:24.016 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:24.016 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:24.016 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:24.016 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:24.016 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:24.016 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:24.016 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.016 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.017 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:24.017 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:24.017 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:24.017 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:24.017 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:24.017 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.017 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:24.017 INFO:teuthology.orchestra.run.smithi006.stdout:______ ERROR at teardown of test_multipart_upload_on_a_bucket_with_policy ______ 2024-05-30T17:13:24.017 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.017 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:24.017 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.017 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:24.017 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:24.017 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:24.017 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:24.018 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.018 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:24.018 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:24.018 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:24.018 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:24.018 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:24.018 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:24.018 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:24.018 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:24.018 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:24.018 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:24.018 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:24.018 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:24.019 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:24.019 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:24.019 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:24.019 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.019 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:24.019 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:24.019 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:24.019 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.019 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:24.019 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:24.019 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:24.019 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:24.019 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:24.020 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:24.020 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:24.020 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:24.020 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:24.020 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:24.020 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.020 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:24.020 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:24.020 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:24.020 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.020 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:24.020 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:24.020 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:24.021 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:24.021 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:24.021 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:24.021 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:24.021 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:24.021 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.021 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:24.021 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.021 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:24.021 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:24.021 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:24.021 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:24.021 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:24.022 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:24.022 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:24.022 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:24.022 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:24.022 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.022 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.022 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:24.022 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:24.022 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:24.022 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:24.022 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:24.022 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:24.022 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.022 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.022 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:24.023 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:24.023 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:24.023 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:24.023 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:24.023 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:24.023 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:24.023 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:24.023 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.023 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.023 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:24.023 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:24.023 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:24.023 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:24.023 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:24.024 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.024 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:24.024 INFO:teuthology.orchestra.run.smithi006.stdout:_____________ ERROR at teardown of test_put_bucket_encryption_kms ______________ 2024-05-30T17:13:24.024 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.024 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:24.024 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.024 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:24.024 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:24.024 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:24.024 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:24.024 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.024 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:24.024 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:24.025 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:24.025 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:24.025 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:24.025 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:24.025 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:24.025 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:24.025 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:24.025 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:24.025 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:24.025 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:24.025 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:24.025 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:24.025 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:24.025 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.026 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:24.026 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:24.026 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:24.026 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.026 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:24.026 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:24.026 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:24.026 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:24.026 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:24.026 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:24.026 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:24.026 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:24.026 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:24.027 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:24.027 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.027 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:24.027 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:24.027 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:24.027 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.027 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:24.027 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:24.027 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:24.027 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:24.027 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:24.027 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:24.027 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:24.028 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:24.028 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.028 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:24.028 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.028 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:24.028 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:24.028 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:24.028 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:24.028 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:24.028 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:24.028 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:24.028 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:24.029 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:24.029 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.029 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.029 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:24.029 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:24.029 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:24.029 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:24.029 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:24.029 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:24.029 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.029 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.029 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:24.029 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:24.030 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:24.030 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:24.030 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:24.030 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:24.030 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:24.030 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:24.030 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.030 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.030 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:24.030 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:24.030 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:24.030 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:24.030 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:24.031 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.031 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:24.031 INFO:teuthology.orchestra.run.smithi006.stdout:_____________ ERROR at teardown of test_get_bucket_encryption_kms ______________ 2024-05-30T17:13:24.031 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.031 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:24.031 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.031 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:24.031 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:24.031 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:24.031 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:24.031 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.031 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:24.031 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:24.031 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:24.032 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:24.032 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:24.032 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:24.032 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:24.032 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:24.032 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:24.032 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:24.032 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:24.032 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:24.032 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:24.032 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:24.032 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:24.032 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.033 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:24.033 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:24.033 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:24.033 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.033 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:24.033 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:24.033 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:24.033 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:24.033 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:24.033 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:24.033 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:24.033 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:24.033 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:24.033 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:24.034 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.034 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:24.034 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:24.034 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:24.034 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.034 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:24.034 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:24.034 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:24.034 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:24.034 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:24.034 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:24.034 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:24.034 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:24.035 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.035 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:24.035 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.035 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:24.035 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:24.035 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:24.035 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:24.035 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:24.035 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:24.035 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:24.035 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:24.035 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:24.035 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.035 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.036 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:24.036 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:24.036 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:24.036 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:24.036 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:24.036 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:24.036 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.036 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.036 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:24.036 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:24.036 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:24.036 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:24.036 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:24.036 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:24.037 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:24.037 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:24.037 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.037 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.037 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:24.037 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:24.037 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:24.037 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:24.037 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:24.037 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.037 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:24.037 INFO:teuthology.orchestra.run.smithi006.stdout:____________ ERROR at teardown of test_delete_bucket_encryption_kms ____________ 2024-05-30T17:13:24.037 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.038 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:24.038 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.038 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:24.038 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:24.038 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:24.038 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:24.038 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.038 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:24.038 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:24.038 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:24.038 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:24.038 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:24.039 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:24.039 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:24.039 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:24.039 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:24.039 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:24.039 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:24.039 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:24.039 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:24.039 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:24.039 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:24.039 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.039 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:24.039 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:24.040 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:24.040 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.040 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:24.040 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:24.040 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:24.040 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:24.040 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:24.040 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:24.040 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:24.040 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:24.040 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:24.040 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:24.040 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.041 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:24.041 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:24.041 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:24.041 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.041 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:24.041 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:24.041 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:24.041 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:24.041 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:24.041 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:24.041 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:24.041 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:24.041 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.041 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:24.041 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.042 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:24.042 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:24.042 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:24.042 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:24.042 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:24.042 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:24.042 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:24.042 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:24.042 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:24.042 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.042 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.042 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:24.042 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:24.042 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:24.042 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:24.043 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:24.043 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:24.043 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.043 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.043 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:24.043 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:24.043 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:24.043 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:24.043 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:24.043 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:24.043 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:24.043 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:24.043 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.043 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.044 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:24.044 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:24.044 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:24.044 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:24.044 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:24.044 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.044 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:24.044 INFO:teuthology.orchestra.run.smithi006.stdout:_ ERROR at teardown of test_sse_kms_default_post_object_authenticated_request __ 2024-05-30T17:13:24.044 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.044 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:24.044 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.044 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:24.044 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:24.044 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:24.045 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:24.045 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.045 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:24.045 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:24.045 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:24.045 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:24.045 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:24.045 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:24.045 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:24.045 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:24.045 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:24.045 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:24.045 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:24.046 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:24.046 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:24.046 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:24.046 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:24.046 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.046 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:24.046 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:24.046 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:24.046 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.046 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:24.046 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:24.046 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:24.046 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:24.047 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:24.047 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:24.047 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:24.047 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:24.047 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:24.047 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:24.047 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.047 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:24.047 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:24.047 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:24.047 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.047 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:24.047 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:24.047 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:24.047 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:24.048 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:24.048 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:24.048 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:24.048 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:24.048 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.048 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:24.048 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.048 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:24.048 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:24.048 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:24.048 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:24.048 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:24.048 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:24.048 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:24.049 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:24.049 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:24.049 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.049 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.049 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:24.049 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:24.049 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:24.049 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:24.049 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:24.049 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:24.049 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.049 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.049 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:24.050 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:24.050 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:24.050 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:24.050 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:24.050 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:24.050 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:24.050 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:24.050 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.050 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.050 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:24.050 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:24.050 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:24.050 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:24.050 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:24.050 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.051 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:24.051 INFO:teuthology.orchestra.run.smithi006.stdout:_________________ ERROR at teardown of test_get_object_torrent _________________ 2024-05-30T17:13:24.051 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.051 INFO:teuthology.orchestra.run.smithi006.stdout:configfile = 2024-05-30T17:13:24.051 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.051 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.fixture(autouse=True) 2024-05-30T17:13:24.051 INFO:teuthology.orchestra.run.smithi006.stdout: def setup_teardown(configfile): 2024-05-30T17:13:24.051 INFO:teuthology.orchestra.run.smithi006.stdout: yield 2024-05-30T17:13:24.051 INFO:teuthology.orchestra.run.smithi006.stdout:> teardown() 2024-05-30T17:13:24.051 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.051 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:332: 2024-05-30T17:13:24.051 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:24.051 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-05-30T17:13:24.051 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-05-30T17:13:24.051 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-05-30T17:13:24.052 INFO:teuthology.orchestra.run.smithi006.stdout: raise err 2024-05-30T17:13:24.052 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-05-30T17:13:24.052 INFO:teuthology.orchestra.run.smithi006.stdout: nuke_bucket(client, bucket_name) 2024-05-30T17:13:24.052 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-05-30T17:13:24.052 INFO:teuthology.orchestra.run.smithi006.stdout: for objects in list_versions(client, bucket, batch_size): 2024-05-30T17:13:24.052 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-05-30T17:13:24.052 INFO:teuthology.orchestra.run.smithi006.stdout: listing = client.list_object_versions(**kwargs) 2024-05-30T17:13:24.052 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-05-30T17:13:24.052 INFO:teuthology.orchestra.run.smithi006.stdout: return self._make_api_call(operation_name, kwargs) 2024-05-30T17:13:24.052 INFO:teuthology.orchestra.run.smithi006.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-05-30T17:13:24.052 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.052 INFO:teuthology.orchestra.run.smithi006.stdout:self = 2024-05-30T17:13:24.052 INFO:teuthology.orchestra.run.smithi006.stdout:operation_name = 'ListObjectVersions' 2024-05-30T17:13:24.052 INFO:teuthology.orchestra.run.smithi006.stdout:api_params = {'Bucket': 'test-zrhryoy189ajjeo9txwotj0u-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-05-30T17:13:24.052 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.053 INFO:teuthology.orchestra.run.smithi006.stdout: def _make_api_call(self, operation_name, api_params): 2024-05-30T17:13:24.053 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-05-30T17:13:24.053 INFO:teuthology.orchestra.run.smithi006.stdout: service_name = self._service_model.service_name 2024-05-30T17:13:24.053 INFO:teuthology.orchestra.run.smithi006.stdout: history_recorder.record( 2024-05-30T17:13:24.053 INFO:teuthology.orchestra.run.smithi006.stdout: 'API_CALL', 2024-05-30T17:13:24.053 INFO:teuthology.orchestra.run.smithi006.stdout: { 2024-05-30T17:13:24.053 INFO:teuthology.orchestra.run.smithi006.stdout: 'service': service_name, 2024-05-30T17:13:24.053 INFO:teuthology.orchestra.run.smithi006.stdout: 'operation': operation_name, 2024-05-30T17:13:24.053 INFO:teuthology.orchestra.run.smithi006.stdout: 'params': api_params, 2024-05-30T17:13:24.053 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2024-05-30T17:13:24.053 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.053 INFO:teuthology.orchestra.run.smithi006.stdout: if operation_model.deprecated: 2024-05-30T17:13:24.053 INFO:teuthology.orchestra.run.smithi006.stdout: logger.debug( 2024-05-30T17:13:24.053 INFO:teuthology.orchestra.run.smithi006.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-05-30T17:13:24.054 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.054 INFO:teuthology.orchestra.run.smithi006.stdout: request_context = { 2024-05-30T17:13:24.054 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_region': self.meta.region_name, 2024-05-30T17:13:24.054 INFO:teuthology.orchestra.run.smithi006.stdout: 'client_config': self.meta.config, 2024-05-30T17:13:24.054 INFO:teuthology.orchestra.run.smithi006.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-05-30T17:13:24.054 INFO:teuthology.orchestra.run.smithi006.stdout: 'auth_type': operation_model.auth_type, 2024-05-30T17:13:24.054 INFO:teuthology.orchestra.run.smithi006.stdout: } 2024-05-30T17:13:24.054 INFO:teuthology.orchestra.run.smithi006.stdout: request_dict = self._convert_to_request_dict( 2024-05-30T17:13:24.054 INFO:teuthology.orchestra.run.smithi006.stdout: api_params, operation_model, context=request_context 2024-05-30T17:13:24.054 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.054 INFO:teuthology.orchestra.run.smithi006.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-05-30T17:13:24.054 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.054 INFO:teuthology.orchestra.run.smithi006.stdout: service_id = self._service_model.service_id.hyphenize() 2024-05-30T17:13:24.054 INFO:teuthology.orchestra.run.smithi006.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-05-30T17:13:24.055 INFO:teuthology.orchestra.run.smithi006.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:24.055 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:24.055 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:24.055 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:24.055 INFO:teuthology.orchestra.run.smithi006.stdout: params=request_dict, 2024-05-30T17:13:24.055 INFO:teuthology.orchestra.run.smithi006.stdout: request_signer=self._request_signer, 2024-05-30T17:13:24.055 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:24.055 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.055 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.055 INFO:teuthology.orchestra.run.smithi006.stdout: if event_response is not None: 2024-05-30T17:13:24.055 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = event_response 2024-05-30T17:13:24.055 INFO:teuthology.orchestra.run.smithi006.stdout: else: 2024-05-30T17:13:24.055 INFO:teuthology.orchestra.run.smithi006.stdout: apply_request_checksum(request_dict) 2024-05-30T17:13:24.055 INFO:teuthology.orchestra.run.smithi006.stdout: http, parsed_response = self._make_request( 2024-05-30T17:13:24.056 INFO:teuthology.orchestra.run.smithi006.stdout: operation_model, request_dict, request_context 2024-05-30T17:13:24.056 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.056 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.056 INFO:teuthology.orchestra.run.smithi006.stdout: self.meta.events.emit( 2024-05-30T17:13:24.056 INFO:teuthology.orchestra.run.smithi006.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-05-30T17:13:24.056 INFO:teuthology.orchestra.run.smithi006.stdout: service_id=service_id, operation_name=operation_name 2024-05-30T17:13:24.056 INFO:teuthology.orchestra.run.smithi006.stdout: ), 2024-05-30T17:13:24.056 INFO:teuthology.orchestra.run.smithi006.stdout: http_response=http, 2024-05-30T17:13:24.056 INFO:teuthology.orchestra.run.smithi006.stdout: parsed=parsed_response, 2024-05-30T17:13:24.056 INFO:teuthology.orchestra.run.smithi006.stdout: model=operation_model, 2024-05-30T17:13:24.056 INFO:teuthology.orchestra.run.smithi006.stdout: context=request_context, 2024-05-30T17:13:24.056 INFO:teuthology.orchestra.run.smithi006.stdout: ) 2024-05-30T17:13:24.056 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.056 INFO:teuthology.orchestra.run.smithi006.stdout: if http.status_code >= 300: 2024-05-30T17:13:24.057 INFO:teuthology.orchestra.run.smithi006.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-05-30T17:13:24.057 INFO:teuthology.orchestra.run.smithi006.stdout: error_class = self.exceptions.from_code(error_code) 2024-05-30T17:13:24.057 INFO:teuthology.orchestra.run.smithi006.stdout:> raise error_class(parsed_response, operation_name) 2024-05-30T17:13:24.057 INFO:teuthology.orchestra.run.smithi006.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-05-30T17:13:24.057 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.057 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-05-30T17:13:24.057 INFO:teuthology.orchestra.run.smithi006.stdout:=============================== warnings summary =============================== 2024-05-30T17:13:24.057 INFO:teuthology.orchestra.run.smithi006.stdout:.tox/py/lib/python3.10/site-packages/boto/plugin.py:40 2024-05-30T17:13:24.057 INFO:teuthology.orchestra.run.smithi006.stdout: /home/ubuntu/cephtest/s3-tests-client.0/.tox/py/lib/python3.10/site-packages/boto/plugin.py:40: DeprecationWarning: the imp module is deprecated in favour of importlib and slated for removal in Python 3.12; see the module's documentation for alternative uses 2024-05-30T17:13:24.057 INFO:teuthology.orchestra.run.smithi006.stdout: import imp 2024-05-30T17:13:24.057 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.057 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py:120 2024-05-30T17:13:24.057 INFO:teuthology.orchestra.run.smithi006.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests/functional/test_headers.py:120: SyntaxWarning: "is" with a literal. Did you mean "=="? 2024-05-30T17:13:24.057 INFO:teuthology.orchestra.run.smithi006.stdout: if boto_type is 'S3Connection': 2024-05-30T17:13:24.057 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.058 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_headers.py:124 2024-05-30T17:13:24.058 INFO:teuthology.orchestra.run.smithi006.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests/functional/test_headers.py:124: SyntaxWarning: "is" with a literal. Did you mean "=="? 2024-05-30T17:13:24.058 INFO:teuthology.orchestra.run.smithi006.stdout: elif boto_type is 'HTTPRequest': 2024-05-30T17:13:24.058 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.058 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3.py:359 2024-05-30T17:13:24.058 INFO:teuthology.orchestra.run.smithi006.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests/functional/test_s3.py:359: DeprecationWarning: invalid escape sequence '\W' 2024-05-30T17:13:24.058 INFO:teuthology.orchestra.run.smithi006.stdout: extra_sc = re.split('\W+', config['main']['storage_classes']) 2024-05-30T17:13:24.058 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.058 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py:176 2024-05-30T17:13:24.058 INFO:teuthology.orchestra.run.smithi006.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests/functional/test_s3_website.py:176: SyntaxWarning: "is" with a literal. Did you mean "=="? 2024-05-30T17:13:24.058 INFO:teuthology.orchestra.run.smithi006.stdout: if k is 'BucketName': 2024-05-30T17:13:24.058 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.059 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py:253 2024-05-30T17:13:24.059 INFO:teuthology.orchestra.run.smithi006.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests/functional/test_s3_website.py:253: PytestUnknownMarkWarning: Unknown pytest.mark.timeout - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html 2024-05-30T17:13:24.059 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.mark.timeout(10) 2024-05-30T17:13:24.059 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.059 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests/functional/test_s3_website.py:1077 2024-05-30T17:13:24.059 INFO:teuthology.orchestra.run.smithi006.stdout: s3tests/functional/test_s3_website.py:1077: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_routing_generator will be ignored 2024-05-30T17:13:24.059 INFO:teuthology.orchestra.run.smithi006.stdout: @pytest.mark.s3website_routing_rules 2024-05-30T17:13:24.059 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.059 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py:2213 2024-05-30T17:13:24.059 INFO:teuthology.orchestra.run.smithi006.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2213: DeprecationWarning: invalid escape sequence '\$' 2024-05-30T17:13:24.059 INFO:teuthology.orchestra.run.smithi006.stdout: ["starts-with", "$key", "\$foo"],\ 2024-05-30T17:13:24.059 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.059 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py:2228 2024-05-30T17:13:24.060 INFO:teuthology.orchestra.run.smithi006.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2228: DeprecationWarning: invalid escape sequence '\$' 2024-05-30T17:13:24.060 INFO:teuthology.orchestra.run.smithi006.stdout: payload = OrderedDict([ ("key" , "\$foo.txt"),("AWSAccessKeyId" , aws_access_key_id),\ 2024-05-30T17:13:24.060 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.060 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py:2234 2024-05-30T17:13:24.060 INFO:teuthology.orchestra.run.smithi006.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2234: DeprecationWarning: invalid escape sequence '\$' 2024-05-30T17:13:24.060 INFO:teuthology.orchestra.run.smithi006.stdout: response = client.get_object(Bucket=bucket_name, Key='\$foo.txt') 2024-05-30T17:13:24.060 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.060 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py:2291 2024-05-30T17:13:24.060 INFO:teuthology.orchestra.run.smithi006.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2291: DeprecationWarning: invalid escape sequence '\$' 2024-05-30T17:13:24.060 INFO:teuthology.orchestra.run.smithi006.stdout: ["starts-with", "$key", "\$foo"],\ 2024-05-30T17:13:24.060 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.060 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py:2306 2024-05-30T17:13:24.060 INFO:teuthology.orchestra.run.smithi006.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2306: DeprecationWarning: invalid escape sequence '\$' 2024-05-30T17:13:24.060 INFO:teuthology.orchestra.run.smithi006.stdout: payload = OrderedDict([ ("key" , "\$foo.txt"),("AWSAccessKeyId" , aws_access_key_id),\ 2024-05-30T17:13:24.061 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.061 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py:2324 2024-05-30T17:13:24.061 INFO:teuthology.orchestra.run.smithi006.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2324: DeprecationWarning: invalid escape sequence '\$' 2024-05-30T17:13:24.061 INFO:teuthology.orchestra.run.smithi006.stdout: ["starts-with", "$key", "\$foo"],\ 2024-05-30T17:13:24.061 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.061 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py:2339 2024-05-30T17:13:24.061 INFO:teuthology.orchestra.run.smithi006.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2339: DeprecationWarning: invalid escape sequence '\$' 2024-05-30T17:13:24.061 INFO:teuthology.orchestra.run.smithi006.stdout: payload = OrderedDict([ ("key" , "\$foo.txt"),("AWSAccessKeyId" , aws_access_key_id[::-1]),\ 2024-05-30T17:13:24.061 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.061 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py:2357 2024-05-30T17:13:24.061 INFO:teuthology.orchestra.run.smithi006.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2357: DeprecationWarning: invalid escape sequence '\$' 2024-05-30T17:13:24.061 INFO:teuthology.orchestra.run.smithi006.stdout: ["starts-with", "$key", "\$foo"],\ 2024-05-30T17:13:24.061 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.061 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py:2372 2024-05-30T17:13:24.062 INFO:teuthology.orchestra.run.smithi006.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2372: DeprecationWarning: invalid escape sequence '\$' 2024-05-30T17:13:24.062 INFO:teuthology.orchestra.run.smithi006.stdout: payload = OrderedDict([ ("key" , "\$foo.txt"),("AWSAccessKeyId" , aws_access_key_id),\ 2024-05-30T17:13:24.062 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.062 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py:2422 2024-05-30T17:13:24.062 INFO:teuthology.orchestra.run.smithi006.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2422: DeprecationWarning: invalid escape sequence '\$' 2024-05-30T17:13:24.062 INFO:teuthology.orchestra.run.smithi006.stdout: ["starts-with", "$key", "\$foo"],\ 2024-05-30T17:13:24.062 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.062 INFO:teuthology.orchestra.run.smithi006.stdout:s3tests_boto3/functional/test_s3.py:2454 2024-05-30T17:13:24.062 INFO:teuthology.orchestra.run.smithi006.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2454: DeprecationWarning: invalid escape sequence '\$' 2024-05-30T17:13:24.062 INFO:teuthology.orchestra.run.smithi006.stdout: ["starts-with", "$key", "\$foo"],\ 2024-05-30T17:13:24.062 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:24.062 INFO:teuthology.orchestra.run.smithi006.stdout:-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html 2024-05-30T17:13:24.062 INFO:teuthology.orchestra.run.smithi006.stdout:=========================== short test summary info ============================ 2024-05-30T17:13:24.063 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_write_to_nonexist_bucket 2024-05-30T17:13:24.063 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_write_with_chunked_transfer_encoding 2024-05-30T17:13:24.063 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_delete - botoco... 2024-05-30T17:13:24.063 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_read_not_exist - botoc... 2024-05-30T17:13:24.063 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_requestid_matches_header_on_error 2024-05-30T17:13:24.063 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_concurrent_multi_object_delete 2024-05-30T17:13:24.063 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multi_object_delete - botocor... 2024-05-30T17:13:24.063 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multi_objectv2_delete - botoc... 2024-05-30T17:13:24.063 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multi_object_delete_key_limit 2024-05-30T17:13:24.063 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multi_objectv2_delete_key_limit 2024-05-30T17:13:24.063 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_head_zero_bytes - boto... 2024-05-30T17:13:24.063 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_write_check_etag - bot... 2024-05-30T17:13:24.063 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_write_cache_control - ... 2024-05-30T17:13:24.064 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_write_expires - botoco... 2024-05-30T17:13:24.064 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_write_read_update_read_delete 2024-05-30T17:13:24.064 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_none_to_good 2024-05-30T17:13:24.064 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_none_to_empty 2024-05-30T17:13:24.064 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_overwrite_to_empty 2024-05-30T17:13:24.064 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_metadata_replaced_on_put 2024-05-30T17:13:24.064 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_write_file - botocore.... 2024-05-30T17:13:24.064 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_anonymous_request 2024-05-30T17:13:24.064 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_request 2024-05-30T17:13:24.064 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_no_content_type 2024-05-30T17:13:24.064 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_request_bad_access_key 2024-05-30T17:13:24.064 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_set_success_code 2024-05-30T17:13:24.064 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_set_invalid_success_code 2024-05-30T17:13:24.065 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_upload_larger_than_chunk 2024-05-30T17:13:24.065 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_set_key_from_filename 2024-05-30T17:13:24.065 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_ignored_header - ... 2024-05-30T17:13:24.065 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_case_insensitive_condition_fields 2024-05-30T17:13:24.065 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_escaped_field_values 2024-05-30T17:13:24.065 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_success_redirect_action 2024-05-30T17:13:24.065 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_invalid_signature 2024-05-30T17:13:24.065 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_invalid_access_key 2024-05-30T17:13:24.065 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_invalid_date_format 2024-05-30T17:13:24.065 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_no_key_specified 2024-05-30T17:13:24.065 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_missing_signature 2024-05-30T17:13:24.065 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_missing_policy_condition 2024-05-30T17:13:24.065 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_user_specified_header 2024-05-30T17:13:24.065 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_request_missing_policy_specified_field 2024-05-30T17:13:24.065 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_condition_is_case_sensitive 2024-05-30T17:13:24.066 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_expires_is_case_sensitive 2024-05-30T17:13:24.066 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_expired_policy - ... 2024-05-30T17:13:24.066 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_wrong_bucket - bo... 2024-05-30T17:13:24.066 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_invalid_request_field_value 2024-05-30T17:13:24.066 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_missing_expires_condition 2024-05-30T17:13:24.066 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_missing_conditions_list 2024-05-30T17:13:24.066 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_limit_exceeded 2024-05-30T17:13:24.066 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_missing_content_length_argument 2024-05-30T17:13:24.066 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_invalid_content_length_argument 2024-05-30T17:13:24.066 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_below_minimum 2024-05-30T17:13:24.066 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_rgw_chunk_size_bug 2024-05-30T17:13:24.066 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_empty_conditions 2024-05-30T17:13:24.066 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifmatch_good - bot... 2024-05-30T17:13:24.067 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifmatch_failed - b... 2024-05-30T17:13:24.067 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifnonematch_good 2024-05-30T17:13:24.067 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifnonematch_failed 2024-05-30T17:13:24.067 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifmodifiedsince_good 2024-05-30T17:13:24.067 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifmodifiedsince_failed 2024-05-30T17:13:24.067 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifunmodifiedsince_good 2024-05-30T17:13:24.067 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifunmodifiedsince_failed 2024-05-30T17:13:24.067 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_good - bot... 2024-05-30T17:13:24.067 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_failed - b... 2024-05-30T17:13:24.067 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_overwrite_existed_good 2024-05-30T17:13:24.067 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_nonexisted_failed 2024-05-30T17:13:24.067 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_good - ... 2024-05-30T17:13:24.067 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_failed 2024-05-30T17:13:24.067 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_nonexisted_good 2024-05-30T17:13:24.068 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_overwrite_existed_failed 2024-05-30T17:13:24.068 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get - botocore.err... 2024-05-30T17:13:24.068 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_bucket_gone - ... 2024-05-30T17:13:24.068 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_delete_key_bucket_gone 2024-05-30T17:13:24.068 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_object_gone - ... 2024-05-30T17:13:24.068 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_head - botocore.errorf... 2024-05-30T17:13:24.068 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_head_notexist - botoco... 2024-05-30T17:13:24.068 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_head_extended - botoco... 2024-05-30T17:13:24.068 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_bucket_acl - b... 2024-05-30T17:13:24.068 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_object_acl - b... 2024-05-30T17:13:24.068 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_put_acl_mtime - botoco... 2024-05-30T17:13:24.068 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated - bo... 2024-05-30T17:13:24.068 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_response_headers 2024-05-30T17:13:24.069 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_bucket_acl 2024-05-30T17:13:24.069 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_object_acl 2024-05-30T17:13:24.069 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_bucket_gone 2024-05-30T17:13:24.069 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_object_gone 2024-05-30T17:13:24.069 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired 2024-05-30T17:13:24.069 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired_tenant 2024-05-30T17:13:24.069 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_range_zero 2024-05-30T17:13:24.069 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_max_range 2024-05-30T17:13:24.069 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_positive_range 2024-05-30T17:13:24.069 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_anon_put - botocore.er... 2024-05-30T17:13:24.069 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_anon_put_write_access 2024-05-30T17:13:24.069 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_put_authenticated - bo... 2024-05-30T17:13:24.069 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_presigned_put_object_with_acl 2024-05-30T17:13:24.070 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_presigned_put_object_with_acl_tenant 2024-05-30T17:13:24.070 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_put_authenticated_expired 2024-05-30T17:13:24.070 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_starts_nonalpha 2024-05-30T17:13:24.070 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_short_one 2024-05-30T17:13:24.070 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_short_two 2024-05-30T17:13:24.070 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_60 2024-05-30T17:13:24.070 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_61 2024-05-30T17:13:24.070 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_62 2024-05-30T17:13:24.070 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_63 2024-05-30T17:13:24.070 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_list_long_name - botoc... 2024-05-30T17:13:24.070 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_ip 2024-05-30T17:13:24.070 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_underscore 2024-05-30T17:13:24.070 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_long 2024-05-30T17:13:24.070 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dash_at_end 2024-05-30T17:13:24.071 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dot_dot 2024-05-30T17:13:24.071 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dot_dash 2024-05-30T17:13:24.071 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dash_dot 2024-05-30T17:13:24.071 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_exists - botoco... 2024-05-30T17:13:24.071 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_get_location - botocor... 2024-05-30T17:13:24.071 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_exists_nonowner 2024-05-30T17:13:24.071 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_recreate_overwrite_acl 2024-05-30T17:13:24.071 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_recreate_new_acl - bot... 2024-05-30T17:13:24.071 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_default - botocore... 2024-05-30T17:13:24.071 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_during_create 2024-05-30T17:13:24.071 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned - botocore.... 2024-05-30T17:13:24.071 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_publicreadwrite 2024-05-30T17:13:24.071 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_authenticatedread 2024-05-30T17:13:24.071 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_default - botocore... 2024-05-30T17:13:24.071 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_canned_during_create 2024-05-30T17:13:24.072 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_canned - botocore.... 2024-05-30T17:13:24.072 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_canned_publicreadwrite 2024-05-30T17:13:24.072 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_canned_authenticatedread 2024-05-30T17:13:24.072 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_canned_bucketownerread 2024-05-30T17:13:24.072 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_canned_bucketownerfullcontrol 2024-05-30T17:13:24.072 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_full_control_verify_owner 2024-05-30T17:13:24.072 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_full_control_verify_attributes 2024-05-30T17:13:24.072 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_private_to_private 2024-05-30T17:13:24.072 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl - botocore.errorfa... 2024-05-30T17:13:24.072 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_write - botocore.e... 2024-05-30T17:13:24.072 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_writeacp - botocor... 2024-05-30T17:13:24.072 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_read - botocore.er... 2024-05-30T17:13:24.072 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_readacp - botocore... 2024-05-30T17:13:24.072 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_fullcontrol 2024-05-30T17:13:24.073 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_read 2024-05-30T17:13:24.073 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_readacp 2024-05-30T17:13:24.073 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_write 2024-05-30T17:13:24.073 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_writeacp 2024-05-30T17:13:24.073 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_nonexist_user 2024-05-30T17:13:24.073 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_header_acl_grants - bo... 2024-05-30T17:13:24.073 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_header_acl_grants - bo... 2024-05-30T17:13:24.073 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_email - boto... 2024-05-30T17:13:24.073 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_email_not_exist 2024-05-30T17:13:24.073 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_revoke_all - botoc... 2024-05-30T17:13:24.073 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_private 2024-05-30T17:13:24.073 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_private 2024-05-30T17:13:24.073 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_publicread 2024-05-30T17:13:24.073 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_publicread 2024-05-30T17:13:24.074 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_publicreadwrite 2024-05-30T17:13:24.074 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_publicreadwrite 2024-05-30T17:13:24.074 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_private 2024-05-30T17:13:24.074 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_publicread 2024-05-30T17:13:24.074 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_publicreadwrite 2024-05-30T17:13:24.074 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_private 2024-05-30T17:13:24.074 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicread 2024-05-30T17:13:24.074 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicreadwrite 2024-05-30T17:13:24.074 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_buckets_create_then_list - bo... 2024-05-30T17:13:24.074 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_buckets_list_ctime - botocore... 2024-05-30T17:13:24.074 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_list_buckets_anonymous - boto... 2024-05-30T17:13:24.074 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_list_buckets_invalid_auth - b... 2024-05-30T17:13:24.074 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_list_buckets_bad_auth - botoc... 2024-05-30T17:13:24.074 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_starts_alpha 2024-05-30T17:13:24.075 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_starts_digit 2024-05-30T17:13:24.075 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_contains_period 2024-05-30T17:13:24.075 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_contains_hyphen 2024-05-30T17:13:24.075 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_recreate_not_overriding 2024-05-30T17:13:24.075 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_special_key_names 2024-05-30T17:13:24.075 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_list_special_prefix - ... 2024-05-30T17:13:24.075 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_zero_size - botoc... 2024-05-30T17:13:24.075 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_16m - botocore.er... 2024-05-30T17:13:24.075 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_same_bucket - bot... 2024-05-30T17:13:24.075 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_verify_contenttype 2024-05-30T17:13:24.075 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_to_itself - botoc... 2024-05-30T17:13:24.075 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_to_itself_with_metadata 2024-05-30T17:13:24.075 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_diff_bucket - bot... 2024-05-30T17:13:24.075 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_not_owned_bucket 2024-05-30T17:13:24.075 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_not_owned_object_bucket 2024-05-30T17:13:24.076 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_canned_acl - boto... 2024-05-30T17:13:24.076 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_retaining_metadata 2024-05-30T17:13:24.076 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_replacing_metadata 2024-05-30T17:13:24.076 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_bucket_not_found 2024-05-30T17:13:24.076 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_key_not_found - b... 2024-05-30T17:13:24.076 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_versioned_bucket 2024-05-30T17:13:24.076 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_versioned_url_encoding 2024-05-30T17:13:24.076 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_versioning_multipart_upload 2024-05-30T17:13:24.076 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_empty - boto... 2024-05-30T17:13:24.076 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_small - boto... 2024-05-30T17:13:24.076 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_copy_small - botoco... 2024-05-30T17:13:24.076 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_copy_invalid_range 2024-05-30T17:13:24.076 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_copy_without_range 2024-05-30T17:13:24.077 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_copy_special_names 2024-05-30T17:13:24.077 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload - botocore.e... 2024-05-30T17:13:24.077 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_copy_versioned - bo... 2024-05-30T17:13:24.077 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_resend_part 2024-05-30T17:13:24.077 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_multiple_sizes 2024-05-30T17:13:24.077 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_copy_multiple_sizes 2024-05-30T17:13:24.077 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_size_too_small 2024-05-30T17:13:24.077 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_contents - b... 2024-05-30T17:13:24.077 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_overwrite_existing_object 2024-05-30T17:13:24.077 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_abort_multipart_upload - boto... 2024-05-30T17:13:24.077 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_abort_multipart_upload_not_found 2024-05-30T17:13:24.077 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_list_multipart_upload - botoc... 2024-05-30T17:13:24.077 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_list_multipart_upload_owner 2024-05-30T17:13:24.077 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_missing_part 2024-05-30T17:13:24.078 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_incorrect_etag 2024-05-30T17:13:24.078 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_get_part - botocore... 2024-05-30T17:13:24.078 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_non_multipart_get_part - boto... 2024-05-30T17:13:24.078 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_100_continue - botocore.error... 2024-05-30T17:13:24.078 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_set_cors - botocore.errorfact... 2024-05-30T17:13:24.078 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_origin_response - botoco... 2024-05-30T17:13:24.078 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_origin_wildcard - botoco... 2024-05-30T17:13:24.078 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_header_option - botocore... 2024-05-30T17:13:24.078 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_presigned_get_object - b... 2024-05-30T17:13:24.078 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_presigned_get_object_tenant 2024-05-30T17:13:24.078 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object - b... 2024-05-30T17:13:24.078 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_with_acl 2024-05-30T17:13:24.078 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_tenant 2024-05-30T17:13:24.078 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_tenant_with_acl 2024-05-30T17:13:24.079 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_set_bucket_tagging - botocore... 2024-05-30T17:13:24.079 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_read_1mb - botocore.er... 2024-05-30T17:13:24.079 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_read_4mb - botocore.er... 2024-05-30T17:13:24.079 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_read_8mb - botocore.er... 2024-05-30T17:13:24.079 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_write_1mb - botocore.e... 2024-05-30T17:13:24.079 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_write_4mb - botocore.e... 2024-05-30T17:13:24.079 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_write_8mb - botocore.e... 2024-05-30T17:13:24.079 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_1mb - botoc... 2024-05-30T17:13:24.079 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_4mb - botoc... 2024-05-30T17:13:24.079 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_8mb - botoc... 2024-05-30T17:13:24.079 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_conditional_write_1mb 2024-05-30T17:13:24.079 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_multipart_upload_write 2024-05-30T17:13:24.079 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_resend_first_finishes_last 2024-05-30T17:13:24.080 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_ranged_request_response_code 2024-05-30T17:13:24.080 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_ranged_big_request_response_code 2024-05-30T17:13:24.080 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_ranged_request_skip_leading_bytes_response_code 2024-05-30T17:13:24.080 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_ranged_request_return_trailing_bytes_response_code 2024-05-30T17:13:24.080 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_ranged_request_invalid_range 2024-05-30T17:13:24.080 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_ranged_request_empty_object 2024-05-30T17:13:24.080 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_bucket_create_suspend 2024-05-30T17:13:24.080 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_read_remove 2024-05-30T17:13:24.080 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_read_remove_head 2024-05-30T17:13:24.080 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_removal 2024-05-30T17:13:24.080 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_overwrite 2024-05-30T17:13:24.080 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_overwrite_suspended 2024-05-30T17:13:24.080 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_suspend_versions 2024-05-30T17:13:24.080 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_suspended_copy 2024-05-30T17:13:24.081 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_versions_remove_all 2024-05-30T17:13:24.081 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_versions_remove_special_names 2024-05-30T17:13:24.081 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_overwrite_multipart 2024-05-30T17:13:24.081 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_list_marker - ... 2024-05-30T17:13:24.081 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_copy_obj_version 2024-05-30T17:13:24.081 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete 2024-05-30T17:13:24.081 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete_with_marker 2024-05-30T17:13:24.081 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete_with_marker_create 2024-05-30T17:13:24.081 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioned_object_acl - botoco... 2024-05-30T17:13:24.081 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioned_object_acl_no_version_specified 2024-05-30T17:13:24.081 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioned_concurrent_object_create_and_remove 2024-05-30T17:13:24.081 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set - botocore.erro... 2024-05-30T17:13:24.081 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_get - botocore.erro... 2024-05-30T17:13:24.081 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_get_no_id - botocor... 2024-05-30T17:13:24.082 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_id_too_long - botoc... 2024-05-30T17:13:24.082 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_same_id - botocore.... 2024-05-30T17:13:24.082 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_invalid_status - bo... 2024-05-30T17:13:24.082 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set_date - botocore... 2024-05-30T17:13:24.082 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set_invalid_date - ... 2024-05-30T17:13:24.082 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set_noncurrent - bo... 2024-05-30T17:13:24.082 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set_deletemarker - ... 2024-05-30T17:13:24.082 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set_filter - botoco... 2024-05-30T17:13:24.082 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set_empty_filter - ... 2024-05-30T17:13:24.082 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set_multipart - bot... 2024-05-30T17:13:24.082 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_transition_set_invalid_date 2024-05-30T17:13:24.082 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1b - botoc... 2024-05-30T17:13:24.083 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1kb - boto... 2024-05-30T17:13:24.083 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1MB - boto... 2024-05-30T17:13:24.083 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_13b - boto... 2024-05-30T17:13:24.083 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_method_head 2024-05-30T17:13:24.083 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_present - bo... 2024-05-30T17:13:24.083 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_other_key - ... 2024-05-30T17:13:24.083 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_invalid_md5 2024-05-30T17:13:24.083 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_no_md5 - bot... 2024-05-30T17:13:24.083 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_no_key - bot... 2024-05-30T17:13:24.083 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_key_no_sse_c - bot... 2024-05-30T17:13:24.083 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_multipart_upload 2024-05-30T17:13:24.083 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_unaligned_multipart_upload 2024-05-30T17:13:24.084 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_multipart_bad_download 2024-05-30T17:13:24.084 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_post_object_authenticated_request 2024-05-30T17:13:24.084 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_method_head - botocor... 2024-05-30T17:13:24.084 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_present - botocore.er... 2024-05-30T17:13:24.084 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_no_key - botocore.err... 2024-05-30T17:13:24.084 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_not_declared - botoco... 2024-05-30T17:13:24.084 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_upload - bo... 2024-05-30T17:13:24.084 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_1 2024-05-30T17:13:24.084 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_2 2024-05-30T17:13:24.084 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_post_object_authenticated_request 2024-05-30T17:13:24.084 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1b - botocor... 2024-05-30T17:13:24.084 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1kb - botoco... 2024-05-30T17:13:24.084 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1MB - botoco... 2024-05-30T17:13:24.085 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_13b - botoco... 2024-05-30T17:13:24.085 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_read_declare - botoco... 2024-05-30T17:13:24.085 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy - botocore.erro... 2024-05-30T17:13:24.085 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucketv2_policy - botocore.er... 2024-05-30T17:13:24.085 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_acl - botocore.... 2024-05-30T17:13:24.085 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucketv2_policy_acl - botocor... 2024-05-30T17:13:24.085 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_another_bucket 2024-05-30T17:13:24.085 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucketv2_policy_another_bucket 2024-05-30T17:13:24.085 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_obj_tagging - botocore.er... 2024-05-30T17:13:24.085 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_obj_head_tagging - botoco... 2024-05-30T17:13:24.085 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_max_tags - botocore.error... 2024-05-30T17:13:24.085 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_excess_tags - botocore.er... 2024-05-30T17:13:24.085 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_max_kvsize_tags - botocor... 2024-05-30T17:13:24.086 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_excess_key_tags - botocor... 2024-05-30T17:13:24.086 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_excess_val_tags - botocor... 2024-05-30T17:13:24.086 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_modify_tags - botocore.er... 2024-05-30T17:13:24.086 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_delete_tags - botocore.er... 2024-05-30T17:13:24.086 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_tags_anonymous_request 2024-05-30T17:13:24.086 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_tags_authenticated_request 2024-05-30T17:13:24.086 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_obj_with_tags - botocore.... 2024-05-30T17:13:24.086 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_tags_acl_public - botocor... 2024-05-30T17:13:24.086 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_tags_acl_public - botocor... 2024-05-30T17:13:24.086 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_delete_tags_obj_public - boto... 2024-05-30T17:13:24.086 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_bucket_atomic_upload_return_version_id 2024-05-30T17:13:24.086 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_bucket_multipart_upload_return_version_id 2024-05-30T17:13:24.086 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_existing_tag 2024-05-30T17:13:24.087 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_tagging_existing_tag 2024-05-30T17:13:24.087 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_tagging_existing_tag 2024-05-30T17:13:24.087 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_copy_source 2024-05-30T17:13:24.087 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_copy_source_meta 2024-05-30T17:13:24.087 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_acl - b... 2024-05-30T17:13:24.087 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_grant 2024-05-30T17:13:24.087 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_c_s3 - b... 2024-05-30T17:13:24.087 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_c_kms - ... 2024-05-30T17:13:24.087 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_s3_kms 2024-05-30T17:13:24.087 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_bad_enc_kms 2024-05-30T17:13:24.087 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_kms_noenc 2024-05-30T17:13:24.087 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_kms_s3 2024-05-30T17:13:24.087 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_acl_existing_tag 2024-05-30T17:13:24.087 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock - bo... 2024-05-30T17:13:24.088 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_bucket 2024-05-30T17:13:24.088 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_with_days_and_years 2024-05-30T17:13:24.088 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_days 2024-05-30T17:13:24.088 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_years 2024-05-30T17:13:24.088 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_mode 2024-05-30T17:13:24.088 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_status 2024-05-30T17:13:24.088 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_suspend_versioning 2024-05-30T17:13:24.088 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_lock - bo... 2024-05-30T17:13:24.088 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_lock_invalid_bucket 2024-05-30T17:13:24.088 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention 2024-05-30T17:13:24.088 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_invalid_bucket 2024-05-30T17:13:24.088 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_invalid_mode 2024-05-30T17:13:24.088 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention 2024-05-30T17:13:24.089 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention_iso8601 2024-05-30T17:13:24.089 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention_invalid_bucket 2024-05-30T17:13:24.089 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_versionid 2024-05-30T17:13:24.089 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_override_default_retention 2024-05-30T17:13:24.089 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_increase_period 2024-05-30T17:13:24.089 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_shorten_period 2024-05-30T17:13:24.089 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_shorten_period_bypass 2024-05-30T17:13:24.089 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_retention 2024-05-30T17:13:24.089 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_delete_multipart_object_with_retention 2024-05-30T17:13:24.089 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_retention_and_marker 2024-05-30T17:13:24.089 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_multi_delete_object_with_retention 2024-05-30T17:13:24.089 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold - ... 2024-05-30T17:13:24.089 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold_invalid_bucket 2024-05-30T17:13:24.089 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold_invalid_status 2024-05-30T17:13:24.089 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_legal_hold - ... 2024-05-30T17:13:24.090 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_legal_hold_invalid_bucket 2024-05-30T17:13:24.090 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_legal_hold_on 2024-05-30T17:13:24.090 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_delete_multipart_object_with_legal_hold_on 2024-05-30T17:13:24.090 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_legal_hold_off 2024-05-30T17:13:24.090 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_metadata 2024-05-30T17:13:24.090 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_uploading_obj - b... 2024-05-30T17:13:24.090 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_governance_with_bypass 2024-05-30T17:13:24.090 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_governance_without_bypass 2024-05-30T17:13:24.090 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_compliance 2024-05-30T17:13:24.090 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_copy_object_ifmatch_good - bo... 2024-05-30T17:13:24.090 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_copy_object_ifnonematch_failed 2024-05-30T17:13:24.090 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_bucket_policy_status - bo... 2024-05-30T17:13:24.090 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_public_acl_bucket_policy_status 2024-05-30T17:13:24.090 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_authpublic_acl_bucket_policy_status 2024-05-30T17:13:24.091 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_publicpolicy_acl_bucket_policy_status 2024-05-30T17:13:24.091 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_nonpublicpolicy_acl_bucket_policy_status 2024-05-30T17:13:24.091 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_nonpublicpolicy_deny_bucket_policy_status 2024-05-30T17:13:24.091 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_undefined_public_block - ... 2024-05-30T17:13:24.091 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_public_block_deny_bucket_policy 2024-05-30T17:13:24.091 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_public_block - botocore.e... 2024-05-30T17:13:24.091 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_block_public_put_bucket_acls 2024-05-30T17:13:24.091 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_block_public_object_canned_acls 2024-05-30T17:13:24.091 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_block_public_policy - botocor... 2024-05-30T17:13:24.091 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_ignore_public_acls - botocore... 2024-05-30T17:13:24.091 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_on_a_bucket_with_policy 2024-05-30T17:13:24.091 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_bucket_encryption_kms - b... 2024-05-30T17:13:24.091 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_bucket_encryption_kms - b... 2024-05-30T17:13:24.091 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_delete_bucket_encryption_kms 2024-05-30T17:13:24.091 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_default_post_object_authenticated_request 2024-05-30T17:13:24.092 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_torrent - botocore... 2024-05-30T17:13:24.092 INFO:teuthology.orchestra.run.smithi006.stdout:= 666 passed, 65 skipped, 127 deselected, 1 xfailed, 18 warnings, 397 errors in 2185.69s (0:36:25) = 2024-05-30T17:13:24.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-30T17:13:24.093 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR: InvocationError for command /home/ubuntu/cephtest/s3-tests-client.0/.tox/py/bin/pytest -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3' (exited with code 1) 2024-05-30T17:13:24.093 INFO:teuthology.orchestra.run.smithi006.stdout:___________________________________ summary ____________________________________ 2024-05-30T17:13:24.093 INFO:teuthology.orchestra.run.smithi006.stdout:ERROR: py: commands failed 2024-05-30T17:13:24.094 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/qa/tasks/s3tests.py", line 462, in run_tests toxvenv_sh(ctx, remote, args, label="s3 tests against rgw") File "/home/teuthworker/src/git.ceph.com_ceph-c_da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/qa/tasks/s3tests.py", line 405, in toxvenv_sh return remote.sh(['source', activate, run.Raw('&&')] + args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (s3 tests against rgw) on smithi006 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" 2024-05-30T17:13:24.096 INFO:tasks.s3tests:Cleaning up boto... 2024-05-30T17:13:24.096 DEBUG:teuthology.orchestra.run.smithi006:> rm /home/ubuntu/cephtest/boto-client.0.cfg 2024-05-30T17:13:24.103 INFO:tasks.s3tests:Removing s3-tests... 2024-05-30T17:13:24.103 DEBUG:teuthology.orchestra.run.smithi006:> rm -rf /home/ubuntu/cephtest/s3-tests-client.0 2024-05-30T17:13:24.299 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/qa/tasks/s3tests.py", line 736, in task with contextutil.nested( File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__ self.gen.throw(typ, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/qa/tasks/s3tests.py", line 61, in download yield File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/qa/tasks/s3tests.py", line 462, in run_tests toxvenv_sh(ctx, remote, args, label="s3 tests against rgw") File "/home/teuthworker/src/git.ceph.com_ceph-c_da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/qa/tasks/s3tests.py", line 405, in toxvenv_sh return remote.sh(['source', activate, run.Raw('&&')] + args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (s3 tests against rgw) on smithi006 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" 2024-05-30T17:13:24.517 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=816dc8dd342d46b79778613a2def831d Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/qa/tasks/s3tests.py", line 736, in task with contextutil.nested( File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__ self.gen.throw(typ, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/qa/tasks/s3tests.py", line 61, in download yield File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/qa/tasks/s3tests.py", line 462, in run_tests toxvenv_sh(ctx, remote, args, label="s3 tests against rgw") File "/home/teuthworker/src/git.ceph.com_ceph-c_da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/qa/tasks/s3tests.py", line 405, in toxvenv_sh return remote.sh(['source', activate, run.Raw('&&')] + args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (s3 tests against rgw) on smithi006 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" 2024-05-30T17:13:24.520 DEBUG:teuthology.run_tasks:Unwinding manager s3tests 2024-05-30T17:13:24.530 DEBUG:teuthology.run_tasks:Unwinding manager s3tests-java 2024-05-30T17:13:24.587 DEBUG:tasks.s3tests_java:S3 Tests Java: END 2024-05-30T17:13:24.587 INFO:tasks.s3tests_java:S3 Tests Java: Cleaning up s3-tests-java... 2024-05-30T17:13:24.587 DEBUG:teuthology.orchestra.run.smithi006:> rm -rf /home/ubuntu/cephtest/s3-tests-java 2024-05-30T17:13:24.644 INFO:tasks.s3tests_java:S3 Tests Java: Deleting S3 users... 2024-05-30T17:13:24.645 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid tester.client.0 --purge-data --cluster ceph 2024-05-30T17:13:24.677 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T17:13:24.677 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T17:13:24.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.718+0000 7fcb89454980 20 rados->read ofs=0 len=0 2024-05-30T17:13:24.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.718+0000 7fcb89454980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T17:13:24.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.718+0000 7fcb89454980 20 realm 2024-05-30T17:13:24.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.718+0000 7fcb89454980 20 rados->read ofs=0 len=0 2024-05-30T17:13:24.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.718+0000 7fcb89454980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T17:13:24.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.718+0000 7fcb89454980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T17:13:24.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.718+0000 7fcb89454980 20 rados->read ofs=0 len=0 2024-05-30T17:13:24.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.718+0000 7fcb89454980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T17:13:24.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.718+0000 7fcb89454980 20 rados->read ofs=0 len=0 2024-05-30T17:13:24.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.722+0000 7fcb89454980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T17:13:24.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.722+0000 7fcb89454980 20 rados->read ofs=0 len=0 2024-05-30T17:13:24.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.722+0000 7fcb89454980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T17:13:24.724 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.722+0000 7fcb89454980 20 searching for the correct realm 2024-05-30T17:13:24.728 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.726+0000 7fcb86f65640 0 -- 172.21.15.6:0/2685968131 >> [v2:172.21.15.6:6816/2704410641,v1:172.21.15.6:6817/2704410641] conn(0x7fcb80029040 msgr2=0x7fcb80029420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T17:13:24.735 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.734+0000 7fcb87766640 0 -- 172.21.15.6:0/2685968131 >> [v2:172.21.15.204:6823/3156311527,v1:172.21.15.204:6825/3156311527] conn(0x7fcb780c0020 msgr2=0x7fcb78131070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T17:13:24.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.742+0000 7fcb89454980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T17:13:24.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.742+0000 7fcb89454980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T17:13:24.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.742+0000 7fcb89454980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T17:13:24.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.742+0000 7fcb89454980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T17:13:24.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.742+0000 7fcb89454980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T17:13:24.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.742+0000 7fcb89454980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T17:13:24.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.742+0000 7fcb89454980 20 rados->read ofs=0 len=0 2024-05-30T17:13:24.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.742+0000 7fcb89454980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T17:13:24.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.742+0000 7fcb89454980 20 rados->read ofs=0 len=0 2024-05-30T17:13:24.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.746+0000 7fcb89454980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T17:13:24.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.746+0000 7fcb89454980 20 rados->read ofs=0 len=0 2024-05-30T17:13:24.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.746+0000 7fcb89454980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T17:13:24.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.746+0000 7fcb89454980 20 zone default found 2024-05-30T17:13:24.748 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.746+0000 7fcb89454980 4 Realm: () 2024-05-30T17:13:24.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.746+0000 7fcb89454980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T17:13:24.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.746+0000 7fcb89454980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T17:13:24.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.746+0000 7fcb89454980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T17:13:24.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.746+0000 7fcb89454980 20 zonegroup default 2024-05-30T17:13:24.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.746+0000 7fcb89454980 20 rados->read ofs=0 len=0 2024-05-30T17:13:24.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.746+0000 7fcb89454980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T17:13:24.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.746+0000 7fcb89454980 20 rados->read ofs=0 len=0 2024-05-30T17:13:24.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.746+0000 7fcb89454980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T17:13:24.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.746+0000 7fcb89454980 20 rados->read ofs=0 len=0 2024-05-30T17:13:24.750 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.746+0000 7fcb89454980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T17:13:24.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.746+0000 7fcb89454980 20 started sync module instance, tier type = 2024-05-30T17:13:24.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.746+0000 7fcb89454980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T17:13:24.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.762+0000 7fcb89454980 20 add_watcher() i=0 2024-05-30T17:13:24.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.762+0000 7fcb89454980 20 add_watcher() i=1 2024-05-30T17:13:24.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.762+0000 7fcb89454980 20 add_watcher() i=2 2024-05-30T17:13:24.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.762+0000 7fcb89454980 20 add_watcher() i=3 2024-05-30T17:13:24.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.762+0000 7fcb89454980 20 add_watcher() i=4 2024-05-30T17:13:24.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.762+0000 7fcb89454980 20 add_watcher() i=5 2024-05-30T17:13:24.766 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.762+0000 7fcb89454980 20 add_watcher() i=6 2024-05-30T17:13:24.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.766+0000 7fcb89454980 20 add_watcher() i=7 2024-05-30T17:13:24.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.766+0000 7fcb89454980 2 all 8 watchers are set, enabling cache 2024-05-30T17:13:24.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.766+0000 7fca3d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T17:13:24.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.766+0000 7fcb89454980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T17:13:24.770 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.766+0000 7fcb89454980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T17:13:24.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.846+0000 7fc9b7fff640 20 reqs_thread_entry: start 2024-05-30T17:13:24.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.850+0000 7fcb89454980 20 init_complete bucket index max shards: 11 2024-05-30T17:13:24.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.850+0000 7fcb89454980 20 Filter name: none 2024-05-30T17:13:24.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.850+0000 7fc9b5ffb640 20 reqs_thread_entry: start 2024-05-30T17:13:24.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.850+0000 7fcb89454980 10 cache get: name=default.rgw.meta+users.uid+tester.client.0 : miss 2024-05-30T17:13:24.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.850+0000 7fcb89454980 20 rados->read ofs=0 len=0 2024-05-30T17:13:24.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.850+0000 7fcb89454980 20 rados_obj.operate() r=0 bl.length=482 2024-05-30T17:13:24.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.850+0000 7fcb89454980 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x13 2024-05-30T17:13:24.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.850+0000 7fcb89454980 10 adding default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2024-05-30T17:13:24.853 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.850+0000 7fcb89454980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-30T17:13:24.854 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.850+0000 7fcb89454980 10 removing key index: WMTCEIRCXEZTRMVCZBLU 2024-05-30T17:13:24.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.854+0000 7fcb89454980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.keys:WMTCEIRCXEZTRMVCZBLU, ofs0, ns] 2024-05-30T17:13:24.857 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.854+0000 7fca6a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166334749 cookie 93858207994800 notifier 7926 bl.length()=170 2024-05-30T17:13:24.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.858+0000 7fcb89454980 10 removing email index: tester.client.0_test@test.test 2024-05-30T17:13:24.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.858+0000 7fcb89454980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.email:tester.client.0_test@test.test, ofs0, ns] 2024-05-30T17:13:24.861 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.858+0000 7fca6a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166334750 cookie 93858207994800 notifier 7926 bl.length()=181 2024-05-30T17:13:24.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.862+0000 7fcb89454980 10 removing user buckets index 2024-05-30T17:13:24.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.862+0000 7fcb89454980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:users.uid:tester.client.0.buckets, ofs0, ns] 2024-05-30T17:13:24.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.866+0000 7fca69ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333677 cookie 93858208019568 notifier 7926 bl.length()=172 2024-05-30T17:13:24.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.866+0000 7fcb89454980 10 removing user index: tester.client.0 2024-05-30T17:13:24.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.870+0000 7fcb89454980 10 removing default.rgw.meta+users.uid+tester.client.0 from cache 2024-05-30T17:13:24.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.870+0000 7fcb89454980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:tester.client.0, ofs0, ns] 2024-05-30T17:13:24.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.870+0000 7fca69ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166334019 cookie 93858208016512 notifier 7926 bl.length()=164 2024-05-30T17:13:24.876 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.874+0000 7fca3d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T17:13:24.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.878+0000 7fcb89454980 20 remove_watcher() i=0 2024-05-30T17:13:24.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.878+0000 7fcb89454980 2 removed watcher, disabling cache 2024-05-30T17:13:24.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.882+0000 7fcb89454980 20 remove_watcher() i=1 2024-05-30T17:13:24.886 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.882+0000 7fcb89454980 20 remove_watcher() i=2 2024-05-30T17:13:24.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.886+0000 7fcb89454980 20 remove_watcher() i=3 2024-05-30T17:13:24.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.886+0000 7fcb89454980 20 remove_watcher() i=4 2024-05-30T17:13:24.892 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.890+0000 7fcb89454980 20 remove_watcher() i=5 2024-05-30T17:13:24.894 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.890+0000 7fcb89454980 20 remove_watcher() i=6 2024-05-30T17:13:24.895 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:24.894+0000 7fcb89454980 20 remove_watcher() i=7 2024-05-30T17:13:24.912 DEBUG:teuthology.run_tasks:Unwinding manager ragweed 2024-05-30T17:13:24.922 DEBUG:teuthology.run_tasks:Unwinding manager tox 2024-05-30T17:13:24.971 DEBUG:teuthology.orchestra.run.smithi006:> rm -rf /home/ubuntu/cephtest/tox-venv 2024-05-30T17:13:25.019 DEBUG:teuthology.run_tasks:Unwinding manager rgw 2024-05-30T17:13:25.028 DEBUG:tasks.rgw.client.0:waiting for process to exit 2024-05-30T17:13:25.028 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:13:25.046 INFO:tasks.rgw.client.0.smithi006.stdout:2024-05-30T17:13:25.034+0000 e53b640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/ceph.client.0.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends beast port=80 -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground (PID: 36192) UID: 0 2024-05-30T17:13:25.046 INFO:tasks.rgw.client.0.smithi006.stdout:2024-05-30T17:13:25.038+0000 8ea3f80 -1 shutting down 2024-05-30T17:13:33.412 INFO:tasks.rgw.client.0.smithi006.stderr:==00:01:19:04.298 36196== could not unlink /tmp/vgdb-pipe-from-vgdb-to-36196-by-root-on-??? 2024-05-30T17:13:33.413 INFO:tasks.rgw.client.0.smithi006.stderr:==00:01:19:04.298 36196== could not unlink /tmp/vgdb-pipe-to-vgdb-from-36196-by-root-on-??? 2024-05-30T17:13:33.413 INFO:tasks.rgw.client.0.smithi006.stderr:==00:01:19:04.298 36196== could not unlink /tmp/vgdb-pipe-shared-mem-vgdb-36196-by-root-on-??? 2024-05-30T17:13:37.232 INFO:tasks.rgw.client.0:Stopped 2024-05-30T17:13:37.233 DEBUG:teuthology.orchestra.run.smithi006:> rm -f /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock 2024-05-30T17:13:37.239 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f /etc/ceph/vault-root-token 2024-05-30T17:13:37.296 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f /home/ubuntu/cephtest/url_file 2024-05-30T17:13:37.352 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'process', '--include-all'] 2024-05-30T17:13:37.352 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'process', '--include-all'] 2024-05-30T17:13:37.352 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc process --include-all 2024-05-30T17:13:37.432 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setuser ceph since I am not root 2024-05-30T17:13:37.433 INFO:teuthology.orchestra.run.smithi006.stderr:ignoring --setgroup ceph since I am not root 2024-05-30T17:13:37.477 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.474+0000 7f05851d5980 20 rados->read ofs=0 len=0 2024-05-30T17:13:37.478 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.474+0000 7f05851d5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T17:13:37.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.474+0000 7f05851d5980 20 realm 2024-05-30T17:13:37.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.474+0000 7f05851d5980 20 rados->read ofs=0 len=0 2024-05-30T17:13:37.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.474+0000 7f05851d5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T17:13:37.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.474+0000 7f05851d5980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-30T17:13:37.479 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.474+0000 7f05851d5980 20 rados->read ofs=0 len=0 2024-05-30T17:13:37.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.478+0000 7f05851d5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T17:13:37.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.478+0000 7f05851d5980 20 rados->read ofs=0 len=0 2024-05-30T17:13:37.480 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.478+0000 7f05851d5980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T17:13:37.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.478+0000 7f05851d5980 20 rados->read ofs=0 len=0 2024-05-30T17:13:37.481 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.478+0000 7f05851d5980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-30T17:13:37.482 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.478+0000 7f05851d5980 20 searching for the correct realm 2024-05-30T17:13:37.485 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.482+0000 7f0582ce6640 0 -- 172.21.15.6:0/2758246988 >> [v2:172.21.15.6:6816/2704410641,v1:172.21.15.6:6817/2704410641] conn(0x7f057c027020 msgr2=0x7f057c05cea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-30T17:13:37.699 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.694+0000 7f05834e7640 0 -- 172.21.15.6:0/2758246988 >> [v2:172.21.15.204:6823/3156311527,v1:172.21.15.204:6825/3156311527] conn(0x7f0578030040 msgr2=0x7f05780990b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-30T17:13:37.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.714+0000 7f05851d5980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-30T17:13:37.718 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.714+0000 7f05851d5980 20 RGWRados::pool_iterate: got default.zone. 2024-05-30T17:13:37.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.714+0000 7f05851d5980 20 RGWRados::pool_iterate: got zone_info.601fa7ab-74a8-40e7-9e32-e5827d25af6b 2024-05-30T17:13:37.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.714+0000 7f05851d5980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-30T17:13:37.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.714+0000 7f05851d5980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-30T17:13:37.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.714+0000 7f05851d5980 20 RGWRados::pool_iterate: got zonegroup_info.f94e2a37-9139-4083-a210-14627dc61d51 2024-05-30T17:13:37.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.714+0000 7f05851d5980 20 rados->read ofs=0 len=0 2024-05-30T17:13:37.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.714+0000 7f05851d5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T17:13:37.719 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.714+0000 7f05851d5980 20 rados->read ofs=0 len=0 2024-05-30T17:13:37.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.714+0000 7f05851d5980 20 rados_obj.operate() r=0 bl.length=46 2024-05-30T17:13:37.720 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.714+0000 7f05851d5980 20 rados->read ofs=0 len=0 2024-05-30T17:13:37.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.714+0000 7f05851d5980 20 rados_obj.operate() r=0 bl.length=470 2024-05-30T17:13:37.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.718+0000 7f05851d5980 20 zone default found 2024-05-30T17:13:37.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.718+0000 7f05851d5980 4 Realm: () 2024-05-30T17:13:37.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.718+0000 7f05851d5980 4 ZoneGroup: default (f94e2a37-9139-4083-a210-14627dc61d51) 2024-05-30T17:13:37.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.718+0000 7f05851d5980 4 Zone: default (601fa7ab-74a8-40e7-9e32-e5827d25af6b) 2024-05-30T17:13:37.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.718+0000 7f05851d5980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-30T17:13:37.721 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.718+0000 7f05851d5980 20 zonegroup default 2024-05-30T17:13:37.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.718+0000 7f05851d5980 20 rados->read ofs=0 len=0 2024-05-30T17:13:37.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.718+0000 7f05851d5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T17:13:37.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.718+0000 7f05851d5980 20 rados->read ofs=0 len=0 2024-05-30T17:13:37.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.718+0000 7f05851d5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T17:13:37.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.718+0000 7f05851d5980 20 rados->read ofs=0 len=0 2024-05-30T17:13:37.722 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.718+0000 7f05851d5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-30T17:13:37.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.718+0000 7f05851d5980 20 started sync module instance, tier type = 2024-05-30T17:13:37.723 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.718+0000 7f05851d5980 20 started zone id=601fa7ab-74a8-40e7-9e32-e5827d25af6b (name=default) with tier type = 2024-05-30T17:13:37.744 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.742+0000 7f05851d5980 20 add_watcher() i=0 2024-05-30T17:13:37.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.742+0000 7f05851d5980 20 add_watcher() i=1 2024-05-30T17:13:37.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.742+0000 7f05851d5980 20 add_watcher() i=2 2024-05-30T17:13:37.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.742+0000 7f05851d5980 20 add_watcher() i=3 2024-05-30T17:13:37.745 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.742+0000 7f05851d5980 20 add_watcher() i=4 2024-05-30T17:13:37.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.742+0000 7f05851d5980 20 add_watcher() i=5 2024-05-30T17:13:37.746 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.742+0000 7f05851d5980 20 add_watcher() i=6 2024-05-30T17:13:37.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.742+0000 7f05851d5980 20 add_watcher() i=7 2024-05-30T17:13:37.747 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.742+0000 7f05851d5980 2 all 8 watchers are set, enabling cache 2024-05-30T17:13:37.751 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.750+0000 7f04397e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T17:13:37.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.750+0000 7f05851d5980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-30T17:13:37.752 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.750+0000 7f05851d5980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-30T17:13:37.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.838+0000 7f03b3fff640 20 reqs_thread_entry: start 2024-05-30T17:13:37.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.838+0000 7f05851d5980 20 init_complete bucket index max shards: 11 2024-05-30T17:13:37.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.838+0000 7f05851d5980 20 Filter name: none 2024-05-30T17:13:37.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.838+0000 7f03b1ffb640 20 reqs_thread_entry: start 2024-05-30T17:13:37.843 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.842+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-05-30T17:13:37.845 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.842+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:37.846 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.842+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:37.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.842+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-05-30T17:13:37.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.842+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='_4TR98QE2ViHtewklPu2GU5YLkcHV_db', time=2024-05-30T18:37:35.724873+0000, chain.objs.size()=2 2024-05-30T17:13:37.848 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.842+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.150__shadow_.Du1eVkcah0jbzBRZz-6d4sr_Chy3dNM_0 2024-05-30T17:13:37.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.842+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.150__shadow_.Du1eVkcah0jbzBRZz-6d4sr_Chy3dNM_1 2024-05-30T17:13:37.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.842+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='_rIDWGz1uLB3W9WepRUjSbfnwWfR7z7S', time=2024-05-30T18:37:48.827538+0000, chain.objs.size()=2 2024-05-30T17:13:37.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.842+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.151__shadow_.XtsJzAzgHaY2N9b0e8vP5uUmxYQlqHv_0 2024-05-30T17:13:37.849 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.842+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.151__shadow_.XtsJzAzgHaY2N9b0e8vP5uUmxYQlqHv_1 2024-05-30T17:13:37.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.842+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='2~olRSpL5olBKDMSwDJp_dWUWRc9G1JiP', time=2024-05-30T18:58:25.842714+0000, chain.objs.size()=2 2024-05-30T17:13:37.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.842+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.519__multipart_mymultipart.2~olRSpL5olBKDMSwDJp_dWUWRc9G1JiP.1 2024-05-30T17:13:37.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.846+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.519__multipart_mymultipart.2~olRSpL5olBKDMSwDJp_dWUWRc9G1JiP.2 2024-05-30T17:13:37.850 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.846+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.4749525207632108562', time=2024-05-30T19:03:45.269321+0000, chain.objs.size()=6 2024-05-30T17:13:37.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.846+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.602__multipart_multipart_enc.2~aR0yIGfg0E8hZGInRGYm1AAuloPFt0w.1 2024-05-30T17:13:37.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.846+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.602__multipart_multipart_enc.2~aR0yIGfg0E8hZGInRGYm1AAuloPFt0w.2 2024-05-30T17:13:37.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.846+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.602__multipart_multipart_enc.2~aR0yIGfg0E8hZGInRGYm1AAuloPFt0w.3 2024-05-30T17:13:37.851 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.846+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.602__multipart_multipart_enc.2~aR0yIGfg0E8hZGInRGYm1AAuloPFt0w.4 2024-05-30T17:13:37.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.846+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.602__multipart_multipart_enc.2~aR0yIGfg0E8hZGInRGYm1AAuloPFt0w.5 2024-05-30T17:13:37.852 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.846+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.602__multipart_multipart_enc.2~aR0yIGfg0E8hZGInRGYm1AAuloPFt0w.6 2024-05-30T17:13:37.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.862+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:37.869 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.866+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-05-30T17:13:37.872 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.870+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:37.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.870+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:37.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.870+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T17:13:37.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.870+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15742787892371305442', time=2024-05-30T19:06:10.144578+0000, chain.objs.size()=1 2024-05-30T17:13:37.875 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.870+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.634__shadow_.aurVlZ17WKPvrXR-h6tnAiFDiKHTv2g_1 2024-05-30T17:13:37.879 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.874+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:37.884 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.882+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-05-30T17:13:37.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.882+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:37.887 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.886+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:37.888 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.886+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T17:13:37.889 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.886+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.877452246033577873', time=2024-05-30T18:55:24.057158+0000, chain.objs.size()=1 2024-05-30T17:13:37.889 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.886+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.508__multipart_mymultipart.2~chLzr7L_CZflDN23wC-xDhT6KmjWA8F.1 2024-05-30T17:13:37.891 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.890+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:37.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.894+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-05-30T17:13:37.901 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.898+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:37.902 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.898+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:37.903 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.902+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-30T17:13:37.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.902+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16715467775355810078', time=2024-05-30T18:55:26.824370+0000, chain.objs.size()=1 2024-05-30T17:13:37.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.902+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.508__multipart_mymultipart.2~5MgcZJh3g_UTPYUf0cZj9Eo6lVcBNlL.1 2024-05-30T17:13:37.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.902+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.4782010479199129465', time=2024-05-30T19:07:57.120507+0000, chain.objs.size()=1 2024-05-30T17:13:37.904 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.902+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.696__multipart_file1.2~zrnOw2HzTmaqCsZapQz-EsRdpDbISlh.1 2024-05-30T17:13:37.907 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.906+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:37.912 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.910+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-05-30T17:13:37.915 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.914+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:37.917 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.914+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:37.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.914+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-05-30T17:13:37.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.914+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.2_b546df9f00570ec354d090c655e79a27', time=2024-05-30T18:55:08.545258+0000, chain.objs.size()=6 2024-05-30T17:13:37.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.914+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.1 2024-05-30T17:13:37.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.914+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.2 2024-05-30T17:13:37.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.914+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.3 2024-05-30T17:13:37.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.914+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.4 2024-05-30T17:13:37.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.914+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.5 2024-05-30T17:13:37.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.914+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.6 2024-05-30T17:13:37.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.914+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17865351391998543059', time=2024-05-30T18:56:46.726229+0000, chain.objs.size()=6 2024-05-30T17:13:37.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.914+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.2~l0doLYA3bTTDgeI-B7Q5iUhXZJ_s3Ro.1 2024-05-30T17:13:37.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.914+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.d7Abem0cRiLH6AL7rtqlEnCmDctPJuQU.2 2024-05-30T17:13:37.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.914+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.2~l0doLYA3bTTDgeI-B7Q5iUhXZJ_s3Ro.3 2024-05-30T17:13:37.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.914+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.2~l0doLYA3bTTDgeI-B7Q5iUhXZJ_s3Ro.4 2024-05-30T17:13:37.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.914+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.2~l0doLYA3bTTDgeI-B7Q5iUhXZJ_s3Ro.5 2024-05-30T17:13:37.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.914+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.2~l0doLYA3bTTDgeI-B7Q5iUhXZJ_s3Ro.6 2024-05-30T17:13:37.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.922+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.6325485337466245060', time=2024-05-30T18:58:57.216225+0000, chain.objs.size()=4 2024-05-30T17:13:37.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.922+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.525__multipart_mymultipart.2~sRwU-A3fEHtYYMUI9qM1cZCrjb0dZ8Z.1 2024-05-30T17:13:37.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.922+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.525__multipart_mymultipart.2~sRwU-A3fEHtYYMUI9qM1cZCrjb0dZ8Z.2 2024-05-30T17:13:37.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.922+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.525__multipart_mymultipart.s1_j2BXwFJzHEKsDKWx-vvlvTjDmuidq.3 2024-05-30T17:13:37.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.922+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.525__multipart_mymultipart.2~sRwU-A3fEHtYYMUI9qM1cZCrjb0dZ8Z.4 2024-05-30T17:13:37.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.930+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:37.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.938+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-05-30T17:13:37.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.942+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:37.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.942+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:37.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.942+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=6, truncated=0, next_marker='' 2024-05-30T17:13:37.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.942+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='_W8kyRI4LzdK1eRYZ9s1zvXW9p8zHSF0', time=2024-05-30T18:37:39.181491+0000, chain.objs.size()=1 2024-05-30T17:13:37.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.942+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.151__shadow_.Ph6YNwW1ct-cOb9_eStEINsUmGvUumC_0 2024-05-30T17:13:37.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.942+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='2~n-_cGl5798fRcpDhcK4w0fHsvcnboZy', time=2024-05-30T18:58:32.037981+0000, chain.objs.size()=1 2024-05-30T17:13:37.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.942+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.521__multipart_mymultipart.2~n-_cGl5798fRcpDhcK4w0fHsvcnboZy.1 2024-05-30T17:13:37.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.942+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='2~osboXhCICCzPBTW2aWA3UuodKB-EgUo', time=2024-05-30T18:58:32.236727+0000, chain.objs.size()=1 2024-05-30T17:13:37.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.942+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.521__multipart_mymultipart2.2~osboXhCICCzPBTW2aWA3UuodKB-EgUo.1 2024-05-30T17:13:37.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.942+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.10893887015370536794', time=2024-05-30T19:02:07.582308+0000, chain.objs.size()=4 2024-05-30T17:13:37.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.942+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.567__multipart_testobj.2~FAr5FDsDpLNUESt05p8SBuJj-ps2ZyY.1 2024-05-30T17:13:37.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.942+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.567__multipart_testobj.2~FAr5FDsDpLNUESt05p8SBuJj-ps2ZyY.2 2024-05-30T17:13:37.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.942+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.567__multipart_testobj.2~FAr5FDsDpLNUESt05p8SBuJj-ps2ZyY.3 2024-05-30T17:13:37.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.942+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.567__multipart_testobj.2~FAr5FDsDpLNUESt05p8SBuJj-ps2ZyY.4 2024-05-30T17:13:37.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.942+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.14130154766301508307', time=2024-05-30T19:02:09.117829+0000, chain.objs.size()=4 2024-05-30T17:13:37.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.942+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.567__multipart_testobj.2~bdl6GnQfGiVqDFu_cu4C9YotA2Splpm.1 2024-05-30T17:13:37.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.942+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.567__multipart_testobj.2~bdl6GnQfGiVqDFu_cu4C9YotA2Splpm.2 2024-05-30T17:13:37.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.942+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.567__multipart_testobj.2~bdl6GnQfGiVqDFu_cu4C9YotA2Splpm.3 2024-05-30T17:13:37.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.942+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.567__multipart_testobj.2~bdl6GnQfGiVqDFu_cu4C9YotA2Splpm.4 2024-05-30T17:13:37.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.942+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.4209446320028204061', time=2024-05-30T19:06:49.439992+0000, chain.objs.size()=6 2024-05-30T17:13:37.949 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.942+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.646__multipart_baz.2~mRwYSC1uHUqv3micKu4DoahRfisJ_hL.1 2024-05-30T17:13:37.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.946+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.646__multipart_baz.2~mRwYSC1uHUqv3micKu4DoahRfisJ_hL.2 2024-05-30T17:13:37.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.950+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.646__multipart_baz.2~mRwYSC1uHUqv3micKu4DoahRfisJ_hL.3 2024-05-30T17:13:37.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.954+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.646__multipart_baz.2~mRwYSC1uHUqv3micKu4DoahRfisJ_hL.4 2024-05-30T17:13:37.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.954+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.646__multipart_baz.2~mRwYSC1uHUqv3micKu4DoahRfisJ_hL.5 2024-05-30T17:13:37.958 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.954+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.646__multipart_baz.2~mRwYSC1uHUqv3micKu4DoahRfisJ_hL.6 2024-05-30T17:13:37.964 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.962+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:37.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.966+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-05-30T17:13:37.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.970+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:37.974 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.970+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:37.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.974+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-05-30T17:13:37.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.974+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.7223728937704388998', time=2024-05-30T18:37:47.124673+0000, chain.objs.size()=3 2024-05-30T17:13:37.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.974+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.151__multipart_foo-FROZEN.2~hYWgCKHNoHZtgRqa2FdWDKJ42tzh4Dj.1 2024-05-30T17:13:37.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.974+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.151__multipart_foo-FROZEN.2~hYWgCKHNoHZtgRqa2FdWDKJ42tzh4Dj.2 2024-05-30T17:13:37.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.974+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.151__multipart_foo-FROZEN.2~hYWgCKHNoHZtgRqa2FdWDKJ42tzh4Dj.3 2024-05-30T17:13:37.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.974+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.8802561832582199787', time=2024-05-30T18:55:07.862655+0000, chain.objs.size()=6 2024-05-30T17:13:37.976 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.974+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.1 2024-05-30T17:13:37.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.974+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.2 2024-05-30T17:13:37.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.974+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.3 2024-05-30T17:13:37.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.974+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.4 2024-05-30T17:13:37.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.974+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.5 2024-05-30T17:13:37.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.974+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.6 2024-05-30T17:13:37.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.974+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='2~b-0hnWQ1OWRKqykHt86jEKu9496mShH', time=2024-05-30T18:57:05.748508+0000, chain.objs.size()=1 2024-05-30T17:13:37.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.974+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.2~b-0hnWQ1OWRKqykHt86jEKu9496mShH.2 2024-05-30T17:13:37.977 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.974+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='2~b-0hnWQ1OWRKqykHt86jEKu9496mShH', time=2024-05-30T18:57:05.758545+0000, chain.objs.size()=1 2024-05-30T17:13:37.978 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.974+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.2~b-0hnWQ1OWRKqykHt86jEKu9496mShH.3 2024-05-30T17:13:37.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.982+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:37.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.986+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-05-30T17:13:37.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.990+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:37.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.990+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:37.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.990+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T17:13:37.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.990+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.8189865895765806651', time=2024-05-30T18:37:38.695261+0000, chain.objs.size()=3 2024-05-30T17:13:37.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.990+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.151__multipart_foo-STANDARD.2~uX6cY7GDcYcPHTlV9_jSJ6TJQR1vUEO.1 2024-05-30T17:13:37.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.990+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.151__multipart_foo-STANDARD.2~uX6cY7GDcYcPHTlV9_jSJ6TJQR1vUEO.2 2024-05-30T17:13:37.994 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.990+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.151__multipart_foo-STANDARD.2~uX6cY7GDcYcPHTlV9_jSJ6TJQR1vUEO.3 2024-05-30T17:13:37.997 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:37.994+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:38.004 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.002+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-05-30T17:13:38.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.006+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:38.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.006+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:38.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.006+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=6, truncated=0, next_marker='' 2024-05-30T17:13:38.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.006+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='_fSpwa7kUaCLoByo1LyQ-40bWDQnMGSZ', time=2024-05-30T18:37:28.853384+0000, chain.objs.size()=1 2024-05-30T17:13:38.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.006+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.150__shadow_.S5L7pUsetWlef_ZyhJop_fPMqmG0Tae_0 2024-05-30T17:13:38.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.006+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.8536047012553968822', time=2024-05-30T18:37:57.068505+0000, chain.objs.size()=2 2024-05-30T17:13:38.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.006+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.154__multipart_foo.2~YluqmZD1E_Lhi7ulHMBNZ3gole_dSs6.1 2024-05-30T17:13:38.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.006+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.154__multipart_foo.2~YluqmZD1E_Lhi7ulHMBNZ3gole_dSs6.2 2024-05-30T17:13:38.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.006+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.11280741734905977669', time=2024-05-30T18:55:31.258337+0000, chain.objs.size()=1 2024-05-30T17:13:38.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.006+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.508__multipart_mymultipart.2~FOAZWAyfUYiP86-8CutamPIkx-Y8Bbc.1 2024-05-30T17:13:38.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.006+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.4894495741178988691', time=2024-05-30T18:57:42.415727+0000, chain.objs.size()=6 2024-05-30T17:13:38.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.006+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.b_rr1q47UCO6wNZzpy3xPBnHQRdpG9yF.1 2024-05-30T17:13:38.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.006+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.pMeAx1ko-N-_Bns7xOP3WK6gKlXtsHqX.2 2024-05-30T17:13:38.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.006+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.NYU9MCPyVnqD57xGD9wOHovm3iHAr104.3 2024-05-30T17:13:38.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.006+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.u-GhkWggCgO47BuoFD-lel63XzvCjPsP.4 2024-05-30T17:13:38.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.006+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.SrHyc58TyISEzYIDOW3Dv5SaAK2EfHPM.5 2024-05-30T17:13:38.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.006+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.9hNS5rUsXS_vJc3uXGJ8sVJtUyFU8TNU.6 2024-05-30T17:13:38.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.006+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='2~CvqDn5Omktl9a6wDn75phVKaFPiLDdC', time=2024-05-30T19:05:33.271919+0000, chain.objs.size()=6 2024-05-30T17:13:38.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.006+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.612__multipart_multipart_enc.2~CvqDn5Omktl9a6wDn75phVKaFPiLDdC.1 2024-05-30T17:13:38.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.006+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.612__multipart_multipart_enc.2~CvqDn5Omktl9a6wDn75phVKaFPiLDdC.2 2024-05-30T17:13:38.015 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.014+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.612__multipart_multipart_enc.2~CvqDn5Omktl9a6wDn75phVKaFPiLDdC.3 2024-05-30T17:13:38.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.014+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.612__multipart_multipart_enc.2~CvqDn5Omktl9a6wDn75phVKaFPiLDdC.4 2024-05-30T17:13:38.018 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.014+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.612__multipart_multipart_enc.2~CvqDn5Omktl9a6wDn75phVKaFPiLDdC.5 2024-05-30T17:13:38.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.014+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.612__multipart_multipart_enc.2~CvqDn5Omktl9a6wDn75phVKaFPiLDdC.6 2024-05-30T17:13:38.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.014+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.6923297242849549733', time=2024-05-30T19:08:56.375981+0000, chain.objs.size()=10 2024-05-30T17:13:38.019 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.014+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.718__multipart_foo.2~esDPzMJMJLb6Pk50BfQDLs5Ay8N3BwO.1 2024-05-30T17:13:38.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.018+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.718__multipart_foo.2~esDPzMJMJLb6Pk50BfQDLs5Ay8N3BwO.2 2024-05-30T17:13:38.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.018+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.718__multipart_foo.2~esDPzMJMJLb6Pk50BfQDLs5Ay8N3BwO.3 2024-05-30T17:13:38.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.018+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.718__multipart_foo.2~esDPzMJMJLb6Pk50BfQDLs5Ay8N3BwO.4 2024-05-30T17:13:38.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.018+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.718__multipart_foo.2~esDPzMJMJLb6Pk50BfQDLs5Ay8N3BwO.5 2024-05-30T17:13:38.021 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.018+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.718__multipart_foo.2~esDPzMJMJLb6Pk50BfQDLs5Ay8N3BwO.6 2024-05-30T17:13:38.022 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.018+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.718__multipart_foo.2~esDPzMJMJLb6Pk50BfQDLs5Ay8N3BwO.7 2024-05-30T17:13:38.024 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.022+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.718__multipart_foo.2~esDPzMJMJLb6Pk50BfQDLs5Ay8N3BwO.8 2024-05-30T17:13:38.028 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.026+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.718__multipart_foo.2~esDPzMJMJLb6Pk50BfQDLs5Ay8N3BwO.9 2024-05-30T17:13:38.029 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.026+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.718__multipart_foo.2~esDPzMJMJLb6Pk50BfQDLs5Ay8N3BwO.10 2024-05-30T17:13:38.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.030+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:38.039 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.038+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-05-30T17:13:38.042 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.038+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:38.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.042+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:38.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.042+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-05-30T17:13:38.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.042+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.3105234742287453778', time=2024-05-30T18:55:11.775581+0000, chain.objs.size()=1 2024-05-30T17:13:38.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.042+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.501__multipart_mymultipart.2~qiJ3b48zYGWV2J2NDm1i97xTAWNJ_DH.1 2024-05-30T17:13:38.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.042+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.6182427614785476468', time=2024-05-30T18:55:29.580224+0000, chain.objs.size()=1 2024-05-30T17:13:38.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.042+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.508__multipart_mymultipart.2~Yjn0IdcaBYJb8LKMue2TvG-xyjyMU3k.1 2024-05-30T17:13:38.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.042+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.1254967037270772473', time=2024-05-30T18:57:53.828253+0000, chain.objs.size()=2 2024-05-30T17:13:38.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.042+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.513__multipart_mymultipart.2~ir92k8WMStNCYnOyk_8aeNg8PAJBRe3.1 2024-05-30T17:13:38.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.042+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.513__multipart_mymultipart.2~ir92k8WMStNCYnOyk_8aeNg8PAJBRe3.2 2024-05-30T17:13:38.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.042+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.1185806961272709507', time=2024-05-30T19:06:48.729358+0000, chain.objs.size()=6 2024-05-30T17:13:38.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.042+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.645__multipart_bar.2~SY5HQjCi12lodRLx5tiqMSGRpP-1tLR.1 2024-05-30T17:13:38.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.042+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.645__multipart_bar.2~SY5HQjCi12lodRLx5tiqMSGRpP-1tLR.2 2024-05-30T17:13:38.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.042+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.645__multipart_bar.2~SY5HQjCi12lodRLx5tiqMSGRpP-1tLR.3 2024-05-30T17:13:38.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.042+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.645__multipart_bar.2~SY5HQjCi12lodRLx5tiqMSGRpP-1tLR.4 2024-05-30T17:13:38.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.042+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.645__multipart_bar.2~SY5HQjCi12lodRLx5tiqMSGRpP-1tLR.5 2024-05-30T17:13:38.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.042+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.645__multipart_bar.2~SY5HQjCi12lodRLx5tiqMSGRpP-1tLR.6 2024-05-30T17:13:38.052 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.050+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:38.057 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.054+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-05-30T17:13:38.060 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.058+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:38.061 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.058+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:38.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.058+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-05-30T17:13:38.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.058+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='_kqGkqrIA2aWBjGwULaovougYWs5pE9L', time=2024-05-30T18:37:35.635477+0000, chain.objs.size()=2 2024-05-30T17:13:38.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.058+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.150__shadow_.WTbnZlYvvYxFGZ1NGRTnFnBtM2zHzIR_0 2024-05-30T17:13:38.062 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.058+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.150__shadow_.WTbnZlYvvYxFGZ1NGRTnFnBtM2zHzIR_1 2024-05-30T17:13:38.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.058+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.9587933957916498815', time=2024-05-30T19:00:10.676431+0000, chain.objs.size()=1 2024-05-30T17:13:38.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.058+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.552__shadow_.1DjR54jvA4O26RjNjs0ldFMy0X-FzE4_1 2024-05-30T17:13:38.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.058+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16808398379249559550', time=2024-05-30T19:06:07.416096+0000, chain.objs.size()=1 2024-05-30T17:13:38.063 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.058+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.633__shadow_.ytvEzg9mJggB3ZU59ia8K6QQp6Vb5jy_1 2024-05-30T17:13:38.067 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.066+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:38.072 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.070+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-05-30T17:13:38.074 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.070+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:38.075 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.074+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:38.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.074+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T17:13:38.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.074+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12704584222050181870', time=2024-05-30T18:55:46.331825+0000, chain.objs.size()=6 2024-05-30T17:13:38.076 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.074+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.509__multipart_mymultipart.2~Xk1fgFbWvC8TVFzwD0nvZMDPaDyHjE2.1 2024-05-30T17:13:38.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.074+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.509__multipart_mymultipart.2~Xk1fgFbWvC8TVFzwD0nvZMDPaDyHjE2.2 2024-05-30T17:13:38.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.074+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.509__multipart_mymultipart.2~Xk1fgFbWvC8TVFzwD0nvZMDPaDyHjE2.3 2024-05-30T17:13:38.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.074+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.509__multipart_mymultipart.2~Xk1fgFbWvC8TVFzwD0nvZMDPaDyHjE2.4 2024-05-30T17:13:38.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.074+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.509__multipart_mymultipart.2~Xk1fgFbWvC8TVFzwD0nvZMDPaDyHjE2.5 2024-05-30T17:13:38.077 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.074+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.509__multipart_mymultipart.2~Xk1fgFbWvC8TVFzwD0nvZMDPaDyHjE2.6 2024-05-30T17:13:38.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.078+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:38.086 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.082+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-05-30T17:13:38.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.086+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:38.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.086+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:38.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.090+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T17:13:38.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.090+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.3619464156199498443', time=2024-05-30T18:37:19.375674+0000, chain.objs.size()=2 2024-05-30T17:13:38.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.090+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.148__shadow_.q7jak4gC6mMhq-uhKpUGFXk-AE5kR1N_0 2024-05-30T17:13:38.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.090+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.148__shadow_.q7jak4gC6mMhq-uhKpUGFXk-AE5kR1N_1 2024-05-30T17:13:38.096 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.094+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:38.101 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.098+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-05-30T17:13:38.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.102+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:38.104 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.102+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:38.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.102+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-30T17:13:38.105 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.102+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.8667309004875789040', time=2024-05-30T18:58:18.678331+0000, chain.objs.size()=4 2024-05-30T17:13:38.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.102+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.517__multipart_mymultipart.2~lFllfhNUPEj7UH14YTgffAK95tw8cBU.1 2024-05-30T17:13:38.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.102+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.517__multipart_mymultipart.2~lFllfhNUPEj7UH14YTgffAK95tw8cBU.2 2024-05-30T17:13:38.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.102+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.517__multipart_mymultipart.2~lFllfhNUPEj7UH14YTgffAK95tw8cBU.3 2024-05-30T17:13:38.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.102+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.517__multipart_mymultipart.2~lFllfhNUPEj7UH14YTgffAK95tw8cBU.4 2024-05-30T17:13:38.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.102+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.5964871449957718287', time=2024-05-30T19:06:12.784971+0000, chain.objs.size()=1 2024-05-30T17:13:38.106 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.102+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.635__shadow_.LGAA9bDFPVV4L1Ml6j3WBg8mG_TJE0u_1 2024-05-30T17:13:38.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.110+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:38.117 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.114+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-05-30T17:13:38.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.118+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:38.121 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.118+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:38.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.118+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T17:13:38.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.118+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='_LD3zUy7VXqlOIOInYZMUpQOzghqxAYH', time=2024-05-30T18:37:34.976053+0000, chain.objs.size()=1 2024-05-30T17:13:38.122 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.118+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.150__shadow_.XQfSybRpmh4zN6Kys2fwY-PFtZlDpsI_1 2024-05-30T17:13:38.126 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.122+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:38.132 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.130+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-05-30T17:13:38.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.130+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:38.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.134+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:38.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.134+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-30T17:13:38.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.134+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.2_3a104a19adca4e0d2727789e1afb51d1', time=2024-05-30T18:55:07.824514+0000, chain.objs.size()=6 2024-05-30T17:13:38.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.134+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.1 2024-05-30T17:13:38.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.134+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.2 2024-05-30T17:13:38.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.134+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.3 2024-05-30T17:13:38.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.134+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.4 2024-05-30T17:13:38.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.134+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.5 2024-05-30T17:13:38.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.134+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.6 2024-05-30T17:13:38.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.134+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15866550351398198503', time=2024-05-30T18:57:46.786082+0000, chain.objs.size()=2 2024-05-30T17:13:38.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.134+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.513__multipart_mymultipart.2~4rDwfBirDbmzEo-Z6f8uHElSbEK0cy_.1 2024-05-30T17:13:38.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.134+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.513__multipart_mymultipart.2~4rDwfBirDbmzEo-Z6f8uHElSbEK0cy_.2 2024-05-30T17:13:38.145 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.142+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:38.149 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.146+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-05-30T17:13:38.153 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.150+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:38.154 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.150+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:38.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.154+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-05-30T17:13:38.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.154+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='2~UHxAIBJcc1x_ID-altuu2PMqwGbs8sh', time=2024-05-30T18:56:46.660974+0000, chain.objs.size()=1 2024-05-30T17:13:38.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.154+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.2~UHxAIBJcc1x_ID-altuu2PMqwGbs8sh.3 2024-05-30T17:13:38.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.154+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.14861645151776525420', time=2024-05-30T18:58:00.916297+0000, chain.objs.size()=2 2024-05-30T17:13:38.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.154+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.515__multipart_mymultipart.2~-plpGbNpuazCf16jlUBpexW1wX4QYHi.1 2024-05-30T17:13:38.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.154+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.515__multipart_mymultipart.2~-plpGbNpuazCf16jlUBpexW1wX4QYHi.2 2024-05-30T17:13:38.156 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.154+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12589433010057791324', time=2024-05-30T18:58:08.551496+0000, chain.objs.size()=3 2024-05-30T17:13:38.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.154+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.515__multipart_mymultipart.2~JuV3xS6CbsW85GVOaH1v4ov8A2SzEeh.1 2024-05-30T17:13:38.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.154+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.515__multipart_mymultipart.2~JuV3xS6CbsW85GVOaH1v4ov8A2SzEeh.2 2024-05-30T17:13:38.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.154+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.515__multipart_mymultipart.2~JuV3xS6CbsW85GVOaH1v4ov8A2SzEeh.3 2024-05-30T17:13:38.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.154+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='2~2CL5SdlxD4R_iM0IUPSrwsMKJ__FX2F', time=2024-05-30T18:58:13.137931+0000, chain.objs.size()=10 2024-05-30T17:13:38.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.154+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.516__multipart_mymultipart.2~2CL5SdlxD4R_iM0IUPSrwsMKJ__FX2F.1 2024-05-30T17:13:38.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.154+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.516__multipart_mymultipart.2~2CL5SdlxD4R_iM0IUPSrwsMKJ__FX2F.2 2024-05-30T17:13:38.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.154+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.516__multipart_mymultipart.2~2CL5SdlxD4R_iM0IUPSrwsMKJ__FX2F.3 2024-05-30T17:13:38.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.154+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.516__multipart_mymultipart.2~2CL5SdlxD4R_iM0IUPSrwsMKJ__FX2F.4 2024-05-30T17:13:38.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.154+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.516__multipart_mymultipart.2~2CL5SdlxD4R_iM0IUPSrwsMKJ__FX2F.5 2024-05-30T17:13:38.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.154+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.516__multipart_mymultipart.2~2CL5SdlxD4R_iM0IUPSrwsMKJ__FX2F.6 2024-05-30T17:13:38.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.162+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.516__multipart_mymultipart.2~2CL5SdlxD4R_iM0IUPSrwsMKJ__FX2F.7 2024-05-30T17:13:38.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.162+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.516__multipart_mymultipart.2~2CL5SdlxD4R_iM0IUPSrwsMKJ__FX2F.8 2024-05-30T17:13:38.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.162+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.516__multipart_mymultipart.2~2CL5SdlxD4R_iM0IUPSrwsMKJ__FX2F.9 2024-05-30T17:13:38.164 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.162+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.516__multipart_mymultipart.2~2CL5SdlxD4R_iM0IUPSrwsMKJ__FX2F.10 2024-05-30T17:13:38.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.166+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:38.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.174+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-05-30T17:13:38.179 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.178+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:38.180 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.178+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:38.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.178+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-05-30T17:13:38.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.178+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='_oONZ9_S4NHCAgdE3BZ9vrNCba8W0s_I', time=2024-05-30T18:37:35.798648+0000, chain.objs.size()=1 2024-05-30T17:13:38.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.178+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.150__shadow_.FQTh98_jRmF7RJFtmmtUEyekHjhHtLB_0 2024-05-30T17:13:38.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.178+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='_Ahy69Z3757GZ9a2u55bmA_qT5lPxGV1', time=2024-05-30T18:37:43.602383+0000, chain.objs.size()=2 2024-05-30T17:13:38.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.178+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.151__shadow_.xcb6pD2KRgOCYMtMDB8HLqe9ixr1qkx_1 2024-05-30T17:13:38.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.178+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.151__shadow_.xcb6pD2KRgOCYMtMDB8HLqe9ixr1qkx_2 2024-05-30T17:13:38.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.178+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='2~vizXKVZg1FMC6unYshwnxH80DsY-V_W', time=2024-05-30T18:56:13.743243+0000, chain.objs.size()=1 2024-05-30T17:13:38.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.178+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.2~vizXKVZg1FMC6unYshwnxH80DsY-V_W.1 2024-05-30T17:13:38.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.178+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.1421180436992495529', time=2024-05-30T19:00:01.379172+0000, chain.objs.size()=1 2024-05-30T17:13:38.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.178+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.550__multipart_mymultipart.2~BWCfoPCLm_PLV0CRfuc-vMcaFPNUqzx.1 2024-05-30T17:13:38.187 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.186+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:38.191 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.190+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-05-30T17:13:38.194 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.190+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:38.196 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.194+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:38.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.194+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-05-30T17:13:38.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.194+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='_MdQ81B8QYEWd-jMv7dZ20ckgnFLK5K0', time=2024-05-30T18:37:48.900508+0000, chain.objs.size()=2 2024-05-30T17:13:38.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.194+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.151__shadow_.lIoFl2FgRbmCCCY0jvYjpQkPPmPCSKn_0 2024-05-30T17:13:38.197 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.194+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.151__shadow_.lIoFl2FgRbmCCCY0jvYjpQkPPmPCSKn_1 2024-05-30T17:13:38.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.194+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='_VlhViqsAF-XrqnC7l1KTsQ2YRvnDFO4', time=2024-05-30T18:37:48.975210+0000, chain.objs.size()=1 2024-05-30T17:13:38.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.194+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.151__shadow_.EGCMBzbrikZQtIevHN-03yJTaErbRq9_0 2024-05-30T17:13:38.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.194+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='_1efU0Wb9KEYWI5QnYBKu_0fE23r62uz', time=2024-05-30T18:37:52.691450+0000, chain.objs.size()=1 2024-05-30T17:13:38.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.194+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.153__shadow_.Ffpzfoo1wZZxusDIipMcNQ86nXK9BCg_0 2024-05-30T17:13:38.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.194+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.14986770919253582766', time=2024-05-30T19:02:10.381240+0000, chain.objs.size()=4 2024-05-30T17:13:38.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.194+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.567__multipart_testobj.2~m1eaPESGhoV70TelNTiyo4gGHELy3jI.1 2024-05-30T17:13:38.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.194+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.567__multipart_testobj.2~m1eaPESGhoV70TelNTiyo4gGHELy3jI.2 2024-05-30T17:13:38.198 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.194+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.567__multipart_testobj.2~m1eaPESGhoV70TelNTiyo4gGHELy3jI.3 2024-05-30T17:13:38.199 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.194+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.567__multipart_testobj.2~m1eaPESGhoV70TelNTiyo4gGHELy3jI.4 2024-05-30T17:13:38.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.202+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:38.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.206+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-05-30T17:13:38.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.210+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:38.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.210+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:38.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.210+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-05-30T17:13:38.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.210+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.7848977628854469768', time=2024-05-30T18:37:42.552044+0000, chain.objs.size()=3 2024-05-30T17:13:38.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.210+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.151__multipart_foo-LUKEWARM.2~Xmhx_LPop9YlBTl0pSQwKN0kKF3IJKM.1 2024-05-30T17:13:38.214 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.210+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.151__multipart_foo-LUKEWARM.2~Xmhx_LPop9YlBTl0pSQwKN0kKF3IJKM.2 2024-05-30T17:13:38.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.210+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.151__multipart_foo-LUKEWARM.2~Xmhx_LPop9YlBTl0pSQwKN0kKF3IJKM.3 2024-05-30T17:13:38.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.210+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='_TOIlujHFVhkvWTz6RT1uh1_neofwR27', time=2024-05-30T18:37:48.144074+0000, chain.objs.size()=2 2024-05-30T17:13:38.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.210+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.151__shadow_._VcVDMVlD5jKUMlejGzJ5Z8PnEDkL2U_1 2024-05-30T17:13:38.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.210+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.151__shadow_._VcVDMVlD5jKUMlejGzJ5Z8PnEDkL2U_2 2024-05-30T17:13:38.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.210+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='2~1EoJZEjwo8iuJTXHRNAaiKBNgTGSlPE', time=2024-05-30T18:39:09.409204+0000, chain.objs.size()=1 2024-05-30T17:13:38.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.210+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.191__multipart_mymultipart.2~1EoJZEjwo8iuJTXHRNAaiKBNgTGSlPE.1 2024-05-30T17:13:38.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.210+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.13766243744037255847', time=2024-05-30T18:57:45.413516+0000, chain.objs.size()=1 2024-05-30T17:13:38.215 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.210+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.513__multipart_mymultipart.2~SpgXIQBtTPmHZ6XUQIw347nunZ57t48.1 2024-05-30T17:13:38.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.218+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:38.226 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.222+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-05-30T17:13:38.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.226+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:38.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.226+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:38.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.226+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-05-30T17:13:38.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.226+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.7119643252229381889', time=2024-05-30T18:59:56.100034+0000, chain.objs.size()=1 2024-05-30T17:13:38.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.226+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.547__shadow_.ADXPfX-rKbBXETV0ylfQgBrm_f0ToT8_1 2024-05-30T17:13:38.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.226+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16263355662460767957', time=2024-05-30T19:05:14.373961+0000, chain.objs.size()=6 2024-05-30T17:13:38.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.226+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.610__multipart_multipart_enc.2~ylpDmfqLJJWo3sYrmkFg2PuEPD3ufEb.1 2024-05-30T17:13:38.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.226+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.610__multipart_multipart_enc.2~ylpDmfqLJJWo3sYrmkFg2PuEPD3ufEb.2 2024-05-30T17:13:38.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.226+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.610__multipart_multipart_enc.2~ylpDmfqLJJWo3sYrmkFg2PuEPD3ufEb.3 2024-05-30T17:13:38.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.226+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.610__multipart_multipart_enc.2~ylpDmfqLJJWo3sYrmkFg2PuEPD3ufEb.4 2024-05-30T17:13:38.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.226+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.610__multipart_multipart_enc.2~ylpDmfqLJJWo3sYrmkFg2PuEPD3ufEb.5 2024-05-30T17:13:38.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.226+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.610__multipart_multipart_enc.2~ylpDmfqLJJWo3sYrmkFg2PuEPD3ufEb.6 2024-05-30T17:13:38.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.226+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.8291256569104651991', time=2024-05-30T19:06:50.044584+0000, chain.objs.size()=6 2024-05-30T17:13:38.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.226+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.647__multipart_foo.2~UlOWPpwUezPlbI-k8_p6RPtnCh_asiS.1 2024-05-30T17:13:38.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.230+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.647__multipart_foo.2~UlOWPpwUezPlbI-k8_p6RPtnCh_asiS.2 2024-05-30T17:13:38.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.230+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.647__multipart_foo.2~UlOWPpwUezPlbI-k8_p6RPtnCh_asiS.3 2024-05-30T17:13:38.233 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.230+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.647__multipart_foo.2~UlOWPpwUezPlbI-k8_p6RPtnCh_asiS.4 2024-05-30T17:13:38.234 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.230+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.647__multipart_foo.2~UlOWPpwUezPlbI-k8_p6RPtnCh_asiS.5 2024-05-30T17:13:38.238 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.234+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.647__multipart_foo.2~UlOWPpwUezPlbI-k8_p6RPtnCh_asiS.6 2024-05-30T17:13:38.248 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.246+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:38.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.250+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-05-30T17:13:38.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.254+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:38.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.254+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:38.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.254+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=10, truncated=0, next_marker='' 2024-05-30T17:13:38.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.254+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='__Hm__55Vuj-CFRfVOiihTSa4d_qGNL5', time=2024-05-30T18:37:57.577666+0000, chain.objs.size()=1 2024-05-30T17:13:38.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.254+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.155__shadow_.m60J6lK9naGqwsazo9s1xrkA-xhxI3j_0 2024-05-30T17:13:38.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.254+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.9887816444562952889', time=2024-05-30T18:56:04.239771+0000, chain.objs.size()=3 2024-05-30T17:13:38.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.254+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.511__multipart_mymultipart.2~Dc250cjiVDlnLJGe14xZ-oiub2BvIYw.1 2024-05-30T17:13:38.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.254+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.511__multipart_mymultipart.2~Dc250cjiVDlnLJGe14xZ-oiub2BvIYw.2 2024-05-30T17:13:38.259 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.254+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.511__multipart_mymultipart.2~Dc250cjiVDlnLJGe14xZ-oiub2BvIYw.3 2024-05-30T17:13:38.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.254+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='2~fGQNf8du2OvFbqMNRQAr8qu3C-RC-HY', time=2024-05-30T18:57:31.382292+0000, chain.objs.size()=1 2024-05-30T17:13:38.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.254+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.2~fGQNf8du2OvFbqMNRQAr8qu3C-RC-HY.1 2024-05-30T17:13:38.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.254+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='2~fGQNf8du2OvFbqMNRQAr8qu3C-RC-HY', time=2024-05-30T18:57:31.392077+0000, chain.objs.size()=1 2024-05-30T17:13:38.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.254+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.2~fGQNf8du2OvFbqMNRQAr8qu3C-RC-HY.2 2024-05-30T17:13:38.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.254+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='2~fGQNf8du2OvFbqMNRQAr8qu3C-RC-HY', time=2024-05-30T18:57:31.401344+0000, chain.objs.size()=1 2024-05-30T17:13:38.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.254+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.2~fGQNf8du2OvFbqMNRQAr8qu3C-RC-HY.3 2024-05-30T17:13:38.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.254+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='2~fGQNf8du2OvFbqMNRQAr8qu3C-RC-HY', time=2024-05-30T18:57:31.410807+0000, chain.objs.size()=1 2024-05-30T17:13:38.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.254+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.2~fGQNf8du2OvFbqMNRQAr8qu3C-RC-HY.4 2024-05-30T17:13:38.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.254+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='2~fGQNf8du2OvFbqMNRQAr8qu3C-RC-HY', time=2024-05-30T18:57:31.420266+0000, chain.objs.size()=1 2024-05-30T17:13:38.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.254+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.2~fGQNf8du2OvFbqMNRQAr8qu3C-RC-HY.5 2024-05-30T17:13:38.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.254+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='2~fGQNf8du2OvFbqMNRQAr8qu3C-RC-HY', time=2024-05-30T18:57:31.430075+0000, chain.objs.size()=1 2024-05-30T17:13:38.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.254+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.2~fGQNf8du2OvFbqMNRQAr8qu3C-RC-HY.6 2024-05-30T17:13:38.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.254+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.835362317344934414', time=2024-05-30T18:58:03.397628+0000, chain.objs.size()=2 2024-05-30T17:13:38.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.254+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.515__multipart_mymultipart.2~wRdO8TPHJCJfhEe0DXyaET1v18--Sbu.1 2024-05-30T17:13:38.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.254+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.515__multipart_mymultipart.2~wRdO8TPHJCJfhEe0DXyaET1v18--Sbu.2 2024-05-30T17:13:38.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.262+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='2~539VtanHeAMrZ_q76sB5C3kPZCwYUz6', time=2024-05-30T19:00:01.845632+0000, chain.objs.size()=1 2024-05-30T17:13:38.266 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.262+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.550__multipart_mymultipart.2~539VtanHeAMrZ_q76sB5C3kPZCwYUz6.1 2024-05-30T17:13:38.270 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.266+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:38.274 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.270+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-05-30T17:13:38.277 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.274+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:38.278 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.274+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:38.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.274+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-05-30T17:13:38.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.274+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12061593685321492132', time=2024-05-30T18:37:31.181312+0000, chain.objs.size()=2 2024-05-30T17:13:38.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.274+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.150__shadow_.xqAkIMzOY87XHSbj47DUl0anF1elQnT_0 2024-05-30T17:13:38.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.278+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.150__shadow_.xqAkIMzOY87XHSbj47DUl0anF1elQnT_1 2024-05-30T17:13:38.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.278+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='_LjxbitSsZOTaVxpxSNFmNPuvyTNrMLg', time=2024-05-30T18:37:52.764776+0000, chain.objs.size()=1 2024-05-30T17:13:38.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.278+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.153__shadow_.i7o4zCxIpwFV1BY96R0LDLjksNjXQNH_0 2024-05-30T17:13:38.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.278+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18221143088714560354', time=2024-05-30T18:56:00.447721+0000, chain.objs.size()=3 2024-05-30T17:13:38.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.278+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.511__multipart_mymultipart.2~O5LQ6bttbmf_Pc5HBNgbX_u3Myn40MB.1 2024-05-30T17:13:38.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.278+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.511__multipart_mymultipart.2~O5LQ6bttbmf_Pc5HBNgbX_u3Myn40MB.2 2024-05-30T17:13:38.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.278+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.511__multipart_mymultipart.2~O5LQ6bttbmf_Pc5HBNgbX_u3Myn40MB.3 2024-05-30T17:13:38.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.278+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.6260570747456506169', time=2024-05-30T18:57:58.802191+0000, chain.objs.size()=1 2024-05-30T17:13:38.280 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.278+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.515__multipart_mymultipart.2~pMRfSY-G4RiQGpEwdgvPrJaOh-KSvom.1 2024-05-30T17:13:38.285 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.282+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:38.289 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.286+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-05-30T17:13:38.291 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.290+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:38.292 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.290+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:38.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.290+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=5, truncated=0, next_marker='' 2024-05-30T17:13:38.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.290+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.8592090830099443342', time=2024-05-30T18:37:19.303239+0000, chain.objs.size()=2 2024-05-30T17:13:38.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.290+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.148__shadow_.gZDfYmM9EBk73PmHXeHqvm0S11CJ-Sx_0 2024-05-30T17:13:38.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.290+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.148__shadow_.gZDfYmM9EBk73PmHXeHqvm0S11CJ-Sx_1 2024-05-30T17:13:38.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.290+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.9513099827088681272', time=2024-05-30T18:37:26.480465+0000, chain.objs.size()=3 2024-05-30T17:13:38.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.290+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.149__multipart_mymultipart-STANDARD.2~qR19X-dfv5F9U6ixODF5mN6AmyDKy99.1 2024-05-30T17:13:38.294 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.290+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.149__multipart_mymultipart-STANDARD.2~qR19X-dfv5F9U6ixODF5mN6AmyDKy99.2 2024-05-30T17:13:38.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.290+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.149__multipart_mymultipart-STANDARD.2~qR19X-dfv5F9U6ixODF5mN6AmyDKy99.3 2024-05-30T17:13:38.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.290+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.1104081544440803874', time=2024-05-30T18:55:15.735242+0000, chain.objs.size()=1 2024-05-30T17:13:38.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.290+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.503__multipart_mymultipart.2~IkNraxnfjZkESDaOZIiBtx791hwolE4.1 2024-05-30T17:13:38.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.290+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.6306115993636742841', time=2024-05-30T18:57:05.822086+0000, chain.objs.size()=6 2024-05-30T17:13:38.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.290+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.2~UHxAIBJcc1x_ID-altuu2PMqwGbs8sh.1 2024-05-30T17:13:38.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.290+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.2~UHxAIBJcc1x_ID-altuu2PMqwGbs8sh.2 2024-05-30T17:13:38.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.290+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.FVVA-V4K9Qmi1YAKZAEejZFBh5S0jLbs.3 2024-05-30T17:13:38.295 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.290+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.2~UHxAIBJcc1x_ID-altuu2PMqwGbs8sh.4 2024-05-30T17:13:38.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.290+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.2~UHxAIBJcc1x_ID-altuu2PMqwGbs8sh.5 2024-05-30T17:13:38.296 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.290+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.2~UHxAIBJcc1x_ID-altuu2PMqwGbs8sh.6 2024-05-30T17:13:38.300 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.298+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15928721584741594495', time=2024-05-30T19:00:02.537880+0000, chain.objs.size()=1 2024-05-30T17:13:38.301 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.298+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.550__multipart_mymultipart.r3Y-HLLeykSA8Q6KQzm6xEttLmN2BLfp.1 2024-05-30T17:13:38.306 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.302+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:38.310 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.306+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-05-30T17:13:38.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.310+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:38.313 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.310+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:38.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.310+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-30T17:13:38.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.310+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='_ohWd-i38B-OBmosf4Tw8hKmbS8RO4bx', time=2024-05-30T18:37:57.637386+0000, chain.objs.size()=1 2024-05-30T17:13:38.314 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.310+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.155__shadow_.qaGWQNGmO7zCg7hdXPs1FZaafoHzczQ_0 2024-05-30T17:13:38.316 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.314+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:38.320 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.318+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-05-30T17:13:38.322 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.318+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:38.325 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.322+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:38.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.322+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-05-30T17:13:38.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.322+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.2_98e9556cf6289b92594d42263b3b87a5', time=2024-05-30T18:55:07.802901+0000, chain.objs.size()=6 2024-05-30T17:13:38.326 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.322+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.1 2024-05-30T17:13:38.327 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.322+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.2 2024-05-30T17:13:38.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.322+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.3 2024-05-30T17:13:38.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.322+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.4 2024-05-30T17:13:38.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.322+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.5 2024-05-30T17:13:38.329 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.322+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.6 2024-05-30T17:13:38.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.322+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.12584752938697777739', time=2024-05-30T18:57:31.495562+0000, chain.objs.size()=6 2024-05-30T17:13:38.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.322+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.2~b-0hnWQ1OWRKqykHt86jEKu9496mShH.1 2024-05-30T17:13:38.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.322+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.XWf8-Hl94yn9VVLY2EWHuTdE0ZClIVzP.2 2024-05-30T17:13:38.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.322+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.cBrJidYftbBiZ2Gw6EbNiJHjc5GvIqky.3 2024-05-30T17:13:38.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.322+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.2~b-0hnWQ1OWRKqykHt86jEKu9496mShH.4 2024-05-30T17:13:38.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.322+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.2~b-0hnWQ1OWRKqykHt86jEKu9496mShH.5 2024-05-30T17:13:38.330 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.322+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.2~b-0hnWQ1OWRKqykHt86jEKu9496mShH.6 2024-05-30T17:13:38.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.326+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='2~mtgDhKqt_Syokmuy3zNN2oFSXGugBdc', time=2024-05-30T18:58:35.586232+0000, chain.objs.size()=1 2024-05-30T17:13:38.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.326+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.523__multipart_mymultipart.2~mtgDhKqt_Syokmuy3zNN2oFSXGugBdc.1 2024-05-30T17:13:38.334 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.330+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:38.338 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.334+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-05-30T17:13:38.341 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.338+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:38.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.338+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:38.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.342+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=6, truncated=0, next_marker='' 2024-05-30T17:13:38.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.342+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='_jrP8QqLvZoa4AfrYst0z5ymD4-s5wQt', time=2024-05-30T18:37:31.987873+0000, chain.objs.size()=1 2024-05-30T17:13:38.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.342+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.150__shadow_.hEV9C7OfQDuhKeGU3beKxvkToXqE30-_1 2024-05-30T17:13:38.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.342+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.15267613928877617686', time=2024-05-30T18:57:48.999222+0000, chain.objs.size()=2 2024-05-30T17:13:38.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.342+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.513__multipart_mymultipart.2~MDYt6G3_ehU7ufhSM1-pCCO8UTdJvdD.1 2024-05-30T17:13:38.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.342+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.513__multipart_mymultipart.2~MDYt6G3_ehU7ufhSM1-pCCO8UTdJvdD.2 2024-05-30T17:13:38.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.342+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='2~cSG-HBi2nqM00G-PsntYOs7esfM3dMd', time=2024-05-30T18:58:32.138382+0000, chain.objs.size()=2 2024-05-30T17:13:38.344 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.342+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.521__multipart_mymultipart.2~cSG-HBi2nqM00G-PsntYOs7esfM3dMd.1 2024-05-30T17:13:38.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.342+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.521__multipart_mymultipart.2~cSG-HBi2nqM00G-PsntYOs7esfM3dMd.2 2024-05-30T17:13:38.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.342+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='2~sRwU-A3fEHtYYMUI9qM1cZCrjb0dZ8Z', time=2024-05-30T18:58:41.442159+0000, chain.objs.size()=1 2024-05-30T17:13:38.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.342+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.525__multipart_mymultipart.2~sRwU-A3fEHtYYMUI9qM1cZCrjb0dZ8Z.3 2024-05-30T17:13:38.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.342+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.16961672695748568019', time=2024-05-30T18:59:30.511920+0000, chain.objs.size()=1 2024-05-30T17:13:38.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.342+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.541__shadow_.O_Hoy8-295SI4xWwzE34_I2wJ0PoF9V_1 2024-05-30T17:13:38.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.342+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='2~ipV5Zy7hV7N-1j84nDLSjPMPxts6VGv', time=2024-05-30T19:05:23.950780+0000, chain.objs.size()=6 2024-05-30T17:13:38.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.342+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.611__multipart_multipart_enc.2~ipV5Zy7hV7N-1j84nDLSjPMPxts6VGv.1 2024-05-30T17:13:38.345 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.342+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.611__multipart_multipart_enc.2~ipV5Zy7hV7N-1j84nDLSjPMPxts6VGv.2 2024-05-30T17:13:38.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.342+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.611__multipart_multipart_enc.2~ipV5Zy7hV7N-1j84nDLSjPMPxts6VGv.3 2024-05-30T17:13:38.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.342+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.611__multipart_multipart_enc.2~ipV5Zy7hV7N-1j84nDLSjPMPxts6VGv.4 2024-05-30T17:13:38.346 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.342+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.611__multipart_multipart_enc.2~ipV5Zy7hV7N-1j84nDLSjPMPxts6VGv.5 2024-05-30T17:13:38.349 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.346+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.611__multipart_multipart_enc.2~ipV5Zy7hV7N-1j84nDLSjPMPxts6VGv.6 2024-05-30T17:13:38.356 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.354+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:38.360 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.358+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-05-30T17:13:38.362 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.358+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:38.363 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.362+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:38.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.362+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-05-30T17:13:38.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.362+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='2~nTYuVFdSwE93Kv2jyLGjUHrRroy07Gn', time=2024-05-30T18:39:12.591196+0000, chain.objs.size()=1 2024-05-30T17:13:38.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.362+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.192__multipart_mymultipart.2~nTYuVFdSwE93Kv2jyLGjUHrRroy07Gn.1 2024-05-30T17:13:38.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.362+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.2_5798f39d1a5f8db134293cc2893a2c71', time=2024-05-30T18:55:09.223477+0000, chain.objs.size()=6 2024-05-30T17:13:38.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.362+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.1 2024-05-30T17:13:38.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.362+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.2 2024-05-30T17:13:38.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.362+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.3 2024-05-30T17:13:38.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.362+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.4 2024-05-30T17:13:38.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.362+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.5 2024-05-30T17:13:38.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.362+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.6 2024-05-30T17:13:38.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.362+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.345562894357718919', time=2024-05-30T18:56:29.970787+0000, chain.objs.size()=6 2024-05-30T17:13:38.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.362+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.1Qxyazil3jHGc-i40XjbcdX-_WjgpY9l.1 2024-05-30T17:13:38.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.362+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.2~vizXKVZg1FMC6unYshwnxH80DsY-V_W.2 2024-05-30T17:13:38.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.362+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.2~vizXKVZg1FMC6unYshwnxH80DsY-V_W.3 2024-05-30T17:13:38.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.362+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.2~vizXKVZg1FMC6unYshwnxH80DsY-V_W.4 2024-05-30T17:13:38.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.362+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.2~vizXKVZg1FMC6unYshwnxH80DsY-V_W.5 2024-05-30T17:13:38.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.366+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.2~vizXKVZg1FMC6unYshwnxH80DsY-V_W.6 2024-05-30T17:13:38.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.366+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.1685311724792802816', time=2024-05-30T19:04:26.619447+0000, chain.objs.size()=6 2024-05-30T17:13:38.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.366+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.603__multipart_multipart_enc.2~1BzbPw9iDgBlcIQke26zZyrlQT_0v53.1 2024-05-30T17:13:38.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.366+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.603__multipart_multipart_enc.2~1BzbPw9iDgBlcIQke26zZyrlQT_0v53.2 2024-05-30T17:13:38.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.366+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.603__multipart_multipart_enc.2~1BzbPw9iDgBlcIQke26zZyrlQT_0v53.3 2024-05-30T17:13:38.371 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.366+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.603__multipart_multipart_enc.2~1BzbPw9iDgBlcIQke26zZyrlQT_0v53.4 2024-05-30T17:13:38.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.366+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.603__multipart_multipart_enc.2~1BzbPw9iDgBlcIQke26zZyrlQT_0v53.5 2024-05-30T17:13:38.372 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.366+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.603__multipart_multipart_enc.2~1BzbPw9iDgBlcIQke26zZyrlQT_0v53.6 2024-05-30T17:13:38.379 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.378+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:38.385 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.382+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-05-30T17:13:38.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.386+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:38.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.386+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:38.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.386+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=5, truncated=0, next_marker='' 2024-05-30T17:13:38.389 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.386+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.7387770982537945682', time=2024-05-30T18:37:26.403647+0000, chain.objs.size()=3 2024-05-30T17:13:38.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.386+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.149__multipart_mymultipart-LUKEWARM.2~tWyzWqj9q_qfiMK4EJ3D5mnC6cWBE74.1 2024-05-30T17:13:38.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.386+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.149__multipart_mymultipart-LUKEWARM.2~tWyzWqj9q_qfiMK4EJ3D5mnC6cWBE74.2 2024-05-30T17:13:38.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.386+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.149__multipart_mymultipart-LUKEWARM.2~tWyzWqj9q_qfiMK4EJ3D5mnC6cWBE74.3 2024-05-30T17:13:38.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.386+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.17284998113391587549', time=2024-05-30T18:56:06.746461+0000, chain.objs.size()=3 2024-05-30T17:13:38.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.386+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.511__multipart_mymultipart.2~1MQM2qWmO0axs3iFGj5A0-e6VXDcAsL.1 2024-05-30T17:13:38.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.386+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.511__multipart_mymultipart.2~1MQM2qWmO0axs3iFGj5A0-e6VXDcAsL.2 2024-05-30T17:13:38.390 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.386+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.511__multipart_mymultipart.2~1MQM2qWmO0axs3iFGj5A0-e6VXDcAsL.3 2024-05-30T17:13:38.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.386+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.6896707989611605034', time=2024-05-30T18:57:53.402761+0000, chain.objs.size()=3 2024-05-30T17:13:38.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.386+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.513__multipart_mymultipart.2~7IcZnlHhUSoOn0rmezhkSA6Lp02WmUm.1 2024-05-30T17:13:38.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.386+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.513__multipart_mymultipart.2~7IcZnlHhUSoOn0rmezhkSA6Lp02WmUm.2 2024-05-30T17:13:38.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.386+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.513__multipart_mymultipart.2~7IcZnlHhUSoOn0rmezhkSA6Lp02WmUm.3 2024-05-30T17:13:38.391 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.386+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.330404440284226139', time=2024-05-30T19:04:37.430075+0000, chain.objs.size()=6 2024-05-30T17:13:38.392 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.386+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.604__multipart_multipart_enc.2~6G2FOQbYcW4xGyz9LW77d1vyAjizEgn.1 2024-05-30T17:13:38.392 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.386+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.604__multipart_multipart_enc.2~6G2FOQbYcW4xGyz9LW77d1vyAjizEgn.2 2024-05-30T17:13:38.392 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.386+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.604__multipart_multipart_enc.2~6G2FOQbYcW4xGyz9LW77d1vyAjizEgn.3 2024-05-30T17:13:38.394 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.390+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.604__multipart_multipart_enc.2~6G2FOQbYcW4xGyz9LW77d1vyAjizEgn.4 2024-05-30T17:13:38.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.394+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.604__multipart_multipart_enc.2~6G2FOQbYcW4xGyz9LW77d1vyAjizEgn.5 2024-05-30T17:13:38.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.394+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.604__multipart_multipart_enc.2~6G2FOQbYcW4xGyz9LW77d1vyAjizEgn.6 2024-05-30T17:13:38.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.394+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.8760749275052082614', time=2024-05-30T19:07:43.251792+0000, chain.objs.size()=1 2024-05-30T17:13:38.396 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.394+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.687__multipart_file1.2~_961TGMKAHiUpBIyLqlxrpBIjivwGZB.1 2024-05-30T17:13:38.402 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.398+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:38.406 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.402+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-05-30T17:13:38.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.406+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:38.408 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.406+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:38.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.406+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-05-30T17:13:38.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.406+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.2_b3917b9c684ab76e7f2a5a60fa390fb0', time=2024-05-30T18:55:09.216236+0000, chain.objs.size()=6 2024-05-30T17:13:38.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.406+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.1 2024-05-30T17:13:38.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.406+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.2 2024-05-30T17:13:38.410 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.406+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.3 2024-05-30T17:13:38.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.406+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.4 2024-05-30T17:13:38.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.406+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.5 2024-05-30T17:13:38.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.406+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.497__multipart_srcmultipart.2~mP8LESyKmidFknz8JzfQ9dPAcfxeghc.6 2024-05-30T17:13:38.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.406+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.18262993051407584687', time=2024-05-30T18:58:06.012415+0000, chain.objs.size()=3 2024-05-30T17:13:38.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.406+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.515__multipart_mymultipart.2~-mp-1QbZSVMrv35Nh_WTR5Ax-u2pnOR.1 2024-05-30T17:13:38.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.406+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.515__multipart_mymultipart.2~-mp-1QbZSVMrv35Nh_WTR5Ax-u2pnOR.2 2024-05-30T17:13:38.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.406+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.515__multipart_mymultipart.2~-mp-1QbZSVMrv35Nh_WTR5Ax-u2pnOR.3 2024-05-30T17:13:38.411 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.406+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.5271994548570311612', time=2024-05-30T18:58:23.455796+0000, chain.objs.size()=2 2024-05-30T17:13:38.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.406+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.518__multipart_mymultipart.2~HYdknzflnMuiQMY4btp1NRIIzVdtpDm.1 2024-05-30T17:13:38.412 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.406+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.518__multipart_mymultipart.2~HYdknzflnMuiQMY4btp1NRIIzVdtpDm.2 2024-05-30T17:13:38.414 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.410+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:38.418 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.414+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-05-30T17:13:38.420 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.418+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:38.421 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.418+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:38.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.418+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=6, truncated=0, next_marker='' 2024-05-30T17:13:38.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.418+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.13550558485899269814', time=2024-05-30T18:37:34.048877+0000, chain.objs.size()=2 2024-05-30T17:13:38.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.418+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.150__shadow_.BTTxI-zuQHHOHqO67yp5UeXQdhC7w4K_0 2024-05-30T17:13:38.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.418+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.150__shadow_.BTTxI-zuQHHOHqO67yp5UeXQdhC7w4K_1 2024-05-30T17:13:38.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.418+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.2506496791758756027', time=2024-05-30T18:57:51.322126+0000, chain.objs.size()=3 2024-05-30T17:13:38.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.418+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.513__multipart_mymultipart.2~Ji1YKZmjV5Bcy8e77UwtRBt0D98yGb8.1 2024-05-30T17:13:38.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.418+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.513__multipart_mymultipart.2~Ji1YKZmjV5Bcy8e77UwtRBt0D98yGb8.2 2024-05-30T17:13:38.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.418+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.513__multipart_mymultipart.2~Ji1YKZmjV5Bcy8e77UwtRBt0D98yGb8.3 2024-05-30T17:13:38.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.418+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.1266075376357186918', time=2024-05-30T18:58:10.404883+0000, chain.objs.size()=2 2024-05-30T17:13:38.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.418+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.515__multipart_mymultipart.2~qVjhQLGZGWXL-454si0v0_ulBYQeORn.1 2024-05-30T17:13:38.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.418+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.515__multipart_mymultipart.2~qVjhQLGZGWXL-454si0v0_ulBYQeORn.2 2024-05-30T17:13:38.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.418+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='2~D75734APknTwlrGir5oKtB0qgH7XX9g', time=2024-05-30T18:58:36.951834+0000, chain.objs.size()=1 2024-05-30T17:13:38.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.418+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.524__multipart_mymultipart.2~D75734APknTwlrGir5oKtB0qgH7XX9g.1 2024-05-30T17:13:38.423 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.418+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.10263594736868744511', time=2024-05-30T18:59:53.421849+0000, chain.objs.size()=1 2024-05-30T17:13:38.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.418+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.547__shadow_.VqOED0OBeoNpx3RCQXtMS5Od7xGxaJC_1 2024-05-30T17:13:38.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.418+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.1372420500746441100', time=2024-05-30T19:06:21.513952+0000, chain.objs.size()=1 2024-05-30T17:13:38.424 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.418+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.640__shadow_.dcPT3Gqu_yCfs4SOOSF9zL-NP1z-ifd_1 2024-05-30T17:13:38.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.426+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:38.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.430+0000 7f05851d5980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-05-30T17:13:38.433 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.430+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-30T17:13:38.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.430+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-30T17:13:38.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.430+0000 7f05851d5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-05-30T17:13:38.434 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.430+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.5697633897241981566', time=2024-05-30T18:37:26.331265+0000, chain.objs.size()=3 2024-05-30T17:13:38.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.430+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.149__multipart_mymultipart-FROZEN.2~MOXtSAuQLXQ5r21pQslryLTtTtGpRdu.1 2024-05-30T17:13:38.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.430+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.149__multipart_mymultipart-FROZEN.2~MOXtSAuQLXQ5r21pQslryLTtTtGpRdu.2 2024-05-30T17:13:38.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.430+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.149__multipart_mymultipart-FROZEN.2~MOXtSAuQLXQ5r21pQslryLTtTtGpRdu.3 2024-05-30T17:13:38.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.430+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.3875020820438028677', time=2024-05-30T18:55:19.702117+0000, chain.objs.size()=1 2024-05-30T17:13:38.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.430+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.506__multipart_mymultipartcopy.2~zbJ0Jvw6G4qZSqpVzUJy4csO8eTf8i5.1 2024-05-30T17:13:38.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.430+0000 7f05851d5980 20 garbage collection: RGWGC::process iterating over entry tag='2~l0doLYA3bTTDgeI-B7Q5iUhXZJ_s3Ro', time=2024-05-30T18:56:29.905644+0000, chain.objs.size()=1 2024-05-30T17:13:38.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.430+0000 7f05851d5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:601fa7ab-74a8-40e7-9e32-e5827d25af6b.4474.512__multipart_mymultipart.2~l0doLYA3bTTDgeI-B7Q5iUhXZJ_s3Ro.2 2024-05-30T17:13:38.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.434+0000 7f05851d5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-30T17:13:38.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.438+0000 7f04397e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-30T17:13:38.445 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.442+0000 7f05851d5980 20 remove_watcher() i=0 2024-05-30T17:13:38.446 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.442+0000 7f05851d5980 2 removed watcher, disabling cache 2024-05-30T17:13:38.447 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.446+0000 7f05851d5980 20 remove_watcher() i=1 2024-05-30T17:13:38.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.446+0000 7f05851d5980 20 remove_watcher() i=2 2024-05-30T17:13:38.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.446+0000 7f05851d5980 20 remove_watcher() i=3 2024-05-30T17:13:38.452 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.450+0000 7f05851d5980 20 remove_watcher() i=4 2024-05-30T17:13:38.454 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.450+0000 7f05851d5980 20 remove_watcher() i=5 2024-05-30T17:13:38.455 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.454+0000 7f05851d5980 20 remove_watcher() i=6 2024-05-30T17:13:38.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-05-30T17:13:38.454+0000 7f05851d5980 20 remove_watcher() i=7 2024-05-30T17:13:38.474 DEBUG:teuthology.run_tasks:Unwinding manager openssl_keys 2024-05-30T17:13:38.485 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-30T17:13:38.529 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-30T17:13:38.529 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T17:13:38.893 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:38.910 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-05-30T17:13:38.921 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":2645,"stamp":"2024-05-30T17:13:38.440189+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":670635562,"num_objects":1096,"num_object_clones":0,"num_object_copies":2676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1096,"num_whiteouts":0,"num_read":253424,"num_read_kb":2692954,"num_write":146665,"num_write_kb":2342828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":289,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13825,"ondisk_log_size":13825,"up":850,"acting":850,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":850,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2431244,"kb_used_data":1099020,"kb_used_omap":12,"kb_used_meta":1332147,"kb_avail":752543476,"statfs":{"total":773094113280,"available":770604519424,"internally_reserved":0,"allocated":1125396480,"data_stored":1109228324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":1364119089},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-520,"num_objects":-4,"num_object_clones":0,"num_object_copies":-8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4,"num_whiteouts":0,"num_read":23,"num_read_kb":25,"num_write":15,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007603"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":27,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:57.133920+0000","last_change":"2024-05-30T16:34:55.869930+0000","last_active":"2024-05-30T17:09:57.133920+0000","last_peered":"2024-05-30T17:09:57.133920+0000","last_clean":"2024-05-30T17:09:57.133920+0000","last_became_active":"2024-05-30T16:34:55.869226+0000","last_became_peered":"2024-05-30T16:34:55.869226+0000","last_unstale":"2024-05-30T17:09:57.133920+0000","last_undegraded":"2024-05-30T17:09:57.133920+0000","last_fullsized":"2024-05-30T17:09:57.133920+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:37:02.394930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"265'1","reported_seq":47,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:07.136134+0000","last_change":"2024-05-30T16:17:35.249630+0000","last_active":"2024-05-30T17:10:07.136134+0000","last_peered":"2024-05-30T17:10:07.136134+0000","last_clean":"2024-05-30T17:10:07.136134+0000","last_became_active":"2024-05-30T16:17:35.248740+0000","last_became_peered":"2024-05-30T16:17:35.248740+0000","last_unstale":"2024-05-30T17:10:07.136134+0000","last_undegraded":"2024-05-30T17:10:07.136134+0000","last_fullsized":"2024-05-30T17:10:07.136134+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:05:10.176331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":6144,"num_write":1,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"265'7","reported_seq":65,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:01.569510+0000","last_change":"2024-05-30T16:17:31.184110+0000","last_active":"2024-05-30T17:10:01.569510+0000","last_peered":"2024-05-30T17:10:01.569510+0000","last_clean":"2024-05-30T17:10:01.569510+0000","last_became_active":"2024-05-30T16:17:31.183349+0000","last_became_peered":"2024-05-30T16:17:31.183349+0000","last_unstale":"2024-05-30T17:10:01.569510+0000","last_undegraded":"2024-05-30T17:10:01.569510+0000","last_fullsized":"2024-05-30T17:10:01.569510+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:06:40.372126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":16384,"num_write":7,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"265'45","reported_seq":184,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.581199+0000","last_change":"2024-05-30T16:12:50.806342+0000","last_active":"2024-05-30T17:11:11.581199+0000","last_peered":"2024-05-30T17:11:11.581199+0000","last_clean":"2024-05-30T17:11:11.581199+0000","last_became_active":"2024-05-30T16:12:50.803352+0000","last_became_peered":"2024-05-30T16:12:50.803352+0000","last_unstale":"2024-05-30T17:11:11.581199+0000","last_undegraded":"2024-05-30T17:11:11.581199+0000","last_fullsized":"2024-05-30T17:11:11.581199+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:23:40.247207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":225,"num_read_kb":156,"num_write":101,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":27,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.543407+0000","last_change":"2024-05-30T16:34:55.887503+0000","last_active":"2024-05-30T17:09:56.543407+0000","last_peered":"2024-05-30T17:09:56.543407+0000","last_clean":"2024-05-30T17:09:56.543407+0000","last_became_active":"2024-05-30T16:34:55.886731+0000","last_became_peered":"2024-05-30T16:34:55.886731+0000","last_unstale":"2024-05-30T17:09:56.543407+0000","last_undegraded":"2024-05-30T17:09:56.543407+0000","last_fullsized":"2024-05-30T17:09:56.543407+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:47:35.283261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"0'0","reported_seq":44,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:06.732701+0000","last_change":"2024-05-30T16:17:35.248384+0000","last_active":"2024-05-30T17:10:06.732701+0000","last_peered":"2024-05-30T17:10:06.732701+0000","last_clean":"2024-05-30T17:10:06.732701+0000","last_became_active":"2024-05-30T16:17:35.247810+0000","last_became_peered":"2024-05-30T16:17:35.247810+0000","last_unstale":"2024-05-30T17:10:06.732701+0000","last_undegraded":"2024-05-30T17:10:06.732701+0000","last_fullsized":"2024-05-30T17:10:06.732701+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:56:26.172595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"0'0","reported_seq":50,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:01.773778+0000","last_change":"2024-05-30T16:17:31.215393+0000","last_active":"2024-05-30T17:10:01.773778+0000","last_peered":"2024-05-30T17:10:01.773778+0000","last_clean":"2024-05-30T17:10:01.773778+0000","last_became_active":"2024-05-30T16:17:31.212337+0000","last_became_peered":"2024-05-30T16:17:31.212337+0000","last_unstale":"2024-05-30T17:10:01.773778+0000","last_undegraded":"2024-05-30T17:10:01.773778+0000","last_fullsized":"2024-05-30T17:10:01.773778+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:48:30.989819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"265'55","reported_seq":210,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:06.556856+0000","last_change":"2024-05-30T16:12:50.810909+0000","last_active":"2024-05-30T17:11:06.556856+0000","last_peered":"2024-05-30T17:11:06.556856+0000","last_clean":"2024-05-30T17:11:06.556856+0000","last_became_active":"2024-05-30T16:12:50.810207+0000","last_became_peered":"2024-05-30T16:12:50.810207+0000","last_unstale":"2024-05-30T17:11:06.556856+0000","last_undegraded":"2024-05-30T17:11:06.556856+0000","last_fullsized":"2024-05-30T17:11:06.556856+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:37:48.188833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":277,"num_read_kb":197,"num_write":99,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.556933+0000","last_change":"2024-05-30T16:34:55.876451+0000","last_active":"2024-05-30T17:11:11.556933+0000","last_peered":"2024-05-30T17:11:11.556933+0000","last_clean":"2024-05-30T17:11:11.556933+0000","last_became_active":"2024-05-30T16:34:55.874359+0000","last_became_peered":"2024-05-30T16:34:55.874359+0000","last_unstale":"2024-05-30T17:11:11.556933+0000","last_undegraded":"2024-05-30T17:11:11.556933+0000","last_fullsized":"2024-05-30T17:11:11.556933+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:37:31.196070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"0'0","reported_seq":48,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:41.561910+0000","last_change":"2024-05-30T16:17:35.743986+0000","last_active":"2024-05-30T17:09:41.561910+0000","last_peered":"2024-05-30T17:09:41.561910+0000","last_clean":"2024-05-30T17:09:41.561910+0000","last_became_active":"2024-05-30T16:17:35.743231+0000","last_became_peered":"2024-05-30T16:17:35.743231+0000","last_unstale":"2024-05-30T17:09:41.561910+0000","last_undegraded":"2024-05-30T17:09:41.561910+0000","last_fullsized":"2024-05-30T17:09:41.561910+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:52:11.845318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"265'2","reported_seq":59,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:46.541550+0000","last_change":"2024-05-30T16:17:31.205642+0000","last_active":"2024-05-30T17:09:46.541550+0000","last_peered":"2024-05-30T17:09:46.541550+0000","last_clean":"2024-05-30T17:09:46.541550+0000","last_became_active":"2024-05-30T16:17:31.204806+0000","last_became_peered":"2024-05-30T16:17:31.204806+0000","last_unstale":"2024-05-30T17:09:46.541550+0000","last_undegraded":"2024-05-30T17:09:46.541550+0000","last_fullsized":"2024-05-30T17:09:46.541550+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:50:34.702667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":6144,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"265'29","reported_seq":134,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.570270+0000","last_change":"2024-05-30T16:12:50.811279+0000","last_active":"2024-05-30T17:10:21.570270+0000","last_peered":"2024-05-30T17:10:21.570270+0000","last_clean":"2024-05-30T17:10:21.570270+0000","last_became_active":"2024-05-30T16:12:50.810062+0000","last_became_peered":"2024-05-30T16:12:50.810062+0000","last_unstale":"2024-05-30T17:10:21.570270+0000","last_undegraded":"2024-05-30T17:10:21.570270+0000","last_fullsized":"2024-05-30T17:10:21.570270+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:23:45.234198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":98,"num_write":65,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":27,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:08:56.553929+0000","last_change":"2024-05-30T16:34:55.888292+0000","last_active":"2024-05-30T17:08:56.553929+0000","last_peered":"2024-05-30T17:08:56.553929+0000","last_clean":"2024-05-30T17:08:56.553929+0000","last_became_active":"2024-05-30T16:34:55.887532+0000","last_became_peered":"2024-05-30T16:34:55.887532+0000","last_unstale":"2024-05-30T17:08:56.553929+0000","last_undegraded":"2024-05-30T17:08:56.553929+0000","last_fullsized":"2024-05-30T17:08:56.553929+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:47:12.792997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"265'1","reported_seq":52,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:51.567346+0000","last_change":"2024-05-30T16:17:35.746712+0000","last_active":"2024-05-30T17:09:51.567346+0000","last_peered":"2024-05-30T17:09:51.567346+0000","last_clean":"2024-05-30T17:09:51.567346+0000","last_became_active":"2024-05-30T16:17:35.745709+0000","last_became_peered":"2024-05-30T16:17:35.745709+0000","last_unstale":"2024-05-30T17:09:51.567346+0000","last_undegraded":"2024-05-30T17:09:51.567346+0000","last_fullsized":"2024-05-30T17:09:51.567346+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:58:32.058273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":584363,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1142,"num_write":1,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1f","version":"265'2","reported_seq":56,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.726378+0000","last_change":"2024-05-30T16:17:31.183141+0000","last_active":"2024-05-30T17:09:36.726378+0000","last_peered":"2024-05-30T17:09:36.726378+0000","last_clean":"2024-05-30T17:09:36.726378+0000","last_became_active":"2024-05-30T16:17:31.182344+0000","last_became_peered":"2024-05-30T16:17:31.182344+0000","last_unstale":"2024-05-30T17:09:36.726378+0000","last_undegraded":"2024-05-30T17:09:36.726378+0000","last_fullsized":"2024-05-30T17:09:36.726378+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:20:08.705563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"260'6","reported_seq":76,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.573922+0000","last_change":"2024-05-30T16:12:50.830384+0000","last_active":"2024-05-30T17:10:21.573922+0000","last_peered":"2024-05-30T17:10:21.573922+0000","last_clean":"2024-05-30T17:10:21.573922+0000","last_became_active":"2024-05-30T16:12:50.826974+0000","last_became_peered":"2024-05-30T16:12:50.826974+0000","last_unstale":"2024-05-30T17:10:21.573922+0000","last_undegraded":"2024-05-30T17:10:21.573922+0000","last_fullsized":"2024-05-30T17:10:21.573922+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:27:49.529929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":24,"num_write":11,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":31,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:06.583992+0000","last_change":"2024-05-30T16:34:55.899516+0000","last_active":"2024-05-30T17:11:06.583992+0000","last_peered":"2024-05-30T17:11:06.583992+0000","last_clean":"2024-05-30T17:11:06.583992+0000","last_became_active":"2024-05-30T16:34:55.898864+0000","last_became_peered":"2024-05-30T16:34:55.898864+0000","last_unstale":"2024-05-30T17:11:06.583992+0000","last_undegraded":"2024-05-30T17:11:06.583992+0000","last_fullsized":"2024-05-30T17:11:06.583992+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:07:16.296580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"265'6","reported_seq":62,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:52.146144+0000","last_change":"2024-05-30T16:17:35.233543+0000","last_active":"2024-05-30T17:10:52.146144+0000","last_peered":"2024-05-30T17:10:52.146144+0000","last_clean":"2024-05-30T17:10:52.146144+0000","last_became_active":"2024-05-30T16:17:35.232744+0000","last_became_peered":"2024-05-30T16:17:35.232744+0000","last_unstale":"2024-05-30T17:10:52.146144+0000","last_undegraded":"2024-05-30T17:10:52.146144+0000","last_fullsized":"2024-05-30T17:10:52.146144+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:40:45.244827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":23554,"num_write":5,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"0'0","reported_seq":51,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:41.574504+0000","last_change":"2024-05-30T16:17:31.186983+0000","last_active":"2024-05-30T17:10:41.574504+0000","last_peered":"2024-05-30T17:10:41.574504+0000","last_clean":"2024-05-30T17:10:41.574504+0000","last_became_active":"2024-05-30T16:17:31.186229+0000","last_became_peered":"2024-05-30T16:17:31.186229+0000","last_unstale":"2024-05-30T17:10:41.574504+0000","last_undegraded":"2024-05-30T17:10:41.574504+0000","last_fullsized":"2024-05-30T17:10:41.574504+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:33:19.747591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"265'27","reported_seq":134,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.573828+0000","last_change":"2024-05-30T16:12:50.830804+0000","last_active":"2024-05-30T17:10:21.573828+0000","last_peered":"2024-05-30T17:10:21.573828+0000","last_clean":"2024-05-30T17:10:21.573828+0000","last_became_active":"2024-05-30T16:12:50.827008+0000","last_became_peered":"2024-05-30T16:12:50.827008+0000","last_unstale":"2024-05-30T17:10:21.573828+0000","last_undegraded":"2024-05-30T17:10:21.573828+0000","last_fullsized":"2024-05-30T17:10:21.573828+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:26:44.426182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":100,"num_write":57,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.778503+0000","last_change":"2024-05-30T16:34:55.870968+0000","last_active":"2024-05-30T17:10:21.778503+0000","last_peered":"2024-05-30T17:10:21.778503+0000","last_clean":"2024-05-30T17:10:21.778503+0000","last_became_active":"2024-05-30T16:34:55.870139+0000","last_became_peered":"2024-05-30T16:34:55.870139+0000","last_unstale":"2024-05-30T17:10:21.778503+0000","last_undegraded":"2024-05-30T17:10:21.778503+0000","last_fullsized":"2024-05-30T17:10:21.778503+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:55:17.556265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"265'5","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.551727+0000","last_change":"2024-05-30T16:17:35.234843+0000","last_active":"2024-05-30T17:10:46.551727+0000","last_peered":"2024-05-30T17:10:46.551727+0000","last_clean":"2024-05-30T17:10:46.551727+0000","last_became_active":"2024-05-30T16:17:35.233040+0000","last_became_peered":"2024-05-30T16:17:35.233040+0000","last_unstale":"2024-05-30T17:10:46.551727+0000","last_undegraded":"2024-05-30T17:10:46.551727+0000","last_fullsized":"2024-05-30T17:10:46.551727+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:42:31.077865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11582751,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":12336,"num_write":10,"num_write_kb":11312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"265'2","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.742318+0000","last_change":"2024-05-30T16:17:31.174675+0000","last_active":"2024-05-30T17:10:51.742318+0000","last_peered":"2024-05-30T17:10:51.742318+0000","last_clean":"2024-05-30T17:10:51.742318+0000","last_became_active":"2024-05-30T16:17:31.173900+0000","last_became_peered":"2024-05-30T16:17:31.173900+0000","last_unstale":"2024-05-30T17:10:51.742318+0000","last_undegraded":"2024-05-30T17:10:51.742318+0000","last_fullsized":"2024-05-30T17:10:51.742318+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:03:12.426092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":6144,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"265'75","reported_seq":266,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.546161+0000","last_change":"2024-05-30T16:12:50.815789+0000","last_active":"2024-05-30T17:10:21.546161+0000","last_peered":"2024-05-30T17:10:21.546161+0000","last_clean":"2024-05-30T17:10:21.546161+0000","last_became_active":"2024-05-30T16:12:50.815032+0000","last_became_peered":"2024-05-30T16:12:50.815032+0000","last_unstale":"2024-05-30T17:10:21.546161+0000","last_undegraded":"2024-05-30T17:10:21.546161+0000","last_fullsized":"2024-05-30T17:10:21.546161+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:53:44.633843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":396,"num_read_kb":277,"num_write":137,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"265'1","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.548662+0000","last_change":"2024-05-30T16:34:55.882343+0000","last_active":"2024-05-30T17:09:56.548662+0000","last_peered":"2024-05-30T17:09:56.548662+0000","last_clean":"2024-05-30T17:09:56.548662+0000","last_became_active":"2024-05-30T16:34:55.881414+0000","last_became_peered":"2024-05-30T16:34:55.881414+0000","last_unstale":"2024-05-30T17:09:56.548662+0000","last_undegraded":"2024-05-30T17:09:56.548662+0000","last_fullsized":"2024-05-30T17:09:56.548662+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:58:23.360902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"262'2","reported_seq":48,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:06.542967+0000","last_change":"2024-05-30T16:17:35.238116+0000","last_active":"2024-05-30T17:10:06.542967+0000","last_peered":"2024-05-30T17:10:06.542967+0000","last_clean":"2024-05-30T17:10:06.542967+0000","last_became_active":"2024-05-30T16:17:35.237424+0000","last_became_peered":"2024-05-30T16:17:35.237424+0000","last_unstale":"2024-05-30T17:10:06.542967+0000","last_undegraded":"2024-05-30T17:10:06.542967+0000","last_fullsized":"2024-05-30T17:10:06.542967+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:30:19.299666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"0'0","reported_seq":50,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:01.731526+0000","last_change":"2024-05-30T16:17:31.192981+0000","last_active":"2024-05-30T17:10:01.731526+0000","last_peered":"2024-05-30T17:10:01.731526+0000","last_clean":"2024-05-30T17:10:01.731526+0000","last_became_active":"2024-05-30T16:17:31.192269+0000","last_became_peered":"2024-05-30T16:17:31.192269+0000","last_unstale":"2024-05-30T17:10:01.731526+0000","last_undegraded":"2024-05-30T17:10:01.731526+0000","last_fullsized":"2024-05-30T17:10:01.731526+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:47:56.727573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"265'25","reported_seq":125,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:06.579930+0000","last_change":"2024-05-30T16:12:50.817479+0000","last_active":"2024-05-30T17:11:06.579930+0000","last_peered":"2024-05-30T17:11:06.579930+0000","last_clean":"2024-05-30T17:11:06.579930+0000","last_became_active":"2024-05-30T16:12:50.811732+0000","last_became_peered":"2024-05-30T16:12:50.811732+0000","last_unstale":"2024-05-30T17:11:06.579930+0000","last_undegraded":"2024-05-30T17:11:06.579930+0000","last_fullsized":"2024-05-30T17:11:06.579930+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:22:47.071307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":85,"num_write":53,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":27,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.548778+0000","last_change":"2024-05-30T16:34:55.899801+0000","last_active":"2024-05-30T17:09:56.548778+0000","last_peered":"2024-05-30T17:09:56.548778+0000","last_clean":"2024-05-30T17:09:56.548778+0000","last_became_active":"2024-05-30T16:34:55.899241+0000","last_became_peered":"2024-05-30T16:34:55.899241+0000","last_unstale":"2024-05-30T17:09:56.548778+0000","last_undegraded":"2024-05-30T17:09:56.548778+0000","last_fullsized":"2024-05-30T17:09:56.548778+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:25:21.668824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"0'0","reported_seq":44,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:06.570530+0000","last_change":"2024-05-30T16:17:35.229918+0000","last_active":"2024-05-30T17:10:06.570530+0000","last_peered":"2024-05-30T17:10:06.570530+0000","last_clean":"2024-05-30T17:10:06.570530+0000","last_became_active":"2024-05-30T16:17:35.229034+0000","last_became_peered":"2024-05-30T16:17:35.229034+0000","last_unstale":"2024-05-30T17:10:06.570530+0000","last_undegraded":"2024-05-30T17:10:06.570530+0000","last_fullsized":"2024-05-30T17:10:06.570530+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:01:27.231981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"265'7","reported_seq":71,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:01.565640+0000","last_change":"2024-05-30T16:17:31.191604+0000","last_active":"2024-05-30T17:10:01.565640+0000","last_peered":"2024-05-30T17:10:01.565640+0000","last_clean":"2024-05-30T17:10:01.565640+0000","last_became_active":"2024-05-30T16:17:31.190742+0000","last_became_peered":"2024-05-30T16:17:31.190742+0000","last_unstale":"2024-05-30T17:10:01.565640+0000","last_undegraded":"2024-05-30T17:10:01.565640+0000","last_fullsized":"2024-05-30T17:10:01.565640+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:37:20.245898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485760,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":30722,"num_write":16,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"265'33","reported_seq":143,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.785481+0000","last_change":"2024-05-30T16:12:50.821208+0000","last_active":"2024-05-30T17:10:51.785481+0000","last_peered":"2024-05-30T17:10:51.785481+0000","last_clean":"2024-05-30T17:10:51.785481+0000","last_became_active":"2024-05-30T16:12:50.820343+0000","last_became_peered":"2024-05-30T16:12:50.820343+0000","last_unstale":"2024-05-30T17:10:51.785481+0000","last_undegraded":"2024-05-30T17:10:51.785481+0000","last_fullsized":"2024-05-30T17:10:51.785481+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:46:09.411694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":154,"num_read_kb":114,"num_write":68,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.772692+0000","last_change":"2024-05-30T16:34:55.889437+0000","last_active":"2024-05-30T17:09:56.772692+0000","last_peered":"2024-05-30T17:09:56.772692+0000","last_clean":"2024-05-30T17:09:56.772692+0000","last_became_active":"2024-05-30T16:34:55.888238+0000","last_became_peered":"2024-05-30T16:34:55.888238+0000","last_unstale":"2024-05-30T17:09:56.772692+0000","last_undegraded":"2024-05-30T17:09:56.772692+0000","last_fullsized":"2024-05-30T17:09:56.772692+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:23:02.664317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"262'6","reported_seq":56,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.579797+0000","last_change":"2024-05-30T16:17:35.740004+0000","last_active":"2024-05-30T17:10:46.579797+0000","last_peered":"2024-05-30T17:10:46.579797+0000","last_clean":"2024-05-30T17:10:46.579797+0000","last_became_active":"2024-05-30T16:17:35.738420+0000","last_became_peered":"2024-05-30T16:17:35.738420+0000","last_unstale":"2024-05-30T17:10:46.579797+0000","last_undegraded":"2024-05-30T17:10:46.579797+0000","last_fullsized":"2024-05-30T17:10:46.579797+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:38:21.023993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6144,"num_write":16,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"265'4","reported_seq":59,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:08:57.120072+0000","last_change":"2024-05-30T16:17:31.195609+0000","last_active":"2024-05-30T17:08:57.120072+0000","last_peered":"2024-05-30T17:08:57.120072+0000","last_clean":"2024-05-30T17:08:57.120072+0000","last_became_active":"2024-05-30T16:17:31.195031+0000","last_became_peered":"2024-05-30T16:17:31.195031+0000","last_unstale":"2024-05-30T17:08:57.120072+0000","last_undegraded":"2024-05-30T17:08:57.120072+0000","last_fullsized":"2024-05-30T17:08:57.120072+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:38:54.212446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":584363,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":6715,"num_write":4,"num_write_kb":6715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"265'65","reported_seq":237,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.577047+0000","last_change":"2024-05-30T16:12:50.820938+0000","last_active":"2024-05-30T17:10:51.577047+0000","last_peered":"2024-05-30T17:10:51.577047+0000","last_clean":"2024-05-30T17:10:51.577047+0000","last_became_active":"2024-05-30T16:12:50.807771+0000","last_became_peered":"2024-05-30T16:12:50.807771+0000","last_unstale":"2024-05-30T17:10:51.577047+0000","last_undegraded":"2024-05-30T17:10:51.577047+0000","last_fullsized":"2024-05-30T17:10:51.577047+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:20:23.418802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":343,"num_read_kb":242,"num_write":116,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"265'1","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.543341+0000","last_change":"2024-05-30T16:34:55.873393+0000","last_active":"2024-05-30T17:09:56.543341+0000","last_peered":"2024-05-30T17:09:56.543341+0000","last_clean":"2024-05-30T17:09:56.543341+0000","last_became_active":"2024-05-30T16:34:55.872581+0000","last_became_peered":"2024-05-30T16:34:55.872581+0000","last_unstale":"2024-05-30T17:09:56.543341+0000","last_undegraded":"2024-05-30T17:09:56.543341+0000","last_fullsized":"2024-05-30T17:09:56.543341+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:23:46.116567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"0'0","reported_seq":45,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:11.557772+0000","last_change":"2024-05-30T16:17:35.739267+0000","last_active":"2024-05-30T17:09:11.557772+0000","last_peered":"2024-05-30T17:09:11.557772+0000","last_clean":"2024-05-30T17:09:11.557772+0000","last_became_active":"2024-05-30T16:17:35.738397+0000","last_became_peered":"2024-05-30T16:17:35.738397+0000","last_unstale":"2024-05-30T17:09:11.557772+0000","last_undegraded":"2024-05-30T17:09:11.557772+0000","last_fullsized":"2024-05-30T17:09:11.557772+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:09:46.285921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"265'2","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:41.559336+0000","last_change":"2024-05-30T16:17:31.197103+0000","last_active":"2024-05-30T17:10:41.559336+0000","last_peered":"2024-05-30T17:10:41.559336+0000","last_clean":"2024-05-30T17:10:41.559336+0000","last_became_active":"2024-05-30T16:17:31.196345+0000","last_became_peered":"2024-05-30T16:17:31.196345+0000","last_unstale":"2024-05-30T17:10:41.559336+0000","last_undegraded":"2024-05-30T17:10:41.559336+0000","last_fullsized":"2024-05-30T17:10:41.559336+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:39:32.522011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":5120,"num_write":2,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"265'38","reported_seq":159,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.581145+0000","last_change":"2024-05-30T16:12:50.827196+0000","last_active":"2024-05-30T17:10:51.581145+0000","last_peered":"2024-05-30T17:10:51.581145+0000","last_clean":"2024-05-30T17:10:51.581145+0000","last_became_active":"2024-05-30T16:12:50.826729+0000","last_became_peered":"2024-05-30T16:12:50.826729+0000","last_unstale":"2024-05-30T17:10:51.581145+0000","last_undegraded":"2024-05-30T17:10:51.581145+0000","last_fullsized":"2024-05-30T17:10:51.581145+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:46:31.837238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":187,"num_read_kb":132,"num_write":81,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.548483+0000","last_change":"2024-05-30T16:34:55.876495+0000","last_active":"2024-05-30T17:09:56.548483+0000","last_peered":"2024-05-30T17:09:56.548483+0000","last_clean":"2024-05-30T17:09:56.548483+0000","last_became_active":"2024-05-30T16:34:55.875788+0000","last_became_peered":"2024-05-30T16:34:55.875788+0000","last_unstale":"2024-05-30T17:09:56.548483+0000","last_undegraded":"2024-05-30T17:09:56.548483+0000","last_fullsized":"2024-05-30T17:09:56.548483+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:15:31.389508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"0'0","reported_seq":45,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.579917+0000","last_change":"2024-05-30T16:17:35.746542+0000","last_active":"2024-05-30T17:10:46.579917+0000","last_peered":"2024-05-30T17:10:46.579917+0000","last_clean":"2024-05-30T17:10:46.579917+0000","last_became_active":"2024-05-30T16:17:35.745769+0000","last_became_peered":"2024-05-30T16:17:35.745769+0000","last_unstale":"2024-05-30T17:10:46.579917+0000","last_undegraded":"2024-05-30T17:10:46.579917+0000","last_fullsized":"2024-05-30T17:10:46.579917+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:03:42.283507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"265'1","reported_seq":53,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:41.552055+0000","last_change":"2024-05-30T16:17:31.206243+0000","last_active":"2024-05-30T17:10:41.552055+0000","last_peered":"2024-05-30T17:10:41.552055+0000","last_clean":"2024-05-30T17:10:41.552055+0000","last_became_active":"2024-05-30T16:17:31.204434+0000","last_became_peered":"2024-05-30T16:17:31.204434+0000","last_unstale":"2024-05-30T17:10:41.552055+0000","last_undegraded":"2024-05-30T17:10:41.552055+0000","last_fullsized":"2024-05-30T17:10:41.552055+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:33:27.089723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":3072,"num_write":1,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"265'26","reported_seq":125,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.554131+0000","last_change":"2024-05-30T16:12:50.814716+0000","last_active":"2024-05-30T17:10:51.554131+0000","last_peered":"2024-05-30T17:10:51.554131+0000","last_clean":"2024-05-30T17:10:51.554131+0000","last_became_active":"2024-05-30T16:12:50.813176+0000","last_became_peered":"2024-05-30T16:12:50.813176+0000","last_unstale":"2024-05-30T17:10:51.554131+0000","last_undegraded":"2024-05-30T17:10:51.554131+0000","last_fullsized":"2024-05-30T17:10:51.554131+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:45:17.112612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":85,"num_write":57,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:57.133868+0000","last_change":"2024-05-30T16:34:55.872909+0000","last_active":"2024-05-30T17:09:57.133868+0000","last_peered":"2024-05-30T17:09:57.133868+0000","last_clean":"2024-05-30T17:09:57.133868+0000","last_became_active":"2024-05-30T16:34:55.871800+0000","last_became_peered":"2024-05-30T16:34:55.871800+0000","last_unstale":"2024-05-30T17:09:57.133868+0000","last_undegraded":"2024-05-30T17:09:57.133868+0000","last_fullsized":"2024-05-30T17:09:57.133868+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:10:45.189484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"265'1","reported_seq":47,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.741327+0000","last_change":"2024-05-30T16:17:35.237595+0000","last_active":"2024-05-30T17:10:46.741327+0000","last_peered":"2024-05-30T17:10:46.741327+0000","last_clean":"2024-05-30T17:10:46.741327+0000","last_became_active":"2024-05-30T16:17:35.236867+0000","last_became_peered":"2024-05-30T16:17:35.236867+0000","last_unstale":"2024-05-30T17:10:46.741327+0000","last_undegraded":"2024-05-30T17:10:46.741327+0000","last_fullsized":"2024-05-30T17:10:46.741327+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:47:01.389981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":584363,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":571,"num_write":1,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"0'0","reported_seq":51,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:41.574782+0000","last_change":"2024-05-30T16:17:31.210713+0000","last_active":"2024-05-30T17:10:41.574782+0000","last_peered":"2024-05-30T17:10:41.574782+0000","last_clean":"2024-05-30T17:10:41.574782+0000","last_became_active":"2024-05-30T16:17:31.209882+0000","last_became_peered":"2024-05-30T16:17:31.209882+0000","last_unstale":"2024-05-30T17:10:41.574782+0000","last_undegraded":"2024-05-30T17:10:41.574782+0000","last_fullsized":"2024-05-30T17:10:41.574782+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:17:20.852336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"265'56","reported_seq":218,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:01.749663+0000","last_change":"2024-05-30T16:12:50.825181+0000","last_active":"2024-05-30T17:09:01.749663+0000","last_peered":"2024-05-30T17:09:01.749663+0000","last_clean":"2024-05-30T17:09:01.749663+0000","last_became_active":"2024-05-30T16:12:50.824043+0000","last_became_peered":"2024-05-30T16:12:50.824043+0000","last_unstale":"2024-05-30T17:09:01.749663+0000","last_undegraded":"2024-05-30T17:09:01.749663+0000","last_fullsized":"2024-05-30T17:09:01.749663+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:29.566016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":308,"num_read_kb":221,"num_write":92,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.585310+0000","last_change":"2024-05-30T16:34:55.893966+0000","last_active":"2024-05-30T17:11:11.585310+0000","last_peered":"2024-05-30T17:11:11.585310+0000","last_clean":"2024-05-30T17:11:11.585310+0000","last_became_active":"2024-05-30T16:34:55.893040+0000","last_became_peered":"2024-05-30T16:34:55.893040+0000","last_unstale":"2024-05-30T17:11:11.585310+0000","last_undegraded":"2024-05-30T17:11:11.585310+0000","last_fullsized":"2024-05-30T17:11:11.585310+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:55:00.349712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"0'0","reported_seq":46,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.790902+0000","last_change":"2024-05-30T16:17:35.246948+0000","last_active":"2024-05-30T17:11:16.790902+0000","last_peered":"2024-05-30T17:11:16.790902+0000","last_clean":"2024-05-30T17:11:16.790902+0000","last_became_active":"2024-05-30T16:17:35.246187+0000","last_became_peered":"2024-05-30T16:17:35.246187+0000","last_unstale":"2024-05-30T17:11:16.790902+0000","last_undegraded":"2024-05-30T17:11:16.790902+0000","last_fullsized":"2024-05-30T17:11:16.790902+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:23:58.794231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"265'6","reported_seq":69,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.746911+0000","last_change":"2024-05-30T16:17:31.189808+0000","last_active":"2024-05-30T17:11:11.746911+0000","last_peered":"2024-05-30T17:11:11.746911+0000","last_clean":"2024-05-30T17:11:11.746911+0000","last_became_active":"2024-05-30T16:17:31.189091+0000","last_became_peered":"2024-05-30T16:17:31.189091+0000","last_unstale":"2024-05-30T17:11:11.746911+0000","last_undegraded":"2024-05-30T17:11:11.746911+0000","last_fullsized":"2024-05-30T17:11:11.746911+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:35:04.039725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5827243,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":21053,"num_write":5,"num_write_kb":5691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"265'9","reported_seq":84,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.566377+0000","last_change":"2024-05-30T16:12:50.824521+0000","last_active":"2024-05-30T17:11:11.566377+0000","last_peered":"2024-05-30T17:11:11.566377+0000","last_clean":"2024-05-30T17:11:11.566377+0000","last_became_active":"2024-05-30T16:12:50.823759+0000","last_became_peered":"2024-05-30T16:12:50.823759+0000","last_unstale":"2024-05-30T17:11:11.566377+0000","last_undegraded":"2024-05-30T17:11:11.566377+0000","last_fullsized":"2024-05-30T17:11:11.566377+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:41:59.872702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":33,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.581162+0000","last_change":"2024-05-30T16:34:55.861418+0000","last_active":"2024-05-30T17:11:11.581162+0000","last_peered":"2024-05-30T17:11:11.581162+0000","last_clean":"2024-05-30T17:11:11.581162+0000","last_became_active":"2024-05-30T16:34:55.860695+0000","last_became_peered":"2024-05-30T16:34:55.860695+0000","last_unstale":"2024-05-30T17:11:11.581162+0000","last_undegraded":"2024-05-30T17:11:11.581162+0000","last_fullsized":"2024-05-30T17:11:11.581162+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:45:54.013286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"0'0","reported_seq":46,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.790812+0000","last_change":"2024-05-30T16:17:35.239935+0000","last_active":"2024-05-30T17:11:16.790812+0000","last_peered":"2024-05-30T17:11:16.790812+0000","last_clean":"2024-05-30T17:11:16.790812+0000","last_became_active":"2024-05-30T16:17:35.239157+0000","last_became_peered":"2024-05-30T16:17:35.239157+0000","last_unstale":"2024-05-30T17:11:16.790812+0000","last_undegraded":"2024-05-30T17:11:16.790812+0000","last_fullsized":"2024-05-30T17:11:16.790812+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:53:43.552214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"265'1","reported_seq":54,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.585076+0000","last_change":"2024-05-30T16:17:31.185375+0000","last_active":"2024-05-30T17:11:11.585076+0000","last_peered":"2024-05-30T17:11:11.585076+0000","last_clean":"2024-05-30T17:11:11.585076+0000","last_became_active":"2024-05-30T16:17:31.184675+0000","last_became_peered":"2024-05-30T16:17:31.184675+0000","last_unstale":"2024-05-30T17:11:11.585076+0000","last_undegraded":"2024-05-30T17:11:11.585076+0000","last_fullsized":"2024-05-30T17:11:11.585076+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:13:30.582952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":714225,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":698,"num_write":1,"num_write_kb":698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"262'96","reported_seq":357,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.581035+0000","last_change":"2024-05-30T16:12:50.802561+0000","last_active":"2024-05-30T17:11:11.581035+0000","last_peered":"2024-05-30T17:11:11.581035+0000","last_clean":"2024-05-30T17:11:11.581035+0000","last_became_active":"2024-05-30T16:12:50.801791+0000","last_became_peered":"2024-05-30T16:12:50.801791+0000","last_unstale":"2024-05-30T17:11:11.581035+0000","last_undegraded":"2024-05-30T17:11:11.581035+0000","last_fullsized":"2024-05-30T17:11:11.581035+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:19:14.883602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":591,"num_read_kb":446,"num_write":104,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"264'1","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:06.566810+0000","last_change":"2024-05-30T16:34:55.864834+0000","last_active":"2024-05-30T17:10:06.566810+0000","last_peered":"2024-05-30T17:10:06.566810+0000","last_clean":"2024-05-30T17:10:06.566810+0000","last_became_active":"2024-05-30T16:34:55.863483+0000","last_became_peered":"2024-05-30T16:34:55.863483+0000","last_unstale":"2024-05-30T17:10:06.566810+0000","last_undegraded":"2024-05-30T17:10:06.566810+0000","last_fullsized":"2024-05-30T17:10:06.566810+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:46:20.034900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"0'0","reported_seq":46,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.586383+0000","last_change":"2024-05-30T16:17:35.236085+0000","last_active":"2024-05-30T17:11:16.586383+0000","last_peered":"2024-05-30T17:11:16.586383+0000","last_clean":"2024-05-30T17:11:16.586383+0000","last_became_active":"2024-05-30T16:17:35.235283+0000","last_became_peered":"2024-05-30T16:17:35.235283+0000","last_unstale":"2024-05-30T17:11:16.586383+0000","last_undegraded":"2024-05-30T17:11:16.586383+0000","last_fullsized":"2024-05-30T17:11:16.586383+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:19:44.053076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"0'0","reported_seq":52,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.557098+0000","last_change":"2024-05-30T16:17:31.204968+0000","last_active":"2024-05-30T17:11:11.557098+0000","last_peered":"2024-05-30T17:11:11.557098+0000","last_clean":"2024-05-30T17:11:11.557098+0000","last_became_active":"2024-05-30T16:17:31.199499+0000","last_became_peered":"2024-05-30T16:17:31.199499+0000","last_unstale":"2024-05-30T17:11:11.557098+0000","last_undegraded":"2024-05-30T17:11:11.557098+0000","last_fullsized":"2024-05-30T17:11:11.557098+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:18:34.513889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"265'22","reported_seq":118,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.746972+0000","last_change":"2024-05-30T16:12:50.807855+0000","last_active":"2024-05-30T17:11:11.746972+0000","last_peered":"2024-05-30T17:11:11.746972+0000","last_clean":"2024-05-30T17:11:11.746972+0000","last_became_active":"2024-05-30T16:12:50.807156+0000","last_became_peered":"2024-05-30T16:12:50.807156+0000","last_unstale":"2024-05-30T17:11:11.746972+0000","last_undegraded":"2024-05-30T17:11:11.746972+0000","last_fullsized":"2024-05-30T17:11:11.746972+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:18:22.683470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":76,"num_write":50,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"265'1","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.585231+0000","last_change":"2024-05-30T16:34:55.889162+0000","last_active":"2024-05-30T17:11:11.585231+0000","last_peered":"2024-05-30T17:11:11.585231+0000","last_clean":"2024-05-30T17:11:11.585231+0000","last_became_active":"2024-05-30T16:34:55.887845+0000","last_became_peered":"2024-05-30T16:34:55.887845+0000","last_unstale":"2024-05-30T17:11:11.585231+0000","last_undegraded":"2024-05-30T17:11:11.585231+0000","last_fullsized":"2024-05-30T17:11:11.585231+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:36:50.835387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"0'0","reported_seq":46,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.558367+0000","last_change":"2024-05-30T16:17:35.246711+0000","last_active":"2024-05-30T17:11:16.558367+0000","last_peered":"2024-05-30T17:11:16.558367+0000","last_clean":"2024-05-30T17:11:16.558367+0000","last_became_active":"2024-05-30T16:17:35.245852+0000","last_became_peered":"2024-05-30T16:17:35.245852+0000","last_unstale":"2024-05-30T17:11:16.558367+0000","last_undegraded":"2024-05-30T17:11:16.558367+0000","last_fullsized":"2024-05-30T17:11:16.558367+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:50:22.430770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"262'5","reported_seq":67,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:12.150705+0000","last_change":"2024-05-30T16:17:31.219308+0000","last_active":"2024-05-30T17:11:12.150705+0000","last_peered":"2024-05-30T17:11:12.150705+0000","last_clean":"2024-05-30T17:11:12.150705+0000","last_became_active":"2024-05-30T16:17:31.218752+0000","last_became_peered":"2024-05-30T16:17:31.218752+0000","last_unstale":"2024-05-30T17:11:12.150705+0000","last_undegraded":"2024-05-30T17:11:12.150705+0000","last_fullsized":"2024-05-30T17:11:12.150705+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:47:57.704377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"265'17","reported_seq":103,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.584982+0000","last_change":"2024-05-30T16:12:50.829602+0000","last_active":"2024-05-30T17:11:11.584982+0000","last_peered":"2024-05-30T17:11:11.584982+0000","last_clean":"2024-05-30T17:11:11.584982+0000","last_became_active":"2024-05-30T16:12:50.817114+0000","last_became_peered":"2024-05-30T16:12:50.817114+0000","last_unstale":"2024-05-30T17:11:11.584982+0000","last_undegraded":"2024-05-30T17:11:11.584982+0000","last_fullsized":"2024-05-30T17:11:11.584982+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:05:48.646667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":55,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:46.541438+0000","last_change":"2024-05-30T16:34:55.867126+0000","last_active":"2024-05-30T17:09:46.541438+0000","last_peered":"2024-05-30T17:09:46.541438+0000","last_clean":"2024-05-30T17:09:46.541438+0000","last_became_active":"2024-05-30T16:34:55.866415+0000","last_became_peered":"2024-05-30T16:34:55.866415+0000","last_unstale":"2024-05-30T17:09:46.541438+0000","last_undegraded":"2024-05-30T17:09:46.541438+0000","last_fullsized":"2024-05-30T17:09:46.541438+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:05:20.422821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"0'0","reported_seq":46,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.560610+0000","last_change":"2024-05-30T16:17:35.228446+0000","last_active":"2024-05-30T17:09:21.560610+0000","last_peered":"2024-05-30T17:09:21.560610+0000","last_clean":"2024-05-30T17:09:21.560610+0000","last_became_active":"2024-05-30T16:17:35.227270+0000","last_became_peered":"2024-05-30T16:17:35.227270+0000","last_unstale":"2024-05-30T17:09:21.560610+0000","last_undegraded":"2024-05-30T17:09:21.560610+0000","last_fullsized":"2024-05-30T17:09:21.560610+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:47:07.315021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"0'0","reported_seq":52,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:16.753258+0000","last_change":"2024-05-30T16:17:31.204447+0000","last_active":"2024-05-30T17:09:16.753258+0000","last_peered":"2024-05-30T17:09:16.753258+0000","last_clean":"2024-05-30T17:09:16.753258+0000","last_became_active":"2024-05-30T16:17:31.203731+0000","last_became_peered":"2024-05-30T16:17:31.203731+0000","last_unstale":"2024-05-30T17:09:16.753258+0000","last_undegraded":"2024-05-30T17:09:16.753258+0000","last_fullsized":"2024-05-30T17:09:16.753258+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:02:51.039638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"265'29","reported_seq":136,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.560856+0000","last_change":"2024-05-30T16:12:50.825208+0000","last_active":"2024-05-30T17:09:21.560856+0000","last_peered":"2024-05-30T17:09:21.560856+0000","last_clean":"2024-05-30T17:09:21.560856+0000","last_became_active":"2024-05-30T16:12:50.824809+0000","last_became_peered":"2024-05-30T16:12:50.824809+0000","last_unstale":"2024-05-30T17:09:21.560856+0000","last_undegraded":"2024-05-30T17:09:21.560856+0000","last_fullsized":"2024-05-30T17:09:21.560856+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:38:19.384141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":98,"num_write":77,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:56.562832+0000","last_change":"2024-05-30T16:34:55.897061+0000","last_active":"2024-05-30T17:10:56.562832+0000","last_peered":"2024-05-30T17:10:56.562832+0000","last_clean":"2024-05-30T17:10:56.562832+0000","last_became_active":"2024-05-30T16:34:55.896357+0000","last_became_peered":"2024-05-30T16:34:55.896357+0000","last_unstale":"2024-05-30T17:10:56.562832+0000","last_undegraded":"2024-05-30T17:10:56.562832+0000","last_fullsized":"2024-05-30T17:10:56.562832+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:23:44.881356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"265'2","reported_seq":44,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:42.130647+0000","last_change":"2024-05-30T16:17:55.551397+0000","last_active":"2024-05-30T17:09:42.130647+0000","last_peered":"2024-05-30T17:09:42.130647+0000","last_clean":"2024-05-30T17:09:42.130647+0000","last_became_active":"2024-05-30T16:17:55.550782+0000","last_became_peered":"2024-05-30T16:17:55.550782+0000","last_unstale":"2024-05-30T17:09:42.130647+0000","last_undegraded":"2024-05-30T17:09:42.130647+0000","last_fullsized":"2024-05-30T17:09:42.130647+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:44:58.931548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2048,"num_write":7,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.d","version":"265'5","reported_seq":62,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:16.535485+0000","last_change":"2024-05-30T16:17:31.202846+0000","last_active":"2024-05-30T17:09:16.535485+0000","last_peered":"2024-05-30T17:09:16.535485+0000","last_clean":"2024-05-30T17:09:16.535485+0000","last_became_active":"2024-05-30T16:17:31.202042+0000","last_became_peered":"2024-05-30T16:17:31.202042+0000","last_unstale":"2024-05-30T17:09:16.535485+0000","last_undegraded":"2024-05-30T17:09:16.535485+0000","last_fullsized":"2024-05-30T17:09:16.535485+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:38:37.711263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6339871,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":6194,"num_write":5,"num_write_kb":6194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"265'24","reported_seq":130,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:22.153145+0000","last_change":"2024-05-30T16:12:50.810823+0000","last_active":"2024-05-30T17:11:22.153145+0000","last_peered":"2024-05-30T17:11:22.153145+0000","last_clean":"2024-05-30T17:11:22.153145+0000","last_became_active":"2024-05-30T16:12:50.810095+0000","last_became_peered":"2024-05-30T16:12:50.810095+0000","last_unstale":"2024-05-30T17:11:22.153145+0000","last_undegraded":"2024-05-30T17:11:22.153145+0000","last_fullsized":"2024-05-30T17:11:22.153145+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:29:53.744702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":90,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.536265+0000","last_change":"2024-05-30T16:34:55.863218+0000","last_active":"2024-05-30T17:09:36.536265+0000","last_peered":"2024-05-30T17:09:36.536265+0000","last_clean":"2024-05-30T17:09:36.536265+0000","last_became_active":"2024-05-30T16:34:55.862478+0000","last_became_peered":"2024-05-30T16:34:55.862478+0000","last_unstale":"2024-05-30T17:09:36.536265+0000","last_undegraded":"2024-05-30T17:09:36.536265+0000","last_fullsized":"2024-05-30T17:09:36.536265+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:30.967780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"262'3","reported_seq":52,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:31.768287+0000","last_change":"2024-05-30T16:17:35.743152+0000","last_active":"2024-05-30T17:09:31.768287+0000","last_peered":"2024-05-30T17:09:31.768287+0000","last_clean":"2024-05-30T17:09:31.768287+0000","last_became_active":"2024-05-30T16:17:35.742255+0000","last_became_peered":"2024-05-30T16:17:35.742255+0000","last_unstale":"2024-05-30T17:09:31.768287+0000","last_undegraded":"2024-05-30T17:09:31.768287+0000","last_fullsized":"2024-05-30T17:09:31.768287+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:41:50.476109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"265'1","reported_seq":54,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:16.556005+0000","last_change":"2024-05-30T16:17:31.199481+0000","last_active":"2024-05-30T17:09:16.556005+0000","last_peered":"2024-05-30T17:09:16.556005+0000","last_clean":"2024-05-30T17:09:16.556005+0000","last_became_active":"2024-05-30T16:17:31.198744+0000","last_became_peered":"2024-05-30T16:17:31.198744+0000","last_unstale":"2024-05-30T17:09:16.556005+0000","last_undegraded":"2024-05-30T17:09:16.556005+0000","last_fullsized":"2024-05-30T17:09:16.556005+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:39:35.328874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":3072,"num_write":1,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"265'36","reported_seq":163,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.559692+0000","last_change":"2024-05-30T16:12:50.821672+0000","last_active":"2024-05-30T17:11:21.559692+0000","last_peered":"2024-05-30T17:11:21.559692+0000","last_clean":"2024-05-30T17:11:21.559692+0000","last_became_active":"2024-05-30T16:12:50.820936+0000","last_became_peered":"2024-05-30T16:12:50.820936+0000","last_unstale":"2024-05-30T17:11:21.559692+0000","last_undegraded":"2024-05-30T17:11:21.559692+0000","last_fullsized":"2024-05-30T17:11:21.559692+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:52:42.146262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":194,"num_read_kb":136,"num_write":73,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.543696+0000","last_change":"2024-05-30T16:34:55.877239+0000","last_active":"2024-05-30T17:09:36.543696+0000","last_peered":"2024-05-30T17:09:36.543696+0000","last_clean":"2024-05-30T17:09:36.543696+0000","last_became_active":"2024-05-30T16:34:55.875965+0000","last_became_peered":"2024-05-30T16:34:55.875965+0000","last_unstale":"2024-05-30T17:09:36.543696+0000","last_undegraded":"2024-05-30T17:09:36.543696+0000","last_fullsized":"2024-05-30T17:09:36.543696+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:10:50.846236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"265'2","reported_seq":50,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.560723+0000","last_change":"2024-05-30T16:17:35.234741+0000","last_active":"2024-05-30T17:09:21.560723+0000","last_peered":"2024-05-30T17:09:21.560723+0000","last_clean":"2024-05-30T17:09:21.560723+0000","last_became_active":"2024-05-30T16:17:35.229643+0000","last_became_peered":"2024-05-30T16:17:35.229643+0000","last_unstale":"2024-05-30T17:09:21.560723+0000","last_undegraded":"2024-05-30T17:09:21.560723+0000","last_fullsized":"2024-05-30T17:09:21.560723+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:55:14.642511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":6144,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.f","version":"262'6","reported_seq":63,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:26.767116+0000","last_change":"2024-05-30T16:17:31.215290+0000","last_active":"2024-05-30T17:09:26.767116+0000","last_peered":"2024-05-30T17:09:26.767116+0000","last_clean":"2024-05-30T17:09:26.767116+0000","last_became_active":"2024-05-30T16:17:31.212894+0000","last_became_peered":"2024-05-30T16:17:31.212894+0000","last_unstale":"2024-05-30T17:09:26.767116+0000","last_undegraded":"2024-05-30T17:09:26.767116+0000","last_fullsized":"2024-05-30T17:09:26.767116+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:31:09.789041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6144,"num_write":11,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"262'21","reported_seq":121,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.559938+0000","last_change":"2024-05-30T16:12:50.806577+0000","last_active":"2024-05-30T17:11:21.559938+0000","last_peered":"2024-05-30T17:11:21.559938+0000","last_clean":"2024-05-30T17:11:21.559938+0000","last_became_active":"2024-05-30T16:12:50.805772+0000","last_became_peered":"2024-05-30T16:12:50.805772+0000","last_unstale":"2024-05-30T17:11:21.559938+0000","last_undegraded":"2024-05-30T17:11:21.559938+0000","last_fullsized":"2024-05-30T17:11:21.559938+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:11:36.510014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":79,"num_write":45,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:56.582343+0000","last_change":"2024-05-30T16:34:55.891074+0000","last_active":"2024-05-30T17:10:56.582343+0000","last_peered":"2024-05-30T17:10:56.582343+0000","last_clean":"2024-05-30T17:10:56.582343+0000","last_became_active":"2024-05-30T16:34:55.890313+0000","last_became_peered":"2024-05-30T16:34:55.890313+0000","last_unstale":"2024-05-30T17:10:56.582343+0000","last_undegraded":"2024-05-30T17:10:56.582343+0000","last_fullsized":"2024-05-30T17:10:56.582343+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:22:36.639758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"265'2","reported_seq":53,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.726171+0000","last_change":"2024-05-30T16:17:35.243089+0000","last_active":"2024-05-30T17:09:36.726171+0000","last_peered":"2024-05-30T17:09:36.726171+0000","last_clean":"2024-05-30T17:09:36.726171+0000","last_became_active":"2024-05-30T16:17:35.242271+0000","last_became_peered":"2024-05-30T16:17:35.242271+0000","last_unstale":"2024-05-30T17:09:36.726171+0000","last_undegraded":"2024-05-30T17:09:36.726171+0000","last_fullsized":"2024-05-30T17:09:36.726171+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:44:49.304400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":10240,"num_write":7,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"265'2","reported_seq":59,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:31.559803+0000","last_change":"2024-05-30T16:17:31.195571+0000","last_active":"2024-05-30T17:09:31.559803+0000","last_peered":"2024-05-30T17:09:31.559803+0000","last_clean":"2024-05-30T17:09:31.559803+0000","last_became_active":"2024-05-30T16:17:31.194774+0000","last_became_peered":"2024-05-30T16:17:31.194774+0000","last_unstale":"2024-05-30T17:09:31.559803+0000","last_undegraded":"2024-05-30T17:09:31.559803+0000","last_fullsized":"2024-05-30T17:09:31.559803+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:24:47.216863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":12288,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"265'18","reported_seq":107,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.584196+0000","last_change":"2024-05-30T16:12:50.820954+0000","last_active":"2024-05-30T17:11:21.584196+0000","last_peered":"2024-05-30T17:11:21.584196+0000","last_clean":"2024-05-30T17:11:21.584196+0000","last_became_active":"2024-05-30T16:12:50.820526+0000","last_became_peered":"2024-05-30T16:12:50.820526+0000","last_unstale":"2024-05-30T17:11:21.584196+0000","last_undegraded":"2024-05-30T17:11:21.584196+0000","last_fullsized":"2024-05-30T17:11:21.584196+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:08:41.493544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":60,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:06.565282+0000","last_change":"2024-05-30T16:34:55.884293+0000","last_active":"2024-05-30T17:11:06.565282+0000","last_peered":"2024-05-30T17:11:06.565282+0000","last_clean":"2024-05-30T17:11:06.565282+0000","last_became_active":"2024-05-30T16:34:55.884065+0000","last_became_peered":"2024-05-30T16:34:55.884065+0000","last_unstale":"2024-05-30T17:11:06.565282+0000","last_undegraded":"2024-05-30T17:11:06.565282+0000","last_fullsized":"2024-05-30T17:11:06.565282+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:13:34.815638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"265'9","reported_seq":71,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.543613+0000","last_change":"2024-05-30T16:17:35.739756+0000","last_active":"2024-05-30T17:09:36.543613+0000","last_peered":"2024-05-30T17:09:36.543613+0000","last_clean":"2024-05-30T17:09:36.543613+0000","last_became_active":"2024-05-30T16:17:35.738991+0000","last_became_peered":"2024-05-30T16:17:35.738991+0000","last_unstale":"2024-05-30T17:09:36.543613+0000","last_undegraded":"2024-05-30T17:09:36.543613+0000","last_fullsized":"2024-05-30T17:09:36.543613+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:50:21.381841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5957105,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":26998,"num_write":13,"num_write_kb":10938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"262'6","reported_seq":65,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:31.768080+0000","last_change":"2024-05-30T16:17:31.216617+0000","last_active":"2024-05-30T17:09:31.768080+0000","last_peered":"2024-05-30T17:09:31.768080+0000","last_clean":"2024-05-30T17:09:31.768080+0000","last_became_active":"2024-05-30T16:17:31.212954+0000","last_became_peered":"2024-05-30T16:17:31.212954+0000","last_unstale":"2024-05-30T17:09:31.768080+0000","last_undegraded":"2024-05-30T17:09:31.768080+0000","last_fullsized":"2024-05-30T17:09:31.768080+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:53:12.730929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":11264,"num_write":11,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"262'102","reported_seq":385,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.754543+0000","last_change":"2024-05-30T16:12:50.821280+0000","last_active":"2024-05-30T17:09:21.754543+0000","last_peered":"2024-05-30T17:09:21.754543+0000","last_clean":"2024-05-30T17:09:21.754543+0000","last_became_active":"2024-05-30T16:12:50.820250+0000","last_became_peered":"2024-05-30T16:12:50.820250+0000","last_unstale":"2024-05-30T17:09:21.754543+0000","last_undegraded":"2024-05-30T17:09:21.754543+0000","last_fullsized":"2024-05-30T17:09:21.754543+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:43:31.770630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":642,"num_read_kb":499,"num_write":114,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"265'1","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:56.562900+0000","last_change":"2024-05-30T16:34:55.880548+0000","last_active":"2024-05-30T17:10:56.562900+0000","last_peered":"2024-05-30T17:10:56.562900+0000","last_clean":"2024-05-30T17:10:56.562900+0000","last_became_active":"2024-05-30T16:34:55.878171+0000","last_became_peered":"2024-05-30T16:34:55.878171+0000","last_unstale":"2024-05-30T17:10:56.562900+0000","last_undegraded":"2024-05-30T17:10:56.562900+0000","last_fullsized":"2024-05-30T17:10:56.562900+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:56:14.165458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"265'11","reported_seq":71,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.765556+0000","last_change":"2024-05-30T16:17:35.245798+0000","last_active":"2024-05-30T17:09:21.765556+0000","last_peered":"2024-05-30T17:09:21.765556+0000","last_clean":"2024-05-30T17:09:21.765556+0000","last_became_active":"2024-05-30T16:17:35.245008+0000","last_became_peered":"2024-05-30T16:17:35.245008+0000","last_unstale":"2024-05-30T17:09:21.765556+0000","last_undegraded":"2024-05-30T17:09:21.765556+0000","last_fullsized":"2024-05-30T17:09:21.765556+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:28:02.631261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17825792,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":25602,"num_write":20,"num_write_kb":17408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"262'3","reported_seq":59,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:46.562533+0000","last_change":"2024-05-30T16:17:31.211050+0000","last_active":"2024-05-30T17:09:46.562533+0000","last_peered":"2024-05-30T17:09:46.562533+0000","last_clean":"2024-05-30T17:09:46.562533+0000","last_became_active":"2024-05-30T16:17:31.210195+0000","last_became_peered":"2024-05-30T16:17:31.210195+0000","last_unstale":"2024-05-30T17:09:46.562533+0000","last_undegraded":"2024-05-30T17:09:46.562533+0000","last_fullsized":"2024-05-30T17:09:46.562533+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:19:11.621024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"265'43","reported_seq":176,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.587423+0000","last_change":"2024-05-30T16:12:50.831498+0000","last_active":"2024-05-30T17:11:21.587423+0000","last_peered":"2024-05-30T17:11:21.587423+0000","last_clean":"2024-05-30T17:11:21.587423+0000","last_became_active":"2024-05-30T16:12:50.830770+0000","last_became_peered":"2024-05-30T16:12:50.830770+0000","last_unstale":"2024-05-30T17:11:21.587423+0000","last_undegraded":"2024-05-30T17:11:21.587423+0000","last_fullsized":"2024-05-30T17:11:21.587423+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:10:09.450788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":214,"num_read_kb":148,"num_write":89,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:56.578000+0000","last_change":"2024-05-30T16:34:55.887780+0000","last_active":"2024-05-30T17:10:56.578000+0000","last_peered":"2024-05-30T17:10:56.578000+0000","last_clean":"2024-05-30T17:10:56.578000+0000","last_became_active":"2024-05-30T16:34:55.887101+0000","last_became_peered":"2024-05-30T16:34:55.887101+0000","last_unstale":"2024-05-30T17:10:56.578000+0000","last_undegraded":"2024-05-30T17:10:56.578000+0000","last_fullsized":"2024-05-30T17:10:56.578000+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:43:47.335383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"262'3","reported_seq":53,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:51.547299+0000","last_change":"2024-05-30T16:17:35.213121+0000","last_active":"2024-05-30T17:09:51.547299+0000","last_peered":"2024-05-30T17:09:51.547299+0000","last_clean":"2024-05-30T17:09:51.547299+0000","last_became_active":"2024-05-30T16:17:35.212403+0000","last_became_peered":"2024-05-30T16:17:35.212403+0000","last_unstale":"2024-05-30T17:09:51.547299+0000","last_undegraded":"2024-05-30T17:09:51.547299+0000","last_fullsized":"2024-05-30T17:09:51.547299+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:52:56.999514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"0'0","reported_seq":52,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:16.721715+0000","last_change":"2024-05-30T16:17:31.192265+0000","last_active":"2024-05-30T17:09:16.721715+0000","last_peered":"2024-05-30T17:09:16.721715+0000","last_clean":"2024-05-30T17:09:16.721715+0000","last_became_active":"2024-05-30T16:17:31.191955+0000","last_became_peered":"2024-05-30T16:17:31.191955+0000","last_unstale":"2024-05-30T17:09:16.721715+0000","last_undegraded":"2024-05-30T17:09:16.721715+0000","last_fullsized":"2024-05-30T17:09:16.721715+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:16:24.140324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"262'8","reported_seq":83,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.559507+0000","last_change":"2024-05-30T16:12:50.820140+0000","last_active":"2024-05-30T17:11:21.559507+0000","last_peered":"2024-05-30T17:11:21.559507+0000","last_clean":"2024-05-30T17:11:21.559507+0000","last_became_active":"2024-05-30T16:12:50.819871+0000","last_became_peered":"2024-05-30T16:12:50.819871+0000","last_unstale":"2024-05-30T17:11:21.559507+0000","last_undegraded":"2024-05-30T17:11:21.559507+0000","last_fullsized":"2024-05-30T17:11:21.559507+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:14:08.269589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":28,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.554460+0000","last_change":"2024-05-30T16:34:55.883102+0000","last_active":"2024-05-30T17:10:21.554460+0000","last_peered":"2024-05-30T17:10:21.554460+0000","last_clean":"2024-05-30T17:10:21.554460+0000","last_became_active":"2024-05-30T16:34:55.882443+0000","last_became_peered":"2024-05-30T16:34:55.882443+0000","last_unstale":"2024-05-30T17:10:21.554460+0000","last_undegraded":"2024-05-30T17:10:21.554460+0000","last_fullsized":"2024-05-30T17:10:21.554460+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:01:28.745649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"262'3","reported_seq":54,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.564614+0000","last_change":"2024-05-30T16:17:35.248377+0000","last_active":"2024-05-30T17:09:56.564614+0000","last_peered":"2024-05-30T17:09:56.564614+0000","last_clean":"2024-05-30T17:09:56.564614+0000","last_became_active":"2024-05-30T16:17:35.247419+0000","last_became_peered":"2024-05-30T16:17:35.247419+0000","last_unstale":"2024-05-30T17:09:56.564614+0000","last_undegraded":"2024-05-30T17:09:56.564614+0000","last_fullsized":"2024-05-30T17:09:56.564614+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:09:58.043053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"262'2","reported_seq":58,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.543844+0000","last_change":"2024-05-30T16:17:31.189996+0000","last_active":"2024-05-30T17:09:36.543844+0000","last_peered":"2024-05-30T17:09:36.543844+0000","last_clean":"2024-05-30T17:09:36.543844+0000","last_became_active":"2024-05-30T16:17:31.189122+0000","last_became_peered":"2024-05-30T16:17:31.189122+0000","last_unstale":"2024-05-30T17:09:36.543844+0000","last_undegraded":"2024-05-30T17:09:36.543844+0000","last_fullsized":"2024-05-30T17:09:36.543844+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:53:04.697503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"265'53","reported_seq":201,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.584475+0000","last_change":"2024-05-30T16:12:50.821561+0000","last_active":"2024-05-30T17:11:21.584475+0000","last_peered":"2024-05-30T17:11:21.584475+0000","last_clean":"2024-05-30T17:11:21.584475+0000","last_became_active":"2024-05-30T16:12:50.809372+0000","last_became_peered":"2024-05-30T16:12:50.809372+0000","last_unstale":"2024-05-30T17:11:21.584475+0000","last_undegraded":"2024-05-30T17:11:21.584475+0000","last_fullsized":"2024-05-30T17:11:21.584475+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:22:57.030056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":186,"num_write":103,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:22.139649+0000","last_change":"2024-05-30T16:34:55.881085+0000","last_active":"2024-05-30T17:10:22.139649+0000","last_peered":"2024-05-30T17:10:22.139649+0000","last_clean":"2024-05-30T17:10:22.139649+0000","last_became_active":"2024-05-30T16:34:55.880627+0000","last_became_peered":"2024-05-30T16:34:55.880627+0000","last_unstale":"2024-05-30T17:10:22.139649+0000","last_undegraded":"2024-05-30T17:10:22.139649+0000","last_fullsized":"2024-05-30T17:10:22.139649+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:34:40.635399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"0'0","reported_seq":48,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:41.561793+0000","last_change":"2024-05-30T16:17:35.743808+0000","last_active":"2024-05-30T17:09:41.561793+0000","last_peered":"2024-05-30T17:09:41.561793+0000","last_clean":"2024-05-30T17:09:41.561793+0000","last_became_active":"2024-05-30T16:17:35.743036+0000","last_became_peered":"2024-05-30T16:17:35.743036+0000","last_unstale":"2024-05-30T17:09:41.561793+0000","last_undegraded":"2024-05-30T17:09:41.561793+0000","last_fullsized":"2024-05-30T17:09:41.561793+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:12:42.745946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"265'2","reported_seq":58,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:51.542424+0000","last_change":"2024-05-30T16:17:31.202885+0000","last_active":"2024-05-30T17:09:51.542424+0000","last_peered":"2024-05-30T17:09:51.542424+0000","last_clean":"2024-05-30T17:09:51.542424+0000","last_became_active":"2024-05-30T16:17:31.202064+0000","last_became_peered":"2024-05-30T16:17:31.202064+0000","last_unstale":"2024-05-30T17:09:51.542424+0000","last_undegraded":"2024-05-30T17:09:51.542424+0000","last_fullsized":"2024-05-30T17:09:51.542424+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:48:01.223505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2048,"num_write":7,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"265'34","reported_seq":149,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.559731+0000","last_change":"2024-05-30T16:12:50.816644+0000","last_active":"2024-05-30T17:11:21.559731+0000","last_peered":"2024-05-30T17:11:21.559731+0000","last_clean":"2024-05-30T17:11:21.559731+0000","last_became_active":"2024-05-30T16:12:50.815895+0000","last_became_peered":"2024-05-30T16:12:50.815895+0000","last_unstale":"2024-05-30T17:11:21.559731+0000","last_undegraded":"2024-05-30T17:11:21.559731+0000","last_fullsized":"2024-05-30T17:11:21.559731+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:33:57.609516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":160,"num_read_kb":111,"num_write":71,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":31,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:01.787658+0000","last_change":"2024-05-30T16:34:55.883831+0000","last_active":"2024-05-30T17:11:01.787658+0000","last_peered":"2024-05-30T17:11:01.787658+0000","last_clean":"2024-05-30T17:11:01.787658+0000","last_became_active":"2024-05-30T16:34:55.883112+0000","last_became_peered":"2024-05-30T16:34:55.883112+0000","last_unstale":"2024-05-30T17:11:01.787658+0000","last_undegraded":"2024-05-30T17:11:01.787658+0000","last_fullsized":"2024-05-30T17:11:01.787658+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:50:45.079747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"265'2","reported_seq":51,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.543941+0000","last_change":"2024-05-30T16:17:35.744284+0000","last_active":"2024-05-30T17:09:36.543941+0000","last_peered":"2024-05-30T17:09:36.543941+0000","last_clean":"2024-05-30T17:09:36.543941+0000","last_became_active":"2024-05-30T16:17:35.743608+0000","last_became_peered":"2024-05-30T16:17:35.743608+0000","last_unstale":"2024-05-30T17:09:36.543941+0000","last_undegraded":"2024-05-30T17:09:36.543941+0000","last_fullsized":"2024-05-30T17:09:36.543941+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:44:29.493440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":6144,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"0'0","reported_seq":53,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:32.128370+0000","last_change":"2024-05-30T16:17:31.189800+0000","last_active":"2024-05-30T17:09:32.128370+0000","last_peered":"2024-05-30T17:09:32.128370+0000","last_clean":"2024-05-30T17:09:32.128370+0000","last_became_active":"2024-05-30T16:17:31.188810+0000","last_became_peered":"2024-05-30T16:17:31.188810+0000","last_unstale":"2024-05-30T17:09:32.128370+0000","last_undegraded":"2024-05-30T17:09:32.128370+0000","last_fullsized":"2024-05-30T17:09:32.128370+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:27:55.332594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"265'18","reported_seq":111,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.570334+0000","last_change":"2024-05-30T16:12:50.827178+0000","last_active":"2024-05-30T17:11:26.570334+0000","last_peered":"2024-05-30T17:11:26.570334+0000","last_clean":"2024-05-30T17:11:26.570334+0000","last_became_active":"2024-05-30T16:12:50.826880+0000","last_became_peered":"2024-05-30T16:12:50.826880+0000","last_unstale":"2024-05-30T17:11:26.570334+0000","last_undegraded":"2024-05-30T17:11:26.570334+0000","last_fullsized":"2024-05-30T17:11:26.570334+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:22:01.392934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":61,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.735902+0000","last_change":"2024-05-30T16:34:55.886177+0000","last_active":"2024-05-30T17:10:21.735902+0000","last_peered":"2024-05-30T17:10:21.735902+0000","last_clean":"2024-05-30T17:10:21.735902+0000","last_became_active":"2024-05-30T16:34:55.885643+0000","last_became_peered":"2024-05-30T16:34:55.885643+0000","last_unstale":"2024-05-30T17:10:21.735902+0000","last_undegraded":"2024-05-30T17:10:21.735902+0000","last_fullsized":"2024-05-30T17:10:21.735902+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:27:38.430910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"262'2","reported_seq":51,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.726341+0000","last_change":"2024-05-30T16:17:35.248399+0000","last_active":"2024-05-30T17:09:36.726341+0000","last_peered":"2024-05-30T17:09:36.726341+0000","last_clean":"2024-05-30T17:09:36.726341+0000","last_became_active":"2024-05-30T16:17:35.247757+0000","last_became_peered":"2024-05-30T16:17:35.247757+0000","last_unstale":"2024-05-30T17:09:36.726341+0000","last_undegraded":"2024-05-30T17:09:36.726341+0000","last_fullsized":"2024-05-30T17:09:36.726341+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:22:36.927668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"0'0","reported_seq":53,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:31.542620+0000","last_change":"2024-05-30T16:17:31.203988+0000","last_active":"2024-05-30T17:09:31.542620+0000","last_peered":"2024-05-30T17:09:31.542620+0000","last_clean":"2024-05-30T17:09:31.542620+0000","last_became_active":"2024-05-30T16:17:31.203452+0000","last_became_peered":"2024-05-30T16:17:31.203452+0000","last_unstale":"2024-05-30T17:09:31.542620+0000","last_undegraded":"2024-05-30T17:09:31.542620+0000","last_fullsized":"2024-05-30T17:09:31.542620+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:50:29.995893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"265'37","reported_seq":162,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.789790+0000","last_change":"2024-05-30T16:12:50.812560+0000","last_active":"2024-05-30T17:11:11.789790+0000","last_peered":"2024-05-30T17:11:11.789790+0000","last_clean":"2024-05-30T17:11:11.789790+0000","last_became_active":"2024-05-30T16:12:50.811626+0000","last_became_peered":"2024-05-30T16:12:50.811626+0000","last_unstale":"2024-05-30T17:11:11.789790+0000","last_undegraded":"2024-05-30T17:11:11.789790+0000","last_fullsized":"2024-05-30T17:11:11.789790+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:11:40.631744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":184,"num_read_kb":132,"num_write":81,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"264'1","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.539587+0000","last_change":"2024-05-30T16:34:55.862486+0000","last_active":"2024-05-30T17:09:36.539587+0000","last_peered":"2024-05-30T17:09:36.539587+0000","last_clean":"2024-05-30T17:09:36.539587+0000","last_became_active":"2024-05-30T16:34:55.861767+0000","last_became_peered":"2024-05-30T16:34:55.861767+0000","last_unstale":"2024-05-30T17:09:36.539587+0000","last_undegraded":"2024-05-30T17:09:36.539587+0000","last_fullsized":"2024-05-30T17:09:36.539587+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:06:04.837971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"265'7","reported_seq":62,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:41.769918+0000","last_change":"2024-05-30T16:17:35.244195+0000","last_active":"2024-05-30T17:09:41.769918+0000","last_peered":"2024-05-30T17:09:41.769918+0000","last_clean":"2024-05-30T17:09:41.769918+0000","last_became_active":"2024-05-30T16:17:35.243435+0000","last_became_peered":"2024-05-30T16:17:35.243435+0000","last_unstale":"2024-05-30T17:09:41.769918+0000","last_undegraded":"2024-05-30T17:09:41.769918+0000","last_fullsized":"2024-05-30T17:09:41.769918+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:05:13.001812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":13312,"num_write":12,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"265'2","reported_seq":57,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:46.545952+0000","last_change":"2024-05-30T16:17:31.201111+0000","last_active":"2024-05-30T17:09:46.545952+0000","last_peered":"2024-05-30T17:09:46.545952+0000","last_clean":"2024-05-30T17:09:46.545952+0000","last_became_active":"2024-05-30T16:17:31.200663+0000","last_became_peered":"2024-05-30T16:17:31.200663+0000","last_unstale":"2024-05-30T17:09:46.545952+0000","last_undegraded":"2024-05-30T17:09:46.545952+0000","last_fullsized":"2024-05-30T17:09:46.545952+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:38:48.835894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"265'15","reported_seq":103,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:12.150755+0000","last_change":"2024-05-30T16:12:50.814553+0000","last_active":"2024-05-30T17:11:12.150755+0000","last_peered":"2024-05-30T17:11:12.150755+0000","last_clean":"2024-05-30T17:11:12.150755+0000","last_became_active":"2024-05-30T16:12:50.813741+0000","last_became_peered":"2024-05-30T16:12:50.813741+0000","last_unstale":"2024-05-30T17:11:12.150755+0000","last_undegraded":"2024-05-30T17:11:12.150755+0000","last_fullsized":"2024-05-30T17:11:12.150755+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:49:05.503730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":53,"num_write":33,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.560876+0000","last_change":"2024-05-30T16:34:55.859738+0000","last_active":"2024-05-30T17:09:36.560876+0000","last_peered":"2024-05-30T17:09:36.560876+0000","last_clean":"2024-05-30T17:09:36.560876+0000","last_became_active":"2024-05-30T16:34:55.858872+0000","last_became_peered":"2024-05-30T16:34:55.858872+0000","last_unstale":"2024-05-30T17:09:36.560876+0000","last_undegraded":"2024-05-30T17:09:36.560876+0000","last_fullsized":"2024-05-30T17:09:36.560876+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:48:21.475604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"0'0","reported_seq":49,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:42.130510+0000","last_change":"2024-05-30T16:17:35.227683+0000","last_active":"2024-05-30T17:09:42.130510+0000","last_peered":"2024-05-30T17:09:42.130510+0000","last_clean":"2024-05-30T17:09:42.130510+0000","last_became_active":"2024-05-30T16:17:35.226829+0000","last_became_peered":"2024-05-30T16:17:35.226829+0000","last_unstale":"2024-05-30T17:09:42.130510+0000","last_undegraded":"2024-05-30T17:09:42.130510+0000","last_fullsized":"2024-05-30T17:09:42.130510+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:15:44.751214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"0'0","reported_seq":54,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.543735+0000","last_change":"2024-05-30T16:17:31.205159+0000","last_active":"2024-05-30T17:09:36.543735+0000","last_peered":"2024-05-30T17:09:36.543735+0000","last_clean":"2024-05-30T17:09:36.543735+0000","last_became_active":"2024-05-30T16:17:31.203535+0000","last_became_peered":"2024-05-30T16:17:31.203535+0000","last_unstale":"2024-05-30T17:09:36.543735+0000","last_undegraded":"2024-05-30T17:09:36.543735+0000","last_fullsized":"2024-05-30T17:09:36.543735+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:42:29.611842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"265'36","reported_seq":157,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.747034+0000","last_change":"2024-05-30T16:12:50.832592+0000","last_active":"2024-05-30T17:11:11.747034+0000","last_peered":"2024-05-30T17:11:11.747034+0000","last_clean":"2024-05-30T17:11:11.747034+0000","last_became_active":"2024-05-30T16:12:50.831842+0000","last_became_peered":"2024-05-30T16:12:50.831842+0000","last_unstale":"2024-05-30T17:11:11.747034+0000","last_undegraded":"2024-05-30T17:11:11.747034+0000","last_fullsized":"2024-05-30T17:11:11.747034+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:08:18.471042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":177,"num_read_kb":125,"num_write":75,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.570209+0000","last_change":"2024-05-30T16:34:55.863472+0000","last_active":"2024-05-30T17:10:21.570209+0000","last_peered":"2024-05-30T17:10:21.570209+0000","last_clean":"2024-05-30T17:10:21.570209+0000","last_became_active":"2024-05-30T16:34:55.862709+0000","last_became_peered":"2024-05-30T16:34:55.862709+0000","last_unstale":"2024-05-30T17:10:21.570209+0000","last_undegraded":"2024-05-30T17:10:21.570209+0000","last_fullsized":"2024-05-30T17:10:21.570209+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:25:02.100900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"265'2","reported_seq":50,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:41.537534+0000","last_change":"2024-05-30T16:17:35.241271+0000","last_active":"2024-05-30T17:09:41.537534+0000","last_peered":"2024-05-30T17:09:41.537534+0000","last_clean":"2024-05-30T17:09:41.537534+0000","last_became_active":"2024-05-30T16:17:35.240472+0000","last_became_peered":"2024-05-30T16:17:35.240472+0000","last_unstale":"2024-05-30T17:09:41.537534+0000","last_undegraded":"2024-05-30T17:09:41.537534+0000","last_fullsized":"2024-05-30T17:09:41.537534+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:59:40.865645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"265'4","reported_seq":62,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.769181+0000","last_change":"2024-05-30T16:17:31.196715+0000","last_active":"2024-05-30T17:09:36.769181+0000","last_peered":"2024-05-30T17:09:36.769181+0000","last_clean":"2024-05-30T17:09:36.769181+0000","last_became_active":"2024-05-30T16:17:31.187958+0000","last_became_peered":"2024-05-30T16:17:31.187958+0000","last_unstale":"2024-05-30T17:09:36.769181+0000","last_undegraded":"2024-05-30T17:09:36.769181+0000","last_fullsized":"2024-05-30T17:09:36.769181+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:26:54.324514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485760,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":10240,"num_write":14,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"265'18","reported_seq":112,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.557030+0000","last_change":"2024-05-30T16:12:50.823849+0000","last_active":"2024-05-30T17:11:11.557030+0000","last_peered":"2024-05-30T17:11:11.557030+0000","last_clean":"2024-05-30T17:11:11.557030+0000","last_became_active":"2024-05-30T16:12:50.823090+0000","last_became_peered":"2024-05-30T16:12:50.823090+0000","last_unstale":"2024-05-30T17:11:11.557030+0000","last_undegraded":"2024-05-30T17:11:11.557030+0000","last_fullsized":"2024-05-30T17:11:11.557030+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:38:28.480174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":69,"num_write":35,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:31.556926+0000","last_change":"2024-05-30T16:34:55.891440+0000","last_active":"2024-05-30T17:10:31.556926+0000","last_peered":"2024-05-30T17:10:31.556926+0000","last_clean":"2024-05-30T17:10:31.556926+0000","last_became_active":"2024-05-30T16:34:55.890688+0000","last_became_peered":"2024-05-30T16:34:55.890688+0000","last_unstale":"2024-05-30T17:10:31.556926+0000","last_undegraded":"2024-05-30T17:10:31.556926+0000","last_fullsized":"2024-05-30T17:10:31.556926+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:05:55.238651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"265'7","reported_seq":65,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:41.727328+0000","last_change":"2024-05-30T16:17:35.238582+0000","last_active":"2024-05-30T17:09:41.727328+0000","last_peered":"2024-05-30T17:09:41.727328+0000","last_clean":"2024-05-30T17:09:41.727328+0000","last_became_active":"2024-05-30T16:17:35.237031+0000","last_became_peered":"2024-05-30T16:17:35.237031+0000","last_unstale":"2024-05-30T17:09:41.727328+0000","last_undegraded":"2024-05-30T17:09:41.727328+0000","last_fullsized":"2024-05-30T17:09:41.727328+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:11:17.043035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485760,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":16,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"262'5","reported_seq":65,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.536379+0000","last_change":"2024-05-30T16:17:31.204721+0000","last_active":"2024-05-30T17:09:36.536379+0000","last_peered":"2024-05-30T17:09:36.536379+0000","last_clean":"2024-05-30T17:09:36.536379+0000","last_became_active":"2024-05-30T16:17:31.203871+0000","last_became_peered":"2024-05-30T16:17:31.203871+0000","last_unstale":"2024-05-30T17:09:36.536379+0000","last_undegraded":"2024-05-30T17:09:36.536379+0000","last_fullsized":"2024-05-30T17:09:36.536379+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:41:32.481769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":4098,"num_write":9,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"265'24","reported_seq":133,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:06.565051+0000","last_change":"2024-05-30T16:12:50.827488+0000","last_active":"2024-05-30T17:11:06.565051+0000","last_peered":"2024-05-30T17:11:06.565051+0000","last_clean":"2024-05-30T17:11:06.565051+0000","last_became_active":"2024-05-30T16:12:50.827078+0000","last_became_peered":"2024-05-30T16:12:50.827078+0000","last_unstale":"2024-05-30T17:11:06.565051+0000","last_undegraded":"2024-05-30T17:11:06.565051+0000","last_fullsized":"2024-05-30T17:11:06.565051+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:30:01.069671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":93,"num_write":46,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"265'165","reported_seq":776,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.569071+0000","last_change":"2024-05-30T15:54:16.662201+0000","last_active":"2024-05-30T17:11:21.569071+0000","last_peered":"2024-05-30T17:11:21.569071+0000","last_clean":"2024-05-30T17:11:21.569071+0000","last_became_active":"2024-05-30T15:54:16.661334+0000","last_became_peered":"2024-05-30T15:54:16.661334+0000","last_unstale":"2024-05-30T17:11:21.569071+0000","last_undegraded":"2024-05-30T17:11:21.569071+0000","last_fullsized":"2024-05-30T17:11:21.569071+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":100,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:14:32.854488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2159792,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":347,"num_read_kb":7638,"num_write":1029,"num_write_kb":24692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"265'436","reported_seq":2069,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:33.884897+0000","last_change":"2024-05-30T15:54:18.669748+0000","last_active":"2024-05-30T17:12:33.884897+0000","last_peered":"2024-05-30T17:12:33.884897+0000","last_clean":"2024-05-30T17:12:33.884897+0000","last_became_active":"2024-05-30T15:54:18.669016+0000","last_became_peered":"2024-05-30T15:54:18.669016+0000","last_unstale":"2024-05-30T17:12:33.884897+0000","last_undegraded":"2024-05-30T17:12:33.884897+0000","last_fullsized":"2024-05-30T17:12:33.884897+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":400,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:26:42.111749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2479,"num_read_kb":2366,"num_write":506,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"265'232","reported_seq":887,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.792186+0000","last_change":"2024-05-30T15:54:16.653102+0000","last_active":"2024-05-30T17:11:21.792186+0000","last_peered":"2024-05-30T17:11:21.792186+0000","last_clean":"2024-05-30T17:11:21.792186+0000","last_became_active":"2024-05-30T15:54:16.652724+0000","last_became_peered":"2024-05-30T15:54:16.652724+0000","last_unstale":"2024-05-30T17:11:21.792186+0000","last_undegraded":"2024-05-30T17:11:21.792186+0000","last_fullsized":"2024-05-30T17:11:21.792186+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":200,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:48:02.488417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16999710,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":552,"num_read_kb":8631,"num_write":1279,"num_write_kb":24636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"265'758","reported_seq":2113,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:59.057740+0000","last_change":"2024-05-30T15:54:18.649460+0000","last_active":"2024-05-30T17:11:59.057740+0000","last_peered":"2024-05-30T17:11:59.057740+0000","last_clean":"2024-05-30T17:11:59.057740+0000","last_became_active":"2024-05-30T15:54:18.648675+0000","last_became_peered":"2024-05-30T15:54:18.648675+0000","last_unstale":"2024-05-30T17:11:59.057740+0000","last_undegraded":"2024-05-30T17:11:59.057740+0000","last_fullsized":"2024-05-30T17:11:59.057740+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":700,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:30:51.616348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3292,"num_read_kb":2905,"num_write":1031,"num_write_kb":415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"265'200","reported_seq":832,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:57.487564+0000","last_change":"2024-05-30T15:54:16.633538+0000","last_active":"2024-05-30T17:11:57.487564+0000","last_peered":"2024-05-30T17:11:57.487564+0000","last_clean":"2024-05-30T17:11:57.487564+0000","last_became_active":"2024-05-30T15:54:16.633011+0000","last_became_peered":"2024-05-30T15:54:16.633011+0000","last_unstale":"2024-05-30T17:11:57.487564+0000","last_undegraded":"2024-05-30T17:11:57.487564+0000","last_fullsized":"2024-05-30T17:11:57.487564+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":100,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:22:33.463154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17125665,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":396,"num_read_kb":43854,"num_write":995,"num_write_kb":74051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"265'746","reported_seq":2144,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:33.888418+0000","last_change":"2024-05-30T15:54:18.651676+0000","last_active":"2024-05-30T17:12:33.888418+0000","last_peered":"2024-05-30T17:12:33.888418+0000","last_clean":"2024-05-30T17:12:33.888418+0000","last_became_active":"2024-05-30T15:54:18.649608+0000","last_became_peered":"2024-05-30T15:54:18.649608+0000","last_unstale":"2024-05-30T17:12:33.888418+0000","last_undegraded":"2024-05-30T17:12:33.888418+0000","last_fullsized":"2024-05-30T17:12:33.888418+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":700,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:29:56.699785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3275,"num_read_kb":2909,"num_write":996,"num_write_kb":378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"265'152","reported_seq":790,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:44.320980+0000","last_change":"2024-05-30T15:54:16.653236+0000","last_active":"2024-05-30T17:11:44.320980+0000","last_peered":"2024-05-30T17:11:44.320980+0000","last_clean":"2024-05-30T17:11:44.320980+0000","last_became_active":"2024-05-30T15:54:16.652315+0000","last_became_peered":"2024-05-30T15:54:16.652315+0000","last_unstale":"2024-05-30T17:11:44.320980+0000","last_undegraded":"2024-05-30T17:11:44.320980+0000","last_fullsized":"2024-05-30T17:11:44.320980+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":100,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:26:57.491124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1336636,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":435,"num_read_kb":3484,"num_write":909,"num_write_kb":6490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"265'487","reported_seq":1962,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:52.698536+0000","last_change":"2024-05-30T15:54:18.652741+0000","last_active":"2024-05-30T17:12:52.698536+0000","last_peered":"2024-05-30T17:12:52.698536+0000","last_clean":"2024-05-30T17:12:52.698536+0000","last_became_active":"2024-05-30T15:54:18.652451+0000","last_became_peered":"2024-05-30T15:54:18.652451+0000","last_unstale":"2024-05-30T17:12:52.698536+0000","last_undegraded":"2024-05-30T17:12:52.698536+0000","last_fullsized":"2024-05-30T17:12:52.698536+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":400,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:50:10.811052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2387,"num_read_kb":2188,"num_write":616,"num_write_kb":168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"265'242","reported_seq":956,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:58.054740+0000","last_change":"2024-05-30T15:54:16.659161+0000","last_active":"2024-05-30T17:12:58.054740+0000","last_peered":"2024-05-30T17:12:58.054740+0000","last_clean":"2024-05-30T17:12:58.054740+0000","last_became_active":"2024-05-30T15:54:16.658338+0000","last_became_peered":"2024-05-30T15:54:16.658338+0000","last_unstale":"2024-05-30T17:12:58.054740+0000","last_undegraded":"2024-05-30T17:12:58.054740+0000","last_fullsized":"2024-05-30T17:12:58.054740+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":200,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:10:31.510086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11588767,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":827,"num_read_kb":24681,"num_write":1243,"num_write_kb":34031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"265'370","reported_seq":1862,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:12.349707+0000","last_change":"2024-05-30T15:54:18.669822+0000","last_active":"2024-05-30T17:11:12.349707+0000","last_peered":"2024-05-30T17:11:12.349707+0000","last_clean":"2024-05-30T17:11:12.349707+0000","last_became_active":"2024-05-30T15:54:18.669017+0000","last_became_peered":"2024-05-30T15:54:18.669017+0000","last_unstale":"2024-05-30T17:11:12.349707+0000","last_undegraded":"2024-05-30T17:11:12.349707+0000","last_fullsized":"2024-05-30T17:11:12.349707+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":300,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:33:40.482356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1986,"num_read_kb":1906,"num_write":419,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"265'192","reported_seq":817,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:17.124503+0000","last_change":"2024-05-30T15:54:16.673757+0000","last_active":"2024-05-30T17:09:17.124503+0000","last_peered":"2024-05-30T17:09:17.124503+0000","last_clean":"2024-05-30T17:09:17.124503+0000","last_became_active":"2024-05-30T15:54:16.671120+0000","last_became_peered":"2024-05-30T15:54:16.671120+0000","last_unstale":"2024-05-30T17:09:17.124503+0000","last_undegraded":"2024-05-30T17:09:17.124503+0000","last_fullsized":"2024-05-30T17:09:17.124503+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":100,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:07:40.834702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5596650,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":434,"num_read_kb":4718,"num_write":1097,"num_write_kb":20583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"265'725","reported_seq":2201,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.709270+0000","last_change":"2024-05-30T15:54:18.651587+0000","last_active":"2024-05-30T17:13:20.709270+0000","last_peered":"2024-05-30T17:13:20.709270+0000","last_clean":"2024-05-30T17:13:20.709270+0000","last_became_active":"2024-05-30T15:54:18.650543+0000","last_became_peered":"2024-05-30T15:54:18.650543+0000","last_unstale":"2024-05-30T17:13:20.709270+0000","last_undegraded":"2024-05-30T17:13:20.709270+0000","last_fullsized":"2024-05-30T17:13:20.709270+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":700,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:51.313043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3282,"num_read_kb":2952,"num_write":973,"num_write_kb":356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"265'188","reported_seq":828,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:18.810687+0000","last_change":"2024-05-30T15:54:16.679393+0000","last_active":"2024-05-30T17:12:18.810687+0000","last_peered":"2024-05-30T17:12:18.810687+0000","last_clean":"2024-05-30T17:12:18.810687+0000","last_became_active":"2024-05-30T15:54:16.678799+0000","last_became_peered":"2024-05-30T15:54:16.678799+0000","last_unstale":"2024-05-30T17:12:18.810687+0000","last_undegraded":"2024-05-30T17:12:18.810687+0000","last_fullsized":"2024-05-30T17:12:18.810687+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":100,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:07:54.614137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5740256,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":463,"num_read_kb":33008,"num_write":1086,"num_write_kb":27021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"265'850","reported_seq":2539,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.703253+0000","last_change":"2024-05-30T15:54:18.657813+0000","last_active":"2024-05-30T17:13:20.703253+0000","last_peered":"2024-05-30T17:13:20.703253+0000","last_clean":"2024-05-30T17:13:20.703253+0000","last_became_active":"2024-05-30T15:54:18.656475+0000","last_became_peered":"2024-05-30T15:54:18.656475+0000","last_unstale":"2024-05-30T17:13:20.703253+0000","last_undegraded":"2024-05-30T17:13:20.703253+0000","last_fullsized":"2024-05-30T17:13:20.703253+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":800,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:08:46.768265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4088,"num_read_kb":3692,"num_write":1166,"num_write_kb":472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"265'139","reported_seq":735,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:16.558759+0000","last_change":"2024-05-30T15:54:16.669564+0000","last_active":"2024-05-30T17:09:16.558759+0000","last_peered":"2024-05-30T17:09:16.558759+0000","last_clean":"2024-05-30T17:09:16.558759+0000","last_became_active":"2024-05-30T15:54:16.669095+0000","last_became_peered":"2024-05-30T15:54:16.669095+0000","last_unstale":"2024-05-30T17:09:16.558759+0000","last_undegraded":"2024-05-30T17:09:16.558759+0000","last_fullsized":"2024-05-30T17:09:16.558759+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":100,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:34:08.005236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485923,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":302,"num_read_kb":17751,"num_write":816,"num_write_kb":30378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"265'369","reported_seq":1790,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.560600+0000","last_change":"2024-05-30T15:54:18.653529+0000","last_active":"2024-05-30T17:11:26.560600+0000","last_peered":"2024-05-30T17:11:26.560600+0000","last_clean":"2024-05-30T17:11:26.560600+0000","last_became_active":"2024-05-30T15:54:18.653189+0000","last_became_peered":"2024-05-30T15:54:18.653189+0000","last_unstale":"2024-05-30T17:11:26.560600+0000","last_undegraded":"2024-05-30T17:11:26.560600+0000","last_fullsized":"2024-05-30T17:11:26.560600+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":300,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:18:04.600068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1969,"num_read_kb":1855,"num_write":435,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"265'336","reported_seq":1764,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:33.888003+0000","last_change":"2024-05-30T15:54:18.652246+0000","last_active":"2024-05-30T17:12:33.888003+0000","last_peered":"2024-05-30T17:12:33.888003+0000","last_clean":"2024-05-30T17:12:33.888003+0000","last_became_active":"2024-05-30T15:54:18.649994+0000","last_became_peered":"2024-05-30T15:54:18.649994+0000","last_unstale":"2024-05-30T17:12:33.888003+0000","last_undegraded":"2024-05-30T17:12:33.888003+0000","last_fullsized":"2024-05-30T17:12:33.888003+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:39:30.273131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1976,"num_read_kb":1893,"num_write":388,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"265'206","reported_seq":871,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:11.306928+0000","last_change":"2024-05-30T15:54:16.650771+0000","last_active":"2024-05-30T17:12:11.306928+0000","last_peered":"2024-05-30T17:12:11.306928+0000","last_clean":"2024-05-30T17:12:11.306928+0000","last_became_active":"2024-05-30T15:54:16.649872+0000","last_became_peered":"2024-05-30T15:54:16.649872+0000","last_unstale":"2024-05-30T17:12:11.306928+0000","last_undegraded":"2024-05-30T17:12:11.306928+0000","last_fullsized":"2024-05-30T17:12:11.306928+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":100,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:01:29.595780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16404427,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":504,"num_read_kb":59018,"num_write":1160,"num_write_kb":23092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"265'1525","reported_seq":3130,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:28.258461+0000","last_change":"2024-05-30T15:54:18.649618+0000","last_active":"2024-05-30T17:12:28.258461+0000","last_peered":"2024-05-30T17:12:28.258461+0000","last_clean":"2024-05-30T17:12:28.258461+0000","last_became_active":"2024-05-30T15:54:18.648860+0000","last_became_peered":"2024-05-30T15:54:18.648860+0000","last_unstale":"2024-05-30T17:12:28.258461+0000","last_undegraded":"2024-05-30T17:12:28.258461+0000","last_fullsized":"2024-05-30T17:12:28.258461+0000","mapping_epoch":24,"log_start":"265'1500","ondisk_log_start":"265'1500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":1500,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:09:51.963499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6132,"num_read_kb":5297,"num_write":2138,"num_write_kb":912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"265'218","reported_seq":1080,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:28.769161+0000","last_change":"2024-05-30T15:54:16.687260+0000","last_active":"2024-05-30T17:12:28.769161+0000","last_peered":"2024-05-30T17:12:28.769161+0000","last_clean":"2024-05-30T17:12:28.769161+0000","last_became_active":"2024-05-30T15:54:16.686847+0000","last_became_peered":"2024-05-30T15:54:16.686847+0000","last_unstale":"2024-05-30T17:12:28.769161+0000","last_undegraded":"2024-05-30T17:12:28.769161+0000","last_fullsized":"2024-05-30T17:12:28.769161+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":200,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:27:56.085645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2781302,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":922,"num_read_kb":24537,"num_write":1230,"num_write_kb":54861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"265'1164","reported_seq":3003,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:10.547968+0000","last_change":"2024-05-30T15:54:18.666133+0000","last_active":"2024-05-30T17:13:10.547968+0000","last_peered":"2024-05-30T17:13:10.547968+0000","last_clean":"2024-05-30T17:13:10.547968+0000","last_became_active":"2024-05-30T15:54:18.665302+0000","last_became_peered":"2024-05-30T15:54:18.665302+0000","last_unstale":"2024-05-30T17:13:10.547968+0000","last_undegraded":"2024-05-30T17:13:10.547968+0000","last_fullsized":"2024-05-30T17:13:10.547968+0000","mapping_epoch":24,"log_start":"265'1100","ondisk_log_start":"265'1100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":1100,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:19:10.742000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5292,"num_read_kb":4714,"num_write":1593,"num_write_kb":642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"265'194","reported_seq":915,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:16.092608+0000","last_change":"2024-05-30T15:54:16.669114+0000","last_active":"2024-05-30T17:09:16.092608+0000","last_peered":"2024-05-30T17:09:16.092608+0000","last_clean":"2024-05-30T17:09:16.092608+0000","last_became_active":"2024-05-30T15:54:16.668438+0000","last_became_peered":"2024-05-30T15:54:16.668438+0000","last_unstale":"2024-05-30T17:09:16.092608+0000","last_undegraded":"2024-05-30T17:09:16.092608+0000","last_fullsized":"2024-05-30T17:09:16.092608+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":100,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:55:42.428949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10487886,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":915,"num_read_kb":8276,"num_write":1170,"num_write_kb":35310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"265'768","reported_seq":2282,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.649495+0000","last_change":"2024-05-30T15:54:18.651494+0000","last_active":"2024-05-30T17:13:20.649495+0000","last_peered":"2024-05-30T17:13:20.649495+0000","last_clean":"2024-05-30T17:13:20.649495+0000","last_became_active":"2024-05-30T15:54:18.651197+0000","last_became_peered":"2024-05-30T15:54:18.651197+0000","last_unstale":"2024-05-30T17:13:20.649495+0000","last_undegraded":"2024-05-30T17:13:20.649495+0000","last_fullsized":"2024-05-30T17:13:20.649495+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":700,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:01:31.985221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3544,"num_read_kb":3171,"num_write":1051,"num_write_kb":404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"265'181","reported_seq":833,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:11.720441+0000","last_change":"2024-05-30T15:54:16.677382+0000","last_active":"2024-05-30T17:09:11.720441+0000","last_peered":"2024-05-30T17:09:11.720441+0000","last_clean":"2024-05-30T17:09:11.720441+0000","last_became_active":"2024-05-30T15:54:16.676703+0000","last_became_peered":"2024-05-30T15:54:16.676703+0000","last_unstale":"2024-05-30T17:09:11.720441+0000","last_undegraded":"2024-05-30T17:09:11.720441+0000","last_fullsized":"2024-05-30T17:09:11.720441+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":100,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:39:55.585070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16123359,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":357,"num_read_kb":73313,"num_write":1053,"num_write_kb":47694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"265'372","reported_seq":1824,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.655751+0000","last_change":"2024-05-30T15:54:18.650530+0000","last_active":"2024-05-30T17:12:44.655751+0000","last_peered":"2024-05-30T17:12:44.655751+0000","last_clean":"2024-05-30T17:12:44.655751+0000","last_became_active":"2024-05-30T15:54:18.650255+0000","last_became_peered":"2024-05-30T15:54:18.650255+0000","last_unstale":"2024-05-30T17:12:44.655751+0000","last_undegraded":"2024-05-30T17:12:44.655751+0000","last_fullsized":"2024-05-30T17:12:44.655751+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":300,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:22:55.711498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1969,"num_read_kb":1871,"num_write":435,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"265'200","reported_seq":851,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:11.557730+0000","last_change":"2024-05-30T15:54:16.665743+0000","last_active":"2024-05-30T17:09:11.557730+0000","last_peered":"2024-05-30T17:09:11.557730+0000","last_clean":"2024-05-30T17:09:11.557730+0000","last_became_active":"2024-05-30T15:54:16.665344+0000","last_became_peered":"2024-05-30T15:54:16.665344+0000","last_unstale":"2024-05-30T17:09:11.557730+0000","last_undegraded":"2024-05-30T17:09:11.557730+0000","last_fullsized":"2024-05-30T17:09:11.557730+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":100,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:11:01.132558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17335132,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":474,"num_read_kb":7750,"num_write":1074,"num_write_kb":36544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"265'375","reported_seq":1800,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:15.191377+0000","last_change":"2024-05-30T15:54:18.645251+0000","last_active":"2024-05-30T17:12:15.191377+0000","last_peered":"2024-05-30T17:12:15.191377+0000","last_clean":"2024-05-30T17:12:15.191377+0000","last_became_active":"2024-05-30T15:54:18.643825+0000","last_became_peered":"2024-05-30T15:54:18.643825+0000","last_unstale":"2024-05-30T17:12:15.191377+0000","last_undegraded":"2024-05-30T17:12:15.191377+0000","last_fullsized":"2024-05-30T17:12:15.191377+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":300,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:54:45.495400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1936,"num_read_kb":1833,"num_write":434,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"265'234","reported_seq":905,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:16.555742+0000","last_change":"2024-05-30T15:54:16.638560+0000","last_active":"2024-05-30T17:09:16.555742+0000","last_peered":"2024-05-30T17:09:16.555742+0000","last_clean":"2024-05-30T17:09:16.555742+0000","last_became_active":"2024-05-30T15:54:16.637928+0000","last_became_peered":"2024-05-30T15:54:16.637928+0000","last_unstale":"2024-05-30T17:09:16.555742+0000","last_undegraded":"2024-05-30T17:09:16.555742+0000","last_fullsized":"2024-05-30T17:09:16.555742+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":200,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:28:17.148350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11620077,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":522,"num_read_kb":65823,"num_write":1177,"num_write_kb":32044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"265'176","reported_seq":700,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.999380+0000","last_change":"2024-05-30T15:54:26.701638+0000","last_active":"2024-05-30T17:13:20.999380+0000","last_peered":"2024-05-30T17:13:20.999380+0000","last_clean":"2024-05-30T17:13:20.999380+0000","last_became_active":"2024-05-30T15:54:26.700957+0000","last_became_peered":"2024-05-30T15:54:26.700957+0000","last_unstale":"2024-05-30T17:13:20.999380+0000","last_undegraded":"2024-05-30T17:13:20.999380+0000","last_fullsized":"2024-05-30T17:13:20.999380+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":100,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:45:03.540847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":270,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":132,"num_read_kb":126,"num_write":386,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"265'494","reported_seq":2021,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.600563+0000","last_change":"2024-05-30T15:54:18.657002+0000","last_active":"2024-05-30T17:12:44.600563+0000","last_peered":"2024-05-30T17:12:44.600563+0000","last_clean":"2024-05-30T17:12:44.600563+0000","last_became_active":"2024-05-30T15:54:18.652712+0000","last_became_peered":"2024-05-30T15:54:18.652712+0000","last_unstale":"2024-05-30T17:12:44.600563+0000","last_undegraded":"2024-05-30T17:12:44.600563+0000","last_fullsized":"2024-05-30T17:12:44.600563+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":400,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:27:30.279867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2579,"num_read_kb":2377,"num_write":668,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"265'189","reported_seq":793,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:36.550925+0000","last_change":"2024-05-30T15:55:54.564987+0000","last_active":"2024-05-30T17:10:36.550925+0000","last_peered":"2024-05-30T17:10:36.550925+0000","last_clean":"2024-05-30T17:10:36.550925+0000","last_became_active":"2024-05-30T15:55:54.563874+0000","last_became_peered":"2024-05-30T15:55:54.563874+0000","last_unstale":"2024-05-30T17:10:36.550925+0000","last_undegraded":"2024-05-30T17:10:36.550925+0000","last_fullsized":"2024-05-30T17:10:36.550925+0000","mapping_epoch":37,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:22:56.400029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079909,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":469,"num_read_kb":1589,"num_write":1088,"num_write_kb":27979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:17.124719+0000","last_change":"2024-05-30T15:54:20.659788+0000","last_active":"2024-05-30T17:09:17.124719+0000","last_peered":"2024-05-30T17:09:17.124719+0000","last_clean":"2024-05-30T17:09:17.124719+0000","last_became_active":"2024-05-30T15:54:20.658937+0000","last_became_peered":"2024-05-30T15:54:20.658937+0000","last_unstale":"2024-05-30T17:09:17.124719+0000","last_undegraded":"2024-05-30T17:09:17.124719+0000","last_fullsized":"2024-05-30T17:09:17.124719+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:41:24.097539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"265'198","reported_seq":820,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:46.590604+0000","last_change":"2024-05-30T15:54:22.668221+0000","last_active":"2024-05-30T17:11:46.590604+0000","last_peered":"2024-05-30T17:11:46.590604+0000","last_clean":"2024-05-30T17:11:46.590604+0000","last_became_active":"2024-05-30T15:54:22.667496+0000","last_became_peered":"2024-05-30T15:54:22.667496+0000","last_unstale":"2024-05-30T17:11:46.590604+0000","last_undegraded":"2024-05-30T17:11:46.590604+0000","last_fullsized":"2024-05-30T17:11:46.590604+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":100,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:54:14.364376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9413,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":276,"num_read_kb":334,"num_write":164,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":498,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.754793+0000","last_change":"2024-05-30T15:54:24.675232+0000","last_active":"2024-05-30T17:09:21.754793+0000","last_peered":"2024-05-30T17:09:21.754793+0000","last_clean":"2024-05-30T17:09:21.754793+0000","last_became_active":"2024-05-30T15:54:24.674491+0000","last_became_peered":"2024-05-30T15:54:24.674491+0000","last_unstale":"2024-05-30T17:09:21.754793+0000","last_undegraded":"2024-05-30T17:09:21.754793+0000","last_fullsized":"2024-05-30T17:09:21.754793+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:30:36.269404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"265'194","reported_seq":769,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.643659+0000","last_change":"2024-05-30T15:54:26.687210+0000","last_active":"2024-05-30T17:12:44.643659+0000","last_peered":"2024-05-30T17:12:44.643659+0000","last_clean":"2024-05-30T17:12:44.643659+0000","last_became_active":"2024-05-30T15:54:26.686430+0000","last_became_peered":"2024-05-30T15:54:26.686430+0000","last_unstale":"2024-05-30T17:12:44.643659+0000","last_undegraded":"2024-05-30T17:12:44.643659+0000","last_fullsized":"2024-05-30T17:12:44.643659+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":100,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:45:58.373725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":824,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":238,"num_read_kb":223,"num_write":274,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"265'510","reported_seq":2160,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.681680+0000","last_change":"2024-05-30T15:54:18.662470+0000","last_active":"2024-05-30T17:13:18.681680+0000","last_peered":"2024-05-30T17:13:18.681680+0000","last_clean":"2024-05-30T17:13:18.681680+0000","last_became_active":"2024-05-30T15:54:18.661762+0000","last_became_peered":"2024-05-30T15:54:18.661762+0000","last_unstale":"2024-05-30T17:13:18.681680+0000","last_undegraded":"2024-05-30T17:13:18.681680+0000","last_fullsized":"2024-05-30T17:13:18.681680+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":400,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:39:51.492340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2657,"num_read_kb":2481,"num_write":627,"num_write_kb":155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"265'204","reported_seq":860,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.582351+0000","last_change":"2024-05-30T15:54:16.639898+0000","last_active":"2024-05-30T17:11:16.582351+0000","last_peered":"2024-05-30T17:11:16.582351+0000","last_clean":"2024-05-30T17:11:16.582351+0000","last_became_active":"2024-05-30T15:54:16.639284+0000","last_became_peered":"2024-05-30T15:54:16.639284+0000","last_unstale":"2024-05-30T17:11:16.582351+0000","last_undegraded":"2024-05-30T17:11:16.582351+0000","last_fullsized":"2024-05-30T17:11:16.582351+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":100,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:33:35.045719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13950104,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":439,"num_read_kb":15653,"num_write":1153,"num_write_kb":64581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":509,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.560530+0000","last_change":"2024-05-30T15:54:20.666014+0000","last_active":"2024-05-30T17:09:21.560530+0000","last_peered":"2024-05-30T17:09:21.560530+0000","last_clean":"2024-05-30T17:09:21.560530+0000","last_became_active":"2024-05-30T15:54:20.665488+0000","last_became_peered":"2024-05-30T15:54:20.665488+0000","last_unstale":"2024-05-30T17:09:21.560530+0000","last_undegraded":"2024-05-30T17:09:21.560530+0000","last_fullsized":"2024-05-30T17:09:21.560530+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:23:46.612023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"265'102","reported_seq":653,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.723144+0000","last_change":"2024-05-30T15:54:22.666790+0000","last_active":"2024-05-30T17:09:21.723144+0000","last_peered":"2024-05-30T17:09:21.723144+0000","last_clean":"2024-05-30T17:09:21.723144+0000","last_became_active":"2024-05-30T15:54:22.666057+0000","last_became_peered":"2024-05-30T15:54:22.666057+0000","last_unstale":"2024-05-30T17:09:21.723144+0000","last_undegraded":"2024-05-30T17:09:21.723144+0000","last_fullsized":"2024-05-30T17:09:21.723144+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:49:29.070169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":113,"num_write":76,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":507,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.723119+0000","last_change":"2024-05-30T15:54:24.687346+0000","last_active":"2024-05-30T17:09:21.723119+0000","last_peered":"2024-05-30T17:09:21.723119+0000","last_clean":"2024-05-30T17:09:21.723119+0000","last_became_active":"2024-05-30T15:54:24.686595+0000","last_became_peered":"2024-05-30T15:54:24.686595+0000","last_unstale":"2024-05-30T17:09:21.723119+0000","last_undegraded":"2024-05-30T17:09:21.723119+0000","last_fullsized":"2024-05-30T17:09:21.723119+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:49:42.604820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"265'99","reported_seq":612,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.102373+0000","last_change":"2024-05-30T15:54:26.689306+0000","last_active":"2024-05-30T17:13:18.102373+0000","last_peered":"2024-05-30T17:13:18.102373+0000","last_clean":"2024-05-30T17:13:18.102373+0000","last_became_active":"2024-05-30T15:54:26.688376+0000","last_became_peered":"2024-05-30T15:54:26.688376+0000","last_unstale":"2024-05-30T17:13:18.102373+0000","last_undegraded":"2024-05-30T17:13:18.102373+0000","last_fullsized":"2024-05-30T17:13:18.102373+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:10:30.878895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":271,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":103,"num_read_kb":102,"num_write":235,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"265'462","reported_seq":2011,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.638007+0000","last_change":"2024-05-30T15:54:18.658772+0000","last_active":"2024-05-30T17:13:20.638007+0000","last_peered":"2024-05-30T17:13:20.638007+0000","last_clean":"2024-05-30T17:13:20.638007+0000","last_became_active":"2024-05-30T15:54:18.657540+0000","last_became_peered":"2024-05-30T15:54:18.657540+0000","last_unstale":"2024-05-30T17:13:20.638007+0000","last_undegraded":"2024-05-30T17:13:20.638007+0000","last_fullsized":"2024-05-30T17:13:20.638007+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":400,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:48:10.156295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2554,"num_read_kb":2362,"num_write":592,"num_write_kb":196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"265'212","reported_seq":873,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:58.050061+0000","last_change":"2024-05-30T15:54:16.679183+0000","last_active":"2024-05-30T17:12:58.050061+0000","last_peered":"2024-05-30T17:12:58.050061+0000","last_clean":"2024-05-30T17:12:58.050061+0000","last_became_active":"2024-05-30T15:54:16.678659+0000","last_became_peered":"2024-05-30T15:54:16.678659+0000","last_unstale":"2024-05-30T17:12:58.050061+0000","last_undegraded":"2024-05-30T17:12:58.050061+0000","last_fullsized":"2024-05-30T17:12:58.050061+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:12:32.311283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11885555,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":520,"num_read_kb":29194,"num_write":1285,"num_write_kb":17953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":508,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:16.559045+0000","last_change":"2024-05-30T15:54:20.654060+0000","last_active":"2024-05-30T17:09:16.559045+0000","last_peered":"2024-05-30T17:09:16.559045+0000","last_clean":"2024-05-30T17:09:16.559045+0000","last_became_active":"2024-05-30T15:54:20.653316+0000","last_became_peered":"2024-05-30T15:54:20.653316+0000","last_unstale":"2024-05-30T17:09:16.559045+0000","last_undegraded":"2024-05-30T17:09:16.559045+0000","last_fullsized":"2024-05-30T17:09:16.559045+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:11:29.893912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"265'149","reported_seq":711,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:26.537490+0000","last_change":"2024-05-30T15:54:22.665082+0000","last_active":"2024-05-30T17:09:26.537490+0000","last_peered":"2024-05-30T17:09:26.537490+0000","last_clean":"2024-05-30T17:09:26.537490+0000","last_became_active":"2024-05-30T15:54:22.664353+0000","last_became_peered":"2024-05-30T15:54:22.664353+0000","last_unstale":"2024-05-30T17:09:26.537490+0000","last_undegraded":"2024-05-30T17:09:26.537490+0000","last_fullsized":"2024-05-30T17:09:26.537490+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:15:14.584034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":173,"num_read_kb":165,"num_write":120,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"265'527","reported_seq":2477,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.887321+0000","last_change":"2024-05-30T15:54:24.682250+0000","last_active":"2024-05-30T17:13:24.887321+0000","last_peered":"2024-05-30T17:13:24.887321+0000","last_clean":"2024-05-30T17:13:24.887321+0000","last_became_active":"2024-05-30T15:54:24.681503+0000","last_became_peered":"2024-05-30T15:54:24.681503+0000","last_unstale":"2024-05-30T17:13:24.887321+0000","last_undegraded":"2024-05-30T17:13:24.887321+0000","last_fullsized":"2024-05-30T17:13:24.887321+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":500,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:37:06.904193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"265'233","reported_seq":1074,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:19.091217+0000","last_change":"2024-05-30T15:54:26.671113+0000","last_active":"2024-05-30T17:13:19.091217+0000","last_peered":"2024-05-30T17:13:19.091217+0000","last_clean":"2024-05-30T17:13:19.091217+0000","last_became_active":"2024-05-30T15:54:26.670257+0000","last_became_peered":"2024-05-30T15:54:26.670257+0000","last_unstale":"2024-05-30T17:13:19.091217+0000","last_undegraded":"2024-05-30T17:13:19.091217+0000","last_fullsized":"2024-05-30T17:13:19.091217+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":200,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:56:52.779180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":661,"num_read_kb":640,"num_write":540,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"265'406","reported_seq":1988,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.698006+0000","last_change":"2024-05-30T15:54:18.660971+0000","last_active":"2024-05-30T17:13:18.698006+0000","last_peered":"2024-05-30T17:13:18.698006+0000","last_clean":"2024-05-30T17:13:18.698006+0000","last_became_active":"2024-05-30T15:54:18.660656+0000","last_became_peered":"2024-05-30T15:54:18.660656+0000","last_unstale":"2024-05-30T17:13:18.698006+0000","last_undegraded":"2024-05-30T17:13:18.698006+0000","last_fullsized":"2024-05-30T17:13:18.698006+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":300,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:13:06.838766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2368,"num_read_kb":2240,"num_write":473,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"265'190","reported_seq":819,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.558239+0000","last_change":"2024-05-30T15:54:16.668687+0000","last_active":"2024-05-30T17:11:16.558239+0000","last_peered":"2024-05-30T17:11:16.558239+0000","last_clean":"2024-05-30T17:11:16.558239+0000","last_became_active":"2024-05-30T15:54:16.668083+0000","last_became_peered":"2024-05-30T15:54:16.668083+0000","last_unstale":"2024-05-30T17:11:16.558239+0000","last_undegraded":"2024-05-30T17:11:16.558239+0000","last_fullsized":"2024-05-30T17:11:16.558239+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":100,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:37:10.668986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5290687,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":405,"num_read_kb":13963,"num_write":1053,"num_write_kb":66137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:17.124781+0000","last_change":"2024-05-30T15:54:20.659591+0000","last_active":"2024-05-30T17:09:17.124781+0000","last_peered":"2024-05-30T17:09:17.124781+0000","last_clean":"2024-05-30T17:09:17.124781+0000","last_became_active":"2024-05-30T15:54:20.658870+0000","last_became_peered":"2024-05-30T15:54:20.658870+0000","last_unstale":"2024-05-30T17:09:17.124781+0000","last_undegraded":"2024-05-30T17:09:17.124781+0000","last_fullsized":"2024-05-30T17:09:17.124781+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:06:21.446324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"265'138","reported_seq":721,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.727691+0000","last_change":"2024-05-30T15:54:22.851241+0000","last_active":"2024-05-30T17:13:20.727691+0000","last_peered":"2024-05-30T17:13:20.727691+0000","last_clean":"2024-05-30T17:13:20.727691+0000","last_became_active":"2024-05-30T15:54:22.849978+0000","last_became_peered":"2024-05-30T15:54:22.849978+0000","last_unstale":"2024-05-30T17:13:20.727691+0000","last_undegraded":"2024-05-30T17:13:20.727691+0000","last_fullsized":"2024-05-30T17:13:20.727691+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":100,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:42:52.749855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8486,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":180,"num_read_kb":226,"num_write":98,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"265'546","reported_seq":2569,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.895542+0000","last_change":"2024-05-30T15:54:24.688312+0000","last_active":"2024-05-30T17:13:24.895542+0000","last_peered":"2024-05-30T17:13:24.895542+0000","last_clean":"2024-05-30T17:13:24.895542+0000","last_became_active":"2024-05-30T15:54:24.686546+0000","last_became_peered":"2024-05-30T15:54:24.686546+0000","last_unstale":"2024-05-30T17:13:24.895542+0000","last_undegraded":"2024-05-30T17:13:24.895542+0000","last_fullsized":"2024-05-30T17:13:24.895542+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":500,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:12:50.305701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"265'895","reported_seq":2510,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.673971+0000","last_change":"2024-05-30T15:54:18.658824+0000","last_active":"2024-05-30T17:13:18.673971+0000","last_peered":"2024-05-30T17:13:18.673971+0000","last_clean":"2024-05-30T17:13:18.673971+0000","last_became_active":"2024-05-30T15:54:18.658124+0000","last_became_peered":"2024-05-30T15:54:18.658124+0000","last_unstale":"2024-05-30T17:13:18.673971+0000","last_undegraded":"2024-05-30T17:13:18.673971+0000","last_fullsized":"2024-05-30T17:13:18.673971+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":800,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:47:47.836108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4127,"num_read_kb":3657,"num_write":1246,"num_write_kb":491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"265'205","reported_seq":885,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:17.601310+0000","last_change":"2024-05-30T15:54:16.659141+0000","last_active":"2024-05-30T17:11:17.601310+0000","last_peered":"2024-05-30T17:11:17.601310+0000","last_clean":"2024-05-30T17:11:17.601310+0000","last_became_active":"2024-05-30T15:54:16.658279+0000","last_became_peered":"2024-05-30T15:54:16.658279+0000","last_unstale":"2024-05-30T17:11:17.601310+0000","last_undegraded":"2024-05-30T17:11:17.601310+0000","last_fullsized":"2024-05-30T17:11:17.601310+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":100,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:04:21.938813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1308948,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":477,"num_read_kb":111643,"num_write":1131,"num_write_kb":26225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":510,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:11.551997+0000","last_change":"2024-05-30T15:54:20.662333+0000","last_active":"2024-05-30T17:10:11.551997+0000","last_peered":"2024-05-30T17:10:11.551997+0000","last_clean":"2024-05-30T17:10:11.551997+0000","last_became_active":"2024-05-30T15:54:20.661583+0000","last_became_peered":"2024-05-30T15:54:20.661583+0000","last_unstale":"2024-05-30T17:10:11.551997+0000","last_undegraded":"2024-05-30T17:10:11.551997+0000","last_fullsized":"2024-05-30T17:10:11.551997+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:07:17.026476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"265'252","reported_seq":895,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:36.971452+0000","last_change":"2024-05-30T15:54:22.850012+0000","last_active":"2024-05-30T17:12:36.971452+0000","last_peered":"2024-05-30T17:12:36.971452+0000","last_clean":"2024-05-30T17:12:36.971452+0000","last_became_active":"2024-05-30T15:54:22.849126+0000","last_became_peered":"2024-05-30T15:54:22.849126+0000","last_unstale":"2024-05-30T17:12:36.971452+0000","last_undegraded":"2024-05-30T17:12:36.971452+0000","last_fullsized":"2024-05-30T17:12:36.971452+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":200,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:34:44.383862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7991,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":343,"num_read_kb":359,"num_write":206,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":502,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:56.786717+0000","last_change":"2024-05-30T15:54:24.674322+0000","last_active":"2024-05-30T17:10:56.786717+0000","last_peered":"2024-05-30T17:10:56.786717+0000","last_clean":"2024-05-30T17:10:56.786717+0000","last_became_active":"2024-05-30T15:54:24.673506+0000","last_became_peered":"2024-05-30T15:54:24.673506+0000","last_unstale":"2024-05-30T17:10:56.786717+0000","last_undegraded":"2024-05-30T17:10:56.786717+0000","last_fullsized":"2024-05-30T17:10:56.786717+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:07.777377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"265'543","reported_seq":1373,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.867831+0000","last_change":"2024-05-30T15:54:26.687129+0000","last_active":"2024-05-30T17:13:24.867831+0000","last_peered":"2024-05-30T17:13:24.867831+0000","last_clean":"2024-05-30T17:13:24.867831+0000","last_became_active":"2024-05-30T15:54:26.686389+0000","last_became_peered":"2024-05-30T15:54:26.686389+0000","last_unstale":"2024-05-30T17:13:24.867831+0000","last_undegraded":"2024-05-30T17:13:24.867831+0000","last_fullsized":"2024-05-30T17:13:24.867831+0000","mapping_epoch":32,"log_start":"265'500","ondisk_log_start":"265'500","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":500,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:44:22.858001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1297,"num_read_kb":1288,"num_write":1137,"num_write_kb":385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"265'1398","reported_seq":3105,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.685832+0000","last_change":"2024-05-30T15:54:18.640378+0000","last_active":"2024-05-30T17:13:18.685832+0000","last_peered":"2024-05-30T17:13:18.685832+0000","last_clean":"2024-05-30T17:13:18.685832+0000","last_became_active":"2024-05-30T15:54:18.639667+0000","last_became_peered":"2024-05-30T15:54:18.639667+0000","last_unstale":"2024-05-30T17:13:18.685832+0000","last_undegraded":"2024-05-30T17:13:18.685832+0000","last_fullsized":"2024-05-30T17:13:18.685832+0000","mapping_epoch":24,"log_start":"265'1300","ondisk_log_start":"265'1300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":1300,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:30:46.931867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5970,"num_read_kb":5130,"num_write":2044,"num_write_kb":884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"265'208","reported_seq":862,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.560609+0000","last_change":"2024-05-30T15:54:16.661968+0000","last_active":"2024-05-30T17:10:46.560609+0000","last_peered":"2024-05-30T17:10:46.560609+0000","last_clean":"2024-05-30T17:10:46.560609+0000","last_became_active":"2024-05-30T15:54:16.659427+0000","last_became_peered":"2024-05-30T15:54:16.659427+0000","last_unstale":"2024-05-30T17:10:46.560609+0000","last_undegraded":"2024-05-30T17:10:46.560609+0000","last_fullsized":"2024-05-30T17:10:46.560609+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":100,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:29:13.804456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5946564,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":462,"num_read_kb":34119,"num_write":1152,"num_write_kb":42367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":513,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:11.545498+0000","last_change":"2024-05-30T15:54:20.647908+0000","last_active":"2024-05-30T17:10:11.545498+0000","last_peered":"2024-05-30T17:10:11.545498+0000","last_clean":"2024-05-30T17:10:11.545498+0000","last_became_active":"2024-05-30T15:54:20.647565+0000","last_became_peered":"2024-05-30T15:54:20.647565+0000","last_unstale":"2024-05-30T17:10:11.545498+0000","last_undegraded":"2024-05-30T17:10:11.545498+0000","last_fullsized":"2024-05-30T17:10:11.545498+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:19:36.402906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"265'142","reported_seq":730,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:11.612518+0000","last_change":"2024-05-30T15:54:22.671276+0000","last_active":"2024-05-30T17:13:11.612518+0000","last_peered":"2024-05-30T17:13:11.612518+0000","last_clean":"2024-05-30T17:13:11.612518+0000","last_became_active":"2024-05-30T15:54:22.670064+0000","last_became_peered":"2024-05-30T15:54:22.670064+0000","last_unstale":"2024-05-30T17:13:11.612518+0000","last_undegraded":"2024-05-30T17:13:11.612518+0000","last_fullsized":"2024-05-30T17:13:11.612518+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":100,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:40:10.621760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16203,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":188,"num_read_kb":263,"num_write":104,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":504,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:36.577286+0000","last_change":"2024-05-30T15:54:24.688941+0000","last_active":"2024-05-30T17:10:36.577286+0000","last_peered":"2024-05-30T17:10:36.577286+0000","last_clean":"2024-05-30T17:10:36.577286+0000","last_became_active":"2024-05-30T15:54:24.688046+0000","last_became_peered":"2024-05-30T15:54:24.688046+0000","last_unstale":"2024-05-30T17:10:36.577286+0000","last_undegraded":"2024-05-30T17:10:36.577286+0000","last_fullsized":"2024-05-30T17:10:36.577286+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:44:58.051665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"265'2369","reported_seq":4189,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.624292+0000","last_change":"2024-05-30T15:54:26.692267+0000","last_active":"2024-05-30T17:13:18.624292+0000","last_peered":"2024-05-30T17:13:18.624292+0000","last_clean":"2024-05-30T17:13:18.624292+0000","last_became_active":"2024-05-30T15:54:26.691574+0000","last_became_peered":"2024-05-30T15:54:26.691574+0000","last_unstale":"2024-05-30T17:13:18.624292+0000","last_undegraded":"2024-05-30T17:13:18.624292+0000","last_fullsized":"2024-05-30T17:13:18.624292+0000","mapping_epoch":32,"log_start":"265'2300","ondisk_log_start":"265'2300","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":2300,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:33:03.952010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5943,"num_read_kb":5935,"num_write":4785,"num_write_kb":1767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"265'371","reported_seq":1792,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:35.664016+0000","last_change":"2024-05-30T15:54:18.670009+0000","last_active":"2024-05-30T17:11:35.664016+0000","last_peered":"2024-05-30T17:11:35.664016+0000","last_clean":"2024-05-30T17:11:35.664016+0000","last_became_active":"2024-05-30T15:54:18.668963+0000","last_became_peered":"2024-05-30T15:54:18.668963+0000","last_unstale":"2024-05-30T17:11:35.664016+0000","last_undegraded":"2024-05-30T17:11:35.664016+0000","last_fullsized":"2024-05-30T17:11:35.664016+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":300,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:33:28.500306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2011,"num_read_kb":1890,"num_write":455,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"265'230","reported_seq":870,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.560532+0000","last_change":"2024-05-30T15:54:16.650507+0000","last_active":"2024-05-30T17:10:46.560532+0000","last_peered":"2024-05-30T17:10:46.560532+0000","last_clean":"2024-05-30T17:10:46.560532+0000","last_became_active":"2024-05-30T15:54:16.649605+0000","last_became_peered":"2024-05-30T15:54:16.649605+0000","last_unstale":"2024-05-30T17:10:46.560532+0000","last_undegraded":"2024-05-30T17:10:46.560532+0000","last_fullsized":"2024-05-30T17:10:46.560532+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":200,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:10:51.493426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5355180,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":459,"num_read_kb":32396,"num_write":1233,"num_write_kb":40071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":512,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.554033+0000","last_change":"2024-05-30T15:54:20.652270+0000","last_active":"2024-05-30T17:10:51.554033+0000","last_peered":"2024-05-30T17:10:51.554033+0000","last_clean":"2024-05-30T17:10:51.554033+0000","last_became_active":"2024-05-30T15:54:20.651369+0000","last_became_peered":"2024-05-30T15:54:20.651369+0000","last_unstale":"2024-05-30T17:10:51.554033+0000","last_undegraded":"2024-05-30T17:10:51.554033+0000","last_fullsized":"2024-05-30T17:10:51.554033+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:06:27.275568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"265'187","reported_seq":798,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:26.585367+0000","last_change":"2024-05-30T15:54:22.666930+0000","last_active":"2024-05-30T17:12:26.585367+0000","last_peered":"2024-05-30T17:12:26.585367+0000","last_clean":"2024-05-30T17:12:26.585367+0000","last_became_active":"2024-05-30T15:54:22.664874+0000","last_became_peered":"2024-05-30T15:54:22.664874+0000","last_unstale":"2024-05-30T17:12:26.585367+0000","last_undegraded":"2024-05-30T17:12:26.585367+0000","last_fullsized":"2024-05-30T17:12:26.585367+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":100,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:25:56.647990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6891,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":250,"num_read_kb":293,"num_write":144,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"265'546","reported_seq":2479,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.897156+0000","last_change":"2024-05-30T15:54:24.674977+0000","last_active":"2024-05-30T17:13:24.897156+0000","last_peered":"2024-05-30T17:13:24.897156+0000","last_clean":"2024-05-30T17:13:24.897156+0000","last_became_active":"2024-05-30T15:54:24.670141+0000","last_became_peered":"2024-05-30T15:54:24.670141+0000","last_unstale":"2024-05-30T17:13:24.897156+0000","last_undegraded":"2024-05-30T17:13:24.897156+0000","last_fullsized":"2024-05-30T17:13:24.897156+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":500,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:38:58.029893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"265'107","reported_seq":611,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:16.569182+0000","last_change":"2024-05-30T15:54:26.693610+0000","last_active":"2024-05-30T17:10:16.569182+0000","last_peered":"2024-05-30T17:10:16.569182+0000","last_clean":"2024-05-30T17:10:16.569182+0000","last_became_active":"2024-05-30T15:54:26.689987+0000","last_became_peered":"2024-05-30T15:54:26.689987+0000","last_unstale":"2024-05-30T17:10:16.569182+0000","last_undegraded":"2024-05-30T17:10:16.569182+0000","last_fullsized":"2024-05-30T17:10:16.569182+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:15:03.997666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":80,"num_write":254,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"265'1331","reported_seq":2966,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.705722+0000","last_change":"2024-05-30T15:54:18.647985+0000","last_active":"2024-05-30T17:13:18.705722+0000","last_peered":"2024-05-30T17:13:18.705722+0000","last_clean":"2024-05-30T17:13:18.705722+0000","last_became_active":"2024-05-30T15:54:18.647634+0000","last_became_peered":"2024-05-30T15:54:18.647634+0000","last_unstale":"2024-05-30T17:13:18.705722+0000","last_undegraded":"2024-05-30T17:13:18.705722+0000","last_fullsized":"2024-05-30T17:13:18.705722+0000","mapping_epoch":24,"log_start":"265'1300","ondisk_log_start":"265'1300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":1300,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:55:27.116297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5717,"num_read_kb":4967,"num_write":1912,"num_write_kb":881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"265'202","reported_seq":817,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:47.145058+0000","last_change":"2024-05-30T15:54:16.668968+0000","last_active":"2024-05-30T17:10:47.145058+0000","last_peered":"2024-05-30T17:10:47.145058+0000","last_clean":"2024-05-30T17:10:47.145058+0000","last_became_active":"2024-05-30T15:54:16.668336+0000","last_became_peered":"2024-05-30T15:54:16.668336+0000","last_unstale":"2024-05-30T17:10:47.145058+0000","last_undegraded":"2024-05-30T17:10:47.145058+0000","last_fullsized":"2024-05-30T17:10:47.145058+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:22:09.399226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41078,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":399,"num_read_kb":1534,"num_write":1054,"num_write_kb":21253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":513,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:31.549752+0000","last_change":"2024-05-30T15:54:20.656467+0000","last_active":"2024-05-30T17:10:31.549752+0000","last_peered":"2024-05-30T17:10:31.549752+0000","last_clean":"2024-05-30T17:10:31.549752+0000","last_became_active":"2024-05-30T15:54:20.655573+0000","last_became_peered":"2024-05-30T15:54:20.655573+0000","last_unstale":"2024-05-30T17:10:31.549752+0000","last_undegraded":"2024-05-30T17:10:31.549752+0000","last_fullsized":"2024-05-30T17:10:31.549752+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:20:02.252327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"265'199","reported_seq":849,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.783939+0000","last_change":"2024-05-30T15:54:22.659510+0000","last_active":"2024-05-30T17:13:24.783939+0000","last_peered":"2024-05-30T17:13:24.783939+0000","last_clean":"2024-05-30T17:13:24.783939+0000","last_became_active":"2024-05-30T15:54:22.658700+0000","last_became_peered":"2024-05-30T15:54:22.658700+0000","last_unstale":"2024-05-30T17:13:24.783939+0000","last_undegraded":"2024-05-30T17:13:24.783939+0000","last_fullsized":"2024-05-30T17:13:24.783939+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T15:57:00.030682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18300,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":291,"num_read_kb":407,"num_write":146,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":505,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:16.546507+0000","last_change":"2024-05-30T15:54:24.674234+0000","last_active":"2024-05-30T17:10:16.546507+0000","last_peered":"2024-05-30T17:10:16.546507+0000","last_clean":"2024-05-30T17:10:16.546507+0000","last_became_active":"2024-05-30T15:54:24.673505+0000","last_became_peered":"2024-05-30T15:54:24.673505+0000","last_unstale":"2024-05-30T17:10:16.546507+0000","last_undegraded":"2024-05-30T17:10:16.546507+0000","last_fullsized":"2024-05-30T17:10:16.546507+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:16:54.874496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"265'99","reported_seq":616,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:49.195333+0000","last_change":"2024-05-30T15:54:26.701347+0000","last_active":"2024-05-30T17:10:49.195333+0000","last_peered":"2024-05-30T17:10:49.195333+0000","last_clean":"2024-05-30T17:10:49.195333+0000","last_became_active":"2024-05-30T15:54:26.700608+0000","last_became_peered":"2024-05-30T15:54:26.700608+0000","last_unstale":"2024-05-30T17:10:49.195333+0000","last_undegraded":"2024-05-30T17:10:49.195333+0000","last_fullsized":"2024-05-30T17:10:49.195333+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:57:26.315030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":918,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":103,"num_write":225,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"265'539","reported_seq":2167,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:28.313044+0000","last_change":"2024-05-30T15:54:18.658375+0000","last_active":"2024-05-30T17:12:28.313044+0000","last_peered":"2024-05-30T17:12:28.313044+0000","last_clean":"2024-05-30T17:12:28.313044+0000","last_became_active":"2024-05-30T15:54:18.657522+0000","last_became_peered":"2024-05-30T15:54:18.657522+0000","last_unstale":"2024-05-30T17:12:28.313044+0000","last_undegraded":"2024-05-30T17:12:28.313044+0000","last_fullsized":"2024-05-30T17:12:28.313044+0000","mapping_epoch":24,"log_start":"265'500","ondisk_log_start":"265'500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":500,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:59:15.813358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2751,"num_read_kb":2537,"num_write":708,"num_write_kb":252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"265'230","reported_seq":911,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:34.812361+0000","last_change":"2024-05-30T15:54:16.666646+0000","last_active":"2024-05-30T17:11:34.812361+0000","last_peered":"2024-05-30T17:11:34.812361+0000","last_clean":"2024-05-30T17:11:34.812361+0000","last_became_active":"2024-05-30T15:54:16.666244+0000","last_became_peered":"2024-05-30T15:54:16.666244+0000","last_unstale":"2024-05-30T17:11:34.812361+0000","last_undegraded":"2024-05-30T17:11:34.812361+0000","last_fullsized":"2024-05-30T17:11:34.812361+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":200,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:10:10.302094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11062814,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":634,"num_read_kb":32957,"num_write":1221,"num_write_kb":40334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":512,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.554107+0000","last_change":"2024-05-30T15:54:20.662409+0000","last_active":"2024-05-30T17:10:51.554107+0000","last_peered":"2024-05-30T17:10:51.554107+0000","last_clean":"2024-05-30T17:10:51.554107+0000","last_became_active":"2024-05-30T15:54:20.661967+0000","last_became_peered":"2024-05-30T15:54:20.661967+0000","last_unstale":"2024-05-30T17:10:51.554107+0000","last_undegraded":"2024-05-30T17:10:51.554107+0000","last_fullsized":"2024-05-30T17:10:51.554107+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:38:09.118798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"265'177","reported_seq":780,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:11.775063+0000","last_change":"2024-05-30T15:54:22.665466+0000","last_active":"2024-05-30T17:13:11.775063+0000","last_peered":"2024-05-30T17:13:11.775063+0000","last_clean":"2024-05-30T17:13:11.775063+0000","last_became_active":"2024-05-30T15:54:22.664732+0000","last_became_peered":"2024-05-30T15:54:22.664732+0000","last_unstale":"2024-05-30T17:13:11.775063+0000","last_undegraded":"2024-05-30T17:13:11.775063+0000","last_fullsized":"2024-05-30T17:13:11.775063+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":100,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:44:24.095083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19031,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":256,"num_read_kb":341,"num_write":198,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.582702+0000","last_change":"2024-05-30T15:54:24.679521+0000","last_active":"2024-05-30T17:11:16.582702+0000","last_peered":"2024-05-30T17:11:16.582702+0000","last_clean":"2024-05-30T17:11:16.582702+0000","last_became_active":"2024-05-30T15:54:24.679254+0000","last_became_peered":"2024-05-30T15:54:24.679254+0000","last_unstale":"2024-05-30T17:11:16.582702+0000","last_undegraded":"2024-05-30T17:11:16.582702+0000","last_fullsized":"2024-05-30T17:11:16.582702+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:16:16.737529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"265'105","reported_seq":623,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.579506+0000","last_change":"2024-05-30T15:54:26.697980+0000","last_active":"2024-05-30T17:10:46.579506+0000","last_peered":"2024-05-30T17:10:46.579506+0000","last_clean":"2024-05-30T17:10:46.579506+0000","last_became_active":"2024-05-30T15:54:26.697333+0000","last_became_peered":"2024-05-30T15:54:26.697333+0000","last_unstale":"2024-05-30T17:10:46.579506+0000","last_undegraded":"2024-05-30T17:10:46.579506+0000","last_fullsized":"2024-05-30T17:10:46.579506+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:11:10.336832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":270,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":107,"num_read_kb":90,"num_write":242,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"265'397","reported_seq":2059,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:09.478879+0000","last_change":"2024-05-30T15:54:18.636254+0000","last_active":"2024-05-30T17:13:09.478879+0000","last_peered":"2024-05-30T17:13:09.478879+0000","last_clean":"2024-05-30T17:13:09.478879+0000","last_became_active":"2024-05-30T15:54:18.635519+0000","last_became_peered":"2024-05-30T15:54:18.635519+0000","last_unstale":"2024-05-30T17:13:09.478879+0000","last_undegraded":"2024-05-30T17:13:09.478879+0000","last_fullsized":"2024-05-30T17:13:09.478879+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":300,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:25:03.148120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2332,"num_read_kb":2242,"num_write":450,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"265'173","reported_seq":793,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:48.596500+0000","last_change":"2024-05-30T15:54:16.685723+0000","last_active":"2024-05-30T17:11:48.596500+0000","last_peered":"2024-05-30T17:11:48.596500+0000","last_clean":"2024-05-30T17:11:48.596500+0000","last_became_active":"2024-05-30T15:54:16.685162+0000","last_became_peered":"2024-05-30T15:54:16.685162+0000","last_unstale":"2024-05-30T17:11:48.596500+0000","last_undegraded":"2024-05-30T17:11:48.596500+0000","last_fullsized":"2024-05-30T17:11:48.596500+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":100,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:11:20.614231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6251614,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":399,"num_read_kb":10586,"num_write":1025,"num_write_kb":21029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":510,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.785627+0000","last_change":"2024-05-30T15:54:20.646982+0000","last_active":"2024-05-30T17:10:51.785627+0000","last_peered":"2024-05-30T17:10:51.785627+0000","last_clean":"2024-05-30T17:10:51.785627+0000","last_became_active":"2024-05-30T15:54:20.646646+0000","last_became_peered":"2024-05-30T15:54:20.646646+0000","last_unstale":"2024-05-30T17:10:51.785627+0000","last_undegraded":"2024-05-30T17:10:51.785627+0000","last_fullsized":"2024-05-30T17:10:51.785627+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:31:33.090729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"265'179","reported_seq":792,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.851680+0000","last_change":"2024-05-30T15:54:22.670371+0000","last_active":"2024-05-30T17:13:24.851680+0000","last_peered":"2024-05-30T17:13:24.851680+0000","last_clean":"2024-05-30T17:13:24.851680+0000","last_became_active":"2024-05-30T15:54:22.669660+0000","last_became_peered":"2024-05-30T15:54:22.669660+0000","last_unstale":"2024-05-30T17:13:24.851680+0000","last_undegraded":"2024-05-30T17:13:24.851680+0000","last_fullsized":"2024-05-30T17:13:24.851680+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":100,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:45:08.898874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9643,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":249,"num_read_kb":299,"num_write":142,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"265'509","reported_seq":4592,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.889937+0000","last_change":"2024-05-30T15:54:24.673150+0000","last_active":"2024-05-30T17:13:24.889937+0000","last_peered":"2024-05-30T17:13:24.889937+0000","last_clean":"2024-05-30T17:13:24.889937+0000","last_became_active":"2024-05-30T15:54:24.672377+0000","last_became_peered":"2024-05-30T15:54:24.672377+0000","last_unstale":"2024-05-30T17:13:24.889937+0000","last_undegraded":"2024-05-30T17:13:24.889937+0000","last_fullsized":"2024-05-30T17:13:24.889937+0000","mapping_epoch":30,"log_start":"262'400","ondisk_log_start":"262'400","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":400,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:22:29.518162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"265'117","reported_seq":633,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.718032+0000","last_change":"2024-05-30T15:54:26.680642+0000","last_active":"2024-05-30T17:13:18.718032+0000","last_peered":"2024-05-30T17:13:18.718032+0000","last_clean":"2024-05-30T17:13:18.718032+0000","last_became_active":"2024-05-30T15:54:26.679866+0000","last_became_peered":"2024-05-30T15:54:26.679866+0000","last_unstale":"2024-05-30T17:13:18.718032+0000","last_undegraded":"2024-05-30T17:13:18.718032+0000","last_fullsized":"2024-05-30T17:13:18.718032+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:26:55.645297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1256,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":144,"num_read_kb":134,"num_write":293,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"265'389","reported_seq":1796,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:37.615748+0000","last_change":"2024-05-30T15:54:18.671784+0000","last_active":"2024-05-30T17:11:37.615748+0000","last_peered":"2024-05-30T17:11:37.615748+0000","last_clean":"2024-05-30T17:11:37.615748+0000","last_became_active":"2024-05-30T15:54:18.671088+0000","last_became_peered":"2024-05-30T15:54:18.671088+0000","last_unstale":"2024-05-30T17:11:37.615748+0000","last_undegraded":"2024-05-30T17:11:37.615748+0000","last_fullsized":"2024-05-30T17:11:37.615748+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":300,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:23:36.607873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2029,"num_read_kb":1894,"num_write":481,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"265'178","reported_seq":921,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:28.333806+0000","last_change":"2024-05-30T15:54:16.672575+0000","last_active":"2024-05-30T17:11:28.333806+0000","last_peered":"2024-05-30T17:11:28.333806+0000","last_clean":"2024-05-30T17:11:28.333806+0000","last_became_active":"2024-05-30T15:54:16.671477+0000","last_became_peered":"2024-05-30T15:54:16.671477+0000","last_unstale":"2024-05-30T17:11:28.333806+0000","last_undegraded":"2024-05-30T17:11:28.333806+0000","last_fullsized":"2024-05-30T17:11:28.333806+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":100,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:47:10.773488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":132257,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":677,"num_read_kb":167118,"num_write":1080,"num_write_kb":34693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"27'2","reported_seq":1154,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.724885+0000","last_change":"2024-05-30T15:54:20.663952+0000","last_active":"2024-05-30T17:13:24.724885+0000","last_peered":"2024-05-30T17:13:24.724885+0000","last_clean":"2024-05-30T17:13:24.724885+0000","last_became_active":"2024-05-30T15:54:20.662371+0000","last_became_peered":"2024-05-30T15:54:20.662371+0000","last_unstale":"2024-05-30T17:13:24.724885+0000","last_undegraded":"2024-05-30T17:13:24.724885+0000","last_fullsized":"2024-05-30T17:13:24.724885+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:43:22.768148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":643,"num_read_kb":643,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"265'109","reported_seq":656,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:56.582283+0000","last_change":"2024-05-30T15:54:22.849771+0000","last_active":"2024-05-30T17:10:56.582283+0000","last_peered":"2024-05-30T17:10:56.582283+0000","last_clean":"2024-05-30T17:10:56.582283+0000","last_became_active":"2024-05-30T15:54:22.849105+0000","last_became_peered":"2024-05-30T15:54:22.849105+0000","last_unstale":"2024-05-30T17:10:56.582283+0000","last_undegraded":"2024-05-30T17:10:56.582283+0000","last_fullsized":"2024-05-30T17:10:56.582283+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:56:52.974159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":120,"num_read_kb":115,"num_write":80,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"265'538","reported_seq":2314,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.891869+0000","last_change":"2024-05-30T15:54:24.678108+0000","last_active":"2024-05-30T17:13:24.891869+0000","last_peered":"2024-05-30T17:13:24.891869+0000","last_clean":"2024-05-30T17:13:24.891869+0000","last_became_active":"2024-05-30T15:54:24.677291+0000","last_became_peered":"2024-05-30T15:54:24.677291+0000","last_unstale":"2024-05-30T17:13:24.891869+0000","last_undegraded":"2024-05-30T17:13:24.891869+0000","last_fullsized":"2024-05-30T17:13:24.891869+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":500,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:59:57.567508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"265'135","reported_seq":656,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:15.308427+0000","last_change":"2024-05-30T15:54:26.675238+0000","last_active":"2024-05-30T17:13:15.308427+0000","last_peered":"2024-05-30T17:13:15.308427+0000","last_clean":"2024-05-30T17:13:15.308427+0000","last_became_active":"2024-05-30T15:54:26.674517+0000","last_became_peered":"2024-05-30T15:54:26.674517+0000","last_unstale":"2024-05-30T17:13:15.308427+0000","last_undegraded":"2024-05-30T17:13:15.308427+0000","last_fullsized":"2024-05-30T17:13:15.308427+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":100,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:38:58.206221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":141,"num_read_kb":129,"num_write":332,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"265'402","reported_seq":1940,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:12.347239+0000","last_change":"2024-05-30T15:54:18.657775+0000","last_active":"2024-05-30T17:13:12.347239+0000","last_peered":"2024-05-30T17:13:12.347239+0000","last_clean":"2024-05-30T17:13:12.347239+0000","last_became_active":"2024-05-30T15:54:18.657247+0000","last_became_peered":"2024-05-30T15:54:18.657247+0000","last_unstale":"2024-05-30T17:13:12.347239+0000","last_undegraded":"2024-05-30T17:13:12.347239+0000","last_fullsized":"2024-05-30T17:13:12.347239+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":300,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:33:29.912796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2106,"num_read_kb":2008,"num_write":461,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"265'180","reported_seq":790,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.553032+0000","last_change":"2024-05-30T15:54:54.427377+0000","last_active":"2024-05-30T17:10:46.553032+0000","last_peered":"2024-05-30T17:10:46.553032+0000","last_clean":"2024-05-30T17:10:46.553032+0000","last_became_active":"2024-05-30T15:54:54.426538+0000","last_became_peered":"2024-05-30T15:54:54.426538+0000","last_unstale":"2024-05-30T17:10:46.553032+0000","last_undegraded":"2024-05-30T17:10:46.553032+0000","last_fullsized":"2024-05-30T17:10:46.553032+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":100,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:57:32.635925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5292382,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":438,"num_read_kb":549,"num_write":1007,"num_write_kb":15096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,7],"acting":[2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":518,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.742564+0000","last_change":"2024-05-30T15:54:20.670270+0000","last_active":"2024-05-30T17:10:51.742564+0000","last_peered":"2024-05-30T17:10:51.742564+0000","last_clean":"2024-05-30T17:10:51.742564+0000","last_became_active":"2024-05-30T15:54:20.669492+0000","last_became_peered":"2024-05-30T15:54:20.669492+0000","last_unstale":"2024-05-30T17:10:51.742564+0000","last_undegraded":"2024-05-30T17:10:51.742564+0000","last_fullsized":"2024-05-30T17:10:51.742564+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:01:58.108666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"265'232","reported_seq":866,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:11.979664+0000","last_change":"2024-05-30T15:54:22.672749+0000","last_active":"2024-05-30T17:13:11.979664+0000","last_peered":"2024-05-30T17:13:11.979664+0000","last_clean":"2024-05-30T17:13:11.979664+0000","last_became_active":"2024-05-30T15:54:22.671943+0000","last_became_peered":"2024-05-30T15:54:22.671943+0000","last_unstale":"2024-05-30T17:13:11.979664+0000","last_undegraded":"2024-05-30T17:13:11.979664+0000","last_fullsized":"2024-05-30T17:13:11.979664+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":200,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:43:33.730358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4994,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":304,"num_read_kb":297,"num_write":174,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":500,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:56.553870+0000","last_change":"2024-05-30T15:54:24.682341+0000","last_active":"2024-05-30T17:10:56.553870+0000","last_peered":"2024-05-30T17:10:56.553870+0000","last_clean":"2024-05-30T17:10:56.553870+0000","last_became_active":"2024-05-30T15:54:24.681595+0000","last_became_peered":"2024-05-30T15:54:24.681595+0000","last_unstale":"2024-05-30T17:10:56.553870+0000","last_undegraded":"2024-05-30T17:10:56.553870+0000","last_fullsized":"2024-05-30T17:10:56.553870+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:03:03.587984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"265'115","reported_seq":622,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:31.794301+0000","last_change":"2024-05-30T15:54:26.689376+0000","last_active":"2024-05-30T17:11:31.794301+0000","last_peered":"2024-05-30T17:11:31.794301+0000","last_clean":"2024-05-30T17:11:31.794301+0000","last_became_active":"2024-05-30T15:54:26.688518+0000","last_became_peered":"2024-05-30T15:54:26.688518+0000","last_unstale":"2024-05-30T17:11:31.794301+0000","last_undegraded":"2024-05-30T17:11:31.794301+0000","last_fullsized":"2024-05-30T17:11:31.794301+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":100,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:13:29.397423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":619,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":113,"num_read_kb":112,"num_write":261,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"265'756","reported_seq":2265,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.637015+0000","last_change":"2024-05-30T15:54:18.678584+0000","last_active":"2024-05-30T17:11:21.637015+0000","last_peered":"2024-05-30T17:11:21.637015+0000","last_clean":"2024-05-30T17:11:21.637015+0000","last_became_active":"2024-05-30T15:54:18.677957+0000","last_became_peered":"2024-05-30T15:54:18.677957+0000","last_unstale":"2024-05-30T17:11:21.637015+0000","last_undegraded":"2024-05-30T17:11:21.637015+0000","last_fullsized":"2024-05-30T17:11:21.637015+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":700,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:51:57.717964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3440,"num_read_kb":3078,"num_write":1010,"num_write_kb":372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":627,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:26.555530+0000","last_change":"2024-05-30T15:54:03.031292+0000","last_active":"2024-05-30T17:10:26.555530+0000","last_peered":"2024-05-30T17:10:26.555530+0000","last_clean":"2024-05-30T17:10:26.555530+0000","last_became_active":"2024-05-30T15:54:03.029214+0000","last_became_peered":"2024-05-30T15:54:03.029214+0000","last_unstale":"2024-05-30T17:10:26.555530+0000","last_undegraded":"2024-05-30T17:10:26.555530+0000","last_fullsized":"2024-05-30T17:10:26.555530+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:01.512251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:01.512251+0000","last_clean_scrub_stamp":"2024-05-30T15:54:01.512251+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:40:07.389706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":530,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:17.151931+0000","last_change":"2024-05-30T15:54:08.578526+0000","last_active":"2024-05-30T17:11:17.151931+0000","last_peered":"2024-05-30T17:11:17.151931+0000","last_clean":"2024-05-30T17:11:17.151931+0000","last_became_active":"2024-05-30T15:54:08.577556+0000","last_became_peered":"2024-05-30T15:54:08.577556+0000","last_unstale":"2024-05-30T17:11:17.151931+0000","last_undegraded":"2024-05-30T17:11:17.151931+0000","last_fullsized":"2024-05-30T17:11:17.151931+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:36:52.339688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"265'186","reported_seq":841,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.587587+0000","last_change":"2024-05-30T15:54:16.674631+0000","last_active":"2024-05-30T17:11:21.587587+0000","last_peered":"2024-05-30T17:11:21.587587+0000","last_clean":"2024-05-30T17:11:21.587587+0000","last_became_active":"2024-05-30T15:54:16.674229+0000","last_became_peered":"2024-05-30T15:54:16.674229+0000","last_unstale":"2024-05-30T17:11:21.587587+0000","last_undegraded":"2024-05-30T17:11:21.587587+0000","last_fullsized":"2024-05-30T17:11:21.587587+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":100,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:38:06.634901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10797837,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":444,"num_read_kb":11470,"num_write":1071,"num_write_kb":25445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":512,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.570297+0000","last_change":"2024-05-30T15:54:20.665207+0000","last_active":"2024-05-30T17:11:26.570297+0000","last_peered":"2024-05-30T17:11:26.570297+0000","last_clean":"2024-05-30T17:11:26.570297+0000","last_became_active":"2024-05-30T15:54:20.664317+0000","last_became_peered":"2024-05-30T15:54:20.664317+0000","last_unstale":"2024-05-30T17:11:26.570297+0000","last_undegraded":"2024-05-30T17:11:26.570297+0000","last_fullsized":"2024-05-30T17:11:26.570297+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:57:42.587325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"265'322","reported_seq":1031,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.820569+0000","last_change":"2024-05-30T15:54:22.670731+0000","last_active":"2024-05-30T17:13:24.820569+0000","last_peered":"2024-05-30T17:13:24.820569+0000","last_clean":"2024-05-30T17:13:24.820569+0000","last_became_active":"2024-05-30T15:54:22.669871+0000","last_became_peered":"2024-05-30T15:54:22.669871+0000","last_unstale":"2024-05-30T17:13:24.820569+0000","last_undegraded":"2024-05-30T17:13:24.820569+0000","last_fullsized":"2024-05-30T17:13:24.820569+0000","mapping_epoch":28,"log_start":"265'300","ondisk_log_start":"265'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":300,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:48:31.269839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59120,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":621,"num_read_kb":899,"num_write":586,"num_write_kb":457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":507,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:16.569157+0000","last_change":"2024-05-30T15:54:24.685392+0000","last_active":"2024-05-30T17:10:16.569157+0000","last_peered":"2024-05-30T17:10:16.569157+0000","last_clean":"2024-05-30T17:10:16.569157+0000","last_became_active":"2024-05-30T15:54:24.684709+0000","last_became_peered":"2024-05-30T15:54:24.684709+0000","last_unstale":"2024-05-30T17:10:16.569157+0000","last_undegraded":"2024-05-30T17:10:16.569157+0000","last_fullsized":"2024-05-30T17:10:16.569157+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:57:38.351236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"265'174","reported_seq":878,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:16.948647+0000","last_change":"2024-05-30T15:54:26.687357+0000","last_active":"2024-05-30T17:13:16.948647+0000","last_peered":"2024-05-30T17:13:16.948647+0000","last_clean":"2024-05-30T17:13:16.948647+0000","last_became_active":"2024-05-30T15:54:26.686647+0000","last_became_peered":"2024-05-30T15:54:26.686647+0000","last_unstale":"2024-05-30T17:13:16.948647+0000","last_undegraded":"2024-05-30T17:13:16.948647+0000","last_fullsized":"2024-05-30T17:13:16.948647+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:16:53.692909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1020,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":416,"num_read_kb":404,"num_write":359,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"265'388","reported_seq":1862,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:28.256178+0000","last_change":"2024-05-30T15:54:18.649340+0000","last_active":"2024-05-30T17:12:28.256178+0000","last_peered":"2024-05-30T17:12:28.256178+0000","last_clean":"2024-05-30T17:12:28.256178+0000","last_became_active":"2024-05-30T15:54:18.648529+0000","last_became_peered":"2024-05-30T15:54:18.648529+0000","last_unstale":"2024-05-30T17:12:28.256178+0000","last_undegraded":"2024-05-30T17:12:28.256178+0000","last_fullsized":"2024-05-30T17:12:28.256178+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":300,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:56:22.361251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2082,"num_read_kb":1967,"num_write":465,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":531,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:31.556722+0000","last_change":"2024-05-30T15:54:08.575348+0000","last_active":"2024-05-30T17:10:31.556722+0000","last_peered":"2024-05-30T17:10:31.556722+0000","last_clean":"2024-05-30T17:10:31.556722+0000","last_became_active":"2024-05-30T15:54:08.574645+0000","last_became_peered":"2024-05-30T15:54:08.574645+0000","last_unstale":"2024-05-30T17:10:31.556722+0000","last_undegraded":"2024-05-30T17:10:31.556722+0000","last_fullsized":"2024-05-30T17:10:31.556722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:41:49.462483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"265'171","reported_seq":806,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.584098+0000","last_change":"2024-05-30T15:54:16.678919+0000","last_active":"2024-05-30T17:11:21.584098+0000","last_peered":"2024-05-30T17:11:21.584098+0000","last_clean":"2024-05-30T17:11:21.584098+0000","last_became_active":"2024-05-30T15:54:16.678386+0000","last_became_peered":"2024-05-30T15:54:16.678386+0000","last_unstale":"2024-05-30T17:11:21.584098+0000","last_undegraded":"2024-05-30T17:11:21.584098+0000","last_fullsized":"2024-05-30T17:11:21.584098+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:06:51.673817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6324848,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":361,"num_read_kb":7665,"num_write":1079,"num_write_kb":33234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":510,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:12.137521+0000","last_change":"2024-05-30T15:54:20.663148+0000","last_active":"2024-05-30T17:10:12.137521+0000","last_peered":"2024-05-30T17:10:12.137521+0000","last_clean":"2024-05-30T17:10:12.137521+0000","last_became_active":"2024-05-30T15:54:20.661502+0000","last_became_peered":"2024-05-30T15:54:20.661502+0000","last_unstale":"2024-05-30T17:10:12.137521+0000","last_undegraded":"2024-05-30T17:10:12.137521+0000","last_fullsized":"2024-05-30T17:10:12.137521+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:53:08.492976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"265'706","reported_seq":2523,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:28.415096+0000","last_change":"2024-05-30T15:54:22.672464+0000","last_active":"2024-05-30T17:13:28.415096+0000","last_peered":"2024-05-30T17:13:28.415096+0000","last_clean":"2024-05-30T17:13:28.415096+0000","last_became_active":"2024-05-30T15:54:22.671588+0000","last_became_peered":"2024-05-30T15:54:22.671588+0000","last_unstale":"2024-05-30T17:13:28.415096+0000","last_undegraded":"2024-05-30T17:13:28.415096+0000","last_fullsized":"2024-05-30T17:13:28.415096+0000","mapping_epoch":28,"log_start":"265'600","ondisk_log_start":"265'600","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":600,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:46:49.368553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28054,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":978,"num_read_kb":1146,"num_write":312,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":503,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:46.546239+0000","last_change":"2024-05-30T15:54:24.688069+0000","last_active":"2024-05-30T17:09:46.546239+0000","last_peered":"2024-05-30T17:09:46.546239+0000","last_clean":"2024-05-30T17:09:46.546239+0000","last_became_active":"2024-05-30T15:54:24.686641+0000","last_became_peered":"2024-05-30T15:54:24.686641+0000","last_unstale":"2024-05-30T17:09:46.546239+0000","last_undegraded":"2024-05-30T17:09:46.546239+0000","last_fullsized":"2024-05-30T17:09:46.546239+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:27:01.277040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"265'123","reported_seq":634,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:31.780995+0000","last_change":"2024-05-30T15:54:26.668522+0000","last_active":"2024-05-30T17:10:31.780995+0000","last_peered":"2024-05-30T17:10:31.780995+0000","last_clean":"2024-05-30T17:10:31.780995+0000","last_became_active":"2024-05-30T15:54:26.667751+0000","last_became_peered":"2024-05-30T15:54:26.667751+0000","last_unstale":"2024-05-30T17:10:31.780995+0000","last_undegraded":"2024-05-30T17:10:31.780995+0000","last_fullsized":"2024-05-30T17:10:31.780995+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":100,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:32:24.025165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":438,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":115,"num_write":291,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"265'326","reported_seq":1646,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:33.881393+0000","last_change":"2024-05-30T15:54:18.652138+0000","last_active":"2024-05-30T17:12:33.881393+0000","last_peered":"2024-05-30T17:12:33.881393+0000","last_clean":"2024-05-30T17:12:33.881393+0000","last_became_active":"2024-05-30T15:54:18.651853+0000","last_became_peered":"2024-05-30T15:54:18.651853+0000","last_unstale":"2024-05-30T17:12:33.881393+0000","last_undegraded":"2024-05-30T17:12:33.881393+0000","last_fullsized":"2024-05-30T17:12:33.881393+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":300,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:07:58.160840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1727,"num_read_kb":1636,"num_write":373,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":533,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.557705+0000","last_change":"2024-05-30T15:54:08.745221+0000","last_active":"2024-05-30T17:11:11.557705+0000","last_peered":"2024-05-30T17:11:11.557705+0000","last_clean":"2024-05-30T17:11:11.557705+0000","last_became_active":"2024-05-30T15:54:08.744898+0000","last_became_peered":"2024-05-30T15:54:08.744898+0000","last_unstale":"2024-05-30T17:11:11.557705+0000","last_undegraded":"2024-05-30T17:11:11.557705+0000","last_fullsized":"2024-05-30T17:11:11.557705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:38:33.683752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"265'183","reported_seq":811,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:31.780836+0000","last_change":"2024-05-30T15:54:16.650710+0000","last_active":"2024-05-30T17:10:31.780836+0000","last_peered":"2024-05-30T17:10:31.780836+0000","last_clean":"2024-05-30T17:10:31.780836+0000","last_became_active":"2024-05-30T15:54:16.650214+0000","last_became_peered":"2024-05-30T15:54:16.650214+0000","last_unstale":"2024-05-30T17:10:31.780836+0000","last_undegraded":"2024-05-30T17:10:31.780836+0000","last_fullsized":"2024-05-30T17:10:31.780836+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":100,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:41:20.055617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7478597,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":410,"num_read_kb":8332,"num_write":1152,"num_write_kb":25808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"27'1","reported_seq":514,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:11.545623+0000","last_change":"2024-05-30T15:54:20.667150+0000","last_active":"2024-05-30T17:10:11.545623+0000","last_peered":"2024-05-30T17:10:11.545623+0000","last_clean":"2024-05-30T17:10:11.545623+0000","last_became_active":"2024-05-30T15:54:20.666715+0000","last_became_peered":"2024-05-30T15:54:20.666715+0000","last_unstale":"2024-05-30T17:10:11.545623+0000","last_undegraded":"2024-05-30T17:10:11.545623+0000","last_fullsized":"2024-05-30T17:10:11.545623+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:57:01.669091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"265'259","reported_seq":1771,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.166581+0000","last_change":"2024-05-30T15:54:22.853392+0000","last_active":"2024-05-30T17:13:24.166581+0000","last_peered":"2024-05-30T17:13:24.166581+0000","last_clean":"2024-05-30T17:13:24.166581+0000","last_became_active":"2024-05-30T15:54:22.852604+0000","last_became_peered":"2024-05-30T15:54:22.852604+0000","last_unstale":"2024-05-30T17:13:24.166581+0000","last_undegraded":"2024-05-30T17:13:24.166581+0000","last_fullsized":"2024-05-30T17:13:24.166581+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":200,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:13:39.908798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":232,"num_read_kb":224,"num_write":183,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":509,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:31.586843+0000","last_change":"2024-05-30T15:54:24.680439+0000","last_active":"2024-05-30T17:11:31.586843+0000","last_peered":"2024-05-30T17:11:31.586843+0000","last_clean":"2024-05-30T17:11:31.586843+0000","last_became_active":"2024-05-30T15:54:24.679830+0000","last_became_peered":"2024-05-30T15:54:24.679830+0000","last_unstale":"2024-05-30T17:11:31.586843+0000","last_undegraded":"2024-05-30T17:11:31.586843+0000","last_fullsized":"2024-05-30T17:11:31.586843+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:05:18.168600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"265'251","reported_seq":860,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:19.239543+0000","last_change":"2024-05-30T15:54:26.699114+0000","last_active":"2024-05-30T17:13:19.239543+0000","last_peered":"2024-05-30T17:13:19.239543+0000","last_clean":"2024-05-30T17:13:19.239543+0000","last_became_active":"2024-05-30T15:54:26.696477+0000","last_became_peered":"2024-05-30T15:54:26.696477+0000","last_unstale":"2024-05-30T17:13:19.239543+0000","last_undegraded":"2024-05-30T17:13:19.239543+0000","last_fullsized":"2024-05-30T17:13:19.239543+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":200,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:15:58.988777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":476,"num_read_kb":500,"num_write":542,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"265'761","reported_seq":2321,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:35.663643+0000","last_change":"2024-05-30T15:54:18.640139+0000","last_active":"2024-05-30T17:11:35.663643+0000","last_peered":"2024-05-30T17:11:35.663643+0000","last_clean":"2024-05-30T17:11:35.663643+0000","last_became_active":"2024-05-30T15:54:18.639830+0000","last_became_peered":"2024-05-30T15:54:18.639830+0000","last_unstale":"2024-05-30T17:11:35.663643+0000","last_undegraded":"2024-05-30T17:11:35.663643+0000","last_fullsized":"2024-05-30T17:11:35.663643+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":700,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:05:32.448981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3481,"num_read_kb":3132,"num_write":1014,"num_write_kb":367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":530,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:17.151871+0000","last_change":"2024-05-30T15:54:08.575969+0000","last_active":"2024-05-30T17:11:17.151871+0000","last_peered":"2024-05-30T17:11:17.151871+0000","last_clean":"2024-05-30T17:11:17.151871+0000","last_became_active":"2024-05-30T15:54:08.574817+0000","last_became_peered":"2024-05-30T15:54:08.574817+0000","last_unstale":"2024-05-30T17:11:17.151871+0000","last_undegraded":"2024-05-30T17:11:17.151871+0000","last_fullsized":"2024-05-30T17:11:17.151871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:39:30.889837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"265'194","reported_seq":909,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.749242+0000","last_change":"2024-05-30T15:54:16.656169+0000","last_active":"2024-05-30T17:11:21.749242+0000","last_peered":"2024-05-30T17:11:21.749242+0000","last_clean":"2024-05-30T17:11:21.749242+0000","last_became_active":"2024-05-30T15:54:16.655347+0000","last_became_peered":"2024-05-30T15:54:16.655347+0000","last_unstale":"2024-05-30T17:11:21.749242+0000","last_undegraded":"2024-05-30T17:11:21.749242+0000","last_fullsized":"2024-05-30T17:11:21.749242+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":100,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:46:03.699500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8740487,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":567,"num_read_kb":42208,"num_write":1143,"num_write_kb":36370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":516,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:41.561839+0000","last_change":"2024-05-30T15:54:20.651897+0000","last_active":"2024-05-30T17:09:41.561839+0000","last_peered":"2024-05-30T17:09:41.561839+0000","last_clean":"2024-05-30T17:09:41.561839+0000","last_became_active":"2024-05-30T15:54:20.651449+0000","last_became_peered":"2024-05-30T15:54:20.651449+0000","last_unstale":"2024-05-30T17:09:41.561839+0000","last_undegraded":"2024-05-30T17:09:41.561839+0000","last_fullsized":"2024-05-30T17:09:41.561839+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:14:31.569912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"265'160","reported_seq":732,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:16.572838+0000","last_change":"2024-05-30T15:54:22.679074+0000","last_active":"2024-05-30T17:10:16.572838+0000","last_peered":"2024-05-30T17:10:16.572838+0000","last_clean":"2024-05-30T17:10:16.572838+0000","last_became_active":"2024-05-30T15:54:22.673881+0000","last_became_peered":"2024-05-30T15:54:22.673881+0000","last_unstale":"2024-05-30T17:10:16.572838+0000","last_undegraded":"2024-05-30T17:10:16.572838+0000","last_fullsized":"2024-05-30T17:10:16.572838+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":100,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:18:38.849518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":200,"num_read_kb":192,"num_write":144,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":501,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:16.545213+0000","last_change":"2024-05-30T15:54:24.678710+0000","last_active":"2024-05-30T17:10:16.545213+0000","last_peered":"2024-05-30T17:10:16.545213+0000","last_clean":"2024-05-30T17:10:16.545213+0000","last_became_active":"2024-05-30T15:54:24.675916+0000","last_became_peered":"2024-05-30T15:54:24.675916+0000","last_unstale":"2024-05-30T17:10:16.545213+0000","last_undegraded":"2024-05-30T17:10:16.545213+0000","last_fullsized":"2024-05-30T17:10:16.545213+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:25:04.329603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"265'137","reported_seq":655,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.579761+0000","last_change":"2024-05-30T15:54:26.705809+0000","last_active":"2024-05-30T17:13:20.579761+0000","last_peered":"2024-05-30T17:13:20.579761+0000","last_clean":"2024-05-30T17:13:20.579761+0000","last_became_active":"2024-05-30T15:54:26.704217+0000","last_became_peered":"2024-05-30T15:54:26.704217+0000","last_unstale":"2024-05-30T17:13:20.579761+0000","last_undegraded":"2024-05-30T17:13:20.579761+0000","last_fullsized":"2024-05-30T17:13:20.579761+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":100,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:47:09.950969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":834,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":138,"num_read_kb":136,"num_write":341,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"265'396","reported_seq":1827,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.793186+0000","last_change":"2024-05-30T15:54:18.665286+0000","last_active":"2024-05-30T17:11:26.793186+0000","last_peered":"2024-05-30T17:11:26.793186+0000","last_clean":"2024-05-30T17:11:26.793186+0000","last_became_active":"2024-05-30T15:54:18.665034+0000","last_became_peered":"2024-05-30T15:54:18.665034+0000","last_unstale":"2024-05-30T17:11:26.793186+0000","last_undegraded":"2024-05-30T17:11:26.793186+0000","last_fullsized":"2024-05-30T17:11:26.793186+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":300,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:02:55.348044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2053,"num_read_kb":1920,"num_write":476,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"265'215","reported_seq":856,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:27.154326+0000","last_change":"2024-05-30T15:54:16.651678+0000","last_active":"2024-05-30T17:11:27.154326+0000","last_peered":"2024-05-30T17:11:27.154326+0000","last_clean":"2024-05-30T17:11:27.154326+0000","last_became_active":"2024-05-30T15:54:16.650735+0000","last_became_peered":"2024-05-30T15:54:16.650735+0000","last_unstale":"2024-05-30T17:11:27.154326+0000","last_undegraded":"2024-05-30T17:11:27.154326+0000","last_fullsized":"2024-05-30T17:11:27.154326+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":200,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:36:29.849007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6324891,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":418,"num_read_kb":5617,"num_write":981,"num_write_kb":15732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":531,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.790986+0000","last_change":"2024-05-30T15:54:08.568214+0000","last_active":"2024-05-30T17:11:16.790986+0000","last_peered":"2024-05-30T17:11:16.790986+0000","last_clean":"2024-05-30T17:11:16.790986+0000","last_became_active":"2024-05-30T15:54:08.567900+0000","last_became_peered":"2024-05-30T15:54:08.567900+0000","last_unstale":"2024-05-30T17:11:16.790986+0000","last_undegraded":"2024-05-30T17:11:16.790986+0000","last_fullsized":"2024-05-30T17:11:16.790986+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:02:38.154765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":517,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.585770+0000","last_change":"2024-05-30T15:54:20.654924+0000","last_active":"2024-05-30T17:11:26.585770+0000","last_peered":"2024-05-30T17:11:26.585770+0000","last_clean":"2024-05-30T17:11:26.585770+0000","last_became_active":"2024-05-30T15:54:20.653790+0000","last_became_peered":"2024-05-30T15:54:20.653790+0000","last_unstale":"2024-05-30T17:11:26.585770+0000","last_undegraded":"2024-05-30T17:11:26.585770+0000","last_fullsized":"2024-05-30T17:11:26.585770+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:40:52.633245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"265'353","reported_seq":1045,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.849366+0000","last_change":"2024-05-30T15:54:22.674716+0000","last_active":"2024-05-30T17:13:24.849366+0000","last_peered":"2024-05-30T17:13:24.849366+0000","last_clean":"2024-05-30T17:13:24.849366+0000","last_became_active":"2024-05-30T15:54:22.673903+0000","last_became_peered":"2024-05-30T15:54:22.673903+0000","last_unstale":"2024-05-30T17:13:24.849366+0000","last_undegraded":"2024-05-30T17:13:24.849366+0000","last_fullsized":"2024-05-30T17:13:24.849366+0000","mapping_epoch":28,"log_start":"265'300","ondisk_log_start":"265'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":300,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:29:10.448451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13965,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1122,"num_read_kb":1192,"num_write":1666,"num_write_kb":1473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":513,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:31.751534+0000","last_change":"2024-05-30T15:54:24.679592+0000","last_active":"2024-05-30T17:11:31.751534+0000","last_peered":"2024-05-30T17:11:31.751534+0000","last_clean":"2024-05-30T17:11:31.751534+0000","last_became_active":"2024-05-30T15:54:24.677198+0000","last_became_peered":"2024-05-30T15:54:24.677198+0000","last_unstale":"2024-05-30T17:11:31.751534+0000","last_undegraded":"2024-05-30T17:11:31.751534+0000","last_fullsized":"2024-05-30T17:11:31.751534+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:47:11.591445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"265'117","reported_seq":635,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.126888+0000","last_change":"2024-05-30T15:54:26.676873+0000","last_active":"2024-05-30T17:13:20.126888+0000","last_peered":"2024-05-30T17:13:20.126888+0000","last_clean":"2024-05-30T17:13:20.126888+0000","last_became_active":"2024-05-30T15:54:26.676078+0000","last_became_peered":"2024-05-30T15:54:26.676078+0000","last_unstale":"2024-05-30T17:13:20.126888+0000","last_undegraded":"2024-05-30T17:13:20.126888+0000","last_fullsized":"2024-05-30T17:13:20.126888+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:52:51.891250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":727,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":114,"num_read_kb":108,"num_write":274,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"265'791","reported_seq":2423,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.705651+0000","last_change":"2024-05-30T15:54:18.644630+0000","last_active":"2024-05-30T17:13:20.705651+0000","last_peered":"2024-05-30T17:13:20.705651+0000","last_clean":"2024-05-30T17:13:20.705651+0000","last_became_active":"2024-05-30T15:54:18.642683+0000","last_became_peered":"2024-05-30T15:54:18.642683+0000","last_unstale":"2024-05-30T17:13:20.705651+0000","last_undegraded":"2024-05-30T17:13:20.705651+0000","last_fullsized":"2024-05-30T17:13:20.705651+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":700,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:23:29.409655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3596,"num_read_kb":3243,"num_write":1037,"num_write_kb":360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"265'179","reported_seq":822,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:53.428135+0000","last_change":"2024-05-30T15:54:16.664218+0000","last_active":"2024-05-30T17:11:53.428135+0000","last_peered":"2024-05-30T17:11:53.428135+0000","last_clean":"2024-05-30T17:11:53.428135+0000","last_became_active":"2024-05-30T15:54:16.663360+0000","last_became_peered":"2024-05-30T15:54:16.663360+0000","last_unstale":"2024-05-30T17:11:53.428135+0000","last_undegraded":"2024-05-30T17:11:53.428135+0000","last_fullsized":"2024-05-30T17:11:53.428135+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":100,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:22:13.108688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7656467,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":402,"num_read_kb":30298,"num_write":1058,"num_write_kb":42840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":529,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:26.755968+0000","last_change":"2024-05-30T15:54:08.577379+0000","last_active":"2024-05-30T17:09:26.755968+0000","last_peered":"2024-05-30T17:09:26.755968+0000","last_clean":"2024-05-30T17:09:26.755968+0000","last_became_active":"2024-05-30T15:54:08.576779+0000","last_became_peered":"2024-05-30T15:54:08.576779+0000","last_unstale":"2024-05-30T17:09:26.755968+0000","last_undegraded":"2024-05-30T17:09:26.755968+0000","last_fullsized":"2024-05-30T17:09:26.755968+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:09:01.046338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":511,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.560832+0000","last_change":"2024-05-30T15:54:20.655937+0000","last_active":"2024-05-30T17:11:26.560832+0000","last_peered":"2024-05-30T17:11:26.560832+0000","last_clean":"2024-05-30T17:11:26.560832+0000","last_became_active":"2024-05-30T15:54:20.655220+0000","last_became_peered":"2024-05-30T15:54:20.655220+0000","last_unstale":"2024-05-30T17:11:26.560832+0000","last_undegraded":"2024-05-30T17:11:26.560832+0000","last_fullsized":"2024-05-30T17:11:26.560832+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:42:12.608224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"265'197","reported_seq":794,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:31.794205+0000","last_change":"2024-05-30T15:54:22.851840+0000","last_active":"2024-05-30T17:11:31.794205+0000","last_peered":"2024-05-30T17:11:31.794205+0000","last_clean":"2024-05-30T17:11:31.794205+0000","last_became_active":"2024-05-30T15:54:22.851338+0000","last_became_peered":"2024-05-30T15:54:22.851338+0000","last_unstale":"2024-05-30T17:11:31.794205+0000","last_undegraded":"2024-05-30T17:11:31.794205+0000","last_fullsized":"2024-05-30T17:11:31.794205+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":100,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:25:07.146907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":256,"num_read_kb":249,"num_write":168,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"265'554","reported_seq":2762,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.884870+0000","last_change":"2024-05-30T15:54:24.677979+0000","last_active":"2024-05-30T17:13:24.884870+0000","last_peered":"2024-05-30T17:13:24.884870+0000","last_clean":"2024-05-30T17:13:24.884870+0000","last_became_active":"2024-05-30T15:54:24.677176+0000","last_became_peered":"2024-05-30T15:54:24.677176+0000","last_unstale":"2024-05-30T17:13:24.884870+0000","last_undegraded":"2024-05-30T17:13:24.884870+0000","last_fullsized":"2024-05-30T17:13:24.884870+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":500,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:01:33.668829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"265'124","reported_seq":631,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.862550+0000","last_change":"2024-05-30T15:54:26.697967+0000","last_active":"2024-05-30T17:13:24.862550+0000","last_peered":"2024-05-30T17:13:24.862550+0000","last_clean":"2024-05-30T17:13:24.862550+0000","last_became_active":"2024-05-30T15:54:26.697294+0000","last_became_peered":"2024-05-30T15:54:26.697294+0000","last_unstale":"2024-05-30T17:13:24.862550+0000","last_undegraded":"2024-05-30T17:13:24.862550+0000","last_fullsized":"2024-05-30T17:13:24.862550+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":100,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:31:13.852916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1933,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":112,"num_read_kb":106,"num_write":288,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"265'716","reported_seq":2110,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.659105+0000","last_change":"2024-05-30T15:54:18.653261+0000","last_active":"2024-05-30T17:12:44.659105+0000","last_peered":"2024-05-30T17:12:44.659105+0000","last_clean":"2024-05-30T17:12:44.659105+0000","last_became_active":"2024-05-30T15:54:18.652981+0000","last_became_peered":"2024-05-30T15:54:18.652981+0000","last_unstale":"2024-05-30T17:12:44.659105+0000","last_undegraded":"2024-05-30T17:12:44.659105+0000","last_fullsized":"2024-05-30T17:12:44.659105+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":700,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:28:55.055044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3098,"num_read_kb":2766,"num_write":954,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"265'195","reported_seq":857,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:25.480699+0000","last_change":"2024-05-30T15:54:16.652512+0000","last_active":"2024-05-30T17:10:25.480699+0000","last_peered":"2024-05-30T17:10:25.480699+0000","last_clean":"2024-05-30T17:10:25.480699+0000","last_became_active":"2024-05-30T15:54:16.652033+0000","last_became_peered":"2024-05-30T15:54:16.652033+0000","last_unstale":"2024-05-30T17:10:25.480699+0000","last_undegraded":"2024-05-30T17:10:25.480699+0000","last_fullsized":"2024-05-30T17:10:25.480699+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":100,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:21:56.815021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11145910,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":510,"num_read_kb":15333,"num_write":1047,"num_write_kb":22470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":529,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.772799+0000","last_change":"2024-05-30T15:54:08.574181+0000","last_active":"2024-05-30T17:09:56.772799+0000","last_peered":"2024-05-30T17:09:56.772799+0000","last_clean":"2024-05-30T17:09:56.772799+0000","last_became_active":"2024-05-30T15:54:08.573754+0000","last_became_peered":"2024-05-30T15:54:08.573754+0000","last_unstale":"2024-05-30T17:09:56.772799+0000","last_undegraded":"2024-05-30T17:09:56.772799+0000","last_fullsized":"2024-05-30T17:09:56.772799+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:35:01.844361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":521,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.750400+0000","last_change":"2024-05-30T15:54:20.657666+0000","last_active":"2024-05-30T17:11:26.750400+0000","last_peered":"2024-05-30T17:11:26.750400+0000","last_clean":"2024-05-30T17:11:26.750400+0000","last_became_active":"2024-05-30T15:54:20.656941+0000","last_became_peered":"2024-05-30T15:54:20.656941+0000","last_unstale":"2024-05-30T17:11:26.750400+0000","last_undegraded":"2024-05-30T17:11:26.750400+0000","last_fullsized":"2024-05-30T17:11:26.750400+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:45:33.081783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"265'130","reported_seq":703,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:31.751592+0000","last_change":"2024-05-30T15:54:22.674809+0000","last_active":"2024-05-30T17:11:31.751592+0000","last_peered":"2024-05-30T17:11:31.751592+0000","last_clean":"2024-05-30T17:11:31.751592+0000","last_became_active":"2024-05-30T15:54:22.674027+0000","last_became_peered":"2024-05-30T15:54:22.674027+0000","last_unstale":"2024-05-30T17:11:31.751592+0000","last_undegraded":"2024-05-30T17:11:31.751592+0000","last_fullsized":"2024-05-30T17:11:31.751592+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:34:39.215953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":164,"num_read_kb":159,"num_write":108,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"265'547","reported_seq":2630,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:28.474744+0000","last_change":"2024-05-30T15:54:24.677368+0000","last_active":"2024-05-30T17:13:28.474744+0000","last_peered":"2024-05-30T17:13:28.474744+0000","last_clean":"2024-05-30T17:13:28.474744+0000","last_became_active":"2024-05-30T15:54:24.676608+0000","last_became_peered":"2024-05-30T15:54:24.676608+0000","last_unstale":"2024-05-30T17:13:28.474744+0000","last_undegraded":"2024-05-30T17:13:28.474744+0000","last_fullsized":"2024-05-30T17:13:28.474744+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":500,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:44:44.176765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"265'227","reported_seq":774,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:11.610342+0000","last_change":"2024-05-30T15:54:26.700019+0000","last_active":"2024-05-30T17:13:11.610342+0000","last_peered":"2024-05-30T17:13:11.610342+0000","last_clean":"2024-05-30T17:13:11.610342+0000","last_became_active":"2024-05-30T15:54:26.699148+0000","last_became_peered":"2024-05-30T15:54:26.699148+0000","last_unstale":"2024-05-30T17:13:11.610342+0000","last_undegraded":"2024-05-30T17:13:11.610342+0000","last_fullsized":"2024-05-30T17:13:11.610342+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":200,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:16:32.958184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":326,"num_read_kb":298,"num_write":504,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"265'747","reported_seq":2339,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:28.311866+0000","last_change":"2024-05-30T15:54:18.658639+0000","last_active":"2024-05-30T17:12:28.311866+0000","last_peered":"2024-05-30T17:12:28.311866+0000","last_clean":"2024-05-30T17:12:28.311866+0000","last_became_active":"2024-05-30T15:54:18.657842+0000","last_became_peered":"2024-05-30T15:54:18.657842+0000","last_unstale":"2024-05-30T17:12:28.311866+0000","last_undegraded":"2024-05-30T17:12:28.311866+0000","last_fullsized":"2024-05-30T17:12:28.311866+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":700,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:08:00.261523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3632,"num_read_kb":3285,"num_write":997,"num_write_kb":370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"265'227","reported_seq":983,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:40.014659+0000","last_change":"2024-05-30T15:54:16.657515+0000","last_active":"2024-05-30T17:10:40.014659+0000","last_peered":"2024-05-30T17:10:40.014659+0000","last_clean":"2024-05-30T17:10:40.014659+0000","last_became_active":"2024-05-30T15:54:16.657067+0000","last_became_peered":"2024-05-30T15:54:16.657067+0000","last_unstale":"2024-05-30T17:10:40.014659+0000","last_undegraded":"2024-05-30T17:10:40.014659+0000","last_fullsized":"2024-05-30T17:10:40.014659+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":200,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:03:23.557850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14883328,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":674,"num_read_kb":72431,"num_write":1151,"num_write_kb":30547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":527,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:01.541851+0000","last_change":"2024-05-30T15:54:08.578053+0000","last_active":"2024-05-30T17:10:01.541851+0000","last_peered":"2024-05-30T17:10:01.541851+0000","last_clean":"2024-05-30T17:10:01.541851+0000","last_became_active":"2024-05-30T15:54:08.577191+0000","last_became_peered":"2024-05-30T15:54:08.577191+0000","last_unstale":"2024-05-30T17:10:01.541851+0000","last_undegraded":"2024-05-30T17:10:01.541851+0000","last_fullsized":"2024-05-30T17:10:01.541851+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:38:36.674998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":515,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:41.578396+0000","last_change":"2024-05-30T15:54:20.653740+0000","last_active":"2024-05-30T17:10:41.578396+0000","last_peered":"2024-05-30T17:10:41.578396+0000","last_clean":"2024-05-30T17:10:41.578396+0000","last_became_active":"2024-05-30T15:54:20.653404+0000","last_became_peered":"2024-05-30T15:54:20.653404+0000","last_unstale":"2024-05-30T17:10:41.578396+0000","last_undegraded":"2024-05-30T17:10:41.578396+0000","last_fullsized":"2024-05-30T17:10:41.578396+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:17:55.928914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"265'334","reported_seq":1069,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.809682+0000","last_change":"2024-05-30T15:54:22.849490+0000","last_active":"2024-05-30T17:13:24.809682+0000","last_peered":"2024-05-30T17:13:24.809682+0000","last_clean":"2024-05-30T17:13:24.809682+0000","last_became_active":"2024-05-30T15:54:22.848761+0000","last_became_peered":"2024-05-30T15:54:22.848761+0000","last_unstale":"2024-05-30T17:13:24.809682+0000","last_undegraded":"2024-05-30T17:13:24.809682+0000","last_fullsized":"2024-05-30T17:13:24.809682+0000","mapping_epoch":28,"log_start":"265'300","ondisk_log_start":"265'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:40:14.623631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34688,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":498,"num_read_kb":700,"num_write":262,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:31.589903+0000","last_change":"2024-05-30T15:54:24.687584+0000","last_active":"2024-05-30T17:11:31.589903+0000","last_peered":"2024-05-30T17:11:31.589903+0000","last_clean":"2024-05-30T17:11:31.589903+0000","last_became_active":"2024-05-30T15:54:24.686801+0000","last_became_peered":"2024-05-30T15:54:24.686801+0000","last_unstale":"2024-05-30T17:11:31.589903+0000","last_undegraded":"2024-05-30T17:11:31.589903+0000","last_fullsized":"2024-05-30T17:11:31.589903+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:41:00.549932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"265'150","reported_seq":651,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:55.992645+0000","last_change":"2024-05-30T15:54:26.701426+0000","last_active":"2024-05-30T17:11:55.992645+0000","last_peered":"2024-05-30T17:11:55.992645+0000","last_clean":"2024-05-30T17:11:55.992645+0000","last_became_active":"2024-05-30T15:54:26.700325+0000","last_became_peered":"2024-05-30T15:54:26.700325+0000","last_unstale":"2024-05-30T17:11:55.992645+0000","last_undegraded":"2024-05-30T17:11:55.992645+0000","last_fullsized":"2024-05-30T17:11:55.992645+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":100,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:05:22.451018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":204,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":115,"num_write":342,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"265'149","reported_seq":722,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:21.183321+0000","last_change":"2024-05-30T15:54:26.687046+0000","last_active":"2024-05-30T17:13:21.183321+0000","last_peered":"2024-05-30T17:13:21.183321+0000","last_clean":"2024-05-30T17:13:21.183321+0000","last_became_active":"2024-05-30T15:54:26.686310+0000","last_became_peered":"2024-05-30T15:54:26.686310+0000","last_unstale":"2024-05-30T17:13:21.183321+0000","last_undegraded":"2024-05-30T17:13:21.183321+0000","last_fullsized":"2024-05-30T17:13:21.183321+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:16:45.128093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":227,"num_read_kb":222,"num_write":347,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"265'746","reported_seq":2045,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.689780+0000","last_change":"2024-05-30T15:54:18.674169+0000","last_active":"2024-05-30T17:13:18.689780+0000","last_peered":"2024-05-30T17:13:18.689780+0000","last_clean":"2024-05-30T17:13:18.689780+0000","last_became_active":"2024-05-30T15:54:18.673418+0000","last_became_peered":"2024-05-30T15:54:18.673418+0000","last_unstale":"2024-05-30T17:13:18.689780+0000","last_undegraded":"2024-05-30T17:13:18.689780+0000","last_fullsized":"2024-05-30T17:13:18.689780+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":700,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:47:19.313441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3258,"num_read_kb":2851,"num_write":1044,"num_write_kb":425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"265'197","reported_seq":816,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.543474+0000","last_change":"2024-05-30T15:54:54.451649+0000","last_active":"2024-05-30T17:09:56.543474+0000","last_peered":"2024-05-30T17:09:56.543474+0000","last_clean":"2024-05-30T17:09:56.543474+0000","last_became_active":"2024-05-30T15:54:54.450966+0000","last_became_peered":"2024-05-30T15:54:54.450966+0000","last_unstale":"2024-05-30T17:09:56.543474+0000","last_undegraded":"2024-05-30T17:09:56.543474+0000","last_fullsized":"2024-05-30T17:09:56.543474+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":100,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:16:23.208005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5857853,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":396,"num_read_kb":38159,"num_write":1159,"num_write_kb":35962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":508,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:01.760976+0000","last_change":"2024-05-30T15:54:20.665689+0000","last_active":"2024-05-30T17:09:01.760976+0000","last_peered":"2024-05-30T17:09:01.760976+0000","last_clean":"2024-05-30T17:09:01.760976+0000","last_became_active":"2024-05-30T15:54:20.664882+0000","last_became_peered":"2024-05-30T15:54:20.664882+0000","last_unstale":"2024-05-30T17:09:01.760976+0000","last_undegraded":"2024-05-30T17:09:01.760976+0000","last_fullsized":"2024-05-30T17:09:01.760976+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:08:16.396333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"265'149","reported_seq":743,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.588979+0000","last_change":"2024-05-30T15:54:22.672900+0000","last_active":"2024-05-30T17:11:26.588979+0000","last_peered":"2024-05-30T17:11:26.588979+0000","last_clean":"2024-05-30T17:11:26.588979+0000","last_became_active":"2024-05-30T15:54:22.672170+0000","last_became_peered":"2024-05-30T15:54:22.672170+0000","last_unstale":"2024-05-30T17:11:26.588979+0000","last_undegraded":"2024-05-30T17:11:26.588979+0000","last_fullsized":"2024-05-30T17:11:26.588979+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:07:18.306116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7618,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":196,"num_read_kb":242,"num_write":106,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":501,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:06.556338+0000","last_change":"2024-05-30T15:54:24.688552+0000","last_active":"2024-05-30T17:09:06.556338+0000","last_peered":"2024-05-30T17:09:06.556338+0000","last_clean":"2024-05-30T17:09:06.556338+0000","last_became_active":"2024-05-30T15:54:24.688053+0000","last_became_peered":"2024-05-30T15:54:24.688053+0000","last_unstale":"2024-05-30T17:09:06.556338+0000","last_undegraded":"2024-05-30T17:09:06.556338+0000","last_fullsized":"2024-05-30T17:09:06.556338+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:53:25.256385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"265'117","reported_seq":642,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.579861+0000","last_change":"2024-05-30T15:54:26.702292+0000","last_active":"2024-05-30T17:10:46.579861+0000","last_peered":"2024-05-30T17:10:46.579861+0000","last_clean":"2024-05-30T17:10:46.579861+0000","last_became_active":"2024-05-30T15:54:26.701588+0000","last_became_peered":"2024-05-30T15:54:26.701588+0000","last_unstale":"2024-05-30T17:10:46.579861+0000","last_undegraded":"2024-05-30T17:10:46.579861+0000","last_fullsized":"2024-05-30T17:10:46.579861+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:13:38.040108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":826,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":172,"num_read_kb":153,"num_write":293,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"265'321","reported_seq":1758,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:55.923011+0000","last_change":"2024-05-30T15:54:18.671021+0000","last_active":"2024-05-30T17:11:55.923011+0000","last_peered":"2024-05-30T17:11:55.923011+0000","last_clean":"2024-05-30T17:11:55.923011+0000","last_became_active":"2024-05-30T15:54:18.670707+0000","last_became_peered":"2024-05-30T15:54:18.670707+0000","last_unstale":"2024-05-30T17:11:55.923011+0000","last_undegraded":"2024-05-30T17:11:55.923011+0000","last_fullsized":"2024-05-30T17:11:55.923011+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":300,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:20:52.055626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1910,"num_read_kb":1828,"num_write":376,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"265'215","reported_seq":879,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:11.752074+0000","last_change":"2024-05-30T15:54:16.663769+0000","last_active":"2024-05-30T17:09:11.752074+0000","last_peered":"2024-05-30T17:09:11.752074+0000","last_clean":"2024-05-30T17:09:11.752074+0000","last_became_active":"2024-05-30T15:54:16.660427+0000","last_became_peered":"2024-05-30T15:54:16.660427+0000","last_unstale":"2024-05-30T17:09:11.752074+0000","last_undegraded":"2024-05-30T17:09:11.752074+0000","last_fullsized":"2024-05-30T17:09:11.752074+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":200,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:37:35.566472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5852361,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":514,"num_read_kb":33224,"num_write":1144,"num_write_kb":46290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":509,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:01.555309+0000","last_change":"2024-05-30T15:54:20.649980+0000","last_active":"2024-05-30T17:09:01.555309+0000","last_peered":"2024-05-30T17:09:01.555309+0000","last_clean":"2024-05-30T17:09:01.555309+0000","last_became_active":"2024-05-30T15:54:20.649330+0000","last_became_peered":"2024-05-30T15:54:20.649330+0000","last_unstale":"2024-05-30T17:09:01.555309+0000","last_undegraded":"2024-05-30T17:09:01.555309+0000","last_fullsized":"2024-05-30T17:09:01.555309+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:16:31.107653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"265'71","reported_seq":595,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:06.552569+0000","last_change":"2024-05-30T15:54:22.663453+0000","last_active":"2024-05-30T17:09:06.552569+0000","last_peered":"2024-05-30T17:09:06.552569+0000","last_clean":"2024-05-30T17:09:06.552569+0000","last_became_active":"2024-05-30T15:54:22.662738+0000","last_became_peered":"2024-05-30T15:54:22.662738+0000","last_unstale":"2024-05-30T17:09:06.552569+0000","last_undegraded":"2024-05-30T17:09:06.552569+0000","last_fullsized":"2024-05-30T17:09:06.552569+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:42:21.227185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":50,"num_write":40,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":499,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:06.750855+0000","last_change":"2024-05-30T15:54:24.682005+0000","last_active":"2024-05-30T17:09:06.750855+0000","last_peered":"2024-05-30T17:09:06.750855+0000","last_clean":"2024-05-30T17:09:06.750855+0000","last_became_active":"2024-05-30T15:54:24.680578+0000","last_became_peered":"2024-05-30T15:54:24.680578+0000","last_unstale":"2024-05-30T17:09:06.750855+0000","last_undegraded":"2024-05-30T17:09:06.750855+0000","last_fullsized":"2024-05-30T17:09:06.750855+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:54:25.321085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"265'212","reported_seq":772,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.665178+0000","last_change":"2024-05-30T15:54:26.687158+0000","last_active":"2024-05-30T17:13:18.665178+0000","last_peered":"2024-05-30T17:13:18.665178+0000","last_clean":"2024-05-30T17:13:18.665178+0000","last_became_active":"2024-05-30T15:54:26.686553+0000","last_became_peered":"2024-05-30T15:54:26.686553+0000","last_unstale":"2024-05-30T17:13:18.665178+0000","last_undegraded":"2024-05-30T17:13:18.665178+0000","last_fullsized":"2024-05-30T17:13:18.665178+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:03:50.128325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":816,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":414,"num_read_kb":383,"num_write":513,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"265'413","reported_seq":1789,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.600546+0000","last_change":"2024-05-30T15:54:18.660456+0000","last_active":"2024-05-30T17:12:44.600546+0000","last_peered":"2024-05-30T17:12:44.600546+0000","last_clean":"2024-05-30T17:12:44.600546+0000","last_became_active":"2024-05-30T15:54:18.660089+0000","last_became_peered":"2024-05-30T15:54:18.660089+0000","last_unstale":"2024-05-30T17:12:44.600546+0000","last_undegraded":"2024-05-30T17:12:44.600546+0000","last_fullsized":"2024-05-30T17:12:44.600546+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":400,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:29:46.343317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2036,"num_read_kb":1890,"num_write":519,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"265'214","reported_seq":883,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:12.123493+0000","last_change":"2024-05-30T15:54:16.650752+0000","last_active":"2024-05-30T17:09:12.123493+0000","last_peered":"2024-05-30T17:09:12.123493+0000","last_clean":"2024-05-30T17:09:12.123493+0000","last_became_active":"2024-05-30T15:54:16.650025+0000","last_became_peered":"2024-05-30T15:54:16.650025+0000","last_unstale":"2024-05-30T17:09:12.123493+0000","last_undegraded":"2024-05-30T17:09:12.123493+0000","last_fullsized":"2024-05-30T17:09:12.123493+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":200,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:14:35.914964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5917240,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":497,"num_read_kb":102670,"num_write":1110,"num_write_kb":39714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":507,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:01.749537+0000","last_change":"2024-05-30T15:54:20.657385+0000","last_active":"2024-05-30T17:09:01.749537+0000","last_peered":"2024-05-30T17:09:01.749537+0000","last_clean":"2024-05-30T17:09:01.749537+0000","last_became_active":"2024-05-30T15:54:20.656663+0000","last_became_peered":"2024-05-30T15:54:20.656663+0000","last_unstale":"2024-05-30T17:09:01.749537+0000","last_undegraded":"2024-05-30T17:09:01.749537+0000","last_fullsized":"2024-05-30T17:09:01.749537+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:31:26.705085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"265'182","reported_seq":792,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.589082+0000","last_change":"2024-05-30T15:54:22.676934+0000","last_active":"2024-05-30T17:11:26.589082+0000","last_peered":"2024-05-30T17:11:26.589082+0000","last_clean":"2024-05-30T17:11:26.589082+0000","last_became_active":"2024-05-30T15:54:22.673673+0000","last_became_peered":"2024-05-30T15:54:22.673673+0000","last_unstale":"2024-05-30T17:11:26.589082+0000","last_undegraded":"2024-05-30T17:11:26.589082+0000","last_fullsized":"2024-05-30T17:11:26.589082+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":100,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:34:48.241130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10523,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":250,"num_read_kb":311,"num_write":142,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":498,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:06.529823+0000","last_change":"2024-05-30T15:54:24.676055+0000","last_active":"2024-05-30T17:09:06.529823+0000","last_peered":"2024-05-30T17:09:06.529823+0000","last_clean":"2024-05-30T17:09:06.529823+0000","last_became_active":"2024-05-30T15:54:24.675289+0000","last_became_peered":"2024-05-30T15:54:24.675289+0000","last_unstale":"2024-05-30T17:09:06.529823+0000","last_undegraded":"2024-05-30T17:09:06.529823+0000","last_fullsized":"2024-05-30T17:09:06.529823+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:37:01.468842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"265'138","reported_seq":634,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:27.127163+0000","last_change":"2024-05-30T15:54:26.704882+0000","last_active":"2024-05-30T17:09:27.127163+0000","last_peered":"2024-05-30T17:09:27.127163+0000","last_clean":"2024-05-30T17:09:27.127163+0000","last_became_active":"2024-05-30T15:54:26.703433+0000","last_became_peered":"2024-05-30T15:54:26.703433+0000","last_unstale":"2024-05-30T17:09:27.127163+0000","last_undegraded":"2024-05-30T17:09:27.127163+0000","last_fullsized":"2024-05-30T17:09:27.127163+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":100,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:19:12.878679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":856,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":110,"num_write":329,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"265'863","reported_seq":2490,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:33.948079+0000","last_change":"2024-05-30T15:54:18.649560+0000","last_active":"2024-05-30T17:12:33.948079+0000","last_peered":"2024-05-30T17:12:33.948079+0000","last_clean":"2024-05-30T17:12:33.948079+0000","last_became_active":"2024-05-30T15:54:18.649273+0000","last_became_peered":"2024-05-30T15:54:18.649273+0000","last_unstale":"2024-05-30T17:12:33.948079+0000","last_undegraded":"2024-05-30T17:12:33.948079+0000","last_fullsized":"2024-05-30T17:12:33.948079+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":800,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:31:47.570692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3918,"num_read_kb":3498,"num_write":1187,"num_write_kb":458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"265'202","reported_seq":862,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:44.661866+0000","last_change":"2024-05-30T15:54:16.649854+0000","last_active":"2024-05-30T17:09:44.661866+0000","last_peered":"2024-05-30T17:09:44.661866+0000","last_clean":"2024-05-30T17:09:44.661866+0000","last_became_active":"2024-05-30T15:54:16.649121+0000","last_became_peered":"2024-05-30T15:54:16.649121+0000","last_unstale":"2024-05-30T17:09:44.661866+0000","last_undegraded":"2024-05-30T17:09:44.661866+0000","last_fullsized":"2024-05-30T17:09:44.661866+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:25:59.798639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12165036,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":488,"num_read_kb":65907,"num_write":1115,"num_write_kb":54558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:01.528692+0000","last_change":"2024-05-30T15:54:20.668691+0000","last_active":"2024-05-30T17:09:01.528692+0000","last_peered":"2024-05-30T17:09:01.528692+0000","last_clean":"2024-05-30T17:09:01.528692+0000","last_became_active":"2024-05-30T15:54:20.667863+0000","last_became_peered":"2024-05-30T15:54:20.667863+0000","last_unstale":"2024-05-30T17:09:01.528692+0000","last_undegraded":"2024-05-30T17:09:01.528692+0000","last_fullsized":"2024-05-30T17:09:01.528692+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:31.629459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"265'139","reported_seq":724,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.570167+0000","last_change":"2024-05-30T15:54:22.665742+0000","last_active":"2024-05-30T17:11:26.570167+0000","last_peered":"2024-05-30T17:11:26.570167+0000","last_clean":"2024-05-30T17:11:26.570167+0000","last_became_active":"2024-05-30T15:54:22.664977+0000","last_became_peered":"2024-05-30T15:54:22.664977+0000","last_unstale":"2024-05-30T17:11:26.570167+0000","last_undegraded":"2024-05-30T17:11:26.570167+0000","last_fullsized":"2024-05-30T17:11:26.570167+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":100,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:20:57.960795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10584,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":178,"num_read_kb":239,"num_write":94,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":504,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:06.552717+0000","last_change":"2024-05-30T15:54:24.677137+0000","last_active":"2024-05-30T17:09:06.552717+0000","last_peered":"2024-05-30T17:09:06.552717+0000","last_clean":"2024-05-30T17:09:06.552717+0000","last_became_active":"2024-05-30T15:54:24.676707+0000","last_became_peered":"2024-05-30T15:54:24.676707+0000","last_unstale":"2024-05-30T17:09:06.552717+0000","last_undegraded":"2024-05-30T17:09:06.552717+0000","last_fullsized":"2024-05-30T17:09:06.552717+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:37:14.112070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"265'442","reported_seq":2099,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:28.309213+0000","last_change":"2024-05-30T15:54:18.671501+0000","last_active":"2024-05-30T17:12:28.309213+0000","last_peered":"2024-05-30T17:12:28.309213+0000","last_clean":"2024-05-30T17:12:28.309213+0000","last_became_active":"2024-05-30T15:54:18.670756+0000","last_became_peered":"2024-05-30T15:54:18.670756+0000","last_unstale":"2024-05-30T17:12:28.309213+0000","last_undegraded":"2024-05-30T17:12:28.309213+0000","last_fullsized":"2024-05-30T17:12:28.309213+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":400,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:50:30.677735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2383,"num_read_kb":2277,"num_write":522,"num_write_kb":97,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"265'154","reported_seq":758,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:22.227972+0000","last_change":"2024-05-30T15:54:16.654636+0000","last_active":"2024-05-30T17:10:22.227972+0000","last_peered":"2024-05-30T17:10:22.227972+0000","last_clean":"2024-05-30T17:10:22.227972+0000","last_became_active":"2024-05-30T15:54:16.653560+0000","last_became_peered":"2024-05-30T15:54:16.653560+0000","last_unstale":"2024-05-30T17:10:22.227972+0000","last_undegraded":"2024-05-30T17:10:22.227972+0000","last_fullsized":"2024-05-30T17:10:22.227972+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":100,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:47:43.349582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5814859,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":316,"num_read_kb":31907,"num_write":883,"num_write_kb":20958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":518,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:41.574743+0000","last_change":"2024-05-30T15:54:20.662992+0000","last_active":"2024-05-30T17:10:41.574743+0000","last_peered":"2024-05-30T17:10:41.574743+0000","last_clean":"2024-05-30T17:10:41.574743+0000","last_became_active":"2024-05-30T15:54:20.662669+0000","last_became_peered":"2024-05-30T15:54:20.662669+0000","last_unstale":"2024-05-30T17:10:41.574743+0000","last_undegraded":"2024-05-30T17:10:41.574743+0000","last_fullsized":"2024-05-30T17:10:41.574743+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:46:17.417038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"265'216","reported_seq":857,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.843270+0000","last_change":"2024-05-30T15:54:22.664232+0000","last_active":"2024-05-30T17:13:24.843270+0000","last_peered":"2024-05-30T17:13:24.843270+0000","last_clean":"2024-05-30T17:13:24.843270+0000","last_became_active":"2024-05-30T15:54:22.663518+0000","last_became_peered":"2024-05-30T15:54:22.663518+0000","last_unstale":"2024-05-30T17:13:24.843270+0000","last_undegraded":"2024-05-30T17:13:24.843270+0000","last_fullsized":"2024-05-30T17:13:24.843270+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":200,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:51:09.801047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18037,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":324,"num_read_kb":450,"num_write":214,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":497,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:27.127216+0000","last_change":"2024-05-30T15:54:24.683358+0000","last_active":"2024-05-30T17:09:27.127216+0000","last_peered":"2024-05-30T17:09:27.127216+0000","last_clean":"2024-05-30T17:09:27.127216+0000","last_became_active":"2024-05-30T15:54:24.682609+0000","last_became_peered":"2024-05-30T15:54:24.682609+0000","last_unstale":"2024-05-30T17:09:27.127216+0000","last_undegraded":"2024-05-30T17:09:27.127216+0000","last_fullsized":"2024-05-30T17:09:27.127216+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:15:33.566334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"265'149","reported_seq":660,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:12.359173+0000","last_change":"2024-05-30T15:54:26.695969+0000","last_active":"2024-05-30T17:11:12.359173+0000","last_peered":"2024-05-30T17:11:12.359173+0000","last_clean":"2024-05-30T17:11:12.359173+0000","last_became_active":"2024-05-30T15:54:26.695210+0000","last_became_peered":"2024-05-30T15:54:26.695210+0000","last_unstale":"2024-05-30T17:11:12.359173+0000","last_undegraded":"2024-05-30T17:11:12.359173+0000","last_fullsized":"2024-05-30T17:11:12.359173+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:13:43.303060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":643,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":167,"num_read_kb":179,"num_write":338,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"265'854","reported_seq":2475,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.655415+0000","last_change":"2024-05-30T15:54:18.641464+0000","last_active":"2024-05-30T17:12:44.655415+0000","last_peered":"2024-05-30T17:12:44.655415+0000","last_clean":"2024-05-30T17:12:44.655415+0000","last_became_active":"2024-05-30T15:54:18.640739+0000","last_became_peered":"2024-05-30T15:54:18.640739+0000","last_unstale":"2024-05-30T17:12:44.655415+0000","last_undegraded":"2024-05-30T17:12:44.655415+0000","last_fullsized":"2024-05-30T17:12:44.655415+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":800,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:05:20.428299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4053,"num_read_kb":3626,"num_write":1150,"num_write_kb":447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"265'241","reported_seq":974,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:28.400625+0000","last_change":"2024-05-30T15:54:54.425702+0000","last_active":"2024-05-30T17:09:28.400625+0000","last_peered":"2024-05-30T17:09:28.400625+0000","last_clean":"2024-05-30T17:09:28.400625+0000","last_became_active":"2024-05-30T15:54:54.424825+0000","last_became_peered":"2024-05-30T15:54:54.424825+0000","last_unstale":"2024-05-30T17:09:28.400625+0000","last_undegraded":"2024-05-30T17:09:28.400625+0000","last_fullsized":"2024-05-30T17:09:28.400625+0000","mapping_epoch":35,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":200,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:43:34.336982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11204870,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":883,"num_read_kb":12756,"num_write":1226,"num_write_kb":34615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"34'3","reported_seq":836,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.748812+0000","last_change":"2024-05-30T15:54:20.664818+0000","last_active":"2024-05-30T17:13:24.748812+0000","last_peered":"2024-05-30T17:13:24.748812+0000","last_clean":"2024-05-30T17:13:24.748812+0000","last_became_active":"2024-05-30T15:54:20.664095+0000","last_became_peered":"2024-05-30T15:54:20.664095+0000","last_unstale":"2024-05-30T17:13:24.748812+0000","last_undegraded":"2024-05-30T17:13:24.748812+0000","last_fullsized":"2024-05-30T17:13:24.748812+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:27:22.067783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":491,"num_read_kb":489,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"265'190","reported_seq":796,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:56.578238+0000","last_change":"2024-05-30T15:54:22.665708+0000","last_active":"2024-05-30T17:11:56.578238+0000","last_peered":"2024-05-30T17:11:56.578238+0000","last_clean":"2024-05-30T17:11:56.578238+0000","last_became_active":"2024-05-30T15:54:22.664997+0000","last_became_peered":"2024-05-30T15:54:22.664997+0000","last_unstale":"2024-05-30T17:11:56.578238+0000","last_undegraded":"2024-05-30T17:11:56.578238+0000","last_fullsized":"2024-05-30T17:11:56.578238+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":100,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:45:04.949797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28724,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":281,"num_read_kb":391,"num_write":188,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":498,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:26.767043+0000","last_change":"2024-05-30T15:54:24.678998+0000","last_active":"2024-05-30T17:09:26.767043+0000","last_peered":"2024-05-30T17:09:26.767043+0000","last_clean":"2024-05-30T17:09:26.767043+0000","last_became_active":"2024-05-30T15:54:24.675583+0000","last_became_peered":"2024-05-30T15:54:24.675583+0000","last_unstale":"2024-05-30T17:09:26.767043+0000","last_undegraded":"2024-05-30T17:09:26.767043+0000","last_fullsized":"2024-05-30T17:09:26.767043+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:17:21.969608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"265'140","reported_seq":655,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.560874+0000","last_change":"2024-05-30T15:54:26.703108+0000","last_active":"2024-05-30T17:11:26.560874+0000","last_peered":"2024-05-30T17:11:26.560874+0000","last_clean":"2024-05-30T17:11:26.560874+0000","last_became_active":"2024-05-30T15:54:26.700392+0000","last_became_peered":"2024-05-30T15:54:26.700392+0000","last_unstale":"2024-05-30T17:11:26.560874+0000","last_undegraded":"2024-05-30T17:11:26.560874+0000","last_fullsized":"2024-05-30T17:11:26.560874+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":100,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:25:14.903459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2627,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":177,"num_read_kb":174,"num_write":403,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"265'368","reported_seq":1738,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:33.884093+0000","last_change":"2024-05-30T15:54:54.420129+0000","last_active":"2024-05-30T17:12:33.884093+0000","last_peered":"2024-05-30T17:12:33.884093+0000","last_clean":"2024-05-30T17:12:33.884093+0000","last_became_active":"2024-05-30T15:54:54.419468+0000","last_became_peered":"2024-05-30T15:54:54.419468+0000","last_unstale":"2024-05-30T17:12:33.884093+0000","last_undegraded":"2024-05-30T17:12:33.884093+0000","last_fullsized":"2024-05-30T17:12:33.884093+0000","mapping_epoch":35,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":300,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:08:55.802490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1898,"num_read_kb":1782,"num_write":455,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"265'156","reported_seq":881,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:52.778205+0000","last_change":"2024-05-30T15:54:16.677283+0000","last_active":"2024-05-30T17:12:52.778205+0000","last_peered":"2024-05-30T17:12:52.778205+0000","last_clean":"2024-05-30T17:12:52.778205+0000","last_became_active":"2024-05-30T15:54:16.676440+0000","last_became_peered":"2024-05-30T15:54:16.676440+0000","last_unstale":"2024-05-30T17:12:52.778205+0000","last_undegraded":"2024-05-30T17:12:52.778205+0000","last_fullsized":"2024-05-30T17:12:52.778205+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":100,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:16:12.516128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9724052,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":866,"num_read_kb":45599,"num_write":922,"num_write_kb":34636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":514,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.723009+0000","last_change":"2024-05-30T15:54:20.656111+0000","last_active":"2024-05-30T17:09:21.723009+0000","last_peered":"2024-05-30T17:09:21.723009+0000","last_clean":"2024-05-30T17:09:21.723009+0000","last_became_active":"2024-05-30T15:54:20.655388+0000","last_became_peered":"2024-05-30T15:54:20.655388+0000","last_unstale":"2024-05-30T17:09:21.723009+0000","last_undegraded":"2024-05-30T17:09:21.723009+0000","last_fullsized":"2024-05-30T17:09:21.723009+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:07:28.254294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"265'161","reported_seq":760,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.837510+0000","last_change":"2024-05-30T15:54:22.849861+0000","last_active":"2024-05-30T17:13:24.837510+0000","last_peered":"2024-05-30T17:13:24.837510+0000","last_clean":"2024-05-30T17:13:24.837510+0000","last_became_active":"2024-05-30T15:54:22.849193+0000","last_became_peered":"2024-05-30T15:54:22.849193+0000","last_unstale":"2024-05-30T17:13:24.837510+0000","last_undegraded":"2024-05-30T17:13:24.837510+0000","last_fullsized":"2024-05-30T17:13:24.837510+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":100,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:51:38.667242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13488,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":212,"num_read_kb":283,"num_write":120,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:41.783470+0000","last_change":"2024-05-30T15:54:24.682047+0000","last_active":"2024-05-30T17:10:41.783470+0000","last_peered":"2024-05-30T17:10:41.783470+0000","last_clean":"2024-05-30T17:10:41.783470+0000","last_became_active":"2024-05-30T15:54:24.681609+0000","last_became_peered":"2024-05-30T15:54:24.681609+0000","last_unstale":"2024-05-30T17:10:41.783470+0000","last_undegraded":"2024-05-30T17:10:41.783470+0000","last_fullsized":"2024-05-30T17:10:41.783470+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:36:01.484941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"265'129","reported_seq":631,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.853790+0000","last_change":"2024-05-30T15:54:26.706376+0000","last_active":"2024-05-30T17:13:24.853790+0000","last_peered":"2024-05-30T17:13:24.853790+0000","last_clean":"2024-05-30T17:13:24.853790+0000","last_became_active":"2024-05-30T15:54:26.704842+0000","last_became_peered":"2024-05-30T15:54:26.704842+0000","last_unstale":"2024-05-30T17:13:24.853790+0000","last_undegraded":"2024-05-30T17:13:24.853790+0000","last_fullsized":"2024-05-30T17:13:24.853790+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":100,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:31:31.301239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":383,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":127,"num_read_kb":122,"num_write":322,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"265'753","reported_seq":2251,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:51.565345+0000","last_change":"2024-05-30T15:54:18.635089+0000","last_active":"2024-05-30T17:11:51.565345+0000","last_peered":"2024-05-30T17:11:51.565345+0000","last_clean":"2024-05-30T17:11:51.565345+0000","last_became_active":"2024-05-30T15:54:18.634803+0000","last_became_peered":"2024-05-30T15:54:18.634803+0000","last_unstale":"2024-05-30T17:11:51.565345+0000","last_undegraded":"2024-05-30T17:11:51.565345+0000","last_fullsized":"2024-05-30T17:11:51.565345+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":700,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:29:09.056772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":3476,"num_read_kb":3109,"num_write":1006,"num_write_kb":389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"265'186","reported_seq":810,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.543434+0000","last_change":"2024-05-30T15:54:54.431157+0000","last_active":"2024-05-30T17:09:56.543434+0000","last_peered":"2024-05-30T17:09:56.543434+0000","last_clean":"2024-05-30T17:09:56.543434+0000","last_became_active":"2024-05-30T15:54:54.430490+0000","last_became_peered":"2024-05-30T15:54:54.430490+0000","last_unstale":"2024-05-30T17:09:56.543434+0000","last_undegraded":"2024-05-30T17:09:56.543434+0000","last_fullsized":"2024-05-30T17:09:56.543434+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":100,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:47:17.028289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6643361,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":477,"num_read_kb":40323,"num_write":1071,"num_write_kb":31617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.765694+0000","last_change":"2024-05-30T15:54:20.651307+0000","last_active":"2024-05-30T17:09:21.765694+0000","last_peered":"2024-05-30T17:09:21.765694+0000","last_clean":"2024-05-30T17:09:21.765694+0000","last_became_active":"2024-05-30T15:54:20.650944+0000","last_became_peered":"2024-05-30T15:54:20.650944+0000","last_unstale":"2024-05-30T17:09:21.765694+0000","last_undegraded":"2024-05-30T17:09:21.765694+0000","last_fullsized":"2024-05-30T17:09:21.765694+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:44:01.827195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"265'113","reported_seq":671,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:41.740281+0000","last_change":"2024-05-30T15:54:22.851946+0000","last_active":"2024-05-30T17:10:41.740281+0000","last_peered":"2024-05-30T17:10:41.740281+0000","last_clean":"2024-05-30T17:10:41.740281+0000","last_became_active":"2024-05-30T15:54:22.850865+0000","last_became_peered":"2024-05-30T15:54:22.850865+0000","last_unstale":"2024-05-30T17:10:41.740281+0000","last_undegraded":"2024-05-30T17:10:41.740281+0000","last_fullsized":"2024-05-30T17:10:41.740281+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":100,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:49:42.844967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":127,"num_read_kb":121,"num_write":92,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"265'538","reported_seq":4166,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:28.422661+0000","last_change":"2024-05-30T15:54:24.681428+0000","last_active":"2024-05-30T17:13:28.422661+0000","last_peered":"2024-05-30T17:13:28.422661+0000","last_clean":"2024-05-30T17:13:28.422661+0000","last_became_active":"2024-05-30T15:54:24.680695+0000","last_became_peered":"2024-05-30T15:54:24.680695+0000","last_unstale":"2024-05-30T17:13:28.422661+0000","last_undegraded":"2024-05-30T17:13:28.422661+0000","last_fullsized":"2024-05-30T17:13:28.422661+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":500,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:53:20.536261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"265'134","reported_seq":641,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:35.623018+0000","last_change":"2024-05-30T15:54:26.696600+0000","last_active":"2024-05-30T17:11:35.623018+0000","last_peered":"2024-05-30T17:11:35.623018+0000","last_clean":"2024-05-30T17:11:35.623018+0000","last_became_active":"2024-05-30T15:54:26.695842+0000","last_became_peered":"2024-05-30T15:54:26.695842+0000","last_unstale":"2024-05-30T17:11:35.623018+0000","last_undegraded":"2024-05-30T17:11:35.623018+0000","last_fullsized":"2024-05-30T17:11:35.623018+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:01:21.719509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":418,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":133,"num_read_kb":127,"num_write":334,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"265'423","reported_seq":1942,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.652772+0000","last_change":"2024-05-30T15:54:18.661318+0000","last_active":"2024-05-30T17:13:20.652772+0000","last_peered":"2024-05-30T17:13:20.652772+0000","last_clean":"2024-05-30T17:13:20.652772+0000","last_became_active":"2024-05-30T15:54:18.660567+0000","last_became_peered":"2024-05-30T15:54:18.660567+0000","last_unstale":"2024-05-30T17:13:20.652772+0000","last_undegraded":"2024-05-30T17:13:20.652772+0000","last_fullsized":"2024-05-30T17:13:20.652772+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":400,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:24:20.096151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2226,"num_read_kb":2104,"num_write":504,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"265'303","reported_seq":994,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:17.124619+0000","last_change":"2024-05-30T15:54:16.686321+0000","last_active":"2024-05-30T17:09:17.124619+0000","last_peered":"2024-05-30T17:09:17.124619+0000","last_clean":"2024-05-30T17:09:17.124619+0000","last_became_active":"2024-05-30T15:54:16.685691+0000","last_became_peered":"2024-05-30T15:54:16.685691+0000","last_unstale":"2024-05-30T17:09:17.124619+0000","last_undegraded":"2024-05-30T17:09:17.124619+0000","last_fullsized":"2024-05-30T17:09:17.124619+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":200,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:37:07.773513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":379054,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":761,"num_read_kb":1764,"num_write":1250,"num_write_kb":18095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":505,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.754576+0000","last_change":"2024-05-30T15:54:20.670328+0000","last_active":"2024-05-30T17:09:21.754576+0000","last_peered":"2024-05-30T17:09:21.754576+0000","last_clean":"2024-05-30T17:09:21.754576+0000","last_became_active":"2024-05-30T15:54:20.669751+0000","last_became_peered":"2024-05-30T15:54:20.669751+0000","last_unstale":"2024-05-30T17:09:21.754576+0000","last_undegraded":"2024-05-30T17:09:21.754576+0000","last_fullsized":"2024-05-30T17:09:21.754576+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:06:46.261517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"265'213","reported_seq":820,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:52.160194+0000","last_change":"2024-05-30T15:54:22.672899+0000","last_active":"2024-05-30T17:11:52.160194+0000","last_peered":"2024-05-30T17:11:52.160194+0000","last_clean":"2024-05-30T17:11:52.160194+0000","last_became_active":"2024-05-30T15:54:22.672078+0000","last_became_peered":"2024-05-30T15:54:22.672078+0000","last_unstale":"2024-05-30T17:11:52.160194+0000","last_undegraded":"2024-05-30T17:11:52.160194+0000","last_fullsized":"2024-05-30T17:11:52.160194+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":200,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:23:16.149546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14727,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":288,"num_read_kb":379,"num_write":182,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":503,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.557376+0000","last_change":"2024-05-30T15:54:24.676069+0000","last_active":"2024-05-30T17:09:21.557376+0000","last_peered":"2024-05-30T17:09:21.557376+0000","last_clean":"2024-05-30T17:09:21.557376+0000","last_became_active":"2024-05-30T15:54:24.675467+0000","last_became_peered":"2024-05-30T15:54:24.675467+0000","last_unstale":"2024-05-30T17:09:21.557376+0000","last_undegraded":"2024-05-30T17:09:21.557376+0000","last_fullsized":"2024-05-30T17:09:21.557376+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:10.271248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"265'102","reported_seq":619,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:19.803711+0000","last_change":"2024-05-30T15:54:26.705986+0000","last_active":"2024-05-30T17:13:19.803711+0000","last_peered":"2024-05-30T17:13:19.803711+0000","last_clean":"2024-05-30T17:13:19.803711+0000","last_became_active":"2024-05-30T15:54:26.705264+0000","last_became_peered":"2024-05-30T15:54:26.705264+0000","last_unstale":"2024-05-30T17:13:19.803711+0000","last_undegraded":"2024-05-30T17:13:19.803711+0000","last_fullsized":"2024-05-30T17:13:19.803711+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:18:43.441191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":805,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":151,"num_read_kb":135,"num_write":247,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"265'698","reported_seq":2167,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.677934+0000","last_change":"2024-05-30T15:54:54.422775+0000","last_active":"2024-05-30T17:13:18.677934+0000","last_peered":"2024-05-30T17:13:18.677934+0000","last_clean":"2024-05-30T17:13:18.677934+0000","last_became_active":"2024-05-30T15:54:54.421604+0000","last_became_peered":"2024-05-30T15:54:54.421604+0000","last_unstale":"2024-05-30T17:13:18.677934+0000","last_undegraded":"2024-05-30T17:13:18.677934+0000","last_fullsized":"2024-05-30T17:13:18.677934+0000","mapping_epoch":35,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":600,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:32:11.130243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3204,"num_read_kb":2893,"num_write":920,"num_write_kb":324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"265'163","reported_seq":804,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:17.124667+0000","last_change":"2024-05-30T15:54:54.424699+0000","last_active":"2024-05-30T17:09:17.124667+0000","last_peered":"2024-05-30T17:09:17.124667+0000","last_clean":"2024-05-30T17:09:17.124667+0000","last_became_active":"2024-05-30T15:54:54.422916+0000","last_became_peered":"2024-05-30T15:54:54.422916+0000","last_unstale":"2024-05-30T17:09:17.124667+0000","last_undegraded":"2024-05-30T17:09:17.124667+0000","last_fullsized":"2024-05-30T17:09:17.124667+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":100,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:06:41.620026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15294568,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":395,"num_read_kb":71105,"num_write":966,"num_write_kb":25767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"34'4","reported_seq":829,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.708703+0000","last_change":"2024-05-30T15:54:20.652854+0000","last_active":"2024-05-30T17:13:24.708703+0000","last_peered":"2024-05-30T17:13:24.708703+0000","last_clean":"2024-05-30T17:13:24.708703+0000","last_became_active":"2024-05-30T15:54:20.652134+0000","last_became_peered":"2024-05-30T15:54:20.652134+0000","last_unstale":"2024-05-30T17:13:24.708703+0000","last_undegraded":"2024-05-30T17:13:24.708703+0000","last_fullsized":"2024-05-30T17:13:24.708703+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:12:04.911743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1139,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":490,"num_read_kb":802,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"265'147","reported_seq":736,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:46.590296+0000","last_change":"2024-05-30T15:54:22.673262+0000","last_active":"2024-05-30T17:11:46.590296+0000","last_peered":"2024-05-30T17:11:46.590296+0000","last_clean":"2024-05-30T17:11:46.590296+0000","last_became_active":"2024-05-30T15:54:22.672169+0000","last_became_peered":"2024-05-30T15:54:22.672169+0000","last_unstale":"2024-05-30T17:11:46.590296+0000","last_undegraded":"2024-05-30T17:11:46.590296+0000","last_fullsized":"2024-05-30T17:11:46.590296+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":100,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:04:32.418773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8883,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":183,"num_read_kb":244,"num_write":98,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":497,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:41.537447+0000","last_change":"2024-05-30T15:54:24.676446+0000","last_active":"2024-05-30T17:09:41.537447+0000","last_peered":"2024-05-30T17:09:41.537447+0000","last_clean":"2024-05-30T17:09:41.537447+0000","last_became_active":"2024-05-30T15:54:24.675683+0000","last_became_peered":"2024-05-30T15:54:24.675683+0000","last_unstale":"2024-05-30T17:09:41.537447+0000","last_undegraded":"2024-05-30T17:09:41.537447+0000","last_fullsized":"2024-05-30T17:09:41.537447+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:10:10.728748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"265'127","reported_seq":655,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:17.801147+0000","last_change":"2024-05-30T15:54:26.689830+0000","last_active":"2024-05-30T17:13:17.801147+0000","last_peered":"2024-05-30T17:13:17.801147+0000","last_clean":"2024-05-30T17:13:17.801147+0000","last_became_active":"2024-05-30T15:54:26.689049+0000","last_became_peered":"2024-05-30T15:54:26.689049+0000","last_unstale":"2024-05-30T17:13:17.801147+0000","last_undegraded":"2024-05-30T17:13:17.801147+0000","last_fullsized":"2024-05-30T17:13:17.801147+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:41:39.309741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":539,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":167,"num_read_kb":166,"num_write":309,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"265'652","reported_seq":2069,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.759093+0000","last_change":"2024-05-30T15:54:18.653246+0000","last_active":"2024-05-30T17:13:18.759093+0000","last_peered":"2024-05-30T17:13:18.759093+0000","last_clean":"2024-05-30T17:13:18.759093+0000","last_became_active":"2024-05-30T15:54:18.651778+0000","last_became_peered":"2024-05-30T15:54:18.651778+0000","last_unstale":"2024-05-30T17:13:18.759093+0000","last_undegraded":"2024-05-30T17:13:18.759093+0000","last_fullsized":"2024-05-30T17:13:18.759093+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":600,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:42:03.298813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3068,"num_read_kb":2749,"num_write":930,"num_write_kb":367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"265'151","reported_seq":765,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:36.577419+0000","last_change":"2024-05-30T15:54:16.656872+0000","last_active":"2024-05-30T17:10:36.577419+0000","last_peered":"2024-05-30T17:10:36.577419+0000","last_clean":"2024-05-30T17:10:36.577419+0000","last_became_active":"2024-05-30T15:54:16.656081+0000","last_became_peered":"2024-05-30T15:54:16.656081+0000","last_unstale":"2024-05-30T17:10:36.577419+0000","last_undegraded":"2024-05-30T17:10:36.577419+0000","last_fullsized":"2024-05-30T17:10:36.577419+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:25:32.809071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":369,"num_read_kb":8201,"num_write":842,"num_write_kb":22196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":515,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:16.721503+0000","last_change":"2024-05-30T15:54:20.661844+0000","last_active":"2024-05-30T17:09:16.721503+0000","last_peered":"2024-05-30T17:09:16.721503+0000","last_clean":"2024-05-30T17:09:16.721503+0000","last_became_active":"2024-05-30T15:54:20.661090+0000","last_became_peered":"2024-05-30T15:54:20.661090+0000","last_unstale":"2024-05-30T17:09:16.721503+0000","last_undegraded":"2024-05-30T17:09:16.721503+0000","last_fullsized":"2024-05-30T17:09:16.721503+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:39:52.559621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"265'112","reported_seq":680,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.784725+0000","last_change":"2024-05-30T15:54:22.670767+0000","last_active":"2024-05-30T17:13:18.784725+0000","last_peered":"2024-05-30T17:13:18.784725+0000","last_clean":"2024-05-30T17:13:18.784725+0000","last_became_active":"2024-05-30T15:54:22.669073+0000","last_became_peered":"2024-05-30T15:54:22.669073+0000","last_unstale":"2024-05-30T17:13:18.784725+0000","last_undegraded":"2024-05-30T17:13:18.784725+0000","last_fullsized":"2024-05-30T17:13:18.784725+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:43:33.730358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8403,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":171,"num_write":70,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:26.724184+0000","last_change":"2024-05-30T15:54:24.680707+0000","last_active":"2024-05-30T17:09:26.724184+0000","last_peered":"2024-05-30T17:09:26.724184+0000","last_clean":"2024-05-30T17:09:26.724184+0000","last_became_active":"2024-05-30T15:54:24.678292+0000","last_became_peered":"2024-05-30T15:54:24.678292+0000","last_unstale":"2024-05-30T17:09:26.724184+0000","last_undegraded":"2024-05-30T17:09:26.724184+0000","last_fullsized":"2024-05-30T17:09:26.724184+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:56:06.653666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"265'164","reported_seq":901,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:14.442195+0000","last_change":"2024-05-30T15:54:26.704198+0000","last_active":"2024-05-30T17:13:14.442195+0000","last_peered":"2024-05-30T17:13:14.442195+0000","last_clean":"2024-05-30T17:13:14.442195+0000","last_became_active":"2024-05-30T15:54:26.703440+0000","last_became_peered":"2024-05-30T15:54:26.703440+0000","last_unstale":"2024-05-30T17:13:14.442195+0000","last_undegraded":"2024-05-30T17:13:14.442195+0000","last_fullsized":"2024-05-30T17:13:14.442195+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":100,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:28:50.587698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1074,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":420,"num_read_kb":450,"num_write":403,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"265'327","reported_seq":1681,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:15.190426+0000","last_change":"2024-05-30T15:54:18.672371+0000","last_active":"2024-05-30T17:12:15.190426+0000","last_peered":"2024-05-30T17:12:15.190426+0000","last_clean":"2024-05-30T17:12:15.190426+0000","last_became_active":"2024-05-30T15:54:18.671650+0000","last_became_peered":"2024-05-30T15:54:18.671650+0000","last_unstale":"2024-05-30T17:12:15.190426+0000","last_undegraded":"2024-05-30T17:12:15.190426+0000","last_fullsized":"2024-05-30T17:12:15.190426+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":300,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:52:16.244837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1672,"num_read_kb":1605,"num_write":361,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"265'205","reported_seq":866,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:34.743973+0000","last_change":"2024-05-30T15:54:16.661703+0000","last_active":"2024-05-30T17:12:34.743973+0000","last_peered":"2024-05-30T17:12:34.743973+0000","last_clean":"2024-05-30T17:12:34.743973+0000","last_became_active":"2024-05-30T15:54:16.660393+0000","last_became_peered":"2024-05-30T15:54:16.660393+0000","last_unstale":"2024-05-30T17:12:34.743973+0000","last_undegraded":"2024-05-30T17:12:34.743973+0000","last_fullsized":"2024-05-30T17:12:34.743973+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":100,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:51:03.780593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4788806,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":518,"num_read_kb":9177,"num_write":1140,"num_write_kb":26248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"27'1","reported_seq":508,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:31.768150+0000","last_change":"2024-05-30T15:54:20.661745+0000","last_active":"2024-05-30T17:09:31.768150+0000","last_peered":"2024-05-30T17:09:31.768150+0000","last_clean":"2024-05-30T17:09:31.768150+0000","last_became_active":"2024-05-30T15:54:20.661311+0000","last_became_peered":"2024-05-30T15:54:20.661311+0000","last_unstale":"2024-05-30T17:09:31.768150+0000","last_undegraded":"2024-05-30T17:09:31.768150+0000","last_fullsized":"2024-05-30T17:09:31.768150+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:08:29.614757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"265'171","reported_seq":778,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:56.757203+0000","last_change":"2024-05-30T15:54:22.663985+0000","last_active":"2024-05-30T17:11:56.757203+0000","last_peered":"2024-05-30T17:11:56.757203+0000","last_clean":"2024-05-30T17:11:56.757203+0000","last_became_active":"2024-05-30T15:54:22.663231+0000","last_became_peered":"2024-05-30T15:54:22.663231+0000","last_unstale":"2024-05-30T17:11:56.757203+0000","last_undegraded":"2024-05-30T17:11:56.757203+0000","last_fullsized":"2024-05-30T17:11:56.757203+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:32:51.856936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9029,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":225,"num_read_kb":274,"num_write":130,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":502,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:26.558679+0000","last_change":"2024-05-30T15:54:24.676660+0000","last_active":"2024-05-30T17:09:26.558679+0000","last_peered":"2024-05-30T17:09:26.558679+0000","last_clean":"2024-05-30T17:09:26.558679+0000","last_became_active":"2024-05-30T15:54:24.673674+0000","last_became_peered":"2024-05-30T15:54:24.673674+0000","last_unstale":"2024-05-30T17:09:26.558679+0000","last_undegraded":"2024-05-30T17:09:26.558679+0000","last_fullsized":"2024-05-30T17:09:26.558679+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:37:07.948924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"265'111","reported_seq":616,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.723032+0000","last_change":"2024-05-30T15:54:26.678237+0000","last_active":"2024-05-30T17:09:21.723032+0000","last_peered":"2024-05-30T17:09:21.723032+0000","last_clean":"2024-05-30T17:09:21.723032+0000","last_became_active":"2024-05-30T15:54:26.677418+0000","last_became_peered":"2024-05-30T15:54:26.677418+0000","last_unstale":"2024-05-30T17:09:21.723032+0000","last_undegraded":"2024-05-30T17:09:21.723032+0000","last_fullsized":"2024-05-30T17:09:21.723032+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":100,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:04:15.881662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":435,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":67,"num_write":244,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"265'305","reported_seq":1708,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:29.784726+0000","last_change":"2024-05-30T15:54:18.671642+0000","last_active":"2024-05-30T17:11:29.784726+0000","last_peered":"2024-05-30T17:11:29.784726+0000","last_clean":"2024-05-30T17:11:29.784726+0000","last_became_active":"2024-05-30T15:54:18.671335+0000","last_became_peered":"2024-05-30T15:54:18.671335+0000","last_unstale":"2024-05-30T17:11:29.784726+0000","last_undegraded":"2024-05-30T17:11:29.784726+0000","last_fullsized":"2024-05-30T17:11:29.784726+0000","mapping_epoch":24,"log_start":"265'200","ondisk_log_start":"265'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:01:34.225524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1771,"num_read_kb":1705,"num_write":340,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"265'202","reported_seq":854,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:26.132685+0000","last_change":"2024-05-30T15:54:16.675257+0000","last_active":"2024-05-30T17:12:26.132685+0000","last_peered":"2024-05-30T17:12:26.132685+0000","last_clean":"2024-05-30T17:12:26.132685+0000","last_became_active":"2024-05-30T15:54:16.674678+0000","last_became_peered":"2024-05-30T15:54:16.674678+0000","last_unstale":"2024-05-30T17:12:26.132685+0000","last_undegraded":"2024-05-30T17:12:26.132685+0000","last_fullsized":"2024-05-30T17:12:26.132685+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:49:35.316396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5919937,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":489,"num_read_kb":10347,"num_write":1032,"num_write_kb":23027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"265'559","reported_seq":2275,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.621937+0000","last_change":"2024-05-30T15:54:18.635401+0000","last_active":"2024-05-30T17:13:20.621937+0000","last_peered":"2024-05-30T17:13:20.621937+0000","last_clean":"2024-05-30T17:13:20.621937+0000","last_became_active":"2024-05-30T15:54:18.635092+0000","last_became_peered":"2024-05-30T15:54:18.635092+0000","last_unstale":"2024-05-30T17:13:20.621937+0000","last_undegraded":"2024-05-30T17:13:20.621937+0000","last_fullsized":"2024-05-30T17:13:20.621937+0000","mapping_epoch":24,"log_start":"265'500","ondisk_log_start":"265'500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":500,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:59:22.401801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2950,"num_read_kb":2739,"num_write":700,"num_write_kb":188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"265'159","reported_seq":773,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.559861+0000","last_change":"2024-05-30T15:54:16.658618+0000","last_active":"2024-05-30T17:11:21.559861+0000","last_peered":"2024-05-30T17:11:21.559861+0000","last_clean":"2024-05-30T17:11:21.559861+0000","last_became_active":"2024-05-30T15:54:16.658183+0000","last_became_peered":"2024-05-30T15:54:16.658183+0000","last_unstale":"2024-05-30T17:11:21.559861+0000","last_undegraded":"2024-05-30T17:11:21.559861+0000","last_fullsized":"2024-05-30T17:11:21.559861+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:41:18.697341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7709997,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":302,"num_read_kb":6656,"num_write":920,"num_write_kb":27882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"265'625","reported_seq":2181,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:20.469750+0000","last_change":"2024-05-30T15:54:18.662889+0000","last_active":"2024-05-30T17:12:20.469750+0000","last_peered":"2024-05-30T17:12:20.469750+0000","last_clean":"2024-05-30T17:12:20.469750+0000","last_became_active":"2024-05-30T15:54:18.662159+0000","last_became_peered":"2024-05-30T15:54:18.662159+0000","last_unstale":"2024-05-30T17:12:20.469750+0000","last_undegraded":"2024-05-30T17:12:20.469750+0000","last_fullsized":"2024-05-30T17:12:20.469750+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":600,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:00:12.906772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3191,"num_read_kb":2926,"num_write":899,"num_write_kb":342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"265'205","reported_seq":858,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:22.153006+0000","last_change":"2024-05-30T15:54:16.670343+0000","last_active":"2024-05-30T17:11:22.153006+0000","last_peered":"2024-05-30T17:11:22.153006+0000","last_clean":"2024-05-30T17:11:22.153006+0000","last_became_active":"2024-05-30T15:54:16.652664+0000","last_became_peered":"2024-05-30T15:54:16.652664+0000","last_unstale":"2024-05-30T17:11:22.153006+0000","last_undegraded":"2024-05-30T17:11:22.153006+0000","last_fullsized":"2024-05-30T17:11:22.153006+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":100,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:16:45.470104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5823370,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":446,"num_read_kb":27404,"num_write":1151,"num_write_kb":29020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"265'1297","reported_seq":2932,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:08.892803+0000","last_change":"2024-05-30T15:54:18.643433+0000","last_active":"2024-05-30T17:13:08.892803+0000","last_peered":"2024-05-30T17:13:08.892803+0000","last_clean":"2024-05-30T17:13:08.892803+0000","last_became_active":"2024-05-30T15:54:18.642697+0000","last_became_peered":"2024-05-30T15:54:18.642697+0000","last_unstale":"2024-05-30T17:13:08.892803+0000","last_undegraded":"2024-05-30T17:13:08.892803+0000","last_fullsized":"2024-05-30T17:13:08.892803+0000","mapping_epoch":24,"log_start":"265'1200","ondisk_log_start":"265'1200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":1200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:46:33.063462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":5484,"num_read_kb":4752,"num_write":1832,"num_write_kb":807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"265'255","reported_seq":987,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:36.401960+0000","last_change":"2024-05-30T15:54:16.660154+0000","last_active":"2024-05-30T17:11:36.401960+0000","last_peered":"2024-05-30T17:11:36.401960+0000","last_clean":"2024-05-30T17:11:36.401960+0000","last_became_active":"2024-05-30T15:54:16.659589+0000","last_became_peered":"2024-05-30T15:54:16.659589+0000","last_unstale":"2024-05-30T17:11:36.401960+0000","last_undegraded":"2024-05-30T17:11:36.401960+0000","last_fullsized":"2024-05-30T17:11:36.401960+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:55:43.508632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3249994,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":792,"num_read_kb":47588,"num_write":1563,"num_write_kb":36318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"265'1101","reported_seq":2680,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:20.420887+0000","last_change":"2024-05-30T15:54:18.653159+0000","last_active":"2024-05-30T17:12:20.420887+0000","last_peered":"2024-05-30T17:12:20.420887+0000","last_clean":"2024-05-30T17:12:20.420887+0000","last_became_active":"2024-05-30T15:54:18.652794+0000","last_became_peered":"2024-05-30T15:54:18.652794+0000","last_unstale":"2024-05-30T17:12:20.420887+0000","last_undegraded":"2024-05-30T17:12:20.420887+0000","last_fullsized":"2024-05-30T17:12:20.420887+0000","mapping_epoch":24,"log_start":"265'1000","ondisk_log_start":"265'1000","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":1000,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:14:33.580719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4733,"num_read_kb":4179,"num_write":1497,"num_write_kb":596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"265'223","reported_seq":891,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:06.570629+0000","last_change":"2024-05-30T15:54:54.452063+0000","last_active":"2024-05-30T17:10:06.570629+0000","last_peered":"2024-05-30T17:10:06.570629+0000","last_clean":"2024-05-30T17:10:06.570629+0000","last_became_active":"2024-05-30T15:54:54.451265+0000","last_became_peered":"2024-05-30T15:54:54.451265+0000","last_unstale":"2024-05-30T17:10:06.570629+0000","last_undegraded":"2024-05-30T17:10:06.570629+0000","last_fullsized":"2024-05-30T17:10:06.570629+0000","mapping_epoch":35,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":200,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:41:58.936344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":538764,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":478,"num_read_kb":2675,"num_write":1163,"num_write_kb":43073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"265'385","reported_seq":1948,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:33.943140+0000","last_change":"2024-05-30T15:54:18.671626+0000","last_active":"2024-05-30T17:12:33.943140+0000","last_peered":"2024-05-30T17:12:33.943140+0000","last_clean":"2024-05-30T17:12:33.943140+0000","last_became_active":"2024-05-30T15:54:18.670801+0000","last_became_peered":"2024-05-30T15:54:18.670801+0000","last_unstale":"2024-05-30T17:12:33.943140+0000","last_undegraded":"2024-05-30T17:12:33.943140+0000","last_fullsized":"2024-05-30T17:12:33.943140+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":300,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:08:04.360448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2215,"num_read_kb":2115,"num_write":443,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"265'219","reported_seq":870,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:20.785441+0000","last_change":"2024-05-30T15:54:16.651535+0000","last_active":"2024-05-30T17:11:20.785441+0000","last_peered":"2024-05-30T17:11:20.785441+0000","last_clean":"2024-05-30T17:11:20.785441+0000","last_became_active":"2024-05-30T15:54:16.649854+0000","last_became_peered":"2024-05-30T15:54:16.649854+0000","last_unstale":"2024-05-30T17:11:20.785441+0000","last_undegraded":"2024-05-30T17:11:20.785441+0000","last_fullsized":"2024-05-30T17:11:20.785441+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":200,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:56:05.559917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":635176,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":496,"num_read_kb":6249,"num_write":1148,"num_write_kb":30962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"265'405","reported_seq":1925,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:12.349529+0000","last_change":"2024-05-30T15:54:18.653103+0000","last_active":"2024-05-30T17:13:12.349529+0000","last_peered":"2024-05-30T17:13:12.349529+0000","last_clean":"2024-05-30T17:13:12.349529+0000","last_became_active":"2024-05-30T15:54:18.650832+0000","last_became_peered":"2024-05-30T15:54:18.650832+0000","last_unstale":"2024-05-30T17:13:12.349529+0000","last_undegraded":"2024-05-30T17:13:12.349529+0000","last_fullsized":"2024-05-30T17:13:12.349529+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":300,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:14:11.681719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2240,"num_read_kb":2125,"num_write":507,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"265'185","reported_seq":809,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:06.543039+0000","last_change":"2024-05-30T15:54:16.685235+0000","last_active":"2024-05-30T17:10:06.543039+0000","last_peered":"2024-05-30T17:10:06.543039+0000","last_clean":"2024-05-30T17:10:06.543039+0000","last_became_active":"2024-05-30T15:54:16.684670+0000","last_became_peered":"2024-05-30T15:54:16.684670+0000","last_unstale":"2024-05-30T17:10:06.543039+0000","last_undegraded":"2024-05-30T17:10:06.543039+0000","last_fullsized":"2024-05-30T17:10:06.543039+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:27.693225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5498449,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":371,"num_read_kb":34719,"num_write":1088,"num_write_kb":29777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"265'818","reported_seq":2410,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.705414+0000","last_change":"2024-05-30T15:54:18.669886+0000","last_active":"2024-05-30T17:13:20.705414+0000","last_peered":"2024-05-30T17:13:20.705414+0000","last_clean":"2024-05-30T17:13:20.705414+0000","last_became_active":"2024-05-30T15:54:18.669105+0000","last_became_peered":"2024-05-30T15:54:18.669105+0000","last_unstale":"2024-05-30T17:13:20.705414+0000","last_undegraded":"2024-05-30T17:13:20.705414+0000","last_fullsized":"2024-05-30T17:13:20.705414+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":800,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:33:49.110943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3660,"num_read_kb":3297,"num_write":1114,"num_write_kb":434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"265'218","reported_seq":916,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.337981+0000","last_change":"2024-05-30T15:54:54.417648+0000","last_active":"2024-05-30T17:11:11.337981+0000","last_peered":"2024-05-30T17:11:11.337981+0000","last_clean":"2024-05-30T17:11:11.337981+0000","last_became_active":"2024-05-30T15:54:54.416482+0000","last_became_peered":"2024-05-30T15:54:54.416482+0000","last_unstale":"2024-05-30T17:11:11.337981+0000","last_undegraded":"2024-05-30T17:11:11.337981+0000","last_fullsized":"2024-05-30T17:11:11.337981+0000","mapping_epoch":35,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":200,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:41:15.267706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5736484,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":727,"num_read_kb":137610,"num_write":1203,"num_write_kb":64841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2f","version":"265'385","reported_seq":1972,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:55.924733+0000","last_change":"2024-05-30T15:54:18.657515+0000","last_active":"2024-05-30T17:11:55.924733+0000","last_peered":"2024-05-30T17:11:55.924733+0000","last_clean":"2024-05-30T17:11:55.924733+0000","last_became_active":"2024-05-30T15:54:18.657236+0000","last_became_peered":"2024-05-30T15:54:18.657236+0000","last_unstale":"2024-05-30T17:11:55.924733+0000","last_undegraded":"2024-05-30T17:11:55.924733+0000","last_fullsized":"2024-05-30T17:11:55.924733+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":300,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:04:47.974078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2258,"num_read_kb":2156,"num_write":442,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"265'201","reported_seq":880,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:12.137218+0000","last_change":"2024-05-30T15:54:16.661242+0000","last_active":"2024-05-30T17:10:12.137218+0000","last_peered":"2024-05-30T17:10:12.137218+0000","last_clean":"2024-05-30T17:10:12.137218+0000","last_became_active":"2024-05-30T15:54:16.653037+0000","last_became_peered":"2024-05-30T15:54:16.653037+0000","last_unstale":"2024-05-30T17:10:12.137218+0000","last_undegraded":"2024-05-30T17:10:12.137218+0000","last_fullsized":"2024-05-30T17:10:12.137218+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":100,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:51:51.448976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2129592,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":541,"num_read_kb":29183,"num_write":975,"num_write_kb":32330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"265'330","reported_seq":1827,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:20.410822+0000","last_change":"2024-05-30T15:54:18.636049+0000","last_active":"2024-05-30T17:12:20.410822+0000","last_peered":"2024-05-30T17:12:20.410822+0000","last_clean":"2024-05-30T17:12:20.410822+0000","last_became_active":"2024-05-30T15:54:18.635382+0000","last_became_peered":"2024-05-30T15:54:18.635382+0000","last_unstale":"2024-05-30T17:12:20.410822+0000","last_undegraded":"2024-05-30T17:12:20.410822+0000","last_fullsized":"2024-05-30T17:12:20.410822+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":300,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:50:22.797101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2071,"num_read_kb":1981,"num_write":382,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"265'155","reported_seq":772,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.579598+0000","last_change":"2024-05-30T15:54:16.683072+0000","last_active":"2024-05-30T17:10:46.579598+0000","last_peered":"2024-05-30T17:10:46.579598+0000","last_clean":"2024-05-30T17:10:46.579598+0000","last_became_active":"2024-05-30T15:54:16.682629+0000","last_became_peered":"2024-05-30T15:54:16.682629+0000","last_unstale":"2024-05-30T17:10:46.579598+0000","last_undegraded":"2024-05-30T17:10:46.579598+0000","last_fullsized":"2024-05-30T17:10:46.579598+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:49:01.501502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7203749,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":304,"num_read_kb":22431,"num_write":818,"num_write_kb":23734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"265'743","reported_seq":2312,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.707890+0000","last_change":"2024-05-30T15:54:18.669167+0000","last_active":"2024-05-30T17:13:20.707890+0000","last_peered":"2024-05-30T17:13:20.707890+0000","last_clean":"2024-05-30T17:13:20.707890+0000","last_became_active":"2024-05-30T15:54:18.668748+0000","last_became_peered":"2024-05-30T15:54:18.668748+0000","last_unstale":"2024-05-30T17:13:20.707890+0000","last_undegraded":"2024-05-30T17:13:20.707890+0000","last_fullsized":"2024-05-30T17:13:20.707890+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":700,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:32:48.662121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3385,"num_read_kb":3066,"num_write":980,"num_write_kb":340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"265'161","reported_seq":896,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:58.685815+0000","last_change":"2024-05-30T15:54:16.676031+0000","last_active":"2024-05-30T17:12:58.685815+0000","last_peered":"2024-05-30T17:12:58.685815+0000","last_clean":"2024-05-30T17:12:58.685815+0000","last_became_active":"2024-05-30T15:54:16.674993+0000","last_became_peered":"2024-05-30T15:54:16.674993+0000","last_unstale":"2024-05-30T17:12:58.685815+0000","last_undegraded":"2024-05-30T17:12:58.685815+0000","last_fullsized":"2024-05-30T17:12:58.685815+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":100,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:09:30.788276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11799386,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":563,"num_read_kb":112844,"num_write":921,"num_write_kb":45704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"265'471","reported_seq":2112,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.654074+0000","last_change":"2024-05-30T15:54:18.672327+0000","last_active":"2024-05-30T17:12:44.654074+0000","last_peered":"2024-05-30T17:12:44.654074+0000","last_clean":"2024-05-30T17:12:44.654074+0000","last_became_active":"2024-05-30T15:54:18.671434+0000","last_became_peered":"2024-05-30T15:54:18.671434+0000","last_unstale":"2024-05-30T17:12:44.654074+0000","last_undegraded":"2024-05-30T17:12:44.654074+0000","last_fullsized":"2024-05-30T17:12:44.654074+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":400,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:07:26.503748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2525,"num_read_kb":2390,"num_write":596,"num_write_kb":183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"265'200","reported_seq":854,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.575766+0000","last_change":"2024-05-30T15:54:16.630459+0000","last_active":"2024-05-30T17:10:46.575766+0000","last_peered":"2024-05-30T17:10:46.575766+0000","last_clean":"2024-05-30T17:10:46.575766+0000","last_became_active":"2024-05-30T15:54:16.629843+0000","last_became_peered":"2024-05-30T15:54:16.629843+0000","last_unstale":"2024-05-30T17:10:46.575766+0000","last_undegraded":"2024-05-30T17:10:46.575766+0000","last_fullsized":"2024-05-30T17:10:46.575766+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":100,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:56:59.908915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":905553,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":430,"num_read_kb":42078,"num_write":1148,"num_write_kb":26102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"265'509","reported_seq":2134,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.660255+0000","last_change":"2024-05-30T15:54:18.660822+0000","last_active":"2024-05-30T17:12:44.660255+0000","last_peered":"2024-05-30T17:12:44.660255+0000","last_clean":"2024-05-30T17:12:44.660255+0000","last_became_active":"2024-05-30T15:54:18.660101+0000","last_became_peered":"2024-05-30T15:54:18.660101+0000","last_unstale":"2024-05-30T17:12:44.660255+0000","last_undegraded":"2024-05-30T17:12:44.660255+0000","last_fullsized":"2024-05-30T17:12:44.660255+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":400,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:23:12.134888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2760,"num_read_kb":2546,"num_write":661,"num_write_kb":201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"265'203","reported_seq":838,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.551832+0000","last_change":"2024-05-30T15:54:16.686381+0000","last_active":"2024-05-30T17:10:46.551832+0000","last_peered":"2024-05-30T17:10:46.551832+0000","last_clean":"2024-05-30T17:10:46.551832+0000","last_became_active":"2024-05-30T15:54:16.685969+0000","last_became_peered":"2024-05-30T15:54:16.685969+0000","last_unstale":"2024-05-30T17:10:46.551832+0000","last_undegraded":"2024-05-30T17:10:46.551832+0000","last_fullsized":"2024-05-30T17:10:46.551832+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":100,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:32:55.261508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18487302,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":412,"num_read_kb":18996,"num_write":1141,"num_write_kb":34757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"265'670","reported_seq":2427,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:55.980437+0000","last_change":"2024-05-30T15:54:18.646120+0000","last_active":"2024-05-30T17:11:55.980437+0000","last_peered":"2024-05-30T17:11:55.980437+0000","last_clean":"2024-05-30T17:11:55.980437+0000","last_became_active":"2024-05-30T15:54:18.645400+0000","last_became_peered":"2024-05-30T15:54:18.645400+0000","last_unstale":"2024-05-30T17:11:55.980437+0000","last_undegraded":"2024-05-30T17:11:55.980437+0000","last_fullsized":"2024-05-30T17:11:55.980437+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":600,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:37:15.286662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3447,"num_read_kb":3120,"num_write":968,"num_write_kb":434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"265'206","reported_seq":840,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:06.565221+0000","last_change":"2024-05-30T15:54:16.664235+0000","last_active":"2024-05-30T17:11:06.565221+0000","last_peered":"2024-05-30T17:11:06.565221+0000","last_clean":"2024-05-30T17:11:06.565221+0000","last_became_active":"2024-05-30T15:54:16.663255+0000","last_became_peered":"2024-05-30T15:54:16.663255+0000","last_unstale":"2024-05-30T17:11:06.565221+0000","last_undegraded":"2024-05-30T17:11:06.565221+0000","last_fullsized":"2024-05-30T17:11:06.565221+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":100,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:14:20.452277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11362066,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":420,"num_read_kb":27117,"num_write":1157,"num_write_kb":19484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"265'360","reported_seq":1779,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:08.893650+0000","last_change":"2024-05-30T15:54:18.658517+0000","last_active":"2024-05-30T17:13:08.893650+0000","last_peered":"2024-05-30T17:13:08.893650+0000","last_clean":"2024-05-30T17:13:08.893650+0000","last_became_active":"2024-05-30T15:54:18.658230+0000","last_became_peered":"2024-05-30T15:54:18.658230+0000","last_unstale":"2024-05-30T17:13:08.893650+0000","last_undegraded":"2024-05-30T17:13:08.893650+0000","last_fullsized":"2024-05-30T17:13:08.893650+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":300,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:13:42.767006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1934,"num_read_kb":1839,"num_write":414,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"265'166","reported_seq":779,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.558787+0000","last_change":"2024-05-30T15:54:16.655014+0000","last_active":"2024-05-30T17:11:16.558787+0000","last_peered":"2024-05-30T17:11:16.558787+0000","last_clean":"2024-05-30T17:11:16.558787+0000","last_became_active":"2024-05-30T15:54:16.654365+0000","last_became_peered":"2024-05-30T15:54:16.654365+0000","last_unstale":"2024-05-30T17:11:16.558787+0000","last_undegraded":"2024-05-30T17:11:16.558787+0000","last_fullsized":"2024-05-30T17:11:16.558787+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":100,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:31:10.427380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14702715,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":354,"num_read_kb":31154,"num_write":911,"num_write_kb":41557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"265'698","reported_seq":2128,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.605807+0000","last_change":"2024-05-30T15:54:18.661710+0000","last_active":"2024-05-30T17:12:44.605807+0000","last_peered":"2024-05-30T17:12:44.605807+0000","last_clean":"2024-05-30T17:12:44.605807+0000","last_became_active":"2024-05-30T15:54:18.660543+0000","last_became_peered":"2024-05-30T15:54:18.660543+0000","last_unstale":"2024-05-30T17:12:44.605807+0000","last_undegraded":"2024-05-30T17:12:44.605807+0000","last_fullsized":"2024-05-30T17:12:44.605807+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":600,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:39:46.092642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3130,"num_read_kb":2812,"num_write":919,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"265'195","reported_seq":837,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:17.152049+0000","last_change":"2024-05-30T15:54:16.666386+0000","last_active":"2024-05-30T17:11:17.152049+0000","last_peered":"2024-05-30T17:11:17.152049+0000","last_clean":"2024-05-30T17:11:17.152049+0000","last_became_active":"2024-05-30T15:54:16.661983+0000","last_became_peered":"2024-05-30T15:54:16.661983+0000","last_unstale":"2024-05-30T17:11:17.152049+0000","last_undegraded":"2024-05-30T17:11:17.152049+0000","last_fullsized":"2024-05-30T17:11:17.152049+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":100,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:48:35.955662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8616050,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":413,"num_read_kb":7620,"num_write":1148,"num_write_kb":9531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"265'152","reported_seq":782,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:10.550602+0000","last_change":"2024-05-30T15:55:54.768358+0000","last_active":"2024-05-30T17:13:10.550602+0000","last_peered":"2024-05-30T17:13:10.550602+0000","last_clean":"2024-05-30T17:13:10.550602+0000","last_became_active":"2024-05-30T15:55:54.767665+0000","last_became_peered":"2024-05-30T15:55:54.767665+0000","last_unstale":"2024-05-30T17:13:10.550602+0000","last_undegraded":"2024-05-30T17:13:10.550602+0000","last_fullsized":"2024-05-30T17:13:10.550602+0000","mapping_epoch":37,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":100,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:31:33.212759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11250466,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":358,"num_read_kb":14674,"num_write":888,"num_write_kb":17371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,0],"acting":[5,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"265'765","reported_seq":2419,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:12.341733+0000","last_change":"2024-05-30T15:54:18.660816+0000","last_active":"2024-05-30T17:11:12.341733+0000","last_peered":"2024-05-30T17:11:12.341733+0000","last_clean":"2024-05-30T17:11:12.341733+0000","last_became_active":"2024-05-30T15:54:18.660501+0000","last_became_peered":"2024-05-30T15:54:18.660501+0000","last_unstale":"2024-05-30T17:11:12.341733+0000","last_undegraded":"2024-05-30T17:11:12.341733+0000","last_fullsized":"2024-05-30T17:11:12.341733+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":700,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:58:48.235676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3837,"num_read_kb":3483,"num_write":1021,"num_write_kb":383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"265'179","reported_seq":830,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.790863+0000","last_change":"2024-05-30T15:54:16.668650+0000","last_active":"2024-05-30T17:11:16.790863+0000","last_peered":"2024-05-30T17:11:16.790863+0000","last_clean":"2024-05-30T17:11:16.790863+0000","last_became_active":"2024-05-30T15:54:16.668178+0000","last_became_peered":"2024-05-30T15:54:16.668178+0000","last_unstale":"2024-05-30T17:11:16.790863+0000","last_undegraded":"2024-05-30T17:11:16.790863+0000","last_fullsized":"2024-05-30T17:11:16.790863+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":100,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:22:19.325731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14464063,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":423,"num_read_kb":57498,"num_write":996,"num_write_kb":21705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"265'474","reported_seq":2144,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.703669+0000","last_change":"2024-05-30T15:54:18.656743+0000","last_active":"2024-05-30T17:13:20.703669+0000","last_peered":"2024-05-30T17:13:20.703669+0000","last_clean":"2024-05-30T17:13:20.703669+0000","last_became_active":"2024-05-30T15:54:18.655872+0000","last_became_peered":"2024-05-30T15:54:18.655872+0000","last_unstale":"2024-05-30T17:13:20.703669+0000","last_undegraded":"2024-05-30T17:13:20.703669+0000","last_fullsized":"2024-05-30T17:13:20.703669+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":400,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:51:05.525906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2571,"num_read_kb":2440,"num_write":587,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5999,"num_read_kb":4309,"num_write":2135,"num_write_kb":880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1140,"ondisk_log_size":1140,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19930,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":13226,"num_read_kb":13032,"num_write":15987,"num_write_kb":4690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":704512,"data_stored":39860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1567,"ondisk_log_size":1567,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":405,"ondisk_log_size":405,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":366949,"num_objects":197,"num_object_clones":0,"num_object_copies":394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":197,"num_whiteouts":0,"num_read":9271,"num_read_kb":11128,"num_write":6557,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":745472,"data_stored":181518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1889,"ondisk_log_size":1889,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1624,"num_read_kb":1934,"num_write":20,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":242,"num_object_clones":0,"num_object_copies":484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":242,"num_whiteouts":0,"num_read":191181,"num_read_kb":174247,"num_write":51900,"num_write_kb":17923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":242,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4116,"ondisk_log_size":4116,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":506410238,"num_objects":484,"num_object_clones":0,"num_object_copies":1452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":484,"num_whiteouts":0,"num_read":31813,"num_read_kb":2124772,"num_write":69681,"num_write_kb":2069422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":762998784,"data_stored":762998784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4465,"ondisk_log_size":4465,"up":192,"acting":192,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":81623142,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":117,"num_read_kb":182367,"num_write":133,"num_write_kb":122722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163258368,"data_stored":163246284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":81623142,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":115,"num_read_kb":181098,"num_write":133,"num_write_kb":122722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163258368,"data_stored":163246284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575810,"num_pgs":106,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":391144,"kb_used_data":129504,"kb_used_omap":1,"kb_used_meta":261630,"kb_avail":93980696,"statfs":{"total":96636764160,"available":96236232704,"internally_reserved":0,"allocated":132612096,"data_stored":130525109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":267909578},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575809,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395632,"kb_used_data":147880,"kb_used_omap":1,"kb_used_meta":247742,"kb_avail":93976208,"statfs":{"total":96636764160,"available":96231636992,"internally_reserved":0,"allocated":151429120,"data_stored":149446612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":253688262},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575809,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":220408,"kb_used_data":135088,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94151432,"statfs":{"total":96636764160,"available":96411066368,"internally_reserved":0,"allocated":138330112,"data_stored":136241754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87357893},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575809,"num_pgs":104,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":225900,"kb_used_data":132260,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94145940,"statfs":{"total":96636764160,"available":96405442560,"internally_reserved":0,"allocated":135434240,"data_stored":133486846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":95877574},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575807,"num_pgs":124,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":376304,"kb_used_data":150760,"kb_used_omap":1,"kb_used_meta":225534,"kb_avail":93995536,"statfs":{"total":96636764160,"available":96251428864,"internally_reserved":0,"allocated":154378240,"data_stored":152309513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":230947271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575807,"num_pgs":91,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":179456,"kb_used_data":108024,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94192384,"statfs":{"total":96636764160,"available":96453001216,"internally_reserved":0,"allocated":110616576,"data_stored":108657566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575811,"num_pgs":112,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":263376,"kb_used_data":146184,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94108464,"statfs":{"total":96636764160,"available":96367067136,"internally_reserved":0,"allocated":149692416,"data_stored":147687855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575806,"num_pgs":109,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":379024,"kb_used_data":149320,"kb_used_omap":1,"kb_used_meta":229694,"kb_avail":93992816,"statfs":{"total":96636764160,"available":96248643584,"internally_reserved":0,"allocated":152903680,"data_stored":150873069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":235207109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":114364416,"data_stored":114364416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":99225600,"data_stored":99225600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":74932224,"data_stored":74932224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":93523968,"data_stored":93523968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":97972224,"data_stored":97972224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":92889088,"data_stored":92889088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":109338624,"data_stored":109338624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":80752640,"data_stored":80752640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":18103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":28335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":7835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":30127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":10287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":151552,"data_stored":39861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":15040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":31930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":4787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":5121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":4696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":6691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":5459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":3918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":7193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32555008,"data_stored":32554271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22020096,"data_stored":22020096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18591744,"data_stored":18588432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":23322624,"data_stored":23318684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16900096,"data_stored":16897366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":21557248,"data_stored":21555883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7340032,"data_stored":7340032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20971520,"data_stored":20971520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24117248,"data_stored":24117248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12836864,"data_stored":12832924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":33140736,"data_stored":33138634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16314368,"data_stored":16313003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":19460096,"data_stored":19458731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":30457856,"data_stored":30457119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":25882624,"data_stored":25880049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T17:13:38.937 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T17:13:39.316 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:39.330 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-05-30T17:13:39.349 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":2645,"stamp":"2024-05-30T17:13:38.440189+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":670635562,"num_objects":1096,"num_object_clones":0,"num_object_copies":2676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1096,"num_whiteouts":0,"num_read":253424,"num_read_kb":2692954,"num_write":146665,"num_write_kb":2342828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":289,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13825,"ondisk_log_size":13825,"up":850,"acting":850,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":850,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2431244,"kb_used_data":1099020,"kb_used_omap":12,"kb_used_meta":1332147,"kb_avail":752543476,"statfs":{"total":773094113280,"available":770604519424,"internally_reserved":0,"allocated":1125396480,"data_stored":1109228324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":1364119089},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-520,"num_objects":-4,"num_object_clones":0,"num_object_copies":-8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4,"num_whiteouts":0,"num_read":23,"num_read_kb":25,"num_write":15,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007603"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":27,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:57.133920+0000","last_change":"2024-05-30T16:34:55.869930+0000","last_active":"2024-05-30T17:09:57.133920+0000","last_peered":"2024-05-30T17:09:57.133920+0000","last_clean":"2024-05-30T17:09:57.133920+0000","last_became_active":"2024-05-30T16:34:55.869226+0000","last_became_peered":"2024-05-30T16:34:55.869226+0000","last_unstale":"2024-05-30T17:09:57.133920+0000","last_undegraded":"2024-05-30T17:09:57.133920+0000","last_fullsized":"2024-05-30T17:09:57.133920+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:37:02.394930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"265'1","reported_seq":47,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:07.136134+0000","last_change":"2024-05-30T16:17:35.249630+0000","last_active":"2024-05-30T17:10:07.136134+0000","last_peered":"2024-05-30T17:10:07.136134+0000","last_clean":"2024-05-30T17:10:07.136134+0000","last_became_active":"2024-05-30T16:17:35.248740+0000","last_became_peered":"2024-05-30T16:17:35.248740+0000","last_unstale":"2024-05-30T17:10:07.136134+0000","last_undegraded":"2024-05-30T17:10:07.136134+0000","last_fullsized":"2024-05-30T17:10:07.136134+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:05:10.176331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":6144,"num_write":1,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"265'7","reported_seq":65,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:01.569510+0000","last_change":"2024-05-30T16:17:31.184110+0000","last_active":"2024-05-30T17:10:01.569510+0000","last_peered":"2024-05-30T17:10:01.569510+0000","last_clean":"2024-05-30T17:10:01.569510+0000","last_became_active":"2024-05-30T16:17:31.183349+0000","last_became_peered":"2024-05-30T16:17:31.183349+0000","last_unstale":"2024-05-30T17:10:01.569510+0000","last_undegraded":"2024-05-30T17:10:01.569510+0000","last_fullsized":"2024-05-30T17:10:01.569510+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:06:40.372126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":16384,"num_write":7,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"265'45","reported_seq":184,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.581199+0000","last_change":"2024-05-30T16:12:50.806342+0000","last_active":"2024-05-30T17:11:11.581199+0000","last_peered":"2024-05-30T17:11:11.581199+0000","last_clean":"2024-05-30T17:11:11.581199+0000","last_became_active":"2024-05-30T16:12:50.803352+0000","last_became_peered":"2024-05-30T16:12:50.803352+0000","last_unstale":"2024-05-30T17:11:11.581199+0000","last_undegraded":"2024-05-30T17:11:11.581199+0000","last_fullsized":"2024-05-30T17:11:11.581199+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:23:40.247207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":225,"num_read_kb":156,"num_write":101,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":27,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.543407+0000","last_change":"2024-05-30T16:34:55.887503+0000","last_active":"2024-05-30T17:09:56.543407+0000","last_peered":"2024-05-30T17:09:56.543407+0000","last_clean":"2024-05-30T17:09:56.543407+0000","last_became_active":"2024-05-30T16:34:55.886731+0000","last_became_peered":"2024-05-30T16:34:55.886731+0000","last_unstale":"2024-05-30T17:09:56.543407+0000","last_undegraded":"2024-05-30T17:09:56.543407+0000","last_fullsized":"2024-05-30T17:09:56.543407+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:47:35.283261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"0'0","reported_seq":44,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:06.732701+0000","last_change":"2024-05-30T16:17:35.248384+0000","last_active":"2024-05-30T17:10:06.732701+0000","last_peered":"2024-05-30T17:10:06.732701+0000","last_clean":"2024-05-30T17:10:06.732701+0000","last_became_active":"2024-05-30T16:17:35.247810+0000","last_became_peered":"2024-05-30T16:17:35.247810+0000","last_unstale":"2024-05-30T17:10:06.732701+0000","last_undegraded":"2024-05-30T17:10:06.732701+0000","last_fullsized":"2024-05-30T17:10:06.732701+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:56:26.172595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"0'0","reported_seq":50,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:01.773778+0000","last_change":"2024-05-30T16:17:31.215393+0000","last_active":"2024-05-30T17:10:01.773778+0000","last_peered":"2024-05-30T17:10:01.773778+0000","last_clean":"2024-05-30T17:10:01.773778+0000","last_became_active":"2024-05-30T16:17:31.212337+0000","last_became_peered":"2024-05-30T16:17:31.212337+0000","last_unstale":"2024-05-30T17:10:01.773778+0000","last_undegraded":"2024-05-30T17:10:01.773778+0000","last_fullsized":"2024-05-30T17:10:01.773778+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:48:30.989819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"265'55","reported_seq":210,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:06.556856+0000","last_change":"2024-05-30T16:12:50.810909+0000","last_active":"2024-05-30T17:11:06.556856+0000","last_peered":"2024-05-30T17:11:06.556856+0000","last_clean":"2024-05-30T17:11:06.556856+0000","last_became_active":"2024-05-30T16:12:50.810207+0000","last_became_peered":"2024-05-30T16:12:50.810207+0000","last_unstale":"2024-05-30T17:11:06.556856+0000","last_undegraded":"2024-05-30T17:11:06.556856+0000","last_fullsized":"2024-05-30T17:11:06.556856+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:37:48.188833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":277,"num_read_kb":197,"num_write":99,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.556933+0000","last_change":"2024-05-30T16:34:55.876451+0000","last_active":"2024-05-30T17:11:11.556933+0000","last_peered":"2024-05-30T17:11:11.556933+0000","last_clean":"2024-05-30T17:11:11.556933+0000","last_became_active":"2024-05-30T16:34:55.874359+0000","last_became_peered":"2024-05-30T16:34:55.874359+0000","last_unstale":"2024-05-30T17:11:11.556933+0000","last_undegraded":"2024-05-30T17:11:11.556933+0000","last_fullsized":"2024-05-30T17:11:11.556933+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:37:31.196070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"0'0","reported_seq":48,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:41.561910+0000","last_change":"2024-05-30T16:17:35.743986+0000","last_active":"2024-05-30T17:09:41.561910+0000","last_peered":"2024-05-30T17:09:41.561910+0000","last_clean":"2024-05-30T17:09:41.561910+0000","last_became_active":"2024-05-30T16:17:35.743231+0000","last_became_peered":"2024-05-30T16:17:35.743231+0000","last_unstale":"2024-05-30T17:09:41.561910+0000","last_undegraded":"2024-05-30T17:09:41.561910+0000","last_fullsized":"2024-05-30T17:09:41.561910+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:52:11.845318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"265'2","reported_seq":59,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:46.541550+0000","last_change":"2024-05-30T16:17:31.205642+0000","last_active":"2024-05-30T17:09:46.541550+0000","last_peered":"2024-05-30T17:09:46.541550+0000","last_clean":"2024-05-30T17:09:46.541550+0000","last_became_active":"2024-05-30T16:17:31.204806+0000","last_became_peered":"2024-05-30T16:17:31.204806+0000","last_unstale":"2024-05-30T17:09:46.541550+0000","last_undegraded":"2024-05-30T17:09:46.541550+0000","last_fullsized":"2024-05-30T17:09:46.541550+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:50:34.702667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":6144,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"265'29","reported_seq":134,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.570270+0000","last_change":"2024-05-30T16:12:50.811279+0000","last_active":"2024-05-30T17:10:21.570270+0000","last_peered":"2024-05-30T17:10:21.570270+0000","last_clean":"2024-05-30T17:10:21.570270+0000","last_became_active":"2024-05-30T16:12:50.810062+0000","last_became_peered":"2024-05-30T16:12:50.810062+0000","last_unstale":"2024-05-30T17:10:21.570270+0000","last_undegraded":"2024-05-30T17:10:21.570270+0000","last_fullsized":"2024-05-30T17:10:21.570270+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:23:45.234198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":98,"num_write":65,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":27,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:08:56.553929+0000","last_change":"2024-05-30T16:34:55.888292+0000","last_active":"2024-05-30T17:08:56.553929+0000","last_peered":"2024-05-30T17:08:56.553929+0000","last_clean":"2024-05-30T17:08:56.553929+0000","last_became_active":"2024-05-30T16:34:55.887532+0000","last_became_peered":"2024-05-30T16:34:55.887532+0000","last_unstale":"2024-05-30T17:08:56.553929+0000","last_undegraded":"2024-05-30T17:08:56.553929+0000","last_fullsized":"2024-05-30T17:08:56.553929+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:47:12.792997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"265'1","reported_seq":52,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:51.567346+0000","last_change":"2024-05-30T16:17:35.746712+0000","last_active":"2024-05-30T17:09:51.567346+0000","last_peered":"2024-05-30T17:09:51.567346+0000","last_clean":"2024-05-30T17:09:51.567346+0000","last_became_active":"2024-05-30T16:17:35.745709+0000","last_became_peered":"2024-05-30T16:17:35.745709+0000","last_unstale":"2024-05-30T17:09:51.567346+0000","last_undegraded":"2024-05-30T17:09:51.567346+0000","last_fullsized":"2024-05-30T17:09:51.567346+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:58:32.058273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":584363,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1142,"num_write":1,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1f","version":"265'2","reported_seq":56,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.726378+0000","last_change":"2024-05-30T16:17:31.183141+0000","last_active":"2024-05-30T17:09:36.726378+0000","last_peered":"2024-05-30T17:09:36.726378+0000","last_clean":"2024-05-30T17:09:36.726378+0000","last_became_active":"2024-05-30T16:17:31.182344+0000","last_became_peered":"2024-05-30T16:17:31.182344+0000","last_unstale":"2024-05-30T17:09:36.726378+0000","last_undegraded":"2024-05-30T17:09:36.726378+0000","last_fullsized":"2024-05-30T17:09:36.726378+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:20:08.705563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"260'6","reported_seq":76,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.573922+0000","last_change":"2024-05-30T16:12:50.830384+0000","last_active":"2024-05-30T17:10:21.573922+0000","last_peered":"2024-05-30T17:10:21.573922+0000","last_clean":"2024-05-30T17:10:21.573922+0000","last_became_active":"2024-05-30T16:12:50.826974+0000","last_became_peered":"2024-05-30T16:12:50.826974+0000","last_unstale":"2024-05-30T17:10:21.573922+0000","last_undegraded":"2024-05-30T17:10:21.573922+0000","last_fullsized":"2024-05-30T17:10:21.573922+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:27:49.529929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":24,"num_write":11,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":31,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:06.583992+0000","last_change":"2024-05-30T16:34:55.899516+0000","last_active":"2024-05-30T17:11:06.583992+0000","last_peered":"2024-05-30T17:11:06.583992+0000","last_clean":"2024-05-30T17:11:06.583992+0000","last_became_active":"2024-05-30T16:34:55.898864+0000","last_became_peered":"2024-05-30T16:34:55.898864+0000","last_unstale":"2024-05-30T17:11:06.583992+0000","last_undegraded":"2024-05-30T17:11:06.583992+0000","last_fullsized":"2024-05-30T17:11:06.583992+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:07:16.296580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"265'6","reported_seq":62,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:52.146144+0000","last_change":"2024-05-30T16:17:35.233543+0000","last_active":"2024-05-30T17:10:52.146144+0000","last_peered":"2024-05-30T17:10:52.146144+0000","last_clean":"2024-05-30T17:10:52.146144+0000","last_became_active":"2024-05-30T16:17:35.232744+0000","last_became_peered":"2024-05-30T16:17:35.232744+0000","last_unstale":"2024-05-30T17:10:52.146144+0000","last_undegraded":"2024-05-30T17:10:52.146144+0000","last_fullsized":"2024-05-30T17:10:52.146144+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:40:45.244827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":23554,"num_write":5,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"0'0","reported_seq":51,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:41.574504+0000","last_change":"2024-05-30T16:17:31.186983+0000","last_active":"2024-05-30T17:10:41.574504+0000","last_peered":"2024-05-30T17:10:41.574504+0000","last_clean":"2024-05-30T17:10:41.574504+0000","last_became_active":"2024-05-30T16:17:31.186229+0000","last_became_peered":"2024-05-30T16:17:31.186229+0000","last_unstale":"2024-05-30T17:10:41.574504+0000","last_undegraded":"2024-05-30T17:10:41.574504+0000","last_fullsized":"2024-05-30T17:10:41.574504+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:33:19.747591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"265'27","reported_seq":134,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.573828+0000","last_change":"2024-05-30T16:12:50.830804+0000","last_active":"2024-05-30T17:10:21.573828+0000","last_peered":"2024-05-30T17:10:21.573828+0000","last_clean":"2024-05-30T17:10:21.573828+0000","last_became_active":"2024-05-30T16:12:50.827008+0000","last_became_peered":"2024-05-30T16:12:50.827008+0000","last_unstale":"2024-05-30T17:10:21.573828+0000","last_undegraded":"2024-05-30T17:10:21.573828+0000","last_fullsized":"2024-05-30T17:10:21.573828+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:26:44.426182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":100,"num_write":57,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.778503+0000","last_change":"2024-05-30T16:34:55.870968+0000","last_active":"2024-05-30T17:10:21.778503+0000","last_peered":"2024-05-30T17:10:21.778503+0000","last_clean":"2024-05-30T17:10:21.778503+0000","last_became_active":"2024-05-30T16:34:55.870139+0000","last_became_peered":"2024-05-30T16:34:55.870139+0000","last_unstale":"2024-05-30T17:10:21.778503+0000","last_undegraded":"2024-05-30T17:10:21.778503+0000","last_fullsized":"2024-05-30T17:10:21.778503+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:55:17.556265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"265'5","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.551727+0000","last_change":"2024-05-30T16:17:35.234843+0000","last_active":"2024-05-30T17:10:46.551727+0000","last_peered":"2024-05-30T17:10:46.551727+0000","last_clean":"2024-05-30T17:10:46.551727+0000","last_became_active":"2024-05-30T16:17:35.233040+0000","last_became_peered":"2024-05-30T16:17:35.233040+0000","last_unstale":"2024-05-30T17:10:46.551727+0000","last_undegraded":"2024-05-30T17:10:46.551727+0000","last_fullsized":"2024-05-30T17:10:46.551727+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:42:31.077865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11582751,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":12336,"num_write":10,"num_write_kb":11312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"265'2","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.742318+0000","last_change":"2024-05-30T16:17:31.174675+0000","last_active":"2024-05-30T17:10:51.742318+0000","last_peered":"2024-05-30T17:10:51.742318+0000","last_clean":"2024-05-30T17:10:51.742318+0000","last_became_active":"2024-05-30T16:17:31.173900+0000","last_became_peered":"2024-05-30T16:17:31.173900+0000","last_unstale":"2024-05-30T17:10:51.742318+0000","last_undegraded":"2024-05-30T17:10:51.742318+0000","last_fullsized":"2024-05-30T17:10:51.742318+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:03:12.426092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":6144,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"265'75","reported_seq":266,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.546161+0000","last_change":"2024-05-30T16:12:50.815789+0000","last_active":"2024-05-30T17:10:21.546161+0000","last_peered":"2024-05-30T17:10:21.546161+0000","last_clean":"2024-05-30T17:10:21.546161+0000","last_became_active":"2024-05-30T16:12:50.815032+0000","last_became_peered":"2024-05-30T16:12:50.815032+0000","last_unstale":"2024-05-30T17:10:21.546161+0000","last_undegraded":"2024-05-30T17:10:21.546161+0000","last_fullsized":"2024-05-30T17:10:21.546161+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:53:44.633843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":396,"num_read_kb":277,"num_write":137,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"265'1","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.548662+0000","last_change":"2024-05-30T16:34:55.882343+0000","last_active":"2024-05-30T17:09:56.548662+0000","last_peered":"2024-05-30T17:09:56.548662+0000","last_clean":"2024-05-30T17:09:56.548662+0000","last_became_active":"2024-05-30T16:34:55.881414+0000","last_became_peered":"2024-05-30T16:34:55.881414+0000","last_unstale":"2024-05-30T17:09:56.548662+0000","last_undegraded":"2024-05-30T17:09:56.548662+0000","last_fullsized":"2024-05-30T17:09:56.548662+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:58:23.360902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"262'2","reported_seq":48,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:06.542967+0000","last_change":"2024-05-30T16:17:35.238116+0000","last_active":"2024-05-30T17:10:06.542967+0000","last_peered":"2024-05-30T17:10:06.542967+0000","last_clean":"2024-05-30T17:10:06.542967+0000","last_became_active":"2024-05-30T16:17:35.237424+0000","last_became_peered":"2024-05-30T16:17:35.237424+0000","last_unstale":"2024-05-30T17:10:06.542967+0000","last_undegraded":"2024-05-30T17:10:06.542967+0000","last_fullsized":"2024-05-30T17:10:06.542967+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:30:19.299666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"0'0","reported_seq":50,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:01.731526+0000","last_change":"2024-05-30T16:17:31.192981+0000","last_active":"2024-05-30T17:10:01.731526+0000","last_peered":"2024-05-30T17:10:01.731526+0000","last_clean":"2024-05-30T17:10:01.731526+0000","last_became_active":"2024-05-30T16:17:31.192269+0000","last_became_peered":"2024-05-30T16:17:31.192269+0000","last_unstale":"2024-05-30T17:10:01.731526+0000","last_undegraded":"2024-05-30T17:10:01.731526+0000","last_fullsized":"2024-05-30T17:10:01.731526+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:47:56.727573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"265'25","reported_seq":125,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:06.579930+0000","last_change":"2024-05-30T16:12:50.817479+0000","last_active":"2024-05-30T17:11:06.579930+0000","last_peered":"2024-05-30T17:11:06.579930+0000","last_clean":"2024-05-30T17:11:06.579930+0000","last_became_active":"2024-05-30T16:12:50.811732+0000","last_became_peered":"2024-05-30T16:12:50.811732+0000","last_unstale":"2024-05-30T17:11:06.579930+0000","last_undegraded":"2024-05-30T17:11:06.579930+0000","last_fullsized":"2024-05-30T17:11:06.579930+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:22:47.071307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":85,"num_write":53,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":27,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.548778+0000","last_change":"2024-05-30T16:34:55.899801+0000","last_active":"2024-05-30T17:09:56.548778+0000","last_peered":"2024-05-30T17:09:56.548778+0000","last_clean":"2024-05-30T17:09:56.548778+0000","last_became_active":"2024-05-30T16:34:55.899241+0000","last_became_peered":"2024-05-30T16:34:55.899241+0000","last_unstale":"2024-05-30T17:09:56.548778+0000","last_undegraded":"2024-05-30T17:09:56.548778+0000","last_fullsized":"2024-05-30T17:09:56.548778+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:25:21.668824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"0'0","reported_seq":44,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:06.570530+0000","last_change":"2024-05-30T16:17:35.229918+0000","last_active":"2024-05-30T17:10:06.570530+0000","last_peered":"2024-05-30T17:10:06.570530+0000","last_clean":"2024-05-30T17:10:06.570530+0000","last_became_active":"2024-05-30T16:17:35.229034+0000","last_became_peered":"2024-05-30T16:17:35.229034+0000","last_unstale":"2024-05-30T17:10:06.570530+0000","last_undegraded":"2024-05-30T17:10:06.570530+0000","last_fullsized":"2024-05-30T17:10:06.570530+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:01:27.231981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"265'7","reported_seq":71,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:01.565640+0000","last_change":"2024-05-30T16:17:31.191604+0000","last_active":"2024-05-30T17:10:01.565640+0000","last_peered":"2024-05-30T17:10:01.565640+0000","last_clean":"2024-05-30T17:10:01.565640+0000","last_became_active":"2024-05-30T16:17:31.190742+0000","last_became_peered":"2024-05-30T16:17:31.190742+0000","last_unstale":"2024-05-30T17:10:01.565640+0000","last_undegraded":"2024-05-30T17:10:01.565640+0000","last_fullsized":"2024-05-30T17:10:01.565640+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:37:20.245898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485760,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":30722,"num_write":16,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"265'33","reported_seq":143,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.785481+0000","last_change":"2024-05-30T16:12:50.821208+0000","last_active":"2024-05-30T17:10:51.785481+0000","last_peered":"2024-05-30T17:10:51.785481+0000","last_clean":"2024-05-30T17:10:51.785481+0000","last_became_active":"2024-05-30T16:12:50.820343+0000","last_became_peered":"2024-05-30T16:12:50.820343+0000","last_unstale":"2024-05-30T17:10:51.785481+0000","last_undegraded":"2024-05-30T17:10:51.785481+0000","last_fullsized":"2024-05-30T17:10:51.785481+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:46:09.411694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":154,"num_read_kb":114,"num_write":68,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.772692+0000","last_change":"2024-05-30T16:34:55.889437+0000","last_active":"2024-05-30T17:09:56.772692+0000","last_peered":"2024-05-30T17:09:56.772692+0000","last_clean":"2024-05-30T17:09:56.772692+0000","last_became_active":"2024-05-30T16:34:55.888238+0000","last_became_peered":"2024-05-30T16:34:55.888238+0000","last_unstale":"2024-05-30T17:09:56.772692+0000","last_undegraded":"2024-05-30T17:09:56.772692+0000","last_fullsized":"2024-05-30T17:09:56.772692+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:23:02.664317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"262'6","reported_seq":56,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.579797+0000","last_change":"2024-05-30T16:17:35.740004+0000","last_active":"2024-05-30T17:10:46.579797+0000","last_peered":"2024-05-30T17:10:46.579797+0000","last_clean":"2024-05-30T17:10:46.579797+0000","last_became_active":"2024-05-30T16:17:35.738420+0000","last_became_peered":"2024-05-30T16:17:35.738420+0000","last_unstale":"2024-05-30T17:10:46.579797+0000","last_undegraded":"2024-05-30T17:10:46.579797+0000","last_fullsized":"2024-05-30T17:10:46.579797+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:38:21.023993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6144,"num_write":16,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"265'4","reported_seq":59,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:08:57.120072+0000","last_change":"2024-05-30T16:17:31.195609+0000","last_active":"2024-05-30T17:08:57.120072+0000","last_peered":"2024-05-30T17:08:57.120072+0000","last_clean":"2024-05-30T17:08:57.120072+0000","last_became_active":"2024-05-30T16:17:31.195031+0000","last_became_peered":"2024-05-30T16:17:31.195031+0000","last_unstale":"2024-05-30T17:08:57.120072+0000","last_undegraded":"2024-05-30T17:08:57.120072+0000","last_fullsized":"2024-05-30T17:08:57.120072+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:38:54.212446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":584363,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":6715,"num_write":4,"num_write_kb":6715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"265'65","reported_seq":237,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.577047+0000","last_change":"2024-05-30T16:12:50.820938+0000","last_active":"2024-05-30T17:10:51.577047+0000","last_peered":"2024-05-30T17:10:51.577047+0000","last_clean":"2024-05-30T17:10:51.577047+0000","last_became_active":"2024-05-30T16:12:50.807771+0000","last_became_peered":"2024-05-30T16:12:50.807771+0000","last_unstale":"2024-05-30T17:10:51.577047+0000","last_undegraded":"2024-05-30T17:10:51.577047+0000","last_fullsized":"2024-05-30T17:10:51.577047+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:20:23.418802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":343,"num_read_kb":242,"num_write":116,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"265'1","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.543341+0000","last_change":"2024-05-30T16:34:55.873393+0000","last_active":"2024-05-30T17:09:56.543341+0000","last_peered":"2024-05-30T17:09:56.543341+0000","last_clean":"2024-05-30T17:09:56.543341+0000","last_became_active":"2024-05-30T16:34:55.872581+0000","last_became_peered":"2024-05-30T16:34:55.872581+0000","last_unstale":"2024-05-30T17:09:56.543341+0000","last_undegraded":"2024-05-30T17:09:56.543341+0000","last_fullsized":"2024-05-30T17:09:56.543341+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:23:46.116567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"0'0","reported_seq":45,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:11.557772+0000","last_change":"2024-05-30T16:17:35.739267+0000","last_active":"2024-05-30T17:09:11.557772+0000","last_peered":"2024-05-30T17:09:11.557772+0000","last_clean":"2024-05-30T17:09:11.557772+0000","last_became_active":"2024-05-30T16:17:35.738397+0000","last_became_peered":"2024-05-30T16:17:35.738397+0000","last_unstale":"2024-05-30T17:09:11.557772+0000","last_undegraded":"2024-05-30T17:09:11.557772+0000","last_fullsized":"2024-05-30T17:09:11.557772+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:09:46.285921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"265'2","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:41.559336+0000","last_change":"2024-05-30T16:17:31.197103+0000","last_active":"2024-05-30T17:10:41.559336+0000","last_peered":"2024-05-30T17:10:41.559336+0000","last_clean":"2024-05-30T17:10:41.559336+0000","last_became_active":"2024-05-30T16:17:31.196345+0000","last_became_peered":"2024-05-30T16:17:31.196345+0000","last_unstale":"2024-05-30T17:10:41.559336+0000","last_undegraded":"2024-05-30T17:10:41.559336+0000","last_fullsized":"2024-05-30T17:10:41.559336+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:39:32.522011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":5120,"num_write":2,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"265'38","reported_seq":159,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.581145+0000","last_change":"2024-05-30T16:12:50.827196+0000","last_active":"2024-05-30T17:10:51.581145+0000","last_peered":"2024-05-30T17:10:51.581145+0000","last_clean":"2024-05-30T17:10:51.581145+0000","last_became_active":"2024-05-30T16:12:50.826729+0000","last_became_peered":"2024-05-30T16:12:50.826729+0000","last_unstale":"2024-05-30T17:10:51.581145+0000","last_undegraded":"2024-05-30T17:10:51.581145+0000","last_fullsized":"2024-05-30T17:10:51.581145+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:46:31.837238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":187,"num_read_kb":132,"num_write":81,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.548483+0000","last_change":"2024-05-30T16:34:55.876495+0000","last_active":"2024-05-30T17:09:56.548483+0000","last_peered":"2024-05-30T17:09:56.548483+0000","last_clean":"2024-05-30T17:09:56.548483+0000","last_became_active":"2024-05-30T16:34:55.875788+0000","last_became_peered":"2024-05-30T16:34:55.875788+0000","last_unstale":"2024-05-30T17:09:56.548483+0000","last_undegraded":"2024-05-30T17:09:56.548483+0000","last_fullsized":"2024-05-30T17:09:56.548483+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:15:31.389508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"0'0","reported_seq":45,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.579917+0000","last_change":"2024-05-30T16:17:35.746542+0000","last_active":"2024-05-30T17:10:46.579917+0000","last_peered":"2024-05-30T17:10:46.579917+0000","last_clean":"2024-05-30T17:10:46.579917+0000","last_became_active":"2024-05-30T16:17:35.745769+0000","last_became_peered":"2024-05-30T16:17:35.745769+0000","last_unstale":"2024-05-30T17:10:46.579917+0000","last_undegraded":"2024-05-30T17:10:46.579917+0000","last_fullsized":"2024-05-30T17:10:46.579917+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:03:42.283507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"265'1","reported_seq":53,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:41.552055+0000","last_change":"2024-05-30T16:17:31.206243+0000","last_active":"2024-05-30T17:10:41.552055+0000","last_peered":"2024-05-30T17:10:41.552055+0000","last_clean":"2024-05-30T17:10:41.552055+0000","last_became_active":"2024-05-30T16:17:31.204434+0000","last_became_peered":"2024-05-30T16:17:31.204434+0000","last_unstale":"2024-05-30T17:10:41.552055+0000","last_undegraded":"2024-05-30T17:10:41.552055+0000","last_fullsized":"2024-05-30T17:10:41.552055+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:33:27.089723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":3072,"num_write":1,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"265'26","reported_seq":125,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.554131+0000","last_change":"2024-05-30T16:12:50.814716+0000","last_active":"2024-05-30T17:10:51.554131+0000","last_peered":"2024-05-30T17:10:51.554131+0000","last_clean":"2024-05-30T17:10:51.554131+0000","last_became_active":"2024-05-30T16:12:50.813176+0000","last_became_peered":"2024-05-30T16:12:50.813176+0000","last_unstale":"2024-05-30T17:10:51.554131+0000","last_undegraded":"2024-05-30T17:10:51.554131+0000","last_fullsized":"2024-05-30T17:10:51.554131+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:45:17.112612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":85,"num_write":57,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:57.133868+0000","last_change":"2024-05-30T16:34:55.872909+0000","last_active":"2024-05-30T17:09:57.133868+0000","last_peered":"2024-05-30T17:09:57.133868+0000","last_clean":"2024-05-30T17:09:57.133868+0000","last_became_active":"2024-05-30T16:34:55.871800+0000","last_became_peered":"2024-05-30T16:34:55.871800+0000","last_unstale":"2024-05-30T17:09:57.133868+0000","last_undegraded":"2024-05-30T17:09:57.133868+0000","last_fullsized":"2024-05-30T17:09:57.133868+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:10:45.189484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"265'1","reported_seq":47,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.741327+0000","last_change":"2024-05-30T16:17:35.237595+0000","last_active":"2024-05-30T17:10:46.741327+0000","last_peered":"2024-05-30T17:10:46.741327+0000","last_clean":"2024-05-30T17:10:46.741327+0000","last_became_active":"2024-05-30T16:17:35.236867+0000","last_became_peered":"2024-05-30T16:17:35.236867+0000","last_unstale":"2024-05-30T17:10:46.741327+0000","last_undegraded":"2024-05-30T17:10:46.741327+0000","last_fullsized":"2024-05-30T17:10:46.741327+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:47:01.389981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":584363,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":571,"num_write":1,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"0'0","reported_seq":51,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:41.574782+0000","last_change":"2024-05-30T16:17:31.210713+0000","last_active":"2024-05-30T17:10:41.574782+0000","last_peered":"2024-05-30T17:10:41.574782+0000","last_clean":"2024-05-30T17:10:41.574782+0000","last_became_active":"2024-05-30T16:17:31.209882+0000","last_became_peered":"2024-05-30T16:17:31.209882+0000","last_unstale":"2024-05-30T17:10:41.574782+0000","last_undegraded":"2024-05-30T17:10:41.574782+0000","last_fullsized":"2024-05-30T17:10:41.574782+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:17:20.852336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"265'56","reported_seq":218,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:01.749663+0000","last_change":"2024-05-30T16:12:50.825181+0000","last_active":"2024-05-30T17:09:01.749663+0000","last_peered":"2024-05-30T17:09:01.749663+0000","last_clean":"2024-05-30T17:09:01.749663+0000","last_became_active":"2024-05-30T16:12:50.824043+0000","last_became_peered":"2024-05-30T16:12:50.824043+0000","last_unstale":"2024-05-30T17:09:01.749663+0000","last_undegraded":"2024-05-30T17:09:01.749663+0000","last_fullsized":"2024-05-30T17:09:01.749663+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:29.566016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":308,"num_read_kb":221,"num_write":92,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.585310+0000","last_change":"2024-05-30T16:34:55.893966+0000","last_active":"2024-05-30T17:11:11.585310+0000","last_peered":"2024-05-30T17:11:11.585310+0000","last_clean":"2024-05-30T17:11:11.585310+0000","last_became_active":"2024-05-30T16:34:55.893040+0000","last_became_peered":"2024-05-30T16:34:55.893040+0000","last_unstale":"2024-05-30T17:11:11.585310+0000","last_undegraded":"2024-05-30T17:11:11.585310+0000","last_fullsized":"2024-05-30T17:11:11.585310+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:55:00.349712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"0'0","reported_seq":46,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.790902+0000","last_change":"2024-05-30T16:17:35.246948+0000","last_active":"2024-05-30T17:11:16.790902+0000","last_peered":"2024-05-30T17:11:16.790902+0000","last_clean":"2024-05-30T17:11:16.790902+0000","last_became_active":"2024-05-30T16:17:35.246187+0000","last_became_peered":"2024-05-30T16:17:35.246187+0000","last_unstale":"2024-05-30T17:11:16.790902+0000","last_undegraded":"2024-05-30T17:11:16.790902+0000","last_fullsized":"2024-05-30T17:11:16.790902+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:23:58.794231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"265'6","reported_seq":69,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.746911+0000","last_change":"2024-05-30T16:17:31.189808+0000","last_active":"2024-05-30T17:11:11.746911+0000","last_peered":"2024-05-30T17:11:11.746911+0000","last_clean":"2024-05-30T17:11:11.746911+0000","last_became_active":"2024-05-30T16:17:31.189091+0000","last_became_peered":"2024-05-30T16:17:31.189091+0000","last_unstale":"2024-05-30T17:11:11.746911+0000","last_undegraded":"2024-05-30T17:11:11.746911+0000","last_fullsized":"2024-05-30T17:11:11.746911+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:35:04.039725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5827243,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":21053,"num_write":5,"num_write_kb":5691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"265'9","reported_seq":84,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.566377+0000","last_change":"2024-05-30T16:12:50.824521+0000","last_active":"2024-05-30T17:11:11.566377+0000","last_peered":"2024-05-30T17:11:11.566377+0000","last_clean":"2024-05-30T17:11:11.566377+0000","last_became_active":"2024-05-30T16:12:50.823759+0000","last_became_peered":"2024-05-30T16:12:50.823759+0000","last_unstale":"2024-05-30T17:11:11.566377+0000","last_undegraded":"2024-05-30T17:11:11.566377+0000","last_fullsized":"2024-05-30T17:11:11.566377+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:41:59.872702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":33,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.581162+0000","last_change":"2024-05-30T16:34:55.861418+0000","last_active":"2024-05-30T17:11:11.581162+0000","last_peered":"2024-05-30T17:11:11.581162+0000","last_clean":"2024-05-30T17:11:11.581162+0000","last_became_active":"2024-05-30T16:34:55.860695+0000","last_became_peered":"2024-05-30T16:34:55.860695+0000","last_unstale":"2024-05-30T17:11:11.581162+0000","last_undegraded":"2024-05-30T17:11:11.581162+0000","last_fullsized":"2024-05-30T17:11:11.581162+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:45:54.013286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"0'0","reported_seq":46,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.790812+0000","last_change":"2024-05-30T16:17:35.239935+0000","last_active":"2024-05-30T17:11:16.790812+0000","last_peered":"2024-05-30T17:11:16.790812+0000","last_clean":"2024-05-30T17:11:16.790812+0000","last_became_active":"2024-05-30T16:17:35.239157+0000","last_became_peered":"2024-05-30T16:17:35.239157+0000","last_unstale":"2024-05-30T17:11:16.790812+0000","last_undegraded":"2024-05-30T17:11:16.790812+0000","last_fullsized":"2024-05-30T17:11:16.790812+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:53:43.552214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"265'1","reported_seq":54,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.585076+0000","last_change":"2024-05-30T16:17:31.185375+0000","last_active":"2024-05-30T17:11:11.585076+0000","last_peered":"2024-05-30T17:11:11.585076+0000","last_clean":"2024-05-30T17:11:11.585076+0000","last_became_active":"2024-05-30T16:17:31.184675+0000","last_became_peered":"2024-05-30T16:17:31.184675+0000","last_unstale":"2024-05-30T17:11:11.585076+0000","last_undegraded":"2024-05-30T17:11:11.585076+0000","last_fullsized":"2024-05-30T17:11:11.585076+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:13:30.582952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":714225,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":698,"num_write":1,"num_write_kb":698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"262'96","reported_seq":357,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.581035+0000","last_change":"2024-05-30T16:12:50.802561+0000","last_active":"2024-05-30T17:11:11.581035+0000","last_peered":"2024-05-30T17:11:11.581035+0000","last_clean":"2024-05-30T17:11:11.581035+0000","last_became_active":"2024-05-30T16:12:50.801791+0000","last_became_peered":"2024-05-30T16:12:50.801791+0000","last_unstale":"2024-05-30T17:11:11.581035+0000","last_undegraded":"2024-05-30T17:11:11.581035+0000","last_fullsized":"2024-05-30T17:11:11.581035+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:19:14.883602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":591,"num_read_kb":446,"num_write":104,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"264'1","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:06.566810+0000","last_change":"2024-05-30T16:34:55.864834+0000","last_active":"2024-05-30T17:10:06.566810+0000","last_peered":"2024-05-30T17:10:06.566810+0000","last_clean":"2024-05-30T17:10:06.566810+0000","last_became_active":"2024-05-30T16:34:55.863483+0000","last_became_peered":"2024-05-30T16:34:55.863483+0000","last_unstale":"2024-05-30T17:10:06.566810+0000","last_undegraded":"2024-05-30T17:10:06.566810+0000","last_fullsized":"2024-05-30T17:10:06.566810+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:46:20.034900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"0'0","reported_seq":46,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.586383+0000","last_change":"2024-05-30T16:17:35.236085+0000","last_active":"2024-05-30T17:11:16.586383+0000","last_peered":"2024-05-30T17:11:16.586383+0000","last_clean":"2024-05-30T17:11:16.586383+0000","last_became_active":"2024-05-30T16:17:35.235283+0000","last_became_peered":"2024-05-30T16:17:35.235283+0000","last_unstale":"2024-05-30T17:11:16.586383+0000","last_undegraded":"2024-05-30T17:11:16.586383+0000","last_fullsized":"2024-05-30T17:11:16.586383+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:19:44.053076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"0'0","reported_seq":52,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.557098+0000","last_change":"2024-05-30T16:17:31.204968+0000","last_active":"2024-05-30T17:11:11.557098+0000","last_peered":"2024-05-30T17:11:11.557098+0000","last_clean":"2024-05-30T17:11:11.557098+0000","last_became_active":"2024-05-30T16:17:31.199499+0000","last_became_peered":"2024-05-30T16:17:31.199499+0000","last_unstale":"2024-05-30T17:11:11.557098+0000","last_undegraded":"2024-05-30T17:11:11.557098+0000","last_fullsized":"2024-05-30T17:11:11.557098+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:18:34.513889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"265'22","reported_seq":118,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.746972+0000","last_change":"2024-05-30T16:12:50.807855+0000","last_active":"2024-05-30T17:11:11.746972+0000","last_peered":"2024-05-30T17:11:11.746972+0000","last_clean":"2024-05-30T17:11:11.746972+0000","last_became_active":"2024-05-30T16:12:50.807156+0000","last_became_peered":"2024-05-30T16:12:50.807156+0000","last_unstale":"2024-05-30T17:11:11.746972+0000","last_undegraded":"2024-05-30T17:11:11.746972+0000","last_fullsized":"2024-05-30T17:11:11.746972+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:18:22.683470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":76,"num_write":50,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"265'1","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.585231+0000","last_change":"2024-05-30T16:34:55.889162+0000","last_active":"2024-05-30T17:11:11.585231+0000","last_peered":"2024-05-30T17:11:11.585231+0000","last_clean":"2024-05-30T17:11:11.585231+0000","last_became_active":"2024-05-30T16:34:55.887845+0000","last_became_peered":"2024-05-30T16:34:55.887845+0000","last_unstale":"2024-05-30T17:11:11.585231+0000","last_undegraded":"2024-05-30T17:11:11.585231+0000","last_fullsized":"2024-05-30T17:11:11.585231+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:36:50.835387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"0'0","reported_seq":46,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.558367+0000","last_change":"2024-05-30T16:17:35.246711+0000","last_active":"2024-05-30T17:11:16.558367+0000","last_peered":"2024-05-30T17:11:16.558367+0000","last_clean":"2024-05-30T17:11:16.558367+0000","last_became_active":"2024-05-30T16:17:35.245852+0000","last_became_peered":"2024-05-30T16:17:35.245852+0000","last_unstale":"2024-05-30T17:11:16.558367+0000","last_undegraded":"2024-05-30T17:11:16.558367+0000","last_fullsized":"2024-05-30T17:11:16.558367+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:50:22.430770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"262'5","reported_seq":67,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:12.150705+0000","last_change":"2024-05-30T16:17:31.219308+0000","last_active":"2024-05-30T17:11:12.150705+0000","last_peered":"2024-05-30T17:11:12.150705+0000","last_clean":"2024-05-30T17:11:12.150705+0000","last_became_active":"2024-05-30T16:17:31.218752+0000","last_became_peered":"2024-05-30T16:17:31.218752+0000","last_unstale":"2024-05-30T17:11:12.150705+0000","last_undegraded":"2024-05-30T17:11:12.150705+0000","last_fullsized":"2024-05-30T17:11:12.150705+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:47:57.704377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"265'17","reported_seq":103,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.584982+0000","last_change":"2024-05-30T16:12:50.829602+0000","last_active":"2024-05-30T17:11:11.584982+0000","last_peered":"2024-05-30T17:11:11.584982+0000","last_clean":"2024-05-30T17:11:11.584982+0000","last_became_active":"2024-05-30T16:12:50.817114+0000","last_became_peered":"2024-05-30T16:12:50.817114+0000","last_unstale":"2024-05-30T17:11:11.584982+0000","last_undegraded":"2024-05-30T17:11:11.584982+0000","last_fullsized":"2024-05-30T17:11:11.584982+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:05:48.646667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":55,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:46.541438+0000","last_change":"2024-05-30T16:34:55.867126+0000","last_active":"2024-05-30T17:09:46.541438+0000","last_peered":"2024-05-30T17:09:46.541438+0000","last_clean":"2024-05-30T17:09:46.541438+0000","last_became_active":"2024-05-30T16:34:55.866415+0000","last_became_peered":"2024-05-30T16:34:55.866415+0000","last_unstale":"2024-05-30T17:09:46.541438+0000","last_undegraded":"2024-05-30T17:09:46.541438+0000","last_fullsized":"2024-05-30T17:09:46.541438+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:05:20.422821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"0'0","reported_seq":46,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.560610+0000","last_change":"2024-05-30T16:17:35.228446+0000","last_active":"2024-05-30T17:09:21.560610+0000","last_peered":"2024-05-30T17:09:21.560610+0000","last_clean":"2024-05-30T17:09:21.560610+0000","last_became_active":"2024-05-30T16:17:35.227270+0000","last_became_peered":"2024-05-30T16:17:35.227270+0000","last_unstale":"2024-05-30T17:09:21.560610+0000","last_undegraded":"2024-05-30T17:09:21.560610+0000","last_fullsized":"2024-05-30T17:09:21.560610+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:47:07.315021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"0'0","reported_seq":52,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:16.753258+0000","last_change":"2024-05-30T16:17:31.204447+0000","last_active":"2024-05-30T17:09:16.753258+0000","last_peered":"2024-05-30T17:09:16.753258+0000","last_clean":"2024-05-30T17:09:16.753258+0000","last_became_active":"2024-05-30T16:17:31.203731+0000","last_became_peered":"2024-05-30T16:17:31.203731+0000","last_unstale":"2024-05-30T17:09:16.753258+0000","last_undegraded":"2024-05-30T17:09:16.753258+0000","last_fullsized":"2024-05-30T17:09:16.753258+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:02:51.039638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"265'29","reported_seq":136,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.560856+0000","last_change":"2024-05-30T16:12:50.825208+0000","last_active":"2024-05-30T17:09:21.560856+0000","last_peered":"2024-05-30T17:09:21.560856+0000","last_clean":"2024-05-30T17:09:21.560856+0000","last_became_active":"2024-05-30T16:12:50.824809+0000","last_became_peered":"2024-05-30T16:12:50.824809+0000","last_unstale":"2024-05-30T17:09:21.560856+0000","last_undegraded":"2024-05-30T17:09:21.560856+0000","last_fullsized":"2024-05-30T17:09:21.560856+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:38:19.384141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":98,"num_write":77,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:56.562832+0000","last_change":"2024-05-30T16:34:55.897061+0000","last_active":"2024-05-30T17:10:56.562832+0000","last_peered":"2024-05-30T17:10:56.562832+0000","last_clean":"2024-05-30T17:10:56.562832+0000","last_became_active":"2024-05-30T16:34:55.896357+0000","last_became_peered":"2024-05-30T16:34:55.896357+0000","last_unstale":"2024-05-30T17:10:56.562832+0000","last_undegraded":"2024-05-30T17:10:56.562832+0000","last_fullsized":"2024-05-30T17:10:56.562832+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:23:44.881356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"265'2","reported_seq":44,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:42.130647+0000","last_change":"2024-05-30T16:17:55.551397+0000","last_active":"2024-05-30T17:09:42.130647+0000","last_peered":"2024-05-30T17:09:42.130647+0000","last_clean":"2024-05-30T17:09:42.130647+0000","last_became_active":"2024-05-30T16:17:55.550782+0000","last_became_peered":"2024-05-30T16:17:55.550782+0000","last_unstale":"2024-05-30T17:09:42.130647+0000","last_undegraded":"2024-05-30T17:09:42.130647+0000","last_fullsized":"2024-05-30T17:09:42.130647+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:44:58.931548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2048,"num_write":7,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.d","version":"265'5","reported_seq":62,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:16.535485+0000","last_change":"2024-05-30T16:17:31.202846+0000","last_active":"2024-05-30T17:09:16.535485+0000","last_peered":"2024-05-30T17:09:16.535485+0000","last_clean":"2024-05-30T17:09:16.535485+0000","last_became_active":"2024-05-30T16:17:31.202042+0000","last_became_peered":"2024-05-30T16:17:31.202042+0000","last_unstale":"2024-05-30T17:09:16.535485+0000","last_undegraded":"2024-05-30T17:09:16.535485+0000","last_fullsized":"2024-05-30T17:09:16.535485+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:38:37.711263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6339871,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":6194,"num_write":5,"num_write_kb":6194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"265'24","reported_seq":130,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:22.153145+0000","last_change":"2024-05-30T16:12:50.810823+0000","last_active":"2024-05-30T17:11:22.153145+0000","last_peered":"2024-05-30T17:11:22.153145+0000","last_clean":"2024-05-30T17:11:22.153145+0000","last_became_active":"2024-05-30T16:12:50.810095+0000","last_became_peered":"2024-05-30T16:12:50.810095+0000","last_unstale":"2024-05-30T17:11:22.153145+0000","last_undegraded":"2024-05-30T17:11:22.153145+0000","last_fullsized":"2024-05-30T17:11:22.153145+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:29:53.744702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":90,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.536265+0000","last_change":"2024-05-30T16:34:55.863218+0000","last_active":"2024-05-30T17:09:36.536265+0000","last_peered":"2024-05-30T17:09:36.536265+0000","last_clean":"2024-05-30T17:09:36.536265+0000","last_became_active":"2024-05-30T16:34:55.862478+0000","last_became_peered":"2024-05-30T16:34:55.862478+0000","last_unstale":"2024-05-30T17:09:36.536265+0000","last_undegraded":"2024-05-30T17:09:36.536265+0000","last_fullsized":"2024-05-30T17:09:36.536265+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:30.967780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"262'3","reported_seq":52,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:31.768287+0000","last_change":"2024-05-30T16:17:35.743152+0000","last_active":"2024-05-30T17:09:31.768287+0000","last_peered":"2024-05-30T17:09:31.768287+0000","last_clean":"2024-05-30T17:09:31.768287+0000","last_became_active":"2024-05-30T16:17:35.742255+0000","last_became_peered":"2024-05-30T16:17:35.742255+0000","last_unstale":"2024-05-30T17:09:31.768287+0000","last_undegraded":"2024-05-30T17:09:31.768287+0000","last_fullsized":"2024-05-30T17:09:31.768287+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:41:50.476109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"265'1","reported_seq":54,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:16.556005+0000","last_change":"2024-05-30T16:17:31.199481+0000","last_active":"2024-05-30T17:09:16.556005+0000","last_peered":"2024-05-30T17:09:16.556005+0000","last_clean":"2024-05-30T17:09:16.556005+0000","last_became_active":"2024-05-30T16:17:31.198744+0000","last_became_peered":"2024-05-30T16:17:31.198744+0000","last_unstale":"2024-05-30T17:09:16.556005+0000","last_undegraded":"2024-05-30T17:09:16.556005+0000","last_fullsized":"2024-05-30T17:09:16.556005+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:39:35.328874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":3072,"num_write":1,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"265'36","reported_seq":163,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.559692+0000","last_change":"2024-05-30T16:12:50.821672+0000","last_active":"2024-05-30T17:11:21.559692+0000","last_peered":"2024-05-30T17:11:21.559692+0000","last_clean":"2024-05-30T17:11:21.559692+0000","last_became_active":"2024-05-30T16:12:50.820936+0000","last_became_peered":"2024-05-30T16:12:50.820936+0000","last_unstale":"2024-05-30T17:11:21.559692+0000","last_undegraded":"2024-05-30T17:11:21.559692+0000","last_fullsized":"2024-05-30T17:11:21.559692+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:52:42.146262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":194,"num_read_kb":136,"num_write":73,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.543696+0000","last_change":"2024-05-30T16:34:55.877239+0000","last_active":"2024-05-30T17:09:36.543696+0000","last_peered":"2024-05-30T17:09:36.543696+0000","last_clean":"2024-05-30T17:09:36.543696+0000","last_became_active":"2024-05-30T16:34:55.875965+0000","last_became_peered":"2024-05-30T16:34:55.875965+0000","last_unstale":"2024-05-30T17:09:36.543696+0000","last_undegraded":"2024-05-30T17:09:36.543696+0000","last_fullsized":"2024-05-30T17:09:36.543696+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:10:50.846236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"265'2","reported_seq":50,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.560723+0000","last_change":"2024-05-30T16:17:35.234741+0000","last_active":"2024-05-30T17:09:21.560723+0000","last_peered":"2024-05-30T17:09:21.560723+0000","last_clean":"2024-05-30T17:09:21.560723+0000","last_became_active":"2024-05-30T16:17:35.229643+0000","last_became_peered":"2024-05-30T16:17:35.229643+0000","last_unstale":"2024-05-30T17:09:21.560723+0000","last_undegraded":"2024-05-30T17:09:21.560723+0000","last_fullsized":"2024-05-30T17:09:21.560723+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:55:14.642511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":6144,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.f","version":"262'6","reported_seq":63,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:26.767116+0000","last_change":"2024-05-30T16:17:31.215290+0000","last_active":"2024-05-30T17:09:26.767116+0000","last_peered":"2024-05-30T17:09:26.767116+0000","last_clean":"2024-05-30T17:09:26.767116+0000","last_became_active":"2024-05-30T16:17:31.212894+0000","last_became_peered":"2024-05-30T16:17:31.212894+0000","last_unstale":"2024-05-30T17:09:26.767116+0000","last_undegraded":"2024-05-30T17:09:26.767116+0000","last_fullsized":"2024-05-30T17:09:26.767116+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:31:09.789041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6144,"num_write":11,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"262'21","reported_seq":121,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.559938+0000","last_change":"2024-05-30T16:12:50.806577+0000","last_active":"2024-05-30T17:11:21.559938+0000","last_peered":"2024-05-30T17:11:21.559938+0000","last_clean":"2024-05-30T17:11:21.559938+0000","last_became_active":"2024-05-30T16:12:50.805772+0000","last_became_peered":"2024-05-30T16:12:50.805772+0000","last_unstale":"2024-05-30T17:11:21.559938+0000","last_undegraded":"2024-05-30T17:11:21.559938+0000","last_fullsized":"2024-05-30T17:11:21.559938+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:11:36.510014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":79,"num_write":45,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:56.582343+0000","last_change":"2024-05-30T16:34:55.891074+0000","last_active":"2024-05-30T17:10:56.582343+0000","last_peered":"2024-05-30T17:10:56.582343+0000","last_clean":"2024-05-30T17:10:56.582343+0000","last_became_active":"2024-05-30T16:34:55.890313+0000","last_became_peered":"2024-05-30T16:34:55.890313+0000","last_unstale":"2024-05-30T17:10:56.582343+0000","last_undegraded":"2024-05-30T17:10:56.582343+0000","last_fullsized":"2024-05-30T17:10:56.582343+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:22:36.639758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"265'2","reported_seq":53,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.726171+0000","last_change":"2024-05-30T16:17:35.243089+0000","last_active":"2024-05-30T17:09:36.726171+0000","last_peered":"2024-05-30T17:09:36.726171+0000","last_clean":"2024-05-30T17:09:36.726171+0000","last_became_active":"2024-05-30T16:17:35.242271+0000","last_became_peered":"2024-05-30T16:17:35.242271+0000","last_unstale":"2024-05-30T17:09:36.726171+0000","last_undegraded":"2024-05-30T17:09:36.726171+0000","last_fullsized":"2024-05-30T17:09:36.726171+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:44:49.304400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":10240,"num_write":7,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"265'2","reported_seq":59,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:31.559803+0000","last_change":"2024-05-30T16:17:31.195571+0000","last_active":"2024-05-30T17:09:31.559803+0000","last_peered":"2024-05-30T17:09:31.559803+0000","last_clean":"2024-05-30T17:09:31.559803+0000","last_became_active":"2024-05-30T16:17:31.194774+0000","last_became_peered":"2024-05-30T16:17:31.194774+0000","last_unstale":"2024-05-30T17:09:31.559803+0000","last_undegraded":"2024-05-30T17:09:31.559803+0000","last_fullsized":"2024-05-30T17:09:31.559803+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:24:47.216863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":12288,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"265'18","reported_seq":107,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.584196+0000","last_change":"2024-05-30T16:12:50.820954+0000","last_active":"2024-05-30T17:11:21.584196+0000","last_peered":"2024-05-30T17:11:21.584196+0000","last_clean":"2024-05-30T17:11:21.584196+0000","last_became_active":"2024-05-30T16:12:50.820526+0000","last_became_peered":"2024-05-30T16:12:50.820526+0000","last_unstale":"2024-05-30T17:11:21.584196+0000","last_undegraded":"2024-05-30T17:11:21.584196+0000","last_fullsized":"2024-05-30T17:11:21.584196+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:08:41.493544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":60,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:06.565282+0000","last_change":"2024-05-30T16:34:55.884293+0000","last_active":"2024-05-30T17:11:06.565282+0000","last_peered":"2024-05-30T17:11:06.565282+0000","last_clean":"2024-05-30T17:11:06.565282+0000","last_became_active":"2024-05-30T16:34:55.884065+0000","last_became_peered":"2024-05-30T16:34:55.884065+0000","last_unstale":"2024-05-30T17:11:06.565282+0000","last_undegraded":"2024-05-30T17:11:06.565282+0000","last_fullsized":"2024-05-30T17:11:06.565282+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:13:34.815638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"265'9","reported_seq":71,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.543613+0000","last_change":"2024-05-30T16:17:35.739756+0000","last_active":"2024-05-30T17:09:36.543613+0000","last_peered":"2024-05-30T17:09:36.543613+0000","last_clean":"2024-05-30T17:09:36.543613+0000","last_became_active":"2024-05-30T16:17:35.738991+0000","last_became_peered":"2024-05-30T16:17:35.738991+0000","last_unstale":"2024-05-30T17:09:36.543613+0000","last_undegraded":"2024-05-30T17:09:36.543613+0000","last_fullsized":"2024-05-30T17:09:36.543613+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:50:21.381841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5957105,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":26998,"num_write":13,"num_write_kb":10938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"262'6","reported_seq":65,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:31.768080+0000","last_change":"2024-05-30T16:17:31.216617+0000","last_active":"2024-05-30T17:09:31.768080+0000","last_peered":"2024-05-30T17:09:31.768080+0000","last_clean":"2024-05-30T17:09:31.768080+0000","last_became_active":"2024-05-30T16:17:31.212954+0000","last_became_peered":"2024-05-30T16:17:31.212954+0000","last_unstale":"2024-05-30T17:09:31.768080+0000","last_undegraded":"2024-05-30T17:09:31.768080+0000","last_fullsized":"2024-05-30T17:09:31.768080+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:53:12.730929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":11264,"num_write":11,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"262'102","reported_seq":385,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.754543+0000","last_change":"2024-05-30T16:12:50.821280+0000","last_active":"2024-05-30T17:09:21.754543+0000","last_peered":"2024-05-30T17:09:21.754543+0000","last_clean":"2024-05-30T17:09:21.754543+0000","last_became_active":"2024-05-30T16:12:50.820250+0000","last_became_peered":"2024-05-30T16:12:50.820250+0000","last_unstale":"2024-05-30T17:09:21.754543+0000","last_undegraded":"2024-05-30T17:09:21.754543+0000","last_fullsized":"2024-05-30T17:09:21.754543+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:43:31.770630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":642,"num_read_kb":499,"num_write":114,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"265'1","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:56.562900+0000","last_change":"2024-05-30T16:34:55.880548+0000","last_active":"2024-05-30T17:10:56.562900+0000","last_peered":"2024-05-30T17:10:56.562900+0000","last_clean":"2024-05-30T17:10:56.562900+0000","last_became_active":"2024-05-30T16:34:55.878171+0000","last_became_peered":"2024-05-30T16:34:55.878171+0000","last_unstale":"2024-05-30T17:10:56.562900+0000","last_undegraded":"2024-05-30T17:10:56.562900+0000","last_fullsized":"2024-05-30T17:10:56.562900+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:56:14.165458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"265'11","reported_seq":71,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.765556+0000","last_change":"2024-05-30T16:17:35.245798+0000","last_active":"2024-05-30T17:09:21.765556+0000","last_peered":"2024-05-30T17:09:21.765556+0000","last_clean":"2024-05-30T17:09:21.765556+0000","last_became_active":"2024-05-30T16:17:35.245008+0000","last_became_peered":"2024-05-30T16:17:35.245008+0000","last_unstale":"2024-05-30T17:09:21.765556+0000","last_undegraded":"2024-05-30T17:09:21.765556+0000","last_fullsized":"2024-05-30T17:09:21.765556+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:28:02.631261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17825792,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":25602,"num_write":20,"num_write_kb":17408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"262'3","reported_seq":59,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:46.562533+0000","last_change":"2024-05-30T16:17:31.211050+0000","last_active":"2024-05-30T17:09:46.562533+0000","last_peered":"2024-05-30T17:09:46.562533+0000","last_clean":"2024-05-30T17:09:46.562533+0000","last_became_active":"2024-05-30T16:17:31.210195+0000","last_became_peered":"2024-05-30T16:17:31.210195+0000","last_unstale":"2024-05-30T17:09:46.562533+0000","last_undegraded":"2024-05-30T17:09:46.562533+0000","last_fullsized":"2024-05-30T17:09:46.562533+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:19:11.621024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"265'43","reported_seq":176,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.587423+0000","last_change":"2024-05-30T16:12:50.831498+0000","last_active":"2024-05-30T17:11:21.587423+0000","last_peered":"2024-05-30T17:11:21.587423+0000","last_clean":"2024-05-30T17:11:21.587423+0000","last_became_active":"2024-05-30T16:12:50.830770+0000","last_became_peered":"2024-05-30T16:12:50.830770+0000","last_unstale":"2024-05-30T17:11:21.587423+0000","last_undegraded":"2024-05-30T17:11:21.587423+0000","last_fullsized":"2024-05-30T17:11:21.587423+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:10:09.450788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":214,"num_read_kb":148,"num_write":89,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:56.578000+0000","last_change":"2024-05-30T16:34:55.887780+0000","last_active":"2024-05-30T17:10:56.578000+0000","last_peered":"2024-05-30T17:10:56.578000+0000","last_clean":"2024-05-30T17:10:56.578000+0000","last_became_active":"2024-05-30T16:34:55.887101+0000","last_became_peered":"2024-05-30T16:34:55.887101+0000","last_unstale":"2024-05-30T17:10:56.578000+0000","last_undegraded":"2024-05-30T17:10:56.578000+0000","last_fullsized":"2024-05-30T17:10:56.578000+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:43:47.335383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"262'3","reported_seq":53,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:51.547299+0000","last_change":"2024-05-30T16:17:35.213121+0000","last_active":"2024-05-30T17:09:51.547299+0000","last_peered":"2024-05-30T17:09:51.547299+0000","last_clean":"2024-05-30T17:09:51.547299+0000","last_became_active":"2024-05-30T16:17:35.212403+0000","last_became_peered":"2024-05-30T16:17:35.212403+0000","last_unstale":"2024-05-30T17:09:51.547299+0000","last_undegraded":"2024-05-30T17:09:51.547299+0000","last_fullsized":"2024-05-30T17:09:51.547299+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:52:56.999514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"0'0","reported_seq":52,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:16.721715+0000","last_change":"2024-05-30T16:17:31.192265+0000","last_active":"2024-05-30T17:09:16.721715+0000","last_peered":"2024-05-30T17:09:16.721715+0000","last_clean":"2024-05-30T17:09:16.721715+0000","last_became_active":"2024-05-30T16:17:31.191955+0000","last_became_peered":"2024-05-30T16:17:31.191955+0000","last_unstale":"2024-05-30T17:09:16.721715+0000","last_undegraded":"2024-05-30T17:09:16.721715+0000","last_fullsized":"2024-05-30T17:09:16.721715+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:16:24.140324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"262'8","reported_seq":83,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.559507+0000","last_change":"2024-05-30T16:12:50.820140+0000","last_active":"2024-05-30T17:11:21.559507+0000","last_peered":"2024-05-30T17:11:21.559507+0000","last_clean":"2024-05-30T17:11:21.559507+0000","last_became_active":"2024-05-30T16:12:50.819871+0000","last_became_peered":"2024-05-30T16:12:50.819871+0000","last_unstale":"2024-05-30T17:11:21.559507+0000","last_undegraded":"2024-05-30T17:11:21.559507+0000","last_fullsized":"2024-05-30T17:11:21.559507+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:14:08.269589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":28,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.554460+0000","last_change":"2024-05-30T16:34:55.883102+0000","last_active":"2024-05-30T17:10:21.554460+0000","last_peered":"2024-05-30T17:10:21.554460+0000","last_clean":"2024-05-30T17:10:21.554460+0000","last_became_active":"2024-05-30T16:34:55.882443+0000","last_became_peered":"2024-05-30T16:34:55.882443+0000","last_unstale":"2024-05-30T17:10:21.554460+0000","last_undegraded":"2024-05-30T17:10:21.554460+0000","last_fullsized":"2024-05-30T17:10:21.554460+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:01:28.745649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"262'3","reported_seq":54,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.564614+0000","last_change":"2024-05-30T16:17:35.248377+0000","last_active":"2024-05-30T17:09:56.564614+0000","last_peered":"2024-05-30T17:09:56.564614+0000","last_clean":"2024-05-30T17:09:56.564614+0000","last_became_active":"2024-05-30T16:17:35.247419+0000","last_became_peered":"2024-05-30T16:17:35.247419+0000","last_unstale":"2024-05-30T17:09:56.564614+0000","last_undegraded":"2024-05-30T17:09:56.564614+0000","last_fullsized":"2024-05-30T17:09:56.564614+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:09:58.043053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"262'2","reported_seq":58,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.543844+0000","last_change":"2024-05-30T16:17:31.189996+0000","last_active":"2024-05-30T17:09:36.543844+0000","last_peered":"2024-05-30T17:09:36.543844+0000","last_clean":"2024-05-30T17:09:36.543844+0000","last_became_active":"2024-05-30T16:17:31.189122+0000","last_became_peered":"2024-05-30T16:17:31.189122+0000","last_unstale":"2024-05-30T17:09:36.543844+0000","last_undegraded":"2024-05-30T17:09:36.543844+0000","last_fullsized":"2024-05-30T17:09:36.543844+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:53:04.697503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"265'53","reported_seq":201,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.584475+0000","last_change":"2024-05-30T16:12:50.821561+0000","last_active":"2024-05-30T17:11:21.584475+0000","last_peered":"2024-05-30T17:11:21.584475+0000","last_clean":"2024-05-30T17:11:21.584475+0000","last_became_active":"2024-05-30T16:12:50.809372+0000","last_became_peered":"2024-05-30T16:12:50.809372+0000","last_unstale":"2024-05-30T17:11:21.584475+0000","last_undegraded":"2024-05-30T17:11:21.584475+0000","last_fullsized":"2024-05-30T17:11:21.584475+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:22:57.030056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":186,"num_write":103,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:22.139649+0000","last_change":"2024-05-30T16:34:55.881085+0000","last_active":"2024-05-30T17:10:22.139649+0000","last_peered":"2024-05-30T17:10:22.139649+0000","last_clean":"2024-05-30T17:10:22.139649+0000","last_became_active":"2024-05-30T16:34:55.880627+0000","last_became_peered":"2024-05-30T16:34:55.880627+0000","last_unstale":"2024-05-30T17:10:22.139649+0000","last_undegraded":"2024-05-30T17:10:22.139649+0000","last_fullsized":"2024-05-30T17:10:22.139649+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:34:40.635399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"0'0","reported_seq":48,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:41.561793+0000","last_change":"2024-05-30T16:17:35.743808+0000","last_active":"2024-05-30T17:09:41.561793+0000","last_peered":"2024-05-30T17:09:41.561793+0000","last_clean":"2024-05-30T17:09:41.561793+0000","last_became_active":"2024-05-30T16:17:35.743036+0000","last_became_peered":"2024-05-30T16:17:35.743036+0000","last_unstale":"2024-05-30T17:09:41.561793+0000","last_undegraded":"2024-05-30T17:09:41.561793+0000","last_fullsized":"2024-05-30T17:09:41.561793+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:12:42.745946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"265'2","reported_seq":58,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:51.542424+0000","last_change":"2024-05-30T16:17:31.202885+0000","last_active":"2024-05-30T17:09:51.542424+0000","last_peered":"2024-05-30T17:09:51.542424+0000","last_clean":"2024-05-30T17:09:51.542424+0000","last_became_active":"2024-05-30T16:17:31.202064+0000","last_became_peered":"2024-05-30T16:17:31.202064+0000","last_unstale":"2024-05-30T17:09:51.542424+0000","last_undegraded":"2024-05-30T17:09:51.542424+0000","last_fullsized":"2024-05-30T17:09:51.542424+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:48:01.223505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2048,"num_write":7,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"265'34","reported_seq":149,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.559731+0000","last_change":"2024-05-30T16:12:50.816644+0000","last_active":"2024-05-30T17:11:21.559731+0000","last_peered":"2024-05-30T17:11:21.559731+0000","last_clean":"2024-05-30T17:11:21.559731+0000","last_became_active":"2024-05-30T16:12:50.815895+0000","last_became_peered":"2024-05-30T16:12:50.815895+0000","last_unstale":"2024-05-30T17:11:21.559731+0000","last_undegraded":"2024-05-30T17:11:21.559731+0000","last_fullsized":"2024-05-30T17:11:21.559731+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:33:57.609516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":160,"num_read_kb":111,"num_write":71,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":31,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:01.787658+0000","last_change":"2024-05-30T16:34:55.883831+0000","last_active":"2024-05-30T17:11:01.787658+0000","last_peered":"2024-05-30T17:11:01.787658+0000","last_clean":"2024-05-30T17:11:01.787658+0000","last_became_active":"2024-05-30T16:34:55.883112+0000","last_became_peered":"2024-05-30T16:34:55.883112+0000","last_unstale":"2024-05-30T17:11:01.787658+0000","last_undegraded":"2024-05-30T17:11:01.787658+0000","last_fullsized":"2024-05-30T17:11:01.787658+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:50:45.079747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"265'2","reported_seq":51,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.543941+0000","last_change":"2024-05-30T16:17:35.744284+0000","last_active":"2024-05-30T17:09:36.543941+0000","last_peered":"2024-05-30T17:09:36.543941+0000","last_clean":"2024-05-30T17:09:36.543941+0000","last_became_active":"2024-05-30T16:17:35.743608+0000","last_became_peered":"2024-05-30T16:17:35.743608+0000","last_unstale":"2024-05-30T17:09:36.543941+0000","last_undegraded":"2024-05-30T17:09:36.543941+0000","last_fullsized":"2024-05-30T17:09:36.543941+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:44:29.493440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":6144,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"0'0","reported_seq":53,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:32.128370+0000","last_change":"2024-05-30T16:17:31.189800+0000","last_active":"2024-05-30T17:09:32.128370+0000","last_peered":"2024-05-30T17:09:32.128370+0000","last_clean":"2024-05-30T17:09:32.128370+0000","last_became_active":"2024-05-30T16:17:31.188810+0000","last_became_peered":"2024-05-30T16:17:31.188810+0000","last_unstale":"2024-05-30T17:09:32.128370+0000","last_undegraded":"2024-05-30T17:09:32.128370+0000","last_fullsized":"2024-05-30T17:09:32.128370+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:27:55.332594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"265'18","reported_seq":111,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.570334+0000","last_change":"2024-05-30T16:12:50.827178+0000","last_active":"2024-05-30T17:11:26.570334+0000","last_peered":"2024-05-30T17:11:26.570334+0000","last_clean":"2024-05-30T17:11:26.570334+0000","last_became_active":"2024-05-30T16:12:50.826880+0000","last_became_peered":"2024-05-30T16:12:50.826880+0000","last_unstale":"2024-05-30T17:11:26.570334+0000","last_undegraded":"2024-05-30T17:11:26.570334+0000","last_fullsized":"2024-05-30T17:11:26.570334+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:22:01.392934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":61,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.735902+0000","last_change":"2024-05-30T16:34:55.886177+0000","last_active":"2024-05-30T17:10:21.735902+0000","last_peered":"2024-05-30T17:10:21.735902+0000","last_clean":"2024-05-30T17:10:21.735902+0000","last_became_active":"2024-05-30T16:34:55.885643+0000","last_became_peered":"2024-05-30T16:34:55.885643+0000","last_unstale":"2024-05-30T17:10:21.735902+0000","last_undegraded":"2024-05-30T17:10:21.735902+0000","last_fullsized":"2024-05-30T17:10:21.735902+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:27:38.430910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"262'2","reported_seq":51,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.726341+0000","last_change":"2024-05-30T16:17:35.248399+0000","last_active":"2024-05-30T17:09:36.726341+0000","last_peered":"2024-05-30T17:09:36.726341+0000","last_clean":"2024-05-30T17:09:36.726341+0000","last_became_active":"2024-05-30T16:17:35.247757+0000","last_became_peered":"2024-05-30T16:17:35.247757+0000","last_unstale":"2024-05-30T17:09:36.726341+0000","last_undegraded":"2024-05-30T17:09:36.726341+0000","last_fullsized":"2024-05-30T17:09:36.726341+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:22:36.927668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"0'0","reported_seq":53,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:31.542620+0000","last_change":"2024-05-30T16:17:31.203988+0000","last_active":"2024-05-30T17:09:31.542620+0000","last_peered":"2024-05-30T17:09:31.542620+0000","last_clean":"2024-05-30T17:09:31.542620+0000","last_became_active":"2024-05-30T16:17:31.203452+0000","last_became_peered":"2024-05-30T16:17:31.203452+0000","last_unstale":"2024-05-30T17:09:31.542620+0000","last_undegraded":"2024-05-30T17:09:31.542620+0000","last_fullsized":"2024-05-30T17:09:31.542620+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:50:29.995893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"265'37","reported_seq":162,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.789790+0000","last_change":"2024-05-30T16:12:50.812560+0000","last_active":"2024-05-30T17:11:11.789790+0000","last_peered":"2024-05-30T17:11:11.789790+0000","last_clean":"2024-05-30T17:11:11.789790+0000","last_became_active":"2024-05-30T16:12:50.811626+0000","last_became_peered":"2024-05-30T16:12:50.811626+0000","last_unstale":"2024-05-30T17:11:11.789790+0000","last_undegraded":"2024-05-30T17:11:11.789790+0000","last_fullsized":"2024-05-30T17:11:11.789790+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:11:40.631744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":184,"num_read_kb":132,"num_write":81,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"264'1","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.539587+0000","last_change":"2024-05-30T16:34:55.862486+0000","last_active":"2024-05-30T17:09:36.539587+0000","last_peered":"2024-05-30T17:09:36.539587+0000","last_clean":"2024-05-30T17:09:36.539587+0000","last_became_active":"2024-05-30T16:34:55.861767+0000","last_became_peered":"2024-05-30T16:34:55.861767+0000","last_unstale":"2024-05-30T17:09:36.539587+0000","last_undegraded":"2024-05-30T17:09:36.539587+0000","last_fullsized":"2024-05-30T17:09:36.539587+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:06:04.837971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"265'7","reported_seq":62,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:41.769918+0000","last_change":"2024-05-30T16:17:35.244195+0000","last_active":"2024-05-30T17:09:41.769918+0000","last_peered":"2024-05-30T17:09:41.769918+0000","last_clean":"2024-05-30T17:09:41.769918+0000","last_became_active":"2024-05-30T16:17:35.243435+0000","last_became_peered":"2024-05-30T16:17:35.243435+0000","last_unstale":"2024-05-30T17:09:41.769918+0000","last_undegraded":"2024-05-30T17:09:41.769918+0000","last_fullsized":"2024-05-30T17:09:41.769918+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:05:13.001812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":13312,"num_write":12,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"265'2","reported_seq":57,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:46.545952+0000","last_change":"2024-05-30T16:17:31.201111+0000","last_active":"2024-05-30T17:09:46.545952+0000","last_peered":"2024-05-30T17:09:46.545952+0000","last_clean":"2024-05-30T17:09:46.545952+0000","last_became_active":"2024-05-30T16:17:31.200663+0000","last_became_peered":"2024-05-30T16:17:31.200663+0000","last_unstale":"2024-05-30T17:09:46.545952+0000","last_undegraded":"2024-05-30T17:09:46.545952+0000","last_fullsized":"2024-05-30T17:09:46.545952+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:38:48.835894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"265'15","reported_seq":103,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:12.150755+0000","last_change":"2024-05-30T16:12:50.814553+0000","last_active":"2024-05-30T17:11:12.150755+0000","last_peered":"2024-05-30T17:11:12.150755+0000","last_clean":"2024-05-30T17:11:12.150755+0000","last_became_active":"2024-05-30T16:12:50.813741+0000","last_became_peered":"2024-05-30T16:12:50.813741+0000","last_unstale":"2024-05-30T17:11:12.150755+0000","last_undegraded":"2024-05-30T17:11:12.150755+0000","last_fullsized":"2024-05-30T17:11:12.150755+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:49:05.503730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":53,"num_write":33,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.560876+0000","last_change":"2024-05-30T16:34:55.859738+0000","last_active":"2024-05-30T17:09:36.560876+0000","last_peered":"2024-05-30T17:09:36.560876+0000","last_clean":"2024-05-30T17:09:36.560876+0000","last_became_active":"2024-05-30T16:34:55.858872+0000","last_became_peered":"2024-05-30T16:34:55.858872+0000","last_unstale":"2024-05-30T17:09:36.560876+0000","last_undegraded":"2024-05-30T17:09:36.560876+0000","last_fullsized":"2024-05-30T17:09:36.560876+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:48:21.475604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"0'0","reported_seq":49,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:42.130510+0000","last_change":"2024-05-30T16:17:35.227683+0000","last_active":"2024-05-30T17:09:42.130510+0000","last_peered":"2024-05-30T17:09:42.130510+0000","last_clean":"2024-05-30T17:09:42.130510+0000","last_became_active":"2024-05-30T16:17:35.226829+0000","last_became_peered":"2024-05-30T16:17:35.226829+0000","last_unstale":"2024-05-30T17:09:42.130510+0000","last_undegraded":"2024-05-30T17:09:42.130510+0000","last_fullsized":"2024-05-30T17:09:42.130510+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:15:44.751214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"0'0","reported_seq":54,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.543735+0000","last_change":"2024-05-30T16:17:31.205159+0000","last_active":"2024-05-30T17:09:36.543735+0000","last_peered":"2024-05-30T17:09:36.543735+0000","last_clean":"2024-05-30T17:09:36.543735+0000","last_became_active":"2024-05-30T16:17:31.203535+0000","last_became_peered":"2024-05-30T16:17:31.203535+0000","last_unstale":"2024-05-30T17:09:36.543735+0000","last_undegraded":"2024-05-30T17:09:36.543735+0000","last_fullsized":"2024-05-30T17:09:36.543735+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:42:29.611842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"265'36","reported_seq":157,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.747034+0000","last_change":"2024-05-30T16:12:50.832592+0000","last_active":"2024-05-30T17:11:11.747034+0000","last_peered":"2024-05-30T17:11:11.747034+0000","last_clean":"2024-05-30T17:11:11.747034+0000","last_became_active":"2024-05-30T16:12:50.831842+0000","last_became_peered":"2024-05-30T16:12:50.831842+0000","last_unstale":"2024-05-30T17:11:11.747034+0000","last_undegraded":"2024-05-30T17:11:11.747034+0000","last_fullsized":"2024-05-30T17:11:11.747034+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:08:18.471042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":177,"num_read_kb":125,"num_write":75,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.570209+0000","last_change":"2024-05-30T16:34:55.863472+0000","last_active":"2024-05-30T17:10:21.570209+0000","last_peered":"2024-05-30T17:10:21.570209+0000","last_clean":"2024-05-30T17:10:21.570209+0000","last_became_active":"2024-05-30T16:34:55.862709+0000","last_became_peered":"2024-05-30T16:34:55.862709+0000","last_unstale":"2024-05-30T17:10:21.570209+0000","last_undegraded":"2024-05-30T17:10:21.570209+0000","last_fullsized":"2024-05-30T17:10:21.570209+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:25:02.100900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"265'2","reported_seq":50,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:41.537534+0000","last_change":"2024-05-30T16:17:35.241271+0000","last_active":"2024-05-30T17:09:41.537534+0000","last_peered":"2024-05-30T17:09:41.537534+0000","last_clean":"2024-05-30T17:09:41.537534+0000","last_became_active":"2024-05-30T16:17:35.240472+0000","last_became_peered":"2024-05-30T16:17:35.240472+0000","last_unstale":"2024-05-30T17:09:41.537534+0000","last_undegraded":"2024-05-30T17:09:41.537534+0000","last_fullsized":"2024-05-30T17:09:41.537534+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:59:40.865645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"265'4","reported_seq":62,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.769181+0000","last_change":"2024-05-30T16:17:31.196715+0000","last_active":"2024-05-30T17:09:36.769181+0000","last_peered":"2024-05-30T17:09:36.769181+0000","last_clean":"2024-05-30T17:09:36.769181+0000","last_became_active":"2024-05-30T16:17:31.187958+0000","last_became_peered":"2024-05-30T16:17:31.187958+0000","last_unstale":"2024-05-30T17:09:36.769181+0000","last_undegraded":"2024-05-30T17:09:36.769181+0000","last_fullsized":"2024-05-30T17:09:36.769181+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:26:54.324514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485760,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":10240,"num_write":14,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"265'18","reported_seq":112,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.557030+0000","last_change":"2024-05-30T16:12:50.823849+0000","last_active":"2024-05-30T17:11:11.557030+0000","last_peered":"2024-05-30T17:11:11.557030+0000","last_clean":"2024-05-30T17:11:11.557030+0000","last_became_active":"2024-05-30T16:12:50.823090+0000","last_became_peered":"2024-05-30T16:12:50.823090+0000","last_unstale":"2024-05-30T17:11:11.557030+0000","last_undegraded":"2024-05-30T17:11:11.557030+0000","last_fullsized":"2024-05-30T17:11:11.557030+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:38:28.480174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":69,"num_write":35,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:31.556926+0000","last_change":"2024-05-30T16:34:55.891440+0000","last_active":"2024-05-30T17:10:31.556926+0000","last_peered":"2024-05-30T17:10:31.556926+0000","last_clean":"2024-05-30T17:10:31.556926+0000","last_became_active":"2024-05-30T16:34:55.890688+0000","last_became_peered":"2024-05-30T16:34:55.890688+0000","last_unstale":"2024-05-30T17:10:31.556926+0000","last_undegraded":"2024-05-30T17:10:31.556926+0000","last_fullsized":"2024-05-30T17:10:31.556926+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:05:55.238651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"265'7","reported_seq":65,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:41.727328+0000","last_change":"2024-05-30T16:17:35.238582+0000","last_active":"2024-05-30T17:09:41.727328+0000","last_peered":"2024-05-30T17:09:41.727328+0000","last_clean":"2024-05-30T17:09:41.727328+0000","last_became_active":"2024-05-30T16:17:35.237031+0000","last_became_peered":"2024-05-30T16:17:35.237031+0000","last_unstale":"2024-05-30T17:09:41.727328+0000","last_undegraded":"2024-05-30T17:09:41.727328+0000","last_fullsized":"2024-05-30T17:09:41.727328+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:11:17.043035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485760,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":16,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"262'5","reported_seq":65,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.536379+0000","last_change":"2024-05-30T16:17:31.204721+0000","last_active":"2024-05-30T17:09:36.536379+0000","last_peered":"2024-05-30T17:09:36.536379+0000","last_clean":"2024-05-30T17:09:36.536379+0000","last_became_active":"2024-05-30T16:17:31.203871+0000","last_became_peered":"2024-05-30T16:17:31.203871+0000","last_unstale":"2024-05-30T17:09:36.536379+0000","last_undegraded":"2024-05-30T17:09:36.536379+0000","last_fullsized":"2024-05-30T17:09:36.536379+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:41:32.481769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":4098,"num_write":9,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"265'24","reported_seq":133,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:06.565051+0000","last_change":"2024-05-30T16:12:50.827488+0000","last_active":"2024-05-30T17:11:06.565051+0000","last_peered":"2024-05-30T17:11:06.565051+0000","last_clean":"2024-05-30T17:11:06.565051+0000","last_became_active":"2024-05-30T16:12:50.827078+0000","last_became_peered":"2024-05-30T16:12:50.827078+0000","last_unstale":"2024-05-30T17:11:06.565051+0000","last_undegraded":"2024-05-30T17:11:06.565051+0000","last_fullsized":"2024-05-30T17:11:06.565051+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:30:01.069671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":93,"num_write":46,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"265'165","reported_seq":776,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.569071+0000","last_change":"2024-05-30T15:54:16.662201+0000","last_active":"2024-05-30T17:11:21.569071+0000","last_peered":"2024-05-30T17:11:21.569071+0000","last_clean":"2024-05-30T17:11:21.569071+0000","last_became_active":"2024-05-30T15:54:16.661334+0000","last_became_peered":"2024-05-30T15:54:16.661334+0000","last_unstale":"2024-05-30T17:11:21.569071+0000","last_undegraded":"2024-05-30T17:11:21.569071+0000","last_fullsized":"2024-05-30T17:11:21.569071+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":100,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:14:32.854488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2159792,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":347,"num_read_kb":7638,"num_write":1029,"num_write_kb":24692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"265'436","reported_seq":2069,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:33.884897+0000","last_change":"2024-05-30T15:54:18.669748+0000","last_active":"2024-05-30T17:12:33.884897+0000","last_peered":"2024-05-30T17:12:33.884897+0000","last_clean":"2024-05-30T17:12:33.884897+0000","last_became_active":"2024-05-30T15:54:18.669016+0000","last_became_peered":"2024-05-30T15:54:18.669016+0000","last_unstale":"2024-05-30T17:12:33.884897+0000","last_undegraded":"2024-05-30T17:12:33.884897+0000","last_fullsized":"2024-05-30T17:12:33.884897+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":400,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:26:42.111749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2479,"num_read_kb":2366,"num_write":506,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"265'232","reported_seq":887,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.792186+0000","last_change":"2024-05-30T15:54:16.653102+0000","last_active":"2024-05-30T17:11:21.792186+0000","last_peered":"2024-05-30T17:11:21.792186+0000","last_clean":"2024-05-30T17:11:21.792186+0000","last_became_active":"2024-05-30T15:54:16.652724+0000","last_became_peered":"2024-05-30T15:54:16.652724+0000","last_unstale":"2024-05-30T17:11:21.792186+0000","last_undegraded":"2024-05-30T17:11:21.792186+0000","last_fullsized":"2024-05-30T17:11:21.792186+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":200,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:48:02.488417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16999710,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":552,"num_read_kb":8631,"num_write":1279,"num_write_kb":24636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"265'758","reported_seq":2113,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:59.057740+0000","last_change":"2024-05-30T15:54:18.649460+0000","last_active":"2024-05-30T17:11:59.057740+0000","last_peered":"2024-05-30T17:11:59.057740+0000","last_clean":"2024-05-30T17:11:59.057740+0000","last_became_active":"2024-05-30T15:54:18.648675+0000","last_became_peered":"2024-05-30T15:54:18.648675+0000","last_unstale":"2024-05-30T17:11:59.057740+0000","last_undegraded":"2024-05-30T17:11:59.057740+0000","last_fullsized":"2024-05-30T17:11:59.057740+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":700,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:30:51.616348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3292,"num_read_kb":2905,"num_write":1031,"num_write_kb":415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"265'200","reported_seq":832,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:57.487564+0000","last_change":"2024-05-30T15:54:16.633538+0000","last_active":"2024-05-30T17:11:57.487564+0000","last_peered":"2024-05-30T17:11:57.487564+0000","last_clean":"2024-05-30T17:11:57.487564+0000","last_became_active":"2024-05-30T15:54:16.633011+0000","last_became_peered":"2024-05-30T15:54:16.633011+0000","last_unstale":"2024-05-30T17:11:57.487564+0000","last_undegraded":"2024-05-30T17:11:57.487564+0000","last_fullsized":"2024-05-30T17:11:57.487564+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":100,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:22:33.463154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17125665,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":396,"num_read_kb":43854,"num_write":995,"num_write_kb":74051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"265'746","reported_seq":2144,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:33.888418+0000","last_change":"2024-05-30T15:54:18.651676+0000","last_active":"2024-05-30T17:12:33.888418+0000","last_peered":"2024-05-30T17:12:33.888418+0000","last_clean":"2024-05-30T17:12:33.888418+0000","last_became_active":"2024-05-30T15:54:18.649608+0000","last_became_peered":"2024-05-30T15:54:18.649608+0000","last_unstale":"2024-05-30T17:12:33.888418+0000","last_undegraded":"2024-05-30T17:12:33.888418+0000","last_fullsized":"2024-05-30T17:12:33.888418+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":700,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:29:56.699785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3275,"num_read_kb":2909,"num_write":996,"num_write_kb":378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"265'152","reported_seq":790,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:44.320980+0000","last_change":"2024-05-30T15:54:16.653236+0000","last_active":"2024-05-30T17:11:44.320980+0000","last_peered":"2024-05-30T17:11:44.320980+0000","last_clean":"2024-05-30T17:11:44.320980+0000","last_became_active":"2024-05-30T15:54:16.652315+0000","last_became_peered":"2024-05-30T15:54:16.652315+0000","last_unstale":"2024-05-30T17:11:44.320980+0000","last_undegraded":"2024-05-30T17:11:44.320980+0000","last_fullsized":"2024-05-30T17:11:44.320980+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":100,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:26:57.491124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1336636,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":435,"num_read_kb":3484,"num_write":909,"num_write_kb":6490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"265'487","reported_seq":1962,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:52.698536+0000","last_change":"2024-05-30T15:54:18.652741+0000","last_active":"2024-05-30T17:12:52.698536+0000","last_peered":"2024-05-30T17:12:52.698536+0000","last_clean":"2024-05-30T17:12:52.698536+0000","last_became_active":"2024-05-30T15:54:18.652451+0000","last_became_peered":"2024-05-30T15:54:18.652451+0000","last_unstale":"2024-05-30T17:12:52.698536+0000","last_undegraded":"2024-05-30T17:12:52.698536+0000","last_fullsized":"2024-05-30T17:12:52.698536+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":400,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:50:10.811052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2387,"num_read_kb":2188,"num_write":616,"num_write_kb":168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"265'242","reported_seq":956,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:58.054740+0000","last_change":"2024-05-30T15:54:16.659161+0000","last_active":"2024-05-30T17:12:58.054740+0000","last_peered":"2024-05-30T17:12:58.054740+0000","last_clean":"2024-05-30T17:12:58.054740+0000","last_became_active":"2024-05-30T15:54:16.658338+0000","last_became_peered":"2024-05-30T15:54:16.658338+0000","last_unstale":"2024-05-30T17:12:58.054740+0000","last_undegraded":"2024-05-30T17:12:58.054740+0000","last_fullsized":"2024-05-30T17:12:58.054740+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":200,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:10:31.510086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11588767,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":827,"num_read_kb":24681,"num_write":1243,"num_write_kb":34031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"265'370","reported_seq":1862,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:12.349707+0000","last_change":"2024-05-30T15:54:18.669822+0000","last_active":"2024-05-30T17:11:12.349707+0000","last_peered":"2024-05-30T17:11:12.349707+0000","last_clean":"2024-05-30T17:11:12.349707+0000","last_became_active":"2024-05-30T15:54:18.669017+0000","last_became_peered":"2024-05-30T15:54:18.669017+0000","last_unstale":"2024-05-30T17:11:12.349707+0000","last_undegraded":"2024-05-30T17:11:12.349707+0000","last_fullsized":"2024-05-30T17:11:12.349707+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":300,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:33:40.482356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1986,"num_read_kb":1906,"num_write":419,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"265'192","reported_seq":817,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:17.124503+0000","last_change":"2024-05-30T15:54:16.673757+0000","last_active":"2024-05-30T17:09:17.124503+0000","last_peered":"2024-05-30T17:09:17.124503+0000","last_clean":"2024-05-30T17:09:17.124503+0000","last_became_active":"2024-05-30T15:54:16.671120+0000","last_became_peered":"2024-05-30T15:54:16.671120+0000","last_unstale":"2024-05-30T17:09:17.124503+0000","last_undegraded":"2024-05-30T17:09:17.124503+0000","last_fullsized":"2024-05-30T17:09:17.124503+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":100,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:07:40.834702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5596650,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":434,"num_read_kb":4718,"num_write":1097,"num_write_kb":20583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"265'725","reported_seq":2201,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.709270+0000","last_change":"2024-05-30T15:54:18.651587+0000","last_active":"2024-05-30T17:13:20.709270+0000","last_peered":"2024-05-30T17:13:20.709270+0000","last_clean":"2024-05-30T17:13:20.709270+0000","last_became_active":"2024-05-30T15:54:18.650543+0000","last_became_peered":"2024-05-30T15:54:18.650543+0000","last_unstale":"2024-05-30T17:13:20.709270+0000","last_undegraded":"2024-05-30T17:13:20.709270+0000","last_fullsized":"2024-05-30T17:13:20.709270+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":700,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:51.313043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3282,"num_read_kb":2952,"num_write":973,"num_write_kb":356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"265'188","reported_seq":828,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:18.810687+0000","last_change":"2024-05-30T15:54:16.679393+0000","last_active":"2024-05-30T17:12:18.810687+0000","last_peered":"2024-05-30T17:12:18.810687+0000","last_clean":"2024-05-30T17:12:18.810687+0000","last_became_active":"2024-05-30T15:54:16.678799+0000","last_became_peered":"2024-05-30T15:54:16.678799+0000","last_unstale":"2024-05-30T17:12:18.810687+0000","last_undegraded":"2024-05-30T17:12:18.810687+0000","last_fullsized":"2024-05-30T17:12:18.810687+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":100,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:07:54.614137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5740256,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":463,"num_read_kb":33008,"num_write":1086,"num_write_kb":27021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"265'850","reported_seq":2539,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.703253+0000","last_change":"2024-05-30T15:54:18.657813+0000","last_active":"2024-05-30T17:13:20.703253+0000","last_peered":"2024-05-30T17:13:20.703253+0000","last_clean":"2024-05-30T17:13:20.703253+0000","last_became_active":"2024-05-30T15:54:18.656475+0000","last_became_peered":"2024-05-30T15:54:18.656475+0000","last_unstale":"2024-05-30T17:13:20.703253+0000","last_undegraded":"2024-05-30T17:13:20.703253+0000","last_fullsized":"2024-05-30T17:13:20.703253+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":800,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:08:46.768265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4088,"num_read_kb":3692,"num_write":1166,"num_write_kb":472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"265'139","reported_seq":735,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:16.558759+0000","last_change":"2024-05-30T15:54:16.669564+0000","last_active":"2024-05-30T17:09:16.558759+0000","last_peered":"2024-05-30T17:09:16.558759+0000","last_clean":"2024-05-30T17:09:16.558759+0000","last_became_active":"2024-05-30T15:54:16.669095+0000","last_became_peered":"2024-05-30T15:54:16.669095+0000","last_unstale":"2024-05-30T17:09:16.558759+0000","last_undegraded":"2024-05-30T17:09:16.558759+0000","last_fullsized":"2024-05-30T17:09:16.558759+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":100,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:34:08.005236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485923,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":302,"num_read_kb":17751,"num_write":816,"num_write_kb":30378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"265'369","reported_seq":1790,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.560600+0000","last_change":"2024-05-30T15:54:18.653529+0000","last_active":"2024-05-30T17:11:26.560600+0000","last_peered":"2024-05-30T17:11:26.560600+0000","last_clean":"2024-05-30T17:11:26.560600+0000","last_became_active":"2024-05-30T15:54:18.653189+0000","last_became_peered":"2024-05-30T15:54:18.653189+0000","last_unstale":"2024-05-30T17:11:26.560600+0000","last_undegraded":"2024-05-30T17:11:26.560600+0000","last_fullsized":"2024-05-30T17:11:26.560600+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":300,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:18:04.600068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1969,"num_read_kb":1855,"num_write":435,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"265'336","reported_seq":1764,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:33.888003+0000","last_change":"2024-05-30T15:54:18.652246+0000","last_active":"2024-05-30T17:12:33.888003+0000","last_peered":"2024-05-30T17:12:33.888003+0000","last_clean":"2024-05-30T17:12:33.888003+0000","last_became_active":"2024-05-30T15:54:18.649994+0000","last_became_peered":"2024-05-30T15:54:18.649994+0000","last_unstale":"2024-05-30T17:12:33.888003+0000","last_undegraded":"2024-05-30T17:12:33.888003+0000","last_fullsized":"2024-05-30T17:12:33.888003+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:39:30.273131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1976,"num_read_kb":1893,"num_write":388,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"265'206","reported_seq":871,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:11.306928+0000","last_change":"2024-05-30T15:54:16.650771+0000","last_active":"2024-05-30T17:12:11.306928+0000","last_peered":"2024-05-30T17:12:11.306928+0000","last_clean":"2024-05-30T17:12:11.306928+0000","last_became_active":"2024-05-30T15:54:16.649872+0000","last_became_peered":"2024-05-30T15:54:16.649872+0000","last_unstale":"2024-05-30T17:12:11.306928+0000","last_undegraded":"2024-05-30T17:12:11.306928+0000","last_fullsized":"2024-05-30T17:12:11.306928+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":100,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:01:29.595780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16404427,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":504,"num_read_kb":59018,"num_write":1160,"num_write_kb":23092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"265'1525","reported_seq":3130,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:28.258461+0000","last_change":"2024-05-30T15:54:18.649618+0000","last_active":"2024-05-30T17:12:28.258461+0000","last_peered":"2024-05-30T17:12:28.258461+0000","last_clean":"2024-05-30T17:12:28.258461+0000","last_became_active":"2024-05-30T15:54:18.648860+0000","last_became_peered":"2024-05-30T15:54:18.648860+0000","last_unstale":"2024-05-30T17:12:28.258461+0000","last_undegraded":"2024-05-30T17:12:28.258461+0000","last_fullsized":"2024-05-30T17:12:28.258461+0000","mapping_epoch":24,"log_start":"265'1500","ondisk_log_start":"265'1500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":1500,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:09:51.963499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6132,"num_read_kb":5297,"num_write":2138,"num_write_kb":912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"265'218","reported_seq":1080,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:28.769161+0000","last_change":"2024-05-30T15:54:16.687260+0000","last_active":"2024-05-30T17:12:28.769161+0000","last_peered":"2024-05-30T17:12:28.769161+0000","last_clean":"2024-05-30T17:12:28.769161+0000","last_became_active":"2024-05-30T15:54:16.686847+0000","last_became_peered":"2024-05-30T15:54:16.686847+0000","last_unstale":"2024-05-30T17:12:28.769161+0000","last_undegraded":"2024-05-30T17:12:28.769161+0000","last_fullsized":"2024-05-30T17:12:28.769161+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":200,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:27:56.085645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2781302,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":922,"num_read_kb":24537,"num_write":1230,"num_write_kb":54861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"265'1164","reported_seq":3003,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:10.547968+0000","last_change":"2024-05-30T15:54:18.666133+0000","last_active":"2024-05-30T17:13:10.547968+0000","last_peered":"2024-05-30T17:13:10.547968+0000","last_clean":"2024-05-30T17:13:10.547968+0000","last_became_active":"2024-05-30T15:54:18.665302+0000","last_became_peered":"2024-05-30T15:54:18.665302+0000","last_unstale":"2024-05-30T17:13:10.547968+0000","last_undegraded":"2024-05-30T17:13:10.547968+0000","last_fullsized":"2024-05-30T17:13:10.547968+0000","mapping_epoch":24,"log_start":"265'1100","ondisk_log_start":"265'1100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":1100,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:19:10.742000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5292,"num_read_kb":4714,"num_write":1593,"num_write_kb":642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"265'194","reported_seq":915,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:16.092608+0000","last_change":"2024-05-30T15:54:16.669114+0000","last_active":"2024-05-30T17:09:16.092608+0000","last_peered":"2024-05-30T17:09:16.092608+0000","last_clean":"2024-05-30T17:09:16.092608+0000","last_became_active":"2024-05-30T15:54:16.668438+0000","last_became_peered":"2024-05-30T15:54:16.668438+0000","last_unstale":"2024-05-30T17:09:16.092608+0000","last_undegraded":"2024-05-30T17:09:16.092608+0000","last_fullsized":"2024-05-30T17:09:16.092608+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":100,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:55:42.428949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10487886,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":915,"num_read_kb":8276,"num_write":1170,"num_write_kb":35310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"265'768","reported_seq":2282,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.649495+0000","last_change":"2024-05-30T15:54:18.651494+0000","last_active":"2024-05-30T17:13:20.649495+0000","last_peered":"2024-05-30T17:13:20.649495+0000","last_clean":"2024-05-30T17:13:20.649495+0000","last_became_active":"2024-05-30T15:54:18.651197+0000","last_became_peered":"2024-05-30T15:54:18.651197+0000","last_unstale":"2024-05-30T17:13:20.649495+0000","last_undegraded":"2024-05-30T17:13:20.649495+0000","last_fullsized":"2024-05-30T17:13:20.649495+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":700,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:01:31.985221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3544,"num_read_kb":3171,"num_write":1051,"num_write_kb":404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"265'181","reported_seq":833,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:11.720441+0000","last_change":"2024-05-30T15:54:16.677382+0000","last_active":"2024-05-30T17:09:11.720441+0000","last_peered":"2024-05-30T17:09:11.720441+0000","last_clean":"2024-05-30T17:09:11.720441+0000","last_became_active":"2024-05-30T15:54:16.676703+0000","last_became_peered":"2024-05-30T15:54:16.676703+0000","last_unstale":"2024-05-30T17:09:11.720441+0000","last_undegraded":"2024-05-30T17:09:11.720441+0000","last_fullsized":"2024-05-30T17:09:11.720441+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":100,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:39:55.585070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16123359,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":357,"num_read_kb":73313,"num_write":1053,"num_write_kb":47694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"265'372","reported_seq":1824,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.655751+0000","last_change":"2024-05-30T15:54:18.650530+0000","last_active":"2024-05-30T17:12:44.655751+0000","last_peered":"2024-05-30T17:12:44.655751+0000","last_clean":"2024-05-30T17:12:44.655751+0000","last_became_active":"2024-05-30T15:54:18.650255+0000","last_became_peered":"2024-05-30T15:54:18.650255+0000","last_unstale":"2024-05-30T17:12:44.655751+0000","last_undegraded":"2024-05-30T17:12:44.655751+0000","last_fullsized":"2024-05-30T17:12:44.655751+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":300,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:22:55.711498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1969,"num_read_kb":1871,"num_write":435,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"265'200","reported_seq":851,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:11.557730+0000","last_change":"2024-05-30T15:54:16.665743+0000","last_active":"2024-05-30T17:09:11.557730+0000","last_peered":"2024-05-30T17:09:11.557730+0000","last_clean":"2024-05-30T17:09:11.557730+0000","last_became_active":"2024-05-30T15:54:16.665344+0000","last_became_peered":"2024-05-30T15:54:16.665344+0000","last_unstale":"2024-05-30T17:09:11.557730+0000","last_undegraded":"2024-05-30T17:09:11.557730+0000","last_fullsized":"2024-05-30T17:09:11.557730+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":100,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:11:01.132558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17335132,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":474,"num_read_kb":7750,"num_write":1074,"num_write_kb":36544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"265'375","reported_seq":1800,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:15.191377+0000","last_change":"2024-05-30T15:54:18.645251+0000","last_active":"2024-05-30T17:12:15.191377+0000","last_peered":"2024-05-30T17:12:15.191377+0000","last_clean":"2024-05-30T17:12:15.191377+0000","last_became_active":"2024-05-30T15:54:18.643825+0000","last_became_peered":"2024-05-30T15:54:18.643825+0000","last_unstale":"2024-05-30T17:12:15.191377+0000","last_undegraded":"2024-05-30T17:12:15.191377+0000","last_fullsized":"2024-05-30T17:12:15.191377+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":300,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:54:45.495400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1936,"num_read_kb":1833,"num_write":434,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"265'234","reported_seq":905,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:16.555742+0000","last_change":"2024-05-30T15:54:16.638560+0000","last_active":"2024-05-30T17:09:16.555742+0000","last_peered":"2024-05-30T17:09:16.555742+0000","last_clean":"2024-05-30T17:09:16.555742+0000","last_became_active":"2024-05-30T15:54:16.637928+0000","last_became_peered":"2024-05-30T15:54:16.637928+0000","last_unstale":"2024-05-30T17:09:16.555742+0000","last_undegraded":"2024-05-30T17:09:16.555742+0000","last_fullsized":"2024-05-30T17:09:16.555742+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":200,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:28:17.148350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11620077,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":522,"num_read_kb":65823,"num_write":1177,"num_write_kb":32044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"265'176","reported_seq":700,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.999380+0000","last_change":"2024-05-30T15:54:26.701638+0000","last_active":"2024-05-30T17:13:20.999380+0000","last_peered":"2024-05-30T17:13:20.999380+0000","last_clean":"2024-05-30T17:13:20.999380+0000","last_became_active":"2024-05-30T15:54:26.700957+0000","last_became_peered":"2024-05-30T15:54:26.700957+0000","last_unstale":"2024-05-30T17:13:20.999380+0000","last_undegraded":"2024-05-30T17:13:20.999380+0000","last_fullsized":"2024-05-30T17:13:20.999380+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":100,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:45:03.540847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":270,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":132,"num_read_kb":126,"num_write":386,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"265'494","reported_seq":2021,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.600563+0000","last_change":"2024-05-30T15:54:18.657002+0000","last_active":"2024-05-30T17:12:44.600563+0000","last_peered":"2024-05-30T17:12:44.600563+0000","last_clean":"2024-05-30T17:12:44.600563+0000","last_became_active":"2024-05-30T15:54:18.652712+0000","last_became_peered":"2024-05-30T15:54:18.652712+0000","last_unstale":"2024-05-30T17:12:44.600563+0000","last_undegraded":"2024-05-30T17:12:44.600563+0000","last_fullsized":"2024-05-30T17:12:44.600563+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":400,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:27:30.279867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2579,"num_read_kb":2377,"num_write":668,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"265'189","reported_seq":793,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:36.550925+0000","last_change":"2024-05-30T15:55:54.564987+0000","last_active":"2024-05-30T17:10:36.550925+0000","last_peered":"2024-05-30T17:10:36.550925+0000","last_clean":"2024-05-30T17:10:36.550925+0000","last_became_active":"2024-05-30T15:55:54.563874+0000","last_became_peered":"2024-05-30T15:55:54.563874+0000","last_unstale":"2024-05-30T17:10:36.550925+0000","last_undegraded":"2024-05-30T17:10:36.550925+0000","last_fullsized":"2024-05-30T17:10:36.550925+0000","mapping_epoch":37,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:22:56.400029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079909,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":469,"num_read_kb":1589,"num_write":1088,"num_write_kb":27979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:17.124719+0000","last_change":"2024-05-30T15:54:20.659788+0000","last_active":"2024-05-30T17:09:17.124719+0000","last_peered":"2024-05-30T17:09:17.124719+0000","last_clean":"2024-05-30T17:09:17.124719+0000","last_became_active":"2024-05-30T15:54:20.658937+0000","last_became_peered":"2024-05-30T15:54:20.658937+0000","last_unstale":"2024-05-30T17:09:17.124719+0000","last_undegraded":"2024-05-30T17:09:17.124719+0000","last_fullsized":"2024-05-30T17:09:17.124719+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:41:24.097539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"265'198","reported_seq":820,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:46.590604+0000","last_change":"2024-05-30T15:54:22.668221+0000","last_active":"2024-05-30T17:11:46.590604+0000","last_peered":"2024-05-30T17:11:46.590604+0000","last_clean":"2024-05-30T17:11:46.590604+0000","last_became_active":"2024-05-30T15:54:22.667496+0000","last_became_peered":"2024-05-30T15:54:22.667496+0000","last_unstale":"2024-05-30T17:11:46.590604+0000","last_undegraded":"2024-05-30T17:11:46.590604+0000","last_fullsized":"2024-05-30T17:11:46.590604+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":100,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:54:14.364376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9413,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":276,"num_read_kb":334,"num_write":164,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":498,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.754793+0000","last_change":"2024-05-30T15:54:24.675232+0000","last_active":"2024-05-30T17:09:21.754793+0000","last_peered":"2024-05-30T17:09:21.754793+0000","last_clean":"2024-05-30T17:09:21.754793+0000","last_became_active":"2024-05-30T15:54:24.674491+0000","last_became_peered":"2024-05-30T15:54:24.674491+0000","last_unstale":"2024-05-30T17:09:21.754793+0000","last_undegraded":"2024-05-30T17:09:21.754793+0000","last_fullsized":"2024-05-30T17:09:21.754793+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:30:36.269404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"265'194","reported_seq":769,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.643659+0000","last_change":"2024-05-30T15:54:26.687210+0000","last_active":"2024-05-30T17:12:44.643659+0000","last_peered":"2024-05-30T17:12:44.643659+0000","last_clean":"2024-05-30T17:12:44.643659+0000","last_became_active":"2024-05-30T15:54:26.686430+0000","last_became_peered":"2024-05-30T15:54:26.686430+0000","last_unstale":"2024-05-30T17:12:44.643659+0000","last_undegraded":"2024-05-30T17:12:44.643659+0000","last_fullsized":"2024-05-30T17:12:44.643659+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":100,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:45:58.373725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":824,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":238,"num_read_kb":223,"num_write":274,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"265'510","reported_seq":2160,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.681680+0000","last_change":"2024-05-30T15:54:18.662470+0000","last_active":"2024-05-30T17:13:18.681680+0000","last_peered":"2024-05-30T17:13:18.681680+0000","last_clean":"2024-05-30T17:13:18.681680+0000","last_became_active":"2024-05-30T15:54:18.661762+0000","last_became_peered":"2024-05-30T15:54:18.661762+0000","last_unstale":"2024-05-30T17:13:18.681680+0000","last_undegraded":"2024-05-30T17:13:18.681680+0000","last_fullsized":"2024-05-30T17:13:18.681680+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":400,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:39:51.492340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2657,"num_read_kb":2481,"num_write":627,"num_write_kb":155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"265'204","reported_seq":860,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.582351+0000","last_change":"2024-05-30T15:54:16.639898+0000","last_active":"2024-05-30T17:11:16.582351+0000","last_peered":"2024-05-30T17:11:16.582351+0000","last_clean":"2024-05-30T17:11:16.582351+0000","last_became_active":"2024-05-30T15:54:16.639284+0000","last_became_peered":"2024-05-30T15:54:16.639284+0000","last_unstale":"2024-05-30T17:11:16.582351+0000","last_undegraded":"2024-05-30T17:11:16.582351+0000","last_fullsized":"2024-05-30T17:11:16.582351+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":100,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:33:35.045719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13950104,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":439,"num_read_kb":15653,"num_write":1153,"num_write_kb":64581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":509,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.560530+0000","last_change":"2024-05-30T15:54:20.666014+0000","last_active":"2024-05-30T17:09:21.560530+0000","last_peered":"2024-05-30T17:09:21.560530+0000","last_clean":"2024-05-30T17:09:21.560530+0000","last_became_active":"2024-05-30T15:54:20.665488+0000","last_became_peered":"2024-05-30T15:54:20.665488+0000","last_unstale":"2024-05-30T17:09:21.560530+0000","last_undegraded":"2024-05-30T17:09:21.560530+0000","last_fullsized":"2024-05-30T17:09:21.560530+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:23:46.612023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"265'102","reported_seq":653,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.723144+0000","last_change":"2024-05-30T15:54:22.666790+0000","last_active":"2024-05-30T17:09:21.723144+0000","last_peered":"2024-05-30T17:09:21.723144+0000","last_clean":"2024-05-30T17:09:21.723144+0000","last_became_active":"2024-05-30T15:54:22.666057+0000","last_became_peered":"2024-05-30T15:54:22.666057+0000","last_unstale":"2024-05-30T17:09:21.723144+0000","last_undegraded":"2024-05-30T17:09:21.723144+0000","last_fullsized":"2024-05-30T17:09:21.723144+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:49:29.070169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":113,"num_write":76,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":507,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.723119+0000","last_change":"2024-05-30T15:54:24.687346+0000","last_active":"2024-05-30T17:09:21.723119+0000","last_peered":"2024-05-30T17:09:21.723119+0000","last_clean":"2024-05-30T17:09:21.723119+0000","last_became_active":"2024-05-30T15:54:24.686595+0000","last_became_peered":"2024-05-30T15:54:24.686595+0000","last_unstale":"2024-05-30T17:09:21.723119+0000","last_undegraded":"2024-05-30T17:09:21.723119+0000","last_fullsized":"2024-05-30T17:09:21.723119+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:49:42.604820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"265'99","reported_seq":612,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.102373+0000","last_change":"2024-05-30T15:54:26.689306+0000","last_active":"2024-05-30T17:13:18.102373+0000","last_peered":"2024-05-30T17:13:18.102373+0000","last_clean":"2024-05-30T17:13:18.102373+0000","last_became_active":"2024-05-30T15:54:26.688376+0000","last_became_peered":"2024-05-30T15:54:26.688376+0000","last_unstale":"2024-05-30T17:13:18.102373+0000","last_undegraded":"2024-05-30T17:13:18.102373+0000","last_fullsized":"2024-05-30T17:13:18.102373+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:10:30.878895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":271,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":103,"num_read_kb":102,"num_write":235,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"265'462","reported_seq":2011,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.638007+0000","last_change":"2024-05-30T15:54:18.658772+0000","last_active":"2024-05-30T17:13:20.638007+0000","last_peered":"2024-05-30T17:13:20.638007+0000","last_clean":"2024-05-30T17:13:20.638007+0000","last_became_active":"2024-05-30T15:54:18.657540+0000","last_became_peered":"2024-05-30T15:54:18.657540+0000","last_unstale":"2024-05-30T17:13:20.638007+0000","last_undegraded":"2024-05-30T17:13:20.638007+0000","last_fullsized":"2024-05-30T17:13:20.638007+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":400,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:48:10.156295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2554,"num_read_kb":2362,"num_write":592,"num_write_kb":196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"265'212","reported_seq":873,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:58.050061+0000","last_change":"2024-05-30T15:54:16.679183+0000","last_active":"2024-05-30T17:12:58.050061+0000","last_peered":"2024-05-30T17:12:58.050061+0000","last_clean":"2024-05-30T17:12:58.050061+0000","last_became_active":"2024-05-30T15:54:16.678659+0000","last_became_peered":"2024-05-30T15:54:16.678659+0000","last_unstale":"2024-05-30T17:12:58.050061+0000","last_undegraded":"2024-05-30T17:12:58.050061+0000","last_fullsized":"2024-05-30T17:12:58.050061+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:12:32.311283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11885555,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":520,"num_read_kb":29194,"num_write":1285,"num_write_kb":17953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":508,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:16.559045+0000","last_change":"2024-05-30T15:54:20.654060+0000","last_active":"2024-05-30T17:09:16.559045+0000","last_peered":"2024-05-30T17:09:16.559045+0000","last_clean":"2024-05-30T17:09:16.559045+0000","last_became_active":"2024-05-30T15:54:20.653316+0000","last_became_peered":"2024-05-30T15:54:20.653316+0000","last_unstale":"2024-05-30T17:09:16.559045+0000","last_undegraded":"2024-05-30T17:09:16.559045+0000","last_fullsized":"2024-05-30T17:09:16.559045+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:11:29.893912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"265'149","reported_seq":711,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:26.537490+0000","last_change":"2024-05-30T15:54:22.665082+0000","last_active":"2024-05-30T17:09:26.537490+0000","last_peered":"2024-05-30T17:09:26.537490+0000","last_clean":"2024-05-30T17:09:26.537490+0000","last_became_active":"2024-05-30T15:54:22.664353+0000","last_became_peered":"2024-05-30T15:54:22.664353+0000","last_unstale":"2024-05-30T17:09:26.537490+0000","last_undegraded":"2024-05-30T17:09:26.537490+0000","last_fullsized":"2024-05-30T17:09:26.537490+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:15:14.584034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":173,"num_read_kb":165,"num_write":120,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"265'527","reported_seq":2477,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.887321+0000","last_change":"2024-05-30T15:54:24.682250+0000","last_active":"2024-05-30T17:13:24.887321+0000","last_peered":"2024-05-30T17:13:24.887321+0000","last_clean":"2024-05-30T17:13:24.887321+0000","last_became_active":"2024-05-30T15:54:24.681503+0000","last_became_peered":"2024-05-30T15:54:24.681503+0000","last_unstale":"2024-05-30T17:13:24.887321+0000","last_undegraded":"2024-05-30T17:13:24.887321+0000","last_fullsized":"2024-05-30T17:13:24.887321+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":500,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:37:06.904193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"265'233","reported_seq":1074,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:19.091217+0000","last_change":"2024-05-30T15:54:26.671113+0000","last_active":"2024-05-30T17:13:19.091217+0000","last_peered":"2024-05-30T17:13:19.091217+0000","last_clean":"2024-05-30T17:13:19.091217+0000","last_became_active":"2024-05-30T15:54:26.670257+0000","last_became_peered":"2024-05-30T15:54:26.670257+0000","last_unstale":"2024-05-30T17:13:19.091217+0000","last_undegraded":"2024-05-30T17:13:19.091217+0000","last_fullsized":"2024-05-30T17:13:19.091217+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":200,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:56:52.779180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":661,"num_read_kb":640,"num_write":540,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"265'406","reported_seq":1988,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.698006+0000","last_change":"2024-05-30T15:54:18.660971+0000","last_active":"2024-05-30T17:13:18.698006+0000","last_peered":"2024-05-30T17:13:18.698006+0000","last_clean":"2024-05-30T17:13:18.698006+0000","last_became_active":"2024-05-30T15:54:18.660656+0000","last_became_peered":"2024-05-30T15:54:18.660656+0000","last_unstale":"2024-05-30T17:13:18.698006+0000","last_undegraded":"2024-05-30T17:13:18.698006+0000","last_fullsized":"2024-05-30T17:13:18.698006+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":300,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:13:06.838766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2368,"num_read_kb":2240,"num_write":473,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"265'190","reported_seq":819,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.558239+0000","last_change":"2024-05-30T15:54:16.668687+0000","last_active":"2024-05-30T17:11:16.558239+0000","last_peered":"2024-05-30T17:11:16.558239+0000","last_clean":"2024-05-30T17:11:16.558239+0000","last_became_active":"2024-05-30T15:54:16.668083+0000","last_became_peered":"2024-05-30T15:54:16.668083+0000","last_unstale":"2024-05-30T17:11:16.558239+0000","last_undegraded":"2024-05-30T17:11:16.558239+0000","last_fullsized":"2024-05-30T17:11:16.558239+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":100,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:37:10.668986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5290687,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":405,"num_read_kb":13963,"num_write":1053,"num_write_kb":66137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:17.124781+0000","last_change":"2024-05-30T15:54:20.659591+0000","last_active":"2024-05-30T17:09:17.124781+0000","last_peered":"2024-05-30T17:09:17.124781+0000","last_clean":"2024-05-30T17:09:17.124781+0000","last_became_active":"2024-05-30T15:54:20.658870+0000","last_became_peered":"2024-05-30T15:54:20.658870+0000","last_unstale":"2024-05-30T17:09:17.124781+0000","last_undegraded":"2024-05-30T17:09:17.124781+0000","last_fullsized":"2024-05-30T17:09:17.124781+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:06:21.446324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"265'138","reported_seq":721,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.727691+0000","last_change":"2024-05-30T15:54:22.851241+0000","last_active":"2024-05-30T17:13:20.727691+0000","last_peered":"2024-05-30T17:13:20.727691+0000","last_clean":"2024-05-30T17:13:20.727691+0000","last_became_active":"2024-05-30T15:54:22.849978+0000","last_became_peered":"2024-05-30T15:54:22.849978+0000","last_unstale":"2024-05-30T17:13:20.727691+0000","last_undegraded":"2024-05-30T17:13:20.727691+0000","last_fullsized":"2024-05-30T17:13:20.727691+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":100,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:42:52.749855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8486,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":180,"num_read_kb":226,"num_write":98,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"265'546","reported_seq":2569,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.895542+0000","last_change":"2024-05-30T15:54:24.688312+0000","last_active":"2024-05-30T17:13:24.895542+0000","last_peered":"2024-05-30T17:13:24.895542+0000","last_clean":"2024-05-30T17:13:24.895542+0000","last_became_active":"2024-05-30T15:54:24.686546+0000","last_became_peered":"2024-05-30T15:54:24.686546+0000","last_unstale":"2024-05-30T17:13:24.895542+0000","last_undegraded":"2024-05-30T17:13:24.895542+0000","last_fullsized":"2024-05-30T17:13:24.895542+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":500,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:12:50.305701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"265'895","reported_seq":2510,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.673971+0000","last_change":"2024-05-30T15:54:18.658824+0000","last_active":"2024-05-30T17:13:18.673971+0000","last_peered":"2024-05-30T17:13:18.673971+0000","last_clean":"2024-05-30T17:13:18.673971+0000","last_became_active":"2024-05-30T15:54:18.658124+0000","last_became_peered":"2024-05-30T15:54:18.658124+0000","last_unstale":"2024-05-30T17:13:18.673971+0000","last_undegraded":"2024-05-30T17:13:18.673971+0000","last_fullsized":"2024-05-30T17:13:18.673971+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":800,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:47:47.836108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4127,"num_read_kb":3657,"num_write":1246,"num_write_kb":491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"265'205","reported_seq":885,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:17.601310+0000","last_change":"2024-05-30T15:54:16.659141+0000","last_active":"2024-05-30T17:11:17.601310+0000","last_peered":"2024-05-30T17:11:17.601310+0000","last_clean":"2024-05-30T17:11:17.601310+0000","last_became_active":"2024-05-30T15:54:16.658279+0000","last_became_peered":"2024-05-30T15:54:16.658279+0000","last_unstale":"2024-05-30T17:11:17.601310+0000","last_undegraded":"2024-05-30T17:11:17.601310+0000","last_fullsized":"2024-05-30T17:11:17.601310+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":100,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:04:21.938813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1308948,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":477,"num_read_kb":111643,"num_write":1131,"num_write_kb":26225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":510,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:11.551997+0000","last_change":"2024-05-30T15:54:20.662333+0000","last_active":"2024-05-30T17:10:11.551997+0000","last_peered":"2024-05-30T17:10:11.551997+0000","last_clean":"2024-05-30T17:10:11.551997+0000","last_became_active":"2024-05-30T15:54:20.661583+0000","last_became_peered":"2024-05-30T15:54:20.661583+0000","last_unstale":"2024-05-30T17:10:11.551997+0000","last_undegraded":"2024-05-30T17:10:11.551997+0000","last_fullsized":"2024-05-30T17:10:11.551997+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:07:17.026476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"265'252","reported_seq":895,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:36.971452+0000","last_change":"2024-05-30T15:54:22.850012+0000","last_active":"2024-05-30T17:12:36.971452+0000","last_peered":"2024-05-30T17:12:36.971452+0000","last_clean":"2024-05-30T17:12:36.971452+0000","last_became_active":"2024-05-30T15:54:22.849126+0000","last_became_peered":"2024-05-30T15:54:22.849126+0000","last_unstale":"2024-05-30T17:12:36.971452+0000","last_undegraded":"2024-05-30T17:12:36.971452+0000","last_fullsized":"2024-05-30T17:12:36.971452+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":200,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:34:44.383862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7991,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":343,"num_read_kb":359,"num_write":206,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":502,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:56.786717+0000","last_change":"2024-05-30T15:54:24.674322+0000","last_active":"2024-05-30T17:10:56.786717+0000","last_peered":"2024-05-30T17:10:56.786717+0000","last_clean":"2024-05-30T17:10:56.786717+0000","last_became_active":"2024-05-30T15:54:24.673506+0000","last_became_peered":"2024-05-30T15:54:24.673506+0000","last_unstale":"2024-05-30T17:10:56.786717+0000","last_undegraded":"2024-05-30T17:10:56.786717+0000","last_fullsized":"2024-05-30T17:10:56.786717+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:07.777377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"265'543","reported_seq":1373,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.867831+0000","last_change":"2024-05-30T15:54:26.687129+0000","last_active":"2024-05-30T17:13:24.867831+0000","last_peered":"2024-05-30T17:13:24.867831+0000","last_clean":"2024-05-30T17:13:24.867831+0000","last_became_active":"2024-05-30T15:54:26.686389+0000","last_became_peered":"2024-05-30T15:54:26.686389+0000","last_unstale":"2024-05-30T17:13:24.867831+0000","last_undegraded":"2024-05-30T17:13:24.867831+0000","last_fullsized":"2024-05-30T17:13:24.867831+0000","mapping_epoch":32,"log_start":"265'500","ondisk_log_start":"265'500","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":500,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:44:22.858001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1297,"num_read_kb":1288,"num_write":1137,"num_write_kb":385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"265'1398","reported_seq":3105,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.685832+0000","last_change":"2024-05-30T15:54:18.640378+0000","last_active":"2024-05-30T17:13:18.685832+0000","last_peered":"2024-05-30T17:13:18.685832+0000","last_clean":"2024-05-30T17:13:18.685832+0000","last_became_active":"2024-05-30T15:54:18.639667+0000","last_became_peered":"2024-05-30T15:54:18.639667+0000","last_unstale":"2024-05-30T17:13:18.685832+0000","last_undegraded":"2024-05-30T17:13:18.685832+0000","last_fullsized":"2024-05-30T17:13:18.685832+0000","mapping_epoch":24,"log_start":"265'1300","ondisk_log_start":"265'1300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":1300,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:30:46.931867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5970,"num_read_kb":5130,"num_write":2044,"num_write_kb":884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"265'208","reported_seq":862,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.560609+0000","last_change":"2024-05-30T15:54:16.661968+0000","last_active":"2024-05-30T17:10:46.560609+0000","last_peered":"2024-05-30T17:10:46.560609+0000","last_clean":"2024-05-30T17:10:46.560609+0000","last_became_active":"2024-05-30T15:54:16.659427+0000","last_became_peered":"2024-05-30T15:54:16.659427+0000","last_unstale":"2024-05-30T17:10:46.560609+0000","last_undegraded":"2024-05-30T17:10:46.560609+0000","last_fullsized":"2024-05-30T17:10:46.560609+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":100,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:29:13.804456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5946564,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":462,"num_read_kb":34119,"num_write":1152,"num_write_kb":42367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":513,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:11.545498+0000","last_change":"2024-05-30T15:54:20.647908+0000","last_active":"2024-05-30T17:10:11.545498+0000","last_peered":"2024-05-30T17:10:11.545498+0000","last_clean":"2024-05-30T17:10:11.545498+0000","last_became_active":"2024-05-30T15:54:20.647565+0000","last_became_peered":"2024-05-30T15:54:20.647565+0000","last_unstale":"2024-05-30T17:10:11.545498+0000","last_undegraded":"2024-05-30T17:10:11.545498+0000","last_fullsized":"2024-05-30T17:10:11.545498+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:19:36.402906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"265'142","reported_seq":730,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:11.612518+0000","last_change":"2024-05-30T15:54:22.671276+0000","last_active":"2024-05-30T17:13:11.612518+0000","last_peered":"2024-05-30T17:13:11.612518+0000","last_clean":"2024-05-30T17:13:11.612518+0000","last_became_active":"2024-05-30T15:54:22.670064+0000","last_became_peered":"2024-05-30T15:54:22.670064+0000","last_unstale":"2024-05-30T17:13:11.612518+0000","last_undegraded":"2024-05-30T17:13:11.612518+0000","last_fullsized":"2024-05-30T17:13:11.612518+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":100,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:40:10.621760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16203,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":188,"num_read_kb":263,"num_write":104,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":504,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:36.577286+0000","last_change":"2024-05-30T15:54:24.688941+0000","last_active":"2024-05-30T17:10:36.577286+0000","last_peered":"2024-05-30T17:10:36.577286+0000","last_clean":"2024-05-30T17:10:36.577286+0000","last_became_active":"2024-05-30T15:54:24.688046+0000","last_became_peered":"2024-05-30T15:54:24.688046+0000","last_unstale":"2024-05-30T17:10:36.577286+0000","last_undegraded":"2024-05-30T17:10:36.577286+0000","last_fullsized":"2024-05-30T17:10:36.577286+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:44:58.051665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"265'2369","reported_seq":4189,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.624292+0000","last_change":"2024-05-30T15:54:26.692267+0000","last_active":"2024-05-30T17:13:18.624292+0000","last_peered":"2024-05-30T17:13:18.624292+0000","last_clean":"2024-05-30T17:13:18.624292+0000","last_became_active":"2024-05-30T15:54:26.691574+0000","last_became_peered":"2024-05-30T15:54:26.691574+0000","last_unstale":"2024-05-30T17:13:18.624292+0000","last_undegraded":"2024-05-30T17:13:18.624292+0000","last_fullsized":"2024-05-30T17:13:18.624292+0000","mapping_epoch":32,"log_start":"265'2300","ondisk_log_start":"265'2300","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":2300,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:33:03.952010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5943,"num_read_kb":5935,"num_write":4785,"num_write_kb":1767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"265'371","reported_seq":1792,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:35.664016+0000","last_change":"2024-05-30T15:54:18.670009+0000","last_active":"2024-05-30T17:11:35.664016+0000","last_peered":"2024-05-30T17:11:35.664016+0000","last_clean":"2024-05-30T17:11:35.664016+0000","last_became_active":"2024-05-30T15:54:18.668963+0000","last_became_peered":"2024-05-30T15:54:18.668963+0000","last_unstale":"2024-05-30T17:11:35.664016+0000","last_undegraded":"2024-05-30T17:11:35.664016+0000","last_fullsized":"2024-05-30T17:11:35.664016+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":300,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:33:28.500306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2011,"num_read_kb":1890,"num_write":455,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"265'230","reported_seq":870,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.560532+0000","last_change":"2024-05-30T15:54:16.650507+0000","last_active":"2024-05-30T17:10:46.560532+0000","last_peered":"2024-05-30T17:10:46.560532+0000","last_clean":"2024-05-30T17:10:46.560532+0000","last_became_active":"2024-05-30T15:54:16.649605+0000","last_became_peered":"2024-05-30T15:54:16.649605+0000","last_unstale":"2024-05-30T17:10:46.560532+0000","last_undegraded":"2024-05-30T17:10:46.560532+0000","last_fullsized":"2024-05-30T17:10:46.560532+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":200,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:10:51.493426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5355180,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":459,"num_read_kb":32396,"num_write":1233,"num_write_kb":40071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":512,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.554033+0000","last_change":"2024-05-30T15:54:20.652270+0000","last_active":"2024-05-30T17:10:51.554033+0000","last_peered":"2024-05-30T17:10:51.554033+0000","last_clean":"2024-05-30T17:10:51.554033+0000","last_became_active":"2024-05-30T15:54:20.651369+0000","last_became_peered":"2024-05-30T15:54:20.651369+0000","last_unstale":"2024-05-30T17:10:51.554033+0000","last_undegraded":"2024-05-30T17:10:51.554033+0000","last_fullsized":"2024-05-30T17:10:51.554033+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:06:27.275568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"265'187","reported_seq":798,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:26.585367+0000","last_change":"2024-05-30T15:54:22.666930+0000","last_active":"2024-05-30T17:12:26.585367+0000","last_peered":"2024-05-30T17:12:26.585367+0000","last_clean":"2024-05-30T17:12:26.585367+0000","last_became_active":"2024-05-30T15:54:22.664874+0000","last_became_peered":"2024-05-30T15:54:22.664874+0000","last_unstale":"2024-05-30T17:12:26.585367+0000","last_undegraded":"2024-05-30T17:12:26.585367+0000","last_fullsized":"2024-05-30T17:12:26.585367+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":100,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:25:56.647990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6891,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":250,"num_read_kb":293,"num_write":144,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"265'546","reported_seq":2479,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.897156+0000","last_change":"2024-05-30T15:54:24.674977+0000","last_active":"2024-05-30T17:13:24.897156+0000","last_peered":"2024-05-30T17:13:24.897156+0000","last_clean":"2024-05-30T17:13:24.897156+0000","last_became_active":"2024-05-30T15:54:24.670141+0000","last_became_peered":"2024-05-30T15:54:24.670141+0000","last_unstale":"2024-05-30T17:13:24.897156+0000","last_undegraded":"2024-05-30T17:13:24.897156+0000","last_fullsized":"2024-05-30T17:13:24.897156+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":500,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:38:58.029893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"265'107","reported_seq":611,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:16.569182+0000","last_change":"2024-05-30T15:54:26.693610+0000","last_active":"2024-05-30T17:10:16.569182+0000","last_peered":"2024-05-30T17:10:16.569182+0000","last_clean":"2024-05-30T17:10:16.569182+0000","last_became_active":"2024-05-30T15:54:26.689987+0000","last_became_peered":"2024-05-30T15:54:26.689987+0000","last_unstale":"2024-05-30T17:10:16.569182+0000","last_undegraded":"2024-05-30T17:10:16.569182+0000","last_fullsized":"2024-05-30T17:10:16.569182+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:15:03.997666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":80,"num_write":254,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"265'1331","reported_seq":2966,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.705722+0000","last_change":"2024-05-30T15:54:18.647985+0000","last_active":"2024-05-30T17:13:18.705722+0000","last_peered":"2024-05-30T17:13:18.705722+0000","last_clean":"2024-05-30T17:13:18.705722+0000","last_became_active":"2024-05-30T15:54:18.647634+0000","last_became_peered":"2024-05-30T15:54:18.647634+0000","last_unstale":"2024-05-30T17:13:18.705722+0000","last_undegraded":"2024-05-30T17:13:18.705722+0000","last_fullsized":"2024-05-30T17:13:18.705722+0000","mapping_epoch":24,"log_start":"265'1300","ondisk_log_start":"265'1300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":1300,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:55:27.116297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5717,"num_read_kb":4967,"num_write":1912,"num_write_kb":881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"265'202","reported_seq":817,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:47.145058+0000","last_change":"2024-05-30T15:54:16.668968+0000","last_active":"2024-05-30T17:10:47.145058+0000","last_peered":"2024-05-30T17:10:47.145058+0000","last_clean":"2024-05-30T17:10:47.145058+0000","last_became_active":"2024-05-30T15:54:16.668336+0000","last_became_peered":"2024-05-30T15:54:16.668336+0000","last_unstale":"2024-05-30T17:10:47.145058+0000","last_undegraded":"2024-05-30T17:10:47.145058+0000","last_fullsized":"2024-05-30T17:10:47.145058+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:22:09.399226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41078,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":399,"num_read_kb":1534,"num_write":1054,"num_write_kb":21253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":513,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:31.549752+0000","last_change":"2024-05-30T15:54:20.656467+0000","last_active":"2024-05-30T17:10:31.549752+0000","last_peered":"2024-05-30T17:10:31.549752+0000","last_clean":"2024-05-30T17:10:31.549752+0000","last_became_active":"2024-05-30T15:54:20.655573+0000","last_became_peered":"2024-05-30T15:54:20.655573+0000","last_unstale":"2024-05-30T17:10:31.549752+0000","last_undegraded":"2024-05-30T17:10:31.549752+0000","last_fullsized":"2024-05-30T17:10:31.549752+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:20:02.252327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"265'199","reported_seq":849,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.783939+0000","last_change":"2024-05-30T15:54:22.659510+0000","last_active":"2024-05-30T17:13:24.783939+0000","last_peered":"2024-05-30T17:13:24.783939+0000","last_clean":"2024-05-30T17:13:24.783939+0000","last_became_active":"2024-05-30T15:54:22.658700+0000","last_became_peered":"2024-05-30T15:54:22.658700+0000","last_unstale":"2024-05-30T17:13:24.783939+0000","last_undegraded":"2024-05-30T17:13:24.783939+0000","last_fullsized":"2024-05-30T17:13:24.783939+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T15:57:00.030682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18300,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":291,"num_read_kb":407,"num_write":146,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":505,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:16.546507+0000","last_change":"2024-05-30T15:54:24.674234+0000","last_active":"2024-05-30T17:10:16.546507+0000","last_peered":"2024-05-30T17:10:16.546507+0000","last_clean":"2024-05-30T17:10:16.546507+0000","last_became_active":"2024-05-30T15:54:24.673505+0000","last_became_peered":"2024-05-30T15:54:24.673505+0000","last_unstale":"2024-05-30T17:10:16.546507+0000","last_undegraded":"2024-05-30T17:10:16.546507+0000","last_fullsized":"2024-05-30T17:10:16.546507+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:16:54.874496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"265'99","reported_seq":616,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:49.195333+0000","last_change":"2024-05-30T15:54:26.701347+0000","last_active":"2024-05-30T17:10:49.195333+0000","last_peered":"2024-05-30T17:10:49.195333+0000","last_clean":"2024-05-30T17:10:49.195333+0000","last_became_active":"2024-05-30T15:54:26.700608+0000","last_became_peered":"2024-05-30T15:54:26.700608+0000","last_unstale":"2024-05-30T17:10:49.195333+0000","last_undegraded":"2024-05-30T17:10:49.195333+0000","last_fullsized":"2024-05-30T17:10:49.195333+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:57:26.315030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":918,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":103,"num_write":225,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"265'539","reported_seq":2167,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:28.313044+0000","last_change":"2024-05-30T15:54:18.658375+0000","last_active":"2024-05-30T17:12:28.313044+0000","last_peered":"2024-05-30T17:12:28.313044+0000","last_clean":"2024-05-30T17:12:28.313044+0000","last_became_active":"2024-05-30T15:54:18.657522+0000","last_became_peered":"2024-05-30T15:54:18.657522+0000","last_unstale":"2024-05-30T17:12:28.313044+0000","last_undegraded":"2024-05-30T17:12:28.313044+0000","last_fullsized":"2024-05-30T17:12:28.313044+0000","mapping_epoch":24,"log_start":"265'500","ondisk_log_start":"265'500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":500,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:59:15.813358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2751,"num_read_kb":2537,"num_write":708,"num_write_kb":252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"265'230","reported_seq":911,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:34.812361+0000","last_change":"2024-05-30T15:54:16.666646+0000","last_active":"2024-05-30T17:11:34.812361+0000","last_peered":"2024-05-30T17:11:34.812361+0000","last_clean":"2024-05-30T17:11:34.812361+0000","last_became_active":"2024-05-30T15:54:16.666244+0000","last_became_peered":"2024-05-30T15:54:16.666244+0000","last_unstale":"2024-05-30T17:11:34.812361+0000","last_undegraded":"2024-05-30T17:11:34.812361+0000","last_fullsized":"2024-05-30T17:11:34.812361+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":200,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:10:10.302094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11062814,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":634,"num_read_kb":32957,"num_write":1221,"num_write_kb":40334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":512,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.554107+0000","last_change":"2024-05-30T15:54:20.662409+0000","last_active":"2024-05-30T17:10:51.554107+0000","last_peered":"2024-05-30T17:10:51.554107+0000","last_clean":"2024-05-30T17:10:51.554107+0000","last_became_active":"2024-05-30T15:54:20.661967+0000","last_became_peered":"2024-05-30T15:54:20.661967+0000","last_unstale":"2024-05-30T17:10:51.554107+0000","last_undegraded":"2024-05-30T17:10:51.554107+0000","last_fullsized":"2024-05-30T17:10:51.554107+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:38:09.118798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"265'177","reported_seq":780,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:11.775063+0000","last_change":"2024-05-30T15:54:22.665466+0000","last_active":"2024-05-30T17:13:11.775063+0000","last_peered":"2024-05-30T17:13:11.775063+0000","last_clean":"2024-05-30T17:13:11.775063+0000","last_became_active":"2024-05-30T15:54:22.664732+0000","last_became_peered":"2024-05-30T15:54:22.664732+0000","last_unstale":"2024-05-30T17:13:11.775063+0000","last_undegraded":"2024-05-30T17:13:11.775063+0000","last_fullsized":"2024-05-30T17:13:11.775063+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":100,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:44:24.095083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19031,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":256,"num_read_kb":341,"num_write":198,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.582702+0000","last_change":"2024-05-30T15:54:24.679521+0000","last_active":"2024-05-30T17:11:16.582702+0000","last_peered":"2024-05-30T17:11:16.582702+0000","last_clean":"2024-05-30T17:11:16.582702+0000","last_became_active":"2024-05-30T15:54:24.679254+0000","last_became_peered":"2024-05-30T15:54:24.679254+0000","last_unstale":"2024-05-30T17:11:16.582702+0000","last_undegraded":"2024-05-30T17:11:16.582702+0000","last_fullsized":"2024-05-30T17:11:16.582702+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:16:16.737529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"265'105","reported_seq":623,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.579506+0000","last_change":"2024-05-30T15:54:26.697980+0000","last_active":"2024-05-30T17:10:46.579506+0000","last_peered":"2024-05-30T17:10:46.579506+0000","last_clean":"2024-05-30T17:10:46.579506+0000","last_became_active":"2024-05-30T15:54:26.697333+0000","last_became_peered":"2024-05-30T15:54:26.697333+0000","last_unstale":"2024-05-30T17:10:46.579506+0000","last_undegraded":"2024-05-30T17:10:46.579506+0000","last_fullsized":"2024-05-30T17:10:46.579506+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:11:10.336832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":270,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":107,"num_read_kb":90,"num_write":242,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"265'397","reported_seq":2059,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:09.478879+0000","last_change":"2024-05-30T15:54:18.636254+0000","last_active":"2024-05-30T17:13:09.478879+0000","last_peered":"2024-05-30T17:13:09.478879+0000","last_clean":"2024-05-30T17:13:09.478879+0000","last_became_active":"2024-05-30T15:54:18.635519+0000","last_became_peered":"2024-05-30T15:54:18.635519+0000","last_unstale":"2024-05-30T17:13:09.478879+0000","last_undegraded":"2024-05-30T17:13:09.478879+0000","last_fullsized":"2024-05-30T17:13:09.478879+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":300,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:25:03.148120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2332,"num_read_kb":2242,"num_write":450,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"265'173","reported_seq":793,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:48.596500+0000","last_change":"2024-05-30T15:54:16.685723+0000","last_active":"2024-05-30T17:11:48.596500+0000","last_peered":"2024-05-30T17:11:48.596500+0000","last_clean":"2024-05-30T17:11:48.596500+0000","last_became_active":"2024-05-30T15:54:16.685162+0000","last_became_peered":"2024-05-30T15:54:16.685162+0000","last_unstale":"2024-05-30T17:11:48.596500+0000","last_undegraded":"2024-05-30T17:11:48.596500+0000","last_fullsized":"2024-05-30T17:11:48.596500+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":100,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:11:20.614231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6251614,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":399,"num_read_kb":10586,"num_write":1025,"num_write_kb":21029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":510,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.785627+0000","last_change":"2024-05-30T15:54:20.646982+0000","last_active":"2024-05-30T17:10:51.785627+0000","last_peered":"2024-05-30T17:10:51.785627+0000","last_clean":"2024-05-30T17:10:51.785627+0000","last_became_active":"2024-05-30T15:54:20.646646+0000","last_became_peered":"2024-05-30T15:54:20.646646+0000","last_unstale":"2024-05-30T17:10:51.785627+0000","last_undegraded":"2024-05-30T17:10:51.785627+0000","last_fullsized":"2024-05-30T17:10:51.785627+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:31:33.090729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"265'179","reported_seq":792,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.851680+0000","last_change":"2024-05-30T15:54:22.670371+0000","last_active":"2024-05-30T17:13:24.851680+0000","last_peered":"2024-05-30T17:13:24.851680+0000","last_clean":"2024-05-30T17:13:24.851680+0000","last_became_active":"2024-05-30T15:54:22.669660+0000","last_became_peered":"2024-05-30T15:54:22.669660+0000","last_unstale":"2024-05-30T17:13:24.851680+0000","last_undegraded":"2024-05-30T17:13:24.851680+0000","last_fullsized":"2024-05-30T17:13:24.851680+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":100,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:45:08.898874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9643,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":249,"num_read_kb":299,"num_write":142,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"265'509","reported_seq":4592,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.889937+0000","last_change":"2024-05-30T15:54:24.673150+0000","last_active":"2024-05-30T17:13:24.889937+0000","last_peered":"2024-05-30T17:13:24.889937+0000","last_clean":"2024-05-30T17:13:24.889937+0000","last_became_active":"2024-05-30T15:54:24.672377+0000","last_became_peered":"2024-05-30T15:54:24.672377+0000","last_unstale":"2024-05-30T17:13:24.889937+0000","last_undegraded":"2024-05-30T17:13:24.889937+0000","last_fullsized":"2024-05-30T17:13:24.889937+0000","mapping_epoch":30,"log_start":"262'400","ondisk_log_start":"262'400","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":400,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:22:29.518162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"265'117","reported_seq":633,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.718032+0000","last_change":"2024-05-30T15:54:26.680642+0000","last_active":"2024-05-30T17:13:18.718032+0000","last_peered":"2024-05-30T17:13:18.718032+0000","last_clean":"2024-05-30T17:13:18.718032+0000","last_became_active":"2024-05-30T15:54:26.679866+0000","last_became_peered":"2024-05-30T15:54:26.679866+0000","last_unstale":"2024-05-30T17:13:18.718032+0000","last_undegraded":"2024-05-30T17:13:18.718032+0000","last_fullsized":"2024-05-30T17:13:18.718032+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:26:55.645297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1256,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":144,"num_read_kb":134,"num_write":293,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"265'389","reported_seq":1796,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:37.615748+0000","last_change":"2024-05-30T15:54:18.671784+0000","last_active":"2024-05-30T17:11:37.615748+0000","last_peered":"2024-05-30T17:11:37.615748+0000","last_clean":"2024-05-30T17:11:37.615748+0000","last_became_active":"2024-05-30T15:54:18.671088+0000","last_became_peered":"2024-05-30T15:54:18.671088+0000","last_unstale":"2024-05-30T17:11:37.615748+0000","last_undegraded":"2024-05-30T17:11:37.615748+0000","last_fullsized":"2024-05-30T17:11:37.615748+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":300,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:23:36.607873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2029,"num_read_kb":1894,"num_write":481,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"265'178","reported_seq":921,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:28.333806+0000","last_change":"2024-05-30T15:54:16.672575+0000","last_active":"2024-05-30T17:11:28.333806+0000","last_peered":"2024-05-30T17:11:28.333806+0000","last_clean":"2024-05-30T17:11:28.333806+0000","last_became_active":"2024-05-30T15:54:16.671477+0000","last_became_peered":"2024-05-30T15:54:16.671477+0000","last_unstale":"2024-05-30T17:11:28.333806+0000","last_undegraded":"2024-05-30T17:11:28.333806+0000","last_fullsized":"2024-05-30T17:11:28.333806+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":100,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:47:10.773488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":132257,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":677,"num_read_kb":167118,"num_write":1080,"num_write_kb":34693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"27'2","reported_seq":1154,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.724885+0000","last_change":"2024-05-30T15:54:20.663952+0000","last_active":"2024-05-30T17:13:24.724885+0000","last_peered":"2024-05-30T17:13:24.724885+0000","last_clean":"2024-05-30T17:13:24.724885+0000","last_became_active":"2024-05-30T15:54:20.662371+0000","last_became_peered":"2024-05-30T15:54:20.662371+0000","last_unstale":"2024-05-30T17:13:24.724885+0000","last_undegraded":"2024-05-30T17:13:24.724885+0000","last_fullsized":"2024-05-30T17:13:24.724885+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:43:22.768148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":643,"num_read_kb":643,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"265'109","reported_seq":656,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:56.582283+0000","last_change":"2024-05-30T15:54:22.849771+0000","last_active":"2024-05-30T17:10:56.582283+0000","last_peered":"2024-05-30T17:10:56.582283+0000","last_clean":"2024-05-30T17:10:56.582283+0000","last_became_active":"2024-05-30T15:54:22.849105+0000","last_became_peered":"2024-05-30T15:54:22.849105+0000","last_unstale":"2024-05-30T17:10:56.582283+0000","last_undegraded":"2024-05-30T17:10:56.582283+0000","last_fullsized":"2024-05-30T17:10:56.582283+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:56:52.974159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":120,"num_read_kb":115,"num_write":80,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"265'538","reported_seq":2314,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.891869+0000","last_change":"2024-05-30T15:54:24.678108+0000","last_active":"2024-05-30T17:13:24.891869+0000","last_peered":"2024-05-30T17:13:24.891869+0000","last_clean":"2024-05-30T17:13:24.891869+0000","last_became_active":"2024-05-30T15:54:24.677291+0000","last_became_peered":"2024-05-30T15:54:24.677291+0000","last_unstale":"2024-05-30T17:13:24.891869+0000","last_undegraded":"2024-05-30T17:13:24.891869+0000","last_fullsized":"2024-05-30T17:13:24.891869+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":500,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:59:57.567508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"265'135","reported_seq":656,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:15.308427+0000","last_change":"2024-05-30T15:54:26.675238+0000","last_active":"2024-05-30T17:13:15.308427+0000","last_peered":"2024-05-30T17:13:15.308427+0000","last_clean":"2024-05-30T17:13:15.308427+0000","last_became_active":"2024-05-30T15:54:26.674517+0000","last_became_peered":"2024-05-30T15:54:26.674517+0000","last_unstale":"2024-05-30T17:13:15.308427+0000","last_undegraded":"2024-05-30T17:13:15.308427+0000","last_fullsized":"2024-05-30T17:13:15.308427+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":100,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:38:58.206221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":141,"num_read_kb":129,"num_write":332,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"265'402","reported_seq":1940,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:12.347239+0000","last_change":"2024-05-30T15:54:18.657775+0000","last_active":"2024-05-30T17:13:12.347239+0000","last_peered":"2024-05-30T17:13:12.347239+0000","last_clean":"2024-05-30T17:13:12.347239+0000","last_became_active":"2024-05-30T15:54:18.657247+0000","last_became_peered":"2024-05-30T15:54:18.657247+0000","last_unstale":"2024-05-30T17:13:12.347239+0000","last_undegraded":"2024-05-30T17:13:12.347239+0000","last_fullsized":"2024-05-30T17:13:12.347239+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":300,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:33:29.912796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2106,"num_read_kb":2008,"num_write":461,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"265'180","reported_seq":790,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.553032+0000","last_change":"2024-05-30T15:54:54.427377+0000","last_active":"2024-05-30T17:10:46.553032+0000","last_peered":"2024-05-30T17:10:46.553032+0000","last_clean":"2024-05-30T17:10:46.553032+0000","last_became_active":"2024-05-30T15:54:54.426538+0000","last_became_peered":"2024-05-30T15:54:54.426538+0000","last_unstale":"2024-05-30T17:10:46.553032+0000","last_undegraded":"2024-05-30T17:10:46.553032+0000","last_fullsized":"2024-05-30T17:10:46.553032+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":100,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:57:32.635925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5292382,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":438,"num_read_kb":549,"num_write":1007,"num_write_kb":15096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,7],"acting":[2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":518,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.742564+0000","last_change":"2024-05-30T15:54:20.670270+0000","last_active":"2024-05-30T17:10:51.742564+0000","last_peered":"2024-05-30T17:10:51.742564+0000","last_clean":"2024-05-30T17:10:51.742564+0000","last_became_active":"2024-05-30T15:54:20.669492+0000","last_became_peered":"2024-05-30T15:54:20.669492+0000","last_unstale":"2024-05-30T17:10:51.742564+0000","last_undegraded":"2024-05-30T17:10:51.742564+0000","last_fullsized":"2024-05-30T17:10:51.742564+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:01:58.108666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"265'232","reported_seq":866,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:11.979664+0000","last_change":"2024-05-30T15:54:22.672749+0000","last_active":"2024-05-30T17:13:11.979664+0000","last_peered":"2024-05-30T17:13:11.979664+0000","last_clean":"2024-05-30T17:13:11.979664+0000","last_became_active":"2024-05-30T15:54:22.671943+0000","last_became_peered":"2024-05-30T15:54:22.671943+0000","last_unstale":"2024-05-30T17:13:11.979664+0000","last_undegraded":"2024-05-30T17:13:11.979664+0000","last_fullsized":"2024-05-30T17:13:11.979664+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":200,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:43:33.730358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4994,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":304,"num_read_kb":297,"num_write":174,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":500,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:56.553870+0000","last_change":"2024-05-30T15:54:24.682341+0000","last_active":"2024-05-30T17:10:56.553870+0000","last_peered":"2024-05-30T17:10:56.553870+0000","last_clean":"2024-05-30T17:10:56.553870+0000","last_became_active":"2024-05-30T15:54:24.681595+0000","last_became_peered":"2024-05-30T15:54:24.681595+0000","last_unstale":"2024-05-30T17:10:56.553870+0000","last_undegraded":"2024-05-30T17:10:56.553870+0000","last_fullsized":"2024-05-30T17:10:56.553870+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:03:03.587984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"265'115","reported_seq":622,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:31.794301+0000","last_change":"2024-05-30T15:54:26.689376+0000","last_active":"2024-05-30T17:11:31.794301+0000","last_peered":"2024-05-30T17:11:31.794301+0000","last_clean":"2024-05-30T17:11:31.794301+0000","last_became_active":"2024-05-30T15:54:26.688518+0000","last_became_peered":"2024-05-30T15:54:26.688518+0000","last_unstale":"2024-05-30T17:11:31.794301+0000","last_undegraded":"2024-05-30T17:11:31.794301+0000","last_fullsized":"2024-05-30T17:11:31.794301+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":100,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:13:29.397423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":619,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":113,"num_read_kb":112,"num_write":261,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"265'756","reported_seq":2265,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.637015+0000","last_change":"2024-05-30T15:54:18.678584+0000","last_active":"2024-05-30T17:11:21.637015+0000","last_peered":"2024-05-30T17:11:21.637015+0000","last_clean":"2024-05-30T17:11:21.637015+0000","last_became_active":"2024-05-30T15:54:18.677957+0000","last_became_peered":"2024-05-30T15:54:18.677957+0000","last_unstale":"2024-05-30T17:11:21.637015+0000","last_undegraded":"2024-05-30T17:11:21.637015+0000","last_fullsized":"2024-05-30T17:11:21.637015+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":700,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:51:57.717964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3440,"num_read_kb":3078,"num_write":1010,"num_write_kb":372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":627,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:26.555530+0000","last_change":"2024-05-30T15:54:03.031292+0000","last_active":"2024-05-30T17:10:26.555530+0000","last_peered":"2024-05-30T17:10:26.555530+0000","last_clean":"2024-05-30T17:10:26.555530+0000","last_became_active":"2024-05-30T15:54:03.029214+0000","last_became_peered":"2024-05-30T15:54:03.029214+0000","last_unstale":"2024-05-30T17:10:26.555530+0000","last_undegraded":"2024-05-30T17:10:26.555530+0000","last_fullsized":"2024-05-30T17:10:26.555530+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:01.512251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:01.512251+0000","last_clean_scrub_stamp":"2024-05-30T15:54:01.512251+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:40:07.389706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":530,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:17.151931+0000","last_change":"2024-05-30T15:54:08.578526+0000","last_active":"2024-05-30T17:11:17.151931+0000","last_peered":"2024-05-30T17:11:17.151931+0000","last_clean":"2024-05-30T17:11:17.151931+0000","last_became_active":"2024-05-30T15:54:08.577556+0000","last_became_peered":"2024-05-30T15:54:08.577556+0000","last_unstale":"2024-05-30T17:11:17.151931+0000","last_undegraded":"2024-05-30T17:11:17.151931+0000","last_fullsized":"2024-05-30T17:11:17.151931+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:36:52.339688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"265'186","reported_seq":841,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.587587+0000","last_change":"2024-05-30T15:54:16.674631+0000","last_active":"2024-05-30T17:11:21.587587+0000","last_peered":"2024-05-30T17:11:21.587587+0000","last_clean":"2024-05-30T17:11:21.587587+0000","last_became_active":"2024-05-30T15:54:16.674229+0000","last_became_peered":"2024-05-30T15:54:16.674229+0000","last_unstale":"2024-05-30T17:11:21.587587+0000","last_undegraded":"2024-05-30T17:11:21.587587+0000","last_fullsized":"2024-05-30T17:11:21.587587+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":100,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:38:06.634901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10797837,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":444,"num_read_kb":11470,"num_write":1071,"num_write_kb":25445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":512,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.570297+0000","last_change":"2024-05-30T15:54:20.665207+0000","last_active":"2024-05-30T17:11:26.570297+0000","last_peered":"2024-05-30T17:11:26.570297+0000","last_clean":"2024-05-30T17:11:26.570297+0000","last_became_active":"2024-05-30T15:54:20.664317+0000","last_became_peered":"2024-05-30T15:54:20.664317+0000","last_unstale":"2024-05-30T17:11:26.570297+0000","last_undegraded":"2024-05-30T17:11:26.570297+0000","last_fullsized":"2024-05-30T17:11:26.570297+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:57:42.587325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"265'322","reported_seq":1031,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.820569+0000","last_change":"2024-05-30T15:54:22.670731+0000","last_active":"2024-05-30T17:13:24.820569+0000","last_peered":"2024-05-30T17:13:24.820569+0000","last_clean":"2024-05-30T17:13:24.820569+0000","last_became_active":"2024-05-30T15:54:22.669871+0000","last_became_peered":"2024-05-30T15:54:22.669871+0000","last_unstale":"2024-05-30T17:13:24.820569+0000","last_undegraded":"2024-05-30T17:13:24.820569+0000","last_fullsized":"2024-05-30T17:13:24.820569+0000","mapping_epoch":28,"log_start":"265'300","ondisk_log_start":"265'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":300,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:48:31.269839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59120,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":621,"num_read_kb":899,"num_write":586,"num_write_kb":457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":507,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:16.569157+0000","last_change":"2024-05-30T15:54:24.685392+0000","last_active":"2024-05-30T17:10:16.569157+0000","last_peered":"2024-05-30T17:10:16.569157+0000","last_clean":"2024-05-30T17:10:16.569157+0000","last_became_active":"2024-05-30T15:54:24.684709+0000","last_became_peered":"2024-05-30T15:54:24.684709+0000","last_unstale":"2024-05-30T17:10:16.569157+0000","last_undegraded":"2024-05-30T17:10:16.569157+0000","last_fullsized":"2024-05-30T17:10:16.569157+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:57:38.351236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"265'174","reported_seq":878,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:16.948647+0000","last_change":"2024-05-30T15:54:26.687357+0000","last_active":"2024-05-30T17:13:16.948647+0000","last_peered":"2024-05-30T17:13:16.948647+0000","last_clean":"2024-05-30T17:13:16.948647+0000","last_became_active":"2024-05-30T15:54:26.686647+0000","last_became_peered":"2024-05-30T15:54:26.686647+0000","last_unstale":"2024-05-30T17:13:16.948647+0000","last_undegraded":"2024-05-30T17:13:16.948647+0000","last_fullsized":"2024-05-30T17:13:16.948647+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:16:53.692909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1020,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":416,"num_read_kb":404,"num_write":359,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"265'388","reported_seq":1862,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:28.256178+0000","last_change":"2024-05-30T15:54:18.649340+0000","last_active":"2024-05-30T17:12:28.256178+0000","last_peered":"2024-05-30T17:12:28.256178+0000","last_clean":"2024-05-30T17:12:28.256178+0000","last_became_active":"2024-05-30T15:54:18.648529+0000","last_became_peered":"2024-05-30T15:54:18.648529+0000","last_unstale":"2024-05-30T17:12:28.256178+0000","last_undegraded":"2024-05-30T17:12:28.256178+0000","last_fullsized":"2024-05-30T17:12:28.256178+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":300,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:56:22.361251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2082,"num_read_kb":1967,"num_write":465,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":531,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:31.556722+0000","last_change":"2024-05-30T15:54:08.575348+0000","last_active":"2024-05-30T17:10:31.556722+0000","last_peered":"2024-05-30T17:10:31.556722+0000","last_clean":"2024-05-30T17:10:31.556722+0000","last_became_active":"2024-05-30T15:54:08.574645+0000","last_became_peered":"2024-05-30T15:54:08.574645+0000","last_unstale":"2024-05-30T17:10:31.556722+0000","last_undegraded":"2024-05-30T17:10:31.556722+0000","last_fullsized":"2024-05-30T17:10:31.556722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:41:49.462483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"265'171","reported_seq":806,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.584098+0000","last_change":"2024-05-30T15:54:16.678919+0000","last_active":"2024-05-30T17:11:21.584098+0000","last_peered":"2024-05-30T17:11:21.584098+0000","last_clean":"2024-05-30T17:11:21.584098+0000","last_became_active":"2024-05-30T15:54:16.678386+0000","last_became_peered":"2024-05-30T15:54:16.678386+0000","last_unstale":"2024-05-30T17:11:21.584098+0000","last_undegraded":"2024-05-30T17:11:21.584098+0000","last_fullsized":"2024-05-30T17:11:21.584098+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:06:51.673817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6324848,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":361,"num_read_kb":7665,"num_write":1079,"num_write_kb":33234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":510,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:12.137521+0000","last_change":"2024-05-30T15:54:20.663148+0000","last_active":"2024-05-30T17:10:12.137521+0000","last_peered":"2024-05-30T17:10:12.137521+0000","last_clean":"2024-05-30T17:10:12.137521+0000","last_became_active":"2024-05-30T15:54:20.661502+0000","last_became_peered":"2024-05-30T15:54:20.661502+0000","last_unstale":"2024-05-30T17:10:12.137521+0000","last_undegraded":"2024-05-30T17:10:12.137521+0000","last_fullsized":"2024-05-30T17:10:12.137521+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:53:08.492976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"265'706","reported_seq":2523,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:28.415096+0000","last_change":"2024-05-30T15:54:22.672464+0000","last_active":"2024-05-30T17:13:28.415096+0000","last_peered":"2024-05-30T17:13:28.415096+0000","last_clean":"2024-05-30T17:13:28.415096+0000","last_became_active":"2024-05-30T15:54:22.671588+0000","last_became_peered":"2024-05-30T15:54:22.671588+0000","last_unstale":"2024-05-30T17:13:28.415096+0000","last_undegraded":"2024-05-30T17:13:28.415096+0000","last_fullsized":"2024-05-30T17:13:28.415096+0000","mapping_epoch":28,"log_start":"265'600","ondisk_log_start":"265'600","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":600,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:46:49.368553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28054,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":978,"num_read_kb":1146,"num_write":312,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":503,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:46.546239+0000","last_change":"2024-05-30T15:54:24.688069+0000","last_active":"2024-05-30T17:09:46.546239+0000","last_peered":"2024-05-30T17:09:46.546239+0000","last_clean":"2024-05-30T17:09:46.546239+0000","last_became_active":"2024-05-30T15:54:24.686641+0000","last_became_peered":"2024-05-30T15:54:24.686641+0000","last_unstale":"2024-05-30T17:09:46.546239+0000","last_undegraded":"2024-05-30T17:09:46.546239+0000","last_fullsized":"2024-05-30T17:09:46.546239+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:27:01.277040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"265'123","reported_seq":634,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:31.780995+0000","last_change":"2024-05-30T15:54:26.668522+0000","last_active":"2024-05-30T17:10:31.780995+0000","last_peered":"2024-05-30T17:10:31.780995+0000","last_clean":"2024-05-30T17:10:31.780995+0000","last_became_active":"2024-05-30T15:54:26.667751+0000","last_became_peered":"2024-05-30T15:54:26.667751+0000","last_unstale":"2024-05-30T17:10:31.780995+0000","last_undegraded":"2024-05-30T17:10:31.780995+0000","last_fullsized":"2024-05-30T17:10:31.780995+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":100,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:32:24.025165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":438,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":115,"num_write":291,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"265'326","reported_seq":1646,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:33.881393+0000","last_change":"2024-05-30T15:54:18.652138+0000","last_active":"2024-05-30T17:12:33.881393+0000","last_peered":"2024-05-30T17:12:33.881393+0000","last_clean":"2024-05-30T17:12:33.881393+0000","last_became_active":"2024-05-30T15:54:18.651853+0000","last_became_peered":"2024-05-30T15:54:18.651853+0000","last_unstale":"2024-05-30T17:12:33.881393+0000","last_undegraded":"2024-05-30T17:12:33.881393+0000","last_fullsized":"2024-05-30T17:12:33.881393+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":300,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:07:58.160840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1727,"num_read_kb":1636,"num_write":373,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":533,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.557705+0000","last_change":"2024-05-30T15:54:08.745221+0000","last_active":"2024-05-30T17:11:11.557705+0000","last_peered":"2024-05-30T17:11:11.557705+0000","last_clean":"2024-05-30T17:11:11.557705+0000","last_became_active":"2024-05-30T15:54:08.744898+0000","last_became_peered":"2024-05-30T15:54:08.744898+0000","last_unstale":"2024-05-30T17:11:11.557705+0000","last_undegraded":"2024-05-30T17:11:11.557705+0000","last_fullsized":"2024-05-30T17:11:11.557705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:38:33.683752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"265'183","reported_seq":811,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:31.780836+0000","last_change":"2024-05-30T15:54:16.650710+0000","last_active":"2024-05-30T17:10:31.780836+0000","last_peered":"2024-05-30T17:10:31.780836+0000","last_clean":"2024-05-30T17:10:31.780836+0000","last_became_active":"2024-05-30T15:54:16.650214+0000","last_became_peered":"2024-05-30T15:54:16.650214+0000","last_unstale":"2024-05-30T17:10:31.780836+0000","last_undegraded":"2024-05-30T17:10:31.780836+0000","last_fullsized":"2024-05-30T17:10:31.780836+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":100,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:41:20.055617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7478597,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":410,"num_read_kb":8332,"num_write":1152,"num_write_kb":25808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"27'1","reported_seq":514,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:11.545623+0000","last_change":"2024-05-30T15:54:20.667150+0000","last_active":"2024-05-30T17:10:11.545623+0000","last_peered":"2024-05-30T17:10:11.545623+0000","last_clean":"2024-05-30T17:10:11.545623+0000","last_became_active":"2024-05-30T15:54:20.666715+0000","last_became_peered":"2024-05-30T15:54:20.666715+0000","last_unstale":"2024-05-30T17:10:11.545623+0000","last_undegraded":"2024-05-30T17:10:11.545623+0000","last_fullsized":"2024-05-30T17:10:11.545623+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:57:01.669091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"265'259","reported_seq":1771,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.166581+0000","last_change":"2024-05-30T15:54:22.853392+0000","last_active":"2024-05-30T17:13:24.166581+0000","last_peered":"2024-05-30T17:13:24.166581+0000","last_clean":"2024-05-30T17:13:24.166581+0000","last_became_active":"2024-05-30T15:54:22.852604+0000","last_became_peered":"2024-05-30T15:54:22.852604+0000","last_unstale":"2024-05-30T17:13:24.166581+0000","last_undegraded":"2024-05-30T17:13:24.166581+0000","last_fullsized":"2024-05-30T17:13:24.166581+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":200,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:13:39.908798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":232,"num_read_kb":224,"num_write":183,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":509,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:31.586843+0000","last_change":"2024-05-30T15:54:24.680439+0000","last_active":"2024-05-30T17:11:31.586843+0000","last_peered":"2024-05-30T17:11:31.586843+0000","last_clean":"2024-05-30T17:11:31.586843+0000","last_became_active":"2024-05-30T15:54:24.679830+0000","last_became_peered":"2024-05-30T15:54:24.679830+0000","last_unstale":"2024-05-30T17:11:31.586843+0000","last_undegraded":"2024-05-30T17:11:31.586843+0000","last_fullsized":"2024-05-30T17:11:31.586843+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:05:18.168600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"265'251","reported_seq":860,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:19.239543+0000","last_change":"2024-05-30T15:54:26.699114+0000","last_active":"2024-05-30T17:13:19.239543+0000","last_peered":"2024-05-30T17:13:19.239543+0000","last_clean":"2024-05-30T17:13:19.239543+0000","last_became_active":"2024-05-30T15:54:26.696477+0000","last_became_peered":"2024-05-30T15:54:26.696477+0000","last_unstale":"2024-05-30T17:13:19.239543+0000","last_undegraded":"2024-05-30T17:13:19.239543+0000","last_fullsized":"2024-05-30T17:13:19.239543+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":200,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:15:58.988777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":476,"num_read_kb":500,"num_write":542,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"265'761","reported_seq":2321,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:35.663643+0000","last_change":"2024-05-30T15:54:18.640139+0000","last_active":"2024-05-30T17:11:35.663643+0000","last_peered":"2024-05-30T17:11:35.663643+0000","last_clean":"2024-05-30T17:11:35.663643+0000","last_became_active":"2024-05-30T15:54:18.639830+0000","last_became_peered":"2024-05-30T15:54:18.639830+0000","last_unstale":"2024-05-30T17:11:35.663643+0000","last_undegraded":"2024-05-30T17:11:35.663643+0000","last_fullsized":"2024-05-30T17:11:35.663643+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":700,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:05:32.448981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3481,"num_read_kb":3132,"num_write":1014,"num_write_kb":367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":530,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:17.151871+0000","last_change":"2024-05-30T15:54:08.575969+0000","last_active":"2024-05-30T17:11:17.151871+0000","last_peered":"2024-05-30T17:11:17.151871+0000","last_clean":"2024-05-30T17:11:17.151871+0000","last_became_active":"2024-05-30T15:54:08.574817+0000","last_became_peered":"2024-05-30T15:54:08.574817+0000","last_unstale":"2024-05-30T17:11:17.151871+0000","last_undegraded":"2024-05-30T17:11:17.151871+0000","last_fullsized":"2024-05-30T17:11:17.151871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:39:30.889837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"265'194","reported_seq":909,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.749242+0000","last_change":"2024-05-30T15:54:16.656169+0000","last_active":"2024-05-30T17:11:21.749242+0000","last_peered":"2024-05-30T17:11:21.749242+0000","last_clean":"2024-05-30T17:11:21.749242+0000","last_became_active":"2024-05-30T15:54:16.655347+0000","last_became_peered":"2024-05-30T15:54:16.655347+0000","last_unstale":"2024-05-30T17:11:21.749242+0000","last_undegraded":"2024-05-30T17:11:21.749242+0000","last_fullsized":"2024-05-30T17:11:21.749242+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":100,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:46:03.699500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8740487,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":567,"num_read_kb":42208,"num_write":1143,"num_write_kb":36370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":516,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:41.561839+0000","last_change":"2024-05-30T15:54:20.651897+0000","last_active":"2024-05-30T17:09:41.561839+0000","last_peered":"2024-05-30T17:09:41.561839+0000","last_clean":"2024-05-30T17:09:41.561839+0000","last_became_active":"2024-05-30T15:54:20.651449+0000","last_became_peered":"2024-05-30T15:54:20.651449+0000","last_unstale":"2024-05-30T17:09:41.561839+0000","last_undegraded":"2024-05-30T17:09:41.561839+0000","last_fullsized":"2024-05-30T17:09:41.561839+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:14:31.569912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"265'160","reported_seq":732,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:16.572838+0000","last_change":"2024-05-30T15:54:22.679074+0000","last_active":"2024-05-30T17:10:16.572838+0000","last_peered":"2024-05-30T17:10:16.572838+0000","last_clean":"2024-05-30T17:10:16.572838+0000","last_became_active":"2024-05-30T15:54:22.673881+0000","last_became_peered":"2024-05-30T15:54:22.673881+0000","last_unstale":"2024-05-30T17:10:16.572838+0000","last_undegraded":"2024-05-30T17:10:16.572838+0000","last_fullsized":"2024-05-30T17:10:16.572838+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":100,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:18:38.849518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":200,"num_read_kb":192,"num_write":144,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":501,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:16.545213+0000","last_change":"2024-05-30T15:54:24.678710+0000","last_active":"2024-05-30T17:10:16.545213+0000","last_peered":"2024-05-30T17:10:16.545213+0000","last_clean":"2024-05-30T17:10:16.545213+0000","last_became_active":"2024-05-30T15:54:24.675916+0000","last_became_peered":"2024-05-30T15:54:24.675916+0000","last_unstale":"2024-05-30T17:10:16.545213+0000","last_undegraded":"2024-05-30T17:10:16.545213+0000","last_fullsized":"2024-05-30T17:10:16.545213+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:25:04.329603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"265'137","reported_seq":655,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.579761+0000","last_change":"2024-05-30T15:54:26.705809+0000","last_active":"2024-05-30T17:13:20.579761+0000","last_peered":"2024-05-30T17:13:20.579761+0000","last_clean":"2024-05-30T17:13:20.579761+0000","last_became_active":"2024-05-30T15:54:26.704217+0000","last_became_peered":"2024-05-30T15:54:26.704217+0000","last_unstale":"2024-05-30T17:13:20.579761+0000","last_undegraded":"2024-05-30T17:13:20.579761+0000","last_fullsized":"2024-05-30T17:13:20.579761+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":100,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:47:09.950969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":834,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":138,"num_read_kb":136,"num_write":341,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"265'396","reported_seq":1827,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.793186+0000","last_change":"2024-05-30T15:54:18.665286+0000","last_active":"2024-05-30T17:11:26.793186+0000","last_peered":"2024-05-30T17:11:26.793186+0000","last_clean":"2024-05-30T17:11:26.793186+0000","last_became_active":"2024-05-30T15:54:18.665034+0000","last_became_peered":"2024-05-30T15:54:18.665034+0000","last_unstale":"2024-05-30T17:11:26.793186+0000","last_undegraded":"2024-05-30T17:11:26.793186+0000","last_fullsized":"2024-05-30T17:11:26.793186+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":300,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:02:55.348044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2053,"num_read_kb":1920,"num_write":476,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"265'215","reported_seq":856,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:27.154326+0000","last_change":"2024-05-30T15:54:16.651678+0000","last_active":"2024-05-30T17:11:27.154326+0000","last_peered":"2024-05-30T17:11:27.154326+0000","last_clean":"2024-05-30T17:11:27.154326+0000","last_became_active":"2024-05-30T15:54:16.650735+0000","last_became_peered":"2024-05-30T15:54:16.650735+0000","last_unstale":"2024-05-30T17:11:27.154326+0000","last_undegraded":"2024-05-30T17:11:27.154326+0000","last_fullsized":"2024-05-30T17:11:27.154326+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":200,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:36:29.849007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6324891,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":418,"num_read_kb":5617,"num_write":981,"num_write_kb":15732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":531,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.790986+0000","last_change":"2024-05-30T15:54:08.568214+0000","last_active":"2024-05-30T17:11:16.790986+0000","last_peered":"2024-05-30T17:11:16.790986+0000","last_clean":"2024-05-30T17:11:16.790986+0000","last_became_active":"2024-05-30T15:54:08.567900+0000","last_became_peered":"2024-05-30T15:54:08.567900+0000","last_unstale":"2024-05-30T17:11:16.790986+0000","last_undegraded":"2024-05-30T17:11:16.790986+0000","last_fullsized":"2024-05-30T17:11:16.790986+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:02:38.154765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":517,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.585770+0000","last_change":"2024-05-30T15:54:20.654924+0000","last_active":"2024-05-30T17:11:26.585770+0000","last_peered":"2024-05-30T17:11:26.585770+0000","last_clean":"2024-05-30T17:11:26.585770+0000","last_became_active":"2024-05-30T15:54:20.653790+0000","last_became_peered":"2024-05-30T15:54:20.653790+0000","last_unstale":"2024-05-30T17:11:26.585770+0000","last_undegraded":"2024-05-30T17:11:26.585770+0000","last_fullsized":"2024-05-30T17:11:26.585770+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:40:52.633245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"265'353","reported_seq":1045,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.849366+0000","last_change":"2024-05-30T15:54:22.674716+0000","last_active":"2024-05-30T17:13:24.849366+0000","last_peered":"2024-05-30T17:13:24.849366+0000","last_clean":"2024-05-30T17:13:24.849366+0000","last_became_active":"2024-05-30T15:54:22.673903+0000","last_became_peered":"2024-05-30T15:54:22.673903+0000","last_unstale":"2024-05-30T17:13:24.849366+0000","last_undegraded":"2024-05-30T17:13:24.849366+0000","last_fullsized":"2024-05-30T17:13:24.849366+0000","mapping_epoch":28,"log_start":"265'300","ondisk_log_start":"265'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":300,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:29:10.448451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13965,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1122,"num_read_kb":1192,"num_write":1666,"num_write_kb":1473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":513,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:31.751534+0000","last_change":"2024-05-30T15:54:24.679592+0000","last_active":"2024-05-30T17:11:31.751534+0000","last_peered":"2024-05-30T17:11:31.751534+0000","last_clean":"2024-05-30T17:11:31.751534+0000","last_became_active":"2024-05-30T15:54:24.677198+0000","last_became_peered":"2024-05-30T15:54:24.677198+0000","last_unstale":"2024-05-30T17:11:31.751534+0000","last_undegraded":"2024-05-30T17:11:31.751534+0000","last_fullsized":"2024-05-30T17:11:31.751534+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:47:11.591445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"265'117","reported_seq":635,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.126888+0000","last_change":"2024-05-30T15:54:26.676873+0000","last_active":"2024-05-30T17:13:20.126888+0000","last_peered":"2024-05-30T17:13:20.126888+0000","last_clean":"2024-05-30T17:13:20.126888+0000","last_became_active":"2024-05-30T15:54:26.676078+0000","last_became_peered":"2024-05-30T15:54:26.676078+0000","last_unstale":"2024-05-30T17:13:20.126888+0000","last_undegraded":"2024-05-30T17:13:20.126888+0000","last_fullsized":"2024-05-30T17:13:20.126888+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:52:51.891250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":727,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":114,"num_read_kb":108,"num_write":274,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"265'791","reported_seq":2423,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.705651+0000","last_change":"2024-05-30T15:54:18.644630+0000","last_active":"2024-05-30T17:13:20.705651+0000","last_peered":"2024-05-30T17:13:20.705651+0000","last_clean":"2024-05-30T17:13:20.705651+0000","last_became_active":"2024-05-30T15:54:18.642683+0000","last_became_peered":"2024-05-30T15:54:18.642683+0000","last_unstale":"2024-05-30T17:13:20.705651+0000","last_undegraded":"2024-05-30T17:13:20.705651+0000","last_fullsized":"2024-05-30T17:13:20.705651+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":700,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:23:29.409655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3596,"num_read_kb":3243,"num_write":1037,"num_write_kb":360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"265'179","reported_seq":822,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:53.428135+0000","last_change":"2024-05-30T15:54:16.664218+0000","last_active":"2024-05-30T17:11:53.428135+0000","last_peered":"2024-05-30T17:11:53.428135+0000","last_clean":"2024-05-30T17:11:53.428135+0000","last_became_active":"2024-05-30T15:54:16.663360+0000","last_became_peered":"2024-05-30T15:54:16.663360+0000","last_unstale":"2024-05-30T17:11:53.428135+0000","last_undegraded":"2024-05-30T17:11:53.428135+0000","last_fullsized":"2024-05-30T17:11:53.428135+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":100,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:22:13.108688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7656467,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":402,"num_read_kb":30298,"num_write":1058,"num_write_kb":42840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":529,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:26.755968+0000","last_change":"2024-05-30T15:54:08.577379+0000","last_active":"2024-05-30T17:09:26.755968+0000","last_peered":"2024-05-30T17:09:26.755968+0000","last_clean":"2024-05-30T17:09:26.755968+0000","last_became_active":"2024-05-30T15:54:08.576779+0000","last_became_peered":"2024-05-30T15:54:08.576779+0000","last_unstale":"2024-05-30T17:09:26.755968+0000","last_undegraded":"2024-05-30T17:09:26.755968+0000","last_fullsized":"2024-05-30T17:09:26.755968+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:09:01.046338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":511,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.560832+0000","last_change":"2024-05-30T15:54:20.655937+0000","last_active":"2024-05-30T17:11:26.560832+0000","last_peered":"2024-05-30T17:11:26.560832+0000","last_clean":"2024-05-30T17:11:26.560832+0000","last_became_active":"2024-05-30T15:54:20.655220+0000","last_became_peered":"2024-05-30T15:54:20.655220+0000","last_unstale":"2024-05-30T17:11:26.560832+0000","last_undegraded":"2024-05-30T17:11:26.560832+0000","last_fullsized":"2024-05-30T17:11:26.560832+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:42:12.608224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"265'197","reported_seq":794,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:31.794205+0000","last_change":"2024-05-30T15:54:22.851840+0000","last_active":"2024-05-30T17:11:31.794205+0000","last_peered":"2024-05-30T17:11:31.794205+0000","last_clean":"2024-05-30T17:11:31.794205+0000","last_became_active":"2024-05-30T15:54:22.851338+0000","last_became_peered":"2024-05-30T15:54:22.851338+0000","last_unstale":"2024-05-30T17:11:31.794205+0000","last_undegraded":"2024-05-30T17:11:31.794205+0000","last_fullsized":"2024-05-30T17:11:31.794205+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":100,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:25:07.146907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":256,"num_read_kb":249,"num_write":168,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"265'554","reported_seq":2762,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.884870+0000","last_change":"2024-05-30T15:54:24.677979+0000","last_active":"2024-05-30T17:13:24.884870+0000","last_peered":"2024-05-30T17:13:24.884870+0000","last_clean":"2024-05-30T17:13:24.884870+0000","last_became_active":"2024-05-30T15:54:24.677176+0000","last_became_peered":"2024-05-30T15:54:24.677176+0000","last_unstale":"2024-05-30T17:13:24.884870+0000","last_undegraded":"2024-05-30T17:13:24.884870+0000","last_fullsized":"2024-05-30T17:13:24.884870+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":500,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:01:33.668829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"265'124","reported_seq":631,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.862550+0000","last_change":"2024-05-30T15:54:26.697967+0000","last_active":"2024-05-30T17:13:24.862550+0000","last_peered":"2024-05-30T17:13:24.862550+0000","last_clean":"2024-05-30T17:13:24.862550+0000","last_became_active":"2024-05-30T15:54:26.697294+0000","last_became_peered":"2024-05-30T15:54:26.697294+0000","last_unstale":"2024-05-30T17:13:24.862550+0000","last_undegraded":"2024-05-30T17:13:24.862550+0000","last_fullsized":"2024-05-30T17:13:24.862550+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":100,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:31:13.852916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1933,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":112,"num_read_kb":106,"num_write":288,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"265'716","reported_seq":2110,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.659105+0000","last_change":"2024-05-30T15:54:18.653261+0000","last_active":"2024-05-30T17:12:44.659105+0000","last_peered":"2024-05-30T17:12:44.659105+0000","last_clean":"2024-05-30T17:12:44.659105+0000","last_became_active":"2024-05-30T15:54:18.652981+0000","last_became_peered":"2024-05-30T15:54:18.652981+0000","last_unstale":"2024-05-30T17:12:44.659105+0000","last_undegraded":"2024-05-30T17:12:44.659105+0000","last_fullsized":"2024-05-30T17:12:44.659105+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":700,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:28:55.055044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3098,"num_read_kb":2766,"num_write":954,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"265'195","reported_seq":857,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:25.480699+0000","last_change":"2024-05-30T15:54:16.652512+0000","last_active":"2024-05-30T17:10:25.480699+0000","last_peered":"2024-05-30T17:10:25.480699+0000","last_clean":"2024-05-30T17:10:25.480699+0000","last_became_active":"2024-05-30T15:54:16.652033+0000","last_became_peered":"2024-05-30T15:54:16.652033+0000","last_unstale":"2024-05-30T17:10:25.480699+0000","last_undegraded":"2024-05-30T17:10:25.480699+0000","last_fullsized":"2024-05-30T17:10:25.480699+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":100,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:21:56.815021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11145910,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":510,"num_read_kb":15333,"num_write":1047,"num_write_kb":22470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":529,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.772799+0000","last_change":"2024-05-30T15:54:08.574181+0000","last_active":"2024-05-30T17:09:56.772799+0000","last_peered":"2024-05-30T17:09:56.772799+0000","last_clean":"2024-05-30T17:09:56.772799+0000","last_became_active":"2024-05-30T15:54:08.573754+0000","last_became_peered":"2024-05-30T15:54:08.573754+0000","last_unstale":"2024-05-30T17:09:56.772799+0000","last_undegraded":"2024-05-30T17:09:56.772799+0000","last_fullsized":"2024-05-30T17:09:56.772799+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:35:01.844361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":521,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.750400+0000","last_change":"2024-05-30T15:54:20.657666+0000","last_active":"2024-05-30T17:11:26.750400+0000","last_peered":"2024-05-30T17:11:26.750400+0000","last_clean":"2024-05-30T17:11:26.750400+0000","last_became_active":"2024-05-30T15:54:20.656941+0000","last_became_peered":"2024-05-30T15:54:20.656941+0000","last_unstale":"2024-05-30T17:11:26.750400+0000","last_undegraded":"2024-05-30T17:11:26.750400+0000","last_fullsized":"2024-05-30T17:11:26.750400+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:45:33.081783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"265'130","reported_seq":703,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:31.751592+0000","last_change":"2024-05-30T15:54:22.674809+0000","last_active":"2024-05-30T17:11:31.751592+0000","last_peered":"2024-05-30T17:11:31.751592+0000","last_clean":"2024-05-30T17:11:31.751592+0000","last_became_active":"2024-05-30T15:54:22.674027+0000","last_became_peered":"2024-05-30T15:54:22.674027+0000","last_unstale":"2024-05-30T17:11:31.751592+0000","last_undegraded":"2024-05-30T17:11:31.751592+0000","last_fullsized":"2024-05-30T17:11:31.751592+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:34:39.215953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":164,"num_read_kb":159,"num_write":108,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"265'547","reported_seq":2630,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:28.474744+0000","last_change":"2024-05-30T15:54:24.677368+0000","last_active":"2024-05-30T17:13:28.474744+0000","last_peered":"2024-05-30T17:13:28.474744+0000","last_clean":"2024-05-30T17:13:28.474744+0000","last_became_active":"2024-05-30T15:54:24.676608+0000","last_became_peered":"2024-05-30T15:54:24.676608+0000","last_unstale":"2024-05-30T17:13:28.474744+0000","last_undegraded":"2024-05-30T17:13:28.474744+0000","last_fullsized":"2024-05-30T17:13:28.474744+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":500,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:44:44.176765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"265'227","reported_seq":774,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:11.610342+0000","last_change":"2024-05-30T15:54:26.700019+0000","last_active":"2024-05-30T17:13:11.610342+0000","last_peered":"2024-05-30T17:13:11.610342+0000","last_clean":"2024-05-30T17:13:11.610342+0000","last_became_active":"2024-05-30T15:54:26.699148+0000","last_became_peered":"2024-05-30T15:54:26.699148+0000","last_unstale":"2024-05-30T17:13:11.610342+0000","last_undegraded":"2024-05-30T17:13:11.610342+0000","last_fullsized":"2024-05-30T17:13:11.610342+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":200,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:16:32.958184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":326,"num_read_kb":298,"num_write":504,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"265'747","reported_seq":2339,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:28.311866+0000","last_change":"2024-05-30T15:54:18.658639+0000","last_active":"2024-05-30T17:12:28.311866+0000","last_peered":"2024-05-30T17:12:28.311866+0000","last_clean":"2024-05-30T17:12:28.311866+0000","last_became_active":"2024-05-30T15:54:18.657842+0000","last_became_peered":"2024-05-30T15:54:18.657842+0000","last_unstale":"2024-05-30T17:12:28.311866+0000","last_undegraded":"2024-05-30T17:12:28.311866+0000","last_fullsized":"2024-05-30T17:12:28.311866+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":700,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:08:00.261523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3632,"num_read_kb":3285,"num_write":997,"num_write_kb":370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"265'227","reported_seq":983,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:40.014659+0000","last_change":"2024-05-30T15:54:16.657515+0000","last_active":"2024-05-30T17:10:40.014659+0000","last_peered":"2024-05-30T17:10:40.014659+0000","last_clean":"2024-05-30T17:10:40.014659+0000","last_became_active":"2024-05-30T15:54:16.657067+0000","last_became_peered":"2024-05-30T15:54:16.657067+0000","last_unstale":"2024-05-30T17:10:40.014659+0000","last_undegraded":"2024-05-30T17:10:40.014659+0000","last_fullsized":"2024-05-30T17:10:40.014659+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":200,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:03:23.557850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14883328,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":674,"num_read_kb":72431,"num_write":1151,"num_write_kb":30547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":527,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:01.541851+0000","last_change":"2024-05-30T15:54:08.578053+0000","last_active":"2024-05-30T17:10:01.541851+0000","last_peered":"2024-05-30T17:10:01.541851+0000","last_clean":"2024-05-30T17:10:01.541851+0000","last_became_active":"2024-05-30T15:54:08.577191+0000","last_became_peered":"2024-05-30T15:54:08.577191+0000","last_unstale":"2024-05-30T17:10:01.541851+0000","last_undegraded":"2024-05-30T17:10:01.541851+0000","last_fullsized":"2024-05-30T17:10:01.541851+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:38:36.674998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":515,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:41.578396+0000","last_change":"2024-05-30T15:54:20.653740+0000","last_active":"2024-05-30T17:10:41.578396+0000","last_peered":"2024-05-30T17:10:41.578396+0000","last_clean":"2024-05-30T17:10:41.578396+0000","last_became_active":"2024-05-30T15:54:20.653404+0000","last_became_peered":"2024-05-30T15:54:20.653404+0000","last_unstale":"2024-05-30T17:10:41.578396+0000","last_undegraded":"2024-05-30T17:10:41.578396+0000","last_fullsized":"2024-05-30T17:10:41.578396+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:17:55.928914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"265'334","reported_seq":1069,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.809682+0000","last_change":"2024-05-30T15:54:22.849490+0000","last_active":"2024-05-30T17:13:24.809682+0000","last_peered":"2024-05-30T17:13:24.809682+0000","last_clean":"2024-05-30T17:13:24.809682+0000","last_became_active":"2024-05-30T15:54:22.848761+0000","last_became_peered":"2024-05-30T15:54:22.848761+0000","last_unstale":"2024-05-30T17:13:24.809682+0000","last_undegraded":"2024-05-30T17:13:24.809682+0000","last_fullsized":"2024-05-30T17:13:24.809682+0000","mapping_epoch":28,"log_start":"265'300","ondisk_log_start":"265'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:40:14.623631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34688,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":498,"num_read_kb":700,"num_write":262,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:31.589903+0000","last_change":"2024-05-30T15:54:24.687584+0000","last_active":"2024-05-30T17:11:31.589903+0000","last_peered":"2024-05-30T17:11:31.589903+0000","last_clean":"2024-05-30T17:11:31.589903+0000","last_became_active":"2024-05-30T15:54:24.686801+0000","last_became_peered":"2024-05-30T15:54:24.686801+0000","last_unstale":"2024-05-30T17:11:31.589903+0000","last_undegraded":"2024-05-30T17:11:31.589903+0000","last_fullsized":"2024-05-30T17:11:31.589903+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:41:00.549932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"265'150","reported_seq":651,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:55.992645+0000","last_change":"2024-05-30T15:54:26.701426+0000","last_active":"2024-05-30T17:11:55.992645+0000","last_peered":"2024-05-30T17:11:55.992645+0000","last_clean":"2024-05-30T17:11:55.992645+0000","last_became_active":"2024-05-30T15:54:26.700325+0000","last_became_peered":"2024-05-30T15:54:26.700325+0000","last_unstale":"2024-05-30T17:11:55.992645+0000","last_undegraded":"2024-05-30T17:11:55.992645+0000","last_fullsized":"2024-05-30T17:11:55.992645+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":100,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:05:22.451018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":204,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":115,"num_write":342,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"265'149","reported_seq":722,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:21.183321+0000","last_change":"2024-05-30T15:54:26.687046+0000","last_active":"2024-05-30T17:13:21.183321+0000","last_peered":"2024-05-30T17:13:21.183321+0000","last_clean":"2024-05-30T17:13:21.183321+0000","last_became_active":"2024-05-30T15:54:26.686310+0000","last_became_peered":"2024-05-30T15:54:26.686310+0000","last_unstale":"2024-05-30T17:13:21.183321+0000","last_undegraded":"2024-05-30T17:13:21.183321+0000","last_fullsized":"2024-05-30T17:13:21.183321+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:16:45.128093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":227,"num_read_kb":222,"num_write":347,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"265'746","reported_seq":2045,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.689780+0000","last_change":"2024-05-30T15:54:18.674169+0000","last_active":"2024-05-30T17:13:18.689780+0000","last_peered":"2024-05-30T17:13:18.689780+0000","last_clean":"2024-05-30T17:13:18.689780+0000","last_became_active":"2024-05-30T15:54:18.673418+0000","last_became_peered":"2024-05-30T15:54:18.673418+0000","last_unstale":"2024-05-30T17:13:18.689780+0000","last_undegraded":"2024-05-30T17:13:18.689780+0000","last_fullsized":"2024-05-30T17:13:18.689780+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":700,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:47:19.313441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3258,"num_read_kb":2851,"num_write":1044,"num_write_kb":425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"265'197","reported_seq":816,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.543474+0000","last_change":"2024-05-30T15:54:54.451649+0000","last_active":"2024-05-30T17:09:56.543474+0000","last_peered":"2024-05-30T17:09:56.543474+0000","last_clean":"2024-05-30T17:09:56.543474+0000","last_became_active":"2024-05-30T15:54:54.450966+0000","last_became_peered":"2024-05-30T15:54:54.450966+0000","last_unstale":"2024-05-30T17:09:56.543474+0000","last_undegraded":"2024-05-30T17:09:56.543474+0000","last_fullsized":"2024-05-30T17:09:56.543474+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":100,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:16:23.208005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5857853,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":396,"num_read_kb":38159,"num_write":1159,"num_write_kb":35962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":508,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:01.760976+0000","last_change":"2024-05-30T15:54:20.665689+0000","last_active":"2024-05-30T17:09:01.760976+0000","last_peered":"2024-05-30T17:09:01.760976+0000","last_clean":"2024-05-30T17:09:01.760976+0000","last_became_active":"2024-05-30T15:54:20.664882+0000","last_became_peered":"2024-05-30T15:54:20.664882+0000","last_unstale":"2024-05-30T17:09:01.760976+0000","last_undegraded":"2024-05-30T17:09:01.760976+0000","last_fullsized":"2024-05-30T17:09:01.760976+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:08:16.396333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"265'149","reported_seq":743,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.588979+0000","last_change":"2024-05-30T15:54:22.672900+0000","last_active":"2024-05-30T17:11:26.588979+0000","last_peered":"2024-05-30T17:11:26.588979+0000","last_clean":"2024-05-30T17:11:26.588979+0000","last_became_active":"2024-05-30T15:54:22.672170+0000","last_became_peered":"2024-05-30T15:54:22.672170+0000","last_unstale":"2024-05-30T17:11:26.588979+0000","last_undegraded":"2024-05-30T17:11:26.588979+0000","last_fullsized":"2024-05-30T17:11:26.588979+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:07:18.306116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7618,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":196,"num_read_kb":242,"num_write":106,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":501,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:06.556338+0000","last_change":"2024-05-30T15:54:24.688552+0000","last_active":"2024-05-30T17:09:06.556338+0000","last_peered":"2024-05-30T17:09:06.556338+0000","last_clean":"2024-05-30T17:09:06.556338+0000","last_became_active":"2024-05-30T15:54:24.688053+0000","last_became_peered":"2024-05-30T15:54:24.688053+0000","last_unstale":"2024-05-30T17:09:06.556338+0000","last_undegraded":"2024-05-30T17:09:06.556338+0000","last_fullsized":"2024-05-30T17:09:06.556338+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:53:25.256385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"265'117","reported_seq":642,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.579861+0000","last_change":"2024-05-30T15:54:26.702292+0000","last_active":"2024-05-30T17:10:46.579861+0000","last_peered":"2024-05-30T17:10:46.579861+0000","last_clean":"2024-05-30T17:10:46.579861+0000","last_became_active":"2024-05-30T15:54:26.701588+0000","last_became_peered":"2024-05-30T15:54:26.701588+0000","last_unstale":"2024-05-30T17:10:46.579861+0000","last_undegraded":"2024-05-30T17:10:46.579861+0000","last_fullsized":"2024-05-30T17:10:46.579861+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:13:38.040108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":826,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":172,"num_read_kb":153,"num_write":293,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"265'321","reported_seq":1758,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:55.923011+0000","last_change":"2024-05-30T15:54:18.671021+0000","last_active":"2024-05-30T17:11:55.923011+0000","last_peered":"2024-05-30T17:11:55.923011+0000","last_clean":"2024-05-30T17:11:55.923011+0000","last_became_active":"2024-05-30T15:54:18.670707+0000","last_became_peered":"2024-05-30T15:54:18.670707+0000","last_unstale":"2024-05-30T17:11:55.923011+0000","last_undegraded":"2024-05-30T17:11:55.923011+0000","last_fullsized":"2024-05-30T17:11:55.923011+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":300,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:20:52.055626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1910,"num_read_kb":1828,"num_write":376,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"265'215","reported_seq":879,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:11.752074+0000","last_change":"2024-05-30T15:54:16.663769+0000","last_active":"2024-05-30T17:09:11.752074+0000","last_peered":"2024-05-30T17:09:11.752074+0000","last_clean":"2024-05-30T17:09:11.752074+0000","last_became_active":"2024-05-30T15:54:16.660427+0000","last_became_peered":"2024-05-30T15:54:16.660427+0000","last_unstale":"2024-05-30T17:09:11.752074+0000","last_undegraded":"2024-05-30T17:09:11.752074+0000","last_fullsized":"2024-05-30T17:09:11.752074+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":200,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:37:35.566472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5852361,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":514,"num_read_kb":33224,"num_write":1144,"num_write_kb":46290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":509,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:01.555309+0000","last_change":"2024-05-30T15:54:20.649980+0000","last_active":"2024-05-30T17:09:01.555309+0000","last_peered":"2024-05-30T17:09:01.555309+0000","last_clean":"2024-05-30T17:09:01.555309+0000","last_became_active":"2024-05-30T15:54:20.649330+0000","last_became_peered":"2024-05-30T15:54:20.649330+0000","last_unstale":"2024-05-30T17:09:01.555309+0000","last_undegraded":"2024-05-30T17:09:01.555309+0000","last_fullsized":"2024-05-30T17:09:01.555309+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:16:31.107653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"265'71","reported_seq":595,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:06.552569+0000","last_change":"2024-05-30T15:54:22.663453+0000","last_active":"2024-05-30T17:09:06.552569+0000","last_peered":"2024-05-30T17:09:06.552569+0000","last_clean":"2024-05-30T17:09:06.552569+0000","last_became_active":"2024-05-30T15:54:22.662738+0000","last_became_peered":"2024-05-30T15:54:22.662738+0000","last_unstale":"2024-05-30T17:09:06.552569+0000","last_undegraded":"2024-05-30T17:09:06.552569+0000","last_fullsized":"2024-05-30T17:09:06.552569+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:42:21.227185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":50,"num_write":40,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":499,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:06.750855+0000","last_change":"2024-05-30T15:54:24.682005+0000","last_active":"2024-05-30T17:09:06.750855+0000","last_peered":"2024-05-30T17:09:06.750855+0000","last_clean":"2024-05-30T17:09:06.750855+0000","last_became_active":"2024-05-30T15:54:24.680578+0000","last_became_peered":"2024-05-30T15:54:24.680578+0000","last_unstale":"2024-05-30T17:09:06.750855+0000","last_undegraded":"2024-05-30T17:09:06.750855+0000","last_fullsized":"2024-05-30T17:09:06.750855+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:54:25.321085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"265'212","reported_seq":772,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.665178+0000","last_change":"2024-05-30T15:54:26.687158+0000","last_active":"2024-05-30T17:13:18.665178+0000","last_peered":"2024-05-30T17:13:18.665178+0000","last_clean":"2024-05-30T17:13:18.665178+0000","last_became_active":"2024-05-30T15:54:26.686553+0000","last_became_peered":"2024-05-30T15:54:26.686553+0000","last_unstale":"2024-05-30T17:13:18.665178+0000","last_undegraded":"2024-05-30T17:13:18.665178+0000","last_fullsized":"2024-05-30T17:13:18.665178+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:03:50.128325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":816,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":414,"num_read_kb":383,"num_write":513,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"265'413","reported_seq":1789,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.600546+0000","last_change":"2024-05-30T15:54:18.660456+0000","last_active":"2024-05-30T17:12:44.600546+0000","last_peered":"2024-05-30T17:12:44.600546+0000","last_clean":"2024-05-30T17:12:44.600546+0000","last_became_active":"2024-05-30T15:54:18.660089+0000","last_became_peered":"2024-05-30T15:54:18.660089+0000","last_unstale":"2024-05-30T17:12:44.600546+0000","last_undegraded":"2024-05-30T17:12:44.600546+0000","last_fullsized":"2024-05-30T17:12:44.600546+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":400,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:29:46.343317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2036,"num_read_kb":1890,"num_write":519,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"265'214","reported_seq":883,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:12.123493+0000","last_change":"2024-05-30T15:54:16.650752+0000","last_active":"2024-05-30T17:09:12.123493+0000","last_peered":"2024-05-30T17:09:12.123493+0000","last_clean":"2024-05-30T17:09:12.123493+0000","last_became_active":"2024-05-30T15:54:16.650025+0000","last_became_peered":"2024-05-30T15:54:16.650025+0000","last_unstale":"2024-05-30T17:09:12.123493+0000","last_undegraded":"2024-05-30T17:09:12.123493+0000","last_fullsized":"2024-05-30T17:09:12.123493+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":200,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:14:35.914964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5917240,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":497,"num_read_kb":102670,"num_write":1110,"num_write_kb":39714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":507,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:01.749537+0000","last_change":"2024-05-30T15:54:20.657385+0000","last_active":"2024-05-30T17:09:01.749537+0000","last_peered":"2024-05-30T17:09:01.749537+0000","last_clean":"2024-05-30T17:09:01.749537+0000","last_became_active":"2024-05-30T15:54:20.656663+0000","last_became_peered":"2024-05-30T15:54:20.656663+0000","last_unstale":"2024-05-30T17:09:01.749537+0000","last_undegraded":"2024-05-30T17:09:01.749537+0000","last_fullsized":"2024-05-30T17:09:01.749537+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:31:26.705085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"265'182","reported_seq":792,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.589082+0000","last_change":"2024-05-30T15:54:22.676934+0000","last_active":"2024-05-30T17:11:26.589082+0000","last_peered":"2024-05-30T17:11:26.589082+0000","last_clean":"2024-05-30T17:11:26.589082+0000","last_became_active":"2024-05-30T15:54:22.673673+0000","last_became_peered":"2024-05-30T15:54:22.673673+0000","last_unstale":"2024-05-30T17:11:26.589082+0000","last_undegraded":"2024-05-30T17:11:26.589082+0000","last_fullsized":"2024-05-30T17:11:26.589082+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":100,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:34:48.241130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10523,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":250,"num_read_kb":311,"num_write":142,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":498,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:06.529823+0000","last_change":"2024-05-30T15:54:24.676055+0000","last_active":"2024-05-30T17:09:06.529823+0000","last_peered":"2024-05-30T17:09:06.529823+0000","last_clean":"2024-05-30T17:09:06.529823+0000","last_became_active":"2024-05-30T15:54:24.675289+0000","last_became_peered":"2024-05-30T15:54:24.675289+0000","last_unstale":"2024-05-30T17:09:06.529823+0000","last_undegraded":"2024-05-30T17:09:06.529823+0000","last_fullsized":"2024-05-30T17:09:06.529823+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:37:01.468842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"265'138","reported_seq":634,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:27.127163+0000","last_change":"2024-05-30T15:54:26.704882+0000","last_active":"2024-05-30T17:09:27.127163+0000","last_peered":"2024-05-30T17:09:27.127163+0000","last_clean":"2024-05-30T17:09:27.127163+0000","last_became_active":"2024-05-30T15:54:26.703433+0000","last_became_peered":"2024-05-30T15:54:26.703433+0000","last_unstale":"2024-05-30T17:09:27.127163+0000","last_undegraded":"2024-05-30T17:09:27.127163+0000","last_fullsized":"2024-05-30T17:09:27.127163+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":100,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:19:12.878679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":856,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":110,"num_write":329,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"265'863","reported_seq":2490,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:33.948079+0000","last_change":"2024-05-30T15:54:18.649560+0000","last_active":"2024-05-30T17:12:33.948079+0000","last_peered":"2024-05-30T17:12:33.948079+0000","last_clean":"2024-05-30T17:12:33.948079+0000","last_became_active":"2024-05-30T15:54:18.649273+0000","last_became_peered":"2024-05-30T15:54:18.649273+0000","last_unstale":"2024-05-30T17:12:33.948079+0000","last_undegraded":"2024-05-30T17:12:33.948079+0000","last_fullsized":"2024-05-30T17:12:33.948079+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":800,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:31:47.570692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3918,"num_read_kb":3498,"num_write":1187,"num_write_kb":458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"265'202","reported_seq":862,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:44.661866+0000","last_change":"2024-05-30T15:54:16.649854+0000","last_active":"2024-05-30T17:09:44.661866+0000","last_peered":"2024-05-30T17:09:44.661866+0000","last_clean":"2024-05-30T17:09:44.661866+0000","last_became_active":"2024-05-30T15:54:16.649121+0000","last_became_peered":"2024-05-30T15:54:16.649121+0000","last_unstale":"2024-05-30T17:09:44.661866+0000","last_undegraded":"2024-05-30T17:09:44.661866+0000","last_fullsized":"2024-05-30T17:09:44.661866+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:25:59.798639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12165036,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":488,"num_read_kb":65907,"num_write":1115,"num_write_kb":54558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:01.528692+0000","last_change":"2024-05-30T15:54:20.668691+0000","last_active":"2024-05-30T17:09:01.528692+0000","last_peered":"2024-05-30T17:09:01.528692+0000","last_clean":"2024-05-30T17:09:01.528692+0000","last_became_active":"2024-05-30T15:54:20.667863+0000","last_became_peered":"2024-05-30T15:54:20.667863+0000","last_unstale":"2024-05-30T17:09:01.528692+0000","last_undegraded":"2024-05-30T17:09:01.528692+0000","last_fullsized":"2024-05-30T17:09:01.528692+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:31.629459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"265'139","reported_seq":724,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.570167+0000","last_change":"2024-05-30T15:54:22.665742+0000","last_active":"2024-05-30T17:11:26.570167+0000","last_peered":"2024-05-30T17:11:26.570167+0000","last_clean":"2024-05-30T17:11:26.570167+0000","last_became_active":"2024-05-30T15:54:22.664977+0000","last_became_peered":"2024-05-30T15:54:22.664977+0000","last_unstale":"2024-05-30T17:11:26.570167+0000","last_undegraded":"2024-05-30T17:11:26.570167+0000","last_fullsized":"2024-05-30T17:11:26.570167+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":100,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:20:57.960795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10584,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":178,"num_read_kb":239,"num_write":94,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":504,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:06.552717+0000","last_change":"2024-05-30T15:54:24.677137+0000","last_active":"2024-05-30T17:09:06.552717+0000","last_peered":"2024-05-30T17:09:06.552717+0000","last_clean":"2024-05-30T17:09:06.552717+0000","last_became_active":"2024-05-30T15:54:24.676707+0000","last_became_peered":"2024-05-30T15:54:24.676707+0000","last_unstale":"2024-05-30T17:09:06.552717+0000","last_undegraded":"2024-05-30T17:09:06.552717+0000","last_fullsized":"2024-05-30T17:09:06.552717+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:37:14.112070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"265'442","reported_seq":2099,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:28.309213+0000","last_change":"2024-05-30T15:54:18.671501+0000","last_active":"2024-05-30T17:12:28.309213+0000","last_peered":"2024-05-30T17:12:28.309213+0000","last_clean":"2024-05-30T17:12:28.309213+0000","last_became_active":"2024-05-30T15:54:18.670756+0000","last_became_peered":"2024-05-30T15:54:18.670756+0000","last_unstale":"2024-05-30T17:12:28.309213+0000","last_undegraded":"2024-05-30T17:12:28.309213+0000","last_fullsized":"2024-05-30T17:12:28.309213+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":400,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:50:30.677735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2383,"num_read_kb":2277,"num_write":522,"num_write_kb":97,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"265'154","reported_seq":758,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:22.227972+0000","last_change":"2024-05-30T15:54:16.654636+0000","last_active":"2024-05-30T17:10:22.227972+0000","last_peered":"2024-05-30T17:10:22.227972+0000","last_clean":"2024-05-30T17:10:22.227972+0000","last_became_active":"2024-05-30T15:54:16.653560+0000","last_became_peered":"2024-05-30T15:54:16.653560+0000","last_unstale":"2024-05-30T17:10:22.227972+0000","last_undegraded":"2024-05-30T17:10:22.227972+0000","last_fullsized":"2024-05-30T17:10:22.227972+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":100,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:47:43.349582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5814859,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":316,"num_read_kb":31907,"num_write":883,"num_write_kb":20958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":518,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:41.574743+0000","last_change":"2024-05-30T15:54:20.662992+0000","last_active":"2024-05-30T17:10:41.574743+0000","last_peered":"2024-05-30T17:10:41.574743+0000","last_clean":"2024-05-30T17:10:41.574743+0000","last_became_active":"2024-05-30T15:54:20.662669+0000","last_became_peered":"2024-05-30T15:54:20.662669+0000","last_unstale":"2024-05-30T17:10:41.574743+0000","last_undegraded":"2024-05-30T17:10:41.574743+0000","last_fullsized":"2024-05-30T17:10:41.574743+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:46:17.417038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"265'216","reported_seq":857,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.843270+0000","last_change":"2024-05-30T15:54:22.664232+0000","last_active":"2024-05-30T17:13:24.843270+0000","last_peered":"2024-05-30T17:13:24.843270+0000","last_clean":"2024-05-30T17:13:24.843270+0000","last_became_active":"2024-05-30T15:54:22.663518+0000","last_became_peered":"2024-05-30T15:54:22.663518+0000","last_unstale":"2024-05-30T17:13:24.843270+0000","last_undegraded":"2024-05-30T17:13:24.843270+0000","last_fullsized":"2024-05-30T17:13:24.843270+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":200,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:51:09.801047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18037,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":324,"num_read_kb":450,"num_write":214,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":497,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:27.127216+0000","last_change":"2024-05-30T15:54:24.683358+0000","last_active":"2024-05-30T17:09:27.127216+0000","last_peered":"2024-05-30T17:09:27.127216+0000","last_clean":"2024-05-30T17:09:27.127216+0000","last_became_active":"2024-05-30T15:54:24.682609+0000","last_became_peered":"2024-05-30T15:54:24.682609+0000","last_unstale":"2024-05-30T17:09:27.127216+0000","last_undegraded":"2024-05-30T17:09:27.127216+0000","last_fullsized":"2024-05-30T17:09:27.127216+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:15:33.566334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"265'149","reported_seq":660,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:12.359173+0000","last_change":"2024-05-30T15:54:26.695969+0000","last_active":"2024-05-30T17:11:12.359173+0000","last_peered":"2024-05-30T17:11:12.359173+0000","last_clean":"2024-05-30T17:11:12.359173+0000","last_became_active":"2024-05-30T15:54:26.695210+0000","last_became_peered":"2024-05-30T15:54:26.695210+0000","last_unstale":"2024-05-30T17:11:12.359173+0000","last_undegraded":"2024-05-30T17:11:12.359173+0000","last_fullsized":"2024-05-30T17:11:12.359173+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:13:43.303060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":643,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":167,"num_read_kb":179,"num_write":338,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"265'854","reported_seq":2475,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.655415+0000","last_change":"2024-05-30T15:54:18.641464+0000","last_active":"2024-05-30T17:12:44.655415+0000","last_peered":"2024-05-30T17:12:44.655415+0000","last_clean":"2024-05-30T17:12:44.655415+0000","last_became_active":"2024-05-30T15:54:18.640739+0000","last_became_peered":"2024-05-30T15:54:18.640739+0000","last_unstale":"2024-05-30T17:12:44.655415+0000","last_undegraded":"2024-05-30T17:12:44.655415+0000","last_fullsized":"2024-05-30T17:12:44.655415+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":800,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:05:20.428299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4053,"num_read_kb":3626,"num_write":1150,"num_write_kb":447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"265'241","reported_seq":974,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:28.400625+0000","last_change":"2024-05-30T15:54:54.425702+0000","last_active":"2024-05-30T17:09:28.400625+0000","last_peered":"2024-05-30T17:09:28.400625+0000","last_clean":"2024-05-30T17:09:28.400625+0000","last_became_active":"2024-05-30T15:54:54.424825+0000","last_became_peered":"2024-05-30T15:54:54.424825+0000","last_unstale":"2024-05-30T17:09:28.400625+0000","last_undegraded":"2024-05-30T17:09:28.400625+0000","last_fullsized":"2024-05-30T17:09:28.400625+0000","mapping_epoch":35,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":200,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:43:34.336982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11204870,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":883,"num_read_kb":12756,"num_write":1226,"num_write_kb":34615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"34'3","reported_seq":836,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.748812+0000","last_change":"2024-05-30T15:54:20.664818+0000","last_active":"2024-05-30T17:13:24.748812+0000","last_peered":"2024-05-30T17:13:24.748812+0000","last_clean":"2024-05-30T17:13:24.748812+0000","last_became_active":"2024-05-30T15:54:20.664095+0000","last_became_peered":"2024-05-30T15:54:20.664095+0000","last_unstale":"2024-05-30T17:13:24.748812+0000","last_undegraded":"2024-05-30T17:13:24.748812+0000","last_fullsized":"2024-05-30T17:13:24.748812+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:27:22.067783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":491,"num_read_kb":489,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"265'190","reported_seq":796,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:56.578238+0000","last_change":"2024-05-30T15:54:22.665708+0000","last_active":"2024-05-30T17:11:56.578238+0000","last_peered":"2024-05-30T17:11:56.578238+0000","last_clean":"2024-05-30T17:11:56.578238+0000","last_became_active":"2024-05-30T15:54:22.664997+0000","last_became_peered":"2024-05-30T15:54:22.664997+0000","last_unstale":"2024-05-30T17:11:56.578238+0000","last_undegraded":"2024-05-30T17:11:56.578238+0000","last_fullsized":"2024-05-30T17:11:56.578238+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":100,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:45:04.949797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28724,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":281,"num_read_kb":391,"num_write":188,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":498,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:26.767043+0000","last_change":"2024-05-30T15:54:24.678998+0000","last_active":"2024-05-30T17:09:26.767043+0000","last_peered":"2024-05-30T17:09:26.767043+0000","last_clean":"2024-05-30T17:09:26.767043+0000","last_became_active":"2024-05-30T15:54:24.675583+0000","last_became_peered":"2024-05-30T15:54:24.675583+0000","last_unstale":"2024-05-30T17:09:26.767043+0000","last_undegraded":"2024-05-30T17:09:26.767043+0000","last_fullsized":"2024-05-30T17:09:26.767043+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:17:21.969608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"265'140","reported_seq":655,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.560874+0000","last_change":"2024-05-30T15:54:26.703108+0000","last_active":"2024-05-30T17:11:26.560874+0000","last_peered":"2024-05-30T17:11:26.560874+0000","last_clean":"2024-05-30T17:11:26.560874+0000","last_became_active":"2024-05-30T15:54:26.700392+0000","last_became_peered":"2024-05-30T15:54:26.700392+0000","last_unstale":"2024-05-30T17:11:26.560874+0000","last_undegraded":"2024-05-30T17:11:26.560874+0000","last_fullsized":"2024-05-30T17:11:26.560874+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":100,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:25:14.903459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2627,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":177,"num_read_kb":174,"num_write":403,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"265'368","reported_seq":1738,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:33.884093+0000","last_change":"2024-05-30T15:54:54.420129+0000","last_active":"2024-05-30T17:12:33.884093+0000","last_peered":"2024-05-30T17:12:33.884093+0000","last_clean":"2024-05-30T17:12:33.884093+0000","last_became_active":"2024-05-30T15:54:54.419468+0000","last_became_peered":"2024-05-30T15:54:54.419468+0000","last_unstale":"2024-05-30T17:12:33.884093+0000","last_undegraded":"2024-05-30T17:12:33.884093+0000","last_fullsized":"2024-05-30T17:12:33.884093+0000","mapping_epoch":35,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":300,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:08:55.802490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1898,"num_read_kb":1782,"num_write":455,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"265'156","reported_seq":881,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:52.778205+0000","last_change":"2024-05-30T15:54:16.677283+0000","last_active":"2024-05-30T17:12:52.778205+0000","last_peered":"2024-05-30T17:12:52.778205+0000","last_clean":"2024-05-30T17:12:52.778205+0000","last_became_active":"2024-05-30T15:54:16.676440+0000","last_became_peered":"2024-05-30T15:54:16.676440+0000","last_unstale":"2024-05-30T17:12:52.778205+0000","last_undegraded":"2024-05-30T17:12:52.778205+0000","last_fullsized":"2024-05-30T17:12:52.778205+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":100,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:16:12.516128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9724052,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":866,"num_read_kb":45599,"num_write":922,"num_write_kb":34636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":514,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.723009+0000","last_change":"2024-05-30T15:54:20.656111+0000","last_active":"2024-05-30T17:09:21.723009+0000","last_peered":"2024-05-30T17:09:21.723009+0000","last_clean":"2024-05-30T17:09:21.723009+0000","last_became_active":"2024-05-30T15:54:20.655388+0000","last_became_peered":"2024-05-30T15:54:20.655388+0000","last_unstale":"2024-05-30T17:09:21.723009+0000","last_undegraded":"2024-05-30T17:09:21.723009+0000","last_fullsized":"2024-05-30T17:09:21.723009+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:07:28.254294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"265'161","reported_seq":760,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.837510+0000","last_change":"2024-05-30T15:54:22.849861+0000","last_active":"2024-05-30T17:13:24.837510+0000","last_peered":"2024-05-30T17:13:24.837510+0000","last_clean":"2024-05-30T17:13:24.837510+0000","last_became_active":"2024-05-30T15:54:22.849193+0000","last_became_peered":"2024-05-30T15:54:22.849193+0000","last_unstale":"2024-05-30T17:13:24.837510+0000","last_undegraded":"2024-05-30T17:13:24.837510+0000","last_fullsized":"2024-05-30T17:13:24.837510+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":100,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:51:38.667242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13488,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":212,"num_read_kb":283,"num_write":120,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:41.783470+0000","last_change":"2024-05-30T15:54:24.682047+0000","last_active":"2024-05-30T17:10:41.783470+0000","last_peered":"2024-05-30T17:10:41.783470+0000","last_clean":"2024-05-30T17:10:41.783470+0000","last_became_active":"2024-05-30T15:54:24.681609+0000","last_became_peered":"2024-05-30T15:54:24.681609+0000","last_unstale":"2024-05-30T17:10:41.783470+0000","last_undegraded":"2024-05-30T17:10:41.783470+0000","last_fullsized":"2024-05-30T17:10:41.783470+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:36:01.484941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"265'129","reported_seq":631,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.853790+0000","last_change":"2024-05-30T15:54:26.706376+0000","last_active":"2024-05-30T17:13:24.853790+0000","last_peered":"2024-05-30T17:13:24.853790+0000","last_clean":"2024-05-30T17:13:24.853790+0000","last_became_active":"2024-05-30T15:54:26.704842+0000","last_became_peered":"2024-05-30T15:54:26.704842+0000","last_unstale":"2024-05-30T17:13:24.853790+0000","last_undegraded":"2024-05-30T17:13:24.853790+0000","last_fullsized":"2024-05-30T17:13:24.853790+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":100,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:31:31.301239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":383,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":127,"num_read_kb":122,"num_write":322,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"265'753","reported_seq":2251,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:51.565345+0000","last_change":"2024-05-30T15:54:18.635089+0000","last_active":"2024-05-30T17:11:51.565345+0000","last_peered":"2024-05-30T17:11:51.565345+0000","last_clean":"2024-05-30T17:11:51.565345+0000","last_became_active":"2024-05-30T15:54:18.634803+0000","last_became_peered":"2024-05-30T15:54:18.634803+0000","last_unstale":"2024-05-30T17:11:51.565345+0000","last_undegraded":"2024-05-30T17:11:51.565345+0000","last_fullsized":"2024-05-30T17:11:51.565345+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":700,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:29:09.056772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":3476,"num_read_kb":3109,"num_write":1006,"num_write_kb":389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"265'186","reported_seq":810,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.543434+0000","last_change":"2024-05-30T15:54:54.431157+0000","last_active":"2024-05-30T17:09:56.543434+0000","last_peered":"2024-05-30T17:09:56.543434+0000","last_clean":"2024-05-30T17:09:56.543434+0000","last_became_active":"2024-05-30T15:54:54.430490+0000","last_became_peered":"2024-05-30T15:54:54.430490+0000","last_unstale":"2024-05-30T17:09:56.543434+0000","last_undegraded":"2024-05-30T17:09:56.543434+0000","last_fullsized":"2024-05-30T17:09:56.543434+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":100,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:47:17.028289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6643361,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":477,"num_read_kb":40323,"num_write":1071,"num_write_kb":31617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.765694+0000","last_change":"2024-05-30T15:54:20.651307+0000","last_active":"2024-05-30T17:09:21.765694+0000","last_peered":"2024-05-30T17:09:21.765694+0000","last_clean":"2024-05-30T17:09:21.765694+0000","last_became_active":"2024-05-30T15:54:20.650944+0000","last_became_peered":"2024-05-30T15:54:20.650944+0000","last_unstale":"2024-05-30T17:09:21.765694+0000","last_undegraded":"2024-05-30T17:09:21.765694+0000","last_fullsized":"2024-05-30T17:09:21.765694+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:44:01.827195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"265'113","reported_seq":671,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:41.740281+0000","last_change":"2024-05-30T15:54:22.851946+0000","last_active":"2024-05-30T17:10:41.740281+0000","last_peered":"2024-05-30T17:10:41.740281+0000","last_clean":"2024-05-30T17:10:41.740281+0000","last_became_active":"2024-05-30T15:54:22.850865+0000","last_became_peered":"2024-05-30T15:54:22.850865+0000","last_unstale":"2024-05-30T17:10:41.740281+0000","last_undegraded":"2024-05-30T17:10:41.740281+0000","last_fullsized":"2024-05-30T17:10:41.740281+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":100,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:49:42.844967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":127,"num_read_kb":121,"num_write":92,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"265'538","reported_seq":4166,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:28.422661+0000","last_change":"2024-05-30T15:54:24.681428+0000","last_active":"2024-05-30T17:13:28.422661+0000","last_peered":"2024-05-30T17:13:28.422661+0000","last_clean":"2024-05-30T17:13:28.422661+0000","last_became_active":"2024-05-30T15:54:24.680695+0000","last_became_peered":"2024-05-30T15:54:24.680695+0000","last_unstale":"2024-05-30T17:13:28.422661+0000","last_undegraded":"2024-05-30T17:13:28.422661+0000","last_fullsized":"2024-05-30T17:13:28.422661+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":500,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:53:20.536261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"265'134","reported_seq":641,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:35.623018+0000","last_change":"2024-05-30T15:54:26.696600+0000","last_active":"2024-05-30T17:11:35.623018+0000","last_peered":"2024-05-30T17:11:35.623018+0000","last_clean":"2024-05-30T17:11:35.623018+0000","last_became_active":"2024-05-30T15:54:26.695842+0000","last_became_peered":"2024-05-30T15:54:26.695842+0000","last_unstale":"2024-05-30T17:11:35.623018+0000","last_undegraded":"2024-05-30T17:11:35.623018+0000","last_fullsized":"2024-05-30T17:11:35.623018+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:01:21.719509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":418,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":133,"num_read_kb":127,"num_write":334,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"265'423","reported_seq":1942,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.652772+0000","last_change":"2024-05-30T15:54:18.661318+0000","last_active":"2024-05-30T17:13:20.652772+0000","last_peered":"2024-05-30T17:13:20.652772+0000","last_clean":"2024-05-30T17:13:20.652772+0000","last_became_active":"2024-05-30T15:54:18.660567+0000","last_became_peered":"2024-05-30T15:54:18.660567+0000","last_unstale":"2024-05-30T17:13:20.652772+0000","last_undegraded":"2024-05-30T17:13:20.652772+0000","last_fullsized":"2024-05-30T17:13:20.652772+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":400,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:24:20.096151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2226,"num_read_kb":2104,"num_write":504,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"265'303","reported_seq":994,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:17.124619+0000","last_change":"2024-05-30T15:54:16.686321+0000","last_active":"2024-05-30T17:09:17.124619+0000","last_peered":"2024-05-30T17:09:17.124619+0000","last_clean":"2024-05-30T17:09:17.124619+0000","last_became_active":"2024-05-30T15:54:16.685691+0000","last_became_peered":"2024-05-30T15:54:16.685691+0000","last_unstale":"2024-05-30T17:09:17.124619+0000","last_undegraded":"2024-05-30T17:09:17.124619+0000","last_fullsized":"2024-05-30T17:09:17.124619+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":200,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:37:07.773513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":379054,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":761,"num_read_kb":1764,"num_write":1250,"num_write_kb":18095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":505,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.754576+0000","last_change":"2024-05-30T15:54:20.670328+0000","last_active":"2024-05-30T17:09:21.754576+0000","last_peered":"2024-05-30T17:09:21.754576+0000","last_clean":"2024-05-30T17:09:21.754576+0000","last_became_active":"2024-05-30T15:54:20.669751+0000","last_became_peered":"2024-05-30T15:54:20.669751+0000","last_unstale":"2024-05-30T17:09:21.754576+0000","last_undegraded":"2024-05-30T17:09:21.754576+0000","last_fullsized":"2024-05-30T17:09:21.754576+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:06:46.261517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"265'213","reported_seq":820,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:52.160194+0000","last_change":"2024-05-30T15:54:22.672899+0000","last_active":"2024-05-30T17:11:52.160194+0000","last_peered":"2024-05-30T17:11:52.160194+0000","last_clean":"2024-05-30T17:11:52.160194+0000","last_became_active":"2024-05-30T15:54:22.672078+0000","last_became_peered":"2024-05-30T15:54:22.672078+0000","last_unstale":"2024-05-30T17:11:52.160194+0000","last_undegraded":"2024-05-30T17:11:52.160194+0000","last_fullsized":"2024-05-30T17:11:52.160194+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":200,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:23:16.149546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14727,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":288,"num_read_kb":379,"num_write":182,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":503,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.557376+0000","last_change":"2024-05-30T15:54:24.676069+0000","last_active":"2024-05-30T17:09:21.557376+0000","last_peered":"2024-05-30T17:09:21.557376+0000","last_clean":"2024-05-30T17:09:21.557376+0000","last_became_active":"2024-05-30T15:54:24.675467+0000","last_became_peered":"2024-05-30T15:54:24.675467+0000","last_unstale":"2024-05-30T17:09:21.557376+0000","last_undegraded":"2024-05-30T17:09:21.557376+0000","last_fullsized":"2024-05-30T17:09:21.557376+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:10.271248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"265'102","reported_seq":619,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:19.803711+0000","last_change":"2024-05-30T15:54:26.705986+0000","last_active":"2024-05-30T17:13:19.803711+0000","last_peered":"2024-05-30T17:13:19.803711+0000","last_clean":"2024-05-30T17:13:19.803711+0000","last_became_active":"2024-05-30T15:54:26.705264+0000","last_became_peered":"2024-05-30T15:54:26.705264+0000","last_unstale":"2024-05-30T17:13:19.803711+0000","last_undegraded":"2024-05-30T17:13:19.803711+0000","last_fullsized":"2024-05-30T17:13:19.803711+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:18:43.441191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":805,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":151,"num_read_kb":135,"num_write":247,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"265'698","reported_seq":2167,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.677934+0000","last_change":"2024-05-30T15:54:54.422775+0000","last_active":"2024-05-30T17:13:18.677934+0000","last_peered":"2024-05-30T17:13:18.677934+0000","last_clean":"2024-05-30T17:13:18.677934+0000","last_became_active":"2024-05-30T15:54:54.421604+0000","last_became_peered":"2024-05-30T15:54:54.421604+0000","last_unstale":"2024-05-30T17:13:18.677934+0000","last_undegraded":"2024-05-30T17:13:18.677934+0000","last_fullsized":"2024-05-30T17:13:18.677934+0000","mapping_epoch":35,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":600,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:32:11.130243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3204,"num_read_kb":2893,"num_write":920,"num_write_kb":324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"265'163","reported_seq":804,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:17.124667+0000","last_change":"2024-05-30T15:54:54.424699+0000","last_active":"2024-05-30T17:09:17.124667+0000","last_peered":"2024-05-30T17:09:17.124667+0000","last_clean":"2024-05-30T17:09:17.124667+0000","last_became_active":"2024-05-30T15:54:54.422916+0000","last_became_peered":"2024-05-30T15:54:54.422916+0000","last_unstale":"2024-05-30T17:09:17.124667+0000","last_undegraded":"2024-05-30T17:09:17.124667+0000","last_fullsized":"2024-05-30T17:09:17.124667+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":100,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:06:41.620026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15294568,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":395,"num_read_kb":71105,"num_write":966,"num_write_kb":25767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"34'4","reported_seq":829,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.708703+0000","last_change":"2024-05-30T15:54:20.652854+0000","last_active":"2024-05-30T17:13:24.708703+0000","last_peered":"2024-05-30T17:13:24.708703+0000","last_clean":"2024-05-30T17:13:24.708703+0000","last_became_active":"2024-05-30T15:54:20.652134+0000","last_became_peered":"2024-05-30T15:54:20.652134+0000","last_unstale":"2024-05-30T17:13:24.708703+0000","last_undegraded":"2024-05-30T17:13:24.708703+0000","last_fullsized":"2024-05-30T17:13:24.708703+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:12:04.911743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1139,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":490,"num_read_kb":802,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"265'147","reported_seq":736,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:46.590296+0000","last_change":"2024-05-30T15:54:22.673262+0000","last_active":"2024-05-30T17:11:46.590296+0000","last_peered":"2024-05-30T17:11:46.590296+0000","last_clean":"2024-05-30T17:11:46.590296+0000","last_became_active":"2024-05-30T15:54:22.672169+0000","last_became_peered":"2024-05-30T15:54:22.672169+0000","last_unstale":"2024-05-30T17:11:46.590296+0000","last_undegraded":"2024-05-30T17:11:46.590296+0000","last_fullsized":"2024-05-30T17:11:46.590296+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":100,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:04:32.418773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8883,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":183,"num_read_kb":244,"num_write":98,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":497,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:41.537447+0000","last_change":"2024-05-30T15:54:24.676446+0000","last_active":"2024-05-30T17:09:41.537447+0000","last_peered":"2024-05-30T17:09:41.537447+0000","last_clean":"2024-05-30T17:09:41.537447+0000","last_became_active":"2024-05-30T15:54:24.675683+0000","last_became_peered":"2024-05-30T15:54:24.675683+0000","last_unstale":"2024-05-30T17:09:41.537447+0000","last_undegraded":"2024-05-30T17:09:41.537447+0000","last_fullsized":"2024-05-30T17:09:41.537447+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:10:10.728748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"265'127","reported_seq":655,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:17.801147+0000","last_change":"2024-05-30T15:54:26.689830+0000","last_active":"2024-05-30T17:13:17.801147+0000","last_peered":"2024-05-30T17:13:17.801147+0000","last_clean":"2024-05-30T17:13:17.801147+0000","last_became_active":"2024-05-30T15:54:26.689049+0000","last_became_peered":"2024-05-30T15:54:26.689049+0000","last_unstale":"2024-05-30T17:13:17.801147+0000","last_undegraded":"2024-05-30T17:13:17.801147+0000","last_fullsized":"2024-05-30T17:13:17.801147+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:41:39.309741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":539,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":167,"num_read_kb":166,"num_write":309,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"265'652","reported_seq":2069,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.759093+0000","last_change":"2024-05-30T15:54:18.653246+0000","last_active":"2024-05-30T17:13:18.759093+0000","last_peered":"2024-05-30T17:13:18.759093+0000","last_clean":"2024-05-30T17:13:18.759093+0000","last_became_active":"2024-05-30T15:54:18.651778+0000","last_became_peered":"2024-05-30T15:54:18.651778+0000","last_unstale":"2024-05-30T17:13:18.759093+0000","last_undegraded":"2024-05-30T17:13:18.759093+0000","last_fullsized":"2024-05-30T17:13:18.759093+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":600,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:42:03.298813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3068,"num_read_kb":2749,"num_write":930,"num_write_kb":367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"265'151","reported_seq":765,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:36.577419+0000","last_change":"2024-05-30T15:54:16.656872+0000","last_active":"2024-05-30T17:10:36.577419+0000","last_peered":"2024-05-30T17:10:36.577419+0000","last_clean":"2024-05-30T17:10:36.577419+0000","last_became_active":"2024-05-30T15:54:16.656081+0000","last_became_peered":"2024-05-30T15:54:16.656081+0000","last_unstale":"2024-05-30T17:10:36.577419+0000","last_undegraded":"2024-05-30T17:10:36.577419+0000","last_fullsized":"2024-05-30T17:10:36.577419+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:25:32.809071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":369,"num_read_kb":8201,"num_write":842,"num_write_kb":22196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":515,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:16.721503+0000","last_change":"2024-05-30T15:54:20.661844+0000","last_active":"2024-05-30T17:09:16.721503+0000","last_peered":"2024-05-30T17:09:16.721503+0000","last_clean":"2024-05-30T17:09:16.721503+0000","last_became_active":"2024-05-30T15:54:20.661090+0000","last_became_peered":"2024-05-30T15:54:20.661090+0000","last_unstale":"2024-05-30T17:09:16.721503+0000","last_undegraded":"2024-05-30T17:09:16.721503+0000","last_fullsized":"2024-05-30T17:09:16.721503+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:39:52.559621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"265'112","reported_seq":680,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.784725+0000","last_change":"2024-05-30T15:54:22.670767+0000","last_active":"2024-05-30T17:13:18.784725+0000","last_peered":"2024-05-30T17:13:18.784725+0000","last_clean":"2024-05-30T17:13:18.784725+0000","last_became_active":"2024-05-30T15:54:22.669073+0000","last_became_peered":"2024-05-30T15:54:22.669073+0000","last_unstale":"2024-05-30T17:13:18.784725+0000","last_undegraded":"2024-05-30T17:13:18.784725+0000","last_fullsized":"2024-05-30T17:13:18.784725+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:43:33.730358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8403,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":171,"num_write":70,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:26.724184+0000","last_change":"2024-05-30T15:54:24.680707+0000","last_active":"2024-05-30T17:09:26.724184+0000","last_peered":"2024-05-30T17:09:26.724184+0000","last_clean":"2024-05-30T17:09:26.724184+0000","last_became_active":"2024-05-30T15:54:24.678292+0000","last_became_peered":"2024-05-30T15:54:24.678292+0000","last_unstale":"2024-05-30T17:09:26.724184+0000","last_undegraded":"2024-05-30T17:09:26.724184+0000","last_fullsized":"2024-05-30T17:09:26.724184+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:56:06.653666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"265'164","reported_seq":901,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:14.442195+0000","last_change":"2024-05-30T15:54:26.704198+0000","last_active":"2024-05-30T17:13:14.442195+0000","last_peered":"2024-05-30T17:13:14.442195+0000","last_clean":"2024-05-30T17:13:14.442195+0000","last_became_active":"2024-05-30T15:54:26.703440+0000","last_became_peered":"2024-05-30T15:54:26.703440+0000","last_unstale":"2024-05-30T17:13:14.442195+0000","last_undegraded":"2024-05-30T17:13:14.442195+0000","last_fullsized":"2024-05-30T17:13:14.442195+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":100,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:28:50.587698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1074,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":420,"num_read_kb":450,"num_write":403,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"265'327","reported_seq":1681,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:15.190426+0000","last_change":"2024-05-30T15:54:18.672371+0000","last_active":"2024-05-30T17:12:15.190426+0000","last_peered":"2024-05-30T17:12:15.190426+0000","last_clean":"2024-05-30T17:12:15.190426+0000","last_became_active":"2024-05-30T15:54:18.671650+0000","last_became_peered":"2024-05-30T15:54:18.671650+0000","last_unstale":"2024-05-30T17:12:15.190426+0000","last_undegraded":"2024-05-30T17:12:15.190426+0000","last_fullsized":"2024-05-30T17:12:15.190426+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":300,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:52:16.244837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1672,"num_read_kb":1605,"num_write":361,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"265'205","reported_seq":866,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:34.743973+0000","last_change":"2024-05-30T15:54:16.661703+0000","last_active":"2024-05-30T17:12:34.743973+0000","last_peered":"2024-05-30T17:12:34.743973+0000","last_clean":"2024-05-30T17:12:34.743973+0000","last_became_active":"2024-05-30T15:54:16.660393+0000","last_became_peered":"2024-05-30T15:54:16.660393+0000","last_unstale":"2024-05-30T17:12:34.743973+0000","last_undegraded":"2024-05-30T17:12:34.743973+0000","last_fullsized":"2024-05-30T17:12:34.743973+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":100,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:51:03.780593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4788806,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":518,"num_read_kb":9177,"num_write":1140,"num_write_kb":26248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"27'1","reported_seq":508,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:31.768150+0000","last_change":"2024-05-30T15:54:20.661745+0000","last_active":"2024-05-30T17:09:31.768150+0000","last_peered":"2024-05-30T17:09:31.768150+0000","last_clean":"2024-05-30T17:09:31.768150+0000","last_became_active":"2024-05-30T15:54:20.661311+0000","last_became_peered":"2024-05-30T15:54:20.661311+0000","last_unstale":"2024-05-30T17:09:31.768150+0000","last_undegraded":"2024-05-30T17:09:31.768150+0000","last_fullsized":"2024-05-30T17:09:31.768150+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:08:29.614757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"265'171","reported_seq":778,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:56.757203+0000","last_change":"2024-05-30T15:54:22.663985+0000","last_active":"2024-05-30T17:11:56.757203+0000","last_peered":"2024-05-30T17:11:56.757203+0000","last_clean":"2024-05-30T17:11:56.757203+0000","last_became_active":"2024-05-30T15:54:22.663231+0000","last_became_peered":"2024-05-30T15:54:22.663231+0000","last_unstale":"2024-05-30T17:11:56.757203+0000","last_undegraded":"2024-05-30T17:11:56.757203+0000","last_fullsized":"2024-05-30T17:11:56.757203+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:32:51.856936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9029,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":225,"num_read_kb":274,"num_write":130,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":502,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:26.558679+0000","last_change":"2024-05-30T15:54:24.676660+0000","last_active":"2024-05-30T17:09:26.558679+0000","last_peered":"2024-05-30T17:09:26.558679+0000","last_clean":"2024-05-30T17:09:26.558679+0000","last_became_active":"2024-05-30T15:54:24.673674+0000","last_became_peered":"2024-05-30T15:54:24.673674+0000","last_unstale":"2024-05-30T17:09:26.558679+0000","last_undegraded":"2024-05-30T17:09:26.558679+0000","last_fullsized":"2024-05-30T17:09:26.558679+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:37:07.948924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"265'111","reported_seq":616,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.723032+0000","last_change":"2024-05-30T15:54:26.678237+0000","last_active":"2024-05-30T17:09:21.723032+0000","last_peered":"2024-05-30T17:09:21.723032+0000","last_clean":"2024-05-30T17:09:21.723032+0000","last_became_active":"2024-05-30T15:54:26.677418+0000","last_became_peered":"2024-05-30T15:54:26.677418+0000","last_unstale":"2024-05-30T17:09:21.723032+0000","last_undegraded":"2024-05-30T17:09:21.723032+0000","last_fullsized":"2024-05-30T17:09:21.723032+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":100,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:04:15.881662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":435,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":67,"num_write":244,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"265'305","reported_seq":1708,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:29.784726+0000","last_change":"2024-05-30T15:54:18.671642+0000","last_active":"2024-05-30T17:11:29.784726+0000","last_peered":"2024-05-30T17:11:29.784726+0000","last_clean":"2024-05-30T17:11:29.784726+0000","last_became_active":"2024-05-30T15:54:18.671335+0000","last_became_peered":"2024-05-30T15:54:18.671335+0000","last_unstale":"2024-05-30T17:11:29.784726+0000","last_undegraded":"2024-05-30T17:11:29.784726+0000","last_fullsized":"2024-05-30T17:11:29.784726+0000","mapping_epoch":24,"log_start":"265'200","ondisk_log_start":"265'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:01:34.225524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1771,"num_read_kb":1705,"num_write":340,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"265'202","reported_seq":854,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:26.132685+0000","last_change":"2024-05-30T15:54:16.675257+0000","last_active":"2024-05-30T17:12:26.132685+0000","last_peered":"2024-05-30T17:12:26.132685+0000","last_clean":"2024-05-30T17:12:26.132685+0000","last_became_active":"2024-05-30T15:54:16.674678+0000","last_became_peered":"2024-05-30T15:54:16.674678+0000","last_unstale":"2024-05-30T17:12:26.132685+0000","last_undegraded":"2024-05-30T17:12:26.132685+0000","last_fullsized":"2024-05-30T17:12:26.132685+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:49:35.316396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5919937,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":489,"num_read_kb":10347,"num_write":1032,"num_write_kb":23027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"265'559","reported_seq":2275,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.621937+0000","last_change":"2024-05-30T15:54:18.635401+0000","last_active":"2024-05-30T17:13:20.621937+0000","last_peered":"2024-05-30T17:13:20.621937+0000","last_clean":"2024-05-30T17:13:20.621937+0000","last_became_active":"2024-05-30T15:54:18.635092+0000","last_became_peered":"2024-05-30T15:54:18.635092+0000","last_unstale":"2024-05-30T17:13:20.621937+0000","last_undegraded":"2024-05-30T17:13:20.621937+0000","last_fullsized":"2024-05-30T17:13:20.621937+0000","mapping_epoch":24,"log_start":"265'500","ondisk_log_start":"265'500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":500,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:59:22.401801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2950,"num_read_kb":2739,"num_write":700,"num_write_kb":188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"265'159","reported_seq":773,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.559861+0000","last_change":"2024-05-30T15:54:16.658618+0000","last_active":"2024-05-30T17:11:21.559861+0000","last_peered":"2024-05-30T17:11:21.559861+0000","last_clean":"2024-05-30T17:11:21.559861+0000","last_became_active":"2024-05-30T15:54:16.658183+0000","last_became_peered":"2024-05-30T15:54:16.658183+0000","last_unstale":"2024-05-30T17:11:21.559861+0000","last_undegraded":"2024-05-30T17:11:21.559861+0000","last_fullsized":"2024-05-30T17:11:21.559861+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:41:18.697341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7709997,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":302,"num_read_kb":6656,"num_write":920,"num_write_kb":27882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"265'625","reported_seq":2181,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:20.469750+0000","last_change":"2024-05-30T15:54:18.662889+0000","last_active":"2024-05-30T17:12:20.469750+0000","last_peered":"2024-05-30T17:12:20.469750+0000","last_clean":"2024-05-30T17:12:20.469750+0000","last_became_active":"2024-05-30T15:54:18.662159+0000","last_became_peered":"2024-05-30T15:54:18.662159+0000","last_unstale":"2024-05-30T17:12:20.469750+0000","last_undegraded":"2024-05-30T17:12:20.469750+0000","last_fullsized":"2024-05-30T17:12:20.469750+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":600,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:00:12.906772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3191,"num_read_kb":2926,"num_write":899,"num_write_kb":342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"265'205","reported_seq":858,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:22.153006+0000","last_change":"2024-05-30T15:54:16.670343+0000","last_active":"2024-05-30T17:11:22.153006+0000","last_peered":"2024-05-30T17:11:22.153006+0000","last_clean":"2024-05-30T17:11:22.153006+0000","last_became_active":"2024-05-30T15:54:16.652664+0000","last_became_peered":"2024-05-30T15:54:16.652664+0000","last_unstale":"2024-05-30T17:11:22.153006+0000","last_undegraded":"2024-05-30T17:11:22.153006+0000","last_fullsized":"2024-05-30T17:11:22.153006+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":100,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:16:45.470104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5823370,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":446,"num_read_kb":27404,"num_write":1151,"num_write_kb":29020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"265'1297","reported_seq":2932,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:08.892803+0000","last_change":"2024-05-30T15:54:18.643433+0000","last_active":"2024-05-30T17:13:08.892803+0000","last_peered":"2024-05-30T17:13:08.892803+0000","last_clean":"2024-05-30T17:13:08.892803+0000","last_became_active":"2024-05-30T15:54:18.642697+0000","last_became_peered":"2024-05-30T15:54:18.642697+0000","last_unstale":"2024-05-30T17:13:08.892803+0000","last_undegraded":"2024-05-30T17:13:08.892803+0000","last_fullsized":"2024-05-30T17:13:08.892803+0000","mapping_epoch":24,"log_start":"265'1200","ondisk_log_start":"265'1200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":1200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:46:33.063462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":5484,"num_read_kb":4752,"num_write":1832,"num_write_kb":807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"265'255","reported_seq":987,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:36.401960+0000","last_change":"2024-05-30T15:54:16.660154+0000","last_active":"2024-05-30T17:11:36.401960+0000","last_peered":"2024-05-30T17:11:36.401960+0000","last_clean":"2024-05-30T17:11:36.401960+0000","last_became_active":"2024-05-30T15:54:16.659589+0000","last_became_peered":"2024-05-30T15:54:16.659589+0000","last_unstale":"2024-05-30T17:11:36.401960+0000","last_undegraded":"2024-05-30T17:11:36.401960+0000","last_fullsized":"2024-05-30T17:11:36.401960+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:55:43.508632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3249994,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":792,"num_read_kb":47588,"num_write":1563,"num_write_kb":36318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"265'1101","reported_seq":2680,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:20.420887+0000","last_change":"2024-05-30T15:54:18.653159+0000","last_active":"2024-05-30T17:12:20.420887+0000","last_peered":"2024-05-30T17:12:20.420887+0000","last_clean":"2024-05-30T17:12:20.420887+0000","last_became_active":"2024-05-30T15:54:18.652794+0000","last_became_peered":"2024-05-30T15:54:18.652794+0000","last_unstale":"2024-05-30T17:12:20.420887+0000","last_undegraded":"2024-05-30T17:12:20.420887+0000","last_fullsized":"2024-05-30T17:12:20.420887+0000","mapping_epoch":24,"log_start":"265'1000","ondisk_log_start":"265'1000","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":1000,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:14:33.580719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4733,"num_read_kb":4179,"num_write":1497,"num_write_kb":596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"265'223","reported_seq":891,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:06.570629+0000","last_change":"2024-05-30T15:54:54.452063+0000","last_active":"2024-05-30T17:10:06.570629+0000","last_peered":"2024-05-30T17:10:06.570629+0000","last_clean":"2024-05-30T17:10:06.570629+0000","last_became_active":"2024-05-30T15:54:54.451265+0000","last_became_peered":"2024-05-30T15:54:54.451265+0000","last_unstale":"2024-05-30T17:10:06.570629+0000","last_undegraded":"2024-05-30T17:10:06.570629+0000","last_fullsized":"2024-05-30T17:10:06.570629+0000","mapping_epoch":35,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":200,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:41:58.936344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":538764,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":478,"num_read_kb":2675,"num_write":1163,"num_write_kb":43073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"265'385","reported_seq":1948,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:33.943140+0000","last_change":"2024-05-30T15:54:18.671626+0000","last_active":"2024-05-30T17:12:33.943140+0000","last_peered":"2024-05-30T17:12:33.943140+0000","last_clean":"2024-05-30T17:12:33.943140+0000","last_became_active":"2024-05-30T15:54:18.670801+0000","last_became_peered":"2024-05-30T15:54:18.670801+0000","last_unstale":"2024-05-30T17:12:33.943140+0000","last_undegraded":"2024-05-30T17:12:33.943140+0000","last_fullsized":"2024-05-30T17:12:33.943140+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":300,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:08:04.360448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2215,"num_read_kb":2115,"num_write":443,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"265'219","reported_seq":870,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:20.785441+0000","last_change":"2024-05-30T15:54:16.651535+0000","last_active":"2024-05-30T17:11:20.785441+0000","last_peered":"2024-05-30T17:11:20.785441+0000","last_clean":"2024-05-30T17:11:20.785441+0000","last_became_active":"2024-05-30T15:54:16.649854+0000","last_became_peered":"2024-05-30T15:54:16.649854+0000","last_unstale":"2024-05-30T17:11:20.785441+0000","last_undegraded":"2024-05-30T17:11:20.785441+0000","last_fullsized":"2024-05-30T17:11:20.785441+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":200,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:56:05.559917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":635176,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":496,"num_read_kb":6249,"num_write":1148,"num_write_kb":30962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"265'405","reported_seq":1925,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:12.349529+0000","last_change":"2024-05-30T15:54:18.653103+0000","last_active":"2024-05-30T17:13:12.349529+0000","last_peered":"2024-05-30T17:13:12.349529+0000","last_clean":"2024-05-30T17:13:12.349529+0000","last_became_active":"2024-05-30T15:54:18.650832+0000","last_became_peered":"2024-05-30T15:54:18.650832+0000","last_unstale":"2024-05-30T17:13:12.349529+0000","last_undegraded":"2024-05-30T17:13:12.349529+0000","last_fullsized":"2024-05-30T17:13:12.349529+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":300,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:14:11.681719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2240,"num_read_kb":2125,"num_write":507,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"265'185","reported_seq":809,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:06.543039+0000","last_change":"2024-05-30T15:54:16.685235+0000","last_active":"2024-05-30T17:10:06.543039+0000","last_peered":"2024-05-30T17:10:06.543039+0000","last_clean":"2024-05-30T17:10:06.543039+0000","last_became_active":"2024-05-30T15:54:16.684670+0000","last_became_peered":"2024-05-30T15:54:16.684670+0000","last_unstale":"2024-05-30T17:10:06.543039+0000","last_undegraded":"2024-05-30T17:10:06.543039+0000","last_fullsized":"2024-05-30T17:10:06.543039+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:27.693225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5498449,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":371,"num_read_kb":34719,"num_write":1088,"num_write_kb":29777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"265'818","reported_seq":2410,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.705414+0000","last_change":"2024-05-30T15:54:18.669886+0000","last_active":"2024-05-30T17:13:20.705414+0000","last_peered":"2024-05-30T17:13:20.705414+0000","last_clean":"2024-05-30T17:13:20.705414+0000","last_became_active":"2024-05-30T15:54:18.669105+0000","last_became_peered":"2024-05-30T15:54:18.669105+0000","last_unstale":"2024-05-30T17:13:20.705414+0000","last_undegraded":"2024-05-30T17:13:20.705414+0000","last_fullsized":"2024-05-30T17:13:20.705414+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":800,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:33:49.110943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3660,"num_read_kb":3297,"num_write":1114,"num_write_kb":434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"265'218","reported_seq":916,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.337981+0000","last_change":"2024-05-30T15:54:54.417648+0000","last_active":"2024-05-30T17:11:11.337981+0000","last_peered":"2024-05-30T17:11:11.337981+0000","last_clean":"2024-05-30T17:11:11.337981+0000","last_became_active":"2024-05-30T15:54:54.416482+0000","last_became_peered":"2024-05-30T15:54:54.416482+0000","last_unstale":"2024-05-30T17:11:11.337981+0000","last_undegraded":"2024-05-30T17:11:11.337981+0000","last_fullsized":"2024-05-30T17:11:11.337981+0000","mapping_epoch":35,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":200,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:41:15.267706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5736484,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":727,"num_read_kb":137610,"num_write":1203,"num_write_kb":64841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2f","version":"265'385","reported_seq":1972,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:55.924733+0000","last_change":"2024-05-30T15:54:18.657515+0000","last_active":"2024-05-30T17:11:55.924733+0000","last_peered":"2024-05-30T17:11:55.924733+0000","last_clean":"2024-05-30T17:11:55.924733+0000","last_became_active":"2024-05-30T15:54:18.657236+0000","last_became_peered":"2024-05-30T15:54:18.657236+0000","last_unstale":"2024-05-30T17:11:55.924733+0000","last_undegraded":"2024-05-30T17:11:55.924733+0000","last_fullsized":"2024-05-30T17:11:55.924733+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":300,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:04:47.974078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2258,"num_read_kb":2156,"num_write":442,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"265'201","reported_seq":880,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:12.137218+0000","last_change":"2024-05-30T15:54:16.661242+0000","last_active":"2024-05-30T17:10:12.137218+0000","last_peered":"2024-05-30T17:10:12.137218+0000","last_clean":"2024-05-30T17:10:12.137218+0000","last_became_active":"2024-05-30T15:54:16.653037+0000","last_became_peered":"2024-05-30T15:54:16.653037+0000","last_unstale":"2024-05-30T17:10:12.137218+0000","last_undegraded":"2024-05-30T17:10:12.137218+0000","last_fullsized":"2024-05-30T17:10:12.137218+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":100,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:51:51.448976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2129592,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":541,"num_read_kb":29183,"num_write":975,"num_write_kb":32330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"265'330","reported_seq":1827,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:20.410822+0000","last_change":"2024-05-30T15:54:18.636049+0000","last_active":"2024-05-30T17:12:20.410822+0000","last_peered":"2024-05-30T17:12:20.410822+0000","last_clean":"2024-05-30T17:12:20.410822+0000","last_became_active":"2024-05-30T15:54:18.635382+0000","last_became_peered":"2024-05-30T15:54:18.635382+0000","last_unstale":"2024-05-30T17:12:20.410822+0000","last_undegraded":"2024-05-30T17:12:20.410822+0000","last_fullsized":"2024-05-30T17:12:20.410822+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":300,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:50:22.797101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2071,"num_read_kb":1981,"num_write":382,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"265'155","reported_seq":772,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.579598+0000","last_change":"2024-05-30T15:54:16.683072+0000","last_active":"2024-05-30T17:10:46.579598+0000","last_peered":"2024-05-30T17:10:46.579598+0000","last_clean":"2024-05-30T17:10:46.579598+0000","last_became_active":"2024-05-30T15:54:16.682629+0000","last_became_peered":"2024-05-30T15:54:16.682629+0000","last_unstale":"2024-05-30T17:10:46.579598+0000","last_undegraded":"2024-05-30T17:10:46.579598+0000","last_fullsized":"2024-05-30T17:10:46.579598+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:49:01.501502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7203749,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":304,"num_read_kb":22431,"num_write":818,"num_write_kb":23734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"265'743","reported_seq":2312,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.707890+0000","last_change":"2024-05-30T15:54:18.669167+0000","last_active":"2024-05-30T17:13:20.707890+0000","last_peered":"2024-05-30T17:13:20.707890+0000","last_clean":"2024-05-30T17:13:20.707890+0000","last_became_active":"2024-05-30T15:54:18.668748+0000","last_became_peered":"2024-05-30T15:54:18.668748+0000","last_unstale":"2024-05-30T17:13:20.707890+0000","last_undegraded":"2024-05-30T17:13:20.707890+0000","last_fullsized":"2024-05-30T17:13:20.707890+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":700,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:32:48.662121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3385,"num_read_kb":3066,"num_write":980,"num_write_kb":340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"265'161","reported_seq":896,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:58.685815+0000","last_change":"2024-05-30T15:54:16.676031+0000","last_active":"2024-05-30T17:12:58.685815+0000","last_peered":"2024-05-30T17:12:58.685815+0000","last_clean":"2024-05-30T17:12:58.685815+0000","last_became_active":"2024-05-30T15:54:16.674993+0000","last_became_peered":"2024-05-30T15:54:16.674993+0000","last_unstale":"2024-05-30T17:12:58.685815+0000","last_undegraded":"2024-05-30T17:12:58.685815+0000","last_fullsized":"2024-05-30T17:12:58.685815+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":100,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:09:30.788276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11799386,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":563,"num_read_kb":112844,"num_write":921,"num_write_kb":45704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"265'471","reported_seq":2112,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.654074+0000","last_change":"2024-05-30T15:54:18.672327+0000","last_active":"2024-05-30T17:12:44.654074+0000","last_peered":"2024-05-30T17:12:44.654074+0000","last_clean":"2024-05-30T17:12:44.654074+0000","last_became_active":"2024-05-30T15:54:18.671434+0000","last_became_peered":"2024-05-30T15:54:18.671434+0000","last_unstale":"2024-05-30T17:12:44.654074+0000","last_undegraded":"2024-05-30T17:12:44.654074+0000","last_fullsized":"2024-05-30T17:12:44.654074+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":400,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:07:26.503748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2525,"num_read_kb":2390,"num_write":596,"num_write_kb":183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"265'200","reported_seq":854,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.575766+0000","last_change":"2024-05-30T15:54:16.630459+0000","last_active":"2024-05-30T17:10:46.575766+0000","last_peered":"2024-05-30T17:10:46.575766+0000","last_clean":"2024-05-30T17:10:46.575766+0000","last_became_active":"2024-05-30T15:54:16.629843+0000","last_became_peered":"2024-05-30T15:54:16.629843+0000","last_unstale":"2024-05-30T17:10:46.575766+0000","last_undegraded":"2024-05-30T17:10:46.575766+0000","last_fullsized":"2024-05-30T17:10:46.575766+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":100,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:56:59.908915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":905553,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":430,"num_read_kb":42078,"num_write":1148,"num_write_kb":26102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"265'509","reported_seq":2134,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.660255+0000","last_change":"2024-05-30T15:54:18.660822+0000","last_active":"2024-05-30T17:12:44.660255+0000","last_peered":"2024-05-30T17:12:44.660255+0000","last_clean":"2024-05-30T17:12:44.660255+0000","last_became_active":"2024-05-30T15:54:18.660101+0000","last_became_peered":"2024-05-30T15:54:18.660101+0000","last_unstale":"2024-05-30T17:12:44.660255+0000","last_undegraded":"2024-05-30T17:12:44.660255+0000","last_fullsized":"2024-05-30T17:12:44.660255+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":400,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:23:12.134888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2760,"num_read_kb":2546,"num_write":661,"num_write_kb":201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"265'203","reported_seq":838,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.551832+0000","last_change":"2024-05-30T15:54:16.686381+0000","last_active":"2024-05-30T17:10:46.551832+0000","last_peered":"2024-05-30T17:10:46.551832+0000","last_clean":"2024-05-30T17:10:46.551832+0000","last_became_active":"2024-05-30T15:54:16.685969+0000","last_became_peered":"2024-05-30T15:54:16.685969+0000","last_unstale":"2024-05-30T17:10:46.551832+0000","last_undegraded":"2024-05-30T17:10:46.551832+0000","last_fullsized":"2024-05-30T17:10:46.551832+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":100,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:32:55.261508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18487302,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":412,"num_read_kb":18996,"num_write":1141,"num_write_kb":34757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"265'670","reported_seq":2427,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:55.980437+0000","last_change":"2024-05-30T15:54:18.646120+0000","last_active":"2024-05-30T17:11:55.980437+0000","last_peered":"2024-05-30T17:11:55.980437+0000","last_clean":"2024-05-30T17:11:55.980437+0000","last_became_active":"2024-05-30T15:54:18.645400+0000","last_became_peered":"2024-05-30T15:54:18.645400+0000","last_unstale":"2024-05-30T17:11:55.980437+0000","last_undegraded":"2024-05-30T17:11:55.980437+0000","last_fullsized":"2024-05-30T17:11:55.980437+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":600,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:37:15.286662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3447,"num_read_kb":3120,"num_write":968,"num_write_kb":434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"265'206","reported_seq":840,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:06.565221+0000","last_change":"2024-05-30T15:54:16.664235+0000","last_active":"2024-05-30T17:11:06.565221+0000","last_peered":"2024-05-30T17:11:06.565221+0000","last_clean":"2024-05-30T17:11:06.565221+0000","last_became_active":"2024-05-30T15:54:16.663255+0000","last_became_peered":"2024-05-30T15:54:16.663255+0000","last_unstale":"2024-05-30T17:11:06.565221+0000","last_undegraded":"2024-05-30T17:11:06.565221+0000","last_fullsized":"2024-05-30T17:11:06.565221+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":100,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:14:20.452277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11362066,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":420,"num_read_kb":27117,"num_write":1157,"num_write_kb":19484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"265'360","reported_seq":1779,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:08.893650+0000","last_change":"2024-05-30T15:54:18.658517+0000","last_active":"2024-05-30T17:13:08.893650+0000","last_peered":"2024-05-30T17:13:08.893650+0000","last_clean":"2024-05-30T17:13:08.893650+0000","last_became_active":"2024-05-30T15:54:18.658230+0000","last_became_peered":"2024-05-30T15:54:18.658230+0000","last_unstale":"2024-05-30T17:13:08.893650+0000","last_undegraded":"2024-05-30T17:13:08.893650+0000","last_fullsized":"2024-05-30T17:13:08.893650+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":300,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:13:42.767006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1934,"num_read_kb":1839,"num_write":414,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"265'166","reported_seq":779,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.558787+0000","last_change":"2024-05-30T15:54:16.655014+0000","last_active":"2024-05-30T17:11:16.558787+0000","last_peered":"2024-05-30T17:11:16.558787+0000","last_clean":"2024-05-30T17:11:16.558787+0000","last_became_active":"2024-05-30T15:54:16.654365+0000","last_became_peered":"2024-05-30T15:54:16.654365+0000","last_unstale":"2024-05-30T17:11:16.558787+0000","last_undegraded":"2024-05-30T17:11:16.558787+0000","last_fullsized":"2024-05-30T17:11:16.558787+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":100,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:31:10.427380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14702715,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":354,"num_read_kb":31154,"num_write":911,"num_write_kb":41557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"265'698","reported_seq":2128,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.605807+0000","last_change":"2024-05-30T15:54:18.661710+0000","last_active":"2024-05-30T17:12:44.605807+0000","last_peered":"2024-05-30T17:12:44.605807+0000","last_clean":"2024-05-30T17:12:44.605807+0000","last_became_active":"2024-05-30T15:54:18.660543+0000","last_became_peered":"2024-05-30T15:54:18.660543+0000","last_unstale":"2024-05-30T17:12:44.605807+0000","last_undegraded":"2024-05-30T17:12:44.605807+0000","last_fullsized":"2024-05-30T17:12:44.605807+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":600,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:39:46.092642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3130,"num_read_kb":2812,"num_write":919,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"265'195","reported_seq":837,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:17.152049+0000","last_change":"2024-05-30T15:54:16.666386+0000","last_active":"2024-05-30T17:11:17.152049+0000","last_peered":"2024-05-30T17:11:17.152049+0000","last_clean":"2024-05-30T17:11:17.152049+0000","last_became_active":"2024-05-30T15:54:16.661983+0000","last_became_peered":"2024-05-30T15:54:16.661983+0000","last_unstale":"2024-05-30T17:11:17.152049+0000","last_undegraded":"2024-05-30T17:11:17.152049+0000","last_fullsized":"2024-05-30T17:11:17.152049+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":100,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:48:35.955662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8616050,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":413,"num_read_kb":7620,"num_write":1148,"num_write_kb":9531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"265'152","reported_seq":782,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:10.550602+0000","last_change":"2024-05-30T15:55:54.768358+0000","last_active":"2024-05-30T17:13:10.550602+0000","last_peered":"2024-05-30T17:13:10.550602+0000","last_clean":"2024-05-30T17:13:10.550602+0000","last_became_active":"2024-05-30T15:55:54.767665+0000","last_became_peered":"2024-05-30T15:55:54.767665+0000","last_unstale":"2024-05-30T17:13:10.550602+0000","last_undegraded":"2024-05-30T17:13:10.550602+0000","last_fullsized":"2024-05-30T17:13:10.550602+0000","mapping_epoch":37,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":100,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:31:33.212759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11250466,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":358,"num_read_kb":14674,"num_write":888,"num_write_kb":17371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,0],"acting":[5,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"265'765","reported_seq":2419,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:12.341733+0000","last_change":"2024-05-30T15:54:18.660816+0000","last_active":"2024-05-30T17:11:12.341733+0000","last_peered":"2024-05-30T17:11:12.341733+0000","last_clean":"2024-05-30T17:11:12.341733+0000","last_became_active":"2024-05-30T15:54:18.660501+0000","last_became_peered":"2024-05-30T15:54:18.660501+0000","last_unstale":"2024-05-30T17:11:12.341733+0000","last_undegraded":"2024-05-30T17:11:12.341733+0000","last_fullsized":"2024-05-30T17:11:12.341733+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":700,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:58:48.235676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3837,"num_read_kb":3483,"num_write":1021,"num_write_kb":383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"265'179","reported_seq":830,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.790863+0000","last_change":"2024-05-30T15:54:16.668650+0000","last_active":"2024-05-30T17:11:16.790863+0000","last_peered":"2024-05-30T17:11:16.790863+0000","last_clean":"2024-05-30T17:11:16.790863+0000","last_became_active":"2024-05-30T15:54:16.668178+0000","last_became_peered":"2024-05-30T15:54:16.668178+0000","last_unstale":"2024-05-30T17:11:16.790863+0000","last_undegraded":"2024-05-30T17:11:16.790863+0000","last_fullsized":"2024-05-30T17:11:16.790863+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":100,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:22:19.325731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14464063,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":423,"num_read_kb":57498,"num_write":996,"num_write_kb":21705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"265'474","reported_seq":2144,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.703669+0000","last_change":"2024-05-30T15:54:18.656743+0000","last_active":"2024-05-30T17:13:20.703669+0000","last_peered":"2024-05-30T17:13:20.703669+0000","last_clean":"2024-05-30T17:13:20.703669+0000","last_became_active":"2024-05-30T15:54:18.655872+0000","last_became_peered":"2024-05-30T15:54:18.655872+0000","last_unstale":"2024-05-30T17:13:20.703669+0000","last_undegraded":"2024-05-30T17:13:20.703669+0000","last_fullsized":"2024-05-30T17:13:20.703669+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":400,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:51:05.525906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2571,"num_read_kb":2440,"num_write":587,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5999,"num_read_kb":4309,"num_write":2135,"num_write_kb":880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1140,"ondisk_log_size":1140,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19930,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":13226,"num_read_kb":13032,"num_write":15987,"num_write_kb":4690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":704512,"data_stored":39860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1567,"ondisk_log_size":1567,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":405,"ondisk_log_size":405,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":366949,"num_objects":197,"num_object_clones":0,"num_object_copies":394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":197,"num_whiteouts":0,"num_read":9271,"num_read_kb":11128,"num_write":6557,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":745472,"data_stored":181518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1889,"ondisk_log_size":1889,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1624,"num_read_kb":1934,"num_write":20,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":242,"num_object_clones":0,"num_object_copies":484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":242,"num_whiteouts":0,"num_read":191181,"num_read_kb":174247,"num_write":51900,"num_write_kb":17923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":242,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4116,"ondisk_log_size":4116,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":506410238,"num_objects":484,"num_object_clones":0,"num_object_copies":1452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":484,"num_whiteouts":0,"num_read":31813,"num_read_kb":2124772,"num_write":69681,"num_write_kb":2069422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":762998784,"data_stored":762998784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4465,"ondisk_log_size":4465,"up":192,"acting":192,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":81623142,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":117,"num_read_kb":182367,"num_write":133,"num_write_kb":122722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163258368,"data_stored":163246284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":81623142,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":115,"num_read_kb":181098,"num_write":133,"num_write_kb":122722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163258368,"data_stored":163246284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575810,"num_pgs":106,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":391144,"kb_used_data":129504,"kb_used_omap":1,"kb_used_meta":261630,"kb_avail":93980696,"statfs":{"total":96636764160,"available":96236232704,"internally_reserved":0,"allocated":132612096,"data_stored":130525109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":267909578},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575809,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395632,"kb_used_data":147880,"kb_used_omap":1,"kb_used_meta":247742,"kb_avail":93976208,"statfs":{"total":96636764160,"available":96231636992,"internally_reserved":0,"allocated":151429120,"data_stored":149446612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":253688262},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575809,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":220408,"kb_used_data":135088,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94151432,"statfs":{"total":96636764160,"available":96411066368,"internally_reserved":0,"allocated":138330112,"data_stored":136241754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87357893},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575809,"num_pgs":104,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":225900,"kb_used_data":132260,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94145940,"statfs":{"total":96636764160,"available":96405442560,"internally_reserved":0,"allocated":135434240,"data_stored":133486846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":95877574},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575807,"num_pgs":124,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":376304,"kb_used_data":150760,"kb_used_omap":1,"kb_used_meta":225534,"kb_avail":93995536,"statfs":{"total":96636764160,"available":96251428864,"internally_reserved":0,"allocated":154378240,"data_stored":152309513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":230947271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575807,"num_pgs":91,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":179456,"kb_used_data":108024,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94192384,"statfs":{"total":96636764160,"available":96453001216,"internally_reserved":0,"allocated":110616576,"data_stored":108657566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575811,"num_pgs":112,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":263376,"kb_used_data":146184,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94108464,"statfs":{"total":96636764160,"available":96367067136,"internally_reserved":0,"allocated":149692416,"data_stored":147687855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575806,"num_pgs":109,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":379024,"kb_used_data":149320,"kb_used_omap":1,"kb_used_meta":229694,"kb_avail":93992816,"statfs":{"total":96636764160,"available":96248643584,"internally_reserved":0,"allocated":152903680,"data_stored":150873069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":235207109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":114364416,"data_stored":114364416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":99225600,"data_stored":99225600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":74932224,"data_stored":74932224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":93523968,"data_stored":93523968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":97972224,"data_stored":97972224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":92889088,"data_stored":92889088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":109338624,"data_stored":109338624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":80752640,"data_stored":80752640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":18103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":28335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":7835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":30127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":10287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":151552,"data_stored":39861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":15040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":31930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":4787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":5121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":4696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":6691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":5459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":3918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":7193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32555008,"data_stored":32554271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22020096,"data_stored":22020096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18591744,"data_stored":18588432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":23322624,"data_stored":23318684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16900096,"data_stored":16897366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":21557248,"data_stored":21555883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7340032,"data_stored":7340032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20971520,"data_stored":20971520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24117248,"data_stored":24117248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12836864,"data_stored":12832924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":33140736,"data_stored":33138634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16314368,"data_stored":16313003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":19460096,"data_stored":19458731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":30457856,"data_stored":30457119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":25882624,"data_stored":25880049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T17:13:39.367 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-30T17:13:39.367 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T17:13:39.759 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:39.776 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-05-30T17:13:39.791 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":2645,"stamp":"2024-05-30T17:13:38.440189+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":670635562,"num_objects":1096,"num_object_clones":0,"num_object_copies":2676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1096,"num_whiteouts":0,"num_read":253424,"num_read_kb":2692954,"num_write":146665,"num_write_kb":2342828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":289,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13825,"ondisk_log_size":13825,"up":850,"acting":850,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":850,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2431244,"kb_used_data":1099020,"kb_used_omap":12,"kb_used_meta":1332147,"kb_avail":752543476,"statfs":{"total":773094113280,"available":770604519424,"internally_reserved":0,"allocated":1125396480,"data_stored":1109228324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":1364119089},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-520,"num_objects":-4,"num_object_clones":0,"num_object_copies":-8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4,"num_whiteouts":0,"num_read":23,"num_read_kb":25,"num_write":15,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007603"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":27,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:57.133920+0000","last_change":"2024-05-30T16:34:55.869930+0000","last_active":"2024-05-30T17:09:57.133920+0000","last_peered":"2024-05-30T17:09:57.133920+0000","last_clean":"2024-05-30T17:09:57.133920+0000","last_became_active":"2024-05-30T16:34:55.869226+0000","last_became_peered":"2024-05-30T16:34:55.869226+0000","last_unstale":"2024-05-30T17:09:57.133920+0000","last_undegraded":"2024-05-30T17:09:57.133920+0000","last_fullsized":"2024-05-30T17:09:57.133920+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:37:02.394930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"265'1","reported_seq":47,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:07.136134+0000","last_change":"2024-05-30T16:17:35.249630+0000","last_active":"2024-05-30T17:10:07.136134+0000","last_peered":"2024-05-30T17:10:07.136134+0000","last_clean":"2024-05-30T17:10:07.136134+0000","last_became_active":"2024-05-30T16:17:35.248740+0000","last_became_peered":"2024-05-30T16:17:35.248740+0000","last_unstale":"2024-05-30T17:10:07.136134+0000","last_undegraded":"2024-05-30T17:10:07.136134+0000","last_fullsized":"2024-05-30T17:10:07.136134+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:05:10.176331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":6144,"num_write":1,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"265'7","reported_seq":65,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:01.569510+0000","last_change":"2024-05-30T16:17:31.184110+0000","last_active":"2024-05-30T17:10:01.569510+0000","last_peered":"2024-05-30T17:10:01.569510+0000","last_clean":"2024-05-30T17:10:01.569510+0000","last_became_active":"2024-05-30T16:17:31.183349+0000","last_became_peered":"2024-05-30T16:17:31.183349+0000","last_unstale":"2024-05-30T17:10:01.569510+0000","last_undegraded":"2024-05-30T17:10:01.569510+0000","last_fullsized":"2024-05-30T17:10:01.569510+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:06:40.372126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":16384,"num_write":7,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"265'45","reported_seq":184,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.581199+0000","last_change":"2024-05-30T16:12:50.806342+0000","last_active":"2024-05-30T17:11:11.581199+0000","last_peered":"2024-05-30T17:11:11.581199+0000","last_clean":"2024-05-30T17:11:11.581199+0000","last_became_active":"2024-05-30T16:12:50.803352+0000","last_became_peered":"2024-05-30T16:12:50.803352+0000","last_unstale":"2024-05-30T17:11:11.581199+0000","last_undegraded":"2024-05-30T17:11:11.581199+0000","last_fullsized":"2024-05-30T17:11:11.581199+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:23:40.247207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":225,"num_read_kb":156,"num_write":101,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":27,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.543407+0000","last_change":"2024-05-30T16:34:55.887503+0000","last_active":"2024-05-30T17:09:56.543407+0000","last_peered":"2024-05-30T17:09:56.543407+0000","last_clean":"2024-05-30T17:09:56.543407+0000","last_became_active":"2024-05-30T16:34:55.886731+0000","last_became_peered":"2024-05-30T16:34:55.886731+0000","last_unstale":"2024-05-30T17:09:56.543407+0000","last_undegraded":"2024-05-30T17:09:56.543407+0000","last_fullsized":"2024-05-30T17:09:56.543407+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:47:35.283261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"0'0","reported_seq":44,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:06.732701+0000","last_change":"2024-05-30T16:17:35.248384+0000","last_active":"2024-05-30T17:10:06.732701+0000","last_peered":"2024-05-30T17:10:06.732701+0000","last_clean":"2024-05-30T17:10:06.732701+0000","last_became_active":"2024-05-30T16:17:35.247810+0000","last_became_peered":"2024-05-30T16:17:35.247810+0000","last_unstale":"2024-05-30T17:10:06.732701+0000","last_undegraded":"2024-05-30T17:10:06.732701+0000","last_fullsized":"2024-05-30T17:10:06.732701+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:56:26.172595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"0'0","reported_seq":50,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:01.773778+0000","last_change":"2024-05-30T16:17:31.215393+0000","last_active":"2024-05-30T17:10:01.773778+0000","last_peered":"2024-05-30T17:10:01.773778+0000","last_clean":"2024-05-30T17:10:01.773778+0000","last_became_active":"2024-05-30T16:17:31.212337+0000","last_became_peered":"2024-05-30T16:17:31.212337+0000","last_unstale":"2024-05-30T17:10:01.773778+0000","last_undegraded":"2024-05-30T17:10:01.773778+0000","last_fullsized":"2024-05-30T17:10:01.773778+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:48:30.989819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"265'55","reported_seq":210,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:06.556856+0000","last_change":"2024-05-30T16:12:50.810909+0000","last_active":"2024-05-30T17:11:06.556856+0000","last_peered":"2024-05-30T17:11:06.556856+0000","last_clean":"2024-05-30T17:11:06.556856+0000","last_became_active":"2024-05-30T16:12:50.810207+0000","last_became_peered":"2024-05-30T16:12:50.810207+0000","last_unstale":"2024-05-30T17:11:06.556856+0000","last_undegraded":"2024-05-30T17:11:06.556856+0000","last_fullsized":"2024-05-30T17:11:06.556856+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:37:48.188833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":277,"num_read_kb":197,"num_write":99,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.556933+0000","last_change":"2024-05-30T16:34:55.876451+0000","last_active":"2024-05-30T17:11:11.556933+0000","last_peered":"2024-05-30T17:11:11.556933+0000","last_clean":"2024-05-30T17:11:11.556933+0000","last_became_active":"2024-05-30T16:34:55.874359+0000","last_became_peered":"2024-05-30T16:34:55.874359+0000","last_unstale":"2024-05-30T17:11:11.556933+0000","last_undegraded":"2024-05-30T17:11:11.556933+0000","last_fullsized":"2024-05-30T17:11:11.556933+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:37:31.196070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"0'0","reported_seq":48,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:41.561910+0000","last_change":"2024-05-30T16:17:35.743986+0000","last_active":"2024-05-30T17:09:41.561910+0000","last_peered":"2024-05-30T17:09:41.561910+0000","last_clean":"2024-05-30T17:09:41.561910+0000","last_became_active":"2024-05-30T16:17:35.743231+0000","last_became_peered":"2024-05-30T16:17:35.743231+0000","last_unstale":"2024-05-30T17:09:41.561910+0000","last_undegraded":"2024-05-30T17:09:41.561910+0000","last_fullsized":"2024-05-30T17:09:41.561910+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:52:11.845318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"265'2","reported_seq":59,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:46.541550+0000","last_change":"2024-05-30T16:17:31.205642+0000","last_active":"2024-05-30T17:09:46.541550+0000","last_peered":"2024-05-30T17:09:46.541550+0000","last_clean":"2024-05-30T17:09:46.541550+0000","last_became_active":"2024-05-30T16:17:31.204806+0000","last_became_peered":"2024-05-30T16:17:31.204806+0000","last_unstale":"2024-05-30T17:09:46.541550+0000","last_undegraded":"2024-05-30T17:09:46.541550+0000","last_fullsized":"2024-05-30T17:09:46.541550+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:50:34.702667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":6144,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"265'29","reported_seq":134,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.570270+0000","last_change":"2024-05-30T16:12:50.811279+0000","last_active":"2024-05-30T17:10:21.570270+0000","last_peered":"2024-05-30T17:10:21.570270+0000","last_clean":"2024-05-30T17:10:21.570270+0000","last_became_active":"2024-05-30T16:12:50.810062+0000","last_became_peered":"2024-05-30T16:12:50.810062+0000","last_unstale":"2024-05-30T17:10:21.570270+0000","last_undegraded":"2024-05-30T17:10:21.570270+0000","last_fullsized":"2024-05-30T17:10:21.570270+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:23:45.234198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":98,"num_write":65,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":27,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:08:56.553929+0000","last_change":"2024-05-30T16:34:55.888292+0000","last_active":"2024-05-30T17:08:56.553929+0000","last_peered":"2024-05-30T17:08:56.553929+0000","last_clean":"2024-05-30T17:08:56.553929+0000","last_became_active":"2024-05-30T16:34:55.887532+0000","last_became_peered":"2024-05-30T16:34:55.887532+0000","last_unstale":"2024-05-30T17:08:56.553929+0000","last_undegraded":"2024-05-30T17:08:56.553929+0000","last_fullsized":"2024-05-30T17:08:56.553929+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:47:12.792997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"265'1","reported_seq":52,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:51.567346+0000","last_change":"2024-05-30T16:17:35.746712+0000","last_active":"2024-05-30T17:09:51.567346+0000","last_peered":"2024-05-30T17:09:51.567346+0000","last_clean":"2024-05-30T17:09:51.567346+0000","last_became_active":"2024-05-30T16:17:35.745709+0000","last_became_peered":"2024-05-30T16:17:35.745709+0000","last_unstale":"2024-05-30T17:09:51.567346+0000","last_undegraded":"2024-05-30T17:09:51.567346+0000","last_fullsized":"2024-05-30T17:09:51.567346+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:58:32.058273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":584363,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1142,"num_write":1,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1f","version":"265'2","reported_seq":56,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.726378+0000","last_change":"2024-05-30T16:17:31.183141+0000","last_active":"2024-05-30T17:09:36.726378+0000","last_peered":"2024-05-30T17:09:36.726378+0000","last_clean":"2024-05-30T17:09:36.726378+0000","last_became_active":"2024-05-30T16:17:31.182344+0000","last_became_peered":"2024-05-30T16:17:31.182344+0000","last_unstale":"2024-05-30T17:09:36.726378+0000","last_undegraded":"2024-05-30T17:09:36.726378+0000","last_fullsized":"2024-05-30T17:09:36.726378+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:20:08.705563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"260'6","reported_seq":76,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.573922+0000","last_change":"2024-05-30T16:12:50.830384+0000","last_active":"2024-05-30T17:10:21.573922+0000","last_peered":"2024-05-30T17:10:21.573922+0000","last_clean":"2024-05-30T17:10:21.573922+0000","last_became_active":"2024-05-30T16:12:50.826974+0000","last_became_peered":"2024-05-30T16:12:50.826974+0000","last_unstale":"2024-05-30T17:10:21.573922+0000","last_undegraded":"2024-05-30T17:10:21.573922+0000","last_fullsized":"2024-05-30T17:10:21.573922+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:27:49.529929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":24,"num_write":11,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":31,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:06.583992+0000","last_change":"2024-05-30T16:34:55.899516+0000","last_active":"2024-05-30T17:11:06.583992+0000","last_peered":"2024-05-30T17:11:06.583992+0000","last_clean":"2024-05-30T17:11:06.583992+0000","last_became_active":"2024-05-30T16:34:55.898864+0000","last_became_peered":"2024-05-30T16:34:55.898864+0000","last_unstale":"2024-05-30T17:11:06.583992+0000","last_undegraded":"2024-05-30T17:11:06.583992+0000","last_fullsized":"2024-05-30T17:11:06.583992+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:07:16.296580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"265'6","reported_seq":62,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:52.146144+0000","last_change":"2024-05-30T16:17:35.233543+0000","last_active":"2024-05-30T17:10:52.146144+0000","last_peered":"2024-05-30T17:10:52.146144+0000","last_clean":"2024-05-30T17:10:52.146144+0000","last_became_active":"2024-05-30T16:17:35.232744+0000","last_became_peered":"2024-05-30T16:17:35.232744+0000","last_unstale":"2024-05-30T17:10:52.146144+0000","last_undegraded":"2024-05-30T17:10:52.146144+0000","last_fullsized":"2024-05-30T17:10:52.146144+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:40:45.244827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":23554,"num_write":5,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"0'0","reported_seq":51,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:41.574504+0000","last_change":"2024-05-30T16:17:31.186983+0000","last_active":"2024-05-30T17:10:41.574504+0000","last_peered":"2024-05-30T17:10:41.574504+0000","last_clean":"2024-05-30T17:10:41.574504+0000","last_became_active":"2024-05-30T16:17:31.186229+0000","last_became_peered":"2024-05-30T16:17:31.186229+0000","last_unstale":"2024-05-30T17:10:41.574504+0000","last_undegraded":"2024-05-30T17:10:41.574504+0000","last_fullsized":"2024-05-30T17:10:41.574504+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:33:19.747591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"265'27","reported_seq":134,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.573828+0000","last_change":"2024-05-30T16:12:50.830804+0000","last_active":"2024-05-30T17:10:21.573828+0000","last_peered":"2024-05-30T17:10:21.573828+0000","last_clean":"2024-05-30T17:10:21.573828+0000","last_became_active":"2024-05-30T16:12:50.827008+0000","last_became_peered":"2024-05-30T16:12:50.827008+0000","last_unstale":"2024-05-30T17:10:21.573828+0000","last_undegraded":"2024-05-30T17:10:21.573828+0000","last_fullsized":"2024-05-30T17:10:21.573828+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:26:44.426182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":100,"num_write":57,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.778503+0000","last_change":"2024-05-30T16:34:55.870968+0000","last_active":"2024-05-30T17:10:21.778503+0000","last_peered":"2024-05-30T17:10:21.778503+0000","last_clean":"2024-05-30T17:10:21.778503+0000","last_became_active":"2024-05-30T16:34:55.870139+0000","last_became_peered":"2024-05-30T16:34:55.870139+0000","last_unstale":"2024-05-30T17:10:21.778503+0000","last_undegraded":"2024-05-30T17:10:21.778503+0000","last_fullsized":"2024-05-30T17:10:21.778503+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:55:17.556265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"265'5","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.551727+0000","last_change":"2024-05-30T16:17:35.234843+0000","last_active":"2024-05-30T17:10:46.551727+0000","last_peered":"2024-05-30T17:10:46.551727+0000","last_clean":"2024-05-30T17:10:46.551727+0000","last_became_active":"2024-05-30T16:17:35.233040+0000","last_became_peered":"2024-05-30T16:17:35.233040+0000","last_unstale":"2024-05-30T17:10:46.551727+0000","last_undegraded":"2024-05-30T17:10:46.551727+0000","last_fullsized":"2024-05-30T17:10:46.551727+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:42:31.077865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11582751,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":12336,"num_write":10,"num_write_kb":11312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"265'2","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.742318+0000","last_change":"2024-05-30T16:17:31.174675+0000","last_active":"2024-05-30T17:10:51.742318+0000","last_peered":"2024-05-30T17:10:51.742318+0000","last_clean":"2024-05-30T17:10:51.742318+0000","last_became_active":"2024-05-30T16:17:31.173900+0000","last_became_peered":"2024-05-30T16:17:31.173900+0000","last_unstale":"2024-05-30T17:10:51.742318+0000","last_undegraded":"2024-05-30T17:10:51.742318+0000","last_fullsized":"2024-05-30T17:10:51.742318+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:03:12.426092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":6144,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"265'75","reported_seq":266,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.546161+0000","last_change":"2024-05-30T16:12:50.815789+0000","last_active":"2024-05-30T17:10:21.546161+0000","last_peered":"2024-05-30T17:10:21.546161+0000","last_clean":"2024-05-30T17:10:21.546161+0000","last_became_active":"2024-05-30T16:12:50.815032+0000","last_became_peered":"2024-05-30T16:12:50.815032+0000","last_unstale":"2024-05-30T17:10:21.546161+0000","last_undegraded":"2024-05-30T17:10:21.546161+0000","last_fullsized":"2024-05-30T17:10:21.546161+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:53:44.633843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":396,"num_read_kb":277,"num_write":137,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"265'1","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.548662+0000","last_change":"2024-05-30T16:34:55.882343+0000","last_active":"2024-05-30T17:09:56.548662+0000","last_peered":"2024-05-30T17:09:56.548662+0000","last_clean":"2024-05-30T17:09:56.548662+0000","last_became_active":"2024-05-30T16:34:55.881414+0000","last_became_peered":"2024-05-30T16:34:55.881414+0000","last_unstale":"2024-05-30T17:09:56.548662+0000","last_undegraded":"2024-05-30T17:09:56.548662+0000","last_fullsized":"2024-05-30T17:09:56.548662+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:58:23.360902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"262'2","reported_seq":48,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:06.542967+0000","last_change":"2024-05-30T16:17:35.238116+0000","last_active":"2024-05-30T17:10:06.542967+0000","last_peered":"2024-05-30T17:10:06.542967+0000","last_clean":"2024-05-30T17:10:06.542967+0000","last_became_active":"2024-05-30T16:17:35.237424+0000","last_became_peered":"2024-05-30T16:17:35.237424+0000","last_unstale":"2024-05-30T17:10:06.542967+0000","last_undegraded":"2024-05-30T17:10:06.542967+0000","last_fullsized":"2024-05-30T17:10:06.542967+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:30:19.299666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"0'0","reported_seq":50,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:01.731526+0000","last_change":"2024-05-30T16:17:31.192981+0000","last_active":"2024-05-30T17:10:01.731526+0000","last_peered":"2024-05-30T17:10:01.731526+0000","last_clean":"2024-05-30T17:10:01.731526+0000","last_became_active":"2024-05-30T16:17:31.192269+0000","last_became_peered":"2024-05-30T16:17:31.192269+0000","last_unstale":"2024-05-30T17:10:01.731526+0000","last_undegraded":"2024-05-30T17:10:01.731526+0000","last_fullsized":"2024-05-30T17:10:01.731526+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:47:56.727573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"265'25","reported_seq":125,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:06.579930+0000","last_change":"2024-05-30T16:12:50.817479+0000","last_active":"2024-05-30T17:11:06.579930+0000","last_peered":"2024-05-30T17:11:06.579930+0000","last_clean":"2024-05-30T17:11:06.579930+0000","last_became_active":"2024-05-30T16:12:50.811732+0000","last_became_peered":"2024-05-30T16:12:50.811732+0000","last_unstale":"2024-05-30T17:11:06.579930+0000","last_undegraded":"2024-05-30T17:11:06.579930+0000","last_fullsized":"2024-05-30T17:11:06.579930+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:22:47.071307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":85,"num_write":53,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":27,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.548778+0000","last_change":"2024-05-30T16:34:55.899801+0000","last_active":"2024-05-30T17:09:56.548778+0000","last_peered":"2024-05-30T17:09:56.548778+0000","last_clean":"2024-05-30T17:09:56.548778+0000","last_became_active":"2024-05-30T16:34:55.899241+0000","last_became_peered":"2024-05-30T16:34:55.899241+0000","last_unstale":"2024-05-30T17:09:56.548778+0000","last_undegraded":"2024-05-30T17:09:56.548778+0000","last_fullsized":"2024-05-30T17:09:56.548778+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:25:21.668824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"0'0","reported_seq":44,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:06.570530+0000","last_change":"2024-05-30T16:17:35.229918+0000","last_active":"2024-05-30T17:10:06.570530+0000","last_peered":"2024-05-30T17:10:06.570530+0000","last_clean":"2024-05-30T17:10:06.570530+0000","last_became_active":"2024-05-30T16:17:35.229034+0000","last_became_peered":"2024-05-30T16:17:35.229034+0000","last_unstale":"2024-05-30T17:10:06.570530+0000","last_undegraded":"2024-05-30T17:10:06.570530+0000","last_fullsized":"2024-05-30T17:10:06.570530+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:01:27.231981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"265'7","reported_seq":71,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:01.565640+0000","last_change":"2024-05-30T16:17:31.191604+0000","last_active":"2024-05-30T17:10:01.565640+0000","last_peered":"2024-05-30T17:10:01.565640+0000","last_clean":"2024-05-30T17:10:01.565640+0000","last_became_active":"2024-05-30T16:17:31.190742+0000","last_became_peered":"2024-05-30T16:17:31.190742+0000","last_unstale":"2024-05-30T17:10:01.565640+0000","last_undegraded":"2024-05-30T17:10:01.565640+0000","last_fullsized":"2024-05-30T17:10:01.565640+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:37:20.245898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485760,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":30722,"num_write":16,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"265'33","reported_seq":143,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.785481+0000","last_change":"2024-05-30T16:12:50.821208+0000","last_active":"2024-05-30T17:10:51.785481+0000","last_peered":"2024-05-30T17:10:51.785481+0000","last_clean":"2024-05-30T17:10:51.785481+0000","last_became_active":"2024-05-30T16:12:50.820343+0000","last_became_peered":"2024-05-30T16:12:50.820343+0000","last_unstale":"2024-05-30T17:10:51.785481+0000","last_undegraded":"2024-05-30T17:10:51.785481+0000","last_fullsized":"2024-05-30T17:10:51.785481+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:46:09.411694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":154,"num_read_kb":114,"num_write":68,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.772692+0000","last_change":"2024-05-30T16:34:55.889437+0000","last_active":"2024-05-30T17:09:56.772692+0000","last_peered":"2024-05-30T17:09:56.772692+0000","last_clean":"2024-05-30T17:09:56.772692+0000","last_became_active":"2024-05-30T16:34:55.888238+0000","last_became_peered":"2024-05-30T16:34:55.888238+0000","last_unstale":"2024-05-30T17:09:56.772692+0000","last_undegraded":"2024-05-30T17:09:56.772692+0000","last_fullsized":"2024-05-30T17:09:56.772692+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:23:02.664317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"262'6","reported_seq":56,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.579797+0000","last_change":"2024-05-30T16:17:35.740004+0000","last_active":"2024-05-30T17:10:46.579797+0000","last_peered":"2024-05-30T17:10:46.579797+0000","last_clean":"2024-05-30T17:10:46.579797+0000","last_became_active":"2024-05-30T16:17:35.738420+0000","last_became_peered":"2024-05-30T16:17:35.738420+0000","last_unstale":"2024-05-30T17:10:46.579797+0000","last_undegraded":"2024-05-30T17:10:46.579797+0000","last_fullsized":"2024-05-30T17:10:46.579797+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:38:21.023993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6144,"num_write":16,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"265'4","reported_seq":59,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:08:57.120072+0000","last_change":"2024-05-30T16:17:31.195609+0000","last_active":"2024-05-30T17:08:57.120072+0000","last_peered":"2024-05-30T17:08:57.120072+0000","last_clean":"2024-05-30T17:08:57.120072+0000","last_became_active":"2024-05-30T16:17:31.195031+0000","last_became_peered":"2024-05-30T16:17:31.195031+0000","last_unstale":"2024-05-30T17:08:57.120072+0000","last_undegraded":"2024-05-30T17:08:57.120072+0000","last_fullsized":"2024-05-30T17:08:57.120072+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:38:54.212446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":584363,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":6715,"num_write":4,"num_write_kb":6715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"265'65","reported_seq":237,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.577047+0000","last_change":"2024-05-30T16:12:50.820938+0000","last_active":"2024-05-30T17:10:51.577047+0000","last_peered":"2024-05-30T17:10:51.577047+0000","last_clean":"2024-05-30T17:10:51.577047+0000","last_became_active":"2024-05-30T16:12:50.807771+0000","last_became_peered":"2024-05-30T16:12:50.807771+0000","last_unstale":"2024-05-30T17:10:51.577047+0000","last_undegraded":"2024-05-30T17:10:51.577047+0000","last_fullsized":"2024-05-30T17:10:51.577047+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:20:23.418802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":343,"num_read_kb":242,"num_write":116,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"265'1","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.543341+0000","last_change":"2024-05-30T16:34:55.873393+0000","last_active":"2024-05-30T17:09:56.543341+0000","last_peered":"2024-05-30T17:09:56.543341+0000","last_clean":"2024-05-30T17:09:56.543341+0000","last_became_active":"2024-05-30T16:34:55.872581+0000","last_became_peered":"2024-05-30T16:34:55.872581+0000","last_unstale":"2024-05-30T17:09:56.543341+0000","last_undegraded":"2024-05-30T17:09:56.543341+0000","last_fullsized":"2024-05-30T17:09:56.543341+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:23:46.116567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"0'0","reported_seq":45,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:11.557772+0000","last_change":"2024-05-30T16:17:35.739267+0000","last_active":"2024-05-30T17:09:11.557772+0000","last_peered":"2024-05-30T17:09:11.557772+0000","last_clean":"2024-05-30T17:09:11.557772+0000","last_became_active":"2024-05-30T16:17:35.738397+0000","last_became_peered":"2024-05-30T16:17:35.738397+0000","last_unstale":"2024-05-30T17:09:11.557772+0000","last_undegraded":"2024-05-30T17:09:11.557772+0000","last_fullsized":"2024-05-30T17:09:11.557772+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:09:46.285921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"265'2","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:41.559336+0000","last_change":"2024-05-30T16:17:31.197103+0000","last_active":"2024-05-30T17:10:41.559336+0000","last_peered":"2024-05-30T17:10:41.559336+0000","last_clean":"2024-05-30T17:10:41.559336+0000","last_became_active":"2024-05-30T16:17:31.196345+0000","last_became_peered":"2024-05-30T16:17:31.196345+0000","last_unstale":"2024-05-30T17:10:41.559336+0000","last_undegraded":"2024-05-30T17:10:41.559336+0000","last_fullsized":"2024-05-30T17:10:41.559336+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:39:32.522011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":5120,"num_write":2,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"265'38","reported_seq":159,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.581145+0000","last_change":"2024-05-30T16:12:50.827196+0000","last_active":"2024-05-30T17:10:51.581145+0000","last_peered":"2024-05-30T17:10:51.581145+0000","last_clean":"2024-05-30T17:10:51.581145+0000","last_became_active":"2024-05-30T16:12:50.826729+0000","last_became_peered":"2024-05-30T16:12:50.826729+0000","last_unstale":"2024-05-30T17:10:51.581145+0000","last_undegraded":"2024-05-30T17:10:51.581145+0000","last_fullsized":"2024-05-30T17:10:51.581145+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:46:31.837238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":187,"num_read_kb":132,"num_write":81,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.548483+0000","last_change":"2024-05-30T16:34:55.876495+0000","last_active":"2024-05-30T17:09:56.548483+0000","last_peered":"2024-05-30T17:09:56.548483+0000","last_clean":"2024-05-30T17:09:56.548483+0000","last_became_active":"2024-05-30T16:34:55.875788+0000","last_became_peered":"2024-05-30T16:34:55.875788+0000","last_unstale":"2024-05-30T17:09:56.548483+0000","last_undegraded":"2024-05-30T17:09:56.548483+0000","last_fullsized":"2024-05-30T17:09:56.548483+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:15:31.389508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"0'0","reported_seq":45,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.579917+0000","last_change":"2024-05-30T16:17:35.746542+0000","last_active":"2024-05-30T17:10:46.579917+0000","last_peered":"2024-05-30T17:10:46.579917+0000","last_clean":"2024-05-30T17:10:46.579917+0000","last_became_active":"2024-05-30T16:17:35.745769+0000","last_became_peered":"2024-05-30T16:17:35.745769+0000","last_unstale":"2024-05-30T17:10:46.579917+0000","last_undegraded":"2024-05-30T17:10:46.579917+0000","last_fullsized":"2024-05-30T17:10:46.579917+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:03:42.283507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"265'1","reported_seq":53,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:41.552055+0000","last_change":"2024-05-30T16:17:31.206243+0000","last_active":"2024-05-30T17:10:41.552055+0000","last_peered":"2024-05-30T17:10:41.552055+0000","last_clean":"2024-05-30T17:10:41.552055+0000","last_became_active":"2024-05-30T16:17:31.204434+0000","last_became_peered":"2024-05-30T16:17:31.204434+0000","last_unstale":"2024-05-30T17:10:41.552055+0000","last_undegraded":"2024-05-30T17:10:41.552055+0000","last_fullsized":"2024-05-30T17:10:41.552055+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:33:27.089723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":3072,"num_write":1,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"265'26","reported_seq":125,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.554131+0000","last_change":"2024-05-30T16:12:50.814716+0000","last_active":"2024-05-30T17:10:51.554131+0000","last_peered":"2024-05-30T17:10:51.554131+0000","last_clean":"2024-05-30T17:10:51.554131+0000","last_became_active":"2024-05-30T16:12:50.813176+0000","last_became_peered":"2024-05-30T16:12:50.813176+0000","last_unstale":"2024-05-30T17:10:51.554131+0000","last_undegraded":"2024-05-30T17:10:51.554131+0000","last_fullsized":"2024-05-30T17:10:51.554131+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:45:17.112612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":85,"num_write":57,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:57.133868+0000","last_change":"2024-05-30T16:34:55.872909+0000","last_active":"2024-05-30T17:09:57.133868+0000","last_peered":"2024-05-30T17:09:57.133868+0000","last_clean":"2024-05-30T17:09:57.133868+0000","last_became_active":"2024-05-30T16:34:55.871800+0000","last_became_peered":"2024-05-30T16:34:55.871800+0000","last_unstale":"2024-05-30T17:09:57.133868+0000","last_undegraded":"2024-05-30T17:09:57.133868+0000","last_fullsized":"2024-05-30T17:09:57.133868+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:10:45.189484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"265'1","reported_seq":47,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.741327+0000","last_change":"2024-05-30T16:17:35.237595+0000","last_active":"2024-05-30T17:10:46.741327+0000","last_peered":"2024-05-30T17:10:46.741327+0000","last_clean":"2024-05-30T17:10:46.741327+0000","last_became_active":"2024-05-30T16:17:35.236867+0000","last_became_peered":"2024-05-30T16:17:35.236867+0000","last_unstale":"2024-05-30T17:10:46.741327+0000","last_undegraded":"2024-05-30T17:10:46.741327+0000","last_fullsized":"2024-05-30T17:10:46.741327+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:47:01.389981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":584363,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":571,"num_write":1,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"0'0","reported_seq":51,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:41.574782+0000","last_change":"2024-05-30T16:17:31.210713+0000","last_active":"2024-05-30T17:10:41.574782+0000","last_peered":"2024-05-30T17:10:41.574782+0000","last_clean":"2024-05-30T17:10:41.574782+0000","last_became_active":"2024-05-30T16:17:31.209882+0000","last_became_peered":"2024-05-30T16:17:31.209882+0000","last_unstale":"2024-05-30T17:10:41.574782+0000","last_undegraded":"2024-05-30T17:10:41.574782+0000","last_fullsized":"2024-05-30T17:10:41.574782+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:17:20.852336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"265'56","reported_seq":218,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:01.749663+0000","last_change":"2024-05-30T16:12:50.825181+0000","last_active":"2024-05-30T17:09:01.749663+0000","last_peered":"2024-05-30T17:09:01.749663+0000","last_clean":"2024-05-30T17:09:01.749663+0000","last_became_active":"2024-05-30T16:12:50.824043+0000","last_became_peered":"2024-05-30T16:12:50.824043+0000","last_unstale":"2024-05-30T17:09:01.749663+0000","last_undegraded":"2024-05-30T17:09:01.749663+0000","last_fullsized":"2024-05-30T17:09:01.749663+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:29.566016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":308,"num_read_kb":221,"num_write":92,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.585310+0000","last_change":"2024-05-30T16:34:55.893966+0000","last_active":"2024-05-30T17:11:11.585310+0000","last_peered":"2024-05-30T17:11:11.585310+0000","last_clean":"2024-05-30T17:11:11.585310+0000","last_became_active":"2024-05-30T16:34:55.893040+0000","last_became_peered":"2024-05-30T16:34:55.893040+0000","last_unstale":"2024-05-30T17:11:11.585310+0000","last_undegraded":"2024-05-30T17:11:11.585310+0000","last_fullsized":"2024-05-30T17:11:11.585310+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:55:00.349712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"0'0","reported_seq":46,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.790902+0000","last_change":"2024-05-30T16:17:35.246948+0000","last_active":"2024-05-30T17:11:16.790902+0000","last_peered":"2024-05-30T17:11:16.790902+0000","last_clean":"2024-05-30T17:11:16.790902+0000","last_became_active":"2024-05-30T16:17:35.246187+0000","last_became_peered":"2024-05-30T16:17:35.246187+0000","last_unstale":"2024-05-30T17:11:16.790902+0000","last_undegraded":"2024-05-30T17:11:16.790902+0000","last_fullsized":"2024-05-30T17:11:16.790902+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:23:58.794231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"265'6","reported_seq":69,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.746911+0000","last_change":"2024-05-30T16:17:31.189808+0000","last_active":"2024-05-30T17:11:11.746911+0000","last_peered":"2024-05-30T17:11:11.746911+0000","last_clean":"2024-05-30T17:11:11.746911+0000","last_became_active":"2024-05-30T16:17:31.189091+0000","last_became_peered":"2024-05-30T16:17:31.189091+0000","last_unstale":"2024-05-30T17:11:11.746911+0000","last_undegraded":"2024-05-30T17:11:11.746911+0000","last_fullsized":"2024-05-30T17:11:11.746911+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:35:04.039725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5827243,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":21053,"num_write":5,"num_write_kb":5691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"265'9","reported_seq":84,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.566377+0000","last_change":"2024-05-30T16:12:50.824521+0000","last_active":"2024-05-30T17:11:11.566377+0000","last_peered":"2024-05-30T17:11:11.566377+0000","last_clean":"2024-05-30T17:11:11.566377+0000","last_became_active":"2024-05-30T16:12:50.823759+0000","last_became_peered":"2024-05-30T16:12:50.823759+0000","last_unstale":"2024-05-30T17:11:11.566377+0000","last_undegraded":"2024-05-30T17:11:11.566377+0000","last_fullsized":"2024-05-30T17:11:11.566377+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:41:59.872702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":33,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.581162+0000","last_change":"2024-05-30T16:34:55.861418+0000","last_active":"2024-05-30T17:11:11.581162+0000","last_peered":"2024-05-30T17:11:11.581162+0000","last_clean":"2024-05-30T17:11:11.581162+0000","last_became_active":"2024-05-30T16:34:55.860695+0000","last_became_peered":"2024-05-30T16:34:55.860695+0000","last_unstale":"2024-05-30T17:11:11.581162+0000","last_undegraded":"2024-05-30T17:11:11.581162+0000","last_fullsized":"2024-05-30T17:11:11.581162+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:45:54.013286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"0'0","reported_seq":46,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.790812+0000","last_change":"2024-05-30T16:17:35.239935+0000","last_active":"2024-05-30T17:11:16.790812+0000","last_peered":"2024-05-30T17:11:16.790812+0000","last_clean":"2024-05-30T17:11:16.790812+0000","last_became_active":"2024-05-30T16:17:35.239157+0000","last_became_peered":"2024-05-30T16:17:35.239157+0000","last_unstale":"2024-05-30T17:11:16.790812+0000","last_undegraded":"2024-05-30T17:11:16.790812+0000","last_fullsized":"2024-05-30T17:11:16.790812+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:53:43.552214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"265'1","reported_seq":54,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.585076+0000","last_change":"2024-05-30T16:17:31.185375+0000","last_active":"2024-05-30T17:11:11.585076+0000","last_peered":"2024-05-30T17:11:11.585076+0000","last_clean":"2024-05-30T17:11:11.585076+0000","last_became_active":"2024-05-30T16:17:31.184675+0000","last_became_peered":"2024-05-30T16:17:31.184675+0000","last_unstale":"2024-05-30T17:11:11.585076+0000","last_undegraded":"2024-05-30T17:11:11.585076+0000","last_fullsized":"2024-05-30T17:11:11.585076+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:13:30.582952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":714225,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":698,"num_write":1,"num_write_kb":698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"262'96","reported_seq":357,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.581035+0000","last_change":"2024-05-30T16:12:50.802561+0000","last_active":"2024-05-30T17:11:11.581035+0000","last_peered":"2024-05-30T17:11:11.581035+0000","last_clean":"2024-05-30T17:11:11.581035+0000","last_became_active":"2024-05-30T16:12:50.801791+0000","last_became_peered":"2024-05-30T16:12:50.801791+0000","last_unstale":"2024-05-30T17:11:11.581035+0000","last_undegraded":"2024-05-30T17:11:11.581035+0000","last_fullsized":"2024-05-30T17:11:11.581035+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:19:14.883602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":591,"num_read_kb":446,"num_write":104,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"264'1","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:06.566810+0000","last_change":"2024-05-30T16:34:55.864834+0000","last_active":"2024-05-30T17:10:06.566810+0000","last_peered":"2024-05-30T17:10:06.566810+0000","last_clean":"2024-05-30T17:10:06.566810+0000","last_became_active":"2024-05-30T16:34:55.863483+0000","last_became_peered":"2024-05-30T16:34:55.863483+0000","last_unstale":"2024-05-30T17:10:06.566810+0000","last_undegraded":"2024-05-30T17:10:06.566810+0000","last_fullsized":"2024-05-30T17:10:06.566810+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:46:20.034900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"0'0","reported_seq":46,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.586383+0000","last_change":"2024-05-30T16:17:35.236085+0000","last_active":"2024-05-30T17:11:16.586383+0000","last_peered":"2024-05-30T17:11:16.586383+0000","last_clean":"2024-05-30T17:11:16.586383+0000","last_became_active":"2024-05-30T16:17:35.235283+0000","last_became_peered":"2024-05-30T16:17:35.235283+0000","last_unstale":"2024-05-30T17:11:16.586383+0000","last_undegraded":"2024-05-30T17:11:16.586383+0000","last_fullsized":"2024-05-30T17:11:16.586383+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:19:44.053076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"0'0","reported_seq":52,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.557098+0000","last_change":"2024-05-30T16:17:31.204968+0000","last_active":"2024-05-30T17:11:11.557098+0000","last_peered":"2024-05-30T17:11:11.557098+0000","last_clean":"2024-05-30T17:11:11.557098+0000","last_became_active":"2024-05-30T16:17:31.199499+0000","last_became_peered":"2024-05-30T16:17:31.199499+0000","last_unstale":"2024-05-30T17:11:11.557098+0000","last_undegraded":"2024-05-30T17:11:11.557098+0000","last_fullsized":"2024-05-30T17:11:11.557098+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:18:34.513889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"265'22","reported_seq":118,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.746972+0000","last_change":"2024-05-30T16:12:50.807855+0000","last_active":"2024-05-30T17:11:11.746972+0000","last_peered":"2024-05-30T17:11:11.746972+0000","last_clean":"2024-05-30T17:11:11.746972+0000","last_became_active":"2024-05-30T16:12:50.807156+0000","last_became_peered":"2024-05-30T16:12:50.807156+0000","last_unstale":"2024-05-30T17:11:11.746972+0000","last_undegraded":"2024-05-30T17:11:11.746972+0000","last_fullsized":"2024-05-30T17:11:11.746972+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:18:22.683470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":76,"num_write":50,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"265'1","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.585231+0000","last_change":"2024-05-30T16:34:55.889162+0000","last_active":"2024-05-30T17:11:11.585231+0000","last_peered":"2024-05-30T17:11:11.585231+0000","last_clean":"2024-05-30T17:11:11.585231+0000","last_became_active":"2024-05-30T16:34:55.887845+0000","last_became_peered":"2024-05-30T16:34:55.887845+0000","last_unstale":"2024-05-30T17:11:11.585231+0000","last_undegraded":"2024-05-30T17:11:11.585231+0000","last_fullsized":"2024-05-30T17:11:11.585231+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:36:50.835387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"0'0","reported_seq":46,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.558367+0000","last_change":"2024-05-30T16:17:35.246711+0000","last_active":"2024-05-30T17:11:16.558367+0000","last_peered":"2024-05-30T17:11:16.558367+0000","last_clean":"2024-05-30T17:11:16.558367+0000","last_became_active":"2024-05-30T16:17:35.245852+0000","last_became_peered":"2024-05-30T16:17:35.245852+0000","last_unstale":"2024-05-30T17:11:16.558367+0000","last_undegraded":"2024-05-30T17:11:16.558367+0000","last_fullsized":"2024-05-30T17:11:16.558367+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:50:22.430770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"262'5","reported_seq":67,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:12.150705+0000","last_change":"2024-05-30T16:17:31.219308+0000","last_active":"2024-05-30T17:11:12.150705+0000","last_peered":"2024-05-30T17:11:12.150705+0000","last_clean":"2024-05-30T17:11:12.150705+0000","last_became_active":"2024-05-30T16:17:31.218752+0000","last_became_peered":"2024-05-30T16:17:31.218752+0000","last_unstale":"2024-05-30T17:11:12.150705+0000","last_undegraded":"2024-05-30T17:11:12.150705+0000","last_fullsized":"2024-05-30T17:11:12.150705+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:47:57.704377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"265'17","reported_seq":103,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.584982+0000","last_change":"2024-05-30T16:12:50.829602+0000","last_active":"2024-05-30T17:11:11.584982+0000","last_peered":"2024-05-30T17:11:11.584982+0000","last_clean":"2024-05-30T17:11:11.584982+0000","last_became_active":"2024-05-30T16:12:50.817114+0000","last_became_peered":"2024-05-30T16:12:50.817114+0000","last_unstale":"2024-05-30T17:11:11.584982+0000","last_undegraded":"2024-05-30T17:11:11.584982+0000","last_fullsized":"2024-05-30T17:11:11.584982+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:05:48.646667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":55,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:46.541438+0000","last_change":"2024-05-30T16:34:55.867126+0000","last_active":"2024-05-30T17:09:46.541438+0000","last_peered":"2024-05-30T17:09:46.541438+0000","last_clean":"2024-05-30T17:09:46.541438+0000","last_became_active":"2024-05-30T16:34:55.866415+0000","last_became_peered":"2024-05-30T16:34:55.866415+0000","last_unstale":"2024-05-30T17:09:46.541438+0000","last_undegraded":"2024-05-30T17:09:46.541438+0000","last_fullsized":"2024-05-30T17:09:46.541438+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:05:20.422821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"0'0","reported_seq":46,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.560610+0000","last_change":"2024-05-30T16:17:35.228446+0000","last_active":"2024-05-30T17:09:21.560610+0000","last_peered":"2024-05-30T17:09:21.560610+0000","last_clean":"2024-05-30T17:09:21.560610+0000","last_became_active":"2024-05-30T16:17:35.227270+0000","last_became_peered":"2024-05-30T16:17:35.227270+0000","last_unstale":"2024-05-30T17:09:21.560610+0000","last_undegraded":"2024-05-30T17:09:21.560610+0000","last_fullsized":"2024-05-30T17:09:21.560610+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:47:07.315021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"0'0","reported_seq":52,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:16.753258+0000","last_change":"2024-05-30T16:17:31.204447+0000","last_active":"2024-05-30T17:09:16.753258+0000","last_peered":"2024-05-30T17:09:16.753258+0000","last_clean":"2024-05-30T17:09:16.753258+0000","last_became_active":"2024-05-30T16:17:31.203731+0000","last_became_peered":"2024-05-30T16:17:31.203731+0000","last_unstale":"2024-05-30T17:09:16.753258+0000","last_undegraded":"2024-05-30T17:09:16.753258+0000","last_fullsized":"2024-05-30T17:09:16.753258+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:02:51.039638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"265'29","reported_seq":136,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.560856+0000","last_change":"2024-05-30T16:12:50.825208+0000","last_active":"2024-05-30T17:09:21.560856+0000","last_peered":"2024-05-30T17:09:21.560856+0000","last_clean":"2024-05-30T17:09:21.560856+0000","last_became_active":"2024-05-30T16:12:50.824809+0000","last_became_peered":"2024-05-30T16:12:50.824809+0000","last_unstale":"2024-05-30T17:09:21.560856+0000","last_undegraded":"2024-05-30T17:09:21.560856+0000","last_fullsized":"2024-05-30T17:09:21.560856+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:38:19.384141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":98,"num_write":77,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:56.562832+0000","last_change":"2024-05-30T16:34:55.897061+0000","last_active":"2024-05-30T17:10:56.562832+0000","last_peered":"2024-05-30T17:10:56.562832+0000","last_clean":"2024-05-30T17:10:56.562832+0000","last_became_active":"2024-05-30T16:34:55.896357+0000","last_became_peered":"2024-05-30T16:34:55.896357+0000","last_unstale":"2024-05-30T17:10:56.562832+0000","last_undegraded":"2024-05-30T17:10:56.562832+0000","last_fullsized":"2024-05-30T17:10:56.562832+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:23:44.881356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"265'2","reported_seq":44,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:42.130647+0000","last_change":"2024-05-30T16:17:55.551397+0000","last_active":"2024-05-30T17:09:42.130647+0000","last_peered":"2024-05-30T17:09:42.130647+0000","last_clean":"2024-05-30T17:09:42.130647+0000","last_became_active":"2024-05-30T16:17:55.550782+0000","last_became_peered":"2024-05-30T16:17:55.550782+0000","last_unstale":"2024-05-30T17:09:42.130647+0000","last_undegraded":"2024-05-30T17:09:42.130647+0000","last_fullsized":"2024-05-30T17:09:42.130647+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:44:58.931548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2048,"num_write":7,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.d","version":"265'5","reported_seq":62,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:16.535485+0000","last_change":"2024-05-30T16:17:31.202846+0000","last_active":"2024-05-30T17:09:16.535485+0000","last_peered":"2024-05-30T17:09:16.535485+0000","last_clean":"2024-05-30T17:09:16.535485+0000","last_became_active":"2024-05-30T16:17:31.202042+0000","last_became_peered":"2024-05-30T16:17:31.202042+0000","last_unstale":"2024-05-30T17:09:16.535485+0000","last_undegraded":"2024-05-30T17:09:16.535485+0000","last_fullsized":"2024-05-30T17:09:16.535485+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:38:37.711263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6339871,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":6194,"num_write":5,"num_write_kb":6194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"265'24","reported_seq":130,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:22.153145+0000","last_change":"2024-05-30T16:12:50.810823+0000","last_active":"2024-05-30T17:11:22.153145+0000","last_peered":"2024-05-30T17:11:22.153145+0000","last_clean":"2024-05-30T17:11:22.153145+0000","last_became_active":"2024-05-30T16:12:50.810095+0000","last_became_peered":"2024-05-30T16:12:50.810095+0000","last_unstale":"2024-05-30T17:11:22.153145+0000","last_undegraded":"2024-05-30T17:11:22.153145+0000","last_fullsized":"2024-05-30T17:11:22.153145+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:29:53.744702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":90,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.536265+0000","last_change":"2024-05-30T16:34:55.863218+0000","last_active":"2024-05-30T17:09:36.536265+0000","last_peered":"2024-05-30T17:09:36.536265+0000","last_clean":"2024-05-30T17:09:36.536265+0000","last_became_active":"2024-05-30T16:34:55.862478+0000","last_became_peered":"2024-05-30T16:34:55.862478+0000","last_unstale":"2024-05-30T17:09:36.536265+0000","last_undegraded":"2024-05-30T17:09:36.536265+0000","last_fullsized":"2024-05-30T17:09:36.536265+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:30.967780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"262'3","reported_seq":52,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:31.768287+0000","last_change":"2024-05-30T16:17:35.743152+0000","last_active":"2024-05-30T17:09:31.768287+0000","last_peered":"2024-05-30T17:09:31.768287+0000","last_clean":"2024-05-30T17:09:31.768287+0000","last_became_active":"2024-05-30T16:17:35.742255+0000","last_became_peered":"2024-05-30T16:17:35.742255+0000","last_unstale":"2024-05-30T17:09:31.768287+0000","last_undegraded":"2024-05-30T17:09:31.768287+0000","last_fullsized":"2024-05-30T17:09:31.768287+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:41:50.476109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"265'1","reported_seq":54,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:16.556005+0000","last_change":"2024-05-30T16:17:31.199481+0000","last_active":"2024-05-30T17:09:16.556005+0000","last_peered":"2024-05-30T17:09:16.556005+0000","last_clean":"2024-05-30T17:09:16.556005+0000","last_became_active":"2024-05-30T16:17:31.198744+0000","last_became_peered":"2024-05-30T16:17:31.198744+0000","last_unstale":"2024-05-30T17:09:16.556005+0000","last_undegraded":"2024-05-30T17:09:16.556005+0000","last_fullsized":"2024-05-30T17:09:16.556005+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:39:35.328874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":3072,"num_write":1,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"265'36","reported_seq":163,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.559692+0000","last_change":"2024-05-30T16:12:50.821672+0000","last_active":"2024-05-30T17:11:21.559692+0000","last_peered":"2024-05-30T17:11:21.559692+0000","last_clean":"2024-05-30T17:11:21.559692+0000","last_became_active":"2024-05-30T16:12:50.820936+0000","last_became_peered":"2024-05-30T16:12:50.820936+0000","last_unstale":"2024-05-30T17:11:21.559692+0000","last_undegraded":"2024-05-30T17:11:21.559692+0000","last_fullsized":"2024-05-30T17:11:21.559692+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:52:42.146262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":194,"num_read_kb":136,"num_write":73,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.543696+0000","last_change":"2024-05-30T16:34:55.877239+0000","last_active":"2024-05-30T17:09:36.543696+0000","last_peered":"2024-05-30T17:09:36.543696+0000","last_clean":"2024-05-30T17:09:36.543696+0000","last_became_active":"2024-05-30T16:34:55.875965+0000","last_became_peered":"2024-05-30T16:34:55.875965+0000","last_unstale":"2024-05-30T17:09:36.543696+0000","last_undegraded":"2024-05-30T17:09:36.543696+0000","last_fullsized":"2024-05-30T17:09:36.543696+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:10:50.846236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"265'2","reported_seq":50,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.560723+0000","last_change":"2024-05-30T16:17:35.234741+0000","last_active":"2024-05-30T17:09:21.560723+0000","last_peered":"2024-05-30T17:09:21.560723+0000","last_clean":"2024-05-30T17:09:21.560723+0000","last_became_active":"2024-05-30T16:17:35.229643+0000","last_became_peered":"2024-05-30T16:17:35.229643+0000","last_unstale":"2024-05-30T17:09:21.560723+0000","last_undegraded":"2024-05-30T17:09:21.560723+0000","last_fullsized":"2024-05-30T17:09:21.560723+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:55:14.642511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":6144,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.f","version":"262'6","reported_seq":63,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:26.767116+0000","last_change":"2024-05-30T16:17:31.215290+0000","last_active":"2024-05-30T17:09:26.767116+0000","last_peered":"2024-05-30T17:09:26.767116+0000","last_clean":"2024-05-30T17:09:26.767116+0000","last_became_active":"2024-05-30T16:17:31.212894+0000","last_became_peered":"2024-05-30T16:17:31.212894+0000","last_unstale":"2024-05-30T17:09:26.767116+0000","last_undegraded":"2024-05-30T17:09:26.767116+0000","last_fullsized":"2024-05-30T17:09:26.767116+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:31:09.789041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6144,"num_write":11,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"262'21","reported_seq":121,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.559938+0000","last_change":"2024-05-30T16:12:50.806577+0000","last_active":"2024-05-30T17:11:21.559938+0000","last_peered":"2024-05-30T17:11:21.559938+0000","last_clean":"2024-05-30T17:11:21.559938+0000","last_became_active":"2024-05-30T16:12:50.805772+0000","last_became_peered":"2024-05-30T16:12:50.805772+0000","last_unstale":"2024-05-30T17:11:21.559938+0000","last_undegraded":"2024-05-30T17:11:21.559938+0000","last_fullsized":"2024-05-30T17:11:21.559938+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:11:36.510014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":79,"num_write":45,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:56.582343+0000","last_change":"2024-05-30T16:34:55.891074+0000","last_active":"2024-05-30T17:10:56.582343+0000","last_peered":"2024-05-30T17:10:56.582343+0000","last_clean":"2024-05-30T17:10:56.582343+0000","last_became_active":"2024-05-30T16:34:55.890313+0000","last_became_peered":"2024-05-30T16:34:55.890313+0000","last_unstale":"2024-05-30T17:10:56.582343+0000","last_undegraded":"2024-05-30T17:10:56.582343+0000","last_fullsized":"2024-05-30T17:10:56.582343+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:22:36.639758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"265'2","reported_seq":53,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.726171+0000","last_change":"2024-05-30T16:17:35.243089+0000","last_active":"2024-05-30T17:09:36.726171+0000","last_peered":"2024-05-30T17:09:36.726171+0000","last_clean":"2024-05-30T17:09:36.726171+0000","last_became_active":"2024-05-30T16:17:35.242271+0000","last_became_peered":"2024-05-30T16:17:35.242271+0000","last_unstale":"2024-05-30T17:09:36.726171+0000","last_undegraded":"2024-05-30T17:09:36.726171+0000","last_fullsized":"2024-05-30T17:09:36.726171+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:44:49.304400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":10240,"num_write":7,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"265'2","reported_seq":59,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:31.559803+0000","last_change":"2024-05-30T16:17:31.195571+0000","last_active":"2024-05-30T17:09:31.559803+0000","last_peered":"2024-05-30T17:09:31.559803+0000","last_clean":"2024-05-30T17:09:31.559803+0000","last_became_active":"2024-05-30T16:17:31.194774+0000","last_became_peered":"2024-05-30T16:17:31.194774+0000","last_unstale":"2024-05-30T17:09:31.559803+0000","last_undegraded":"2024-05-30T17:09:31.559803+0000","last_fullsized":"2024-05-30T17:09:31.559803+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:24:47.216863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":12288,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"265'18","reported_seq":107,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.584196+0000","last_change":"2024-05-30T16:12:50.820954+0000","last_active":"2024-05-30T17:11:21.584196+0000","last_peered":"2024-05-30T17:11:21.584196+0000","last_clean":"2024-05-30T17:11:21.584196+0000","last_became_active":"2024-05-30T16:12:50.820526+0000","last_became_peered":"2024-05-30T16:12:50.820526+0000","last_unstale":"2024-05-30T17:11:21.584196+0000","last_undegraded":"2024-05-30T17:11:21.584196+0000","last_fullsized":"2024-05-30T17:11:21.584196+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:08:41.493544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":60,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:06.565282+0000","last_change":"2024-05-30T16:34:55.884293+0000","last_active":"2024-05-30T17:11:06.565282+0000","last_peered":"2024-05-30T17:11:06.565282+0000","last_clean":"2024-05-30T17:11:06.565282+0000","last_became_active":"2024-05-30T16:34:55.884065+0000","last_became_peered":"2024-05-30T16:34:55.884065+0000","last_unstale":"2024-05-30T17:11:06.565282+0000","last_undegraded":"2024-05-30T17:11:06.565282+0000","last_fullsized":"2024-05-30T17:11:06.565282+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:13:34.815638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"265'9","reported_seq":71,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.543613+0000","last_change":"2024-05-30T16:17:35.739756+0000","last_active":"2024-05-30T17:09:36.543613+0000","last_peered":"2024-05-30T17:09:36.543613+0000","last_clean":"2024-05-30T17:09:36.543613+0000","last_became_active":"2024-05-30T16:17:35.738991+0000","last_became_peered":"2024-05-30T16:17:35.738991+0000","last_unstale":"2024-05-30T17:09:36.543613+0000","last_undegraded":"2024-05-30T17:09:36.543613+0000","last_fullsized":"2024-05-30T17:09:36.543613+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:50:21.381841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5957105,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":26998,"num_write":13,"num_write_kb":10938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"262'6","reported_seq":65,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:31.768080+0000","last_change":"2024-05-30T16:17:31.216617+0000","last_active":"2024-05-30T17:09:31.768080+0000","last_peered":"2024-05-30T17:09:31.768080+0000","last_clean":"2024-05-30T17:09:31.768080+0000","last_became_active":"2024-05-30T16:17:31.212954+0000","last_became_peered":"2024-05-30T16:17:31.212954+0000","last_unstale":"2024-05-30T17:09:31.768080+0000","last_undegraded":"2024-05-30T17:09:31.768080+0000","last_fullsized":"2024-05-30T17:09:31.768080+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:53:12.730929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":11264,"num_write":11,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"262'102","reported_seq":385,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.754543+0000","last_change":"2024-05-30T16:12:50.821280+0000","last_active":"2024-05-30T17:09:21.754543+0000","last_peered":"2024-05-30T17:09:21.754543+0000","last_clean":"2024-05-30T17:09:21.754543+0000","last_became_active":"2024-05-30T16:12:50.820250+0000","last_became_peered":"2024-05-30T16:12:50.820250+0000","last_unstale":"2024-05-30T17:09:21.754543+0000","last_undegraded":"2024-05-30T17:09:21.754543+0000","last_fullsized":"2024-05-30T17:09:21.754543+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:43:31.770630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":642,"num_read_kb":499,"num_write":114,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"265'1","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:56.562900+0000","last_change":"2024-05-30T16:34:55.880548+0000","last_active":"2024-05-30T17:10:56.562900+0000","last_peered":"2024-05-30T17:10:56.562900+0000","last_clean":"2024-05-30T17:10:56.562900+0000","last_became_active":"2024-05-30T16:34:55.878171+0000","last_became_peered":"2024-05-30T16:34:55.878171+0000","last_unstale":"2024-05-30T17:10:56.562900+0000","last_undegraded":"2024-05-30T17:10:56.562900+0000","last_fullsized":"2024-05-30T17:10:56.562900+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:56:14.165458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"265'11","reported_seq":71,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.765556+0000","last_change":"2024-05-30T16:17:35.245798+0000","last_active":"2024-05-30T17:09:21.765556+0000","last_peered":"2024-05-30T17:09:21.765556+0000","last_clean":"2024-05-30T17:09:21.765556+0000","last_became_active":"2024-05-30T16:17:35.245008+0000","last_became_peered":"2024-05-30T16:17:35.245008+0000","last_unstale":"2024-05-30T17:09:21.765556+0000","last_undegraded":"2024-05-30T17:09:21.765556+0000","last_fullsized":"2024-05-30T17:09:21.765556+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:28:02.631261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17825792,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":25602,"num_write":20,"num_write_kb":17408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"262'3","reported_seq":59,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:46.562533+0000","last_change":"2024-05-30T16:17:31.211050+0000","last_active":"2024-05-30T17:09:46.562533+0000","last_peered":"2024-05-30T17:09:46.562533+0000","last_clean":"2024-05-30T17:09:46.562533+0000","last_became_active":"2024-05-30T16:17:31.210195+0000","last_became_peered":"2024-05-30T16:17:31.210195+0000","last_unstale":"2024-05-30T17:09:46.562533+0000","last_undegraded":"2024-05-30T17:09:46.562533+0000","last_fullsized":"2024-05-30T17:09:46.562533+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:19:11.621024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"265'43","reported_seq":176,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.587423+0000","last_change":"2024-05-30T16:12:50.831498+0000","last_active":"2024-05-30T17:11:21.587423+0000","last_peered":"2024-05-30T17:11:21.587423+0000","last_clean":"2024-05-30T17:11:21.587423+0000","last_became_active":"2024-05-30T16:12:50.830770+0000","last_became_peered":"2024-05-30T16:12:50.830770+0000","last_unstale":"2024-05-30T17:11:21.587423+0000","last_undegraded":"2024-05-30T17:11:21.587423+0000","last_fullsized":"2024-05-30T17:11:21.587423+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:10:09.450788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":214,"num_read_kb":148,"num_write":89,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:56.578000+0000","last_change":"2024-05-30T16:34:55.887780+0000","last_active":"2024-05-30T17:10:56.578000+0000","last_peered":"2024-05-30T17:10:56.578000+0000","last_clean":"2024-05-30T17:10:56.578000+0000","last_became_active":"2024-05-30T16:34:55.887101+0000","last_became_peered":"2024-05-30T16:34:55.887101+0000","last_unstale":"2024-05-30T17:10:56.578000+0000","last_undegraded":"2024-05-30T17:10:56.578000+0000","last_fullsized":"2024-05-30T17:10:56.578000+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:43:47.335383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"262'3","reported_seq":53,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:51.547299+0000","last_change":"2024-05-30T16:17:35.213121+0000","last_active":"2024-05-30T17:09:51.547299+0000","last_peered":"2024-05-30T17:09:51.547299+0000","last_clean":"2024-05-30T17:09:51.547299+0000","last_became_active":"2024-05-30T16:17:35.212403+0000","last_became_peered":"2024-05-30T16:17:35.212403+0000","last_unstale":"2024-05-30T17:09:51.547299+0000","last_undegraded":"2024-05-30T17:09:51.547299+0000","last_fullsized":"2024-05-30T17:09:51.547299+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:52:56.999514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"0'0","reported_seq":52,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:16.721715+0000","last_change":"2024-05-30T16:17:31.192265+0000","last_active":"2024-05-30T17:09:16.721715+0000","last_peered":"2024-05-30T17:09:16.721715+0000","last_clean":"2024-05-30T17:09:16.721715+0000","last_became_active":"2024-05-30T16:17:31.191955+0000","last_became_peered":"2024-05-30T16:17:31.191955+0000","last_unstale":"2024-05-30T17:09:16.721715+0000","last_undegraded":"2024-05-30T17:09:16.721715+0000","last_fullsized":"2024-05-30T17:09:16.721715+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:16:24.140324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"262'8","reported_seq":83,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.559507+0000","last_change":"2024-05-30T16:12:50.820140+0000","last_active":"2024-05-30T17:11:21.559507+0000","last_peered":"2024-05-30T17:11:21.559507+0000","last_clean":"2024-05-30T17:11:21.559507+0000","last_became_active":"2024-05-30T16:12:50.819871+0000","last_became_peered":"2024-05-30T16:12:50.819871+0000","last_unstale":"2024-05-30T17:11:21.559507+0000","last_undegraded":"2024-05-30T17:11:21.559507+0000","last_fullsized":"2024-05-30T17:11:21.559507+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:14:08.269589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":28,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.554460+0000","last_change":"2024-05-30T16:34:55.883102+0000","last_active":"2024-05-30T17:10:21.554460+0000","last_peered":"2024-05-30T17:10:21.554460+0000","last_clean":"2024-05-30T17:10:21.554460+0000","last_became_active":"2024-05-30T16:34:55.882443+0000","last_became_peered":"2024-05-30T16:34:55.882443+0000","last_unstale":"2024-05-30T17:10:21.554460+0000","last_undegraded":"2024-05-30T17:10:21.554460+0000","last_fullsized":"2024-05-30T17:10:21.554460+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:01:28.745649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"262'3","reported_seq":54,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.564614+0000","last_change":"2024-05-30T16:17:35.248377+0000","last_active":"2024-05-30T17:09:56.564614+0000","last_peered":"2024-05-30T17:09:56.564614+0000","last_clean":"2024-05-30T17:09:56.564614+0000","last_became_active":"2024-05-30T16:17:35.247419+0000","last_became_peered":"2024-05-30T16:17:35.247419+0000","last_unstale":"2024-05-30T17:09:56.564614+0000","last_undegraded":"2024-05-30T17:09:56.564614+0000","last_fullsized":"2024-05-30T17:09:56.564614+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:09:58.043053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"262'2","reported_seq":58,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.543844+0000","last_change":"2024-05-30T16:17:31.189996+0000","last_active":"2024-05-30T17:09:36.543844+0000","last_peered":"2024-05-30T17:09:36.543844+0000","last_clean":"2024-05-30T17:09:36.543844+0000","last_became_active":"2024-05-30T16:17:31.189122+0000","last_became_peered":"2024-05-30T16:17:31.189122+0000","last_unstale":"2024-05-30T17:09:36.543844+0000","last_undegraded":"2024-05-30T17:09:36.543844+0000","last_fullsized":"2024-05-30T17:09:36.543844+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:53:04.697503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"265'53","reported_seq":201,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.584475+0000","last_change":"2024-05-30T16:12:50.821561+0000","last_active":"2024-05-30T17:11:21.584475+0000","last_peered":"2024-05-30T17:11:21.584475+0000","last_clean":"2024-05-30T17:11:21.584475+0000","last_became_active":"2024-05-30T16:12:50.809372+0000","last_became_peered":"2024-05-30T16:12:50.809372+0000","last_unstale":"2024-05-30T17:11:21.584475+0000","last_undegraded":"2024-05-30T17:11:21.584475+0000","last_fullsized":"2024-05-30T17:11:21.584475+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:22:57.030056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":186,"num_write":103,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:22.139649+0000","last_change":"2024-05-30T16:34:55.881085+0000","last_active":"2024-05-30T17:10:22.139649+0000","last_peered":"2024-05-30T17:10:22.139649+0000","last_clean":"2024-05-30T17:10:22.139649+0000","last_became_active":"2024-05-30T16:34:55.880627+0000","last_became_peered":"2024-05-30T16:34:55.880627+0000","last_unstale":"2024-05-30T17:10:22.139649+0000","last_undegraded":"2024-05-30T17:10:22.139649+0000","last_fullsized":"2024-05-30T17:10:22.139649+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:34:40.635399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"0'0","reported_seq":48,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:41.561793+0000","last_change":"2024-05-30T16:17:35.743808+0000","last_active":"2024-05-30T17:09:41.561793+0000","last_peered":"2024-05-30T17:09:41.561793+0000","last_clean":"2024-05-30T17:09:41.561793+0000","last_became_active":"2024-05-30T16:17:35.743036+0000","last_became_peered":"2024-05-30T16:17:35.743036+0000","last_unstale":"2024-05-30T17:09:41.561793+0000","last_undegraded":"2024-05-30T17:09:41.561793+0000","last_fullsized":"2024-05-30T17:09:41.561793+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:12:42.745946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"265'2","reported_seq":58,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:51.542424+0000","last_change":"2024-05-30T16:17:31.202885+0000","last_active":"2024-05-30T17:09:51.542424+0000","last_peered":"2024-05-30T17:09:51.542424+0000","last_clean":"2024-05-30T17:09:51.542424+0000","last_became_active":"2024-05-30T16:17:31.202064+0000","last_became_peered":"2024-05-30T16:17:31.202064+0000","last_unstale":"2024-05-30T17:09:51.542424+0000","last_undegraded":"2024-05-30T17:09:51.542424+0000","last_fullsized":"2024-05-30T17:09:51.542424+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:48:01.223505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2048,"num_write":7,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"265'34","reported_seq":149,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.559731+0000","last_change":"2024-05-30T16:12:50.816644+0000","last_active":"2024-05-30T17:11:21.559731+0000","last_peered":"2024-05-30T17:11:21.559731+0000","last_clean":"2024-05-30T17:11:21.559731+0000","last_became_active":"2024-05-30T16:12:50.815895+0000","last_became_peered":"2024-05-30T16:12:50.815895+0000","last_unstale":"2024-05-30T17:11:21.559731+0000","last_undegraded":"2024-05-30T17:11:21.559731+0000","last_fullsized":"2024-05-30T17:11:21.559731+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:33:57.609516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":160,"num_read_kb":111,"num_write":71,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":31,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:01.787658+0000","last_change":"2024-05-30T16:34:55.883831+0000","last_active":"2024-05-30T17:11:01.787658+0000","last_peered":"2024-05-30T17:11:01.787658+0000","last_clean":"2024-05-30T17:11:01.787658+0000","last_became_active":"2024-05-30T16:34:55.883112+0000","last_became_peered":"2024-05-30T16:34:55.883112+0000","last_unstale":"2024-05-30T17:11:01.787658+0000","last_undegraded":"2024-05-30T17:11:01.787658+0000","last_fullsized":"2024-05-30T17:11:01.787658+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:50:45.079747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"265'2","reported_seq":51,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.543941+0000","last_change":"2024-05-30T16:17:35.744284+0000","last_active":"2024-05-30T17:09:36.543941+0000","last_peered":"2024-05-30T17:09:36.543941+0000","last_clean":"2024-05-30T17:09:36.543941+0000","last_became_active":"2024-05-30T16:17:35.743608+0000","last_became_peered":"2024-05-30T16:17:35.743608+0000","last_unstale":"2024-05-30T17:09:36.543941+0000","last_undegraded":"2024-05-30T17:09:36.543941+0000","last_fullsized":"2024-05-30T17:09:36.543941+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:44:29.493440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":6144,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"0'0","reported_seq":53,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:32.128370+0000","last_change":"2024-05-30T16:17:31.189800+0000","last_active":"2024-05-30T17:09:32.128370+0000","last_peered":"2024-05-30T17:09:32.128370+0000","last_clean":"2024-05-30T17:09:32.128370+0000","last_became_active":"2024-05-30T16:17:31.188810+0000","last_became_peered":"2024-05-30T16:17:31.188810+0000","last_unstale":"2024-05-30T17:09:32.128370+0000","last_undegraded":"2024-05-30T17:09:32.128370+0000","last_fullsized":"2024-05-30T17:09:32.128370+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:27:55.332594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"265'18","reported_seq":111,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.570334+0000","last_change":"2024-05-30T16:12:50.827178+0000","last_active":"2024-05-30T17:11:26.570334+0000","last_peered":"2024-05-30T17:11:26.570334+0000","last_clean":"2024-05-30T17:11:26.570334+0000","last_became_active":"2024-05-30T16:12:50.826880+0000","last_became_peered":"2024-05-30T16:12:50.826880+0000","last_unstale":"2024-05-30T17:11:26.570334+0000","last_undegraded":"2024-05-30T17:11:26.570334+0000","last_fullsized":"2024-05-30T17:11:26.570334+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:22:01.392934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":61,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.735902+0000","last_change":"2024-05-30T16:34:55.886177+0000","last_active":"2024-05-30T17:10:21.735902+0000","last_peered":"2024-05-30T17:10:21.735902+0000","last_clean":"2024-05-30T17:10:21.735902+0000","last_became_active":"2024-05-30T16:34:55.885643+0000","last_became_peered":"2024-05-30T16:34:55.885643+0000","last_unstale":"2024-05-30T17:10:21.735902+0000","last_undegraded":"2024-05-30T17:10:21.735902+0000","last_fullsized":"2024-05-30T17:10:21.735902+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:27:38.430910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"262'2","reported_seq":51,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.726341+0000","last_change":"2024-05-30T16:17:35.248399+0000","last_active":"2024-05-30T17:09:36.726341+0000","last_peered":"2024-05-30T17:09:36.726341+0000","last_clean":"2024-05-30T17:09:36.726341+0000","last_became_active":"2024-05-30T16:17:35.247757+0000","last_became_peered":"2024-05-30T16:17:35.247757+0000","last_unstale":"2024-05-30T17:09:36.726341+0000","last_undegraded":"2024-05-30T17:09:36.726341+0000","last_fullsized":"2024-05-30T17:09:36.726341+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:22:36.927668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"0'0","reported_seq":53,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:31.542620+0000","last_change":"2024-05-30T16:17:31.203988+0000","last_active":"2024-05-30T17:09:31.542620+0000","last_peered":"2024-05-30T17:09:31.542620+0000","last_clean":"2024-05-30T17:09:31.542620+0000","last_became_active":"2024-05-30T16:17:31.203452+0000","last_became_peered":"2024-05-30T16:17:31.203452+0000","last_unstale":"2024-05-30T17:09:31.542620+0000","last_undegraded":"2024-05-30T17:09:31.542620+0000","last_fullsized":"2024-05-30T17:09:31.542620+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:50:29.995893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"265'37","reported_seq":162,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.789790+0000","last_change":"2024-05-30T16:12:50.812560+0000","last_active":"2024-05-30T17:11:11.789790+0000","last_peered":"2024-05-30T17:11:11.789790+0000","last_clean":"2024-05-30T17:11:11.789790+0000","last_became_active":"2024-05-30T16:12:50.811626+0000","last_became_peered":"2024-05-30T16:12:50.811626+0000","last_unstale":"2024-05-30T17:11:11.789790+0000","last_undegraded":"2024-05-30T17:11:11.789790+0000","last_fullsized":"2024-05-30T17:11:11.789790+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:11:40.631744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":184,"num_read_kb":132,"num_write":81,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"264'1","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.539587+0000","last_change":"2024-05-30T16:34:55.862486+0000","last_active":"2024-05-30T17:09:36.539587+0000","last_peered":"2024-05-30T17:09:36.539587+0000","last_clean":"2024-05-30T17:09:36.539587+0000","last_became_active":"2024-05-30T16:34:55.861767+0000","last_became_peered":"2024-05-30T16:34:55.861767+0000","last_unstale":"2024-05-30T17:09:36.539587+0000","last_undegraded":"2024-05-30T17:09:36.539587+0000","last_fullsized":"2024-05-30T17:09:36.539587+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:06:04.837971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"265'7","reported_seq":62,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:41.769918+0000","last_change":"2024-05-30T16:17:35.244195+0000","last_active":"2024-05-30T17:09:41.769918+0000","last_peered":"2024-05-30T17:09:41.769918+0000","last_clean":"2024-05-30T17:09:41.769918+0000","last_became_active":"2024-05-30T16:17:35.243435+0000","last_became_peered":"2024-05-30T16:17:35.243435+0000","last_unstale":"2024-05-30T17:09:41.769918+0000","last_undegraded":"2024-05-30T17:09:41.769918+0000","last_fullsized":"2024-05-30T17:09:41.769918+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:05:13.001812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":13312,"num_write":12,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"265'2","reported_seq":57,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:46.545952+0000","last_change":"2024-05-30T16:17:31.201111+0000","last_active":"2024-05-30T17:09:46.545952+0000","last_peered":"2024-05-30T17:09:46.545952+0000","last_clean":"2024-05-30T17:09:46.545952+0000","last_became_active":"2024-05-30T16:17:31.200663+0000","last_became_peered":"2024-05-30T16:17:31.200663+0000","last_unstale":"2024-05-30T17:09:46.545952+0000","last_undegraded":"2024-05-30T17:09:46.545952+0000","last_fullsized":"2024-05-30T17:09:46.545952+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:38:48.835894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"265'15","reported_seq":103,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:12.150755+0000","last_change":"2024-05-30T16:12:50.814553+0000","last_active":"2024-05-30T17:11:12.150755+0000","last_peered":"2024-05-30T17:11:12.150755+0000","last_clean":"2024-05-30T17:11:12.150755+0000","last_became_active":"2024-05-30T16:12:50.813741+0000","last_became_peered":"2024-05-30T16:12:50.813741+0000","last_unstale":"2024-05-30T17:11:12.150755+0000","last_undegraded":"2024-05-30T17:11:12.150755+0000","last_fullsized":"2024-05-30T17:11:12.150755+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:49:05.503730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":53,"num_write":33,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.560876+0000","last_change":"2024-05-30T16:34:55.859738+0000","last_active":"2024-05-30T17:09:36.560876+0000","last_peered":"2024-05-30T17:09:36.560876+0000","last_clean":"2024-05-30T17:09:36.560876+0000","last_became_active":"2024-05-30T16:34:55.858872+0000","last_became_peered":"2024-05-30T16:34:55.858872+0000","last_unstale":"2024-05-30T17:09:36.560876+0000","last_undegraded":"2024-05-30T17:09:36.560876+0000","last_fullsized":"2024-05-30T17:09:36.560876+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:48:21.475604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"0'0","reported_seq":49,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:42.130510+0000","last_change":"2024-05-30T16:17:35.227683+0000","last_active":"2024-05-30T17:09:42.130510+0000","last_peered":"2024-05-30T17:09:42.130510+0000","last_clean":"2024-05-30T17:09:42.130510+0000","last_became_active":"2024-05-30T16:17:35.226829+0000","last_became_peered":"2024-05-30T16:17:35.226829+0000","last_unstale":"2024-05-30T17:09:42.130510+0000","last_undegraded":"2024-05-30T17:09:42.130510+0000","last_fullsized":"2024-05-30T17:09:42.130510+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:15:44.751214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"0'0","reported_seq":54,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.543735+0000","last_change":"2024-05-30T16:17:31.205159+0000","last_active":"2024-05-30T17:09:36.543735+0000","last_peered":"2024-05-30T17:09:36.543735+0000","last_clean":"2024-05-30T17:09:36.543735+0000","last_became_active":"2024-05-30T16:17:31.203535+0000","last_became_peered":"2024-05-30T16:17:31.203535+0000","last_unstale":"2024-05-30T17:09:36.543735+0000","last_undegraded":"2024-05-30T17:09:36.543735+0000","last_fullsized":"2024-05-30T17:09:36.543735+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:42:29.611842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"265'36","reported_seq":157,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.747034+0000","last_change":"2024-05-30T16:12:50.832592+0000","last_active":"2024-05-30T17:11:11.747034+0000","last_peered":"2024-05-30T17:11:11.747034+0000","last_clean":"2024-05-30T17:11:11.747034+0000","last_became_active":"2024-05-30T16:12:50.831842+0000","last_became_peered":"2024-05-30T16:12:50.831842+0000","last_unstale":"2024-05-30T17:11:11.747034+0000","last_undegraded":"2024-05-30T17:11:11.747034+0000","last_fullsized":"2024-05-30T17:11:11.747034+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:08:18.471042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":177,"num_read_kb":125,"num_write":75,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.570209+0000","last_change":"2024-05-30T16:34:55.863472+0000","last_active":"2024-05-30T17:10:21.570209+0000","last_peered":"2024-05-30T17:10:21.570209+0000","last_clean":"2024-05-30T17:10:21.570209+0000","last_became_active":"2024-05-30T16:34:55.862709+0000","last_became_peered":"2024-05-30T16:34:55.862709+0000","last_unstale":"2024-05-30T17:10:21.570209+0000","last_undegraded":"2024-05-30T17:10:21.570209+0000","last_fullsized":"2024-05-30T17:10:21.570209+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:25:02.100900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"265'2","reported_seq":50,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:41.537534+0000","last_change":"2024-05-30T16:17:35.241271+0000","last_active":"2024-05-30T17:09:41.537534+0000","last_peered":"2024-05-30T17:09:41.537534+0000","last_clean":"2024-05-30T17:09:41.537534+0000","last_became_active":"2024-05-30T16:17:35.240472+0000","last_became_peered":"2024-05-30T16:17:35.240472+0000","last_unstale":"2024-05-30T17:09:41.537534+0000","last_undegraded":"2024-05-30T17:09:41.537534+0000","last_fullsized":"2024-05-30T17:09:41.537534+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:59:40.865645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"265'4","reported_seq":62,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.769181+0000","last_change":"2024-05-30T16:17:31.196715+0000","last_active":"2024-05-30T17:09:36.769181+0000","last_peered":"2024-05-30T17:09:36.769181+0000","last_clean":"2024-05-30T17:09:36.769181+0000","last_became_active":"2024-05-30T16:17:31.187958+0000","last_became_peered":"2024-05-30T16:17:31.187958+0000","last_unstale":"2024-05-30T17:09:36.769181+0000","last_undegraded":"2024-05-30T17:09:36.769181+0000","last_fullsized":"2024-05-30T17:09:36.769181+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:26:54.324514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485760,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":10240,"num_write":14,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"265'18","reported_seq":112,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.557030+0000","last_change":"2024-05-30T16:12:50.823849+0000","last_active":"2024-05-30T17:11:11.557030+0000","last_peered":"2024-05-30T17:11:11.557030+0000","last_clean":"2024-05-30T17:11:11.557030+0000","last_became_active":"2024-05-30T16:12:50.823090+0000","last_became_peered":"2024-05-30T16:12:50.823090+0000","last_unstale":"2024-05-30T17:11:11.557030+0000","last_undegraded":"2024-05-30T17:11:11.557030+0000","last_fullsized":"2024-05-30T17:11:11.557030+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:38:28.480174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":69,"num_write":35,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:31.556926+0000","last_change":"2024-05-30T16:34:55.891440+0000","last_active":"2024-05-30T17:10:31.556926+0000","last_peered":"2024-05-30T17:10:31.556926+0000","last_clean":"2024-05-30T17:10:31.556926+0000","last_became_active":"2024-05-30T16:34:55.890688+0000","last_became_peered":"2024-05-30T16:34:55.890688+0000","last_unstale":"2024-05-30T17:10:31.556926+0000","last_undegraded":"2024-05-30T17:10:31.556926+0000","last_fullsized":"2024-05-30T17:10:31.556926+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:05:55.238651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"265'7","reported_seq":65,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:41.727328+0000","last_change":"2024-05-30T16:17:35.238582+0000","last_active":"2024-05-30T17:09:41.727328+0000","last_peered":"2024-05-30T17:09:41.727328+0000","last_clean":"2024-05-30T17:09:41.727328+0000","last_became_active":"2024-05-30T16:17:35.237031+0000","last_became_peered":"2024-05-30T16:17:35.237031+0000","last_unstale":"2024-05-30T17:09:41.727328+0000","last_undegraded":"2024-05-30T17:09:41.727328+0000","last_fullsized":"2024-05-30T17:09:41.727328+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:11:17.043035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485760,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":16,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"262'5","reported_seq":65,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.536379+0000","last_change":"2024-05-30T16:17:31.204721+0000","last_active":"2024-05-30T17:09:36.536379+0000","last_peered":"2024-05-30T17:09:36.536379+0000","last_clean":"2024-05-30T17:09:36.536379+0000","last_became_active":"2024-05-30T16:17:31.203871+0000","last_became_peered":"2024-05-30T16:17:31.203871+0000","last_unstale":"2024-05-30T17:09:36.536379+0000","last_undegraded":"2024-05-30T17:09:36.536379+0000","last_fullsized":"2024-05-30T17:09:36.536379+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:41:32.481769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":4098,"num_write":9,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"265'24","reported_seq":133,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:06.565051+0000","last_change":"2024-05-30T16:12:50.827488+0000","last_active":"2024-05-30T17:11:06.565051+0000","last_peered":"2024-05-30T17:11:06.565051+0000","last_clean":"2024-05-30T17:11:06.565051+0000","last_became_active":"2024-05-30T16:12:50.827078+0000","last_became_peered":"2024-05-30T16:12:50.827078+0000","last_unstale":"2024-05-30T17:11:06.565051+0000","last_undegraded":"2024-05-30T17:11:06.565051+0000","last_fullsized":"2024-05-30T17:11:06.565051+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:30:01.069671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":93,"num_write":46,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"265'165","reported_seq":776,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.569071+0000","last_change":"2024-05-30T15:54:16.662201+0000","last_active":"2024-05-30T17:11:21.569071+0000","last_peered":"2024-05-30T17:11:21.569071+0000","last_clean":"2024-05-30T17:11:21.569071+0000","last_became_active":"2024-05-30T15:54:16.661334+0000","last_became_peered":"2024-05-30T15:54:16.661334+0000","last_unstale":"2024-05-30T17:11:21.569071+0000","last_undegraded":"2024-05-30T17:11:21.569071+0000","last_fullsized":"2024-05-30T17:11:21.569071+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":100,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:14:32.854488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2159792,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":347,"num_read_kb":7638,"num_write":1029,"num_write_kb":24692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"265'436","reported_seq":2069,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:33.884897+0000","last_change":"2024-05-30T15:54:18.669748+0000","last_active":"2024-05-30T17:12:33.884897+0000","last_peered":"2024-05-30T17:12:33.884897+0000","last_clean":"2024-05-30T17:12:33.884897+0000","last_became_active":"2024-05-30T15:54:18.669016+0000","last_became_peered":"2024-05-30T15:54:18.669016+0000","last_unstale":"2024-05-30T17:12:33.884897+0000","last_undegraded":"2024-05-30T17:12:33.884897+0000","last_fullsized":"2024-05-30T17:12:33.884897+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":400,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:26:42.111749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2479,"num_read_kb":2366,"num_write":506,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"265'232","reported_seq":887,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.792186+0000","last_change":"2024-05-30T15:54:16.653102+0000","last_active":"2024-05-30T17:11:21.792186+0000","last_peered":"2024-05-30T17:11:21.792186+0000","last_clean":"2024-05-30T17:11:21.792186+0000","last_became_active":"2024-05-30T15:54:16.652724+0000","last_became_peered":"2024-05-30T15:54:16.652724+0000","last_unstale":"2024-05-30T17:11:21.792186+0000","last_undegraded":"2024-05-30T17:11:21.792186+0000","last_fullsized":"2024-05-30T17:11:21.792186+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":200,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:48:02.488417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16999710,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":552,"num_read_kb":8631,"num_write":1279,"num_write_kb":24636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"265'758","reported_seq":2113,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:59.057740+0000","last_change":"2024-05-30T15:54:18.649460+0000","last_active":"2024-05-30T17:11:59.057740+0000","last_peered":"2024-05-30T17:11:59.057740+0000","last_clean":"2024-05-30T17:11:59.057740+0000","last_became_active":"2024-05-30T15:54:18.648675+0000","last_became_peered":"2024-05-30T15:54:18.648675+0000","last_unstale":"2024-05-30T17:11:59.057740+0000","last_undegraded":"2024-05-30T17:11:59.057740+0000","last_fullsized":"2024-05-30T17:11:59.057740+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":700,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:30:51.616348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3292,"num_read_kb":2905,"num_write":1031,"num_write_kb":415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"265'200","reported_seq":832,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:57.487564+0000","last_change":"2024-05-30T15:54:16.633538+0000","last_active":"2024-05-30T17:11:57.487564+0000","last_peered":"2024-05-30T17:11:57.487564+0000","last_clean":"2024-05-30T17:11:57.487564+0000","last_became_active":"2024-05-30T15:54:16.633011+0000","last_became_peered":"2024-05-30T15:54:16.633011+0000","last_unstale":"2024-05-30T17:11:57.487564+0000","last_undegraded":"2024-05-30T17:11:57.487564+0000","last_fullsized":"2024-05-30T17:11:57.487564+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":100,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:22:33.463154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17125665,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":396,"num_read_kb":43854,"num_write":995,"num_write_kb":74051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"265'746","reported_seq":2144,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:33.888418+0000","last_change":"2024-05-30T15:54:18.651676+0000","last_active":"2024-05-30T17:12:33.888418+0000","last_peered":"2024-05-30T17:12:33.888418+0000","last_clean":"2024-05-30T17:12:33.888418+0000","last_became_active":"2024-05-30T15:54:18.649608+0000","last_became_peered":"2024-05-30T15:54:18.649608+0000","last_unstale":"2024-05-30T17:12:33.888418+0000","last_undegraded":"2024-05-30T17:12:33.888418+0000","last_fullsized":"2024-05-30T17:12:33.888418+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":700,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:29:56.699785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3275,"num_read_kb":2909,"num_write":996,"num_write_kb":378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"265'152","reported_seq":790,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:44.320980+0000","last_change":"2024-05-30T15:54:16.653236+0000","last_active":"2024-05-30T17:11:44.320980+0000","last_peered":"2024-05-30T17:11:44.320980+0000","last_clean":"2024-05-30T17:11:44.320980+0000","last_became_active":"2024-05-30T15:54:16.652315+0000","last_became_peered":"2024-05-30T15:54:16.652315+0000","last_unstale":"2024-05-30T17:11:44.320980+0000","last_undegraded":"2024-05-30T17:11:44.320980+0000","last_fullsized":"2024-05-30T17:11:44.320980+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":100,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:26:57.491124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1336636,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":435,"num_read_kb":3484,"num_write":909,"num_write_kb":6490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"265'487","reported_seq":1962,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:52.698536+0000","last_change":"2024-05-30T15:54:18.652741+0000","last_active":"2024-05-30T17:12:52.698536+0000","last_peered":"2024-05-30T17:12:52.698536+0000","last_clean":"2024-05-30T17:12:52.698536+0000","last_became_active":"2024-05-30T15:54:18.652451+0000","last_became_peered":"2024-05-30T15:54:18.652451+0000","last_unstale":"2024-05-30T17:12:52.698536+0000","last_undegraded":"2024-05-30T17:12:52.698536+0000","last_fullsized":"2024-05-30T17:12:52.698536+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":400,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:50:10.811052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2387,"num_read_kb":2188,"num_write":616,"num_write_kb":168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"265'242","reported_seq":956,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:58.054740+0000","last_change":"2024-05-30T15:54:16.659161+0000","last_active":"2024-05-30T17:12:58.054740+0000","last_peered":"2024-05-30T17:12:58.054740+0000","last_clean":"2024-05-30T17:12:58.054740+0000","last_became_active":"2024-05-30T15:54:16.658338+0000","last_became_peered":"2024-05-30T15:54:16.658338+0000","last_unstale":"2024-05-30T17:12:58.054740+0000","last_undegraded":"2024-05-30T17:12:58.054740+0000","last_fullsized":"2024-05-30T17:12:58.054740+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":200,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:10:31.510086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11588767,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":827,"num_read_kb":24681,"num_write":1243,"num_write_kb":34031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"265'370","reported_seq":1862,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:12.349707+0000","last_change":"2024-05-30T15:54:18.669822+0000","last_active":"2024-05-30T17:11:12.349707+0000","last_peered":"2024-05-30T17:11:12.349707+0000","last_clean":"2024-05-30T17:11:12.349707+0000","last_became_active":"2024-05-30T15:54:18.669017+0000","last_became_peered":"2024-05-30T15:54:18.669017+0000","last_unstale":"2024-05-30T17:11:12.349707+0000","last_undegraded":"2024-05-30T17:11:12.349707+0000","last_fullsized":"2024-05-30T17:11:12.349707+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":300,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:33:40.482356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1986,"num_read_kb":1906,"num_write":419,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"265'192","reported_seq":817,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:17.124503+0000","last_change":"2024-05-30T15:54:16.673757+0000","last_active":"2024-05-30T17:09:17.124503+0000","last_peered":"2024-05-30T17:09:17.124503+0000","last_clean":"2024-05-30T17:09:17.124503+0000","last_became_active":"2024-05-30T15:54:16.671120+0000","last_became_peered":"2024-05-30T15:54:16.671120+0000","last_unstale":"2024-05-30T17:09:17.124503+0000","last_undegraded":"2024-05-30T17:09:17.124503+0000","last_fullsized":"2024-05-30T17:09:17.124503+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":100,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:07:40.834702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5596650,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":434,"num_read_kb":4718,"num_write":1097,"num_write_kb":20583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"265'725","reported_seq":2201,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.709270+0000","last_change":"2024-05-30T15:54:18.651587+0000","last_active":"2024-05-30T17:13:20.709270+0000","last_peered":"2024-05-30T17:13:20.709270+0000","last_clean":"2024-05-30T17:13:20.709270+0000","last_became_active":"2024-05-30T15:54:18.650543+0000","last_became_peered":"2024-05-30T15:54:18.650543+0000","last_unstale":"2024-05-30T17:13:20.709270+0000","last_undegraded":"2024-05-30T17:13:20.709270+0000","last_fullsized":"2024-05-30T17:13:20.709270+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":700,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:51.313043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3282,"num_read_kb":2952,"num_write":973,"num_write_kb":356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"265'188","reported_seq":828,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:18.810687+0000","last_change":"2024-05-30T15:54:16.679393+0000","last_active":"2024-05-30T17:12:18.810687+0000","last_peered":"2024-05-30T17:12:18.810687+0000","last_clean":"2024-05-30T17:12:18.810687+0000","last_became_active":"2024-05-30T15:54:16.678799+0000","last_became_peered":"2024-05-30T15:54:16.678799+0000","last_unstale":"2024-05-30T17:12:18.810687+0000","last_undegraded":"2024-05-30T17:12:18.810687+0000","last_fullsized":"2024-05-30T17:12:18.810687+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":100,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:07:54.614137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5740256,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":463,"num_read_kb":33008,"num_write":1086,"num_write_kb":27021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"265'850","reported_seq":2539,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.703253+0000","last_change":"2024-05-30T15:54:18.657813+0000","last_active":"2024-05-30T17:13:20.703253+0000","last_peered":"2024-05-30T17:13:20.703253+0000","last_clean":"2024-05-30T17:13:20.703253+0000","last_became_active":"2024-05-30T15:54:18.656475+0000","last_became_peered":"2024-05-30T15:54:18.656475+0000","last_unstale":"2024-05-30T17:13:20.703253+0000","last_undegraded":"2024-05-30T17:13:20.703253+0000","last_fullsized":"2024-05-30T17:13:20.703253+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":800,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:08:46.768265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4088,"num_read_kb":3692,"num_write":1166,"num_write_kb":472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"265'139","reported_seq":735,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:16.558759+0000","last_change":"2024-05-30T15:54:16.669564+0000","last_active":"2024-05-30T17:09:16.558759+0000","last_peered":"2024-05-30T17:09:16.558759+0000","last_clean":"2024-05-30T17:09:16.558759+0000","last_became_active":"2024-05-30T15:54:16.669095+0000","last_became_peered":"2024-05-30T15:54:16.669095+0000","last_unstale":"2024-05-30T17:09:16.558759+0000","last_undegraded":"2024-05-30T17:09:16.558759+0000","last_fullsized":"2024-05-30T17:09:16.558759+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":100,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:34:08.005236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485923,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":302,"num_read_kb":17751,"num_write":816,"num_write_kb":30378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"265'369","reported_seq":1790,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.560600+0000","last_change":"2024-05-30T15:54:18.653529+0000","last_active":"2024-05-30T17:11:26.560600+0000","last_peered":"2024-05-30T17:11:26.560600+0000","last_clean":"2024-05-30T17:11:26.560600+0000","last_became_active":"2024-05-30T15:54:18.653189+0000","last_became_peered":"2024-05-30T15:54:18.653189+0000","last_unstale":"2024-05-30T17:11:26.560600+0000","last_undegraded":"2024-05-30T17:11:26.560600+0000","last_fullsized":"2024-05-30T17:11:26.560600+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":300,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:18:04.600068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1969,"num_read_kb":1855,"num_write":435,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"265'336","reported_seq":1764,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:33.888003+0000","last_change":"2024-05-30T15:54:18.652246+0000","last_active":"2024-05-30T17:12:33.888003+0000","last_peered":"2024-05-30T17:12:33.888003+0000","last_clean":"2024-05-30T17:12:33.888003+0000","last_became_active":"2024-05-30T15:54:18.649994+0000","last_became_peered":"2024-05-30T15:54:18.649994+0000","last_unstale":"2024-05-30T17:12:33.888003+0000","last_undegraded":"2024-05-30T17:12:33.888003+0000","last_fullsized":"2024-05-30T17:12:33.888003+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:39:30.273131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1976,"num_read_kb":1893,"num_write":388,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"265'206","reported_seq":871,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:11.306928+0000","last_change":"2024-05-30T15:54:16.650771+0000","last_active":"2024-05-30T17:12:11.306928+0000","last_peered":"2024-05-30T17:12:11.306928+0000","last_clean":"2024-05-30T17:12:11.306928+0000","last_became_active":"2024-05-30T15:54:16.649872+0000","last_became_peered":"2024-05-30T15:54:16.649872+0000","last_unstale":"2024-05-30T17:12:11.306928+0000","last_undegraded":"2024-05-30T17:12:11.306928+0000","last_fullsized":"2024-05-30T17:12:11.306928+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":100,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:01:29.595780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16404427,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":504,"num_read_kb":59018,"num_write":1160,"num_write_kb":23092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"265'1525","reported_seq":3130,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:28.258461+0000","last_change":"2024-05-30T15:54:18.649618+0000","last_active":"2024-05-30T17:12:28.258461+0000","last_peered":"2024-05-30T17:12:28.258461+0000","last_clean":"2024-05-30T17:12:28.258461+0000","last_became_active":"2024-05-30T15:54:18.648860+0000","last_became_peered":"2024-05-30T15:54:18.648860+0000","last_unstale":"2024-05-30T17:12:28.258461+0000","last_undegraded":"2024-05-30T17:12:28.258461+0000","last_fullsized":"2024-05-30T17:12:28.258461+0000","mapping_epoch":24,"log_start":"265'1500","ondisk_log_start":"265'1500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":1500,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:09:51.963499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6132,"num_read_kb":5297,"num_write":2138,"num_write_kb":912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"265'218","reported_seq":1080,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:28.769161+0000","last_change":"2024-05-30T15:54:16.687260+0000","last_active":"2024-05-30T17:12:28.769161+0000","last_peered":"2024-05-30T17:12:28.769161+0000","last_clean":"2024-05-30T17:12:28.769161+0000","last_became_active":"2024-05-30T15:54:16.686847+0000","last_became_peered":"2024-05-30T15:54:16.686847+0000","last_unstale":"2024-05-30T17:12:28.769161+0000","last_undegraded":"2024-05-30T17:12:28.769161+0000","last_fullsized":"2024-05-30T17:12:28.769161+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":200,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:27:56.085645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2781302,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":922,"num_read_kb":24537,"num_write":1230,"num_write_kb":54861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"265'1164","reported_seq":3003,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:10.547968+0000","last_change":"2024-05-30T15:54:18.666133+0000","last_active":"2024-05-30T17:13:10.547968+0000","last_peered":"2024-05-30T17:13:10.547968+0000","last_clean":"2024-05-30T17:13:10.547968+0000","last_became_active":"2024-05-30T15:54:18.665302+0000","last_became_peered":"2024-05-30T15:54:18.665302+0000","last_unstale":"2024-05-30T17:13:10.547968+0000","last_undegraded":"2024-05-30T17:13:10.547968+0000","last_fullsized":"2024-05-30T17:13:10.547968+0000","mapping_epoch":24,"log_start":"265'1100","ondisk_log_start":"265'1100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":1100,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:19:10.742000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5292,"num_read_kb":4714,"num_write":1593,"num_write_kb":642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"265'194","reported_seq":915,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:16.092608+0000","last_change":"2024-05-30T15:54:16.669114+0000","last_active":"2024-05-30T17:09:16.092608+0000","last_peered":"2024-05-30T17:09:16.092608+0000","last_clean":"2024-05-30T17:09:16.092608+0000","last_became_active":"2024-05-30T15:54:16.668438+0000","last_became_peered":"2024-05-30T15:54:16.668438+0000","last_unstale":"2024-05-30T17:09:16.092608+0000","last_undegraded":"2024-05-30T17:09:16.092608+0000","last_fullsized":"2024-05-30T17:09:16.092608+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":100,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:55:42.428949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10487886,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":915,"num_read_kb":8276,"num_write":1170,"num_write_kb":35310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"265'768","reported_seq":2282,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.649495+0000","last_change":"2024-05-30T15:54:18.651494+0000","last_active":"2024-05-30T17:13:20.649495+0000","last_peered":"2024-05-30T17:13:20.649495+0000","last_clean":"2024-05-30T17:13:20.649495+0000","last_became_active":"2024-05-30T15:54:18.651197+0000","last_became_peered":"2024-05-30T15:54:18.651197+0000","last_unstale":"2024-05-30T17:13:20.649495+0000","last_undegraded":"2024-05-30T17:13:20.649495+0000","last_fullsized":"2024-05-30T17:13:20.649495+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":700,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:01:31.985221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3544,"num_read_kb":3171,"num_write":1051,"num_write_kb":404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"265'181","reported_seq":833,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:11.720441+0000","last_change":"2024-05-30T15:54:16.677382+0000","last_active":"2024-05-30T17:09:11.720441+0000","last_peered":"2024-05-30T17:09:11.720441+0000","last_clean":"2024-05-30T17:09:11.720441+0000","last_became_active":"2024-05-30T15:54:16.676703+0000","last_became_peered":"2024-05-30T15:54:16.676703+0000","last_unstale":"2024-05-30T17:09:11.720441+0000","last_undegraded":"2024-05-30T17:09:11.720441+0000","last_fullsized":"2024-05-30T17:09:11.720441+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":100,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:39:55.585070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16123359,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":357,"num_read_kb":73313,"num_write":1053,"num_write_kb":47694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"265'372","reported_seq":1824,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.655751+0000","last_change":"2024-05-30T15:54:18.650530+0000","last_active":"2024-05-30T17:12:44.655751+0000","last_peered":"2024-05-30T17:12:44.655751+0000","last_clean":"2024-05-30T17:12:44.655751+0000","last_became_active":"2024-05-30T15:54:18.650255+0000","last_became_peered":"2024-05-30T15:54:18.650255+0000","last_unstale":"2024-05-30T17:12:44.655751+0000","last_undegraded":"2024-05-30T17:12:44.655751+0000","last_fullsized":"2024-05-30T17:12:44.655751+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":300,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:22:55.711498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1969,"num_read_kb":1871,"num_write":435,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"265'200","reported_seq":851,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:11.557730+0000","last_change":"2024-05-30T15:54:16.665743+0000","last_active":"2024-05-30T17:09:11.557730+0000","last_peered":"2024-05-30T17:09:11.557730+0000","last_clean":"2024-05-30T17:09:11.557730+0000","last_became_active":"2024-05-30T15:54:16.665344+0000","last_became_peered":"2024-05-30T15:54:16.665344+0000","last_unstale":"2024-05-30T17:09:11.557730+0000","last_undegraded":"2024-05-30T17:09:11.557730+0000","last_fullsized":"2024-05-30T17:09:11.557730+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":100,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:11:01.132558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17335132,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":474,"num_read_kb":7750,"num_write":1074,"num_write_kb":36544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"265'375","reported_seq":1800,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:15.191377+0000","last_change":"2024-05-30T15:54:18.645251+0000","last_active":"2024-05-30T17:12:15.191377+0000","last_peered":"2024-05-30T17:12:15.191377+0000","last_clean":"2024-05-30T17:12:15.191377+0000","last_became_active":"2024-05-30T15:54:18.643825+0000","last_became_peered":"2024-05-30T15:54:18.643825+0000","last_unstale":"2024-05-30T17:12:15.191377+0000","last_undegraded":"2024-05-30T17:12:15.191377+0000","last_fullsized":"2024-05-30T17:12:15.191377+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":300,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:54:45.495400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1936,"num_read_kb":1833,"num_write":434,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"265'234","reported_seq":905,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:16.555742+0000","last_change":"2024-05-30T15:54:16.638560+0000","last_active":"2024-05-30T17:09:16.555742+0000","last_peered":"2024-05-30T17:09:16.555742+0000","last_clean":"2024-05-30T17:09:16.555742+0000","last_became_active":"2024-05-30T15:54:16.637928+0000","last_became_peered":"2024-05-30T15:54:16.637928+0000","last_unstale":"2024-05-30T17:09:16.555742+0000","last_undegraded":"2024-05-30T17:09:16.555742+0000","last_fullsized":"2024-05-30T17:09:16.555742+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":200,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:28:17.148350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11620077,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":522,"num_read_kb":65823,"num_write":1177,"num_write_kb":32044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"265'176","reported_seq":700,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.999380+0000","last_change":"2024-05-30T15:54:26.701638+0000","last_active":"2024-05-30T17:13:20.999380+0000","last_peered":"2024-05-30T17:13:20.999380+0000","last_clean":"2024-05-30T17:13:20.999380+0000","last_became_active":"2024-05-30T15:54:26.700957+0000","last_became_peered":"2024-05-30T15:54:26.700957+0000","last_unstale":"2024-05-30T17:13:20.999380+0000","last_undegraded":"2024-05-30T17:13:20.999380+0000","last_fullsized":"2024-05-30T17:13:20.999380+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":100,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:45:03.540847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":270,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":132,"num_read_kb":126,"num_write":386,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"265'494","reported_seq":2021,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.600563+0000","last_change":"2024-05-30T15:54:18.657002+0000","last_active":"2024-05-30T17:12:44.600563+0000","last_peered":"2024-05-30T17:12:44.600563+0000","last_clean":"2024-05-30T17:12:44.600563+0000","last_became_active":"2024-05-30T15:54:18.652712+0000","last_became_peered":"2024-05-30T15:54:18.652712+0000","last_unstale":"2024-05-30T17:12:44.600563+0000","last_undegraded":"2024-05-30T17:12:44.600563+0000","last_fullsized":"2024-05-30T17:12:44.600563+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":400,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:27:30.279867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2579,"num_read_kb":2377,"num_write":668,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"265'189","reported_seq":793,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:36.550925+0000","last_change":"2024-05-30T15:55:54.564987+0000","last_active":"2024-05-30T17:10:36.550925+0000","last_peered":"2024-05-30T17:10:36.550925+0000","last_clean":"2024-05-30T17:10:36.550925+0000","last_became_active":"2024-05-30T15:55:54.563874+0000","last_became_peered":"2024-05-30T15:55:54.563874+0000","last_unstale":"2024-05-30T17:10:36.550925+0000","last_undegraded":"2024-05-30T17:10:36.550925+0000","last_fullsized":"2024-05-30T17:10:36.550925+0000","mapping_epoch":37,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:22:56.400029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079909,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":469,"num_read_kb":1589,"num_write":1088,"num_write_kb":27979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:17.124719+0000","last_change":"2024-05-30T15:54:20.659788+0000","last_active":"2024-05-30T17:09:17.124719+0000","last_peered":"2024-05-30T17:09:17.124719+0000","last_clean":"2024-05-30T17:09:17.124719+0000","last_became_active":"2024-05-30T15:54:20.658937+0000","last_became_peered":"2024-05-30T15:54:20.658937+0000","last_unstale":"2024-05-30T17:09:17.124719+0000","last_undegraded":"2024-05-30T17:09:17.124719+0000","last_fullsized":"2024-05-30T17:09:17.124719+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:41:24.097539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"265'198","reported_seq":820,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:46.590604+0000","last_change":"2024-05-30T15:54:22.668221+0000","last_active":"2024-05-30T17:11:46.590604+0000","last_peered":"2024-05-30T17:11:46.590604+0000","last_clean":"2024-05-30T17:11:46.590604+0000","last_became_active":"2024-05-30T15:54:22.667496+0000","last_became_peered":"2024-05-30T15:54:22.667496+0000","last_unstale":"2024-05-30T17:11:46.590604+0000","last_undegraded":"2024-05-30T17:11:46.590604+0000","last_fullsized":"2024-05-30T17:11:46.590604+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":100,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:54:14.364376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9413,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":276,"num_read_kb":334,"num_write":164,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":498,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.754793+0000","last_change":"2024-05-30T15:54:24.675232+0000","last_active":"2024-05-30T17:09:21.754793+0000","last_peered":"2024-05-30T17:09:21.754793+0000","last_clean":"2024-05-30T17:09:21.754793+0000","last_became_active":"2024-05-30T15:54:24.674491+0000","last_became_peered":"2024-05-30T15:54:24.674491+0000","last_unstale":"2024-05-30T17:09:21.754793+0000","last_undegraded":"2024-05-30T17:09:21.754793+0000","last_fullsized":"2024-05-30T17:09:21.754793+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:30:36.269404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"265'194","reported_seq":769,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.643659+0000","last_change":"2024-05-30T15:54:26.687210+0000","last_active":"2024-05-30T17:12:44.643659+0000","last_peered":"2024-05-30T17:12:44.643659+0000","last_clean":"2024-05-30T17:12:44.643659+0000","last_became_active":"2024-05-30T15:54:26.686430+0000","last_became_peered":"2024-05-30T15:54:26.686430+0000","last_unstale":"2024-05-30T17:12:44.643659+0000","last_undegraded":"2024-05-30T17:12:44.643659+0000","last_fullsized":"2024-05-30T17:12:44.643659+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":100,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:45:58.373725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":824,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":238,"num_read_kb":223,"num_write":274,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"265'510","reported_seq":2160,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.681680+0000","last_change":"2024-05-30T15:54:18.662470+0000","last_active":"2024-05-30T17:13:18.681680+0000","last_peered":"2024-05-30T17:13:18.681680+0000","last_clean":"2024-05-30T17:13:18.681680+0000","last_became_active":"2024-05-30T15:54:18.661762+0000","last_became_peered":"2024-05-30T15:54:18.661762+0000","last_unstale":"2024-05-30T17:13:18.681680+0000","last_undegraded":"2024-05-30T17:13:18.681680+0000","last_fullsized":"2024-05-30T17:13:18.681680+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":400,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:39:51.492340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2657,"num_read_kb":2481,"num_write":627,"num_write_kb":155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"265'204","reported_seq":860,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.582351+0000","last_change":"2024-05-30T15:54:16.639898+0000","last_active":"2024-05-30T17:11:16.582351+0000","last_peered":"2024-05-30T17:11:16.582351+0000","last_clean":"2024-05-30T17:11:16.582351+0000","last_became_active":"2024-05-30T15:54:16.639284+0000","last_became_peered":"2024-05-30T15:54:16.639284+0000","last_unstale":"2024-05-30T17:11:16.582351+0000","last_undegraded":"2024-05-30T17:11:16.582351+0000","last_fullsized":"2024-05-30T17:11:16.582351+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":100,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:33:35.045719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13950104,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":439,"num_read_kb":15653,"num_write":1153,"num_write_kb":64581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":509,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.560530+0000","last_change":"2024-05-30T15:54:20.666014+0000","last_active":"2024-05-30T17:09:21.560530+0000","last_peered":"2024-05-30T17:09:21.560530+0000","last_clean":"2024-05-30T17:09:21.560530+0000","last_became_active":"2024-05-30T15:54:20.665488+0000","last_became_peered":"2024-05-30T15:54:20.665488+0000","last_unstale":"2024-05-30T17:09:21.560530+0000","last_undegraded":"2024-05-30T17:09:21.560530+0000","last_fullsized":"2024-05-30T17:09:21.560530+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:23:46.612023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"265'102","reported_seq":653,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.723144+0000","last_change":"2024-05-30T15:54:22.666790+0000","last_active":"2024-05-30T17:09:21.723144+0000","last_peered":"2024-05-30T17:09:21.723144+0000","last_clean":"2024-05-30T17:09:21.723144+0000","last_became_active":"2024-05-30T15:54:22.666057+0000","last_became_peered":"2024-05-30T15:54:22.666057+0000","last_unstale":"2024-05-30T17:09:21.723144+0000","last_undegraded":"2024-05-30T17:09:21.723144+0000","last_fullsized":"2024-05-30T17:09:21.723144+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:49:29.070169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":113,"num_write":76,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":507,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.723119+0000","last_change":"2024-05-30T15:54:24.687346+0000","last_active":"2024-05-30T17:09:21.723119+0000","last_peered":"2024-05-30T17:09:21.723119+0000","last_clean":"2024-05-30T17:09:21.723119+0000","last_became_active":"2024-05-30T15:54:24.686595+0000","last_became_peered":"2024-05-30T15:54:24.686595+0000","last_unstale":"2024-05-30T17:09:21.723119+0000","last_undegraded":"2024-05-30T17:09:21.723119+0000","last_fullsized":"2024-05-30T17:09:21.723119+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:49:42.604820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"265'99","reported_seq":612,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.102373+0000","last_change":"2024-05-30T15:54:26.689306+0000","last_active":"2024-05-30T17:13:18.102373+0000","last_peered":"2024-05-30T17:13:18.102373+0000","last_clean":"2024-05-30T17:13:18.102373+0000","last_became_active":"2024-05-30T15:54:26.688376+0000","last_became_peered":"2024-05-30T15:54:26.688376+0000","last_unstale":"2024-05-30T17:13:18.102373+0000","last_undegraded":"2024-05-30T17:13:18.102373+0000","last_fullsized":"2024-05-30T17:13:18.102373+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:10:30.878895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":271,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":103,"num_read_kb":102,"num_write":235,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"265'462","reported_seq":2011,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.638007+0000","last_change":"2024-05-30T15:54:18.658772+0000","last_active":"2024-05-30T17:13:20.638007+0000","last_peered":"2024-05-30T17:13:20.638007+0000","last_clean":"2024-05-30T17:13:20.638007+0000","last_became_active":"2024-05-30T15:54:18.657540+0000","last_became_peered":"2024-05-30T15:54:18.657540+0000","last_unstale":"2024-05-30T17:13:20.638007+0000","last_undegraded":"2024-05-30T17:13:20.638007+0000","last_fullsized":"2024-05-30T17:13:20.638007+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":400,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:48:10.156295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2554,"num_read_kb":2362,"num_write":592,"num_write_kb":196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"265'212","reported_seq":873,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:58.050061+0000","last_change":"2024-05-30T15:54:16.679183+0000","last_active":"2024-05-30T17:12:58.050061+0000","last_peered":"2024-05-30T17:12:58.050061+0000","last_clean":"2024-05-30T17:12:58.050061+0000","last_became_active":"2024-05-30T15:54:16.678659+0000","last_became_peered":"2024-05-30T15:54:16.678659+0000","last_unstale":"2024-05-30T17:12:58.050061+0000","last_undegraded":"2024-05-30T17:12:58.050061+0000","last_fullsized":"2024-05-30T17:12:58.050061+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:12:32.311283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11885555,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":520,"num_read_kb":29194,"num_write":1285,"num_write_kb":17953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":508,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:16.559045+0000","last_change":"2024-05-30T15:54:20.654060+0000","last_active":"2024-05-30T17:09:16.559045+0000","last_peered":"2024-05-30T17:09:16.559045+0000","last_clean":"2024-05-30T17:09:16.559045+0000","last_became_active":"2024-05-30T15:54:20.653316+0000","last_became_peered":"2024-05-30T15:54:20.653316+0000","last_unstale":"2024-05-30T17:09:16.559045+0000","last_undegraded":"2024-05-30T17:09:16.559045+0000","last_fullsized":"2024-05-30T17:09:16.559045+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:11:29.893912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"265'149","reported_seq":711,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:26.537490+0000","last_change":"2024-05-30T15:54:22.665082+0000","last_active":"2024-05-30T17:09:26.537490+0000","last_peered":"2024-05-30T17:09:26.537490+0000","last_clean":"2024-05-30T17:09:26.537490+0000","last_became_active":"2024-05-30T15:54:22.664353+0000","last_became_peered":"2024-05-30T15:54:22.664353+0000","last_unstale":"2024-05-30T17:09:26.537490+0000","last_undegraded":"2024-05-30T17:09:26.537490+0000","last_fullsized":"2024-05-30T17:09:26.537490+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:15:14.584034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":173,"num_read_kb":165,"num_write":120,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"265'527","reported_seq":2477,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.887321+0000","last_change":"2024-05-30T15:54:24.682250+0000","last_active":"2024-05-30T17:13:24.887321+0000","last_peered":"2024-05-30T17:13:24.887321+0000","last_clean":"2024-05-30T17:13:24.887321+0000","last_became_active":"2024-05-30T15:54:24.681503+0000","last_became_peered":"2024-05-30T15:54:24.681503+0000","last_unstale":"2024-05-30T17:13:24.887321+0000","last_undegraded":"2024-05-30T17:13:24.887321+0000","last_fullsized":"2024-05-30T17:13:24.887321+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":500,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:37:06.904193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"265'233","reported_seq":1074,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:19.091217+0000","last_change":"2024-05-30T15:54:26.671113+0000","last_active":"2024-05-30T17:13:19.091217+0000","last_peered":"2024-05-30T17:13:19.091217+0000","last_clean":"2024-05-30T17:13:19.091217+0000","last_became_active":"2024-05-30T15:54:26.670257+0000","last_became_peered":"2024-05-30T15:54:26.670257+0000","last_unstale":"2024-05-30T17:13:19.091217+0000","last_undegraded":"2024-05-30T17:13:19.091217+0000","last_fullsized":"2024-05-30T17:13:19.091217+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":200,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:56:52.779180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":661,"num_read_kb":640,"num_write":540,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"265'406","reported_seq":1988,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.698006+0000","last_change":"2024-05-30T15:54:18.660971+0000","last_active":"2024-05-30T17:13:18.698006+0000","last_peered":"2024-05-30T17:13:18.698006+0000","last_clean":"2024-05-30T17:13:18.698006+0000","last_became_active":"2024-05-30T15:54:18.660656+0000","last_became_peered":"2024-05-30T15:54:18.660656+0000","last_unstale":"2024-05-30T17:13:18.698006+0000","last_undegraded":"2024-05-30T17:13:18.698006+0000","last_fullsized":"2024-05-30T17:13:18.698006+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":300,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:13:06.838766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2368,"num_read_kb":2240,"num_write":473,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"265'190","reported_seq":819,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.558239+0000","last_change":"2024-05-30T15:54:16.668687+0000","last_active":"2024-05-30T17:11:16.558239+0000","last_peered":"2024-05-30T17:11:16.558239+0000","last_clean":"2024-05-30T17:11:16.558239+0000","last_became_active":"2024-05-30T15:54:16.668083+0000","last_became_peered":"2024-05-30T15:54:16.668083+0000","last_unstale":"2024-05-30T17:11:16.558239+0000","last_undegraded":"2024-05-30T17:11:16.558239+0000","last_fullsized":"2024-05-30T17:11:16.558239+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":100,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:37:10.668986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5290687,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":405,"num_read_kb":13963,"num_write":1053,"num_write_kb":66137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:17.124781+0000","last_change":"2024-05-30T15:54:20.659591+0000","last_active":"2024-05-30T17:09:17.124781+0000","last_peered":"2024-05-30T17:09:17.124781+0000","last_clean":"2024-05-30T17:09:17.124781+0000","last_became_active":"2024-05-30T15:54:20.658870+0000","last_became_peered":"2024-05-30T15:54:20.658870+0000","last_unstale":"2024-05-30T17:09:17.124781+0000","last_undegraded":"2024-05-30T17:09:17.124781+0000","last_fullsized":"2024-05-30T17:09:17.124781+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:06:21.446324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"265'138","reported_seq":721,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.727691+0000","last_change":"2024-05-30T15:54:22.851241+0000","last_active":"2024-05-30T17:13:20.727691+0000","last_peered":"2024-05-30T17:13:20.727691+0000","last_clean":"2024-05-30T17:13:20.727691+0000","last_became_active":"2024-05-30T15:54:22.849978+0000","last_became_peered":"2024-05-30T15:54:22.849978+0000","last_unstale":"2024-05-30T17:13:20.727691+0000","last_undegraded":"2024-05-30T17:13:20.727691+0000","last_fullsized":"2024-05-30T17:13:20.727691+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":100,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:42:52.749855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8486,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":180,"num_read_kb":226,"num_write":98,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"265'546","reported_seq":2569,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.895542+0000","last_change":"2024-05-30T15:54:24.688312+0000","last_active":"2024-05-30T17:13:24.895542+0000","last_peered":"2024-05-30T17:13:24.895542+0000","last_clean":"2024-05-30T17:13:24.895542+0000","last_became_active":"2024-05-30T15:54:24.686546+0000","last_became_peered":"2024-05-30T15:54:24.686546+0000","last_unstale":"2024-05-30T17:13:24.895542+0000","last_undegraded":"2024-05-30T17:13:24.895542+0000","last_fullsized":"2024-05-30T17:13:24.895542+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":500,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:12:50.305701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"265'895","reported_seq":2510,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.673971+0000","last_change":"2024-05-30T15:54:18.658824+0000","last_active":"2024-05-30T17:13:18.673971+0000","last_peered":"2024-05-30T17:13:18.673971+0000","last_clean":"2024-05-30T17:13:18.673971+0000","last_became_active":"2024-05-30T15:54:18.658124+0000","last_became_peered":"2024-05-30T15:54:18.658124+0000","last_unstale":"2024-05-30T17:13:18.673971+0000","last_undegraded":"2024-05-30T17:13:18.673971+0000","last_fullsized":"2024-05-30T17:13:18.673971+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":800,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:47:47.836108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4127,"num_read_kb":3657,"num_write":1246,"num_write_kb":491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"265'205","reported_seq":885,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:17.601310+0000","last_change":"2024-05-30T15:54:16.659141+0000","last_active":"2024-05-30T17:11:17.601310+0000","last_peered":"2024-05-30T17:11:17.601310+0000","last_clean":"2024-05-30T17:11:17.601310+0000","last_became_active":"2024-05-30T15:54:16.658279+0000","last_became_peered":"2024-05-30T15:54:16.658279+0000","last_unstale":"2024-05-30T17:11:17.601310+0000","last_undegraded":"2024-05-30T17:11:17.601310+0000","last_fullsized":"2024-05-30T17:11:17.601310+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":100,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:04:21.938813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1308948,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":477,"num_read_kb":111643,"num_write":1131,"num_write_kb":26225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":510,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:11.551997+0000","last_change":"2024-05-30T15:54:20.662333+0000","last_active":"2024-05-30T17:10:11.551997+0000","last_peered":"2024-05-30T17:10:11.551997+0000","last_clean":"2024-05-30T17:10:11.551997+0000","last_became_active":"2024-05-30T15:54:20.661583+0000","last_became_peered":"2024-05-30T15:54:20.661583+0000","last_unstale":"2024-05-30T17:10:11.551997+0000","last_undegraded":"2024-05-30T17:10:11.551997+0000","last_fullsized":"2024-05-30T17:10:11.551997+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:07:17.026476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"265'252","reported_seq":895,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:36.971452+0000","last_change":"2024-05-30T15:54:22.850012+0000","last_active":"2024-05-30T17:12:36.971452+0000","last_peered":"2024-05-30T17:12:36.971452+0000","last_clean":"2024-05-30T17:12:36.971452+0000","last_became_active":"2024-05-30T15:54:22.849126+0000","last_became_peered":"2024-05-30T15:54:22.849126+0000","last_unstale":"2024-05-30T17:12:36.971452+0000","last_undegraded":"2024-05-30T17:12:36.971452+0000","last_fullsized":"2024-05-30T17:12:36.971452+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":200,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:34:44.383862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7991,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":343,"num_read_kb":359,"num_write":206,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":502,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:56.786717+0000","last_change":"2024-05-30T15:54:24.674322+0000","last_active":"2024-05-30T17:10:56.786717+0000","last_peered":"2024-05-30T17:10:56.786717+0000","last_clean":"2024-05-30T17:10:56.786717+0000","last_became_active":"2024-05-30T15:54:24.673506+0000","last_became_peered":"2024-05-30T15:54:24.673506+0000","last_unstale":"2024-05-30T17:10:56.786717+0000","last_undegraded":"2024-05-30T17:10:56.786717+0000","last_fullsized":"2024-05-30T17:10:56.786717+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:07.777377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"265'543","reported_seq":1373,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.867831+0000","last_change":"2024-05-30T15:54:26.687129+0000","last_active":"2024-05-30T17:13:24.867831+0000","last_peered":"2024-05-30T17:13:24.867831+0000","last_clean":"2024-05-30T17:13:24.867831+0000","last_became_active":"2024-05-30T15:54:26.686389+0000","last_became_peered":"2024-05-30T15:54:26.686389+0000","last_unstale":"2024-05-30T17:13:24.867831+0000","last_undegraded":"2024-05-30T17:13:24.867831+0000","last_fullsized":"2024-05-30T17:13:24.867831+0000","mapping_epoch":32,"log_start":"265'500","ondisk_log_start":"265'500","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":500,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:44:22.858001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1297,"num_read_kb":1288,"num_write":1137,"num_write_kb":385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"265'1398","reported_seq":3105,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.685832+0000","last_change":"2024-05-30T15:54:18.640378+0000","last_active":"2024-05-30T17:13:18.685832+0000","last_peered":"2024-05-30T17:13:18.685832+0000","last_clean":"2024-05-30T17:13:18.685832+0000","last_became_active":"2024-05-30T15:54:18.639667+0000","last_became_peered":"2024-05-30T15:54:18.639667+0000","last_unstale":"2024-05-30T17:13:18.685832+0000","last_undegraded":"2024-05-30T17:13:18.685832+0000","last_fullsized":"2024-05-30T17:13:18.685832+0000","mapping_epoch":24,"log_start":"265'1300","ondisk_log_start":"265'1300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":1300,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:30:46.931867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5970,"num_read_kb":5130,"num_write":2044,"num_write_kb":884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"265'208","reported_seq":862,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.560609+0000","last_change":"2024-05-30T15:54:16.661968+0000","last_active":"2024-05-30T17:10:46.560609+0000","last_peered":"2024-05-30T17:10:46.560609+0000","last_clean":"2024-05-30T17:10:46.560609+0000","last_became_active":"2024-05-30T15:54:16.659427+0000","last_became_peered":"2024-05-30T15:54:16.659427+0000","last_unstale":"2024-05-30T17:10:46.560609+0000","last_undegraded":"2024-05-30T17:10:46.560609+0000","last_fullsized":"2024-05-30T17:10:46.560609+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":100,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:29:13.804456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5946564,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":462,"num_read_kb":34119,"num_write":1152,"num_write_kb":42367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":513,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:11.545498+0000","last_change":"2024-05-30T15:54:20.647908+0000","last_active":"2024-05-30T17:10:11.545498+0000","last_peered":"2024-05-30T17:10:11.545498+0000","last_clean":"2024-05-30T17:10:11.545498+0000","last_became_active":"2024-05-30T15:54:20.647565+0000","last_became_peered":"2024-05-30T15:54:20.647565+0000","last_unstale":"2024-05-30T17:10:11.545498+0000","last_undegraded":"2024-05-30T17:10:11.545498+0000","last_fullsized":"2024-05-30T17:10:11.545498+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:19:36.402906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"265'142","reported_seq":730,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:11.612518+0000","last_change":"2024-05-30T15:54:22.671276+0000","last_active":"2024-05-30T17:13:11.612518+0000","last_peered":"2024-05-30T17:13:11.612518+0000","last_clean":"2024-05-30T17:13:11.612518+0000","last_became_active":"2024-05-30T15:54:22.670064+0000","last_became_peered":"2024-05-30T15:54:22.670064+0000","last_unstale":"2024-05-30T17:13:11.612518+0000","last_undegraded":"2024-05-30T17:13:11.612518+0000","last_fullsized":"2024-05-30T17:13:11.612518+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":100,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:40:10.621760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16203,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":188,"num_read_kb":263,"num_write":104,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":504,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:36.577286+0000","last_change":"2024-05-30T15:54:24.688941+0000","last_active":"2024-05-30T17:10:36.577286+0000","last_peered":"2024-05-30T17:10:36.577286+0000","last_clean":"2024-05-30T17:10:36.577286+0000","last_became_active":"2024-05-30T15:54:24.688046+0000","last_became_peered":"2024-05-30T15:54:24.688046+0000","last_unstale":"2024-05-30T17:10:36.577286+0000","last_undegraded":"2024-05-30T17:10:36.577286+0000","last_fullsized":"2024-05-30T17:10:36.577286+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:44:58.051665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"265'2369","reported_seq":4189,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.624292+0000","last_change":"2024-05-30T15:54:26.692267+0000","last_active":"2024-05-30T17:13:18.624292+0000","last_peered":"2024-05-30T17:13:18.624292+0000","last_clean":"2024-05-30T17:13:18.624292+0000","last_became_active":"2024-05-30T15:54:26.691574+0000","last_became_peered":"2024-05-30T15:54:26.691574+0000","last_unstale":"2024-05-30T17:13:18.624292+0000","last_undegraded":"2024-05-30T17:13:18.624292+0000","last_fullsized":"2024-05-30T17:13:18.624292+0000","mapping_epoch":32,"log_start":"265'2300","ondisk_log_start":"265'2300","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":2300,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:33:03.952010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5943,"num_read_kb":5935,"num_write":4785,"num_write_kb":1767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"265'371","reported_seq":1792,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:35.664016+0000","last_change":"2024-05-30T15:54:18.670009+0000","last_active":"2024-05-30T17:11:35.664016+0000","last_peered":"2024-05-30T17:11:35.664016+0000","last_clean":"2024-05-30T17:11:35.664016+0000","last_became_active":"2024-05-30T15:54:18.668963+0000","last_became_peered":"2024-05-30T15:54:18.668963+0000","last_unstale":"2024-05-30T17:11:35.664016+0000","last_undegraded":"2024-05-30T17:11:35.664016+0000","last_fullsized":"2024-05-30T17:11:35.664016+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":300,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:33:28.500306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2011,"num_read_kb":1890,"num_write":455,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"265'230","reported_seq":870,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.560532+0000","last_change":"2024-05-30T15:54:16.650507+0000","last_active":"2024-05-30T17:10:46.560532+0000","last_peered":"2024-05-30T17:10:46.560532+0000","last_clean":"2024-05-30T17:10:46.560532+0000","last_became_active":"2024-05-30T15:54:16.649605+0000","last_became_peered":"2024-05-30T15:54:16.649605+0000","last_unstale":"2024-05-30T17:10:46.560532+0000","last_undegraded":"2024-05-30T17:10:46.560532+0000","last_fullsized":"2024-05-30T17:10:46.560532+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":200,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:10:51.493426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5355180,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":459,"num_read_kb":32396,"num_write":1233,"num_write_kb":40071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":512,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.554033+0000","last_change":"2024-05-30T15:54:20.652270+0000","last_active":"2024-05-30T17:10:51.554033+0000","last_peered":"2024-05-30T17:10:51.554033+0000","last_clean":"2024-05-30T17:10:51.554033+0000","last_became_active":"2024-05-30T15:54:20.651369+0000","last_became_peered":"2024-05-30T15:54:20.651369+0000","last_unstale":"2024-05-30T17:10:51.554033+0000","last_undegraded":"2024-05-30T17:10:51.554033+0000","last_fullsized":"2024-05-30T17:10:51.554033+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:06:27.275568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"265'187","reported_seq":798,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:26.585367+0000","last_change":"2024-05-30T15:54:22.666930+0000","last_active":"2024-05-30T17:12:26.585367+0000","last_peered":"2024-05-30T17:12:26.585367+0000","last_clean":"2024-05-30T17:12:26.585367+0000","last_became_active":"2024-05-30T15:54:22.664874+0000","last_became_peered":"2024-05-30T15:54:22.664874+0000","last_unstale":"2024-05-30T17:12:26.585367+0000","last_undegraded":"2024-05-30T17:12:26.585367+0000","last_fullsized":"2024-05-30T17:12:26.585367+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":100,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:25:56.647990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6891,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":250,"num_read_kb":293,"num_write":144,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"265'546","reported_seq":2479,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.897156+0000","last_change":"2024-05-30T15:54:24.674977+0000","last_active":"2024-05-30T17:13:24.897156+0000","last_peered":"2024-05-30T17:13:24.897156+0000","last_clean":"2024-05-30T17:13:24.897156+0000","last_became_active":"2024-05-30T15:54:24.670141+0000","last_became_peered":"2024-05-30T15:54:24.670141+0000","last_unstale":"2024-05-30T17:13:24.897156+0000","last_undegraded":"2024-05-30T17:13:24.897156+0000","last_fullsized":"2024-05-30T17:13:24.897156+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":500,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:38:58.029893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"265'107","reported_seq":611,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:16.569182+0000","last_change":"2024-05-30T15:54:26.693610+0000","last_active":"2024-05-30T17:10:16.569182+0000","last_peered":"2024-05-30T17:10:16.569182+0000","last_clean":"2024-05-30T17:10:16.569182+0000","last_became_active":"2024-05-30T15:54:26.689987+0000","last_became_peered":"2024-05-30T15:54:26.689987+0000","last_unstale":"2024-05-30T17:10:16.569182+0000","last_undegraded":"2024-05-30T17:10:16.569182+0000","last_fullsized":"2024-05-30T17:10:16.569182+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:15:03.997666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":80,"num_write":254,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"265'1331","reported_seq":2966,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.705722+0000","last_change":"2024-05-30T15:54:18.647985+0000","last_active":"2024-05-30T17:13:18.705722+0000","last_peered":"2024-05-30T17:13:18.705722+0000","last_clean":"2024-05-30T17:13:18.705722+0000","last_became_active":"2024-05-30T15:54:18.647634+0000","last_became_peered":"2024-05-30T15:54:18.647634+0000","last_unstale":"2024-05-30T17:13:18.705722+0000","last_undegraded":"2024-05-30T17:13:18.705722+0000","last_fullsized":"2024-05-30T17:13:18.705722+0000","mapping_epoch":24,"log_start":"265'1300","ondisk_log_start":"265'1300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":1300,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:55:27.116297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5717,"num_read_kb":4967,"num_write":1912,"num_write_kb":881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"265'202","reported_seq":817,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:47.145058+0000","last_change":"2024-05-30T15:54:16.668968+0000","last_active":"2024-05-30T17:10:47.145058+0000","last_peered":"2024-05-30T17:10:47.145058+0000","last_clean":"2024-05-30T17:10:47.145058+0000","last_became_active":"2024-05-30T15:54:16.668336+0000","last_became_peered":"2024-05-30T15:54:16.668336+0000","last_unstale":"2024-05-30T17:10:47.145058+0000","last_undegraded":"2024-05-30T17:10:47.145058+0000","last_fullsized":"2024-05-30T17:10:47.145058+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:22:09.399226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41078,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":399,"num_read_kb":1534,"num_write":1054,"num_write_kb":21253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":513,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:31.549752+0000","last_change":"2024-05-30T15:54:20.656467+0000","last_active":"2024-05-30T17:10:31.549752+0000","last_peered":"2024-05-30T17:10:31.549752+0000","last_clean":"2024-05-30T17:10:31.549752+0000","last_became_active":"2024-05-30T15:54:20.655573+0000","last_became_peered":"2024-05-30T15:54:20.655573+0000","last_unstale":"2024-05-30T17:10:31.549752+0000","last_undegraded":"2024-05-30T17:10:31.549752+0000","last_fullsized":"2024-05-30T17:10:31.549752+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:20:02.252327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"265'199","reported_seq":849,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.783939+0000","last_change":"2024-05-30T15:54:22.659510+0000","last_active":"2024-05-30T17:13:24.783939+0000","last_peered":"2024-05-30T17:13:24.783939+0000","last_clean":"2024-05-30T17:13:24.783939+0000","last_became_active":"2024-05-30T15:54:22.658700+0000","last_became_peered":"2024-05-30T15:54:22.658700+0000","last_unstale":"2024-05-30T17:13:24.783939+0000","last_undegraded":"2024-05-30T17:13:24.783939+0000","last_fullsized":"2024-05-30T17:13:24.783939+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T15:57:00.030682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18300,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":291,"num_read_kb":407,"num_write":146,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":505,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:16.546507+0000","last_change":"2024-05-30T15:54:24.674234+0000","last_active":"2024-05-30T17:10:16.546507+0000","last_peered":"2024-05-30T17:10:16.546507+0000","last_clean":"2024-05-30T17:10:16.546507+0000","last_became_active":"2024-05-30T15:54:24.673505+0000","last_became_peered":"2024-05-30T15:54:24.673505+0000","last_unstale":"2024-05-30T17:10:16.546507+0000","last_undegraded":"2024-05-30T17:10:16.546507+0000","last_fullsized":"2024-05-30T17:10:16.546507+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:16:54.874496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"265'99","reported_seq":616,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:49.195333+0000","last_change":"2024-05-30T15:54:26.701347+0000","last_active":"2024-05-30T17:10:49.195333+0000","last_peered":"2024-05-30T17:10:49.195333+0000","last_clean":"2024-05-30T17:10:49.195333+0000","last_became_active":"2024-05-30T15:54:26.700608+0000","last_became_peered":"2024-05-30T15:54:26.700608+0000","last_unstale":"2024-05-30T17:10:49.195333+0000","last_undegraded":"2024-05-30T17:10:49.195333+0000","last_fullsized":"2024-05-30T17:10:49.195333+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:57:26.315030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":918,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":103,"num_write":225,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"265'539","reported_seq":2167,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:28.313044+0000","last_change":"2024-05-30T15:54:18.658375+0000","last_active":"2024-05-30T17:12:28.313044+0000","last_peered":"2024-05-30T17:12:28.313044+0000","last_clean":"2024-05-30T17:12:28.313044+0000","last_became_active":"2024-05-30T15:54:18.657522+0000","last_became_peered":"2024-05-30T15:54:18.657522+0000","last_unstale":"2024-05-30T17:12:28.313044+0000","last_undegraded":"2024-05-30T17:12:28.313044+0000","last_fullsized":"2024-05-30T17:12:28.313044+0000","mapping_epoch":24,"log_start":"265'500","ondisk_log_start":"265'500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":500,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:59:15.813358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2751,"num_read_kb":2537,"num_write":708,"num_write_kb":252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"265'230","reported_seq":911,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:34.812361+0000","last_change":"2024-05-30T15:54:16.666646+0000","last_active":"2024-05-30T17:11:34.812361+0000","last_peered":"2024-05-30T17:11:34.812361+0000","last_clean":"2024-05-30T17:11:34.812361+0000","last_became_active":"2024-05-30T15:54:16.666244+0000","last_became_peered":"2024-05-30T15:54:16.666244+0000","last_unstale":"2024-05-30T17:11:34.812361+0000","last_undegraded":"2024-05-30T17:11:34.812361+0000","last_fullsized":"2024-05-30T17:11:34.812361+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":200,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:10:10.302094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11062814,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":634,"num_read_kb":32957,"num_write":1221,"num_write_kb":40334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":512,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.554107+0000","last_change":"2024-05-30T15:54:20.662409+0000","last_active":"2024-05-30T17:10:51.554107+0000","last_peered":"2024-05-30T17:10:51.554107+0000","last_clean":"2024-05-30T17:10:51.554107+0000","last_became_active":"2024-05-30T15:54:20.661967+0000","last_became_peered":"2024-05-30T15:54:20.661967+0000","last_unstale":"2024-05-30T17:10:51.554107+0000","last_undegraded":"2024-05-30T17:10:51.554107+0000","last_fullsized":"2024-05-30T17:10:51.554107+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:38:09.118798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"265'177","reported_seq":780,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:11.775063+0000","last_change":"2024-05-30T15:54:22.665466+0000","last_active":"2024-05-30T17:13:11.775063+0000","last_peered":"2024-05-30T17:13:11.775063+0000","last_clean":"2024-05-30T17:13:11.775063+0000","last_became_active":"2024-05-30T15:54:22.664732+0000","last_became_peered":"2024-05-30T15:54:22.664732+0000","last_unstale":"2024-05-30T17:13:11.775063+0000","last_undegraded":"2024-05-30T17:13:11.775063+0000","last_fullsized":"2024-05-30T17:13:11.775063+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":100,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:44:24.095083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19031,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":256,"num_read_kb":341,"num_write":198,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.582702+0000","last_change":"2024-05-30T15:54:24.679521+0000","last_active":"2024-05-30T17:11:16.582702+0000","last_peered":"2024-05-30T17:11:16.582702+0000","last_clean":"2024-05-30T17:11:16.582702+0000","last_became_active":"2024-05-30T15:54:24.679254+0000","last_became_peered":"2024-05-30T15:54:24.679254+0000","last_unstale":"2024-05-30T17:11:16.582702+0000","last_undegraded":"2024-05-30T17:11:16.582702+0000","last_fullsized":"2024-05-30T17:11:16.582702+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:16:16.737529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"265'105","reported_seq":623,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.579506+0000","last_change":"2024-05-30T15:54:26.697980+0000","last_active":"2024-05-30T17:10:46.579506+0000","last_peered":"2024-05-30T17:10:46.579506+0000","last_clean":"2024-05-30T17:10:46.579506+0000","last_became_active":"2024-05-30T15:54:26.697333+0000","last_became_peered":"2024-05-30T15:54:26.697333+0000","last_unstale":"2024-05-30T17:10:46.579506+0000","last_undegraded":"2024-05-30T17:10:46.579506+0000","last_fullsized":"2024-05-30T17:10:46.579506+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:11:10.336832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":270,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":107,"num_read_kb":90,"num_write":242,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"265'397","reported_seq":2059,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:09.478879+0000","last_change":"2024-05-30T15:54:18.636254+0000","last_active":"2024-05-30T17:13:09.478879+0000","last_peered":"2024-05-30T17:13:09.478879+0000","last_clean":"2024-05-30T17:13:09.478879+0000","last_became_active":"2024-05-30T15:54:18.635519+0000","last_became_peered":"2024-05-30T15:54:18.635519+0000","last_unstale":"2024-05-30T17:13:09.478879+0000","last_undegraded":"2024-05-30T17:13:09.478879+0000","last_fullsized":"2024-05-30T17:13:09.478879+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":300,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:25:03.148120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2332,"num_read_kb":2242,"num_write":450,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"265'173","reported_seq":793,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:48.596500+0000","last_change":"2024-05-30T15:54:16.685723+0000","last_active":"2024-05-30T17:11:48.596500+0000","last_peered":"2024-05-30T17:11:48.596500+0000","last_clean":"2024-05-30T17:11:48.596500+0000","last_became_active":"2024-05-30T15:54:16.685162+0000","last_became_peered":"2024-05-30T15:54:16.685162+0000","last_unstale":"2024-05-30T17:11:48.596500+0000","last_undegraded":"2024-05-30T17:11:48.596500+0000","last_fullsized":"2024-05-30T17:11:48.596500+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":100,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:11:20.614231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6251614,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":399,"num_read_kb":10586,"num_write":1025,"num_write_kb":21029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":510,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.785627+0000","last_change":"2024-05-30T15:54:20.646982+0000","last_active":"2024-05-30T17:10:51.785627+0000","last_peered":"2024-05-30T17:10:51.785627+0000","last_clean":"2024-05-30T17:10:51.785627+0000","last_became_active":"2024-05-30T15:54:20.646646+0000","last_became_peered":"2024-05-30T15:54:20.646646+0000","last_unstale":"2024-05-30T17:10:51.785627+0000","last_undegraded":"2024-05-30T17:10:51.785627+0000","last_fullsized":"2024-05-30T17:10:51.785627+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:31:33.090729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"265'179","reported_seq":792,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.851680+0000","last_change":"2024-05-30T15:54:22.670371+0000","last_active":"2024-05-30T17:13:24.851680+0000","last_peered":"2024-05-30T17:13:24.851680+0000","last_clean":"2024-05-30T17:13:24.851680+0000","last_became_active":"2024-05-30T15:54:22.669660+0000","last_became_peered":"2024-05-30T15:54:22.669660+0000","last_unstale":"2024-05-30T17:13:24.851680+0000","last_undegraded":"2024-05-30T17:13:24.851680+0000","last_fullsized":"2024-05-30T17:13:24.851680+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":100,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:45:08.898874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9643,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":249,"num_read_kb":299,"num_write":142,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"265'509","reported_seq":4592,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.889937+0000","last_change":"2024-05-30T15:54:24.673150+0000","last_active":"2024-05-30T17:13:24.889937+0000","last_peered":"2024-05-30T17:13:24.889937+0000","last_clean":"2024-05-30T17:13:24.889937+0000","last_became_active":"2024-05-30T15:54:24.672377+0000","last_became_peered":"2024-05-30T15:54:24.672377+0000","last_unstale":"2024-05-30T17:13:24.889937+0000","last_undegraded":"2024-05-30T17:13:24.889937+0000","last_fullsized":"2024-05-30T17:13:24.889937+0000","mapping_epoch":30,"log_start":"262'400","ondisk_log_start":"262'400","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":400,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:22:29.518162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"265'117","reported_seq":633,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.718032+0000","last_change":"2024-05-30T15:54:26.680642+0000","last_active":"2024-05-30T17:13:18.718032+0000","last_peered":"2024-05-30T17:13:18.718032+0000","last_clean":"2024-05-30T17:13:18.718032+0000","last_became_active":"2024-05-30T15:54:26.679866+0000","last_became_peered":"2024-05-30T15:54:26.679866+0000","last_unstale":"2024-05-30T17:13:18.718032+0000","last_undegraded":"2024-05-30T17:13:18.718032+0000","last_fullsized":"2024-05-30T17:13:18.718032+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:26:55.645297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1256,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":144,"num_read_kb":134,"num_write":293,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"265'389","reported_seq":1796,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:37.615748+0000","last_change":"2024-05-30T15:54:18.671784+0000","last_active":"2024-05-30T17:11:37.615748+0000","last_peered":"2024-05-30T17:11:37.615748+0000","last_clean":"2024-05-30T17:11:37.615748+0000","last_became_active":"2024-05-30T15:54:18.671088+0000","last_became_peered":"2024-05-30T15:54:18.671088+0000","last_unstale":"2024-05-30T17:11:37.615748+0000","last_undegraded":"2024-05-30T17:11:37.615748+0000","last_fullsized":"2024-05-30T17:11:37.615748+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":300,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:23:36.607873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2029,"num_read_kb":1894,"num_write":481,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"265'178","reported_seq":921,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:28.333806+0000","last_change":"2024-05-30T15:54:16.672575+0000","last_active":"2024-05-30T17:11:28.333806+0000","last_peered":"2024-05-30T17:11:28.333806+0000","last_clean":"2024-05-30T17:11:28.333806+0000","last_became_active":"2024-05-30T15:54:16.671477+0000","last_became_peered":"2024-05-30T15:54:16.671477+0000","last_unstale":"2024-05-30T17:11:28.333806+0000","last_undegraded":"2024-05-30T17:11:28.333806+0000","last_fullsized":"2024-05-30T17:11:28.333806+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":100,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:47:10.773488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":132257,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":677,"num_read_kb":167118,"num_write":1080,"num_write_kb":34693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"27'2","reported_seq":1154,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.724885+0000","last_change":"2024-05-30T15:54:20.663952+0000","last_active":"2024-05-30T17:13:24.724885+0000","last_peered":"2024-05-30T17:13:24.724885+0000","last_clean":"2024-05-30T17:13:24.724885+0000","last_became_active":"2024-05-30T15:54:20.662371+0000","last_became_peered":"2024-05-30T15:54:20.662371+0000","last_unstale":"2024-05-30T17:13:24.724885+0000","last_undegraded":"2024-05-30T17:13:24.724885+0000","last_fullsized":"2024-05-30T17:13:24.724885+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:43:22.768148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":643,"num_read_kb":643,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"265'109","reported_seq":656,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:56.582283+0000","last_change":"2024-05-30T15:54:22.849771+0000","last_active":"2024-05-30T17:10:56.582283+0000","last_peered":"2024-05-30T17:10:56.582283+0000","last_clean":"2024-05-30T17:10:56.582283+0000","last_became_active":"2024-05-30T15:54:22.849105+0000","last_became_peered":"2024-05-30T15:54:22.849105+0000","last_unstale":"2024-05-30T17:10:56.582283+0000","last_undegraded":"2024-05-30T17:10:56.582283+0000","last_fullsized":"2024-05-30T17:10:56.582283+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:56:52.974159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":120,"num_read_kb":115,"num_write":80,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"265'538","reported_seq":2314,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.891869+0000","last_change":"2024-05-30T15:54:24.678108+0000","last_active":"2024-05-30T17:13:24.891869+0000","last_peered":"2024-05-30T17:13:24.891869+0000","last_clean":"2024-05-30T17:13:24.891869+0000","last_became_active":"2024-05-30T15:54:24.677291+0000","last_became_peered":"2024-05-30T15:54:24.677291+0000","last_unstale":"2024-05-30T17:13:24.891869+0000","last_undegraded":"2024-05-30T17:13:24.891869+0000","last_fullsized":"2024-05-30T17:13:24.891869+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":500,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:59:57.567508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"265'135","reported_seq":656,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:15.308427+0000","last_change":"2024-05-30T15:54:26.675238+0000","last_active":"2024-05-30T17:13:15.308427+0000","last_peered":"2024-05-30T17:13:15.308427+0000","last_clean":"2024-05-30T17:13:15.308427+0000","last_became_active":"2024-05-30T15:54:26.674517+0000","last_became_peered":"2024-05-30T15:54:26.674517+0000","last_unstale":"2024-05-30T17:13:15.308427+0000","last_undegraded":"2024-05-30T17:13:15.308427+0000","last_fullsized":"2024-05-30T17:13:15.308427+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":100,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:38:58.206221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":141,"num_read_kb":129,"num_write":332,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"265'402","reported_seq":1940,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:12.347239+0000","last_change":"2024-05-30T15:54:18.657775+0000","last_active":"2024-05-30T17:13:12.347239+0000","last_peered":"2024-05-30T17:13:12.347239+0000","last_clean":"2024-05-30T17:13:12.347239+0000","last_became_active":"2024-05-30T15:54:18.657247+0000","last_became_peered":"2024-05-30T15:54:18.657247+0000","last_unstale":"2024-05-30T17:13:12.347239+0000","last_undegraded":"2024-05-30T17:13:12.347239+0000","last_fullsized":"2024-05-30T17:13:12.347239+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":300,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:33:29.912796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2106,"num_read_kb":2008,"num_write":461,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"265'180","reported_seq":790,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.553032+0000","last_change":"2024-05-30T15:54:54.427377+0000","last_active":"2024-05-30T17:10:46.553032+0000","last_peered":"2024-05-30T17:10:46.553032+0000","last_clean":"2024-05-30T17:10:46.553032+0000","last_became_active":"2024-05-30T15:54:54.426538+0000","last_became_peered":"2024-05-30T15:54:54.426538+0000","last_unstale":"2024-05-30T17:10:46.553032+0000","last_undegraded":"2024-05-30T17:10:46.553032+0000","last_fullsized":"2024-05-30T17:10:46.553032+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":100,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:57:32.635925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5292382,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":438,"num_read_kb":549,"num_write":1007,"num_write_kb":15096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,7],"acting":[2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":518,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.742564+0000","last_change":"2024-05-30T15:54:20.670270+0000","last_active":"2024-05-30T17:10:51.742564+0000","last_peered":"2024-05-30T17:10:51.742564+0000","last_clean":"2024-05-30T17:10:51.742564+0000","last_became_active":"2024-05-30T15:54:20.669492+0000","last_became_peered":"2024-05-30T15:54:20.669492+0000","last_unstale":"2024-05-30T17:10:51.742564+0000","last_undegraded":"2024-05-30T17:10:51.742564+0000","last_fullsized":"2024-05-30T17:10:51.742564+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:01:58.108666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"265'232","reported_seq":866,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:11.979664+0000","last_change":"2024-05-30T15:54:22.672749+0000","last_active":"2024-05-30T17:13:11.979664+0000","last_peered":"2024-05-30T17:13:11.979664+0000","last_clean":"2024-05-30T17:13:11.979664+0000","last_became_active":"2024-05-30T15:54:22.671943+0000","last_became_peered":"2024-05-30T15:54:22.671943+0000","last_unstale":"2024-05-30T17:13:11.979664+0000","last_undegraded":"2024-05-30T17:13:11.979664+0000","last_fullsized":"2024-05-30T17:13:11.979664+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":200,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:43:33.730358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4994,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":304,"num_read_kb":297,"num_write":174,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":500,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:56.553870+0000","last_change":"2024-05-30T15:54:24.682341+0000","last_active":"2024-05-30T17:10:56.553870+0000","last_peered":"2024-05-30T17:10:56.553870+0000","last_clean":"2024-05-30T17:10:56.553870+0000","last_became_active":"2024-05-30T15:54:24.681595+0000","last_became_peered":"2024-05-30T15:54:24.681595+0000","last_unstale":"2024-05-30T17:10:56.553870+0000","last_undegraded":"2024-05-30T17:10:56.553870+0000","last_fullsized":"2024-05-30T17:10:56.553870+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:03:03.587984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"265'115","reported_seq":622,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:31.794301+0000","last_change":"2024-05-30T15:54:26.689376+0000","last_active":"2024-05-30T17:11:31.794301+0000","last_peered":"2024-05-30T17:11:31.794301+0000","last_clean":"2024-05-30T17:11:31.794301+0000","last_became_active":"2024-05-30T15:54:26.688518+0000","last_became_peered":"2024-05-30T15:54:26.688518+0000","last_unstale":"2024-05-30T17:11:31.794301+0000","last_undegraded":"2024-05-30T17:11:31.794301+0000","last_fullsized":"2024-05-30T17:11:31.794301+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":100,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:13:29.397423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":619,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":113,"num_read_kb":112,"num_write":261,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"265'756","reported_seq":2265,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.637015+0000","last_change":"2024-05-30T15:54:18.678584+0000","last_active":"2024-05-30T17:11:21.637015+0000","last_peered":"2024-05-30T17:11:21.637015+0000","last_clean":"2024-05-30T17:11:21.637015+0000","last_became_active":"2024-05-30T15:54:18.677957+0000","last_became_peered":"2024-05-30T15:54:18.677957+0000","last_unstale":"2024-05-30T17:11:21.637015+0000","last_undegraded":"2024-05-30T17:11:21.637015+0000","last_fullsized":"2024-05-30T17:11:21.637015+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":700,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:51:57.717964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3440,"num_read_kb":3078,"num_write":1010,"num_write_kb":372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":627,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:26.555530+0000","last_change":"2024-05-30T15:54:03.031292+0000","last_active":"2024-05-30T17:10:26.555530+0000","last_peered":"2024-05-30T17:10:26.555530+0000","last_clean":"2024-05-30T17:10:26.555530+0000","last_became_active":"2024-05-30T15:54:03.029214+0000","last_became_peered":"2024-05-30T15:54:03.029214+0000","last_unstale":"2024-05-30T17:10:26.555530+0000","last_undegraded":"2024-05-30T17:10:26.555530+0000","last_fullsized":"2024-05-30T17:10:26.555530+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:01.512251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:01.512251+0000","last_clean_scrub_stamp":"2024-05-30T15:54:01.512251+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:40:07.389706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":530,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:17.151931+0000","last_change":"2024-05-30T15:54:08.578526+0000","last_active":"2024-05-30T17:11:17.151931+0000","last_peered":"2024-05-30T17:11:17.151931+0000","last_clean":"2024-05-30T17:11:17.151931+0000","last_became_active":"2024-05-30T15:54:08.577556+0000","last_became_peered":"2024-05-30T15:54:08.577556+0000","last_unstale":"2024-05-30T17:11:17.151931+0000","last_undegraded":"2024-05-30T17:11:17.151931+0000","last_fullsized":"2024-05-30T17:11:17.151931+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:36:52.339688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"265'186","reported_seq":841,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.587587+0000","last_change":"2024-05-30T15:54:16.674631+0000","last_active":"2024-05-30T17:11:21.587587+0000","last_peered":"2024-05-30T17:11:21.587587+0000","last_clean":"2024-05-30T17:11:21.587587+0000","last_became_active":"2024-05-30T15:54:16.674229+0000","last_became_peered":"2024-05-30T15:54:16.674229+0000","last_unstale":"2024-05-30T17:11:21.587587+0000","last_undegraded":"2024-05-30T17:11:21.587587+0000","last_fullsized":"2024-05-30T17:11:21.587587+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":100,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:38:06.634901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10797837,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":444,"num_read_kb":11470,"num_write":1071,"num_write_kb":25445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":512,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.570297+0000","last_change":"2024-05-30T15:54:20.665207+0000","last_active":"2024-05-30T17:11:26.570297+0000","last_peered":"2024-05-30T17:11:26.570297+0000","last_clean":"2024-05-30T17:11:26.570297+0000","last_became_active":"2024-05-30T15:54:20.664317+0000","last_became_peered":"2024-05-30T15:54:20.664317+0000","last_unstale":"2024-05-30T17:11:26.570297+0000","last_undegraded":"2024-05-30T17:11:26.570297+0000","last_fullsized":"2024-05-30T17:11:26.570297+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:57:42.587325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"265'322","reported_seq":1031,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.820569+0000","last_change":"2024-05-30T15:54:22.670731+0000","last_active":"2024-05-30T17:13:24.820569+0000","last_peered":"2024-05-30T17:13:24.820569+0000","last_clean":"2024-05-30T17:13:24.820569+0000","last_became_active":"2024-05-30T15:54:22.669871+0000","last_became_peered":"2024-05-30T15:54:22.669871+0000","last_unstale":"2024-05-30T17:13:24.820569+0000","last_undegraded":"2024-05-30T17:13:24.820569+0000","last_fullsized":"2024-05-30T17:13:24.820569+0000","mapping_epoch":28,"log_start":"265'300","ondisk_log_start":"265'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":300,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:48:31.269839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59120,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":621,"num_read_kb":899,"num_write":586,"num_write_kb":457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":507,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:16.569157+0000","last_change":"2024-05-30T15:54:24.685392+0000","last_active":"2024-05-30T17:10:16.569157+0000","last_peered":"2024-05-30T17:10:16.569157+0000","last_clean":"2024-05-30T17:10:16.569157+0000","last_became_active":"2024-05-30T15:54:24.684709+0000","last_became_peered":"2024-05-30T15:54:24.684709+0000","last_unstale":"2024-05-30T17:10:16.569157+0000","last_undegraded":"2024-05-30T17:10:16.569157+0000","last_fullsized":"2024-05-30T17:10:16.569157+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:57:38.351236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"265'174","reported_seq":878,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:16.948647+0000","last_change":"2024-05-30T15:54:26.687357+0000","last_active":"2024-05-30T17:13:16.948647+0000","last_peered":"2024-05-30T17:13:16.948647+0000","last_clean":"2024-05-30T17:13:16.948647+0000","last_became_active":"2024-05-30T15:54:26.686647+0000","last_became_peered":"2024-05-30T15:54:26.686647+0000","last_unstale":"2024-05-30T17:13:16.948647+0000","last_undegraded":"2024-05-30T17:13:16.948647+0000","last_fullsized":"2024-05-30T17:13:16.948647+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:16:53.692909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1020,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":416,"num_read_kb":404,"num_write":359,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"265'388","reported_seq":1862,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:28.256178+0000","last_change":"2024-05-30T15:54:18.649340+0000","last_active":"2024-05-30T17:12:28.256178+0000","last_peered":"2024-05-30T17:12:28.256178+0000","last_clean":"2024-05-30T17:12:28.256178+0000","last_became_active":"2024-05-30T15:54:18.648529+0000","last_became_peered":"2024-05-30T15:54:18.648529+0000","last_unstale":"2024-05-30T17:12:28.256178+0000","last_undegraded":"2024-05-30T17:12:28.256178+0000","last_fullsized":"2024-05-30T17:12:28.256178+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":300,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:56:22.361251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2082,"num_read_kb":1967,"num_write":465,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":531,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:31.556722+0000","last_change":"2024-05-30T15:54:08.575348+0000","last_active":"2024-05-30T17:10:31.556722+0000","last_peered":"2024-05-30T17:10:31.556722+0000","last_clean":"2024-05-30T17:10:31.556722+0000","last_became_active":"2024-05-30T15:54:08.574645+0000","last_became_peered":"2024-05-30T15:54:08.574645+0000","last_unstale":"2024-05-30T17:10:31.556722+0000","last_undegraded":"2024-05-30T17:10:31.556722+0000","last_fullsized":"2024-05-30T17:10:31.556722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:41:49.462483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"265'171","reported_seq":806,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.584098+0000","last_change":"2024-05-30T15:54:16.678919+0000","last_active":"2024-05-30T17:11:21.584098+0000","last_peered":"2024-05-30T17:11:21.584098+0000","last_clean":"2024-05-30T17:11:21.584098+0000","last_became_active":"2024-05-30T15:54:16.678386+0000","last_became_peered":"2024-05-30T15:54:16.678386+0000","last_unstale":"2024-05-30T17:11:21.584098+0000","last_undegraded":"2024-05-30T17:11:21.584098+0000","last_fullsized":"2024-05-30T17:11:21.584098+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:06:51.673817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6324848,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":361,"num_read_kb":7665,"num_write":1079,"num_write_kb":33234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":510,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:12.137521+0000","last_change":"2024-05-30T15:54:20.663148+0000","last_active":"2024-05-30T17:10:12.137521+0000","last_peered":"2024-05-30T17:10:12.137521+0000","last_clean":"2024-05-30T17:10:12.137521+0000","last_became_active":"2024-05-30T15:54:20.661502+0000","last_became_peered":"2024-05-30T15:54:20.661502+0000","last_unstale":"2024-05-30T17:10:12.137521+0000","last_undegraded":"2024-05-30T17:10:12.137521+0000","last_fullsized":"2024-05-30T17:10:12.137521+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:53:08.492976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"265'706","reported_seq":2523,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:28.415096+0000","last_change":"2024-05-30T15:54:22.672464+0000","last_active":"2024-05-30T17:13:28.415096+0000","last_peered":"2024-05-30T17:13:28.415096+0000","last_clean":"2024-05-30T17:13:28.415096+0000","last_became_active":"2024-05-30T15:54:22.671588+0000","last_became_peered":"2024-05-30T15:54:22.671588+0000","last_unstale":"2024-05-30T17:13:28.415096+0000","last_undegraded":"2024-05-30T17:13:28.415096+0000","last_fullsized":"2024-05-30T17:13:28.415096+0000","mapping_epoch":28,"log_start":"265'600","ondisk_log_start":"265'600","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":600,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:46:49.368553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28054,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":978,"num_read_kb":1146,"num_write":312,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":503,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:46.546239+0000","last_change":"2024-05-30T15:54:24.688069+0000","last_active":"2024-05-30T17:09:46.546239+0000","last_peered":"2024-05-30T17:09:46.546239+0000","last_clean":"2024-05-30T17:09:46.546239+0000","last_became_active":"2024-05-30T15:54:24.686641+0000","last_became_peered":"2024-05-30T15:54:24.686641+0000","last_unstale":"2024-05-30T17:09:46.546239+0000","last_undegraded":"2024-05-30T17:09:46.546239+0000","last_fullsized":"2024-05-30T17:09:46.546239+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:27:01.277040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"265'123","reported_seq":634,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:31.780995+0000","last_change":"2024-05-30T15:54:26.668522+0000","last_active":"2024-05-30T17:10:31.780995+0000","last_peered":"2024-05-30T17:10:31.780995+0000","last_clean":"2024-05-30T17:10:31.780995+0000","last_became_active":"2024-05-30T15:54:26.667751+0000","last_became_peered":"2024-05-30T15:54:26.667751+0000","last_unstale":"2024-05-30T17:10:31.780995+0000","last_undegraded":"2024-05-30T17:10:31.780995+0000","last_fullsized":"2024-05-30T17:10:31.780995+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":100,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:32:24.025165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":438,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":115,"num_write":291,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"265'326","reported_seq":1646,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:33.881393+0000","last_change":"2024-05-30T15:54:18.652138+0000","last_active":"2024-05-30T17:12:33.881393+0000","last_peered":"2024-05-30T17:12:33.881393+0000","last_clean":"2024-05-30T17:12:33.881393+0000","last_became_active":"2024-05-30T15:54:18.651853+0000","last_became_peered":"2024-05-30T15:54:18.651853+0000","last_unstale":"2024-05-30T17:12:33.881393+0000","last_undegraded":"2024-05-30T17:12:33.881393+0000","last_fullsized":"2024-05-30T17:12:33.881393+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":300,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:07:58.160840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1727,"num_read_kb":1636,"num_write":373,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":533,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.557705+0000","last_change":"2024-05-30T15:54:08.745221+0000","last_active":"2024-05-30T17:11:11.557705+0000","last_peered":"2024-05-30T17:11:11.557705+0000","last_clean":"2024-05-30T17:11:11.557705+0000","last_became_active":"2024-05-30T15:54:08.744898+0000","last_became_peered":"2024-05-30T15:54:08.744898+0000","last_unstale":"2024-05-30T17:11:11.557705+0000","last_undegraded":"2024-05-30T17:11:11.557705+0000","last_fullsized":"2024-05-30T17:11:11.557705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:38:33.683752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"265'183","reported_seq":811,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:31.780836+0000","last_change":"2024-05-30T15:54:16.650710+0000","last_active":"2024-05-30T17:10:31.780836+0000","last_peered":"2024-05-30T17:10:31.780836+0000","last_clean":"2024-05-30T17:10:31.780836+0000","last_became_active":"2024-05-30T15:54:16.650214+0000","last_became_peered":"2024-05-30T15:54:16.650214+0000","last_unstale":"2024-05-30T17:10:31.780836+0000","last_undegraded":"2024-05-30T17:10:31.780836+0000","last_fullsized":"2024-05-30T17:10:31.780836+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":100,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:41:20.055617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7478597,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":410,"num_read_kb":8332,"num_write":1152,"num_write_kb":25808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"27'1","reported_seq":514,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:11.545623+0000","last_change":"2024-05-30T15:54:20.667150+0000","last_active":"2024-05-30T17:10:11.545623+0000","last_peered":"2024-05-30T17:10:11.545623+0000","last_clean":"2024-05-30T17:10:11.545623+0000","last_became_active":"2024-05-30T15:54:20.666715+0000","last_became_peered":"2024-05-30T15:54:20.666715+0000","last_unstale":"2024-05-30T17:10:11.545623+0000","last_undegraded":"2024-05-30T17:10:11.545623+0000","last_fullsized":"2024-05-30T17:10:11.545623+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:57:01.669091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"265'259","reported_seq":1771,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.166581+0000","last_change":"2024-05-30T15:54:22.853392+0000","last_active":"2024-05-30T17:13:24.166581+0000","last_peered":"2024-05-30T17:13:24.166581+0000","last_clean":"2024-05-30T17:13:24.166581+0000","last_became_active":"2024-05-30T15:54:22.852604+0000","last_became_peered":"2024-05-30T15:54:22.852604+0000","last_unstale":"2024-05-30T17:13:24.166581+0000","last_undegraded":"2024-05-30T17:13:24.166581+0000","last_fullsized":"2024-05-30T17:13:24.166581+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":200,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:13:39.908798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":232,"num_read_kb":224,"num_write":183,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":509,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:31.586843+0000","last_change":"2024-05-30T15:54:24.680439+0000","last_active":"2024-05-30T17:11:31.586843+0000","last_peered":"2024-05-30T17:11:31.586843+0000","last_clean":"2024-05-30T17:11:31.586843+0000","last_became_active":"2024-05-30T15:54:24.679830+0000","last_became_peered":"2024-05-30T15:54:24.679830+0000","last_unstale":"2024-05-30T17:11:31.586843+0000","last_undegraded":"2024-05-30T17:11:31.586843+0000","last_fullsized":"2024-05-30T17:11:31.586843+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:05:18.168600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"265'251","reported_seq":860,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:19.239543+0000","last_change":"2024-05-30T15:54:26.699114+0000","last_active":"2024-05-30T17:13:19.239543+0000","last_peered":"2024-05-30T17:13:19.239543+0000","last_clean":"2024-05-30T17:13:19.239543+0000","last_became_active":"2024-05-30T15:54:26.696477+0000","last_became_peered":"2024-05-30T15:54:26.696477+0000","last_unstale":"2024-05-30T17:13:19.239543+0000","last_undegraded":"2024-05-30T17:13:19.239543+0000","last_fullsized":"2024-05-30T17:13:19.239543+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":200,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:15:58.988777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":476,"num_read_kb":500,"num_write":542,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"265'761","reported_seq":2321,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:35.663643+0000","last_change":"2024-05-30T15:54:18.640139+0000","last_active":"2024-05-30T17:11:35.663643+0000","last_peered":"2024-05-30T17:11:35.663643+0000","last_clean":"2024-05-30T17:11:35.663643+0000","last_became_active":"2024-05-30T15:54:18.639830+0000","last_became_peered":"2024-05-30T15:54:18.639830+0000","last_unstale":"2024-05-30T17:11:35.663643+0000","last_undegraded":"2024-05-30T17:11:35.663643+0000","last_fullsized":"2024-05-30T17:11:35.663643+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":700,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:05:32.448981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3481,"num_read_kb":3132,"num_write":1014,"num_write_kb":367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":530,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:17.151871+0000","last_change":"2024-05-30T15:54:08.575969+0000","last_active":"2024-05-30T17:11:17.151871+0000","last_peered":"2024-05-30T17:11:17.151871+0000","last_clean":"2024-05-30T17:11:17.151871+0000","last_became_active":"2024-05-30T15:54:08.574817+0000","last_became_peered":"2024-05-30T15:54:08.574817+0000","last_unstale":"2024-05-30T17:11:17.151871+0000","last_undegraded":"2024-05-30T17:11:17.151871+0000","last_fullsized":"2024-05-30T17:11:17.151871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:39:30.889837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"265'194","reported_seq":909,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.749242+0000","last_change":"2024-05-30T15:54:16.656169+0000","last_active":"2024-05-30T17:11:21.749242+0000","last_peered":"2024-05-30T17:11:21.749242+0000","last_clean":"2024-05-30T17:11:21.749242+0000","last_became_active":"2024-05-30T15:54:16.655347+0000","last_became_peered":"2024-05-30T15:54:16.655347+0000","last_unstale":"2024-05-30T17:11:21.749242+0000","last_undegraded":"2024-05-30T17:11:21.749242+0000","last_fullsized":"2024-05-30T17:11:21.749242+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":100,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:46:03.699500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8740487,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":567,"num_read_kb":42208,"num_write":1143,"num_write_kb":36370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":516,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:41.561839+0000","last_change":"2024-05-30T15:54:20.651897+0000","last_active":"2024-05-30T17:09:41.561839+0000","last_peered":"2024-05-30T17:09:41.561839+0000","last_clean":"2024-05-30T17:09:41.561839+0000","last_became_active":"2024-05-30T15:54:20.651449+0000","last_became_peered":"2024-05-30T15:54:20.651449+0000","last_unstale":"2024-05-30T17:09:41.561839+0000","last_undegraded":"2024-05-30T17:09:41.561839+0000","last_fullsized":"2024-05-30T17:09:41.561839+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:14:31.569912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"265'160","reported_seq":732,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:16.572838+0000","last_change":"2024-05-30T15:54:22.679074+0000","last_active":"2024-05-30T17:10:16.572838+0000","last_peered":"2024-05-30T17:10:16.572838+0000","last_clean":"2024-05-30T17:10:16.572838+0000","last_became_active":"2024-05-30T15:54:22.673881+0000","last_became_peered":"2024-05-30T15:54:22.673881+0000","last_unstale":"2024-05-30T17:10:16.572838+0000","last_undegraded":"2024-05-30T17:10:16.572838+0000","last_fullsized":"2024-05-30T17:10:16.572838+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":100,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:18:38.849518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":200,"num_read_kb":192,"num_write":144,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":501,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:16.545213+0000","last_change":"2024-05-30T15:54:24.678710+0000","last_active":"2024-05-30T17:10:16.545213+0000","last_peered":"2024-05-30T17:10:16.545213+0000","last_clean":"2024-05-30T17:10:16.545213+0000","last_became_active":"2024-05-30T15:54:24.675916+0000","last_became_peered":"2024-05-30T15:54:24.675916+0000","last_unstale":"2024-05-30T17:10:16.545213+0000","last_undegraded":"2024-05-30T17:10:16.545213+0000","last_fullsized":"2024-05-30T17:10:16.545213+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:25:04.329603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"265'137","reported_seq":655,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.579761+0000","last_change":"2024-05-30T15:54:26.705809+0000","last_active":"2024-05-30T17:13:20.579761+0000","last_peered":"2024-05-30T17:13:20.579761+0000","last_clean":"2024-05-30T17:13:20.579761+0000","last_became_active":"2024-05-30T15:54:26.704217+0000","last_became_peered":"2024-05-30T15:54:26.704217+0000","last_unstale":"2024-05-30T17:13:20.579761+0000","last_undegraded":"2024-05-30T17:13:20.579761+0000","last_fullsized":"2024-05-30T17:13:20.579761+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":100,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:47:09.950969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":834,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":138,"num_read_kb":136,"num_write":341,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"265'396","reported_seq":1827,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.793186+0000","last_change":"2024-05-30T15:54:18.665286+0000","last_active":"2024-05-30T17:11:26.793186+0000","last_peered":"2024-05-30T17:11:26.793186+0000","last_clean":"2024-05-30T17:11:26.793186+0000","last_became_active":"2024-05-30T15:54:18.665034+0000","last_became_peered":"2024-05-30T15:54:18.665034+0000","last_unstale":"2024-05-30T17:11:26.793186+0000","last_undegraded":"2024-05-30T17:11:26.793186+0000","last_fullsized":"2024-05-30T17:11:26.793186+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":300,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:02:55.348044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2053,"num_read_kb":1920,"num_write":476,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"265'215","reported_seq":856,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:27.154326+0000","last_change":"2024-05-30T15:54:16.651678+0000","last_active":"2024-05-30T17:11:27.154326+0000","last_peered":"2024-05-30T17:11:27.154326+0000","last_clean":"2024-05-30T17:11:27.154326+0000","last_became_active":"2024-05-30T15:54:16.650735+0000","last_became_peered":"2024-05-30T15:54:16.650735+0000","last_unstale":"2024-05-30T17:11:27.154326+0000","last_undegraded":"2024-05-30T17:11:27.154326+0000","last_fullsized":"2024-05-30T17:11:27.154326+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":200,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:36:29.849007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6324891,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":418,"num_read_kb":5617,"num_write":981,"num_write_kb":15732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":531,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.790986+0000","last_change":"2024-05-30T15:54:08.568214+0000","last_active":"2024-05-30T17:11:16.790986+0000","last_peered":"2024-05-30T17:11:16.790986+0000","last_clean":"2024-05-30T17:11:16.790986+0000","last_became_active":"2024-05-30T15:54:08.567900+0000","last_became_peered":"2024-05-30T15:54:08.567900+0000","last_unstale":"2024-05-30T17:11:16.790986+0000","last_undegraded":"2024-05-30T17:11:16.790986+0000","last_fullsized":"2024-05-30T17:11:16.790986+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:02:38.154765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":517,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.585770+0000","last_change":"2024-05-30T15:54:20.654924+0000","last_active":"2024-05-30T17:11:26.585770+0000","last_peered":"2024-05-30T17:11:26.585770+0000","last_clean":"2024-05-30T17:11:26.585770+0000","last_became_active":"2024-05-30T15:54:20.653790+0000","last_became_peered":"2024-05-30T15:54:20.653790+0000","last_unstale":"2024-05-30T17:11:26.585770+0000","last_undegraded":"2024-05-30T17:11:26.585770+0000","last_fullsized":"2024-05-30T17:11:26.585770+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:40:52.633245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"265'353","reported_seq":1045,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.849366+0000","last_change":"2024-05-30T15:54:22.674716+0000","last_active":"2024-05-30T17:13:24.849366+0000","last_peered":"2024-05-30T17:13:24.849366+0000","last_clean":"2024-05-30T17:13:24.849366+0000","last_became_active":"2024-05-30T15:54:22.673903+0000","last_became_peered":"2024-05-30T15:54:22.673903+0000","last_unstale":"2024-05-30T17:13:24.849366+0000","last_undegraded":"2024-05-30T17:13:24.849366+0000","last_fullsized":"2024-05-30T17:13:24.849366+0000","mapping_epoch":28,"log_start":"265'300","ondisk_log_start":"265'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":300,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:29:10.448451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13965,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1122,"num_read_kb":1192,"num_write":1666,"num_write_kb":1473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":513,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:31.751534+0000","last_change":"2024-05-30T15:54:24.679592+0000","last_active":"2024-05-30T17:11:31.751534+0000","last_peered":"2024-05-30T17:11:31.751534+0000","last_clean":"2024-05-30T17:11:31.751534+0000","last_became_active":"2024-05-30T15:54:24.677198+0000","last_became_peered":"2024-05-30T15:54:24.677198+0000","last_unstale":"2024-05-30T17:11:31.751534+0000","last_undegraded":"2024-05-30T17:11:31.751534+0000","last_fullsized":"2024-05-30T17:11:31.751534+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:47:11.591445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"265'117","reported_seq":635,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.126888+0000","last_change":"2024-05-30T15:54:26.676873+0000","last_active":"2024-05-30T17:13:20.126888+0000","last_peered":"2024-05-30T17:13:20.126888+0000","last_clean":"2024-05-30T17:13:20.126888+0000","last_became_active":"2024-05-30T15:54:26.676078+0000","last_became_peered":"2024-05-30T15:54:26.676078+0000","last_unstale":"2024-05-30T17:13:20.126888+0000","last_undegraded":"2024-05-30T17:13:20.126888+0000","last_fullsized":"2024-05-30T17:13:20.126888+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:52:51.891250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":727,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":114,"num_read_kb":108,"num_write":274,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"265'791","reported_seq":2423,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.705651+0000","last_change":"2024-05-30T15:54:18.644630+0000","last_active":"2024-05-30T17:13:20.705651+0000","last_peered":"2024-05-30T17:13:20.705651+0000","last_clean":"2024-05-30T17:13:20.705651+0000","last_became_active":"2024-05-30T15:54:18.642683+0000","last_became_peered":"2024-05-30T15:54:18.642683+0000","last_unstale":"2024-05-30T17:13:20.705651+0000","last_undegraded":"2024-05-30T17:13:20.705651+0000","last_fullsized":"2024-05-30T17:13:20.705651+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":700,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:23:29.409655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3596,"num_read_kb":3243,"num_write":1037,"num_write_kb":360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"265'179","reported_seq":822,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:53.428135+0000","last_change":"2024-05-30T15:54:16.664218+0000","last_active":"2024-05-30T17:11:53.428135+0000","last_peered":"2024-05-30T17:11:53.428135+0000","last_clean":"2024-05-30T17:11:53.428135+0000","last_became_active":"2024-05-30T15:54:16.663360+0000","last_became_peered":"2024-05-30T15:54:16.663360+0000","last_unstale":"2024-05-30T17:11:53.428135+0000","last_undegraded":"2024-05-30T17:11:53.428135+0000","last_fullsized":"2024-05-30T17:11:53.428135+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":100,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:22:13.108688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7656467,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":402,"num_read_kb":30298,"num_write":1058,"num_write_kb":42840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":529,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:26.755968+0000","last_change":"2024-05-30T15:54:08.577379+0000","last_active":"2024-05-30T17:09:26.755968+0000","last_peered":"2024-05-30T17:09:26.755968+0000","last_clean":"2024-05-30T17:09:26.755968+0000","last_became_active":"2024-05-30T15:54:08.576779+0000","last_became_peered":"2024-05-30T15:54:08.576779+0000","last_unstale":"2024-05-30T17:09:26.755968+0000","last_undegraded":"2024-05-30T17:09:26.755968+0000","last_fullsized":"2024-05-30T17:09:26.755968+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:09:01.046338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":511,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.560832+0000","last_change":"2024-05-30T15:54:20.655937+0000","last_active":"2024-05-30T17:11:26.560832+0000","last_peered":"2024-05-30T17:11:26.560832+0000","last_clean":"2024-05-30T17:11:26.560832+0000","last_became_active":"2024-05-30T15:54:20.655220+0000","last_became_peered":"2024-05-30T15:54:20.655220+0000","last_unstale":"2024-05-30T17:11:26.560832+0000","last_undegraded":"2024-05-30T17:11:26.560832+0000","last_fullsized":"2024-05-30T17:11:26.560832+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:42:12.608224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"265'197","reported_seq":794,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:31.794205+0000","last_change":"2024-05-30T15:54:22.851840+0000","last_active":"2024-05-30T17:11:31.794205+0000","last_peered":"2024-05-30T17:11:31.794205+0000","last_clean":"2024-05-30T17:11:31.794205+0000","last_became_active":"2024-05-30T15:54:22.851338+0000","last_became_peered":"2024-05-30T15:54:22.851338+0000","last_unstale":"2024-05-30T17:11:31.794205+0000","last_undegraded":"2024-05-30T17:11:31.794205+0000","last_fullsized":"2024-05-30T17:11:31.794205+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":100,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:25:07.146907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":256,"num_read_kb":249,"num_write":168,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"265'554","reported_seq":2762,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.884870+0000","last_change":"2024-05-30T15:54:24.677979+0000","last_active":"2024-05-30T17:13:24.884870+0000","last_peered":"2024-05-30T17:13:24.884870+0000","last_clean":"2024-05-30T17:13:24.884870+0000","last_became_active":"2024-05-30T15:54:24.677176+0000","last_became_peered":"2024-05-30T15:54:24.677176+0000","last_unstale":"2024-05-30T17:13:24.884870+0000","last_undegraded":"2024-05-30T17:13:24.884870+0000","last_fullsized":"2024-05-30T17:13:24.884870+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":500,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:01:33.668829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"265'124","reported_seq":631,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.862550+0000","last_change":"2024-05-30T15:54:26.697967+0000","last_active":"2024-05-30T17:13:24.862550+0000","last_peered":"2024-05-30T17:13:24.862550+0000","last_clean":"2024-05-30T17:13:24.862550+0000","last_became_active":"2024-05-30T15:54:26.697294+0000","last_became_peered":"2024-05-30T15:54:26.697294+0000","last_unstale":"2024-05-30T17:13:24.862550+0000","last_undegraded":"2024-05-30T17:13:24.862550+0000","last_fullsized":"2024-05-30T17:13:24.862550+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":100,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:31:13.852916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1933,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":112,"num_read_kb":106,"num_write":288,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"265'716","reported_seq":2110,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.659105+0000","last_change":"2024-05-30T15:54:18.653261+0000","last_active":"2024-05-30T17:12:44.659105+0000","last_peered":"2024-05-30T17:12:44.659105+0000","last_clean":"2024-05-30T17:12:44.659105+0000","last_became_active":"2024-05-30T15:54:18.652981+0000","last_became_peered":"2024-05-30T15:54:18.652981+0000","last_unstale":"2024-05-30T17:12:44.659105+0000","last_undegraded":"2024-05-30T17:12:44.659105+0000","last_fullsized":"2024-05-30T17:12:44.659105+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":700,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:28:55.055044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3098,"num_read_kb":2766,"num_write":954,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"265'195","reported_seq":857,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:25.480699+0000","last_change":"2024-05-30T15:54:16.652512+0000","last_active":"2024-05-30T17:10:25.480699+0000","last_peered":"2024-05-30T17:10:25.480699+0000","last_clean":"2024-05-30T17:10:25.480699+0000","last_became_active":"2024-05-30T15:54:16.652033+0000","last_became_peered":"2024-05-30T15:54:16.652033+0000","last_unstale":"2024-05-30T17:10:25.480699+0000","last_undegraded":"2024-05-30T17:10:25.480699+0000","last_fullsized":"2024-05-30T17:10:25.480699+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":100,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:21:56.815021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11145910,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":510,"num_read_kb":15333,"num_write":1047,"num_write_kb":22470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":529,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.772799+0000","last_change":"2024-05-30T15:54:08.574181+0000","last_active":"2024-05-30T17:09:56.772799+0000","last_peered":"2024-05-30T17:09:56.772799+0000","last_clean":"2024-05-30T17:09:56.772799+0000","last_became_active":"2024-05-30T15:54:08.573754+0000","last_became_peered":"2024-05-30T15:54:08.573754+0000","last_unstale":"2024-05-30T17:09:56.772799+0000","last_undegraded":"2024-05-30T17:09:56.772799+0000","last_fullsized":"2024-05-30T17:09:56.772799+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:35:01.844361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":521,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.750400+0000","last_change":"2024-05-30T15:54:20.657666+0000","last_active":"2024-05-30T17:11:26.750400+0000","last_peered":"2024-05-30T17:11:26.750400+0000","last_clean":"2024-05-30T17:11:26.750400+0000","last_became_active":"2024-05-30T15:54:20.656941+0000","last_became_peered":"2024-05-30T15:54:20.656941+0000","last_unstale":"2024-05-30T17:11:26.750400+0000","last_undegraded":"2024-05-30T17:11:26.750400+0000","last_fullsized":"2024-05-30T17:11:26.750400+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:45:33.081783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"265'130","reported_seq":703,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:31.751592+0000","last_change":"2024-05-30T15:54:22.674809+0000","last_active":"2024-05-30T17:11:31.751592+0000","last_peered":"2024-05-30T17:11:31.751592+0000","last_clean":"2024-05-30T17:11:31.751592+0000","last_became_active":"2024-05-30T15:54:22.674027+0000","last_became_peered":"2024-05-30T15:54:22.674027+0000","last_unstale":"2024-05-30T17:11:31.751592+0000","last_undegraded":"2024-05-30T17:11:31.751592+0000","last_fullsized":"2024-05-30T17:11:31.751592+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:34:39.215953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":164,"num_read_kb":159,"num_write":108,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"265'547","reported_seq":2630,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:28.474744+0000","last_change":"2024-05-30T15:54:24.677368+0000","last_active":"2024-05-30T17:13:28.474744+0000","last_peered":"2024-05-30T17:13:28.474744+0000","last_clean":"2024-05-30T17:13:28.474744+0000","last_became_active":"2024-05-30T15:54:24.676608+0000","last_became_peered":"2024-05-30T15:54:24.676608+0000","last_unstale":"2024-05-30T17:13:28.474744+0000","last_undegraded":"2024-05-30T17:13:28.474744+0000","last_fullsized":"2024-05-30T17:13:28.474744+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":500,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:44:44.176765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"265'227","reported_seq":774,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:11.610342+0000","last_change":"2024-05-30T15:54:26.700019+0000","last_active":"2024-05-30T17:13:11.610342+0000","last_peered":"2024-05-30T17:13:11.610342+0000","last_clean":"2024-05-30T17:13:11.610342+0000","last_became_active":"2024-05-30T15:54:26.699148+0000","last_became_peered":"2024-05-30T15:54:26.699148+0000","last_unstale":"2024-05-30T17:13:11.610342+0000","last_undegraded":"2024-05-30T17:13:11.610342+0000","last_fullsized":"2024-05-30T17:13:11.610342+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":200,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:16:32.958184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":326,"num_read_kb":298,"num_write":504,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"265'747","reported_seq":2339,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:28.311866+0000","last_change":"2024-05-30T15:54:18.658639+0000","last_active":"2024-05-30T17:12:28.311866+0000","last_peered":"2024-05-30T17:12:28.311866+0000","last_clean":"2024-05-30T17:12:28.311866+0000","last_became_active":"2024-05-30T15:54:18.657842+0000","last_became_peered":"2024-05-30T15:54:18.657842+0000","last_unstale":"2024-05-30T17:12:28.311866+0000","last_undegraded":"2024-05-30T17:12:28.311866+0000","last_fullsized":"2024-05-30T17:12:28.311866+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":700,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:08:00.261523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3632,"num_read_kb":3285,"num_write":997,"num_write_kb":370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"265'227","reported_seq":983,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:40.014659+0000","last_change":"2024-05-30T15:54:16.657515+0000","last_active":"2024-05-30T17:10:40.014659+0000","last_peered":"2024-05-30T17:10:40.014659+0000","last_clean":"2024-05-30T17:10:40.014659+0000","last_became_active":"2024-05-30T15:54:16.657067+0000","last_became_peered":"2024-05-30T15:54:16.657067+0000","last_unstale":"2024-05-30T17:10:40.014659+0000","last_undegraded":"2024-05-30T17:10:40.014659+0000","last_fullsized":"2024-05-30T17:10:40.014659+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":200,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:03:23.557850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14883328,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":674,"num_read_kb":72431,"num_write":1151,"num_write_kb":30547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":527,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:01.541851+0000","last_change":"2024-05-30T15:54:08.578053+0000","last_active":"2024-05-30T17:10:01.541851+0000","last_peered":"2024-05-30T17:10:01.541851+0000","last_clean":"2024-05-30T17:10:01.541851+0000","last_became_active":"2024-05-30T15:54:08.577191+0000","last_became_peered":"2024-05-30T15:54:08.577191+0000","last_unstale":"2024-05-30T17:10:01.541851+0000","last_undegraded":"2024-05-30T17:10:01.541851+0000","last_fullsized":"2024-05-30T17:10:01.541851+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:38:36.674998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":515,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:41.578396+0000","last_change":"2024-05-30T15:54:20.653740+0000","last_active":"2024-05-30T17:10:41.578396+0000","last_peered":"2024-05-30T17:10:41.578396+0000","last_clean":"2024-05-30T17:10:41.578396+0000","last_became_active":"2024-05-30T15:54:20.653404+0000","last_became_peered":"2024-05-30T15:54:20.653404+0000","last_unstale":"2024-05-30T17:10:41.578396+0000","last_undegraded":"2024-05-30T17:10:41.578396+0000","last_fullsized":"2024-05-30T17:10:41.578396+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:17:55.928914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"265'334","reported_seq":1069,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.809682+0000","last_change":"2024-05-30T15:54:22.849490+0000","last_active":"2024-05-30T17:13:24.809682+0000","last_peered":"2024-05-30T17:13:24.809682+0000","last_clean":"2024-05-30T17:13:24.809682+0000","last_became_active":"2024-05-30T15:54:22.848761+0000","last_became_peered":"2024-05-30T15:54:22.848761+0000","last_unstale":"2024-05-30T17:13:24.809682+0000","last_undegraded":"2024-05-30T17:13:24.809682+0000","last_fullsized":"2024-05-30T17:13:24.809682+0000","mapping_epoch":28,"log_start":"265'300","ondisk_log_start":"265'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:40:14.623631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34688,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":498,"num_read_kb":700,"num_write":262,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:31.589903+0000","last_change":"2024-05-30T15:54:24.687584+0000","last_active":"2024-05-30T17:11:31.589903+0000","last_peered":"2024-05-30T17:11:31.589903+0000","last_clean":"2024-05-30T17:11:31.589903+0000","last_became_active":"2024-05-30T15:54:24.686801+0000","last_became_peered":"2024-05-30T15:54:24.686801+0000","last_unstale":"2024-05-30T17:11:31.589903+0000","last_undegraded":"2024-05-30T17:11:31.589903+0000","last_fullsized":"2024-05-30T17:11:31.589903+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:41:00.549932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"265'150","reported_seq":651,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:55.992645+0000","last_change":"2024-05-30T15:54:26.701426+0000","last_active":"2024-05-30T17:11:55.992645+0000","last_peered":"2024-05-30T17:11:55.992645+0000","last_clean":"2024-05-30T17:11:55.992645+0000","last_became_active":"2024-05-30T15:54:26.700325+0000","last_became_peered":"2024-05-30T15:54:26.700325+0000","last_unstale":"2024-05-30T17:11:55.992645+0000","last_undegraded":"2024-05-30T17:11:55.992645+0000","last_fullsized":"2024-05-30T17:11:55.992645+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":100,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:05:22.451018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":204,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":115,"num_write":342,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"265'149","reported_seq":722,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:21.183321+0000","last_change":"2024-05-30T15:54:26.687046+0000","last_active":"2024-05-30T17:13:21.183321+0000","last_peered":"2024-05-30T17:13:21.183321+0000","last_clean":"2024-05-30T17:13:21.183321+0000","last_became_active":"2024-05-30T15:54:26.686310+0000","last_became_peered":"2024-05-30T15:54:26.686310+0000","last_unstale":"2024-05-30T17:13:21.183321+0000","last_undegraded":"2024-05-30T17:13:21.183321+0000","last_fullsized":"2024-05-30T17:13:21.183321+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:16:45.128093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":227,"num_read_kb":222,"num_write":347,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"265'746","reported_seq":2045,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.689780+0000","last_change":"2024-05-30T15:54:18.674169+0000","last_active":"2024-05-30T17:13:18.689780+0000","last_peered":"2024-05-30T17:13:18.689780+0000","last_clean":"2024-05-30T17:13:18.689780+0000","last_became_active":"2024-05-30T15:54:18.673418+0000","last_became_peered":"2024-05-30T15:54:18.673418+0000","last_unstale":"2024-05-30T17:13:18.689780+0000","last_undegraded":"2024-05-30T17:13:18.689780+0000","last_fullsized":"2024-05-30T17:13:18.689780+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":700,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:47:19.313441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3258,"num_read_kb":2851,"num_write":1044,"num_write_kb":425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"265'197","reported_seq":816,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.543474+0000","last_change":"2024-05-30T15:54:54.451649+0000","last_active":"2024-05-30T17:09:56.543474+0000","last_peered":"2024-05-30T17:09:56.543474+0000","last_clean":"2024-05-30T17:09:56.543474+0000","last_became_active":"2024-05-30T15:54:54.450966+0000","last_became_peered":"2024-05-30T15:54:54.450966+0000","last_unstale":"2024-05-30T17:09:56.543474+0000","last_undegraded":"2024-05-30T17:09:56.543474+0000","last_fullsized":"2024-05-30T17:09:56.543474+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":100,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:16:23.208005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5857853,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":396,"num_read_kb":38159,"num_write":1159,"num_write_kb":35962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":508,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:01.760976+0000","last_change":"2024-05-30T15:54:20.665689+0000","last_active":"2024-05-30T17:09:01.760976+0000","last_peered":"2024-05-30T17:09:01.760976+0000","last_clean":"2024-05-30T17:09:01.760976+0000","last_became_active":"2024-05-30T15:54:20.664882+0000","last_became_peered":"2024-05-30T15:54:20.664882+0000","last_unstale":"2024-05-30T17:09:01.760976+0000","last_undegraded":"2024-05-30T17:09:01.760976+0000","last_fullsized":"2024-05-30T17:09:01.760976+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:08:16.396333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"265'149","reported_seq":743,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.588979+0000","last_change":"2024-05-30T15:54:22.672900+0000","last_active":"2024-05-30T17:11:26.588979+0000","last_peered":"2024-05-30T17:11:26.588979+0000","last_clean":"2024-05-30T17:11:26.588979+0000","last_became_active":"2024-05-30T15:54:22.672170+0000","last_became_peered":"2024-05-30T15:54:22.672170+0000","last_unstale":"2024-05-30T17:11:26.588979+0000","last_undegraded":"2024-05-30T17:11:26.588979+0000","last_fullsized":"2024-05-30T17:11:26.588979+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:07:18.306116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7618,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":196,"num_read_kb":242,"num_write":106,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":501,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:06.556338+0000","last_change":"2024-05-30T15:54:24.688552+0000","last_active":"2024-05-30T17:09:06.556338+0000","last_peered":"2024-05-30T17:09:06.556338+0000","last_clean":"2024-05-30T17:09:06.556338+0000","last_became_active":"2024-05-30T15:54:24.688053+0000","last_became_peered":"2024-05-30T15:54:24.688053+0000","last_unstale":"2024-05-30T17:09:06.556338+0000","last_undegraded":"2024-05-30T17:09:06.556338+0000","last_fullsized":"2024-05-30T17:09:06.556338+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:53:25.256385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"265'117","reported_seq":642,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.579861+0000","last_change":"2024-05-30T15:54:26.702292+0000","last_active":"2024-05-30T17:10:46.579861+0000","last_peered":"2024-05-30T17:10:46.579861+0000","last_clean":"2024-05-30T17:10:46.579861+0000","last_became_active":"2024-05-30T15:54:26.701588+0000","last_became_peered":"2024-05-30T15:54:26.701588+0000","last_unstale":"2024-05-30T17:10:46.579861+0000","last_undegraded":"2024-05-30T17:10:46.579861+0000","last_fullsized":"2024-05-30T17:10:46.579861+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:13:38.040108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":826,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":172,"num_read_kb":153,"num_write":293,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"265'321","reported_seq":1758,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:55.923011+0000","last_change":"2024-05-30T15:54:18.671021+0000","last_active":"2024-05-30T17:11:55.923011+0000","last_peered":"2024-05-30T17:11:55.923011+0000","last_clean":"2024-05-30T17:11:55.923011+0000","last_became_active":"2024-05-30T15:54:18.670707+0000","last_became_peered":"2024-05-30T15:54:18.670707+0000","last_unstale":"2024-05-30T17:11:55.923011+0000","last_undegraded":"2024-05-30T17:11:55.923011+0000","last_fullsized":"2024-05-30T17:11:55.923011+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":300,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:20:52.055626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1910,"num_read_kb":1828,"num_write":376,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"265'215","reported_seq":879,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:11.752074+0000","last_change":"2024-05-30T15:54:16.663769+0000","last_active":"2024-05-30T17:09:11.752074+0000","last_peered":"2024-05-30T17:09:11.752074+0000","last_clean":"2024-05-30T17:09:11.752074+0000","last_became_active":"2024-05-30T15:54:16.660427+0000","last_became_peered":"2024-05-30T15:54:16.660427+0000","last_unstale":"2024-05-30T17:09:11.752074+0000","last_undegraded":"2024-05-30T17:09:11.752074+0000","last_fullsized":"2024-05-30T17:09:11.752074+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":200,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:37:35.566472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5852361,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":514,"num_read_kb":33224,"num_write":1144,"num_write_kb":46290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":509,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:01.555309+0000","last_change":"2024-05-30T15:54:20.649980+0000","last_active":"2024-05-30T17:09:01.555309+0000","last_peered":"2024-05-30T17:09:01.555309+0000","last_clean":"2024-05-30T17:09:01.555309+0000","last_became_active":"2024-05-30T15:54:20.649330+0000","last_became_peered":"2024-05-30T15:54:20.649330+0000","last_unstale":"2024-05-30T17:09:01.555309+0000","last_undegraded":"2024-05-30T17:09:01.555309+0000","last_fullsized":"2024-05-30T17:09:01.555309+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:16:31.107653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"265'71","reported_seq":595,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:06.552569+0000","last_change":"2024-05-30T15:54:22.663453+0000","last_active":"2024-05-30T17:09:06.552569+0000","last_peered":"2024-05-30T17:09:06.552569+0000","last_clean":"2024-05-30T17:09:06.552569+0000","last_became_active":"2024-05-30T15:54:22.662738+0000","last_became_peered":"2024-05-30T15:54:22.662738+0000","last_unstale":"2024-05-30T17:09:06.552569+0000","last_undegraded":"2024-05-30T17:09:06.552569+0000","last_fullsized":"2024-05-30T17:09:06.552569+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:42:21.227185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":50,"num_write":40,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":499,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:06.750855+0000","last_change":"2024-05-30T15:54:24.682005+0000","last_active":"2024-05-30T17:09:06.750855+0000","last_peered":"2024-05-30T17:09:06.750855+0000","last_clean":"2024-05-30T17:09:06.750855+0000","last_became_active":"2024-05-30T15:54:24.680578+0000","last_became_peered":"2024-05-30T15:54:24.680578+0000","last_unstale":"2024-05-30T17:09:06.750855+0000","last_undegraded":"2024-05-30T17:09:06.750855+0000","last_fullsized":"2024-05-30T17:09:06.750855+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:54:25.321085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"265'212","reported_seq":772,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.665178+0000","last_change":"2024-05-30T15:54:26.687158+0000","last_active":"2024-05-30T17:13:18.665178+0000","last_peered":"2024-05-30T17:13:18.665178+0000","last_clean":"2024-05-30T17:13:18.665178+0000","last_became_active":"2024-05-30T15:54:26.686553+0000","last_became_peered":"2024-05-30T15:54:26.686553+0000","last_unstale":"2024-05-30T17:13:18.665178+0000","last_undegraded":"2024-05-30T17:13:18.665178+0000","last_fullsized":"2024-05-30T17:13:18.665178+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:03:50.128325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":816,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":414,"num_read_kb":383,"num_write":513,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"265'413","reported_seq":1789,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.600546+0000","last_change":"2024-05-30T15:54:18.660456+0000","last_active":"2024-05-30T17:12:44.600546+0000","last_peered":"2024-05-30T17:12:44.600546+0000","last_clean":"2024-05-30T17:12:44.600546+0000","last_became_active":"2024-05-30T15:54:18.660089+0000","last_became_peered":"2024-05-30T15:54:18.660089+0000","last_unstale":"2024-05-30T17:12:44.600546+0000","last_undegraded":"2024-05-30T17:12:44.600546+0000","last_fullsized":"2024-05-30T17:12:44.600546+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":400,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:29:46.343317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2036,"num_read_kb":1890,"num_write":519,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"265'214","reported_seq":883,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:12.123493+0000","last_change":"2024-05-30T15:54:16.650752+0000","last_active":"2024-05-30T17:09:12.123493+0000","last_peered":"2024-05-30T17:09:12.123493+0000","last_clean":"2024-05-30T17:09:12.123493+0000","last_became_active":"2024-05-30T15:54:16.650025+0000","last_became_peered":"2024-05-30T15:54:16.650025+0000","last_unstale":"2024-05-30T17:09:12.123493+0000","last_undegraded":"2024-05-30T17:09:12.123493+0000","last_fullsized":"2024-05-30T17:09:12.123493+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":200,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:14:35.914964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5917240,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":497,"num_read_kb":102670,"num_write":1110,"num_write_kb":39714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":507,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:01.749537+0000","last_change":"2024-05-30T15:54:20.657385+0000","last_active":"2024-05-30T17:09:01.749537+0000","last_peered":"2024-05-30T17:09:01.749537+0000","last_clean":"2024-05-30T17:09:01.749537+0000","last_became_active":"2024-05-30T15:54:20.656663+0000","last_became_peered":"2024-05-30T15:54:20.656663+0000","last_unstale":"2024-05-30T17:09:01.749537+0000","last_undegraded":"2024-05-30T17:09:01.749537+0000","last_fullsized":"2024-05-30T17:09:01.749537+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:31:26.705085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"265'182","reported_seq":792,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.589082+0000","last_change":"2024-05-30T15:54:22.676934+0000","last_active":"2024-05-30T17:11:26.589082+0000","last_peered":"2024-05-30T17:11:26.589082+0000","last_clean":"2024-05-30T17:11:26.589082+0000","last_became_active":"2024-05-30T15:54:22.673673+0000","last_became_peered":"2024-05-30T15:54:22.673673+0000","last_unstale":"2024-05-30T17:11:26.589082+0000","last_undegraded":"2024-05-30T17:11:26.589082+0000","last_fullsized":"2024-05-30T17:11:26.589082+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":100,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:34:48.241130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10523,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":250,"num_read_kb":311,"num_write":142,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":498,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:06.529823+0000","last_change":"2024-05-30T15:54:24.676055+0000","last_active":"2024-05-30T17:09:06.529823+0000","last_peered":"2024-05-30T17:09:06.529823+0000","last_clean":"2024-05-30T17:09:06.529823+0000","last_became_active":"2024-05-30T15:54:24.675289+0000","last_became_peered":"2024-05-30T15:54:24.675289+0000","last_unstale":"2024-05-30T17:09:06.529823+0000","last_undegraded":"2024-05-30T17:09:06.529823+0000","last_fullsized":"2024-05-30T17:09:06.529823+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:37:01.468842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"265'138","reported_seq":634,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:27.127163+0000","last_change":"2024-05-30T15:54:26.704882+0000","last_active":"2024-05-30T17:09:27.127163+0000","last_peered":"2024-05-30T17:09:27.127163+0000","last_clean":"2024-05-30T17:09:27.127163+0000","last_became_active":"2024-05-30T15:54:26.703433+0000","last_became_peered":"2024-05-30T15:54:26.703433+0000","last_unstale":"2024-05-30T17:09:27.127163+0000","last_undegraded":"2024-05-30T17:09:27.127163+0000","last_fullsized":"2024-05-30T17:09:27.127163+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":100,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:19:12.878679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":856,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":110,"num_write":329,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"265'863","reported_seq":2490,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:33.948079+0000","last_change":"2024-05-30T15:54:18.649560+0000","last_active":"2024-05-30T17:12:33.948079+0000","last_peered":"2024-05-30T17:12:33.948079+0000","last_clean":"2024-05-30T17:12:33.948079+0000","last_became_active":"2024-05-30T15:54:18.649273+0000","last_became_peered":"2024-05-30T15:54:18.649273+0000","last_unstale":"2024-05-30T17:12:33.948079+0000","last_undegraded":"2024-05-30T17:12:33.948079+0000","last_fullsized":"2024-05-30T17:12:33.948079+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":800,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:31:47.570692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3918,"num_read_kb":3498,"num_write":1187,"num_write_kb":458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"265'202","reported_seq":862,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:44.661866+0000","last_change":"2024-05-30T15:54:16.649854+0000","last_active":"2024-05-30T17:09:44.661866+0000","last_peered":"2024-05-30T17:09:44.661866+0000","last_clean":"2024-05-30T17:09:44.661866+0000","last_became_active":"2024-05-30T15:54:16.649121+0000","last_became_peered":"2024-05-30T15:54:16.649121+0000","last_unstale":"2024-05-30T17:09:44.661866+0000","last_undegraded":"2024-05-30T17:09:44.661866+0000","last_fullsized":"2024-05-30T17:09:44.661866+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:25:59.798639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12165036,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":488,"num_read_kb":65907,"num_write":1115,"num_write_kb":54558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:01.528692+0000","last_change":"2024-05-30T15:54:20.668691+0000","last_active":"2024-05-30T17:09:01.528692+0000","last_peered":"2024-05-30T17:09:01.528692+0000","last_clean":"2024-05-30T17:09:01.528692+0000","last_became_active":"2024-05-30T15:54:20.667863+0000","last_became_peered":"2024-05-30T15:54:20.667863+0000","last_unstale":"2024-05-30T17:09:01.528692+0000","last_undegraded":"2024-05-30T17:09:01.528692+0000","last_fullsized":"2024-05-30T17:09:01.528692+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:31.629459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"265'139","reported_seq":724,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.570167+0000","last_change":"2024-05-30T15:54:22.665742+0000","last_active":"2024-05-30T17:11:26.570167+0000","last_peered":"2024-05-30T17:11:26.570167+0000","last_clean":"2024-05-30T17:11:26.570167+0000","last_became_active":"2024-05-30T15:54:22.664977+0000","last_became_peered":"2024-05-30T15:54:22.664977+0000","last_unstale":"2024-05-30T17:11:26.570167+0000","last_undegraded":"2024-05-30T17:11:26.570167+0000","last_fullsized":"2024-05-30T17:11:26.570167+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":100,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:20:57.960795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10584,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":178,"num_read_kb":239,"num_write":94,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":504,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:06.552717+0000","last_change":"2024-05-30T15:54:24.677137+0000","last_active":"2024-05-30T17:09:06.552717+0000","last_peered":"2024-05-30T17:09:06.552717+0000","last_clean":"2024-05-30T17:09:06.552717+0000","last_became_active":"2024-05-30T15:54:24.676707+0000","last_became_peered":"2024-05-30T15:54:24.676707+0000","last_unstale":"2024-05-30T17:09:06.552717+0000","last_undegraded":"2024-05-30T17:09:06.552717+0000","last_fullsized":"2024-05-30T17:09:06.552717+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:37:14.112070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"265'442","reported_seq":2099,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:28.309213+0000","last_change":"2024-05-30T15:54:18.671501+0000","last_active":"2024-05-30T17:12:28.309213+0000","last_peered":"2024-05-30T17:12:28.309213+0000","last_clean":"2024-05-30T17:12:28.309213+0000","last_became_active":"2024-05-30T15:54:18.670756+0000","last_became_peered":"2024-05-30T15:54:18.670756+0000","last_unstale":"2024-05-30T17:12:28.309213+0000","last_undegraded":"2024-05-30T17:12:28.309213+0000","last_fullsized":"2024-05-30T17:12:28.309213+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":400,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:50:30.677735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2383,"num_read_kb":2277,"num_write":522,"num_write_kb":97,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"265'154","reported_seq":758,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:22.227972+0000","last_change":"2024-05-30T15:54:16.654636+0000","last_active":"2024-05-30T17:10:22.227972+0000","last_peered":"2024-05-30T17:10:22.227972+0000","last_clean":"2024-05-30T17:10:22.227972+0000","last_became_active":"2024-05-30T15:54:16.653560+0000","last_became_peered":"2024-05-30T15:54:16.653560+0000","last_unstale":"2024-05-30T17:10:22.227972+0000","last_undegraded":"2024-05-30T17:10:22.227972+0000","last_fullsized":"2024-05-30T17:10:22.227972+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":100,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:47:43.349582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5814859,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":316,"num_read_kb":31907,"num_write":883,"num_write_kb":20958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":518,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:41.574743+0000","last_change":"2024-05-30T15:54:20.662992+0000","last_active":"2024-05-30T17:10:41.574743+0000","last_peered":"2024-05-30T17:10:41.574743+0000","last_clean":"2024-05-30T17:10:41.574743+0000","last_became_active":"2024-05-30T15:54:20.662669+0000","last_became_peered":"2024-05-30T15:54:20.662669+0000","last_unstale":"2024-05-30T17:10:41.574743+0000","last_undegraded":"2024-05-30T17:10:41.574743+0000","last_fullsized":"2024-05-30T17:10:41.574743+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:46:17.417038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"265'216","reported_seq":857,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.843270+0000","last_change":"2024-05-30T15:54:22.664232+0000","last_active":"2024-05-30T17:13:24.843270+0000","last_peered":"2024-05-30T17:13:24.843270+0000","last_clean":"2024-05-30T17:13:24.843270+0000","last_became_active":"2024-05-30T15:54:22.663518+0000","last_became_peered":"2024-05-30T15:54:22.663518+0000","last_unstale":"2024-05-30T17:13:24.843270+0000","last_undegraded":"2024-05-30T17:13:24.843270+0000","last_fullsized":"2024-05-30T17:13:24.843270+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":200,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:51:09.801047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18037,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":324,"num_read_kb":450,"num_write":214,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":497,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:27.127216+0000","last_change":"2024-05-30T15:54:24.683358+0000","last_active":"2024-05-30T17:09:27.127216+0000","last_peered":"2024-05-30T17:09:27.127216+0000","last_clean":"2024-05-30T17:09:27.127216+0000","last_became_active":"2024-05-30T15:54:24.682609+0000","last_became_peered":"2024-05-30T15:54:24.682609+0000","last_unstale":"2024-05-30T17:09:27.127216+0000","last_undegraded":"2024-05-30T17:09:27.127216+0000","last_fullsized":"2024-05-30T17:09:27.127216+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:15:33.566334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"265'149","reported_seq":660,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:12.359173+0000","last_change":"2024-05-30T15:54:26.695969+0000","last_active":"2024-05-30T17:11:12.359173+0000","last_peered":"2024-05-30T17:11:12.359173+0000","last_clean":"2024-05-30T17:11:12.359173+0000","last_became_active":"2024-05-30T15:54:26.695210+0000","last_became_peered":"2024-05-30T15:54:26.695210+0000","last_unstale":"2024-05-30T17:11:12.359173+0000","last_undegraded":"2024-05-30T17:11:12.359173+0000","last_fullsized":"2024-05-30T17:11:12.359173+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:13:43.303060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":643,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":167,"num_read_kb":179,"num_write":338,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"265'854","reported_seq":2475,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.655415+0000","last_change":"2024-05-30T15:54:18.641464+0000","last_active":"2024-05-30T17:12:44.655415+0000","last_peered":"2024-05-30T17:12:44.655415+0000","last_clean":"2024-05-30T17:12:44.655415+0000","last_became_active":"2024-05-30T15:54:18.640739+0000","last_became_peered":"2024-05-30T15:54:18.640739+0000","last_unstale":"2024-05-30T17:12:44.655415+0000","last_undegraded":"2024-05-30T17:12:44.655415+0000","last_fullsized":"2024-05-30T17:12:44.655415+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":800,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:05:20.428299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4053,"num_read_kb":3626,"num_write":1150,"num_write_kb":447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"265'241","reported_seq":974,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:28.400625+0000","last_change":"2024-05-30T15:54:54.425702+0000","last_active":"2024-05-30T17:09:28.400625+0000","last_peered":"2024-05-30T17:09:28.400625+0000","last_clean":"2024-05-30T17:09:28.400625+0000","last_became_active":"2024-05-30T15:54:54.424825+0000","last_became_peered":"2024-05-30T15:54:54.424825+0000","last_unstale":"2024-05-30T17:09:28.400625+0000","last_undegraded":"2024-05-30T17:09:28.400625+0000","last_fullsized":"2024-05-30T17:09:28.400625+0000","mapping_epoch":35,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":200,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:43:34.336982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11204870,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":883,"num_read_kb":12756,"num_write":1226,"num_write_kb":34615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"34'3","reported_seq":836,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.748812+0000","last_change":"2024-05-30T15:54:20.664818+0000","last_active":"2024-05-30T17:13:24.748812+0000","last_peered":"2024-05-30T17:13:24.748812+0000","last_clean":"2024-05-30T17:13:24.748812+0000","last_became_active":"2024-05-30T15:54:20.664095+0000","last_became_peered":"2024-05-30T15:54:20.664095+0000","last_unstale":"2024-05-30T17:13:24.748812+0000","last_undegraded":"2024-05-30T17:13:24.748812+0000","last_fullsized":"2024-05-30T17:13:24.748812+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:27:22.067783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":491,"num_read_kb":489,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"265'190","reported_seq":796,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:56.578238+0000","last_change":"2024-05-30T15:54:22.665708+0000","last_active":"2024-05-30T17:11:56.578238+0000","last_peered":"2024-05-30T17:11:56.578238+0000","last_clean":"2024-05-30T17:11:56.578238+0000","last_became_active":"2024-05-30T15:54:22.664997+0000","last_became_peered":"2024-05-30T15:54:22.664997+0000","last_unstale":"2024-05-30T17:11:56.578238+0000","last_undegraded":"2024-05-30T17:11:56.578238+0000","last_fullsized":"2024-05-30T17:11:56.578238+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":100,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:45:04.949797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28724,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":281,"num_read_kb":391,"num_write":188,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":498,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:26.767043+0000","last_change":"2024-05-30T15:54:24.678998+0000","last_active":"2024-05-30T17:09:26.767043+0000","last_peered":"2024-05-30T17:09:26.767043+0000","last_clean":"2024-05-30T17:09:26.767043+0000","last_became_active":"2024-05-30T15:54:24.675583+0000","last_became_peered":"2024-05-30T15:54:24.675583+0000","last_unstale":"2024-05-30T17:09:26.767043+0000","last_undegraded":"2024-05-30T17:09:26.767043+0000","last_fullsized":"2024-05-30T17:09:26.767043+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:17:21.969608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"265'140","reported_seq":655,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.560874+0000","last_change":"2024-05-30T15:54:26.703108+0000","last_active":"2024-05-30T17:11:26.560874+0000","last_peered":"2024-05-30T17:11:26.560874+0000","last_clean":"2024-05-30T17:11:26.560874+0000","last_became_active":"2024-05-30T15:54:26.700392+0000","last_became_peered":"2024-05-30T15:54:26.700392+0000","last_unstale":"2024-05-30T17:11:26.560874+0000","last_undegraded":"2024-05-30T17:11:26.560874+0000","last_fullsized":"2024-05-30T17:11:26.560874+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":100,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:25:14.903459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2627,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":177,"num_read_kb":174,"num_write":403,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"265'368","reported_seq":1738,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:33.884093+0000","last_change":"2024-05-30T15:54:54.420129+0000","last_active":"2024-05-30T17:12:33.884093+0000","last_peered":"2024-05-30T17:12:33.884093+0000","last_clean":"2024-05-30T17:12:33.884093+0000","last_became_active":"2024-05-30T15:54:54.419468+0000","last_became_peered":"2024-05-30T15:54:54.419468+0000","last_unstale":"2024-05-30T17:12:33.884093+0000","last_undegraded":"2024-05-30T17:12:33.884093+0000","last_fullsized":"2024-05-30T17:12:33.884093+0000","mapping_epoch":35,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":300,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:08:55.802490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1898,"num_read_kb":1782,"num_write":455,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"265'156","reported_seq":881,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:52.778205+0000","last_change":"2024-05-30T15:54:16.677283+0000","last_active":"2024-05-30T17:12:52.778205+0000","last_peered":"2024-05-30T17:12:52.778205+0000","last_clean":"2024-05-30T17:12:52.778205+0000","last_became_active":"2024-05-30T15:54:16.676440+0000","last_became_peered":"2024-05-30T15:54:16.676440+0000","last_unstale":"2024-05-30T17:12:52.778205+0000","last_undegraded":"2024-05-30T17:12:52.778205+0000","last_fullsized":"2024-05-30T17:12:52.778205+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":100,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:16:12.516128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9724052,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":866,"num_read_kb":45599,"num_write":922,"num_write_kb":34636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":514,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.723009+0000","last_change":"2024-05-30T15:54:20.656111+0000","last_active":"2024-05-30T17:09:21.723009+0000","last_peered":"2024-05-30T17:09:21.723009+0000","last_clean":"2024-05-30T17:09:21.723009+0000","last_became_active":"2024-05-30T15:54:20.655388+0000","last_became_peered":"2024-05-30T15:54:20.655388+0000","last_unstale":"2024-05-30T17:09:21.723009+0000","last_undegraded":"2024-05-30T17:09:21.723009+0000","last_fullsized":"2024-05-30T17:09:21.723009+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:07:28.254294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"265'161","reported_seq":760,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.837510+0000","last_change":"2024-05-30T15:54:22.849861+0000","last_active":"2024-05-30T17:13:24.837510+0000","last_peered":"2024-05-30T17:13:24.837510+0000","last_clean":"2024-05-30T17:13:24.837510+0000","last_became_active":"2024-05-30T15:54:22.849193+0000","last_became_peered":"2024-05-30T15:54:22.849193+0000","last_unstale":"2024-05-30T17:13:24.837510+0000","last_undegraded":"2024-05-30T17:13:24.837510+0000","last_fullsized":"2024-05-30T17:13:24.837510+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":100,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:51:38.667242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13488,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":212,"num_read_kb":283,"num_write":120,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:41.783470+0000","last_change":"2024-05-30T15:54:24.682047+0000","last_active":"2024-05-30T17:10:41.783470+0000","last_peered":"2024-05-30T17:10:41.783470+0000","last_clean":"2024-05-30T17:10:41.783470+0000","last_became_active":"2024-05-30T15:54:24.681609+0000","last_became_peered":"2024-05-30T15:54:24.681609+0000","last_unstale":"2024-05-30T17:10:41.783470+0000","last_undegraded":"2024-05-30T17:10:41.783470+0000","last_fullsized":"2024-05-30T17:10:41.783470+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:36:01.484941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"265'129","reported_seq":631,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.853790+0000","last_change":"2024-05-30T15:54:26.706376+0000","last_active":"2024-05-30T17:13:24.853790+0000","last_peered":"2024-05-30T17:13:24.853790+0000","last_clean":"2024-05-30T17:13:24.853790+0000","last_became_active":"2024-05-30T15:54:26.704842+0000","last_became_peered":"2024-05-30T15:54:26.704842+0000","last_unstale":"2024-05-30T17:13:24.853790+0000","last_undegraded":"2024-05-30T17:13:24.853790+0000","last_fullsized":"2024-05-30T17:13:24.853790+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":100,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:31:31.301239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":383,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":127,"num_read_kb":122,"num_write":322,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"265'753","reported_seq":2251,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:51.565345+0000","last_change":"2024-05-30T15:54:18.635089+0000","last_active":"2024-05-30T17:11:51.565345+0000","last_peered":"2024-05-30T17:11:51.565345+0000","last_clean":"2024-05-30T17:11:51.565345+0000","last_became_active":"2024-05-30T15:54:18.634803+0000","last_became_peered":"2024-05-30T15:54:18.634803+0000","last_unstale":"2024-05-30T17:11:51.565345+0000","last_undegraded":"2024-05-30T17:11:51.565345+0000","last_fullsized":"2024-05-30T17:11:51.565345+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":700,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:29:09.056772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":3476,"num_read_kb":3109,"num_write":1006,"num_write_kb":389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"265'186","reported_seq":810,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.543434+0000","last_change":"2024-05-30T15:54:54.431157+0000","last_active":"2024-05-30T17:09:56.543434+0000","last_peered":"2024-05-30T17:09:56.543434+0000","last_clean":"2024-05-30T17:09:56.543434+0000","last_became_active":"2024-05-30T15:54:54.430490+0000","last_became_peered":"2024-05-30T15:54:54.430490+0000","last_unstale":"2024-05-30T17:09:56.543434+0000","last_undegraded":"2024-05-30T17:09:56.543434+0000","last_fullsized":"2024-05-30T17:09:56.543434+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":100,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:47:17.028289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6643361,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":477,"num_read_kb":40323,"num_write":1071,"num_write_kb":31617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.765694+0000","last_change":"2024-05-30T15:54:20.651307+0000","last_active":"2024-05-30T17:09:21.765694+0000","last_peered":"2024-05-30T17:09:21.765694+0000","last_clean":"2024-05-30T17:09:21.765694+0000","last_became_active":"2024-05-30T15:54:20.650944+0000","last_became_peered":"2024-05-30T15:54:20.650944+0000","last_unstale":"2024-05-30T17:09:21.765694+0000","last_undegraded":"2024-05-30T17:09:21.765694+0000","last_fullsized":"2024-05-30T17:09:21.765694+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:44:01.827195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"265'113","reported_seq":671,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:41.740281+0000","last_change":"2024-05-30T15:54:22.851946+0000","last_active":"2024-05-30T17:10:41.740281+0000","last_peered":"2024-05-30T17:10:41.740281+0000","last_clean":"2024-05-30T17:10:41.740281+0000","last_became_active":"2024-05-30T15:54:22.850865+0000","last_became_peered":"2024-05-30T15:54:22.850865+0000","last_unstale":"2024-05-30T17:10:41.740281+0000","last_undegraded":"2024-05-30T17:10:41.740281+0000","last_fullsized":"2024-05-30T17:10:41.740281+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":100,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:49:42.844967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":127,"num_read_kb":121,"num_write":92,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"265'538","reported_seq":4166,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:28.422661+0000","last_change":"2024-05-30T15:54:24.681428+0000","last_active":"2024-05-30T17:13:28.422661+0000","last_peered":"2024-05-30T17:13:28.422661+0000","last_clean":"2024-05-30T17:13:28.422661+0000","last_became_active":"2024-05-30T15:54:24.680695+0000","last_became_peered":"2024-05-30T15:54:24.680695+0000","last_unstale":"2024-05-30T17:13:28.422661+0000","last_undegraded":"2024-05-30T17:13:28.422661+0000","last_fullsized":"2024-05-30T17:13:28.422661+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":500,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:53:20.536261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"265'134","reported_seq":641,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:35.623018+0000","last_change":"2024-05-30T15:54:26.696600+0000","last_active":"2024-05-30T17:11:35.623018+0000","last_peered":"2024-05-30T17:11:35.623018+0000","last_clean":"2024-05-30T17:11:35.623018+0000","last_became_active":"2024-05-30T15:54:26.695842+0000","last_became_peered":"2024-05-30T15:54:26.695842+0000","last_unstale":"2024-05-30T17:11:35.623018+0000","last_undegraded":"2024-05-30T17:11:35.623018+0000","last_fullsized":"2024-05-30T17:11:35.623018+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:01:21.719509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":418,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":133,"num_read_kb":127,"num_write":334,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"265'423","reported_seq":1942,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.652772+0000","last_change":"2024-05-30T15:54:18.661318+0000","last_active":"2024-05-30T17:13:20.652772+0000","last_peered":"2024-05-30T17:13:20.652772+0000","last_clean":"2024-05-30T17:13:20.652772+0000","last_became_active":"2024-05-30T15:54:18.660567+0000","last_became_peered":"2024-05-30T15:54:18.660567+0000","last_unstale":"2024-05-30T17:13:20.652772+0000","last_undegraded":"2024-05-30T17:13:20.652772+0000","last_fullsized":"2024-05-30T17:13:20.652772+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":400,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:24:20.096151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2226,"num_read_kb":2104,"num_write":504,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"265'303","reported_seq":994,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:17.124619+0000","last_change":"2024-05-30T15:54:16.686321+0000","last_active":"2024-05-30T17:09:17.124619+0000","last_peered":"2024-05-30T17:09:17.124619+0000","last_clean":"2024-05-30T17:09:17.124619+0000","last_became_active":"2024-05-30T15:54:16.685691+0000","last_became_peered":"2024-05-30T15:54:16.685691+0000","last_unstale":"2024-05-30T17:09:17.124619+0000","last_undegraded":"2024-05-30T17:09:17.124619+0000","last_fullsized":"2024-05-30T17:09:17.124619+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":200,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:37:07.773513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":379054,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":761,"num_read_kb":1764,"num_write":1250,"num_write_kb":18095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":505,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.754576+0000","last_change":"2024-05-30T15:54:20.670328+0000","last_active":"2024-05-30T17:09:21.754576+0000","last_peered":"2024-05-30T17:09:21.754576+0000","last_clean":"2024-05-30T17:09:21.754576+0000","last_became_active":"2024-05-30T15:54:20.669751+0000","last_became_peered":"2024-05-30T15:54:20.669751+0000","last_unstale":"2024-05-30T17:09:21.754576+0000","last_undegraded":"2024-05-30T17:09:21.754576+0000","last_fullsized":"2024-05-30T17:09:21.754576+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:06:46.261517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"265'213","reported_seq":820,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:52.160194+0000","last_change":"2024-05-30T15:54:22.672899+0000","last_active":"2024-05-30T17:11:52.160194+0000","last_peered":"2024-05-30T17:11:52.160194+0000","last_clean":"2024-05-30T17:11:52.160194+0000","last_became_active":"2024-05-30T15:54:22.672078+0000","last_became_peered":"2024-05-30T15:54:22.672078+0000","last_unstale":"2024-05-30T17:11:52.160194+0000","last_undegraded":"2024-05-30T17:11:52.160194+0000","last_fullsized":"2024-05-30T17:11:52.160194+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":200,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:23:16.149546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14727,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":288,"num_read_kb":379,"num_write":182,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":503,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.557376+0000","last_change":"2024-05-30T15:54:24.676069+0000","last_active":"2024-05-30T17:09:21.557376+0000","last_peered":"2024-05-30T17:09:21.557376+0000","last_clean":"2024-05-30T17:09:21.557376+0000","last_became_active":"2024-05-30T15:54:24.675467+0000","last_became_peered":"2024-05-30T15:54:24.675467+0000","last_unstale":"2024-05-30T17:09:21.557376+0000","last_undegraded":"2024-05-30T17:09:21.557376+0000","last_fullsized":"2024-05-30T17:09:21.557376+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:10.271248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"265'102","reported_seq":619,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:19.803711+0000","last_change":"2024-05-30T15:54:26.705986+0000","last_active":"2024-05-30T17:13:19.803711+0000","last_peered":"2024-05-30T17:13:19.803711+0000","last_clean":"2024-05-30T17:13:19.803711+0000","last_became_active":"2024-05-30T15:54:26.705264+0000","last_became_peered":"2024-05-30T15:54:26.705264+0000","last_unstale":"2024-05-30T17:13:19.803711+0000","last_undegraded":"2024-05-30T17:13:19.803711+0000","last_fullsized":"2024-05-30T17:13:19.803711+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:18:43.441191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":805,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":151,"num_read_kb":135,"num_write":247,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"265'698","reported_seq":2167,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.677934+0000","last_change":"2024-05-30T15:54:54.422775+0000","last_active":"2024-05-30T17:13:18.677934+0000","last_peered":"2024-05-30T17:13:18.677934+0000","last_clean":"2024-05-30T17:13:18.677934+0000","last_became_active":"2024-05-30T15:54:54.421604+0000","last_became_peered":"2024-05-30T15:54:54.421604+0000","last_unstale":"2024-05-30T17:13:18.677934+0000","last_undegraded":"2024-05-30T17:13:18.677934+0000","last_fullsized":"2024-05-30T17:13:18.677934+0000","mapping_epoch":35,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":600,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:32:11.130243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3204,"num_read_kb":2893,"num_write":920,"num_write_kb":324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"265'163","reported_seq":804,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:17.124667+0000","last_change":"2024-05-30T15:54:54.424699+0000","last_active":"2024-05-30T17:09:17.124667+0000","last_peered":"2024-05-30T17:09:17.124667+0000","last_clean":"2024-05-30T17:09:17.124667+0000","last_became_active":"2024-05-30T15:54:54.422916+0000","last_became_peered":"2024-05-30T15:54:54.422916+0000","last_unstale":"2024-05-30T17:09:17.124667+0000","last_undegraded":"2024-05-30T17:09:17.124667+0000","last_fullsized":"2024-05-30T17:09:17.124667+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":100,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:06:41.620026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15294568,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":395,"num_read_kb":71105,"num_write":966,"num_write_kb":25767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"34'4","reported_seq":829,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.708703+0000","last_change":"2024-05-30T15:54:20.652854+0000","last_active":"2024-05-30T17:13:24.708703+0000","last_peered":"2024-05-30T17:13:24.708703+0000","last_clean":"2024-05-30T17:13:24.708703+0000","last_became_active":"2024-05-30T15:54:20.652134+0000","last_became_peered":"2024-05-30T15:54:20.652134+0000","last_unstale":"2024-05-30T17:13:24.708703+0000","last_undegraded":"2024-05-30T17:13:24.708703+0000","last_fullsized":"2024-05-30T17:13:24.708703+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:12:04.911743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1139,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":490,"num_read_kb":802,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"265'147","reported_seq":736,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:46.590296+0000","last_change":"2024-05-30T15:54:22.673262+0000","last_active":"2024-05-30T17:11:46.590296+0000","last_peered":"2024-05-30T17:11:46.590296+0000","last_clean":"2024-05-30T17:11:46.590296+0000","last_became_active":"2024-05-30T15:54:22.672169+0000","last_became_peered":"2024-05-30T15:54:22.672169+0000","last_unstale":"2024-05-30T17:11:46.590296+0000","last_undegraded":"2024-05-30T17:11:46.590296+0000","last_fullsized":"2024-05-30T17:11:46.590296+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":100,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:04:32.418773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8883,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":183,"num_read_kb":244,"num_write":98,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":497,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:41.537447+0000","last_change":"2024-05-30T15:54:24.676446+0000","last_active":"2024-05-30T17:09:41.537447+0000","last_peered":"2024-05-30T17:09:41.537447+0000","last_clean":"2024-05-30T17:09:41.537447+0000","last_became_active":"2024-05-30T15:54:24.675683+0000","last_became_peered":"2024-05-30T15:54:24.675683+0000","last_unstale":"2024-05-30T17:09:41.537447+0000","last_undegraded":"2024-05-30T17:09:41.537447+0000","last_fullsized":"2024-05-30T17:09:41.537447+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:10:10.728748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"265'127","reported_seq":655,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:17.801147+0000","last_change":"2024-05-30T15:54:26.689830+0000","last_active":"2024-05-30T17:13:17.801147+0000","last_peered":"2024-05-30T17:13:17.801147+0000","last_clean":"2024-05-30T17:13:17.801147+0000","last_became_active":"2024-05-30T15:54:26.689049+0000","last_became_peered":"2024-05-30T15:54:26.689049+0000","last_unstale":"2024-05-30T17:13:17.801147+0000","last_undegraded":"2024-05-30T17:13:17.801147+0000","last_fullsized":"2024-05-30T17:13:17.801147+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:41:39.309741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":539,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":167,"num_read_kb":166,"num_write":309,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"265'652","reported_seq":2069,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.759093+0000","last_change":"2024-05-30T15:54:18.653246+0000","last_active":"2024-05-30T17:13:18.759093+0000","last_peered":"2024-05-30T17:13:18.759093+0000","last_clean":"2024-05-30T17:13:18.759093+0000","last_became_active":"2024-05-30T15:54:18.651778+0000","last_became_peered":"2024-05-30T15:54:18.651778+0000","last_unstale":"2024-05-30T17:13:18.759093+0000","last_undegraded":"2024-05-30T17:13:18.759093+0000","last_fullsized":"2024-05-30T17:13:18.759093+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":600,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:42:03.298813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3068,"num_read_kb":2749,"num_write":930,"num_write_kb":367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"265'151","reported_seq":765,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:36.577419+0000","last_change":"2024-05-30T15:54:16.656872+0000","last_active":"2024-05-30T17:10:36.577419+0000","last_peered":"2024-05-30T17:10:36.577419+0000","last_clean":"2024-05-30T17:10:36.577419+0000","last_became_active":"2024-05-30T15:54:16.656081+0000","last_became_peered":"2024-05-30T15:54:16.656081+0000","last_unstale":"2024-05-30T17:10:36.577419+0000","last_undegraded":"2024-05-30T17:10:36.577419+0000","last_fullsized":"2024-05-30T17:10:36.577419+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:25:32.809071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":369,"num_read_kb":8201,"num_write":842,"num_write_kb":22196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":515,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:16.721503+0000","last_change":"2024-05-30T15:54:20.661844+0000","last_active":"2024-05-30T17:09:16.721503+0000","last_peered":"2024-05-30T17:09:16.721503+0000","last_clean":"2024-05-30T17:09:16.721503+0000","last_became_active":"2024-05-30T15:54:20.661090+0000","last_became_peered":"2024-05-30T15:54:20.661090+0000","last_unstale":"2024-05-30T17:09:16.721503+0000","last_undegraded":"2024-05-30T17:09:16.721503+0000","last_fullsized":"2024-05-30T17:09:16.721503+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:39:52.559621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"265'112","reported_seq":680,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.784725+0000","last_change":"2024-05-30T15:54:22.670767+0000","last_active":"2024-05-30T17:13:18.784725+0000","last_peered":"2024-05-30T17:13:18.784725+0000","last_clean":"2024-05-30T17:13:18.784725+0000","last_became_active":"2024-05-30T15:54:22.669073+0000","last_became_peered":"2024-05-30T15:54:22.669073+0000","last_unstale":"2024-05-30T17:13:18.784725+0000","last_undegraded":"2024-05-30T17:13:18.784725+0000","last_fullsized":"2024-05-30T17:13:18.784725+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:43:33.730358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8403,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":171,"num_write":70,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:26.724184+0000","last_change":"2024-05-30T15:54:24.680707+0000","last_active":"2024-05-30T17:09:26.724184+0000","last_peered":"2024-05-30T17:09:26.724184+0000","last_clean":"2024-05-30T17:09:26.724184+0000","last_became_active":"2024-05-30T15:54:24.678292+0000","last_became_peered":"2024-05-30T15:54:24.678292+0000","last_unstale":"2024-05-30T17:09:26.724184+0000","last_undegraded":"2024-05-30T17:09:26.724184+0000","last_fullsized":"2024-05-30T17:09:26.724184+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:56:06.653666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"265'164","reported_seq":901,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:14.442195+0000","last_change":"2024-05-30T15:54:26.704198+0000","last_active":"2024-05-30T17:13:14.442195+0000","last_peered":"2024-05-30T17:13:14.442195+0000","last_clean":"2024-05-30T17:13:14.442195+0000","last_became_active":"2024-05-30T15:54:26.703440+0000","last_became_peered":"2024-05-30T15:54:26.703440+0000","last_unstale":"2024-05-30T17:13:14.442195+0000","last_undegraded":"2024-05-30T17:13:14.442195+0000","last_fullsized":"2024-05-30T17:13:14.442195+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":100,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:28:50.587698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1074,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":420,"num_read_kb":450,"num_write":403,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"265'327","reported_seq":1681,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:15.190426+0000","last_change":"2024-05-30T15:54:18.672371+0000","last_active":"2024-05-30T17:12:15.190426+0000","last_peered":"2024-05-30T17:12:15.190426+0000","last_clean":"2024-05-30T17:12:15.190426+0000","last_became_active":"2024-05-30T15:54:18.671650+0000","last_became_peered":"2024-05-30T15:54:18.671650+0000","last_unstale":"2024-05-30T17:12:15.190426+0000","last_undegraded":"2024-05-30T17:12:15.190426+0000","last_fullsized":"2024-05-30T17:12:15.190426+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":300,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:52:16.244837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1672,"num_read_kb":1605,"num_write":361,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"265'205","reported_seq":866,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:34.743973+0000","last_change":"2024-05-30T15:54:16.661703+0000","last_active":"2024-05-30T17:12:34.743973+0000","last_peered":"2024-05-30T17:12:34.743973+0000","last_clean":"2024-05-30T17:12:34.743973+0000","last_became_active":"2024-05-30T15:54:16.660393+0000","last_became_peered":"2024-05-30T15:54:16.660393+0000","last_unstale":"2024-05-30T17:12:34.743973+0000","last_undegraded":"2024-05-30T17:12:34.743973+0000","last_fullsized":"2024-05-30T17:12:34.743973+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":100,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:51:03.780593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4788806,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":518,"num_read_kb":9177,"num_write":1140,"num_write_kb":26248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"27'1","reported_seq":508,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:31.768150+0000","last_change":"2024-05-30T15:54:20.661745+0000","last_active":"2024-05-30T17:09:31.768150+0000","last_peered":"2024-05-30T17:09:31.768150+0000","last_clean":"2024-05-30T17:09:31.768150+0000","last_became_active":"2024-05-30T15:54:20.661311+0000","last_became_peered":"2024-05-30T15:54:20.661311+0000","last_unstale":"2024-05-30T17:09:31.768150+0000","last_undegraded":"2024-05-30T17:09:31.768150+0000","last_fullsized":"2024-05-30T17:09:31.768150+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:08:29.614757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"265'171","reported_seq":778,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:56.757203+0000","last_change":"2024-05-30T15:54:22.663985+0000","last_active":"2024-05-30T17:11:56.757203+0000","last_peered":"2024-05-30T17:11:56.757203+0000","last_clean":"2024-05-30T17:11:56.757203+0000","last_became_active":"2024-05-30T15:54:22.663231+0000","last_became_peered":"2024-05-30T15:54:22.663231+0000","last_unstale":"2024-05-30T17:11:56.757203+0000","last_undegraded":"2024-05-30T17:11:56.757203+0000","last_fullsized":"2024-05-30T17:11:56.757203+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:32:51.856936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9029,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":225,"num_read_kb":274,"num_write":130,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":502,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:26.558679+0000","last_change":"2024-05-30T15:54:24.676660+0000","last_active":"2024-05-30T17:09:26.558679+0000","last_peered":"2024-05-30T17:09:26.558679+0000","last_clean":"2024-05-30T17:09:26.558679+0000","last_became_active":"2024-05-30T15:54:24.673674+0000","last_became_peered":"2024-05-30T15:54:24.673674+0000","last_unstale":"2024-05-30T17:09:26.558679+0000","last_undegraded":"2024-05-30T17:09:26.558679+0000","last_fullsized":"2024-05-30T17:09:26.558679+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:37:07.948924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"265'111","reported_seq":616,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.723032+0000","last_change":"2024-05-30T15:54:26.678237+0000","last_active":"2024-05-30T17:09:21.723032+0000","last_peered":"2024-05-30T17:09:21.723032+0000","last_clean":"2024-05-30T17:09:21.723032+0000","last_became_active":"2024-05-30T15:54:26.677418+0000","last_became_peered":"2024-05-30T15:54:26.677418+0000","last_unstale":"2024-05-30T17:09:21.723032+0000","last_undegraded":"2024-05-30T17:09:21.723032+0000","last_fullsized":"2024-05-30T17:09:21.723032+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":100,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:04:15.881662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":435,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":67,"num_write":244,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"265'305","reported_seq":1708,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:29.784726+0000","last_change":"2024-05-30T15:54:18.671642+0000","last_active":"2024-05-30T17:11:29.784726+0000","last_peered":"2024-05-30T17:11:29.784726+0000","last_clean":"2024-05-30T17:11:29.784726+0000","last_became_active":"2024-05-30T15:54:18.671335+0000","last_became_peered":"2024-05-30T15:54:18.671335+0000","last_unstale":"2024-05-30T17:11:29.784726+0000","last_undegraded":"2024-05-30T17:11:29.784726+0000","last_fullsized":"2024-05-30T17:11:29.784726+0000","mapping_epoch":24,"log_start":"265'200","ondisk_log_start":"265'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:01:34.225524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1771,"num_read_kb":1705,"num_write":340,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"265'202","reported_seq":854,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:26.132685+0000","last_change":"2024-05-30T15:54:16.675257+0000","last_active":"2024-05-30T17:12:26.132685+0000","last_peered":"2024-05-30T17:12:26.132685+0000","last_clean":"2024-05-30T17:12:26.132685+0000","last_became_active":"2024-05-30T15:54:16.674678+0000","last_became_peered":"2024-05-30T15:54:16.674678+0000","last_unstale":"2024-05-30T17:12:26.132685+0000","last_undegraded":"2024-05-30T17:12:26.132685+0000","last_fullsized":"2024-05-30T17:12:26.132685+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:49:35.316396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5919937,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":489,"num_read_kb":10347,"num_write":1032,"num_write_kb":23027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"265'559","reported_seq":2275,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.621937+0000","last_change":"2024-05-30T15:54:18.635401+0000","last_active":"2024-05-30T17:13:20.621937+0000","last_peered":"2024-05-30T17:13:20.621937+0000","last_clean":"2024-05-30T17:13:20.621937+0000","last_became_active":"2024-05-30T15:54:18.635092+0000","last_became_peered":"2024-05-30T15:54:18.635092+0000","last_unstale":"2024-05-30T17:13:20.621937+0000","last_undegraded":"2024-05-30T17:13:20.621937+0000","last_fullsized":"2024-05-30T17:13:20.621937+0000","mapping_epoch":24,"log_start":"265'500","ondisk_log_start":"265'500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":500,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:59:22.401801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2950,"num_read_kb":2739,"num_write":700,"num_write_kb":188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"265'159","reported_seq":773,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.559861+0000","last_change":"2024-05-30T15:54:16.658618+0000","last_active":"2024-05-30T17:11:21.559861+0000","last_peered":"2024-05-30T17:11:21.559861+0000","last_clean":"2024-05-30T17:11:21.559861+0000","last_became_active":"2024-05-30T15:54:16.658183+0000","last_became_peered":"2024-05-30T15:54:16.658183+0000","last_unstale":"2024-05-30T17:11:21.559861+0000","last_undegraded":"2024-05-30T17:11:21.559861+0000","last_fullsized":"2024-05-30T17:11:21.559861+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:41:18.697341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7709997,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":302,"num_read_kb":6656,"num_write":920,"num_write_kb":27882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"265'625","reported_seq":2181,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:20.469750+0000","last_change":"2024-05-30T15:54:18.662889+0000","last_active":"2024-05-30T17:12:20.469750+0000","last_peered":"2024-05-30T17:12:20.469750+0000","last_clean":"2024-05-30T17:12:20.469750+0000","last_became_active":"2024-05-30T15:54:18.662159+0000","last_became_peered":"2024-05-30T15:54:18.662159+0000","last_unstale":"2024-05-30T17:12:20.469750+0000","last_undegraded":"2024-05-30T17:12:20.469750+0000","last_fullsized":"2024-05-30T17:12:20.469750+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":600,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:00:12.906772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3191,"num_read_kb":2926,"num_write":899,"num_write_kb":342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"265'205","reported_seq":858,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:22.153006+0000","last_change":"2024-05-30T15:54:16.670343+0000","last_active":"2024-05-30T17:11:22.153006+0000","last_peered":"2024-05-30T17:11:22.153006+0000","last_clean":"2024-05-30T17:11:22.153006+0000","last_became_active":"2024-05-30T15:54:16.652664+0000","last_became_peered":"2024-05-30T15:54:16.652664+0000","last_unstale":"2024-05-30T17:11:22.153006+0000","last_undegraded":"2024-05-30T17:11:22.153006+0000","last_fullsized":"2024-05-30T17:11:22.153006+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":100,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:16:45.470104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5823370,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":446,"num_read_kb":27404,"num_write":1151,"num_write_kb":29020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"265'1297","reported_seq":2932,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:08.892803+0000","last_change":"2024-05-30T15:54:18.643433+0000","last_active":"2024-05-30T17:13:08.892803+0000","last_peered":"2024-05-30T17:13:08.892803+0000","last_clean":"2024-05-30T17:13:08.892803+0000","last_became_active":"2024-05-30T15:54:18.642697+0000","last_became_peered":"2024-05-30T15:54:18.642697+0000","last_unstale":"2024-05-30T17:13:08.892803+0000","last_undegraded":"2024-05-30T17:13:08.892803+0000","last_fullsized":"2024-05-30T17:13:08.892803+0000","mapping_epoch":24,"log_start":"265'1200","ondisk_log_start":"265'1200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":1200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:46:33.063462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":5484,"num_read_kb":4752,"num_write":1832,"num_write_kb":807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"265'255","reported_seq":987,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:36.401960+0000","last_change":"2024-05-30T15:54:16.660154+0000","last_active":"2024-05-30T17:11:36.401960+0000","last_peered":"2024-05-30T17:11:36.401960+0000","last_clean":"2024-05-30T17:11:36.401960+0000","last_became_active":"2024-05-30T15:54:16.659589+0000","last_became_peered":"2024-05-30T15:54:16.659589+0000","last_unstale":"2024-05-30T17:11:36.401960+0000","last_undegraded":"2024-05-30T17:11:36.401960+0000","last_fullsized":"2024-05-30T17:11:36.401960+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:55:43.508632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3249994,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":792,"num_read_kb":47588,"num_write":1563,"num_write_kb":36318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"265'1101","reported_seq":2680,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:20.420887+0000","last_change":"2024-05-30T15:54:18.653159+0000","last_active":"2024-05-30T17:12:20.420887+0000","last_peered":"2024-05-30T17:12:20.420887+0000","last_clean":"2024-05-30T17:12:20.420887+0000","last_became_active":"2024-05-30T15:54:18.652794+0000","last_became_peered":"2024-05-30T15:54:18.652794+0000","last_unstale":"2024-05-30T17:12:20.420887+0000","last_undegraded":"2024-05-30T17:12:20.420887+0000","last_fullsized":"2024-05-30T17:12:20.420887+0000","mapping_epoch":24,"log_start":"265'1000","ondisk_log_start":"265'1000","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":1000,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:14:33.580719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4733,"num_read_kb":4179,"num_write":1497,"num_write_kb":596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"265'223","reported_seq":891,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:06.570629+0000","last_change":"2024-05-30T15:54:54.452063+0000","last_active":"2024-05-30T17:10:06.570629+0000","last_peered":"2024-05-30T17:10:06.570629+0000","last_clean":"2024-05-30T17:10:06.570629+0000","last_became_active":"2024-05-30T15:54:54.451265+0000","last_became_peered":"2024-05-30T15:54:54.451265+0000","last_unstale":"2024-05-30T17:10:06.570629+0000","last_undegraded":"2024-05-30T17:10:06.570629+0000","last_fullsized":"2024-05-30T17:10:06.570629+0000","mapping_epoch":35,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":200,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:41:58.936344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":538764,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":478,"num_read_kb":2675,"num_write":1163,"num_write_kb":43073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"265'385","reported_seq":1948,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:33.943140+0000","last_change":"2024-05-30T15:54:18.671626+0000","last_active":"2024-05-30T17:12:33.943140+0000","last_peered":"2024-05-30T17:12:33.943140+0000","last_clean":"2024-05-30T17:12:33.943140+0000","last_became_active":"2024-05-30T15:54:18.670801+0000","last_became_peered":"2024-05-30T15:54:18.670801+0000","last_unstale":"2024-05-30T17:12:33.943140+0000","last_undegraded":"2024-05-30T17:12:33.943140+0000","last_fullsized":"2024-05-30T17:12:33.943140+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":300,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:08:04.360448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2215,"num_read_kb":2115,"num_write":443,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"265'219","reported_seq":870,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:20.785441+0000","last_change":"2024-05-30T15:54:16.651535+0000","last_active":"2024-05-30T17:11:20.785441+0000","last_peered":"2024-05-30T17:11:20.785441+0000","last_clean":"2024-05-30T17:11:20.785441+0000","last_became_active":"2024-05-30T15:54:16.649854+0000","last_became_peered":"2024-05-30T15:54:16.649854+0000","last_unstale":"2024-05-30T17:11:20.785441+0000","last_undegraded":"2024-05-30T17:11:20.785441+0000","last_fullsized":"2024-05-30T17:11:20.785441+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":200,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:56:05.559917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":635176,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":496,"num_read_kb":6249,"num_write":1148,"num_write_kb":30962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"265'405","reported_seq":1925,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:12.349529+0000","last_change":"2024-05-30T15:54:18.653103+0000","last_active":"2024-05-30T17:13:12.349529+0000","last_peered":"2024-05-30T17:13:12.349529+0000","last_clean":"2024-05-30T17:13:12.349529+0000","last_became_active":"2024-05-30T15:54:18.650832+0000","last_became_peered":"2024-05-30T15:54:18.650832+0000","last_unstale":"2024-05-30T17:13:12.349529+0000","last_undegraded":"2024-05-30T17:13:12.349529+0000","last_fullsized":"2024-05-30T17:13:12.349529+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":300,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:14:11.681719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2240,"num_read_kb":2125,"num_write":507,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"265'185","reported_seq":809,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:06.543039+0000","last_change":"2024-05-30T15:54:16.685235+0000","last_active":"2024-05-30T17:10:06.543039+0000","last_peered":"2024-05-30T17:10:06.543039+0000","last_clean":"2024-05-30T17:10:06.543039+0000","last_became_active":"2024-05-30T15:54:16.684670+0000","last_became_peered":"2024-05-30T15:54:16.684670+0000","last_unstale":"2024-05-30T17:10:06.543039+0000","last_undegraded":"2024-05-30T17:10:06.543039+0000","last_fullsized":"2024-05-30T17:10:06.543039+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:27.693225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5498449,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":371,"num_read_kb":34719,"num_write":1088,"num_write_kb":29777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"265'818","reported_seq":2410,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.705414+0000","last_change":"2024-05-30T15:54:18.669886+0000","last_active":"2024-05-30T17:13:20.705414+0000","last_peered":"2024-05-30T17:13:20.705414+0000","last_clean":"2024-05-30T17:13:20.705414+0000","last_became_active":"2024-05-30T15:54:18.669105+0000","last_became_peered":"2024-05-30T15:54:18.669105+0000","last_unstale":"2024-05-30T17:13:20.705414+0000","last_undegraded":"2024-05-30T17:13:20.705414+0000","last_fullsized":"2024-05-30T17:13:20.705414+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":800,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:33:49.110943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3660,"num_read_kb":3297,"num_write":1114,"num_write_kb":434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"265'218","reported_seq":916,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.337981+0000","last_change":"2024-05-30T15:54:54.417648+0000","last_active":"2024-05-30T17:11:11.337981+0000","last_peered":"2024-05-30T17:11:11.337981+0000","last_clean":"2024-05-30T17:11:11.337981+0000","last_became_active":"2024-05-30T15:54:54.416482+0000","last_became_peered":"2024-05-30T15:54:54.416482+0000","last_unstale":"2024-05-30T17:11:11.337981+0000","last_undegraded":"2024-05-30T17:11:11.337981+0000","last_fullsized":"2024-05-30T17:11:11.337981+0000","mapping_epoch":35,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":200,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:41:15.267706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5736484,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":727,"num_read_kb":137610,"num_write":1203,"num_write_kb":64841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2f","version":"265'385","reported_seq":1972,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:55.924733+0000","last_change":"2024-05-30T15:54:18.657515+0000","last_active":"2024-05-30T17:11:55.924733+0000","last_peered":"2024-05-30T17:11:55.924733+0000","last_clean":"2024-05-30T17:11:55.924733+0000","last_became_active":"2024-05-30T15:54:18.657236+0000","last_became_peered":"2024-05-30T15:54:18.657236+0000","last_unstale":"2024-05-30T17:11:55.924733+0000","last_undegraded":"2024-05-30T17:11:55.924733+0000","last_fullsized":"2024-05-30T17:11:55.924733+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":300,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:04:47.974078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2258,"num_read_kb":2156,"num_write":442,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"265'201","reported_seq":880,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:12.137218+0000","last_change":"2024-05-30T15:54:16.661242+0000","last_active":"2024-05-30T17:10:12.137218+0000","last_peered":"2024-05-30T17:10:12.137218+0000","last_clean":"2024-05-30T17:10:12.137218+0000","last_became_active":"2024-05-30T15:54:16.653037+0000","last_became_peered":"2024-05-30T15:54:16.653037+0000","last_unstale":"2024-05-30T17:10:12.137218+0000","last_undegraded":"2024-05-30T17:10:12.137218+0000","last_fullsized":"2024-05-30T17:10:12.137218+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":100,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:51:51.448976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2129592,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":541,"num_read_kb":29183,"num_write":975,"num_write_kb":32330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"265'330","reported_seq":1827,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:20.410822+0000","last_change":"2024-05-30T15:54:18.636049+0000","last_active":"2024-05-30T17:12:20.410822+0000","last_peered":"2024-05-30T17:12:20.410822+0000","last_clean":"2024-05-30T17:12:20.410822+0000","last_became_active":"2024-05-30T15:54:18.635382+0000","last_became_peered":"2024-05-30T15:54:18.635382+0000","last_unstale":"2024-05-30T17:12:20.410822+0000","last_undegraded":"2024-05-30T17:12:20.410822+0000","last_fullsized":"2024-05-30T17:12:20.410822+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":300,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:50:22.797101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2071,"num_read_kb":1981,"num_write":382,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"265'155","reported_seq":772,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.579598+0000","last_change":"2024-05-30T15:54:16.683072+0000","last_active":"2024-05-30T17:10:46.579598+0000","last_peered":"2024-05-30T17:10:46.579598+0000","last_clean":"2024-05-30T17:10:46.579598+0000","last_became_active":"2024-05-30T15:54:16.682629+0000","last_became_peered":"2024-05-30T15:54:16.682629+0000","last_unstale":"2024-05-30T17:10:46.579598+0000","last_undegraded":"2024-05-30T17:10:46.579598+0000","last_fullsized":"2024-05-30T17:10:46.579598+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:49:01.501502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7203749,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":304,"num_read_kb":22431,"num_write":818,"num_write_kb":23734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"265'743","reported_seq":2312,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.707890+0000","last_change":"2024-05-30T15:54:18.669167+0000","last_active":"2024-05-30T17:13:20.707890+0000","last_peered":"2024-05-30T17:13:20.707890+0000","last_clean":"2024-05-30T17:13:20.707890+0000","last_became_active":"2024-05-30T15:54:18.668748+0000","last_became_peered":"2024-05-30T15:54:18.668748+0000","last_unstale":"2024-05-30T17:13:20.707890+0000","last_undegraded":"2024-05-30T17:13:20.707890+0000","last_fullsized":"2024-05-30T17:13:20.707890+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":700,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:32:48.662121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3385,"num_read_kb":3066,"num_write":980,"num_write_kb":340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"265'161","reported_seq":896,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:58.685815+0000","last_change":"2024-05-30T15:54:16.676031+0000","last_active":"2024-05-30T17:12:58.685815+0000","last_peered":"2024-05-30T17:12:58.685815+0000","last_clean":"2024-05-30T17:12:58.685815+0000","last_became_active":"2024-05-30T15:54:16.674993+0000","last_became_peered":"2024-05-30T15:54:16.674993+0000","last_unstale":"2024-05-30T17:12:58.685815+0000","last_undegraded":"2024-05-30T17:12:58.685815+0000","last_fullsized":"2024-05-30T17:12:58.685815+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":100,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:09:30.788276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11799386,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":563,"num_read_kb":112844,"num_write":921,"num_write_kb":45704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"265'471","reported_seq":2112,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.654074+0000","last_change":"2024-05-30T15:54:18.672327+0000","last_active":"2024-05-30T17:12:44.654074+0000","last_peered":"2024-05-30T17:12:44.654074+0000","last_clean":"2024-05-30T17:12:44.654074+0000","last_became_active":"2024-05-30T15:54:18.671434+0000","last_became_peered":"2024-05-30T15:54:18.671434+0000","last_unstale":"2024-05-30T17:12:44.654074+0000","last_undegraded":"2024-05-30T17:12:44.654074+0000","last_fullsized":"2024-05-30T17:12:44.654074+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":400,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:07:26.503748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2525,"num_read_kb":2390,"num_write":596,"num_write_kb":183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"265'200","reported_seq":854,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.575766+0000","last_change":"2024-05-30T15:54:16.630459+0000","last_active":"2024-05-30T17:10:46.575766+0000","last_peered":"2024-05-30T17:10:46.575766+0000","last_clean":"2024-05-30T17:10:46.575766+0000","last_became_active":"2024-05-30T15:54:16.629843+0000","last_became_peered":"2024-05-30T15:54:16.629843+0000","last_unstale":"2024-05-30T17:10:46.575766+0000","last_undegraded":"2024-05-30T17:10:46.575766+0000","last_fullsized":"2024-05-30T17:10:46.575766+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":100,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:56:59.908915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":905553,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":430,"num_read_kb":42078,"num_write":1148,"num_write_kb":26102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"265'509","reported_seq":2134,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.660255+0000","last_change":"2024-05-30T15:54:18.660822+0000","last_active":"2024-05-30T17:12:44.660255+0000","last_peered":"2024-05-30T17:12:44.660255+0000","last_clean":"2024-05-30T17:12:44.660255+0000","last_became_active":"2024-05-30T15:54:18.660101+0000","last_became_peered":"2024-05-30T15:54:18.660101+0000","last_unstale":"2024-05-30T17:12:44.660255+0000","last_undegraded":"2024-05-30T17:12:44.660255+0000","last_fullsized":"2024-05-30T17:12:44.660255+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":400,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:23:12.134888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2760,"num_read_kb":2546,"num_write":661,"num_write_kb":201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"265'203","reported_seq":838,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.551832+0000","last_change":"2024-05-30T15:54:16.686381+0000","last_active":"2024-05-30T17:10:46.551832+0000","last_peered":"2024-05-30T17:10:46.551832+0000","last_clean":"2024-05-30T17:10:46.551832+0000","last_became_active":"2024-05-30T15:54:16.685969+0000","last_became_peered":"2024-05-30T15:54:16.685969+0000","last_unstale":"2024-05-30T17:10:46.551832+0000","last_undegraded":"2024-05-30T17:10:46.551832+0000","last_fullsized":"2024-05-30T17:10:46.551832+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":100,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:32:55.261508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18487302,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":412,"num_read_kb":18996,"num_write":1141,"num_write_kb":34757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"265'670","reported_seq":2427,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:55.980437+0000","last_change":"2024-05-30T15:54:18.646120+0000","last_active":"2024-05-30T17:11:55.980437+0000","last_peered":"2024-05-30T17:11:55.980437+0000","last_clean":"2024-05-30T17:11:55.980437+0000","last_became_active":"2024-05-30T15:54:18.645400+0000","last_became_peered":"2024-05-30T15:54:18.645400+0000","last_unstale":"2024-05-30T17:11:55.980437+0000","last_undegraded":"2024-05-30T17:11:55.980437+0000","last_fullsized":"2024-05-30T17:11:55.980437+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":600,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:37:15.286662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3447,"num_read_kb":3120,"num_write":968,"num_write_kb":434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"265'206","reported_seq":840,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:06.565221+0000","last_change":"2024-05-30T15:54:16.664235+0000","last_active":"2024-05-30T17:11:06.565221+0000","last_peered":"2024-05-30T17:11:06.565221+0000","last_clean":"2024-05-30T17:11:06.565221+0000","last_became_active":"2024-05-30T15:54:16.663255+0000","last_became_peered":"2024-05-30T15:54:16.663255+0000","last_unstale":"2024-05-30T17:11:06.565221+0000","last_undegraded":"2024-05-30T17:11:06.565221+0000","last_fullsized":"2024-05-30T17:11:06.565221+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":100,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:14:20.452277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11362066,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":420,"num_read_kb":27117,"num_write":1157,"num_write_kb":19484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"265'360","reported_seq":1779,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:08.893650+0000","last_change":"2024-05-30T15:54:18.658517+0000","last_active":"2024-05-30T17:13:08.893650+0000","last_peered":"2024-05-30T17:13:08.893650+0000","last_clean":"2024-05-30T17:13:08.893650+0000","last_became_active":"2024-05-30T15:54:18.658230+0000","last_became_peered":"2024-05-30T15:54:18.658230+0000","last_unstale":"2024-05-30T17:13:08.893650+0000","last_undegraded":"2024-05-30T17:13:08.893650+0000","last_fullsized":"2024-05-30T17:13:08.893650+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":300,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:13:42.767006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1934,"num_read_kb":1839,"num_write":414,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"265'166","reported_seq":779,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.558787+0000","last_change":"2024-05-30T15:54:16.655014+0000","last_active":"2024-05-30T17:11:16.558787+0000","last_peered":"2024-05-30T17:11:16.558787+0000","last_clean":"2024-05-30T17:11:16.558787+0000","last_became_active":"2024-05-30T15:54:16.654365+0000","last_became_peered":"2024-05-30T15:54:16.654365+0000","last_unstale":"2024-05-30T17:11:16.558787+0000","last_undegraded":"2024-05-30T17:11:16.558787+0000","last_fullsized":"2024-05-30T17:11:16.558787+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":100,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:31:10.427380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14702715,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":354,"num_read_kb":31154,"num_write":911,"num_write_kb":41557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"265'698","reported_seq":2128,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.605807+0000","last_change":"2024-05-30T15:54:18.661710+0000","last_active":"2024-05-30T17:12:44.605807+0000","last_peered":"2024-05-30T17:12:44.605807+0000","last_clean":"2024-05-30T17:12:44.605807+0000","last_became_active":"2024-05-30T15:54:18.660543+0000","last_became_peered":"2024-05-30T15:54:18.660543+0000","last_unstale":"2024-05-30T17:12:44.605807+0000","last_undegraded":"2024-05-30T17:12:44.605807+0000","last_fullsized":"2024-05-30T17:12:44.605807+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":600,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:39:46.092642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3130,"num_read_kb":2812,"num_write":919,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"265'195","reported_seq":837,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:17.152049+0000","last_change":"2024-05-30T15:54:16.666386+0000","last_active":"2024-05-30T17:11:17.152049+0000","last_peered":"2024-05-30T17:11:17.152049+0000","last_clean":"2024-05-30T17:11:17.152049+0000","last_became_active":"2024-05-30T15:54:16.661983+0000","last_became_peered":"2024-05-30T15:54:16.661983+0000","last_unstale":"2024-05-30T17:11:17.152049+0000","last_undegraded":"2024-05-30T17:11:17.152049+0000","last_fullsized":"2024-05-30T17:11:17.152049+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":100,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:48:35.955662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8616050,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":413,"num_read_kb":7620,"num_write":1148,"num_write_kb":9531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"265'152","reported_seq":782,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:10.550602+0000","last_change":"2024-05-30T15:55:54.768358+0000","last_active":"2024-05-30T17:13:10.550602+0000","last_peered":"2024-05-30T17:13:10.550602+0000","last_clean":"2024-05-30T17:13:10.550602+0000","last_became_active":"2024-05-30T15:55:54.767665+0000","last_became_peered":"2024-05-30T15:55:54.767665+0000","last_unstale":"2024-05-30T17:13:10.550602+0000","last_undegraded":"2024-05-30T17:13:10.550602+0000","last_fullsized":"2024-05-30T17:13:10.550602+0000","mapping_epoch":37,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":100,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:31:33.212759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11250466,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":358,"num_read_kb":14674,"num_write":888,"num_write_kb":17371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,0],"acting":[5,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"265'765","reported_seq":2419,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:12.341733+0000","last_change":"2024-05-30T15:54:18.660816+0000","last_active":"2024-05-30T17:11:12.341733+0000","last_peered":"2024-05-30T17:11:12.341733+0000","last_clean":"2024-05-30T17:11:12.341733+0000","last_became_active":"2024-05-30T15:54:18.660501+0000","last_became_peered":"2024-05-30T15:54:18.660501+0000","last_unstale":"2024-05-30T17:11:12.341733+0000","last_undegraded":"2024-05-30T17:11:12.341733+0000","last_fullsized":"2024-05-30T17:11:12.341733+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":700,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:58:48.235676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3837,"num_read_kb":3483,"num_write":1021,"num_write_kb":383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"265'179","reported_seq":830,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.790863+0000","last_change":"2024-05-30T15:54:16.668650+0000","last_active":"2024-05-30T17:11:16.790863+0000","last_peered":"2024-05-30T17:11:16.790863+0000","last_clean":"2024-05-30T17:11:16.790863+0000","last_became_active":"2024-05-30T15:54:16.668178+0000","last_became_peered":"2024-05-30T15:54:16.668178+0000","last_unstale":"2024-05-30T17:11:16.790863+0000","last_undegraded":"2024-05-30T17:11:16.790863+0000","last_fullsized":"2024-05-30T17:11:16.790863+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":100,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:22:19.325731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14464063,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":423,"num_read_kb":57498,"num_write":996,"num_write_kb":21705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"265'474","reported_seq":2144,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.703669+0000","last_change":"2024-05-30T15:54:18.656743+0000","last_active":"2024-05-30T17:13:20.703669+0000","last_peered":"2024-05-30T17:13:20.703669+0000","last_clean":"2024-05-30T17:13:20.703669+0000","last_became_active":"2024-05-30T15:54:18.655872+0000","last_became_peered":"2024-05-30T15:54:18.655872+0000","last_unstale":"2024-05-30T17:13:20.703669+0000","last_undegraded":"2024-05-30T17:13:20.703669+0000","last_fullsized":"2024-05-30T17:13:20.703669+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":400,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:51:05.525906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2571,"num_read_kb":2440,"num_write":587,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5999,"num_read_kb":4309,"num_write":2135,"num_write_kb":880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1140,"ondisk_log_size":1140,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19930,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":13226,"num_read_kb":13032,"num_write":15987,"num_write_kb":4690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":704512,"data_stored":39860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1567,"ondisk_log_size":1567,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":405,"ondisk_log_size":405,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":366949,"num_objects":197,"num_object_clones":0,"num_object_copies":394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":197,"num_whiteouts":0,"num_read":9271,"num_read_kb":11128,"num_write":6557,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":745472,"data_stored":181518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1889,"ondisk_log_size":1889,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1624,"num_read_kb":1934,"num_write":20,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":242,"num_object_clones":0,"num_object_copies":484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":242,"num_whiteouts":0,"num_read":191181,"num_read_kb":174247,"num_write":51900,"num_write_kb":17923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":242,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4116,"ondisk_log_size":4116,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":506410238,"num_objects":484,"num_object_clones":0,"num_object_copies":1452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":484,"num_whiteouts":0,"num_read":31813,"num_read_kb":2124772,"num_write":69681,"num_write_kb":2069422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":762998784,"data_stored":762998784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4465,"ondisk_log_size":4465,"up":192,"acting":192,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":81623142,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":117,"num_read_kb":182367,"num_write":133,"num_write_kb":122722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163258368,"data_stored":163246284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":81623142,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":115,"num_read_kb":181098,"num_write":133,"num_write_kb":122722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163258368,"data_stored":163246284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575810,"num_pgs":106,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":391144,"kb_used_data":129504,"kb_used_omap":1,"kb_used_meta":261630,"kb_avail":93980696,"statfs":{"total":96636764160,"available":96236232704,"internally_reserved":0,"allocated":132612096,"data_stored":130525109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":267909578},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575809,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395632,"kb_used_data":147880,"kb_used_omap":1,"kb_used_meta":247742,"kb_avail":93976208,"statfs":{"total":96636764160,"available":96231636992,"internally_reserved":0,"allocated":151429120,"data_stored":149446612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":253688262},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575809,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":220408,"kb_used_data":135088,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94151432,"statfs":{"total":96636764160,"available":96411066368,"internally_reserved":0,"allocated":138330112,"data_stored":136241754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87357893},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575809,"num_pgs":104,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":225900,"kb_used_data":132260,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94145940,"statfs":{"total":96636764160,"available":96405442560,"internally_reserved":0,"allocated":135434240,"data_stored":133486846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":95877574},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575807,"num_pgs":124,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":376304,"kb_used_data":150760,"kb_used_omap":1,"kb_used_meta":225534,"kb_avail":93995536,"statfs":{"total":96636764160,"available":96251428864,"internally_reserved":0,"allocated":154378240,"data_stored":152309513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":230947271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575807,"num_pgs":91,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":179456,"kb_used_data":108024,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94192384,"statfs":{"total":96636764160,"available":96453001216,"internally_reserved":0,"allocated":110616576,"data_stored":108657566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575811,"num_pgs":112,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":263376,"kb_used_data":146184,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94108464,"statfs":{"total":96636764160,"available":96367067136,"internally_reserved":0,"allocated":149692416,"data_stored":147687855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575806,"num_pgs":109,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":379024,"kb_used_data":149320,"kb_used_omap":1,"kb_used_meta":229694,"kb_avail":93992816,"statfs":{"total":96636764160,"available":96248643584,"internally_reserved":0,"allocated":152903680,"data_stored":150873069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":235207109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":114364416,"data_stored":114364416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":99225600,"data_stored":99225600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":74932224,"data_stored":74932224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":93523968,"data_stored":93523968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":97972224,"data_stored":97972224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":92889088,"data_stored":92889088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":109338624,"data_stored":109338624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":80752640,"data_stored":80752640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":18103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":28335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":7835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":30127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":10287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":151552,"data_stored":39861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":15040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":31930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":4787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":5121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":4696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":6691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":5459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":3918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":7193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32555008,"data_stored":32554271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22020096,"data_stored":22020096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18591744,"data_stored":18588432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":23322624,"data_stored":23318684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16900096,"data_stored":16897366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":21557248,"data_stored":21555883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7340032,"data_stored":7340032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20971520,"data_stored":20971520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24117248,"data_stored":24117248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12836864,"data_stored":12832924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":33140736,"data_stored":33138634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16314368,"data_stored":16313003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":19460096,"data_stored":19458731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":30457856,"data_stored":30457119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":25882624,"data_stored":25880049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T17:13:39.805 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T17:13:40.144 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:40.145 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":265,"fsid":"87f2315a-4430-477b-ae32-1664164ce9e8","created":"2024-05-30T15:53:48.501791+0000","modified":"2024-05-30T16:34:56.836938+0000","last_up_change":"2024-05-30T15:54:00.488066+0000","last_in_change":"2024-05-30T15:53:51.589084+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":15,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":68,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T15:54:01.322096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-30T15:54:07.446935+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.buckets.data","create_time":"2024-05-30T15:54:14.611790+0000","flags":1,"flags_names":"hashpspool","type":3,"size":3,"min_size":2,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"client.0","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.buckets.index","create_time":"2024-05-30T15:54:16.987559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":".rgw.root","create_time":"2024-05-30T15:54:18.730106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.log","create_time":"2024-05-30T15:54:20.973977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"default.rgw.control","create_time":"2024-05-30T15:54:22.687188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"default.rgw.meta","create_time":"2024-05-30T15:54:24.800212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":65,"pool_name":"default.rgw.buckets.non-ec","create_time":"2024-05-30T16:12:49.488694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"254","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":66,"pool_name":"default.rgw.buckets.data.frozen","create_time":"2024-05-30T16:17:29.417030+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"257","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":67,"pool_name":"default.rgw.buckets.data.lukewarm","create_time":"2024-05-30T16:17:33.634680+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"260","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":68,"pool_name":"default.rgw.otp","create_time":"2024-05-30T16:34:54.143659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"265","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9b54fda-6c4d-49b5-b6f7-9baf01edc4fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":177258275},{"type":"v1","addr":"172.21.15.6:6801","nonce":177258275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":177258275},{"type":"v1","addr":"172.21.15.6:6803","nonce":177258275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":177258275},{"type":"v1","addr":"172.21.15.6:6807","nonce":177258275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":177258275},{"type":"v1","addr":"172.21.15.6:6805","nonce":177258275}]},"public_addr":"172.21.15.6:6801/177258275","cluster_addr":"172.21.15.6:6803/177258275","heartbeat_back_addr":"172.21.15.6:6807/177258275","heartbeat_front_addr":"172.21.15.6:6805/177258275","state":["exists","up"]},{"osd":1,"uuid":"476e6529-66eb-4479-8ca9-b646750fb07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":1897362715},{"type":"v1","addr":"172.21.15.6:6809","nonce":1897362715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":1897362715},{"type":"v1","addr":"172.21.15.6:6811","nonce":1897362715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":1897362715},{"type":"v1","addr":"172.21.15.6:6815","nonce":1897362715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":1897362715},{"type":"v1","addr":"172.21.15.6:6813","nonce":1897362715}]},"public_addr":"172.21.15.6:6809/1897362715","cluster_addr":"172.21.15.6:6811/1897362715","heartbeat_back_addr":"172.21.15.6:6815/1897362715","heartbeat_front_addr":"172.21.15.6:6813/1897362715","state":["exists","up"]},{"osd":2,"uuid":"c8461074-51a0-4209-aed4-e05343440267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":2704410641},{"type":"v1","addr":"172.21.15.6:6817","nonce":2704410641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2704410641},{"type":"v1","addr":"172.21.15.6:6820","nonce":2704410641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6825","nonce":2704410641},{"type":"v1","addr":"172.21.15.6:6827","nonce":2704410641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2704410641},{"type":"v1","addr":"172.21.15.6:6824","nonce":2704410641}]},"public_addr":"172.21.15.6:6817/2704410641","cluster_addr":"172.21.15.6:6820/2704410641","heartbeat_back_addr":"172.21.15.6:6827/2704410641","heartbeat_front_addr":"172.21.15.6:6824/2704410641","state":["exists","up"]},{"osd":3,"uuid":"c4775e06-d8e2-44c0-a0c8-9f4e8e21f5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":1141406589},{"type":"v1","addr":"172.21.15.6:6821","nonce":1141406589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6823","nonce":1141406589},{"type":"v1","addr":"172.21.15.6:6826","nonce":1141406589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1141406589},{"type":"v1","addr":"172.21.15.6:6831","nonce":1141406589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1141406589},{"type":"v1","addr":"172.21.15.6:6829","nonce":1141406589}]},"public_addr":"172.21.15.6:6821/1141406589","cluster_addr":"172.21.15.6:6826/1141406589","heartbeat_back_addr":"172.21.15.6:6831/1141406589","heartbeat_front_addr":"172.21.15.6:6829/1141406589","state":["exists","up"]},{"osd":4,"uuid":"bfc9fd12-63b8-4711-a306-23a17a9aaf0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":986196749},{"type":"v1","addr":"172.21.15.204:6808","nonce":986196749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":986196749},{"type":"v1","addr":"172.21.15.204:6811","nonce":986196749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":986196749},{"type":"v1","addr":"172.21.15.204:6815","nonce":986196749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":986196749},{"type":"v1","addr":"172.21.15.204:6813","nonce":986196749}]},"public_addr":"172.21.15.204:6808/986196749","cluster_addr":"172.21.15.204:6811/986196749","heartbeat_back_addr":"172.21.15.204:6815/986196749","heartbeat_front_addr":"172.21.15.204:6813/986196749","state":["exists","up"]},{"osd":5,"uuid":"e21db988-15e6-488e-8bc1-e5ba0a2f354b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2677861390},{"type":"v1","addr":"172.21.15.204:6817","nonce":2677861390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2677861390},{"type":"v1","addr":"172.21.15.204:6819","nonce":2677861390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2677861390},{"type":"v1","addr":"172.21.15.204:6824","nonce":2677861390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2677861390},{"type":"v1","addr":"172.21.15.204:6821","nonce":2677861390}]},"public_addr":"172.21.15.204:6817/2677861390","cluster_addr":"172.21.15.204:6819/2677861390","heartbeat_back_addr":"172.21.15.204:6824/2677861390","heartbeat_front_addr":"172.21.15.204:6821/2677861390","state":["exists","up"]},{"osd":6,"uuid":"85a75312-8faf-4279-9301-134fcb19cbd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3687190393},{"type":"v1","addr":"172.21.15.204:6801","nonce":3687190393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3687190393},{"type":"v1","addr":"172.21.15.204:6803","nonce":3687190393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":3687190393},{"type":"v1","addr":"172.21.15.204:6809","nonce":3687190393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3687190393},{"type":"v1","addr":"172.21.15.204:6805","nonce":3687190393}]},"public_addr":"172.21.15.204:6801/3687190393","cluster_addr":"172.21.15.204:6803/3687190393","heartbeat_back_addr":"172.21.15.204:6809/3687190393","heartbeat_front_addr":"172.21.15.204:6805/3687190393","state":["exists","up"]},{"osd":7,"uuid":"f0550e43-5653-4433-8f78-17af5e82316d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":3156311527},{"type":"v1","addr":"172.21.15.204:6825","nonce":3156311527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3156311527},{"type":"v1","addr":"172.21.15.204:6827","nonce":3156311527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3156311527},{"type":"v1","addr":"172.21.15.204:6831","nonce":3156311527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3156311527},{"type":"v1","addr":"172.21.15.204:6829","nonce":3156311527}]},"public_addr":"172.21.15.204:6825/3156311527","cluster_addr":"172.21.15.204:6827/3156311527","heartbeat_back_addr":"172.21.15.204:6831/3156311527","heartbeat_front_addr":"172.21.15.204:6829/3156311527","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:57.123142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:57.210132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:57.107006+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:57.217713+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:57.079193+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:57.281733+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:57.111373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T15:53:56.985566+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.13","mappings":[{"from":6,"to":1},{"from":7,"to":2}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.1b","mappings":[{"from":0,"to":2}]},{"pgid":"3.1d","mappings":[{"from":4,"to":2}]},{"pgid":"3.24","mappings":[{"from":4,"to":1}]},{"pgid":"3.27","mappings":[{"from":3,"to":1}]},{"pgid":"3.30","mappings":[{"from":4,"to":2}]},{"pgid":"4.1a","mappings":[{"from":3,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]},{"pgid":"67.c","mappings":[{"from":3,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"client.0":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":26,"snaps":[{"begin":1,"length":2}]},{"pool":31,"snaps":[{"begin":1,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T17:13:41.166 INFO:tasks.ceph:Scrubbing osd.0 2024-05-30T17:13:41.167 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-30T17:13:41.333 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-30T17:13:41.333 INFO:teuthology.orchestra.run.smithi006.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-30T17:13:41.333 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-30T17:13:41.350 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-30T17:13:41.645 INFO:teuthology.orchestra.run.smithi006.stderr:instructed osd(s) 0 to deep-scrub 2024-05-30T17:13:41.665 INFO:tasks.ceph:Scrubbing osd.1 2024-05-30T17:13:41.665 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-30T17:13:41.822 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-30T17:13:41.823 INFO:teuthology.orchestra.run.smithi006.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-30T17:13:41.823 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-30T17:13:41.838 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-30T17:13:42.127 INFO:teuthology.orchestra.run.smithi006.stderr:instructed osd(s) 1 to deep-scrub 2024-05-30T17:13:42.145 INFO:tasks.ceph:Scrubbing osd.2 2024-05-30T17:13:42.146 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-30T17:13:42.295 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-30T17:13:42.295 INFO:teuthology.orchestra.run.smithi006.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-30T17:13:42.296 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-30T17:13:42.311 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-30T17:13:42.599 INFO:teuthology.orchestra.run.smithi006.stderr:instructed osd(s) 2 to deep-scrub 2024-05-30T17:13:42.620 INFO:tasks.ceph:Scrubbing osd.3 2024-05-30T17:13:42.620 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-30T17:13:42.770 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-30T17:13:42.770 INFO:teuthology.orchestra.run.smithi006.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-30T17:13:42.770 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-30T17:13:42.785 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-30T17:13:43.072 INFO:teuthology.orchestra.run.smithi006.stderr:instructed osd(s) 3 to deep-scrub 2024-05-30T17:13:43.093 INFO:tasks.ceph:Scrubbing osd.4 2024-05-30T17:13:43.093 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-30T17:13:43.251 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-30T17:13:43.252 INFO:teuthology.orchestra.run.smithi006.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-30T17:13:43.252 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-30T17:13:43.268 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-30T17:13:43.564 INFO:teuthology.orchestra.run.smithi006.stderr:instructed osd(s) 4 to deep-scrub 2024-05-30T17:13:43.585 INFO:tasks.ceph:Scrubbing osd.5 2024-05-30T17:13:43.585 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-30T17:13:43.748 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-30T17:13:43.748 INFO:teuthology.orchestra.run.smithi006.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-30T17:13:43.749 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-30T17:13:43.766 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-30T17:13:44.065 INFO:teuthology.orchestra.run.smithi006.stderr:instructed osd(s) 5 to deep-scrub 2024-05-30T17:13:44.085 INFO:tasks.ceph:Scrubbing osd.6 2024-05-30T17:13:44.085 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-30T17:13:44.243 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-30T17:13:44.243 INFO:teuthology.orchestra.run.smithi006.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-30T17:13:44.243 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-30T17:13:44.259 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-30T17:13:44.564 INFO:teuthology.orchestra.run.smithi006.stderr:instructed osd(s) 6 to deep-scrub 2024-05-30T17:13:44.584 INFO:tasks.ceph:Scrubbing osd.7 2024-05-30T17:13:44.584 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-30T17:13:44.739 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-05-30T17:13:44.740 INFO:teuthology.orchestra.run.smithi006.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-30T17:13:44.740 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-05-30T17:13:44.755 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-30T17:13:45.050 INFO:teuthology.orchestra.run.smithi006.stderr:instructed osd(s) 7 to deep-scrub 2024-05-30T17:13:45.069 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T17:13:45.407 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:13:45.422 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-05-30T17:13:45.439 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":2648,"stamp":"2024-05-30T17:13:44.444750+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":191800421,"num_objects":837,"num_object_clones":0,"num_object_copies":1935,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":837,"num_whiteouts":0,"num_read":254020,"num_read_kb":2693627,"num_write":147058,"num_write_kb":2342860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":289,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13778,"ondisk_log_size":13778,"up":850,"acting":850,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":850,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1669580,"kb_used_data":332740,"kb_used_omap":12,"kb_used_meta":1336307,"kb_avail":753305140,"statfs":{"total":773094113280,"available":771384463360,"internally_reserved":0,"allocated":340725760,"data_stored":324874602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":1368378929},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4],"upper_bound":2},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-478835141,"num_objects":-259,"num_object_clones":0,"num_object_copies":-741,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-259,"num_whiteouts":0,"num_read":596,"num_read_kb":673,"num_write":393,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008250"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":27,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:57.133920+0000","last_change":"2024-05-30T16:34:55.869930+0000","last_active":"2024-05-30T17:09:57.133920+0000","last_peered":"2024-05-30T17:09:57.133920+0000","last_clean":"2024-05-30T17:09:57.133920+0000","last_became_active":"2024-05-30T16:34:55.869226+0000","last_became_peered":"2024-05-30T16:34:55.869226+0000","last_unstale":"2024-05-30T17:09:57.133920+0000","last_undegraded":"2024-05-30T17:09:57.133920+0000","last_fullsized":"2024-05-30T17:09:57.133920+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:37:02.394930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"265'2","reported_seq":48,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:36.971525+0000","last_change":"2024-05-30T16:17:35.249630+0000","last_active":"2024-05-30T17:12:36.971525+0000","last_peered":"2024-05-30T17:12:36.971525+0000","last_clean":"2024-05-30T17:12:36.971525+0000","last_became_active":"2024-05-30T16:17:35.248740+0000","last_became_peered":"2024-05-30T16:17:35.248740+0000","last_unstale":"2024-05-30T17:12:36.971525+0000","last_undegraded":"2024-05-30T17:12:36.971525+0000","last_fullsized":"2024-05-30T17:12:36.971525+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:05:10.176331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"265'8","reported_seq":66,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:31.604038+0000","last_change":"2024-05-30T16:17:31.184110+0000","last_active":"2024-05-30T17:12:31.604038+0000","last_peered":"2024-05-30T17:12:31.604038+0000","last_clean":"2024-05-30T17:12:31.604038+0000","last_became_active":"2024-05-30T16:17:31.183349+0000","last_became_peered":"2024-05-30T16:17:31.183349+0000","last_unstale":"2024-05-30T17:12:31.604038+0000","last_undegraded":"2024-05-30T17:12:31.604038+0000","last_fullsized":"2024-05-30T17:12:31.604038+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:06:40.372126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":16384,"num_write":8,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"265'45","reported_seq":184,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.581199+0000","last_change":"2024-05-30T16:12:50.806342+0000","last_active":"2024-05-30T17:11:11.581199+0000","last_peered":"2024-05-30T17:11:11.581199+0000","last_clean":"2024-05-30T17:11:11.581199+0000","last_became_active":"2024-05-30T16:12:50.803352+0000","last_became_peered":"2024-05-30T16:12:50.803352+0000","last_unstale":"2024-05-30T17:11:11.581199+0000","last_undegraded":"2024-05-30T17:11:11.581199+0000","last_fullsized":"2024-05-30T17:11:11.581199+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:23:40.247207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":225,"num_read_kb":156,"num_write":101,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":27,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.543407+0000","last_change":"2024-05-30T16:34:55.887503+0000","last_active":"2024-05-30T17:09:56.543407+0000","last_peered":"2024-05-30T17:09:56.543407+0000","last_clean":"2024-05-30T17:09:56.543407+0000","last_became_active":"2024-05-30T16:34:55.886731+0000","last_became_peered":"2024-05-30T16:34:55.886731+0000","last_unstale":"2024-05-30T17:09:56.543407+0000","last_undegraded":"2024-05-30T17:09:56.543407+0000","last_fullsized":"2024-05-30T17:09:56.543407+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:47:35.283261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"0'0","reported_seq":44,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:06.732701+0000","last_change":"2024-05-30T16:17:35.248384+0000","last_active":"2024-05-30T17:10:06.732701+0000","last_peered":"2024-05-30T17:10:06.732701+0000","last_clean":"2024-05-30T17:10:06.732701+0000","last_became_active":"2024-05-30T16:17:35.247810+0000","last_became_peered":"2024-05-30T16:17:35.247810+0000","last_unstale":"2024-05-30T17:10:06.732701+0000","last_undegraded":"2024-05-30T17:10:06.732701+0000","last_fullsized":"2024-05-30T17:10:06.732701+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:56:26.172595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"0'0","reported_seq":50,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:01.773778+0000","last_change":"2024-05-30T16:17:31.215393+0000","last_active":"2024-05-30T17:10:01.773778+0000","last_peered":"2024-05-30T17:10:01.773778+0000","last_clean":"2024-05-30T17:10:01.773778+0000","last_became_active":"2024-05-30T16:17:31.212337+0000","last_became_peered":"2024-05-30T16:17:31.212337+0000","last_unstale":"2024-05-30T17:10:01.773778+0000","last_undegraded":"2024-05-30T17:10:01.773778+0000","last_fullsized":"2024-05-30T17:10:01.773778+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:48:30.989819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"265'55","reported_seq":210,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:06.556856+0000","last_change":"2024-05-30T16:12:50.810909+0000","last_active":"2024-05-30T17:11:06.556856+0000","last_peered":"2024-05-30T17:11:06.556856+0000","last_clean":"2024-05-30T17:11:06.556856+0000","last_became_active":"2024-05-30T16:12:50.810207+0000","last_became_peered":"2024-05-30T16:12:50.810207+0000","last_unstale":"2024-05-30T17:11:06.556856+0000","last_undegraded":"2024-05-30T17:11:06.556856+0000","last_fullsized":"2024-05-30T17:11:06.556856+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:37:48.188833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":277,"num_read_kb":197,"num_write":99,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.556933+0000","last_change":"2024-05-30T16:34:55.876451+0000","last_active":"2024-05-30T17:11:11.556933+0000","last_peered":"2024-05-30T17:11:11.556933+0000","last_clean":"2024-05-30T17:11:11.556933+0000","last_became_active":"2024-05-30T16:34:55.874359+0000","last_became_peered":"2024-05-30T16:34:55.874359+0000","last_unstale":"2024-05-30T17:11:11.556933+0000","last_undegraded":"2024-05-30T17:11:11.556933+0000","last_fullsized":"2024-05-30T17:11:11.556933+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:37:31.196070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"0'0","reported_seq":48,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:41.561910+0000","last_change":"2024-05-30T16:17:35.743986+0000","last_active":"2024-05-30T17:09:41.561910+0000","last_peered":"2024-05-30T17:09:41.561910+0000","last_clean":"2024-05-30T17:09:41.561910+0000","last_became_active":"2024-05-30T16:17:35.743231+0000","last_became_peered":"2024-05-30T16:17:35.743231+0000","last_unstale":"2024-05-30T17:09:41.561910+0000","last_undegraded":"2024-05-30T17:09:41.561910+0000","last_fullsized":"2024-05-30T17:09:41.561910+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:52:11.845318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"265'3","reported_seq":60,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:46.564402+0000","last_change":"2024-05-30T16:17:31.205642+0000","last_active":"2024-05-30T17:11:46.564402+0000","last_peered":"2024-05-30T17:11:46.564402+0000","last_clean":"2024-05-30T17:11:46.564402+0000","last_became_active":"2024-05-30T16:17:31.204806+0000","last_became_peered":"2024-05-30T16:17:31.204806+0000","last_unstale":"2024-05-30T17:11:46.564402+0000","last_undegraded":"2024-05-30T17:11:46.564402+0000","last_fullsized":"2024-05-30T17:11:46.564402+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:50:34.702667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"265'29","reported_seq":134,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.570270+0000","last_change":"2024-05-30T16:12:50.811279+0000","last_active":"2024-05-30T17:10:21.570270+0000","last_peered":"2024-05-30T17:10:21.570270+0000","last_clean":"2024-05-30T17:10:21.570270+0000","last_became_active":"2024-05-30T16:12:50.810062+0000","last_became_peered":"2024-05-30T16:12:50.810062+0000","last_unstale":"2024-05-30T17:10:21.570270+0000","last_undegraded":"2024-05-30T17:10:21.570270+0000","last_fullsized":"2024-05-30T17:10:21.570270+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:23:45.234198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":98,"num_write":65,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":27,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:08:56.553929+0000","last_change":"2024-05-30T16:34:55.888292+0000","last_active":"2024-05-30T17:08:56.553929+0000","last_peered":"2024-05-30T17:08:56.553929+0000","last_clean":"2024-05-30T17:08:56.553929+0000","last_became_active":"2024-05-30T16:34:55.887532+0000","last_became_peered":"2024-05-30T16:34:55.887532+0000","last_unstale":"2024-05-30T17:08:56.553929+0000","last_undegraded":"2024-05-30T17:08:56.553929+0000","last_fullsized":"2024-05-30T17:08:56.553929+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:47:12.792997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"265'2","reported_seq":53,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:51.594565+0000","last_change":"2024-05-30T16:17:35.746712+0000","last_active":"2024-05-30T17:11:51.594565+0000","last_peered":"2024-05-30T17:11:51.594565+0000","last_clean":"2024-05-30T17:11:51.594565+0000","last_became_active":"2024-05-30T16:17:35.745709+0000","last_became_peered":"2024-05-30T16:17:35.745709+0000","last_unstale":"2024-05-30T17:11:51.594565+0000","last_undegraded":"2024-05-30T17:11:51.594565+0000","last_fullsized":"2024-05-30T17:11:51.594565+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:58:32.058273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1142,"num_write":2,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1f","version":"265'3","reported_seq":57,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:41.753957+0000","last_change":"2024-05-30T16:17:31.183141+0000","last_active":"2024-05-30T17:11:41.753957+0000","last_peered":"2024-05-30T17:11:41.753957+0000","last_clean":"2024-05-30T17:11:41.753957+0000","last_became_active":"2024-05-30T16:17:31.182344+0000","last_became_peered":"2024-05-30T16:17:31.182344+0000","last_unstale":"2024-05-30T17:11:41.753957+0000","last_undegraded":"2024-05-30T17:11:41.753957+0000","last_fullsized":"2024-05-30T17:11:41.753957+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:20:08.705563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"260'6","reported_seq":76,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.573922+0000","last_change":"2024-05-30T16:12:50.830384+0000","last_active":"2024-05-30T17:10:21.573922+0000","last_peered":"2024-05-30T17:10:21.573922+0000","last_clean":"2024-05-30T17:10:21.573922+0000","last_became_active":"2024-05-30T16:12:50.826974+0000","last_became_peered":"2024-05-30T16:12:50.826974+0000","last_unstale":"2024-05-30T17:10:21.573922+0000","last_undegraded":"2024-05-30T17:10:21.573922+0000","last_fullsized":"2024-05-30T17:10:21.573922+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:27:49.529929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":24,"num_write":11,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":31,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:06.583992+0000","last_change":"2024-05-30T16:34:55.899516+0000","last_active":"2024-05-30T17:11:06.583992+0000","last_peered":"2024-05-30T17:11:06.583992+0000","last_clean":"2024-05-30T17:11:06.583992+0000","last_became_active":"2024-05-30T16:34:55.898864+0000","last_became_peered":"2024-05-30T16:34:55.898864+0000","last_unstale":"2024-05-30T17:11:06.583992+0000","last_undegraded":"2024-05-30T17:11:06.583992+0000","last_fullsized":"2024-05-30T17:11:06.583992+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:07:16.296580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"265'7","reported_seq":63,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:21.981834+0000","last_change":"2024-05-30T16:17:35.233543+0000","last_active":"2024-05-30T17:13:21.981834+0000","last_peered":"2024-05-30T17:13:21.981834+0000","last_clean":"2024-05-30T17:13:21.981834+0000","last_became_active":"2024-05-30T16:17:35.232744+0000","last_became_peered":"2024-05-30T16:17:35.232744+0000","last_unstale":"2024-05-30T17:13:21.981834+0000","last_undegraded":"2024-05-30T17:13:21.981834+0000","last_fullsized":"2024-05-30T17:13:21.981834+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:40:45.244827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":23554,"num_write":6,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"0'0","reported_seq":51,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:41.574504+0000","last_change":"2024-05-30T16:17:31.186983+0000","last_active":"2024-05-30T17:10:41.574504+0000","last_peered":"2024-05-30T17:10:41.574504+0000","last_clean":"2024-05-30T17:10:41.574504+0000","last_became_active":"2024-05-30T16:17:31.186229+0000","last_became_peered":"2024-05-30T16:17:31.186229+0000","last_unstale":"2024-05-30T17:10:41.574504+0000","last_undegraded":"2024-05-30T17:10:41.574504+0000","last_fullsized":"2024-05-30T17:10:41.574504+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:33:19.747591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"265'27","reported_seq":134,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.573828+0000","last_change":"2024-05-30T16:12:50.830804+0000","last_active":"2024-05-30T17:10:21.573828+0000","last_peered":"2024-05-30T17:10:21.573828+0000","last_clean":"2024-05-30T17:10:21.573828+0000","last_became_active":"2024-05-30T16:12:50.827008+0000","last_became_peered":"2024-05-30T16:12:50.827008+0000","last_unstale":"2024-05-30T17:10:21.573828+0000","last_undegraded":"2024-05-30T17:10:21.573828+0000","last_fullsized":"2024-05-30T17:10:21.573828+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:26:44.426182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":100,"num_write":57,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.778503+0000","last_change":"2024-05-30T16:34:55.870968+0000","last_active":"2024-05-30T17:10:21.778503+0000","last_peered":"2024-05-30T17:10:21.778503+0000","last_clean":"2024-05-30T17:10:21.778503+0000","last_became_active":"2024-05-30T16:34:55.870139+0000","last_became_peered":"2024-05-30T16:34:55.870139+0000","last_unstale":"2024-05-30T17:10:21.778503+0000","last_undegraded":"2024-05-30T17:10:21.778503+0000","last_fullsized":"2024-05-30T17:10:21.778503+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:55:17.556265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"265'8","reported_seq":58,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.316893+0000","last_change":"2024-05-30T16:17:35.234843+0000","last_active":"2024-05-30T17:13:38.316893+0000","last_peered":"2024-05-30T17:13:38.316893+0000","last_clean":"2024-05-30T17:13:38.316893+0000","last_became_active":"2024-05-30T16:17:35.233040+0000","last_became_peered":"2024-05-30T16:17:35.233040+0000","last_unstale":"2024-05-30T17:13:38.316893+0000","last_undegraded":"2024-05-30T17:13:38.316893+0000","last_fullsized":"2024-05-30T17:13:38.316893+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:42:31.077865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":12336,"num_write":13,"num_write_kb":11312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"265'3","reported_seq":56,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:16.775960+0000","last_change":"2024-05-30T16:17:31.174675+0000","last_active":"2024-05-30T17:13:16.775960+0000","last_peered":"2024-05-30T17:13:16.775960+0000","last_clean":"2024-05-30T17:13:16.775960+0000","last_became_active":"2024-05-30T16:17:31.173900+0000","last_became_peered":"2024-05-30T16:17:31.173900+0000","last_unstale":"2024-05-30T17:13:16.775960+0000","last_undegraded":"2024-05-30T17:13:16.775960+0000","last_fullsized":"2024-05-30T17:13:16.775960+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:03:12.426092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"265'75","reported_seq":266,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.546161+0000","last_change":"2024-05-30T16:12:50.815789+0000","last_active":"2024-05-30T17:10:21.546161+0000","last_peered":"2024-05-30T17:10:21.546161+0000","last_clean":"2024-05-30T17:10:21.546161+0000","last_became_active":"2024-05-30T16:12:50.815032+0000","last_became_peered":"2024-05-30T16:12:50.815032+0000","last_unstale":"2024-05-30T17:10:21.546161+0000","last_undegraded":"2024-05-30T17:10:21.546161+0000","last_fullsized":"2024-05-30T17:10:21.546161+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:53:44.633843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":396,"num_read_kb":277,"num_write":137,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"265'1","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.548662+0000","last_change":"2024-05-30T16:34:55.882343+0000","last_active":"2024-05-30T17:09:56.548662+0000","last_peered":"2024-05-30T17:09:56.548662+0000","last_clean":"2024-05-30T17:09:56.548662+0000","last_became_active":"2024-05-30T16:34:55.881414+0000","last_became_peered":"2024-05-30T16:34:55.881414+0000","last_unstale":"2024-05-30T17:09:56.548662+0000","last_undegraded":"2024-05-30T17:09:56.548662+0000","last_fullsized":"2024-05-30T17:09:56.548662+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:58:23.360902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"262'2","reported_seq":48,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:06.542967+0000","last_change":"2024-05-30T16:17:35.238116+0000","last_active":"2024-05-30T17:10:06.542967+0000","last_peered":"2024-05-30T17:10:06.542967+0000","last_clean":"2024-05-30T17:10:06.542967+0000","last_became_active":"2024-05-30T16:17:35.237424+0000","last_became_peered":"2024-05-30T16:17:35.237424+0000","last_unstale":"2024-05-30T17:10:06.542967+0000","last_undegraded":"2024-05-30T17:10:06.542967+0000","last_fullsized":"2024-05-30T17:10:06.542967+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:30:19.299666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"0'0","reported_seq":50,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:01.731526+0000","last_change":"2024-05-30T16:17:31.192981+0000","last_active":"2024-05-30T17:10:01.731526+0000","last_peered":"2024-05-30T17:10:01.731526+0000","last_clean":"2024-05-30T17:10:01.731526+0000","last_became_active":"2024-05-30T16:17:31.192269+0000","last_became_peered":"2024-05-30T16:17:31.192269+0000","last_unstale":"2024-05-30T17:10:01.731526+0000","last_undegraded":"2024-05-30T17:10:01.731526+0000","last_fullsized":"2024-05-30T17:10:01.731526+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:47:56.727573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"265'25","reported_seq":125,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:06.579930+0000","last_change":"2024-05-30T16:12:50.817479+0000","last_active":"2024-05-30T17:11:06.579930+0000","last_peered":"2024-05-30T17:11:06.579930+0000","last_clean":"2024-05-30T17:11:06.579930+0000","last_became_active":"2024-05-30T16:12:50.811732+0000","last_became_peered":"2024-05-30T16:12:50.811732+0000","last_unstale":"2024-05-30T17:11:06.579930+0000","last_undegraded":"2024-05-30T17:11:06.579930+0000","last_fullsized":"2024-05-30T17:11:06.579930+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:22:47.071307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":85,"num_write":53,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":27,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.548778+0000","last_change":"2024-05-30T16:34:55.899801+0000","last_active":"2024-05-30T17:09:56.548778+0000","last_peered":"2024-05-30T17:09:56.548778+0000","last_clean":"2024-05-30T17:09:56.548778+0000","last_became_active":"2024-05-30T16:34:55.899241+0000","last_became_peered":"2024-05-30T16:34:55.899241+0000","last_unstale":"2024-05-30T17:09:56.548778+0000","last_undegraded":"2024-05-30T17:09:56.548778+0000","last_fullsized":"2024-05-30T17:09:56.548778+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:25:21.668824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"0'0","reported_seq":44,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:06.570530+0000","last_change":"2024-05-30T16:17:35.229918+0000","last_active":"2024-05-30T17:10:06.570530+0000","last_peered":"2024-05-30T17:10:06.570530+0000","last_clean":"2024-05-30T17:10:06.570530+0000","last_became_active":"2024-05-30T16:17:35.229034+0000","last_became_peered":"2024-05-30T16:17:35.229034+0000","last_unstale":"2024-05-30T17:10:06.570530+0000","last_undegraded":"2024-05-30T17:10:06.570530+0000","last_fullsized":"2024-05-30T17:10:06.570530+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:01:27.231981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"265'8","reported_seq":72,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:31.601927+0000","last_change":"2024-05-30T16:17:31.191604+0000","last_active":"2024-05-30T17:12:31.601927+0000","last_peered":"2024-05-30T17:12:31.601927+0000","last_clean":"2024-05-30T17:12:31.601927+0000","last_became_active":"2024-05-30T16:17:31.190742+0000","last_became_peered":"2024-05-30T16:17:31.190742+0000","last_unstale":"2024-05-30T17:12:31.601927+0000","last_undegraded":"2024-05-30T17:12:31.601927+0000","last_fullsized":"2024-05-30T17:12:31.601927+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:37:20.245898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":30722,"num_write":17,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"265'33","reported_seq":143,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.785481+0000","last_change":"2024-05-30T16:12:50.821208+0000","last_active":"2024-05-30T17:10:51.785481+0000","last_peered":"2024-05-30T17:10:51.785481+0000","last_clean":"2024-05-30T17:10:51.785481+0000","last_became_active":"2024-05-30T16:12:50.820343+0000","last_became_peered":"2024-05-30T16:12:50.820343+0000","last_unstale":"2024-05-30T17:10:51.785481+0000","last_undegraded":"2024-05-30T17:10:51.785481+0000","last_fullsized":"2024-05-30T17:10:51.785481+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:46:09.411694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":154,"num_read_kb":114,"num_write":68,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.772692+0000","last_change":"2024-05-30T16:34:55.889437+0000","last_active":"2024-05-30T17:09:56.772692+0000","last_peered":"2024-05-30T17:09:56.772692+0000","last_clean":"2024-05-30T17:09:56.772692+0000","last_became_active":"2024-05-30T16:34:55.888238+0000","last_became_peered":"2024-05-30T16:34:55.888238+0000","last_unstale":"2024-05-30T17:09:56.772692+0000","last_undegraded":"2024-05-30T17:09:56.772692+0000","last_fullsized":"2024-05-30T17:09:56.772692+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:23:02.664317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"262'6","reported_seq":56,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.579797+0000","last_change":"2024-05-30T16:17:35.740004+0000","last_active":"2024-05-30T17:10:46.579797+0000","last_peered":"2024-05-30T17:10:46.579797+0000","last_clean":"2024-05-30T17:10:46.579797+0000","last_became_active":"2024-05-30T16:17:35.738420+0000","last_became_peered":"2024-05-30T16:17:35.738420+0000","last_unstale":"2024-05-30T17:10:46.579797+0000","last_undegraded":"2024-05-30T17:10:46.579797+0000","last_fullsized":"2024-05-30T17:10:46.579797+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:38:21.023993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6144,"num_write":16,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"265'5","reported_seq":61,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.186136+0000","last_change":"2024-05-30T16:17:31.195609+0000","last_active":"2024-05-30T17:13:38.186136+0000","last_peered":"2024-05-30T17:13:38.186136+0000","last_clean":"2024-05-30T17:13:38.186136+0000","last_became_active":"2024-05-30T16:17:31.195031+0000","last_became_peered":"2024-05-30T16:17:31.195031+0000","last_unstale":"2024-05-30T17:13:38.186136+0000","last_undegraded":"2024-05-30T17:13:38.186136+0000","last_fullsized":"2024-05-30T17:13:38.186136+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:38:54.212446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6715,"num_write":5,"num_write_kb":6715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"265'65","reported_seq":237,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.577047+0000","last_change":"2024-05-30T16:12:50.820938+0000","last_active":"2024-05-30T17:10:51.577047+0000","last_peered":"2024-05-30T17:10:51.577047+0000","last_clean":"2024-05-30T17:10:51.577047+0000","last_became_active":"2024-05-30T16:12:50.807771+0000","last_became_peered":"2024-05-30T16:12:50.807771+0000","last_unstale":"2024-05-30T17:10:51.577047+0000","last_undegraded":"2024-05-30T17:10:51.577047+0000","last_fullsized":"2024-05-30T17:10:51.577047+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:20:23.418802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":343,"num_read_kb":242,"num_write":116,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"265'1","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.543341+0000","last_change":"2024-05-30T16:34:55.873393+0000","last_active":"2024-05-30T17:09:56.543341+0000","last_peered":"2024-05-30T17:09:56.543341+0000","last_clean":"2024-05-30T17:09:56.543341+0000","last_became_active":"2024-05-30T16:34:55.872581+0000","last_became_peered":"2024-05-30T16:34:55.872581+0000","last_unstale":"2024-05-30T17:09:56.543341+0000","last_undegraded":"2024-05-30T17:09:56.543341+0000","last_fullsized":"2024-05-30T17:09:56.543341+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:23:46.116567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"0'0","reported_seq":45,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:11.557772+0000","last_change":"2024-05-30T16:17:35.739267+0000","last_active":"2024-05-30T17:09:11.557772+0000","last_peered":"2024-05-30T17:09:11.557772+0000","last_clean":"2024-05-30T17:09:11.557772+0000","last_became_active":"2024-05-30T16:17:35.738397+0000","last_became_peered":"2024-05-30T16:17:35.738397+0000","last_unstale":"2024-05-30T17:09:11.557772+0000","last_undegraded":"2024-05-30T17:09:11.557772+0000","last_fullsized":"2024-05-30T17:09:11.557772+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:09:46.285921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"265'3","reported_seq":56,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:06.595753+0000","last_change":"2024-05-30T16:17:31.197103+0000","last_active":"2024-05-30T17:13:06.595753+0000","last_peered":"2024-05-30T17:13:06.595753+0000","last_clean":"2024-05-30T17:13:06.595753+0000","last_became_active":"2024-05-30T16:17:31.196345+0000","last_became_peered":"2024-05-30T16:17:31.196345+0000","last_unstale":"2024-05-30T17:13:06.595753+0000","last_undegraded":"2024-05-30T17:13:06.595753+0000","last_fullsized":"2024-05-30T17:13:06.595753+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:39:32.522011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"265'38","reported_seq":159,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.581145+0000","last_change":"2024-05-30T16:12:50.827196+0000","last_active":"2024-05-30T17:10:51.581145+0000","last_peered":"2024-05-30T17:10:51.581145+0000","last_clean":"2024-05-30T17:10:51.581145+0000","last_became_active":"2024-05-30T16:12:50.826729+0000","last_became_peered":"2024-05-30T16:12:50.826729+0000","last_unstale":"2024-05-30T17:10:51.581145+0000","last_undegraded":"2024-05-30T17:10:51.581145+0000","last_fullsized":"2024-05-30T17:10:51.581145+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:46:31.837238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":187,"num_read_kb":132,"num_write":81,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.548483+0000","last_change":"2024-05-30T16:34:55.876495+0000","last_active":"2024-05-30T17:09:56.548483+0000","last_peered":"2024-05-30T17:09:56.548483+0000","last_clean":"2024-05-30T17:09:56.548483+0000","last_became_active":"2024-05-30T16:34:55.875788+0000","last_became_peered":"2024-05-30T16:34:55.875788+0000","last_unstale":"2024-05-30T17:09:56.548483+0000","last_undegraded":"2024-05-30T17:09:56.548483+0000","last_fullsized":"2024-05-30T17:09:56.548483+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:15:31.389508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"0'0","reported_seq":45,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.579917+0000","last_change":"2024-05-30T16:17:35.746542+0000","last_active":"2024-05-30T17:10:46.579917+0000","last_peered":"2024-05-30T17:10:46.579917+0000","last_clean":"2024-05-30T17:10:46.579917+0000","last_became_active":"2024-05-30T16:17:35.745769+0000","last_became_peered":"2024-05-30T16:17:35.745769+0000","last_unstale":"2024-05-30T17:10:46.579917+0000","last_undegraded":"2024-05-30T17:10:46.579917+0000","last_fullsized":"2024-05-30T17:10:46.579917+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:03:42.283507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"265'2","reported_seq":54,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:06.580648+0000","last_change":"2024-05-30T16:17:31.206243+0000","last_active":"2024-05-30T17:13:06.580648+0000","last_peered":"2024-05-30T17:13:06.580648+0000","last_clean":"2024-05-30T17:13:06.580648+0000","last_became_active":"2024-05-30T16:17:31.204434+0000","last_became_peered":"2024-05-30T16:17:31.204434+0000","last_unstale":"2024-05-30T17:13:06.580648+0000","last_undegraded":"2024-05-30T17:13:06.580648+0000","last_fullsized":"2024-05-30T17:13:06.580648+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:33:27.089723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"265'26","reported_seq":125,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.554131+0000","last_change":"2024-05-30T16:12:50.814716+0000","last_active":"2024-05-30T17:10:51.554131+0000","last_peered":"2024-05-30T17:10:51.554131+0000","last_clean":"2024-05-30T17:10:51.554131+0000","last_became_active":"2024-05-30T16:12:50.813176+0000","last_became_peered":"2024-05-30T16:12:50.813176+0000","last_unstale":"2024-05-30T17:10:51.554131+0000","last_undegraded":"2024-05-30T17:10:51.554131+0000","last_fullsized":"2024-05-30T17:10:51.554131+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:45:17.112612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":85,"num_write":57,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:57.133868+0000","last_change":"2024-05-30T16:34:55.872909+0000","last_active":"2024-05-30T17:09:57.133868+0000","last_peered":"2024-05-30T17:09:57.133868+0000","last_clean":"2024-05-30T17:09:57.133868+0000","last_became_active":"2024-05-30T16:34:55.871800+0000","last_became_peered":"2024-05-30T16:34:55.871800+0000","last_unstale":"2024-05-30T17:09:57.133868+0000","last_undegraded":"2024-05-30T17:09:57.133868+0000","last_fullsized":"2024-05-30T17:09:57.133868+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:10:45.189484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"265'2","reported_seq":48,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:11.775019+0000","last_change":"2024-05-30T16:17:35.237595+0000","last_active":"2024-05-30T17:13:11.775019+0000","last_peered":"2024-05-30T17:13:11.775019+0000","last_clean":"2024-05-30T17:13:11.775019+0000","last_became_active":"2024-05-30T16:17:35.236867+0000","last_became_peered":"2024-05-30T16:17:35.236867+0000","last_unstale":"2024-05-30T17:13:11.775019+0000","last_undegraded":"2024-05-30T17:13:11.775019+0000","last_fullsized":"2024-05-30T17:13:11.775019+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:47:01.389981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":571,"num_write":2,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"0'0","reported_seq":51,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:41.574782+0000","last_change":"2024-05-30T16:17:31.210713+0000","last_active":"2024-05-30T17:10:41.574782+0000","last_peered":"2024-05-30T17:10:41.574782+0000","last_clean":"2024-05-30T17:10:41.574782+0000","last_became_active":"2024-05-30T16:17:31.209882+0000","last_became_peered":"2024-05-30T16:17:31.209882+0000","last_unstale":"2024-05-30T17:10:41.574782+0000","last_undegraded":"2024-05-30T17:10:41.574782+0000","last_fullsized":"2024-05-30T17:10:41.574782+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:17:20.852336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"265'56","reported_seq":218,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:01.749663+0000","last_change":"2024-05-30T16:12:50.825181+0000","last_active":"2024-05-30T17:09:01.749663+0000","last_peered":"2024-05-30T17:09:01.749663+0000","last_clean":"2024-05-30T17:09:01.749663+0000","last_became_active":"2024-05-30T16:12:50.824043+0000","last_became_peered":"2024-05-30T16:12:50.824043+0000","last_unstale":"2024-05-30T17:09:01.749663+0000","last_undegraded":"2024-05-30T17:09:01.749663+0000","last_fullsized":"2024-05-30T17:09:01.749663+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:29.566016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":308,"num_read_kb":221,"num_write":92,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.585310+0000","last_change":"2024-05-30T16:34:55.893966+0000","last_active":"2024-05-30T17:11:11.585310+0000","last_peered":"2024-05-30T17:11:11.585310+0000","last_clean":"2024-05-30T17:11:11.585310+0000","last_became_active":"2024-05-30T16:34:55.893040+0000","last_became_peered":"2024-05-30T16:34:55.893040+0000","last_unstale":"2024-05-30T17:11:11.585310+0000","last_undegraded":"2024-05-30T17:11:11.585310+0000","last_fullsized":"2024-05-30T17:11:11.585310+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:55:00.349712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"0'0","reported_seq":46,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.790902+0000","last_change":"2024-05-30T16:17:35.246948+0000","last_active":"2024-05-30T17:11:16.790902+0000","last_peered":"2024-05-30T17:11:16.790902+0000","last_clean":"2024-05-30T17:11:16.790902+0000","last_became_active":"2024-05-30T16:17:35.246187+0000","last_became_peered":"2024-05-30T16:17:35.246187+0000","last_unstale":"2024-05-30T17:11:16.790902+0000","last_undegraded":"2024-05-30T17:11:16.790902+0000","last_fullsized":"2024-05-30T17:11:16.790902+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:23:58.794231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"265'7","reported_seq":70,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:31.779060+0000","last_change":"2024-05-30T16:17:31.189808+0000","last_active":"2024-05-30T17:13:31.779060+0000","last_peered":"2024-05-30T17:13:31.779060+0000","last_clean":"2024-05-30T17:13:31.779060+0000","last_became_active":"2024-05-30T16:17:31.189091+0000","last_became_peered":"2024-05-30T16:17:31.189091+0000","last_unstale":"2024-05-30T17:13:31.779060+0000","last_undegraded":"2024-05-30T17:13:31.779060+0000","last_fullsized":"2024-05-30T17:13:31.779060+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:35:04.039725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":21053,"num_write":6,"num_write_kb":5691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"265'9","reported_seq":84,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.566377+0000","last_change":"2024-05-30T16:12:50.824521+0000","last_active":"2024-05-30T17:11:11.566377+0000","last_peered":"2024-05-30T17:11:11.566377+0000","last_clean":"2024-05-30T17:11:11.566377+0000","last_became_active":"2024-05-30T16:12:50.823759+0000","last_became_peered":"2024-05-30T16:12:50.823759+0000","last_unstale":"2024-05-30T17:11:11.566377+0000","last_undegraded":"2024-05-30T17:11:11.566377+0000","last_fullsized":"2024-05-30T17:11:11.566377+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:41:59.872702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":33,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.581162+0000","last_change":"2024-05-30T16:34:55.861418+0000","last_active":"2024-05-30T17:11:11.581162+0000","last_peered":"2024-05-30T17:11:11.581162+0000","last_clean":"2024-05-30T17:11:11.581162+0000","last_became_active":"2024-05-30T16:34:55.860695+0000","last_became_peered":"2024-05-30T16:34:55.860695+0000","last_unstale":"2024-05-30T17:11:11.581162+0000","last_undegraded":"2024-05-30T17:11:11.581162+0000","last_fullsized":"2024-05-30T17:11:11.581162+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:45:54.013286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"0'0","reported_seq":46,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.790812+0000","last_change":"2024-05-30T16:17:35.239935+0000","last_active":"2024-05-30T17:11:16.790812+0000","last_peered":"2024-05-30T17:11:16.790812+0000","last_clean":"2024-05-30T17:11:16.790812+0000","last_became_active":"2024-05-30T16:17:35.239157+0000","last_became_peered":"2024-05-30T16:17:35.239157+0000","last_unstale":"2024-05-30T17:11:16.790812+0000","last_undegraded":"2024-05-30T17:11:16.790812+0000","last_fullsized":"2024-05-30T17:11:16.790812+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:53:43.552214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"265'2","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:31.616705+0000","last_change":"2024-05-30T16:17:31.185375+0000","last_active":"2024-05-30T17:13:31.616705+0000","last_peered":"2024-05-30T17:13:31.616705+0000","last_clean":"2024-05-30T17:13:31.616705+0000","last_became_active":"2024-05-30T16:17:31.184675+0000","last_became_peered":"2024-05-30T16:17:31.184675+0000","last_unstale":"2024-05-30T17:13:31.616705+0000","last_undegraded":"2024-05-30T17:13:31.616705+0000","last_fullsized":"2024-05-30T17:13:31.616705+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:13:30.582952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":698,"num_write":2,"num_write_kb":698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"262'96","reported_seq":357,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.581035+0000","last_change":"2024-05-30T16:12:50.802561+0000","last_active":"2024-05-30T17:11:11.581035+0000","last_peered":"2024-05-30T17:11:11.581035+0000","last_clean":"2024-05-30T17:11:11.581035+0000","last_became_active":"2024-05-30T16:12:50.801791+0000","last_became_peered":"2024-05-30T16:12:50.801791+0000","last_unstale":"2024-05-30T17:11:11.581035+0000","last_undegraded":"2024-05-30T17:11:11.581035+0000","last_fullsized":"2024-05-30T17:11:11.581035+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:19:14.883602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":591,"num_read_kb":446,"num_write":104,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"264'1","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:06.566810+0000","last_change":"2024-05-30T16:34:55.864834+0000","last_active":"2024-05-30T17:10:06.566810+0000","last_peered":"2024-05-30T17:10:06.566810+0000","last_clean":"2024-05-30T17:10:06.566810+0000","last_became_active":"2024-05-30T16:34:55.863483+0000","last_became_peered":"2024-05-30T16:34:55.863483+0000","last_unstale":"2024-05-30T17:10:06.566810+0000","last_undegraded":"2024-05-30T17:10:06.566810+0000","last_fullsized":"2024-05-30T17:10:06.566810+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:46:20.034900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"0'0","reported_seq":46,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.586383+0000","last_change":"2024-05-30T16:17:35.236085+0000","last_active":"2024-05-30T17:11:16.586383+0000","last_peered":"2024-05-30T17:11:16.586383+0000","last_clean":"2024-05-30T17:11:16.586383+0000","last_became_active":"2024-05-30T16:17:35.235283+0000","last_became_peered":"2024-05-30T16:17:35.235283+0000","last_unstale":"2024-05-30T17:11:16.586383+0000","last_undegraded":"2024-05-30T17:11:16.586383+0000","last_fullsized":"2024-05-30T17:11:16.586383+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:19:44.053076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"0'0","reported_seq":52,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.557098+0000","last_change":"2024-05-30T16:17:31.204968+0000","last_active":"2024-05-30T17:11:11.557098+0000","last_peered":"2024-05-30T17:11:11.557098+0000","last_clean":"2024-05-30T17:11:11.557098+0000","last_became_active":"2024-05-30T16:17:31.199499+0000","last_became_peered":"2024-05-30T16:17:31.199499+0000","last_unstale":"2024-05-30T17:11:11.557098+0000","last_undegraded":"2024-05-30T17:11:11.557098+0000","last_fullsized":"2024-05-30T17:11:11.557098+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:18:34.513889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"265'22","reported_seq":118,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.746972+0000","last_change":"2024-05-30T16:12:50.807855+0000","last_active":"2024-05-30T17:11:11.746972+0000","last_peered":"2024-05-30T17:11:11.746972+0000","last_clean":"2024-05-30T17:11:11.746972+0000","last_became_active":"2024-05-30T16:12:50.807156+0000","last_became_peered":"2024-05-30T16:12:50.807156+0000","last_unstale":"2024-05-30T17:11:11.746972+0000","last_undegraded":"2024-05-30T17:11:11.746972+0000","last_fullsized":"2024-05-30T17:11:11.746972+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:18:22.683470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":76,"num_write":50,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"265'1","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.585231+0000","last_change":"2024-05-30T16:34:55.889162+0000","last_active":"2024-05-30T17:11:11.585231+0000","last_peered":"2024-05-30T17:11:11.585231+0000","last_clean":"2024-05-30T17:11:11.585231+0000","last_became_active":"2024-05-30T16:34:55.887845+0000","last_became_peered":"2024-05-30T16:34:55.887845+0000","last_unstale":"2024-05-30T17:11:11.585231+0000","last_undegraded":"2024-05-30T17:11:11.585231+0000","last_fullsized":"2024-05-30T17:11:11.585231+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:36:50.835387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"0'0","reported_seq":46,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.558367+0000","last_change":"2024-05-30T16:17:35.246711+0000","last_active":"2024-05-30T17:11:16.558367+0000","last_peered":"2024-05-30T17:11:16.558367+0000","last_clean":"2024-05-30T17:11:16.558367+0000","last_became_active":"2024-05-30T16:17:35.245852+0000","last_became_peered":"2024-05-30T16:17:35.245852+0000","last_unstale":"2024-05-30T17:11:16.558367+0000","last_undegraded":"2024-05-30T17:11:16.558367+0000","last_fullsized":"2024-05-30T17:11:16.558367+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:50:22.430770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"262'5","reported_seq":67,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:12.150705+0000","last_change":"2024-05-30T16:17:31.219308+0000","last_active":"2024-05-30T17:11:12.150705+0000","last_peered":"2024-05-30T17:11:12.150705+0000","last_clean":"2024-05-30T17:11:12.150705+0000","last_became_active":"2024-05-30T16:17:31.218752+0000","last_became_peered":"2024-05-30T16:17:31.218752+0000","last_unstale":"2024-05-30T17:11:12.150705+0000","last_undegraded":"2024-05-30T17:11:12.150705+0000","last_fullsized":"2024-05-30T17:11:12.150705+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:47:57.704377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"265'17","reported_seq":103,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.584982+0000","last_change":"2024-05-30T16:12:50.829602+0000","last_active":"2024-05-30T17:11:11.584982+0000","last_peered":"2024-05-30T17:11:11.584982+0000","last_clean":"2024-05-30T17:11:11.584982+0000","last_became_active":"2024-05-30T16:12:50.817114+0000","last_became_peered":"2024-05-30T16:12:50.817114+0000","last_unstale":"2024-05-30T17:11:11.584982+0000","last_undegraded":"2024-05-30T17:11:11.584982+0000","last_fullsized":"2024-05-30T17:11:11.584982+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:05:48.646667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":55,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:46.541438+0000","last_change":"2024-05-30T16:34:55.867126+0000","last_active":"2024-05-30T17:09:46.541438+0000","last_peered":"2024-05-30T17:09:46.541438+0000","last_clean":"2024-05-30T17:09:46.541438+0000","last_became_active":"2024-05-30T16:34:55.866415+0000","last_became_peered":"2024-05-30T16:34:55.866415+0000","last_unstale":"2024-05-30T17:09:46.541438+0000","last_undegraded":"2024-05-30T17:09:46.541438+0000","last_fullsized":"2024-05-30T17:09:46.541438+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:05:20.422821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"0'0","reported_seq":46,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.560610+0000","last_change":"2024-05-30T16:17:35.228446+0000","last_active":"2024-05-30T17:09:21.560610+0000","last_peered":"2024-05-30T17:09:21.560610+0000","last_clean":"2024-05-30T17:09:21.560610+0000","last_became_active":"2024-05-30T16:17:35.227270+0000","last_became_peered":"2024-05-30T16:17:35.227270+0000","last_unstale":"2024-05-30T17:09:21.560610+0000","last_undegraded":"2024-05-30T17:09:21.560610+0000","last_fullsized":"2024-05-30T17:09:21.560610+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:47:07.315021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"0'0","reported_seq":52,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:16.753258+0000","last_change":"2024-05-30T16:17:31.204447+0000","last_active":"2024-05-30T17:09:16.753258+0000","last_peered":"2024-05-30T17:09:16.753258+0000","last_clean":"2024-05-30T17:09:16.753258+0000","last_became_active":"2024-05-30T16:17:31.203731+0000","last_became_peered":"2024-05-30T16:17:31.203731+0000","last_unstale":"2024-05-30T17:09:16.753258+0000","last_undegraded":"2024-05-30T17:09:16.753258+0000","last_fullsized":"2024-05-30T17:09:16.753258+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:02:51.039638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"265'29","reported_seq":136,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.560856+0000","last_change":"2024-05-30T16:12:50.825208+0000","last_active":"2024-05-30T17:09:21.560856+0000","last_peered":"2024-05-30T17:09:21.560856+0000","last_clean":"2024-05-30T17:09:21.560856+0000","last_became_active":"2024-05-30T16:12:50.824809+0000","last_became_peered":"2024-05-30T16:12:50.824809+0000","last_unstale":"2024-05-30T17:09:21.560856+0000","last_undegraded":"2024-05-30T17:09:21.560856+0000","last_fullsized":"2024-05-30T17:09:21.560856+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:38:19.384141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":98,"num_write":77,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:56.562832+0000","last_change":"2024-05-30T16:34:55.897061+0000","last_active":"2024-05-30T17:10:56.562832+0000","last_peered":"2024-05-30T17:10:56.562832+0000","last_clean":"2024-05-30T17:10:56.562832+0000","last_became_active":"2024-05-30T16:34:55.896357+0000","last_became_peered":"2024-05-30T16:34:55.896357+0000","last_unstale":"2024-05-30T17:10:56.562832+0000","last_undegraded":"2024-05-30T17:10:56.562832+0000","last_fullsized":"2024-05-30T17:10:56.562832+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:23:44.881356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"265'3","reported_seq":45,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:57.161541+0000","last_change":"2024-05-30T16:17:55.551397+0000","last_active":"2024-05-30T17:11:57.161541+0000","last_peered":"2024-05-30T17:11:57.161541+0000","last_clean":"2024-05-30T17:11:57.161541+0000","last_became_active":"2024-05-30T16:17:55.550782+0000","last_became_peered":"2024-05-30T16:17:55.550782+0000","last_unstale":"2024-05-30T17:11:57.161541+0000","last_undegraded":"2024-05-30T17:11:57.161541+0000","last_fullsized":"2024-05-30T17:11:57.161541+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:44:58.931548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2048,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.d","version":"265'7","reported_seq":64,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.266955+0000","last_change":"2024-05-30T16:17:31.202846+0000","last_active":"2024-05-30T17:13:38.266955+0000","last_peered":"2024-05-30T17:13:38.266955+0000","last_clean":"2024-05-30T17:13:38.266955+0000","last_became_active":"2024-05-30T16:17:31.202042+0000","last_became_peered":"2024-05-30T16:17:31.202042+0000","last_unstale":"2024-05-30T17:13:38.266955+0000","last_undegraded":"2024-05-30T17:13:38.266955+0000","last_fullsized":"2024-05-30T17:13:38.266955+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:38:37.711263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":6194,"num_write":7,"num_write_kb":6194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"265'24","reported_seq":130,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:22.153145+0000","last_change":"2024-05-30T16:12:50.810823+0000","last_active":"2024-05-30T17:11:22.153145+0000","last_peered":"2024-05-30T17:11:22.153145+0000","last_clean":"2024-05-30T17:11:22.153145+0000","last_became_active":"2024-05-30T16:12:50.810095+0000","last_became_peered":"2024-05-30T16:12:50.810095+0000","last_unstale":"2024-05-30T17:11:22.153145+0000","last_undegraded":"2024-05-30T17:11:22.153145+0000","last_fullsized":"2024-05-30T17:11:22.153145+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:29:53.744702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":90,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.536265+0000","last_change":"2024-05-30T16:34:55.863218+0000","last_active":"2024-05-30T17:09:36.536265+0000","last_peered":"2024-05-30T17:09:36.536265+0000","last_clean":"2024-05-30T17:09:36.536265+0000","last_became_active":"2024-05-30T16:34:55.862478+0000","last_became_peered":"2024-05-30T16:34:55.862478+0000","last_unstale":"2024-05-30T17:09:36.536265+0000","last_undegraded":"2024-05-30T17:09:36.536265+0000","last_fullsized":"2024-05-30T17:09:36.536265+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:30.967780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"262'3","reported_seq":52,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:31.768287+0000","last_change":"2024-05-30T16:17:35.743152+0000","last_active":"2024-05-30T17:09:31.768287+0000","last_peered":"2024-05-30T17:09:31.768287+0000","last_clean":"2024-05-30T17:09:31.768287+0000","last_became_active":"2024-05-30T16:17:35.742255+0000","last_became_peered":"2024-05-30T16:17:35.742255+0000","last_unstale":"2024-05-30T17:09:31.768287+0000","last_undegraded":"2024-05-30T17:09:31.768287+0000","last_fullsized":"2024-05-30T17:09:31.768287+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:41:50.476109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"265'2","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:31.586951+0000","last_change":"2024-05-30T16:17:31.199481+0000","last_active":"2024-05-30T17:11:31.586951+0000","last_peered":"2024-05-30T17:11:31.586951+0000","last_clean":"2024-05-30T17:11:31.586951+0000","last_became_active":"2024-05-30T16:17:31.198744+0000","last_became_peered":"2024-05-30T16:17:31.198744+0000","last_unstale":"2024-05-30T17:11:31.586951+0000","last_undegraded":"2024-05-30T17:11:31.586951+0000","last_fullsized":"2024-05-30T17:11:31.586951+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:39:35.328874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"265'36","reported_seq":163,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.559692+0000","last_change":"2024-05-30T16:12:50.821672+0000","last_active":"2024-05-30T17:11:21.559692+0000","last_peered":"2024-05-30T17:11:21.559692+0000","last_clean":"2024-05-30T17:11:21.559692+0000","last_became_active":"2024-05-30T16:12:50.820936+0000","last_became_peered":"2024-05-30T16:12:50.820936+0000","last_unstale":"2024-05-30T17:11:21.559692+0000","last_undegraded":"2024-05-30T17:11:21.559692+0000","last_fullsized":"2024-05-30T17:11:21.559692+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:52:42.146262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":194,"num_read_kb":136,"num_write":73,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.543696+0000","last_change":"2024-05-30T16:34:55.877239+0000","last_active":"2024-05-30T17:09:36.543696+0000","last_peered":"2024-05-30T17:09:36.543696+0000","last_clean":"2024-05-30T17:09:36.543696+0000","last_became_active":"2024-05-30T16:34:55.875965+0000","last_became_peered":"2024-05-30T16:34:55.875965+0000","last_unstale":"2024-05-30T17:09:36.543696+0000","last_undegraded":"2024-05-30T17:09:36.543696+0000","last_fullsized":"2024-05-30T17:09:36.543696+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:10:50.846236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"265'3","reported_seq":51,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:36.591125+0000","last_change":"2024-05-30T16:17:35.234741+0000","last_active":"2024-05-30T17:11:36.591125+0000","last_peered":"2024-05-30T17:11:36.591125+0000","last_clean":"2024-05-30T17:11:36.591125+0000","last_became_active":"2024-05-30T16:17:35.229643+0000","last_became_peered":"2024-05-30T16:17:35.229643+0000","last_unstale":"2024-05-30T17:11:36.591125+0000","last_undegraded":"2024-05-30T17:11:36.591125+0000","last_fullsized":"2024-05-30T17:11:36.591125+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:55:14.642511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.f","version":"262'6","reported_seq":63,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:26.767116+0000","last_change":"2024-05-30T16:17:31.215290+0000","last_active":"2024-05-30T17:09:26.767116+0000","last_peered":"2024-05-30T17:09:26.767116+0000","last_clean":"2024-05-30T17:09:26.767116+0000","last_became_active":"2024-05-30T16:17:31.212894+0000","last_became_peered":"2024-05-30T16:17:31.212894+0000","last_unstale":"2024-05-30T17:09:26.767116+0000","last_undegraded":"2024-05-30T17:09:26.767116+0000","last_fullsized":"2024-05-30T17:09:26.767116+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:31:09.789041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6144,"num_write":11,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"262'21","reported_seq":121,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.559938+0000","last_change":"2024-05-30T16:12:50.806577+0000","last_active":"2024-05-30T17:11:21.559938+0000","last_peered":"2024-05-30T17:11:21.559938+0000","last_clean":"2024-05-30T17:11:21.559938+0000","last_became_active":"2024-05-30T16:12:50.805772+0000","last_became_peered":"2024-05-30T16:12:50.805772+0000","last_unstale":"2024-05-30T17:11:21.559938+0000","last_undegraded":"2024-05-30T17:11:21.559938+0000","last_fullsized":"2024-05-30T17:11:21.559938+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:11:36.510014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":79,"num_write":45,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:56.582343+0000","last_change":"2024-05-30T16:34:55.891074+0000","last_active":"2024-05-30T17:10:56.582343+0000","last_peered":"2024-05-30T17:10:56.582343+0000","last_clean":"2024-05-30T17:10:56.582343+0000","last_became_active":"2024-05-30T16:34:55.890313+0000","last_became_peered":"2024-05-30T16:34:55.890313+0000","last_unstale":"2024-05-30T17:10:56.582343+0000","last_undegraded":"2024-05-30T17:10:56.582343+0000","last_fullsized":"2024-05-30T17:10:56.582343+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:22:36.639758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"265'3","reported_seq":54,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:46.754914+0000","last_change":"2024-05-30T16:17:35.243089+0000","last_active":"2024-05-30T17:11:46.754914+0000","last_peered":"2024-05-30T17:11:46.754914+0000","last_clean":"2024-05-30T17:11:46.754914+0000","last_became_active":"2024-05-30T16:17:35.242271+0000","last_became_peered":"2024-05-30T16:17:35.242271+0000","last_unstale":"2024-05-30T17:11:46.754914+0000","last_undegraded":"2024-05-30T17:11:46.754914+0000","last_fullsized":"2024-05-30T17:11:46.754914+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:44:49.304400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10240,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"265'3","reported_seq":60,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:41.589088+0000","last_change":"2024-05-30T16:17:31.195571+0000","last_active":"2024-05-30T17:11:41.589088+0000","last_peered":"2024-05-30T17:11:41.589088+0000","last_clean":"2024-05-30T17:11:41.589088+0000","last_became_active":"2024-05-30T16:17:31.194774+0000","last_became_peered":"2024-05-30T16:17:31.194774+0000","last_unstale":"2024-05-30T17:11:41.589088+0000","last_undegraded":"2024-05-30T17:11:41.589088+0000","last_fullsized":"2024-05-30T17:11:41.589088+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:24:47.216863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":12288,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"265'18","reported_seq":107,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.584196+0000","last_change":"2024-05-30T16:12:50.820954+0000","last_active":"2024-05-30T17:11:21.584196+0000","last_peered":"2024-05-30T17:11:21.584196+0000","last_clean":"2024-05-30T17:11:21.584196+0000","last_became_active":"2024-05-30T16:12:50.820526+0000","last_became_peered":"2024-05-30T16:12:50.820526+0000","last_unstale":"2024-05-30T17:11:21.584196+0000","last_undegraded":"2024-05-30T17:11:21.584196+0000","last_fullsized":"2024-05-30T17:11:21.584196+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:08:41.493544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":60,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:06.565282+0000","last_change":"2024-05-30T16:34:55.884293+0000","last_active":"2024-05-30T17:11:06.565282+0000","last_peered":"2024-05-30T17:11:06.565282+0000","last_clean":"2024-05-30T17:11:06.565282+0000","last_became_active":"2024-05-30T16:34:55.884065+0000","last_became_peered":"2024-05-30T16:34:55.884065+0000","last_unstale":"2024-05-30T17:11:06.565282+0000","last_undegraded":"2024-05-30T17:11:06.565282+0000","last_fullsized":"2024-05-30T17:11:06.565282+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:13:34.815638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"265'10","reported_seq":72,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:46.575265+0000","last_change":"2024-05-30T16:17:35.739756+0000","last_active":"2024-05-30T17:11:46.575265+0000","last_peered":"2024-05-30T17:11:46.575265+0000","last_clean":"2024-05-30T17:11:46.575265+0000","last_became_active":"2024-05-30T16:17:35.738991+0000","last_became_peered":"2024-05-30T16:17:35.738991+0000","last_unstale":"2024-05-30T17:11:46.575265+0000","last_undegraded":"2024-05-30T17:11:46.575265+0000","last_fullsized":"2024-05-30T17:11:46.575265+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:50:21.381841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":26998,"num_write":14,"num_write_kb":10938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"262'6","reported_seq":65,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:31.768080+0000","last_change":"2024-05-30T16:17:31.216617+0000","last_active":"2024-05-30T17:09:31.768080+0000","last_peered":"2024-05-30T17:09:31.768080+0000","last_clean":"2024-05-30T17:09:31.768080+0000","last_became_active":"2024-05-30T16:17:31.212954+0000","last_became_peered":"2024-05-30T16:17:31.212954+0000","last_unstale":"2024-05-30T17:09:31.768080+0000","last_undegraded":"2024-05-30T17:09:31.768080+0000","last_fullsized":"2024-05-30T17:09:31.768080+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:53:12.730929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":11264,"num_write":11,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"262'102","reported_seq":385,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.754543+0000","last_change":"2024-05-30T16:12:50.821280+0000","last_active":"2024-05-30T17:09:21.754543+0000","last_peered":"2024-05-30T17:09:21.754543+0000","last_clean":"2024-05-30T17:09:21.754543+0000","last_became_active":"2024-05-30T16:12:50.820250+0000","last_became_peered":"2024-05-30T16:12:50.820250+0000","last_unstale":"2024-05-30T17:09:21.754543+0000","last_undegraded":"2024-05-30T17:09:21.754543+0000","last_fullsized":"2024-05-30T17:09:21.754543+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:43:31.770630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":642,"num_read_kb":499,"num_write":114,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"265'1","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:56.562900+0000","last_change":"2024-05-30T16:34:55.880548+0000","last_active":"2024-05-30T17:10:56.562900+0000","last_peered":"2024-05-30T17:10:56.562900+0000","last_clean":"2024-05-30T17:10:56.562900+0000","last_became_active":"2024-05-30T16:34:55.878171+0000","last_became_peered":"2024-05-30T16:34:55.878171+0000","last_unstale":"2024-05-30T17:10:56.562900+0000","last_undegraded":"2024-05-30T17:10:56.562900+0000","last_fullsized":"2024-05-30T17:10:56.562900+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:56:14.165458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"265'14","reported_seq":74,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.069136+0000","last_change":"2024-05-30T16:17:35.245798+0000","last_active":"2024-05-30T17:13:38.069136+0000","last_peered":"2024-05-30T17:13:38.069136+0000","last_clean":"2024-05-30T17:13:38.069136+0000","last_became_active":"2024-05-30T16:17:35.245008+0000","last_became_peered":"2024-05-30T16:17:35.245008+0000","last_unstale":"2024-05-30T17:13:38.069136+0000","last_undegraded":"2024-05-30T17:13:38.069136+0000","last_fullsized":"2024-05-30T17:13:38.069136+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:28:02.631261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":25602,"num_write":23,"num_write_kb":17408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"262'3","reported_seq":59,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:46.562533+0000","last_change":"2024-05-30T16:17:31.211050+0000","last_active":"2024-05-30T17:09:46.562533+0000","last_peered":"2024-05-30T17:09:46.562533+0000","last_clean":"2024-05-30T17:09:46.562533+0000","last_became_active":"2024-05-30T16:17:31.210195+0000","last_became_peered":"2024-05-30T16:17:31.210195+0000","last_unstale":"2024-05-30T17:09:46.562533+0000","last_undegraded":"2024-05-30T17:09:46.562533+0000","last_fullsized":"2024-05-30T17:09:46.562533+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:19:11.621024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"265'43","reported_seq":176,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.587423+0000","last_change":"2024-05-30T16:12:50.831498+0000","last_active":"2024-05-30T17:11:21.587423+0000","last_peered":"2024-05-30T17:11:21.587423+0000","last_clean":"2024-05-30T17:11:21.587423+0000","last_became_active":"2024-05-30T16:12:50.830770+0000","last_became_peered":"2024-05-30T16:12:50.830770+0000","last_unstale":"2024-05-30T17:11:21.587423+0000","last_undegraded":"2024-05-30T17:11:21.587423+0000","last_fullsized":"2024-05-30T17:11:21.587423+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:10:09.450788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":214,"num_read_kb":148,"num_write":89,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:56.578000+0000","last_change":"2024-05-30T16:34:55.887780+0000","last_active":"2024-05-30T17:10:56.578000+0000","last_peered":"2024-05-30T17:10:56.578000+0000","last_clean":"2024-05-30T17:10:56.578000+0000","last_became_active":"2024-05-30T16:34:55.887101+0000","last_became_peered":"2024-05-30T16:34:55.887101+0000","last_unstale":"2024-05-30T17:10:56.578000+0000","last_undegraded":"2024-05-30T17:10:56.578000+0000","last_fullsized":"2024-05-30T17:10:56.578000+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:43:47.335383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"262'3","reported_seq":53,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:51.547299+0000","last_change":"2024-05-30T16:17:35.213121+0000","last_active":"2024-05-30T17:09:51.547299+0000","last_peered":"2024-05-30T17:09:51.547299+0000","last_clean":"2024-05-30T17:09:51.547299+0000","last_became_active":"2024-05-30T16:17:35.212403+0000","last_became_peered":"2024-05-30T16:17:35.212403+0000","last_unstale":"2024-05-30T17:09:51.547299+0000","last_undegraded":"2024-05-30T17:09:51.547299+0000","last_fullsized":"2024-05-30T17:09:51.547299+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:52:56.999514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"0'0","reported_seq":52,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:16.721715+0000","last_change":"2024-05-30T16:17:31.192265+0000","last_active":"2024-05-30T17:09:16.721715+0000","last_peered":"2024-05-30T17:09:16.721715+0000","last_clean":"2024-05-30T17:09:16.721715+0000","last_became_active":"2024-05-30T16:17:31.191955+0000","last_became_peered":"2024-05-30T16:17:31.191955+0000","last_unstale":"2024-05-30T17:09:16.721715+0000","last_undegraded":"2024-05-30T17:09:16.721715+0000","last_fullsized":"2024-05-30T17:09:16.721715+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:16:24.140324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"262'8","reported_seq":83,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.559507+0000","last_change":"2024-05-30T16:12:50.820140+0000","last_active":"2024-05-30T17:11:21.559507+0000","last_peered":"2024-05-30T17:11:21.559507+0000","last_clean":"2024-05-30T17:11:21.559507+0000","last_became_active":"2024-05-30T16:12:50.819871+0000","last_became_peered":"2024-05-30T16:12:50.819871+0000","last_unstale":"2024-05-30T17:11:21.559507+0000","last_undegraded":"2024-05-30T17:11:21.559507+0000","last_fullsized":"2024-05-30T17:11:21.559507+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:14:08.269589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":28,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.554460+0000","last_change":"2024-05-30T16:34:55.883102+0000","last_active":"2024-05-30T17:10:21.554460+0000","last_peered":"2024-05-30T17:10:21.554460+0000","last_clean":"2024-05-30T17:10:21.554460+0000","last_became_active":"2024-05-30T16:34:55.882443+0000","last_became_peered":"2024-05-30T16:34:55.882443+0000","last_unstale":"2024-05-30T17:10:21.554460+0000","last_undegraded":"2024-05-30T17:10:21.554460+0000","last_fullsized":"2024-05-30T17:10:21.554460+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:01:28.745649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"262'3","reported_seq":54,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.564614+0000","last_change":"2024-05-30T16:17:35.248377+0000","last_active":"2024-05-30T17:09:56.564614+0000","last_peered":"2024-05-30T17:09:56.564614+0000","last_clean":"2024-05-30T17:09:56.564614+0000","last_became_active":"2024-05-30T16:17:35.247419+0000","last_became_peered":"2024-05-30T16:17:35.247419+0000","last_unstale":"2024-05-30T17:09:56.564614+0000","last_undegraded":"2024-05-30T17:09:56.564614+0000","last_fullsized":"2024-05-30T17:09:56.564614+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:09:58.043053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"262'2","reported_seq":58,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.543844+0000","last_change":"2024-05-30T16:17:31.189996+0000","last_active":"2024-05-30T17:09:36.543844+0000","last_peered":"2024-05-30T17:09:36.543844+0000","last_clean":"2024-05-30T17:09:36.543844+0000","last_became_active":"2024-05-30T16:17:31.189122+0000","last_became_peered":"2024-05-30T16:17:31.189122+0000","last_unstale":"2024-05-30T17:09:36.543844+0000","last_undegraded":"2024-05-30T17:09:36.543844+0000","last_fullsized":"2024-05-30T17:09:36.543844+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:53:04.697503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"265'53","reported_seq":201,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.584475+0000","last_change":"2024-05-30T16:12:50.821561+0000","last_active":"2024-05-30T17:11:21.584475+0000","last_peered":"2024-05-30T17:11:21.584475+0000","last_clean":"2024-05-30T17:11:21.584475+0000","last_became_active":"2024-05-30T16:12:50.809372+0000","last_became_peered":"2024-05-30T16:12:50.809372+0000","last_unstale":"2024-05-30T17:11:21.584475+0000","last_undegraded":"2024-05-30T17:11:21.584475+0000","last_fullsized":"2024-05-30T17:11:21.584475+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:22:57.030056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":186,"num_write":103,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:22.139649+0000","last_change":"2024-05-30T16:34:55.881085+0000","last_active":"2024-05-30T17:10:22.139649+0000","last_peered":"2024-05-30T17:10:22.139649+0000","last_clean":"2024-05-30T17:10:22.139649+0000","last_became_active":"2024-05-30T16:34:55.880627+0000","last_became_peered":"2024-05-30T16:34:55.880627+0000","last_unstale":"2024-05-30T17:10:22.139649+0000","last_undegraded":"2024-05-30T17:10:22.139649+0000","last_fullsized":"2024-05-30T17:10:22.139649+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:34:40.635399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"0'0","reported_seq":48,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:41.561793+0000","last_change":"2024-05-30T16:17:35.743808+0000","last_active":"2024-05-30T17:09:41.561793+0000","last_peered":"2024-05-30T17:09:41.561793+0000","last_clean":"2024-05-30T17:09:41.561793+0000","last_became_active":"2024-05-30T16:17:35.743036+0000","last_became_peered":"2024-05-30T16:17:35.743036+0000","last_unstale":"2024-05-30T17:09:41.561793+0000","last_undegraded":"2024-05-30T17:09:41.561793+0000","last_fullsized":"2024-05-30T17:09:41.561793+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:12:42.745946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"265'3","reported_seq":59,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:56.566418+0000","last_change":"2024-05-30T16:17:31.202885+0000","last_active":"2024-05-30T17:11:56.566418+0000","last_peered":"2024-05-30T17:11:56.566418+0000","last_clean":"2024-05-30T17:11:56.566418+0000","last_became_active":"2024-05-30T16:17:31.202064+0000","last_became_peered":"2024-05-30T16:17:31.202064+0000","last_unstale":"2024-05-30T17:11:56.566418+0000","last_undegraded":"2024-05-30T17:11:56.566418+0000","last_fullsized":"2024-05-30T17:11:56.566418+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:48:01.223505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2048,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"265'34","reported_seq":149,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.559731+0000","last_change":"2024-05-30T16:12:50.816644+0000","last_active":"2024-05-30T17:11:21.559731+0000","last_peered":"2024-05-30T17:11:21.559731+0000","last_clean":"2024-05-30T17:11:21.559731+0000","last_became_active":"2024-05-30T16:12:50.815895+0000","last_became_peered":"2024-05-30T16:12:50.815895+0000","last_unstale":"2024-05-30T17:11:21.559731+0000","last_undegraded":"2024-05-30T17:11:21.559731+0000","last_fullsized":"2024-05-30T17:11:21.559731+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:33:57.609516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":160,"num_read_kb":111,"num_write":71,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":31,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:01.787658+0000","last_change":"2024-05-30T16:34:55.883831+0000","last_active":"2024-05-30T17:11:01.787658+0000","last_peered":"2024-05-30T17:11:01.787658+0000","last_clean":"2024-05-30T17:11:01.787658+0000","last_became_active":"2024-05-30T16:34:55.883112+0000","last_became_peered":"2024-05-30T16:34:55.883112+0000","last_unstale":"2024-05-30T17:11:01.787658+0000","last_undegraded":"2024-05-30T17:11:01.787658+0000","last_fullsized":"2024-05-30T17:11:01.787658+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:50:45.079747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"265'3","reported_seq":52,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:46.575746+0000","last_change":"2024-05-30T16:17:35.744284+0000","last_active":"2024-05-30T17:11:46.575746+0000","last_peered":"2024-05-30T17:11:46.575746+0000","last_clean":"2024-05-30T17:11:46.575746+0000","last_became_active":"2024-05-30T16:17:35.743608+0000","last_became_peered":"2024-05-30T16:17:35.743608+0000","last_unstale":"2024-05-30T17:11:46.575746+0000","last_undegraded":"2024-05-30T17:11:46.575746+0000","last_fullsized":"2024-05-30T17:11:46.575746+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:44:29.493440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"0'0","reported_seq":53,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:32.128370+0000","last_change":"2024-05-30T16:17:31.189800+0000","last_active":"2024-05-30T17:09:32.128370+0000","last_peered":"2024-05-30T17:09:32.128370+0000","last_clean":"2024-05-30T17:09:32.128370+0000","last_became_active":"2024-05-30T16:17:31.188810+0000","last_became_peered":"2024-05-30T16:17:31.188810+0000","last_unstale":"2024-05-30T17:09:32.128370+0000","last_undegraded":"2024-05-30T17:09:32.128370+0000","last_fullsized":"2024-05-30T17:09:32.128370+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:27:55.332594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"265'18","reported_seq":111,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.570334+0000","last_change":"2024-05-30T16:12:50.827178+0000","last_active":"2024-05-30T17:11:26.570334+0000","last_peered":"2024-05-30T17:11:26.570334+0000","last_clean":"2024-05-30T17:11:26.570334+0000","last_became_active":"2024-05-30T16:12:50.826880+0000","last_became_peered":"2024-05-30T16:12:50.826880+0000","last_unstale":"2024-05-30T17:11:26.570334+0000","last_undegraded":"2024-05-30T17:11:26.570334+0000","last_fullsized":"2024-05-30T17:11:26.570334+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:22:01.392934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":61,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.735902+0000","last_change":"2024-05-30T16:34:55.886177+0000","last_active":"2024-05-30T17:10:21.735902+0000","last_peered":"2024-05-30T17:10:21.735902+0000","last_clean":"2024-05-30T17:10:21.735902+0000","last_became_active":"2024-05-30T16:34:55.885643+0000","last_became_peered":"2024-05-30T16:34:55.885643+0000","last_unstale":"2024-05-30T17:10:21.735902+0000","last_undegraded":"2024-05-30T17:10:21.735902+0000","last_fullsized":"2024-05-30T17:10:21.735902+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:27:38.430910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"262'2","reported_seq":51,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.726341+0000","last_change":"2024-05-30T16:17:35.248399+0000","last_active":"2024-05-30T17:09:36.726341+0000","last_peered":"2024-05-30T17:09:36.726341+0000","last_clean":"2024-05-30T17:09:36.726341+0000","last_became_active":"2024-05-30T16:17:35.247757+0000","last_became_peered":"2024-05-30T16:17:35.247757+0000","last_unstale":"2024-05-30T17:09:36.726341+0000","last_undegraded":"2024-05-30T17:09:36.726341+0000","last_fullsized":"2024-05-30T17:09:36.726341+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:22:36.927668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"0'0","reported_seq":53,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:31.542620+0000","last_change":"2024-05-30T16:17:31.203988+0000","last_active":"2024-05-30T17:09:31.542620+0000","last_peered":"2024-05-30T17:09:31.542620+0000","last_clean":"2024-05-30T17:09:31.542620+0000","last_became_active":"2024-05-30T16:17:31.203452+0000","last_became_peered":"2024-05-30T16:17:31.203452+0000","last_unstale":"2024-05-30T17:09:31.542620+0000","last_undegraded":"2024-05-30T17:09:31.542620+0000","last_fullsized":"2024-05-30T17:09:31.542620+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:50:29.995893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"265'37","reported_seq":162,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.789790+0000","last_change":"2024-05-30T16:12:50.812560+0000","last_active":"2024-05-30T17:11:11.789790+0000","last_peered":"2024-05-30T17:11:11.789790+0000","last_clean":"2024-05-30T17:11:11.789790+0000","last_became_active":"2024-05-30T16:12:50.811626+0000","last_became_peered":"2024-05-30T16:12:50.811626+0000","last_unstale":"2024-05-30T17:11:11.789790+0000","last_undegraded":"2024-05-30T17:11:11.789790+0000","last_fullsized":"2024-05-30T17:11:11.789790+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:11:40.631744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":184,"num_read_kb":132,"num_write":81,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"264'1","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.539587+0000","last_change":"2024-05-30T16:34:55.862486+0000","last_active":"2024-05-30T17:09:36.539587+0000","last_peered":"2024-05-30T17:09:36.539587+0000","last_clean":"2024-05-30T17:09:36.539587+0000","last_became_active":"2024-05-30T16:34:55.861767+0000","last_became_peered":"2024-05-30T16:34:55.861767+0000","last_unstale":"2024-05-30T17:09:36.539587+0000","last_undegraded":"2024-05-30T17:09:36.539587+0000","last_fullsized":"2024-05-30T17:09:36.539587+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:06:04.837971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"265'8","reported_seq":63,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:46.797203+0000","last_change":"2024-05-30T16:17:35.244195+0000","last_active":"2024-05-30T17:11:46.797203+0000","last_peered":"2024-05-30T17:11:46.797203+0000","last_clean":"2024-05-30T17:11:46.797203+0000","last_became_active":"2024-05-30T16:17:35.243435+0000","last_became_peered":"2024-05-30T16:17:35.243435+0000","last_unstale":"2024-05-30T17:11:46.797203+0000","last_undegraded":"2024-05-30T17:11:46.797203+0000","last_fullsized":"2024-05-30T17:11:46.797203+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:05:13.001812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":13312,"num_write":13,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"265'3","reported_seq":58,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:46.575316+0000","last_change":"2024-05-30T16:17:31.201111+0000","last_active":"2024-05-30T17:11:46.575316+0000","last_peered":"2024-05-30T17:11:46.575316+0000","last_clean":"2024-05-30T17:11:46.575316+0000","last_became_active":"2024-05-30T16:17:31.200663+0000","last_became_peered":"2024-05-30T16:17:31.200663+0000","last_unstale":"2024-05-30T17:11:46.575316+0000","last_undegraded":"2024-05-30T17:11:46.575316+0000","last_fullsized":"2024-05-30T17:11:46.575316+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:38:48.835894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"265'15","reported_seq":103,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:12.150755+0000","last_change":"2024-05-30T16:12:50.814553+0000","last_active":"2024-05-30T17:11:12.150755+0000","last_peered":"2024-05-30T17:11:12.150755+0000","last_clean":"2024-05-30T17:11:12.150755+0000","last_became_active":"2024-05-30T16:12:50.813741+0000","last_became_peered":"2024-05-30T16:12:50.813741+0000","last_unstale":"2024-05-30T17:11:12.150755+0000","last_undegraded":"2024-05-30T17:11:12.150755+0000","last_fullsized":"2024-05-30T17:11:12.150755+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:49:05.503730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":53,"num_write":33,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.560876+0000","last_change":"2024-05-30T16:34:55.859738+0000","last_active":"2024-05-30T17:09:36.560876+0000","last_peered":"2024-05-30T17:09:36.560876+0000","last_clean":"2024-05-30T17:09:36.560876+0000","last_became_active":"2024-05-30T16:34:55.858872+0000","last_became_peered":"2024-05-30T16:34:55.858872+0000","last_unstale":"2024-05-30T17:09:36.560876+0000","last_undegraded":"2024-05-30T17:09:36.560876+0000","last_fullsized":"2024-05-30T17:09:36.560876+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:48:21.475604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"0'0","reported_seq":49,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:42.130510+0000","last_change":"2024-05-30T16:17:35.227683+0000","last_active":"2024-05-30T17:09:42.130510+0000","last_peered":"2024-05-30T17:09:42.130510+0000","last_clean":"2024-05-30T17:09:42.130510+0000","last_became_active":"2024-05-30T16:17:35.226829+0000","last_became_peered":"2024-05-30T16:17:35.226829+0000","last_unstale":"2024-05-30T17:09:42.130510+0000","last_undegraded":"2024-05-30T17:09:42.130510+0000","last_fullsized":"2024-05-30T17:09:42.130510+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:15:44.751214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"0'0","reported_seq":54,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.543735+0000","last_change":"2024-05-30T16:17:31.205159+0000","last_active":"2024-05-30T17:09:36.543735+0000","last_peered":"2024-05-30T17:09:36.543735+0000","last_clean":"2024-05-30T17:09:36.543735+0000","last_became_active":"2024-05-30T16:17:31.203535+0000","last_became_peered":"2024-05-30T16:17:31.203535+0000","last_unstale":"2024-05-30T17:09:36.543735+0000","last_undegraded":"2024-05-30T17:09:36.543735+0000","last_fullsized":"2024-05-30T17:09:36.543735+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:42:29.611842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"265'36","reported_seq":157,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.747034+0000","last_change":"2024-05-30T16:12:50.832592+0000","last_active":"2024-05-30T17:11:11.747034+0000","last_peered":"2024-05-30T17:11:11.747034+0000","last_clean":"2024-05-30T17:11:11.747034+0000","last_became_active":"2024-05-30T16:12:50.831842+0000","last_became_peered":"2024-05-30T16:12:50.831842+0000","last_unstale":"2024-05-30T17:11:11.747034+0000","last_undegraded":"2024-05-30T17:11:11.747034+0000","last_fullsized":"2024-05-30T17:11:11.747034+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:08:18.471042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":177,"num_read_kb":125,"num_write":75,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:21.570209+0000","last_change":"2024-05-30T16:34:55.863472+0000","last_active":"2024-05-30T17:10:21.570209+0000","last_peered":"2024-05-30T17:10:21.570209+0000","last_clean":"2024-05-30T17:10:21.570209+0000","last_became_active":"2024-05-30T16:34:55.862709+0000","last_became_peered":"2024-05-30T16:34:55.862709+0000","last_unstale":"2024-05-30T17:10:21.570209+0000","last_undegraded":"2024-05-30T17:10:21.570209+0000","last_fullsized":"2024-05-30T17:10:21.570209+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:25:02.100900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"265'3","reported_seq":51,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:46.565279+0000","last_change":"2024-05-30T16:17:35.241271+0000","last_active":"2024-05-30T17:11:46.565279+0000","last_peered":"2024-05-30T17:11:46.565279+0000","last_clean":"2024-05-30T17:11:46.565279+0000","last_became_active":"2024-05-30T16:17:35.240472+0000","last_became_peered":"2024-05-30T16:17:35.240472+0000","last_unstale":"2024-05-30T17:11:46.565279+0000","last_undegraded":"2024-05-30T17:11:46.565279+0000","last_fullsized":"2024-05-30T17:11:46.565279+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:59:40.865645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"265'6","reported_seq":64,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:37.984537+0000","last_change":"2024-05-30T16:17:31.196715+0000","last_active":"2024-05-30T17:13:37.984537+0000","last_peered":"2024-05-30T17:13:37.984537+0000","last_clean":"2024-05-30T17:13:37.984537+0000","last_became_active":"2024-05-30T16:17:31.187958+0000","last_became_peered":"2024-05-30T16:17:31.187958+0000","last_unstale":"2024-05-30T17:13:37.984537+0000","last_undegraded":"2024-05-30T17:13:37.984537+0000","last_fullsized":"2024-05-30T17:13:37.984537+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:26:54.324514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":10240,"num_write":16,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"265'18","reported_seq":112,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.557030+0000","last_change":"2024-05-30T16:12:50.823849+0000","last_active":"2024-05-30T17:11:11.557030+0000","last_peered":"2024-05-30T17:11:11.557030+0000","last_clean":"2024-05-30T17:11:11.557030+0000","last_became_active":"2024-05-30T16:12:50.823090+0000","last_became_peered":"2024-05-30T16:12:50.823090+0000","last_unstale":"2024-05-30T17:11:11.557030+0000","last_undegraded":"2024-05-30T17:11:11.557030+0000","last_fullsized":"2024-05-30T17:11:11.557030+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:38:28.480174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":69,"num_write":35,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:31.556926+0000","last_change":"2024-05-30T16:34:55.891440+0000","last_active":"2024-05-30T17:10:31.556926+0000","last_peered":"2024-05-30T17:10:31.556926+0000","last_clean":"2024-05-30T17:10:31.556926+0000","last_became_active":"2024-05-30T16:34:55.890688+0000","last_became_peered":"2024-05-30T16:34:55.890688+0000","last_unstale":"2024-05-30T17:10:31.556926+0000","last_undegraded":"2024-05-30T17:10:31.556926+0000","last_fullsized":"2024-05-30T17:10:31.556926+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:05:55.238651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"265'8","reported_seq":66,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:46.755004+0000","last_change":"2024-05-30T16:17:35.238582+0000","last_active":"2024-05-30T17:11:46.755004+0000","last_peered":"2024-05-30T17:11:46.755004+0000","last_clean":"2024-05-30T17:11:46.755004+0000","last_became_active":"2024-05-30T16:17:35.237031+0000","last_became_peered":"2024-05-30T16:17:35.237031+0000","last_unstale":"2024-05-30T17:11:46.755004+0000","last_undegraded":"2024-05-30T17:11:46.755004+0000","last_fullsized":"2024-05-30T17:11:46.755004+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:11:17.043035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":20482,"num_write":17,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"262'5","reported_seq":65,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:36.536379+0000","last_change":"2024-05-30T16:17:31.204721+0000","last_active":"2024-05-30T17:09:36.536379+0000","last_peered":"2024-05-30T17:09:36.536379+0000","last_clean":"2024-05-30T17:09:36.536379+0000","last_became_active":"2024-05-30T16:17:31.203871+0000","last_became_peered":"2024-05-30T16:17:31.203871+0000","last_unstale":"2024-05-30T17:09:36.536379+0000","last_undegraded":"2024-05-30T17:09:36.536379+0000","last_fullsized":"2024-05-30T17:09:36.536379+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:41:32.481769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":4098,"num_write":9,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"265'24","reported_seq":133,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:06.565051+0000","last_change":"2024-05-30T16:12:50.827488+0000","last_active":"2024-05-30T17:11:06.565051+0000","last_peered":"2024-05-30T17:11:06.565051+0000","last_clean":"2024-05-30T17:11:06.565051+0000","last_became_active":"2024-05-30T16:12:50.827078+0000","last_became_peered":"2024-05-30T16:12:50.827078+0000","last_unstale":"2024-05-30T17:11:06.565051+0000","last_undegraded":"2024-05-30T17:11:06.565051+0000","last_fullsized":"2024-05-30T17:11:06.565051+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:12:49.772872+0000","last_clean_scrub_stamp":"2024-05-30T16:12:49.772872+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:30:01.069671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":93,"num_write":46,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"265'165","reported_seq":776,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.569071+0000","last_change":"2024-05-30T15:54:16.662201+0000","last_active":"2024-05-30T17:11:21.569071+0000","last_peered":"2024-05-30T17:11:21.569071+0000","last_clean":"2024-05-30T17:11:21.569071+0000","last_became_active":"2024-05-30T15:54:16.661334+0000","last_became_peered":"2024-05-30T15:54:16.661334+0000","last_unstale":"2024-05-30T17:11:21.569071+0000","last_undegraded":"2024-05-30T17:11:21.569071+0000","last_fullsized":"2024-05-30T17:11:21.569071+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":100,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:14:32.854488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2159792,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":347,"num_read_kb":7638,"num_write":1029,"num_write_kb":24692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"265'436","reported_seq":2069,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:33.884897+0000","last_change":"2024-05-30T15:54:18.669748+0000","last_active":"2024-05-30T17:12:33.884897+0000","last_peered":"2024-05-30T17:12:33.884897+0000","last_clean":"2024-05-30T17:12:33.884897+0000","last_became_active":"2024-05-30T15:54:18.669016+0000","last_became_peered":"2024-05-30T15:54:18.669016+0000","last_unstale":"2024-05-30T17:12:33.884897+0000","last_undegraded":"2024-05-30T17:12:33.884897+0000","last_fullsized":"2024-05-30T17:12:33.884897+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":400,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:26:42.111749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2479,"num_read_kb":2366,"num_write":506,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"265'234","reported_seq":889,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.349387+0000","last_change":"2024-05-30T15:54:16.653102+0000","last_active":"2024-05-30T17:13:38.349387+0000","last_peered":"2024-05-30T17:13:38.349387+0000","last_clean":"2024-05-30T17:13:38.349387+0000","last_became_active":"2024-05-30T15:54:16.652724+0000","last_became_peered":"2024-05-30T15:54:16.652724+0000","last_unstale":"2024-05-30T17:13:38.349387+0000","last_undegraded":"2024-05-30T17:13:38.349387+0000","last_fullsized":"2024-05-30T17:13:38.349387+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":200,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:48:02.488417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6513950,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":554,"num_read_kb":8631,"num_write":1281,"num_write_kb":24636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"265'758","reported_seq":2113,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:59.057740+0000","last_change":"2024-05-30T15:54:18.649460+0000","last_active":"2024-05-30T17:11:59.057740+0000","last_peered":"2024-05-30T17:11:59.057740+0000","last_clean":"2024-05-30T17:11:59.057740+0000","last_became_active":"2024-05-30T15:54:18.648675+0000","last_became_peered":"2024-05-30T15:54:18.648675+0000","last_unstale":"2024-05-30T17:11:59.057740+0000","last_undegraded":"2024-05-30T17:11:59.057740+0000","last_fullsized":"2024-05-30T17:11:59.057740+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":700,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:30:51.616348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3292,"num_read_kb":2905,"num_write":1031,"num_write_kb":415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"265'208","reported_seq":840,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.414087+0000","last_change":"2024-05-30T15:54:16.633538+0000","last_active":"2024-05-30T17:13:38.414087+0000","last_peered":"2024-05-30T17:13:38.414087+0000","last_clean":"2024-05-30T17:13:38.414087+0000","last_became_active":"2024-05-30T15:54:16.633011+0000","last_became_peered":"2024-05-30T15:54:16.633011+0000","last_unstale":"2024-05-30T17:13:38.414087+0000","last_undegraded":"2024-05-30T17:13:38.414087+0000","last_fullsized":"2024-05-30T17:13:38.414087+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":100,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:22:33.463154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11855586,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":404,"num_read_kb":43860,"num_write":998,"num_write_kb":74051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"265'746","reported_seq":2144,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:33.888418+0000","last_change":"2024-05-30T15:54:18.651676+0000","last_active":"2024-05-30T17:12:33.888418+0000","last_peered":"2024-05-30T17:12:33.888418+0000","last_clean":"2024-05-30T17:12:33.888418+0000","last_became_active":"2024-05-30T15:54:18.649608+0000","last_became_peered":"2024-05-30T15:54:18.649608+0000","last_unstale":"2024-05-30T17:12:33.888418+0000","last_undegraded":"2024-05-30T17:12:33.888418+0000","last_fullsized":"2024-05-30T17:12:33.888418+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":700,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:29:56.699785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3275,"num_read_kb":2909,"num_write":996,"num_write_kb":378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"265'154","reported_seq":792,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.017937+0000","last_change":"2024-05-30T15:54:16.653236+0000","last_active":"2024-05-30T17:13:38.017937+0000","last_peered":"2024-05-30T17:13:38.017937+0000","last_clean":"2024-05-30T17:13:38.017937+0000","last_became_active":"2024-05-30T15:54:16.652315+0000","last_became_peered":"2024-05-30T15:54:16.652315+0000","last_unstale":"2024-05-30T17:13:38.017937+0000","last_undegraded":"2024-05-30T17:13:38.017937+0000","last_fullsized":"2024-05-30T17:13:38.017937+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":100,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:26:57.491124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1250479,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":437,"num_read_kb":3484,"num_write":911,"num_write_kb":6490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"265'487","reported_seq":1962,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:52.698536+0000","last_change":"2024-05-30T15:54:18.652741+0000","last_active":"2024-05-30T17:12:52.698536+0000","last_peered":"2024-05-30T17:12:52.698536+0000","last_clean":"2024-05-30T17:12:52.698536+0000","last_became_active":"2024-05-30T15:54:18.652451+0000","last_became_peered":"2024-05-30T15:54:18.652451+0000","last_unstale":"2024-05-30T17:12:52.698536+0000","last_undegraded":"2024-05-30T17:12:52.698536+0000","last_fullsized":"2024-05-30T17:12:52.698536+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":400,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:50:10.811052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2387,"num_read_kb":2188,"num_write":616,"num_write_kb":168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"265'245","reported_seq":959,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.112123+0000","last_change":"2024-05-30T15:54:16.659161+0000","last_active":"2024-05-30T17:13:38.112123+0000","last_peered":"2024-05-30T17:13:38.112123+0000","last_clean":"2024-05-30T17:13:38.112123+0000","last_became_active":"2024-05-30T15:54:16.658338+0000","last_became_peered":"2024-05-30T15:54:16.658338+0000","last_unstale":"2024-05-30T17:13:38.112123+0000","last_undegraded":"2024-05-30T17:13:38.112123+0000","last_fullsized":"2024-05-30T17:13:38.112123+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":200,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:10:31.510086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291491,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":830,"num_read_kb":24681,"num_write":1246,"num_write_kb":34031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"265'370","reported_seq":1862,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:12.349707+0000","last_change":"2024-05-30T15:54:18.669822+0000","last_active":"2024-05-30T17:11:12.349707+0000","last_peered":"2024-05-30T17:11:12.349707+0000","last_clean":"2024-05-30T17:11:12.349707+0000","last_became_active":"2024-05-30T15:54:18.669017+0000","last_became_peered":"2024-05-30T15:54:18.669017+0000","last_unstale":"2024-05-30T17:11:12.349707+0000","last_undegraded":"2024-05-30T17:11:12.349707+0000","last_fullsized":"2024-05-30T17:11:12.349707+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":300,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:33:40.482356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1986,"num_read_kb":1906,"num_write":419,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"265'196","reported_seq":821,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.304069+0000","last_change":"2024-05-30T15:54:16.673757+0000","last_active":"2024-05-30T17:13:38.304069+0000","last_peered":"2024-05-30T17:13:38.304069+0000","last_clean":"2024-05-30T17:13:38.304069+0000","last_became_active":"2024-05-30T15:54:16.671120+0000","last_became_peered":"2024-05-30T15:54:16.671120+0000","last_unstale":"2024-05-30T17:13:38.304069+0000","last_undegraded":"2024-05-30T17:13:38.304069+0000","last_fullsized":"2024-05-30T17:13:38.304069+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":100,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:07:40.834702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":81,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":438,"num_read_kb":4718,"num_write":1101,"num_write_kb":20583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"265'725","reported_seq":2201,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.709270+0000","last_change":"2024-05-30T15:54:18.651587+0000","last_active":"2024-05-30T17:13:20.709270+0000","last_peered":"2024-05-30T17:13:20.709270+0000","last_clean":"2024-05-30T17:13:20.709270+0000","last_became_active":"2024-05-30T15:54:18.650543+0000","last_became_peered":"2024-05-30T15:54:18.650543+0000","last_unstale":"2024-05-30T17:13:20.709270+0000","last_undegraded":"2024-05-30T17:13:20.709270+0000","last_fullsized":"2024-05-30T17:13:20.709270+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":700,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:51.313043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3282,"num_read_kb":2952,"num_write":973,"num_write_kb":356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"265'192","reported_seq":832,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.016262+0000","last_change":"2024-05-30T15:54:16.679393+0000","last_active":"2024-05-30T17:13:38.016262+0000","last_peered":"2024-05-30T17:13:38.016262+0000","last_clean":"2024-05-30T17:13:38.016262+0000","last_became_active":"2024-05-30T15:54:16.678799+0000","last_became_peered":"2024-05-30T15:54:16.678799+0000","last_unstale":"2024-05-30T17:13:38.016262+0000","last_undegraded":"2024-05-30T17:13:38.016262+0000","last_fullsized":"2024-05-30T17:13:38.016262+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":100,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:07:54.614137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":491377,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":467,"num_read_kb":33008,"num_write":1090,"num_write_kb":27021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"265'850","reported_seq":2539,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.703253+0000","last_change":"2024-05-30T15:54:18.657813+0000","last_active":"2024-05-30T17:13:20.703253+0000","last_peered":"2024-05-30T17:13:20.703253+0000","last_clean":"2024-05-30T17:13:20.703253+0000","last_became_active":"2024-05-30T15:54:18.656475+0000","last_became_peered":"2024-05-30T15:54:18.656475+0000","last_unstale":"2024-05-30T17:13:20.703253+0000","last_undegraded":"2024-05-30T17:13:20.703253+0000","last_fullsized":"2024-05-30T17:13:20.703253+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":800,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:08:46.768265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4088,"num_read_kb":3692,"num_write":1166,"num_write_kb":472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"265'140","reported_seq":736,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:46.593214+0000","last_change":"2024-05-30T15:54:16.669564+0000","last_active":"2024-05-30T17:11:46.593214+0000","last_peered":"2024-05-30T17:11:46.593214+0000","last_clean":"2024-05-30T17:11:46.593214+0000","last_became_active":"2024-05-30T15:54:16.669095+0000","last_became_peered":"2024-05-30T15:54:16.669095+0000","last_unstale":"2024-05-30T17:11:46.593214+0000","last_undegraded":"2024-05-30T17:11:46.593214+0000","last_fullsized":"2024-05-30T17:11:46.593214+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":100,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:34:08.005236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5243043,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":303,"num_read_kb":17751,"num_write":817,"num_write_kb":30378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"265'369","reported_seq":1790,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.560600+0000","last_change":"2024-05-30T15:54:18.653529+0000","last_active":"2024-05-30T17:11:26.560600+0000","last_peered":"2024-05-30T17:11:26.560600+0000","last_clean":"2024-05-30T17:11:26.560600+0000","last_became_active":"2024-05-30T15:54:18.653189+0000","last_became_peered":"2024-05-30T15:54:18.653189+0000","last_unstale":"2024-05-30T17:11:26.560600+0000","last_undegraded":"2024-05-30T17:11:26.560600+0000","last_fullsized":"2024-05-30T17:11:26.560600+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":300,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:18:04.600068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1969,"num_read_kb":1855,"num_write":435,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"265'336","reported_seq":1765,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:33.943343+0000","last_change":"2024-05-30T15:54:18.652246+0000","last_active":"2024-05-30T17:12:33.943343+0000","last_peered":"2024-05-30T17:12:33.943343+0000","last_clean":"2024-05-30T17:12:33.943343+0000","last_became_active":"2024-05-30T15:54:18.649994+0000","last_became_peered":"2024-05-30T15:54:18.649994+0000","last_unstale":"2024-05-30T17:12:33.943343+0000","last_undegraded":"2024-05-30T17:12:33.943343+0000","last_fullsized":"2024-05-30T17:12:33.943343+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:39:30.273131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1976,"num_read_kb":1893,"num_write":388,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"265'212","reported_seq":877,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.380534+0000","last_change":"2024-05-30T15:54:16.650771+0000","last_active":"2024-05-30T17:13:38.380534+0000","last_peered":"2024-05-30T17:13:38.380534+0000","last_clean":"2024-05-30T17:13:38.380534+0000","last_became_active":"2024-05-30T15:54:16.649872+0000","last_became_peered":"2024-05-30T15:54:16.649872+0000","last_unstale":"2024-05-30T17:13:38.380534+0000","last_undegraded":"2024-05-30T17:13:38.380534+0000","last_fullsized":"2024-05-30T17:13:38.380534+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:01:29.595780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33349,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":510,"num_read_kb":59018,"num_write":1166,"num_write_kb":23092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"265'1525","reported_seq":3131,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:28.308320+0000","last_change":"2024-05-30T15:54:18.649618+0000","last_active":"2024-05-30T17:12:28.308320+0000","last_peered":"2024-05-30T17:12:28.308320+0000","last_clean":"2024-05-30T17:12:28.308320+0000","last_became_active":"2024-05-30T15:54:18.648860+0000","last_became_peered":"2024-05-30T15:54:18.648860+0000","last_unstale":"2024-05-30T17:12:28.308320+0000","last_undegraded":"2024-05-30T17:12:28.308320+0000","last_fullsized":"2024-05-30T17:12:28.308320+0000","mapping_epoch":24,"log_start":"265'1500","ondisk_log_start":"265'1500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":1500,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:09:51.963499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6132,"num_read_kb":5297,"num_write":2138,"num_write_kb":912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"265'221","reported_seq":1083,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.081900+0000","last_change":"2024-05-30T15:54:16.687260+0000","last_active":"2024-05-30T17:13:38.081900+0000","last_peered":"2024-05-30T17:13:38.081900+0000","last_clean":"2024-05-30T17:13:38.081900+0000","last_became_active":"2024-05-30T15:54:16.686847+0000","last_became_peered":"2024-05-30T15:54:16.686847+0000","last_unstale":"2024-05-30T17:13:38.081900+0000","last_undegraded":"2024-05-30T17:13:38.081900+0000","last_fullsized":"2024-05-30T17:13:38.081900+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:27:56.085645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2728232,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":925,"num_read_kb":24537,"num_write":1233,"num_write_kb":54861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"265'1164","reported_seq":3003,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:10.547968+0000","last_change":"2024-05-30T15:54:18.666133+0000","last_active":"2024-05-30T17:13:10.547968+0000","last_peered":"2024-05-30T17:13:10.547968+0000","last_clean":"2024-05-30T17:13:10.547968+0000","last_became_active":"2024-05-30T15:54:18.665302+0000","last_became_peered":"2024-05-30T15:54:18.665302+0000","last_unstale":"2024-05-30T17:13:10.547968+0000","last_undegraded":"2024-05-30T17:13:10.547968+0000","last_fullsized":"2024-05-30T17:13:10.547968+0000","mapping_epoch":24,"log_start":"265'1100","ondisk_log_start":"265'1100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":1100,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:19:10.742000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5292,"num_read_kb":4714,"num_write":1593,"num_write_kb":642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"265'197","reported_seq":918,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.395089+0000","last_change":"2024-05-30T15:54:16.669114+0000","last_active":"2024-05-30T17:13:38.395089+0000","last_peered":"2024-05-30T17:13:38.395089+0000","last_clean":"2024-05-30T17:13:38.395089+0000","last_became_active":"2024-05-30T15:54:16.668438+0000","last_became_peered":"2024-05-30T15:54:16.668438+0000","last_unstale":"2024-05-30T17:13:38.395089+0000","last_undegraded":"2024-05-30T17:13:38.395089+0000","last_fullsized":"2024-05-30T17:13:38.395089+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":100,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:55:42.428949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":918,"num_read_kb":8276,"num_write":1173,"num_write_kb":35310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"265'768","reported_seq":2282,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.649495+0000","last_change":"2024-05-30T15:54:18.651494+0000","last_active":"2024-05-30T17:13:20.649495+0000","last_peered":"2024-05-30T17:13:20.649495+0000","last_clean":"2024-05-30T17:13:20.649495+0000","last_became_active":"2024-05-30T15:54:18.651197+0000","last_became_peered":"2024-05-30T15:54:18.651197+0000","last_unstale":"2024-05-30T17:13:20.649495+0000","last_undegraded":"2024-05-30T17:13:20.649495+0000","last_fullsized":"2024-05-30T17:13:20.649495+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":700,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:01:31.985221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3544,"num_read_kb":3171,"num_write":1051,"num_write_kb":404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"265'190","reported_seq":842,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.380623+0000","last_change":"2024-05-30T15:54:16.677382+0000","last_active":"2024-05-30T17:13:38.380623+0000","last_peered":"2024-05-30T17:13:38.380623+0000","last_clean":"2024-05-30T17:13:38.380623+0000","last_became_active":"2024-05-30T15:54:16.676703+0000","last_became_peered":"2024-05-30T15:54:16.676703+0000","last_unstale":"2024-05-30T17:13:38.380623+0000","last_undegraded":"2024-05-30T17:13:38.380623+0000","last_fullsized":"2024-05-30T17:13:38.380623+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":100,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:39:55.585070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":366,"num_read_kb":73313,"num_write":1062,"num_write_kb":47694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"265'372","reported_seq":1824,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.655751+0000","last_change":"2024-05-30T15:54:18.650530+0000","last_active":"2024-05-30T17:12:44.655751+0000","last_peered":"2024-05-30T17:12:44.655751+0000","last_clean":"2024-05-30T17:12:44.655751+0000","last_became_active":"2024-05-30T15:54:18.650255+0000","last_became_peered":"2024-05-30T15:54:18.650255+0000","last_unstale":"2024-05-30T17:12:44.655751+0000","last_undegraded":"2024-05-30T17:12:44.655751+0000","last_fullsized":"2024-05-30T17:12:44.655751+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":300,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:22:55.711498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1969,"num_read_kb":1871,"num_write":435,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"265'203","reported_seq":854,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.082220+0000","last_change":"2024-05-30T15:54:16.665743+0000","last_active":"2024-05-30T17:13:38.082220+0000","last_peered":"2024-05-30T17:13:38.082220+0000","last_clean":"2024-05-30T17:13:38.082220+0000","last_became_active":"2024-05-30T15:54:16.665344+0000","last_became_peered":"2024-05-30T15:54:16.665344+0000","last_unstale":"2024-05-30T17:13:38.082220+0000","last_undegraded":"2024-05-30T17:13:38.082220+0000","last_fullsized":"2024-05-30T17:13:38.082220+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":100,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:11:01.132558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6847332,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":477,"num_read_kb":7750,"num_write":1077,"num_write_kb":36544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"265'375","reported_seq":1800,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:15.191377+0000","last_change":"2024-05-30T15:54:18.645251+0000","last_active":"2024-05-30T17:12:15.191377+0000","last_peered":"2024-05-30T17:12:15.191377+0000","last_clean":"2024-05-30T17:12:15.191377+0000","last_became_active":"2024-05-30T15:54:18.643825+0000","last_became_peered":"2024-05-30T15:54:18.643825+0000","last_unstale":"2024-05-30T17:12:15.191377+0000","last_undegraded":"2024-05-30T17:12:15.191377+0000","last_fullsized":"2024-05-30T17:12:15.191377+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":300,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:54:45.495400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1936,"num_read_kb":1833,"num_write":434,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"265'238","reported_seq":909,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.265938+0000","last_change":"2024-05-30T15:54:16.638560+0000","last_active":"2024-05-30T17:13:38.265938+0000","last_peered":"2024-05-30T17:13:38.265938+0000","last_clean":"2024-05-30T17:13:38.265938+0000","last_became_active":"2024-05-30T15:54:16.637928+0000","last_became_peered":"2024-05-30T15:54:16.637928+0000","last_unstale":"2024-05-30T17:13:38.265938+0000","last_undegraded":"2024-05-30T17:13:38.265938+0000","last_fullsized":"2024-05-30T17:13:38.265938+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":200,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:28:17.148350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079922,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":526,"num_read_kb":65823,"num_write":1181,"num_write_kb":32044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"265'176","reported_seq":700,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.999380+0000","last_change":"2024-05-30T15:54:26.701638+0000","last_active":"2024-05-30T17:13:20.999380+0000","last_peered":"2024-05-30T17:13:20.999380+0000","last_clean":"2024-05-30T17:13:20.999380+0000","last_became_active":"2024-05-30T15:54:26.700957+0000","last_became_peered":"2024-05-30T15:54:26.700957+0000","last_unstale":"2024-05-30T17:13:20.999380+0000","last_undegraded":"2024-05-30T17:13:20.999380+0000","last_fullsized":"2024-05-30T17:13:20.999380+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":100,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:45:03.540847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":270,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":132,"num_read_kb":126,"num_write":386,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"265'494","reported_seq":2021,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.600563+0000","last_change":"2024-05-30T15:54:18.657002+0000","last_active":"2024-05-30T17:12:44.600563+0000","last_peered":"2024-05-30T17:12:44.600563+0000","last_clean":"2024-05-30T17:12:44.600563+0000","last_became_active":"2024-05-30T15:54:18.652712+0000","last_became_peered":"2024-05-30T15:54:18.652712+0000","last_unstale":"2024-05-30T17:12:44.600563+0000","last_undegraded":"2024-05-30T17:12:44.600563+0000","last_fullsized":"2024-05-30T17:12:44.600563+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":400,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:27:30.279867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2579,"num_read_kb":2377,"num_write":668,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"265'190","reported_seq":794,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:56.578483+0000","last_change":"2024-05-30T15:55:54.564987+0000","last_active":"2024-05-30T17:12:56.578483+0000","last_peered":"2024-05-30T17:12:56.578483+0000","last_clean":"2024-05-30T17:12:56.578483+0000","last_became_active":"2024-05-30T15:55:54.563874+0000","last_became_peered":"2024-05-30T15:55:54.563874+0000","last_unstale":"2024-05-30T17:12:56.578483+0000","last_undegraded":"2024-05-30T17:12:56.578483+0000","last_fullsized":"2024-05-30T17:12:56.578483+0000","mapping_epoch":37,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":100,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:22:56.400029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079905,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":470,"num_read_kb":1589,"num_write":1089,"num_write_kb":27979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:17.124719+0000","last_change":"2024-05-30T15:54:20.659788+0000","last_active":"2024-05-30T17:09:17.124719+0000","last_peered":"2024-05-30T17:09:17.124719+0000","last_clean":"2024-05-30T17:09:17.124719+0000","last_became_active":"2024-05-30T15:54:20.658937+0000","last_became_peered":"2024-05-30T15:54:20.658937+0000","last_unstale":"2024-05-30T17:09:17.124719+0000","last_undegraded":"2024-05-30T17:09:17.124719+0000","last_fullsized":"2024-05-30T17:09:17.124719+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:41:24.097539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"265'202","reported_seq":828,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:37.988099+0000","last_change":"2024-05-30T15:54:22.668221+0000","last_active":"2024-05-30T17:13:37.988099+0000","last_peered":"2024-05-30T17:13:37.988099+0000","last_clean":"2024-05-30T17:13:37.988099+0000","last_became_active":"2024-05-30T15:54:22.667496+0000","last_became_peered":"2024-05-30T15:54:22.667496+0000","last_unstale":"2024-05-30T17:13:37.988099+0000","last_undegraded":"2024-05-30T17:13:37.988099+0000","last_fullsized":"2024-05-30T17:13:37.988099+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:54:14.364376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6140,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":285,"num_read_kb":355,"num_write":168,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":498,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.754793+0000","last_change":"2024-05-30T15:54:24.675232+0000","last_active":"2024-05-30T17:09:21.754793+0000","last_peered":"2024-05-30T17:09:21.754793+0000","last_clean":"2024-05-30T17:09:21.754793+0000","last_became_active":"2024-05-30T15:54:24.674491+0000","last_became_peered":"2024-05-30T15:54:24.674491+0000","last_unstale":"2024-05-30T17:09:21.754793+0000","last_undegraded":"2024-05-30T17:09:21.754793+0000","last_fullsized":"2024-05-30T17:09:21.754793+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:30:36.269404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"265'194","reported_seq":769,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.643659+0000","last_change":"2024-05-30T15:54:26.687210+0000","last_active":"2024-05-30T17:12:44.643659+0000","last_peered":"2024-05-30T17:12:44.643659+0000","last_clean":"2024-05-30T17:12:44.643659+0000","last_became_active":"2024-05-30T15:54:26.686430+0000","last_became_peered":"2024-05-30T15:54:26.686430+0000","last_unstale":"2024-05-30T17:12:44.643659+0000","last_undegraded":"2024-05-30T17:12:44.643659+0000","last_fullsized":"2024-05-30T17:12:44.643659+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":100,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:45:58.373725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":824,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":238,"num_read_kb":223,"num_write":274,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"265'510","reported_seq":2160,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.681680+0000","last_change":"2024-05-30T15:54:18.662470+0000","last_active":"2024-05-30T17:13:18.681680+0000","last_peered":"2024-05-30T17:13:18.681680+0000","last_clean":"2024-05-30T17:13:18.681680+0000","last_became_active":"2024-05-30T15:54:18.661762+0000","last_became_peered":"2024-05-30T15:54:18.661762+0000","last_unstale":"2024-05-30T17:13:18.681680+0000","last_undegraded":"2024-05-30T17:13:18.681680+0000","last_fullsized":"2024-05-30T17:13:18.681680+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":400,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:39:51.492340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2657,"num_read_kb":2481,"num_write":627,"num_write_kb":155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"265'210","reported_seq":866,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.331138+0000","last_change":"2024-05-30T15:54:16.639898+0000","last_active":"2024-05-30T17:13:38.331138+0000","last_peered":"2024-05-30T17:13:38.331138+0000","last_clean":"2024-05-30T17:13:38.331138+0000","last_became_active":"2024-05-30T15:54:16.639284+0000","last_became_peered":"2024-05-30T15:54:16.639284+0000","last_unstale":"2024-05-30T17:13:38.331138+0000","last_undegraded":"2024-05-30T17:13:38.331138+0000","last_fullsized":"2024-05-30T17:13:38.331138+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":100,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:33:35.045719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262178,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":445,"num_read_kb":15653,"num_write":1159,"num_write_kb":64581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":509,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.560530+0000","last_change":"2024-05-30T15:54:20.666014+0000","last_active":"2024-05-30T17:09:21.560530+0000","last_peered":"2024-05-30T17:09:21.560530+0000","last_clean":"2024-05-30T17:09:21.560530+0000","last_became_active":"2024-05-30T15:54:20.665488+0000","last_became_peered":"2024-05-30T15:54:20.665488+0000","last_unstale":"2024-05-30T17:09:21.560530+0000","last_undegraded":"2024-05-30T17:09:21.560530+0000","last_fullsized":"2024-05-30T17:09:21.560530+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:23:46.612023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"265'102","reported_seq":653,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.723144+0000","last_change":"2024-05-30T15:54:22.666790+0000","last_active":"2024-05-30T17:09:21.723144+0000","last_peered":"2024-05-30T17:09:21.723144+0000","last_clean":"2024-05-30T17:09:21.723144+0000","last_became_active":"2024-05-30T15:54:22.666057+0000","last_became_peered":"2024-05-30T15:54:22.666057+0000","last_unstale":"2024-05-30T17:09:21.723144+0000","last_undegraded":"2024-05-30T17:09:21.723144+0000","last_fullsized":"2024-05-30T17:09:21.723144+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:49:29.070169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":113,"num_write":76,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":507,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.723119+0000","last_change":"2024-05-30T15:54:24.687346+0000","last_active":"2024-05-30T17:09:21.723119+0000","last_peered":"2024-05-30T17:09:21.723119+0000","last_clean":"2024-05-30T17:09:21.723119+0000","last_became_active":"2024-05-30T15:54:24.686595+0000","last_became_peered":"2024-05-30T15:54:24.686595+0000","last_unstale":"2024-05-30T17:09:21.723119+0000","last_undegraded":"2024-05-30T17:09:21.723119+0000","last_fullsized":"2024-05-30T17:09:21.723119+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:49:42.604820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"265'99","reported_seq":612,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.102373+0000","last_change":"2024-05-30T15:54:26.689306+0000","last_active":"2024-05-30T17:13:18.102373+0000","last_peered":"2024-05-30T17:13:18.102373+0000","last_clean":"2024-05-30T17:13:18.102373+0000","last_became_active":"2024-05-30T15:54:26.688376+0000","last_became_peered":"2024-05-30T15:54:26.688376+0000","last_unstale":"2024-05-30T17:13:18.102373+0000","last_undegraded":"2024-05-30T17:13:18.102373+0000","last_fullsized":"2024-05-30T17:13:18.102373+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:10:30.878895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":271,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":103,"num_read_kb":102,"num_write":235,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"265'462","reported_seq":2011,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.638007+0000","last_change":"2024-05-30T15:54:18.658772+0000","last_active":"2024-05-30T17:13:20.638007+0000","last_peered":"2024-05-30T17:13:20.638007+0000","last_clean":"2024-05-30T17:13:20.638007+0000","last_became_active":"2024-05-30T15:54:18.657540+0000","last_became_peered":"2024-05-30T15:54:18.657540+0000","last_unstale":"2024-05-30T17:13:20.638007+0000","last_undegraded":"2024-05-30T17:13:20.638007+0000","last_fullsized":"2024-05-30T17:13:20.638007+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":400,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:48:10.156295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2554,"num_read_kb":2362,"num_write":592,"num_write_kb":196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"265'216","reported_seq":877,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.270251+0000","last_change":"2024-05-30T15:54:16.679183+0000","last_active":"2024-05-30T17:13:38.270251+0000","last_peered":"2024-05-30T17:13:38.270251+0000","last_clean":"2024-05-30T17:13:38.270251+0000","last_became_active":"2024-05-30T15:54:16.678659+0000","last_became_peered":"2024-05-30T15:54:16.678659+0000","last_unstale":"2024-05-30T17:13:38.270251+0000","last_undegraded":"2024-05-30T17:13:38.270251+0000","last_fullsized":"2024-05-30T17:13:38.270251+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":200,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:12:32.311283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291507,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":524,"num_read_kb":29194,"num_write":1289,"num_write_kb":17953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":508,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:16.559045+0000","last_change":"2024-05-30T15:54:20.654060+0000","last_active":"2024-05-30T17:09:16.559045+0000","last_peered":"2024-05-30T17:09:16.559045+0000","last_clean":"2024-05-30T17:09:16.559045+0000","last_became_active":"2024-05-30T15:54:20.653316+0000","last_became_peered":"2024-05-30T15:54:20.653316+0000","last_unstale":"2024-05-30T17:09:16.559045+0000","last_undegraded":"2024-05-30T17:09:16.559045+0000","last_fullsized":"2024-05-30T17:09:16.559045+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:11:29.893912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"265'149","reported_seq":711,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:26.537490+0000","last_change":"2024-05-30T15:54:22.665082+0000","last_active":"2024-05-30T17:09:26.537490+0000","last_peered":"2024-05-30T17:09:26.537490+0000","last_clean":"2024-05-30T17:09:26.537490+0000","last_became_active":"2024-05-30T15:54:22.664353+0000","last_became_peered":"2024-05-30T15:54:22.664353+0000","last_unstale":"2024-05-30T17:09:26.537490+0000","last_undegraded":"2024-05-30T17:09:26.537490+0000","last_fullsized":"2024-05-30T17:09:26.537490+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:15:14.584034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":173,"num_read_kb":165,"num_write":120,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"265'532","reported_seq":2482,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.330016+0000","last_change":"2024-05-30T15:54:24.682250+0000","last_active":"2024-05-30T17:13:38.330016+0000","last_peered":"2024-05-30T17:13:38.330016+0000","last_clean":"2024-05-30T17:13:38.330016+0000","last_became_active":"2024-05-30T15:54:24.681503+0000","last_became_peered":"2024-05-30T15:54:24.681503+0000","last_unstale":"2024-05-30T17:13:38.330016+0000","last_undegraded":"2024-05-30T17:13:38.330016+0000","last_fullsized":"2024-05-30T17:13:38.330016+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":500,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:37:06.904193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"265'233","reported_seq":1074,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:19.091217+0000","last_change":"2024-05-30T15:54:26.671113+0000","last_active":"2024-05-30T17:13:19.091217+0000","last_peered":"2024-05-30T17:13:19.091217+0000","last_clean":"2024-05-30T17:13:19.091217+0000","last_became_active":"2024-05-30T15:54:26.670257+0000","last_became_peered":"2024-05-30T15:54:26.670257+0000","last_unstale":"2024-05-30T17:13:19.091217+0000","last_undegraded":"2024-05-30T17:13:19.091217+0000","last_fullsized":"2024-05-30T17:13:19.091217+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":200,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:56:52.779180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":661,"num_read_kb":640,"num_write":540,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"265'406","reported_seq":1988,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.698006+0000","last_change":"2024-05-30T15:54:18.660971+0000","last_active":"2024-05-30T17:13:18.698006+0000","last_peered":"2024-05-30T17:13:18.698006+0000","last_clean":"2024-05-30T17:13:18.698006+0000","last_became_active":"2024-05-30T15:54:18.660656+0000","last_became_peered":"2024-05-30T15:54:18.660656+0000","last_unstale":"2024-05-30T17:13:18.698006+0000","last_undegraded":"2024-05-30T17:13:18.698006+0000","last_fullsized":"2024-05-30T17:13:18.698006+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":300,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:13:06.838766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2368,"num_read_kb":2240,"num_write":473,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"265'191","reported_seq":820,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:36.588487+0000","last_change":"2024-05-30T15:54:16.668687+0000","last_active":"2024-05-30T17:13:36.588487+0000","last_peered":"2024-05-30T17:13:36.588487+0000","last_clean":"2024-05-30T17:13:36.588487+0000","last_became_active":"2024-05-30T15:54:16.668083+0000","last_became_peered":"2024-05-30T15:54:16.668083+0000","last_unstale":"2024-05-30T17:13:36.588487+0000","last_undegraded":"2024-05-30T17:13:36.588487+0000","last_fullsized":"2024-05-30T17:13:36.588487+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":100,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:37:10.668986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5263487,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":406,"num_read_kb":13963,"num_write":1054,"num_write_kb":66137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:17.124781+0000","last_change":"2024-05-30T15:54:20.659591+0000","last_active":"2024-05-30T17:09:17.124781+0000","last_peered":"2024-05-30T17:09:17.124781+0000","last_clean":"2024-05-30T17:09:17.124781+0000","last_became_active":"2024-05-30T15:54:20.658870+0000","last_became_peered":"2024-05-30T15:54:20.658870+0000","last_unstale":"2024-05-30T17:09:17.124781+0000","last_undegraded":"2024-05-30T17:09:17.124781+0000","last_fullsized":"2024-05-30T17:09:17.124781+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:06:21.446324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"265'142","reported_seq":729,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:37.882914+0000","last_change":"2024-05-30T15:54:22.851241+0000","last_active":"2024-05-30T17:13:37.882914+0000","last_peered":"2024-05-30T17:13:37.882914+0000","last_clean":"2024-05-30T17:13:37.882914+0000","last_became_active":"2024-05-30T15:54:22.849978+0000","last_became_peered":"2024-05-30T15:54:22.849978+0000","last_unstale":"2024-05-30T17:13:37.882914+0000","last_undegraded":"2024-05-30T17:13:37.882914+0000","last_fullsized":"2024-05-30T17:13:37.882914+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":100,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:42:52.749855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8153,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":189,"num_read_kb":241,"num_write":102,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"265'549","reported_seq":2572,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:37.745868+0000","last_change":"2024-05-30T15:54:24.688312+0000","last_active":"2024-05-30T17:13:37.745868+0000","last_peered":"2024-05-30T17:13:37.745868+0000","last_clean":"2024-05-30T17:13:37.745868+0000","last_became_active":"2024-05-30T15:54:24.686546+0000","last_became_peered":"2024-05-30T15:54:24.686546+0000","last_unstale":"2024-05-30T17:13:37.745868+0000","last_undegraded":"2024-05-30T17:13:37.745868+0000","last_fullsized":"2024-05-30T17:13:37.745868+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":500,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:12:50.305701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"265'895","reported_seq":2510,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.673971+0000","last_change":"2024-05-30T15:54:18.658824+0000","last_active":"2024-05-30T17:13:18.673971+0000","last_peered":"2024-05-30T17:13:18.673971+0000","last_clean":"2024-05-30T17:13:18.673971+0000","last_became_active":"2024-05-30T15:54:18.658124+0000","last_became_peered":"2024-05-30T15:54:18.658124+0000","last_unstale":"2024-05-30T17:13:18.673971+0000","last_undegraded":"2024-05-30T17:13:18.673971+0000","last_fullsized":"2024-05-30T17:13:18.673971+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":800,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:47:47.836108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4127,"num_read_kb":3657,"num_write":1246,"num_write_kb":491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"265'208","reported_seq":888,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.244131+0000","last_change":"2024-05-30T15:54:16.659141+0000","last_active":"2024-05-30T17:13:38.244131+0000","last_peered":"2024-05-30T17:13:38.244131+0000","last_clean":"2024-05-30T17:13:38.244131+0000","last_became_active":"2024-05-30T15:54:16.658279+0000","last_became_peered":"2024-05-30T15:54:16.658279+0000","last_unstale":"2024-05-30T17:13:38.244131+0000","last_undegraded":"2024-05-30T17:13:38.244131+0000","last_fullsized":"2024-05-30T17:13:38.244131+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":100,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:04:21.938813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1254547,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":480,"num_read_kb":111643,"num_write":1134,"num_write_kb":26225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":510,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:11.551997+0000","last_change":"2024-05-30T15:54:20.662333+0000","last_active":"2024-05-30T17:10:11.551997+0000","last_peered":"2024-05-30T17:10:11.551997+0000","last_clean":"2024-05-30T17:10:11.551997+0000","last_became_active":"2024-05-30T15:54:20.661583+0000","last_became_peered":"2024-05-30T15:54:20.661583+0000","last_unstale":"2024-05-30T17:10:11.551997+0000","last_undegraded":"2024-05-30T17:10:11.551997+0000","last_fullsized":"2024-05-30T17:10:11.551997+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:07:17.026476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"265'256","reported_seq":903,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.071103+0000","last_change":"2024-05-30T15:54:22.850012+0000","last_active":"2024-05-30T17:13:38.071103+0000","last_peered":"2024-05-30T17:13:38.071103+0000","last_clean":"2024-05-30T17:13:38.071103+0000","last_became_active":"2024-05-30T15:54:22.849126+0000","last_became_peered":"2024-05-30T15:54:22.849126+0000","last_unstale":"2024-05-30T17:13:38.071103+0000","last_undegraded":"2024-05-30T17:13:38.071103+0000","last_fullsized":"2024-05-30T17:13:38.071103+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":200,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:34:44.383862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6773,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":352,"num_read_kb":376,"num_write":210,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":502,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:56.786717+0000","last_change":"2024-05-30T15:54:24.674322+0000","last_active":"2024-05-30T17:10:56.786717+0000","last_peered":"2024-05-30T17:10:56.786717+0000","last_clean":"2024-05-30T17:10:56.786717+0000","last_became_active":"2024-05-30T15:54:24.673506+0000","last_became_peered":"2024-05-30T15:54:24.673506+0000","last_unstale":"2024-05-30T17:10:56.786717+0000","last_undegraded":"2024-05-30T17:10:56.786717+0000","last_fullsized":"2024-05-30T17:10:56.786717+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:07.777377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"265'543","reported_seq":1373,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.867831+0000","last_change":"2024-05-30T15:54:26.687129+0000","last_active":"2024-05-30T17:13:24.867831+0000","last_peered":"2024-05-30T17:13:24.867831+0000","last_clean":"2024-05-30T17:13:24.867831+0000","last_became_active":"2024-05-30T15:54:26.686389+0000","last_became_peered":"2024-05-30T15:54:26.686389+0000","last_unstale":"2024-05-30T17:13:24.867831+0000","last_undegraded":"2024-05-30T17:13:24.867831+0000","last_fullsized":"2024-05-30T17:13:24.867831+0000","mapping_epoch":32,"log_start":"265'500","ondisk_log_start":"265'500","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":500,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:44:22.858001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1297,"num_read_kb":1288,"num_write":1137,"num_write_kb":385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"265'1398","reported_seq":3105,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.685832+0000","last_change":"2024-05-30T15:54:18.640378+0000","last_active":"2024-05-30T17:13:18.685832+0000","last_peered":"2024-05-30T17:13:18.685832+0000","last_clean":"2024-05-30T17:13:18.685832+0000","last_became_active":"2024-05-30T15:54:18.639667+0000","last_became_peered":"2024-05-30T15:54:18.639667+0000","last_unstale":"2024-05-30T17:13:18.685832+0000","last_undegraded":"2024-05-30T17:13:18.685832+0000","last_fullsized":"2024-05-30T17:13:18.685832+0000","mapping_epoch":24,"log_start":"265'1300","ondisk_log_start":"265'1300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":1300,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:30:46.931867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5970,"num_read_kb":5130,"num_write":2044,"num_write_kb":884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"265'213","reported_seq":867,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.299735+0000","last_change":"2024-05-30T15:54:16.661968+0000","last_active":"2024-05-30T17:13:38.299735+0000","last_peered":"2024-05-30T17:13:38.299735+0000","last_clean":"2024-05-30T17:13:38.299735+0000","last_became_active":"2024-05-30T15:54:16.659427+0000","last_became_peered":"2024-05-30T15:54:16.659427+0000","last_unstale":"2024-05-30T17:13:38.299735+0000","last_undegraded":"2024-05-30T17:13:38.299735+0000","last_fullsized":"2024-05-30T17:13:38.299735+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":200,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:29:13.804456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":467,"num_read_kb":34119,"num_write":1157,"num_write_kb":42367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":513,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:11.545498+0000","last_change":"2024-05-30T15:54:20.647908+0000","last_active":"2024-05-30T17:10:11.545498+0000","last_peered":"2024-05-30T17:10:11.545498+0000","last_clean":"2024-05-30T17:10:11.545498+0000","last_became_active":"2024-05-30T15:54:20.647565+0000","last_became_peered":"2024-05-30T15:54:20.647565+0000","last_unstale":"2024-05-30T17:10:11.545498+0000","last_undegraded":"2024-05-30T17:10:11.545498+0000","last_fullsized":"2024-05-30T17:10:11.545498+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:19:36.402906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"265'146","reported_seq":738,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.310374+0000","last_change":"2024-05-30T15:54:22.671276+0000","last_active":"2024-05-30T17:13:38.310374+0000","last_peered":"2024-05-30T17:13:38.310374+0000","last_clean":"2024-05-30T17:13:38.310374+0000","last_became_active":"2024-05-30T15:54:22.670064+0000","last_became_peered":"2024-05-30T15:54:22.670064+0000","last_unstale":"2024-05-30T17:13:38.310374+0000","last_undegraded":"2024-05-30T17:13:38.310374+0000","last_fullsized":"2024-05-30T17:13:38.310374+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:40:10.621760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12270,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":197,"num_read_kb":284,"num_write":108,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":504,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:36.577286+0000","last_change":"2024-05-30T15:54:24.688941+0000","last_active":"2024-05-30T17:10:36.577286+0000","last_peered":"2024-05-30T17:10:36.577286+0000","last_clean":"2024-05-30T17:10:36.577286+0000","last_became_active":"2024-05-30T15:54:24.688046+0000","last_became_peered":"2024-05-30T15:54:24.688046+0000","last_unstale":"2024-05-30T17:10:36.577286+0000","last_undegraded":"2024-05-30T17:10:36.577286+0000","last_fullsized":"2024-05-30T17:10:36.577286+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:44:58.051665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"265'2369","reported_seq":4189,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.624292+0000","last_change":"2024-05-30T15:54:26.692267+0000","last_active":"2024-05-30T17:13:18.624292+0000","last_peered":"2024-05-30T17:13:18.624292+0000","last_clean":"2024-05-30T17:13:18.624292+0000","last_became_active":"2024-05-30T15:54:26.691574+0000","last_became_peered":"2024-05-30T15:54:26.691574+0000","last_unstale":"2024-05-30T17:13:18.624292+0000","last_undegraded":"2024-05-30T17:13:18.624292+0000","last_fullsized":"2024-05-30T17:13:18.624292+0000","mapping_epoch":32,"log_start":"265'2300","ondisk_log_start":"265'2300","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":2300,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:33:03.952010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5943,"num_read_kb":5935,"num_write":4785,"num_write_kb":1767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"265'371","reported_seq":1792,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:35.664016+0000","last_change":"2024-05-30T15:54:18.670009+0000","last_active":"2024-05-30T17:11:35.664016+0000","last_peered":"2024-05-30T17:11:35.664016+0000","last_clean":"2024-05-30T17:11:35.664016+0000","last_became_active":"2024-05-30T15:54:18.668963+0000","last_became_peered":"2024-05-30T15:54:18.668963+0000","last_unstale":"2024-05-30T17:11:35.664016+0000","last_undegraded":"2024-05-30T17:11:35.664016+0000","last_fullsized":"2024-05-30T17:11:35.664016+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":300,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:33:28.500306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2011,"num_read_kb":1890,"num_write":455,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"265'240","reported_seq":880,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.413331+0000","last_change":"2024-05-30T15:54:16.650507+0000","last_active":"2024-05-30T17:13:38.413331+0000","last_peered":"2024-05-30T17:13:38.413331+0000","last_clean":"2024-05-30T17:13:38.413331+0000","last_became_active":"2024-05-30T15:54:16.649605+0000","last_became_peered":"2024-05-30T15:54:16.649605+0000","last_unstale":"2024-05-30T17:13:38.413331+0000","last_undegraded":"2024-05-30T17:13:38.413331+0000","last_fullsized":"2024-05-30T17:13:38.413331+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":200,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:10:51.493426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242911,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":469,"num_read_kb":32402,"num_write":1238,"num_write_kb":40071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":512,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.554033+0000","last_change":"2024-05-30T15:54:20.652270+0000","last_active":"2024-05-30T17:10:51.554033+0000","last_peered":"2024-05-30T17:10:51.554033+0000","last_clean":"2024-05-30T17:10:51.554033+0000","last_became_active":"2024-05-30T15:54:20.651369+0000","last_became_peered":"2024-05-30T15:54:20.651369+0000","last_unstale":"2024-05-30T17:10:51.554033+0000","last_undegraded":"2024-05-30T17:10:51.554033+0000","last_fullsized":"2024-05-30T17:10:51.554033+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:06:27.275568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"265'191","reported_seq":806,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.099946+0000","last_change":"2024-05-30T15:54:22.666930+0000","last_active":"2024-05-30T17:13:38.099946+0000","last_peered":"2024-05-30T17:13:38.099946+0000","last_clean":"2024-05-30T17:13:38.099946+0000","last_became_active":"2024-05-30T15:54:22.664874+0000","last_became_peered":"2024-05-30T15:54:22.664874+0000","last_unstale":"2024-05-30T17:13:38.099946+0000","last_undegraded":"2024-05-30T17:13:38.099946+0000","last_fullsized":"2024-05-30T17:13:38.099946+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":100,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:25:56.647990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6309,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":259,"num_read_kb":308,"num_write":148,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"265'549","reported_seq":2482,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:37.748220+0000","last_change":"2024-05-30T15:54:24.674977+0000","last_active":"2024-05-30T17:13:37.748220+0000","last_peered":"2024-05-30T17:13:37.748220+0000","last_clean":"2024-05-30T17:13:37.748220+0000","last_became_active":"2024-05-30T15:54:24.670141+0000","last_became_peered":"2024-05-30T15:54:24.670141+0000","last_unstale":"2024-05-30T17:13:37.748220+0000","last_undegraded":"2024-05-30T17:13:37.748220+0000","last_fullsized":"2024-05-30T17:13:37.748220+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":500,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:38:58.029893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"265'107","reported_seq":611,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:16.569182+0000","last_change":"2024-05-30T15:54:26.693610+0000","last_active":"2024-05-30T17:10:16.569182+0000","last_peered":"2024-05-30T17:10:16.569182+0000","last_clean":"2024-05-30T17:10:16.569182+0000","last_became_active":"2024-05-30T15:54:26.689987+0000","last_became_peered":"2024-05-30T15:54:26.689987+0000","last_unstale":"2024-05-30T17:10:16.569182+0000","last_undegraded":"2024-05-30T17:10:16.569182+0000","last_fullsized":"2024-05-30T17:10:16.569182+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:15:03.997666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":80,"num_write":254,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"265'1331","reported_seq":2966,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.705722+0000","last_change":"2024-05-30T15:54:18.647985+0000","last_active":"2024-05-30T17:13:18.705722+0000","last_peered":"2024-05-30T17:13:18.705722+0000","last_clean":"2024-05-30T17:13:18.705722+0000","last_became_active":"2024-05-30T15:54:18.647634+0000","last_became_peered":"2024-05-30T15:54:18.647634+0000","last_unstale":"2024-05-30T17:13:18.705722+0000","last_undegraded":"2024-05-30T17:13:18.705722+0000","last_fullsized":"2024-05-30T17:13:18.705722+0000","mapping_epoch":24,"log_start":"265'1300","ondisk_log_start":"265'1300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":1300,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:55:27.116297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5717,"num_read_kb":4967,"num_write":1912,"num_write_kb":881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"265'206","reported_seq":821,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.427671+0000","last_change":"2024-05-30T15:54:16.668968+0000","last_active":"2024-05-30T17:13:38.427671+0000","last_peered":"2024-05-30T17:13:38.427671+0000","last_clean":"2024-05-30T17:13:38.427671+0000","last_became_active":"2024-05-30T15:54:16.668336+0000","last_became_peered":"2024-05-30T15:54:16.668336+0000","last_unstale":"2024-05-30T17:13:38.427671+0000","last_undegraded":"2024-05-30T17:13:38.427671+0000","last_fullsized":"2024-05-30T17:13:38.427671+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":100,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:22:09.399226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":403,"num_read_kb":1534,"num_write":1058,"num_write_kb":21253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":513,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:31.549752+0000","last_change":"2024-05-30T15:54:20.656467+0000","last_active":"2024-05-30T17:10:31.549752+0000","last_peered":"2024-05-30T17:10:31.549752+0000","last_clean":"2024-05-30T17:10:31.549752+0000","last_became_active":"2024-05-30T15:54:20.655573+0000","last_became_peered":"2024-05-30T15:54:20.655573+0000","last_unstale":"2024-05-30T17:10:31.549752+0000","last_undegraded":"2024-05-30T17:10:31.549752+0000","last_fullsized":"2024-05-30T17:10:31.549752+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:20:02.252327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"265'207","reported_seq":865,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.360174+0000","last_change":"2024-05-30T15:54:22.659510+0000","last_active":"2024-05-30T17:13:38.360174+0000","last_peered":"2024-05-30T17:13:38.360174+0000","last_clean":"2024-05-30T17:13:38.360174+0000","last_became_active":"2024-05-30T15:54:22.658700+0000","last_became_peered":"2024-05-30T15:54:22.658700+0000","last_unstale":"2024-05-30T17:13:38.360174+0000","last_undegraded":"2024-05-30T17:13:38.360174+0000","last_fullsized":"2024-05-30T17:13:38.360174+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":100,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T15:57:00.030682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14119,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":309,"num_read_kb":443,"num_write":154,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":505,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:16.546507+0000","last_change":"2024-05-30T15:54:24.674234+0000","last_active":"2024-05-30T17:10:16.546507+0000","last_peered":"2024-05-30T17:10:16.546507+0000","last_clean":"2024-05-30T17:10:16.546507+0000","last_became_active":"2024-05-30T15:54:24.673505+0000","last_became_peered":"2024-05-30T15:54:24.673505+0000","last_unstale":"2024-05-30T17:10:16.546507+0000","last_undegraded":"2024-05-30T17:10:16.546507+0000","last_fullsized":"2024-05-30T17:10:16.546507+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:16:54.874496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"265'99","reported_seq":616,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:49.195333+0000","last_change":"2024-05-30T15:54:26.701347+0000","last_active":"2024-05-30T17:10:49.195333+0000","last_peered":"2024-05-30T17:10:49.195333+0000","last_clean":"2024-05-30T17:10:49.195333+0000","last_became_active":"2024-05-30T15:54:26.700608+0000","last_became_peered":"2024-05-30T15:54:26.700608+0000","last_unstale":"2024-05-30T17:10:49.195333+0000","last_undegraded":"2024-05-30T17:10:49.195333+0000","last_fullsized":"2024-05-30T17:10:49.195333+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:57:26.315030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":918,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":103,"num_write":225,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"265'539","reported_seq":2167,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:28.313044+0000","last_change":"2024-05-30T15:54:18.658375+0000","last_active":"2024-05-30T17:12:28.313044+0000","last_peered":"2024-05-30T17:12:28.313044+0000","last_clean":"2024-05-30T17:12:28.313044+0000","last_became_active":"2024-05-30T15:54:18.657522+0000","last_became_peered":"2024-05-30T15:54:18.657522+0000","last_unstale":"2024-05-30T17:12:28.313044+0000","last_undegraded":"2024-05-30T17:12:28.313044+0000","last_fullsized":"2024-05-30T17:12:28.313044+0000","mapping_epoch":24,"log_start":"265'500","ondisk_log_start":"265'500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":500,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:59:15.813358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2751,"num_read_kb":2537,"num_write":708,"num_write_kb":252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"265'234","reported_seq":915,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.300691+0000","last_change":"2024-05-30T15:54:16.666646+0000","last_active":"2024-05-30T17:13:38.300691+0000","last_peered":"2024-05-30T17:13:38.300691+0000","last_clean":"2024-05-30T17:13:38.300691+0000","last_became_active":"2024-05-30T15:54:16.666244+0000","last_became_peered":"2024-05-30T15:54:16.666244+0000","last_unstale":"2024-05-30T17:13:38.300691+0000","last_undegraded":"2024-05-30T17:13:38.300691+0000","last_fullsized":"2024-05-30T17:13:38.300691+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":200,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:10:10.302094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":549856,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":638,"num_read_kb":32957,"num_write":1225,"num_write_kb":40334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":512,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.554107+0000","last_change":"2024-05-30T15:54:20.662409+0000","last_active":"2024-05-30T17:10:51.554107+0000","last_peered":"2024-05-30T17:10:51.554107+0000","last_clean":"2024-05-30T17:10:51.554107+0000","last_became_active":"2024-05-30T15:54:20.661967+0000","last_became_peered":"2024-05-30T15:54:20.661967+0000","last_unstale":"2024-05-30T17:10:51.554107+0000","last_undegraded":"2024-05-30T17:10:51.554107+0000","last_fullsized":"2024-05-30T17:10:51.554107+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:38:09.118798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"265'181","reported_seq":788,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.384417+0000","last_change":"2024-05-30T15:54:22.665466+0000","last_active":"2024-05-30T17:13:38.384417+0000","last_peered":"2024-05-30T17:13:38.384417+0000","last_clean":"2024-05-30T17:13:38.384417+0000","last_became_active":"2024-05-30T15:54:22.664732+0000","last_became_peered":"2024-05-30T15:54:22.664732+0000","last_unstale":"2024-05-30T17:13:38.384417+0000","last_undegraded":"2024-05-30T17:13:38.384417+0000","last_fullsized":"2024-05-30T17:13:38.384417+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":100,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:44:24.095083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13534,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":265,"num_read_kb":366,"num_write":202,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.582702+0000","last_change":"2024-05-30T15:54:24.679521+0000","last_active":"2024-05-30T17:11:16.582702+0000","last_peered":"2024-05-30T17:11:16.582702+0000","last_clean":"2024-05-30T17:11:16.582702+0000","last_became_active":"2024-05-30T15:54:24.679254+0000","last_became_peered":"2024-05-30T15:54:24.679254+0000","last_unstale":"2024-05-30T17:11:16.582702+0000","last_undegraded":"2024-05-30T17:11:16.582702+0000","last_fullsized":"2024-05-30T17:11:16.582702+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:16:16.737529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"265'105","reported_seq":623,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.579506+0000","last_change":"2024-05-30T15:54:26.697980+0000","last_active":"2024-05-30T17:10:46.579506+0000","last_peered":"2024-05-30T17:10:46.579506+0000","last_clean":"2024-05-30T17:10:46.579506+0000","last_became_active":"2024-05-30T15:54:26.697333+0000","last_became_peered":"2024-05-30T15:54:26.697333+0000","last_unstale":"2024-05-30T17:10:46.579506+0000","last_undegraded":"2024-05-30T17:10:46.579506+0000","last_fullsized":"2024-05-30T17:10:46.579506+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:11:10.336832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":270,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":107,"num_read_kb":90,"num_write":242,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"265'397","reported_seq":2059,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:09.478879+0000","last_change":"2024-05-30T15:54:18.636254+0000","last_active":"2024-05-30T17:13:09.478879+0000","last_peered":"2024-05-30T17:13:09.478879+0000","last_clean":"2024-05-30T17:13:09.478879+0000","last_became_active":"2024-05-30T15:54:18.635519+0000","last_became_peered":"2024-05-30T15:54:18.635519+0000","last_unstale":"2024-05-30T17:13:09.478879+0000","last_undegraded":"2024-05-30T17:13:09.478879+0000","last_fullsized":"2024-05-30T17:13:09.478879+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":300,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:25:03.148120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2332,"num_read_kb":2242,"num_write":450,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"265'175","reported_seq":795,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.428021+0000","last_change":"2024-05-30T15:54:16.685723+0000","last_active":"2024-05-30T17:13:38.428021+0000","last_peered":"2024-05-30T17:13:38.428021+0000","last_clean":"2024-05-30T17:13:38.428021+0000","last_became_active":"2024-05-30T15:54:16.685162+0000","last_became_peered":"2024-05-30T15:54:16.685162+0000","last_unstale":"2024-05-30T17:13:38.428021+0000","last_undegraded":"2024-05-30T17:13:38.428021+0000","last_fullsized":"2024-05-30T17:13:38.428021+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":100,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:11:20.614231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5904316,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":401,"num_read_kb":10586,"num_write":1027,"num_write_kb":21029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":510,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.785627+0000","last_change":"2024-05-30T15:54:20.646982+0000","last_active":"2024-05-30T17:10:51.785627+0000","last_peered":"2024-05-30T17:10:51.785627+0000","last_clean":"2024-05-30T17:10:51.785627+0000","last_became_active":"2024-05-30T15:54:20.646646+0000","last_became_peered":"2024-05-30T15:54:20.646646+0000","last_unstale":"2024-05-30T17:10:51.785627+0000","last_undegraded":"2024-05-30T17:10:51.785627+0000","last_fullsized":"2024-05-30T17:10:51.785627+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:31:33.090729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"265'183","reported_seq":800,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.252053+0000","last_change":"2024-05-30T15:54:22.670371+0000","last_active":"2024-05-30T17:13:38.252053+0000","last_peered":"2024-05-30T17:13:38.252053+0000","last_clean":"2024-05-30T17:13:38.252053+0000","last_became_active":"2024-05-30T15:54:22.669660+0000","last_became_peered":"2024-05-30T15:54:22.669660+0000","last_unstale":"2024-05-30T17:13:38.252053+0000","last_undegraded":"2024-05-30T17:13:38.252053+0000","last_fullsized":"2024-05-30T17:13:38.252053+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":100,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:45:08.898874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6014,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":258,"num_read_kb":320,"num_write":146,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"265'512","reported_seq":4595,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:37.739440+0000","last_change":"2024-05-30T15:54:24.673150+0000","last_active":"2024-05-30T17:13:37.739440+0000","last_peered":"2024-05-30T17:13:37.739440+0000","last_clean":"2024-05-30T17:13:37.739440+0000","last_became_active":"2024-05-30T15:54:24.672377+0000","last_became_peered":"2024-05-30T15:54:24.672377+0000","last_unstale":"2024-05-30T17:13:37.739440+0000","last_undegraded":"2024-05-30T17:13:37.739440+0000","last_fullsized":"2024-05-30T17:13:37.739440+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":500,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:22:29.518162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"265'117","reported_seq":633,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.718032+0000","last_change":"2024-05-30T15:54:26.680642+0000","last_active":"2024-05-30T17:13:18.718032+0000","last_peered":"2024-05-30T17:13:18.718032+0000","last_clean":"2024-05-30T17:13:18.718032+0000","last_became_active":"2024-05-30T15:54:26.679866+0000","last_became_peered":"2024-05-30T15:54:26.679866+0000","last_unstale":"2024-05-30T17:13:18.718032+0000","last_undegraded":"2024-05-30T17:13:18.718032+0000","last_fullsized":"2024-05-30T17:13:18.718032+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:26:55.645297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1256,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":144,"num_read_kb":134,"num_write":293,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"265'389","reported_seq":1796,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:37.615748+0000","last_change":"2024-05-30T15:54:18.671784+0000","last_active":"2024-05-30T17:11:37.615748+0000","last_peered":"2024-05-30T17:11:37.615748+0000","last_clean":"2024-05-30T17:11:37.615748+0000","last_became_active":"2024-05-30T15:54:18.671088+0000","last_became_peered":"2024-05-30T15:54:18.671088+0000","last_unstale":"2024-05-30T17:11:37.615748+0000","last_undegraded":"2024-05-30T17:11:37.615748+0000","last_fullsized":"2024-05-30T17:11:37.615748+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":300,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:23:36.607873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2029,"num_read_kb":1894,"num_write":481,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"265'180","reported_seq":923,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.170040+0000","last_change":"2024-05-30T15:54:16.672575+0000","last_active":"2024-05-30T17:13:38.170040+0000","last_peered":"2024-05-30T17:13:38.170040+0000","last_clean":"2024-05-30T17:13:38.170040+0000","last_became_active":"2024-05-30T15:54:16.671477+0000","last_became_peered":"2024-05-30T15:54:16.671477+0000","last_unstale":"2024-05-30T17:13:38.170040+0000","last_undegraded":"2024-05-30T17:13:38.170040+0000","last_fullsized":"2024-05-30T17:13:38.170040+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":100,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:47:10.773488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110807,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":679,"num_read_kb":167118,"num_write":1082,"num_write_kb":34693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"27'2","reported_seq":1158,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:37.481745+0000","last_change":"2024-05-30T15:54:20.663952+0000","last_active":"2024-05-30T17:13:37.481745+0000","last_peered":"2024-05-30T17:13:37.481745+0000","last_clean":"2024-05-30T17:13:37.481745+0000","last_became_active":"2024-05-30T15:54:20.662371+0000","last_became_peered":"2024-05-30T15:54:20.662371+0000","last_unstale":"2024-05-30T17:13:37.481745+0000","last_undegraded":"2024-05-30T17:13:37.481745+0000","last_fullsized":"2024-05-30T17:13:37.481745+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:43:22.768148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":647,"num_read_kb":647,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"265'109","reported_seq":656,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:56.582283+0000","last_change":"2024-05-30T15:54:22.849771+0000","last_active":"2024-05-30T17:10:56.582283+0000","last_peered":"2024-05-30T17:10:56.582283+0000","last_clean":"2024-05-30T17:10:56.582283+0000","last_became_active":"2024-05-30T15:54:22.849105+0000","last_became_peered":"2024-05-30T15:54:22.849105+0000","last_unstale":"2024-05-30T17:10:56.582283+0000","last_undegraded":"2024-05-30T17:10:56.582283+0000","last_fullsized":"2024-05-30T17:10:56.582283+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:56:52.974159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":120,"num_read_kb":115,"num_write":80,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"265'541","reported_seq":2317,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:37.740199+0000","last_change":"2024-05-30T15:54:24.678108+0000","last_active":"2024-05-30T17:13:37.740199+0000","last_peered":"2024-05-30T17:13:37.740199+0000","last_clean":"2024-05-30T17:13:37.740199+0000","last_became_active":"2024-05-30T15:54:24.677291+0000","last_became_peered":"2024-05-30T15:54:24.677291+0000","last_unstale":"2024-05-30T17:13:37.740199+0000","last_undegraded":"2024-05-30T17:13:37.740199+0000","last_fullsized":"2024-05-30T17:13:37.740199+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":500,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:59:57.567508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"265'135","reported_seq":656,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:15.308427+0000","last_change":"2024-05-30T15:54:26.675238+0000","last_active":"2024-05-30T17:13:15.308427+0000","last_peered":"2024-05-30T17:13:15.308427+0000","last_clean":"2024-05-30T17:13:15.308427+0000","last_became_active":"2024-05-30T15:54:26.674517+0000","last_became_peered":"2024-05-30T15:54:26.674517+0000","last_unstale":"2024-05-30T17:13:15.308427+0000","last_undegraded":"2024-05-30T17:13:15.308427+0000","last_fullsized":"2024-05-30T17:13:15.308427+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":100,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:38:58.206221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":141,"num_read_kb":129,"num_write":332,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"265'402","reported_seq":1940,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:12.347239+0000","last_change":"2024-05-30T15:54:18.657775+0000","last_active":"2024-05-30T17:13:12.347239+0000","last_peered":"2024-05-30T17:13:12.347239+0000","last_clean":"2024-05-30T17:13:12.347239+0000","last_became_active":"2024-05-30T15:54:18.657247+0000","last_became_peered":"2024-05-30T15:54:18.657247+0000","last_unstale":"2024-05-30T17:13:12.347239+0000","last_undegraded":"2024-05-30T17:13:12.347239+0000","last_fullsized":"2024-05-30T17:13:12.347239+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":300,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:33:29.912796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2106,"num_read_kb":2008,"num_write":461,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"265'185","reported_seq":795,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.221751+0000","last_change":"2024-05-30T15:54:54.427377+0000","last_active":"2024-05-30T17:13:38.221751+0000","last_peered":"2024-05-30T17:13:38.221751+0000","last_clean":"2024-05-30T17:13:38.221751+0000","last_became_active":"2024-05-30T15:54:54.426538+0000","last_became_peered":"2024-05-30T15:54:54.426538+0000","last_unstale":"2024-05-30T17:13:38.221751+0000","last_undegraded":"2024-05-30T17:13:38.221751+0000","last_fullsized":"2024-05-30T17:13:38.221751+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:57:32.635925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":443,"num_read_kb":549,"num_write":1012,"num_write_kb":15096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,7],"acting":[2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":518,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:51.742564+0000","last_change":"2024-05-30T15:54:20.670270+0000","last_active":"2024-05-30T17:10:51.742564+0000","last_peered":"2024-05-30T17:10:51.742564+0000","last_clean":"2024-05-30T17:10:51.742564+0000","last_became_active":"2024-05-30T15:54:20.669492+0000","last_became_peered":"2024-05-30T15:54:20.669492+0000","last_unstale":"2024-05-30T17:10:51.742564+0000","last_undegraded":"2024-05-30T17:10:51.742564+0000","last_fullsized":"2024-05-30T17:10:51.742564+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:01:58.108666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"265'236","reported_seq":874,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.002745+0000","last_change":"2024-05-30T15:54:22.672749+0000","last_active":"2024-05-30T17:13:38.002745+0000","last_peered":"2024-05-30T17:13:38.002745+0000","last_clean":"2024-05-30T17:13:38.002745+0000","last_became_active":"2024-05-30T15:54:22.671943+0000","last_became_peered":"2024-05-30T15:54:22.671943+0000","last_unstale":"2024-05-30T17:13:38.002745+0000","last_undegraded":"2024-05-30T17:13:38.002745+0000","last_fullsized":"2024-05-30T17:13:38.002745+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":200,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:43:33.730358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":313,"num_read_kb":312,"num_write":178,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":500,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:56.553870+0000","last_change":"2024-05-30T15:54:24.682341+0000","last_active":"2024-05-30T17:10:56.553870+0000","last_peered":"2024-05-30T17:10:56.553870+0000","last_clean":"2024-05-30T17:10:56.553870+0000","last_became_active":"2024-05-30T15:54:24.681595+0000","last_became_peered":"2024-05-30T15:54:24.681595+0000","last_unstale":"2024-05-30T17:10:56.553870+0000","last_undegraded":"2024-05-30T17:10:56.553870+0000","last_fullsized":"2024-05-30T17:10:56.553870+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:03:03.587984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"265'115","reported_seq":622,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:31.794301+0000","last_change":"2024-05-30T15:54:26.689376+0000","last_active":"2024-05-30T17:11:31.794301+0000","last_peered":"2024-05-30T17:11:31.794301+0000","last_clean":"2024-05-30T17:11:31.794301+0000","last_became_active":"2024-05-30T15:54:26.688518+0000","last_became_peered":"2024-05-30T15:54:26.688518+0000","last_unstale":"2024-05-30T17:11:31.794301+0000","last_undegraded":"2024-05-30T17:11:31.794301+0000","last_fullsized":"2024-05-30T17:11:31.794301+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":100,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:13:29.397423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":619,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":113,"num_read_kb":112,"num_write":261,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"265'756","reported_seq":2265,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.637015+0000","last_change":"2024-05-30T15:54:18.678584+0000","last_active":"2024-05-30T17:11:21.637015+0000","last_peered":"2024-05-30T17:11:21.637015+0000","last_clean":"2024-05-30T17:11:21.637015+0000","last_became_active":"2024-05-30T15:54:18.677957+0000","last_became_peered":"2024-05-30T15:54:18.677957+0000","last_unstale":"2024-05-30T17:11:21.637015+0000","last_undegraded":"2024-05-30T17:11:21.637015+0000","last_fullsized":"2024-05-30T17:11:21.637015+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":700,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:51:57.717964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3440,"num_read_kb":3078,"num_write":1010,"num_write_kb":372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":627,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:26.555530+0000","last_change":"2024-05-30T15:54:03.031292+0000","last_active":"2024-05-30T17:10:26.555530+0000","last_peered":"2024-05-30T17:10:26.555530+0000","last_clean":"2024-05-30T17:10:26.555530+0000","last_became_active":"2024-05-30T15:54:03.029214+0000","last_became_peered":"2024-05-30T15:54:03.029214+0000","last_unstale":"2024-05-30T17:10:26.555530+0000","last_undegraded":"2024-05-30T17:10:26.555530+0000","last_fullsized":"2024-05-30T17:10:26.555530+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:01.512251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:01.512251+0000","last_clean_scrub_stamp":"2024-05-30T15:54:01.512251+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:40:07.389706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":530,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:17.151931+0000","last_change":"2024-05-30T15:54:08.578526+0000","last_active":"2024-05-30T17:11:17.151931+0000","last_peered":"2024-05-30T17:11:17.151931+0000","last_clean":"2024-05-30T17:11:17.151931+0000","last_became_active":"2024-05-30T15:54:08.577556+0000","last_became_peered":"2024-05-30T15:54:08.577556+0000","last_unstale":"2024-05-30T17:11:17.151931+0000","last_undegraded":"2024-05-30T17:11:17.151931+0000","last_fullsized":"2024-05-30T17:11:17.151931+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:36:52.339688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"265'189","reported_seq":844,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.333424+0000","last_change":"2024-05-30T15:54:16.674631+0000","last_active":"2024-05-30T17:13:38.333424+0000","last_peered":"2024-05-30T17:13:38.333424+0000","last_clean":"2024-05-30T17:13:38.333424+0000","last_became_active":"2024-05-30T15:54:16.674229+0000","last_became_peered":"2024-05-30T15:54:16.674229+0000","last_unstale":"2024-05-30T17:13:38.333424+0000","last_undegraded":"2024-05-30T17:13:38.333424+0000","last_fullsized":"2024-05-30T17:13:38.333424+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:38:06.634901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5525719,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":447,"num_read_kb":11470,"num_write":1074,"num_write_kb":25445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":512,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.570297+0000","last_change":"2024-05-30T15:54:20.665207+0000","last_active":"2024-05-30T17:11:26.570297+0000","last_peered":"2024-05-30T17:11:26.570297+0000","last_clean":"2024-05-30T17:11:26.570297+0000","last_became_active":"2024-05-30T15:54:20.664317+0000","last_became_peered":"2024-05-30T15:54:20.664317+0000","last_unstale":"2024-05-30T17:11:26.570297+0000","last_undegraded":"2024-05-30T17:11:26.570297+0000","last_fullsized":"2024-05-30T17:11:26.570297+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:57:42.587325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"265'334","reported_seq":1055,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.207803+0000","last_change":"2024-05-30T15:54:22.670731+0000","last_active":"2024-05-30T17:13:38.207803+0000","last_peered":"2024-05-30T17:13:38.207803+0000","last_clean":"2024-05-30T17:13:38.207803+0000","last_became_active":"2024-05-30T15:54:22.669871+0000","last_became_peered":"2024-05-30T15:54:22.669871+0000","last_unstale":"2024-05-30T17:13:38.207803+0000","last_undegraded":"2024-05-30T17:13:38.207803+0000","last_fullsized":"2024-05-30T17:13:38.207803+0000","mapping_epoch":28,"log_start":"265'300","ondisk_log_start":"265'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:48:31.269839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49272,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":648,"num_read_kb":960,"num_write":598,"num_write_kb":460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":507,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:16.569157+0000","last_change":"2024-05-30T15:54:24.685392+0000","last_active":"2024-05-30T17:10:16.569157+0000","last_peered":"2024-05-30T17:10:16.569157+0000","last_clean":"2024-05-30T17:10:16.569157+0000","last_became_active":"2024-05-30T15:54:24.684709+0000","last_became_peered":"2024-05-30T15:54:24.684709+0000","last_unstale":"2024-05-30T17:10:16.569157+0000","last_undegraded":"2024-05-30T17:10:16.569157+0000","last_fullsized":"2024-05-30T17:10:16.569157+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:57:38.351236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"265'174","reported_seq":878,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:16.948647+0000","last_change":"2024-05-30T15:54:26.687357+0000","last_active":"2024-05-30T17:13:16.948647+0000","last_peered":"2024-05-30T17:13:16.948647+0000","last_clean":"2024-05-30T17:13:16.948647+0000","last_became_active":"2024-05-30T15:54:26.686647+0000","last_became_peered":"2024-05-30T15:54:26.686647+0000","last_unstale":"2024-05-30T17:13:16.948647+0000","last_undegraded":"2024-05-30T17:13:16.948647+0000","last_fullsized":"2024-05-30T17:13:16.948647+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:16:53.692909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1020,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":416,"num_read_kb":404,"num_write":359,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"265'388","reported_seq":1862,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:28.256178+0000","last_change":"2024-05-30T15:54:18.649340+0000","last_active":"2024-05-30T17:12:28.256178+0000","last_peered":"2024-05-30T17:12:28.256178+0000","last_clean":"2024-05-30T17:12:28.256178+0000","last_became_active":"2024-05-30T15:54:18.648529+0000","last_became_peered":"2024-05-30T15:54:18.648529+0000","last_unstale":"2024-05-30T17:12:28.256178+0000","last_undegraded":"2024-05-30T17:12:28.256178+0000","last_fullsized":"2024-05-30T17:12:28.256178+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":300,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:56:22.361251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2082,"num_read_kb":1967,"num_write":465,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":531,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:31.556722+0000","last_change":"2024-05-30T15:54:08.575348+0000","last_active":"2024-05-30T17:10:31.556722+0000","last_peered":"2024-05-30T17:10:31.556722+0000","last_clean":"2024-05-30T17:10:31.556722+0000","last_became_active":"2024-05-30T15:54:08.574645+0000","last_became_peered":"2024-05-30T15:54:08.574645+0000","last_unstale":"2024-05-30T17:10:31.556722+0000","last_undegraded":"2024-05-30T17:10:31.556722+0000","last_fullsized":"2024-05-30T17:10:31.556722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:41:49.462483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"265'174","reported_seq":809,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.395671+0000","last_change":"2024-05-30T15:54:16.678919+0000","last_active":"2024-05-30T17:13:38.395671+0000","last_peered":"2024-05-30T17:13:38.395671+0000","last_clean":"2024-05-30T17:13:38.395671+0000","last_became_active":"2024-05-30T15:54:16.678386+0000","last_became_peered":"2024-05-30T15:54:16.678386+0000","last_unstale":"2024-05-30T17:13:38.395671+0000","last_undegraded":"2024-05-30T17:13:38.395671+0000","last_fullsized":"2024-05-30T17:13:38.395671+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:06:51.673817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079916,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":364,"num_read_kb":7665,"num_write":1082,"num_write_kb":33234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":510,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:12.137521+0000","last_change":"2024-05-30T15:54:20.663148+0000","last_active":"2024-05-30T17:10:12.137521+0000","last_peered":"2024-05-30T17:10:12.137521+0000","last_clean":"2024-05-30T17:10:12.137521+0000","last_became_active":"2024-05-30T15:54:20.661502+0000","last_became_peered":"2024-05-30T15:54:20.661502+0000","last_unstale":"2024-05-30T17:10:12.137521+0000","last_undegraded":"2024-05-30T17:10:12.137521+0000","last_fullsized":"2024-05-30T17:10:12.137521+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:53:08.492976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"265'722","reported_seq":2552,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.393245+0000","last_change":"2024-05-30T15:54:22.672464+0000","last_active":"2024-05-30T17:13:38.393245+0000","last_peered":"2024-05-30T17:13:38.393245+0000","last_clean":"2024-05-30T17:13:38.393245+0000","last_became_active":"2024-05-30T15:54:22.671588+0000","last_became_peered":"2024-05-30T15:54:22.671588+0000","last_unstale":"2024-05-30T17:13:38.393245+0000","last_undegraded":"2024-05-30T17:13:38.393245+0000","last_fullsized":"2024-05-30T17:13:38.393245+0000","mapping_epoch":28,"log_start":"265'700","ondisk_log_start":"265'700","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":700,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:46:49.368553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19819,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":1008,"num_read_kb":1208,"num_write":324,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":503,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:46.546239+0000","last_change":"2024-05-30T15:54:24.688069+0000","last_active":"2024-05-30T17:09:46.546239+0000","last_peered":"2024-05-30T17:09:46.546239+0000","last_clean":"2024-05-30T17:09:46.546239+0000","last_became_active":"2024-05-30T15:54:24.686641+0000","last_became_peered":"2024-05-30T15:54:24.686641+0000","last_unstale":"2024-05-30T17:09:46.546239+0000","last_undegraded":"2024-05-30T17:09:46.546239+0000","last_fullsized":"2024-05-30T17:09:46.546239+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:27:01.277040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"265'123","reported_seq":634,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:31.780995+0000","last_change":"2024-05-30T15:54:26.668522+0000","last_active":"2024-05-30T17:10:31.780995+0000","last_peered":"2024-05-30T17:10:31.780995+0000","last_clean":"2024-05-30T17:10:31.780995+0000","last_became_active":"2024-05-30T15:54:26.667751+0000","last_became_peered":"2024-05-30T15:54:26.667751+0000","last_unstale":"2024-05-30T17:10:31.780995+0000","last_undegraded":"2024-05-30T17:10:31.780995+0000","last_fullsized":"2024-05-30T17:10:31.780995+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":100,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:32:24.025165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":438,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":115,"num_write":291,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"265'326","reported_seq":1646,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:33.881393+0000","last_change":"2024-05-30T15:54:18.652138+0000","last_active":"2024-05-30T17:12:33.881393+0000","last_peered":"2024-05-30T17:12:33.881393+0000","last_clean":"2024-05-30T17:12:33.881393+0000","last_became_active":"2024-05-30T15:54:18.651853+0000","last_became_peered":"2024-05-30T15:54:18.651853+0000","last_unstale":"2024-05-30T17:12:33.881393+0000","last_undegraded":"2024-05-30T17:12:33.881393+0000","last_fullsized":"2024-05-30T17:12:33.881393+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":300,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:07:58.160840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1727,"num_read_kb":1636,"num_write":373,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":533,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:11.557705+0000","last_change":"2024-05-30T15:54:08.745221+0000","last_active":"2024-05-30T17:11:11.557705+0000","last_peered":"2024-05-30T17:11:11.557705+0000","last_clean":"2024-05-30T17:11:11.557705+0000","last_became_active":"2024-05-30T15:54:08.744898+0000","last_became_peered":"2024-05-30T15:54:08.744898+0000","last_unstale":"2024-05-30T17:11:11.557705+0000","last_undegraded":"2024-05-30T17:11:11.557705+0000","last_fullsized":"2024-05-30T17:11:11.557705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:38:33.683752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"265'185","reported_seq":813,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.301673+0000","last_change":"2024-05-30T15:54:16.650710+0000","last_active":"2024-05-30T17:13:38.301673+0000","last_peered":"2024-05-30T17:13:38.301673+0000","last_clean":"2024-05-30T17:13:38.301673+0000","last_became_active":"2024-05-30T15:54:16.650214+0000","last_became_peered":"2024-05-30T15:54:16.650214+0000","last_unstale":"2024-05-30T17:13:38.301673+0000","last_undegraded":"2024-05-30T17:13:38.301673+0000","last_fullsized":"2024-05-30T17:13:38.301673+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:41:20.055617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1911070,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":412,"num_read_kb":8332,"num_write":1154,"num_write_kb":25808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"27'1","reported_seq":514,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:11.545623+0000","last_change":"2024-05-30T15:54:20.667150+0000","last_active":"2024-05-30T17:10:11.545623+0000","last_peered":"2024-05-30T17:10:11.545623+0000","last_clean":"2024-05-30T17:10:11.545623+0000","last_became_active":"2024-05-30T15:54:20.666715+0000","last_became_peered":"2024-05-30T15:54:20.666715+0000","last_unstale":"2024-05-30T17:10:11.545623+0000","last_undegraded":"2024-05-30T17:10:11.545623+0000","last_fullsized":"2024-05-30T17:10:11.545623+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:57:01.669091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"265'259","reported_seq":1771,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.166581+0000","last_change":"2024-05-30T15:54:22.853392+0000","last_active":"2024-05-30T17:13:24.166581+0000","last_peered":"2024-05-30T17:13:24.166581+0000","last_clean":"2024-05-30T17:13:24.166581+0000","last_became_active":"2024-05-30T15:54:22.852604+0000","last_became_peered":"2024-05-30T15:54:22.852604+0000","last_unstale":"2024-05-30T17:13:24.166581+0000","last_undegraded":"2024-05-30T17:13:24.166581+0000","last_fullsized":"2024-05-30T17:13:24.166581+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":200,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:13:39.908798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":232,"num_read_kb":224,"num_write":183,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":509,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:31.586843+0000","last_change":"2024-05-30T15:54:24.680439+0000","last_active":"2024-05-30T17:11:31.586843+0000","last_peered":"2024-05-30T17:11:31.586843+0000","last_clean":"2024-05-30T17:11:31.586843+0000","last_became_active":"2024-05-30T15:54:24.679830+0000","last_became_peered":"2024-05-30T15:54:24.679830+0000","last_unstale":"2024-05-30T17:11:31.586843+0000","last_undegraded":"2024-05-30T17:11:31.586843+0000","last_fullsized":"2024-05-30T17:11:31.586843+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:05:18.168600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"265'251","reported_seq":860,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:19.239543+0000","last_change":"2024-05-30T15:54:26.699114+0000","last_active":"2024-05-30T17:13:19.239543+0000","last_peered":"2024-05-30T17:13:19.239543+0000","last_clean":"2024-05-30T17:13:19.239543+0000","last_became_active":"2024-05-30T15:54:26.696477+0000","last_became_peered":"2024-05-30T15:54:26.696477+0000","last_unstale":"2024-05-30T17:13:19.239543+0000","last_undegraded":"2024-05-30T17:13:19.239543+0000","last_fullsized":"2024-05-30T17:13:19.239543+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":200,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:15:58.988777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":476,"num_read_kb":500,"num_write":542,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"265'761","reported_seq":2321,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:35.663643+0000","last_change":"2024-05-30T15:54:18.640139+0000","last_active":"2024-05-30T17:11:35.663643+0000","last_peered":"2024-05-30T17:11:35.663643+0000","last_clean":"2024-05-30T17:11:35.663643+0000","last_became_active":"2024-05-30T15:54:18.639830+0000","last_became_peered":"2024-05-30T15:54:18.639830+0000","last_unstale":"2024-05-30T17:11:35.663643+0000","last_undegraded":"2024-05-30T17:11:35.663643+0000","last_fullsized":"2024-05-30T17:11:35.663643+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":700,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:05:32.448981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3481,"num_read_kb":3132,"num_write":1014,"num_write_kb":367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":530,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:17.151871+0000","last_change":"2024-05-30T15:54:08.575969+0000","last_active":"2024-05-30T17:11:17.151871+0000","last_peered":"2024-05-30T17:11:17.151871+0000","last_clean":"2024-05-30T17:11:17.151871+0000","last_became_active":"2024-05-30T15:54:08.574817+0000","last_became_peered":"2024-05-30T15:54:08.574817+0000","last_unstale":"2024-05-30T17:11:17.151871+0000","last_undegraded":"2024-05-30T17:11:17.151871+0000","last_fullsized":"2024-05-30T17:11:17.151871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:39:30.889837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"265'199","reported_seq":914,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.270675+0000","last_change":"2024-05-30T15:54:16.656169+0000","last_active":"2024-05-30T17:13:38.270675+0000","last_peered":"2024-05-30T17:13:38.270675+0000","last_clean":"2024-05-30T17:13:38.270675+0000","last_became_active":"2024-05-30T15:54:16.655347+0000","last_became_peered":"2024-05-30T15:54:16.655347+0000","last_unstale":"2024-05-30T17:13:38.270675+0000","last_undegraded":"2024-05-30T17:13:38.270675+0000","last_fullsized":"2024-05-30T17:13:38.270675+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:46:03.699500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":572,"num_read_kb":42208,"num_write":1148,"num_write_kb":36370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":516,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:41.561839+0000","last_change":"2024-05-30T15:54:20.651897+0000","last_active":"2024-05-30T17:09:41.561839+0000","last_peered":"2024-05-30T17:09:41.561839+0000","last_clean":"2024-05-30T17:09:41.561839+0000","last_became_active":"2024-05-30T15:54:20.651449+0000","last_became_peered":"2024-05-30T15:54:20.651449+0000","last_unstale":"2024-05-30T17:09:41.561839+0000","last_undegraded":"2024-05-30T17:09:41.561839+0000","last_fullsized":"2024-05-30T17:09:41.561839+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:14:31.569912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"265'160","reported_seq":732,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:16.572838+0000","last_change":"2024-05-30T15:54:22.679074+0000","last_active":"2024-05-30T17:10:16.572838+0000","last_peered":"2024-05-30T17:10:16.572838+0000","last_clean":"2024-05-30T17:10:16.572838+0000","last_became_active":"2024-05-30T15:54:22.673881+0000","last_became_peered":"2024-05-30T15:54:22.673881+0000","last_unstale":"2024-05-30T17:10:16.572838+0000","last_undegraded":"2024-05-30T17:10:16.572838+0000","last_fullsized":"2024-05-30T17:10:16.572838+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":100,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:18:38.849518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":200,"num_read_kb":192,"num_write":144,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":501,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:16.545213+0000","last_change":"2024-05-30T15:54:24.678710+0000","last_active":"2024-05-30T17:10:16.545213+0000","last_peered":"2024-05-30T17:10:16.545213+0000","last_clean":"2024-05-30T17:10:16.545213+0000","last_became_active":"2024-05-30T15:54:24.675916+0000","last_became_peered":"2024-05-30T15:54:24.675916+0000","last_unstale":"2024-05-30T17:10:16.545213+0000","last_undegraded":"2024-05-30T17:10:16.545213+0000","last_fullsized":"2024-05-30T17:10:16.545213+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:25:04.329603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"265'137","reported_seq":655,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.579761+0000","last_change":"2024-05-30T15:54:26.705809+0000","last_active":"2024-05-30T17:13:20.579761+0000","last_peered":"2024-05-30T17:13:20.579761+0000","last_clean":"2024-05-30T17:13:20.579761+0000","last_became_active":"2024-05-30T15:54:26.704217+0000","last_became_peered":"2024-05-30T15:54:26.704217+0000","last_unstale":"2024-05-30T17:13:20.579761+0000","last_undegraded":"2024-05-30T17:13:20.579761+0000","last_fullsized":"2024-05-30T17:13:20.579761+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":100,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:47:09.950969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":834,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":138,"num_read_kb":136,"num_write":341,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"265'396","reported_seq":1827,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.793186+0000","last_change":"2024-05-30T15:54:18.665286+0000","last_active":"2024-05-30T17:11:26.793186+0000","last_peered":"2024-05-30T17:11:26.793186+0000","last_clean":"2024-05-30T17:11:26.793186+0000","last_became_active":"2024-05-30T15:54:18.665034+0000","last_became_peered":"2024-05-30T15:54:18.665034+0000","last_unstale":"2024-05-30T17:11:26.793186+0000","last_undegraded":"2024-05-30T17:11:26.793186+0000","last_fullsized":"2024-05-30T17:11:26.793186+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":300,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:02:55.348044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2053,"num_read_kb":1920,"num_write":476,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"265'217","reported_seq":858,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.025951+0000","last_change":"2024-05-30T15:54:16.651678+0000","last_active":"2024-05-30T17:13:38.025951+0000","last_peered":"2024-05-30T17:13:38.025951+0000","last_clean":"2024-05-30T17:13:38.025951+0000","last_became_active":"2024-05-30T15:54:16.650735+0000","last_became_peered":"2024-05-30T15:54:16.650735+0000","last_unstale":"2024-05-30T17:13:38.025951+0000","last_undegraded":"2024-05-30T17:13:38.025951+0000","last_fullsized":"2024-05-30T17:13:38.025951+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":200,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:36:29.849007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079967,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":420,"num_read_kb":5617,"num_write":983,"num_write_kb":15732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":531,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:16.790986+0000","last_change":"2024-05-30T15:54:08.568214+0000","last_active":"2024-05-30T17:11:16.790986+0000","last_peered":"2024-05-30T17:11:16.790986+0000","last_clean":"2024-05-30T17:11:16.790986+0000","last_became_active":"2024-05-30T15:54:08.567900+0000","last_became_peered":"2024-05-30T15:54:08.567900+0000","last_unstale":"2024-05-30T17:11:16.790986+0000","last_undegraded":"2024-05-30T17:11:16.790986+0000","last_fullsized":"2024-05-30T17:11:16.790986+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:02:38.154765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":517,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.585770+0000","last_change":"2024-05-30T15:54:20.654924+0000","last_active":"2024-05-30T17:11:26.585770+0000","last_peered":"2024-05-30T17:11:26.585770+0000","last_clean":"2024-05-30T17:11:26.585770+0000","last_became_active":"2024-05-30T15:54:20.653790+0000","last_became_peered":"2024-05-30T15:54:20.653790+0000","last_unstale":"2024-05-30T17:11:26.585770+0000","last_undegraded":"2024-05-30T17:11:26.585770+0000","last_fullsized":"2024-05-30T17:11:26.585770+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:40:52.633245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"265'361","reported_seq":1061,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.225500+0000","last_change":"2024-05-30T15:54:22.674716+0000","last_active":"2024-05-30T17:13:38.225500+0000","last_peered":"2024-05-30T17:13:38.225500+0000","last_clean":"2024-05-30T17:13:38.225500+0000","last_became_active":"2024-05-30T15:54:22.673903+0000","last_became_peered":"2024-05-30T15:54:22.673903+0000","last_unstale":"2024-05-30T17:13:38.225500+0000","last_undegraded":"2024-05-30T17:13:38.225500+0000","last_fullsized":"2024-05-30T17:13:38.225500+0000","mapping_epoch":28,"log_start":"265'300","ondisk_log_start":"265'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":300,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:29:10.448451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11442,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1140,"num_read_kb":1226,"num_write":1674,"num_write_kb":1475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":513,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:31.751534+0000","last_change":"2024-05-30T15:54:24.679592+0000","last_active":"2024-05-30T17:11:31.751534+0000","last_peered":"2024-05-30T17:11:31.751534+0000","last_clean":"2024-05-30T17:11:31.751534+0000","last_became_active":"2024-05-30T15:54:24.677198+0000","last_became_peered":"2024-05-30T15:54:24.677198+0000","last_unstale":"2024-05-30T17:11:31.751534+0000","last_undegraded":"2024-05-30T17:11:31.751534+0000","last_fullsized":"2024-05-30T17:11:31.751534+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:47:11.591445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"265'117","reported_seq":635,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.126888+0000","last_change":"2024-05-30T15:54:26.676873+0000","last_active":"2024-05-30T17:13:20.126888+0000","last_peered":"2024-05-30T17:13:20.126888+0000","last_clean":"2024-05-30T17:13:20.126888+0000","last_became_active":"2024-05-30T15:54:26.676078+0000","last_became_peered":"2024-05-30T15:54:26.676078+0000","last_unstale":"2024-05-30T17:13:20.126888+0000","last_undegraded":"2024-05-30T17:13:20.126888+0000","last_fullsized":"2024-05-30T17:13:20.126888+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:52:51.891250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":727,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":114,"num_read_kb":108,"num_write":274,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"265'791","reported_seq":2423,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.705651+0000","last_change":"2024-05-30T15:54:18.644630+0000","last_active":"2024-05-30T17:13:20.705651+0000","last_peered":"2024-05-30T17:13:20.705651+0000","last_clean":"2024-05-30T17:13:20.705651+0000","last_became_active":"2024-05-30T15:54:18.642683+0000","last_became_peered":"2024-05-30T15:54:18.642683+0000","last_unstale":"2024-05-30T17:13:20.705651+0000","last_undegraded":"2024-05-30T17:13:20.705651+0000","last_fullsized":"2024-05-30T17:13:20.705651+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":700,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:23:29.409655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3596,"num_read_kb":3243,"num_write":1037,"num_write_kb":360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"265'184","reported_seq":827,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.372204+0000","last_change":"2024-05-30T15:54:16.664218+0000","last_active":"2024-05-30T17:13:38.372204+0000","last_peered":"2024-05-30T17:13:38.372204+0000","last_clean":"2024-05-30T17:13:38.372204+0000","last_became_active":"2024-05-30T15:54:16.663360+0000","last_became_peered":"2024-05-30T15:54:16.663360+0000","last_unstale":"2024-05-30T17:13:38.372204+0000","last_undegraded":"2024-05-30T17:13:38.372204+0000","last_fullsized":"2024-05-30T17:13:38.372204+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":100,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:22:13.108688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":661325,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":407,"num_read_kb":30298,"num_write":1063,"num_write_kb":42840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":530,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:36.572903+0000","last_change":"2024-05-30T15:54:08.577379+0000","last_active":"2024-05-30T17:11:36.572903+0000","last_peered":"2024-05-30T17:11:36.572903+0000","last_clean":"2024-05-30T17:11:36.572903+0000","last_became_active":"2024-05-30T15:54:08.576779+0000","last_became_peered":"2024-05-30T15:54:08.576779+0000","last_unstale":"2024-05-30T17:11:36.572903+0000","last_undegraded":"2024-05-30T17:11:36.572903+0000","last_fullsized":"2024-05-30T17:11:36.572903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:09:01.046338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":511,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.560832+0000","last_change":"2024-05-30T15:54:20.655937+0000","last_active":"2024-05-30T17:11:26.560832+0000","last_peered":"2024-05-30T17:11:26.560832+0000","last_clean":"2024-05-30T17:11:26.560832+0000","last_became_active":"2024-05-30T15:54:20.655220+0000","last_became_peered":"2024-05-30T15:54:20.655220+0000","last_unstale":"2024-05-30T17:11:26.560832+0000","last_undegraded":"2024-05-30T17:11:26.560832+0000","last_fullsized":"2024-05-30T17:11:26.560832+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:42:12.608224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"265'197","reported_seq":794,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:31.794205+0000","last_change":"2024-05-30T15:54:22.851840+0000","last_active":"2024-05-30T17:11:31.794205+0000","last_peered":"2024-05-30T17:11:31.794205+0000","last_clean":"2024-05-30T17:11:31.794205+0000","last_became_active":"2024-05-30T15:54:22.851338+0000","last_became_peered":"2024-05-30T15:54:22.851338+0000","last_unstale":"2024-05-30T17:11:31.794205+0000","last_undegraded":"2024-05-30T17:11:31.794205+0000","last_fullsized":"2024-05-30T17:11:31.794205+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":100,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:25:07.146907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":256,"num_read_kb":249,"num_write":168,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"265'557","reported_seq":2765,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:37.731958+0000","last_change":"2024-05-30T15:54:24.677979+0000","last_active":"2024-05-30T17:13:37.731958+0000","last_peered":"2024-05-30T17:13:37.731958+0000","last_clean":"2024-05-30T17:13:37.731958+0000","last_became_active":"2024-05-30T15:54:24.677176+0000","last_became_peered":"2024-05-30T15:54:24.677176+0000","last_unstale":"2024-05-30T17:13:37.731958+0000","last_undegraded":"2024-05-30T17:13:37.731958+0000","last_fullsized":"2024-05-30T17:13:37.731958+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":500,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:01:33.668829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"265'124","reported_seq":631,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.862550+0000","last_change":"2024-05-30T15:54:26.697967+0000","last_active":"2024-05-30T17:13:24.862550+0000","last_peered":"2024-05-30T17:13:24.862550+0000","last_clean":"2024-05-30T17:13:24.862550+0000","last_became_active":"2024-05-30T15:54:26.697294+0000","last_became_peered":"2024-05-30T15:54:26.697294+0000","last_unstale":"2024-05-30T17:13:24.862550+0000","last_undegraded":"2024-05-30T17:13:24.862550+0000","last_fullsized":"2024-05-30T17:13:24.862550+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":100,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:31:13.852916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1933,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":112,"num_read_kb":106,"num_write":288,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"265'716","reported_seq":2110,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.659105+0000","last_change":"2024-05-30T15:54:18.653261+0000","last_active":"2024-05-30T17:12:44.659105+0000","last_peered":"2024-05-30T17:12:44.659105+0000","last_clean":"2024-05-30T17:12:44.659105+0000","last_became_active":"2024-05-30T15:54:18.652981+0000","last_became_peered":"2024-05-30T15:54:18.652981+0000","last_unstale":"2024-05-30T17:12:44.659105+0000","last_undegraded":"2024-05-30T17:12:44.659105+0000","last_fullsized":"2024-05-30T17:12:44.659105+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":700,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:28:55.055044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3098,"num_read_kb":2766,"num_write":954,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"265'200","reported_seq":862,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.395015+0000","last_change":"2024-05-30T15:54:16.652512+0000","last_active":"2024-05-30T17:13:38.395015+0000","last_peered":"2024-05-30T17:13:38.395015+0000","last_clean":"2024-05-30T17:13:38.395015+0000","last_became_active":"2024-05-30T15:54:16.652033+0000","last_became_peered":"2024-05-30T15:54:16.652033+0000","last_unstale":"2024-05-30T17:13:38.395015+0000","last_undegraded":"2024-05-30T17:13:38.395015+0000","last_fullsized":"2024-05-30T17:13:38.395015+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":100,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:21:56.815021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":287689,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":515,"num_read_kb":15333,"num_write":1052,"num_write_kb":22470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":529,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:56.772799+0000","last_change":"2024-05-30T15:54:08.574181+0000","last_active":"2024-05-30T17:09:56.772799+0000","last_peered":"2024-05-30T17:09:56.772799+0000","last_clean":"2024-05-30T17:09:56.772799+0000","last_became_active":"2024-05-30T15:54:08.573754+0000","last_became_peered":"2024-05-30T15:54:08.573754+0000","last_unstale":"2024-05-30T17:09:56.772799+0000","last_undegraded":"2024-05-30T17:09:56.772799+0000","last_fullsized":"2024-05-30T17:09:56.772799+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:35:01.844361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":521,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.750400+0000","last_change":"2024-05-30T15:54:20.657666+0000","last_active":"2024-05-30T17:11:26.750400+0000","last_peered":"2024-05-30T17:11:26.750400+0000","last_clean":"2024-05-30T17:11:26.750400+0000","last_became_active":"2024-05-30T15:54:20.656941+0000","last_became_peered":"2024-05-30T15:54:20.656941+0000","last_unstale":"2024-05-30T17:11:26.750400+0000","last_undegraded":"2024-05-30T17:11:26.750400+0000","last_fullsized":"2024-05-30T17:11:26.750400+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:45:33.081783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"265'130","reported_seq":703,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:31.751592+0000","last_change":"2024-05-30T15:54:22.674809+0000","last_active":"2024-05-30T17:11:31.751592+0000","last_peered":"2024-05-30T17:11:31.751592+0000","last_clean":"2024-05-30T17:11:31.751592+0000","last_became_active":"2024-05-30T15:54:22.674027+0000","last_became_peered":"2024-05-30T15:54:22.674027+0000","last_unstale":"2024-05-30T17:11:31.751592+0000","last_undegraded":"2024-05-30T17:11:31.751592+0000","last_fullsized":"2024-05-30T17:11:31.751592+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:34:39.215953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":164,"num_read_kb":159,"num_write":108,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"265'550","reported_seq":2633,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:37.742773+0000","last_change":"2024-05-30T15:54:24.677368+0000","last_active":"2024-05-30T17:13:37.742773+0000","last_peered":"2024-05-30T17:13:37.742773+0000","last_clean":"2024-05-30T17:13:37.742773+0000","last_became_active":"2024-05-30T15:54:24.676608+0000","last_became_peered":"2024-05-30T15:54:24.676608+0000","last_unstale":"2024-05-30T17:13:37.742773+0000","last_undegraded":"2024-05-30T17:13:37.742773+0000","last_fullsized":"2024-05-30T17:13:37.742773+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":500,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:44:44.176765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"265'227","reported_seq":774,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:11.610342+0000","last_change":"2024-05-30T15:54:26.700019+0000","last_active":"2024-05-30T17:13:11.610342+0000","last_peered":"2024-05-30T17:13:11.610342+0000","last_clean":"2024-05-30T17:13:11.610342+0000","last_became_active":"2024-05-30T15:54:26.699148+0000","last_became_peered":"2024-05-30T15:54:26.699148+0000","last_unstale":"2024-05-30T17:13:11.610342+0000","last_undegraded":"2024-05-30T17:13:11.610342+0000","last_fullsized":"2024-05-30T17:13:11.610342+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":200,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:16:32.958184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":326,"num_read_kb":298,"num_write":504,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"265'747","reported_seq":2339,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:28.311866+0000","last_change":"2024-05-30T15:54:18.658639+0000","last_active":"2024-05-30T17:12:28.311866+0000","last_peered":"2024-05-30T17:12:28.311866+0000","last_clean":"2024-05-30T17:12:28.311866+0000","last_became_active":"2024-05-30T15:54:18.657842+0000","last_became_peered":"2024-05-30T15:54:18.657842+0000","last_unstale":"2024-05-30T17:12:28.311866+0000","last_undegraded":"2024-05-30T17:12:28.311866+0000","last_fullsized":"2024-05-30T17:12:28.311866+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":700,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:08:00.261523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3632,"num_read_kb":3285,"num_write":997,"num_write_kb":370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"265'238","reported_seq":994,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.370882+0000","last_change":"2024-05-30T15:54:16.657515+0000","last_active":"2024-05-30T17:13:38.370882+0000","last_peered":"2024-05-30T17:13:38.370882+0000","last_clean":"2024-05-30T17:13:38.370882+0000","last_became_active":"2024-05-30T15:54:16.657067+0000","last_became_peered":"2024-05-30T15:54:16.657067+0000","last_unstale":"2024-05-30T17:13:38.370882+0000","last_undegraded":"2024-05-30T17:13:38.370882+0000","last_fullsized":"2024-05-30T17:13:38.370882+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":200,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:03:23.557850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5418111,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":685,"num_read_kb":72437,"num_write":1157,"num_write_kb":30547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":527,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:01.541851+0000","last_change":"2024-05-30T15:54:08.578053+0000","last_active":"2024-05-30T17:10:01.541851+0000","last_peered":"2024-05-30T17:10:01.541851+0000","last_clean":"2024-05-30T17:10:01.541851+0000","last_became_active":"2024-05-30T15:54:08.577191+0000","last_became_peered":"2024-05-30T15:54:08.577191+0000","last_unstale":"2024-05-30T17:10:01.541851+0000","last_undegraded":"2024-05-30T17:10:01.541851+0000","last_fullsized":"2024-05-30T17:10:01.541851+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:38:36.674998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":515,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:41.578396+0000","last_change":"2024-05-30T15:54:20.653740+0000","last_active":"2024-05-30T17:10:41.578396+0000","last_peered":"2024-05-30T17:10:41.578396+0000","last_clean":"2024-05-30T17:10:41.578396+0000","last_became_active":"2024-05-30T15:54:20.653404+0000","last_became_peered":"2024-05-30T15:54:20.653404+0000","last_unstale":"2024-05-30T17:10:41.578396+0000","last_undegraded":"2024-05-30T17:10:41.578396+0000","last_fullsized":"2024-05-30T17:10:41.578396+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:17:55.928914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"265'346","reported_seq":1093,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.441027+0000","last_change":"2024-05-30T15:54:22.849490+0000","last_active":"2024-05-30T17:13:38.441027+0000","last_peered":"2024-05-30T17:13:38.441027+0000","last_clean":"2024-05-30T17:13:38.441027+0000","last_became_active":"2024-05-30T15:54:22.848761+0000","last_became_peered":"2024-05-30T15:54:22.848761+0000","last_unstale":"2024-05-30T17:13:38.441027+0000","last_undegraded":"2024-05-30T17:13:38.441027+0000","last_fullsized":"2024-05-30T17:13:38.441027+0000","mapping_epoch":28,"log_start":"265'300","ondisk_log_start":"265'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":300,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:40:14.623631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24415,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":525,"num_read_kb":761,"num_write":274,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:31.589903+0000","last_change":"2024-05-30T15:54:24.687584+0000","last_active":"2024-05-30T17:11:31.589903+0000","last_peered":"2024-05-30T17:11:31.589903+0000","last_clean":"2024-05-30T17:11:31.589903+0000","last_became_active":"2024-05-30T15:54:24.686801+0000","last_became_peered":"2024-05-30T15:54:24.686801+0000","last_unstale":"2024-05-30T17:11:31.589903+0000","last_undegraded":"2024-05-30T17:11:31.589903+0000","last_fullsized":"2024-05-30T17:11:31.589903+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:41:00.549932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"265'150","reported_seq":651,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:55.992645+0000","last_change":"2024-05-30T15:54:26.701426+0000","last_active":"2024-05-30T17:11:55.992645+0000","last_peered":"2024-05-30T17:11:55.992645+0000","last_clean":"2024-05-30T17:11:55.992645+0000","last_became_active":"2024-05-30T15:54:26.700325+0000","last_became_peered":"2024-05-30T15:54:26.700325+0000","last_unstale":"2024-05-30T17:11:55.992645+0000","last_undegraded":"2024-05-30T17:11:55.992645+0000","last_fullsized":"2024-05-30T17:11:55.992645+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":100,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:05:22.451018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":204,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":115,"num_write":342,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"265'149","reported_seq":722,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:21.183321+0000","last_change":"2024-05-30T15:54:26.687046+0000","last_active":"2024-05-30T17:13:21.183321+0000","last_peered":"2024-05-30T17:13:21.183321+0000","last_clean":"2024-05-30T17:13:21.183321+0000","last_became_active":"2024-05-30T15:54:26.686310+0000","last_became_peered":"2024-05-30T15:54:26.686310+0000","last_unstale":"2024-05-30T17:13:21.183321+0000","last_undegraded":"2024-05-30T17:13:21.183321+0000","last_fullsized":"2024-05-30T17:13:21.183321+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:16:45.128093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":227,"num_read_kb":222,"num_write":347,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"265'746","reported_seq":2046,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.763581+0000","last_change":"2024-05-30T15:54:18.674169+0000","last_active":"2024-05-30T17:13:18.763581+0000","last_peered":"2024-05-30T17:13:18.763581+0000","last_clean":"2024-05-30T17:13:18.763581+0000","last_became_active":"2024-05-30T15:54:18.673418+0000","last_became_peered":"2024-05-30T15:54:18.673418+0000","last_unstale":"2024-05-30T17:13:18.763581+0000","last_undegraded":"2024-05-30T17:13:18.763581+0000","last_fullsized":"2024-05-30T17:13:18.763581+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":700,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:47:19.313441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3258,"num_read_kb":2851,"num_write":1044,"num_write_kb":425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"265'202","reported_seq":821,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.163287+0000","last_change":"2024-05-30T15:54:54.451649+0000","last_active":"2024-05-30T17:13:38.163287+0000","last_peered":"2024-05-30T17:13:38.163287+0000","last_clean":"2024-05-30T17:13:38.163287+0000","last_became_active":"2024-05-30T15:54:54.450966+0000","last_became_peered":"2024-05-30T15:54:54.450966+0000","last_unstale":"2024-05-30T17:13:38.163287+0000","last_undegraded":"2024-05-30T17:13:38.163287+0000","last_fullsized":"2024-05-30T17:13:38.163287+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:16:23.208005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262272,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":401,"num_read_kb":38159,"num_write":1164,"num_write_kb":35962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":509,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.792056+0000","last_change":"2024-05-30T15:54:20.665689+0000","last_active":"2024-05-30T17:11:21.792056+0000","last_peered":"2024-05-30T17:11:21.792056+0000","last_clean":"2024-05-30T17:11:21.792056+0000","last_became_active":"2024-05-30T15:54:20.664882+0000","last_became_peered":"2024-05-30T15:54:20.664882+0000","last_unstale":"2024-05-30T17:11:21.792056+0000","last_undegraded":"2024-05-30T17:11:21.792056+0000","last_fullsized":"2024-05-30T17:11:21.792056+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:08:16.396333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"265'153","reported_seq":751,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.086291+0000","last_change":"2024-05-30T15:54:22.672900+0000","last_active":"2024-05-30T17:13:38.086291+0000","last_peered":"2024-05-30T17:13:38.086291+0000","last_clean":"2024-05-30T17:13:38.086291+0000","last_became_active":"2024-05-30T15:54:22.672170+0000","last_became_peered":"2024-05-30T15:54:22.672170+0000","last_unstale":"2024-05-30T17:13:38.086291+0000","last_undegraded":"2024-05-30T17:13:38.086291+0000","last_fullsized":"2024-05-30T17:13:38.086291+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":100,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:07:18.306116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5933,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":205,"num_read_kb":259,"num_write":110,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":501,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:06.556338+0000","last_change":"2024-05-30T15:54:24.688552+0000","last_active":"2024-05-30T17:09:06.556338+0000","last_peered":"2024-05-30T17:09:06.556338+0000","last_clean":"2024-05-30T17:09:06.556338+0000","last_became_active":"2024-05-30T15:54:24.688053+0000","last_became_peered":"2024-05-30T15:54:24.688053+0000","last_unstale":"2024-05-30T17:09:06.556338+0000","last_undegraded":"2024-05-30T17:09:06.556338+0000","last_fullsized":"2024-05-30T17:09:06.556338+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:53:25.256385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"265'117","reported_seq":642,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:46.579861+0000","last_change":"2024-05-30T15:54:26.702292+0000","last_active":"2024-05-30T17:10:46.579861+0000","last_peered":"2024-05-30T17:10:46.579861+0000","last_clean":"2024-05-30T17:10:46.579861+0000","last_became_active":"2024-05-30T15:54:26.701588+0000","last_became_peered":"2024-05-30T15:54:26.701588+0000","last_unstale":"2024-05-30T17:10:46.579861+0000","last_undegraded":"2024-05-30T17:10:46.579861+0000","last_fullsized":"2024-05-30T17:10:46.579861+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:13:38.040108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":826,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":172,"num_read_kb":153,"num_write":293,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"265'321","reported_seq":1759,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:55.974538+0000","last_change":"2024-05-30T15:54:18.671021+0000","last_active":"2024-05-30T17:11:55.974538+0000","last_peered":"2024-05-30T17:11:55.974538+0000","last_clean":"2024-05-30T17:11:55.974538+0000","last_became_active":"2024-05-30T15:54:18.670707+0000","last_became_peered":"2024-05-30T15:54:18.670707+0000","last_unstale":"2024-05-30T17:11:55.974538+0000","last_undegraded":"2024-05-30T17:11:55.974538+0000","last_fullsized":"2024-05-30T17:11:55.974538+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":300,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:20:52.055626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1910,"num_read_kb":1828,"num_write":376,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"265'219","reported_seq":883,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.162489+0000","last_change":"2024-05-30T15:54:16.663769+0000","last_active":"2024-05-30T17:13:38.162489+0000","last_peered":"2024-05-30T17:13:38.162489+0000","last_clean":"2024-05-30T17:13:38.162489+0000","last_became_active":"2024-05-30T15:54:16.660427+0000","last_became_peered":"2024-05-30T15:54:16.660427+0000","last_unstale":"2024-05-30T17:13:38.162489+0000","last_undegraded":"2024-05-30T17:13:38.162489+0000","last_fullsized":"2024-05-30T17:13:38.162489+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":200,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:37:35.566472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262181,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":518,"num_read_kb":33224,"num_write":1148,"num_write_kb":46290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":510,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.587706+0000","last_change":"2024-05-30T15:54:20.649980+0000","last_active":"2024-05-30T17:11:21.587706+0000","last_peered":"2024-05-30T17:11:21.587706+0000","last_clean":"2024-05-30T17:11:21.587706+0000","last_became_active":"2024-05-30T15:54:20.649330+0000","last_became_peered":"2024-05-30T15:54:20.649330+0000","last_unstale":"2024-05-30T17:11:21.587706+0000","last_undegraded":"2024-05-30T17:11:21.587706+0000","last_fullsized":"2024-05-30T17:11:21.587706+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:16:31.107653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"265'71","reported_seq":595,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:06.552569+0000","last_change":"2024-05-30T15:54:22.663453+0000","last_active":"2024-05-30T17:09:06.552569+0000","last_peered":"2024-05-30T17:09:06.552569+0000","last_clean":"2024-05-30T17:09:06.552569+0000","last_became_active":"2024-05-30T15:54:22.662738+0000","last_became_peered":"2024-05-30T15:54:22.662738+0000","last_unstale":"2024-05-30T17:09:06.552569+0000","last_undegraded":"2024-05-30T17:09:06.552569+0000","last_fullsized":"2024-05-30T17:09:06.552569+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:42:21.227185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":50,"num_write":40,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":499,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:06.750855+0000","last_change":"2024-05-30T15:54:24.682005+0000","last_active":"2024-05-30T17:09:06.750855+0000","last_peered":"2024-05-30T17:09:06.750855+0000","last_clean":"2024-05-30T17:09:06.750855+0000","last_became_active":"2024-05-30T15:54:24.680578+0000","last_became_peered":"2024-05-30T15:54:24.680578+0000","last_unstale":"2024-05-30T17:09:06.750855+0000","last_undegraded":"2024-05-30T17:09:06.750855+0000","last_fullsized":"2024-05-30T17:09:06.750855+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:54:25.321085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"265'212","reported_seq":772,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.665178+0000","last_change":"2024-05-30T15:54:26.687158+0000","last_active":"2024-05-30T17:13:18.665178+0000","last_peered":"2024-05-30T17:13:18.665178+0000","last_clean":"2024-05-30T17:13:18.665178+0000","last_became_active":"2024-05-30T15:54:26.686553+0000","last_became_peered":"2024-05-30T15:54:26.686553+0000","last_unstale":"2024-05-30T17:13:18.665178+0000","last_undegraded":"2024-05-30T17:13:18.665178+0000","last_fullsized":"2024-05-30T17:13:18.665178+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:03:50.128325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":816,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":414,"num_read_kb":383,"num_write":513,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"265'413","reported_seq":1790,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.659799+0000","last_change":"2024-05-30T15:54:18.660456+0000","last_active":"2024-05-30T17:12:44.659799+0000","last_peered":"2024-05-30T17:12:44.659799+0000","last_clean":"2024-05-30T17:12:44.659799+0000","last_became_active":"2024-05-30T15:54:18.660089+0000","last_became_peered":"2024-05-30T15:54:18.660089+0000","last_unstale":"2024-05-30T17:12:44.659799+0000","last_undegraded":"2024-05-30T17:12:44.659799+0000","last_fullsized":"2024-05-30T17:12:44.659799+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":400,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:29:46.343317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2036,"num_read_kb":1890,"num_write":519,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"265'217","reported_seq":886,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:37.926216+0000","last_change":"2024-05-30T15:54:16.650752+0000","last_active":"2024-05-30T17:13:37.926216+0000","last_peered":"2024-05-30T17:13:37.926216+0000","last_clean":"2024-05-30T17:13:37.926216+0000","last_became_active":"2024-05-30T15:54:16.650025+0000","last_became_peered":"2024-05-30T15:54:16.650025+0000","last_unstale":"2024-05-30T17:13:37.926216+0000","last_undegraded":"2024-05-30T17:13:37.926216+0000","last_fullsized":"2024-05-30T17:13:37.926216+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":200,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:14:35.914964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5569939,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":500,"num_read_kb":102670,"num_write":1113,"num_write_kb":39714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":508,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.568814+0000","last_change":"2024-05-30T15:54:20.657385+0000","last_active":"2024-05-30T17:11:21.568814+0000","last_peered":"2024-05-30T17:11:21.568814+0000","last_clean":"2024-05-30T17:11:21.568814+0000","last_became_active":"2024-05-30T15:54:20.656663+0000","last_became_peered":"2024-05-30T15:54:20.656663+0000","last_unstale":"2024-05-30T17:11:21.568814+0000","last_undegraded":"2024-05-30T17:11:21.568814+0000","last_fullsized":"2024-05-30T17:11:21.568814+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:31:26.705085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"265'186","reported_seq":800,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:37.869066+0000","last_change":"2024-05-30T15:54:22.676934+0000","last_active":"2024-05-30T17:13:37.869066+0000","last_peered":"2024-05-30T17:13:37.869066+0000","last_clean":"2024-05-30T17:13:37.869066+0000","last_became_active":"2024-05-30T15:54:22.673673+0000","last_became_peered":"2024-05-30T15:54:22.673673+0000","last_unstale":"2024-05-30T17:13:37.869066+0000","last_undegraded":"2024-05-30T17:13:37.869066+0000","last_fullsized":"2024-05-30T17:13:37.869066+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":100,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:34:48.241130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7114,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":259,"num_read_kb":332,"num_write":146,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":498,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:06.529823+0000","last_change":"2024-05-30T15:54:24.676055+0000","last_active":"2024-05-30T17:09:06.529823+0000","last_peered":"2024-05-30T17:09:06.529823+0000","last_clean":"2024-05-30T17:09:06.529823+0000","last_became_active":"2024-05-30T15:54:24.675289+0000","last_became_peered":"2024-05-30T15:54:24.675289+0000","last_unstale":"2024-05-30T17:09:06.529823+0000","last_undegraded":"2024-05-30T17:09:06.529823+0000","last_fullsized":"2024-05-30T17:09:06.529823+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:37:01.468842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"265'138","reported_seq":634,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:27.127163+0000","last_change":"2024-05-30T15:54:26.704882+0000","last_active":"2024-05-30T17:09:27.127163+0000","last_peered":"2024-05-30T17:09:27.127163+0000","last_clean":"2024-05-30T17:09:27.127163+0000","last_became_active":"2024-05-30T15:54:26.703433+0000","last_became_peered":"2024-05-30T15:54:26.703433+0000","last_unstale":"2024-05-30T17:09:27.127163+0000","last_undegraded":"2024-05-30T17:09:27.127163+0000","last_fullsized":"2024-05-30T17:09:27.127163+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":100,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:19:12.878679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":856,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":110,"num_write":329,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"265'863","reported_seq":2491,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:12.349961+0000","last_change":"2024-05-30T15:54:18.649560+0000","last_active":"2024-05-30T17:13:12.349961+0000","last_peered":"2024-05-30T17:13:12.349961+0000","last_clean":"2024-05-30T17:13:12.349961+0000","last_became_active":"2024-05-30T15:54:18.649273+0000","last_became_peered":"2024-05-30T15:54:18.649273+0000","last_unstale":"2024-05-30T17:13:12.349961+0000","last_undegraded":"2024-05-30T17:13:12.349961+0000","last_fullsized":"2024-05-30T17:13:12.349961+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":800,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:31:47.570692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3918,"num_read_kb":3498,"num_write":1187,"num_write_kb":458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"265'207","reported_seq":867,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.237463+0000","last_change":"2024-05-30T15:54:16.649854+0000","last_active":"2024-05-30T17:13:38.237463+0000","last_peered":"2024-05-30T17:13:38.237463+0000","last_clean":"2024-05-30T17:13:38.237463+0000","last_became_active":"2024-05-30T15:54:16.649121+0000","last_became_peered":"2024-05-30T15:54:16.649121+0000","last_unstale":"2024-05-30T17:13:38.237463+0000","last_undegraded":"2024-05-30T17:13:38.237463+0000","last_fullsized":"2024-05-30T17:13:38.237463+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":100,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:25:59.798639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1029196,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":493,"num_read_kb":65907,"num_write":1120,"num_write_kb":54558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":507,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:21.559613+0000","last_change":"2024-05-30T15:54:20.668691+0000","last_active":"2024-05-30T17:11:21.559613+0000","last_peered":"2024-05-30T17:11:21.559613+0000","last_clean":"2024-05-30T17:11:21.559613+0000","last_became_active":"2024-05-30T15:54:20.667863+0000","last_became_peered":"2024-05-30T15:54:20.667863+0000","last_unstale":"2024-05-30T17:11:21.559613+0000","last_undegraded":"2024-05-30T17:11:21.559613+0000","last_fullsized":"2024-05-30T17:11:21.559613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:31.629459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"265'143","reported_seq":732,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.417964+0000","last_change":"2024-05-30T15:54:22.665742+0000","last_active":"2024-05-30T17:13:38.417964+0000","last_peered":"2024-05-30T17:13:38.417964+0000","last_clean":"2024-05-30T17:13:38.417964+0000","last_became_active":"2024-05-30T15:54:22.664977+0000","last_became_peered":"2024-05-30T15:54:22.664977+0000","last_unstale":"2024-05-30T17:13:38.417964+0000","last_undegraded":"2024-05-30T17:13:38.417964+0000","last_fullsized":"2024-05-30T17:13:38.417964+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":100,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:20:57.960795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7290,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":187,"num_read_kb":260,"num_write":98,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":504,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:06.552717+0000","last_change":"2024-05-30T15:54:24.677137+0000","last_active":"2024-05-30T17:09:06.552717+0000","last_peered":"2024-05-30T17:09:06.552717+0000","last_clean":"2024-05-30T17:09:06.552717+0000","last_became_active":"2024-05-30T15:54:24.676707+0000","last_became_peered":"2024-05-30T15:54:24.676707+0000","last_unstale":"2024-05-30T17:09:06.552717+0000","last_undegraded":"2024-05-30T17:09:06.552717+0000","last_fullsized":"2024-05-30T17:09:06.552717+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:37:14.112070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"265'442","reported_seq":2099,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:28.309213+0000","last_change":"2024-05-30T15:54:18.671501+0000","last_active":"2024-05-30T17:12:28.309213+0000","last_peered":"2024-05-30T17:12:28.309213+0000","last_clean":"2024-05-30T17:12:28.309213+0000","last_became_active":"2024-05-30T15:54:18.670756+0000","last_became_peered":"2024-05-30T15:54:18.670756+0000","last_unstale":"2024-05-30T17:12:28.309213+0000","last_undegraded":"2024-05-30T17:12:28.309213+0000","last_fullsized":"2024-05-30T17:12:28.309213+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":400,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:50:30.677735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2383,"num_read_kb":2277,"num_write":522,"num_write_kb":97,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"265'164","reported_seq":768,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.395002+0000","last_change":"2024-05-30T15:54:16.654636+0000","last_active":"2024-05-30T17:13:38.395002+0000","last_peered":"2024-05-30T17:13:38.395002+0000","last_clean":"2024-05-30T17:13:38.395002+0000","last_became_active":"2024-05-30T15:54:16.653560+0000","last_became_peered":"2024-05-30T15:54:16.653560+0000","last_unstale":"2024-05-30T17:13:38.395002+0000","last_undegraded":"2024-05-30T17:13:38.395002+0000","last_fullsized":"2024-05-30T17:13:38.395002+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":100,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:47:43.349582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5417712,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":326,"num_read_kb":31913,"num_write":888,"num_write_kb":20958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":518,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:41.574743+0000","last_change":"2024-05-30T15:54:20.662992+0000","last_active":"2024-05-30T17:10:41.574743+0000","last_peered":"2024-05-30T17:10:41.574743+0000","last_clean":"2024-05-30T17:10:41.574743+0000","last_became_active":"2024-05-30T15:54:20.662669+0000","last_became_peered":"2024-05-30T15:54:20.662669+0000","last_unstale":"2024-05-30T17:10:41.574743+0000","last_undegraded":"2024-05-30T17:10:41.574743+0000","last_fullsized":"2024-05-30T17:10:41.574743+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:46:17.417038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"265'224","reported_seq":873,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.406026+0000","last_change":"2024-05-30T15:54:22.664232+0000","last_active":"2024-05-30T17:13:38.406026+0000","last_peered":"2024-05-30T17:13:38.406026+0000","last_clean":"2024-05-30T17:13:38.406026+0000","last_became_active":"2024-05-30T15:54:22.663518+0000","last_became_peered":"2024-05-30T15:54:22.663518+0000","last_unstale":"2024-05-30T17:13:38.406026+0000","last_undegraded":"2024-05-30T17:13:38.406026+0000","last_fullsized":"2024-05-30T17:13:38.406026+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":200,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:51:09.801047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11676,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":342,"num_read_kb":490,"num_write":222,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":497,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:27.127216+0000","last_change":"2024-05-30T15:54:24.683358+0000","last_active":"2024-05-30T17:09:27.127216+0000","last_peered":"2024-05-30T17:09:27.127216+0000","last_clean":"2024-05-30T17:09:27.127216+0000","last_became_active":"2024-05-30T15:54:24.682609+0000","last_became_peered":"2024-05-30T15:54:24.682609+0000","last_unstale":"2024-05-30T17:09:27.127216+0000","last_undegraded":"2024-05-30T17:09:27.127216+0000","last_fullsized":"2024-05-30T17:09:27.127216+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:15:33.566334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"265'149","reported_seq":660,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:12.359173+0000","last_change":"2024-05-30T15:54:26.695969+0000","last_active":"2024-05-30T17:11:12.359173+0000","last_peered":"2024-05-30T17:11:12.359173+0000","last_clean":"2024-05-30T17:11:12.359173+0000","last_became_active":"2024-05-30T15:54:26.695210+0000","last_became_peered":"2024-05-30T15:54:26.695210+0000","last_unstale":"2024-05-30T17:11:12.359173+0000","last_undegraded":"2024-05-30T17:11:12.359173+0000","last_fullsized":"2024-05-30T17:11:12.359173+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:13:43.303060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":643,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":167,"num_read_kb":179,"num_write":338,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"265'854","reported_seq":2475,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.655415+0000","last_change":"2024-05-30T15:54:18.641464+0000","last_active":"2024-05-30T17:12:44.655415+0000","last_peered":"2024-05-30T17:12:44.655415+0000","last_clean":"2024-05-30T17:12:44.655415+0000","last_became_active":"2024-05-30T15:54:18.640739+0000","last_became_peered":"2024-05-30T15:54:18.640739+0000","last_unstale":"2024-05-30T17:12:44.655415+0000","last_undegraded":"2024-05-30T17:12:44.655415+0000","last_fullsized":"2024-05-30T17:12:44.655415+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":800,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:05:20.428299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4053,"num_read_kb":3626,"num_write":1150,"num_write_kb":447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"265'245","reported_seq":978,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.264572+0000","last_change":"2024-05-30T15:54:54.425702+0000","last_active":"2024-05-30T17:13:38.264572+0000","last_peered":"2024-05-30T17:13:38.264572+0000","last_clean":"2024-05-30T17:13:38.264572+0000","last_became_active":"2024-05-30T15:54:54.424825+0000","last_became_peered":"2024-05-30T15:54:54.424825+0000","last_unstale":"2024-05-30T17:13:38.264572+0000","last_undegraded":"2024-05-30T17:13:38.264572+0000","last_fullsized":"2024-05-30T17:13:38.264572+0000","mapping_epoch":35,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":200,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:43:34.336982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5285497,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":887,"num_read_kb":12756,"num_write":1230,"num_write_kb":34615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"34'3","reported_seq":838,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:37.467959+0000","last_change":"2024-05-30T15:54:20.664818+0000","last_active":"2024-05-30T17:13:37.467959+0000","last_peered":"2024-05-30T17:13:37.467959+0000","last_clean":"2024-05-30T17:13:37.467959+0000","last_became_active":"2024-05-30T15:54:20.664095+0000","last_became_peered":"2024-05-30T15:54:20.664095+0000","last_unstale":"2024-05-30T17:13:37.467959+0000","last_undegraded":"2024-05-30T17:13:37.467959+0000","last_fullsized":"2024-05-30T17:13:37.467959+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:27:22.067783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":494,"num_read_kb":492,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"265'194","reported_seq":804,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.430826+0000","last_change":"2024-05-30T15:54:22.665708+0000","last_active":"2024-05-30T17:13:38.430826+0000","last_peered":"2024-05-30T17:13:38.430826+0000","last_clean":"2024-05-30T17:13:38.430826+0000","last_became_active":"2024-05-30T15:54:22.664997+0000","last_became_peered":"2024-05-30T15:54:22.664997+0000","last_unstale":"2024-05-30T17:13:38.430826+0000","last_undegraded":"2024-05-30T17:13:38.430826+0000","last_fullsized":"2024-05-30T17:13:38.430826+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":100,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:45:04.949797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25625,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":290,"num_read_kb":412,"num_write":192,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":498,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:26.767043+0000","last_change":"2024-05-30T15:54:24.678998+0000","last_active":"2024-05-30T17:09:26.767043+0000","last_peered":"2024-05-30T17:09:26.767043+0000","last_clean":"2024-05-30T17:09:26.767043+0000","last_became_active":"2024-05-30T15:54:24.675583+0000","last_became_peered":"2024-05-30T15:54:24.675583+0000","last_unstale":"2024-05-30T17:09:26.767043+0000","last_undegraded":"2024-05-30T17:09:26.767043+0000","last_fullsized":"2024-05-30T17:09:26.767043+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:17:21.969608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"265'140","reported_seq":655,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:26.560874+0000","last_change":"2024-05-30T15:54:26.703108+0000","last_active":"2024-05-30T17:11:26.560874+0000","last_peered":"2024-05-30T17:11:26.560874+0000","last_clean":"2024-05-30T17:11:26.560874+0000","last_became_active":"2024-05-30T15:54:26.700392+0000","last_became_peered":"2024-05-30T15:54:26.700392+0000","last_unstale":"2024-05-30T17:11:26.560874+0000","last_undegraded":"2024-05-30T17:11:26.560874+0000","last_fullsized":"2024-05-30T17:11:26.560874+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":100,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:25:14.903459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2627,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":177,"num_read_kb":174,"num_write":403,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"265'368","reported_seq":1738,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:33.884093+0000","last_change":"2024-05-30T15:54:54.420129+0000","last_active":"2024-05-30T17:12:33.884093+0000","last_peered":"2024-05-30T17:12:33.884093+0000","last_clean":"2024-05-30T17:12:33.884093+0000","last_became_active":"2024-05-30T15:54:54.419468+0000","last_became_peered":"2024-05-30T15:54:54.419468+0000","last_unstale":"2024-05-30T17:12:33.884093+0000","last_undegraded":"2024-05-30T17:12:33.884093+0000","last_fullsized":"2024-05-30T17:12:33.884093+0000","mapping_epoch":35,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":300,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:08:55.802490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1898,"num_read_kb":1782,"num_write":455,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"265'159","reported_seq":884,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.350347+0000","last_change":"2024-05-30T15:54:16.677283+0000","last_active":"2024-05-30T17:13:38.350347+0000","last_peered":"2024-05-30T17:13:38.350347+0000","last_clean":"2024-05-30T17:13:38.350347+0000","last_became_active":"2024-05-30T15:54:16.676440+0000","last_became_peered":"2024-05-30T15:54:16.676440+0000","last_unstale":"2024-05-30T17:13:38.350347+0000","last_undegraded":"2024-05-30T17:13:38.350347+0000","last_fullsized":"2024-05-30T17:13:38.350347+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:16:12.516128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456485,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":869,"num_read_kb":45599,"num_write":925,"num_write_kb":34636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":514,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.723009+0000","last_change":"2024-05-30T15:54:20.656111+0000","last_active":"2024-05-30T17:09:21.723009+0000","last_peered":"2024-05-30T17:09:21.723009+0000","last_clean":"2024-05-30T17:09:21.723009+0000","last_became_active":"2024-05-30T15:54:20.655388+0000","last_became_peered":"2024-05-30T15:54:20.655388+0000","last_unstale":"2024-05-30T17:09:21.723009+0000","last_undegraded":"2024-05-30T17:09:21.723009+0000","last_fullsized":"2024-05-30T17:09:21.723009+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:07:28.254294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"265'165","reported_seq":768,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.148676+0000","last_change":"2024-05-30T15:54:22.849861+0000","last_active":"2024-05-30T17:13:38.148676+0000","last_peered":"2024-05-30T17:13:38.148676+0000","last_clean":"2024-05-30T17:13:38.148676+0000","last_became_active":"2024-05-30T15:54:22.849193+0000","last_became_peered":"2024-05-30T15:54:22.849193+0000","last_unstale":"2024-05-30T17:13:38.148676+0000","last_undegraded":"2024-05-30T17:13:38.148676+0000","last_fullsized":"2024-05-30T17:13:38.148676+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":100,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:51:38.667242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11086,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":221,"num_read_kb":302,"num_write":124,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:41.783470+0000","last_change":"2024-05-30T15:54:24.682047+0000","last_active":"2024-05-30T17:10:41.783470+0000","last_peered":"2024-05-30T17:10:41.783470+0000","last_clean":"2024-05-30T17:10:41.783470+0000","last_became_active":"2024-05-30T15:54:24.681609+0000","last_became_peered":"2024-05-30T15:54:24.681609+0000","last_unstale":"2024-05-30T17:10:41.783470+0000","last_undegraded":"2024-05-30T17:10:41.783470+0000","last_fullsized":"2024-05-30T17:10:41.783470+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:36:01.484941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"265'129","reported_seq":631,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:24.853790+0000","last_change":"2024-05-30T15:54:26.706376+0000","last_active":"2024-05-30T17:13:24.853790+0000","last_peered":"2024-05-30T17:13:24.853790+0000","last_clean":"2024-05-30T17:13:24.853790+0000","last_became_active":"2024-05-30T15:54:26.704842+0000","last_became_peered":"2024-05-30T15:54:26.704842+0000","last_unstale":"2024-05-30T17:13:24.853790+0000","last_undegraded":"2024-05-30T17:13:24.853790+0000","last_fullsized":"2024-05-30T17:13:24.853790+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":100,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:31:31.301239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":383,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":127,"num_read_kb":122,"num_write":322,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"265'753","reported_seq":2251,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:51.565345+0000","last_change":"2024-05-30T15:54:18.635089+0000","last_active":"2024-05-30T17:11:51.565345+0000","last_peered":"2024-05-30T17:11:51.565345+0000","last_clean":"2024-05-30T17:11:51.565345+0000","last_became_active":"2024-05-30T15:54:18.634803+0000","last_became_peered":"2024-05-30T15:54:18.634803+0000","last_unstale":"2024-05-30T17:11:51.565345+0000","last_undegraded":"2024-05-30T17:11:51.565345+0000","last_fullsized":"2024-05-30T17:11:51.565345+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":700,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:29:09.056772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":3476,"num_read_kb":3109,"num_write":1006,"num_write_kb":389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"265'190","reported_seq":814,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.238930+0000","last_change":"2024-05-30T15:54:54.431157+0000","last_active":"2024-05-30T17:13:38.238930+0000","last_peered":"2024-05-30T17:13:38.238930+0000","last_clean":"2024-05-30T17:13:38.238930+0000","last_became_active":"2024-05-30T15:54:54.430490+0000","last_became_peered":"2024-05-30T15:54:54.430490+0000","last_unstale":"2024-05-30T17:13:38.238930+0000","last_undegraded":"2024-05-30T17:13:38.238930+0000","last_fullsized":"2024-05-30T17:13:38.238930+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":100,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:47:17.028289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":481,"num_read_kb":40323,"num_write":1075,"num_write_kb":31617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.765694+0000","last_change":"2024-05-30T15:54:20.651307+0000","last_active":"2024-05-30T17:09:21.765694+0000","last_peered":"2024-05-30T17:09:21.765694+0000","last_clean":"2024-05-30T17:09:21.765694+0000","last_became_active":"2024-05-30T15:54:20.650944+0000","last_became_peered":"2024-05-30T15:54:20.650944+0000","last_unstale":"2024-05-30T17:09:21.765694+0000","last_undegraded":"2024-05-30T17:09:21.765694+0000","last_fullsized":"2024-05-30T17:09:21.765694+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:44:01.827195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"265'113","reported_seq":671,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:41.740281+0000","last_change":"2024-05-30T15:54:22.851946+0000","last_active":"2024-05-30T17:10:41.740281+0000","last_peered":"2024-05-30T17:10:41.740281+0000","last_clean":"2024-05-30T17:10:41.740281+0000","last_became_active":"2024-05-30T15:54:22.850865+0000","last_became_peered":"2024-05-30T15:54:22.850865+0000","last_unstale":"2024-05-30T17:10:41.740281+0000","last_undegraded":"2024-05-30T17:10:41.740281+0000","last_fullsized":"2024-05-30T17:10:41.740281+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":100,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:49:42.844967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":127,"num_read_kb":121,"num_write":92,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"265'541","reported_seq":4169,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:37.727977+0000","last_change":"2024-05-30T15:54:24.681428+0000","last_active":"2024-05-30T17:13:37.727977+0000","last_peered":"2024-05-30T17:13:37.727977+0000","last_clean":"2024-05-30T17:13:37.727977+0000","last_became_active":"2024-05-30T15:54:24.680695+0000","last_became_peered":"2024-05-30T15:54:24.680695+0000","last_unstale":"2024-05-30T17:13:37.727977+0000","last_undegraded":"2024-05-30T17:13:37.727977+0000","last_fullsized":"2024-05-30T17:13:37.727977+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":500,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:53:20.536261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"265'134","reported_seq":641,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:35.623018+0000","last_change":"2024-05-30T15:54:26.696600+0000","last_active":"2024-05-30T17:11:35.623018+0000","last_peered":"2024-05-30T17:11:35.623018+0000","last_clean":"2024-05-30T17:11:35.623018+0000","last_became_active":"2024-05-30T15:54:26.695842+0000","last_became_peered":"2024-05-30T15:54:26.695842+0000","last_unstale":"2024-05-30T17:11:35.623018+0000","last_undegraded":"2024-05-30T17:11:35.623018+0000","last_fullsized":"2024-05-30T17:11:35.623018+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:01:21.719509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":418,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":133,"num_read_kb":127,"num_write":334,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"265'423","reported_seq":1942,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.652772+0000","last_change":"2024-05-30T15:54:18.661318+0000","last_active":"2024-05-30T17:13:20.652772+0000","last_peered":"2024-05-30T17:13:20.652772+0000","last_clean":"2024-05-30T17:13:20.652772+0000","last_became_active":"2024-05-30T15:54:18.660567+0000","last_became_peered":"2024-05-30T15:54:18.660567+0000","last_unstale":"2024-05-30T17:13:20.652772+0000","last_undegraded":"2024-05-30T17:13:20.652772+0000","last_fullsized":"2024-05-30T17:13:20.652772+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":400,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:24:20.096151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2226,"num_read_kb":2104,"num_write":504,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"265'306","reported_seq":997,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.334529+0000","last_change":"2024-05-30T15:54:16.686321+0000","last_active":"2024-05-30T17:13:38.334529+0000","last_peered":"2024-05-30T17:13:38.334529+0000","last_clean":"2024-05-30T17:13:38.334529+0000","last_became_active":"2024-05-30T15:54:16.685691+0000","last_became_peered":"2024-05-30T15:54:16.685691+0000","last_unstale":"2024-05-30T17:13:38.334529+0000","last_undegraded":"2024-05-30T17:13:38.334529+0000","last_fullsized":"2024-05-30T17:13:38.334529+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":200,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:37:07.773513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":764,"num_read_kb":1764,"num_write":1253,"num_write_kb":18095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":505,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.754576+0000","last_change":"2024-05-30T15:54:20.670328+0000","last_active":"2024-05-30T17:09:21.754576+0000","last_peered":"2024-05-30T17:09:21.754576+0000","last_clean":"2024-05-30T17:09:21.754576+0000","last_became_active":"2024-05-30T15:54:20.669751+0000","last_became_peered":"2024-05-30T15:54:20.669751+0000","last_unstale":"2024-05-30T17:09:21.754576+0000","last_undegraded":"2024-05-30T17:09:21.754576+0000","last_fullsized":"2024-05-30T17:09:21.754576+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:06:46.261517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"265'217","reported_seq":828,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.273680+0000","last_change":"2024-05-30T15:54:22.672899+0000","last_active":"2024-05-30T17:13:38.273680+0000","last_peered":"2024-05-30T17:13:38.273680+0000","last_clean":"2024-05-30T17:13:38.273680+0000","last_became_active":"2024-05-30T15:54:22.672078+0000","last_became_peered":"2024-05-30T15:54:22.672078+0000","last_unstale":"2024-05-30T17:13:38.273680+0000","last_undegraded":"2024-05-30T17:13:38.273680+0000","last_fullsized":"2024-05-30T17:13:38.273680+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":200,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:23:16.149546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10553,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":297,"num_read_kb":402,"num_write":186,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":503,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.557376+0000","last_change":"2024-05-30T15:54:24.676069+0000","last_active":"2024-05-30T17:09:21.557376+0000","last_peered":"2024-05-30T17:09:21.557376+0000","last_clean":"2024-05-30T17:09:21.557376+0000","last_became_active":"2024-05-30T15:54:24.675467+0000","last_became_peered":"2024-05-30T15:54:24.675467+0000","last_unstale":"2024-05-30T17:09:21.557376+0000","last_undegraded":"2024-05-30T17:09:21.557376+0000","last_fullsized":"2024-05-30T17:09:21.557376+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:10.271248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"265'102","reported_seq":619,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:19.803711+0000","last_change":"2024-05-30T15:54:26.705986+0000","last_active":"2024-05-30T17:13:19.803711+0000","last_peered":"2024-05-30T17:13:19.803711+0000","last_clean":"2024-05-30T17:13:19.803711+0000","last_became_active":"2024-05-30T15:54:26.705264+0000","last_became_peered":"2024-05-30T15:54:26.705264+0000","last_unstale":"2024-05-30T17:13:19.803711+0000","last_undegraded":"2024-05-30T17:13:19.803711+0000","last_fullsized":"2024-05-30T17:13:19.803711+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:18:43.441191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":805,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":151,"num_read_kb":135,"num_write":247,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"265'698","reported_seq":2167,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.677934+0000","last_change":"2024-05-30T15:54:54.422775+0000","last_active":"2024-05-30T17:13:18.677934+0000","last_peered":"2024-05-30T17:13:18.677934+0000","last_clean":"2024-05-30T17:13:18.677934+0000","last_became_active":"2024-05-30T15:54:54.421604+0000","last_became_peered":"2024-05-30T15:54:54.421604+0000","last_unstale":"2024-05-30T17:13:18.677934+0000","last_undegraded":"2024-05-30T17:13:18.677934+0000","last_fullsized":"2024-05-30T17:13:18.677934+0000","mapping_epoch":35,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":600,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:32:11.130243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3204,"num_read_kb":2893,"num_write":920,"num_write_kb":324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"265'173","reported_seq":814,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.368667+0000","last_change":"2024-05-30T15:54:54.424699+0000","last_active":"2024-05-30T17:13:38.368667+0000","last_peered":"2024-05-30T17:13:38.368667+0000","last_clean":"2024-05-30T17:13:38.368667+0000","last_became_active":"2024-05-30T15:54:54.422916+0000","last_became_peered":"2024-05-30T15:54:54.422916+0000","last_unstale":"2024-05-30T17:13:38.368667+0000","last_undegraded":"2024-05-30T17:13:38.368667+0000","last_fullsized":"2024-05-30T17:13:38.368667+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":100,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:06:41.620026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505146,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":405,"num_read_kb":71111,"num_write":971,"num_write_kb":25767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"34'4","reported_seq":831,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:37.465461+0000","last_change":"2024-05-30T15:54:20.652854+0000","last_active":"2024-05-30T17:13:37.465461+0000","last_peered":"2024-05-30T17:13:37.465461+0000","last_clean":"2024-05-30T17:13:37.465461+0000","last_became_active":"2024-05-30T15:54:20.652134+0000","last_became_peered":"2024-05-30T15:54:20.652134+0000","last_unstale":"2024-05-30T17:13:37.465461+0000","last_undegraded":"2024-05-30T17:13:37.465461+0000","last_fullsized":"2024-05-30T17:13:37.465461+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:12:04.911743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1139,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":493,"num_read_kb":807,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"265'151","reported_seq":744,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.288958+0000","last_change":"2024-05-30T15:54:22.673262+0000","last_active":"2024-05-30T17:13:38.288958+0000","last_peered":"2024-05-30T17:13:38.288958+0000","last_clean":"2024-05-30T17:13:38.288958+0000","last_became_active":"2024-05-30T15:54:22.672169+0000","last_became_peered":"2024-05-30T15:54:22.672169+0000","last_unstale":"2024-05-30T17:13:38.288958+0000","last_undegraded":"2024-05-30T17:13:38.288958+0000","last_fullsized":"2024-05-30T17:13:38.288958+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:04:32.418773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6731,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":192,"num_read_kb":263,"num_write":102,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":497,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:41.537447+0000","last_change":"2024-05-30T15:54:24.676446+0000","last_active":"2024-05-30T17:09:41.537447+0000","last_peered":"2024-05-30T17:09:41.537447+0000","last_clean":"2024-05-30T17:09:41.537447+0000","last_became_active":"2024-05-30T15:54:24.675683+0000","last_became_peered":"2024-05-30T15:54:24.675683+0000","last_unstale":"2024-05-30T17:09:41.537447+0000","last_undegraded":"2024-05-30T17:09:41.537447+0000","last_fullsized":"2024-05-30T17:09:41.537447+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:10:10.728748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"265'127","reported_seq":655,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:17.801147+0000","last_change":"2024-05-30T15:54:26.689830+0000","last_active":"2024-05-30T17:13:17.801147+0000","last_peered":"2024-05-30T17:13:17.801147+0000","last_clean":"2024-05-30T17:13:17.801147+0000","last_became_active":"2024-05-30T15:54:26.689049+0000","last_became_peered":"2024-05-30T15:54:26.689049+0000","last_unstale":"2024-05-30T17:13:17.801147+0000","last_undegraded":"2024-05-30T17:13:17.801147+0000","last_fullsized":"2024-05-30T17:13:17.801147+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:41:39.309741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":539,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":167,"num_read_kb":166,"num_write":309,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"265'652","reported_seq":2069,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.759093+0000","last_change":"2024-05-30T15:54:18.653246+0000","last_active":"2024-05-30T17:13:18.759093+0000","last_peered":"2024-05-30T17:13:18.759093+0000","last_clean":"2024-05-30T17:13:18.759093+0000","last_became_active":"2024-05-30T15:54:18.651778+0000","last_became_peered":"2024-05-30T15:54:18.651778+0000","last_unstale":"2024-05-30T17:13:18.759093+0000","last_undegraded":"2024-05-30T17:13:18.759093+0000","last_fullsized":"2024-05-30T17:13:18.759093+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":600,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:42:03.298813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3068,"num_read_kb":2749,"num_write":930,"num_write_kb":367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"265'151","reported_seq":765,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:10:36.577419+0000","last_change":"2024-05-30T15:54:16.656872+0000","last_active":"2024-05-30T17:10:36.577419+0000","last_peered":"2024-05-30T17:10:36.577419+0000","last_clean":"2024-05-30T17:10:36.577419+0000","last_became_active":"2024-05-30T15:54:16.656081+0000","last_became_peered":"2024-05-30T15:54:16.656081+0000","last_unstale":"2024-05-30T17:10:36.577419+0000","last_undegraded":"2024-05-30T17:10:36.577419+0000","last_fullsized":"2024-05-30T17:10:36.577419+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:25:32.809071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":369,"num_read_kb":8201,"num_write":842,"num_write_kb":22196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":515,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:16.721503+0000","last_change":"2024-05-30T15:54:20.661844+0000","last_active":"2024-05-30T17:09:16.721503+0000","last_peered":"2024-05-30T17:09:16.721503+0000","last_clean":"2024-05-30T17:09:16.721503+0000","last_became_active":"2024-05-30T15:54:20.661090+0000","last_became_peered":"2024-05-30T15:54:20.661090+0000","last_unstale":"2024-05-30T17:09:16.721503+0000","last_undegraded":"2024-05-30T17:09:16.721503+0000","last_fullsized":"2024-05-30T17:09:16.721503+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:39:52.559621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"265'116","reported_seq":688,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.117383+0000","last_change":"2024-05-30T15:54:22.670767+0000","last_active":"2024-05-30T17:13:38.117383+0000","last_peered":"2024-05-30T17:13:38.117383+0000","last_clean":"2024-05-30T17:13:38.117383+0000","last_became_active":"2024-05-30T15:54:22.669073+0000","last_became_peered":"2024-05-30T15:54:22.669073+0000","last_unstale":"2024-05-30T17:13:38.117383+0000","last_undegraded":"2024-05-30T17:13:38.117383+0000","last_fullsized":"2024-05-30T17:13:38.117383+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":100,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:43:33.730358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6915,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":145,"num_read_kb":188,"num_write":74,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:26.724184+0000","last_change":"2024-05-30T15:54:24.680707+0000","last_active":"2024-05-30T17:09:26.724184+0000","last_peered":"2024-05-30T17:09:26.724184+0000","last_clean":"2024-05-30T17:09:26.724184+0000","last_became_active":"2024-05-30T15:54:24.678292+0000","last_became_peered":"2024-05-30T15:54:24.678292+0000","last_unstale":"2024-05-30T17:09:26.724184+0000","last_undegraded":"2024-05-30T17:09:26.724184+0000","last_fullsized":"2024-05-30T17:09:26.724184+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:56:06.653666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"265'164","reported_seq":901,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:14.442195+0000","last_change":"2024-05-30T15:54:26.704198+0000","last_active":"2024-05-30T17:13:14.442195+0000","last_peered":"2024-05-30T17:13:14.442195+0000","last_clean":"2024-05-30T17:13:14.442195+0000","last_became_active":"2024-05-30T15:54:26.703440+0000","last_became_peered":"2024-05-30T15:54:26.703440+0000","last_unstale":"2024-05-30T17:13:14.442195+0000","last_undegraded":"2024-05-30T17:13:14.442195+0000","last_fullsized":"2024-05-30T17:13:14.442195+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":100,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:28:50.587698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1074,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":420,"num_read_kb":450,"num_write":403,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"265'327","reported_seq":1681,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:15.190426+0000","last_change":"2024-05-30T15:54:18.672371+0000","last_active":"2024-05-30T17:12:15.190426+0000","last_peered":"2024-05-30T17:12:15.190426+0000","last_clean":"2024-05-30T17:12:15.190426+0000","last_became_active":"2024-05-30T15:54:18.671650+0000","last_became_peered":"2024-05-30T15:54:18.671650+0000","last_unstale":"2024-05-30T17:12:15.190426+0000","last_undegraded":"2024-05-30T17:12:15.190426+0000","last_fullsized":"2024-05-30T17:12:15.190426+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":300,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:52:16.244837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1672,"num_read_kb":1605,"num_write":361,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"265'206","reported_seq":867,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:26.613842+0000","last_change":"2024-05-30T15:54:16.661703+0000","last_active":"2024-05-30T17:13:26.613842+0000","last_peered":"2024-05-30T17:13:26.613842+0000","last_clean":"2024-05-30T17:13:26.613842+0000","last_became_active":"2024-05-30T15:54:16.660393+0000","last_became_peered":"2024-05-30T15:54:16.660393+0000","last_unstale":"2024-05-30T17:13:26.613842+0000","last_undegraded":"2024-05-30T17:13:26.613842+0000","last_fullsized":"2024-05-30T17:13:26.613842+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":100,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:51:03.780593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":594502,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":519,"num_read_kb":9177,"num_write":1141,"num_write_kb":26248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"27'1","reported_seq":508,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:31.768150+0000","last_change":"2024-05-30T15:54:20.661745+0000","last_active":"2024-05-30T17:09:31.768150+0000","last_peered":"2024-05-30T17:09:31.768150+0000","last_clean":"2024-05-30T17:09:31.768150+0000","last_became_active":"2024-05-30T15:54:20.661311+0000","last_became_peered":"2024-05-30T15:54:20.661311+0000","last_unstale":"2024-05-30T17:09:31.768150+0000","last_undegraded":"2024-05-30T17:09:31.768150+0000","last_fullsized":"2024-05-30T17:09:31.768150+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:08:29.614757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"265'175","reported_seq":786,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.174593+0000","last_change":"2024-05-30T15:54:22.663985+0000","last_active":"2024-05-30T17:13:38.174593+0000","last_peered":"2024-05-30T17:13:38.174593+0000","last_clean":"2024-05-30T17:13:38.174593+0000","last_became_active":"2024-05-30T15:54:22.663231+0000","last_became_peered":"2024-05-30T15:54:22.663231+0000","last_unstale":"2024-05-30T17:13:38.174593+0000","last_undegraded":"2024-05-30T17:13:38.174593+0000","last_fullsized":"2024-05-30T17:13:38.174593+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":100,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:32:51.856936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4459,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":234,"num_read_kb":297,"num_write":134,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":502,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:26.558679+0000","last_change":"2024-05-30T15:54:24.676660+0000","last_active":"2024-05-30T17:09:26.558679+0000","last_peered":"2024-05-30T17:09:26.558679+0000","last_clean":"2024-05-30T17:09:26.558679+0000","last_became_active":"2024-05-30T15:54:24.673674+0000","last_became_peered":"2024-05-30T15:54:24.673674+0000","last_unstale":"2024-05-30T17:09:26.558679+0000","last_undegraded":"2024-05-30T17:09:26.558679+0000","last_fullsized":"2024-05-30T17:09:26.558679+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:37:07.948924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"265'111","reported_seq":616,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:09:21.723032+0000","last_change":"2024-05-30T15:54:26.678237+0000","last_active":"2024-05-30T17:09:21.723032+0000","last_peered":"2024-05-30T17:09:21.723032+0000","last_clean":"2024-05-30T17:09:21.723032+0000","last_became_active":"2024-05-30T15:54:26.677418+0000","last_became_peered":"2024-05-30T15:54:26.677418+0000","last_unstale":"2024-05-30T17:09:21.723032+0000","last_undegraded":"2024-05-30T17:09:21.723032+0000","last_fullsized":"2024-05-30T17:09:21.723032+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":100,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:04:15.881662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":435,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":67,"num_write":244,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"265'305","reported_seq":1708,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:29.784726+0000","last_change":"2024-05-30T15:54:18.671642+0000","last_active":"2024-05-30T17:11:29.784726+0000","last_peered":"2024-05-30T17:11:29.784726+0000","last_clean":"2024-05-30T17:11:29.784726+0000","last_became_active":"2024-05-30T15:54:18.671335+0000","last_became_peered":"2024-05-30T15:54:18.671335+0000","last_unstale":"2024-05-30T17:11:29.784726+0000","last_undegraded":"2024-05-30T17:11:29.784726+0000","last_fullsized":"2024-05-30T17:11:29.784726+0000","mapping_epoch":24,"log_start":"265'200","ondisk_log_start":"265'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:01:34.225524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1771,"num_read_kb":1705,"num_write":340,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"265'206","reported_seq":858,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.375876+0000","last_change":"2024-05-30T15:54:16.675257+0000","last_active":"2024-05-30T17:13:38.375876+0000","last_peered":"2024-05-30T17:13:38.375876+0000","last_clean":"2024-05-30T17:13:38.375876+0000","last_became_active":"2024-05-30T15:54:16.674678+0000","last_became_peered":"2024-05-30T15:54:16.674678+0000","last_unstale":"2024-05-30T17:13:38.375876+0000","last_undegraded":"2024-05-30T17:13:38.375876+0000","last_fullsized":"2024-05-30T17:13:38.375876+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":100,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:49:35.316396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25713,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":493,"num_read_kb":10347,"num_write":1036,"num_write_kb":23027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"265'559","reported_seq":2275,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.621937+0000","last_change":"2024-05-30T15:54:18.635401+0000","last_active":"2024-05-30T17:13:20.621937+0000","last_peered":"2024-05-30T17:13:20.621937+0000","last_clean":"2024-05-30T17:13:20.621937+0000","last_became_active":"2024-05-30T15:54:18.635092+0000","last_became_peered":"2024-05-30T15:54:18.635092+0000","last_unstale":"2024-05-30T17:13:20.621937+0000","last_undegraded":"2024-05-30T17:13:20.621937+0000","last_fullsized":"2024-05-30T17:13:20.621937+0000","mapping_epoch":24,"log_start":"265'500","ondisk_log_start":"265'500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":500,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:59:22.401801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2950,"num_read_kb":2739,"num_write":700,"num_write_kb":188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"265'165","reported_seq":779,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.284600+0000","last_change":"2024-05-30T15:54:16.658618+0000","last_active":"2024-05-30T17:13:38.284600+0000","last_peered":"2024-05-30T17:13:38.284600+0000","last_clean":"2024-05-30T17:13:38.284600+0000","last_became_active":"2024-05-30T15:54:16.658183+0000","last_became_peered":"2024-05-30T15:54:16.658183+0000","last_unstale":"2024-05-30T17:13:38.284600+0000","last_undegraded":"2024-05-30T17:13:38.284600+0000","last_fullsized":"2024-05-30T17:13:38.284600+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":100,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:41:18.697341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":308,"num_read_kb":6656,"num_write":926,"num_write_kb":27882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"265'625","reported_seq":2181,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:20.469750+0000","last_change":"2024-05-30T15:54:18.662889+0000","last_active":"2024-05-30T17:12:20.469750+0000","last_peered":"2024-05-30T17:12:20.469750+0000","last_clean":"2024-05-30T17:12:20.469750+0000","last_became_active":"2024-05-30T15:54:18.662159+0000","last_became_peered":"2024-05-30T15:54:18.662159+0000","last_unstale":"2024-05-30T17:12:20.469750+0000","last_undegraded":"2024-05-30T17:12:20.469750+0000","last_fullsized":"2024-05-30T17:12:20.469750+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":600,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:00:12.906772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3191,"num_read_kb":2926,"num_write":899,"num_write_kb":342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"265'210","reported_seq":863,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.171802+0000","last_change":"2024-05-30T15:54:16.670343+0000","last_active":"2024-05-30T17:13:38.171802+0000","last_peered":"2024-05-30T17:13:38.171802+0000","last_clean":"2024-05-30T17:13:38.171802+0000","last_became_active":"2024-05-30T15:54:16.652664+0000","last_became_peered":"2024-05-30T15:54:16.652664+0000","last_unstale":"2024-05-30T17:13:38.171802+0000","last_undegraded":"2024-05-30T17:13:38.171802+0000","last_fullsized":"2024-05-30T17:13:38.171802+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":100,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:16:45.470104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524420,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":451,"num_read_kb":27404,"num_write":1156,"num_write_kb":29020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"265'1297","reported_seq":2932,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:08.892803+0000","last_change":"2024-05-30T15:54:18.643433+0000","last_active":"2024-05-30T17:13:08.892803+0000","last_peered":"2024-05-30T17:13:08.892803+0000","last_clean":"2024-05-30T17:13:08.892803+0000","last_became_active":"2024-05-30T15:54:18.642697+0000","last_became_peered":"2024-05-30T15:54:18.642697+0000","last_unstale":"2024-05-30T17:13:08.892803+0000","last_undegraded":"2024-05-30T17:13:08.892803+0000","last_fullsized":"2024-05-30T17:13:08.892803+0000","mapping_epoch":24,"log_start":"265'1200","ondisk_log_start":"265'1200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":1200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:46:33.063462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":5484,"num_read_kb":4752,"num_write":1832,"num_write_kb":807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"265'256","reported_seq":988,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:41.589226+0000","last_change":"2024-05-30T15:54:16.660154+0000","last_active":"2024-05-30T17:11:41.589226+0000","last_peered":"2024-05-30T17:11:41.589226+0000","last_clean":"2024-05-30T17:11:41.589226+0000","last_became_active":"2024-05-30T15:54:16.659589+0000","last_became_peered":"2024-05-30T15:54:16.659589+0000","last_unstale":"2024-05-30T17:11:41.589226+0000","last_undegraded":"2024-05-30T17:11:41.589226+0000","last_fullsized":"2024-05-30T17:11:41.589226+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":200,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:55:43.508632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3222795,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":793,"num_read_kb":47588,"num_write":1564,"num_write_kb":36318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"265'1101","reported_seq":2680,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:20.420887+0000","last_change":"2024-05-30T15:54:18.653159+0000","last_active":"2024-05-30T17:12:20.420887+0000","last_peered":"2024-05-30T17:12:20.420887+0000","last_clean":"2024-05-30T17:12:20.420887+0000","last_became_active":"2024-05-30T15:54:18.652794+0000","last_became_peered":"2024-05-30T15:54:18.652794+0000","last_unstale":"2024-05-30T17:12:20.420887+0000","last_undegraded":"2024-05-30T17:12:20.420887+0000","last_fullsized":"2024-05-30T17:12:20.420887+0000","mapping_epoch":24,"log_start":"265'1000","ondisk_log_start":"265'1000","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":1000,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:14:33.580719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4733,"num_read_kb":4179,"num_write":1497,"num_write_kb":596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"265'225","reported_seq":893,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:37.951296+0000","last_change":"2024-05-30T15:54:54.452063+0000","last_active":"2024-05-30T17:13:37.951296+0000","last_peered":"2024-05-30T17:13:37.951296+0000","last_clean":"2024-05-30T17:13:37.951296+0000","last_became_active":"2024-05-30T15:54:54.451265+0000","last_became_peered":"2024-05-30T15:54:54.451265+0000","last_unstale":"2024-05-30T17:13:37.951296+0000","last_undegraded":"2024-05-30T17:13:37.951296+0000","last_fullsized":"2024-05-30T17:13:37.951296+0000","mapping_epoch":35,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":200,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:41:58.936344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524357,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":480,"num_read_kb":2675,"num_write":1165,"num_write_kb":43073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"265'385","reported_seq":1948,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:33.943140+0000","last_change":"2024-05-30T15:54:18.671626+0000","last_active":"2024-05-30T17:12:33.943140+0000","last_peered":"2024-05-30T17:12:33.943140+0000","last_clean":"2024-05-30T17:12:33.943140+0000","last_became_active":"2024-05-30T15:54:18.670801+0000","last_became_peered":"2024-05-30T15:54:18.670801+0000","last_unstale":"2024-05-30T17:12:33.943140+0000","last_undegraded":"2024-05-30T17:12:33.943140+0000","last_fullsized":"2024-05-30T17:12:33.943140+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":300,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:08:04.360448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2215,"num_read_kb":2115,"num_write":443,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"265'222","reported_seq":873,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:37.962443+0000","last_change":"2024-05-30T15:54:16.651535+0000","last_active":"2024-05-30T17:13:37.962443+0000","last_peered":"2024-05-30T17:13:37.962443+0000","last_clean":"2024-05-30T17:13:37.962443+0000","last_became_active":"2024-05-30T15:54:16.649854+0000","last_became_peered":"2024-05-30T15:54:16.649854+0000","last_unstale":"2024-05-30T17:13:37.962443+0000","last_undegraded":"2024-05-30T17:13:37.962443+0000","last_fullsized":"2024-05-30T17:13:37.962443+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":200,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:56:05.559917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306443,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":499,"num_read_kb":6249,"num_write":1151,"num_write_kb":30962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"265'405","reported_seq":1925,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:12.349529+0000","last_change":"2024-05-30T15:54:18.653103+0000","last_active":"2024-05-30T17:13:12.349529+0000","last_peered":"2024-05-30T17:13:12.349529+0000","last_clean":"2024-05-30T17:13:12.349529+0000","last_became_active":"2024-05-30T15:54:18.650832+0000","last_became_peered":"2024-05-30T15:54:18.650832+0000","last_unstale":"2024-05-30T17:13:12.349529+0000","last_undegraded":"2024-05-30T17:13:12.349529+0000","last_fullsized":"2024-05-30T17:13:12.349529+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":300,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:14:11.681719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2240,"num_read_kb":2125,"num_write":507,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"265'187","reported_seq":811,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:37.935581+0000","last_change":"2024-05-30T15:54:16.685235+0000","last_active":"2024-05-30T17:13:37.935581+0000","last_peered":"2024-05-30T17:13:37.935581+0000","last_clean":"2024-05-30T17:13:37.935581+0000","last_became_active":"2024-05-30T15:54:16.684670+0000","last_became_peered":"2024-05-30T15:54:16.684670+0000","last_unstale":"2024-05-30T17:13:37.935581+0000","last_undegraded":"2024-05-30T17:13:37.935581+0000","last_fullsized":"2024-05-30T17:13:37.935581+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":100,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:27.693225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":254692,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":373,"num_read_kb":34719,"num_write":1090,"num_write_kb":29777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"265'818","reported_seq":2410,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.705414+0000","last_change":"2024-05-30T15:54:18.669886+0000","last_active":"2024-05-30T17:13:20.705414+0000","last_peered":"2024-05-30T17:13:20.705414+0000","last_clean":"2024-05-30T17:13:20.705414+0000","last_became_active":"2024-05-30T15:54:18.669105+0000","last_became_peered":"2024-05-30T15:54:18.669105+0000","last_unstale":"2024-05-30T17:13:20.705414+0000","last_undegraded":"2024-05-30T17:13:20.705414+0000","last_fullsized":"2024-05-30T17:13:20.705414+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":800,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:33:49.110943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3660,"num_read_kb":3297,"num_write":1114,"num_write_kb":434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"265'220","reported_seq":918,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.172156+0000","last_change":"2024-05-30T15:54:54.417648+0000","last_active":"2024-05-30T17:13:38.172156+0000","last_peered":"2024-05-30T17:13:38.172156+0000","last_clean":"2024-05-30T17:13:38.172156+0000","last_became_active":"2024-05-30T15:54:54.416482+0000","last_became_peered":"2024-05-30T15:54:54.416482+0000","last_unstale":"2024-05-30T17:13:38.172156+0000","last_undegraded":"2024-05-30T17:13:38.172156+0000","last_fullsized":"2024-05-30T17:13:38.172156+0000","mapping_epoch":35,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":200,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:41:15.267706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5708449,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":729,"num_read_kb":137610,"num_write":1205,"num_write_kb":64841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2f","version":"265'385","reported_seq":1973,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:55.981122+0000","last_change":"2024-05-30T15:54:18.657515+0000","last_active":"2024-05-30T17:11:55.981122+0000","last_peered":"2024-05-30T17:11:55.981122+0000","last_clean":"2024-05-30T17:11:55.981122+0000","last_became_active":"2024-05-30T15:54:18.657236+0000","last_became_peered":"2024-05-30T15:54:18.657236+0000","last_unstale":"2024-05-30T17:11:55.981122+0000","last_undegraded":"2024-05-30T17:11:55.981122+0000","last_fullsized":"2024-05-30T17:11:55.981122+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":300,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:04:47.974078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2258,"num_read_kb":2156,"num_write":442,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"265'203","reported_seq":882,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:37.952336+0000","last_change":"2024-05-30T15:54:16.661242+0000","last_active":"2024-05-30T17:13:37.952336+0000","last_peered":"2024-05-30T17:13:37.952336+0000","last_clean":"2024-05-30T17:13:37.952336+0000","last_became_active":"2024-05-30T15:54:16.653037+0000","last_became_peered":"2024-05-30T15:54:16.653037+0000","last_unstale":"2024-05-30T17:13:37.952336+0000","last_undegraded":"2024-05-30T17:13:37.952336+0000","last_fullsized":"2024-05-30T17:13:37.952336+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":100,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:51:51.448976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2108965,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":543,"num_read_kb":29183,"num_write":977,"num_write_kb":32330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"265'330","reported_seq":1828,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:20.469201+0000","last_change":"2024-05-30T15:54:18.636049+0000","last_active":"2024-05-30T17:12:20.469201+0000","last_peered":"2024-05-30T17:12:20.469201+0000","last_clean":"2024-05-30T17:12:20.469201+0000","last_became_active":"2024-05-30T15:54:18.635382+0000","last_became_peered":"2024-05-30T15:54:18.635382+0000","last_unstale":"2024-05-30T17:12:20.469201+0000","last_undegraded":"2024-05-30T17:12:20.469201+0000","last_fullsized":"2024-05-30T17:12:20.469201+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":300,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:50:22.797101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2071,"num_read_kb":1981,"num_write":382,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"265'159","reported_seq":776,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.188102+0000","last_change":"2024-05-30T15:54:16.683072+0000","last_active":"2024-05-30T17:13:38.188102+0000","last_peered":"2024-05-30T17:13:38.188102+0000","last_clean":"2024-05-30T17:13:38.188102+0000","last_became_active":"2024-05-30T15:54:16.682629+0000","last_became_peered":"2024-05-30T15:54:16.682629+0000","last_unstale":"2024-05-30T17:13:38.188102+0000","last_undegraded":"2024-05-30T17:13:38.188102+0000","last_fullsized":"2024-05-30T17:13:38.188102+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:49:01.501502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310739,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":22431,"num_write":822,"num_write_kb":23734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"265'743","reported_seq":2312,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.707890+0000","last_change":"2024-05-30T15:54:18.669167+0000","last_active":"2024-05-30T17:13:20.707890+0000","last_peered":"2024-05-30T17:13:20.707890+0000","last_clean":"2024-05-30T17:13:20.707890+0000","last_became_active":"2024-05-30T15:54:18.668748+0000","last_became_peered":"2024-05-30T15:54:18.668748+0000","last_unstale":"2024-05-30T17:13:20.707890+0000","last_undegraded":"2024-05-30T17:13:20.707890+0000","last_fullsized":"2024-05-30T17:13:20.707890+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":700,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:32:48.662121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3385,"num_read_kb":3066,"num_write":980,"num_write_kb":340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"265'165","reported_seq":900,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.186285+0000","last_change":"2024-05-30T15:54:16.676031+0000","last_active":"2024-05-30T17:13:38.186285+0000","last_peered":"2024-05-30T17:13:38.186285+0000","last_clean":"2024-05-30T17:13:38.186285+0000","last_became_active":"2024-05-30T15:54:16.674993+0000","last_became_peered":"2024-05-30T15:54:16.674993+0000","last_unstale":"2024-05-30T17:13:38.186285+0000","last_undegraded":"2024-05-30T17:13:38.186285+0000","last_fullsized":"2024-05-30T17:13:38.186285+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":100,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:09:30.788276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6553619,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":567,"num_read_kb":112844,"num_write":925,"num_write_kb":45704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"265'471","reported_seq":2112,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.654074+0000","last_change":"2024-05-30T15:54:18.672327+0000","last_active":"2024-05-30T17:12:44.654074+0000","last_peered":"2024-05-30T17:12:44.654074+0000","last_clean":"2024-05-30T17:12:44.654074+0000","last_became_active":"2024-05-30T15:54:18.671434+0000","last_became_peered":"2024-05-30T15:54:18.671434+0000","last_unstale":"2024-05-30T17:12:44.654074+0000","last_undegraded":"2024-05-30T17:12:44.654074+0000","last_fullsized":"2024-05-30T17:12:44.654074+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":400,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:07:26.503748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2525,"num_read_kb":2390,"num_write":596,"num_write_kb":183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"265'204","reported_seq":858,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.249598+0000","last_change":"2024-05-30T15:54:16.630459+0000","last_active":"2024-05-30T17:13:38.249598+0000","last_peered":"2024-05-30T17:13:38.249598+0000","last_clean":"2024-05-30T17:13:38.249598+0000","last_became_active":"2024-05-30T15:54:16.629843+0000","last_became_peered":"2024-05-30T15:54:16.629843+0000","last_unstale":"2024-05-30T17:13:38.249598+0000","last_undegraded":"2024-05-30T17:13:38.249598+0000","last_fullsized":"2024-05-30T17:13:38.249598+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":100,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:56:59.908915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524465,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":434,"num_read_kb":42078,"num_write":1152,"num_write_kb":26102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"265'509","reported_seq":2134,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.660255+0000","last_change":"2024-05-30T15:54:18.660822+0000","last_active":"2024-05-30T17:12:44.660255+0000","last_peered":"2024-05-30T17:12:44.660255+0000","last_clean":"2024-05-30T17:12:44.660255+0000","last_became_active":"2024-05-30T15:54:18.660101+0000","last_became_peered":"2024-05-30T15:54:18.660101+0000","last_unstale":"2024-05-30T17:12:44.660255+0000","last_undegraded":"2024-05-30T17:12:44.660255+0000","last_fullsized":"2024-05-30T17:12:44.660255+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":400,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:23:12.134888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2760,"num_read_kb":2546,"num_write":661,"num_write_kb":201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"265'210","reported_seq":845,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.352472+0000","last_change":"2024-05-30T15:54:16.686381+0000","last_active":"2024-05-30T17:13:38.352472+0000","last_peered":"2024-05-30T17:13:38.352472+0000","last_clean":"2024-05-30T17:13:38.352472+0000","last_became_active":"2024-05-30T15:54:16.685969+0000","last_became_peered":"2024-05-30T15:54:16.685969+0000","last_unstale":"2024-05-30T17:13:38.352472+0000","last_undegraded":"2024-05-30T17:13:38.352472+0000","last_fullsized":"2024-05-30T17:13:38.352472+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":100,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:32:55.261508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1342159,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":419,"num_read_kb":18996,"num_write":1148,"num_write_kb":34757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"265'670","reported_seq":2427,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:55.980437+0000","last_change":"2024-05-30T15:54:18.646120+0000","last_active":"2024-05-30T17:11:55.980437+0000","last_peered":"2024-05-30T17:11:55.980437+0000","last_clean":"2024-05-30T17:11:55.980437+0000","last_became_active":"2024-05-30T15:54:18.645400+0000","last_became_peered":"2024-05-30T15:54:18.645400+0000","last_unstale":"2024-05-30T17:11:55.980437+0000","last_undegraded":"2024-05-30T17:11:55.980437+0000","last_fullsized":"2024-05-30T17:11:55.980437+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":600,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:37:15.286662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3447,"num_read_kb":3120,"num_write":968,"num_write_kb":434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"265'215","reported_seq":849,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.368850+0000","last_change":"2024-05-30T15:54:16.664235+0000","last_active":"2024-05-30T17:13:38.368850+0000","last_peered":"2024-05-30T17:13:38.368850+0000","last_clean":"2024-05-30T17:13:38.368850+0000","last_became_active":"2024-05-30T15:54:16.663255+0000","last_became_peered":"2024-05-30T15:54:16.663255+0000","last_unstale":"2024-05-30T17:13:38.368850+0000","last_undegraded":"2024-05-30T17:13:38.368850+0000","last_fullsized":"2024-05-30T17:13:38.368850+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":200,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:14:20.452277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767340,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":429,"num_read_kb":27123,"num_write":1161,"num_write_kb":19484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"265'360","reported_seq":1779,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:08.893650+0000","last_change":"2024-05-30T15:54:18.658517+0000","last_active":"2024-05-30T17:13:08.893650+0000","last_peered":"2024-05-30T17:13:08.893650+0000","last_clean":"2024-05-30T17:13:08.893650+0000","last_became_active":"2024-05-30T15:54:18.658230+0000","last_became_peered":"2024-05-30T15:54:18.658230+0000","last_unstale":"2024-05-30T17:13:08.893650+0000","last_undegraded":"2024-05-30T17:13:08.893650+0000","last_fullsized":"2024-05-30T17:13:08.893650+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":300,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:13:42.767006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1934,"num_read_kb":1839,"num_write":414,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"265'170","reported_seq":783,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.241023+0000","last_change":"2024-05-30T15:54:16.655014+0000","last_active":"2024-05-30T17:13:38.241023+0000","last_peered":"2024-05-30T17:13:38.241023+0000","last_clean":"2024-05-30T17:13:38.241023+0000","last_became_active":"2024-05-30T15:54:16.654365+0000","last_became_peered":"2024-05-30T15:54:16.654365+0000","last_unstale":"2024-05-30T17:13:38.241023+0000","last_undegraded":"2024-05-30T17:13:38.241023+0000","last_fullsized":"2024-05-30T17:13:38.241023+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":100,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:31:10.427380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":358,"num_read_kb":31154,"num_write":915,"num_write_kb":41557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"265'698","reported_seq":2128,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.605807+0000","last_change":"2024-05-30T15:54:18.661710+0000","last_active":"2024-05-30T17:12:44.605807+0000","last_peered":"2024-05-30T17:12:44.605807+0000","last_clean":"2024-05-30T17:12:44.605807+0000","last_became_active":"2024-05-30T15:54:18.660543+0000","last_became_peered":"2024-05-30T15:54:18.660543+0000","last_unstale":"2024-05-30T17:12:44.605807+0000","last_undegraded":"2024-05-30T17:12:44.605807+0000","last_fullsized":"2024-05-30T17:12:44.605807+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":600,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:39:46.092642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3130,"num_read_kb":2812,"num_write":919,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"265'202","reported_seq":844,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.264091+0000","last_change":"2024-05-30T15:54:16.666386+0000","last_active":"2024-05-30T17:13:38.264091+0000","last_peered":"2024-05-30T17:13:38.264091+0000","last_clean":"2024-05-30T17:13:38.264091+0000","last_became_active":"2024-05-30T15:54:16.661983+0000","last_became_peered":"2024-05-30T15:54:16.661983+0000","last_unstale":"2024-05-30T17:13:38.264091+0000","last_undegraded":"2024-05-30T17:13:38.264091+0000","last_fullsized":"2024-05-30T17:13:38.264091+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:48:35.955662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718827,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":420,"num_read_kb":7620,"num_write":1155,"num_write_kb":9531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"265'154","reported_seq":784,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:37.927146+0000","last_change":"2024-05-30T15:55:54.768358+0000","last_active":"2024-05-30T17:13:37.927146+0000","last_peered":"2024-05-30T17:13:37.927146+0000","last_clean":"2024-05-30T17:13:37.927146+0000","last_became_active":"2024-05-30T15:55:54.767665+0000","last_became_peered":"2024-05-30T15:55:54.767665+0000","last_unstale":"2024-05-30T17:13:37.927146+0000","last_undegraded":"2024-05-30T17:13:37.927146+0000","last_fullsized":"2024-05-30T17:13:37.927146+0000","mapping_epoch":37,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":100,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:31:33.212759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5682939,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":360,"num_read_kb":14674,"num_write":890,"num_write_kb":17371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,0],"acting":[5,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"265'765","reported_seq":2419,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:12.341733+0000","last_change":"2024-05-30T15:54:18.660816+0000","last_active":"2024-05-30T17:11:12.341733+0000","last_peered":"2024-05-30T17:11:12.341733+0000","last_clean":"2024-05-30T17:11:12.341733+0000","last_became_active":"2024-05-30T15:54:18.660501+0000","last_became_peered":"2024-05-30T15:54:18.660501+0000","last_unstale":"2024-05-30T17:11:12.341733+0000","last_undegraded":"2024-05-30T17:11:12.341733+0000","last_fullsized":"2024-05-30T17:11:12.341733+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":700,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:58:48.235676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3837,"num_read_kb":3483,"num_write":1021,"num_write_kb":383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"265'183","reported_seq":834,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.402339+0000","last_change":"2024-05-30T15:54:16.668650+0000","last_active":"2024-05-30T17:13:38.402339+0000","last_peered":"2024-05-30T17:13:38.402339+0000","last_clean":"2024-05-30T17:13:38.402339+0000","last_became_active":"2024-05-30T15:54:16.668178+0000","last_became_peered":"2024-05-30T15:54:16.668178+0000","last_unstale":"2024-05-30T17:13:38.402339+0000","last_undegraded":"2024-05-30T17:13:38.402339+0000","last_fullsized":"2024-05-30T17:13:38.402339+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":100,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:22:19.325731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3976850,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":427,"num_read_kb":57498,"num_write":1000,"num_write_kb":21705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"265'474","reported_seq":2144,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.703669+0000","last_change":"2024-05-30T15:54:18.656743+0000","last_active":"2024-05-30T17:13:20.703669+0000","last_peered":"2024-05-30T17:13:20.703669+0000","last_clean":"2024-05-30T17:13:20.703669+0000","last_became_active":"2024-05-30T15:54:18.655872+0000","last_became_peered":"2024-05-30T15:54:18.655872+0000","last_unstale":"2024-05-30T17:13:20.703669+0000","last_undegraded":"2024-05-30T17:13:20.703669+0000","last_fullsized":"2024-05-30T17:13:20.703669+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":400,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:51:05.525906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2571,"num_read_kb":2440,"num_write":587,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5999,"num_read_kb":4309,"num_write":2135,"num_write_kb":880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1140,"ondisk_log_size":1140,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19930,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":13226,"num_read_kb":13032,"num_write":15987,"num_write_kb":4690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":704512,"data_stored":39860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1567,"ondisk_log_size":1567,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":331,"ondisk_log_size":331,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":279892,"num_objects":197,"num_object_clones":0,"num_object_copies":394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":197,"num_whiteouts":0,"num_read":9562,"num_read_kb":11753,"num_write":6685,"num_write_kb":2848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1921,"ondisk_log_size":1921,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1634,"num_read_kb":1946,"num_write":20,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":242,"num_object_clones":0,"num_object_copies":484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":242,"num_whiteouts":0,"num_read":191181,"num_read_kb":174247,"num_write":51900,"num_write_kb":17923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":242,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4116,"ondisk_log_size":4116,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":157354006,"num_objects":261,"num_object_clones":0,"num_object_copies":783,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":261,"num_whiteouts":0,"num_read":32072,"num_read_kb":2124808,"num_write":69910,"num_write_kb":2069422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":238202880,"data_stored":238202880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4424,"ondisk_log_size":4424,"up":192,"acting":192,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":135,"num_read_kb":182367,"num_write":151,"num_write_kb":122722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95,"ondisk_log_size":95,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":133,"num_read_kb":181098,"num_write":151,"num_write_kb":122722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95,"ondisk_log_size":95,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575811,"num_pgs":106,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300280,"kb_used_data":38548,"kb_used_omap":1,"kb_used_meta":261630,"kb_avail":94071560,"statfs":{"total":96636764160,"available":96329277440,"internally_reserved":0,"allocated":39473152,"data_stored":37452492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":267909578},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575810,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":289776,"kb_used_data":37816,"kb_used_omap":1,"kb_used_meta":251902,"kb_avail":94082064,"statfs":{"total":96636764160,"available":96340033536,"internally_reserved":0,"allocated":38723584,"data_stored":36776739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":257948102},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575810,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":133908,"kb_used_data":48496,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94237932,"statfs":{"total":96636764160,"available":96499642368,"internally_reserved":0,"allocated":49659904,"data_stored":47630081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87357893},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575810,"num_pgs":104,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":138308,"kb_used_data":44652,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94233532,"statfs":{"total":96636764160,"available":96495136768,"internally_reserved":0,"allocated":45723648,"data_stored":43786666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":95877574},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575808,"num_pgs":124,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":269204,"kb_used_data":43584,"kb_used_omap":1,"kb_used_meta":225534,"kb_avail":94102636,"statfs":{"total":96636764160,"available":96361099264,"internally_reserved":0,"allocated":44630016,"data_stored":42616238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":230947271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575808,"num_pgs":91,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":107104,"kb_used_data":35652,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94264736,"statfs":{"total":96636764160,"available":96527089664,"internally_reserved":0,"allocated":36507648,"data_stored":34568865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575812,"num_pgs":112,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":164128,"kb_used_data":46872,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94207712,"statfs":{"total":96636764160,"available":96468697088,"internally_reserved":0,"allocated":47996928,"data_stored":46030586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575807,"num_pgs":109,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266872,"kb_used_data":37120,"kb_used_omap":1,"kb_used_meta":229694,"kb_avail":94104968,"statfs":{"total":96636764160,"available":96363487232,"internally_reserved":0,"allocated":38010880,"data_stored":36012935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":235207109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":27881472,"data_stored":27881472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":37441536,"data_stored":37441536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27029504,"data_stored":27029504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24588288,"data_stored":24588288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":31006720,"data_stored":31006720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":33796096,"data_stored":33796096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":32382976,"data_stored":32382976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24076288,"data_stored":24076288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":4787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":5121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":4696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":6691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":5459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":3918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":7193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T17:13:45.457 INFO:tasks.ceph:pgid 68.1a last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.457 INFO:tasks.ceph:pgid 67.1d last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.457 INFO:tasks.ceph:pgid 66.1c last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.457 INFO:tasks.ceph:pgid 65.1f last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.457 INFO:tasks.ceph:pgid 68.1b last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.458 INFO:tasks.ceph:pgid 67.1c last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.458 INFO:tasks.ceph:pgid 66.1d last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.458 INFO:tasks.ceph:pgid 65.1e last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.458 INFO:tasks.ceph:pgid 68.18 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.458 INFO:tasks.ceph:pgid 67.1f last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.459 INFO:tasks.ceph:pgid 66.1e last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.459 INFO:tasks.ceph:pgid 65.1d last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.459 INFO:tasks.ceph:pgid 68.19 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.459 INFO:tasks.ceph:pgid 67.1e last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.459 INFO:tasks.ceph:pgid 66.1f last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.459 INFO:tasks.ceph:pgid 65.1c last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.459 INFO:tasks.ceph:pgid 68.1e last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.460 INFO:tasks.ceph:pgid 67.19 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.460 INFO:tasks.ceph:pgid 66.18 last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.460 INFO:tasks.ceph:pgid 65.1b last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.460 INFO:tasks.ceph:pgid 68.1f last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.460 INFO:tasks.ceph:pgid 67.18 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.460 INFO:tasks.ceph:pgid 66.19 last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.460 INFO:tasks.ceph:pgid 65.1a last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.461 INFO:tasks.ceph:pgid 68.1c last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.461 INFO:tasks.ceph:pgid 67.1b last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.461 INFO:tasks.ceph:pgid 66.1a last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.461 INFO:tasks.ceph:pgid 65.19 last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.461 INFO:tasks.ceph:pgid 68.1d last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.461 INFO:tasks.ceph:pgid 67.1a last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.462 INFO:tasks.ceph:pgid 66.1b last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.462 INFO:tasks.ceph:pgid 65.18 last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.462 INFO:tasks.ceph:pgid 68.12 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.462 INFO:tasks.ceph:pgid 67.15 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.462 INFO:tasks.ceph:pgid 66.14 last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.462 INFO:tasks.ceph:pgid 65.17 last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.462 INFO:tasks.ceph:pgid 68.13 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.463 INFO:tasks.ceph:pgid 67.14 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.463 INFO:tasks.ceph:pgid 66.15 last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.463 INFO:tasks.ceph:pgid 65.16 last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.463 INFO:tasks.ceph:pgid 68.10 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.463 INFO:tasks.ceph:pgid 67.17 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.463 INFO:tasks.ceph:pgid 66.16 last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.464 INFO:tasks.ceph:pgid 65.15 last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.464 INFO:tasks.ceph:pgid 68.11 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.464 INFO:tasks.ceph:pgid 67.16 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.464 INFO:tasks.ceph:pgid 66.17 last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.464 INFO:tasks.ceph:pgid 65.14 last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.464 INFO:tasks.ceph:pgid 68.16 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.464 INFO:tasks.ceph:pgid 67.11 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.465 INFO:tasks.ceph:pgid 66.10 last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.465 INFO:tasks.ceph:pgid 65.13 last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.465 INFO:tasks.ceph:pgid 68.17 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.465 INFO:tasks.ceph:pgid 67.10 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.465 INFO:tasks.ceph:pgid 66.11 last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.465 INFO:tasks.ceph:pgid 65.12 last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.465 INFO:tasks.ceph:pgid 68.14 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.466 INFO:tasks.ceph:pgid 67.13 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.466 INFO:tasks.ceph:pgid 66.12 last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.466 INFO:tasks.ceph:pgid 65.11 last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.466 INFO:tasks.ceph:pgid 68.15 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.466 INFO:tasks.ceph:pgid 67.12 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.466 INFO:tasks.ceph:pgid 66.13 last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.467 INFO:tasks.ceph:pgid 65.10 last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.467 INFO:tasks.ceph:pgid 68.a last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.467 INFO:tasks.ceph:pgid 67.d last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.467 INFO:tasks.ceph:pgid 66.c last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.467 INFO:tasks.ceph:pgid 65.f last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.467 INFO:tasks.ceph:pgid 68.b last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.467 INFO:tasks.ceph:pgid 67.c last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.468 INFO:tasks.ceph:pgid 66.d last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.468 INFO:tasks.ceph:pgid 65.e last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.468 INFO:tasks.ceph:pgid 68.8 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.468 INFO:tasks.ceph:pgid 67.f last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.468 INFO:tasks.ceph:pgid 66.e last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.468 INFO:tasks.ceph:pgid 65.d last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.468 INFO:tasks.ceph:pgid 68.9 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.469 INFO:tasks.ceph:pgid 67.e last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.469 INFO:tasks.ceph:pgid 66.f last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.469 INFO:tasks.ceph:pgid 65.c last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.469 INFO:tasks.ceph:pgid 68.e last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.469 INFO:tasks.ceph:pgid 67.9 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.469 INFO:tasks.ceph:pgid 66.8 last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.470 INFO:tasks.ceph:pgid 65.b last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.470 INFO:tasks.ceph:pgid 68.f last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.470 INFO:tasks.ceph:pgid 67.8 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.470 INFO:tasks.ceph:pgid 66.9 last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.470 INFO:tasks.ceph:pgid 65.a last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.470 INFO:tasks.ceph:pgid 68.c last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.470 INFO:tasks.ceph:pgid 67.b last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.471 INFO:tasks.ceph:pgid 66.a last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.471 INFO:tasks.ceph:pgid 65.9 last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.471 INFO:tasks.ceph:pgid 68.d last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.471 INFO:tasks.ceph:pgid 67.a last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.471 INFO:tasks.ceph:pgid 66.b last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.471 INFO:tasks.ceph:pgid 65.8 last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.471 INFO:tasks.ceph:pgid 68.2 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.472 INFO:tasks.ceph:pgid 67.5 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.472 INFO:tasks.ceph:pgid 66.4 last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.472 INFO:tasks.ceph:pgid 65.7 last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.472 INFO:tasks.ceph:pgid 68.3 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.472 INFO:tasks.ceph:pgid 67.4 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.472 INFO:tasks.ceph:pgid 66.5 last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.473 INFO:tasks.ceph:pgid 65.6 last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.473 INFO:tasks.ceph:pgid 68.0 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.473 INFO:tasks.ceph:pgid 67.7 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.473 INFO:tasks.ceph:pgid 66.6 last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.473 INFO:tasks.ceph:pgid 65.5 last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.473 INFO:tasks.ceph:pgid 68.1 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.474 INFO:tasks.ceph:pgid 67.6 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.474 INFO:tasks.ceph:pgid 66.7 last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.474 INFO:tasks.ceph:pgid 65.4 last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.474 INFO:tasks.ceph:pgid 68.6 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.474 INFO:tasks.ceph:pgid 67.1 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.474 INFO:tasks.ceph:pgid 66.0 last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.475 INFO:tasks.ceph:pgid 65.3 last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.475 INFO:tasks.ceph:pgid 68.7 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.475 INFO:tasks.ceph:pgid 67.0 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.475 INFO:tasks.ceph:pgid 66.1 last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.475 INFO:tasks.ceph:pgid 65.2 last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.475 INFO:tasks.ceph:pgid 68.4 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.475 INFO:tasks.ceph:pgid 67.3 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.476 INFO:tasks.ceph:pgid 66.2 last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.476 INFO:tasks.ceph:pgid 65.1 last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.476 INFO:tasks.ceph:pgid 68.5 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.476 INFO:tasks.ceph:pgid 67.2 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.476 INFO:tasks.ceph:pgid 66.3 last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.476 INFO:tasks.ceph:pgid 65.0 last_scrub_stamp 2024-05-30T16:12:49.772872+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.477 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.477 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.477 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.477 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.477 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.477 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.478 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.478 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.478 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.478 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.478 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.478 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.478 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.479 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.479 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.479 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.479 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.479 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.479 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.480 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.480 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.480 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.480 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.480 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.480 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.480 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.481 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.481 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.481 INFO:tasks.ceph:pgid 8.18 last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.481 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.481 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.481 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.482 INFO:tasks.ceph:pgid 6.16 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.482 INFO:tasks.ceph:pgid 7.17 last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.482 INFO:tasks.ceph:pgid 8.19 last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.482 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.482 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.482 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.483 INFO:tasks.ceph:pgid 6.17 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.483 INFO:tasks.ceph:pgid 7.16 last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.483 INFO:tasks.ceph:pgid 8.1a last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.483 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.483 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.483 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.484 INFO:tasks.ceph:pgid 6.14 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.484 INFO:tasks.ceph:pgid 7.15 last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.484 INFO:tasks.ceph:pgid 8.1b last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.484 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.484 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.484 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.485 INFO:tasks.ceph:pgid 6.15 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.485 INFO:tasks.ceph:pgid 7.14 last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.485 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.485 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.485 INFO:tasks.ceph:pgid 5.9 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.485 INFO:tasks.ceph:pgid 6.a last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.486 INFO:tasks.ceph:pgid 7.b last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.486 INFO:tasks.ceph:pgid 8.4 last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.486 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.486 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.486 INFO:tasks.ceph:pgid 5.8 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.486 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.487 INFO:tasks.ceph:pgid 7.a last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.487 INFO:tasks.ceph:pgid 8.5 last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.487 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.487 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.487 INFO:tasks.ceph:pgid 5.b last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.488 INFO:tasks.ceph:pgid 6.8 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.488 INFO:tasks.ceph:pgid 7.9 last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.488 INFO:tasks.ceph:pgid 8.6 last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.488 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.488 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.488 INFO:tasks.ceph:pgid 5.a last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.489 INFO:tasks.ceph:pgid 6.9 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.489 INFO:tasks.ceph:pgid 7.8 last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.489 INFO:tasks.ceph:pgid 8.7 last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.489 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.489 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.489 INFO:tasks.ceph:pgid 5.d last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.490 INFO:tasks.ceph:pgid 6.e last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.490 INFO:tasks.ceph:pgid 7.f last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.490 INFO:tasks.ceph:pgid 8.0 last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.490 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.490 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.491 INFO:tasks.ceph:pgid 5.c last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.491 INFO:tasks.ceph:pgid 6.f last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.491 INFO:tasks.ceph:pgid 7.e last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.491 INFO:tasks.ceph:pgid 8.1 last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.491 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.491 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.492 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.492 INFO:tasks.ceph:pgid 6.c last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.492 INFO:tasks.ceph:pgid 7.d last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.492 INFO:tasks.ceph:pgid 8.2 last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.492 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.492 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.493 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.493 INFO:tasks.ceph:pgid 6.d last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.493 INFO:tasks.ceph:pgid 7.c last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.493 INFO:tasks.ceph:pgid 8.3 last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.493 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.493 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-30T15:54:01.512251+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=1, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.494 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-05-30T15:54:07.550093+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=7, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.494 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.494 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.494 INFO:tasks.ceph:pgid 6.7 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.494 INFO:tasks.ceph:pgid 7.6 last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.494 INFO:tasks.ceph:pgid 8.9 last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.494 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.495 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-05-30T15:54:07.550093+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=7, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.495 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.495 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.495 INFO:tasks.ceph:pgid 6.4 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.495 INFO:tasks.ceph:pgid 7.5 last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.495 INFO:tasks.ceph:pgid 8.a last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.495 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.496 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-05-30T15:54:07.550093+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=7, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.496 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.496 INFO:tasks.ceph:pgid 5.6 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.496 INFO:tasks.ceph:pgid 6.5 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.496 INFO:tasks.ceph:pgid 7.4 last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.496 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.497 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.497 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-05-30T15:54:07.550093+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=7, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.497 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.498 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.498 INFO:tasks.ceph:pgid 6.6 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.498 INFO:tasks.ceph:pgid 7.7 last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.498 INFO:tasks.ceph:pgid 8.8 last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.498 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.498 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.499 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-05-30T15:54:07.550093+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=7, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.499 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.499 INFO:tasks.ceph:pgid 6.0 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.499 INFO:tasks.ceph:pgid 7.1 last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.499 INFO:tasks.ceph:pgid 8.e last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.499 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.499 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.500 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-05-30T15:54:07.550093+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=7, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.500 INFO:tasks.ceph:pgid 5.2 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.500 INFO:tasks.ceph:pgid 6.1 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.500 INFO:tasks.ceph:pgid 7.0 last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.500 INFO:tasks.ceph:pgid 8.f last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.500 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.500 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.501 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-05-30T15:54:07.550093+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=7, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.501 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.501 INFO:tasks.ceph:pgid 6.2 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.501 INFO:tasks.ceph:pgid 7.3 last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.501 INFO:tasks.ceph:pgid 8.c last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.501 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.502 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.502 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-05-30T15:54:07.550093+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=7, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.502 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.502 INFO:tasks.ceph:pgid 6.3 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.502 INFO:tasks.ceph:pgid 7.2 last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.502 INFO:tasks.ceph:pgid 8.d last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.502 INFO:tasks.ceph:pgid 8.1f last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.503 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.503 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.503 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.503 INFO:tasks.ceph:pgid 6.11 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.503 INFO:tasks.ceph:pgid 7.10 last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.503 INFO:tasks.ceph:pgid 8.1e last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.503 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.504 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.504 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.504 INFO:tasks.ceph:pgid 6.10 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.504 INFO:tasks.ceph:pgid 7.11 last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.504 INFO:tasks.ceph:pgid 8.1d last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.504 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.504 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.505 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.505 INFO:tasks.ceph:pgid 6.13 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.505 INFO:tasks.ceph:pgid 7.12 last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.505 INFO:tasks.ceph:pgid 8.1c last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.505 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.505 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.505 INFO:tasks.ceph:pgid 5.11 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.506 INFO:tasks.ceph:pgid 6.12 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.506 INFO:tasks.ceph:pgid 7.13 last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.506 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.506 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.506 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.506 INFO:tasks.ceph:pgid 6.1d last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.507 INFO:tasks.ceph:pgid 7.1c last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.507 INFO:tasks.ceph:pgid 8.13 last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.507 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.507 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.507 INFO:tasks.ceph:pgid 5.1f last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.507 INFO:tasks.ceph:pgid 6.1c last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.507 INFO:tasks.ceph:pgid 7.1d last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.508 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.508 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.508 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.508 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.508 INFO:tasks.ceph:pgid 6.1f last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.508 INFO:tasks.ceph:pgid 7.1e last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.508 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.509 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.509 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.509 INFO:tasks.ceph:pgid 5.1d last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.509 INFO:tasks.ceph:pgid 6.1e last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.509 INFO:tasks.ceph:pgid 7.1f last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.509 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.509 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.510 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.510 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.510 INFO:tasks.ceph:pgid 6.19 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.510 INFO:tasks.ceph:pgid 7.18 last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.510 INFO:tasks.ceph:pgid 8.17 last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.510 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.510 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.511 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.511 INFO:tasks.ceph:pgid 6.18 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.511 INFO:tasks.ceph:pgid 7.19 last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.511 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.511 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.511 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.512 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.512 INFO:tasks.ceph:pgid 6.1b last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.512 INFO:tasks.ceph:pgid 7.1a last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.512 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.512 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.512 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.512 INFO:tasks.ceph:pgid 5.19 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.513 INFO:tasks.ceph:pgid 6.1a last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.513 INFO:tasks.ceph:pgid 7.1b last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.513 INFO:tasks.ceph:pgid 8.14 last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.513 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.513 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.513 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.513 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.514 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.514 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.514 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.514 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.514 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.514 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.515 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.515 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.515 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.515 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.515 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.515 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.515 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.516 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.516 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.516 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.516 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.516 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.516 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.516 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.517 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.517 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.517 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.517 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.517 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.517 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.517 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.518 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.518 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.518 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.518 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.518 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:13:45.518 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-30T17:14:05.519 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T17:14:05.887 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:14:05.899 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-05-30T17:14:05.903 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":2658,"stamp":"2024-05-30T17:14:04.458870+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":191800421,"num_objects":837,"num_object_clones":0,"num_object_copies":1935,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":837,"num_whiteouts":0,"num_read":254020,"num_read_kb":2693627,"num_write":147058,"num_write_kb":2342860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":289,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":12765,"num_omap_keys":44,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13943,"ondisk_log_size":13943,"up":850,"acting":850,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":850,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1673284,"kb_used_data":332740,"kb_used_omap":12,"kb_used_meta":1340467,"kb_avail":753301436,"statfs":{"total":773094113280,"available":771380670464,"internally_reserved":0,"allocated":340725760,"data_stored":324874602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":1372638769},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9593,"num_omap_keys":37,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009413"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:26.969133+0000","last_change":"2024-05-30T16:34:55.869930+0000","last_active":"2024-05-30T17:12:26.969133+0000","last_peered":"2024-05-30T17:12:26.969133+0000","last_clean":"2024-05-30T17:12:26.969133+0000","last_became_active":"2024-05-30T16:34:55.869226+0000","last_became_peered":"2024-05-30T16:34:55.869226+0000","last_unstale":"2024-05-30T17:12:26.969133+0000","last_undegraded":"2024-05-30T17:12:26.969133+0000","last_fullsized":"2024-05-30T17:12:26.969133+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"265'2","reported_seq":49,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.284428+0000","last_change":"2024-05-30T16:17:35.249630+0000","last_active":"2024-05-30T17:13:38.284428+0000","last_peered":"2024-05-30T17:13:38.284428+0000","last_clean":"2024-05-30T17:13:38.284428+0000","last_became_active":"2024-05-30T16:17:35.248740+0000","last_became_peered":"2024-05-30T16:17:35.248740+0000","last_unstale":"2024-05-30T17:13:38.284428+0000","last_undegraded":"2024-05-30T17:13:38.284428+0000","last_fullsized":"2024-05-30T17:13:38.284428+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"265'8","reported_seq":75,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.443531+0000","last_change":"2024-05-30T17:13:43.443594+0000","last_active":"2024-05-30T17:13:43.443531+0000","last_peered":"2024-05-30T17:13:43.443531+0000","last_clean":"2024-05-30T17:13:43.443531+0000","last_became_active":"2024-05-30T16:17:31.183349+0000","last_became_peered":"2024-05-30T16:17:31.183349+0000","last_unstale":"2024-05-30T17:13:43.443531+0000","last_undegraded":"2024-05-30T17:13:43.443531+0000","last_fullsized":"2024-05-30T17:13:43.443531+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'8","last_scrub_stamp":"2024-05-30T17:13:43.443352+0000","last_deep_scrub":"265'8","last_deep_scrub_stamp":"2024-05-30T17:13:43.443352+0000","last_clean_scrub_stamp":"2024-05-30T17:13:43.443352+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:57:10.662914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":16384,"num_write":8,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"265'45","reported_seq":193,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.684557+0000","last_change":"2024-05-30T17:13:48.684645+0000","last_active":"2024-05-30T17:13:48.684557+0000","last_peered":"2024-05-30T17:13:48.684557+0000","last_clean":"2024-05-30T17:13:48.684557+0000","last_became_active":"2024-05-30T16:12:50.803352+0000","last_became_peered":"2024-05-30T16:12:50.803352+0000","last_unstale":"2024-05-30T17:13:48.684557+0000","last_undegraded":"2024-05-30T17:13:48.684557+0000","last_fullsized":"2024-05-30T17:13:48.684557+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'45","last_scrub_stamp":"2024-05-30T17:13:48.684344+0000","last_deep_scrub":"265'45","last_deep_scrub_stamp":"2024-05-30T17:13:48.684344+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.684344+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:56:59.286905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":225,"num_read_kb":156,"num_write":101,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:26.572657+0000","last_change":"2024-05-30T16:34:55.887503+0000","last_active":"2024-05-30T17:12:26.572657+0000","last_peered":"2024-05-30T17:12:26.572657+0000","last_clean":"2024-05-30T17:12:26.572657+0000","last_became_active":"2024-05-30T16:34:55.886731+0000","last_became_peered":"2024-05-30T16:34:55.886731+0000","last_unstale":"2024-05-30T17:12:26.572657+0000","last_undegraded":"2024-05-30T17:12:26.572657+0000","last_fullsized":"2024-05-30T17:12:26.572657+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"0'0","reported_seq":45,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:36.766484+0000","last_change":"2024-05-30T16:17:35.248384+0000","last_active":"2024-05-30T17:12:36.766484+0000","last_peered":"2024-05-30T17:12:36.766484+0000","last_clean":"2024-05-30T17:12:36.766484+0000","last_became_active":"2024-05-30T16:17:35.247810+0000","last_became_peered":"2024-05-30T16:17:35.247810+0000","last_unstale":"2024-05-30T17:12:36.766484+0000","last_undegraded":"2024-05-30T17:12:36.766484+0000","last_fullsized":"2024-05-30T17:12:36.766484+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"0'0","reported_seq":59,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.953095+0000","last_change":"2024-05-30T17:13:45.953152+0000","last_active":"2024-05-30T17:13:45.953095+0000","last_peered":"2024-05-30T17:13:45.953095+0000","last_clean":"2024-05-30T17:13:45.953095+0000","last_became_active":"2024-05-30T16:17:31.212337+0000","last_became_peered":"2024-05-30T16:17:31.212337+0000","last_unstale":"2024-05-30T17:13:45.953095+0000","last_undegraded":"2024-05-30T17:13:45.953095+0000","last_fullsized":"2024-05-30T17:13:45.953095+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:45.952903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:45.952903+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.952903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:31:16.835383+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"265'55","reported_seq":219,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:55.177804+0000","last_change":"2024-05-30T17:13:55.177870+0000","last_active":"2024-05-30T17:13:55.177804+0000","last_peered":"2024-05-30T17:13:55.177804+0000","last_clean":"2024-05-30T17:13:55.177804+0000","last_became_active":"2024-05-30T16:12:50.810207+0000","last_became_peered":"2024-05-30T16:12:50.810207+0000","last_unstale":"2024-05-30T17:13:55.177804+0000","last_undegraded":"2024-05-30T17:13:55.177804+0000","last_fullsized":"2024-05-30T17:13:55.177804+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'55","last_scrub_stamp":"2024-05-30T17:13:55.177616+0000","last_deep_scrub":"265'55","last_deep_scrub_stamp":"2024-05-30T17:13:55.177616+0000","last_clean_scrub_stamp":"2024-05-30T17:13:55.177616+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:10:07.203727+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":277,"num_read_kb":197,"num_write":99,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:36.588447+0000","last_change":"2024-05-30T16:34:55.876451+0000","last_active":"2024-05-30T17:13:36.588447+0000","last_peered":"2024-05-30T17:13:36.588447+0000","last_clean":"2024-05-30T17:13:36.588447+0000","last_became_active":"2024-05-30T16:34:55.874359+0000","last_became_peered":"2024-05-30T16:34:55.874359+0000","last_unstale":"2024-05-30T17:13:36.588447+0000","last_undegraded":"2024-05-30T17:13:36.588447+0000","last_fullsized":"2024-05-30T17:13:36.588447+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"0'0","reported_seq":50,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.648243+0000","last_change":"2024-05-30T16:17:35.743986+0000","last_active":"2024-05-30T17:13:41.648243+0000","last_peered":"2024-05-30T17:13:41.648243+0000","last_clean":"2024-05-30T17:13:41.648243+0000","last_became_active":"2024-05-30T16:17:35.743231+0000","last_became_peered":"2024-05-30T16:17:35.743231+0000","last_unstale":"2024-05-30T17:13:41.648243+0000","last_undegraded":"2024-05-30T17:13:41.648243+0000","last_fullsized":"2024-05-30T17:13:41.648243+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"265'3","reported_seq":62,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:42.601701+0000","last_change":"2024-05-30T16:17:31.205642+0000","last_active":"2024-05-30T17:13:42.601701+0000","last_peered":"2024-05-30T17:13:42.601701+0000","last_clean":"2024-05-30T17:13:42.601701+0000","last_became_active":"2024-05-30T16:17:31.204806+0000","last_became_peered":"2024-05-30T16:17:31.204806+0000","last_unstale":"2024-05-30T17:13:42.601701+0000","last_undegraded":"2024-05-30T17:13:42.601701+0000","last_fullsized":"2024-05-30T17:13:42.601701+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"265'29","reported_seq":143,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.662503+0000","last_change":"2024-05-30T17:13:46.662582+0000","last_active":"2024-05-30T17:13:46.662503+0000","last_peered":"2024-05-30T17:13:46.662503+0000","last_clean":"2024-05-30T17:13:46.662503+0000","last_became_active":"2024-05-30T16:12:50.810062+0000","last_became_peered":"2024-05-30T16:12:50.810062+0000","last_unstale":"2024-05-30T17:13:46.662503+0000","last_undegraded":"2024-05-30T17:13:46.662503+0000","last_fullsized":"2024-05-30T17:13:46.662503+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'29","last_scrub_stamp":"2024-05-30T17:13:46.662271+0000","last_deep_scrub":"265'29","last_deep_scrub_stamp":"2024-05-30T17:13:46.662271+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.662271+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:14:37.786974+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":98,"num_write":65,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.075706+0000","last_change":"2024-05-30T16:34:55.888292+0000","last_active":"2024-05-30T17:13:43.075706+0000","last_peered":"2024-05-30T17:13:43.075706+0000","last_clean":"2024-05-30T17:13:43.075706+0000","last_became_active":"2024-05-30T16:34:55.887532+0000","last_became_peered":"2024-05-30T16:34:55.887532+0000","last_unstale":"2024-05-30T17:13:43.075706+0000","last_undegraded":"2024-05-30T17:13:43.075706+0000","last_fullsized":"2024-05-30T17:13:43.075706+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"265'2","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.075519+0000","last_change":"2024-05-30T16:17:35.746712+0000","last_active":"2024-05-30T17:13:43.075519+0000","last_peered":"2024-05-30T17:13:43.075519+0000","last_clean":"2024-05-30T17:13:43.075519+0000","last_became_active":"2024-05-30T16:17:35.745709+0000","last_became_peered":"2024-05-30T16:17:35.745709+0000","last_unstale":"2024-05-30T17:13:43.075519+0000","last_undegraded":"2024-05-30T17:13:43.075519+0000","last_fullsized":"2024-05-30T17:13:43.075519+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1142,"num_write":2,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1f","version":"265'3","reported_seq":67,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.831674+0000","last_change":"2024-05-30T17:13:50.831741+0000","last_active":"2024-05-30T17:13:50.831674+0000","last_peered":"2024-05-30T17:13:50.831674+0000","last_clean":"2024-05-30T17:13:50.831674+0000","last_became_active":"2024-05-30T16:17:31.182344+0000","last_became_peered":"2024-05-30T16:17:31.182344+0000","last_unstale":"2024-05-30T17:13:50.831674+0000","last_undegraded":"2024-05-30T17:13:50.831674+0000","last_fullsized":"2024-05-30T17:13:50.831674+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:13:50.831396+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:13:50.831396+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.831396+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:36:02.576103+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"260'6","reported_seq":85,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.454253+0000","last_change":"2024-05-30T17:13:46.454380+0000","last_active":"2024-05-30T17:13:46.454253+0000","last_peered":"2024-05-30T17:13:46.454253+0000","last_clean":"2024-05-30T17:13:46.454253+0000","last_became_active":"2024-05-30T16:12:50.826974+0000","last_became_peered":"2024-05-30T16:12:50.826974+0000","last_unstale":"2024-05-30T17:13:46.454253+0000","last_undegraded":"2024-05-30T17:13:46.454253+0000","last_fullsized":"2024-05-30T17:13:46.454253+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'6","last_scrub_stamp":"2024-05-30T17:13:46.453944+0000","last_deep_scrub":"260'6","last_deep_scrub_stamp":"2024-05-30T17:13:46.453944+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.453944+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:05:54.138613+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":24,"num_write":11,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":32,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:06.611451+0000","last_change":"2024-05-30T16:34:55.899516+0000","last_active":"2024-05-30T17:13:06.611451+0000","last_peered":"2024-05-30T17:13:06.611451+0000","last_clean":"2024-05-30T17:13:06.611451+0000","last_became_active":"2024-05-30T16:34:55.898864+0000","last_became_peered":"2024-05-30T16:34:55.898864+0000","last_unstale":"2024-05-30T17:13:06.611451+0000","last_undegraded":"2024-05-30T17:13:06.611451+0000","last_fullsized":"2024-05-30T17:13:06.611451+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"265'7","reported_seq":64,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.067411+0000","last_change":"2024-05-30T16:17:35.233543+0000","last_active":"2024-05-30T17:13:38.067411+0000","last_peered":"2024-05-30T17:13:38.067411+0000","last_clean":"2024-05-30T17:13:38.067411+0000","last_became_active":"2024-05-30T16:17:35.232744+0000","last_became_peered":"2024-05-30T16:17:35.232744+0000","last_unstale":"2024-05-30T17:13:38.067411+0000","last_undegraded":"2024-05-30T17:13:38.067411+0000","last_fullsized":"2024-05-30T17:13:38.067411+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":23554,"num_write":6,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"0'0","reported_seq":60,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.680121+0000","last_change":"2024-05-30T17:13:43.680163+0000","last_active":"2024-05-30T17:13:43.680121+0000","last_peered":"2024-05-30T17:13:43.680121+0000","last_clean":"2024-05-30T17:13:43.680121+0000","last_became_active":"2024-05-30T16:17:31.186229+0000","last_became_peered":"2024-05-30T16:17:31.186229+0000","last_unstale":"2024-05-30T17:13:43.680121+0000","last_undegraded":"2024-05-30T17:13:43.680121+0000","last_fullsized":"2024-05-30T17:13:43.680121+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:43.679999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:43.679999+0000","last_clean_scrub_stamp":"2024-05-30T17:13:43.679999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:30:30.079637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"265'27","reported_seq":143,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.491839+0000","last_change":"2024-05-30T17:13:45.491914+0000","last_active":"2024-05-30T17:13:45.491839+0000","last_peered":"2024-05-30T17:13:45.491839+0000","last_clean":"2024-05-30T17:13:45.491839+0000","last_became_active":"2024-05-30T16:12:50.827008+0000","last_became_peered":"2024-05-30T16:12:50.827008+0000","last_unstale":"2024-05-30T17:13:45.491839+0000","last_undegraded":"2024-05-30T17:13:45.491839+0000","last_fullsized":"2024-05-30T17:13:45.491839+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'27","last_scrub_stamp":"2024-05-30T17:13:45.491648+0000","last_deep_scrub":"265'27","last_deep_scrub_stamp":"2024-05-30T17:13:45.491648+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.491648+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:27:36.651992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":100,"num_write":57,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":31,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:26.806691+0000","last_change":"2024-05-30T16:34:55.870968+0000","last_active":"2024-05-30T17:12:26.806691+0000","last_peered":"2024-05-30T17:12:26.806691+0000","last_clean":"2024-05-30T17:12:26.806691+0000","last_became_active":"2024-05-30T16:34:55.870139+0000","last_became_peered":"2024-05-30T16:34:55.870139+0000","last_unstale":"2024-05-30T17:12:26.806691+0000","last_undegraded":"2024-05-30T17:12:26.806691+0000","last_fullsized":"2024-05-30T17:12:26.806691+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"265'8","reported_seq":59,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.397095+0000","last_change":"2024-05-30T16:17:35.234843+0000","last_active":"2024-05-30T17:13:38.397095+0000","last_peered":"2024-05-30T17:13:38.397095+0000","last_clean":"2024-05-30T17:13:38.397095+0000","last_became_active":"2024-05-30T16:17:35.233040+0000","last_became_peered":"2024-05-30T16:17:35.233040+0000","last_unstale":"2024-05-30T17:13:38.397095+0000","last_undegraded":"2024-05-30T17:13:38.397095+0000","last_fullsized":"2024-05-30T17:13:38.397095+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":12336,"num_write":13,"num_write_kb":11312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"265'3","reported_seq":65,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.804589+0000","last_change":"2024-05-30T17:13:46.804678+0000","last_active":"2024-05-30T17:13:46.804589+0000","last_peered":"2024-05-30T17:13:46.804589+0000","last_clean":"2024-05-30T17:13:46.804589+0000","last_became_active":"2024-05-30T16:17:31.173900+0000","last_became_peered":"2024-05-30T16:17:31.173900+0000","last_unstale":"2024-05-30T17:13:46.804589+0000","last_undegraded":"2024-05-30T17:13:46.804589+0000","last_fullsized":"2024-05-30T17:13:46.804589+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:13:46.804390+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:13:46.804390+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.804390+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:37:20.477514+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"265'75","reported_seq":275,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.777415+0000","last_change":"2024-05-30T17:13:49.777540+0000","last_active":"2024-05-30T17:13:49.777415+0000","last_peered":"2024-05-30T17:13:49.777415+0000","last_clean":"2024-05-30T17:13:49.777415+0000","last_became_active":"2024-05-30T16:12:50.815032+0000","last_became_peered":"2024-05-30T16:12:50.815032+0000","last_unstale":"2024-05-30T17:13:49.777415+0000","last_undegraded":"2024-05-30T17:13:49.777415+0000","last_fullsized":"2024-05-30T17:13:49.777415+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'75","last_scrub_stamp":"2024-05-30T17:13:49.777169+0000","last_deep_scrub":"265'75","last_deep_scrub_stamp":"2024-05-30T17:13:49.777169+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.777169+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:42:13.290083+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":396,"num_read_kb":277,"num_write":137,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"265'1","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:26.585548+0000","last_change":"2024-05-30T16:34:55.882343+0000","last_active":"2024-05-30T17:12:26.585548+0000","last_peered":"2024-05-30T17:12:26.585548+0000","last_clean":"2024-05-30T17:12:26.585548+0000","last_became_active":"2024-05-30T16:34:55.881414+0000","last_became_peered":"2024-05-30T16:34:55.881414+0000","last_unstale":"2024-05-30T17:12:26.585548+0000","last_undegraded":"2024-05-30T17:12:26.585548+0000","last_fullsized":"2024-05-30T17:12:26.585548+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"262'2","reported_seq":49,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:36.575890+0000","last_change":"2024-05-30T16:17:35.238116+0000","last_active":"2024-05-30T17:12:36.575890+0000","last_peered":"2024-05-30T17:12:36.575890+0000","last_clean":"2024-05-30T17:12:36.575890+0000","last_became_active":"2024-05-30T16:17:35.237424+0000","last_became_peered":"2024-05-30T16:17:35.237424+0000","last_unstale":"2024-05-30T17:12:36.575890+0000","last_undegraded":"2024-05-30T17:12:36.575890+0000","last_fullsized":"2024-05-30T17:12:36.575890+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"0'0","reported_seq":51,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:31.765322+0000","last_change":"2024-05-30T16:17:31.192981+0000","last_active":"2024-05-30T17:12:31.765322+0000","last_peered":"2024-05-30T17:12:31.765322+0000","last_clean":"2024-05-30T17:12:31.765322+0000","last_became_active":"2024-05-30T16:17:31.192269+0000","last_became_peered":"2024-05-30T16:17:31.192269+0000","last_unstale":"2024-05-30T17:12:31.765322+0000","last_undegraded":"2024-05-30T17:12:31.765322+0000","last_fullsized":"2024-05-30T17:12:31.765322+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"265'25","reported_seq":134,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.688810+0000","last_change":"2024-05-30T17:13:45.688906+0000","last_active":"2024-05-30T17:13:45.688810+0000","last_peered":"2024-05-30T17:13:45.688810+0000","last_clean":"2024-05-30T17:13:45.688810+0000","last_became_active":"2024-05-30T16:12:50.811732+0000","last_became_peered":"2024-05-30T16:12:50.811732+0000","last_unstale":"2024-05-30T17:13:45.688810+0000","last_undegraded":"2024-05-30T17:13:45.688810+0000","last_fullsized":"2024-05-30T17:13:45.688810+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'25","last_scrub_stamp":"2024-05-30T17:13:45.688602+0000","last_deep_scrub":"265'25","last_deep_scrub_stamp":"2024-05-30T17:13:45.688602+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.688602+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:20:25.477039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":85,"num_write":53,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:26.585664+0000","last_change":"2024-05-30T16:34:55.899801+0000","last_active":"2024-05-30T17:12:26.585664+0000","last_peered":"2024-05-30T17:12:26.585664+0000","last_clean":"2024-05-30T17:12:26.585664+0000","last_became_active":"2024-05-30T16:34:55.899241+0000","last_became_peered":"2024-05-30T16:34:55.899241+0000","last_unstale":"2024-05-30T17:12:26.585664+0000","last_undegraded":"2024-05-30T17:12:26.585664+0000","last_fullsized":"2024-05-30T17:12:26.585664+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"0'0","reported_seq":53,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:55.376659+0000","last_change":"2024-05-30T17:13:55.376721+0000","last_active":"2024-05-30T17:13:55.376659+0000","last_peered":"2024-05-30T17:13:55.376659+0000","last_clean":"2024-05-30T17:13:55.376659+0000","last_became_active":"2024-05-30T16:17:35.229034+0000","last_became_peered":"2024-05-30T16:17:35.229034+0000","last_unstale":"2024-05-30T17:13:55.376659+0000","last_undegraded":"2024-05-30T17:13:55.376659+0000","last_fullsized":"2024-05-30T17:13:55.376659+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:55.376450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:55.376450+0000","last_clean_scrub_stamp":"2024-05-30T17:13:55.376450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:44:03.105415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"265'9","reported_seq":82,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.700949+0000","last_change":"2024-05-30T17:13:47.701025+0000","last_active":"2024-05-30T17:13:47.700949+0000","last_peered":"2024-05-30T17:13:47.700949+0000","last_clean":"2024-05-30T17:13:47.700949+0000","last_became_active":"2024-05-30T16:17:31.190742+0000","last_became_peered":"2024-05-30T16:17:31.190742+0000","last_unstale":"2024-05-30T17:13:47.700949+0000","last_undegraded":"2024-05-30T17:13:47.700949+0000","last_fullsized":"2024-05-30T17:13:47.700949+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'9","last_scrub_stamp":"2024-05-30T17:13:47.700729+0000","last_deep_scrub":"265'9","last_deep_scrub_stamp":"2024-05-30T17:13:47.700729+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.700729+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:50:23.027430+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":30722,"num_write":17,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"265'33","reported_seq":152,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.992243+0000","last_change":"2024-05-30T17:13:44.992374+0000","last_active":"2024-05-30T17:13:44.992243+0000","last_peered":"2024-05-30T17:13:44.992243+0000","last_clean":"2024-05-30T17:13:44.992243+0000","last_became_active":"2024-05-30T16:12:50.820343+0000","last_became_peered":"2024-05-30T16:12:50.820343+0000","last_unstale":"2024-05-30T17:13:44.992243+0000","last_undegraded":"2024-05-30T17:13:44.992243+0000","last_fullsized":"2024-05-30T17:13:44.992243+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'33","last_scrub_stamp":"2024-05-30T17:13:44.992004+0000","last_deep_scrub":"265'33","last_deep_scrub_stamp":"2024-05-30T17:13:44.992004+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.992004+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:18:13.774581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":154,"num_read_kb":114,"num_write":68,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:16.803864+0000","last_change":"2024-05-30T16:34:55.889437+0000","last_active":"2024-05-30T17:12:16.803864+0000","last_peered":"2024-05-30T17:12:16.803864+0000","last_clean":"2024-05-30T17:12:16.803864+0000","last_became_active":"2024-05-30T16:34:55.888238+0000","last_became_peered":"2024-05-30T16:34:55.888238+0000","last_unstale":"2024-05-30T17:12:16.803864+0000","last_undegraded":"2024-05-30T17:12:16.803864+0000","last_fullsized":"2024-05-30T17:12:16.803864+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"262'6","reported_seq":57,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:11.612638+0000","last_change":"2024-05-30T16:17:35.740004+0000","last_active":"2024-05-30T17:13:11.612638+0000","last_peered":"2024-05-30T17:13:11.612638+0000","last_clean":"2024-05-30T17:13:11.612638+0000","last_became_active":"2024-05-30T16:17:35.738420+0000","last_became_peered":"2024-05-30T16:17:35.738420+0000","last_unstale":"2024-05-30T17:13:11.612638+0000","last_undegraded":"2024-05-30T17:13:11.612638+0000","last_fullsized":"2024-05-30T17:13:11.612638+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6144,"num_write":16,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"265'5","reported_seq":70,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.196417+0000","last_change":"2024-05-30T17:13:56.196484+0000","last_active":"2024-05-30T17:13:56.196417+0000","last_peered":"2024-05-30T17:13:56.196417+0000","last_clean":"2024-05-30T17:13:56.196417+0000","last_became_active":"2024-05-30T16:17:31.195031+0000","last_became_peered":"2024-05-30T16:17:31.195031+0000","last_unstale":"2024-05-30T17:13:56.196417+0000","last_undegraded":"2024-05-30T17:13:56.196417+0000","last_fullsized":"2024-05-30T17:13:56.196417+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'5","last_scrub_stamp":"2024-05-30T17:13:56.196222+0000","last_deep_scrub":"265'5","last_deep_scrub_stamp":"2024-05-30T17:13:56.196222+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.196222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:04:32.713858+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6715,"num_write":5,"num_write_kb":6715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"265'65","reported_seq":246,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:42.646313+0000","last_change":"2024-05-30T17:13:42.646346+0000","last_active":"2024-05-30T17:13:42.646313+0000","last_peered":"2024-05-30T17:13:42.646313+0000","last_clean":"2024-05-30T17:13:42.646313+0000","last_became_active":"2024-05-30T16:12:50.807771+0000","last_became_peered":"2024-05-30T16:12:50.807771+0000","last_unstale":"2024-05-30T17:13:42.646313+0000","last_undegraded":"2024-05-30T17:13:42.646313+0000","last_fullsized":"2024-05-30T17:13:42.646313+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'65","last_scrub_stamp":"2024-05-30T17:13:42.646208+0000","last_deep_scrub":"265'65","last_deep_scrub_stamp":"2024-05-30T17:13:42.646208+0000","last_clean_scrub_stamp":"2024-05-30T17:13:42.646208+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:19:09.986519+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":343,"num_read_kb":242,"num_write":116,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"265'1","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:16.570377+0000","last_change":"2024-05-30T16:34:55.873393+0000","last_active":"2024-05-30T17:12:16.570377+0000","last_peered":"2024-05-30T17:12:16.570377+0000","last_clean":"2024-05-30T17:12:16.570377+0000","last_became_active":"2024-05-30T16:34:55.872581+0000","last_became_peered":"2024-05-30T16:34:55.872581+0000","last_unstale":"2024-05-30T17:12:16.570377+0000","last_undegraded":"2024-05-30T17:12:16.570377+0000","last_fullsized":"2024-05-30T17:12:16.570377+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"0'0","reported_seq":47,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.075685+0000","last_change":"2024-05-30T16:17:35.739267+0000","last_active":"2024-05-30T17:13:43.075685+0000","last_peered":"2024-05-30T17:13:43.075685+0000","last_clean":"2024-05-30T17:13:43.075685+0000","last_became_active":"2024-05-30T16:17:35.738397+0000","last_became_peered":"2024-05-30T16:17:35.738397+0000","last_unstale":"2024-05-30T17:13:43.075685+0000","last_undegraded":"2024-05-30T17:13:43.075685+0000","last_fullsized":"2024-05-30T17:13:43.075685+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"265'3","reported_seq":65,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.455165+0000","last_change":"2024-05-30T17:13:49.455225+0000","last_active":"2024-05-30T17:13:49.455165+0000","last_peered":"2024-05-30T17:13:49.455165+0000","last_clean":"2024-05-30T17:13:49.455165+0000","last_became_active":"2024-05-30T16:17:31.196345+0000","last_became_peered":"2024-05-30T16:17:31.196345+0000","last_unstale":"2024-05-30T17:13:49.455165+0000","last_undegraded":"2024-05-30T17:13:49.455165+0000","last_fullsized":"2024-05-30T17:13:49.455165+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:13:49.454958+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:13:49.454958+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.454958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:19:39.781936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"265'38","reported_seq":168,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.437795+0000","last_change":"2024-05-30T17:13:49.437862+0000","last_active":"2024-05-30T17:13:49.437795+0000","last_peered":"2024-05-30T17:13:49.437795+0000","last_clean":"2024-05-30T17:13:49.437795+0000","last_became_active":"2024-05-30T16:12:50.826729+0000","last_became_peered":"2024-05-30T16:12:50.826729+0000","last_unstale":"2024-05-30T17:13:49.437795+0000","last_undegraded":"2024-05-30T17:13:49.437795+0000","last_fullsized":"2024-05-30T17:13:49.437795+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'38","last_scrub_stamp":"2024-05-30T17:13:49.437536+0000","last_deep_scrub":"265'38","last_deep_scrub_stamp":"2024-05-30T17:13:49.437536+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.437536+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:58:24.201516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":187,"num_read_kb":132,"num_write":81,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:16.582911+0000","last_change":"2024-05-30T16:34:55.876495+0000","last_active":"2024-05-30T17:12:16.582911+0000","last_peered":"2024-05-30T17:12:16.582911+0000","last_clean":"2024-05-30T17:12:16.582911+0000","last_became_active":"2024-05-30T16:34:55.875788+0000","last_became_peered":"2024-05-30T16:34:55.875788+0000","last_unstale":"2024-05-30T17:12:16.582911+0000","last_undegraded":"2024-05-30T17:12:16.582911+0000","last_fullsized":"2024-05-30T17:12:16.582911+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"0'0","reported_seq":46,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:11.612707+0000","last_change":"2024-05-30T16:17:35.746542+0000","last_active":"2024-05-30T17:13:11.612707+0000","last_peered":"2024-05-30T17:13:11.612707+0000","last_clean":"2024-05-30T17:13:11.612707+0000","last_became_active":"2024-05-30T16:17:35.745769+0000","last_became_peered":"2024-05-30T16:17:35.745769+0000","last_unstale":"2024-05-30T17:13:11.612707+0000","last_undegraded":"2024-05-30T17:13:11.612707+0000","last_fullsized":"2024-05-30T17:13:11.612707+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"265'2","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.301305+0000","last_change":"2024-05-30T16:17:31.206243+0000","last_active":"2024-05-30T17:13:38.301305+0000","last_peered":"2024-05-30T17:13:38.301305+0000","last_clean":"2024-05-30T17:13:38.301305+0000","last_became_active":"2024-05-30T16:17:31.204434+0000","last_became_peered":"2024-05-30T16:17:31.204434+0000","last_unstale":"2024-05-30T17:13:38.301305+0000","last_undegraded":"2024-05-30T17:13:38.301305+0000","last_fullsized":"2024-05-30T17:13:38.301305+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"265'26","reported_seq":134,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:57.206156+0000","last_change":"2024-05-30T17:13:57.206337+0000","last_active":"2024-05-30T17:13:57.206156+0000","last_peered":"2024-05-30T17:13:57.206156+0000","last_clean":"2024-05-30T17:13:57.206156+0000","last_became_active":"2024-05-30T16:12:50.813176+0000","last_became_peered":"2024-05-30T16:12:50.813176+0000","last_unstale":"2024-05-30T17:13:57.206156+0000","last_undegraded":"2024-05-30T17:13:57.206156+0000","last_fullsized":"2024-05-30T17:13:57.206156+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'26","last_scrub_stamp":"2024-05-30T17:13:57.205944+0000","last_deep_scrub":"265'26","last_deep_scrub_stamp":"2024-05-30T17:13:57.205944+0000","last_clean_scrub_stamp":"2024-05-30T17:13:57.205944+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:09:07.523880+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":85,"num_write":57,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:16.966656+0000","last_change":"2024-05-30T16:34:55.872909+0000","last_active":"2024-05-30T17:12:16.966656+0000","last_peered":"2024-05-30T17:12:16.966656+0000","last_clean":"2024-05-30T17:12:16.966656+0000","last_became_active":"2024-05-30T16:34:55.871800+0000","last_became_peered":"2024-05-30T16:34:55.871800+0000","last_unstale":"2024-05-30T17:12:16.966656+0000","last_undegraded":"2024-05-30T17:12:16.966656+0000","last_fullsized":"2024-05-30T17:12:16.966656+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"265'2","reported_seq":49,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.284154+0000","last_change":"2024-05-30T16:17:35.237595+0000","last_active":"2024-05-30T17:13:38.284154+0000","last_peered":"2024-05-30T17:13:38.284154+0000","last_clean":"2024-05-30T17:13:38.284154+0000","last_became_active":"2024-05-30T16:17:35.236867+0000","last_became_peered":"2024-05-30T16:17:35.236867+0000","last_unstale":"2024-05-30T17:13:38.284154+0000","last_undegraded":"2024-05-30T17:13:38.284154+0000","last_fullsized":"2024-05-30T17:13:38.284154+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":571,"num_write":2,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"0'0","reported_seq":60,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.914149+0000","last_change":"2024-05-30T17:13:59.914262+0000","last_active":"2024-05-30T17:13:59.914149+0000","last_peered":"2024-05-30T17:13:59.914149+0000","last_clean":"2024-05-30T17:13:59.914149+0000","last_became_active":"2024-05-30T16:17:31.209882+0000","last_became_peered":"2024-05-30T16:17:31.209882+0000","last_unstale":"2024-05-30T17:13:59.914149+0000","last_undegraded":"2024-05-30T17:13:59.914149+0000","last_fullsized":"2024-05-30T17:13:59.914149+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:59.913913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:59.913913+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.913913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:14:43.780738+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"265'56","reported_seq":228,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.549397+0000","last_change":"2024-05-30T17:13:54.549516+0000","last_active":"2024-05-30T17:13:54.549397+0000","last_peered":"2024-05-30T17:13:54.549397+0000","last_clean":"2024-05-30T17:13:54.549397+0000","last_became_active":"2024-05-30T16:12:50.824043+0000","last_became_peered":"2024-05-30T16:12:50.824043+0000","last_unstale":"2024-05-30T17:13:54.549397+0000","last_undegraded":"2024-05-30T17:13:54.549397+0000","last_fullsized":"2024-05-30T17:13:54.549397+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'56","last_scrub_stamp":"2024-05-30T17:13:54.549107+0000","last_deep_scrub":"265'56","last_deep_scrub_stamp":"2024-05-30T17:13:54.549107+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.549107+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:55:26.307978+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":308,"num_read_kb":221,"num_write":92,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:36.618123+0000","last_change":"2024-05-30T16:34:55.893966+0000","last_active":"2024-05-30T17:13:36.618123+0000","last_peered":"2024-05-30T17:13:36.618123+0000","last_clean":"2024-05-30T17:13:36.618123+0000","last_became_active":"2024-05-30T16:34:55.893040+0000","last_became_peered":"2024-05-30T16:34:55.893040+0000","last_unstale":"2024-05-30T17:13:36.618123+0000","last_undegraded":"2024-05-30T17:13:36.618123+0000","last_fullsized":"2024-05-30T17:13:36.618123+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"0'0","reported_seq":47,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:36.821391+0000","last_change":"2024-05-30T16:17:35.246948+0000","last_active":"2024-05-30T17:13:36.821391+0000","last_peered":"2024-05-30T17:13:36.821391+0000","last_clean":"2024-05-30T17:13:36.821391+0000","last_became_active":"2024-05-30T16:17:35.246187+0000","last_became_peered":"2024-05-30T16:17:35.246187+0000","last_unstale":"2024-05-30T17:13:36.821391+0000","last_undegraded":"2024-05-30T17:13:36.821391+0000","last_fullsized":"2024-05-30T17:13:36.821391+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"265'8","reported_seq":80,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.895849+0000","last_change":"2024-05-30T17:13:54.895979+0000","last_active":"2024-05-30T17:13:54.895849+0000","last_peered":"2024-05-30T17:13:54.895849+0000","last_clean":"2024-05-30T17:13:54.895849+0000","last_became_active":"2024-05-30T16:17:31.189091+0000","last_became_peered":"2024-05-30T16:17:31.189091+0000","last_unstale":"2024-05-30T17:13:54.895849+0000","last_undegraded":"2024-05-30T17:13:54.895849+0000","last_fullsized":"2024-05-30T17:13:54.895849+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'8","last_scrub_stamp":"2024-05-30T17:13:54.895517+0000","last_deep_scrub":"265'8","last_deep_scrub_stamp":"2024-05-30T17:13:54.895517+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.895517+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:48:19.231935+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":21053,"num_write":6,"num_write_kb":5691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"265'9","reported_seq":93,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:55.595931+0000","last_change":"2024-05-30T17:13:55.595991+0000","last_active":"2024-05-30T17:13:55.595931+0000","last_peered":"2024-05-30T17:13:55.595931+0000","last_clean":"2024-05-30T17:13:55.595931+0000","last_became_active":"2024-05-30T16:12:50.823759+0000","last_became_peered":"2024-05-30T16:12:50.823759+0000","last_unstale":"2024-05-30T17:13:55.595931+0000","last_undegraded":"2024-05-30T17:13:55.595931+0000","last_fullsized":"2024-05-30T17:13:55.595931+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'9","last_scrub_stamp":"2024-05-30T17:13:55.595737+0000","last_deep_scrub":"265'9","last_deep_scrub_stamp":"2024-05-30T17:13:55.595737+0000","last_clean_scrub_stamp":"2024-05-30T17:13:55.595737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:06:46.930141+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":33,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:36.616253+0000","last_change":"2024-05-30T16:34:55.861418+0000","last_active":"2024-05-30T17:13:36.616253+0000","last_peered":"2024-05-30T17:13:36.616253+0000","last_clean":"2024-05-30T17:13:36.616253+0000","last_became_active":"2024-05-30T16:34:55.860695+0000","last_became_peered":"2024-05-30T16:34:55.860695+0000","last_unstale":"2024-05-30T17:13:36.616253+0000","last_undegraded":"2024-05-30T17:13:36.616253+0000","last_fullsized":"2024-05-30T17:13:36.616253+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"0'0","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.981887+0000","last_change":"2024-05-30T17:13:53.981997+0000","last_active":"2024-05-30T17:13:53.981887+0000","last_peered":"2024-05-30T17:13:53.981887+0000","last_clean":"2024-05-30T17:13:53.981887+0000","last_became_active":"2024-05-30T16:17:35.239157+0000","last_became_peered":"2024-05-30T16:17:35.239157+0000","last_unstale":"2024-05-30T17:13:53.981887+0000","last_undegraded":"2024-05-30T17:13:53.981887+0000","last_fullsized":"2024-05-30T17:13:53.981887+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:53.981644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:53.981644+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.981644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:55:41.438700+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"265'2","reported_seq":64,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.438167+0000","last_change":"2024-05-30T17:13:47.438232+0000","last_active":"2024-05-30T17:13:47.438167+0000","last_peered":"2024-05-30T17:13:47.438167+0000","last_clean":"2024-05-30T17:13:47.438167+0000","last_became_active":"2024-05-30T16:17:31.184675+0000","last_became_peered":"2024-05-30T16:17:31.184675+0000","last_unstale":"2024-05-30T17:13:47.438167+0000","last_undegraded":"2024-05-30T17:13:47.438167+0000","last_fullsized":"2024-05-30T17:13:47.438167+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:13:47.437969+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:13:47.437969+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.437969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:58:28.886391+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":698,"num_write":2,"num_write_kb":698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"265'97","reported_seq":367,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.658941+0000","last_change":"2024-05-30T17:13:49.659011+0000","last_active":"2024-05-30T17:13:49.658941+0000","last_peered":"2024-05-30T17:13:49.658941+0000","last_clean":"2024-05-30T17:13:49.658941+0000","last_became_active":"2024-05-30T16:12:50.801791+0000","last_became_peered":"2024-05-30T16:12:50.801791+0000","last_unstale":"2024-05-30T17:13:49.658941+0000","last_undegraded":"2024-05-30T17:13:49.658941+0000","last_fullsized":"2024-05-30T17:13:49.658941+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'97","last_scrub_stamp":"2024-05-30T17:13:49.658745+0000","last_deep_scrub":"265'97","last_deep_scrub_stamp":"2024-05-30T17:13:49.658745+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.658745+0000","objects_scrubbed":1,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:10:57.618551+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":591,"num_read_kb":446,"num_write":104,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2994,"num_omap_keys":6,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"264'1","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:26.600681+0000","last_change":"2024-05-30T16:34:55.864834+0000","last_active":"2024-05-30T17:12:26.600681+0000","last_peered":"2024-05-30T17:12:26.600681+0000","last_clean":"2024-05-30T17:12:26.600681+0000","last_became_active":"2024-05-30T16:34:55.863483+0000","last_became_peered":"2024-05-30T16:34:55.863483+0000","last_unstale":"2024-05-30T17:12:26.600681+0000","last_undegraded":"2024-05-30T17:12:26.600681+0000","last_fullsized":"2024-05-30T17:12:26.600681+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"0'0","reported_seq":47,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:36.618001+0000","last_change":"2024-05-30T16:17:35.236085+0000","last_active":"2024-05-30T17:13:36.618001+0000","last_peered":"2024-05-30T17:13:36.618001+0000","last_clean":"2024-05-30T17:13:36.618001+0000","last_became_active":"2024-05-30T16:17:35.235283+0000","last_became_peered":"2024-05-30T16:17:35.235283+0000","last_unstale":"2024-05-30T17:13:36.618001+0000","last_undegraded":"2024-05-30T17:13:36.618001+0000","last_fullsized":"2024-05-30T17:13:36.618001+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"0'0","reported_seq":53,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:31.587433+0000","last_change":"2024-05-30T16:17:31.204968+0000","last_active":"2024-05-30T17:13:31.587433+0000","last_peered":"2024-05-30T17:13:31.587433+0000","last_clean":"2024-05-30T17:13:31.587433+0000","last_became_active":"2024-05-30T16:17:31.199499+0000","last_became_peered":"2024-05-30T16:17:31.199499+0000","last_unstale":"2024-05-30T17:13:31.587433+0000","last_undegraded":"2024-05-30T17:13:31.587433+0000","last_fullsized":"2024-05-30T17:13:31.587433+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"265'22","reported_seq":127,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.758712+0000","last_change":"2024-05-30T17:13:44.758825+0000","last_active":"2024-05-30T17:13:44.758712+0000","last_peered":"2024-05-30T17:13:44.758712+0000","last_clean":"2024-05-30T17:13:44.758712+0000","last_became_active":"2024-05-30T16:12:50.807156+0000","last_became_peered":"2024-05-30T16:12:50.807156+0000","last_unstale":"2024-05-30T17:13:44.758712+0000","last_undegraded":"2024-05-30T17:13:44.758712+0000","last_fullsized":"2024-05-30T17:13:44.758712+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'22","last_scrub_stamp":"2024-05-30T17:13:44.758408+0000","last_deep_scrub":"265'22","last_deep_scrub_stamp":"2024-05-30T17:13:44.758408+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.758408+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:20:24.643496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":76,"num_write":50,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"265'1","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:36.618076+0000","last_change":"2024-05-30T16:34:55.889162+0000","last_active":"2024-05-30T17:13:36.618076+0000","last_peered":"2024-05-30T17:13:36.618076+0000","last_clean":"2024-05-30T17:13:36.618076+0000","last_became_active":"2024-05-30T16:34:55.887845+0000","last_became_peered":"2024-05-30T16:34:55.887845+0000","last_unstale":"2024-05-30T17:13:36.618076+0000","last_undegraded":"2024-05-30T17:13:36.618076+0000","last_fullsized":"2024-05-30T17:13:36.618076+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"0'0","reported_seq":47,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:36.588553+0000","last_change":"2024-05-30T16:17:35.246711+0000","last_active":"2024-05-30T17:13:36.588553+0000","last_peered":"2024-05-30T17:13:36.588553+0000","last_clean":"2024-05-30T17:13:36.588553+0000","last_became_active":"2024-05-30T16:17:35.245852+0000","last_became_peered":"2024-05-30T16:17:35.245852+0000","last_unstale":"2024-05-30T17:13:36.588553+0000","last_undegraded":"2024-05-30T17:13:36.588553+0000","last_fullsized":"2024-05-30T17:13:36.588553+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"265'6","reported_seq":77,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:00.249278+0000","last_change":"2024-05-30T17:14:00.249347+0000","last_active":"2024-05-30T17:14:00.249278+0000","last_peered":"2024-05-30T17:14:00.249278+0000","last_clean":"2024-05-30T17:14:00.249278+0000","last_became_active":"2024-05-30T16:17:31.218752+0000","last_became_peered":"2024-05-30T16:17:31.218752+0000","last_unstale":"2024-05-30T17:14:00.249278+0000","last_undegraded":"2024-05-30T17:14:00.249278+0000","last_fullsized":"2024-05-30T17:14:00.249278+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'6","last_scrub_stamp":"2024-05-30T17:14:00.249094+0000","last_deep_scrub":"265'6","last_deep_scrub_stamp":"2024-05-30T17:14:00.249094+0000","last_clean_scrub_stamp":"2024-05-30T17:14:00.249094+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:10:46.604501+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"265'17","reported_seq":112,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.434952+0000","last_change":"2024-05-30T17:13:48.435015+0000","last_active":"2024-05-30T17:13:48.434952+0000","last_peered":"2024-05-30T17:13:48.434952+0000","last_clean":"2024-05-30T17:13:48.434952+0000","last_became_active":"2024-05-30T16:12:50.817114+0000","last_became_peered":"2024-05-30T16:12:50.817114+0000","last_unstale":"2024-05-30T17:13:48.434952+0000","last_undegraded":"2024-05-30T17:13:48.434952+0000","last_fullsized":"2024-05-30T17:13:48.434952+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'17","last_scrub_stamp":"2024-05-30T17:13:48.434761+0000","last_deep_scrub":"265'17","last_deep_scrub_stamp":"2024-05-30T17:13:48.434761+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.434761+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:55.692466+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":55,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:56.566294+0000","last_change":"2024-05-30T16:34:55.867126+0000","last_active":"2024-05-30T17:11:56.566294+0000","last_peered":"2024-05-30T17:11:56.566294+0000","last_clean":"2024-05-30T17:11:56.566294+0000","last_became_active":"2024-05-30T16:34:55.866415+0000","last_became_peered":"2024-05-30T16:34:55.866415+0000","last_unstale":"2024-05-30T17:11:56.566294+0000","last_undegraded":"2024-05-30T17:11:56.566294+0000","last_fullsized":"2024-05-30T17:11:56.566294+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"0'0","reported_seq":56,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.361463+0000","last_change":"2024-05-30T17:13:51.361463+0000","last_active":"2024-05-30T17:13:51.361463+0000","last_peered":"2024-05-30T17:13:51.361463+0000","last_clean":"2024-05-30T17:13:51.361463+0000","last_became_active":"2024-05-30T16:17:35.227270+0000","last_became_peered":"2024-05-30T16:17:35.227270+0000","last_unstale":"2024-05-30T17:13:51.361463+0000","last_undegraded":"2024-05-30T17:13:51.361463+0000","last_fullsized":"2024-05-30T17:13:51.361463+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:51.361210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:51.361210+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.361210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:13:33.987563+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"0'0","reported_seq":62,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.503842+0000","last_change":"2024-05-30T17:13:45.503842+0000","last_active":"2024-05-30T17:13:45.503842+0000","last_peered":"2024-05-30T17:13:45.503842+0000","last_clean":"2024-05-30T17:13:45.503842+0000","last_became_active":"2024-05-30T16:17:31.203731+0000","last_became_peered":"2024-05-30T16:17:31.203731+0000","last_unstale":"2024-05-30T17:13:45.503842+0000","last_undegraded":"2024-05-30T17:13:45.503842+0000","last_fullsized":"2024-05-30T17:13:45.503842+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:45.503591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:45.503591+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.503591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:19:48.974417+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"265'29","reported_seq":146,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.392583+0000","last_change":"2024-05-30T17:13:50.392583+0000","last_active":"2024-05-30T17:13:50.392583+0000","last_peered":"2024-05-30T17:13:50.392583+0000","last_clean":"2024-05-30T17:13:50.392583+0000","last_became_active":"2024-05-30T16:12:50.824809+0000","last_became_peered":"2024-05-30T16:12:50.824809+0000","last_unstale":"2024-05-30T17:13:50.392583+0000","last_undegraded":"2024-05-30T17:13:50.392583+0000","last_fullsized":"2024-05-30T17:13:50.392583+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'29","last_scrub_stamp":"2024-05-30T17:13:50.392236+0000","last_deep_scrub":"265'29","last_deep_scrub_stamp":"2024-05-30T17:13:50.392236+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.392236+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:32:06.254235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":98,"num_write":77,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:11.596984+0000","last_change":"2024-05-30T16:34:55.897061+0000","last_active":"2024-05-30T17:13:11.596984+0000","last_peered":"2024-05-30T17:13:11.596984+0000","last_clean":"2024-05-30T17:13:11.596984+0000","last_became_active":"2024-05-30T16:34:55.896357+0000","last_became_peered":"2024-05-30T16:34:55.896357+0000","last_unstale":"2024-05-30T17:13:11.596984+0000","last_undegraded":"2024-05-30T17:13:11.596984+0000","last_fullsized":"2024-05-30T17:13:11.596984+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"265'3","reported_seq":46,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.218179+0000","last_change":"2024-05-30T16:17:55.551397+0000","last_active":"2024-05-30T17:13:38.218179+0000","last_peered":"2024-05-30T17:13:38.218179+0000","last_clean":"2024-05-30T17:13:38.218179+0000","last_became_active":"2024-05-30T16:17:55.550782+0000","last_became_peered":"2024-05-30T16:17:55.550782+0000","last_unstale":"2024-05-30T17:13:38.218179+0000","last_undegraded":"2024-05-30T17:13:38.218179+0000","last_fullsized":"2024-05-30T17:13:38.218179+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2048,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.d","version":"265'7","reported_seq":66,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:42.601688+0000","last_change":"2024-05-30T16:17:31.202846+0000","last_active":"2024-05-30T17:13:42.601688+0000","last_peered":"2024-05-30T17:13:42.601688+0000","last_clean":"2024-05-30T17:13:42.601688+0000","last_became_active":"2024-05-30T16:17:31.202042+0000","last_became_peered":"2024-05-30T16:17:31.202042+0000","last_unstale":"2024-05-30T17:13:42.601688+0000","last_undegraded":"2024-05-30T17:13:42.601688+0000","last_fullsized":"2024-05-30T17:13:42.601688+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":6194,"num_write":7,"num_write_kb":6194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"265'24","reported_seq":139,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.172777+0000","last_change":"2024-05-30T17:13:44.172845+0000","last_active":"2024-05-30T17:13:44.172777+0000","last_peered":"2024-05-30T17:13:44.172777+0000","last_clean":"2024-05-30T17:13:44.172777+0000","last_became_active":"2024-05-30T16:12:50.810095+0000","last_became_peered":"2024-05-30T16:12:50.810095+0000","last_unstale":"2024-05-30T17:13:44.172777+0000","last_undegraded":"2024-05-30T17:13:44.172777+0000","last_fullsized":"2024-05-30T17:13:44.172777+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'24","last_scrub_stamp":"2024-05-30T17:13:44.172579+0000","last_deep_scrub":"265'24","last_deep_scrub_stamp":"2024-05-30T17:13:44.172579+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.172579+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:00:02.104351+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":90,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":31,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.566730+0000","last_change":"2024-05-30T16:34:55.863218+0000","last_active":"2024-05-30T17:13:44.566730+0000","last_peered":"2024-05-30T17:13:44.566730+0000","last_clean":"2024-05-30T17:13:44.566730+0000","last_became_active":"2024-05-30T16:34:55.862478+0000","last_became_peered":"2024-05-30T16:34:55.862478+0000","last_unstale":"2024-05-30T17:13:44.566730+0000","last_undegraded":"2024-05-30T17:13:44.566730+0000","last_fullsized":"2024-05-30T17:13:44.566730+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"262'3","reported_seq":54,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:42.129963+0000","last_change":"2024-05-30T16:17:35.743152+0000","last_active":"2024-05-30T17:13:42.129963+0000","last_peered":"2024-05-30T17:13:42.129963+0000","last_clean":"2024-05-30T17:13:42.129963+0000","last_became_active":"2024-05-30T16:17:35.742255+0000","last_became_peered":"2024-05-30T16:17:35.742255+0000","last_unstale":"2024-05-30T17:13:42.129963+0000","last_undegraded":"2024-05-30T17:13:42.129963+0000","last_fullsized":"2024-05-30T17:13:42.129963+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"265'2","reported_seq":65,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:55.774110+0000","last_change":"2024-05-30T17:13:55.774249+0000","last_active":"2024-05-30T17:13:55.774110+0000","last_peered":"2024-05-30T17:13:55.774110+0000","last_clean":"2024-05-30T17:13:55.774110+0000","last_became_active":"2024-05-30T16:17:31.198744+0000","last_became_peered":"2024-05-30T16:17:31.198744+0000","last_unstale":"2024-05-30T17:13:55.774110+0000","last_undegraded":"2024-05-30T17:13:55.774110+0000","last_fullsized":"2024-05-30T17:13:55.774110+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:13:55.773876+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:13:55.773876+0000","last_clean_scrub_stamp":"2024-05-30T17:13:55.773876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:52:06.016908+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"265'36","reported_seq":172,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.883144+0000","last_change":"2024-05-30T17:13:45.883208+0000","last_active":"2024-05-30T17:13:45.883144+0000","last_peered":"2024-05-30T17:13:45.883144+0000","last_clean":"2024-05-30T17:13:45.883144+0000","last_became_active":"2024-05-30T16:12:50.820936+0000","last_became_peered":"2024-05-30T16:12:50.820936+0000","last_unstale":"2024-05-30T17:13:45.883144+0000","last_undegraded":"2024-05-30T17:13:45.883144+0000","last_fullsized":"2024-05-30T17:13:45.883144+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'36","last_scrub_stamp":"2024-05-30T17:13:45.882931+0000","last_deep_scrub":"265'36","last_deep_scrub_stamp":"2024-05-30T17:13:45.882931+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.882931+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:49:53.101567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":194,"num_read_kb":136,"num_write":73,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":31,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.053089+0000","last_change":"2024-05-30T16:34:55.877239+0000","last_active":"2024-05-30T17:13:45.053089+0000","last_peered":"2024-05-30T17:13:45.053089+0000","last_clean":"2024-05-30T17:13:45.053089+0000","last_became_active":"2024-05-30T16:34:55.875965+0000","last_became_peered":"2024-05-30T16:34:55.875965+0000","last_unstale":"2024-05-30T17:13:45.053089+0000","last_undegraded":"2024-05-30T17:13:45.053089+0000","last_fullsized":"2024-05-30T17:13:45.053089+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"265'3","reported_seq":61,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.354852+0000","last_change":"2024-05-30T17:13:59.354913+0000","last_active":"2024-05-30T17:13:59.354852+0000","last_peered":"2024-05-30T17:13:59.354852+0000","last_clean":"2024-05-30T17:13:59.354852+0000","last_became_active":"2024-05-30T16:17:35.229643+0000","last_became_peered":"2024-05-30T16:17:35.229643+0000","last_unstale":"2024-05-30T17:13:59.354852+0000","last_undegraded":"2024-05-30T17:13:59.354852+0000","last_fullsized":"2024-05-30T17:13:59.354852+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:13:59.354655+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:13:59.354655+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.354655+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:40:27.564311+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.f","version":"262'6","reported_seq":73,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.021324+0000","last_change":"2024-05-30T17:13:43.021324+0000","last_active":"2024-05-30T17:13:43.021324+0000","last_peered":"2024-05-30T17:13:43.021324+0000","last_clean":"2024-05-30T17:13:43.021324+0000","last_became_active":"2024-05-30T16:17:31.212894+0000","last_became_peered":"2024-05-30T16:17:31.212894+0000","last_unstale":"2024-05-30T17:13:43.021324+0000","last_undegraded":"2024-05-30T17:13:43.021324+0000","last_fullsized":"2024-05-30T17:13:43.021324+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'6","last_scrub_stamp":"2024-05-30T17:13:43.021049+0000","last_deep_scrub":"262'6","last_deep_scrub_stamp":"2024-05-30T17:13:43.021049+0000","last_clean_scrub_stamp":"2024-05-30T17:13:43.021049+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:27:51.042980+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6144,"num_write":11,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"262'21","reported_seq":130,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.118605+0000","last_change":"2024-05-30T17:13:53.118667+0000","last_active":"2024-05-30T17:13:53.118605+0000","last_peered":"2024-05-30T17:13:53.118605+0000","last_clean":"2024-05-30T17:13:53.118605+0000","last_became_active":"2024-05-30T16:12:50.805772+0000","last_became_peered":"2024-05-30T16:12:50.805772+0000","last_unstale":"2024-05-30T17:13:53.118605+0000","last_undegraded":"2024-05-30T17:13:53.118605+0000","last_fullsized":"2024-05-30T17:13:53.118605+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'21","last_scrub_stamp":"2024-05-30T17:13:53.118396+0000","last_deep_scrub":"262'21","last_deep_scrub_stamp":"2024-05-30T17:13:53.118396+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.118396+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:21:33.139209+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":79,"num_write":45,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:11.612685+0000","last_change":"2024-05-30T16:34:55.891074+0000","last_active":"2024-05-30T17:13:11.612685+0000","last_peered":"2024-05-30T17:13:11.612685+0000","last_clean":"2024-05-30T17:13:11.612685+0000","last_became_active":"2024-05-30T16:34:55.890313+0000","last_became_peered":"2024-05-30T16:34:55.890313+0000","last_unstale":"2024-05-30T17:13:11.612685+0000","last_undegraded":"2024-05-30T17:13:11.612685+0000","last_fullsized":"2024-05-30T17:13:11.612685+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"265'3","reported_seq":56,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.567217+0000","last_change":"2024-05-30T16:17:35.243089+0000","last_active":"2024-05-30T17:13:43.567217+0000","last_peered":"2024-05-30T17:13:43.567217+0000","last_clean":"2024-05-30T17:13:43.567217+0000","last_became_active":"2024-05-30T16:17:35.242271+0000","last_became_peered":"2024-05-30T16:17:35.242271+0000","last_unstale":"2024-05-30T17:13:43.567217+0000","last_undegraded":"2024-05-30T17:13:43.567217+0000","last_fullsized":"2024-05-30T17:13:43.567217+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10240,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"265'3","reported_seq":70,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.656221+0000","last_change":"2024-05-30T17:13:51.656292+0000","last_active":"2024-05-30T17:13:51.656221+0000","last_peered":"2024-05-30T17:13:51.656221+0000","last_clean":"2024-05-30T17:13:51.656221+0000","last_became_active":"2024-05-30T16:17:31.194774+0000","last_became_peered":"2024-05-30T16:17:31.194774+0000","last_unstale":"2024-05-30T17:13:51.656221+0000","last_undegraded":"2024-05-30T17:13:51.656221+0000","last_fullsized":"2024-05-30T17:13:51.656221+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:13:51.656016+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:13:51.656016+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.656016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:08:09.027576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":12288,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"265'18","reported_seq":116,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.659792+0000","last_change":"2024-05-30T17:13:44.659853+0000","last_active":"2024-05-30T17:13:44.659792+0000","last_peered":"2024-05-30T17:13:44.659792+0000","last_clean":"2024-05-30T17:13:44.659792+0000","last_became_active":"2024-05-30T16:12:50.820526+0000","last_became_peered":"2024-05-30T16:12:50.820526+0000","last_unstale":"2024-05-30T17:13:44.659792+0000","last_undegraded":"2024-05-30T17:13:44.659792+0000","last_fullsized":"2024-05-30T17:13:44.659792+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'18","last_scrub_stamp":"2024-05-30T17:13:44.659583+0000","last_deep_scrub":"265'18","last_deep_scrub_stamp":"2024-05-30T17:13:44.659583+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.659583+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:01:55.866157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":60,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:21.599695+0000","last_change":"2024-05-30T16:34:55.884293+0000","last_active":"2024-05-30T17:13:21.599695+0000","last_peered":"2024-05-30T17:13:21.599695+0000","last_clean":"2024-05-30T17:13:21.599695+0000","last_became_active":"2024-05-30T16:34:55.884065+0000","last_became_peered":"2024-05-30T16:34:55.884065+0000","last_unstale":"2024-05-30T17:13:21.599695+0000","last_undegraded":"2024-05-30T17:13:21.599695+0000","last_fullsized":"2024-05-30T17:13:21.599695+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"265'11","reported_seq":83,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.610393+0000","last_change":"2024-05-30T17:13:56.610471+0000","last_active":"2024-05-30T17:13:56.610393+0000","last_peered":"2024-05-30T17:13:56.610393+0000","last_clean":"2024-05-30T17:13:56.610393+0000","last_became_active":"2024-05-30T16:17:35.738991+0000","last_became_peered":"2024-05-30T16:17:35.738991+0000","last_unstale":"2024-05-30T17:13:56.610393+0000","last_undegraded":"2024-05-30T17:13:56.610393+0000","last_fullsized":"2024-05-30T17:13:56.610393+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'11","last_scrub_stamp":"2024-05-30T17:13:56.610167+0000","last_deep_scrub":"265'11","last_deep_scrub_stamp":"2024-05-30T17:13:56.610167+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.610167+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:07:32.799346+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":26998,"num_write":14,"num_write_kb":10938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"262'6","reported_seq":75,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.948567+0000","last_change":"2024-05-30T17:13:49.948567+0000","last_active":"2024-05-30T17:13:49.948567+0000","last_peered":"2024-05-30T17:13:49.948567+0000","last_clean":"2024-05-30T17:13:49.948567+0000","last_became_active":"2024-05-30T16:17:31.212954+0000","last_became_peered":"2024-05-30T16:17:31.212954+0000","last_unstale":"2024-05-30T17:13:49.948567+0000","last_undegraded":"2024-05-30T17:13:49.948567+0000","last_fullsized":"2024-05-30T17:13:49.948567+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'6","last_scrub_stamp":"2024-05-30T17:13:49.948291+0000","last_deep_scrub":"262'6","last_deep_scrub_stamp":"2024-05-30T17:13:49.948291+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.948291+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:46:37.622512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":11264,"num_write":11,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"265'103","reported_seq":396,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:57.554791+0000","last_change":"2024-05-30T17:13:57.554869+0000","last_active":"2024-05-30T17:13:57.554791+0000","last_peered":"2024-05-30T17:13:57.554791+0000","last_clean":"2024-05-30T17:13:57.554791+0000","last_became_active":"2024-05-30T16:12:50.820250+0000","last_became_peered":"2024-05-30T16:12:50.820250+0000","last_unstale":"2024-05-30T17:13:57.554791+0000","last_undegraded":"2024-05-30T17:13:57.554791+0000","last_fullsized":"2024-05-30T17:13:57.554791+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'103","last_scrub_stamp":"2024-05-30T17:13:57.554576+0000","last_deep_scrub":"265'103","last_deep_scrub_stamp":"2024-05-30T17:13:57.554576+0000","last_clean_scrub_stamp":"2024-05-30T17:13:57.554576+0000","objects_scrubbed":1,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:49:22.258431+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":642,"num_read_kb":499,"num_write":114,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3960,"num_omap_keys":8,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"265'1","reported_seq":31,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:11.597056+0000","last_change":"2024-05-30T16:34:55.880548+0000","last_active":"2024-05-30T17:13:11.597056+0000","last_peered":"2024-05-30T17:13:11.597056+0000","last_clean":"2024-05-30T17:13:11.597056+0000","last_became_active":"2024-05-30T16:34:55.878171+0000","last_became_peered":"2024-05-30T16:34:55.878171+0000","last_unstale":"2024-05-30T17:13:11.597056+0000","last_undegraded":"2024-05-30T17:13:11.597056+0000","last_fullsized":"2024-05-30T17:13:11.597056+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"265'14","reported_seq":76,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:42.129846+0000","last_change":"2024-05-30T16:17:35.245798+0000","last_active":"2024-05-30T17:13:42.129846+0000","last_peered":"2024-05-30T17:13:42.129846+0000","last_clean":"2024-05-30T17:13:42.129846+0000","last_became_active":"2024-05-30T16:17:35.245008+0000","last_became_peered":"2024-05-30T16:17:35.245008+0000","last_unstale":"2024-05-30T17:13:42.129846+0000","last_undegraded":"2024-05-30T17:13:42.129846+0000","last_fullsized":"2024-05-30T17:13:42.129846+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":25602,"num_write":23,"num_write_kb":17408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"262'3","reported_seq":60,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:56.592820+0000","last_change":"2024-05-30T16:17:31.211050+0000","last_active":"2024-05-30T17:11:56.592820+0000","last_peered":"2024-05-30T17:11:56.592820+0000","last_clean":"2024-05-30T17:11:56.592820+0000","last_became_active":"2024-05-30T16:17:31.210195+0000","last_became_peered":"2024-05-30T16:17:31.210195+0000","last_unstale":"2024-05-30T17:11:56.592820+0000","last_undegraded":"2024-05-30T17:11:56.592820+0000","last_fullsized":"2024-05-30T17:11:56.592820+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"265'43","reported_seq":185,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.468823+0000","last_change":"2024-05-30T17:13:44.468886+0000","last_active":"2024-05-30T17:13:44.468823+0000","last_peered":"2024-05-30T17:13:44.468823+0000","last_clean":"2024-05-30T17:13:44.468823+0000","last_became_active":"2024-05-30T16:12:50.830770+0000","last_became_peered":"2024-05-30T16:12:50.830770+0000","last_unstale":"2024-05-30T17:13:44.468823+0000","last_undegraded":"2024-05-30T17:13:44.468823+0000","last_fullsized":"2024-05-30T17:13:44.468823+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'43","last_scrub_stamp":"2024-05-30T17:13:44.468612+0000","last_deep_scrub":"265'43","last_deep_scrub_stamp":"2024-05-30T17:13:44.468612+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.468612+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:26:30.609648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":214,"num_read_kb":148,"num_write":89,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:11.610397+0000","last_change":"2024-05-30T16:34:55.887780+0000","last_active":"2024-05-30T17:13:11.610397+0000","last_peered":"2024-05-30T17:13:11.610397+0000","last_clean":"2024-05-30T17:13:11.610397+0000","last_became_active":"2024-05-30T16:34:55.887101+0000","last_became_peered":"2024-05-30T16:34:55.887101+0000","last_unstale":"2024-05-30T17:13:11.610397+0000","last_undegraded":"2024-05-30T17:13:11.610397+0000","last_fullsized":"2024-05-30T17:13:11.610397+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"262'3","reported_seq":62,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:52.503910+0000","last_change":"2024-05-30T17:13:52.503977+0000","last_active":"2024-05-30T17:13:52.503910+0000","last_peered":"2024-05-30T17:13:52.503910+0000","last_clean":"2024-05-30T17:13:52.503910+0000","last_became_active":"2024-05-30T16:17:35.212403+0000","last_became_peered":"2024-05-30T16:17:35.212403+0000","last_unstale":"2024-05-30T17:13:52.503910+0000","last_undegraded":"2024-05-30T17:13:52.503910+0000","last_fullsized":"2024-05-30T17:13:52.503910+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'3","last_scrub_stamp":"2024-05-30T17:13:52.503699+0000","last_deep_scrub":"262'3","last_deep_scrub_stamp":"2024-05-30T17:13:52.503699+0000","last_clean_scrub_stamp":"2024-05-30T17:13:52.503699+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:51:26.227521+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"0'0","reported_seq":62,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.933818+0000","last_change":"2024-05-30T17:13:58.933876+0000","last_active":"2024-05-30T17:13:58.933818+0000","last_peered":"2024-05-30T17:13:58.933818+0000","last_clean":"2024-05-30T17:13:58.933818+0000","last_became_active":"2024-05-30T16:17:31.191955+0000","last_became_peered":"2024-05-30T16:17:31.191955+0000","last_unstale":"2024-05-30T17:13:58.933818+0000","last_undegraded":"2024-05-30T17:13:58.933818+0000","last_fullsized":"2024-05-30T17:13:58.933818+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:58.933611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:58.933611+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.933611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:34:51.660228+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"262'8","reported_seq":92,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.839264+0000","last_change":"2024-05-30T17:13:46.839335+0000","last_active":"2024-05-30T17:13:46.839264+0000","last_peered":"2024-05-30T17:13:46.839264+0000","last_clean":"2024-05-30T17:13:46.839264+0000","last_became_active":"2024-05-30T16:12:50.819871+0000","last_became_peered":"2024-05-30T16:12:50.819871+0000","last_unstale":"2024-05-30T17:13:46.839264+0000","last_undegraded":"2024-05-30T17:13:46.839264+0000","last_fullsized":"2024-05-30T17:13:46.839264+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'8","last_scrub_stamp":"2024-05-30T17:13:46.839029+0000","last_deep_scrub":"262'8","last_deep_scrub_stamp":"2024-05-30T17:13:46.839029+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.839029+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:29:46.097404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":28,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":31,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:26.585480+0000","last_change":"2024-05-30T16:34:55.883102+0000","last_active":"2024-05-30T17:12:26.585480+0000","last_peered":"2024-05-30T17:12:26.585480+0000","last_clean":"2024-05-30T17:12:26.585480+0000","last_became_active":"2024-05-30T16:34:55.882443+0000","last_became_peered":"2024-05-30T16:34:55.882443+0000","last_unstale":"2024-05-30T17:12:26.585480+0000","last_undegraded":"2024-05-30T17:12:26.585480+0000","last_fullsized":"2024-05-30T17:12:26.585480+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"262'3","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:01.594411+0000","last_change":"2024-05-30T16:17:35.248377+0000","last_active":"2024-05-30T17:12:01.594411+0000","last_peered":"2024-05-30T17:12:01.594411+0000","last_clean":"2024-05-30T17:12:01.594411+0000","last_became_active":"2024-05-30T16:17:35.247419+0000","last_became_peered":"2024-05-30T16:17:35.247419+0000","last_unstale":"2024-05-30T17:12:01.594411+0000","last_undegraded":"2024-05-30T17:12:01.594411+0000","last_fullsized":"2024-05-30T17:12:01.594411+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"262'2","reported_seq":68,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.502634+0000","last_change":"2024-05-30T17:13:50.502697+0000","last_active":"2024-05-30T17:13:50.502634+0000","last_peered":"2024-05-30T17:13:50.502634+0000","last_clean":"2024-05-30T17:13:50.502634+0000","last_became_active":"2024-05-30T16:17:31.189122+0000","last_became_peered":"2024-05-30T16:17:31.189122+0000","last_unstale":"2024-05-30T17:13:50.502634+0000","last_undegraded":"2024-05-30T17:13:50.502634+0000","last_fullsized":"2024-05-30T17:13:50.502634+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'2","last_scrub_stamp":"2024-05-30T17:13:50.502442+0000","last_deep_scrub":"262'2","last_deep_scrub_stamp":"2024-05-30T17:13:50.502442+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.502442+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:48:21.318011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"265'53","reported_seq":210,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.655149+0000","last_change":"2024-05-30T17:13:41.655171+0000","last_active":"2024-05-30T17:13:41.655149+0000","last_peered":"2024-05-30T17:13:41.655149+0000","last_clean":"2024-05-30T17:13:41.655149+0000","last_became_active":"2024-05-30T16:12:50.809372+0000","last_became_peered":"2024-05-30T16:12:50.809372+0000","last_unstale":"2024-05-30T17:13:41.655149+0000","last_undegraded":"2024-05-30T17:13:41.655149+0000","last_fullsized":"2024-05-30T17:13:41.655149+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'53","last_scrub_stamp":"2024-05-30T17:13:41.655071+0000","last_deep_scrub":"265'53","last_deep_scrub_stamp":"2024-05-30T17:13:41.655071+0000","last_clean_scrub_stamp":"2024-05-30T17:13:41.655071+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:39:32.196268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":186,"num_write":103,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":31,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:26.969180+0000","last_change":"2024-05-30T16:34:55.881085+0000","last_active":"2024-05-30T17:12:26.969180+0000","last_peered":"2024-05-30T17:12:26.969180+0000","last_clean":"2024-05-30T17:12:26.969180+0000","last_became_active":"2024-05-30T16:34:55.880627+0000","last_became_peered":"2024-05-30T16:34:55.880627+0000","last_unstale":"2024-05-30T17:12:26.969180+0000","last_undegraded":"2024-05-30T17:12:26.969180+0000","last_fullsized":"2024-05-30T17:12:26.969180+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"0'0","reported_seq":50,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.648090+0000","last_change":"2024-05-30T16:17:35.743808+0000","last_active":"2024-05-30T17:13:41.648090+0000","last_peered":"2024-05-30T17:13:41.648090+0000","last_clean":"2024-05-30T17:13:41.648090+0000","last_became_active":"2024-05-30T16:17:35.743036+0000","last_became_peered":"2024-05-30T16:17:35.743036+0000","last_unstale":"2024-05-30T17:13:41.648090+0000","last_undegraded":"2024-05-30T17:13:41.648090+0000","last_fullsized":"2024-05-30T17:13:41.648090+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"265'3","reported_seq":61,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:42.601563+0000","last_change":"2024-05-30T16:17:31.202885+0000","last_active":"2024-05-30T17:13:42.601563+0000","last_peered":"2024-05-30T17:13:42.601563+0000","last_clean":"2024-05-30T17:13:42.601563+0000","last_became_active":"2024-05-30T16:17:31.202064+0000","last_became_peered":"2024-05-30T16:17:31.202064+0000","last_unstale":"2024-05-30T17:13:42.601563+0000","last_undegraded":"2024-05-30T17:13:42.601563+0000","last_fullsized":"2024-05-30T17:13:42.601563+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2048,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"265'34","reported_seq":158,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.862253+0000","last_change":"2024-05-30T17:13:47.862374+0000","last_active":"2024-05-30T17:13:47.862253+0000","last_peered":"2024-05-30T17:13:47.862253+0000","last_clean":"2024-05-30T17:13:47.862253+0000","last_became_active":"2024-05-30T16:12:50.815895+0000","last_became_peered":"2024-05-30T16:12:50.815895+0000","last_unstale":"2024-05-30T17:13:47.862253+0000","last_undegraded":"2024-05-30T17:13:47.862253+0000","last_fullsized":"2024-05-30T17:13:47.862253+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'34","last_scrub_stamp":"2024-05-30T17:13:47.861959+0000","last_deep_scrub":"265'34","last_deep_scrub_stamp":"2024-05-30T17:13:47.861959+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.861959+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:37:52.262386+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":160,"num_read_kb":111,"num_write":71,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":32,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:01.813668+0000","last_change":"2024-05-30T16:34:55.883831+0000","last_active":"2024-05-30T17:13:01.813668+0000","last_peered":"2024-05-30T17:13:01.813668+0000","last_clean":"2024-05-30T17:13:01.813668+0000","last_became_active":"2024-05-30T16:34:55.883112+0000","last_became_peered":"2024-05-30T16:34:55.883112+0000","last_unstale":"2024-05-30T17:13:01.813668+0000","last_undegraded":"2024-05-30T17:13:01.813668+0000","last_fullsized":"2024-05-30T17:13:01.813668+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"265'3","reported_seq":62,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:00.593884+0000","last_change":"2024-05-30T17:14:00.593945+0000","last_active":"2024-05-30T17:14:00.593884+0000","last_peered":"2024-05-30T17:14:00.593884+0000","last_clean":"2024-05-30T17:14:00.593884+0000","last_became_active":"2024-05-30T16:17:35.743608+0000","last_became_peered":"2024-05-30T16:17:35.743608+0000","last_unstale":"2024-05-30T17:14:00.593884+0000","last_undegraded":"2024-05-30T17:14:00.593884+0000","last_fullsized":"2024-05-30T17:14:00.593884+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:14:00.593691+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:14:00.593691+0000","last_clean_scrub_stamp":"2024-05-30T17:14:00.593691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:51:53.188748+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"0'0","reported_seq":63,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:52.225937+0000","last_change":"2024-05-30T17:13:52.225991+0000","last_active":"2024-05-30T17:13:52.225937+0000","last_peered":"2024-05-30T17:13:52.225937+0000","last_clean":"2024-05-30T17:13:52.225937+0000","last_became_active":"2024-05-30T16:17:31.188810+0000","last_became_peered":"2024-05-30T16:17:31.188810+0000","last_unstale":"2024-05-30T17:13:52.225937+0000","last_undegraded":"2024-05-30T17:13:52.225937+0000","last_fullsized":"2024-05-30T17:13:52.225937+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:52.225736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:52.225736+0000","last_clean_scrub_stamp":"2024-05-30T17:13:52.225736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:31:29.155002+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"265'18","reported_seq":120,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.502424+0000","last_change":"2024-05-30T17:13:53.502489+0000","last_active":"2024-05-30T17:13:53.502424+0000","last_peered":"2024-05-30T17:13:53.502424+0000","last_clean":"2024-05-30T17:13:53.502424+0000","last_became_active":"2024-05-30T16:12:50.826880+0000","last_became_peered":"2024-05-30T16:12:50.826880+0000","last_unstale":"2024-05-30T17:13:53.502424+0000","last_undegraded":"2024-05-30T17:13:53.502424+0000","last_fullsized":"2024-05-30T17:13:53.502424+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'18","last_scrub_stamp":"2024-05-30T17:13:53.502233+0000","last_deep_scrub":"265'18","last_deep_scrub_stamp":"2024-05-30T17:13:53.502233+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.502233+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:07:36.967499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":61,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":31,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:26.764016+0000","last_change":"2024-05-30T16:34:55.886177+0000","last_active":"2024-05-30T17:12:26.764016+0000","last_peered":"2024-05-30T17:12:26.764016+0000","last_clean":"2024-05-30T17:12:26.764016+0000","last_became_active":"2024-05-30T16:34:55.885643+0000","last_became_peered":"2024-05-30T16:34:55.885643+0000","last_unstale":"2024-05-30T17:12:26.764016+0000","last_undegraded":"2024-05-30T17:12:26.764016+0000","last_fullsized":"2024-05-30T17:12:26.764016+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"262'2","reported_seq":53,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.567194+0000","last_change":"2024-05-30T16:17:35.248399+0000","last_active":"2024-05-30T17:13:43.567194+0000","last_peered":"2024-05-30T17:13:43.567194+0000","last_clean":"2024-05-30T17:13:43.567194+0000","last_became_active":"2024-05-30T16:17:35.247757+0000","last_became_peered":"2024-05-30T16:17:35.247757+0000","last_unstale":"2024-05-30T17:13:43.567194+0000","last_undegraded":"2024-05-30T17:13:43.567194+0000","last_fullsized":"2024-05-30T17:13:43.567194+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"0'0","reported_seq":63,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.456500+0000","last_change":"2024-05-30T17:13:46.456500+0000","last_active":"2024-05-30T17:13:46.456500+0000","last_peered":"2024-05-30T17:13:46.456500+0000","last_clean":"2024-05-30T17:13:46.456500+0000","last_became_active":"2024-05-30T16:17:31.203452+0000","last_became_peered":"2024-05-30T16:17:31.203452+0000","last_unstale":"2024-05-30T17:13:46.456500+0000","last_undegraded":"2024-05-30T17:13:46.456500+0000","last_fullsized":"2024-05-30T17:13:46.456500+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:46.456245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:46.456245+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.456245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:07:15.159368+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"265'37","reported_seq":171,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.919132+0000","last_change":"2024-05-30T17:13:46.919198+0000","last_active":"2024-05-30T17:13:46.919132+0000","last_peered":"2024-05-30T17:13:46.919132+0000","last_clean":"2024-05-30T17:13:46.919132+0000","last_became_active":"2024-05-30T16:12:50.811626+0000","last_became_peered":"2024-05-30T16:12:50.811626+0000","last_unstale":"2024-05-30T17:13:46.919132+0000","last_undegraded":"2024-05-30T17:13:46.919132+0000","last_fullsized":"2024-05-30T17:13:46.919132+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'37","last_scrub_stamp":"2024-05-30T17:13:46.918931+0000","last_deep_scrub":"265'37","last_deep_scrub_stamp":"2024-05-30T17:13:46.918931+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.918931+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:38:14.857169+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":184,"num_read_kb":132,"num_write":81,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"264'1","reported_seq":32,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:42.601726+0000","last_change":"2024-05-30T16:34:55.862486+0000","last_active":"2024-05-30T17:13:42.601726+0000","last_peered":"2024-05-30T17:13:42.601726+0000","last_clean":"2024-05-30T17:13:42.601726+0000","last_became_active":"2024-05-30T16:34:55.861767+0000","last_became_peered":"2024-05-30T16:34:55.861767+0000","last_unstale":"2024-05-30T17:13:42.601726+0000","last_undegraded":"2024-05-30T17:13:42.601726+0000","last_fullsized":"2024-05-30T17:13:42.601726+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"265'8","reported_seq":65,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:42.129827+0000","last_change":"2024-05-30T16:17:35.244195+0000","last_active":"2024-05-30T17:13:42.129827+0000","last_peered":"2024-05-30T17:13:42.129827+0000","last_clean":"2024-05-30T17:13:42.129827+0000","last_became_active":"2024-05-30T16:17:35.243435+0000","last_became_peered":"2024-05-30T16:17:35.243435+0000","last_unstale":"2024-05-30T17:13:42.129827+0000","last_undegraded":"2024-05-30T17:13:42.129827+0000","last_fullsized":"2024-05-30T17:13:42.129827+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":13312,"num_write":13,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"265'3","reported_seq":68,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.458042+0000","last_change":"2024-05-30T17:13:47.458106+0000","last_active":"2024-05-30T17:13:47.458042+0000","last_peered":"2024-05-30T17:13:47.458042+0000","last_clean":"2024-05-30T17:13:47.458042+0000","last_became_active":"2024-05-30T16:17:31.200663+0000","last_became_peered":"2024-05-30T16:17:31.200663+0000","last_unstale":"2024-05-30T17:13:47.458042+0000","last_undegraded":"2024-05-30T17:13:47.458042+0000","last_fullsized":"2024-05-30T17:13:47.458042+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:13:47.457841+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:13:47.457841+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.457841+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:37:54.600403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"265'15","reported_seq":112,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.234861+0000","last_change":"2024-05-30T17:13:48.234982+0000","last_active":"2024-05-30T17:13:48.234861+0000","last_peered":"2024-05-30T17:13:48.234861+0000","last_clean":"2024-05-30T17:13:48.234861+0000","last_became_active":"2024-05-30T16:12:50.813741+0000","last_became_peered":"2024-05-30T16:12:50.813741+0000","last_unstale":"2024-05-30T17:13:48.234861+0000","last_undegraded":"2024-05-30T17:13:48.234861+0000","last_fullsized":"2024-05-30T17:13:48.234861+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'15","last_scrub_stamp":"2024-05-30T17:13:48.234638+0000","last_deep_scrub":"265'15","last_deep_scrub_stamp":"2024-05-30T17:13:48.234638+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.234638+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:15:02.442194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":53,"num_write":33,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":31,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.648361+0000","last_change":"2024-05-30T16:34:55.859738+0000","last_active":"2024-05-30T17:13:41.648361+0000","last_peered":"2024-05-30T17:13:41.648361+0000","last_clean":"2024-05-30T17:13:41.648361+0000","last_became_active":"2024-05-30T16:34:55.858872+0000","last_became_peered":"2024-05-30T16:34:55.858872+0000","last_unstale":"2024-05-30T17:13:41.648361+0000","last_undegraded":"2024-05-30T17:13:41.648361+0000","last_fullsized":"2024-05-30T17:13:41.648361+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"0'0","reported_seq":51,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.067117+0000","last_change":"2024-05-30T16:17:35.227683+0000","last_active":"2024-05-30T17:13:44.067117+0000","last_peered":"2024-05-30T17:13:44.067117+0000","last_clean":"2024-05-30T17:13:44.067117+0000","last_became_active":"2024-05-30T16:17:35.226829+0000","last_became_peered":"2024-05-30T16:17:35.226829+0000","last_unstale":"2024-05-30T17:13:44.067117+0000","last_undegraded":"2024-05-30T17:13:44.067117+0000","last_fullsized":"2024-05-30T17:13:44.067117+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"0'0","reported_seq":64,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.486126+0000","last_change":"2024-05-30T17:13:48.486182+0000","last_active":"2024-05-30T17:13:48.486126+0000","last_peered":"2024-05-30T17:13:48.486126+0000","last_clean":"2024-05-30T17:13:48.486126+0000","last_became_active":"2024-05-30T16:17:31.203535+0000","last_became_peered":"2024-05-30T16:17:31.203535+0000","last_unstale":"2024-05-30T17:13:48.486126+0000","last_undegraded":"2024-05-30T17:13:48.486126+0000","last_fullsized":"2024-05-30T17:13:48.486126+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:48.485936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:48.485936+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.485936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:42:58.343268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"265'36","reported_seq":166,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.750202+0000","last_change":"2024-05-30T17:13:43.750267+0000","last_active":"2024-05-30T17:13:43.750202+0000","last_peered":"2024-05-30T17:13:43.750202+0000","last_clean":"2024-05-30T17:13:43.750202+0000","last_became_active":"2024-05-30T16:12:50.831842+0000","last_became_peered":"2024-05-30T16:12:50.831842+0000","last_unstale":"2024-05-30T17:13:43.750202+0000","last_undegraded":"2024-05-30T17:13:43.750202+0000","last_fullsized":"2024-05-30T17:13:43.750202+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'36","last_scrub_stamp":"2024-05-30T17:13:43.750005+0000","last_deep_scrub":"265'36","last_deep_scrub_stamp":"2024-05-30T17:13:43.750005+0000","last_clean_scrub_stamp":"2024-05-30T17:13:43.750005+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:20:47.390125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":177,"num_read_kb":125,"num_write":75,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":31,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:26.600628+0000","last_change":"2024-05-30T16:34:55.863472+0000","last_active":"2024-05-30T17:12:26.600628+0000","last_peered":"2024-05-30T17:12:26.600628+0000","last_clean":"2024-05-30T17:12:26.600628+0000","last_became_active":"2024-05-30T16:34:55.862709+0000","last_became_peered":"2024-05-30T16:34:55.862709+0000","last_unstale":"2024-05-30T17:12:26.600628+0000","last_undegraded":"2024-05-30T17:12:26.600628+0000","last_fullsized":"2024-05-30T17:12:26.600628+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"265'3","reported_seq":53,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.566694+0000","last_change":"2024-05-30T16:17:35.241271+0000","last_active":"2024-05-30T17:13:44.566694+0000","last_peered":"2024-05-30T17:13:44.566694+0000","last_clean":"2024-05-30T17:13:44.566694+0000","last_became_active":"2024-05-30T16:17:35.240472+0000","last_became_peered":"2024-05-30T16:17:35.240472+0000","last_unstale":"2024-05-30T17:13:44.566694+0000","last_undegraded":"2024-05-30T17:13:44.566694+0000","last_fullsized":"2024-05-30T17:13:44.566694+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"265'6","reported_seq":74,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.031484+0000","last_change":"2024-05-30T17:13:44.031484+0000","last_active":"2024-05-30T17:13:44.031484+0000","last_peered":"2024-05-30T17:13:44.031484+0000","last_clean":"2024-05-30T17:13:44.031484+0000","last_became_active":"2024-05-30T16:17:31.187958+0000","last_became_peered":"2024-05-30T16:17:31.187958+0000","last_unstale":"2024-05-30T17:13:44.031484+0000","last_undegraded":"2024-05-30T17:13:44.031484+0000","last_fullsized":"2024-05-30T17:13:44.031484+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'6","last_scrub_stamp":"2024-05-30T17:13:44.031239+0000","last_deep_scrub":"265'6","last_deep_scrub_stamp":"2024-05-30T17:13:44.031239+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.031239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:29:30.262819+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":10240,"num_write":16,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"265'18","reported_seq":121,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.918092+0000","last_change":"2024-05-30T17:13:44.918156+0000","last_active":"2024-05-30T17:13:44.918092+0000","last_peered":"2024-05-30T17:13:44.918092+0000","last_clean":"2024-05-30T17:13:44.918092+0000","last_became_active":"2024-05-30T16:12:50.823090+0000","last_became_peered":"2024-05-30T16:12:50.823090+0000","last_unstale":"2024-05-30T17:13:44.918092+0000","last_undegraded":"2024-05-30T17:13:44.918092+0000","last_fullsized":"2024-05-30T17:13:44.918092+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'18","last_scrub_stamp":"2024-05-30T17:13:44.917823+0000","last_deep_scrub":"265'18","last_deep_scrub_stamp":"2024-05-30T17:13:44.917823+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.917823+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:31:54.618562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":69,"num_write":35,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":31,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:36.588097+0000","last_change":"2024-05-30T16:34:55.891440+0000","last_active":"2024-05-30T17:12:36.588097+0000","last_peered":"2024-05-30T17:12:36.588097+0000","last_clean":"2024-05-30T17:12:36.588097+0000","last_became_active":"2024-05-30T16:34:55.890688+0000","last_became_peered":"2024-05-30T16:34:55.890688+0000","last_unstale":"2024-05-30T17:12:36.588097+0000","last_undegraded":"2024-05-30T17:12:36.588097+0000","last_fullsized":"2024-05-30T17:12:36.588097+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"265'8","reported_seq":68,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.567154+0000","last_change":"2024-05-30T16:17:35.238582+0000","last_active":"2024-05-30T17:13:43.567154+0000","last_peered":"2024-05-30T17:13:43.567154+0000","last_clean":"2024-05-30T17:13:43.567154+0000","last_became_active":"2024-05-30T16:17:35.237031+0000","last_became_peered":"2024-05-30T16:17:35.237031+0000","last_unstale":"2024-05-30T17:13:43.567154+0000","last_undegraded":"2024-05-30T17:13:43.567154+0000","last_fullsized":"2024-05-30T17:13:43.567154+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":20482,"num_write":17,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"265'6","reported_seq":76,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.818137+0000","last_change":"2024-05-30T17:13:48.818281+0000","last_active":"2024-05-30T17:13:48.818137+0000","last_peered":"2024-05-30T17:13:48.818137+0000","last_clean":"2024-05-30T17:13:48.818137+0000","last_became_active":"2024-05-30T16:17:31.203871+0000","last_became_peered":"2024-05-30T16:17:31.203871+0000","last_unstale":"2024-05-30T17:13:48.818137+0000","last_undegraded":"2024-05-30T17:13:48.818137+0000","last_fullsized":"2024-05-30T17:13:48.818137+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'6","last_scrub_stamp":"2024-05-30T17:13:48.817847+0000","last_deep_scrub":"265'6","last_deep_scrub_stamp":"2024-05-30T17:13:48.817847+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.817847+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:26:36.664287+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":4098,"num_write":9,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"265'24","reported_seq":142,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.471273+0000","last_change":"2024-05-30T17:13:51.471404+0000","last_active":"2024-05-30T17:13:51.471273+0000","last_peered":"2024-05-30T17:13:51.471273+0000","last_clean":"2024-05-30T17:13:51.471273+0000","last_became_active":"2024-05-30T16:12:50.827078+0000","last_became_peered":"2024-05-30T16:12:50.827078+0000","last_unstale":"2024-05-30T17:13:51.471273+0000","last_undegraded":"2024-05-30T17:13:51.471273+0000","last_fullsized":"2024-05-30T17:13:51.471273+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'24","last_scrub_stamp":"2024-05-30T17:13:51.471027+0000","last_deep_scrub":"265'24","last_deep_scrub_stamp":"2024-05-30T17:13:51.471027+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.471027+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:41:56.836293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":93,"num_write":46,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"265'165","reported_seq":777,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:26.601117+0000","last_change":"2024-05-30T15:54:16.662201+0000","last_active":"2024-05-30T17:13:26.601117+0000","last_peered":"2024-05-30T17:13:26.601117+0000","last_clean":"2024-05-30T17:13:26.601117+0000","last_became_active":"2024-05-30T15:54:16.661334+0000","last_became_peered":"2024-05-30T15:54:16.661334+0000","last_unstale":"2024-05-30T17:13:26.601117+0000","last_undegraded":"2024-05-30T17:13:26.601117+0000","last_fullsized":"2024-05-30T17:13:26.601117+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":100,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2159792,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":347,"num_read_kb":7638,"num_write":1029,"num_write_kb":24692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"265'436","reported_seq":2070,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:33.939613+0000","last_change":"2024-05-30T15:54:18.669748+0000","last_active":"2024-05-30T17:12:33.939613+0000","last_peered":"2024-05-30T17:12:33.939613+0000","last_clean":"2024-05-30T17:12:33.939613+0000","last_became_active":"2024-05-30T15:54:18.669016+0000","last_became_peered":"2024-05-30T15:54:18.669016+0000","last_unstale":"2024-05-30T17:12:33.939613+0000","last_undegraded":"2024-05-30T17:12:33.939613+0000","last_fullsized":"2024-05-30T17:12:33.939613+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":400,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2479,"num_read_kb":2366,"num_write":506,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"265'234","reported_seq":890,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.402416+0000","last_change":"2024-05-30T15:54:16.653102+0000","last_active":"2024-05-30T17:13:38.402416+0000","last_peered":"2024-05-30T17:13:38.402416+0000","last_clean":"2024-05-30T17:13:38.402416+0000","last_became_active":"2024-05-30T15:54:16.652724+0000","last_became_peered":"2024-05-30T15:54:16.652724+0000","last_unstale":"2024-05-30T17:13:38.402416+0000","last_undegraded":"2024-05-30T17:13:38.402416+0000","last_fullsized":"2024-05-30T17:13:38.402416+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":200,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6513950,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":554,"num_read_kb":8631,"num_write":1281,"num_write_kb":24636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"265'758","reported_seq":2114,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:59.112000+0000","last_change":"2024-05-30T15:54:18.649460+0000","last_active":"2024-05-30T17:11:59.112000+0000","last_peered":"2024-05-30T17:11:59.112000+0000","last_clean":"2024-05-30T17:11:59.112000+0000","last_became_active":"2024-05-30T15:54:18.648675+0000","last_became_peered":"2024-05-30T15:54:18.648675+0000","last_unstale":"2024-05-30T17:11:59.112000+0000","last_undegraded":"2024-05-30T17:11:59.112000+0000","last_fullsized":"2024-05-30T17:11:59.112000+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":700,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3292,"num_read_kb":2905,"num_write":1031,"num_write_kb":415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"265'208","reported_seq":841,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.415910+0000","last_change":"2024-05-30T15:54:16.633538+0000","last_active":"2024-05-30T17:13:38.415910+0000","last_peered":"2024-05-30T17:13:38.415910+0000","last_clean":"2024-05-30T17:13:38.415910+0000","last_became_active":"2024-05-30T15:54:16.633011+0000","last_became_peered":"2024-05-30T15:54:16.633011+0000","last_unstale":"2024-05-30T17:13:38.415910+0000","last_undegraded":"2024-05-30T17:13:38.415910+0000","last_fullsized":"2024-05-30T17:13:38.415910+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":100,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11855586,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":404,"num_read_kb":43860,"num_write":998,"num_write_kb":74051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"265'746","reported_seq":2145,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:33.944749+0000","last_change":"2024-05-30T15:54:18.651676+0000","last_active":"2024-05-30T17:12:33.944749+0000","last_peered":"2024-05-30T17:12:33.944749+0000","last_clean":"2024-05-30T17:12:33.944749+0000","last_became_active":"2024-05-30T15:54:18.649608+0000","last_became_peered":"2024-05-30T15:54:18.649608+0000","last_unstale":"2024-05-30T17:12:33.944749+0000","last_undegraded":"2024-05-30T17:12:33.944749+0000","last_fullsized":"2024-05-30T17:12:33.944749+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":700,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3275,"num_read_kb":2909,"num_write":996,"num_write_kb":378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"265'154","reported_seq":793,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.303761+0000","last_change":"2024-05-30T15:54:16.653236+0000","last_active":"2024-05-30T17:13:38.303761+0000","last_peered":"2024-05-30T17:13:38.303761+0000","last_clean":"2024-05-30T17:13:38.303761+0000","last_became_active":"2024-05-30T15:54:16.652315+0000","last_became_peered":"2024-05-30T15:54:16.652315+0000","last_unstale":"2024-05-30T17:13:38.303761+0000","last_undegraded":"2024-05-30T17:13:38.303761+0000","last_fullsized":"2024-05-30T17:13:38.303761+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":100,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1250479,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":437,"num_read_kb":3484,"num_write":911,"num_write_kb":6490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"265'487","reported_seq":1963,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:09.101198+0000","last_change":"2024-05-30T15:54:18.652741+0000","last_active":"2024-05-30T17:13:09.101198+0000","last_peered":"2024-05-30T17:13:09.101198+0000","last_clean":"2024-05-30T17:13:09.101198+0000","last_became_active":"2024-05-30T15:54:18.652451+0000","last_became_peered":"2024-05-30T15:54:18.652451+0000","last_unstale":"2024-05-30T17:13:09.101198+0000","last_undegraded":"2024-05-30T17:13:09.101198+0000","last_fullsized":"2024-05-30T17:13:09.101198+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":400,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2387,"num_read_kb":2188,"num_write":616,"num_write_kb":168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"265'245","reported_seq":960,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.351165+0000","last_change":"2024-05-30T15:54:16.659161+0000","last_active":"2024-05-30T17:13:38.351165+0000","last_peered":"2024-05-30T17:13:38.351165+0000","last_clean":"2024-05-30T17:13:38.351165+0000","last_became_active":"2024-05-30T15:54:16.658338+0000","last_became_peered":"2024-05-30T15:54:16.658338+0000","last_unstale":"2024-05-30T17:13:38.351165+0000","last_undegraded":"2024-05-30T17:13:38.351165+0000","last_fullsized":"2024-05-30T17:13:38.351165+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":200,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291491,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":830,"num_read_kb":24681,"num_write":1246,"num_write_kb":34031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"265'370","reported_seq":1863,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:41.576986+0000","last_change":"2024-05-30T15:54:18.669822+0000","last_active":"2024-05-30T17:12:41.576986+0000","last_peered":"2024-05-30T17:12:41.576986+0000","last_clean":"2024-05-30T17:12:41.576986+0000","last_became_active":"2024-05-30T15:54:18.669017+0000","last_became_peered":"2024-05-30T15:54:18.669017+0000","last_unstale":"2024-05-30T17:12:41.576986+0000","last_undegraded":"2024-05-30T17:12:41.576986+0000","last_fullsized":"2024-05-30T17:12:41.576986+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":300,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1986,"num_read_kb":1906,"num_write":419,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"265'196","reported_seq":822,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.356961+0000","last_change":"2024-05-30T15:54:16.673757+0000","last_active":"2024-05-30T17:13:38.356961+0000","last_peered":"2024-05-30T17:13:38.356961+0000","last_clean":"2024-05-30T17:13:38.356961+0000","last_became_active":"2024-05-30T15:54:16.671120+0000","last_became_peered":"2024-05-30T15:54:16.671120+0000","last_unstale":"2024-05-30T17:13:38.356961+0000","last_undegraded":"2024-05-30T17:13:38.356961+0000","last_fullsized":"2024-05-30T17:13:38.356961+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":100,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":81,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":438,"num_read_kb":4718,"num_write":1101,"num_write_kb":20583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"265'725","reported_seq":2202,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:31.779327+0000","last_change":"2024-05-30T15:54:18.651587+0000","last_active":"2024-05-30T17:13:31.779327+0000","last_peered":"2024-05-30T17:13:31.779327+0000","last_clean":"2024-05-30T17:13:31.779327+0000","last_became_active":"2024-05-30T15:54:18.650543+0000","last_became_peered":"2024-05-30T15:54:18.650543+0000","last_unstale":"2024-05-30T17:13:31.779327+0000","last_undegraded":"2024-05-30T17:13:31.779327+0000","last_fullsized":"2024-05-30T17:13:31.779327+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":700,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3282,"num_read_kb":2952,"num_write":973,"num_write_kb":356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"265'192","reported_seq":833,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.350917+0000","last_change":"2024-05-30T15:54:16.679393+0000","last_active":"2024-05-30T17:13:38.350917+0000","last_peered":"2024-05-30T17:13:38.350917+0000","last_clean":"2024-05-30T17:13:38.350917+0000","last_became_active":"2024-05-30T15:54:16.678799+0000","last_became_peered":"2024-05-30T15:54:16.678799+0000","last_unstale":"2024-05-30T17:13:38.350917+0000","last_undegraded":"2024-05-30T17:13:38.350917+0000","last_fullsized":"2024-05-30T17:13:38.350917+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":100,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":491377,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":467,"num_read_kb":33008,"num_write":1090,"num_write_kb":27021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"265'850","reported_seq":2540,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:31.984380+0000","last_change":"2024-05-30T15:54:18.657813+0000","last_active":"2024-05-30T17:13:31.984380+0000","last_peered":"2024-05-30T17:13:31.984380+0000","last_clean":"2024-05-30T17:13:31.984380+0000","last_became_active":"2024-05-30T15:54:18.656475+0000","last_became_peered":"2024-05-30T15:54:18.656475+0000","last_unstale":"2024-05-30T17:13:31.984380+0000","last_undegraded":"2024-05-30T17:13:31.984380+0000","last_fullsized":"2024-05-30T17:13:31.984380+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":800,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4088,"num_read_kb":3692,"num_write":1166,"num_write_kb":472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"265'140","reported_seq":737,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.110035+0000","last_change":"2024-05-30T15:54:16.669564+0000","last_active":"2024-05-30T17:13:38.110035+0000","last_peered":"2024-05-30T17:13:38.110035+0000","last_clean":"2024-05-30T17:13:38.110035+0000","last_became_active":"2024-05-30T15:54:16.669095+0000","last_became_peered":"2024-05-30T15:54:16.669095+0000","last_unstale":"2024-05-30T17:13:38.110035+0000","last_undegraded":"2024-05-30T17:13:38.110035+0000","last_fullsized":"2024-05-30T17:13:38.110035+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":100,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5243043,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":303,"num_read_kb":17751,"num_write":817,"num_write_kb":30378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"265'369","reported_seq":1791,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:31.585378+0000","last_change":"2024-05-30T15:54:18.653529+0000","last_active":"2024-05-30T17:13:31.585378+0000","last_peered":"2024-05-30T17:13:31.585378+0000","last_clean":"2024-05-30T17:13:31.585378+0000","last_became_active":"2024-05-30T15:54:18.653189+0000","last_became_peered":"2024-05-30T15:54:18.653189+0000","last_unstale":"2024-05-30T17:13:31.585378+0000","last_undegraded":"2024-05-30T17:13:31.585378+0000","last_fullsized":"2024-05-30T17:13:31.585378+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":300,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1969,"num_read_kb":1855,"num_write":435,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"265'336","reported_seq":1766,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.781326+0000","last_change":"2024-05-30T15:54:18.652246+0000","last_active":"2024-05-30T17:13:41.781326+0000","last_peered":"2024-05-30T17:13:41.781326+0000","last_clean":"2024-05-30T17:13:41.781326+0000","last_became_active":"2024-05-30T15:54:18.649994+0000","last_became_peered":"2024-05-30T15:54:18.649994+0000","last_unstale":"2024-05-30T17:13:41.781326+0000","last_undegraded":"2024-05-30T17:13:41.781326+0000","last_fullsized":"2024-05-30T17:13:41.781326+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1976,"num_read_kb":1893,"num_write":388,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"265'212","reported_seq":878,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.395120+0000","last_change":"2024-05-30T15:54:16.650771+0000","last_active":"2024-05-30T17:13:38.395120+0000","last_peered":"2024-05-30T17:13:38.395120+0000","last_clean":"2024-05-30T17:13:38.395120+0000","last_became_active":"2024-05-30T15:54:16.649872+0000","last_became_peered":"2024-05-30T15:54:16.649872+0000","last_unstale":"2024-05-30T17:13:38.395120+0000","last_undegraded":"2024-05-30T17:13:38.395120+0000","last_fullsized":"2024-05-30T17:13:38.395120+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33349,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":510,"num_read_kb":59018,"num_write":1166,"num_write_kb":23092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"265'1525","reported_seq":3132,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.604880+0000","last_change":"2024-05-30T15:54:18.649618+0000","last_active":"2024-05-30T17:13:41.604880+0000","last_peered":"2024-05-30T17:13:41.604880+0000","last_clean":"2024-05-30T17:13:41.604880+0000","last_became_active":"2024-05-30T15:54:18.648860+0000","last_became_peered":"2024-05-30T15:54:18.648860+0000","last_unstale":"2024-05-30T17:13:41.604880+0000","last_undegraded":"2024-05-30T17:13:41.604880+0000","last_fullsized":"2024-05-30T17:13:41.604880+0000","mapping_epoch":24,"log_start":"265'1500","ondisk_log_start":"265'1500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":1500,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6132,"num_read_kb":5297,"num_write":2138,"num_write_kb":912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"265'221","reported_seq":1085,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.565561+0000","last_change":"2024-05-30T15:54:16.687260+0000","last_active":"2024-05-30T17:13:44.565561+0000","last_peered":"2024-05-30T17:13:44.565561+0000","last_clean":"2024-05-30T17:13:44.565561+0000","last_became_active":"2024-05-30T15:54:16.686847+0000","last_became_peered":"2024-05-30T15:54:16.686847+0000","last_unstale":"2024-05-30T17:13:44.565561+0000","last_undegraded":"2024-05-30T17:13:44.565561+0000","last_fullsized":"2024-05-30T17:13:44.565561+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2728232,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":925,"num_read_kb":24537,"num_write":1233,"num_write_kb":54861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"265'1164","reported_seq":3005,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.066233+0000","last_change":"2024-05-30T15:54:18.666133+0000","last_active":"2024-05-30T17:13:44.066233+0000","last_peered":"2024-05-30T17:13:44.066233+0000","last_clean":"2024-05-30T17:13:44.066233+0000","last_became_active":"2024-05-30T15:54:18.665302+0000","last_became_peered":"2024-05-30T15:54:18.665302+0000","last_unstale":"2024-05-30T17:13:44.066233+0000","last_undegraded":"2024-05-30T17:13:44.066233+0000","last_fullsized":"2024-05-30T17:13:44.066233+0000","mapping_epoch":24,"log_start":"265'1100","ondisk_log_start":"265'1100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":1100,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5292,"num_read_kb":4714,"num_write":1593,"num_write_kb":642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"265'197","reported_seq":920,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.565552+0000","last_change":"2024-05-30T15:54:16.669114+0000","last_active":"2024-05-30T17:13:44.565552+0000","last_peered":"2024-05-30T17:13:44.565552+0000","last_clean":"2024-05-30T17:13:44.565552+0000","last_became_active":"2024-05-30T15:54:16.668438+0000","last_became_peered":"2024-05-30T15:54:16.668438+0000","last_unstale":"2024-05-30T17:13:44.565552+0000","last_undegraded":"2024-05-30T17:13:44.565552+0000","last_fullsized":"2024-05-30T17:13:44.565552+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":100,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":918,"num_read_kb":8276,"num_write":1173,"num_write_kb":35310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"265'768","reported_seq":2283,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.708088+0000","last_change":"2024-05-30T15:54:18.651494+0000","last_active":"2024-05-30T17:13:20.708088+0000","last_peered":"2024-05-30T17:13:20.708088+0000","last_clean":"2024-05-30T17:13:20.708088+0000","last_became_active":"2024-05-30T15:54:18.651197+0000","last_became_peered":"2024-05-30T15:54:18.651197+0000","last_unstale":"2024-05-30T17:13:20.708088+0000","last_undegraded":"2024-05-30T17:13:20.708088+0000","last_fullsized":"2024-05-30T17:13:20.708088+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":700,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3544,"num_read_kb":3171,"num_write":1051,"num_write_kb":404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"265'190","reported_seq":844,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.565518+0000","last_change":"2024-05-30T15:54:16.677382+0000","last_active":"2024-05-30T17:13:43.565518+0000","last_peered":"2024-05-30T17:13:43.565518+0000","last_clean":"2024-05-30T17:13:43.565518+0000","last_became_active":"2024-05-30T15:54:16.676703+0000","last_became_peered":"2024-05-30T15:54:16.676703+0000","last_unstale":"2024-05-30T17:13:43.565518+0000","last_undegraded":"2024-05-30T17:13:43.565518+0000","last_fullsized":"2024-05-30T17:13:43.565518+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":100,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":366,"num_read_kb":73313,"num_write":1062,"num_write_kb":47694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"265'372","reported_seq":1825,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:09.137986+0000","last_change":"2024-05-30T15:54:18.650530+0000","last_active":"2024-05-30T17:13:09.137986+0000","last_peered":"2024-05-30T17:13:09.137986+0000","last_clean":"2024-05-30T17:13:09.137986+0000","last_became_active":"2024-05-30T15:54:18.650255+0000","last_became_peered":"2024-05-30T15:54:18.650255+0000","last_unstale":"2024-05-30T17:13:09.137986+0000","last_undegraded":"2024-05-30T17:13:09.137986+0000","last_fullsized":"2024-05-30T17:13:09.137986+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":300,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1969,"num_read_kb":1871,"num_write":435,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"265'203","reported_seq":856,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.074027+0000","last_change":"2024-05-30T15:54:16.665743+0000","last_active":"2024-05-30T17:13:43.074027+0000","last_peered":"2024-05-30T17:13:43.074027+0000","last_clean":"2024-05-30T17:13:43.074027+0000","last_became_active":"2024-05-30T15:54:16.665344+0000","last_became_peered":"2024-05-30T15:54:16.665344+0000","last_unstale":"2024-05-30T17:13:43.074027+0000","last_undegraded":"2024-05-30T17:13:43.074027+0000","last_fullsized":"2024-05-30T17:13:43.074027+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":100,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6847332,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":477,"num_read_kb":7750,"num_write":1077,"num_write_kb":36544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"265'375","reported_seq":1801,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:15.255341+0000","last_change":"2024-05-30T15:54:18.645251+0000","last_active":"2024-05-30T17:12:15.255341+0000","last_peered":"2024-05-30T17:12:15.255341+0000","last_clean":"2024-05-30T17:12:15.255341+0000","last_became_active":"2024-05-30T15:54:18.643825+0000","last_became_peered":"2024-05-30T15:54:18.643825+0000","last_unstale":"2024-05-30T17:12:15.255341+0000","last_undegraded":"2024-05-30T17:12:15.255341+0000","last_fullsized":"2024-05-30T17:12:15.255341+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":300,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1936,"num_read_kb":1833,"num_write":434,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"265'238","reported_seq":911,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.647132+0000","last_change":"2024-05-30T15:54:16.638560+0000","last_active":"2024-05-30T17:13:41.647132+0000","last_peered":"2024-05-30T17:13:41.647132+0000","last_clean":"2024-05-30T17:13:41.647132+0000","last_became_active":"2024-05-30T15:54:16.637928+0000","last_became_peered":"2024-05-30T15:54:16.637928+0000","last_unstale":"2024-05-30T17:13:41.647132+0000","last_undegraded":"2024-05-30T17:13:41.647132+0000","last_fullsized":"2024-05-30T17:13:41.647132+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":200,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079922,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":526,"num_read_kb":65823,"num_write":1181,"num_write_kb":32044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"265'180","reported_seq":713,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.837232+0000","last_change":"2024-05-30T17:13:47.837308+0000","last_active":"2024-05-30T17:13:47.837232+0000","last_peered":"2024-05-30T17:13:47.837232+0000","last_clean":"2024-05-30T17:13:47.837232+0000","last_became_active":"2024-05-30T15:54:26.700957+0000","last_became_peered":"2024-05-30T15:54:26.700957+0000","last_unstale":"2024-05-30T17:13:47.837232+0000","last_undegraded":"2024-05-30T17:13:47.837232+0000","last_fullsized":"2024-05-30T17:13:47.837232+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'180","last_scrub_stamp":"2024-05-30T17:13:47.837040+0000","last_deep_scrub":"265'180","last_deep_scrub_stamp":"2024-05-30T17:13:47.837040+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.837040+0000","objects_scrubbed":5,"log_size":80,"log_dups_size":100,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:49:37.253715+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":270,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":132,"num_read_kb":126,"num_write":386,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"265'494","reported_seq":2023,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.074230+0000","last_change":"2024-05-30T15:54:18.657002+0000","last_active":"2024-05-30T17:13:43.074230+0000","last_peered":"2024-05-30T17:13:43.074230+0000","last_clean":"2024-05-30T17:13:43.074230+0000","last_became_active":"2024-05-30T15:54:18.652712+0000","last_became_peered":"2024-05-30T15:54:18.652712+0000","last_unstale":"2024-05-30T17:13:43.074230+0000","last_undegraded":"2024-05-30T17:13:43.074230+0000","last_fullsized":"2024-05-30T17:13:43.074230+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":400,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2579,"num_read_kb":2377,"num_write":668,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"265'190","reported_seq":803,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.106668+0000","last_change":"2024-05-30T17:13:51.106756+0000","last_active":"2024-05-30T17:13:51.106668+0000","last_peered":"2024-05-30T17:13:51.106668+0000","last_clean":"2024-05-30T17:13:51.106668+0000","last_became_active":"2024-05-30T15:55:54.563874+0000","last_became_peered":"2024-05-30T15:55:54.563874+0000","last_unstale":"2024-05-30T17:13:51.106668+0000","last_undegraded":"2024-05-30T17:13:51.106668+0000","last_fullsized":"2024-05-30T17:13:51.106668+0000","mapping_epoch":37,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'190","last_scrub_stamp":"2024-05-30T17:13:51.106465+0000","last_deep_scrub":"265'190","last_deep_scrub_stamp":"2024-05-30T17:13:51.106465+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.106465+0000","objects_scrubbed":2,"log_size":90,"log_dups_size":100,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:21:06.302905+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079905,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":470,"num_read_kb":1589,"num_write":1089,"num_write_kb":27979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":507,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:42.157906+0000","last_change":"2024-05-30T15:54:20.659788+0000","last_active":"2024-05-30T17:11:42.157906+0000","last_peered":"2024-05-30T17:11:42.157906+0000","last_clean":"2024-05-30T17:11:42.157906+0000","last_became_active":"2024-05-30T15:54:20.658937+0000","last_became_peered":"2024-05-30T15:54:20.658937+0000","last_unstale":"2024-05-30T17:11:42.157906+0000","last_undegraded":"2024-05-30T17:11:42.157906+0000","last_fullsized":"2024-05-30T17:11:42.157906+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"265'202","reported_seq":829,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:37.989942+0000","last_change":"2024-05-30T15:54:22.668221+0000","last_active":"2024-05-30T17:13:37.989942+0000","last_peered":"2024-05-30T17:13:37.989942+0000","last_clean":"2024-05-30T17:13:37.989942+0000","last_became_active":"2024-05-30T15:54:22.667496+0000","last_became_peered":"2024-05-30T15:54:22.667496+0000","last_unstale":"2024-05-30T17:13:37.989942+0000","last_undegraded":"2024-05-30T17:13:37.989942+0000","last_fullsized":"2024-05-30T17:13:37.989942+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6140,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":285,"num_read_kb":355,"num_write":168,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":499,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:46.575823+0000","last_change":"2024-05-30T15:54:24.675232+0000","last_active":"2024-05-30T17:11:46.575823+0000","last_peered":"2024-05-30T17:11:46.575823+0000","last_clean":"2024-05-30T17:11:46.575823+0000","last_became_active":"2024-05-30T15:54:24.674491+0000","last_became_peered":"2024-05-30T15:54:24.674491+0000","last_unstale":"2024-05-30T17:11:46.575823+0000","last_undegraded":"2024-05-30T17:11:46.575823+0000","last_fullsized":"2024-05-30T17:11:46.575823+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"265'199","reported_seq":783,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.234801+0000","last_change":"2024-05-30T17:13:53.234943+0000","last_active":"2024-05-30T17:13:53.234801+0000","last_peered":"2024-05-30T17:13:53.234801+0000","last_clean":"2024-05-30T17:13:53.234801+0000","last_became_active":"2024-05-30T15:54:26.686430+0000","last_became_peered":"2024-05-30T15:54:26.686430+0000","last_unstale":"2024-05-30T17:13:53.234801+0000","last_undegraded":"2024-05-30T17:13:53.234801+0000","last_fullsized":"2024-05-30T17:13:53.234801+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'199","last_scrub_stamp":"2024-05-30T17:13:53.234506+0000","last_deep_scrub":"265'199","last_deep_scrub_stamp":"2024-05-30T17:13:53.234506+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.234506+0000","objects_scrubbed":5,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:07:57.530934+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":824,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":238,"num_read_kb":223,"num_write":274,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"265'510","reported_seq":2161,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.759127+0000","last_change":"2024-05-30T15:54:18.662470+0000","last_active":"2024-05-30T17:13:18.759127+0000","last_peered":"2024-05-30T17:13:18.759127+0000","last_clean":"2024-05-30T17:13:18.759127+0000","last_became_active":"2024-05-30T15:54:18.661762+0000","last_became_peered":"2024-05-30T15:54:18.661762+0000","last_unstale":"2024-05-30T17:13:18.759127+0000","last_undegraded":"2024-05-30T17:13:18.759127+0000","last_fullsized":"2024-05-30T17:13:18.759127+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":400,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2657,"num_read_kb":2481,"num_write":627,"num_write_kb":155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"265'210","reported_seq":867,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.427287+0000","last_change":"2024-05-30T15:54:16.639898+0000","last_active":"2024-05-30T17:13:38.427287+0000","last_peered":"2024-05-30T17:13:38.427287+0000","last_clean":"2024-05-30T17:13:38.427287+0000","last_became_active":"2024-05-30T15:54:16.639284+0000","last_became_peered":"2024-05-30T15:54:16.639284+0000","last_unstale":"2024-05-30T17:13:38.427287+0000","last_undegraded":"2024-05-30T17:13:38.427287+0000","last_fullsized":"2024-05-30T17:13:38.427287+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":100,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262178,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":445,"num_read_kb":15653,"num_write":1159,"num_write_kb":64581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":511,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.074361+0000","last_change":"2024-05-30T15:54:20.666014+0000","last_active":"2024-05-30T17:13:43.074361+0000","last_peered":"2024-05-30T17:13:43.074361+0000","last_clean":"2024-05-30T17:13:43.074361+0000","last_became_active":"2024-05-30T15:54:20.665488+0000","last_became_peered":"2024-05-30T15:54:20.665488+0000","last_unstale":"2024-05-30T17:13:43.074361+0000","last_undegraded":"2024-05-30T17:13:43.074361+0000","last_fullsized":"2024-05-30T17:13:43.074361+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"265'105","reported_seq":665,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.970161+0000","last_change":"2024-05-30T17:13:59.970319+0000","last_active":"2024-05-30T17:13:59.970161+0000","last_peered":"2024-05-30T17:13:59.970161+0000","last_clean":"2024-05-30T17:13:59.970161+0000","last_became_active":"2024-05-30T15:54:22.666057+0000","last_became_peered":"2024-05-30T15:54:22.666057+0000","last_unstale":"2024-05-30T17:13:59.970161+0000","last_undegraded":"2024-05-30T17:13:59.970161+0000","last_fullsized":"2024-05-30T17:13:59.970161+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'105","last_scrub_stamp":"2024-05-30T17:13:59.969943+0000","last_deep_scrub":"265'105","last_deep_scrub_stamp":"2024-05-30T17:13:59.969943+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.969943+0000","objects_scrubbed":3,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:45:56.754937+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":113,"num_write":76,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":516,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:52.906699+0000","last_change":"2024-05-30T17:13:52.906758+0000","last_active":"2024-05-30T17:13:52.906699+0000","last_peered":"2024-05-30T17:13:52.906699+0000","last_clean":"2024-05-30T17:13:52.906699+0000","last_became_active":"2024-05-30T15:54:24.686595+0000","last_became_peered":"2024-05-30T15:54:24.686595+0000","last_unstale":"2024-05-30T17:13:52.906699+0000","last_undegraded":"2024-05-30T17:13:52.906699+0000","last_fullsized":"2024-05-30T17:13:52.906699+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:52.906486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:52.906486+0000","last_clean_scrub_stamp":"2024-05-30T17:13:52.906486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:22:22.141892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"265'103","reported_seq":625,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.717277+0000","last_change":"2024-05-30T17:13:53.717354+0000","last_active":"2024-05-30T17:13:53.717277+0000","last_peered":"2024-05-30T17:13:53.717277+0000","last_clean":"2024-05-30T17:13:53.717277+0000","last_became_active":"2024-05-30T15:54:26.688376+0000","last_became_peered":"2024-05-30T15:54:26.688376+0000","last_unstale":"2024-05-30T17:13:53.717277+0000","last_undegraded":"2024-05-30T17:13:53.717277+0000","last_fullsized":"2024-05-30T17:13:53.717277+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'103","last_scrub_stamp":"2024-05-30T17:13:53.717082+0000","last_deep_scrub":"265'103","last_deep_scrub_stamp":"2024-05-30T17:13:53.717082+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.717082+0000","objects_scrubbed":4,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:37:15.362960+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":271,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":103,"num_read_kb":102,"num_write":235,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"265'462","reported_seq":2012,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.706143+0000","last_change":"2024-05-30T15:54:18.658772+0000","last_active":"2024-05-30T17:13:20.706143+0000","last_peered":"2024-05-30T17:13:20.706143+0000","last_clean":"2024-05-30T17:13:20.706143+0000","last_became_active":"2024-05-30T15:54:18.657540+0000","last_became_peered":"2024-05-30T15:54:18.657540+0000","last_unstale":"2024-05-30T17:13:20.706143+0000","last_undegraded":"2024-05-30T17:13:20.706143+0000","last_fullsized":"2024-05-30T17:13:20.706143+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":400,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2554,"num_read_kb":2362,"num_write":592,"num_write_kb":196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"265'216","reported_seq":878,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.414145+0000","last_change":"2024-05-30T15:54:16.679183+0000","last_active":"2024-05-30T17:13:38.414145+0000","last_peered":"2024-05-30T17:13:38.414145+0000","last_clean":"2024-05-30T17:13:38.414145+0000","last_became_active":"2024-05-30T15:54:16.678659+0000","last_became_peered":"2024-05-30T15:54:16.678659+0000","last_unstale":"2024-05-30T17:13:38.414145+0000","last_undegraded":"2024-05-30T17:13:38.414145+0000","last_fullsized":"2024-05-30T17:13:38.414145+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":200,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291507,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":524,"num_read_kb":29194,"num_write":1289,"num_write_kb":17953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":509,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:41.592348+0000","last_change":"2024-05-30T15:54:20.654060+0000","last_active":"2024-05-30T17:11:41.592348+0000","last_peered":"2024-05-30T17:11:41.592348+0000","last_clean":"2024-05-30T17:11:41.592348+0000","last_became_active":"2024-05-30T15:54:20.653316+0000","last_became_peered":"2024-05-30T15:54:20.653316+0000","last_unstale":"2024-05-30T17:11:41.592348+0000","last_undegraded":"2024-05-30T17:11:41.592348+0000","last_fullsized":"2024-05-30T17:11:41.592348+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"265'155","reported_seq":726,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.161794+0000","last_change":"2024-05-30T17:13:48.161878+0000","last_active":"2024-05-30T17:13:48.161794+0000","last_peered":"2024-05-30T17:13:48.161794+0000","last_clean":"2024-05-30T17:13:48.161794+0000","last_became_active":"2024-05-30T15:54:22.664353+0000","last_became_peered":"2024-05-30T15:54:22.664353+0000","last_unstale":"2024-05-30T17:13:48.161794+0000","last_undegraded":"2024-05-30T17:13:48.161794+0000","last_fullsized":"2024-05-30T17:13:48.161794+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'155","last_scrub_stamp":"2024-05-30T17:13:48.161621+0000","last_deep_scrub":"265'155","last_deep_scrub_stamp":"2024-05-30T17:13:48.161621+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.161621+0000","objects_scrubbed":6,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:30:12.200451+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":173,"num_read_kb":165,"num_write":120,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"265'532","reported_seq":2483,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.450074+0000","last_change":"2024-05-30T15:54:24.682250+0000","last_active":"2024-05-30T17:13:38.450074+0000","last_peered":"2024-05-30T17:13:38.450074+0000","last_clean":"2024-05-30T17:13:38.450074+0000","last_became_active":"2024-05-30T15:54:24.681503+0000","last_became_peered":"2024-05-30T15:54:24.681503+0000","last_unstale":"2024-05-30T17:13:38.450074+0000","last_undegraded":"2024-05-30T17:13:38.450074+0000","last_fullsized":"2024-05-30T17:13:38.450074+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":500,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"265'237","reported_seq":1087,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.884623+0000","last_change":"2024-05-30T17:13:51.884697+0000","last_active":"2024-05-30T17:13:51.884623+0000","last_peered":"2024-05-30T17:13:51.884623+0000","last_clean":"2024-05-30T17:13:51.884623+0000","last_became_active":"2024-05-30T15:54:26.670257+0000","last_became_peered":"2024-05-30T15:54:26.670257+0000","last_unstale":"2024-05-30T17:13:51.884623+0000","last_undegraded":"2024-05-30T17:13:51.884623+0000","last_fullsized":"2024-05-30T17:13:51.884623+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'237","last_scrub_stamp":"2024-05-30T17:13:51.884401+0000","last_deep_scrub":"265'237","last_deep_scrub_stamp":"2024-05-30T17:13:51.884401+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.884401+0000","objects_scrubbed":4,"log_size":37,"log_dups_size":200,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:24:37.651399+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":661,"num_read_kb":640,"num_write":540,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":234,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"265'406","reported_seq":1989,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.758594+0000","last_change":"2024-05-30T15:54:18.660971+0000","last_active":"2024-05-30T17:13:18.758594+0000","last_peered":"2024-05-30T17:13:18.758594+0000","last_clean":"2024-05-30T17:13:18.758594+0000","last_became_active":"2024-05-30T15:54:18.660656+0000","last_became_peered":"2024-05-30T15:54:18.660656+0000","last_unstale":"2024-05-30T17:13:18.758594+0000","last_undegraded":"2024-05-30T17:13:18.758594+0000","last_fullsized":"2024-05-30T17:13:18.758594+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":300,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2368,"num_read_kb":2240,"num_write":473,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"265'191","reported_seq":821,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.333050+0000","last_change":"2024-05-30T15:54:16.668687+0000","last_active":"2024-05-30T17:13:38.333050+0000","last_peered":"2024-05-30T17:13:38.333050+0000","last_clean":"2024-05-30T17:13:38.333050+0000","last_became_active":"2024-05-30T15:54:16.668083+0000","last_became_peered":"2024-05-30T15:54:16.668083+0000","last_unstale":"2024-05-30T17:13:38.333050+0000","last_undegraded":"2024-05-30T17:13:38.333050+0000","last_fullsized":"2024-05-30T17:13:38.333050+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":100,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5263487,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":406,"num_read_kb":13963,"num_write":1054,"num_write_kb":66137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":507,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:42.157988+0000","last_change":"2024-05-30T15:54:20.659591+0000","last_active":"2024-05-30T17:11:42.157988+0000","last_peered":"2024-05-30T17:11:42.157988+0000","last_clean":"2024-05-30T17:11:42.157988+0000","last_became_active":"2024-05-30T15:54:20.658870+0000","last_became_peered":"2024-05-30T15:54:20.658870+0000","last_unstale":"2024-05-30T17:11:42.157988+0000","last_undegraded":"2024-05-30T17:11:42.157988+0000","last_fullsized":"2024-05-30T17:11:42.157988+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"265'142","reported_seq":730,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:37.885533+0000","last_change":"2024-05-30T15:54:22.851241+0000","last_active":"2024-05-30T17:13:37.885533+0000","last_peered":"2024-05-30T17:13:37.885533+0000","last_clean":"2024-05-30T17:13:37.885533+0000","last_became_active":"2024-05-30T15:54:22.849978+0000","last_became_peered":"2024-05-30T15:54:22.849978+0000","last_unstale":"2024-05-30T17:13:37.885533+0000","last_undegraded":"2024-05-30T17:13:37.885533+0000","last_fullsized":"2024-05-30T17:13:37.885533+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":100,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8153,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":189,"num_read_kb":241,"num_write":102,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"265'549","reported_seq":2573,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.456888+0000","last_change":"2024-05-30T15:54:24.688312+0000","last_active":"2024-05-30T17:13:38.456888+0000","last_peered":"2024-05-30T17:13:38.456888+0000","last_clean":"2024-05-30T17:13:38.456888+0000","last_became_active":"2024-05-30T15:54:24.686546+0000","last_became_peered":"2024-05-30T15:54:24.686546+0000","last_unstale":"2024-05-30T17:13:38.456888+0000","last_undegraded":"2024-05-30T17:13:38.456888+0000","last_fullsized":"2024-05-30T17:13:38.456888+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":500,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"265'895","reported_seq":2511,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.758636+0000","last_change":"2024-05-30T15:54:18.658824+0000","last_active":"2024-05-30T17:13:18.758636+0000","last_peered":"2024-05-30T17:13:18.758636+0000","last_clean":"2024-05-30T17:13:18.758636+0000","last_became_active":"2024-05-30T15:54:18.658124+0000","last_became_peered":"2024-05-30T15:54:18.658124+0000","last_unstale":"2024-05-30T17:13:18.758636+0000","last_undegraded":"2024-05-30T17:13:18.758636+0000","last_fullsized":"2024-05-30T17:13:18.758636+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":800,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4127,"num_read_kb":3657,"num_write":1246,"num_write_kb":491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"265'208","reported_seq":889,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.307115+0000","last_change":"2024-05-30T15:54:16.659141+0000","last_active":"2024-05-30T17:13:38.307115+0000","last_peered":"2024-05-30T17:13:38.307115+0000","last_clean":"2024-05-30T17:13:38.307115+0000","last_became_active":"2024-05-30T15:54:16.658279+0000","last_became_peered":"2024-05-30T15:54:16.658279+0000","last_unstale":"2024-05-30T17:13:38.307115+0000","last_undegraded":"2024-05-30T17:13:38.307115+0000","last_fullsized":"2024-05-30T17:13:38.307115+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":100,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1254547,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":480,"num_read_kb":111643,"num_write":1134,"num_write_kb":26225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":511,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:21.584081+0000","last_change":"2024-05-30T15:54:20.662333+0000","last_active":"2024-05-30T17:12:21.584081+0000","last_peered":"2024-05-30T17:12:21.584081+0000","last_clean":"2024-05-30T17:12:21.584081+0000","last_became_active":"2024-05-30T15:54:20.661583+0000","last_became_peered":"2024-05-30T15:54:20.661583+0000","last_unstale":"2024-05-30T17:12:21.584081+0000","last_undegraded":"2024-05-30T17:12:21.584081+0000","last_fullsized":"2024-05-30T17:12:21.584081+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"265'265","reported_seq":921,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.257602+0000","last_change":"2024-05-30T17:13:58.257702+0000","last_active":"2024-05-30T17:13:58.257602+0000","last_peered":"2024-05-30T17:13:58.257602+0000","last_clean":"2024-05-30T17:13:58.257602+0000","last_became_active":"2024-05-30T15:54:22.849126+0000","last_became_peered":"2024-05-30T15:54:22.849126+0000","last_unstale":"2024-05-30T17:13:58.257602+0000","last_undegraded":"2024-05-30T17:13:58.257602+0000","last_fullsized":"2024-05-30T17:13:58.257602+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'265","last_scrub_stamp":"2024-05-30T17:13:58.257369+0000","last_deep_scrub":"265'265","last_deep_scrub_stamp":"2024-05-30T17:13:58.257369+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.257369+0000","objects_scrubbed":10,"log_size":65,"log_dups_size":200,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:15:48.980020+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6773,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":352,"num_read_kb":376,"num_write":210,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":503,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:11.815696+0000","last_change":"2024-05-30T15:54:24.674322+0000","last_active":"2024-05-30T17:13:11.815696+0000","last_peered":"2024-05-30T17:13:11.815696+0000","last_clean":"2024-05-30T17:13:11.815696+0000","last_became_active":"2024-05-30T15:54:24.673506+0000","last_became_peered":"2024-05-30T15:54:24.673506+0000","last_unstale":"2024-05-30T17:13:11.815696+0000","last_undegraded":"2024-05-30T17:13:11.815696+0000","last_fullsized":"2024-05-30T17:13:11.815696+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"265'546","reported_seq":1385,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.226322+0000","last_change":"2024-05-30T17:13:54.226466+0000","last_active":"2024-05-30T17:13:54.226322+0000","last_peered":"2024-05-30T17:13:54.226322+0000","last_clean":"2024-05-30T17:13:54.226322+0000","last_became_active":"2024-05-30T15:54:26.686389+0000","last_became_peered":"2024-05-30T15:54:26.686389+0000","last_unstale":"2024-05-30T17:13:54.226322+0000","last_undegraded":"2024-05-30T17:13:54.226322+0000","last_fullsized":"2024-05-30T17:13:54.226322+0000","mapping_epoch":32,"log_start":"265'500","ondisk_log_start":"265'500","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'546","last_scrub_stamp":"2024-05-30T17:13:54.226029+0000","last_deep_scrub":"265'546","last_deep_scrub_stamp":"2024-05-30T17:13:54.226029+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.226029+0000","objects_scrubbed":3,"log_size":46,"log_dups_size":500,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:20:00.749166+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1297,"num_read_kb":1288,"num_write":1137,"num_write_kb":385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":381,"num_omap_keys":2,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"265'1398","reported_seq":3106,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.763641+0000","last_change":"2024-05-30T15:54:18.640378+0000","last_active":"2024-05-30T17:13:18.763641+0000","last_peered":"2024-05-30T17:13:18.763641+0000","last_clean":"2024-05-30T17:13:18.763641+0000","last_became_active":"2024-05-30T15:54:18.639667+0000","last_became_peered":"2024-05-30T15:54:18.639667+0000","last_unstale":"2024-05-30T17:13:18.763641+0000","last_undegraded":"2024-05-30T17:13:18.763641+0000","last_fullsized":"2024-05-30T17:13:18.763641+0000","mapping_epoch":24,"log_start":"265'1300","ondisk_log_start":"265'1300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":1300,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5970,"num_read_kb":5130,"num_write":2044,"num_write_kb":884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"265'213","reported_seq":868,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.371059+0000","last_change":"2024-05-30T15:54:16.661968+0000","last_active":"2024-05-30T17:13:38.371059+0000","last_peered":"2024-05-30T17:13:38.371059+0000","last_clean":"2024-05-30T17:13:38.371059+0000","last_became_active":"2024-05-30T15:54:16.659427+0000","last_became_peered":"2024-05-30T15:54:16.659427+0000","last_unstale":"2024-05-30T17:13:38.371059+0000","last_undegraded":"2024-05-30T17:13:38.371059+0000","last_fullsized":"2024-05-30T17:13:38.371059+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":200,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":467,"num_read_kb":34119,"num_write":1157,"num_write_kb":42367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":522,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.231419+0000","last_change":"2024-05-30T17:13:58.231474+0000","last_active":"2024-05-30T17:13:58.231419+0000","last_peered":"2024-05-30T17:13:58.231419+0000","last_clean":"2024-05-30T17:13:58.231419+0000","last_became_active":"2024-05-30T15:54:20.647565+0000","last_became_peered":"2024-05-30T15:54:20.647565+0000","last_unstale":"2024-05-30T17:13:58.231419+0000","last_undegraded":"2024-05-30T17:13:58.231419+0000","last_fullsized":"2024-05-30T17:13:58.231419+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:58.231223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:58.231223+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.231223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:53:45.579167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"265'146","reported_seq":739,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.312031+0000","last_change":"2024-05-30T15:54:22.671276+0000","last_active":"2024-05-30T17:13:38.312031+0000","last_peered":"2024-05-30T17:13:38.312031+0000","last_clean":"2024-05-30T17:13:38.312031+0000","last_became_active":"2024-05-30T15:54:22.670064+0000","last_became_peered":"2024-05-30T15:54:22.670064+0000","last_unstale":"2024-05-30T17:13:38.312031+0000","last_undegraded":"2024-05-30T17:13:38.312031+0000","last_fullsized":"2024-05-30T17:13:38.312031+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12270,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":197,"num_read_kb":284,"num_write":108,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":505,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:46.607388+0000","last_change":"2024-05-30T15:54:24.688941+0000","last_active":"2024-05-30T17:12:46.607388+0000","last_peered":"2024-05-30T17:12:46.607388+0000","last_clean":"2024-05-30T17:12:46.607388+0000","last_became_active":"2024-05-30T15:54:24.688046+0000","last_became_peered":"2024-05-30T15:54:24.688046+0000","last_unstale":"2024-05-30T17:12:46.607388+0000","last_undegraded":"2024-05-30T17:12:46.607388+0000","last_fullsized":"2024-05-30T17:12:46.607388+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"265'2369","reported_seq":4191,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.566212+0000","last_change":"2024-05-30T15:54:26.692267+0000","last_active":"2024-05-30T17:13:44.566212+0000","last_peered":"2024-05-30T17:13:44.566212+0000","last_clean":"2024-05-30T17:13:44.566212+0000","last_became_active":"2024-05-30T15:54:26.691574+0000","last_became_peered":"2024-05-30T15:54:26.691574+0000","last_unstale":"2024-05-30T17:13:44.566212+0000","last_undegraded":"2024-05-30T17:13:44.566212+0000","last_fullsized":"2024-05-30T17:13:44.566212+0000","mapping_epoch":32,"log_start":"265'2300","ondisk_log_start":"265'2300","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":2300,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5943,"num_read_kb":5935,"num_write":4785,"num_write_kb":1767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"265'371","reported_seq":1793,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:41.576912+0000","last_change":"2024-05-30T15:54:18.670009+0000","last_active":"2024-05-30T17:12:41.576912+0000","last_peered":"2024-05-30T17:12:41.576912+0000","last_clean":"2024-05-30T17:12:41.576912+0000","last_became_active":"2024-05-30T15:54:18.668963+0000","last_became_peered":"2024-05-30T15:54:18.668963+0000","last_unstale":"2024-05-30T17:12:41.576912+0000","last_undegraded":"2024-05-30T17:12:41.576912+0000","last_fullsized":"2024-05-30T17:12:41.576912+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":300,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2011,"num_read_kb":1890,"num_write":455,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"265'240","reported_seq":881,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.426739+0000","last_change":"2024-05-30T15:54:16.650507+0000","last_active":"2024-05-30T17:13:38.426739+0000","last_peered":"2024-05-30T17:13:38.426739+0000","last_clean":"2024-05-30T17:13:38.426739+0000","last_became_active":"2024-05-30T15:54:16.649605+0000","last_became_peered":"2024-05-30T15:54:16.649605+0000","last_unstale":"2024-05-30T17:13:38.426739+0000","last_undegraded":"2024-05-30T17:13:38.426739+0000","last_fullsized":"2024-05-30T17:13:38.426739+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":200,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242911,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":469,"num_read_kb":32402,"num_write":1238,"num_write_kb":40071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":521,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.170085+0000","last_change":"2024-05-30T17:13:56.170146+0000","last_active":"2024-05-30T17:13:56.170085+0000","last_peered":"2024-05-30T17:13:56.170085+0000","last_clean":"2024-05-30T17:13:56.170085+0000","last_became_active":"2024-05-30T15:54:20.651369+0000","last_became_peered":"2024-05-30T15:54:20.651369+0000","last_unstale":"2024-05-30T17:13:56.170085+0000","last_undegraded":"2024-05-30T17:13:56.170085+0000","last_fullsized":"2024-05-30T17:13:56.170085+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:56.169880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:56.169880+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.169880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:32:45.358384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"265'191","reported_seq":807,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.101941+0000","last_change":"2024-05-30T15:54:22.666930+0000","last_active":"2024-05-30T17:13:38.101941+0000","last_peered":"2024-05-30T17:13:38.101941+0000","last_clean":"2024-05-30T17:13:38.101941+0000","last_became_active":"2024-05-30T15:54:22.664874+0000","last_became_peered":"2024-05-30T15:54:22.664874+0000","last_unstale":"2024-05-30T17:13:38.101941+0000","last_undegraded":"2024-05-30T17:13:38.101941+0000","last_fullsized":"2024-05-30T17:13:38.101941+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":100,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6309,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":259,"num_read_kb":308,"num_write":148,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"265'549","reported_seq":2483,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.458419+0000","last_change":"2024-05-30T15:54:24.674977+0000","last_active":"2024-05-30T17:13:38.458419+0000","last_peered":"2024-05-30T17:13:38.458419+0000","last_clean":"2024-05-30T17:13:38.458419+0000","last_became_active":"2024-05-30T15:54:24.670141+0000","last_became_peered":"2024-05-30T15:54:24.670141+0000","last_unstale":"2024-05-30T17:13:38.458419+0000","last_undegraded":"2024-05-30T17:13:38.458419+0000","last_fullsized":"2024-05-30T17:13:38.458419+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":500,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"265'107","reported_seq":620,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:52.677135+0000","last_change":"2024-05-30T17:13:52.677276+0000","last_active":"2024-05-30T17:13:52.677135+0000","last_peered":"2024-05-30T17:13:52.677135+0000","last_clean":"2024-05-30T17:13:52.677135+0000","last_became_active":"2024-05-30T15:54:26.689987+0000","last_became_peered":"2024-05-30T15:54:26.689987+0000","last_unstale":"2024-05-30T17:13:52.677135+0000","last_undegraded":"2024-05-30T17:13:52.677135+0000","last_fullsized":"2024-05-30T17:13:52.677135+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'107","last_scrub_stamp":"2024-05-30T17:13:52.676848+0000","last_deep_scrub":"265'107","last_deep_scrub_stamp":"2024-05-30T17:13:52.676848+0000","last_clean_scrub_stamp":"2024-05-30T17:13:52.676848+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:48:23.065085+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":80,"num_write":254,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"265'1331","reported_seq":2967,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.757798+0000","last_change":"2024-05-30T15:54:18.647985+0000","last_active":"2024-05-30T17:13:18.757798+0000","last_peered":"2024-05-30T17:13:18.757798+0000","last_clean":"2024-05-30T17:13:18.757798+0000","last_became_active":"2024-05-30T15:54:18.647634+0000","last_became_peered":"2024-05-30T15:54:18.647634+0000","last_unstale":"2024-05-30T17:13:18.757798+0000","last_undegraded":"2024-05-30T17:13:18.757798+0000","last_fullsized":"2024-05-30T17:13:18.757798+0000","mapping_epoch":24,"log_start":"265'1300","ondisk_log_start":"265'1300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":1300,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5717,"num_read_kb":4967,"num_write":1912,"num_write_kb":881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"265'206","reported_seq":822,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.427898+0000","last_change":"2024-05-30T15:54:16.668968+0000","last_active":"2024-05-30T17:13:38.427898+0000","last_peered":"2024-05-30T17:13:38.427898+0000","last_clean":"2024-05-30T17:13:38.427898+0000","last_became_active":"2024-05-30T15:54:16.668336+0000","last_became_peered":"2024-05-30T15:54:16.668336+0000","last_unstale":"2024-05-30T17:13:38.427898+0000","last_undegraded":"2024-05-30T17:13:38.427898+0000","last_fullsized":"2024-05-30T17:13:38.427898+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":100,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":403,"num_read_kb":1534,"num_write":1058,"num_write_kb":21253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":522,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.159988+0000","last_change":"2024-05-30T17:13:54.160046+0000","last_active":"2024-05-30T17:13:54.159988+0000","last_peered":"2024-05-30T17:13:54.159988+0000","last_clean":"2024-05-30T17:13:54.159988+0000","last_became_active":"2024-05-30T15:54:20.655573+0000","last_became_peered":"2024-05-30T15:54:20.655573+0000","last_unstale":"2024-05-30T17:13:54.159988+0000","last_undegraded":"2024-05-30T17:13:54.159988+0000","last_fullsized":"2024-05-30T17:13:54.159988+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:54.159789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:54.159789+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.159789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:27:43.050145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"265'207","reported_seq":866,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.361999+0000","last_change":"2024-05-30T15:54:22.659510+0000","last_active":"2024-05-30T17:13:38.361999+0000","last_peered":"2024-05-30T17:13:38.361999+0000","last_clean":"2024-05-30T17:13:38.361999+0000","last_became_active":"2024-05-30T15:54:22.658700+0000","last_became_peered":"2024-05-30T15:54:22.658700+0000","last_unstale":"2024-05-30T17:13:38.361999+0000","last_undegraded":"2024-05-30T17:13:38.361999+0000","last_fullsized":"2024-05-30T17:13:38.361999+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":100,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14119,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":309,"num_read_kb":443,"num_write":154,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":514,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.048165+0000","last_change":"2024-05-30T17:13:43.048221+0000","last_active":"2024-05-30T17:13:43.048165+0000","last_peered":"2024-05-30T17:13:43.048165+0000","last_clean":"2024-05-30T17:13:43.048165+0000","last_became_active":"2024-05-30T15:54:24.673505+0000","last_became_peered":"2024-05-30T15:54:24.673505+0000","last_unstale":"2024-05-30T17:13:43.048165+0000","last_undegraded":"2024-05-30T17:13:43.048165+0000","last_fullsized":"2024-05-30T17:13:43.048165+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:43.047974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:43.047974+0000","last_clean_scrub_stamp":"2024-05-30T17:13:43.047974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:04:22.180317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"265'101","reported_seq":627,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.954947+0000","last_change":"2024-05-30T17:13:50.955055+0000","last_active":"2024-05-30T17:13:50.954947+0000","last_peered":"2024-05-30T17:13:50.954947+0000","last_clean":"2024-05-30T17:13:50.954947+0000","last_became_active":"2024-05-30T15:54:26.700608+0000","last_became_peered":"2024-05-30T15:54:26.700608+0000","last_unstale":"2024-05-30T17:13:50.954947+0000","last_undegraded":"2024-05-30T17:13:50.954947+0000","last_fullsized":"2024-05-30T17:13:50.954947+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'101","last_scrub_stamp":"2024-05-30T17:13:50.954735+0000","last_deep_scrub":"265'101","last_deep_scrub_stamp":"2024-05-30T17:13:50.954735+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.954735+0000","objects_scrubbed":2,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:48:33.984457+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":918,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":103,"num_write":225,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"265'539","reported_seq":2168,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:06.773729+0000","last_change":"2024-05-30T15:54:18.658375+0000","last_active":"2024-05-30T17:13:06.773729+0000","last_peered":"2024-05-30T17:13:06.773729+0000","last_clean":"2024-05-30T17:13:06.773729+0000","last_became_active":"2024-05-30T15:54:18.657522+0000","last_became_peered":"2024-05-30T15:54:18.657522+0000","last_unstale":"2024-05-30T17:13:06.773729+0000","last_undegraded":"2024-05-30T17:13:06.773729+0000","last_fullsized":"2024-05-30T17:13:06.773729+0000","mapping_epoch":24,"log_start":"265'500","ondisk_log_start":"265'500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":500,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2751,"num_read_kb":2537,"num_write":708,"num_write_kb":252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"265'234","reported_seq":916,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.305058+0000","last_change":"2024-05-30T15:54:16.666646+0000","last_active":"2024-05-30T17:13:38.305058+0000","last_peered":"2024-05-30T17:13:38.305058+0000","last_clean":"2024-05-30T17:13:38.305058+0000","last_became_active":"2024-05-30T15:54:16.666244+0000","last_became_peered":"2024-05-30T15:54:16.666244+0000","last_unstale":"2024-05-30T17:13:38.305058+0000","last_undegraded":"2024-05-30T17:13:38.305058+0000","last_fullsized":"2024-05-30T17:13:38.305058+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":200,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":549856,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":638,"num_read_kb":32957,"num_write":1225,"num_write_kb":40334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":521,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:52.077511+0000","last_change":"2024-05-30T17:13:52.077571+0000","last_active":"2024-05-30T17:13:52.077511+0000","last_peered":"2024-05-30T17:13:52.077511+0000","last_clean":"2024-05-30T17:13:52.077511+0000","last_became_active":"2024-05-30T15:54:20.661967+0000","last_became_peered":"2024-05-30T15:54:20.661967+0000","last_unstale":"2024-05-30T17:13:52.077511+0000","last_undegraded":"2024-05-30T17:13:52.077511+0000","last_fullsized":"2024-05-30T17:13:52.077511+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:52.077316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:52.077316+0000","last_clean_scrub_stamp":"2024-05-30T17:13:52.077316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:55:45.987890+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"265'181","reported_seq":789,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.385977+0000","last_change":"2024-05-30T15:54:22.665466+0000","last_active":"2024-05-30T17:13:38.385977+0000","last_peered":"2024-05-30T17:13:38.385977+0000","last_clean":"2024-05-30T17:13:38.385977+0000","last_became_active":"2024-05-30T15:54:22.664732+0000","last_became_peered":"2024-05-30T15:54:22.664732+0000","last_unstale":"2024-05-30T17:13:38.385977+0000","last_undegraded":"2024-05-30T17:13:38.385977+0000","last_fullsized":"2024-05-30T17:13:38.385977+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":100,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13534,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":265,"num_read_kb":366,"num_write":202,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":507,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:31.615083+0000","last_change":"2024-05-30T15:54:24.679521+0000","last_active":"2024-05-30T17:13:31.615083+0000","last_peered":"2024-05-30T17:13:31.615083+0000","last_clean":"2024-05-30T17:13:31.615083+0000","last_became_active":"2024-05-30T15:54:24.679254+0000","last_became_peered":"2024-05-30T15:54:24.679254+0000","last_unstale":"2024-05-30T17:13:31.615083+0000","last_undegraded":"2024-05-30T17:13:31.615083+0000","last_fullsized":"2024-05-30T17:13:31.615083+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"265'109","reported_seq":636,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:57.389981+0000","last_change":"2024-05-30T17:13:57.390054+0000","last_active":"2024-05-30T17:13:57.389981+0000","last_peered":"2024-05-30T17:13:57.389981+0000","last_clean":"2024-05-30T17:13:57.389981+0000","last_became_active":"2024-05-30T15:54:26.697333+0000","last_became_peered":"2024-05-30T15:54:26.697333+0000","last_unstale":"2024-05-30T17:13:57.389981+0000","last_undegraded":"2024-05-30T17:13:57.389981+0000","last_fullsized":"2024-05-30T17:13:57.389981+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'109","last_scrub_stamp":"2024-05-30T17:13:57.389793+0000","last_deep_scrub":"265'109","last_deep_scrub_stamp":"2024-05-30T17:13:57.389793+0000","last_clean_scrub_stamp":"2024-05-30T17:13:57.389793+0000","objects_scrubbed":4,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:34:55.402057+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":270,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":107,"num_read_kb":90,"num_write":242,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"265'397","reported_seq":2060,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:12.231233+0000","last_change":"2024-05-30T15:54:18.636254+0000","last_active":"2024-05-30T17:13:12.231233+0000","last_peered":"2024-05-30T17:13:12.231233+0000","last_clean":"2024-05-30T17:13:12.231233+0000","last_became_active":"2024-05-30T15:54:18.635519+0000","last_became_peered":"2024-05-30T15:54:18.635519+0000","last_unstale":"2024-05-30T17:13:12.231233+0000","last_undegraded":"2024-05-30T17:13:12.231233+0000","last_fullsized":"2024-05-30T17:13:12.231233+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":300,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2332,"num_read_kb":2242,"num_write":450,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"265'175","reported_seq":796,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.437572+0000","last_change":"2024-05-30T15:54:16.685723+0000","last_active":"2024-05-30T17:13:38.437572+0000","last_peered":"2024-05-30T17:13:38.437572+0000","last_clean":"2024-05-30T17:13:38.437572+0000","last_became_active":"2024-05-30T15:54:16.685162+0000","last_became_peered":"2024-05-30T15:54:16.685162+0000","last_unstale":"2024-05-30T17:13:38.437572+0000","last_undegraded":"2024-05-30T17:13:38.437572+0000","last_fullsized":"2024-05-30T17:13:38.437572+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":100,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5904316,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":401,"num_read_kb":10586,"num_write":1027,"num_write_kb":21029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":519,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.991573+0000","last_change":"2024-05-30T17:13:56.991631+0000","last_active":"2024-05-30T17:13:56.991573+0000","last_peered":"2024-05-30T17:13:56.991573+0000","last_clean":"2024-05-30T17:13:56.991573+0000","last_became_active":"2024-05-30T15:54:20.646646+0000","last_became_peered":"2024-05-30T15:54:20.646646+0000","last_unstale":"2024-05-30T17:13:56.991573+0000","last_undegraded":"2024-05-30T17:13:56.991573+0000","last_fullsized":"2024-05-30T17:13:56.991573+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:56.991372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:56.991372+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.991372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:25:19.485498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"265'189","reported_seq":815,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.157684+0000","last_change":"2024-05-30T17:13:46.157858+0000","last_active":"2024-05-30T17:13:46.157684+0000","last_peered":"2024-05-30T17:13:46.157684+0000","last_clean":"2024-05-30T17:13:46.157684+0000","last_became_active":"2024-05-30T15:54:22.669660+0000","last_became_peered":"2024-05-30T15:54:22.669660+0000","last_unstale":"2024-05-30T17:13:46.157684+0000","last_undegraded":"2024-05-30T17:13:46.157684+0000","last_fullsized":"2024-05-30T17:13:46.157684+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'189","last_scrub_stamp":"2024-05-30T17:13:46.157487+0000","last_deep_scrub":"265'189","last_deep_scrub_stamp":"2024-05-30T17:13:46.157487+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.157487+0000","objects_scrubbed":7,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:49:02.360024+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6014,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":258,"num_read_kb":320,"num_write":146,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"265'512","reported_seq":4604,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.085253+0000","last_change":"2024-05-30T17:13:44.085280+0000","last_active":"2024-05-30T17:13:44.085253+0000","last_peered":"2024-05-30T17:13:44.085253+0000","last_clean":"2024-05-30T17:13:44.085253+0000","last_became_active":"2024-05-30T15:54:24.672377+0000","last_became_peered":"2024-05-30T15:54:24.672377+0000","last_unstale":"2024-05-30T17:13:44.085253+0000","last_undegraded":"2024-05-30T17:13:44.085253+0000","last_fullsized":"2024-05-30T17:13:44.085253+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'512","last_scrub_stamp":"2024-05-30T17:13:44.085165+0000","last_deep_scrub":"265'512","last_deep_scrub_stamp":"2024-05-30T17:13:44.085165+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.085165+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":500,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:13:21.462886+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"265'120","reported_seq":645,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.014771+0000","last_change":"2024-05-30T17:13:53.014854+0000","last_active":"2024-05-30T17:13:53.014771+0000","last_peered":"2024-05-30T17:13:53.014771+0000","last_clean":"2024-05-30T17:13:53.014771+0000","last_became_active":"2024-05-30T15:54:26.679866+0000","last_became_peered":"2024-05-30T15:54:26.679866+0000","last_unstale":"2024-05-30T17:13:53.014771+0000","last_undegraded":"2024-05-30T17:13:53.014771+0000","last_fullsized":"2024-05-30T17:13:53.014771+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'120","last_scrub_stamp":"2024-05-30T17:13:53.014561+0000","last_deep_scrub":"265'120","last_deep_scrub_stamp":"2024-05-30T17:13:53.014561+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.014561+0000","objects_scrubbed":3,"log_size":20,"log_dups_size":100,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:47:46.937703+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1256,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":144,"num_read_kb":134,"num_write":293,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"265'389","reported_seq":1797,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:06.773841+0000","last_change":"2024-05-30T15:54:18.671784+0000","last_active":"2024-05-30T17:13:06.773841+0000","last_peered":"2024-05-30T17:13:06.773841+0000","last_clean":"2024-05-30T17:13:06.773841+0000","last_became_active":"2024-05-30T15:54:18.671088+0000","last_became_peered":"2024-05-30T15:54:18.671088+0000","last_unstale":"2024-05-30T17:13:06.773841+0000","last_undegraded":"2024-05-30T17:13:06.773841+0000","last_fullsized":"2024-05-30T17:13:06.773841+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":300,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2029,"num_read_kb":1894,"num_write":481,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"265'180","reported_seq":924,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.202371+0000","last_change":"2024-05-30T15:54:16.672575+0000","last_active":"2024-05-30T17:13:38.202371+0000","last_peered":"2024-05-30T17:13:38.202371+0000","last_clean":"2024-05-30T17:13:38.202371+0000","last_became_active":"2024-05-30T15:54:16.671477+0000","last_became_peered":"2024-05-30T15:54:16.671477+0000","last_unstale":"2024-05-30T17:13:38.202371+0000","last_undegraded":"2024-05-30T17:13:38.202371+0000","last_fullsized":"2024-05-30T17:13:38.202371+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":100,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110807,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":679,"num_read_kb":167118,"num_write":1082,"num_write_kb":34693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"27'2","reported_seq":1159,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:37.718852+0000","last_change":"2024-05-30T15:54:20.663952+0000","last_active":"2024-05-30T17:13:37.718852+0000","last_peered":"2024-05-30T17:13:37.718852+0000","last_clean":"2024-05-30T17:13:37.718852+0000","last_became_active":"2024-05-30T15:54:20.662371+0000","last_became_peered":"2024-05-30T15:54:20.662371+0000","last_unstale":"2024-05-30T17:13:37.718852+0000","last_undegraded":"2024-05-30T17:13:37.718852+0000","last_fullsized":"2024-05-30T17:13:37.718852+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":647,"num_read_kb":647,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"265'109","reported_seq":657,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:11.612581+0000","last_change":"2024-05-30T15:54:22.849771+0000","last_active":"2024-05-30T17:13:11.612581+0000","last_peered":"2024-05-30T17:13:11.612581+0000","last_clean":"2024-05-30T17:13:11.612581+0000","last_became_active":"2024-05-30T15:54:22.849105+0000","last_became_peered":"2024-05-30T15:54:22.849105+0000","last_unstale":"2024-05-30T17:13:11.612581+0000","last_undegraded":"2024-05-30T17:13:11.612581+0000","last_fullsized":"2024-05-30T17:13:11.612581+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":120,"num_read_kb":115,"num_write":80,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"265'541","reported_seq":2326,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.806417+0000","last_change":"2024-05-30T17:13:54.806572+0000","last_active":"2024-05-30T17:13:54.806417+0000","last_peered":"2024-05-30T17:13:54.806417+0000","last_clean":"2024-05-30T17:13:54.806417+0000","last_became_active":"2024-05-30T15:54:24.677291+0000","last_became_peered":"2024-05-30T15:54:24.677291+0000","last_unstale":"2024-05-30T17:13:54.806417+0000","last_undegraded":"2024-05-30T17:13:54.806417+0000","last_fullsized":"2024-05-30T17:13:54.806417+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'541","last_scrub_stamp":"2024-05-30T17:13:54.806190+0000","last_deep_scrub":"265'541","last_deep_scrub_stamp":"2024-05-30T17:13:54.806190+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.806190+0000","objects_scrubbed":1,"log_size":41,"log_dups_size":500,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T05:09:18.353958+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"265'136","reported_seq":666,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.859668+0000","last_change":"2024-05-30T17:13:49.859798+0000","last_active":"2024-05-30T17:13:49.859668+0000","last_peered":"2024-05-30T17:13:49.859668+0000","last_clean":"2024-05-30T17:13:49.859668+0000","last_became_active":"2024-05-30T15:54:26.674517+0000","last_became_peered":"2024-05-30T15:54:26.674517+0000","last_unstale":"2024-05-30T17:13:49.859668+0000","last_undegraded":"2024-05-30T17:13:49.859668+0000","last_fullsized":"2024-05-30T17:13:49.859668+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'136","last_scrub_stamp":"2024-05-30T17:13:49.859451+0000","last_deep_scrub":"265'136","last_deep_scrub_stamp":"2024-05-30T17:13:49.859451+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.859451+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:25:15.350422+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":141,"num_read_kb":129,"num_write":332,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"265'402","reported_seq":1941,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:21.818023+0000","last_change":"2024-05-30T15:54:18.657775+0000","last_active":"2024-05-30T17:13:21.818023+0000","last_peered":"2024-05-30T17:13:21.818023+0000","last_clean":"2024-05-30T17:13:21.818023+0000","last_became_active":"2024-05-30T15:54:18.657247+0000","last_became_peered":"2024-05-30T15:54:18.657247+0000","last_unstale":"2024-05-30T17:13:21.818023+0000","last_undegraded":"2024-05-30T17:13:21.818023+0000","last_fullsized":"2024-05-30T17:13:21.818023+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":300,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2106,"num_read_kb":2008,"num_write":461,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"265'185","reported_seq":804,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.106642+0000","last_change":"2024-05-30T17:13:45.106682+0000","last_active":"2024-05-30T17:13:45.106642+0000","last_peered":"2024-05-30T17:13:45.106642+0000","last_clean":"2024-05-30T17:13:45.106642+0000","last_became_active":"2024-05-30T15:54:54.426538+0000","last_became_peered":"2024-05-30T15:54:54.426538+0000","last_unstale":"2024-05-30T17:13:45.106642+0000","last_undegraded":"2024-05-30T17:13:45.106642+0000","last_fullsized":"2024-05-30T17:13:45.106642+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'185","last_scrub_stamp":"2024-05-30T17:13:45.106515+0000","last_deep_scrub":"265'185","last_deep_scrub_stamp":"2024-05-30T17:13:45.106515+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.106515+0000","objects_scrubbed":1,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:52:29.468974+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":443,"num_read_kb":549,"num_write":1012,"num_write_kb":15096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,7],"acting":[2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":519,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:06.773890+0000","last_change":"2024-05-30T15:54:20.670270+0000","last_active":"2024-05-30T17:13:06.773890+0000","last_peered":"2024-05-30T17:13:06.773890+0000","last_clean":"2024-05-30T17:13:06.773890+0000","last_became_active":"2024-05-30T15:54:20.669492+0000","last_became_peered":"2024-05-30T15:54:20.669492+0000","last_unstale":"2024-05-30T17:13:06.773890+0000","last_undegraded":"2024-05-30T17:13:06.773890+0000","last_fullsized":"2024-05-30T17:13:06.773890+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"265'243","reported_seq":890,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:01.276657+0000","last_change":"2024-05-30T17:14:01.276754+0000","last_active":"2024-05-30T17:14:01.276657+0000","last_peered":"2024-05-30T17:14:01.276657+0000","last_clean":"2024-05-30T17:14:01.276657+0000","last_became_active":"2024-05-30T15:54:22.671943+0000","last_became_peered":"2024-05-30T15:54:22.671943+0000","last_unstale":"2024-05-30T17:14:01.276657+0000","last_undegraded":"2024-05-30T17:14:01.276657+0000","last_fullsized":"2024-05-30T17:14:01.276657+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'243","last_scrub_stamp":"2024-05-30T17:14:01.276461+0000","last_deep_scrub":"265'243","last_deep_scrub_stamp":"2024-05-30T17:14:01.276461+0000","last_clean_scrub_stamp":"2024-05-30T17:14:01.276461+0000","objects_scrubbed":8,"log_size":43,"log_dups_size":200,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:31:23.133344+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":313,"num_read_kb":312,"num_write":178,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":509,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:00.785558+0000","last_change":"2024-05-30T17:14:00.785659+0000","last_active":"2024-05-30T17:14:00.785558+0000","last_peered":"2024-05-30T17:14:00.785558+0000","last_clean":"2024-05-30T17:14:00.785558+0000","last_became_active":"2024-05-30T15:54:24.681595+0000","last_became_peered":"2024-05-30T15:54:24.681595+0000","last_unstale":"2024-05-30T17:14:00.785558+0000","last_undegraded":"2024-05-30T17:14:00.785558+0000","last_fullsized":"2024-05-30T17:14:00.785558+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:00.785264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:00.785264+0000","last_clean_scrub_stamp":"2024-05-30T17:14:00.785264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:38:41.393882+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"265'119","reported_seq":635,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.989353+0000","last_change":"2024-05-30T17:13:51.989431+0000","last_active":"2024-05-30T17:13:51.989353+0000","last_peered":"2024-05-30T17:13:51.989353+0000","last_clean":"2024-05-30T17:13:51.989353+0000","last_became_active":"2024-05-30T15:54:26.688518+0000","last_became_peered":"2024-05-30T15:54:26.688518+0000","last_unstale":"2024-05-30T17:13:51.989353+0000","last_undegraded":"2024-05-30T17:13:51.989353+0000","last_fullsized":"2024-05-30T17:13:51.989353+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'119","last_scrub_stamp":"2024-05-30T17:13:51.989161+0000","last_deep_scrub":"265'119","last_deep_scrub_stamp":"2024-05-30T17:13:51.989161+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.989161+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":100,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:46:17.674786+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":619,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":113,"num_read_kb":112,"num_write":261,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"265'758","reported_seq":2277,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:57.780923+0000","last_change":"2024-05-30T17:13:57.781083+0000","last_active":"2024-05-30T17:13:57.780923+0000","last_peered":"2024-05-30T17:13:57.780923+0000","last_clean":"2024-05-30T17:13:57.780923+0000","last_became_active":"2024-05-30T15:54:18.677957+0000","last_became_peered":"2024-05-30T15:54:18.677957+0000","last_unstale":"2024-05-30T17:13:57.780923+0000","last_undegraded":"2024-05-30T17:13:57.780923+0000","last_fullsized":"2024-05-30T17:13:57.780923+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'758","last_scrub_stamp":"2024-05-30T17:13:57.780707+0000","last_deep_scrub":"265'758","last_deep_scrub_stamp":"2024-05-30T17:13:57.780707+0000","last_clean_scrub_stamp":"2024-05-30T17:13:57.780707+0000","objects_scrubbed":2,"log_size":58,"log_dups_size":700,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:16:34.692139+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3440,"num_read_kb":3078,"num_write":1010,"num_write_kb":372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":628,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:26.585314+0000","last_change":"2024-05-30T15:54:03.031292+0000","last_active":"2024-05-30T17:12:26.585314+0000","last_peered":"2024-05-30T17:12:26.585314+0000","last_clean":"2024-05-30T17:12:26.585314+0000","last_became_active":"2024-05-30T15:54:03.029214+0000","last_became_peered":"2024-05-30T15:54:03.029214+0000","last_unstale":"2024-05-30T17:12:26.585314+0000","last_undegraded":"2024-05-30T17:12:26.585314+0000","last_fullsized":"2024-05-30T17:12:26.585314+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:01.512251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:01.512251+0000","last_clean_scrub_stamp":"2024-05-30T15:54:01.512251+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":531,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:21.981988+0000","last_change":"2024-05-30T15:54:08.578526+0000","last_active":"2024-05-30T17:13:21.981988+0000","last_peered":"2024-05-30T17:13:21.981988+0000","last_clean":"2024-05-30T17:13:21.981988+0000","last_became_active":"2024-05-30T15:54:08.577556+0000","last_became_peered":"2024-05-30T15:54:08.577556+0000","last_unstale":"2024-05-30T17:13:21.981988+0000","last_undegraded":"2024-05-30T17:13:21.981988+0000","last_fullsized":"2024-05-30T17:13:21.981988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"265'189","reported_seq":845,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.415326+0000","last_change":"2024-05-30T15:54:16.674631+0000","last_active":"2024-05-30T17:13:38.415326+0000","last_peered":"2024-05-30T17:13:38.415326+0000","last_clean":"2024-05-30T17:13:38.415326+0000","last_became_active":"2024-05-30T15:54:16.674229+0000","last_became_peered":"2024-05-30T15:54:16.674229+0000","last_unstale":"2024-05-30T17:13:38.415326+0000","last_undegraded":"2024-05-30T17:13:38.415326+0000","last_fullsized":"2024-05-30T17:13:38.415326+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5525719,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":447,"num_read_kb":11470,"num_write":1074,"num_write_kb":25445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":513,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:31.602335+0000","last_change":"2024-05-30T15:54:20.665207+0000","last_active":"2024-05-30T17:13:31.602335+0000","last_peered":"2024-05-30T17:13:31.602335+0000","last_clean":"2024-05-30T17:13:31.602335+0000","last_became_active":"2024-05-30T15:54:20.664317+0000","last_became_peered":"2024-05-30T15:54:20.664317+0000","last_unstale":"2024-05-30T17:13:31.602335+0000","last_undegraded":"2024-05-30T17:13:31.602335+0000","last_fullsized":"2024-05-30T17:13:31.602335+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"265'334","reported_seq":1056,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.210051+0000","last_change":"2024-05-30T15:54:22.670731+0000","last_active":"2024-05-30T17:13:38.210051+0000","last_peered":"2024-05-30T17:13:38.210051+0000","last_clean":"2024-05-30T17:13:38.210051+0000","last_became_active":"2024-05-30T15:54:22.669871+0000","last_became_peered":"2024-05-30T15:54:22.669871+0000","last_unstale":"2024-05-30T17:13:38.210051+0000","last_undegraded":"2024-05-30T17:13:38.210051+0000","last_fullsized":"2024-05-30T17:13:38.210051+0000","mapping_epoch":28,"log_start":"265'300","ondisk_log_start":"265'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49272,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":648,"num_read_kb":960,"num_write":598,"num_write_kb":460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":516,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.868767+0000","last_change":"2024-05-30T17:13:58.868833+0000","last_active":"2024-05-30T17:13:58.868767+0000","last_peered":"2024-05-30T17:13:58.868767+0000","last_clean":"2024-05-30T17:13:58.868767+0000","last_became_active":"2024-05-30T15:54:24.684709+0000","last_became_peered":"2024-05-30T15:54:24.684709+0000","last_unstale":"2024-05-30T17:13:58.868767+0000","last_undegraded":"2024-05-30T17:13:58.868767+0000","last_fullsized":"2024-05-30T17:13:58.868767+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:58.868550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:58.868550+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.868550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:45:56.978968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"265'174","reported_seq":879,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.790400+0000","last_change":"2024-05-30T15:54:26.687357+0000","last_active":"2024-05-30T17:13:20.790400+0000","last_peered":"2024-05-30T17:13:20.790400+0000","last_clean":"2024-05-30T17:13:20.790400+0000","last_became_active":"2024-05-30T15:54:26.686647+0000","last_became_peered":"2024-05-30T15:54:26.686647+0000","last_unstale":"2024-05-30T17:13:20.790400+0000","last_undegraded":"2024-05-30T17:13:20.790400+0000","last_fullsized":"2024-05-30T17:13:20.790400+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1020,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":416,"num_read_kb":404,"num_write":359,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"265'388","reported_seq":1863,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:28.318621+0000","last_change":"2024-05-30T15:54:18.649340+0000","last_active":"2024-05-30T17:12:28.318621+0000","last_peered":"2024-05-30T17:12:28.318621+0000","last_clean":"2024-05-30T17:12:28.318621+0000","last_became_active":"2024-05-30T15:54:18.648529+0000","last_became_peered":"2024-05-30T15:54:18.648529+0000","last_unstale":"2024-05-30T17:12:28.318621+0000","last_undegraded":"2024-05-30T17:12:28.318621+0000","last_fullsized":"2024-05-30T17:12:28.318621+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":300,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2082,"num_read_kb":1967,"num_write":465,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":532,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:31.586728+0000","last_change":"2024-05-30T15:54:08.575348+0000","last_active":"2024-05-30T17:12:31.586728+0000","last_peered":"2024-05-30T17:12:31.586728+0000","last_clean":"2024-05-30T17:12:31.586728+0000","last_became_active":"2024-05-30T15:54:08.574645+0000","last_became_peered":"2024-05-30T15:54:08.574645+0000","last_unstale":"2024-05-30T17:12:31.586728+0000","last_undegraded":"2024-05-30T17:12:31.586728+0000","last_fullsized":"2024-05-30T17:12:31.586728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"265'174","reported_seq":810,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.439442+0000","last_change":"2024-05-30T15:54:16.678919+0000","last_active":"2024-05-30T17:13:38.439442+0000","last_peered":"2024-05-30T17:13:38.439442+0000","last_clean":"2024-05-30T17:13:38.439442+0000","last_became_active":"2024-05-30T15:54:16.678386+0000","last_became_peered":"2024-05-30T15:54:16.678386+0000","last_unstale":"2024-05-30T17:13:38.439442+0000","last_undegraded":"2024-05-30T17:13:38.439442+0000","last_fullsized":"2024-05-30T17:13:38.439442+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079916,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":364,"num_read_kb":7665,"num_write":1082,"num_write_kb":33234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":511,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:21.968039+0000","last_change":"2024-05-30T15:54:20.663148+0000","last_active":"2024-05-30T17:12:21.968039+0000","last_peered":"2024-05-30T17:12:21.968039+0000","last_clean":"2024-05-30T17:12:21.968039+0000","last_became_active":"2024-05-30T15:54:20.661502+0000","last_became_peered":"2024-05-30T15:54:20.661502+0000","last_unstale":"2024-05-30T17:12:21.968039+0000","last_undegraded":"2024-05-30T17:12:21.968039+0000","last_fullsized":"2024-05-30T17:12:21.968039+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"265'722","reported_seq":2553,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.445728+0000","last_change":"2024-05-30T15:54:22.672464+0000","last_active":"2024-05-30T17:13:38.445728+0000","last_peered":"2024-05-30T17:13:38.445728+0000","last_clean":"2024-05-30T17:13:38.445728+0000","last_became_active":"2024-05-30T15:54:22.671588+0000","last_became_peered":"2024-05-30T15:54:22.671588+0000","last_unstale":"2024-05-30T17:13:38.445728+0000","last_undegraded":"2024-05-30T17:13:38.445728+0000","last_fullsized":"2024-05-30T17:13:38.445728+0000","mapping_epoch":28,"log_start":"265'700","ondisk_log_start":"265'700","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":700,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19819,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":1008,"num_read_kb":1208,"num_write":324,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":505,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.052200+0000","last_change":"2024-05-30T15:54:24.688069+0000","last_active":"2024-05-30T17:13:45.052200+0000","last_peered":"2024-05-30T17:13:45.052200+0000","last_clean":"2024-05-30T17:13:45.052200+0000","last_became_active":"2024-05-30T15:54:24.686641+0000","last_became_peered":"2024-05-30T15:54:24.686641+0000","last_unstale":"2024-05-30T17:13:45.052200+0000","last_undegraded":"2024-05-30T17:13:45.052200+0000","last_fullsized":"2024-05-30T17:13:45.052200+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"265'125","reported_seq":645,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.985096+0000","last_change":"2024-05-30T17:13:54.985175+0000","last_active":"2024-05-30T17:13:54.985096+0000","last_peered":"2024-05-30T17:13:54.985096+0000","last_clean":"2024-05-30T17:13:54.985096+0000","last_became_active":"2024-05-30T15:54:26.667751+0000","last_became_peered":"2024-05-30T15:54:26.667751+0000","last_unstale":"2024-05-30T17:13:54.985096+0000","last_undegraded":"2024-05-30T17:13:54.985096+0000","last_fullsized":"2024-05-30T17:13:54.985096+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'125","last_scrub_stamp":"2024-05-30T17:13:54.984877+0000","last_deep_scrub":"265'125","last_deep_scrub_stamp":"2024-05-30T17:13:54.984877+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.984877+0000","objects_scrubbed":2,"log_size":25,"log_dups_size":100,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:40:14.213968+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":438,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":115,"num_write":291,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"265'326","reported_seq":1647,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:33.938973+0000","last_change":"2024-05-30T15:54:18.652138+0000","last_active":"2024-05-30T17:12:33.938973+0000","last_peered":"2024-05-30T17:12:33.938973+0000","last_clean":"2024-05-30T17:12:33.938973+0000","last_became_active":"2024-05-30T15:54:18.651853+0000","last_became_peered":"2024-05-30T15:54:18.651853+0000","last_unstale":"2024-05-30T17:12:33.938973+0000","last_undegraded":"2024-05-30T17:12:33.938973+0000","last_fullsized":"2024-05-30T17:12:33.938973+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":300,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1727,"num_read_kb":1636,"num_write":373,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":542,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.261872+0000","last_change":"2024-05-30T17:13:59.261974+0000","last_active":"2024-05-30T17:13:59.261872+0000","last_peered":"2024-05-30T17:13:59.261872+0000","last_clean":"2024-05-30T17:13:59.261872+0000","last_became_active":"2024-05-30T15:54:08.744898+0000","last_became_peered":"2024-05-30T15:54:08.744898+0000","last_unstale":"2024-05-30T17:13:59.261872+0000","last_undegraded":"2024-05-30T17:13:59.261872+0000","last_fullsized":"2024-05-30T17:13:59.261872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:59.261650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:59.261650+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.261650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:03:48.512476+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"265'185","reported_seq":814,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.402514+0000","last_change":"2024-05-30T15:54:16.650710+0000","last_active":"2024-05-30T17:13:38.402514+0000","last_peered":"2024-05-30T17:13:38.402514+0000","last_clean":"2024-05-30T17:13:38.402514+0000","last_became_active":"2024-05-30T15:54:16.650214+0000","last_became_peered":"2024-05-30T15:54:16.650214+0000","last_unstale":"2024-05-30T17:13:38.402514+0000","last_undegraded":"2024-05-30T17:13:38.402514+0000","last_fullsized":"2024-05-30T17:13:38.402514+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1911070,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":412,"num_read_kb":8332,"num_write":1154,"num_write_kb":25808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"265'2","reported_seq":524,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.146546+0000","last_change":"2024-05-30T17:13:50.146675+0000","last_active":"2024-05-30T17:13:50.146546+0000","last_peered":"2024-05-30T17:13:50.146546+0000","last_clean":"2024-05-30T17:13:50.146546+0000","last_became_active":"2024-05-30T15:54:20.666715+0000","last_became_peered":"2024-05-30T15:54:20.666715+0000","last_unstale":"2024-05-30T17:13:50.146546+0000","last_undegraded":"2024-05-30T17:13:50.146546+0000","last_fullsized":"2024-05-30T17:13:50.146546+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:13:50.146258+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:13:50.146258+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.146258+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:41:50.441025+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"265'259","reported_seq":1773,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.052022+0000","last_change":"2024-05-30T15:54:22.853392+0000","last_active":"2024-05-30T17:13:45.052022+0000","last_peered":"2024-05-30T17:13:45.052022+0000","last_clean":"2024-05-30T17:13:45.052022+0000","last_became_active":"2024-05-30T15:54:22.852604+0000","last_became_peered":"2024-05-30T15:54:22.852604+0000","last_unstale":"2024-05-30T17:13:45.052022+0000","last_undegraded":"2024-05-30T17:13:45.052022+0000","last_fullsized":"2024-05-30T17:13:45.052022+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":200,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":232,"num_read_kb":224,"num_write":183,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":518,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:00.884506+0000","last_change":"2024-05-30T17:14:00.884643+0000","last_active":"2024-05-30T17:14:00.884506+0000","last_peered":"2024-05-30T17:14:00.884506+0000","last_clean":"2024-05-30T17:14:00.884506+0000","last_became_active":"2024-05-30T15:54:24.679830+0000","last_became_peered":"2024-05-30T15:54:24.679830+0000","last_unstale":"2024-05-30T17:14:00.884506+0000","last_undegraded":"2024-05-30T17:14:00.884506+0000","last_fullsized":"2024-05-30T17:14:00.884506+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:00.884270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:00.884270+0000","last_clean_scrub_stamp":"2024-05-30T17:14:00.884270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:18:06.001324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"265'253","reported_seq":871,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.356811+0000","last_change":"2024-05-30T17:13:56.356879+0000","last_active":"2024-05-30T17:13:56.356811+0000","last_peered":"2024-05-30T17:13:56.356811+0000","last_clean":"2024-05-30T17:13:56.356811+0000","last_became_active":"2024-05-30T15:54:26.696477+0000","last_became_peered":"2024-05-30T15:54:26.696477+0000","last_unstale":"2024-05-30T17:13:56.356811+0000","last_undegraded":"2024-05-30T17:13:56.356811+0000","last_fullsized":"2024-05-30T17:13:56.356811+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'253","last_scrub_stamp":"2024-05-30T17:13:56.356623+0000","last_deep_scrub":"265'253","last_deep_scrub_stamp":"2024-05-30T17:13:56.356623+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.356623+0000","objects_scrubbed":2,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:15:54.649548+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":476,"num_read_kb":500,"num_write":542,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1193,"num_omap_keys":7,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"265'761","reported_seq":2322,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:31.820285+0000","last_change":"2024-05-30T15:54:18.640139+0000","last_active":"2024-05-30T17:13:31.820285+0000","last_peered":"2024-05-30T17:13:31.820285+0000","last_clean":"2024-05-30T17:13:31.820285+0000","last_became_active":"2024-05-30T15:54:18.639830+0000","last_became_peered":"2024-05-30T15:54:18.639830+0000","last_unstale":"2024-05-30T17:13:31.820285+0000","last_undegraded":"2024-05-30T17:13:31.820285+0000","last_fullsized":"2024-05-30T17:13:31.820285+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":700,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3481,"num_read_kb":3132,"num_write":1014,"num_write_kb":367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":531,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:21.981904+0000","last_change":"2024-05-30T15:54:08.575969+0000","last_active":"2024-05-30T17:13:21.981904+0000","last_peered":"2024-05-30T17:13:21.981904+0000","last_clean":"2024-05-30T17:13:21.981904+0000","last_became_active":"2024-05-30T15:54:08.574817+0000","last_became_peered":"2024-05-30T15:54:08.574817+0000","last_unstale":"2024-05-30T17:13:21.981904+0000","last_undegraded":"2024-05-30T17:13:21.981904+0000","last_fullsized":"2024-05-30T17:13:21.981904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"265'199","reported_seq":915,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.300141+0000","last_change":"2024-05-30T15:54:16.656169+0000","last_active":"2024-05-30T17:13:38.300141+0000","last_peered":"2024-05-30T17:13:38.300141+0000","last_clean":"2024-05-30T17:13:38.300141+0000","last_became_active":"2024-05-30T15:54:16.655347+0000","last_became_peered":"2024-05-30T15:54:16.655347+0000","last_unstale":"2024-05-30T17:13:38.300141+0000","last_undegraded":"2024-05-30T17:13:38.300141+0000","last_fullsized":"2024-05-30T17:13:38.300141+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":572,"num_read_kb":42208,"num_write":1148,"num_write_kb":36370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":518,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.647322+0000","last_change":"2024-05-30T15:54:20.651897+0000","last_active":"2024-05-30T17:13:41.647322+0000","last_peered":"2024-05-30T17:13:41.647322+0000","last_clean":"2024-05-30T17:13:41.647322+0000","last_became_active":"2024-05-30T15:54:20.651449+0000","last_became_peered":"2024-05-30T15:54:20.651449+0000","last_unstale":"2024-05-30T17:13:41.647322+0000","last_undegraded":"2024-05-30T17:13:41.647322+0000","last_fullsized":"2024-05-30T17:13:41.647322+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"265'160","reported_seq":733,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:26.602969+0000","last_change":"2024-05-30T15:54:22.679074+0000","last_active":"2024-05-30T17:12:26.602969+0000","last_peered":"2024-05-30T17:12:26.602969+0000","last_clean":"2024-05-30T17:12:26.602969+0000","last_became_active":"2024-05-30T15:54:22.673881+0000","last_became_peered":"2024-05-30T15:54:22.673881+0000","last_unstale":"2024-05-30T17:12:26.602969+0000","last_undegraded":"2024-05-30T17:12:26.602969+0000","last_fullsized":"2024-05-30T17:12:26.602969+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":100,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":200,"num_read_kb":192,"num_write":144,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":510,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.801167+0000","last_change":"2024-05-30T17:13:56.801223+0000","last_active":"2024-05-30T17:13:56.801167+0000","last_peered":"2024-05-30T17:13:56.801167+0000","last_clean":"2024-05-30T17:13:56.801167+0000","last_became_active":"2024-05-30T15:54:24.675916+0000","last_became_peered":"2024-05-30T15:54:24.675916+0000","last_unstale":"2024-05-30T17:13:56.801167+0000","last_undegraded":"2024-05-30T17:13:56.801167+0000","last_fullsized":"2024-05-30T17:13:56.801167+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:56.800964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:56.800964+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.800964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:50:34.435231+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"265'139","reported_seq":666,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.262374+0000","last_change":"2024-05-30T17:13:49.262568+0000","last_active":"2024-05-30T17:13:49.262374+0000","last_peered":"2024-05-30T17:13:49.262374+0000","last_clean":"2024-05-30T17:13:49.262374+0000","last_became_active":"2024-05-30T15:54:26.704217+0000","last_became_peered":"2024-05-30T15:54:26.704217+0000","last_unstale":"2024-05-30T17:13:49.262374+0000","last_undegraded":"2024-05-30T17:13:49.262374+0000","last_fullsized":"2024-05-30T17:13:49.262374+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'139","last_scrub_stamp":"2024-05-30T17:13:49.262162+0000","last_deep_scrub":"265'139","last_deep_scrub_stamp":"2024-05-30T17:13:49.262162+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.262162+0000","objects_scrubbed":2,"log_size":39,"log_dups_size":100,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:30:17.555273+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":834,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":138,"num_read_kb":136,"num_write":341,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"265'396","reported_seq":1828,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:31.820215+0000","last_change":"2024-05-30T15:54:18.665286+0000","last_active":"2024-05-30T17:13:31.820215+0000","last_peered":"2024-05-30T17:13:31.820215+0000","last_clean":"2024-05-30T17:13:31.820215+0000","last_became_active":"2024-05-30T15:54:18.665034+0000","last_became_peered":"2024-05-30T15:54:18.665034+0000","last_unstale":"2024-05-30T17:13:31.820215+0000","last_undegraded":"2024-05-30T17:13:31.820215+0000","last_fullsized":"2024-05-30T17:13:31.820215+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":300,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2053,"num_read_kb":1920,"num_write":476,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"265'217","reported_seq":859,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.082974+0000","last_change":"2024-05-30T15:54:16.651678+0000","last_active":"2024-05-30T17:13:38.082974+0000","last_peered":"2024-05-30T17:13:38.082974+0000","last_clean":"2024-05-30T17:13:38.082974+0000","last_became_active":"2024-05-30T15:54:16.650735+0000","last_became_peered":"2024-05-30T15:54:16.650735+0000","last_unstale":"2024-05-30T17:13:38.082974+0000","last_undegraded":"2024-05-30T17:13:38.082974+0000","last_fullsized":"2024-05-30T17:13:38.082974+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":200,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079967,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":420,"num_read_kb":5617,"num_write":983,"num_write_kb":15732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":532,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:21.817928+0000","last_change":"2024-05-30T15:54:08.568214+0000","last_active":"2024-05-30T17:13:21.817928+0000","last_peered":"2024-05-30T17:13:21.817928+0000","last_clean":"2024-05-30T17:13:21.817928+0000","last_became_active":"2024-05-30T15:54:08.567900+0000","last_became_peered":"2024-05-30T15:54:08.567900+0000","last_unstale":"2024-05-30T17:13:21.817928+0000","last_undegraded":"2024-05-30T17:13:21.817928+0000","last_fullsized":"2024-05-30T17:13:21.817928+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":518,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:31.614922+0000","last_change":"2024-05-30T15:54:20.654924+0000","last_active":"2024-05-30T17:13:31.614922+0000","last_peered":"2024-05-30T17:13:31.614922+0000","last_clean":"2024-05-30T17:13:31.614922+0000","last_became_active":"2024-05-30T15:54:20.653790+0000","last_became_peered":"2024-05-30T15:54:20.653790+0000","last_unstale":"2024-05-30T17:13:31.614922+0000","last_undegraded":"2024-05-30T17:13:31.614922+0000","last_fullsized":"2024-05-30T17:13:31.614922+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"265'361","reported_seq":1062,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.227703+0000","last_change":"2024-05-30T15:54:22.674716+0000","last_active":"2024-05-30T17:13:38.227703+0000","last_peered":"2024-05-30T17:13:38.227703+0000","last_clean":"2024-05-30T17:13:38.227703+0000","last_became_active":"2024-05-30T15:54:22.673903+0000","last_became_peered":"2024-05-30T15:54:22.673903+0000","last_unstale":"2024-05-30T17:13:38.227703+0000","last_undegraded":"2024-05-30T17:13:38.227703+0000","last_fullsized":"2024-05-30T17:13:38.227703+0000","mapping_epoch":28,"log_start":"265'300","ondisk_log_start":"265'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":300,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11442,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1140,"num_read_kb":1226,"num_write":1674,"num_write_kb":1475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":522,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:55.903746+0000","last_change":"2024-05-30T17:13:55.903798+0000","last_active":"2024-05-30T17:13:55.903746+0000","last_peered":"2024-05-30T17:13:55.903746+0000","last_clean":"2024-05-30T17:13:55.903746+0000","last_became_active":"2024-05-30T15:54:24.677198+0000","last_became_peered":"2024-05-30T15:54:24.677198+0000","last_unstale":"2024-05-30T17:13:55.903746+0000","last_undegraded":"2024-05-30T17:13:55.903746+0000","last_fullsized":"2024-05-30T17:13:55.903746+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:55.903559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:55.903559+0000","last_clean_scrub_stamp":"2024-05-30T17:13:55.903559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:20:32.475018+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"265'122","reported_seq":649,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:57.847760+0000","last_change":"2024-05-30T17:13:57.847843+0000","last_active":"2024-05-30T17:13:57.847760+0000","last_peered":"2024-05-30T17:13:57.847760+0000","last_clean":"2024-05-30T17:13:57.847760+0000","last_became_active":"2024-05-30T15:54:26.676078+0000","last_became_peered":"2024-05-30T15:54:26.676078+0000","last_unstale":"2024-05-30T17:13:57.847760+0000","last_undegraded":"2024-05-30T17:13:57.847760+0000","last_fullsized":"2024-05-30T17:13:57.847760+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'122","last_scrub_stamp":"2024-05-30T17:13:57.847569+0000","last_deep_scrub":"265'122","last_deep_scrub_stamp":"2024-05-30T17:13:57.847569+0000","last_clean_scrub_stamp":"2024-05-30T17:13:57.847569+0000","objects_scrubbed":5,"log_size":22,"log_dups_size":100,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:27:31.982550+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":727,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":114,"num_read_kb":108,"num_write":274,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":73,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"265'791","reported_seq":2424,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:31.614896+0000","last_change":"2024-05-30T15:54:18.644630+0000","last_active":"2024-05-30T17:13:31.614896+0000","last_peered":"2024-05-30T17:13:31.614896+0000","last_clean":"2024-05-30T17:13:31.614896+0000","last_became_active":"2024-05-30T15:54:18.642683+0000","last_became_peered":"2024-05-30T15:54:18.642683+0000","last_unstale":"2024-05-30T17:13:31.614896+0000","last_undegraded":"2024-05-30T17:13:31.614896+0000","last_fullsized":"2024-05-30T17:13:31.614896+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":700,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3596,"num_read_kb":3243,"num_write":1037,"num_write_kb":360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"265'184","reported_seq":828,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.372573+0000","last_change":"2024-05-30T15:54:16.664218+0000","last_active":"2024-05-30T17:13:38.372573+0000","last_peered":"2024-05-30T17:13:38.372573+0000","last_clean":"2024-05-30T17:13:38.372573+0000","last_became_active":"2024-05-30T15:54:16.663360+0000","last_became_peered":"2024-05-30T15:54:16.663360+0000","last_unstale":"2024-05-30T17:13:38.372573+0000","last_undegraded":"2024-05-30T17:13:38.372573+0000","last_fullsized":"2024-05-30T17:13:38.372573+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":100,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":661325,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":407,"num_read_kb":30298,"num_write":1063,"num_write_kb":42840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":531,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.604977+0000","last_change":"2024-05-30T15:54:08.577379+0000","last_active":"2024-05-30T17:13:41.604977+0000","last_peered":"2024-05-30T17:13:41.604977+0000","last_clean":"2024-05-30T17:13:41.604977+0000","last_became_active":"2024-05-30T15:54:08.576779+0000","last_became_peered":"2024-05-30T15:54:08.576779+0000","last_unstale":"2024-05-30T17:13:41.604977+0000","last_undegraded":"2024-05-30T17:13:41.604977+0000","last_fullsized":"2024-05-30T17:13:41.604977+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":520,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:55.817272+0000","last_change":"2024-05-30T17:13:55.817326+0000","last_active":"2024-05-30T17:13:55.817272+0000","last_peered":"2024-05-30T17:13:55.817272+0000","last_clean":"2024-05-30T17:13:55.817272+0000","last_became_active":"2024-05-30T15:54:20.655220+0000","last_became_peered":"2024-05-30T15:54:20.655220+0000","last_unstale":"2024-05-30T17:13:55.817272+0000","last_undegraded":"2024-05-30T17:13:55.817272+0000","last_fullsized":"2024-05-30T17:13:55.817272+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:55.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:55.817064+0000","last_clean_scrub_stamp":"2024-05-30T17:13:55.817064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:12:59.140457+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"265'197","reported_seq":795,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:36.821345+0000","last_change":"2024-05-30T15:54:22.851840+0000","last_active":"2024-05-30T17:13:36.821345+0000","last_peered":"2024-05-30T17:13:36.821345+0000","last_clean":"2024-05-30T17:13:36.821345+0000","last_became_active":"2024-05-30T15:54:22.851338+0000","last_became_peered":"2024-05-30T15:54:22.851338+0000","last_unstale":"2024-05-30T17:13:36.821345+0000","last_undegraded":"2024-05-30T17:13:36.821345+0000","last_fullsized":"2024-05-30T17:13:36.821345+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":100,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":256,"num_read_kb":249,"num_write":168,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"265'557","reported_seq":2774,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:57.243711+0000","last_change":"2024-05-30T17:13:57.243776+0000","last_active":"2024-05-30T17:13:57.243711+0000","last_peered":"2024-05-30T17:13:57.243711+0000","last_clean":"2024-05-30T17:13:57.243711+0000","last_became_active":"2024-05-30T15:54:24.677176+0000","last_became_peered":"2024-05-30T15:54:24.677176+0000","last_unstale":"2024-05-30T17:13:57.243711+0000","last_undegraded":"2024-05-30T17:13:57.243711+0000","last_fullsized":"2024-05-30T17:13:57.243711+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'557","last_scrub_stamp":"2024-05-30T17:13:57.243509+0000","last_deep_scrub":"265'557","last_deep_scrub_stamp":"2024-05-30T17:13:57.243509+0000","last_clean_scrub_stamp":"2024-05-30T17:13:57.243509+0000","objects_scrubbed":1,"log_size":57,"log_dups_size":500,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:04:49.675199+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"265'131","reported_seq":647,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.355450+0000","last_change":"2024-05-30T17:13:58.355552+0000","last_active":"2024-05-30T17:13:58.355450+0000","last_peered":"2024-05-30T17:13:58.355450+0000","last_clean":"2024-05-30T17:13:58.355450+0000","last_became_active":"2024-05-30T15:54:26.697294+0000","last_became_peered":"2024-05-30T15:54:26.697294+0000","last_unstale":"2024-05-30T17:13:58.355450+0000","last_undegraded":"2024-05-30T17:13:58.355450+0000","last_fullsized":"2024-05-30T17:13:58.355450+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'131","last_scrub_stamp":"2024-05-30T17:13:58.355257+0000","last_deep_scrub":"265'131","last_deep_scrub_stamp":"2024-05-30T17:13:58.355257+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.355257+0000","objects_scrubbed":7,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:43:23.516003+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1933,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":112,"num_read_kb":106,"num_write":288,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":64,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"265'716","reported_seq":2111,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:12.349843+0000","last_change":"2024-05-30T15:54:18.653261+0000","last_active":"2024-05-30T17:13:12.349843+0000","last_peered":"2024-05-30T17:13:12.349843+0000","last_clean":"2024-05-30T17:13:12.349843+0000","last_became_active":"2024-05-30T15:54:18.652981+0000","last_became_peered":"2024-05-30T15:54:18.652981+0000","last_unstale":"2024-05-30T17:13:12.349843+0000","last_undegraded":"2024-05-30T17:13:12.349843+0000","last_fullsized":"2024-05-30T17:13:12.349843+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":700,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3098,"num_read_kb":2766,"num_write":954,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"265'200","reported_seq":863,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.403305+0000","last_change":"2024-05-30T15:54:16.652512+0000","last_active":"2024-05-30T17:13:38.403305+0000","last_peered":"2024-05-30T17:13:38.403305+0000","last_clean":"2024-05-30T17:13:38.403305+0000","last_became_active":"2024-05-30T15:54:16.652033+0000","last_became_peered":"2024-05-30T15:54:16.652033+0000","last_unstale":"2024-05-30T17:13:38.403305+0000","last_undegraded":"2024-05-30T17:13:38.403305+0000","last_fullsized":"2024-05-30T17:13:38.403305+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":100,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":287689,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":515,"num_read_kb":15333,"num_write":1052,"num_write_kb":22470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":530,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:06.801712+0000","last_change":"2024-05-30T15:54:08.574181+0000","last_active":"2024-05-30T17:12:06.801712+0000","last_peered":"2024-05-30T17:12:06.801712+0000","last_clean":"2024-05-30T17:12:06.801712+0000","last_became_active":"2024-05-30T15:54:08.573754+0000","last_became_peered":"2024-05-30T15:54:08.573754+0000","last_unstale":"2024-05-30T17:12:06.801712+0000","last_undegraded":"2024-05-30T17:12:06.801712+0000","last_fullsized":"2024-05-30T17:12:06.801712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":522,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:31.779168+0000","last_change":"2024-05-30T15:54:20.657666+0000","last_active":"2024-05-30T17:13:31.779168+0000","last_peered":"2024-05-30T17:13:31.779168+0000","last_clean":"2024-05-30T17:13:31.779168+0000","last_became_active":"2024-05-30T15:54:20.656941+0000","last_became_peered":"2024-05-30T15:54:20.656941+0000","last_unstale":"2024-05-30T17:13:31.779168+0000","last_undegraded":"2024-05-30T17:13:31.779168+0000","last_fullsized":"2024-05-30T17:13:31.779168+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"265'135","reported_seq":717,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:57.901626+0000","last_change":"2024-05-30T17:13:57.901801+0000","last_active":"2024-05-30T17:13:57.901626+0000","last_peered":"2024-05-30T17:13:57.901626+0000","last_clean":"2024-05-30T17:13:57.901626+0000","last_became_active":"2024-05-30T15:54:22.674027+0000","last_became_peered":"2024-05-30T15:54:22.674027+0000","last_unstale":"2024-05-30T17:13:57.901626+0000","last_undegraded":"2024-05-30T17:13:57.901626+0000","last_fullsized":"2024-05-30T17:13:57.901626+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'135","last_scrub_stamp":"2024-05-30T17:13:57.901361+0000","last_deep_scrub":"265'135","last_deep_scrub_stamp":"2024-05-30T17:13:57.901361+0000","last_clean_scrub_stamp":"2024-05-30T17:13:57.901361+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":100,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:02:52.230529+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":164,"num_read_kb":159,"num_write":108,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"265'550","reported_seq":2634,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.455387+0000","last_change":"2024-05-30T15:54:24.677368+0000","last_active":"2024-05-30T17:13:38.455387+0000","last_peered":"2024-05-30T17:13:38.455387+0000","last_clean":"2024-05-30T17:13:38.455387+0000","last_became_active":"2024-05-30T15:54:24.676608+0000","last_became_peered":"2024-05-30T15:54:24.676608+0000","last_unstale":"2024-05-30T17:13:38.455387+0000","last_undegraded":"2024-05-30T17:13:38.455387+0000","last_fullsized":"2024-05-30T17:13:38.455387+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":500,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"265'233","reported_seq":789,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.655070+0000","last_change":"2024-05-30T17:13:50.655156+0000","last_active":"2024-05-30T17:13:50.655070+0000","last_peered":"2024-05-30T17:13:50.655070+0000","last_clean":"2024-05-30T17:13:50.655070+0000","last_became_active":"2024-05-30T15:54:26.699148+0000","last_became_peered":"2024-05-30T15:54:26.699148+0000","last_unstale":"2024-05-30T17:13:50.655070+0000","last_undegraded":"2024-05-30T17:13:50.655070+0000","last_fullsized":"2024-05-30T17:13:50.655070+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'233","last_scrub_stamp":"2024-05-30T17:13:50.654846+0000","last_deep_scrub":"265'233","last_deep_scrub_stamp":"2024-05-30T17:13:50.654846+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.654846+0000","objects_scrubbed":6,"log_size":33,"log_dups_size":200,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:23:30.288859+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":326,"num_read_kb":298,"num_write":504,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":178,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"265'747","reported_seq":2340,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:31.779194+0000","last_change":"2024-05-30T15:54:18.658639+0000","last_active":"2024-05-30T17:13:31.779194+0000","last_peered":"2024-05-30T17:13:31.779194+0000","last_clean":"2024-05-30T17:13:31.779194+0000","last_became_active":"2024-05-30T15:54:18.657842+0000","last_became_peered":"2024-05-30T15:54:18.657842+0000","last_unstale":"2024-05-30T17:13:31.779194+0000","last_undegraded":"2024-05-30T17:13:31.779194+0000","last_fullsized":"2024-05-30T17:13:31.779194+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":700,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3632,"num_read_kb":3285,"num_write":997,"num_write_kb":370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"265'238","reported_seq":995,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.415399+0000","last_change":"2024-05-30T15:54:16.657515+0000","last_active":"2024-05-30T17:13:38.415399+0000","last_peered":"2024-05-30T17:13:38.415399+0000","last_clean":"2024-05-30T17:13:38.415399+0000","last_became_active":"2024-05-30T15:54:16.657067+0000","last_became_peered":"2024-05-30T15:54:16.657067+0000","last_unstale":"2024-05-30T17:13:38.415399+0000","last_undegraded":"2024-05-30T17:13:38.415399+0000","last_fullsized":"2024-05-30T17:13:38.415399+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":200,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5418111,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":685,"num_read_kb":72437,"num_write":1157,"num_write_kb":30547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":528,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:11.570355+0000","last_change":"2024-05-30T15:54:08.578053+0000","last_active":"2024-05-30T17:12:11.570355+0000","last_peered":"2024-05-30T17:12:11.570355+0000","last_clean":"2024-05-30T17:12:11.570355+0000","last_became_active":"2024-05-30T15:54:08.577191+0000","last_became_peered":"2024-05-30T15:54:08.577191+0000","last_unstale":"2024-05-30T17:12:11.570355+0000","last_undegraded":"2024-05-30T17:12:11.570355+0000","last_fullsized":"2024-05-30T17:12:11.570355+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":516,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:41.606086+0000","last_change":"2024-05-30T15:54:20.653740+0000","last_active":"2024-05-30T17:12:41.606086+0000","last_peered":"2024-05-30T17:12:41.606086+0000","last_clean":"2024-05-30T17:12:41.606086+0000","last_became_active":"2024-05-30T15:54:20.653404+0000","last_became_peered":"2024-05-30T15:54:20.653404+0000","last_unstale":"2024-05-30T17:12:41.606086+0000","last_undegraded":"2024-05-30T17:12:41.606086+0000","last_fullsized":"2024-05-30T17:12:41.606086+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"265'346","reported_seq":1094,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.442403+0000","last_change":"2024-05-30T15:54:22.849490+0000","last_active":"2024-05-30T17:13:38.442403+0000","last_peered":"2024-05-30T17:13:38.442403+0000","last_clean":"2024-05-30T17:13:38.442403+0000","last_became_active":"2024-05-30T15:54:22.848761+0000","last_became_peered":"2024-05-30T15:54:22.848761+0000","last_unstale":"2024-05-30T17:13:38.442403+0000","last_undegraded":"2024-05-30T17:13:38.442403+0000","last_fullsized":"2024-05-30T17:13:38.442403+0000","mapping_epoch":28,"log_start":"265'300","ondisk_log_start":"265'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":300,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24415,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":525,"num_read_kb":761,"num_write":274,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":507,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:36.617928+0000","last_change":"2024-05-30T15:54:24.687584+0000","last_active":"2024-05-30T17:13:36.617928+0000","last_peered":"2024-05-30T17:13:36.617928+0000","last_clean":"2024-05-30T17:13:36.617928+0000","last_became_active":"2024-05-30T15:54:24.686801+0000","last_became_peered":"2024-05-30T15:54:24.686801+0000","last_unstale":"2024-05-30T17:13:36.617928+0000","last_undegraded":"2024-05-30T17:13:36.617928+0000","last_fullsized":"2024-05-30T17:13:36.617928+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"265'152","reported_seq":662,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.241219+0000","last_change":"2024-05-30T17:13:51.241295+0000","last_active":"2024-05-30T17:13:51.241219+0000","last_peered":"2024-05-30T17:13:51.241219+0000","last_clean":"2024-05-30T17:13:51.241219+0000","last_became_active":"2024-05-30T15:54:26.700325+0000","last_became_peered":"2024-05-30T15:54:26.700325+0000","last_unstale":"2024-05-30T17:13:51.241219+0000","last_undegraded":"2024-05-30T17:13:51.241219+0000","last_fullsized":"2024-05-30T17:13:51.241219+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'152","last_scrub_stamp":"2024-05-30T17:13:51.241024+0000","last_deep_scrub":"265'152","last_deep_scrub_stamp":"2024-05-30T17:13:51.241024+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.241024+0000","objects_scrubbed":2,"log_size":52,"log_dups_size":100,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:14:44.872712+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":204,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":115,"num_write":342,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"265'150","reported_seq":732,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.796700+0000","last_change":"2024-05-30T17:13:56.796775+0000","last_active":"2024-05-30T17:13:56.796700+0000","last_peered":"2024-05-30T17:13:56.796700+0000","last_clean":"2024-05-30T17:13:56.796700+0000","last_became_active":"2024-05-30T15:54:26.686310+0000","last_became_peered":"2024-05-30T15:54:26.686310+0000","last_unstale":"2024-05-30T17:13:56.796700+0000","last_undegraded":"2024-05-30T17:13:56.796700+0000","last_fullsized":"2024-05-30T17:13:56.796700+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'150","last_scrub_stamp":"2024-05-30T17:13:56.796467+0000","last_deep_scrub":"265'150","last_deep_scrub_stamp":"2024-05-30T17:13:56.796467+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.796467+0000","objects_scrubbed":1,"log_size":50,"log_dups_size":100,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:01:08.998802+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":227,"num_read_kb":222,"num_write":347,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"265'746","reported_seq":2047,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.781457+0000","last_change":"2024-05-30T15:54:18.674169+0000","last_active":"2024-05-30T17:13:41.781457+0000","last_peered":"2024-05-30T17:13:41.781457+0000","last_clean":"2024-05-30T17:13:41.781457+0000","last_became_active":"2024-05-30T15:54:18.673418+0000","last_became_peered":"2024-05-30T15:54:18.673418+0000","last_unstale":"2024-05-30T17:13:41.781457+0000","last_undegraded":"2024-05-30T17:13:41.781457+0000","last_fullsized":"2024-05-30T17:13:41.781457+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":700,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3258,"num_read_kb":2851,"num_write":1044,"num_write_kb":425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"265'202","reported_seq":830,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.152930+0000","last_change":"2024-05-30T17:13:49.153045+0000","last_active":"2024-05-30T17:13:49.152930+0000","last_peered":"2024-05-30T17:13:49.152930+0000","last_clean":"2024-05-30T17:13:49.152930+0000","last_became_active":"2024-05-30T15:54:54.450966+0000","last_became_peered":"2024-05-30T15:54:54.450966+0000","last_unstale":"2024-05-30T17:13:49.152930+0000","last_undegraded":"2024-05-30T17:13:49.152930+0000","last_fullsized":"2024-05-30T17:13:49.152930+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'202","last_scrub_stamp":"2024-05-30T17:13:49.152709+0000","last_deep_scrub":"265'202","last_deep_scrub_stamp":"2024-05-30T17:13:49.152709+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.152709+0000","objects_scrubbed":6,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:19:50.214056+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262272,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":401,"num_read_kb":38159,"num_write":1164,"num_write_kb":35962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":518,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:00.085249+0000","last_change":"2024-05-30T17:14:00.085304+0000","last_active":"2024-05-30T17:14:00.085249+0000","last_peered":"2024-05-30T17:14:00.085249+0000","last_clean":"2024-05-30T17:14:00.085249+0000","last_became_active":"2024-05-30T15:54:20.664882+0000","last_became_peered":"2024-05-30T15:54:20.664882+0000","last_unstale":"2024-05-30T17:14:00.085249+0000","last_undegraded":"2024-05-30T17:14:00.085249+0000","last_fullsized":"2024-05-30T17:14:00.085249+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:00.085062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:00.085062+0000","last_clean_scrub_stamp":"2024-05-30T17:14:00.085062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:36:44.188047+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"265'153","reported_seq":753,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.074575+0000","last_change":"2024-05-30T15:54:22.672900+0000","last_active":"2024-05-30T17:13:43.074575+0000","last_peered":"2024-05-30T17:13:43.074575+0000","last_clean":"2024-05-30T17:13:43.074575+0000","last_became_active":"2024-05-30T15:54:22.672170+0000","last_became_peered":"2024-05-30T15:54:22.672170+0000","last_unstale":"2024-05-30T17:13:43.074575+0000","last_undegraded":"2024-05-30T17:13:43.074575+0000","last_fullsized":"2024-05-30T17:13:43.074575+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":100,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5933,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":205,"num_read_kb":259,"num_write":110,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":503,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.075229+0000","last_change":"2024-05-30T15:54:24.688552+0000","last_active":"2024-05-30T17:13:43.075229+0000","last_peered":"2024-05-30T17:13:43.075229+0000","last_clean":"2024-05-30T17:13:43.075229+0000","last_became_active":"2024-05-30T15:54:24.688053+0000","last_became_peered":"2024-05-30T15:54:24.688053+0000","last_unstale":"2024-05-30T17:13:43.075229+0000","last_undegraded":"2024-05-30T17:13:43.075229+0000","last_fullsized":"2024-05-30T17:13:43.075229+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"265'120","reported_seq":654,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.374741+0000","last_change":"2024-05-30T17:13:54.374814+0000","last_active":"2024-05-30T17:13:54.374741+0000","last_peered":"2024-05-30T17:13:54.374741+0000","last_clean":"2024-05-30T17:13:54.374741+0000","last_became_active":"2024-05-30T15:54:26.701588+0000","last_became_peered":"2024-05-30T15:54:26.701588+0000","last_unstale":"2024-05-30T17:13:54.374741+0000","last_undegraded":"2024-05-30T17:13:54.374741+0000","last_fullsized":"2024-05-30T17:13:54.374741+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'120","last_scrub_stamp":"2024-05-30T17:13:54.374529+0000","last_deep_scrub":"265'120","last_deep_scrub_stamp":"2024-05-30T17:13:54.374529+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.374529+0000","objects_scrubbed":3,"log_size":20,"log_dups_size":100,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:54:34.109921+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":826,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":172,"num_read_kb":153,"num_write":293,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"265'323","reported_seq":1770,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:01.069483+0000","last_change":"2024-05-30T17:14:01.069557+0000","last_active":"2024-05-30T17:14:01.069483+0000","last_peered":"2024-05-30T17:14:01.069483+0000","last_clean":"2024-05-30T17:14:01.069483+0000","last_became_active":"2024-05-30T15:54:18.670707+0000","last_became_peered":"2024-05-30T15:54:18.670707+0000","last_unstale":"2024-05-30T17:14:01.069483+0000","last_undegraded":"2024-05-30T17:14:01.069483+0000","last_fullsized":"2024-05-30T17:14:01.069483+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'323","last_scrub_stamp":"2024-05-30T17:14:01.069299+0000","last_deep_scrub":"265'323","last_deep_scrub_stamp":"2024-05-30T17:14:01.069299+0000","last_clean_scrub_stamp":"2024-05-30T17:14:01.069299+0000","objects_scrubbed":2,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:28:00.081754+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1910,"num_read_kb":1828,"num_write":376,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"265'219","reported_seq":885,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.051242+0000","last_change":"2024-05-30T15:54:16.663769+0000","last_active":"2024-05-30T17:13:45.051242+0000","last_peered":"2024-05-30T17:13:45.051242+0000","last_clean":"2024-05-30T17:13:45.051242+0000","last_became_active":"2024-05-30T15:54:16.660427+0000","last_became_peered":"2024-05-30T15:54:16.660427+0000","last_unstale":"2024-05-30T17:13:45.051242+0000","last_undegraded":"2024-05-30T17:13:45.051242+0000","last_fullsized":"2024-05-30T17:13:45.051242+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":200,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262181,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":518,"num_read_kb":33224,"num_write":1148,"num_write_kb":46290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":511,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.619149+0000","last_change":"2024-05-30T15:54:20.649980+0000","last_active":"2024-05-30T17:13:41.619149+0000","last_peered":"2024-05-30T17:13:41.619149+0000","last_clean":"2024-05-30T17:13:41.619149+0000","last_became_active":"2024-05-30T15:54:20.649330+0000","last_became_peered":"2024-05-30T15:54:20.649330+0000","last_unstale":"2024-05-30T17:13:41.619149+0000","last_undegraded":"2024-05-30T17:13:41.619149+0000","last_fullsized":"2024-05-30T17:13:41.619149+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"265'71","reported_seq":597,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.647459+0000","last_change":"2024-05-30T15:54:22.663453+0000","last_active":"2024-05-30T17:13:41.647459+0000","last_peered":"2024-05-30T17:13:41.647459+0000","last_clean":"2024-05-30T17:13:41.647459+0000","last_became_active":"2024-05-30T15:54:22.662738+0000","last_became_peered":"2024-05-30T15:54:22.662738+0000","last_unstale":"2024-05-30T17:13:41.647459+0000","last_undegraded":"2024-05-30T17:13:41.647459+0000","last_fullsized":"2024-05-30T17:13:41.647459+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":50,"num_write":40,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":501,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.052203+0000","last_change":"2024-05-30T15:54:24.682005+0000","last_active":"2024-05-30T17:13:45.052203+0000","last_peered":"2024-05-30T17:13:45.052203+0000","last_clean":"2024-05-30T17:13:45.052203+0000","last_became_active":"2024-05-30T15:54:24.680578+0000","last_became_peered":"2024-05-30T15:54:24.680578+0000","last_unstale":"2024-05-30T17:13:45.052203+0000","last_undegraded":"2024-05-30T17:13:45.052203+0000","last_fullsized":"2024-05-30T17:13:45.052203+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"265'215","reported_seq":784,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.750305+0000","last_change":"2024-05-30T17:13:54.750473+0000","last_active":"2024-05-30T17:13:54.750305+0000","last_peered":"2024-05-30T17:13:54.750305+0000","last_clean":"2024-05-30T17:13:54.750305+0000","last_became_active":"2024-05-30T15:54:26.686553+0000","last_became_peered":"2024-05-30T15:54:26.686553+0000","last_unstale":"2024-05-30T17:13:54.750305+0000","last_undegraded":"2024-05-30T17:13:54.750305+0000","last_fullsized":"2024-05-30T17:13:54.750305+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'215","last_scrub_stamp":"2024-05-30T17:13:54.750033+0000","last_deep_scrub":"265'215","last_deep_scrub_stamp":"2024-05-30T17:13:54.750033+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.750033+0000","objects_scrubbed":3,"log_size":15,"log_dups_size":200,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:23:19.977063+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":816,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":414,"num_read_kb":383,"num_write":513,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":398,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"265'413","reported_seq":1791,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.619062+0000","last_change":"2024-05-30T15:54:18.660456+0000","last_active":"2024-05-30T17:13:41.619062+0000","last_peered":"2024-05-30T17:13:41.619062+0000","last_clean":"2024-05-30T17:13:41.619062+0000","last_became_active":"2024-05-30T15:54:18.660089+0000","last_became_peered":"2024-05-30T15:54:18.660089+0000","last_unstale":"2024-05-30T17:13:41.619062+0000","last_undegraded":"2024-05-30T17:13:41.619062+0000","last_fullsized":"2024-05-30T17:13:41.619062+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":400,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2036,"num_read_kb":1890,"num_write":519,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"265'217","reported_seq":888,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.065767+0000","last_change":"2024-05-30T15:54:16.650752+0000","last_active":"2024-05-30T17:13:44.065767+0000","last_peered":"2024-05-30T17:13:44.065767+0000","last_clean":"2024-05-30T17:13:44.065767+0000","last_became_active":"2024-05-30T15:54:16.650025+0000","last_became_peered":"2024-05-30T15:54:16.650025+0000","last_unstale":"2024-05-30T17:13:44.065767+0000","last_undegraded":"2024-05-30T17:13:44.065767+0000","last_fullsized":"2024-05-30T17:13:44.065767+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":200,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5569939,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":500,"num_read_kb":102670,"num_write":1113,"num_write_kb":39714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":509,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.604813+0000","last_change":"2024-05-30T15:54:20.657385+0000","last_active":"2024-05-30T17:13:41.604813+0000","last_peered":"2024-05-30T17:13:41.604813+0000","last_clean":"2024-05-30T17:13:41.604813+0000","last_became_active":"2024-05-30T15:54:20.656663+0000","last_became_peered":"2024-05-30T15:54:20.656663+0000","last_unstale":"2024-05-30T17:13:41.604813+0000","last_undegraded":"2024-05-30T17:13:41.604813+0000","last_fullsized":"2024-05-30T17:13:41.604813+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"265'186","reported_seq":802,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.074553+0000","last_change":"2024-05-30T15:54:22.676934+0000","last_active":"2024-05-30T17:13:43.074553+0000","last_peered":"2024-05-30T17:13:43.074553+0000","last_clean":"2024-05-30T17:13:43.074553+0000","last_became_active":"2024-05-30T15:54:22.673673+0000","last_became_peered":"2024-05-30T15:54:22.673673+0000","last_unstale":"2024-05-30T17:13:43.074553+0000","last_undegraded":"2024-05-30T17:13:43.074553+0000","last_fullsized":"2024-05-30T17:13:43.074553+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":100,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7114,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":259,"num_read_kb":332,"num_write":146,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":508,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.754848+0000","last_change":"2024-05-30T17:13:50.754954+0000","last_active":"2024-05-30T17:13:50.754848+0000","last_peered":"2024-05-30T17:13:50.754848+0000","last_clean":"2024-05-30T17:13:50.754848+0000","last_became_active":"2024-05-30T15:54:24.675289+0000","last_became_peered":"2024-05-30T15:54:24.675289+0000","last_unstale":"2024-05-30T17:13:50.754848+0000","last_undegraded":"2024-05-30T17:13:50.754848+0000","last_fullsized":"2024-05-30T17:13:50.754848+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:50.754554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:50.754554+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.754554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:47:56.533373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"265'141","reported_seq":646,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.256964+0000","last_change":"2024-05-30T17:13:50.257049+0000","last_active":"2024-05-30T17:13:50.256964+0000","last_peered":"2024-05-30T17:13:50.256964+0000","last_clean":"2024-05-30T17:13:50.256964+0000","last_became_active":"2024-05-30T15:54:26.703433+0000","last_became_peered":"2024-05-30T15:54:26.703433+0000","last_unstale":"2024-05-30T17:13:50.256964+0000","last_undegraded":"2024-05-30T17:13:50.256964+0000","last_fullsized":"2024-05-30T17:13:50.256964+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'141","last_scrub_stamp":"2024-05-30T17:13:50.256770+0000","last_deep_scrub":"265'141","last_deep_scrub_stamp":"2024-05-30T17:13:50.256770+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.256770+0000","objects_scrubbed":3,"log_size":41,"log_dups_size":100,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:08:34.180219+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":856,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":110,"num_write":329,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"265'863","reported_seq":2492,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.619000+0000","last_change":"2024-05-30T15:54:18.649560+0000","last_active":"2024-05-30T17:13:41.619000+0000","last_peered":"2024-05-30T17:13:41.619000+0000","last_clean":"2024-05-30T17:13:41.619000+0000","last_became_active":"2024-05-30T15:54:18.649273+0000","last_became_peered":"2024-05-30T15:54:18.649273+0000","last_unstale":"2024-05-30T17:13:41.619000+0000","last_undegraded":"2024-05-30T17:13:41.619000+0000","last_fullsized":"2024-05-30T17:13:41.619000+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":800,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3918,"num_read_kb":3498,"num_write":1187,"num_write_kb":458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"265'207","reported_seq":869,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.646953+0000","last_change":"2024-05-30T15:54:16.649854+0000","last_active":"2024-05-30T17:13:41.646953+0000","last_peered":"2024-05-30T17:13:41.646953+0000","last_clean":"2024-05-30T17:13:41.646953+0000","last_became_active":"2024-05-30T15:54:16.649121+0000","last_became_peered":"2024-05-30T15:54:16.649121+0000","last_unstale":"2024-05-30T17:13:41.646953+0000","last_undegraded":"2024-05-30T17:13:41.646953+0000","last_fullsized":"2024-05-30T17:13:41.646953+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":100,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1029196,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":493,"num_read_kb":65907,"num_write":1120,"num_write_kb":54558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":516,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.736994+0000","last_change":"2024-05-30T17:13:51.737047+0000","last_active":"2024-05-30T17:13:51.736994+0000","last_peered":"2024-05-30T17:13:51.736994+0000","last_clean":"2024-05-30T17:13:51.736994+0000","last_became_active":"2024-05-30T15:54:20.667863+0000","last_became_peered":"2024-05-30T15:54:20.667863+0000","last_unstale":"2024-05-30T17:13:51.736994+0000","last_undegraded":"2024-05-30T17:13:51.736994+0000","last_fullsized":"2024-05-30T17:13:51.736994+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:51.736796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:51.736796+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.736796+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:58:41.051038+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"265'143","reported_seq":734,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.052100+0000","last_change":"2024-05-30T15:54:22.665742+0000","last_active":"2024-05-30T17:13:45.052100+0000","last_peered":"2024-05-30T17:13:45.052100+0000","last_clean":"2024-05-30T17:13:45.052100+0000","last_became_active":"2024-05-30T15:54:22.664977+0000","last_became_peered":"2024-05-30T15:54:22.664977+0000","last_unstale":"2024-05-30T17:13:45.052100+0000","last_undegraded":"2024-05-30T17:13:45.052100+0000","last_fullsized":"2024-05-30T17:13:45.052100+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":100,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7290,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":187,"num_read_kb":260,"num_write":98,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.647676+0000","last_change":"2024-05-30T15:54:24.677137+0000","last_active":"2024-05-30T17:13:41.647676+0000","last_peered":"2024-05-30T17:13:41.647676+0000","last_clean":"2024-05-30T17:13:41.647676+0000","last_became_active":"2024-05-30T15:54:24.676707+0000","last_became_peered":"2024-05-30T15:54:24.676707+0000","last_unstale":"2024-05-30T17:13:41.647676+0000","last_undegraded":"2024-05-30T17:13:41.647676+0000","last_fullsized":"2024-05-30T17:13:41.647676+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"265'443","reported_seq":2109,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.771973+0000","last_change":"2024-05-30T17:13:59.772041+0000","last_active":"2024-05-30T17:13:59.771973+0000","last_peered":"2024-05-30T17:13:59.771973+0000","last_clean":"2024-05-30T17:13:59.771973+0000","last_became_active":"2024-05-30T15:54:18.670756+0000","last_became_peered":"2024-05-30T15:54:18.670756+0000","last_unstale":"2024-05-30T17:13:59.771973+0000","last_undegraded":"2024-05-30T17:13:59.771973+0000","last_fullsized":"2024-05-30T17:13:59.771973+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'443","last_scrub_stamp":"2024-05-30T17:13:59.771783+0000","last_deep_scrub":"265'443","last_deep_scrub_stamp":"2024-05-30T17:13:59.771783+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.771783+0000","objects_scrubbed":1,"log_size":43,"log_dups_size":400,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:50:25.853396+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2383,"num_read_kb":2277,"num_write":522,"num_write_kb":97,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"265'164","reported_seq":770,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.073839+0000","last_change":"2024-05-30T15:54:16.654636+0000","last_active":"2024-05-30T17:13:43.073839+0000","last_peered":"2024-05-30T17:13:43.073839+0000","last_clean":"2024-05-30T17:13:43.073839+0000","last_became_active":"2024-05-30T15:54:16.653560+0000","last_became_peered":"2024-05-30T15:54:16.653560+0000","last_unstale":"2024-05-30T17:13:43.073839+0000","last_undegraded":"2024-05-30T17:13:43.073839+0000","last_fullsized":"2024-05-30T17:13:43.073839+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":100,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5417712,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":326,"num_read_kb":31913,"num_write":888,"num_write_kb":20958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":519,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:41.604301+0000","last_change":"2024-05-30T15:54:20.662992+0000","last_active":"2024-05-30T17:12:41.604301+0000","last_peered":"2024-05-30T17:12:41.604301+0000","last_clean":"2024-05-30T17:12:41.604301+0000","last_became_active":"2024-05-30T15:54:20.662669+0000","last_became_peered":"2024-05-30T15:54:20.662669+0000","last_unstale":"2024-05-30T17:12:41.604301+0000","last_undegraded":"2024-05-30T17:12:41.604301+0000","last_fullsized":"2024-05-30T17:12:41.604301+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"265'224","reported_seq":874,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.407462+0000","last_change":"2024-05-30T15:54:22.664232+0000","last_active":"2024-05-30T17:13:38.407462+0000","last_peered":"2024-05-30T17:13:38.407462+0000","last_clean":"2024-05-30T17:13:38.407462+0000","last_became_active":"2024-05-30T15:54:22.663518+0000","last_became_peered":"2024-05-30T15:54:22.663518+0000","last_unstale":"2024-05-30T17:13:38.407462+0000","last_undegraded":"2024-05-30T17:13:38.407462+0000","last_fullsized":"2024-05-30T17:13:38.407462+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":200,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11676,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":342,"num_read_kb":490,"num_write":222,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:55.195013+0000","last_change":"2024-05-30T17:13:55.195111+0000","last_active":"2024-05-30T17:13:55.195013+0000","last_peered":"2024-05-30T17:13:55.195013+0000","last_clean":"2024-05-30T17:13:55.195013+0000","last_became_active":"2024-05-30T15:54:24.682609+0000","last_became_peered":"2024-05-30T15:54:24.682609+0000","last_unstale":"2024-05-30T17:13:55.195013+0000","last_undegraded":"2024-05-30T17:13:55.195013+0000","last_fullsized":"2024-05-30T17:13:55.195013+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:55.194814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:55.194814+0000","last_clean_scrub_stamp":"2024-05-30T17:13:55.194814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:35:33.515747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"265'151","reported_seq":672,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:00.314401+0000","last_change":"2024-05-30T17:14:00.314480+0000","last_active":"2024-05-30T17:14:00.314401+0000","last_peered":"2024-05-30T17:14:00.314401+0000","last_clean":"2024-05-30T17:14:00.314401+0000","last_became_active":"2024-05-30T15:54:26.695210+0000","last_became_peered":"2024-05-30T15:54:26.695210+0000","last_unstale":"2024-05-30T17:14:00.314401+0000","last_undegraded":"2024-05-30T17:14:00.314401+0000","last_fullsized":"2024-05-30T17:14:00.314401+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'151","last_scrub_stamp":"2024-05-30T17:14:00.314210+0000","last_deep_scrub":"265'151","last_deep_scrub_stamp":"2024-05-30T17:14:00.314210+0000","last_clean_scrub_stamp":"2024-05-30T17:14:00.314210+0000","objects_scrubbed":2,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:57:02.003537+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":643,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":167,"num_read_kb":179,"num_write":338,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"265'854","reported_seq":2476,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.656545+0000","last_change":"2024-05-30T15:54:18.641464+0000","last_active":"2024-05-30T17:12:44.656545+0000","last_peered":"2024-05-30T17:12:44.656545+0000","last_clean":"2024-05-30T17:12:44.656545+0000","last_became_active":"2024-05-30T15:54:18.640739+0000","last_became_peered":"2024-05-30T15:54:18.640739+0000","last_unstale":"2024-05-30T17:12:44.656545+0000","last_undegraded":"2024-05-30T17:12:44.656545+0000","last_fullsized":"2024-05-30T17:12:44.656545+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":800,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4053,"num_read_kb":3626,"num_write":1150,"num_write_kb":447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"265'245","reported_seq":987,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.935720+0000","last_change":"2024-05-30T17:13:47.935811+0000","last_active":"2024-05-30T17:13:47.935720+0000","last_peered":"2024-05-30T17:13:47.935720+0000","last_clean":"2024-05-30T17:13:47.935720+0000","last_became_active":"2024-05-30T15:54:54.424825+0000","last_became_peered":"2024-05-30T15:54:54.424825+0000","last_unstale":"2024-05-30T17:13:47.935720+0000","last_undegraded":"2024-05-30T17:13:47.935720+0000","last_fullsized":"2024-05-30T17:13:47.935720+0000","mapping_epoch":35,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'245","last_scrub_stamp":"2024-05-30T17:13:47.935515+0000","last_deep_scrub":"265'245","last_deep_scrub_stamp":"2024-05-30T17:13:47.935515+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.935515+0000","objects_scrubbed":3,"log_size":45,"log_dups_size":200,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:50:12.821107+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5285497,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":887,"num_read_kb":12756,"num_write":1230,"num_write_kb":34615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"265'4","reported_seq":848,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.789874+0000","last_change":"2024-05-30T17:13:53.789936+0000","last_active":"2024-05-30T17:13:53.789874+0000","last_peered":"2024-05-30T17:13:53.789874+0000","last_clean":"2024-05-30T17:13:53.789874+0000","last_became_active":"2024-05-30T15:54:20.664095+0000","last_became_peered":"2024-05-30T15:54:20.664095+0000","last_unstale":"2024-05-30T17:13:53.789874+0000","last_undegraded":"2024-05-30T17:13:53.789874+0000","last_fullsized":"2024-05-30T17:13:53.789874+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'4","last_scrub_stamp":"2024-05-30T17:13:53.789681+0000","last_deep_scrub":"265'4","last_deep_scrub_stamp":"2024-05-30T17:13:53.789681+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.789681+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:25:01.268608+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":494,"num_read_kb":492,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"265'194","reported_seq":805,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.432347+0000","last_change":"2024-05-30T15:54:22.665708+0000","last_active":"2024-05-30T17:13:38.432347+0000","last_peered":"2024-05-30T17:13:38.432347+0000","last_clean":"2024-05-30T17:13:38.432347+0000","last_became_active":"2024-05-30T15:54:22.664997+0000","last_became_peered":"2024-05-30T15:54:22.664997+0000","last_unstale":"2024-05-30T17:13:38.432347+0000","last_undegraded":"2024-05-30T17:13:38.432347+0000","last_fullsized":"2024-05-30T17:13:38.432347+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":100,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25625,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":290,"num_read_kb":412,"num_write":192,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":499,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:56.799200+0000","last_change":"2024-05-30T15:54:24.678998+0000","last_active":"2024-05-30T17:11:56.799200+0000","last_peered":"2024-05-30T17:11:56.799200+0000","last_clean":"2024-05-30T17:11:56.799200+0000","last_became_active":"2024-05-30T15:54:24.675583+0000","last_became_peered":"2024-05-30T15:54:24.675583+0000","last_unstale":"2024-05-30T17:11:56.799200+0000","last_undegraded":"2024-05-30T17:11:56.799200+0000","last_fullsized":"2024-05-30T17:11:56.799200+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"265'140","reported_seq":657,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.566461+0000","last_change":"2024-05-30T15:54:26.703108+0000","last_active":"2024-05-30T17:13:44.566461+0000","last_peered":"2024-05-30T17:13:44.566461+0000","last_clean":"2024-05-30T17:13:44.566461+0000","last_became_active":"2024-05-30T15:54:26.700392+0000","last_became_peered":"2024-05-30T15:54:26.700392+0000","last_unstale":"2024-05-30T17:13:44.566461+0000","last_undegraded":"2024-05-30T17:13:44.566461+0000","last_fullsized":"2024-05-30T17:13:44.566461+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":100,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2627,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":177,"num_read_kb":174,"num_write":403,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"265'371","reported_seq":1750,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.352543+0000","last_change":"2024-05-30T17:13:53.352650+0000","last_active":"2024-05-30T17:13:53.352543+0000","last_peered":"2024-05-30T17:13:53.352543+0000","last_clean":"2024-05-30T17:13:53.352543+0000","last_became_active":"2024-05-30T15:54:54.419468+0000","last_became_peered":"2024-05-30T15:54:54.419468+0000","last_unstale":"2024-05-30T17:13:53.352543+0000","last_undegraded":"2024-05-30T17:13:53.352543+0000","last_fullsized":"2024-05-30T17:13:53.352543+0000","mapping_epoch":35,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'371","last_scrub_stamp":"2024-05-30T17:13:53.352335+0000","last_deep_scrub":"265'371","last_deep_scrub_stamp":"2024-05-30T17:13:53.352335+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.352335+0000","objects_scrubbed":3,"log_size":71,"log_dups_size":300,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:30:41.234405+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1898,"num_read_kb":1782,"num_write":455,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":310,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"265'159","reported_seq":885,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.394814+0000","last_change":"2024-05-30T15:54:16.677283+0000","last_active":"2024-05-30T17:13:38.394814+0000","last_peered":"2024-05-30T17:13:38.394814+0000","last_clean":"2024-05-30T17:13:38.394814+0000","last_became_active":"2024-05-30T15:54:16.676440+0000","last_became_peered":"2024-05-30T15:54:16.676440+0000","last_unstale":"2024-05-30T17:13:38.394814+0000","last_undegraded":"2024-05-30T17:13:38.394814+0000","last_fullsized":"2024-05-30T17:13:38.394814+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456485,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":869,"num_read_kb":45599,"num_write":925,"num_write_kb":34636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":515,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:51.756199+0000","last_change":"2024-05-30T15:54:20.656111+0000","last_active":"2024-05-30T17:11:51.756199+0000","last_peered":"2024-05-30T17:11:51.756199+0000","last_clean":"2024-05-30T17:11:51.756199+0000","last_became_active":"2024-05-30T15:54:20.655388+0000","last_became_peered":"2024-05-30T15:54:20.655388+0000","last_unstale":"2024-05-30T17:11:51.756199+0000","last_undegraded":"2024-05-30T17:11:51.756199+0000","last_fullsized":"2024-05-30T17:11:51.756199+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"265'165","reported_seq":769,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.150744+0000","last_change":"2024-05-30T15:54:22.849861+0000","last_active":"2024-05-30T17:13:38.150744+0000","last_peered":"2024-05-30T17:13:38.150744+0000","last_clean":"2024-05-30T17:13:38.150744+0000","last_became_active":"2024-05-30T15:54:22.849193+0000","last_became_peered":"2024-05-30T15:54:22.849193+0000","last_unstale":"2024-05-30T17:13:38.150744+0000","last_undegraded":"2024-05-30T17:13:38.150744+0000","last_fullsized":"2024-05-30T17:13:38.150744+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":100,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11086,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":221,"num_read_kb":302,"num_write":124,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":507,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:41.809876+0000","last_change":"2024-05-30T15:54:24.682047+0000","last_active":"2024-05-30T17:12:41.809876+0000","last_peered":"2024-05-30T17:12:41.809876+0000","last_clean":"2024-05-30T17:12:41.809876+0000","last_became_active":"2024-05-30T15:54:24.681609+0000","last_became_peered":"2024-05-30T15:54:24.681609+0000","last_unstale":"2024-05-30T17:12:41.809876+0000","last_undegraded":"2024-05-30T17:12:41.809876+0000","last_fullsized":"2024-05-30T17:12:41.809876+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"265'132","reported_seq":644,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.247687+0000","last_change":"2024-05-30T17:13:47.247831+0000","last_active":"2024-05-30T17:13:47.247687+0000","last_peered":"2024-05-30T17:13:47.247687+0000","last_clean":"2024-05-30T17:13:47.247687+0000","last_became_active":"2024-05-30T15:54:26.704842+0000","last_became_peered":"2024-05-30T15:54:26.704842+0000","last_unstale":"2024-05-30T17:13:47.247687+0000","last_undegraded":"2024-05-30T17:13:47.247687+0000","last_fullsized":"2024-05-30T17:13:47.247687+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'132","last_scrub_stamp":"2024-05-30T17:13:47.247463+0000","last_deep_scrub":"265'132","last_deep_scrub_stamp":"2024-05-30T17:13:47.247463+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.247463+0000","objects_scrubbed":3,"log_size":32,"log_dups_size":100,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:58:57.717875+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":383,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":127,"num_read_kb":122,"num_write":322,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"265'753","reported_seq":2252,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:12.231918+0000","last_change":"2024-05-30T15:54:18.635089+0000","last_active":"2024-05-30T17:13:12.231918+0000","last_peered":"2024-05-30T17:13:12.231918+0000","last_clean":"2024-05-30T17:13:12.231918+0000","last_became_active":"2024-05-30T15:54:18.634803+0000","last_became_peered":"2024-05-30T15:54:18.634803+0000","last_unstale":"2024-05-30T17:13:12.231918+0000","last_undegraded":"2024-05-30T17:13:12.231918+0000","last_fullsized":"2024-05-30T17:13:12.231918+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":700,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":3476,"num_read_kb":3109,"num_write":1006,"num_write_kb":389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"265'190","reported_seq":823,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.153453+0000","last_change":"2024-05-30T17:13:47.153543+0000","last_active":"2024-05-30T17:13:47.153453+0000","last_peered":"2024-05-30T17:13:47.153453+0000","last_clean":"2024-05-30T17:13:47.153453+0000","last_became_active":"2024-05-30T15:54:54.430490+0000","last_became_peered":"2024-05-30T15:54:54.430490+0000","last_unstale":"2024-05-30T17:13:47.153453+0000","last_undegraded":"2024-05-30T17:13:47.153453+0000","last_fullsized":"2024-05-30T17:13:47.153453+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'190","last_scrub_stamp":"2024-05-30T17:13:47.153254+0000","last_deep_scrub":"265'190","last_deep_scrub_stamp":"2024-05-30T17:13:47.153254+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.153254+0000","objects_scrubbed":4,"log_size":90,"log_dups_size":100,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:50:08.663040+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":481,"num_read_kb":40323,"num_write":1075,"num_write_kb":31617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":515,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.000839+0000","last_change":"2024-05-30T17:13:56.000902+0000","last_active":"2024-05-30T17:13:56.000839+0000","last_peered":"2024-05-30T17:13:56.000839+0000","last_clean":"2024-05-30T17:13:56.000839+0000","last_became_active":"2024-05-30T15:54:20.650944+0000","last_became_peered":"2024-05-30T15:54:20.650944+0000","last_unstale":"2024-05-30T17:13:56.000839+0000","last_undegraded":"2024-05-30T17:13:56.000839+0000","last_fullsized":"2024-05-30T17:13:56.000839+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:56.000601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:56.000601+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.000601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:28:54.937679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"265'120","reported_seq":687,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.917769+0000","last_change":"2024-05-30T17:13:56.917856+0000","last_active":"2024-05-30T17:13:56.917769+0000","last_peered":"2024-05-30T17:13:56.917769+0000","last_clean":"2024-05-30T17:13:56.917769+0000","last_became_active":"2024-05-30T15:54:22.850865+0000","last_became_peered":"2024-05-30T15:54:22.850865+0000","last_unstale":"2024-05-30T17:13:56.917769+0000","last_undegraded":"2024-05-30T17:13:56.917769+0000","last_fullsized":"2024-05-30T17:13:56.917769+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'120","last_scrub_stamp":"2024-05-30T17:13:56.917584+0000","last_deep_scrub":"265'120","last_deep_scrub_stamp":"2024-05-30T17:13:56.917584+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.917584+0000","objects_scrubbed":7,"log_size":20,"log_dups_size":100,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:18:29.769236+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":127,"num_read_kb":121,"num_write":92,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1780,"num_omap_keys":6,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"265'541","reported_seq":4178,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.730062+0000","last_change":"2024-05-30T17:13:58.730127+0000","last_active":"2024-05-30T17:13:58.730062+0000","last_peered":"2024-05-30T17:13:58.730062+0000","last_clean":"2024-05-30T17:13:58.730062+0000","last_became_active":"2024-05-30T15:54:24.680695+0000","last_became_peered":"2024-05-30T15:54:24.680695+0000","last_unstale":"2024-05-30T17:13:58.730062+0000","last_undegraded":"2024-05-30T17:13:58.730062+0000","last_fullsized":"2024-05-30T17:13:58.730062+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'541","last_scrub_stamp":"2024-05-30T17:13:58.729875+0000","last_deep_scrub":"265'541","last_deep_scrub_stamp":"2024-05-30T17:13:58.729875+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.729875+0000","objects_scrubbed":1,"log_size":41,"log_dups_size":500,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:30:04.009452+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"265'134","reported_seq":643,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.052419+0000","last_change":"2024-05-30T15:54:26.696600+0000","last_active":"2024-05-30T17:13:45.052419+0000","last_peered":"2024-05-30T17:13:45.052419+0000","last_clean":"2024-05-30T17:13:45.052419+0000","last_became_active":"2024-05-30T15:54:26.695842+0000","last_became_peered":"2024-05-30T15:54:26.695842+0000","last_unstale":"2024-05-30T17:13:45.052419+0000","last_undegraded":"2024-05-30T17:13:45.052419+0000","last_fullsized":"2024-05-30T17:13:45.052419+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:25.645629+0000","last_clean_scrub_stamp":"2024-05-30T15:54:25.645629+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":418,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":133,"num_read_kb":127,"num_write":334,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"265'423","reported_seq":1943,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.709354+0000","last_change":"2024-05-30T15:54:18.661318+0000","last_active":"2024-05-30T17:13:20.709354+0000","last_peered":"2024-05-30T17:13:20.709354+0000","last_clean":"2024-05-30T17:13:20.709354+0000","last_became_active":"2024-05-30T15:54:18.660567+0000","last_became_peered":"2024-05-30T15:54:18.660567+0000","last_unstale":"2024-05-30T17:13:20.709354+0000","last_undegraded":"2024-05-30T17:13:20.709354+0000","last_fullsized":"2024-05-30T17:13:20.709354+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":400,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2226,"num_read_kb":2104,"num_write":504,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"265'306","reported_seq":998,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.394785+0000","last_change":"2024-05-30T15:54:16.686321+0000","last_active":"2024-05-30T17:13:38.394785+0000","last_peered":"2024-05-30T17:13:38.394785+0000","last_clean":"2024-05-30T17:13:38.394785+0000","last_became_active":"2024-05-30T15:54:16.685691+0000","last_became_peered":"2024-05-30T15:54:16.685691+0000","last_unstale":"2024-05-30T17:13:38.394785+0000","last_undegraded":"2024-05-30T17:13:38.394785+0000","last_fullsized":"2024-05-30T17:13:38.394785+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":200,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":764,"num_read_kb":1764,"num_write":1253,"num_write_kb":18095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:51.576939+0000","last_change":"2024-05-30T15:54:20.670328+0000","last_active":"2024-05-30T17:11:51.576939+0000","last_peered":"2024-05-30T17:11:51.576939+0000","last_clean":"2024-05-30T17:11:51.576939+0000","last_became_active":"2024-05-30T15:54:20.669751+0000","last_became_peered":"2024-05-30T15:54:20.669751+0000","last_unstale":"2024-05-30T17:11:51.576939+0000","last_undegraded":"2024-05-30T17:11:51.576939+0000","last_fullsized":"2024-05-30T17:11:51.576939+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"265'225","reported_seq":845,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.220663+0000","last_change":"2024-05-30T17:13:59.220776+0000","last_active":"2024-05-30T17:13:59.220663+0000","last_peered":"2024-05-30T17:13:59.220663+0000","last_clean":"2024-05-30T17:13:59.220663+0000","last_became_active":"2024-05-30T15:54:22.672078+0000","last_became_peered":"2024-05-30T15:54:22.672078+0000","last_unstale":"2024-05-30T17:13:59.220663+0000","last_undegraded":"2024-05-30T17:13:59.220663+0000","last_fullsized":"2024-05-30T17:13:59.220663+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'225","last_scrub_stamp":"2024-05-30T17:13:59.220468+0000","last_deep_scrub":"265'225","last_deep_scrub_stamp":"2024-05-30T17:13:59.220468+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.220468+0000","objects_scrubbed":9,"log_size":25,"log_dups_size":200,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T05:11:49.982867+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10553,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":297,"num_read_kb":402,"num_write":186,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":504,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:46.590351+0000","last_change":"2024-05-30T15:54:24.676069+0000","last_active":"2024-05-30T17:11:46.590351+0000","last_peered":"2024-05-30T17:11:46.590351+0000","last_clean":"2024-05-30T17:11:46.590351+0000","last_became_active":"2024-05-30T15:54:24.675467+0000","last_became_peered":"2024-05-30T15:54:24.675467+0000","last_unstale":"2024-05-30T17:11:46.590351+0000","last_undegraded":"2024-05-30T17:11:46.590351+0000","last_fullsized":"2024-05-30T17:11:46.590351+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"265'106","reported_seq":632,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.532393+0000","last_change":"2024-05-30T17:13:58.532471+0000","last_active":"2024-05-30T17:13:58.532393+0000","last_peered":"2024-05-30T17:13:58.532393+0000","last_clean":"2024-05-30T17:13:58.532393+0000","last_became_active":"2024-05-30T15:54:26.705264+0000","last_became_peered":"2024-05-30T15:54:26.705264+0000","last_unstale":"2024-05-30T17:13:58.532393+0000","last_undegraded":"2024-05-30T17:13:58.532393+0000","last_fullsized":"2024-05-30T17:13:58.532393+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'106","last_scrub_stamp":"2024-05-30T17:13:58.532222+0000","last_deep_scrub":"265'106","last_deep_scrub_stamp":"2024-05-30T17:13:58.532222+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.532222+0000","objects_scrubbed":4,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:07:52.286487+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":805,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":151,"num_read_kb":135,"num_write":247,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"265'700","reported_seq":2178,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.780405+0000","last_change":"2024-05-30T17:13:45.780482+0000","last_active":"2024-05-30T17:13:45.780405+0000","last_peered":"2024-05-30T17:13:45.780405+0000","last_clean":"2024-05-30T17:13:45.780405+0000","last_became_active":"2024-05-30T15:54:54.421604+0000","last_became_peered":"2024-05-30T15:54:54.421604+0000","last_unstale":"2024-05-30T17:13:45.780405+0000","last_undegraded":"2024-05-30T17:13:45.780405+0000","last_fullsized":"2024-05-30T17:13:45.780405+0000","mapping_epoch":35,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'700","last_scrub_stamp":"2024-05-30T17:13:45.780204+0000","last_deep_scrub":"265'700","last_deep_scrub_stamp":"2024-05-30T17:13:45.780204+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.780204+0000","objects_scrubbed":2,"log_size":100,"log_dups_size":600,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:24:04.304931+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3204,"num_read_kb":2893,"num_write":920,"num_write_kb":324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"265'173","reported_seq":823,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.218710+0000","last_change":"2024-05-30T17:13:46.218809+0000","last_active":"2024-05-30T17:13:46.218710+0000","last_peered":"2024-05-30T17:13:46.218710+0000","last_clean":"2024-05-30T17:13:46.218710+0000","last_became_active":"2024-05-30T15:54:54.422916+0000","last_became_peered":"2024-05-30T15:54:54.422916+0000","last_unstale":"2024-05-30T17:13:46.218710+0000","last_undegraded":"2024-05-30T17:13:46.218710+0000","last_fullsized":"2024-05-30T17:13:46.218710+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'173","last_scrub_stamp":"2024-05-30T17:13:46.218474+0000","last_deep_scrub":"265'173","last_deep_scrub_stamp":"2024-05-30T17:13:46.218474+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.218474+0000","objects_scrubbed":4,"log_size":73,"log_dups_size":100,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:17:58.546167+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505146,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":405,"num_read_kb":71111,"num_write":971,"num_write_kb":25767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"34'4","reported_seq":832,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:37.482412+0000","last_change":"2024-05-30T15:54:20.652854+0000","last_active":"2024-05-30T17:13:37.482412+0000","last_peered":"2024-05-30T17:13:37.482412+0000","last_clean":"2024-05-30T17:13:37.482412+0000","last_became_active":"2024-05-30T15:54:20.652134+0000","last_became_peered":"2024-05-30T15:54:20.652134+0000","last_unstale":"2024-05-30T17:13:37.482412+0000","last_undegraded":"2024-05-30T17:13:37.482412+0000","last_fullsized":"2024-05-30T17:13:37.482412+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1139,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":493,"num_read_kb":807,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"265'151","reported_seq":745,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.290755+0000","last_change":"2024-05-30T15:54:22.673262+0000","last_active":"2024-05-30T17:13:38.290755+0000","last_peered":"2024-05-30T17:13:38.290755+0000","last_clean":"2024-05-30T17:13:38.290755+0000","last_became_active":"2024-05-30T15:54:22.672169+0000","last_became_peered":"2024-05-30T15:54:22.672169+0000","last_unstale":"2024-05-30T17:13:38.290755+0000","last_undegraded":"2024-05-30T17:13:38.290755+0000","last_fullsized":"2024-05-30T17:13:38.290755+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6731,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":192,"num_read_kb":263,"num_write":102,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:52.740783+0000","last_change":"2024-05-30T17:13:52.740908+0000","last_active":"2024-05-30T17:13:52.740783+0000","last_peered":"2024-05-30T17:13:52.740783+0000","last_clean":"2024-05-30T17:13:52.740783+0000","last_became_active":"2024-05-30T15:54:24.675683+0000","last_became_peered":"2024-05-30T15:54:24.675683+0000","last_unstale":"2024-05-30T17:13:52.740783+0000","last_undegraded":"2024-05-30T17:13:52.740783+0000","last_fullsized":"2024-05-30T17:13:52.740783+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:52.740555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:52.740555+0000","last_clean_scrub_stamp":"2024-05-30T17:13:52.740555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:52:04.474974+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"265'128","reported_seq":665,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:01.345963+0000","last_change":"2024-05-30T17:14:01.346031+0000","last_active":"2024-05-30T17:14:01.345963+0000","last_peered":"2024-05-30T17:14:01.345963+0000","last_clean":"2024-05-30T17:14:01.345963+0000","last_became_active":"2024-05-30T15:54:26.689049+0000","last_became_peered":"2024-05-30T15:54:26.689049+0000","last_unstale":"2024-05-30T17:14:01.345963+0000","last_undegraded":"2024-05-30T17:14:01.345963+0000","last_fullsized":"2024-05-30T17:14:01.345963+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'128","last_scrub_stamp":"2024-05-30T17:14:01.345770+0000","last_deep_scrub":"265'128","last_deep_scrub_stamp":"2024-05-30T17:14:01.345770+0000","last_clean_scrub_stamp":"2024-05-30T17:14:01.345770+0000","objects_scrubbed":1,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:20:24.034017+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":539,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":167,"num_read_kb":166,"num_write":309,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"265'652","reported_seq":2070,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.761573+0000","last_change":"2024-05-30T15:54:18.653246+0000","last_active":"2024-05-30T17:13:18.761573+0000","last_peered":"2024-05-30T17:13:18.761573+0000","last_clean":"2024-05-30T17:13:18.761573+0000","last_became_active":"2024-05-30T15:54:18.651778+0000","last_became_peered":"2024-05-30T15:54:18.651778+0000","last_unstale":"2024-05-30T17:13:18.761573+0000","last_undegraded":"2024-05-30T17:13:18.761573+0000","last_fullsized":"2024-05-30T17:13:18.761573+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":600,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3068,"num_read_kb":2749,"num_write":930,"num_write_kb":367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"265'151","reported_seq":766,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:36.605206+0000","last_change":"2024-05-30T15:54:16.656872+0000","last_active":"2024-05-30T17:12:36.605206+0000","last_peered":"2024-05-30T17:12:36.605206+0000","last_clean":"2024-05-30T17:12:36.605206+0000","last_became_active":"2024-05-30T15:54:16.656081+0000","last_became_peered":"2024-05-30T15:54:16.656081+0000","last_unstale":"2024-05-30T17:12:36.605206+0000","last_undegraded":"2024-05-30T17:12:36.605206+0000","last_fullsized":"2024-05-30T17:12:36.605206+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":369,"num_read_kb":8201,"num_write":842,"num_write_kb":22196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":516,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:41.753697+0000","last_change":"2024-05-30T15:54:20.661844+0000","last_active":"2024-05-30T17:11:41.753697+0000","last_peered":"2024-05-30T17:11:41.753697+0000","last_clean":"2024-05-30T17:11:41.753697+0000","last_became_active":"2024-05-30T15:54:20.661090+0000","last_became_peered":"2024-05-30T15:54:20.661090+0000","last_unstale":"2024-05-30T17:11:41.753697+0000","last_undegraded":"2024-05-30T17:11:41.753697+0000","last_fullsized":"2024-05-30T17:11:41.753697+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:19.621908+0000","last_clean_scrub_stamp":"2024-05-30T15:54:19.621908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"265'116","reported_seq":689,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.119113+0000","last_change":"2024-05-30T15:54:22.670767+0000","last_active":"2024-05-30T17:13:38.119113+0000","last_peered":"2024-05-30T17:13:38.119113+0000","last_clean":"2024-05-30T17:13:38.119113+0000","last_became_active":"2024-05-30T15:54:22.669073+0000","last_became_peered":"2024-05-30T15:54:22.669073+0000","last_unstale":"2024-05-30T17:13:38.119113+0000","last_undegraded":"2024-05-30T17:13:38.119113+0000","last_fullsized":"2024-05-30T17:13:38.119113+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":100,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6915,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":145,"num_read_kb":188,"num_write":74,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":515,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.881973+0000","last_change":"2024-05-30T17:13:53.882025+0000","last_active":"2024-05-30T17:13:53.881973+0000","last_peered":"2024-05-30T17:13:53.881973+0000","last_clean":"2024-05-30T17:13:53.881973+0000","last_became_active":"2024-05-30T15:54:24.678292+0000","last_became_peered":"2024-05-30T15:54:24.678292+0000","last_unstale":"2024-05-30T17:13:53.881973+0000","last_undegraded":"2024-05-30T17:13:53.881973+0000","last_fullsized":"2024-05-30T17:13:53.881973+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:53.881772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:53.881772+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.881772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:31:21.481218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"265'171","reported_seq":917,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.566227+0000","last_change":"2024-05-30T17:13:59.566327+0000","last_active":"2024-05-30T17:13:59.566227+0000","last_peered":"2024-05-30T17:13:59.566227+0000","last_clean":"2024-05-30T17:13:59.566227+0000","last_became_active":"2024-05-30T15:54:26.703440+0000","last_became_peered":"2024-05-30T15:54:26.703440+0000","last_unstale":"2024-05-30T17:13:59.566227+0000","last_undegraded":"2024-05-30T17:13:59.566227+0000","last_fullsized":"2024-05-30T17:13:59.566227+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'171","last_scrub_stamp":"2024-05-30T17:13:59.565988+0000","last_deep_scrub":"265'171","last_deep_scrub_stamp":"2024-05-30T17:13:59.565988+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.565988+0000","objects_scrubbed":7,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:51:56.619593+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1074,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":420,"num_read_kb":450,"num_write":403,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1200,"num_omap_keys":6,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"265'327","reported_seq":1682,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:15.255156+0000","last_change":"2024-05-30T15:54:18.672371+0000","last_active":"2024-05-30T17:12:15.255156+0000","last_peered":"2024-05-30T17:12:15.255156+0000","last_clean":"2024-05-30T17:12:15.255156+0000","last_became_active":"2024-05-30T15:54:18.671650+0000","last_became_peered":"2024-05-30T15:54:18.671650+0000","last_unstale":"2024-05-30T17:12:15.255156+0000","last_undegraded":"2024-05-30T17:12:15.255156+0000","last_fullsized":"2024-05-30T17:12:15.255156+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":300,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1672,"num_read_kb":1605,"num_write":361,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"265'206","reported_seq":868,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.350185+0000","last_change":"2024-05-30T15:54:16.661703+0000","last_active":"2024-05-30T17:13:38.350185+0000","last_peered":"2024-05-30T17:13:38.350185+0000","last_clean":"2024-05-30T17:13:38.350185+0000","last_became_active":"2024-05-30T15:54:16.660393+0000","last_became_peered":"2024-05-30T15:54:16.660393+0000","last_unstale":"2024-05-30T17:13:38.350185+0000","last_undegraded":"2024-05-30T17:13:38.350185+0000","last_fullsized":"2024-05-30T17:13:38.350185+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":100,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":594502,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":519,"num_read_kb":9177,"num_write":1141,"num_write_kb":26248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"265'2","reported_seq":518,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.016000+0000","last_change":"2024-05-30T17:13:58.016076+0000","last_active":"2024-05-30T17:13:58.016000+0000","last_peered":"2024-05-30T17:13:58.016000+0000","last_clean":"2024-05-30T17:13:58.016000+0000","last_became_active":"2024-05-30T15:54:20.661311+0000","last_became_peered":"2024-05-30T15:54:20.661311+0000","last_unstale":"2024-05-30T17:13:58.016000+0000","last_undegraded":"2024-05-30T17:13:58.016000+0000","last_fullsized":"2024-05-30T17:13:58.016000+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:13:58.015783+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:13:58.015783+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.015783+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:05:29.698239+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"265'175","reported_seq":787,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.176879+0000","last_change":"2024-05-30T15:54:22.663985+0000","last_active":"2024-05-30T17:13:38.176879+0000","last_peered":"2024-05-30T17:13:38.176879+0000","last_clean":"2024-05-30T17:13:38.176879+0000","last_became_active":"2024-05-30T15:54:22.663231+0000","last_became_peered":"2024-05-30T15:54:22.663231+0000","last_unstale":"2024-05-30T17:13:38.176879+0000","last_undegraded":"2024-05-30T17:13:38.176879+0000","last_fullsized":"2024-05-30T17:13:38.176879+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":100,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4459,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":234,"num_read_kb":297,"num_write":134,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":503,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:11:56.592922+0000","last_change":"2024-05-30T15:54:24.676660+0000","last_active":"2024-05-30T17:11:56.592922+0000","last_peered":"2024-05-30T17:11:56.592922+0000","last_clean":"2024-05-30T17:11:56.592922+0000","last_became_active":"2024-05-30T15:54:24.673674+0000","last_became_peered":"2024-05-30T15:54:24.673674+0000","last_unstale":"2024-05-30T17:11:56.592922+0000","last_undegraded":"2024-05-30T17:11:56.592922+0000","last_fullsized":"2024-05-30T17:11:56.592922+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:23.638904+0000","last_clean_scrub_stamp":"2024-05-30T15:54:23.638904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"265'114","reported_seq":629,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.822855+0000","last_change":"2024-05-30T17:13:48.822855+0000","last_active":"2024-05-30T17:13:48.822855+0000","last_peered":"2024-05-30T17:13:48.822855+0000","last_clean":"2024-05-30T17:13:48.822855+0000","last_became_active":"2024-05-30T15:54:26.677418+0000","last_became_peered":"2024-05-30T15:54:26.677418+0000","last_unstale":"2024-05-30T17:13:48.822855+0000","last_undegraded":"2024-05-30T17:13:48.822855+0000","last_fullsized":"2024-05-30T17:13:48.822855+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'114","last_scrub_stamp":"2024-05-30T17:13:48.822470+0000","last_deep_scrub":"265'114","last_deep_scrub_stamp":"2024-05-30T17:13:48.822470+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.822470+0000","objects_scrubbed":3,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:49:43.113999+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":435,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":67,"num_write":244,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"265'308","reported_seq":1720,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.056309+0000","last_change":"2024-05-30T17:13:59.056454+0000","last_active":"2024-05-30T17:13:59.056309+0000","last_peered":"2024-05-30T17:13:59.056309+0000","last_clean":"2024-05-30T17:13:59.056309+0000","last_became_active":"2024-05-30T15:54:18.671335+0000","last_became_peered":"2024-05-30T15:54:18.671335+0000","last_unstale":"2024-05-30T17:13:59.056309+0000","last_undegraded":"2024-05-30T17:13:59.056309+0000","last_fullsized":"2024-05-30T17:13:59.056309+0000","mapping_epoch":24,"log_start":"265'200","ondisk_log_start":"265'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'308","last_scrub_stamp":"2024-05-30T17:13:59.056066+0000","last_deep_scrub":"265'308","last_deep_scrub_stamp":"2024-05-30T17:13:59.056066+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.056066+0000","objects_scrubbed":3,"log_size":108,"log_dups_size":200,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:17:28.319253+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1771,"num_read_kb":1705,"num_write":340,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"265'206","reported_seq":859,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.399777+0000","last_change":"2024-05-30T15:54:16.675257+0000","last_active":"2024-05-30T17:13:38.399777+0000","last_peered":"2024-05-30T17:13:38.399777+0000","last_clean":"2024-05-30T17:13:38.399777+0000","last_became_active":"2024-05-30T15:54:16.674678+0000","last_became_peered":"2024-05-30T15:54:16.674678+0000","last_unstale":"2024-05-30T17:13:38.399777+0000","last_undegraded":"2024-05-30T17:13:38.399777+0000","last_fullsized":"2024-05-30T17:13:38.399777+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":100,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25713,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":493,"num_read_kb":10347,"num_write":1036,"num_write_kb":23027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"265'559","reported_seq":2276,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.705055+0000","last_change":"2024-05-30T15:54:18.635401+0000","last_active":"2024-05-30T17:13:20.705055+0000","last_peered":"2024-05-30T17:13:20.705055+0000","last_clean":"2024-05-30T17:13:20.705055+0000","last_became_active":"2024-05-30T15:54:18.635092+0000","last_became_peered":"2024-05-30T15:54:18.635092+0000","last_unstale":"2024-05-30T17:13:20.705055+0000","last_undegraded":"2024-05-30T17:13:20.705055+0000","last_fullsized":"2024-05-30T17:13:20.705055+0000","mapping_epoch":24,"log_start":"265'500","ondisk_log_start":"265'500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":500,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2950,"num_read_kb":2739,"num_write":700,"num_write_kb":188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"265'165","reported_seq":788,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:01.304037+0000","last_change":"2024-05-30T17:14:01.304121+0000","last_active":"2024-05-30T17:14:01.304037+0000","last_peered":"2024-05-30T17:14:01.304037+0000","last_clean":"2024-05-30T17:14:01.304037+0000","last_became_active":"2024-05-30T15:54:16.658183+0000","last_became_peered":"2024-05-30T15:54:16.658183+0000","last_unstale":"2024-05-30T17:14:01.304037+0000","last_undegraded":"2024-05-30T17:14:01.304037+0000","last_fullsized":"2024-05-30T17:14:01.304037+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'165","last_scrub_stamp":"2024-05-30T17:14:01.303807+0000","last_deep_scrub":"265'165","last_deep_scrub_stamp":"2024-05-30T17:14:01.303807+0000","last_clean_scrub_stamp":"2024-05-30T17:14:01.303807+0000","objects_scrubbed":1,"log_size":65,"log_dups_size":100,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:51:27.141500+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":308,"num_read_kb":6656,"num_write":926,"num_write_kb":27882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"265'625","reported_seq":2182,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:21.967948+0000","last_change":"2024-05-30T15:54:18.662889+0000","last_active":"2024-05-30T17:12:21.967948+0000","last_peered":"2024-05-30T17:12:21.967948+0000","last_clean":"2024-05-30T17:12:21.967948+0000","last_became_active":"2024-05-30T15:54:18.662159+0000","last_became_peered":"2024-05-30T15:54:18.662159+0000","last_unstale":"2024-05-30T17:12:21.967948+0000","last_undegraded":"2024-05-30T17:12:21.967948+0000","last_fullsized":"2024-05-30T17:12:21.967948+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":600,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3191,"num_read_kb":2926,"num_write":899,"num_write_kb":342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"265'210","reported_seq":864,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.303697+0000","last_change":"2024-05-30T15:54:16.670343+0000","last_active":"2024-05-30T17:13:38.303697+0000","last_peered":"2024-05-30T17:13:38.303697+0000","last_clean":"2024-05-30T17:13:38.303697+0000","last_became_active":"2024-05-30T15:54:16.652664+0000","last_became_peered":"2024-05-30T15:54:16.652664+0000","last_unstale":"2024-05-30T17:13:38.303697+0000","last_undegraded":"2024-05-30T17:13:38.303697+0000","last_fullsized":"2024-05-30T17:13:38.303697+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":100,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524420,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":451,"num_read_kb":27404,"num_write":1156,"num_write_kb":29020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"265'1297","reported_seq":2933,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:12.347476+0000","last_change":"2024-05-30T15:54:18.643433+0000","last_active":"2024-05-30T17:13:12.347476+0000","last_peered":"2024-05-30T17:13:12.347476+0000","last_clean":"2024-05-30T17:13:12.347476+0000","last_became_active":"2024-05-30T15:54:18.642697+0000","last_became_peered":"2024-05-30T15:54:18.642697+0000","last_unstale":"2024-05-30T17:13:12.347476+0000","last_undegraded":"2024-05-30T17:13:12.347476+0000","last_fullsized":"2024-05-30T17:13:12.347476+0000","mapping_epoch":24,"log_start":"265'1200","ondisk_log_start":"265'1200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":1200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":5484,"num_read_kb":4752,"num_write":1832,"num_write_kb":807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"265'256","reported_seq":990,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.646944+0000","last_change":"2024-05-30T15:54:16.660154+0000","last_active":"2024-05-30T17:13:41.646944+0000","last_peered":"2024-05-30T17:13:41.646944+0000","last_clean":"2024-05-30T17:13:41.646944+0000","last_became_active":"2024-05-30T15:54:16.659589+0000","last_became_peered":"2024-05-30T15:54:16.659589+0000","last_unstale":"2024-05-30T17:13:41.646944+0000","last_undegraded":"2024-05-30T17:13:41.646944+0000","last_fullsized":"2024-05-30T17:13:41.646944+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":200,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3222795,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":793,"num_read_kb":47588,"num_write":1564,"num_write_kb":36318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"265'1101","reported_seq":2682,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.074235+0000","last_change":"2024-05-30T15:54:18.653159+0000","last_active":"2024-05-30T17:13:43.074235+0000","last_peered":"2024-05-30T17:13:43.074235+0000","last_clean":"2024-05-30T17:13:43.074235+0000","last_became_active":"2024-05-30T15:54:18.652794+0000","last_became_peered":"2024-05-30T15:54:18.652794+0000","last_unstale":"2024-05-30T17:13:43.074235+0000","last_undegraded":"2024-05-30T17:13:43.074235+0000","last_fullsized":"2024-05-30T17:13:43.074235+0000","mapping_epoch":24,"log_start":"265'1000","ondisk_log_start":"265'1000","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":1000,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4733,"num_read_kb":4179,"num_write":1497,"num_write_kb":596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"265'225","reported_seq":902,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:52.361991+0000","last_change":"2024-05-30T17:13:52.362088+0000","last_active":"2024-05-30T17:13:52.361991+0000","last_peered":"2024-05-30T17:13:52.361991+0000","last_clean":"2024-05-30T17:13:52.361991+0000","last_became_active":"2024-05-30T15:54:54.451265+0000","last_became_peered":"2024-05-30T15:54:54.451265+0000","last_unstale":"2024-05-30T17:13:52.361991+0000","last_undegraded":"2024-05-30T17:13:52.361991+0000","last_fullsized":"2024-05-30T17:13:52.361991+0000","mapping_epoch":35,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'225","last_scrub_stamp":"2024-05-30T17:13:52.361791+0000","last_deep_scrub":"265'225","last_deep_scrub_stamp":"2024-05-30T17:13:52.361791+0000","last_clean_scrub_stamp":"2024-05-30T17:13:52.361791+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":200,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:41:32.517363+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524357,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":480,"num_read_kb":2675,"num_write":1165,"num_write_kb":43073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"265'385","reported_seq":1949,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:31.779244+0000","last_change":"2024-05-30T15:54:18.671626+0000","last_active":"2024-05-30T17:13:31.779244+0000","last_peered":"2024-05-30T17:13:31.779244+0000","last_clean":"2024-05-30T17:13:31.779244+0000","last_became_active":"2024-05-30T15:54:18.670801+0000","last_became_peered":"2024-05-30T15:54:18.670801+0000","last_unstale":"2024-05-30T17:13:31.779244+0000","last_undegraded":"2024-05-30T17:13:31.779244+0000","last_fullsized":"2024-05-30T17:13:31.779244+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":300,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2215,"num_read_kb":2115,"num_write":443,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"265'222","reported_seq":874,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.052000+0000","last_change":"2024-05-30T15:54:16.651535+0000","last_active":"2024-05-30T17:13:38.052000+0000","last_peered":"2024-05-30T17:13:38.052000+0000","last_clean":"2024-05-30T17:13:38.052000+0000","last_became_active":"2024-05-30T15:54:16.649854+0000","last_became_peered":"2024-05-30T15:54:16.649854+0000","last_unstale":"2024-05-30T17:13:38.052000+0000","last_undegraded":"2024-05-30T17:13:38.052000+0000","last_fullsized":"2024-05-30T17:13:38.052000+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":200,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306443,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":499,"num_read_kb":6249,"num_write":1151,"num_write_kb":30962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"265'405","reported_seq":1926,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:31.616767+0000","last_change":"2024-05-30T15:54:18.653103+0000","last_active":"2024-05-30T17:13:31.616767+0000","last_peered":"2024-05-30T17:13:31.616767+0000","last_clean":"2024-05-30T17:13:31.616767+0000","last_became_active":"2024-05-30T15:54:18.650832+0000","last_became_peered":"2024-05-30T15:54:18.650832+0000","last_unstale":"2024-05-30T17:13:31.616767+0000","last_undegraded":"2024-05-30T17:13:31.616767+0000","last_fullsized":"2024-05-30T17:13:31.616767+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":300,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2240,"num_read_kb":2125,"num_write":507,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"265'187","reported_seq":812,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.378862+0000","last_change":"2024-05-30T15:54:16.685235+0000","last_active":"2024-05-30T17:13:38.378862+0000","last_peered":"2024-05-30T17:13:38.378862+0000","last_clean":"2024-05-30T17:13:38.378862+0000","last_became_active":"2024-05-30T15:54:16.684670+0000","last_became_peered":"2024-05-30T15:54:16.684670+0000","last_unstale":"2024-05-30T17:13:38.378862+0000","last_undegraded":"2024-05-30T17:13:38.378862+0000","last_fullsized":"2024-05-30T17:13:38.378862+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":100,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":254692,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":373,"num_read_kb":34719,"num_write":1090,"num_write_kb":29777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"265'818","reported_seq":2411,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:31.587370+0000","last_change":"2024-05-30T15:54:18.669886+0000","last_active":"2024-05-30T17:13:31.587370+0000","last_peered":"2024-05-30T17:13:31.587370+0000","last_clean":"2024-05-30T17:13:31.587370+0000","last_became_active":"2024-05-30T15:54:18.669105+0000","last_became_peered":"2024-05-30T15:54:18.669105+0000","last_unstale":"2024-05-30T17:13:31.587370+0000","last_undegraded":"2024-05-30T17:13:31.587370+0000","last_fullsized":"2024-05-30T17:13:31.587370+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":800,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3660,"num_read_kb":3297,"num_write":1114,"num_write_kb":434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"265'220","reported_seq":927,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.975897+0000","last_change":"2024-05-30T17:13:48.976145+0000","last_active":"2024-05-30T17:13:48.975897+0000","last_peered":"2024-05-30T17:13:48.975897+0000","last_clean":"2024-05-30T17:13:48.975897+0000","last_became_active":"2024-05-30T15:54:54.416482+0000","last_became_peered":"2024-05-30T15:54:54.416482+0000","last_unstale":"2024-05-30T17:13:48.975897+0000","last_undegraded":"2024-05-30T17:13:48.975897+0000","last_fullsized":"2024-05-30T17:13:48.975897+0000","mapping_epoch":35,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'220","last_scrub_stamp":"2024-05-30T17:13:48.975457+0000","last_deep_scrub":"265'220","last_deep_scrub_stamp":"2024-05-30T17:13:48.975457+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.975457+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":200,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:44:46.059246+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5708449,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":729,"num_read_kb":137610,"num_write":1205,"num_write_kb":64841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2f","version":"265'385","reported_seq":1974,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.617457+0000","last_change":"2024-05-30T15:54:18.657515+0000","last_active":"2024-05-30T17:13:41.617457+0000","last_peered":"2024-05-30T17:13:41.617457+0000","last_clean":"2024-05-30T17:13:41.617457+0000","last_became_active":"2024-05-30T15:54:18.657236+0000","last_became_peered":"2024-05-30T15:54:18.657236+0000","last_unstale":"2024-05-30T17:13:41.617457+0000","last_undegraded":"2024-05-30T17:13:41.617457+0000","last_fullsized":"2024-05-30T17:13:41.617457+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":300,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2258,"num_read_kb":2156,"num_write":442,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"265'203","reported_seq":883,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.051695+0000","last_change":"2024-05-30T15:54:16.661242+0000","last_active":"2024-05-30T17:13:38.051695+0000","last_peered":"2024-05-30T17:13:38.051695+0000","last_clean":"2024-05-30T17:13:38.051695+0000","last_became_active":"2024-05-30T15:54:16.653037+0000","last_became_peered":"2024-05-30T15:54:16.653037+0000","last_unstale":"2024-05-30T17:13:38.051695+0000","last_undegraded":"2024-05-30T17:13:38.051695+0000","last_fullsized":"2024-05-30T17:13:38.051695+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":100,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2108965,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":543,"num_read_kb":29183,"num_write":977,"num_write_kb":32330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"265'330","reported_seq":1829,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.781532+0000","last_change":"2024-05-30T15:54:18.636049+0000","last_active":"2024-05-30T17:13:41.781532+0000","last_peered":"2024-05-30T17:13:41.781532+0000","last_clean":"2024-05-30T17:13:41.781532+0000","last_became_active":"2024-05-30T15:54:18.635382+0000","last_became_peered":"2024-05-30T15:54:18.635382+0000","last_unstale":"2024-05-30T17:13:41.781532+0000","last_undegraded":"2024-05-30T17:13:41.781532+0000","last_fullsized":"2024-05-30T17:13:41.781532+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":300,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2071,"num_read_kb":1981,"num_write":382,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"265'159","reported_seq":777,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.307596+0000","last_change":"2024-05-30T15:54:16.683072+0000","last_active":"2024-05-30T17:13:38.307596+0000","last_peered":"2024-05-30T17:13:38.307596+0000","last_clean":"2024-05-30T17:13:38.307596+0000","last_became_active":"2024-05-30T15:54:16.682629+0000","last_became_peered":"2024-05-30T15:54:16.682629+0000","last_unstale":"2024-05-30T17:13:38.307596+0000","last_undegraded":"2024-05-30T17:13:38.307596+0000","last_fullsized":"2024-05-30T17:13:38.307596+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310739,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":22431,"num_write":822,"num_write_kb":23734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"265'743","reported_seq":2313,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:26.584411+0000","last_change":"2024-05-30T15:54:18.669167+0000","last_active":"2024-05-30T17:13:26.584411+0000","last_peered":"2024-05-30T17:13:26.584411+0000","last_clean":"2024-05-30T17:13:26.584411+0000","last_became_active":"2024-05-30T15:54:18.668748+0000","last_became_peered":"2024-05-30T15:54:18.668748+0000","last_unstale":"2024-05-30T17:13:26.584411+0000","last_undegraded":"2024-05-30T17:13:26.584411+0000","last_fullsized":"2024-05-30T17:13:26.584411+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":700,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3385,"num_read_kb":3066,"num_write":980,"num_write_kb":340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"265'165","reported_seq":901,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.376677+0000","last_change":"2024-05-30T15:54:16.676031+0000","last_active":"2024-05-30T17:13:38.376677+0000","last_peered":"2024-05-30T17:13:38.376677+0000","last_clean":"2024-05-30T17:13:38.376677+0000","last_became_active":"2024-05-30T15:54:16.674993+0000","last_became_peered":"2024-05-30T15:54:16.674993+0000","last_unstale":"2024-05-30T17:13:38.376677+0000","last_undegraded":"2024-05-30T17:13:38.376677+0000","last_fullsized":"2024-05-30T17:13:38.376677+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":100,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6553619,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":567,"num_read_kb":112844,"num_write":925,"num_write_kb":45704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"265'472","reported_seq":2122,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:01.015512+0000","last_change":"2024-05-30T17:14:01.015577+0000","last_active":"2024-05-30T17:14:01.015512+0000","last_peered":"2024-05-30T17:14:01.015512+0000","last_clean":"2024-05-30T17:14:01.015512+0000","last_became_active":"2024-05-30T15:54:18.671434+0000","last_became_peered":"2024-05-30T15:54:18.671434+0000","last_unstale":"2024-05-30T17:14:01.015512+0000","last_undegraded":"2024-05-30T17:14:01.015512+0000","last_fullsized":"2024-05-30T17:14:01.015512+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'472","last_scrub_stamp":"2024-05-30T17:14:01.015328+0000","last_deep_scrub":"265'472","last_deep_scrub_stamp":"2024-05-30T17:14:01.015328+0000","last_clean_scrub_stamp":"2024-05-30T17:14:01.015328+0000","objects_scrubbed":1,"log_size":72,"log_dups_size":400,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:21:04.896268+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2525,"num_read_kb":2390,"num_write":596,"num_write_kb":183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"265'204","reported_seq":859,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.397498+0000","last_change":"2024-05-30T15:54:16.630459+0000","last_active":"2024-05-30T17:13:38.397498+0000","last_peered":"2024-05-30T17:13:38.397498+0000","last_clean":"2024-05-30T17:13:38.397498+0000","last_became_active":"2024-05-30T15:54:16.629843+0000","last_became_peered":"2024-05-30T15:54:16.629843+0000","last_unstale":"2024-05-30T17:13:38.397498+0000","last_undegraded":"2024-05-30T17:13:38.397498+0000","last_fullsized":"2024-05-30T17:13:38.397498+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":100,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524465,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":434,"num_read_kb":42078,"num_write":1152,"num_write_kb":26102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"265'509","reported_seq":2135,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:06.595630+0000","last_change":"2024-05-30T15:54:18.660822+0000","last_active":"2024-05-30T17:13:06.595630+0000","last_peered":"2024-05-30T17:13:06.595630+0000","last_clean":"2024-05-30T17:13:06.595630+0000","last_became_active":"2024-05-30T15:54:18.660101+0000","last_became_peered":"2024-05-30T15:54:18.660101+0000","last_unstale":"2024-05-30T17:13:06.595630+0000","last_undegraded":"2024-05-30T17:13:06.595630+0000","last_fullsized":"2024-05-30T17:13:06.595630+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":400,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2760,"num_read_kb":2546,"num_write":661,"num_write_kb":201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"265'210","reported_seq":846,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.352776+0000","last_change":"2024-05-30T15:54:16.686381+0000","last_active":"2024-05-30T17:13:38.352776+0000","last_peered":"2024-05-30T17:13:38.352776+0000","last_clean":"2024-05-30T17:13:38.352776+0000","last_became_active":"2024-05-30T15:54:16.685969+0000","last_became_peered":"2024-05-30T15:54:16.685969+0000","last_unstale":"2024-05-30T17:13:38.352776+0000","last_undegraded":"2024-05-30T17:13:38.352776+0000","last_fullsized":"2024-05-30T17:13:38.352776+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":100,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1342159,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":419,"num_read_kb":18996,"num_write":1148,"num_write_kb":34757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"265'670","reported_seq":2428,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:06.609100+0000","last_change":"2024-05-30T15:54:18.646120+0000","last_active":"2024-05-30T17:13:06.609100+0000","last_peered":"2024-05-30T17:13:06.609100+0000","last_clean":"2024-05-30T17:13:06.609100+0000","last_became_active":"2024-05-30T15:54:18.645400+0000","last_became_peered":"2024-05-30T15:54:18.645400+0000","last_unstale":"2024-05-30T17:13:06.609100+0000","last_undegraded":"2024-05-30T17:13:06.609100+0000","last_fullsized":"2024-05-30T17:13:06.609100+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":600,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3447,"num_read_kb":3120,"num_write":968,"num_write_kb":434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"265'215","reported_seq":850,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.413515+0000","last_change":"2024-05-30T15:54:16.664235+0000","last_active":"2024-05-30T17:13:38.413515+0000","last_peered":"2024-05-30T17:13:38.413515+0000","last_clean":"2024-05-30T17:13:38.413515+0000","last_became_active":"2024-05-30T15:54:16.663255+0000","last_became_peered":"2024-05-30T15:54:16.663255+0000","last_unstale":"2024-05-30T17:13:38.413515+0000","last_undegraded":"2024-05-30T17:13:38.413515+0000","last_fullsized":"2024-05-30T17:13:38.413515+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":200,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767340,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":429,"num_read_kb":27123,"num_write":1161,"num_write_kb":19484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"265'360","reported_seq":1780,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:12.350089+0000","last_change":"2024-05-30T15:54:18.658517+0000","last_active":"2024-05-30T17:13:12.350089+0000","last_peered":"2024-05-30T17:13:12.350089+0000","last_clean":"2024-05-30T17:13:12.350089+0000","last_became_active":"2024-05-30T15:54:18.658230+0000","last_became_peered":"2024-05-30T15:54:18.658230+0000","last_unstale":"2024-05-30T17:13:12.350089+0000","last_undegraded":"2024-05-30T17:13:12.350089+0000","last_fullsized":"2024-05-30T17:13:12.350089+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":300,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1934,"num_read_kb":1839,"num_write":414,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"265'170","reported_seq":792,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:00.278603+0000","last_change":"2024-05-30T17:14:00.278671+0000","last_active":"2024-05-30T17:14:00.278603+0000","last_peered":"2024-05-30T17:14:00.278603+0000","last_clean":"2024-05-30T17:14:00.278603+0000","last_became_active":"2024-05-30T15:54:16.654365+0000","last_became_peered":"2024-05-30T15:54:16.654365+0000","last_unstale":"2024-05-30T17:14:00.278603+0000","last_undegraded":"2024-05-30T17:14:00.278603+0000","last_fullsized":"2024-05-30T17:14:00.278603+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'170","last_scrub_stamp":"2024-05-30T17:14:00.278386+0000","last_deep_scrub":"265'170","last_deep_scrub_stamp":"2024-05-30T17:14:00.278386+0000","last_clean_scrub_stamp":"2024-05-30T17:14:00.278386+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":100,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:04:58.427288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":358,"num_read_kb":31154,"num_write":915,"num_write_kb":41557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"265'698","reported_seq":2129,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.660064+0000","last_change":"2024-05-30T15:54:18.661710+0000","last_active":"2024-05-30T17:12:44.660064+0000","last_peered":"2024-05-30T17:12:44.660064+0000","last_clean":"2024-05-30T17:12:44.660064+0000","last_became_active":"2024-05-30T15:54:18.660543+0000","last_became_peered":"2024-05-30T15:54:18.660543+0000","last_unstale":"2024-05-30T17:12:44.660064+0000","last_undegraded":"2024-05-30T17:12:44.660064+0000","last_fullsized":"2024-05-30T17:12:44.660064+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":600,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3130,"num_read_kb":2812,"num_write":919,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"265'202","reported_seq":845,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.368900+0000","last_change":"2024-05-30T15:54:16.666386+0000","last_active":"2024-05-30T17:13:38.368900+0000","last_peered":"2024-05-30T17:13:38.368900+0000","last_clean":"2024-05-30T17:13:38.368900+0000","last_became_active":"2024-05-30T15:54:16.661983+0000","last_became_peered":"2024-05-30T15:54:16.661983+0000","last_unstale":"2024-05-30T17:13:38.368900+0000","last_undegraded":"2024-05-30T17:13:38.368900+0000","last_fullsized":"2024-05-30T17:13:38.368900+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718827,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":420,"num_read_kb":7620,"num_write":1155,"num_write_kb":9531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"265'154","reported_seq":793,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.164212+0000","last_change":"2024-05-30T17:13:45.164368+0000","last_active":"2024-05-30T17:13:45.164212+0000","last_peered":"2024-05-30T17:13:45.164212+0000","last_clean":"2024-05-30T17:13:45.164212+0000","last_became_active":"2024-05-30T15:55:54.767665+0000","last_became_peered":"2024-05-30T15:55:54.767665+0000","last_unstale":"2024-05-30T17:13:45.164212+0000","last_undegraded":"2024-05-30T17:13:45.164212+0000","last_fullsized":"2024-05-30T17:13:45.164212+0000","mapping_epoch":37,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'154","last_scrub_stamp":"2024-05-30T17:13:45.163967+0000","last_deep_scrub":"265'154","last_deep_scrub_stamp":"2024-05-30T17:13:45.163967+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.163967+0000","objects_scrubbed":2,"log_size":54,"log_dups_size":100,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:10:48.513574+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5682939,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":360,"num_read_kb":14674,"num_write":890,"num_write_kb":17371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,0],"acting":[5,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"265'765","reported_seq":2420,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:01.594622+0000","last_change":"2024-05-30T15:54:18.660816+0000","last_active":"2024-05-30T17:12:01.594622+0000","last_peered":"2024-05-30T17:12:01.594622+0000","last_clean":"2024-05-30T17:12:01.594622+0000","last_became_active":"2024-05-30T15:54:18.660501+0000","last_became_peered":"2024-05-30T15:54:18.660501+0000","last_unstale":"2024-05-30T17:12:01.594622+0000","last_undegraded":"2024-05-30T17:12:01.594622+0000","last_fullsized":"2024-05-30T17:12:01.594622+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":700,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3837,"num_read_kb":3483,"num_write":1021,"num_write_kb":383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"265'183","reported_seq":835,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.414790+0000","last_change":"2024-05-30T15:54:16.668650+0000","last_active":"2024-05-30T17:13:38.414790+0000","last_peered":"2024-05-30T17:13:38.414790+0000","last_clean":"2024-05-30T17:13:38.414790+0000","last_became_active":"2024-05-30T15:54:16.668178+0000","last_became_peered":"2024-05-30T15:54:16.668178+0000","last_unstale":"2024-05-30T17:13:38.414790+0000","last_undegraded":"2024-05-30T17:13:38.414790+0000","last_fullsized":"2024-05-30T17:13:38.414790+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":100,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3976850,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":427,"num_read_kb":57498,"num_write":1000,"num_write_kb":21705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"265'474","reported_seq":2145,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:20.704606+0000","last_change":"2024-05-30T15:54:18.656743+0000","last_active":"2024-05-30T17:13:20.704606+0000","last_peered":"2024-05-30T17:13:20.704606+0000","last_clean":"2024-05-30T17:13:20.704606+0000","last_became_active":"2024-05-30T15:54:18.655872+0000","last_became_peered":"2024-05-30T15:54:18.655872+0000","last_unstale":"2024-05-30T17:13:20.704606+0000","last_undegraded":"2024-05-30T17:13:20.704606+0000","last_fullsized":"2024-05-30T17:13:20.704606+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":400,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2571,"num_read_kb":2440,"num_write":587,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5999,"num_read_kb":4309,"num_write":2135,"num_write_kb":880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6954,"num_omap_keys":14,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1142,"ondisk_log_size":1142,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19930,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":13226,"num_read_kb":13032,"num_write":15987,"num_write_kb":4690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3721,"num_omap_keys":22,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":704512,"data_stored":39860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1657,"ondisk_log_size":1657,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":331,"ondisk_log_size":331,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":279892,"num_objects":197,"num_object_clones":0,"num_object_copies":394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":197,"num_whiteouts":0,"num_read":9562,"num_read_kb":11753,"num_write":6685,"num_write_kb":2848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1780,"num_omap_keys":6,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1972,"ondisk_log_size":1972,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1634,"num_read_kb":1946,"num_write":20,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14,"ondisk_log_size":14,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":242,"num_object_clones":0,"num_object_copies":484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":242,"num_whiteouts":0,"num_read":191181,"num_read_kb":174247,"num_write":51900,"num_write_kb":17923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":242,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":310,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4130,"ondisk_log_size":4130,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":157354006,"num_objects":261,"num_object_clones":0,"num_object_copies":783,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":261,"num_whiteouts":0,"num_read":32072,"num_read_kb":2124808,"num_write":69910,"num_write_kb":2069422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":238202880,"data_stored":238202880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4424,"ondisk_log_size":4424,"up":192,"acting":192,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":135,"num_read_kb":182367,"num_write":151,"num_write_kb":122722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":133,"num_read_kb":181098,"num_write":151,"num_write_kb":122722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575815,"num_pgs":106,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300188,"kb_used_data":38548,"kb_used_omap":1,"kb_used_meta":261630,"kb_avail":94071652,"statfs":{"total":96636764160,"available":96329371648,"internally_reserved":0,"allocated":39473152,"data_stored":37452492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":267909578},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575814,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":289728,"kb_used_data":37816,"kb_used_omap":1,"kb_used_meta":251902,"kb_avail":94082112,"statfs":{"total":96636764160,"available":96340082688,"internally_reserved":0,"allocated":38723584,"data_stored":36776739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":257948102},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575814,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":133816,"kb_used_data":48496,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94238024,"statfs":{"total":96636764160,"available":96499736576,"internally_reserved":0,"allocated":49659904,"data_stored":47630081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87357893},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575814,"num_pgs":104,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":142452,"kb_used_data":44652,"kb_used_omap":1,"kb_used_meta":97790,"kb_avail":94229388,"statfs":{"total":96636764160,"available":96490893312,"internally_reserved":0,"allocated":45723648,"data_stored":43786666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":100137414},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575812,"num_pgs":124,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":269128,"kb_used_data":43584,"kb_used_omap":1,"kb_used_meta":225534,"kb_avail":94102712,"statfs":{"total":96636764160,"available":96361177088,"internally_reserved":0,"allocated":44630016,"data_stored":42616238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":230947271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575812,"num_pgs":91,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":107084,"kb_used_data":35652,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94264756,"statfs":{"total":96636764160,"available":96527110144,"internally_reserved":0,"allocated":36507648,"data_stored":34568865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575816,"num_pgs":112,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":164064,"kb_used_data":46872,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94207776,"statfs":{"total":96636764160,"available":96468762624,"internally_reserved":0,"allocated":47996928,"data_stored":46030586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575811,"num_pgs":109,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266824,"kb_used_data":37120,"kb_used_omap":1,"kb_used_meta":229694,"kb_avail":94105016,"statfs":{"total":96636764160,"available":96363536384,"internally_reserved":0,"allocated":38010880,"data_stored":36012935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":235207109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":27881472,"data_stored":27881472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":37441536,"data_stored":37441536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27029504,"data_stored":27029504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24588288,"data_stored":24588288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":31006720,"data_stored":31006720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":33796096,"data_stored":33796096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":32382976,"data_stored":32382976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24076288,"data_stored":24076288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":4787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":5121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":4696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":6691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":5459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":3918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":7193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T17:14:05.918 INFO:tasks.ceph:pgid 68.1a last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.918 INFO:tasks.ceph:pgid 67.1d last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.918 INFO:tasks.ceph:pgid 68.1b last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.918 INFO:tasks.ceph:pgid 67.1c last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.919 INFO:tasks.ceph:pgid 68.18 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.919 INFO:tasks.ceph:pgid 67.1f last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.919 INFO:tasks.ceph:pgid 66.1e last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.919 INFO:tasks.ceph:pgid 68.19 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.919 INFO:tasks.ceph:pgid 67.1e last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.919 INFO:tasks.ceph:pgid 68.1e last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.920 INFO:tasks.ceph:pgid 67.19 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.920 INFO:tasks.ceph:pgid 68.1f last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.920 INFO:tasks.ceph:pgid 67.18 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.920 INFO:tasks.ceph:pgid 68.1c last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.920 INFO:tasks.ceph:pgid 67.1b last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.920 INFO:tasks.ceph:pgid 66.1a last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.920 INFO:tasks.ceph:pgid 68.1d last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.921 INFO:tasks.ceph:pgid 68.12 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.921 INFO:tasks.ceph:pgid 67.15 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.921 INFO:tasks.ceph:pgid 68.13 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.921 INFO:tasks.ceph:pgid 67.14 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.921 INFO:tasks.ceph:pgid 68.10 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.922 INFO:tasks.ceph:pgid 67.17 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.922 INFO:tasks.ceph:pgid 66.16 last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.922 INFO:tasks.ceph:pgid 68.11 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.922 INFO:tasks.ceph:pgid 67.16 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.922 INFO:tasks.ceph:pgid 68.16 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.922 INFO:tasks.ceph:pgid 67.11 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.923 INFO:tasks.ceph:pgid 68.17 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.923 INFO:tasks.ceph:pgid 68.14 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.923 INFO:tasks.ceph:pgid 67.13 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.923 INFO:tasks.ceph:pgid 66.12 last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.923 INFO:tasks.ceph:pgid 68.15 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.923 INFO:tasks.ceph:pgid 67.12 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.924 INFO:tasks.ceph:pgid 68.a last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.924 INFO:tasks.ceph:pgid 68.b last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.924 INFO:tasks.ceph:pgid 67.c last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.924 INFO:tasks.ceph:pgid 66.d last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.924 INFO:tasks.ceph:pgid 68.8 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.924 INFO:tasks.ceph:pgid 67.f last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.925 INFO:tasks.ceph:pgid 68.9 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.925 INFO:tasks.ceph:pgid 68.e last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.925 INFO:tasks.ceph:pgid 67.9 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.925 INFO:tasks.ceph:pgid 68.f last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.925 INFO:tasks.ceph:pgid 68.c last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.926 INFO:tasks.ceph:pgid 67.b last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.926 INFO:tasks.ceph:pgid 66.a last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.926 INFO:tasks.ceph:pgid 68.d last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.926 INFO:tasks.ceph:pgid 68.2 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.926 INFO:tasks.ceph:pgid 67.5 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.927 INFO:tasks.ceph:pgid 68.3 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.927 INFO:tasks.ceph:pgid 67.4 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.927 INFO:tasks.ceph:pgid 66.5 last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.927 INFO:tasks.ceph:pgid 68.0 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.927 INFO:tasks.ceph:pgid 68.1 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.927 INFO:tasks.ceph:pgid 67.6 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.928 INFO:tasks.ceph:pgid 68.6 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.928 INFO:tasks.ceph:pgid 67.1 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.928 INFO:tasks.ceph:pgid 68.7 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.928 INFO:tasks.ceph:pgid 67.0 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.929 INFO:tasks.ceph:pgid 68.4 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.929 INFO:tasks.ceph:pgid 67.3 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.929 INFO:tasks.ceph:pgid 68.5 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.929 INFO:tasks.ceph:pgid 67.2 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.929 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.930 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.930 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.930 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.930 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.930 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.930 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.931 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.931 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.931 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.931 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.931 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.931 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.932 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.932 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.932 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.932 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.932 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.932 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.933 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.933 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.933 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.933 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.933 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.933 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.934 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.934 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.934 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.934 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.934 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.934 INFO:tasks.ceph:pgid 6.16 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.935 INFO:tasks.ceph:pgid 7.17 last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.935 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.935 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.935 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.935 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.935 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.936 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.936 INFO:tasks.ceph:pgid 7.15 last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.936 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.936 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.936 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.936 INFO:tasks.ceph:pgid 6.15 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.937 INFO:tasks.ceph:pgid 7.14 last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.937 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.937 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.937 INFO:tasks.ceph:pgid 5.9 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.937 INFO:tasks.ceph:pgid 7.b last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.937 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.938 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.938 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.938 INFO:tasks.ceph:pgid 7.a last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.938 INFO:tasks.ceph:pgid 8.5 last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.938 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.939 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.939 INFO:tasks.ceph:pgid 6.8 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.939 INFO:tasks.ceph:pgid 7.9 last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.939 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.939 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.939 INFO:tasks.ceph:pgid 6.9 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.940 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.940 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.940 INFO:tasks.ceph:pgid 6.e last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.941 INFO:tasks.ceph:pgid 7.f last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.941 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.941 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.941 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.942 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.942 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.942 INFO:tasks.ceph:pgid 6.c last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.942 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.942 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.943 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-30T15:54:01.512251+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=1, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.943 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-05-30T15:54:07.550093+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=7, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.943 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.943 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.943 INFO:tasks.ceph:pgid 6.7 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.944 INFO:tasks.ceph:pgid 8.9 last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.944 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.944 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-05-30T15:54:07.550093+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=7, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.944 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.944 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.944 INFO:tasks.ceph:pgid 6.4 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.945 INFO:tasks.ceph:pgid 7.5 last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.945 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.945 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.945 INFO:tasks.ceph:pgid 6.5 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.945 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.945 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-05-30T15:54:07.550093+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=7, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.946 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.946 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.946 INFO:tasks.ceph:pgid 6.6 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.946 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.946 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.946 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-05-30T15:54:07.550093+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=7, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.947 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.947 INFO:tasks.ceph:pgid 6.0 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.947 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.947 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.947 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-05-30T15:54:07.550093+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=7, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.947 INFO:tasks.ceph:pgid 6.1 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.948 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.948 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.948 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-05-30T15:54:07.550093+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=7, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.948 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.948 INFO:tasks.ceph:pgid 7.3 last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.948 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.949 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.949 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-05-30T15:54:07.550093+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=7, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.949 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.949 INFO:tasks.ceph:pgid 6.3 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.949 INFO:tasks.ceph:pgid 7.2 last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.949 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.950 INFO:tasks.ceph:pgid 6.11 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.950 INFO:tasks.ceph:pgid 7.10 last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.950 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.950 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.950 INFO:tasks.ceph:pgid 6.10 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.950 INFO:tasks.ceph:pgid 7.11 last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.951 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.951 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.951 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.951 INFO:tasks.ceph:pgid 6.13 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.951 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.951 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.952 INFO:tasks.ceph:pgid 6.12 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.952 INFO:tasks.ceph:pgid 7.13 last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.952 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.952 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.952 INFO:tasks.ceph:pgid 6.1d last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.952 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.953 INFO:tasks.ceph:pgid 6.1c last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.953 INFO:tasks.ceph:pgid 7.1d last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.953 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.953 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.953 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.954 INFO:tasks.ceph:pgid 6.1f last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.954 INFO:tasks.ceph:pgid 7.1e last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.954 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.954 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2024-05-30T15:54:25.645629+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=25, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.954 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.954 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.955 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.955 INFO:tasks.ceph:pgid 7.18 last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.955 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.955 INFO:tasks.ceph:pgid 6.18 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.955 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.956 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.956 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2024-05-30T15:54:19.621908+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=19, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.956 INFO:tasks.ceph:pgid 6.1b last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.956 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.956 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.956 INFO:tasks.ceph:pgid 6.1a last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.957 INFO:tasks.ceph:pgid 7.1b last_scrub_stamp 2024-05-30T15:54:23.638904+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=23, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.957 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.957 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.957 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.957 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.957 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.958 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.958 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.958 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.958 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.958 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.958 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.959 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.959 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.959 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.959 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.959 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.959 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.960 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.960 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.960 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.960 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.960 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.960 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.961 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.961 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.961 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.961 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.961 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.961 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:05.962 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-30T17:14:25.963 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T17:14:26.330 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:14:26.341 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-05-30T17:14:26.349 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":2668,"stamp":"2024-05-30T17:14:24.473067+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":191800421,"num_objects":837,"num_object_clones":0,"num_object_copies":1935,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":837,"num_whiteouts":0,"num_read":254020,"num_read_kb":2693627,"num_write":147058,"num_write_kb":2342860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":289,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":599488,"num_omap_keys":1456,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13837,"ondisk_log_size":13837,"up":850,"acting":850,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":850,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1673284,"kb_used_data":332740,"kb_used_omap":12,"kb_used_meta":1340467,"kb_avail":753301436,"statfs":{"total":773094113280,"available":771380670464,"internally_reserved":0,"allocated":340725760,"data_stored":324874602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":1372638769},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":28143,"num_omap_keys":91,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009546"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:26.969133+0000","last_change":"2024-05-30T16:34:55.869930+0000","last_active":"2024-05-30T17:12:26.969133+0000","last_peered":"2024-05-30T17:12:26.969133+0000","last_clean":"2024-05-30T17:12:26.969133+0000","last_became_active":"2024-05-30T16:34:55.869226+0000","last_became_peered":"2024-05-30T16:34:55.869226+0000","last_unstale":"2024-05-30T17:12:26.969133+0000","last_undegraded":"2024-05-30T17:12:26.969133+0000","last_fullsized":"2024-05-30T17:12:26.969133+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"265'2","reported_seq":49,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.284428+0000","last_change":"2024-05-30T16:17:35.249630+0000","last_active":"2024-05-30T17:13:38.284428+0000","last_peered":"2024-05-30T17:13:38.284428+0000","last_clean":"2024-05-30T17:13:38.284428+0000","last_became_active":"2024-05-30T16:17:35.248740+0000","last_became_peered":"2024-05-30T16:17:35.248740+0000","last_unstale":"2024-05-30T17:13:38.284428+0000","last_undegraded":"2024-05-30T17:13:38.284428+0000","last_fullsized":"2024-05-30T17:13:38.284428+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"265'8","reported_seq":75,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.443531+0000","last_change":"2024-05-30T17:13:43.443594+0000","last_active":"2024-05-30T17:13:43.443531+0000","last_peered":"2024-05-30T17:13:43.443531+0000","last_clean":"2024-05-30T17:13:43.443531+0000","last_became_active":"2024-05-30T16:17:31.183349+0000","last_became_peered":"2024-05-30T16:17:31.183349+0000","last_unstale":"2024-05-30T17:13:43.443531+0000","last_undegraded":"2024-05-30T17:13:43.443531+0000","last_fullsized":"2024-05-30T17:13:43.443531+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'8","last_scrub_stamp":"2024-05-30T17:13:43.443352+0000","last_deep_scrub":"265'8","last_deep_scrub_stamp":"2024-05-30T17:13:43.443352+0000","last_clean_scrub_stamp":"2024-05-30T17:13:43.443352+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:57:10.662914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":16384,"num_write":8,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"265'45","reported_seq":193,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.684557+0000","last_change":"2024-05-30T17:13:48.684645+0000","last_active":"2024-05-30T17:13:48.684557+0000","last_peered":"2024-05-30T17:13:48.684557+0000","last_clean":"2024-05-30T17:13:48.684557+0000","last_became_active":"2024-05-30T16:12:50.803352+0000","last_became_peered":"2024-05-30T16:12:50.803352+0000","last_unstale":"2024-05-30T17:13:48.684557+0000","last_undegraded":"2024-05-30T17:13:48.684557+0000","last_fullsized":"2024-05-30T17:13:48.684557+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'45","last_scrub_stamp":"2024-05-30T17:13:48.684344+0000","last_deep_scrub":"265'45","last_deep_scrub_stamp":"2024-05-30T17:13:48.684344+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.684344+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:56:59.286905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":225,"num_read_kb":156,"num_write":101,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":36,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:14.491141+0000","last_change":"2024-05-30T17:14:14.491199+0000","last_active":"2024-05-30T17:14:14.491141+0000","last_peered":"2024-05-30T17:14:14.491141+0000","last_clean":"2024-05-30T17:14:14.491141+0000","last_became_active":"2024-05-30T16:34:55.886731+0000","last_became_peered":"2024-05-30T16:34:55.886731+0000","last_unstale":"2024-05-30T17:14:14.491141+0000","last_undegraded":"2024-05-30T17:14:14.491141+0000","last_fullsized":"2024-05-30T17:14:14.491141+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:14.490948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:14.490948+0000","last_clean_scrub_stamp":"2024-05-30T17:14:14.490948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:54:47.154526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"0'0","reported_seq":45,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:36.766484+0000","last_change":"2024-05-30T16:17:35.248384+0000","last_active":"2024-05-30T17:12:36.766484+0000","last_peered":"2024-05-30T17:12:36.766484+0000","last_clean":"2024-05-30T17:12:36.766484+0000","last_became_active":"2024-05-30T16:17:35.247810+0000","last_became_peered":"2024-05-30T16:17:35.247810+0000","last_unstale":"2024-05-30T17:12:36.766484+0000","last_undegraded":"2024-05-30T17:12:36.766484+0000","last_fullsized":"2024-05-30T17:12:36.766484+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"0'0","reported_seq":59,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.953095+0000","last_change":"2024-05-30T17:13:45.953152+0000","last_active":"2024-05-30T17:13:45.953095+0000","last_peered":"2024-05-30T17:13:45.953095+0000","last_clean":"2024-05-30T17:13:45.953095+0000","last_became_active":"2024-05-30T16:17:31.212337+0000","last_became_peered":"2024-05-30T16:17:31.212337+0000","last_unstale":"2024-05-30T17:13:45.953095+0000","last_undegraded":"2024-05-30T17:13:45.953095+0000","last_fullsized":"2024-05-30T17:13:45.953095+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:45.952903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:45.952903+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.952903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:31:16.835383+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"265'55","reported_seq":219,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:55.177804+0000","last_change":"2024-05-30T17:13:55.177870+0000","last_active":"2024-05-30T17:13:55.177804+0000","last_peered":"2024-05-30T17:13:55.177804+0000","last_clean":"2024-05-30T17:13:55.177804+0000","last_became_active":"2024-05-30T16:12:50.810207+0000","last_became_peered":"2024-05-30T16:12:50.810207+0000","last_unstale":"2024-05-30T17:13:55.177804+0000","last_undegraded":"2024-05-30T17:13:55.177804+0000","last_fullsized":"2024-05-30T17:13:55.177804+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'55","last_scrub_stamp":"2024-05-30T17:13:55.177616+0000","last_deep_scrub":"265'55","last_deep_scrub_stamp":"2024-05-30T17:13:55.177616+0000","last_clean_scrub_stamp":"2024-05-30T17:13:55.177616+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:10:07.203727+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":277,"num_read_kb":197,"num_write":99,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:36.588447+0000","last_change":"2024-05-30T16:34:55.876451+0000","last_active":"2024-05-30T17:13:36.588447+0000","last_peered":"2024-05-30T17:13:36.588447+0000","last_clean":"2024-05-30T17:13:36.588447+0000","last_became_active":"2024-05-30T16:34:55.874359+0000","last_became_peered":"2024-05-30T16:34:55.874359+0000","last_unstale":"2024-05-30T17:13:36.588447+0000","last_undegraded":"2024-05-30T17:13:36.588447+0000","last_fullsized":"2024-05-30T17:13:36.588447+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"0'0","reported_seq":50,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.648243+0000","last_change":"2024-05-30T16:17:35.743986+0000","last_active":"2024-05-30T17:13:41.648243+0000","last_peered":"2024-05-30T17:13:41.648243+0000","last_clean":"2024-05-30T17:13:41.648243+0000","last_became_active":"2024-05-30T16:17:35.743231+0000","last_became_peered":"2024-05-30T16:17:35.743231+0000","last_unstale":"2024-05-30T17:13:41.648243+0000","last_undegraded":"2024-05-30T17:13:41.648243+0000","last_fullsized":"2024-05-30T17:13:41.648243+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"265'3","reported_seq":70,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:09.442558+0000","last_change":"2024-05-30T17:14:09.442626+0000","last_active":"2024-05-30T17:14:09.442558+0000","last_peered":"2024-05-30T17:14:09.442558+0000","last_clean":"2024-05-30T17:14:09.442558+0000","last_became_active":"2024-05-30T16:17:31.204806+0000","last_became_peered":"2024-05-30T16:17:31.204806+0000","last_unstale":"2024-05-30T17:14:09.442558+0000","last_undegraded":"2024-05-30T17:14:09.442558+0000","last_fullsized":"2024-05-30T17:14:09.442558+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:14:09.442363+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:14:09.442363+0000","last_clean_scrub_stamp":"2024-05-30T17:14:09.442363+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:38:40.295126+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"265'29","reported_seq":143,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.662503+0000","last_change":"2024-05-30T17:13:46.662582+0000","last_active":"2024-05-30T17:13:46.662503+0000","last_peered":"2024-05-30T17:13:46.662503+0000","last_clean":"2024-05-30T17:13:46.662503+0000","last_became_active":"2024-05-30T16:12:50.810062+0000","last_became_peered":"2024-05-30T16:12:50.810062+0000","last_unstale":"2024-05-30T17:13:46.662503+0000","last_undegraded":"2024-05-30T17:13:46.662503+0000","last_fullsized":"2024-05-30T17:13:46.662503+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'29","last_scrub_stamp":"2024-05-30T17:13:46.662271+0000","last_deep_scrub":"265'29","last_deep_scrub_stamp":"2024-05-30T17:13:46.662271+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.662271+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:14:37.786974+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":98,"num_write":65,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.075706+0000","last_change":"2024-05-30T16:34:55.888292+0000","last_active":"2024-05-30T17:13:43.075706+0000","last_peered":"2024-05-30T17:13:43.075706+0000","last_clean":"2024-05-30T17:13:43.075706+0000","last_became_active":"2024-05-30T16:34:55.887532+0000","last_became_peered":"2024-05-30T16:34:55.887532+0000","last_unstale":"2024-05-30T17:13:43.075706+0000","last_undegraded":"2024-05-30T17:13:43.075706+0000","last_fullsized":"2024-05-30T17:13:43.075706+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"265'2","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.075519+0000","last_change":"2024-05-30T16:17:35.746712+0000","last_active":"2024-05-30T17:13:43.075519+0000","last_peered":"2024-05-30T17:13:43.075519+0000","last_clean":"2024-05-30T17:13:43.075519+0000","last_became_active":"2024-05-30T16:17:35.745709+0000","last_became_peered":"2024-05-30T16:17:35.745709+0000","last_unstale":"2024-05-30T17:13:43.075519+0000","last_undegraded":"2024-05-30T17:13:43.075519+0000","last_fullsized":"2024-05-30T17:13:43.075519+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1142,"num_write":2,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1f","version":"265'3","reported_seq":67,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.831674+0000","last_change":"2024-05-30T17:13:50.831741+0000","last_active":"2024-05-30T17:13:50.831674+0000","last_peered":"2024-05-30T17:13:50.831674+0000","last_clean":"2024-05-30T17:13:50.831674+0000","last_became_active":"2024-05-30T16:17:31.182344+0000","last_became_peered":"2024-05-30T16:17:31.182344+0000","last_unstale":"2024-05-30T17:13:50.831674+0000","last_undegraded":"2024-05-30T17:13:50.831674+0000","last_fullsized":"2024-05-30T17:13:50.831674+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:13:50.831396+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:13:50.831396+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.831396+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:36:02.576103+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"260'6","reported_seq":85,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.454253+0000","last_change":"2024-05-30T17:13:46.454380+0000","last_active":"2024-05-30T17:13:46.454253+0000","last_peered":"2024-05-30T17:13:46.454253+0000","last_clean":"2024-05-30T17:13:46.454253+0000","last_became_active":"2024-05-30T16:12:50.826974+0000","last_became_peered":"2024-05-30T16:12:50.826974+0000","last_unstale":"2024-05-30T17:13:46.454253+0000","last_undegraded":"2024-05-30T17:13:46.454253+0000","last_fullsized":"2024-05-30T17:13:46.454253+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'6","last_scrub_stamp":"2024-05-30T17:13:46.453944+0000","last_deep_scrub":"260'6","last_deep_scrub_stamp":"2024-05-30T17:13:46.453944+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.453944+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:05:54.138613+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":24,"num_write":11,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":32,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:06.611451+0000","last_change":"2024-05-30T16:34:55.899516+0000","last_active":"2024-05-30T17:13:06.611451+0000","last_peered":"2024-05-30T17:13:06.611451+0000","last_clean":"2024-05-30T17:13:06.611451+0000","last_became_active":"2024-05-30T16:34:55.898864+0000","last_became_peered":"2024-05-30T16:34:55.898864+0000","last_unstale":"2024-05-30T17:13:06.611451+0000","last_undegraded":"2024-05-30T17:13:06.611451+0000","last_fullsized":"2024-05-30T17:13:06.611451+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"265'7","reported_seq":64,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.067411+0000","last_change":"2024-05-30T16:17:35.233543+0000","last_active":"2024-05-30T17:13:38.067411+0000","last_peered":"2024-05-30T17:13:38.067411+0000","last_clean":"2024-05-30T17:13:38.067411+0000","last_became_active":"2024-05-30T16:17:35.232744+0000","last_became_peered":"2024-05-30T16:17:35.232744+0000","last_unstale":"2024-05-30T17:13:38.067411+0000","last_undegraded":"2024-05-30T17:13:38.067411+0000","last_fullsized":"2024-05-30T17:13:38.067411+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":23554,"num_write":6,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"0'0","reported_seq":60,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.680121+0000","last_change":"2024-05-30T17:13:43.680163+0000","last_active":"2024-05-30T17:13:43.680121+0000","last_peered":"2024-05-30T17:13:43.680121+0000","last_clean":"2024-05-30T17:13:43.680121+0000","last_became_active":"2024-05-30T16:17:31.186229+0000","last_became_peered":"2024-05-30T16:17:31.186229+0000","last_unstale":"2024-05-30T17:13:43.680121+0000","last_undegraded":"2024-05-30T17:13:43.680121+0000","last_fullsized":"2024-05-30T17:13:43.680121+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:43.679999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:43.679999+0000","last_clean_scrub_stamp":"2024-05-30T17:13:43.679999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:30:30.079637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"265'27","reported_seq":143,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.491839+0000","last_change":"2024-05-30T17:13:45.491914+0000","last_active":"2024-05-30T17:13:45.491839+0000","last_peered":"2024-05-30T17:13:45.491839+0000","last_clean":"2024-05-30T17:13:45.491839+0000","last_became_active":"2024-05-30T16:12:50.827008+0000","last_became_peered":"2024-05-30T16:12:50.827008+0000","last_unstale":"2024-05-30T17:13:45.491839+0000","last_undegraded":"2024-05-30T17:13:45.491839+0000","last_fullsized":"2024-05-30T17:13:45.491839+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'27","last_scrub_stamp":"2024-05-30T17:13:45.491648+0000","last_deep_scrub":"265'27","last_deep_scrub_stamp":"2024-05-30T17:13:45.491648+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.491648+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:27:36.651992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":100,"num_write":57,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":31,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:26.806691+0000","last_change":"2024-05-30T16:34:55.870968+0000","last_active":"2024-05-30T17:12:26.806691+0000","last_peered":"2024-05-30T17:12:26.806691+0000","last_clean":"2024-05-30T17:12:26.806691+0000","last_became_active":"2024-05-30T16:34:55.870139+0000","last_became_peered":"2024-05-30T16:34:55.870139+0000","last_unstale":"2024-05-30T17:12:26.806691+0000","last_undegraded":"2024-05-30T17:12:26.806691+0000","last_fullsized":"2024-05-30T17:12:26.806691+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"265'8","reported_seq":67,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:16.849562+0000","last_change":"2024-05-30T17:14:16.849722+0000","last_active":"2024-05-30T17:14:16.849562+0000","last_peered":"2024-05-30T17:14:16.849562+0000","last_clean":"2024-05-30T17:14:16.849562+0000","last_became_active":"2024-05-30T16:17:35.233040+0000","last_became_peered":"2024-05-30T16:17:35.233040+0000","last_unstale":"2024-05-30T17:14:16.849562+0000","last_undegraded":"2024-05-30T17:14:16.849562+0000","last_fullsized":"2024-05-30T17:14:16.849562+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'8","last_scrub_stamp":"2024-05-30T17:14:16.849340+0000","last_deep_scrub":"265'8","last_deep_scrub_stamp":"2024-05-30T17:14:16.849340+0000","last_clean_scrub_stamp":"2024-05-30T17:14:16.849340+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:56:30.841879+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":12336,"num_write":13,"num_write_kb":11312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"265'3","reported_seq":65,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.804589+0000","last_change":"2024-05-30T17:13:46.804678+0000","last_active":"2024-05-30T17:13:46.804589+0000","last_peered":"2024-05-30T17:13:46.804589+0000","last_clean":"2024-05-30T17:13:46.804589+0000","last_became_active":"2024-05-30T16:17:31.173900+0000","last_became_peered":"2024-05-30T16:17:31.173900+0000","last_unstale":"2024-05-30T17:13:46.804589+0000","last_undegraded":"2024-05-30T17:13:46.804589+0000","last_fullsized":"2024-05-30T17:13:46.804589+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:13:46.804390+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:13:46.804390+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.804390+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:37:20.477514+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"265'75","reported_seq":275,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.777415+0000","last_change":"2024-05-30T17:13:49.777540+0000","last_active":"2024-05-30T17:13:49.777415+0000","last_peered":"2024-05-30T17:13:49.777415+0000","last_clean":"2024-05-30T17:13:49.777415+0000","last_became_active":"2024-05-30T16:12:50.815032+0000","last_became_peered":"2024-05-30T16:12:50.815032+0000","last_unstale":"2024-05-30T17:13:49.777415+0000","last_undegraded":"2024-05-30T17:13:49.777415+0000","last_fullsized":"2024-05-30T17:13:49.777415+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'75","last_scrub_stamp":"2024-05-30T17:13:49.777169+0000","last_deep_scrub":"265'75","last_deep_scrub_stamp":"2024-05-30T17:13:49.777169+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.777169+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:42:13.290083+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":396,"num_read_kb":277,"num_write":137,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"265'1","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:26.585548+0000","last_change":"2024-05-30T16:34:55.882343+0000","last_active":"2024-05-30T17:12:26.585548+0000","last_peered":"2024-05-30T17:12:26.585548+0000","last_clean":"2024-05-30T17:12:26.585548+0000","last_became_active":"2024-05-30T16:34:55.881414+0000","last_became_peered":"2024-05-30T16:34:55.881414+0000","last_unstale":"2024-05-30T17:12:26.585548+0000","last_undegraded":"2024-05-30T17:12:26.585548+0000","last_fullsized":"2024-05-30T17:12:26.585548+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"262'2","reported_seq":57,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:17.859737+0000","last_change":"2024-05-30T17:14:17.859865+0000","last_active":"2024-05-30T17:14:17.859737+0000","last_peered":"2024-05-30T17:14:17.859737+0000","last_clean":"2024-05-30T17:14:17.859737+0000","last_became_active":"2024-05-30T16:17:35.237424+0000","last_became_peered":"2024-05-30T16:17:35.237424+0000","last_unstale":"2024-05-30T17:14:17.859737+0000","last_undegraded":"2024-05-30T17:14:17.859737+0000","last_fullsized":"2024-05-30T17:14:17.859737+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'2","last_scrub_stamp":"2024-05-30T17:14:17.859457+0000","last_deep_scrub":"262'2","last_deep_scrub_stamp":"2024-05-30T17:14:17.859457+0000","last_clean_scrub_stamp":"2024-05-30T17:14:17.859457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:38:38.589711+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"0'0","reported_seq":51,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:31.765322+0000","last_change":"2024-05-30T16:17:31.192981+0000","last_active":"2024-05-30T17:12:31.765322+0000","last_peered":"2024-05-30T17:12:31.765322+0000","last_clean":"2024-05-30T17:12:31.765322+0000","last_became_active":"2024-05-30T16:17:31.192269+0000","last_became_peered":"2024-05-30T16:17:31.192269+0000","last_unstale":"2024-05-30T17:12:31.765322+0000","last_undegraded":"2024-05-30T17:12:31.765322+0000","last_fullsized":"2024-05-30T17:12:31.765322+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:30.145887+0000","last_clean_scrub_stamp":"2024-05-30T16:17:30.145887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"265'25","reported_seq":134,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.688810+0000","last_change":"2024-05-30T17:13:45.688906+0000","last_active":"2024-05-30T17:13:45.688810+0000","last_peered":"2024-05-30T17:13:45.688810+0000","last_clean":"2024-05-30T17:13:45.688810+0000","last_became_active":"2024-05-30T16:12:50.811732+0000","last_became_peered":"2024-05-30T16:12:50.811732+0000","last_unstale":"2024-05-30T17:13:45.688810+0000","last_undegraded":"2024-05-30T17:13:45.688810+0000","last_fullsized":"2024-05-30T17:13:45.688810+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'25","last_scrub_stamp":"2024-05-30T17:13:45.688602+0000","last_deep_scrub":"265'25","last_deep_scrub_stamp":"2024-05-30T17:13:45.688602+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.688602+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:20:25.477039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":85,"num_write":53,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:26.585664+0000","last_change":"2024-05-30T16:34:55.899801+0000","last_active":"2024-05-30T17:12:26.585664+0000","last_peered":"2024-05-30T17:12:26.585664+0000","last_clean":"2024-05-30T17:12:26.585664+0000","last_became_active":"2024-05-30T16:34:55.899241+0000","last_became_peered":"2024-05-30T16:34:55.899241+0000","last_unstale":"2024-05-30T17:12:26.585664+0000","last_undegraded":"2024-05-30T17:12:26.585664+0000","last_fullsized":"2024-05-30T17:12:26.585664+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"0'0","reported_seq":53,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:55.376659+0000","last_change":"2024-05-30T17:13:55.376721+0000","last_active":"2024-05-30T17:13:55.376659+0000","last_peered":"2024-05-30T17:13:55.376659+0000","last_clean":"2024-05-30T17:13:55.376659+0000","last_became_active":"2024-05-30T16:17:35.229034+0000","last_became_peered":"2024-05-30T16:17:35.229034+0000","last_unstale":"2024-05-30T17:13:55.376659+0000","last_undegraded":"2024-05-30T17:13:55.376659+0000","last_fullsized":"2024-05-30T17:13:55.376659+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:55.376450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:55.376450+0000","last_clean_scrub_stamp":"2024-05-30T17:13:55.376450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:44:03.105415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"265'9","reported_seq":82,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.700949+0000","last_change":"2024-05-30T17:13:47.701025+0000","last_active":"2024-05-30T17:13:47.700949+0000","last_peered":"2024-05-30T17:13:47.700949+0000","last_clean":"2024-05-30T17:13:47.700949+0000","last_became_active":"2024-05-30T16:17:31.190742+0000","last_became_peered":"2024-05-30T16:17:31.190742+0000","last_unstale":"2024-05-30T17:13:47.700949+0000","last_undegraded":"2024-05-30T17:13:47.700949+0000","last_fullsized":"2024-05-30T17:13:47.700949+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'9","last_scrub_stamp":"2024-05-30T17:13:47.700729+0000","last_deep_scrub":"265'9","last_deep_scrub_stamp":"2024-05-30T17:13:47.700729+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.700729+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:50:23.027430+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":30722,"num_write":17,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"265'33","reported_seq":152,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.992243+0000","last_change":"2024-05-30T17:13:44.992374+0000","last_active":"2024-05-30T17:13:44.992243+0000","last_peered":"2024-05-30T17:13:44.992243+0000","last_clean":"2024-05-30T17:13:44.992243+0000","last_became_active":"2024-05-30T16:12:50.820343+0000","last_became_peered":"2024-05-30T16:12:50.820343+0000","last_unstale":"2024-05-30T17:13:44.992243+0000","last_undegraded":"2024-05-30T17:13:44.992243+0000","last_fullsized":"2024-05-30T17:13:44.992243+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'33","last_scrub_stamp":"2024-05-30T17:13:44.992004+0000","last_deep_scrub":"265'33","last_deep_scrub_stamp":"2024-05-30T17:13:44.992004+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.992004+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:18:13.774581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":154,"num_read_kb":114,"num_write":68,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:16.803864+0000","last_change":"2024-05-30T16:34:55.889437+0000","last_active":"2024-05-30T17:12:16.803864+0000","last_peered":"2024-05-30T17:12:16.803864+0000","last_clean":"2024-05-30T17:12:16.803864+0000","last_became_active":"2024-05-30T16:34:55.888238+0000","last_became_peered":"2024-05-30T16:34:55.888238+0000","last_unstale":"2024-05-30T17:12:16.803864+0000","last_undegraded":"2024-05-30T17:12:16.803864+0000","last_fullsized":"2024-05-30T17:12:16.803864+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"262'6","reported_seq":65,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:11.410160+0000","last_change":"2024-05-30T17:14:11.410224+0000","last_active":"2024-05-30T17:14:11.410160+0000","last_peered":"2024-05-30T17:14:11.410160+0000","last_clean":"2024-05-30T17:14:11.410160+0000","last_became_active":"2024-05-30T16:17:35.738420+0000","last_became_peered":"2024-05-30T16:17:35.738420+0000","last_unstale":"2024-05-30T17:14:11.410160+0000","last_undegraded":"2024-05-30T17:14:11.410160+0000","last_fullsized":"2024-05-30T17:14:11.410160+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'6","last_scrub_stamp":"2024-05-30T17:14:11.409961+0000","last_deep_scrub":"262'6","last_deep_scrub_stamp":"2024-05-30T17:14:11.409961+0000","last_clean_scrub_stamp":"2024-05-30T17:14:11.409961+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:59:36.378123+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6144,"num_write":16,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"265'5","reported_seq":70,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.196417+0000","last_change":"2024-05-30T17:13:56.196484+0000","last_active":"2024-05-30T17:13:56.196417+0000","last_peered":"2024-05-30T17:13:56.196417+0000","last_clean":"2024-05-30T17:13:56.196417+0000","last_became_active":"2024-05-30T16:17:31.195031+0000","last_became_peered":"2024-05-30T16:17:31.195031+0000","last_unstale":"2024-05-30T17:13:56.196417+0000","last_undegraded":"2024-05-30T17:13:56.196417+0000","last_fullsized":"2024-05-30T17:13:56.196417+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'5","last_scrub_stamp":"2024-05-30T17:13:56.196222+0000","last_deep_scrub":"265'5","last_deep_scrub_stamp":"2024-05-30T17:13:56.196222+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.196222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:04:32.713858+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6715,"num_write":5,"num_write_kb":6715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"265'65","reported_seq":246,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:42.646313+0000","last_change":"2024-05-30T17:13:42.646346+0000","last_active":"2024-05-30T17:13:42.646313+0000","last_peered":"2024-05-30T17:13:42.646313+0000","last_clean":"2024-05-30T17:13:42.646313+0000","last_became_active":"2024-05-30T16:12:50.807771+0000","last_became_peered":"2024-05-30T16:12:50.807771+0000","last_unstale":"2024-05-30T17:13:42.646313+0000","last_undegraded":"2024-05-30T17:13:42.646313+0000","last_fullsized":"2024-05-30T17:13:42.646313+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'65","last_scrub_stamp":"2024-05-30T17:13:42.646208+0000","last_deep_scrub":"265'65","last_deep_scrub_stamp":"2024-05-30T17:13:42.646208+0000","last_clean_scrub_stamp":"2024-05-30T17:13:42.646208+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:19:09.986519+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":343,"num_read_kb":242,"num_write":116,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"265'2","reported_seq":39,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:13.464154+0000","last_change":"2024-05-30T17:14:13.464224+0000","last_active":"2024-05-30T17:14:13.464154+0000","last_peered":"2024-05-30T17:14:13.464154+0000","last_clean":"2024-05-30T17:14:13.464154+0000","last_became_active":"2024-05-30T16:34:55.872581+0000","last_became_peered":"2024-05-30T16:34:55.872581+0000","last_unstale":"2024-05-30T17:14:13.464154+0000","last_undegraded":"2024-05-30T17:14:13.464154+0000","last_fullsized":"2024-05-30T17:14:13.464154+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:14:13.463958+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:14:13.463958+0000","last_clean_scrub_stamp":"2024-05-30T17:14:13.463958+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:33:38.750355+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"0'0","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:07.429700+0000","last_change":"2024-05-30T17:14:07.429850+0000","last_active":"2024-05-30T17:14:07.429700+0000","last_peered":"2024-05-30T17:14:07.429700+0000","last_clean":"2024-05-30T17:14:07.429700+0000","last_became_active":"2024-05-30T16:17:35.738397+0000","last_became_peered":"2024-05-30T16:17:35.738397+0000","last_unstale":"2024-05-30T17:14:07.429700+0000","last_undegraded":"2024-05-30T17:14:07.429700+0000","last_fullsized":"2024-05-30T17:14:07.429700+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:07.429474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:07.429474+0000","last_clean_scrub_stamp":"2024-05-30T17:14:07.429474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:01:13.797785+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"265'3","reported_seq":65,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.455165+0000","last_change":"2024-05-30T17:13:49.455225+0000","last_active":"2024-05-30T17:13:49.455165+0000","last_peered":"2024-05-30T17:13:49.455165+0000","last_clean":"2024-05-30T17:13:49.455165+0000","last_became_active":"2024-05-30T16:17:31.196345+0000","last_became_peered":"2024-05-30T16:17:31.196345+0000","last_unstale":"2024-05-30T17:13:49.455165+0000","last_undegraded":"2024-05-30T17:13:49.455165+0000","last_fullsized":"2024-05-30T17:13:49.455165+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:13:49.454958+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:13:49.454958+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.454958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:19:39.781936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"265'38","reported_seq":168,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.437795+0000","last_change":"2024-05-30T17:13:49.437862+0000","last_active":"2024-05-30T17:13:49.437795+0000","last_peered":"2024-05-30T17:13:49.437795+0000","last_clean":"2024-05-30T17:13:49.437795+0000","last_became_active":"2024-05-30T16:12:50.826729+0000","last_became_peered":"2024-05-30T16:12:50.826729+0000","last_unstale":"2024-05-30T17:13:49.437795+0000","last_undegraded":"2024-05-30T17:13:49.437795+0000","last_fullsized":"2024-05-30T17:13:49.437795+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'38","last_scrub_stamp":"2024-05-30T17:13:49.437536+0000","last_deep_scrub":"265'38","last_deep_scrub_stamp":"2024-05-30T17:13:49.437536+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.437536+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:58:24.201516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":187,"num_read_kb":132,"num_write":81,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":37,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:04.725568+0000","last_change":"2024-05-30T17:14:04.725681+0000","last_active":"2024-05-30T17:14:04.725568+0000","last_peered":"2024-05-30T17:14:04.725568+0000","last_clean":"2024-05-30T17:14:04.725568+0000","last_became_active":"2024-05-30T16:34:55.875788+0000","last_became_peered":"2024-05-30T16:34:55.875788+0000","last_unstale":"2024-05-30T17:14:04.725568+0000","last_undegraded":"2024-05-30T17:14:04.725568+0000","last_fullsized":"2024-05-30T17:14:04.725568+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:04.725336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:04.725336+0000","last_clean_scrub_stamp":"2024-05-30T17:14:04.725336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:32:51.275635+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"0'0","reported_seq":46,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:11.612707+0000","last_change":"2024-05-30T16:17:35.746542+0000","last_active":"2024-05-30T17:13:11.612707+0000","last_peered":"2024-05-30T17:13:11.612707+0000","last_clean":"2024-05-30T17:13:11.612707+0000","last_became_active":"2024-05-30T16:17:35.745769+0000","last_became_peered":"2024-05-30T16:17:35.745769+0000","last_unstale":"2024-05-30T17:13:11.612707+0000","last_undegraded":"2024-05-30T17:13:11.612707+0000","last_fullsized":"2024-05-30T17:13:11.612707+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"265'2","reported_seq":63,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:10.398735+0000","last_change":"2024-05-30T17:14:10.398878+0000","last_active":"2024-05-30T17:14:10.398735+0000","last_peered":"2024-05-30T17:14:10.398735+0000","last_clean":"2024-05-30T17:14:10.398735+0000","last_became_active":"2024-05-30T16:17:31.204434+0000","last_became_peered":"2024-05-30T16:17:31.204434+0000","last_unstale":"2024-05-30T17:14:10.398735+0000","last_undegraded":"2024-05-30T17:14:10.398735+0000","last_fullsized":"2024-05-30T17:14:10.398735+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:14:10.398504+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:14:10.398504+0000","last_clean_scrub_stamp":"2024-05-30T17:14:10.398504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:25:37.977228+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"265'26","reported_seq":134,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:57.206156+0000","last_change":"2024-05-30T17:13:57.206337+0000","last_active":"2024-05-30T17:13:57.206156+0000","last_peered":"2024-05-30T17:13:57.206156+0000","last_clean":"2024-05-30T17:13:57.206156+0000","last_became_active":"2024-05-30T16:12:50.813176+0000","last_became_peered":"2024-05-30T16:12:50.813176+0000","last_unstale":"2024-05-30T17:13:57.206156+0000","last_undegraded":"2024-05-30T17:13:57.206156+0000","last_fullsized":"2024-05-30T17:13:57.206156+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'26","last_scrub_stamp":"2024-05-30T17:13:57.205944+0000","last_deep_scrub":"265'26","last_deep_scrub_stamp":"2024-05-30T17:13:57.205944+0000","last_clean_scrub_stamp":"2024-05-30T17:13:57.205944+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:09:07.523880+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":85,"num_write":57,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:16.966656+0000","last_change":"2024-05-30T16:34:55.872909+0000","last_active":"2024-05-30T17:12:16.966656+0000","last_peered":"2024-05-30T17:12:16.966656+0000","last_clean":"2024-05-30T17:12:16.966656+0000","last_became_active":"2024-05-30T16:34:55.871800+0000","last_became_peered":"2024-05-30T16:34:55.871800+0000","last_unstale":"2024-05-30T17:12:16.966656+0000","last_undegraded":"2024-05-30T17:12:16.966656+0000","last_fullsized":"2024-05-30T17:12:16.966656+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"265'2","reported_seq":49,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.284154+0000","last_change":"2024-05-30T16:17:35.237595+0000","last_active":"2024-05-30T17:13:38.284154+0000","last_peered":"2024-05-30T17:13:38.284154+0000","last_clean":"2024-05-30T17:13:38.284154+0000","last_became_active":"2024-05-30T16:17:35.236867+0000","last_became_peered":"2024-05-30T16:17:35.236867+0000","last_unstale":"2024-05-30T17:13:38.284154+0000","last_undegraded":"2024-05-30T17:13:38.284154+0000","last_fullsized":"2024-05-30T17:13:38.284154+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":571,"num_write":2,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"0'0","reported_seq":60,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.914149+0000","last_change":"2024-05-30T17:13:59.914262+0000","last_active":"2024-05-30T17:13:59.914149+0000","last_peered":"2024-05-30T17:13:59.914149+0000","last_clean":"2024-05-30T17:13:59.914149+0000","last_became_active":"2024-05-30T16:17:31.209882+0000","last_became_peered":"2024-05-30T16:17:31.209882+0000","last_unstale":"2024-05-30T17:13:59.914149+0000","last_undegraded":"2024-05-30T17:13:59.914149+0000","last_fullsized":"2024-05-30T17:13:59.914149+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:59.913913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:59.913913+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.913913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:14:43.780738+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"265'56","reported_seq":228,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.549397+0000","last_change":"2024-05-30T17:13:54.549516+0000","last_active":"2024-05-30T17:13:54.549397+0000","last_peered":"2024-05-30T17:13:54.549397+0000","last_clean":"2024-05-30T17:13:54.549397+0000","last_became_active":"2024-05-30T16:12:50.824043+0000","last_became_peered":"2024-05-30T16:12:50.824043+0000","last_unstale":"2024-05-30T17:13:54.549397+0000","last_undegraded":"2024-05-30T17:13:54.549397+0000","last_fullsized":"2024-05-30T17:13:54.549397+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'56","last_scrub_stamp":"2024-05-30T17:13:54.549107+0000","last_deep_scrub":"265'56","last_deep_scrub_stamp":"2024-05-30T17:13:54.549107+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.549107+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:55:26.307978+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":308,"num_read_kb":221,"num_write":92,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:36.618123+0000","last_change":"2024-05-30T16:34:55.893966+0000","last_active":"2024-05-30T17:13:36.618123+0000","last_peered":"2024-05-30T17:13:36.618123+0000","last_clean":"2024-05-30T17:13:36.618123+0000","last_became_active":"2024-05-30T16:34:55.893040+0000","last_became_peered":"2024-05-30T16:34:55.893040+0000","last_unstale":"2024-05-30T17:13:36.618123+0000","last_undegraded":"2024-05-30T17:13:36.618123+0000","last_fullsized":"2024-05-30T17:13:36.618123+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"0'0","reported_seq":47,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:36.821391+0000","last_change":"2024-05-30T16:17:35.246948+0000","last_active":"2024-05-30T17:13:36.821391+0000","last_peered":"2024-05-30T17:13:36.821391+0000","last_clean":"2024-05-30T17:13:36.821391+0000","last_became_active":"2024-05-30T16:17:35.246187+0000","last_became_peered":"2024-05-30T16:17:35.246187+0000","last_unstale":"2024-05-30T17:13:36.821391+0000","last_undegraded":"2024-05-30T17:13:36.821391+0000","last_fullsized":"2024-05-30T17:13:36.821391+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"265'8","reported_seq":80,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.895849+0000","last_change":"2024-05-30T17:13:54.895979+0000","last_active":"2024-05-30T17:13:54.895849+0000","last_peered":"2024-05-30T17:13:54.895849+0000","last_clean":"2024-05-30T17:13:54.895849+0000","last_became_active":"2024-05-30T16:17:31.189091+0000","last_became_peered":"2024-05-30T16:17:31.189091+0000","last_unstale":"2024-05-30T17:13:54.895849+0000","last_undegraded":"2024-05-30T17:13:54.895849+0000","last_fullsized":"2024-05-30T17:13:54.895849+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'8","last_scrub_stamp":"2024-05-30T17:13:54.895517+0000","last_deep_scrub":"265'8","last_deep_scrub_stamp":"2024-05-30T17:13:54.895517+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.895517+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:48:19.231935+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":21053,"num_write":6,"num_write_kb":5691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"265'9","reported_seq":93,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:55.595931+0000","last_change":"2024-05-30T17:13:55.595991+0000","last_active":"2024-05-30T17:13:55.595931+0000","last_peered":"2024-05-30T17:13:55.595931+0000","last_clean":"2024-05-30T17:13:55.595931+0000","last_became_active":"2024-05-30T16:12:50.823759+0000","last_became_peered":"2024-05-30T16:12:50.823759+0000","last_unstale":"2024-05-30T17:13:55.595931+0000","last_undegraded":"2024-05-30T17:13:55.595931+0000","last_fullsized":"2024-05-30T17:13:55.595931+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'9","last_scrub_stamp":"2024-05-30T17:13:55.595737+0000","last_deep_scrub":"265'9","last_deep_scrub_stamp":"2024-05-30T17:13:55.595737+0000","last_clean_scrub_stamp":"2024-05-30T17:13:55.595737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:06:46.930141+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":33,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:36.616253+0000","last_change":"2024-05-30T16:34:55.861418+0000","last_active":"2024-05-30T17:13:36.616253+0000","last_peered":"2024-05-30T17:13:36.616253+0000","last_clean":"2024-05-30T17:13:36.616253+0000","last_became_active":"2024-05-30T16:34:55.860695+0000","last_became_peered":"2024-05-30T16:34:55.860695+0000","last_unstale":"2024-05-30T17:13:36.616253+0000","last_undegraded":"2024-05-30T17:13:36.616253+0000","last_fullsized":"2024-05-30T17:13:36.616253+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"0'0","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.981887+0000","last_change":"2024-05-30T17:13:53.981997+0000","last_active":"2024-05-30T17:13:53.981887+0000","last_peered":"2024-05-30T17:13:53.981887+0000","last_clean":"2024-05-30T17:13:53.981887+0000","last_became_active":"2024-05-30T16:17:35.239157+0000","last_became_peered":"2024-05-30T16:17:35.239157+0000","last_unstale":"2024-05-30T17:13:53.981887+0000","last_undegraded":"2024-05-30T17:13:53.981887+0000","last_fullsized":"2024-05-30T17:13:53.981887+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:53.981644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:53.981644+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.981644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:55:41.438700+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"265'2","reported_seq":64,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.438167+0000","last_change":"2024-05-30T17:13:47.438232+0000","last_active":"2024-05-30T17:13:47.438167+0000","last_peered":"2024-05-30T17:13:47.438167+0000","last_clean":"2024-05-30T17:13:47.438167+0000","last_became_active":"2024-05-30T16:17:31.184675+0000","last_became_peered":"2024-05-30T16:17:31.184675+0000","last_unstale":"2024-05-30T17:13:47.438167+0000","last_undegraded":"2024-05-30T17:13:47.438167+0000","last_fullsized":"2024-05-30T17:13:47.438167+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:13:47.437969+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:13:47.437969+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.437969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:58:28.886391+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":698,"num_write":2,"num_write_kb":698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"265'97","reported_seq":367,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.658941+0000","last_change":"2024-05-30T17:13:49.659011+0000","last_active":"2024-05-30T17:13:49.658941+0000","last_peered":"2024-05-30T17:13:49.658941+0000","last_clean":"2024-05-30T17:13:49.658941+0000","last_became_active":"2024-05-30T16:12:50.801791+0000","last_became_peered":"2024-05-30T16:12:50.801791+0000","last_unstale":"2024-05-30T17:13:49.658941+0000","last_undegraded":"2024-05-30T17:13:49.658941+0000","last_fullsized":"2024-05-30T17:13:49.658941+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'97","last_scrub_stamp":"2024-05-30T17:13:49.658745+0000","last_deep_scrub":"265'97","last_deep_scrub_stamp":"2024-05-30T17:13:49.658745+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.658745+0000","objects_scrubbed":1,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:10:57.618551+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":591,"num_read_kb":446,"num_write":104,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2994,"num_omap_keys":6,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"264'1","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:26.600681+0000","last_change":"2024-05-30T16:34:55.864834+0000","last_active":"2024-05-30T17:12:26.600681+0000","last_peered":"2024-05-30T17:12:26.600681+0000","last_clean":"2024-05-30T17:12:26.600681+0000","last_became_active":"2024-05-30T16:34:55.863483+0000","last_became_peered":"2024-05-30T16:34:55.863483+0000","last_unstale":"2024-05-30T17:12:26.600681+0000","last_undegraded":"2024-05-30T17:12:26.600681+0000","last_fullsized":"2024-05-30T17:12:26.600681+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"0'0","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:03.388657+0000","last_change":"2024-05-30T17:14:03.388717+0000","last_active":"2024-05-30T17:14:03.388657+0000","last_peered":"2024-05-30T17:14:03.388657+0000","last_clean":"2024-05-30T17:14:03.388657+0000","last_became_active":"2024-05-30T16:17:35.235283+0000","last_became_peered":"2024-05-30T16:17:35.235283+0000","last_unstale":"2024-05-30T17:14:03.388657+0000","last_undegraded":"2024-05-30T17:14:03.388657+0000","last_fullsized":"2024-05-30T17:14:03.388657+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:03.388441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:03.388441+0000","last_clean_scrub_stamp":"2024-05-30T17:14:03.388441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:18:19.298873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"0'0","reported_seq":61,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:15.820628+0000","last_change":"2024-05-30T17:14:15.820726+0000","last_active":"2024-05-30T17:14:15.820628+0000","last_peered":"2024-05-30T17:14:15.820628+0000","last_clean":"2024-05-30T17:14:15.820628+0000","last_became_active":"2024-05-30T16:17:31.199499+0000","last_became_peered":"2024-05-30T16:17:31.199499+0000","last_unstale":"2024-05-30T17:14:15.820628+0000","last_undegraded":"2024-05-30T17:14:15.820628+0000","last_fullsized":"2024-05-30T17:14:15.820628+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:15.820409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:15.820409+0000","last_clean_scrub_stamp":"2024-05-30T17:14:15.820409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:58:15.762012+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"265'22","reported_seq":127,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.758712+0000","last_change":"2024-05-30T17:13:44.758825+0000","last_active":"2024-05-30T17:13:44.758712+0000","last_peered":"2024-05-30T17:13:44.758712+0000","last_clean":"2024-05-30T17:13:44.758712+0000","last_became_active":"2024-05-30T16:12:50.807156+0000","last_became_peered":"2024-05-30T16:12:50.807156+0000","last_unstale":"2024-05-30T17:13:44.758712+0000","last_undegraded":"2024-05-30T17:13:44.758712+0000","last_fullsized":"2024-05-30T17:13:44.758712+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'22","last_scrub_stamp":"2024-05-30T17:13:44.758408+0000","last_deep_scrub":"265'22","last_deep_scrub_stamp":"2024-05-30T17:13:44.758408+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.758408+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:20:24.643496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":76,"num_write":50,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"265'1","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:36.618076+0000","last_change":"2024-05-30T16:34:55.889162+0000","last_active":"2024-05-30T17:13:36.618076+0000","last_peered":"2024-05-30T17:13:36.618076+0000","last_clean":"2024-05-30T17:13:36.618076+0000","last_became_active":"2024-05-30T16:34:55.887845+0000","last_became_peered":"2024-05-30T16:34:55.887845+0000","last_unstale":"2024-05-30T17:13:36.618076+0000","last_undegraded":"2024-05-30T17:13:36.618076+0000","last_fullsized":"2024-05-30T17:13:36.618076+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"0'0","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:19.832379+0000","last_change":"2024-05-30T17:14:19.832435+0000","last_active":"2024-05-30T17:14:19.832379+0000","last_peered":"2024-05-30T17:14:19.832379+0000","last_clean":"2024-05-30T17:14:19.832379+0000","last_became_active":"2024-05-30T16:17:35.245852+0000","last_became_peered":"2024-05-30T16:17:35.245852+0000","last_unstale":"2024-05-30T17:14:19.832379+0000","last_undegraded":"2024-05-30T17:14:19.832379+0000","last_fullsized":"2024-05-30T17:14:19.832379+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:19.832182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:19.832182+0000","last_clean_scrub_stamp":"2024-05-30T17:14:19.832182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:18:45.511193+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"265'6","reported_seq":77,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:00.249278+0000","last_change":"2024-05-30T17:14:00.249347+0000","last_active":"2024-05-30T17:14:00.249278+0000","last_peered":"2024-05-30T17:14:00.249278+0000","last_clean":"2024-05-30T17:14:00.249278+0000","last_became_active":"2024-05-30T16:17:31.218752+0000","last_became_peered":"2024-05-30T16:17:31.218752+0000","last_unstale":"2024-05-30T17:14:00.249278+0000","last_undegraded":"2024-05-30T17:14:00.249278+0000","last_fullsized":"2024-05-30T17:14:00.249278+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'6","last_scrub_stamp":"2024-05-30T17:14:00.249094+0000","last_deep_scrub":"265'6","last_deep_scrub_stamp":"2024-05-30T17:14:00.249094+0000","last_clean_scrub_stamp":"2024-05-30T17:14:00.249094+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:10:46.604501+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"265'17","reported_seq":112,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.434952+0000","last_change":"2024-05-30T17:13:48.435015+0000","last_active":"2024-05-30T17:13:48.434952+0000","last_peered":"2024-05-30T17:13:48.434952+0000","last_clean":"2024-05-30T17:13:48.434952+0000","last_became_active":"2024-05-30T16:12:50.817114+0000","last_became_peered":"2024-05-30T16:12:50.817114+0000","last_unstale":"2024-05-30T17:13:48.434952+0000","last_undegraded":"2024-05-30T17:13:48.434952+0000","last_fullsized":"2024-05-30T17:13:48.434952+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'17","last_scrub_stamp":"2024-05-30T17:13:48.434761+0000","last_deep_scrub":"265'17","last_deep_scrub_stamp":"2024-05-30T17:13:48.434761+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.434761+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:55.692466+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":55,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":39,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:12.439006+0000","last_change":"2024-05-30T17:14:12.439066+0000","last_active":"2024-05-30T17:14:12.439006+0000","last_peered":"2024-05-30T17:14:12.439006+0000","last_clean":"2024-05-30T17:14:12.439006+0000","last_became_active":"2024-05-30T16:34:55.866415+0000","last_became_peered":"2024-05-30T16:34:55.866415+0000","last_unstale":"2024-05-30T17:14:12.439006+0000","last_undegraded":"2024-05-30T17:14:12.439006+0000","last_fullsized":"2024-05-30T17:14:12.439006+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:12.438804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:12.438804+0000","last_clean_scrub_stamp":"2024-05-30T17:14:12.438804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:30:46.541635+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"0'0","reported_seq":56,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.361463+0000","last_change":"2024-05-30T17:13:51.361463+0000","last_active":"2024-05-30T17:13:51.361463+0000","last_peered":"2024-05-30T17:13:51.361463+0000","last_clean":"2024-05-30T17:13:51.361463+0000","last_became_active":"2024-05-30T16:17:35.227270+0000","last_became_peered":"2024-05-30T16:17:35.227270+0000","last_unstale":"2024-05-30T17:13:51.361463+0000","last_undegraded":"2024-05-30T17:13:51.361463+0000","last_fullsized":"2024-05-30T17:13:51.361463+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:51.361210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:51.361210+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.361210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:13:33.987563+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"0'0","reported_seq":62,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.503842+0000","last_change":"2024-05-30T17:13:45.503842+0000","last_active":"2024-05-30T17:13:45.503842+0000","last_peered":"2024-05-30T17:13:45.503842+0000","last_clean":"2024-05-30T17:13:45.503842+0000","last_became_active":"2024-05-30T16:17:31.203731+0000","last_became_peered":"2024-05-30T16:17:31.203731+0000","last_unstale":"2024-05-30T17:13:45.503842+0000","last_undegraded":"2024-05-30T17:13:45.503842+0000","last_fullsized":"2024-05-30T17:13:45.503842+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:45.503591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:45.503591+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.503591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:19:48.974417+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"265'29","reported_seq":146,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.392583+0000","last_change":"2024-05-30T17:13:50.392583+0000","last_active":"2024-05-30T17:13:50.392583+0000","last_peered":"2024-05-30T17:13:50.392583+0000","last_clean":"2024-05-30T17:13:50.392583+0000","last_became_active":"2024-05-30T16:12:50.824809+0000","last_became_peered":"2024-05-30T16:12:50.824809+0000","last_unstale":"2024-05-30T17:13:50.392583+0000","last_undegraded":"2024-05-30T17:13:50.392583+0000","last_fullsized":"2024-05-30T17:13:50.392583+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'29","last_scrub_stamp":"2024-05-30T17:13:50.392236+0000","last_deep_scrub":"265'29","last_deep_scrub_stamp":"2024-05-30T17:13:50.392236+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.392236+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:32:06.254235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":98,"num_write":77,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:11.596984+0000","last_change":"2024-05-30T16:34:55.897061+0000","last_active":"2024-05-30T17:13:11.596984+0000","last_peered":"2024-05-30T17:13:11.596984+0000","last_clean":"2024-05-30T17:13:11.596984+0000","last_became_active":"2024-05-30T16:34:55.896357+0000","last_became_peered":"2024-05-30T16:34:55.896357+0000","last_unstale":"2024-05-30T17:13:11.596984+0000","last_undegraded":"2024-05-30T17:13:11.596984+0000","last_fullsized":"2024-05-30T17:13:11.596984+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"265'3","reported_seq":47,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.067209+0000","last_change":"2024-05-30T16:17:55.551397+0000","last_active":"2024-05-30T17:13:44.067209+0000","last_peered":"2024-05-30T17:13:44.067209+0000","last_clean":"2024-05-30T17:13:44.067209+0000","last_became_active":"2024-05-30T16:17:55.550782+0000","last_became_peered":"2024-05-30T16:17:55.550782+0000","last_unstale":"2024-05-30T17:13:44.067209+0000","last_undegraded":"2024-05-30T17:13:44.067209+0000","last_fullsized":"2024-05-30T17:13:44.067209+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2048,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.d","version":"265'7","reported_seq":74,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:07.405375+0000","last_change":"2024-05-30T17:14:07.405503+0000","last_active":"2024-05-30T17:14:07.405375+0000","last_peered":"2024-05-30T17:14:07.405375+0000","last_clean":"2024-05-30T17:14:07.405375+0000","last_became_active":"2024-05-30T16:17:31.202042+0000","last_became_peered":"2024-05-30T16:17:31.202042+0000","last_unstale":"2024-05-30T17:14:07.405375+0000","last_undegraded":"2024-05-30T17:14:07.405375+0000","last_fullsized":"2024-05-30T17:14:07.405375+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'7","last_scrub_stamp":"2024-05-30T17:14:07.405148+0000","last_deep_scrub":"265'7","last_deep_scrub_stamp":"2024-05-30T17:14:07.405148+0000","last_clean_scrub_stamp":"2024-05-30T17:14:07.405148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:09:19.710358+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":6194,"num_write":7,"num_write_kb":6194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"265'24","reported_seq":139,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.172777+0000","last_change":"2024-05-30T17:13:44.172845+0000","last_active":"2024-05-30T17:13:44.172777+0000","last_peered":"2024-05-30T17:13:44.172777+0000","last_clean":"2024-05-30T17:13:44.172777+0000","last_became_active":"2024-05-30T16:12:50.810095+0000","last_became_peered":"2024-05-30T16:12:50.810095+0000","last_unstale":"2024-05-30T17:13:44.172777+0000","last_undegraded":"2024-05-30T17:13:44.172777+0000","last_fullsized":"2024-05-30T17:13:44.172777+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'24","last_scrub_stamp":"2024-05-30T17:13:44.172579+0000","last_deep_scrub":"265'24","last_deep_scrub_stamp":"2024-05-30T17:13:44.172579+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.172579+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:00:02.104351+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":90,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":39,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:20.793663+0000","last_change":"2024-05-30T17:14:20.793717+0000","last_active":"2024-05-30T17:14:20.793663+0000","last_peered":"2024-05-30T17:14:20.793663+0000","last_clean":"2024-05-30T17:14:20.793663+0000","last_became_active":"2024-05-30T16:34:55.862478+0000","last_became_peered":"2024-05-30T16:34:55.862478+0000","last_unstale":"2024-05-30T17:14:20.793663+0000","last_undegraded":"2024-05-30T17:14:20.793663+0000","last_fullsized":"2024-05-30T17:14:20.793663+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:20.793471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:20.793471+0000","last_clean_scrub_stamp":"2024-05-30T17:14:20.793471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:12:26.521954+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"262'3","reported_seq":54,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:42.129963+0000","last_change":"2024-05-30T16:17:35.743152+0000","last_active":"2024-05-30T17:13:42.129963+0000","last_peered":"2024-05-30T17:13:42.129963+0000","last_clean":"2024-05-30T17:13:42.129963+0000","last_became_active":"2024-05-30T16:17:35.742255+0000","last_became_peered":"2024-05-30T16:17:35.742255+0000","last_unstale":"2024-05-30T17:13:42.129963+0000","last_undegraded":"2024-05-30T17:13:42.129963+0000","last_fullsized":"2024-05-30T17:13:42.129963+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"265'2","reported_seq":65,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:55.774110+0000","last_change":"2024-05-30T17:13:55.774249+0000","last_active":"2024-05-30T17:13:55.774110+0000","last_peered":"2024-05-30T17:13:55.774110+0000","last_clean":"2024-05-30T17:13:55.774110+0000","last_became_active":"2024-05-30T16:17:31.198744+0000","last_became_peered":"2024-05-30T16:17:31.198744+0000","last_unstale":"2024-05-30T17:13:55.774110+0000","last_undegraded":"2024-05-30T17:13:55.774110+0000","last_fullsized":"2024-05-30T17:13:55.774110+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:13:55.773876+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:13:55.773876+0000","last_clean_scrub_stamp":"2024-05-30T17:13:55.773876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:52:06.016908+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"265'36","reported_seq":172,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.883144+0000","last_change":"2024-05-30T17:13:45.883208+0000","last_active":"2024-05-30T17:13:45.883144+0000","last_peered":"2024-05-30T17:13:45.883144+0000","last_clean":"2024-05-30T17:13:45.883144+0000","last_became_active":"2024-05-30T16:12:50.820936+0000","last_became_peered":"2024-05-30T16:12:50.820936+0000","last_unstale":"2024-05-30T17:13:45.883144+0000","last_undegraded":"2024-05-30T17:13:45.883144+0000","last_fullsized":"2024-05-30T17:13:45.883144+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'36","last_scrub_stamp":"2024-05-30T17:13:45.882931+0000","last_deep_scrub":"265'36","last_deep_scrub_stamp":"2024-05-30T17:13:45.882931+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.882931+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:49:53.101567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":194,"num_read_kb":136,"num_write":73,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":39,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:08.715919+0000","last_change":"2024-05-30T17:14:08.715975+0000","last_active":"2024-05-30T17:14:08.715919+0000","last_peered":"2024-05-30T17:14:08.715919+0000","last_clean":"2024-05-30T17:14:08.715919+0000","last_became_active":"2024-05-30T16:34:55.875965+0000","last_became_peered":"2024-05-30T16:34:55.875965+0000","last_unstale":"2024-05-30T17:14:08.715919+0000","last_undegraded":"2024-05-30T17:14:08.715919+0000","last_fullsized":"2024-05-30T17:14:08.715919+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:08.715715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:08.715715+0000","last_clean_scrub_stamp":"2024-05-30T17:14:08.715715+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:29:25.928545+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"265'3","reported_seq":61,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.354852+0000","last_change":"2024-05-30T17:13:59.354913+0000","last_active":"2024-05-30T17:13:59.354852+0000","last_peered":"2024-05-30T17:13:59.354852+0000","last_clean":"2024-05-30T17:13:59.354852+0000","last_became_active":"2024-05-30T16:17:35.229643+0000","last_became_peered":"2024-05-30T16:17:35.229643+0000","last_unstale":"2024-05-30T17:13:59.354852+0000","last_undegraded":"2024-05-30T17:13:59.354852+0000","last_fullsized":"2024-05-30T17:13:59.354852+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:13:59.354655+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:13:59.354655+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.354655+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:40:27.564311+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.f","version":"262'6","reported_seq":73,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.021324+0000","last_change":"2024-05-30T17:13:43.021324+0000","last_active":"2024-05-30T17:13:43.021324+0000","last_peered":"2024-05-30T17:13:43.021324+0000","last_clean":"2024-05-30T17:13:43.021324+0000","last_became_active":"2024-05-30T16:17:31.212894+0000","last_became_peered":"2024-05-30T16:17:31.212894+0000","last_unstale":"2024-05-30T17:13:43.021324+0000","last_undegraded":"2024-05-30T17:13:43.021324+0000","last_fullsized":"2024-05-30T17:13:43.021324+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'6","last_scrub_stamp":"2024-05-30T17:13:43.021049+0000","last_deep_scrub":"262'6","last_deep_scrub_stamp":"2024-05-30T17:13:43.021049+0000","last_clean_scrub_stamp":"2024-05-30T17:13:43.021049+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:27:51.042980+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6144,"num_write":11,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"262'21","reported_seq":130,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.118605+0000","last_change":"2024-05-30T17:13:53.118667+0000","last_active":"2024-05-30T17:13:53.118605+0000","last_peered":"2024-05-30T17:13:53.118605+0000","last_clean":"2024-05-30T17:13:53.118605+0000","last_became_active":"2024-05-30T16:12:50.805772+0000","last_became_peered":"2024-05-30T16:12:50.805772+0000","last_unstale":"2024-05-30T17:13:53.118605+0000","last_undegraded":"2024-05-30T17:13:53.118605+0000","last_fullsized":"2024-05-30T17:13:53.118605+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'21","last_scrub_stamp":"2024-05-30T17:13:53.118396+0000","last_deep_scrub":"262'21","last_deep_scrub_stamp":"2024-05-30T17:13:53.118396+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.118396+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:21:33.139209+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":79,"num_write":45,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:11.612685+0000","last_change":"2024-05-30T16:34:55.891074+0000","last_active":"2024-05-30T17:13:11.612685+0000","last_peered":"2024-05-30T17:13:11.612685+0000","last_clean":"2024-05-30T17:13:11.612685+0000","last_became_active":"2024-05-30T16:34:55.890313+0000","last_became_peered":"2024-05-30T16:34:55.890313+0000","last_unstale":"2024-05-30T17:13:11.612685+0000","last_undegraded":"2024-05-30T17:13:11.612685+0000","last_fullsized":"2024-05-30T17:13:11.612685+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"265'3","reported_seq":56,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.567217+0000","last_change":"2024-05-30T16:17:35.243089+0000","last_active":"2024-05-30T17:13:43.567217+0000","last_peered":"2024-05-30T17:13:43.567217+0000","last_clean":"2024-05-30T17:13:43.567217+0000","last_became_active":"2024-05-30T16:17:35.242271+0000","last_became_peered":"2024-05-30T16:17:35.242271+0000","last_unstale":"2024-05-30T17:13:43.567217+0000","last_undegraded":"2024-05-30T17:13:43.567217+0000","last_fullsized":"2024-05-30T17:13:43.567217+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10240,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"265'3","reported_seq":70,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.656221+0000","last_change":"2024-05-30T17:13:51.656292+0000","last_active":"2024-05-30T17:13:51.656221+0000","last_peered":"2024-05-30T17:13:51.656221+0000","last_clean":"2024-05-30T17:13:51.656221+0000","last_became_active":"2024-05-30T16:17:31.194774+0000","last_became_peered":"2024-05-30T16:17:31.194774+0000","last_unstale":"2024-05-30T17:13:51.656221+0000","last_undegraded":"2024-05-30T17:13:51.656221+0000","last_fullsized":"2024-05-30T17:13:51.656221+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:13:51.656016+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:13:51.656016+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.656016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:08:09.027576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":12288,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"265'18","reported_seq":116,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.659792+0000","last_change":"2024-05-30T17:13:44.659853+0000","last_active":"2024-05-30T17:13:44.659792+0000","last_peered":"2024-05-30T17:13:44.659792+0000","last_clean":"2024-05-30T17:13:44.659792+0000","last_became_active":"2024-05-30T16:12:50.820526+0000","last_became_peered":"2024-05-30T16:12:50.820526+0000","last_unstale":"2024-05-30T17:13:44.659792+0000","last_undegraded":"2024-05-30T17:13:44.659792+0000","last_fullsized":"2024-05-30T17:13:44.659792+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'18","last_scrub_stamp":"2024-05-30T17:13:44.659583+0000","last_deep_scrub":"265'18","last_deep_scrub_stamp":"2024-05-30T17:13:44.659583+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.659583+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:01:55.866157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":60,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:21.599695+0000","last_change":"2024-05-30T16:34:55.884293+0000","last_active":"2024-05-30T17:13:21.599695+0000","last_peered":"2024-05-30T17:13:21.599695+0000","last_clean":"2024-05-30T17:13:21.599695+0000","last_became_active":"2024-05-30T16:34:55.884065+0000","last_became_peered":"2024-05-30T16:34:55.884065+0000","last_unstale":"2024-05-30T17:13:21.599695+0000","last_undegraded":"2024-05-30T17:13:21.599695+0000","last_fullsized":"2024-05-30T17:13:21.599695+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"265'11","reported_seq":83,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.610393+0000","last_change":"2024-05-30T17:13:56.610471+0000","last_active":"2024-05-30T17:13:56.610393+0000","last_peered":"2024-05-30T17:13:56.610393+0000","last_clean":"2024-05-30T17:13:56.610393+0000","last_became_active":"2024-05-30T16:17:35.738991+0000","last_became_peered":"2024-05-30T16:17:35.738991+0000","last_unstale":"2024-05-30T17:13:56.610393+0000","last_undegraded":"2024-05-30T17:13:56.610393+0000","last_fullsized":"2024-05-30T17:13:56.610393+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'11","last_scrub_stamp":"2024-05-30T17:13:56.610167+0000","last_deep_scrub":"265'11","last_deep_scrub_stamp":"2024-05-30T17:13:56.610167+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.610167+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:07:32.799346+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":26998,"num_write":14,"num_write_kb":10938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"262'6","reported_seq":75,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.948567+0000","last_change":"2024-05-30T17:13:49.948567+0000","last_active":"2024-05-30T17:13:49.948567+0000","last_peered":"2024-05-30T17:13:49.948567+0000","last_clean":"2024-05-30T17:13:49.948567+0000","last_became_active":"2024-05-30T16:17:31.212954+0000","last_became_peered":"2024-05-30T16:17:31.212954+0000","last_unstale":"2024-05-30T17:13:49.948567+0000","last_undegraded":"2024-05-30T17:13:49.948567+0000","last_fullsized":"2024-05-30T17:13:49.948567+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'6","last_scrub_stamp":"2024-05-30T17:13:49.948291+0000","last_deep_scrub":"262'6","last_deep_scrub_stamp":"2024-05-30T17:13:49.948291+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.948291+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:46:37.622512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":11264,"num_write":11,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"265'103","reported_seq":396,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:57.554791+0000","last_change":"2024-05-30T17:13:57.554869+0000","last_active":"2024-05-30T17:13:57.554791+0000","last_peered":"2024-05-30T17:13:57.554791+0000","last_clean":"2024-05-30T17:13:57.554791+0000","last_became_active":"2024-05-30T16:12:50.820250+0000","last_became_peered":"2024-05-30T16:12:50.820250+0000","last_unstale":"2024-05-30T17:13:57.554791+0000","last_undegraded":"2024-05-30T17:13:57.554791+0000","last_fullsized":"2024-05-30T17:13:57.554791+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'103","last_scrub_stamp":"2024-05-30T17:13:57.554576+0000","last_deep_scrub":"265'103","last_deep_scrub_stamp":"2024-05-30T17:13:57.554576+0000","last_clean_scrub_stamp":"2024-05-30T17:13:57.554576+0000","objects_scrubbed":1,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:49:22.258431+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":642,"num_read_kb":499,"num_write":114,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3960,"num_omap_keys":8,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"265'1","reported_seq":31,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:11.597056+0000","last_change":"2024-05-30T16:34:55.880548+0000","last_active":"2024-05-30T17:13:11.597056+0000","last_peered":"2024-05-30T17:13:11.597056+0000","last_clean":"2024-05-30T17:13:11.597056+0000","last_became_active":"2024-05-30T16:34:55.878171+0000","last_became_peered":"2024-05-30T16:34:55.878171+0000","last_unstale":"2024-05-30T17:13:11.597056+0000","last_undegraded":"2024-05-30T17:13:11.597056+0000","last_fullsized":"2024-05-30T17:13:11.597056+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"265'14","reported_seq":76,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:42.129846+0000","last_change":"2024-05-30T16:17:35.245798+0000","last_active":"2024-05-30T17:13:42.129846+0000","last_peered":"2024-05-30T17:13:42.129846+0000","last_clean":"2024-05-30T17:13:42.129846+0000","last_became_active":"2024-05-30T16:17:35.245008+0000","last_became_peered":"2024-05-30T16:17:35.245008+0000","last_unstale":"2024-05-30T17:13:42.129846+0000","last_undegraded":"2024-05-30T17:13:42.129846+0000","last_fullsized":"2024-05-30T17:13:42.129846+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":25602,"num_write":23,"num_write_kb":17408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"262'3","reported_seq":69,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:03.872899+0000","last_change":"2024-05-30T17:14:03.872899+0000","last_active":"2024-05-30T17:14:03.872899+0000","last_peered":"2024-05-30T17:14:03.872899+0000","last_clean":"2024-05-30T17:14:03.872899+0000","last_became_active":"2024-05-30T16:17:31.210195+0000","last_became_peered":"2024-05-30T16:17:31.210195+0000","last_unstale":"2024-05-30T17:14:03.872899+0000","last_undegraded":"2024-05-30T17:14:03.872899+0000","last_fullsized":"2024-05-30T17:14:03.872899+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'3","last_scrub_stamp":"2024-05-30T17:14:03.872614+0000","last_deep_scrub":"262'3","last_deep_scrub_stamp":"2024-05-30T17:14:03.872614+0000","last_clean_scrub_stamp":"2024-05-30T17:14:03.872614+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:58:00.016236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"265'43","reported_seq":185,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.468823+0000","last_change":"2024-05-30T17:13:44.468886+0000","last_active":"2024-05-30T17:13:44.468823+0000","last_peered":"2024-05-30T17:13:44.468823+0000","last_clean":"2024-05-30T17:13:44.468823+0000","last_became_active":"2024-05-30T16:12:50.830770+0000","last_became_peered":"2024-05-30T16:12:50.830770+0000","last_unstale":"2024-05-30T17:13:44.468823+0000","last_undegraded":"2024-05-30T17:13:44.468823+0000","last_fullsized":"2024-05-30T17:13:44.468823+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'43","last_scrub_stamp":"2024-05-30T17:13:44.468612+0000","last_deep_scrub":"265'43","last_deep_scrub_stamp":"2024-05-30T17:13:44.468612+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.468612+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:26:30.609648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":214,"num_read_kb":148,"num_write":89,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:11.610397+0000","last_change":"2024-05-30T16:34:55.887780+0000","last_active":"2024-05-30T17:13:11.610397+0000","last_peered":"2024-05-30T17:13:11.610397+0000","last_clean":"2024-05-30T17:13:11.610397+0000","last_became_active":"2024-05-30T16:34:55.887101+0000","last_became_peered":"2024-05-30T16:34:55.887101+0000","last_unstale":"2024-05-30T17:13:11.610397+0000","last_undegraded":"2024-05-30T17:13:11.610397+0000","last_fullsized":"2024-05-30T17:13:11.610397+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"262'3","reported_seq":63,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:52.503977+0000","last_change":"2024-05-30T17:13:52.503977+0000","last_active":"2024-05-30T17:13:52.503977+0000","last_peered":"2024-05-30T17:13:52.503977+0000","last_clean":"2024-05-30T17:13:52.503977+0000","last_became_active":"2024-05-30T16:17:35.212403+0000","last_became_peered":"2024-05-30T16:17:35.212403+0000","last_unstale":"2024-05-30T17:13:52.503977+0000","last_undegraded":"2024-05-30T17:13:52.503977+0000","last_fullsized":"2024-05-30T17:13:52.503977+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'3","last_scrub_stamp":"2024-05-30T17:13:52.503699+0000","last_deep_scrub":"262'3","last_deep_scrub_stamp":"2024-05-30T17:13:52.503699+0000","last_clean_scrub_stamp":"2024-05-30T17:13:52.503699+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:51:26.227521+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"0'0","reported_seq":62,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.933818+0000","last_change":"2024-05-30T17:13:58.933876+0000","last_active":"2024-05-30T17:13:58.933818+0000","last_peered":"2024-05-30T17:13:58.933818+0000","last_clean":"2024-05-30T17:13:58.933818+0000","last_became_active":"2024-05-30T16:17:31.191955+0000","last_became_peered":"2024-05-30T16:17:31.191955+0000","last_unstale":"2024-05-30T17:13:58.933818+0000","last_undegraded":"2024-05-30T17:13:58.933818+0000","last_fullsized":"2024-05-30T17:13:58.933818+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:58.933611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:58.933611+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.933611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:34:51.660228+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"262'8","reported_seq":92,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.839264+0000","last_change":"2024-05-30T17:13:46.839335+0000","last_active":"2024-05-30T17:13:46.839264+0000","last_peered":"2024-05-30T17:13:46.839264+0000","last_clean":"2024-05-30T17:13:46.839264+0000","last_became_active":"2024-05-30T16:12:50.819871+0000","last_became_peered":"2024-05-30T16:12:50.819871+0000","last_unstale":"2024-05-30T17:13:46.839264+0000","last_undegraded":"2024-05-30T17:13:46.839264+0000","last_fullsized":"2024-05-30T17:13:46.839264+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'8","last_scrub_stamp":"2024-05-30T17:13:46.839029+0000","last_deep_scrub":"262'8","last_deep_scrub_stamp":"2024-05-30T17:13:46.839029+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.839029+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:29:46.097404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":28,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":31,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:26.585480+0000","last_change":"2024-05-30T16:34:55.883102+0000","last_active":"2024-05-30T17:12:26.585480+0000","last_peered":"2024-05-30T17:12:26.585480+0000","last_clean":"2024-05-30T17:12:26.585480+0000","last_became_active":"2024-05-30T16:34:55.882443+0000","last_became_peered":"2024-05-30T16:34:55.882443+0000","last_unstale":"2024-05-30T17:12:26.585480+0000","last_undegraded":"2024-05-30T17:12:26.585480+0000","last_fullsized":"2024-05-30T17:12:26.585480+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"262'3","reported_seq":64,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:07.870394+0000","last_change":"2024-05-30T17:14:07.870537+0000","last_active":"2024-05-30T17:14:07.870394+0000","last_peered":"2024-05-30T17:14:07.870394+0000","last_clean":"2024-05-30T17:14:07.870394+0000","last_became_active":"2024-05-30T16:17:35.247419+0000","last_became_peered":"2024-05-30T16:17:35.247419+0000","last_unstale":"2024-05-30T17:14:07.870394+0000","last_undegraded":"2024-05-30T17:14:07.870394+0000","last_fullsized":"2024-05-30T17:14:07.870394+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'3","last_scrub_stamp":"2024-05-30T17:14:07.870141+0000","last_deep_scrub":"262'3","last_deep_scrub_stamp":"2024-05-30T17:14:07.870141+0000","last_clean_scrub_stamp":"2024-05-30T17:14:07.870141+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:18:01.008022+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"262'2","reported_seq":68,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.502634+0000","last_change":"2024-05-30T17:13:50.502697+0000","last_active":"2024-05-30T17:13:50.502634+0000","last_peered":"2024-05-30T17:13:50.502634+0000","last_clean":"2024-05-30T17:13:50.502634+0000","last_became_active":"2024-05-30T16:17:31.189122+0000","last_became_peered":"2024-05-30T16:17:31.189122+0000","last_unstale":"2024-05-30T17:13:50.502634+0000","last_undegraded":"2024-05-30T17:13:50.502634+0000","last_fullsized":"2024-05-30T17:13:50.502634+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'2","last_scrub_stamp":"2024-05-30T17:13:50.502442+0000","last_deep_scrub":"262'2","last_deep_scrub_stamp":"2024-05-30T17:13:50.502442+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.502442+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:48:21.318011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"265'53","reported_seq":210,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.655149+0000","last_change":"2024-05-30T17:13:41.655171+0000","last_active":"2024-05-30T17:13:41.655149+0000","last_peered":"2024-05-30T17:13:41.655149+0000","last_clean":"2024-05-30T17:13:41.655149+0000","last_became_active":"2024-05-30T16:12:50.809372+0000","last_became_peered":"2024-05-30T16:12:50.809372+0000","last_unstale":"2024-05-30T17:13:41.655149+0000","last_undegraded":"2024-05-30T17:13:41.655149+0000","last_fullsized":"2024-05-30T17:13:41.655149+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'53","last_scrub_stamp":"2024-05-30T17:13:41.655071+0000","last_deep_scrub":"265'53","last_deep_scrub_stamp":"2024-05-30T17:13:41.655071+0000","last_clean_scrub_stamp":"2024-05-30T17:13:41.655071+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:39:32.196268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":186,"num_write":103,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":31,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:26.969180+0000","last_change":"2024-05-30T16:34:55.881085+0000","last_active":"2024-05-30T17:12:26.969180+0000","last_peered":"2024-05-30T17:12:26.969180+0000","last_clean":"2024-05-30T17:12:26.969180+0000","last_became_active":"2024-05-30T16:34:55.880627+0000","last_became_peered":"2024-05-30T16:34:55.880627+0000","last_unstale":"2024-05-30T17:12:26.969180+0000","last_undegraded":"2024-05-30T17:12:26.969180+0000","last_fullsized":"2024-05-30T17:12:26.969180+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"0'0","reported_seq":50,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.648090+0000","last_change":"2024-05-30T16:17:35.743808+0000","last_active":"2024-05-30T17:13:41.648090+0000","last_peered":"2024-05-30T17:13:41.648090+0000","last_clean":"2024-05-30T17:13:41.648090+0000","last_became_active":"2024-05-30T16:17:35.743036+0000","last_became_peered":"2024-05-30T16:17:35.743036+0000","last_unstale":"2024-05-30T17:13:41.648090+0000","last_undegraded":"2024-05-30T17:13:41.648090+0000","last_fullsized":"2024-05-30T17:13:41.648090+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"265'3","reported_seq":69,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:08.435518+0000","last_change":"2024-05-30T17:14:08.435583+0000","last_active":"2024-05-30T17:14:08.435518+0000","last_peered":"2024-05-30T17:14:08.435518+0000","last_clean":"2024-05-30T17:14:08.435518+0000","last_became_active":"2024-05-30T16:17:31.202064+0000","last_became_peered":"2024-05-30T16:17:31.202064+0000","last_unstale":"2024-05-30T17:14:08.435518+0000","last_undegraded":"2024-05-30T17:14:08.435518+0000","last_fullsized":"2024-05-30T17:14:08.435518+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:14:08.435314+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:14:08.435314+0000","last_clean_scrub_stamp":"2024-05-30T17:14:08.435314+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:58:19.406401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2048,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"265'34","reported_seq":158,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.862253+0000","last_change":"2024-05-30T17:13:47.862374+0000","last_active":"2024-05-30T17:13:47.862253+0000","last_peered":"2024-05-30T17:13:47.862253+0000","last_clean":"2024-05-30T17:13:47.862253+0000","last_became_active":"2024-05-30T16:12:50.815895+0000","last_became_peered":"2024-05-30T16:12:50.815895+0000","last_unstale":"2024-05-30T17:13:47.862253+0000","last_undegraded":"2024-05-30T17:13:47.862253+0000","last_fullsized":"2024-05-30T17:13:47.862253+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'34","last_scrub_stamp":"2024-05-30T17:13:47.861959+0000","last_deep_scrub":"265'34","last_deep_scrub_stamp":"2024-05-30T17:13:47.861959+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.861959+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:37:52.262386+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":160,"num_read_kb":111,"num_write":71,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":32,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:01.813668+0000","last_change":"2024-05-30T16:34:55.883831+0000","last_active":"2024-05-30T17:13:01.813668+0000","last_peered":"2024-05-30T17:13:01.813668+0000","last_clean":"2024-05-30T17:13:01.813668+0000","last_became_active":"2024-05-30T16:34:55.883112+0000","last_became_peered":"2024-05-30T16:34:55.883112+0000","last_unstale":"2024-05-30T17:13:01.813668+0000","last_undegraded":"2024-05-30T17:13:01.813668+0000","last_fullsized":"2024-05-30T17:13:01.813668+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"265'3","reported_seq":62,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:00.593884+0000","last_change":"2024-05-30T17:14:00.593945+0000","last_active":"2024-05-30T17:14:00.593884+0000","last_peered":"2024-05-30T17:14:00.593884+0000","last_clean":"2024-05-30T17:14:00.593884+0000","last_became_active":"2024-05-30T16:17:35.743608+0000","last_became_peered":"2024-05-30T16:17:35.743608+0000","last_unstale":"2024-05-30T17:14:00.593884+0000","last_undegraded":"2024-05-30T17:14:00.593884+0000","last_fullsized":"2024-05-30T17:14:00.593884+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:14:00.593691+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:14:00.593691+0000","last_clean_scrub_stamp":"2024-05-30T17:14:00.593691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:51:53.188748+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"0'0","reported_seq":63,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:52.225937+0000","last_change":"2024-05-30T17:13:52.225991+0000","last_active":"2024-05-30T17:13:52.225937+0000","last_peered":"2024-05-30T17:13:52.225937+0000","last_clean":"2024-05-30T17:13:52.225937+0000","last_became_active":"2024-05-30T16:17:31.188810+0000","last_became_peered":"2024-05-30T16:17:31.188810+0000","last_unstale":"2024-05-30T17:13:52.225937+0000","last_undegraded":"2024-05-30T17:13:52.225937+0000","last_fullsized":"2024-05-30T17:13:52.225937+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:52.225736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:52.225736+0000","last_clean_scrub_stamp":"2024-05-30T17:13:52.225736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:31:29.155002+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"265'18","reported_seq":120,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.502424+0000","last_change":"2024-05-30T17:13:53.502489+0000","last_active":"2024-05-30T17:13:53.502424+0000","last_peered":"2024-05-30T17:13:53.502424+0000","last_clean":"2024-05-30T17:13:53.502424+0000","last_became_active":"2024-05-30T16:12:50.826880+0000","last_became_peered":"2024-05-30T16:12:50.826880+0000","last_unstale":"2024-05-30T17:13:53.502424+0000","last_undegraded":"2024-05-30T17:13:53.502424+0000","last_fullsized":"2024-05-30T17:13:53.502424+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'18","last_scrub_stamp":"2024-05-30T17:13:53.502233+0000","last_deep_scrub":"265'18","last_deep_scrub_stamp":"2024-05-30T17:13:53.502233+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.502233+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:07:36.967499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":61,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":31,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:26.764016+0000","last_change":"2024-05-30T16:34:55.886177+0000","last_active":"2024-05-30T17:12:26.764016+0000","last_peered":"2024-05-30T17:12:26.764016+0000","last_clean":"2024-05-30T17:12:26.764016+0000","last_became_active":"2024-05-30T16:34:55.885643+0000","last_became_peered":"2024-05-30T16:34:55.885643+0000","last_unstale":"2024-05-30T17:12:26.764016+0000","last_undegraded":"2024-05-30T17:12:26.764016+0000","last_fullsized":"2024-05-30T17:12:26.764016+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"262'2","reported_seq":53,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.567194+0000","last_change":"2024-05-30T16:17:35.248399+0000","last_active":"2024-05-30T17:13:43.567194+0000","last_peered":"2024-05-30T17:13:43.567194+0000","last_clean":"2024-05-30T17:13:43.567194+0000","last_became_active":"2024-05-30T16:17:35.247757+0000","last_became_peered":"2024-05-30T16:17:35.247757+0000","last_unstale":"2024-05-30T17:13:43.567194+0000","last_undegraded":"2024-05-30T17:13:43.567194+0000","last_fullsized":"2024-05-30T17:13:43.567194+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"0'0","reported_seq":63,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.456500+0000","last_change":"2024-05-30T17:13:46.456500+0000","last_active":"2024-05-30T17:13:46.456500+0000","last_peered":"2024-05-30T17:13:46.456500+0000","last_clean":"2024-05-30T17:13:46.456500+0000","last_became_active":"2024-05-30T16:17:31.203452+0000","last_became_peered":"2024-05-30T16:17:31.203452+0000","last_unstale":"2024-05-30T17:13:46.456500+0000","last_undegraded":"2024-05-30T17:13:46.456500+0000","last_fullsized":"2024-05-30T17:13:46.456500+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:46.456245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:46.456245+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.456245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:07:15.159368+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"265'37","reported_seq":171,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.919132+0000","last_change":"2024-05-30T17:13:46.919198+0000","last_active":"2024-05-30T17:13:46.919132+0000","last_peered":"2024-05-30T17:13:46.919132+0000","last_clean":"2024-05-30T17:13:46.919132+0000","last_became_active":"2024-05-30T16:12:50.811626+0000","last_became_peered":"2024-05-30T16:12:50.811626+0000","last_unstale":"2024-05-30T17:13:46.919132+0000","last_undegraded":"2024-05-30T17:13:46.919132+0000","last_fullsized":"2024-05-30T17:13:46.919132+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'37","last_scrub_stamp":"2024-05-30T17:13:46.918931+0000","last_deep_scrub":"265'37","last_deep_scrub_stamp":"2024-05-30T17:13:46.918931+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.918931+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:38:14.857169+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":184,"num_read_kb":132,"num_write":81,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"265'2","reported_seq":41,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:11.395523+0000","last_change":"2024-05-30T17:14:11.395649+0000","last_active":"2024-05-30T17:14:11.395523+0000","last_peered":"2024-05-30T17:14:11.395523+0000","last_clean":"2024-05-30T17:14:11.395523+0000","last_became_active":"2024-05-30T16:34:55.861767+0000","last_became_peered":"2024-05-30T16:34:55.861767+0000","last_unstale":"2024-05-30T17:14:11.395523+0000","last_undegraded":"2024-05-30T17:14:11.395523+0000","last_fullsized":"2024-05-30T17:14:11.395523+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:14:11.395245+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:14:11.395245+0000","last_clean_scrub_stamp":"2024-05-30T17:14:11.395245+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:31:20.732589+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"265'8","reported_seq":73,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:20.954829+0000","last_change":"2024-05-30T17:14:20.954890+0000","last_active":"2024-05-30T17:14:20.954829+0000","last_peered":"2024-05-30T17:14:20.954829+0000","last_clean":"2024-05-30T17:14:20.954829+0000","last_became_active":"2024-05-30T16:17:35.243435+0000","last_became_peered":"2024-05-30T16:17:35.243435+0000","last_unstale":"2024-05-30T17:14:20.954829+0000","last_undegraded":"2024-05-30T17:14:20.954829+0000","last_fullsized":"2024-05-30T17:14:20.954829+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'8","last_scrub_stamp":"2024-05-30T17:14:20.954635+0000","last_deep_scrub":"265'8","last_deep_scrub_stamp":"2024-05-30T17:14:20.954635+0000","last_clean_scrub_stamp":"2024-05-30T17:14:20.954635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:57:35.836726+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":13312,"num_write":13,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"265'3","reported_seq":68,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.458042+0000","last_change":"2024-05-30T17:13:47.458106+0000","last_active":"2024-05-30T17:13:47.458042+0000","last_peered":"2024-05-30T17:13:47.458042+0000","last_clean":"2024-05-30T17:13:47.458042+0000","last_became_active":"2024-05-30T16:17:31.200663+0000","last_became_peered":"2024-05-30T16:17:31.200663+0000","last_unstale":"2024-05-30T17:13:47.458042+0000","last_undegraded":"2024-05-30T17:13:47.458042+0000","last_fullsized":"2024-05-30T17:13:47.458042+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:13:47.457841+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:13:47.457841+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.457841+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:37:54.600403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"265'15","reported_seq":112,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.234861+0000","last_change":"2024-05-30T17:13:48.234982+0000","last_active":"2024-05-30T17:13:48.234861+0000","last_peered":"2024-05-30T17:13:48.234861+0000","last_clean":"2024-05-30T17:13:48.234861+0000","last_became_active":"2024-05-30T16:12:50.813741+0000","last_became_peered":"2024-05-30T16:12:50.813741+0000","last_unstale":"2024-05-30T17:13:48.234861+0000","last_undegraded":"2024-05-30T17:13:48.234861+0000","last_fullsized":"2024-05-30T17:13:48.234861+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'15","last_scrub_stamp":"2024-05-30T17:13:48.234638+0000","last_deep_scrub":"265'15","last_deep_scrub_stamp":"2024-05-30T17:13:48.234638+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.234638+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:15:02.442194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":53,"num_write":33,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":31,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.648361+0000","last_change":"2024-05-30T16:34:55.859738+0000","last_active":"2024-05-30T17:13:41.648361+0000","last_peered":"2024-05-30T17:13:41.648361+0000","last_clean":"2024-05-30T17:13:41.648361+0000","last_became_active":"2024-05-30T16:34:55.858872+0000","last_became_peered":"2024-05-30T16:34:55.858872+0000","last_unstale":"2024-05-30T17:13:41.648361+0000","last_undegraded":"2024-05-30T17:13:41.648361+0000","last_fullsized":"2024-05-30T17:13:41.648361+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"0'0","reported_seq":51,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.067117+0000","last_change":"2024-05-30T16:17:35.227683+0000","last_active":"2024-05-30T17:13:44.067117+0000","last_peered":"2024-05-30T17:13:44.067117+0000","last_clean":"2024-05-30T17:13:44.067117+0000","last_became_active":"2024-05-30T16:17:35.226829+0000","last_became_peered":"2024-05-30T16:17:35.226829+0000","last_unstale":"2024-05-30T17:13:44.067117+0000","last_undegraded":"2024-05-30T17:13:44.067117+0000","last_fullsized":"2024-05-30T17:13:44.067117+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"0'0","reported_seq":64,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.486126+0000","last_change":"2024-05-30T17:13:48.486182+0000","last_active":"2024-05-30T17:13:48.486126+0000","last_peered":"2024-05-30T17:13:48.486126+0000","last_clean":"2024-05-30T17:13:48.486126+0000","last_became_active":"2024-05-30T16:17:31.203535+0000","last_became_peered":"2024-05-30T16:17:31.203535+0000","last_unstale":"2024-05-30T17:13:48.486126+0000","last_undegraded":"2024-05-30T17:13:48.486126+0000","last_fullsized":"2024-05-30T17:13:48.486126+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:48.485936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:48.485936+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.485936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:42:58.343268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"265'36","reported_seq":166,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.750202+0000","last_change":"2024-05-30T17:13:43.750267+0000","last_active":"2024-05-30T17:13:43.750202+0000","last_peered":"2024-05-30T17:13:43.750202+0000","last_clean":"2024-05-30T17:13:43.750202+0000","last_became_active":"2024-05-30T16:12:50.831842+0000","last_became_peered":"2024-05-30T16:12:50.831842+0000","last_unstale":"2024-05-30T17:13:43.750202+0000","last_undegraded":"2024-05-30T17:13:43.750202+0000","last_fullsized":"2024-05-30T17:13:43.750202+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'36","last_scrub_stamp":"2024-05-30T17:13:43.750005+0000","last_deep_scrub":"265'36","last_deep_scrub_stamp":"2024-05-30T17:13:43.750005+0000","last_clean_scrub_stamp":"2024-05-30T17:13:43.750005+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:20:47.390125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":177,"num_read_kb":125,"num_write":75,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":31,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:26.600628+0000","last_change":"2024-05-30T16:34:55.863472+0000","last_active":"2024-05-30T17:12:26.600628+0000","last_peered":"2024-05-30T17:12:26.600628+0000","last_clean":"2024-05-30T17:12:26.600628+0000","last_became_active":"2024-05-30T16:34:55.862709+0000","last_became_peered":"2024-05-30T16:34:55.862709+0000","last_unstale":"2024-05-30T17:12:26.600628+0000","last_undegraded":"2024-05-30T17:12:26.600628+0000","last_fullsized":"2024-05-30T17:12:26.600628+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"265'3","reported_seq":61,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:18.813164+0000","last_change":"2024-05-30T17:14:18.813250+0000","last_active":"2024-05-30T17:14:18.813164+0000","last_peered":"2024-05-30T17:14:18.813164+0000","last_clean":"2024-05-30T17:14:18.813164+0000","last_became_active":"2024-05-30T16:17:35.240472+0000","last_became_peered":"2024-05-30T16:17:35.240472+0000","last_unstale":"2024-05-30T17:14:18.813164+0000","last_undegraded":"2024-05-30T17:14:18.813164+0000","last_fullsized":"2024-05-30T17:14:18.813164+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:14:18.812962+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:14:18.812962+0000","last_clean_scrub_stamp":"2024-05-30T17:14:18.812962+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:06:37.629144+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"265'6","reported_seq":74,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.031484+0000","last_change":"2024-05-30T17:13:44.031484+0000","last_active":"2024-05-30T17:13:44.031484+0000","last_peered":"2024-05-30T17:13:44.031484+0000","last_clean":"2024-05-30T17:13:44.031484+0000","last_became_active":"2024-05-30T16:17:31.187958+0000","last_became_peered":"2024-05-30T16:17:31.187958+0000","last_unstale":"2024-05-30T17:13:44.031484+0000","last_undegraded":"2024-05-30T17:13:44.031484+0000","last_fullsized":"2024-05-30T17:13:44.031484+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'6","last_scrub_stamp":"2024-05-30T17:13:44.031239+0000","last_deep_scrub":"265'6","last_deep_scrub_stamp":"2024-05-30T17:13:44.031239+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.031239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:29:30.262819+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":10240,"num_write":16,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"265'18","reported_seq":121,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.918092+0000","last_change":"2024-05-30T17:13:44.918156+0000","last_active":"2024-05-30T17:13:44.918092+0000","last_peered":"2024-05-30T17:13:44.918092+0000","last_clean":"2024-05-30T17:13:44.918092+0000","last_became_active":"2024-05-30T16:12:50.823090+0000","last_became_peered":"2024-05-30T16:12:50.823090+0000","last_unstale":"2024-05-30T17:13:44.918092+0000","last_undegraded":"2024-05-30T17:13:44.918092+0000","last_fullsized":"2024-05-30T17:13:44.918092+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'18","last_scrub_stamp":"2024-05-30T17:13:44.917823+0000","last_deep_scrub":"265'18","last_deep_scrub_stamp":"2024-05-30T17:13:44.917823+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.917823+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:31:54.618562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":69,"num_write":35,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":31,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:36.588097+0000","last_change":"2024-05-30T16:34:55.891440+0000","last_active":"2024-05-30T17:12:36.588097+0000","last_peered":"2024-05-30T17:12:36.588097+0000","last_clean":"2024-05-30T17:12:36.588097+0000","last_became_active":"2024-05-30T16:34:55.890688+0000","last_became_peered":"2024-05-30T16:34:55.890688+0000","last_unstale":"2024-05-30T17:12:36.588097+0000","last_undegraded":"2024-05-30T17:12:36.588097+0000","last_fullsized":"2024-05-30T17:12:36.588097+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"265'8","reported_seq":68,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.567154+0000","last_change":"2024-05-30T16:17:35.238582+0000","last_active":"2024-05-30T17:13:43.567154+0000","last_peered":"2024-05-30T17:13:43.567154+0000","last_clean":"2024-05-30T17:13:43.567154+0000","last_became_active":"2024-05-30T16:17:35.237031+0000","last_became_peered":"2024-05-30T16:17:35.237031+0000","last_unstale":"2024-05-30T17:13:43.567154+0000","last_undegraded":"2024-05-30T17:13:43.567154+0000","last_fullsized":"2024-05-30T17:13:43.567154+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:17:34.184545+0000","last_clean_scrub_stamp":"2024-05-30T16:17:34.184545+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":20482,"num_write":17,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"265'6","reported_seq":76,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.818137+0000","last_change":"2024-05-30T17:13:48.818281+0000","last_active":"2024-05-30T17:13:48.818137+0000","last_peered":"2024-05-30T17:13:48.818137+0000","last_clean":"2024-05-30T17:13:48.818137+0000","last_became_active":"2024-05-30T16:17:31.203871+0000","last_became_peered":"2024-05-30T16:17:31.203871+0000","last_unstale":"2024-05-30T17:13:48.818137+0000","last_undegraded":"2024-05-30T17:13:48.818137+0000","last_fullsized":"2024-05-30T17:13:48.818137+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'6","last_scrub_stamp":"2024-05-30T17:13:48.817847+0000","last_deep_scrub":"265'6","last_deep_scrub_stamp":"2024-05-30T17:13:48.817847+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.817847+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:26:36.664287+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":4098,"num_write":9,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"265'24","reported_seq":142,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.471273+0000","last_change":"2024-05-30T17:13:51.471404+0000","last_active":"2024-05-30T17:13:51.471273+0000","last_peered":"2024-05-30T17:13:51.471273+0000","last_clean":"2024-05-30T17:13:51.471273+0000","last_became_active":"2024-05-30T16:12:50.827078+0000","last_became_peered":"2024-05-30T16:12:50.827078+0000","last_unstale":"2024-05-30T17:13:51.471273+0000","last_undegraded":"2024-05-30T17:13:51.471273+0000","last_fullsized":"2024-05-30T17:13:51.471273+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'24","last_scrub_stamp":"2024-05-30T17:13:51.471027+0000","last_deep_scrub":"265'24","last_deep_scrub_stamp":"2024-05-30T17:13:51.471027+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.471027+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:41:56.836293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":93,"num_write":46,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"265'165","reported_seq":777,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:26.601117+0000","last_change":"2024-05-30T15:54:16.662201+0000","last_active":"2024-05-30T17:13:26.601117+0000","last_peered":"2024-05-30T17:13:26.601117+0000","last_clean":"2024-05-30T17:13:26.601117+0000","last_became_active":"2024-05-30T15:54:16.661334+0000","last_became_peered":"2024-05-30T15:54:16.661334+0000","last_unstale":"2024-05-30T17:13:26.601117+0000","last_undegraded":"2024-05-30T17:13:26.601117+0000","last_fullsized":"2024-05-30T17:13:26.601117+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":100,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2159792,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":347,"num_read_kb":7638,"num_write":1029,"num_write_kb":24692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"265'441","reported_seq":2084,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:06.288519+0000","last_change":"2024-05-30T17:14:06.288519+0000","last_active":"2024-05-30T17:14:06.288519+0000","last_peered":"2024-05-30T17:14:06.288519+0000","last_clean":"2024-05-30T17:14:06.288519+0000","last_became_active":"2024-05-30T15:54:18.669016+0000","last_became_peered":"2024-05-30T15:54:18.669016+0000","last_unstale":"2024-05-30T17:14:06.288519+0000","last_undegraded":"2024-05-30T17:14:06.288519+0000","last_fullsized":"2024-05-30T17:14:06.288519+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'441","last_scrub_stamp":"2024-05-30T17:14:06.288225+0000","last_deep_scrub":"265'441","last_deep_scrub_stamp":"2024-05-30T17:14:06.288225+0000","last_clean_scrub_stamp":"2024-05-30T17:14:06.288225+0000","objects_scrubbed":5,"log_size":41,"log_dups_size":400,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:34:29.289562+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2479,"num_read_kb":2366,"num_write":506,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"265'234","reported_seq":898,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:15.986498+0000","last_change":"2024-05-30T17:14:15.986759+0000","last_active":"2024-05-30T17:14:15.986498+0000","last_peered":"2024-05-30T17:14:15.986498+0000","last_clean":"2024-05-30T17:14:15.986498+0000","last_became_active":"2024-05-30T15:54:16.652724+0000","last_became_peered":"2024-05-30T15:54:16.652724+0000","last_unstale":"2024-05-30T17:14:15.986498+0000","last_undegraded":"2024-05-30T17:14:15.986498+0000","last_fullsized":"2024-05-30T17:14:15.986498+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'234","last_scrub_stamp":"2024-05-30T17:14:15.986234+0000","last_deep_scrub":"265'234","last_deep_scrub_stamp":"2024-05-30T17:14:15.986234+0000","last_clean_scrub_stamp":"2024-05-30T17:14:15.986234+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":200,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:24:51.351668+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6513950,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":554,"num_read_kb":8631,"num_write":1281,"num_write_kb":24636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"265'762","reported_seq":2127,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:15.778412+0000","last_change":"2024-05-30T17:14:15.778412+0000","last_active":"2024-05-30T17:14:15.778412+0000","last_peered":"2024-05-30T17:14:15.778412+0000","last_clean":"2024-05-30T17:14:15.778412+0000","last_became_active":"2024-05-30T15:54:18.648675+0000","last_became_peered":"2024-05-30T15:54:18.648675+0000","last_unstale":"2024-05-30T17:14:15.778412+0000","last_undegraded":"2024-05-30T17:14:15.778412+0000","last_fullsized":"2024-05-30T17:14:15.778412+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'762","last_scrub_stamp":"2024-05-30T17:14:15.778159+0000","last_deep_scrub":"265'762","last_deep_scrub_stamp":"2024-05-30T17:14:15.778159+0000","last_clean_scrub_stamp":"2024-05-30T17:14:15.778159+0000","objects_scrubbed":4,"log_size":62,"log_dups_size":700,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:29:08.620212+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3292,"num_read_kb":2905,"num_write":1031,"num_write_kb":415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"265'208","reported_seq":841,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.415910+0000","last_change":"2024-05-30T15:54:16.633538+0000","last_active":"2024-05-30T17:13:38.415910+0000","last_peered":"2024-05-30T17:13:38.415910+0000","last_clean":"2024-05-30T17:13:38.415910+0000","last_became_active":"2024-05-30T15:54:16.633011+0000","last_became_peered":"2024-05-30T15:54:16.633011+0000","last_unstale":"2024-05-30T17:13:38.415910+0000","last_undegraded":"2024-05-30T17:13:38.415910+0000","last_fullsized":"2024-05-30T17:13:38.415910+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":100,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11855586,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":404,"num_read_kb":43860,"num_write":998,"num_write_kb":74051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"265'747","reported_seq":2155,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:14.074411+0000","last_change":"2024-05-30T17:14:14.074411+0000","last_active":"2024-05-30T17:14:14.074411+0000","last_peered":"2024-05-30T17:14:14.074411+0000","last_clean":"2024-05-30T17:14:14.074411+0000","last_became_active":"2024-05-30T15:54:18.649608+0000","last_became_peered":"2024-05-30T15:54:18.649608+0000","last_unstale":"2024-05-30T17:14:14.074411+0000","last_undegraded":"2024-05-30T17:14:14.074411+0000","last_fullsized":"2024-05-30T17:14:14.074411+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'747","last_scrub_stamp":"2024-05-30T17:14:14.074153+0000","last_deep_scrub":"265'747","last_deep_scrub_stamp":"2024-05-30T17:14:14.074153+0000","last_clean_scrub_stamp":"2024-05-30T17:14:14.074153+0000","objects_scrubbed":1,"log_size":47,"log_dups_size":700,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:05:55.305461+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3275,"num_read_kb":2909,"num_write":996,"num_write_kb":378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"265'154","reported_seq":793,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.303761+0000","last_change":"2024-05-30T15:54:16.653236+0000","last_active":"2024-05-30T17:13:38.303761+0000","last_peered":"2024-05-30T17:13:38.303761+0000","last_clean":"2024-05-30T17:13:38.303761+0000","last_became_active":"2024-05-30T15:54:16.652315+0000","last_became_peered":"2024-05-30T15:54:16.652315+0000","last_unstale":"2024-05-30T17:13:38.303761+0000","last_undegraded":"2024-05-30T17:13:38.303761+0000","last_fullsized":"2024-05-30T17:13:38.303761+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":100,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1250479,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":437,"num_read_kb":3484,"num_write":911,"num_write_kb":6490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"265'490","reported_seq":1975,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:12.850064+0000","last_change":"2024-05-30T17:14:12.850064+0000","last_active":"2024-05-30T17:14:12.850064+0000","last_peered":"2024-05-30T17:14:12.850064+0000","last_clean":"2024-05-30T17:14:12.850064+0000","last_became_active":"2024-05-30T15:54:18.652451+0000","last_became_peered":"2024-05-30T15:54:18.652451+0000","last_unstale":"2024-05-30T17:14:12.850064+0000","last_undegraded":"2024-05-30T17:14:12.850064+0000","last_fullsized":"2024-05-30T17:14:12.850064+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'490","last_scrub_stamp":"2024-05-30T17:14:12.849644+0000","last_deep_scrub":"265'490","last_deep_scrub_stamp":"2024-05-30T17:14:12.849644+0000","last_clean_scrub_stamp":"2024-05-30T17:14:12.849644+0000","objects_scrubbed":3,"log_size":90,"log_dups_size":400,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:33:13.009731+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2387,"num_read_kb":2188,"num_write":616,"num_write_kb":168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":505,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"265'245","reported_seq":961,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.051393+0000","last_change":"2024-05-30T15:54:16.659161+0000","last_active":"2024-05-30T17:13:45.051393+0000","last_peered":"2024-05-30T17:13:45.051393+0000","last_clean":"2024-05-30T17:13:45.051393+0000","last_became_active":"2024-05-30T15:54:16.658338+0000","last_became_peered":"2024-05-30T15:54:16.658338+0000","last_unstale":"2024-05-30T17:13:45.051393+0000","last_undegraded":"2024-05-30T17:13:45.051393+0000","last_fullsized":"2024-05-30T17:13:45.051393+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":200,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291491,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":830,"num_read_kb":24681,"num_write":1246,"num_write_kb":34031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"265'378","reported_seq":1879,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:05.718929+0000","last_change":"2024-05-30T17:14:05.719014+0000","last_active":"2024-05-30T17:14:05.718929+0000","last_peered":"2024-05-30T17:14:05.718929+0000","last_clean":"2024-05-30T17:14:05.718929+0000","last_became_active":"2024-05-30T15:54:18.669017+0000","last_became_peered":"2024-05-30T15:54:18.669017+0000","last_unstale":"2024-05-30T17:14:05.718929+0000","last_undegraded":"2024-05-30T17:14:05.718929+0000","last_fullsized":"2024-05-30T17:14:05.718929+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'378","last_scrub_stamp":"2024-05-30T17:14:05.718728+0000","last_deep_scrub":"265'378","last_deep_scrub_stamp":"2024-05-30T17:14:05.718728+0000","last_clean_scrub_stamp":"2024-05-30T17:14:05.718728+0000","objects_scrubbed":8,"log_size":78,"log_dups_size":300,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:31:23.767712+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1986,"num_read_kb":1906,"num_write":419,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":465,"num_omap_keys":3,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"265'196","reported_seq":831,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:15.302437+0000","last_change":"2024-05-30T17:14:15.302437+0000","last_active":"2024-05-30T17:14:15.302437+0000","last_peered":"2024-05-30T17:14:15.302437+0000","last_clean":"2024-05-30T17:14:15.302437+0000","last_became_active":"2024-05-30T15:54:16.671120+0000","last_became_peered":"2024-05-30T15:54:16.671120+0000","last_unstale":"2024-05-30T17:14:15.302437+0000","last_undegraded":"2024-05-30T17:14:15.302437+0000","last_fullsized":"2024-05-30T17:14:15.302437+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'196","last_scrub_stamp":"2024-05-30T17:14:15.302156+0000","last_deep_scrub":"265'196","last_deep_scrub_stamp":"2024-05-30T17:14:15.302156+0000","last_clean_scrub_stamp":"2024-05-30T17:14:15.302156+0000","objects_scrubbed":2,"log_size":96,"log_dups_size":100,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:02:53.975403+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":81,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":438,"num_read_kb":4718,"num_write":1101,"num_write_kb":20583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"265'727","reported_seq":2212,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:16.031885+0000","last_change":"2024-05-30T17:14:16.032031+0000","last_active":"2024-05-30T17:14:16.031885+0000","last_peered":"2024-05-30T17:14:16.031885+0000","last_clean":"2024-05-30T17:14:16.031885+0000","last_became_active":"2024-05-30T15:54:18.650543+0000","last_became_peered":"2024-05-30T15:54:18.650543+0000","last_unstale":"2024-05-30T17:14:16.031885+0000","last_undegraded":"2024-05-30T17:14:16.031885+0000","last_fullsized":"2024-05-30T17:14:16.031885+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'727","last_scrub_stamp":"2024-05-30T17:14:16.031640+0000","last_deep_scrub":"265'727","last_deep_scrub_stamp":"2024-05-30T17:14:16.031640+0000","last_clean_scrub_stamp":"2024-05-30T17:14:16.031640+0000","objects_scrubbed":2,"log_size":27,"log_dups_size":700,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:30:02.474759+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3282,"num_read_kb":2952,"num_write":973,"num_write_kb":356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"265'192","reported_seq":842,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:17.822847+0000","last_change":"2024-05-30T17:14:17.822942+0000","last_active":"2024-05-30T17:14:17.822847+0000","last_peered":"2024-05-30T17:14:17.822847+0000","last_clean":"2024-05-30T17:14:17.822847+0000","last_became_active":"2024-05-30T15:54:16.678799+0000","last_became_peered":"2024-05-30T15:54:16.678799+0000","last_unstale":"2024-05-30T17:14:17.822847+0000","last_undegraded":"2024-05-30T17:14:17.822847+0000","last_fullsized":"2024-05-30T17:14:17.822847+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'192","last_scrub_stamp":"2024-05-30T17:14:17.822619+0000","last_deep_scrub":"265'192","last_deep_scrub_stamp":"2024-05-30T17:14:17.822619+0000","last_clean_scrub_stamp":"2024-05-30T17:14:17.822619+0000","objects_scrubbed":4,"log_size":92,"log_dups_size":100,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:01:36.730473+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":491377,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":467,"num_read_kb":33008,"num_write":1090,"num_write_kb":27021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"265'851","reported_seq":2549,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:07.288471+0000","last_change":"2024-05-30T17:14:07.288543+0000","last_active":"2024-05-30T17:14:07.288471+0000","last_peered":"2024-05-30T17:14:07.288471+0000","last_clean":"2024-05-30T17:14:07.288471+0000","last_became_active":"2024-05-30T15:54:18.656475+0000","last_became_peered":"2024-05-30T15:54:18.656475+0000","last_unstale":"2024-05-30T17:14:07.288471+0000","last_undegraded":"2024-05-30T17:14:07.288471+0000","last_fullsized":"2024-05-30T17:14:07.288471+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'851","last_scrub_stamp":"2024-05-30T17:14:07.288276+0000","last_deep_scrub":"265'851","last_deep_scrub_stamp":"2024-05-30T17:14:07.288276+0000","last_clean_scrub_stamp":"2024-05-30T17:14:07.288276+0000","objects_scrubbed":1,"log_size":51,"log_dups_size":800,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:53:16.054837+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4088,"num_read_kb":3692,"num_write":1166,"num_write_kb":472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"265'140","reported_seq":738,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.074056+0000","last_change":"2024-05-30T15:54:16.669564+0000","last_active":"2024-05-30T17:13:43.074056+0000","last_peered":"2024-05-30T17:13:43.074056+0000","last_clean":"2024-05-30T17:13:43.074056+0000","last_became_active":"2024-05-30T15:54:16.669095+0000","last_became_peered":"2024-05-30T15:54:16.669095+0000","last_unstale":"2024-05-30T17:13:43.074056+0000","last_undegraded":"2024-05-30T17:13:43.074056+0000","last_fullsized":"2024-05-30T17:13:43.074056+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":100,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5243043,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":303,"num_read_kb":17751,"num_write":817,"num_write_kb":30378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"265'373","reported_seq":1803,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:05.355282+0000","last_change":"2024-05-30T17:14:05.355364+0000","last_active":"2024-05-30T17:14:05.355282+0000","last_peered":"2024-05-30T17:14:05.355282+0000","last_clean":"2024-05-30T17:14:05.355282+0000","last_became_active":"2024-05-30T15:54:18.653189+0000","last_became_peered":"2024-05-30T15:54:18.653189+0000","last_unstale":"2024-05-30T17:14:05.355282+0000","last_undegraded":"2024-05-30T17:14:05.355282+0000","last_fullsized":"2024-05-30T17:14:05.355282+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'373","last_scrub_stamp":"2024-05-30T17:14:05.355099+0000","last_deep_scrub":"265'373","last_deep_scrub_stamp":"2024-05-30T17:14:05.355099+0000","last_clean_scrub_stamp":"2024-05-30T17:14:05.355099+0000","objects_scrubbed":4,"log_size":73,"log_dups_size":300,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:26:58.416740+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1969,"num_read_kb":1855,"num_write":435,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"265'340","reported_seq":1778,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:12.082673+0000","last_change":"2024-05-30T17:14:12.082843+0000","last_active":"2024-05-30T17:14:12.082673+0000","last_peered":"2024-05-30T17:14:12.082673+0000","last_clean":"2024-05-30T17:14:12.082673+0000","last_became_active":"2024-05-30T15:54:18.649994+0000","last_became_peered":"2024-05-30T15:54:18.649994+0000","last_unstale":"2024-05-30T17:14:12.082673+0000","last_undegraded":"2024-05-30T17:14:12.082673+0000","last_fullsized":"2024-05-30T17:14:12.082673+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'340","last_scrub_stamp":"2024-05-30T17:14:12.082448+0000","last_deep_scrub":"265'340","last_deep_scrub_stamp":"2024-05-30T17:14:12.082448+0000","last_clean_scrub_stamp":"2024-05-30T17:14:12.082448+0000","objects_scrubbed":4,"log_size":40,"log_dups_size":300,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:35:49.760153+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1976,"num_read_kb":1893,"num_write":388,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":249,"num_omap_keys":1,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"265'212","reported_seq":879,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.051342+0000","last_change":"2024-05-30T15:54:16.650771+0000","last_active":"2024-05-30T17:13:45.051342+0000","last_peered":"2024-05-30T17:13:45.051342+0000","last_clean":"2024-05-30T17:13:45.051342+0000","last_became_active":"2024-05-30T15:54:16.649872+0000","last_became_peered":"2024-05-30T15:54:16.649872+0000","last_unstale":"2024-05-30T17:13:45.051342+0000","last_undegraded":"2024-05-30T17:13:45.051342+0000","last_fullsized":"2024-05-30T17:13:45.051342+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33349,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":510,"num_read_kb":59018,"num_write":1166,"num_write_kb":23092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"265'1526","reported_seq":3141,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:13.760005+0000","last_change":"2024-05-30T17:14:13.760080+0000","last_active":"2024-05-30T17:14:13.760005+0000","last_peered":"2024-05-30T17:14:13.760005+0000","last_clean":"2024-05-30T17:14:13.760005+0000","last_became_active":"2024-05-30T15:54:18.648860+0000","last_became_peered":"2024-05-30T15:54:18.648860+0000","last_unstale":"2024-05-30T17:14:13.760005+0000","last_undegraded":"2024-05-30T17:14:13.760005+0000","last_fullsized":"2024-05-30T17:14:13.760005+0000","mapping_epoch":24,"log_start":"265'1500","ondisk_log_start":"265'1500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'1526","last_scrub_stamp":"2024-05-30T17:14:13.759812+0000","last_deep_scrub":"265'1526","last_deep_scrub_stamp":"2024-05-30T17:14:13.759812+0000","last_clean_scrub_stamp":"2024-05-30T17:14:13.759812+0000","objects_scrubbed":1,"log_size":26,"log_dups_size":1500,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:47:16.863579+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6132,"num_read_kb":5297,"num_write":2138,"num_write_kb":912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"265'221","reported_seq":1093,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:13.810620+0000","last_change":"2024-05-30T17:14:13.810876+0000","last_active":"2024-05-30T17:14:13.810620+0000","last_peered":"2024-05-30T17:14:13.810620+0000","last_clean":"2024-05-30T17:14:13.810620+0000","last_became_active":"2024-05-30T15:54:16.686847+0000","last_became_peered":"2024-05-30T15:54:16.686847+0000","last_unstale":"2024-05-30T17:14:13.810620+0000","last_undegraded":"2024-05-30T17:14:13.810620+0000","last_fullsized":"2024-05-30T17:14:13.810620+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'221","last_scrub_stamp":"2024-05-30T17:14:13.810368+0000","last_deep_scrub":"265'221","last_deep_scrub_stamp":"2024-05-30T17:14:13.810368+0000","last_clean_scrub_stamp":"2024-05-30T17:14:13.810368+0000","objects_scrubbed":5,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:53:55.581906+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2728232,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":925,"num_read_kb":24537,"num_write":1233,"num_write_kb":54861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"265'1169","reported_seq":3018,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:04.292081+0000","last_change":"2024-05-30T17:14:04.292155+0000","last_active":"2024-05-30T17:14:04.292081+0000","last_peered":"2024-05-30T17:14:04.292081+0000","last_clean":"2024-05-30T17:14:04.292081+0000","last_became_active":"2024-05-30T15:54:18.665302+0000","last_became_peered":"2024-05-30T15:54:18.665302+0000","last_unstale":"2024-05-30T17:14:04.292081+0000","last_undegraded":"2024-05-30T17:14:04.292081+0000","last_fullsized":"2024-05-30T17:14:04.292081+0000","mapping_epoch":24,"log_start":"265'1100","ondisk_log_start":"265'1100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'1169","last_scrub_stamp":"2024-05-30T17:14:04.291893+0000","last_deep_scrub":"265'1169","last_deep_scrub_stamp":"2024-05-30T17:14:04.291893+0000","last_clean_scrub_stamp":"2024-05-30T17:14:04.291893+0000","objects_scrubbed":5,"log_size":69,"log_dups_size":1100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:33:21.886845+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5292,"num_read_kb":4714,"num_write":1593,"num_write_kb":642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3544,"num_omap_keys":14,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"265'197","reported_seq":928,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:08.780887+0000","last_change":"2024-05-30T17:14:08.781004+0000","last_active":"2024-05-30T17:14:08.780887+0000","last_peered":"2024-05-30T17:14:08.780887+0000","last_clean":"2024-05-30T17:14:08.780887+0000","last_became_active":"2024-05-30T15:54:16.668438+0000","last_became_peered":"2024-05-30T15:54:16.668438+0000","last_unstale":"2024-05-30T17:14:08.780887+0000","last_undegraded":"2024-05-30T17:14:08.780887+0000","last_fullsized":"2024-05-30T17:14:08.780887+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'197","last_scrub_stamp":"2024-05-30T17:14:08.780668+0000","last_deep_scrub":"265'197","last_deep_scrub_stamp":"2024-05-30T17:14:08.780668+0000","last_clean_scrub_stamp":"2024-05-30T17:14:08.780668+0000","objects_scrubbed":5,"log_size":97,"log_dups_size":100,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:59:06.022425+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":918,"num_read_kb":8276,"num_write":1173,"num_write_kb":35310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"265'772","reported_seq":2296,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:04.332160+0000","last_change":"2024-05-30T17:14:04.332160+0000","last_active":"2024-05-30T17:14:04.332160+0000","last_peered":"2024-05-30T17:14:04.332160+0000","last_clean":"2024-05-30T17:14:04.332160+0000","last_became_active":"2024-05-30T15:54:18.651197+0000","last_became_peered":"2024-05-30T15:54:18.651197+0000","last_unstale":"2024-05-30T17:14:04.332160+0000","last_undegraded":"2024-05-30T17:14:04.332160+0000","last_fullsized":"2024-05-30T17:14:04.332160+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'772","last_scrub_stamp":"2024-05-30T17:14:04.331893+0000","last_deep_scrub":"265'772","last_deep_scrub_stamp":"2024-05-30T17:14:04.331893+0000","last_clean_scrub_stamp":"2024-05-30T17:14:04.331893+0000","objects_scrubbed":4,"log_size":72,"log_dups_size":700,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:54:58.149438+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3544,"num_read_kb":3171,"num_write":1051,"num_write_kb":404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"265'190","reported_seq":852,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:20.131165+0000","last_change":"2024-05-30T17:14:20.131261+0000","last_active":"2024-05-30T17:14:20.131165+0000","last_peered":"2024-05-30T17:14:20.131165+0000","last_clean":"2024-05-30T17:14:20.131165+0000","last_became_active":"2024-05-30T15:54:16.676703+0000","last_became_peered":"2024-05-30T15:54:16.676703+0000","last_unstale":"2024-05-30T17:14:20.131165+0000","last_undegraded":"2024-05-30T17:14:20.131165+0000","last_fullsized":"2024-05-30T17:14:20.131165+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'190","last_scrub_stamp":"2024-05-30T17:14:20.130939+0000","last_deep_scrub":"265'190","last_deep_scrub_stamp":"2024-05-30T17:14:20.130939+0000","last_clean_scrub_stamp":"2024-05-30T17:14:20.130939+0000","objects_scrubbed":3,"log_size":90,"log_dups_size":100,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:16:27.631887+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":366,"num_read_kb":73313,"num_write":1062,"num_write_kb":47694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"265'376","reported_seq":1838,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:03.319819+0000","last_change":"2024-05-30T17:14:03.319819+0000","last_active":"2024-05-30T17:14:03.319819+0000","last_peered":"2024-05-30T17:14:03.319819+0000","last_clean":"2024-05-30T17:14:03.319819+0000","last_became_active":"2024-05-30T15:54:18.650255+0000","last_became_peered":"2024-05-30T15:54:18.650255+0000","last_unstale":"2024-05-30T17:14:03.319819+0000","last_undegraded":"2024-05-30T17:14:03.319819+0000","last_fullsized":"2024-05-30T17:14:03.319819+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'376","last_scrub_stamp":"2024-05-30T17:14:03.319457+0000","last_deep_scrub":"265'376","last_deep_scrub_stamp":"2024-05-30T17:14:03.319457+0000","last_clean_scrub_stamp":"2024-05-30T17:14:03.319457+0000","objects_scrubbed":4,"log_size":76,"log_dups_size":300,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:24:42.352764+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1969,"num_read_kb":1871,"num_write":435,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1010,"num_omap_keys":4,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"265'203","reported_seq":856,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.074027+0000","last_change":"2024-05-30T15:54:16.665743+0000","last_active":"2024-05-30T17:13:43.074027+0000","last_peered":"2024-05-30T17:13:43.074027+0000","last_clean":"2024-05-30T17:13:43.074027+0000","last_became_active":"2024-05-30T15:54:16.665344+0000","last_became_peered":"2024-05-30T15:54:16.665344+0000","last_unstale":"2024-05-30T17:13:43.074027+0000","last_undegraded":"2024-05-30T17:13:43.074027+0000","last_fullsized":"2024-05-30T17:13:43.074027+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":100,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6847332,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":477,"num_read_kb":7750,"num_write":1077,"num_write_kb":36544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"265'375","reported_seq":1802,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.566102+0000","last_change":"2024-05-30T15:54:18.645251+0000","last_active":"2024-05-30T17:13:43.566102+0000","last_peered":"2024-05-30T17:13:43.566102+0000","last_clean":"2024-05-30T17:13:43.566102+0000","last_became_active":"2024-05-30T15:54:18.643825+0000","last_became_peered":"2024-05-30T15:54:18.643825+0000","last_unstale":"2024-05-30T17:13:43.566102+0000","last_undegraded":"2024-05-30T17:13:43.566102+0000","last_fullsized":"2024-05-30T17:13:43.566102+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":300,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1936,"num_read_kb":1833,"num_write":434,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"265'238","reported_seq":911,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.647132+0000","last_change":"2024-05-30T15:54:16.638560+0000","last_active":"2024-05-30T17:13:41.647132+0000","last_peered":"2024-05-30T17:13:41.647132+0000","last_clean":"2024-05-30T17:13:41.647132+0000","last_became_active":"2024-05-30T15:54:16.637928+0000","last_became_peered":"2024-05-30T15:54:16.637928+0000","last_unstale":"2024-05-30T17:13:41.647132+0000","last_undegraded":"2024-05-30T17:13:41.647132+0000","last_fullsized":"2024-05-30T17:13:41.647132+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":200,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079922,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":526,"num_read_kb":65823,"num_write":1181,"num_write_kb":32044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"265'180","reported_seq":714,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.837308+0000","last_change":"2024-05-30T17:13:47.837308+0000","last_active":"2024-05-30T17:13:47.837308+0000","last_peered":"2024-05-30T17:13:47.837308+0000","last_clean":"2024-05-30T17:13:47.837308+0000","last_became_active":"2024-05-30T15:54:26.700957+0000","last_became_peered":"2024-05-30T15:54:26.700957+0000","last_unstale":"2024-05-30T17:13:47.837308+0000","last_undegraded":"2024-05-30T17:13:47.837308+0000","last_fullsized":"2024-05-30T17:13:47.837308+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'180","last_scrub_stamp":"2024-05-30T17:13:47.837040+0000","last_deep_scrub":"265'180","last_deep_scrub_stamp":"2024-05-30T17:13:47.837040+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.837040+0000","objects_scrubbed":5,"log_size":80,"log_dups_size":100,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:49:37.253715+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":270,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":132,"num_read_kb":126,"num_write":386,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"265'496","reported_seq":2033,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:18.368033+0000","last_change":"2024-05-30T17:14:18.368097+0000","last_active":"2024-05-30T17:14:18.368033+0000","last_peered":"2024-05-30T17:14:18.368033+0000","last_clean":"2024-05-30T17:14:18.368033+0000","last_became_active":"2024-05-30T15:54:18.652712+0000","last_became_peered":"2024-05-30T15:54:18.652712+0000","last_unstale":"2024-05-30T17:14:18.368033+0000","last_undegraded":"2024-05-30T17:14:18.368033+0000","last_fullsized":"2024-05-30T17:14:18.368033+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'496","last_scrub_stamp":"2024-05-30T17:14:18.367832+0000","last_deep_scrub":"265'496","last_deep_scrub_stamp":"2024-05-30T17:14:18.367832+0000","last_clean_scrub_stamp":"2024-05-30T17:14:18.367832+0000","objects_scrubbed":2,"log_size":96,"log_dups_size":400,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:39:11.375061+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2579,"num_read_kb":2377,"num_write":668,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":148,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"265'190","reported_seq":803,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.106668+0000","last_change":"2024-05-30T17:13:51.106756+0000","last_active":"2024-05-30T17:13:51.106668+0000","last_peered":"2024-05-30T17:13:51.106668+0000","last_clean":"2024-05-30T17:13:51.106668+0000","last_became_active":"2024-05-30T15:55:54.563874+0000","last_became_peered":"2024-05-30T15:55:54.563874+0000","last_unstale":"2024-05-30T17:13:51.106668+0000","last_undegraded":"2024-05-30T17:13:51.106668+0000","last_fullsized":"2024-05-30T17:13:51.106668+0000","mapping_epoch":37,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'190","last_scrub_stamp":"2024-05-30T17:13:51.106465+0000","last_deep_scrub":"265'190","last_deep_scrub_stamp":"2024-05-30T17:13:51.106465+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.106465+0000","objects_scrubbed":2,"log_size":90,"log_dups_size":100,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:21:06.302905+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079905,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":470,"num_read_kb":1589,"num_write":1089,"num_write_kb":27979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":516,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:12.375774+0000","last_change":"2024-05-30T17:14:12.375835+0000","last_active":"2024-05-30T17:14:12.375774+0000","last_peered":"2024-05-30T17:14:12.375774+0000","last_clean":"2024-05-30T17:14:12.375774+0000","last_became_active":"2024-05-30T15:54:20.658937+0000","last_became_peered":"2024-05-30T15:54:20.658937+0000","last_unstale":"2024-05-30T17:14:12.375774+0000","last_undegraded":"2024-05-30T17:14:12.375774+0000","last_fullsized":"2024-05-30T17:14:12.375774+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:12.375577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:12.375577+0000","last_clean_scrub_stamp":"2024-05-30T17:14:12.375577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:14:24.728898+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"265'202","reported_seq":830,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.647331+0000","last_change":"2024-05-30T15:54:22.668221+0000","last_active":"2024-05-30T17:13:41.647331+0000","last_peered":"2024-05-30T17:13:41.647331+0000","last_clean":"2024-05-30T17:13:41.647331+0000","last_became_active":"2024-05-30T15:54:22.667496+0000","last_became_peered":"2024-05-30T15:54:22.667496+0000","last_unstale":"2024-05-30T17:13:41.647331+0000","last_undegraded":"2024-05-30T17:13:41.647331+0000","last_fullsized":"2024-05-30T17:13:41.647331+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6140,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":285,"num_read_kb":355,"num_write":168,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":508,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:06.781850+0000","last_change":"2024-05-30T17:14:06.781850+0000","last_active":"2024-05-30T17:14:06.781850+0000","last_peered":"2024-05-30T17:14:06.781850+0000","last_clean":"2024-05-30T17:14:06.781850+0000","last_became_active":"2024-05-30T15:54:24.674491+0000","last_became_peered":"2024-05-30T15:54:24.674491+0000","last_unstale":"2024-05-30T17:14:06.781850+0000","last_undegraded":"2024-05-30T17:14:06.781850+0000","last_fullsized":"2024-05-30T17:14:06.781850+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:06.781591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:06.781591+0000","last_clean_scrub_stamp":"2024-05-30T17:14:06.781591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:22:20.251425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"265'199","reported_seq":784,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.234943+0000","last_change":"2024-05-30T17:13:53.234943+0000","last_active":"2024-05-30T17:13:53.234943+0000","last_peered":"2024-05-30T17:13:53.234943+0000","last_clean":"2024-05-30T17:13:53.234943+0000","last_became_active":"2024-05-30T15:54:26.686430+0000","last_became_peered":"2024-05-30T15:54:26.686430+0000","last_unstale":"2024-05-30T17:13:53.234943+0000","last_undegraded":"2024-05-30T17:13:53.234943+0000","last_fullsized":"2024-05-30T17:13:53.234943+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'199","last_scrub_stamp":"2024-05-30T17:13:53.234506+0000","last_deep_scrub":"265'199","last_deep_scrub_stamp":"2024-05-30T17:13:53.234506+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.234506+0000","objects_scrubbed":5,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:07:57.530934+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":824,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":238,"num_read_kb":223,"num_write":274,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"265'512","reported_seq":2172,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:03.288156+0000","last_change":"2024-05-30T17:14:03.288156+0000","last_active":"2024-05-30T17:14:03.288156+0000","last_peered":"2024-05-30T17:14:03.288156+0000","last_clean":"2024-05-30T17:14:03.288156+0000","last_became_active":"2024-05-30T15:54:18.661762+0000","last_became_peered":"2024-05-30T15:54:18.661762+0000","last_unstale":"2024-05-30T17:14:03.288156+0000","last_undegraded":"2024-05-30T17:14:03.288156+0000","last_fullsized":"2024-05-30T17:14:03.288156+0000","mapping_epoch":24,"log_start":"265'501","ondisk_log_start":"265'501","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'512","last_scrub_stamp":"2024-05-30T17:14:03.287889+0000","last_deep_scrub":"265'512","last_deep_scrub_stamp":"2024-05-30T17:14:03.287889+0000","last_clean_scrub_stamp":"2024-05-30T17:14:03.287889+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":501,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:21:44.477336+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2657,"num_read_kb":2481,"num_write":627,"num_write_kb":155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"265'210","reported_seq":867,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.427287+0000","last_change":"2024-05-30T15:54:16.639898+0000","last_active":"2024-05-30T17:13:38.427287+0000","last_peered":"2024-05-30T17:13:38.427287+0000","last_clean":"2024-05-30T17:13:38.427287+0000","last_became_active":"2024-05-30T15:54:16.639284+0000","last_became_peered":"2024-05-30T15:54:16.639284+0000","last_unstale":"2024-05-30T17:13:38.427287+0000","last_undegraded":"2024-05-30T17:13:38.427287+0000","last_fullsized":"2024-05-30T17:13:38.427287+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":100,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262178,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":445,"num_read_kb":15653,"num_write":1159,"num_write_kb":64581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":519,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:21.409846+0000","last_change":"2024-05-30T17:14:21.409904+0000","last_active":"2024-05-30T17:14:21.409846+0000","last_peered":"2024-05-30T17:14:21.409846+0000","last_clean":"2024-05-30T17:14:21.409846+0000","last_became_active":"2024-05-30T15:54:20.665488+0000","last_became_peered":"2024-05-30T15:54:20.665488+0000","last_unstale":"2024-05-30T17:14:21.409846+0000","last_undegraded":"2024-05-30T17:14:21.409846+0000","last_fullsized":"2024-05-30T17:14:21.409846+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:21.409644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:21.409644+0000","last_clean_scrub_stamp":"2024-05-30T17:14:21.409644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:41:07.021077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"265'105","reported_seq":666,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.970319+0000","last_change":"2024-05-30T17:13:59.970319+0000","last_active":"2024-05-30T17:13:59.970319+0000","last_peered":"2024-05-30T17:13:59.970319+0000","last_clean":"2024-05-30T17:13:59.970319+0000","last_became_active":"2024-05-30T15:54:22.666057+0000","last_became_peered":"2024-05-30T15:54:22.666057+0000","last_unstale":"2024-05-30T17:13:59.970319+0000","last_undegraded":"2024-05-30T17:13:59.970319+0000","last_fullsized":"2024-05-30T17:13:59.970319+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'105","last_scrub_stamp":"2024-05-30T17:13:59.969943+0000","last_deep_scrub":"265'105","last_deep_scrub_stamp":"2024-05-30T17:13:59.969943+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.969943+0000","objects_scrubbed":3,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:45:56.754937+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":113,"num_write":76,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":517,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:52.906758+0000","last_change":"2024-05-30T17:13:52.906758+0000","last_active":"2024-05-30T17:13:52.906758+0000","last_peered":"2024-05-30T17:13:52.906758+0000","last_clean":"2024-05-30T17:13:52.906758+0000","last_became_active":"2024-05-30T15:54:24.686595+0000","last_became_peered":"2024-05-30T15:54:24.686595+0000","last_unstale":"2024-05-30T17:13:52.906758+0000","last_undegraded":"2024-05-30T17:13:52.906758+0000","last_fullsized":"2024-05-30T17:13:52.906758+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:52.906486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:52.906486+0000","last_clean_scrub_stamp":"2024-05-30T17:13:52.906486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:22:22.141892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"265'103","reported_seq":625,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.717277+0000","last_change":"2024-05-30T17:13:53.717354+0000","last_active":"2024-05-30T17:13:53.717277+0000","last_peered":"2024-05-30T17:13:53.717277+0000","last_clean":"2024-05-30T17:13:53.717277+0000","last_became_active":"2024-05-30T15:54:26.688376+0000","last_became_peered":"2024-05-30T15:54:26.688376+0000","last_unstale":"2024-05-30T17:13:53.717277+0000","last_undegraded":"2024-05-30T17:13:53.717277+0000","last_fullsized":"2024-05-30T17:13:53.717277+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'103","last_scrub_stamp":"2024-05-30T17:13:53.717082+0000","last_deep_scrub":"265'103","last_deep_scrub_stamp":"2024-05-30T17:13:53.717082+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.717082+0000","objects_scrubbed":4,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:37:15.362960+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":271,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":103,"num_read_kb":102,"num_write":235,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"265'469","reported_seq":2028,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:09.932800+0000","last_change":"2024-05-30T17:14:09.932883+0000","last_active":"2024-05-30T17:14:09.932800+0000","last_peered":"2024-05-30T17:14:09.932800+0000","last_clean":"2024-05-30T17:14:09.932800+0000","last_became_active":"2024-05-30T15:54:18.657540+0000","last_became_peered":"2024-05-30T15:54:18.657540+0000","last_unstale":"2024-05-30T17:14:09.932800+0000","last_undegraded":"2024-05-30T17:14:09.932800+0000","last_fullsized":"2024-05-30T17:14:09.932800+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'469","last_scrub_stamp":"2024-05-30T17:14:09.932582+0000","last_deep_scrub":"265'469","last_deep_scrub_stamp":"2024-05-30T17:14:09.932582+0000","last_clean_scrub_stamp":"2024-05-30T17:14:09.932582+0000","objects_scrubbed":7,"log_size":69,"log_dups_size":400,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:37:05.257095+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2554,"num_read_kb":2362,"num_write":592,"num_write_kb":196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3556,"num_omap_keys":14,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"265'216","reported_seq":886,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:19.855125+0000","last_change":"2024-05-30T17:14:19.855235+0000","last_active":"2024-05-30T17:14:19.855125+0000","last_peered":"2024-05-30T17:14:19.855125+0000","last_clean":"2024-05-30T17:14:19.855125+0000","last_became_active":"2024-05-30T15:54:16.678659+0000","last_became_peered":"2024-05-30T15:54:16.678659+0000","last_unstale":"2024-05-30T17:14:19.855125+0000","last_undegraded":"2024-05-30T17:14:19.855125+0000","last_fullsized":"2024-05-30T17:14:19.855125+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'216","last_scrub_stamp":"2024-05-30T17:14:19.854896+0000","last_deep_scrub":"265'216","last_deep_scrub_stamp":"2024-05-30T17:14:19.854896+0000","last_clean_scrub_stamp":"2024-05-30T17:14:19.854896+0000","objects_scrubbed":4,"log_size":16,"log_dups_size":200,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:20:48.665014+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291507,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":524,"num_read_kb":29194,"num_write":1289,"num_write_kb":17953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":518,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:19.407506+0000","last_change":"2024-05-30T17:14:19.407560+0000","last_active":"2024-05-30T17:14:19.407506+0000","last_peered":"2024-05-30T17:14:19.407506+0000","last_clean":"2024-05-30T17:14:19.407506+0000","last_became_active":"2024-05-30T15:54:20.653316+0000","last_became_peered":"2024-05-30T15:54:20.653316+0000","last_unstale":"2024-05-30T17:14:19.407506+0000","last_undegraded":"2024-05-30T17:14:19.407506+0000","last_fullsized":"2024-05-30T17:14:19.407506+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:19.407301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:19.407301+0000","last_clean_scrub_stamp":"2024-05-30T17:14:19.407301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:47:55.159952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"265'155","reported_seq":727,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.161878+0000","last_change":"2024-05-30T17:13:48.161878+0000","last_active":"2024-05-30T17:13:48.161878+0000","last_peered":"2024-05-30T17:13:48.161878+0000","last_clean":"2024-05-30T17:13:48.161878+0000","last_became_active":"2024-05-30T15:54:22.664353+0000","last_became_peered":"2024-05-30T15:54:22.664353+0000","last_unstale":"2024-05-30T17:13:48.161878+0000","last_undegraded":"2024-05-30T17:13:48.161878+0000","last_fullsized":"2024-05-30T17:13:48.161878+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'155","last_scrub_stamp":"2024-05-30T17:13:48.161621+0000","last_deep_scrub":"265'155","last_deep_scrub_stamp":"2024-05-30T17:13:48.161621+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.161621+0000","objects_scrubbed":6,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:30:12.200451+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":173,"num_read_kb":165,"num_write":120,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"265'532","reported_seq":2492,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:03.691837+0000","last_change":"2024-05-30T17:14:03.691837+0000","last_active":"2024-05-30T17:14:03.691837+0000","last_peered":"2024-05-30T17:14:03.691837+0000","last_clean":"2024-05-30T17:14:03.691837+0000","last_became_active":"2024-05-30T15:54:24.681503+0000","last_became_peered":"2024-05-30T15:54:24.681503+0000","last_unstale":"2024-05-30T17:14:03.691837+0000","last_undegraded":"2024-05-30T17:14:03.691837+0000","last_fullsized":"2024-05-30T17:14:03.691837+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'532","last_scrub_stamp":"2024-05-30T17:14:03.691579+0000","last_deep_scrub":"265'532","last_deep_scrub_stamp":"2024-05-30T17:14:03.691579+0000","last_clean_scrub_stamp":"2024-05-30T17:14:03.691579+0000","objects_scrubbed":1,"log_size":32,"log_dups_size":500,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:45:49.735882+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"265'237","reported_seq":1087,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.884623+0000","last_change":"2024-05-30T17:13:51.884697+0000","last_active":"2024-05-30T17:13:51.884623+0000","last_peered":"2024-05-30T17:13:51.884623+0000","last_clean":"2024-05-30T17:13:51.884623+0000","last_became_active":"2024-05-30T15:54:26.670257+0000","last_became_peered":"2024-05-30T15:54:26.670257+0000","last_unstale":"2024-05-30T17:13:51.884623+0000","last_undegraded":"2024-05-30T17:13:51.884623+0000","last_fullsized":"2024-05-30T17:13:51.884623+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'237","last_scrub_stamp":"2024-05-30T17:13:51.884401+0000","last_deep_scrub":"265'237","last_deep_scrub_stamp":"2024-05-30T17:13:51.884401+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.884401+0000","objects_scrubbed":4,"log_size":37,"log_dups_size":200,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:24:37.651399+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":661,"num_read_kb":640,"num_write":540,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":234,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"265'409","reported_seq":2001,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:13.365832+0000","last_change":"2024-05-30T17:14:13.365901+0000","last_active":"2024-05-30T17:14:13.365832+0000","last_peered":"2024-05-30T17:14:13.365832+0000","last_clean":"2024-05-30T17:14:13.365832+0000","last_became_active":"2024-05-30T15:54:18.660656+0000","last_became_peered":"2024-05-30T15:54:18.660656+0000","last_unstale":"2024-05-30T17:14:13.365832+0000","last_undegraded":"2024-05-30T17:14:13.365832+0000","last_fullsized":"2024-05-30T17:14:13.365832+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'409","last_scrub_stamp":"2024-05-30T17:14:13.365635+0000","last_deep_scrub":"265'409","last_deep_scrub_stamp":"2024-05-30T17:14:13.365635+0000","last_clean_scrub_stamp":"2024-05-30T17:14:13.365635+0000","objects_scrubbed":3,"log_size":109,"log_dups_size":300,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:12:07.620435+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2368,"num_read_kb":2240,"num_write":473,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3015,"num_omap_keys":13,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"265'191","reported_seq":829,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:07.755507+0000","last_change":"2024-05-30T17:14:07.755692+0000","last_active":"2024-05-30T17:14:07.755507+0000","last_peered":"2024-05-30T17:14:07.755507+0000","last_clean":"2024-05-30T17:14:07.755507+0000","last_became_active":"2024-05-30T15:54:16.668083+0000","last_became_peered":"2024-05-30T15:54:16.668083+0000","last_unstale":"2024-05-30T17:14:07.755507+0000","last_undegraded":"2024-05-30T17:14:07.755507+0000","last_fullsized":"2024-05-30T17:14:07.755507+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'191","last_scrub_stamp":"2024-05-30T17:14:07.755193+0000","last_deep_scrub":"265'191","last_deep_scrub_stamp":"2024-05-30T17:14:07.755193+0000","last_clean_scrub_stamp":"2024-05-30T17:14:07.755193+0000","objects_scrubbed":3,"log_size":91,"log_dups_size":100,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:28:20.087604+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5263487,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":406,"num_read_kb":13963,"num_write":1054,"num_write_kb":66137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":516,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:10.355088+0000","last_change":"2024-05-30T17:14:10.355160+0000","last_active":"2024-05-30T17:14:10.355088+0000","last_peered":"2024-05-30T17:14:10.355088+0000","last_clean":"2024-05-30T17:14:10.355088+0000","last_became_active":"2024-05-30T15:54:20.658870+0000","last_became_peered":"2024-05-30T15:54:20.658870+0000","last_unstale":"2024-05-30T17:14:10.355088+0000","last_undegraded":"2024-05-30T17:14:10.355088+0000","last_fullsized":"2024-05-30T17:14:10.355088+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:10.354868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:10.354868+0000","last_clean_scrub_stamp":"2024-05-30T17:14:10.354868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:31:51.606017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"265'146","reported_seq":743,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:09.748021+0000","last_change":"2024-05-30T17:14:09.748021+0000","last_active":"2024-05-30T17:14:09.748021+0000","last_peered":"2024-05-30T17:14:09.748021+0000","last_clean":"2024-05-30T17:14:09.748021+0000","last_became_active":"2024-05-30T15:54:22.849978+0000","last_became_peered":"2024-05-30T15:54:22.849978+0000","last_unstale":"2024-05-30T17:14:09.748021+0000","last_undegraded":"2024-05-30T17:14:09.748021+0000","last_fullsized":"2024-05-30T17:14:09.748021+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'146","last_scrub_stamp":"2024-05-30T17:14:09.747755+0000","last_deep_scrub":"265'146","last_deep_scrub_stamp":"2024-05-30T17:14:09.747755+0000","last_clean_scrub_stamp":"2024-05-30T17:14:09.747755+0000","objects_scrubbed":4,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:28:48.260787+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8153,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":189,"num_read_kb":241,"num_write":102,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"265'549","reported_seq":2582,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:05.388708+0000","last_change":"2024-05-30T17:14:05.388708+0000","last_active":"2024-05-30T17:14:05.388708+0000","last_peered":"2024-05-30T17:14:05.388708+0000","last_clean":"2024-05-30T17:14:05.388708+0000","last_became_active":"2024-05-30T15:54:24.686546+0000","last_became_peered":"2024-05-30T15:54:24.686546+0000","last_unstale":"2024-05-30T17:14:05.388708+0000","last_undegraded":"2024-05-30T17:14:05.388708+0000","last_fullsized":"2024-05-30T17:14:05.388708+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'549","last_scrub_stamp":"2024-05-30T17:14:05.388354+0000","last_deep_scrub":"265'549","last_deep_scrub_stamp":"2024-05-30T17:14:05.388354+0000","last_clean_scrub_stamp":"2024-05-30T17:14:05.388354+0000","objects_scrubbed":1,"log_size":49,"log_dups_size":500,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:59:56.299339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"265'901","reported_seq":2525,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:05.283950+0000","last_change":"2024-05-30T17:14:05.284034+0000","last_active":"2024-05-30T17:14:05.283950+0000","last_peered":"2024-05-30T17:14:05.283950+0000","last_clean":"2024-05-30T17:14:05.283950+0000","last_became_active":"2024-05-30T15:54:18.658124+0000","last_became_peered":"2024-05-30T15:54:18.658124+0000","last_unstale":"2024-05-30T17:14:05.283950+0000","last_undegraded":"2024-05-30T17:14:05.283950+0000","last_fullsized":"2024-05-30T17:14:05.283950+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'901","last_scrub_stamp":"2024-05-30T17:14:05.283757+0000","last_deep_scrub":"265'901","last_deep_scrub_stamp":"2024-05-30T17:14:05.283757+0000","last_clean_scrub_stamp":"2024-05-30T17:14:05.283757+0000","objects_scrubbed":6,"log_size":101,"log_dups_size":800,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:45:18.502796+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4127,"num_read_kb":3657,"num_write":1246,"num_write_kb":491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2810,"num_omap_keys":11,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"265'208","reported_seq":889,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.307115+0000","last_change":"2024-05-30T15:54:16.659141+0000","last_active":"2024-05-30T17:13:38.307115+0000","last_peered":"2024-05-30T17:13:38.307115+0000","last_clean":"2024-05-30T17:13:38.307115+0000","last_became_active":"2024-05-30T15:54:16.658279+0000","last_became_peered":"2024-05-30T15:54:16.658279+0000","last_unstale":"2024-05-30T17:13:38.307115+0000","last_undegraded":"2024-05-30T17:13:38.307115+0000","last_fullsized":"2024-05-30T17:13:38.307115+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":100,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1254547,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":480,"num_read_kb":111643,"num_write":1134,"num_write_kb":26225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":519,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:14.742285+0000","last_change":"2024-05-30T17:14:14.742349+0000","last_active":"2024-05-30T17:14:14.742285+0000","last_peered":"2024-05-30T17:14:14.742285+0000","last_clean":"2024-05-30T17:14:14.742285+0000","last_became_active":"2024-05-30T15:54:20.661583+0000","last_became_peered":"2024-05-30T15:54:20.661583+0000","last_unstale":"2024-05-30T17:14:14.742285+0000","last_undegraded":"2024-05-30T17:14:14.742285+0000","last_fullsized":"2024-05-30T17:14:14.742285+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:14.742060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:14.742060+0000","last_clean_scrub_stamp":"2024-05-30T17:14:14.742060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:08:49.700663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"265'265","reported_seq":921,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.257602+0000","last_change":"2024-05-30T17:13:58.257702+0000","last_active":"2024-05-30T17:13:58.257602+0000","last_peered":"2024-05-30T17:13:58.257602+0000","last_clean":"2024-05-30T17:13:58.257602+0000","last_became_active":"2024-05-30T15:54:22.849126+0000","last_became_peered":"2024-05-30T15:54:22.849126+0000","last_unstale":"2024-05-30T17:13:58.257602+0000","last_undegraded":"2024-05-30T17:13:58.257602+0000","last_fullsized":"2024-05-30T17:13:58.257602+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'265","last_scrub_stamp":"2024-05-30T17:13:58.257369+0000","last_deep_scrub":"265'265","last_deep_scrub_stamp":"2024-05-30T17:13:58.257369+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.257369+0000","objects_scrubbed":10,"log_size":65,"log_dups_size":200,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:15:48.980020+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6773,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":352,"num_read_kb":376,"num_write":210,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":511,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:07.996809+0000","last_change":"2024-05-30T17:14:07.996872+0000","last_active":"2024-05-30T17:14:07.996809+0000","last_peered":"2024-05-30T17:14:07.996809+0000","last_clean":"2024-05-30T17:14:07.996809+0000","last_became_active":"2024-05-30T15:54:24.673506+0000","last_became_peered":"2024-05-30T15:54:24.673506+0000","last_unstale":"2024-05-30T17:14:07.996809+0000","last_undegraded":"2024-05-30T17:14:07.996809+0000","last_fullsized":"2024-05-30T17:14:07.996809+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:07.996581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:07.996581+0000","last_clean_scrub_stamp":"2024-05-30T17:14:07.996581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:59:09.003040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"265'546","reported_seq":1385,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.226322+0000","last_change":"2024-05-30T17:13:54.226466+0000","last_active":"2024-05-30T17:13:54.226322+0000","last_peered":"2024-05-30T17:13:54.226322+0000","last_clean":"2024-05-30T17:13:54.226322+0000","last_became_active":"2024-05-30T15:54:26.686389+0000","last_became_peered":"2024-05-30T15:54:26.686389+0000","last_unstale":"2024-05-30T17:13:54.226322+0000","last_undegraded":"2024-05-30T17:13:54.226322+0000","last_fullsized":"2024-05-30T17:13:54.226322+0000","mapping_epoch":32,"log_start":"265'500","ondisk_log_start":"265'500","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'546","last_scrub_stamp":"2024-05-30T17:13:54.226029+0000","last_deep_scrub":"265'546","last_deep_scrub_stamp":"2024-05-30T17:13:54.226029+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.226029+0000","objects_scrubbed":3,"log_size":46,"log_dups_size":500,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:20:00.749166+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1297,"num_read_kb":1288,"num_write":1137,"num_write_kb":385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":381,"num_omap_keys":2,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"265'1398","reported_seq":3106,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:18.763641+0000","last_change":"2024-05-30T15:54:18.640378+0000","last_active":"2024-05-30T17:13:18.763641+0000","last_peered":"2024-05-30T17:13:18.763641+0000","last_clean":"2024-05-30T17:13:18.763641+0000","last_became_active":"2024-05-30T15:54:18.639667+0000","last_became_peered":"2024-05-30T15:54:18.639667+0000","last_unstale":"2024-05-30T17:13:18.763641+0000","last_undegraded":"2024-05-30T17:13:18.763641+0000","last_fullsized":"2024-05-30T17:13:18.763641+0000","mapping_epoch":24,"log_start":"265'1300","ondisk_log_start":"265'1300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":1300,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5970,"num_read_kb":5130,"num_write":2044,"num_write_kb":884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"265'213","reported_seq":868,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.371059+0000","last_change":"2024-05-30T15:54:16.661968+0000","last_active":"2024-05-30T17:13:38.371059+0000","last_peered":"2024-05-30T17:13:38.371059+0000","last_clean":"2024-05-30T17:13:38.371059+0000","last_became_active":"2024-05-30T15:54:16.659427+0000","last_became_peered":"2024-05-30T15:54:16.659427+0000","last_unstale":"2024-05-30T17:13:38.371059+0000","last_undegraded":"2024-05-30T17:13:38.371059+0000","last_fullsized":"2024-05-30T17:13:38.371059+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":200,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":467,"num_read_kb":34119,"num_write":1157,"num_write_kb":42367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":522,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.231419+0000","last_change":"2024-05-30T17:13:58.231474+0000","last_active":"2024-05-30T17:13:58.231419+0000","last_peered":"2024-05-30T17:13:58.231419+0000","last_clean":"2024-05-30T17:13:58.231419+0000","last_became_active":"2024-05-30T15:54:20.647565+0000","last_became_peered":"2024-05-30T15:54:20.647565+0000","last_unstale":"2024-05-30T17:13:58.231419+0000","last_undegraded":"2024-05-30T17:13:58.231419+0000","last_fullsized":"2024-05-30T17:13:58.231419+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:58.231223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:58.231223+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.231223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:53:45.579167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"265'146","reported_seq":739,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.312031+0000","last_change":"2024-05-30T15:54:22.671276+0000","last_active":"2024-05-30T17:13:38.312031+0000","last_peered":"2024-05-30T17:13:38.312031+0000","last_clean":"2024-05-30T17:13:38.312031+0000","last_became_active":"2024-05-30T15:54:22.670064+0000","last_became_peered":"2024-05-30T15:54:22.670064+0000","last_unstale":"2024-05-30T17:13:38.312031+0000","last_undegraded":"2024-05-30T17:13:38.312031+0000","last_fullsized":"2024-05-30T17:13:38.312031+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12270,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":197,"num_read_kb":284,"num_write":108,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":513,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:02.359550+0000","last_change":"2024-05-30T17:14:02.359602+0000","last_active":"2024-05-30T17:14:02.359550+0000","last_peered":"2024-05-30T17:14:02.359550+0000","last_clean":"2024-05-30T17:14:02.359550+0000","last_became_active":"2024-05-30T15:54:24.688046+0000","last_became_peered":"2024-05-30T15:54:24.688046+0000","last_unstale":"2024-05-30T17:14:02.359550+0000","last_undegraded":"2024-05-30T17:14:02.359550+0000","last_fullsized":"2024-05-30T17:14:02.359550+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:02.359352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:02.359352+0000","last_clean_scrub_stamp":"2024-05-30T17:14:02.359352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:10:48.308830+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"265'2370","reported_seq":4200,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:04.738831+0000","last_change":"2024-05-30T17:14:04.738949+0000","last_active":"2024-05-30T17:14:04.738831+0000","last_peered":"2024-05-30T17:14:04.738831+0000","last_clean":"2024-05-30T17:14:04.738831+0000","last_became_active":"2024-05-30T15:54:26.691574+0000","last_became_peered":"2024-05-30T15:54:26.691574+0000","last_unstale":"2024-05-30T17:14:04.738831+0000","last_undegraded":"2024-05-30T17:14:04.738831+0000","last_fullsized":"2024-05-30T17:14:04.738831+0000","mapping_epoch":32,"log_start":"265'2300","ondisk_log_start":"265'2300","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2370","last_scrub_stamp":"2024-05-30T17:14:04.738577+0000","last_deep_scrub":"265'2370","last_deep_scrub_stamp":"2024-05-30T17:14:04.738577+0000","last_clean_scrub_stamp":"2024-05-30T17:14:04.738577+0000","objects_scrubbed":1,"log_size":70,"log_dups_size":2300,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:55:13.090443+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5943,"num_read_kb":5935,"num_write":4785,"num_write_kb":1767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":547,"num_omap_keys":3,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"265'373","reported_seq":1803,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:01.825393+0000","last_change":"2024-05-30T17:14:01.825479+0000","last_active":"2024-05-30T17:14:01.825393+0000","last_peered":"2024-05-30T17:14:01.825393+0000","last_clean":"2024-05-30T17:14:01.825393+0000","last_became_active":"2024-05-30T15:54:18.668963+0000","last_became_peered":"2024-05-30T15:54:18.668963+0000","last_unstale":"2024-05-30T17:14:01.825393+0000","last_undegraded":"2024-05-30T17:14:01.825393+0000","last_fullsized":"2024-05-30T17:14:01.825393+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'373","last_scrub_stamp":"2024-05-30T17:14:01.825217+0000","last_deep_scrub":"265'373","last_deep_scrub_stamp":"2024-05-30T17:14:01.825217+0000","last_clean_scrub_stamp":"2024-05-30T17:14:01.825217+0000","objects_scrubbed":2,"log_size":73,"log_dups_size":300,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:14:41.634577+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2011,"num_read_kb":1890,"num_write":455,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":225,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"265'240","reported_seq":881,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.426739+0000","last_change":"2024-05-30T15:54:16.650507+0000","last_active":"2024-05-30T17:13:38.426739+0000","last_peered":"2024-05-30T17:13:38.426739+0000","last_clean":"2024-05-30T17:13:38.426739+0000","last_became_active":"2024-05-30T15:54:16.649605+0000","last_became_peered":"2024-05-30T15:54:16.649605+0000","last_unstale":"2024-05-30T17:13:38.426739+0000","last_undegraded":"2024-05-30T17:13:38.426739+0000","last_fullsized":"2024-05-30T17:13:38.426739+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":200,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242911,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":469,"num_read_kb":32402,"num_write":1238,"num_write_kb":40071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":521,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.170085+0000","last_change":"2024-05-30T17:13:56.170146+0000","last_active":"2024-05-30T17:13:56.170085+0000","last_peered":"2024-05-30T17:13:56.170085+0000","last_clean":"2024-05-30T17:13:56.170085+0000","last_became_active":"2024-05-30T15:54:20.651369+0000","last_became_peered":"2024-05-30T15:54:20.651369+0000","last_unstale":"2024-05-30T17:13:56.170085+0000","last_undegraded":"2024-05-30T17:13:56.170085+0000","last_fullsized":"2024-05-30T17:13:56.170085+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:56.169880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:56.169880+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.169880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:32:45.358384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"265'191","reported_seq":807,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.101941+0000","last_change":"2024-05-30T15:54:22.666930+0000","last_active":"2024-05-30T17:13:38.101941+0000","last_peered":"2024-05-30T17:13:38.101941+0000","last_clean":"2024-05-30T17:13:38.101941+0000","last_became_active":"2024-05-30T15:54:22.664874+0000","last_became_peered":"2024-05-30T15:54:22.664874+0000","last_unstale":"2024-05-30T17:13:38.101941+0000","last_undegraded":"2024-05-30T17:13:38.101941+0000","last_fullsized":"2024-05-30T17:13:38.101941+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":100,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6309,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":259,"num_read_kb":308,"num_write":148,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"265'549","reported_seq":2491,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:06.842878+0000","last_change":"2024-05-30T17:14:06.842942+0000","last_active":"2024-05-30T17:14:06.842878+0000","last_peered":"2024-05-30T17:14:06.842878+0000","last_clean":"2024-05-30T17:14:06.842878+0000","last_became_active":"2024-05-30T15:54:24.670141+0000","last_became_peered":"2024-05-30T15:54:24.670141+0000","last_unstale":"2024-05-30T17:14:06.842878+0000","last_undegraded":"2024-05-30T17:14:06.842878+0000","last_fullsized":"2024-05-30T17:14:06.842878+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'549","last_scrub_stamp":"2024-05-30T17:14:06.842673+0000","last_deep_scrub":"265'549","last_deep_scrub_stamp":"2024-05-30T17:14:06.842673+0000","last_clean_scrub_stamp":"2024-05-30T17:14:06.842673+0000","objects_scrubbed":1,"log_size":49,"log_dups_size":500,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:16:17.488221+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"265'107","reported_seq":620,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:52.677135+0000","last_change":"2024-05-30T17:13:52.677276+0000","last_active":"2024-05-30T17:13:52.677135+0000","last_peered":"2024-05-30T17:13:52.677135+0000","last_clean":"2024-05-30T17:13:52.677135+0000","last_became_active":"2024-05-30T15:54:26.689987+0000","last_became_peered":"2024-05-30T15:54:26.689987+0000","last_unstale":"2024-05-30T17:13:52.677135+0000","last_undegraded":"2024-05-30T17:13:52.677135+0000","last_fullsized":"2024-05-30T17:13:52.677135+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'107","last_scrub_stamp":"2024-05-30T17:13:52.676848+0000","last_deep_scrub":"265'107","last_deep_scrub_stamp":"2024-05-30T17:13:52.676848+0000","last_clean_scrub_stamp":"2024-05-30T17:13:52.676848+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:48:23.065085+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":80,"num_write":254,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"265'1335","reported_seq":2979,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:16.822597+0000","last_change":"2024-05-30T17:14:16.822667+0000","last_active":"2024-05-30T17:14:16.822597+0000","last_peered":"2024-05-30T17:14:16.822597+0000","last_clean":"2024-05-30T17:14:16.822597+0000","last_became_active":"2024-05-30T15:54:18.647634+0000","last_became_peered":"2024-05-30T15:54:18.647634+0000","last_unstale":"2024-05-30T17:14:16.822597+0000","last_undegraded":"2024-05-30T17:14:16.822597+0000","last_fullsized":"2024-05-30T17:14:16.822597+0000","mapping_epoch":24,"log_start":"265'1300","ondisk_log_start":"265'1300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'1335","last_scrub_stamp":"2024-05-30T17:14:16.822401+0000","last_deep_scrub":"265'1335","last_deep_scrub_stamp":"2024-05-30T17:14:16.822401+0000","last_clean_scrub_stamp":"2024-05-30T17:14:16.822401+0000","objects_scrubbed":4,"log_size":35,"log_dups_size":1300,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:44:49.543711+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5717,"num_read_kb":4967,"num_write":1912,"num_write_kb":881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"265'206","reported_seq":822,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.427898+0000","last_change":"2024-05-30T15:54:16.668968+0000","last_active":"2024-05-30T17:13:38.427898+0000","last_peered":"2024-05-30T17:13:38.427898+0000","last_clean":"2024-05-30T17:13:38.427898+0000","last_became_active":"2024-05-30T15:54:16.668336+0000","last_became_peered":"2024-05-30T15:54:16.668336+0000","last_unstale":"2024-05-30T17:13:38.427898+0000","last_undegraded":"2024-05-30T17:13:38.427898+0000","last_fullsized":"2024-05-30T17:13:38.427898+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":100,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":403,"num_read_kb":1534,"num_write":1058,"num_write_kb":21253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":522,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.159988+0000","last_change":"2024-05-30T17:13:54.160046+0000","last_active":"2024-05-30T17:13:54.159988+0000","last_peered":"2024-05-30T17:13:54.159988+0000","last_clean":"2024-05-30T17:13:54.159988+0000","last_became_active":"2024-05-30T15:54:20.655573+0000","last_became_peered":"2024-05-30T15:54:20.655573+0000","last_unstale":"2024-05-30T17:13:54.159988+0000","last_undegraded":"2024-05-30T17:13:54.159988+0000","last_fullsized":"2024-05-30T17:13:54.159988+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:54.159789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:54.159789+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.159789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:27:43.050145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"265'212","reported_seq":879,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:19.794482+0000","last_change":"2024-05-30T17:14:19.794572+0000","last_active":"2024-05-30T17:14:19.794482+0000","last_peered":"2024-05-30T17:14:19.794482+0000","last_clean":"2024-05-30T17:14:19.794482+0000","last_became_active":"2024-05-30T15:54:22.658700+0000","last_became_peered":"2024-05-30T15:54:22.658700+0000","last_unstale":"2024-05-30T17:14:19.794482+0000","last_undegraded":"2024-05-30T17:14:19.794482+0000","last_fullsized":"2024-05-30T17:14:19.794482+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'212","last_scrub_stamp":"2024-05-30T17:14:19.794300+0000","last_deep_scrub":"265'212","last_deep_scrub_stamp":"2024-05-30T17:14:19.794300+0000","last_clean_scrub_stamp":"2024-05-30T17:14:19.794300+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:25:02.091631+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14119,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":309,"num_read_kb":443,"num_write":154,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":514,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.048165+0000","last_change":"2024-05-30T17:13:43.048221+0000","last_active":"2024-05-30T17:13:43.048165+0000","last_peered":"2024-05-30T17:13:43.048165+0000","last_clean":"2024-05-30T17:13:43.048165+0000","last_became_active":"2024-05-30T15:54:24.673505+0000","last_became_peered":"2024-05-30T15:54:24.673505+0000","last_unstale":"2024-05-30T17:13:43.048165+0000","last_undegraded":"2024-05-30T17:13:43.048165+0000","last_fullsized":"2024-05-30T17:13:43.048165+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:43.047974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:43.047974+0000","last_clean_scrub_stamp":"2024-05-30T17:13:43.047974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:04:22.180317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"265'101","reported_seq":627,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.954947+0000","last_change":"2024-05-30T17:13:50.955055+0000","last_active":"2024-05-30T17:13:50.954947+0000","last_peered":"2024-05-30T17:13:50.954947+0000","last_clean":"2024-05-30T17:13:50.954947+0000","last_became_active":"2024-05-30T15:54:26.700608+0000","last_became_peered":"2024-05-30T15:54:26.700608+0000","last_unstale":"2024-05-30T17:13:50.954947+0000","last_undegraded":"2024-05-30T17:13:50.954947+0000","last_fullsized":"2024-05-30T17:13:50.954947+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'101","last_scrub_stamp":"2024-05-30T17:13:50.954735+0000","last_deep_scrub":"265'101","last_deep_scrub_stamp":"2024-05-30T17:13:50.954735+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.954735+0000","objects_scrubbed":2,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:48:33.984457+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":918,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":103,"num_write":225,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"265'544","reported_seq":2181,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:10.057108+0000","last_change":"2024-05-30T17:14:10.057194+0000","last_active":"2024-05-30T17:14:10.057108+0000","last_peered":"2024-05-30T17:14:10.057108+0000","last_clean":"2024-05-30T17:14:10.057108+0000","last_became_active":"2024-05-30T15:54:18.657522+0000","last_became_peered":"2024-05-30T15:54:18.657522+0000","last_unstale":"2024-05-30T17:14:10.057108+0000","last_undegraded":"2024-05-30T17:14:10.057108+0000","last_fullsized":"2024-05-30T17:14:10.057108+0000","mapping_epoch":24,"log_start":"265'500","ondisk_log_start":"265'500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'544","last_scrub_stamp":"2024-05-30T17:14:10.056929+0000","last_deep_scrub":"265'544","last_deep_scrub_stamp":"2024-05-30T17:14:10.056929+0000","last_clean_scrub_stamp":"2024-05-30T17:14:10.056929+0000","objects_scrubbed":5,"log_size":44,"log_dups_size":500,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:06:50.095596+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2751,"num_read_kb":2537,"num_write":708,"num_write_kb":252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":303,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"265'234","reported_seq":916,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.305058+0000","last_change":"2024-05-30T15:54:16.666646+0000","last_active":"2024-05-30T17:13:38.305058+0000","last_peered":"2024-05-30T17:13:38.305058+0000","last_clean":"2024-05-30T17:13:38.305058+0000","last_became_active":"2024-05-30T15:54:16.666244+0000","last_became_peered":"2024-05-30T15:54:16.666244+0000","last_unstale":"2024-05-30T17:13:38.305058+0000","last_undegraded":"2024-05-30T17:13:38.305058+0000","last_fullsized":"2024-05-30T17:13:38.305058+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":200,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":549856,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":638,"num_read_kb":32957,"num_write":1225,"num_write_kb":40334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":521,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:52.077511+0000","last_change":"2024-05-30T17:13:52.077571+0000","last_active":"2024-05-30T17:13:52.077511+0000","last_peered":"2024-05-30T17:13:52.077511+0000","last_clean":"2024-05-30T17:13:52.077511+0000","last_became_active":"2024-05-30T15:54:20.661967+0000","last_became_peered":"2024-05-30T15:54:20.661967+0000","last_unstale":"2024-05-30T17:13:52.077511+0000","last_undegraded":"2024-05-30T17:13:52.077511+0000","last_fullsized":"2024-05-30T17:13:52.077511+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:52.077316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:52.077316+0000","last_clean_scrub_stamp":"2024-05-30T17:13:52.077316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:55:45.987890+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"265'181","reported_seq":789,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.385977+0000","last_change":"2024-05-30T15:54:22.665466+0000","last_active":"2024-05-30T17:13:38.385977+0000","last_peered":"2024-05-30T17:13:38.385977+0000","last_clean":"2024-05-30T17:13:38.385977+0000","last_became_active":"2024-05-30T15:54:22.664732+0000","last_became_peered":"2024-05-30T15:54:22.664732+0000","last_unstale":"2024-05-30T17:13:38.385977+0000","last_undegraded":"2024-05-30T17:13:38.385977+0000","last_fullsized":"2024-05-30T17:13:38.385977+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":100,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13534,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":265,"num_read_kb":366,"num_write":202,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":515,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:01.839426+0000","last_change":"2024-05-30T17:14:01.839579+0000","last_active":"2024-05-30T17:14:01.839426+0000","last_peered":"2024-05-30T17:14:01.839426+0000","last_clean":"2024-05-30T17:14:01.839426+0000","last_became_active":"2024-05-30T15:54:24.679254+0000","last_became_peered":"2024-05-30T15:54:24.679254+0000","last_unstale":"2024-05-30T17:14:01.839426+0000","last_undegraded":"2024-05-30T17:14:01.839426+0000","last_fullsized":"2024-05-30T17:14:01.839426+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:01.839200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:01.839200+0000","last_clean_scrub_stamp":"2024-05-30T17:14:01.839200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:13:39.135342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"265'109","reported_seq":636,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:57.389981+0000","last_change":"2024-05-30T17:13:57.390054+0000","last_active":"2024-05-30T17:13:57.389981+0000","last_peered":"2024-05-30T17:13:57.389981+0000","last_clean":"2024-05-30T17:13:57.389981+0000","last_became_active":"2024-05-30T15:54:26.697333+0000","last_became_peered":"2024-05-30T15:54:26.697333+0000","last_unstale":"2024-05-30T17:13:57.389981+0000","last_undegraded":"2024-05-30T17:13:57.389981+0000","last_fullsized":"2024-05-30T17:13:57.389981+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'109","last_scrub_stamp":"2024-05-30T17:13:57.389793+0000","last_deep_scrub":"265'109","last_deep_scrub_stamp":"2024-05-30T17:13:57.389793+0000","last_clean_scrub_stamp":"2024-05-30T17:13:57.389793+0000","objects_scrubbed":4,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:34:55.402057+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":270,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":107,"num_read_kb":90,"num_write":242,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"265'397","reported_seq":2060,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:12.231233+0000","last_change":"2024-05-30T15:54:18.636254+0000","last_active":"2024-05-30T17:13:12.231233+0000","last_peered":"2024-05-30T17:13:12.231233+0000","last_clean":"2024-05-30T17:13:12.231233+0000","last_became_active":"2024-05-30T15:54:18.635519+0000","last_became_peered":"2024-05-30T15:54:18.635519+0000","last_unstale":"2024-05-30T17:13:12.231233+0000","last_undegraded":"2024-05-30T17:13:12.231233+0000","last_fullsized":"2024-05-30T17:13:12.231233+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":300,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2332,"num_read_kb":2242,"num_write":450,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"265'175","reported_seq":804,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:12.798118+0000","last_change":"2024-05-30T17:14:12.798266+0000","last_active":"2024-05-30T17:14:12.798118+0000","last_peered":"2024-05-30T17:14:12.798118+0000","last_clean":"2024-05-30T17:14:12.798118+0000","last_became_active":"2024-05-30T15:54:16.685162+0000","last_became_peered":"2024-05-30T15:54:16.685162+0000","last_unstale":"2024-05-30T17:14:12.798118+0000","last_undegraded":"2024-05-30T17:14:12.798118+0000","last_fullsized":"2024-05-30T17:14:12.798118+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'175","last_scrub_stamp":"2024-05-30T17:14:12.797857+0000","last_deep_scrub":"265'175","last_deep_scrub_stamp":"2024-05-30T17:14:12.797857+0000","last_clean_scrub_stamp":"2024-05-30T17:14:12.797857+0000","objects_scrubbed":6,"log_size":75,"log_dups_size":100,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:43:33.318612+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5904316,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":401,"num_read_kb":10586,"num_write":1027,"num_write_kb":21029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":519,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.991573+0000","last_change":"2024-05-30T17:13:56.991631+0000","last_active":"2024-05-30T17:13:56.991573+0000","last_peered":"2024-05-30T17:13:56.991573+0000","last_clean":"2024-05-30T17:13:56.991573+0000","last_became_active":"2024-05-30T15:54:20.646646+0000","last_became_peered":"2024-05-30T15:54:20.646646+0000","last_unstale":"2024-05-30T17:13:56.991573+0000","last_undegraded":"2024-05-30T17:13:56.991573+0000","last_fullsized":"2024-05-30T17:13:56.991573+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:56.991372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:56.991372+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.991372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:25:19.485498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"265'189","reported_seq":815,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.157684+0000","last_change":"2024-05-30T17:13:46.157858+0000","last_active":"2024-05-30T17:13:46.157684+0000","last_peered":"2024-05-30T17:13:46.157684+0000","last_clean":"2024-05-30T17:13:46.157684+0000","last_became_active":"2024-05-30T15:54:22.669660+0000","last_became_peered":"2024-05-30T15:54:22.669660+0000","last_unstale":"2024-05-30T17:13:46.157684+0000","last_undegraded":"2024-05-30T17:13:46.157684+0000","last_fullsized":"2024-05-30T17:13:46.157684+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'189","last_scrub_stamp":"2024-05-30T17:13:46.157487+0000","last_deep_scrub":"265'189","last_deep_scrub_stamp":"2024-05-30T17:13:46.157487+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.157487+0000","objects_scrubbed":7,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:49:02.360024+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6014,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":258,"num_read_kb":320,"num_write":146,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"265'512","reported_seq":4604,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.085253+0000","last_change":"2024-05-30T17:13:44.085280+0000","last_active":"2024-05-30T17:13:44.085253+0000","last_peered":"2024-05-30T17:13:44.085253+0000","last_clean":"2024-05-30T17:13:44.085253+0000","last_became_active":"2024-05-30T15:54:24.672377+0000","last_became_peered":"2024-05-30T15:54:24.672377+0000","last_unstale":"2024-05-30T17:13:44.085253+0000","last_undegraded":"2024-05-30T17:13:44.085253+0000","last_fullsized":"2024-05-30T17:13:44.085253+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'512","last_scrub_stamp":"2024-05-30T17:13:44.085165+0000","last_deep_scrub":"265'512","last_deep_scrub_stamp":"2024-05-30T17:13:44.085165+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.085165+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":500,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:13:21.462886+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"265'120","reported_seq":645,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.014771+0000","last_change":"2024-05-30T17:13:53.014854+0000","last_active":"2024-05-30T17:13:53.014771+0000","last_peered":"2024-05-30T17:13:53.014771+0000","last_clean":"2024-05-30T17:13:53.014771+0000","last_became_active":"2024-05-30T15:54:26.679866+0000","last_became_peered":"2024-05-30T15:54:26.679866+0000","last_unstale":"2024-05-30T17:13:53.014771+0000","last_undegraded":"2024-05-30T17:13:53.014771+0000","last_fullsized":"2024-05-30T17:13:53.014771+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'120","last_scrub_stamp":"2024-05-30T17:13:53.014561+0000","last_deep_scrub":"265'120","last_deep_scrub_stamp":"2024-05-30T17:13:53.014561+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.014561+0000","objects_scrubbed":3,"log_size":20,"log_dups_size":100,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:47:46.937703+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1256,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":144,"num_read_kb":134,"num_write":293,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"265'392","reported_seq":1808,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:02.030303+0000","last_change":"2024-05-30T17:14:02.030382+0000","last_active":"2024-05-30T17:14:02.030303+0000","last_peered":"2024-05-30T17:14:02.030303+0000","last_clean":"2024-05-30T17:14:02.030303+0000","last_became_active":"2024-05-30T15:54:18.671088+0000","last_became_peered":"2024-05-30T15:54:18.671088+0000","last_unstale":"2024-05-30T17:14:02.030303+0000","last_undegraded":"2024-05-30T17:14:02.030303+0000","last_fullsized":"2024-05-30T17:14:02.030303+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'392","last_scrub_stamp":"2024-05-30T17:14:02.030076+0000","last_deep_scrub":"265'392","last_deep_scrub_stamp":"2024-05-30T17:14:02.030076+0000","last_clean_scrub_stamp":"2024-05-30T17:14:02.030076+0000","objects_scrubbed":3,"log_size":92,"log_dups_size":300,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:37:05.922490+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2029,"num_read_kb":1894,"num_write":481,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"265'180","reported_seq":932,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:18.060340+0000","last_change":"2024-05-30T17:14:18.060426+0000","last_active":"2024-05-30T17:14:18.060340+0000","last_peered":"2024-05-30T17:14:18.060340+0000","last_clean":"2024-05-30T17:14:18.060340+0000","last_became_active":"2024-05-30T15:54:16.671477+0000","last_became_peered":"2024-05-30T15:54:16.671477+0000","last_unstale":"2024-05-30T17:14:18.060340+0000","last_undegraded":"2024-05-30T17:14:18.060340+0000","last_fullsized":"2024-05-30T17:14:18.060340+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'180","last_scrub_stamp":"2024-05-30T17:14:18.060120+0000","last_deep_scrub":"265'180","last_deep_scrub_stamp":"2024-05-30T17:14:18.060120+0000","last_clean_scrub_stamp":"2024-05-30T17:14:18.060120+0000","objects_scrubbed":3,"log_size":80,"log_dups_size":100,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:36:39.055061+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110807,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":679,"num_read_kb":167118,"num_write":1082,"num_write_kb":34693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"265'4","reported_seq":1169,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:16.307268+0000","last_change":"2024-05-30T17:14:16.307355+0000","last_active":"2024-05-30T17:14:16.307268+0000","last_peered":"2024-05-30T17:14:16.307268+0000","last_clean":"2024-05-30T17:14:16.307268+0000","last_became_active":"2024-05-30T15:54:20.662371+0000","last_became_peered":"2024-05-30T15:54:20.662371+0000","last_unstale":"2024-05-30T17:14:16.307268+0000","last_undegraded":"2024-05-30T17:14:16.307268+0000","last_fullsized":"2024-05-30T17:14:16.307268+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'4","last_scrub_stamp":"2024-05-30T17:14:16.307053+0000","last_deep_scrub":"265'4","last_deep_scrub_stamp":"2024-05-30T17:14:16.307053+0000","last_clean_scrub_stamp":"2024-05-30T17:14:16.307053+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:31:23.411741+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":647,"num_read_kb":647,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"265'114","reported_seq":670,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:10.427586+0000","last_change":"2024-05-30T17:14:10.427657+0000","last_active":"2024-05-30T17:14:10.427586+0000","last_peered":"2024-05-30T17:14:10.427586+0000","last_clean":"2024-05-30T17:14:10.427586+0000","last_became_active":"2024-05-30T15:54:22.849105+0000","last_became_peered":"2024-05-30T15:54:22.849105+0000","last_unstale":"2024-05-30T17:14:10.427586+0000","last_undegraded":"2024-05-30T17:14:10.427586+0000","last_fullsized":"2024-05-30T17:14:10.427586+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'114","last_scrub_stamp":"2024-05-30T17:14:10.427409+0000","last_deep_scrub":"265'114","last_deep_scrub_stamp":"2024-05-30T17:14:10.427409+0000","last_clean_scrub_stamp":"2024-05-30T17:14:10.427409+0000","objects_scrubbed":5,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:10:36.524011+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":120,"num_read_kb":115,"num_write":80,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"265'541","reported_seq":2326,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.806417+0000","last_change":"2024-05-30T17:13:54.806572+0000","last_active":"2024-05-30T17:13:54.806417+0000","last_peered":"2024-05-30T17:13:54.806417+0000","last_clean":"2024-05-30T17:13:54.806417+0000","last_became_active":"2024-05-30T15:54:24.677291+0000","last_became_peered":"2024-05-30T15:54:24.677291+0000","last_unstale":"2024-05-30T17:13:54.806417+0000","last_undegraded":"2024-05-30T17:13:54.806417+0000","last_fullsized":"2024-05-30T17:13:54.806417+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'541","last_scrub_stamp":"2024-05-30T17:13:54.806190+0000","last_deep_scrub":"265'541","last_deep_scrub_stamp":"2024-05-30T17:13:54.806190+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.806190+0000","objects_scrubbed":1,"log_size":41,"log_dups_size":500,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T05:09:18.353958+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"265'136","reported_seq":666,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.859668+0000","last_change":"2024-05-30T17:13:49.859798+0000","last_active":"2024-05-30T17:13:49.859668+0000","last_peered":"2024-05-30T17:13:49.859668+0000","last_clean":"2024-05-30T17:13:49.859668+0000","last_became_active":"2024-05-30T15:54:26.674517+0000","last_became_peered":"2024-05-30T15:54:26.674517+0000","last_unstale":"2024-05-30T17:13:49.859668+0000","last_undegraded":"2024-05-30T17:13:49.859668+0000","last_fullsized":"2024-05-30T17:13:49.859668+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'136","last_scrub_stamp":"2024-05-30T17:13:49.859451+0000","last_deep_scrub":"265'136","last_deep_scrub_stamp":"2024-05-30T17:13:49.859451+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.859451+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:25:15.350422+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":141,"num_read_kb":129,"num_write":332,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"265'410","reported_seq":1957,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:05.044019+0000","last_change":"2024-05-30T17:14:05.044110+0000","last_active":"2024-05-30T17:14:05.044019+0000","last_peered":"2024-05-30T17:14:05.044019+0000","last_clean":"2024-05-30T17:14:05.044019+0000","last_became_active":"2024-05-30T15:54:18.657247+0000","last_became_peered":"2024-05-30T15:54:18.657247+0000","last_unstale":"2024-05-30T17:14:05.044019+0000","last_undegraded":"2024-05-30T17:14:05.044019+0000","last_fullsized":"2024-05-30T17:14:05.044019+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'410","last_scrub_stamp":"2024-05-30T17:14:05.043803+0000","last_deep_scrub":"265'410","last_deep_scrub_stamp":"2024-05-30T17:14:05.043803+0000","last_clean_scrub_stamp":"2024-05-30T17:14:05.043803+0000","objects_scrubbed":8,"log_size":110,"log_dups_size":300,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:16:53.264752+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2106,"num_read_kb":2008,"num_write":461,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":502,"num_omap_keys":2,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"265'185","reported_seq":804,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.106642+0000","last_change":"2024-05-30T17:13:45.106682+0000","last_active":"2024-05-30T17:13:45.106642+0000","last_peered":"2024-05-30T17:13:45.106642+0000","last_clean":"2024-05-30T17:13:45.106642+0000","last_became_active":"2024-05-30T15:54:54.426538+0000","last_became_peered":"2024-05-30T15:54:54.426538+0000","last_unstale":"2024-05-30T17:13:45.106642+0000","last_undegraded":"2024-05-30T17:13:45.106642+0000","last_fullsized":"2024-05-30T17:13:45.106642+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'185","last_scrub_stamp":"2024-05-30T17:13:45.106515+0000","last_deep_scrub":"265'185","last_deep_scrub_stamp":"2024-05-30T17:13:45.106515+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.106515+0000","objects_scrubbed":1,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:52:29.468974+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":443,"num_read_kb":549,"num_write":1012,"num_write_kb":15096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,7],"acting":[2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":527,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:13.044644+0000","last_change":"2024-05-30T17:14:13.044703+0000","last_active":"2024-05-30T17:14:13.044644+0000","last_peered":"2024-05-30T17:14:13.044644+0000","last_clean":"2024-05-30T17:14:13.044644+0000","last_became_active":"2024-05-30T15:54:20.669492+0000","last_became_peered":"2024-05-30T15:54:20.669492+0000","last_unstale":"2024-05-30T17:14:13.044644+0000","last_undegraded":"2024-05-30T17:14:13.044644+0000","last_fullsized":"2024-05-30T17:14:13.044644+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:13.044456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:13.044456+0000","last_clean_scrub_stamp":"2024-05-30T17:14:13.044456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:52:25.523369+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"265'243","reported_seq":890,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:01.276657+0000","last_change":"2024-05-30T17:14:01.276754+0000","last_active":"2024-05-30T17:14:01.276657+0000","last_peered":"2024-05-30T17:14:01.276657+0000","last_clean":"2024-05-30T17:14:01.276657+0000","last_became_active":"2024-05-30T15:54:22.671943+0000","last_became_peered":"2024-05-30T15:54:22.671943+0000","last_unstale":"2024-05-30T17:14:01.276657+0000","last_undegraded":"2024-05-30T17:14:01.276657+0000","last_fullsized":"2024-05-30T17:14:01.276657+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'243","last_scrub_stamp":"2024-05-30T17:14:01.276461+0000","last_deep_scrub":"265'243","last_deep_scrub_stamp":"2024-05-30T17:14:01.276461+0000","last_clean_scrub_stamp":"2024-05-30T17:14:01.276461+0000","objects_scrubbed":8,"log_size":43,"log_dups_size":200,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:31:23.133344+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":313,"num_read_kb":312,"num_write":178,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":509,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:00.785558+0000","last_change":"2024-05-30T17:14:00.785659+0000","last_active":"2024-05-30T17:14:00.785558+0000","last_peered":"2024-05-30T17:14:00.785558+0000","last_clean":"2024-05-30T17:14:00.785558+0000","last_became_active":"2024-05-30T15:54:24.681595+0000","last_became_peered":"2024-05-30T15:54:24.681595+0000","last_unstale":"2024-05-30T17:14:00.785558+0000","last_undegraded":"2024-05-30T17:14:00.785558+0000","last_fullsized":"2024-05-30T17:14:00.785558+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:00.785264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:00.785264+0000","last_clean_scrub_stamp":"2024-05-30T17:14:00.785264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:38:41.393882+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"265'119","reported_seq":635,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.989353+0000","last_change":"2024-05-30T17:13:51.989431+0000","last_active":"2024-05-30T17:13:51.989353+0000","last_peered":"2024-05-30T17:13:51.989353+0000","last_clean":"2024-05-30T17:13:51.989353+0000","last_became_active":"2024-05-30T15:54:26.688518+0000","last_became_peered":"2024-05-30T15:54:26.688518+0000","last_unstale":"2024-05-30T17:13:51.989353+0000","last_undegraded":"2024-05-30T17:13:51.989353+0000","last_fullsized":"2024-05-30T17:13:51.989353+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'119","last_scrub_stamp":"2024-05-30T17:13:51.989161+0000","last_deep_scrub":"265'119","last_deep_scrub_stamp":"2024-05-30T17:13:51.989161+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.989161+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":100,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:46:17.674786+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":619,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":113,"num_read_kb":112,"num_write":261,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"265'758","reported_seq":2277,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:57.780923+0000","last_change":"2024-05-30T17:13:57.781083+0000","last_active":"2024-05-30T17:13:57.780923+0000","last_peered":"2024-05-30T17:13:57.780923+0000","last_clean":"2024-05-30T17:13:57.780923+0000","last_became_active":"2024-05-30T15:54:18.677957+0000","last_became_peered":"2024-05-30T15:54:18.677957+0000","last_unstale":"2024-05-30T17:13:57.780923+0000","last_undegraded":"2024-05-30T17:13:57.780923+0000","last_fullsized":"2024-05-30T17:13:57.780923+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'758","last_scrub_stamp":"2024-05-30T17:13:57.780707+0000","last_deep_scrub":"265'758","last_deep_scrub_stamp":"2024-05-30T17:13:57.780707+0000","last_clean_scrub_stamp":"2024-05-30T17:13:57.780707+0000","objects_scrubbed":2,"log_size":58,"log_dups_size":700,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:16:34.692139+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3440,"num_read_kb":3078,"num_write":1010,"num_write_kb":372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":628,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:26.585314+0000","last_change":"2024-05-30T15:54:03.031292+0000","last_active":"2024-05-30T17:12:26.585314+0000","last_peered":"2024-05-30T17:12:26.585314+0000","last_clean":"2024-05-30T17:12:26.585314+0000","last_became_active":"2024-05-30T15:54:03.029214+0000","last_became_peered":"2024-05-30T15:54:03.029214+0000","last_unstale":"2024-05-30T17:12:26.585314+0000","last_undegraded":"2024-05-30T17:12:26.585314+0000","last_fullsized":"2024-05-30T17:12:26.585314+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:01.512251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:01.512251+0000","last_clean_scrub_stamp":"2024-05-30T15:54:01.512251+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":539,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:19.247044+0000","last_change":"2024-05-30T17:14:19.247196+0000","last_active":"2024-05-30T17:14:19.247044+0000","last_peered":"2024-05-30T17:14:19.247044+0000","last_clean":"2024-05-30T17:14:19.247044+0000","last_became_active":"2024-05-30T15:54:08.577556+0000","last_became_peered":"2024-05-30T15:54:08.577556+0000","last_unstale":"2024-05-30T17:14:19.247044+0000","last_undegraded":"2024-05-30T17:14:19.247044+0000","last_fullsized":"2024-05-30T17:14:19.247044+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:19.246805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:19.246805+0000","last_clean_scrub_stamp":"2024-05-30T17:14:19.246805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:51:24.222735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"265'189","reported_seq":845,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.415326+0000","last_change":"2024-05-30T15:54:16.674631+0000","last_active":"2024-05-30T17:13:38.415326+0000","last_peered":"2024-05-30T17:13:38.415326+0000","last_clean":"2024-05-30T17:13:38.415326+0000","last_became_active":"2024-05-30T15:54:16.674229+0000","last_became_peered":"2024-05-30T15:54:16.674229+0000","last_unstale":"2024-05-30T17:13:38.415326+0000","last_undegraded":"2024-05-30T17:13:38.415326+0000","last_fullsized":"2024-05-30T17:13:38.415326+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5525719,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":447,"num_read_kb":11470,"num_write":1074,"num_write_kb":25445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":521,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:16.814789+0000","last_change":"2024-05-30T17:14:16.814843+0000","last_active":"2024-05-30T17:14:16.814789+0000","last_peered":"2024-05-30T17:14:16.814789+0000","last_clean":"2024-05-30T17:14:16.814789+0000","last_became_active":"2024-05-30T15:54:20.664317+0000","last_became_peered":"2024-05-30T15:54:20.664317+0000","last_unstale":"2024-05-30T17:14:16.814789+0000","last_undegraded":"2024-05-30T17:14:16.814789+0000","last_fullsized":"2024-05-30T17:14:16.814789+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:16.814584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:16.814584+0000","last_clean_scrub_stamp":"2024-05-30T17:14:16.814584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:59:43.265365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"265'334","reported_seq":1056,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.210051+0000","last_change":"2024-05-30T15:54:22.670731+0000","last_active":"2024-05-30T17:13:38.210051+0000","last_peered":"2024-05-30T17:13:38.210051+0000","last_clean":"2024-05-30T17:13:38.210051+0000","last_became_active":"2024-05-30T15:54:22.669871+0000","last_became_peered":"2024-05-30T15:54:22.669871+0000","last_unstale":"2024-05-30T17:13:38.210051+0000","last_undegraded":"2024-05-30T17:13:38.210051+0000","last_fullsized":"2024-05-30T17:13:38.210051+0000","mapping_epoch":28,"log_start":"265'300","ondisk_log_start":"265'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49272,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":648,"num_read_kb":960,"num_write":598,"num_write_kb":460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":516,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.868767+0000","last_change":"2024-05-30T17:13:58.868833+0000","last_active":"2024-05-30T17:13:58.868767+0000","last_peered":"2024-05-30T17:13:58.868767+0000","last_clean":"2024-05-30T17:13:58.868767+0000","last_became_active":"2024-05-30T15:54:24.684709+0000","last_became_peered":"2024-05-30T15:54:24.684709+0000","last_unstale":"2024-05-30T17:13:58.868767+0000","last_undegraded":"2024-05-30T17:13:58.868767+0000","last_fullsized":"2024-05-30T17:13:58.868767+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:58.868550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:58.868550+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.868550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:45:56.978968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"265'178","reported_seq":891,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:02.796474+0000","last_change":"2024-05-30T17:14:02.796551+0000","last_active":"2024-05-30T17:14:02.796474+0000","last_peered":"2024-05-30T17:14:02.796474+0000","last_clean":"2024-05-30T17:14:02.796474+0000","last_became_active":"2024-05-30T15:54:26.686647+0000","last_became_peered":"2024-05-30T15:54:26.686647+0000","last_unstale":"2024-05-30T17:14:02.796474+0000","last_undegraded":"2024-05-30T17:14:02.796474+0000","last_fullsized":"2024-05-30T17:14:02.796474+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'178","last_scrub_stamp":"2024-05-30T17:14:02.796290+0000","last_deep_scrub":"265'178","last_deep_scrub_stamp":"2024-05-30T17:14:02.796290+0000","last_clean_scrub_stamp":"2024-05-30T17:14:02.796290+0000","objects_scrubbed":4,"log_size":78,"log_dups_size":100,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:33:47.206690+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1020,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":416,"num_read_kb":404,"num_write":359,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":66,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"265'389","reported_seq":1872,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:14.815422+0000","last_change":"2024-05-30T17:14:14.815492+0000","last_active":"2024-05-30T17:14:14.815422+0000","last_peered":"2024-05-30T17:14:14.815422+0000","last_clean":"2024-05-30T17:14:14.815422+0000","last_became_active":"2024-05-30T15:54:18.648529+0000","last_became_peered":"2024-05-30T15:54:18.648529+0000","last_unstale":"2024-05-30T17:14:14.815422+0000","last_undegraded":"2024-05-30T17:14:14.815422+0000","last_fullsized":"2024-05-30T17:14:14.815422+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'389","last_scrub_stamp":"2024-05-30T17:14:14.815228+0000","last_deep_scrub":"265'389","last_deep_scrub_stamp":"2024-05-30T17:14:14.815228+0000","last_clean_scrub_stamp":"2024-05-30T17:14:14.815228+0000","objects_scrubbed":1,"log_size":89,"log_dups_size":300,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:17:35.526601+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2082,"num_read_kb":1967,"num_write":465,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":532,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:31.586728+0000","last_change":"2024-05-30T15:54:08.575348+0000","last_active":"2024-05-30T17:12:31.586728+0000","last_peered":"2024-05-30T17:12:31.586728+0000","last_clean":"2024-05-30T17:12:31.586728+0000","last_became_active":"2024-05-30T15:54:08.574645+0000","last_became_peered":"2024-05-30T15:54:08.574645+0000","last_unstale":"2024-05-30T17:12:31.586728+0000","last_undegraded":"2024-05-30T17:12:31.586728+0000","last_fullsized":"2024-05-30T17:12:31.586728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"265'174","reported_seq":810,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.439442+0000","last_change":"2024-05-30T15:54:16.678919+0000","last_active":"2024-05-30T17:13:38.439442+0000","last_peered":"2024-05-30T17:13:38.439442+0000","last_clean":"2024-05-30T17:13:38.439442+0000","last_became_active":"2024-05-30T15:54:16.678386+0000","last_became_peered":"2024-05-30T15:54:16.678386+0000","last_unstale":"2024-05-30T17:13:38.439442+0000","last_undegraded":"2024-05-30T17:13:38.439442+0000","last_fullsized":"2024-05-30T17:13:38.439442+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079916,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":364,"num_read_kb":7665,"num_write":1082,"num_write_kb":33234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":519,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:14.305572+0000","last_change":"2024-05-30T17:14:14.305625+0000","last_active":"2024-05-30T17:14:14.305572+0000","last_peered":"2024-05-30T17:14:14.305572+0000","last_clean":"2024-05-30T17:14:14.305572+0000","last_became_active":"2024-05-30T15:54:20.661502+0000","last_became_peered":"2024-05-30T15:54:20.661502+0000","last_unstale":"2024-05-30T17:14:14.305572+0000","last_undegraded":"2024-05-30T17:14:14.305572+0000","last_fullsized":"2024-05-30T17:14:14.305572+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:14.305372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:14.305372+0000","last_clean_scrub_stamp":"2024-05-30T17:14:14.305372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T05:04:18.082051+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"265'730","reported_seq":2569,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:04.002916+0000","last_change":"2024-05-30T17:14:04.003007+0000","last_active":"2024-05-30T17:14:04.002916+0000","last_peered":"2024-05-30T17:14:04.002916+0000","last_clean":"2024-05-30T17:14:04.002916+0000","last_became_active":"2024-05-30T15:54:22.671588+0000","last_became_peered":"2024-05-30T15:54:22.671588+0000","last_unstale":"2024-05-30T17:14:04.002916+0000","last_undegraded":"2024-05-30T17:14:04.002916+0000","last_fullsized":"2024-05-30T17:14:04.002916+0000","mapping_epoch":28,"log_start":"265'700","ondisk_log_start":"265'700","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'730","last_scrub_stamp":"2024-05-30T17:14:04.002697+0000","last_deep_scrub":"265'730","last_deep_scrub_stamp":"2024-05-30T17:14:04.002697+0000","last_clean_scrub_stamp":"2024-05-30T17:14:04.002697+0000","objects_scrubbed":12,"log_size":30,"log_dups_size":700,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:14:57.807605+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19819,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":1008,"num_read_kb":1208,"num_write":324,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6022,"num_omap_keys":18,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":513,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:02.668077+0000","last_change":"2024-05-30T17:14:02.668129+0000","last_active":"2024-05-30T17:14:02.668077+0000","last_peered":"2024-05-30T17:14:02.668077+0000","last_clean":"2024-05-30T17:14:02.668077+0000","last_became_active":"2024-05-30T15:54:24.686641+0000","last_became_peered":"2024-05-30T15:54:24.686641+0000","last_unstale":"2024-05-30T17:14:02.668077+0000","last_undegraded":"2024-05-30T17:14:02.668077+0000","last_fullsized":"2024-05-30T17:14:02.668077+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:02.667880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:02.667880+0000","last_clean_scrub_stamp":"2024-05-30T17:14:02.667880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:55:41.680344+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"265'125","reported_seq":645,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.985096+0000","last_change":"2024-05-30T17:13:54.985175+0000","last_active":"2024-05-30T17:13:54.985096+0000","last_peered":"2024-05-30T17:13:54.985096+0000","last_clean":"2024-05-30T17:13:54.985096+0000","last_became_active":"2024-05-30T15:54:26.667751+0000","last_became_peered":"2024-05-30T15:54:26.667751+0000","last_unstale":"2024-05-30T17:13:54.985096+0000","last_undegraded":"2024-05-30T17:13:54.985096+0000","last_fullsized":"2024-05-30T17:13:54.985096+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'125","last_scrub_stamp":"2024-05-30T17:13:54.984877+0000","last_deep_scrub":"265'125","last_deep_scrub_stamp":"2024-05-30T17:13:54.984877+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.984877+0000","objects_scrubbed":2,"log_size":25,"log_dups_size":100,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:40:14.213968+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":438,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":115,"num_write":291,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"265'328","reported_seq":1657,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:07.012695+0000","last_change":"2024-05-30T17:14:07.012769+0000","last_active":"2024-05-30T17:14:07.012695+0000","last_peered":"2024-05-30T17:14:07.012695+0000","last_clean":"2024-05-30T17:14:07.012695+0000","last_became_active":"2024-05-30T15:54:18.651853+0000","last_became_peered":"2024-05-30T15:54:18.651853+0000","last_unstale":"2024-05-30T17:14:07.012695+0000","last_undegraded":"2024-05-30T17:14:07.012695+0000","last_fullsized":"2024-05-30T17:14:07.012695+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'328","last_scrub_stamp":"2024-05-30T17:14:07.012480+0000","last_deep_scrub":"265'328","last_deep_scrub_stamp":"2024-05-30T17:14:07.012480+0000","last_clean_scrub_stamp":"2024-05-30T17:14:07.012480+0000","objects_scrubbed":2,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:54:27.007726+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1727,"num_read_kb":1636,"num_write":373,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":542,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.261872+0000","last_change":"2024-05-30T17:13:59.261974+0000","last_active":"2024-05-30T17:13:59.261872+0000","last_peered":"2024-05-30T17:13:59.261872+0000","last_clean":"2024-05-30T17:13:59.261872+0000","last_became_active":"2024-05-30T15:54:08.744898+0000","last_became_peered":"2024-05-30T15:54:08.744898+0000","last_unstale":"2024-05-30T17:13:59.261872+0000","last_undegraded":"2024-05-30T17:13:59.261872+0000","last_fullsized":"2024-05-30T17:13:59.261872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:59.261650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:59.261650+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.261650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:03:48.512476+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"265'185","reported_seq":822,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:13.996981+0000","last_change":"2024-05-30T17:14:13.997191+0000","last_active":"2024-05-30T17:14:13.996981+0000","last_peered":"2024-05-30T17:14:13.996981+0000","last_clean":"2024-05-30T17:14:13.996981+0000","last_became_active":"2024-05-30T15:54:16.650214+0000","last_became_peered":"2024-05-30T15:54:16.650214+0000","last_unstale":"2024-05-30T17:14:13.996981+0000","last_undegraded":"2024-05-30T17:14:13.996981+0000","last_fullsized":"2024-05-30T17:14:13.996981+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'185","last_scrub_stamp":"2024-05-30T17:14:13.996657+0000","last_deep_scrub":"265'185","last_deep_scrub_stamp":"2024-05-30T17:14:13.996657+0000","last_clean_scrub_stamp":"2024-05-30T17:14:13.996657+0000","objects_scrubbed":9,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:48:29.511347+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1911070,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":412,"num_read_kb":8332,"num_write":1154,"num_write_kb":25808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"265'2","reported_seq":524,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.146546+0000","last_change":"2024-05-30T17:13:50.146675+0000","last_active":"2024-05-30T17:13:50.146546+0000","last_peered":"2024-05-30T17:13:50.146546+0000","last_clean":"2024-05-30T17:13:50.146546+0000","last_became_active":"2024-05-30T15:54:20.666715+0000","last_became_peered":"2024-05-30T15:54:20.666715+0000","last_unstale":"2024-05-30T17:13:50.146546+0000","last_undegraded":"2024-05-30T17:13:50.146546+0000","last_fullsized":"2024-05-30T17:13:50.146546+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:13:50.146258+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:13:50.146258+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.146258+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:41:50.441025+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"265'268","reported_seq":1790,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:07.764156+0000","last_change":"2024-05-30T17:14:07.764244+0000","last_active":"2024-05-30T17:14:07.764156+0000","last_peered":"2024-05-30T17:14:07.764156+0000","last_clean":"2024-05-30T17:14:07.764156+0000","last_became_active":"2024-05-30T15:54:22.852604+0000","last_became_peered":"2024-05-30T15:54:22.852604+0000","last_unstale":"2024-05-30T17:14:07.764156+0000","last_undegraded":"2024-05-30T17:14:07.764156+0000","last_fullsized":"2024-05-30T17:14:07.764156+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'268","last_scrub_stamp":"2024-05-30T17:14:07.763964+0000","last_deep_scrub":"265'268","last_deep_scrub_stamp":"2024-05-30T17:14:07.763964+0000","last_clean_scrub_stamp":"2024-05-30T17:14:07.763964+0000","objects_scrubbed":9,"log_size":68,"log_dups_size":200,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:30:31.072037+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":232,"num_read_kb":224,"num_write":183,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5824,"num_omap_keys":14,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":518,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:00.884506+0000","last_change":"2024-05-30T17:14:00.884643+0000","last_active":"2024-05-30T17:14:00.884506+0000","last_peered":"2024-05-30T17:14:00.884506+0000","last_clean":"2024-05-30T17:14:00.884506+0000","last_became_active":"2024-05-30T15:54:24.679830+0000","last_became_peered":"2024-05-30T15:54:24.679830+0000","last_unstale":"2024-05-30T17:14:00.884506+0000","last_undegraded":"2024-05-30T17:14:00.884506+0000","last_fullsized":"2024-05-30T17:14:00.884506+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:00.884270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:00.884270+0000","last_clean_scrub_stamp":"2024-05-30T17:14:00.884270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:18:06.001324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"265'253","reported_seq":871,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.356811+0000","last_change":"2024-05-30T17:13:56.356879+0000","last_active":"2024-05-30T17:13:56.356811+0000","last_peered":"2024-05-30T17:13:56.356811+0000","last_clean":"2024-05-30T17:13:56.356811+0000","last_became_active":"2024-05-30T15:54:26.696477+0000","last_became_peered":"2024-05-30T15:54:26.696477+0000","last_unstale":"2024-05-30T17:13:56.356811+0000","last_undegraded":"2024-05-30T17:13:56.356811+0000","last_fullsized":"2024-05-30T17:13:56.356811+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'253","last_scrub_stamp":"2024-05-30T17:13:56.356623+0000","last_deep_scrub":"265'253","last_deep_scrub_stamp":"2024-05-30T17:13:56.356623+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.356623+0000","objects_scrubbed":2,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:15:54.649548+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":476,"num_read_kb":500,"num_write":542,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1193,"num_omap_keys":7,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"265'766","reported_seq":2335,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:11.094921+0000","last_change":"2024-05-30T17:14:11.095077+0000","last_active":"2024-05-30T17:14:11.094921+0000","last_peered":"2024-05-30T17:14:11.094921+0000","last_clean":"2024-05-30T17:14:11.094921+0000","last_became_active":"2024-05-30T15:54:18.639830+0000","last_became_peered":"2024-05-30T15:54:18.639830+0000","last_unstale":"2024-05-30T17:14:11.094921+0000","last_undegraded":"2024-05-30T17:14:11.094921+0000","last_fullsized":"2024-05-30T17:14:11.094921+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'766","last_scrub_stamp":"2024-05-30T17:14:11.094660+0000","last_deep_scrub":"265'766","last_deep_scrub_stamp":"2024-05-30T17:14:11.094660+0000","last_clean_scrub_stamp":"2024-05-30T17:14:11.094660+0000","objects_scrubbed":5,"log_size":66,"log_dups_size":700,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:57:27.825458+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3481,"num_read_kb":3132,"num_write":1014,"num_write_kb":367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":539,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:18.271712+0000","last_change":"2024-05-30T17:14:18.271770+0000","last_active":"2024-05-30T17:14:18.271712+0000","last_peered":"2024-05-30T17:14:18.271712+0000","last_clean":"2024-05-30T17:14:18.271712+0000","last_became_active":"2024-05-30T15:54:08.574817+0000","last_became_peered":"2024-05-30T15:54:08.574817+0000","last_unstale":"2024-05-30T17:14:18.271712+0000","last_undegraded":"2024-05-30T17:14:18.271712+0000","last_fullsized":"2024-05-30T17:14:18.271712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:18.271517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:18.271517+0000","last_clean_scrub_stamp":"2024-05-30T17:14:18.271517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:45:29.263529+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"265'199","reported_seq":923,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:17.027327+0000","last_change":"2024-05-30T17:14:17.027406+0000","last_active":"2024-05-30T17:14:17.027327+0000","last_peered":"2024-05-30T17:14:17.027327+0000","last_clean":"2024-05-30T17:14:17.027327+0000","last_became_active":"2024-05-30T15:54:16.655347+0000","last_became_peered":"2024-05-30T15:54:16.655347+0000","last_unstale":"2024-05-30T17:14:17.027327+0000","last_undegraded":"2024-05-30T17:14:17.027327+0000","last_fullsized":"2024-05-30T17:14:17.027327+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'199","last_scrub_stamp":"2024-05-30T17:14:17.027124+0000","last_deep_scrub":"265'199","last_deep_scrub_stamp":"2024-05-30T17:14:17.027124+0000","last_clean_scrub_stamp":"2024-05-30T17:14:17.027124+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:00:55.705228+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":572,"num_read_kb":42208,"num_write":1148,"num_write_kb":36370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":526,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:13.829376+0000","last_change":"2024-05-30T17:14:13.829430+0000","last_active":"2024-05-30T17:14:13.829376+0000","last_peered":"2024-05-30T17:14:13.829376+0000","last_clean":"2024-05-30T17:14:13.829376+0000","last_became_active":"2024-05-30T15:54:20.651449+0000","last_became_peered":"2024-05-30T15:54:20.651449+0000","last_unstale":"2024-05-30T17:14:13.829376+0000","last_undegraded":"2024-05-30T17:14:13.829376+0000","last_fullsized":"2024-05-30T17:14:13.829376+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:13.829176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:13.829176+0000","last_clean_scrub_stamp":"2024-05-30T17:14:13.829176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:22:24.775288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"265'167","reported_seq":748,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:12.372999+0000","last_change":"2024-05-30T17:14:12.373071+0000","last_active":"2024-05-30T17:14:12.372999+0000","last_peered":"2024-05-30T17:14:12.372999+0000","last_clean":"2024-05-30T17:14:12.372999+0000","last_became_active":"2024-05-30T15:54:22.673881+0000","last_became_peered":"2024-05-30T15:54:22.673881+0000","last_unstale":"2024-05-30T17:14:12.372999+0000","last_undegraded":"2024-05-30T17:14:12.372999+0000","last_fullsized":"2024-05-30T17:14:12.372999+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'167","last_scrub_stamp":"2024-05-30T17:14:12.372906+0000","last_deep_scrub":"265'167","last_deep_scrub_stamp":"2024-05-30T17:14:12.372906+0000","last_clean_scrub_stamp":"2024-05-30T17:14:12.372906+0000","objects_scrubbed":7,"log_size":67,"log_dups_size":100,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:29:18.523344+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":200,"num_read_kb":192,"num_write":144,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1630,"num_omap_keys":4,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":510,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.801167+0000","last_change":"2024-05-30T17:13:56.801223+0000","last_active":"2024-05-30T17:13:56.801167+0000","last_peered":"2024-05-30T17:13:56.801167+0000","last_clean":"2024-05-30T17:13:56.801167+0000","last_became_active":"2024-05-30T15:54:24.675916+0000","last_became_peered":"2024-05-30T15:54:24.675916+0000","last_unstale":"2024-05-30T17:13:56.801167+0000","last_undegraded":"2024-05-30T17:13:56.801167+0000","last_fullsized":"2024-05-30T17:13:56.801167+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:56.800964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:56.800964+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.800964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:50:34.435231+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"265'139","reported_seq":666,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.262374+0000","last_change":"2024-05-30T17:13:49.262568+0000","last_active":"2024-05-30T17:13:49.262374+0000","last_peered":"2024-05-30T17:13:49.262374+0000","last_clean":"2024-05-30T17:13:49.262374+0000","last_became_active":"2024-05-30T15:54:26.704217+0000","last_became_peered":"2024-05-30T15:54:26.704217+0000","last_unstale":"2024-05-30T17:13:49.262374+0000","last_undegraded":"2024-05-30T17:13:49.262374+0000","last_fullsized":"2024-05-30T17:13:49.262374+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'139","last_scrub_stamp":"2024-05-30T17:13:49.262162+0000","last_deep_scrub":"265'139","last_deep_scrub_stamp":"2024-05-30T17:13:49.262162+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.262162+0000","objects_scrubbed":2,"log_size":39,"log_dups_size":100,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:30:17.555273+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":834,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":138,"num_read_kb":136,"num_write":341,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"265'398","reported_seq":1838,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:03.013211+0000","last_change":"2024-05-30T17:14:03.013307+0000","last_active":"2024-05-30T17:14:03.013211+0000","last_peered":"2024-05-30T17:14:03.013211+0000","last_clean":"2024-05-30T17:14:03.013211+0000","last_became_active":"2024-05-30T15:54:18.665034+0000","last_became_peered":"2024-05-30T15:54:18.665034+0000","last_unstale":"2024-05-30T17:14:03.013211+0000","last_undegraded":"2024-05-30T17:14:03.013211+0000","last_fullsized":"2024-05-30T17:14:03.013211+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'398","last_scrub_stamp":"2024-05-30T17:14:03.013020+0000","last_deep_scrub":"265'398","last_deep_scrub_stamp":"2024-05-30T17:14:03.013020+0000","last_clean_scrub_stamp":"2024-05-30T17:14:03.013020+0000","objects_scrubbed":2,"log_size":98,"log_dups_size":300,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:58:48.490896+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2053,"num_read_kb":1920,"num_write":476,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"265'217","reported_seq":867,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:21.298652+0000","last_change":"2024-05-30T17:14:21.298756+0000","last_active":"2024-05-30T17:14:21.298652+0000","last_peered":"2024-05-30T17:14:21.298652+0000","last_clean":"2024-05-30T17:14:21.298652+0000","last_became_active":"2024-05-30T15:54:16.650735+0000","last_became_peered":"2024-05-30T15:54:16.650735+0000","last_unstale":"2024-05-30T17:14:21.298652+0000","last_undegraded":"2024-05-30T17:14:21.298652+0000","last_fullsized":"2024-05-30T17:14:21.298652+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'217","last_scrub_stamp":"2024-05-30T17:14:21.298453+0000","last_deep_scrub":"265'217","last_deep_scrub_stamp":"2024-05-30T17:14:21.298453+0000","last_clean_scrub_stamp":"2024-05-30T17:14:21.298453+0000","objects_scrubbed":6,"log_size":17,"log_dups_size":200,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:06:06.658095+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079967,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":420,"num_read_kb":5617,"num_write":983,"num_write_kb":15732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":540,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:12.043129+0000","last_change":"2024-05-30T17:14:12.043183+0000","last_active":"2024-05-30T17:14:12.043129+0000","last_peered":"2024-05-30T17:14:12.043129+0000","last_clean":"2024-05-30T17:14:12.043129+0000","last_became_active":"2024-05-30T15:54:08.567900+0000","last_became_peered":"2024-05-30T15:54:08.567900+0000","last_unstale":"2024-05-30T17:14:12.043129+0000","last_undegraded":"2024-05-30T17:14:12.043129+0000","last_fullsized":"2024-05-30T17:14:12.043129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:12.042933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:12.042933+0000","last_clean_scrub_stamp":"2024-05-30T17:14:12.042933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:36:17.591809+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":526,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:15.771566+0000","last_change":"2024-05-30T17:14:15.771620+0000","last_active":"2024-05-30T17:14:15.771566+0000","last_peered":"2024-05-30T17:14:15.771566+0000","last_clean":"2024-05-30T17:14:15.771566+0000","last_became_active":"2024-05-30T15:54:20.653790+0000","last_became_peered":"2024-05-30T15:54:20.653790+0000","last_unstale":"2024-05-30T17:14:15.771566+0000","last_undegraded":"2024-05-30T17:14:15.771566+0000","last_fullsized":"2024-05-30T17:14:15.771566+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:15.771368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:15.771368+0000","last_clean_scrub_stamp":"2024-05-30T17:14:15.771368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:07:25.961584+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"265'371","reported_seq":1080,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:08.931637+0000","last_change":"2024-05-30T17:14:08.931785+0000","last_active":"2024-05-30T17:14:08.931637+0000","last_peered":"2024-05-30T17:14:08.931637+0000","last_clean":"2024-05-30T17:14:08.931637+0000","last_became_active":"2024-05-30T15:54:22.673903+0000","last_became_peered":"2024-05-30T15:54:22.673903+0000","last_unstale":"2024-05-30T17:14:08.931637+0000","last_undegraded":"2024-05-30T17:14:08.931637+0000","last_fullsized":"2024-05-30T17:14:08.931637+0000","mapping_epoch":28,"log_start":"265'300","ondisk_log_start":"265'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'371","last_scrub_stamp":"2024-05-30T17:14:08.931442+0000","last_deep_scrub":"265'371","last_deep_scrub_stamp":"2024-05-30T17:14:08.931442+0000","last_clean_scrub_stamp":"2024-05-30T17:14:08.931442+0000","objects_scrubbed":10,"log_size":71,"log_dups_size":300,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:46:35.284674+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11442,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1140,"num_read_kb":1226,"num_write":1674,"num_write_kb":1475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":496492,"num_omap_keys":1134,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":522,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:55.903746+0000","last_change":"2024-05-30T17:13:55.903798+0000","last_active":"2024-05-30T17:13:55.903746+0000","last_peered":"2024-05-30T17:13:55.903746+0000","last_clean":"2024-05-30T17:13:55.903746+0000","last_became_active":"2024-05-30T15:54:24.677198+0000","last_became_peered":"2024-05-30T15:54:24.677198+0000","last_unstale":"2024-05-30T17:13:55.903746+0000","last_undegraded":"2024-05-30T17:13:55.903746+0000","last_fullsized":"2024-05-30T17:13:55.903746+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:55.903559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:55.903559+0000","last_clean_scrub_stamp":"2024-05-30T17:13:55.903559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:20:32.475018+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"265'122","reported_seq":649,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:57.847760+0000","last_change":"2024-05-30T17:13:57.847843+0000","last_active":"2024-05-30T17:13:57.847760+0000","last_peered":"2024-05-30T17:13:57.847760+0000","last_clean":"2024-05-30T17:13:57.847760+0000","last_became_active":"2024-05-30T15:54:26.676078+0000","last_became_peered":"2024-05-30T15:54:26.676078+0000","last_unstale":"2024-05-30T17:13:57.847760+0000","last_undegraded":"2024-05-30T17:13:57.847760+0000","last_fullsized":"2024-05-30T17:13:57.847760+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'122","last_scrub_stamp":"2024-05-30T17:13:57.847569+0000","last_deep_scrub":"265'122","last_deep_scrub_stamp":"2024-05-30T17:13:57.847569+0000","last_clean_scrub_stamp":"2024-05-30T17:13:57.847569+0000","objects_scrubbed":5,"log_size":22,"log_dups_size":100,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:27:31.982550+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":727,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":114,"num_read_kb":108,"num_write":274,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":73,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"265'797","reported_seq":2438,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:20.820346+0000","last_change":"2024-05-30T17:14:20.820437+0000","last_active":"2024-05-30T17:14:20.820346+0000","last_peered":"2024-05-30T17:14:20.820346+0000","last_clean":"2024-05-30T17:14:20.820346+0000","last_became_active":"2024-05-30T15:54:18.642683+0000","last_became_peered":"2024-05-30T15:54:18.642683+0000","last_unstale":"2024-05-30T17:14:20.820346+0000","last_undegraded":"2024-05-30T17:14:20.820346+0000","last_fullsized":"2024-05-30T17:14:20.820346+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'797","last_scrub_stamp":"2024-05-30T17:14:20.820153+0000","last_deep_scrub":"265'797","last_deep_scrub_stamp":"2024-05-30T17:14:20.820153+0000","last_clean_scrub_stamp":"2024-05-30T17:14:20.820153+0000","objects_scrubbed":6,"log_size":97,"log_dups_size":700,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:01:34.756808+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3596,"num_read_kb":3243,"num_write":1037,"num_write_kb":360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1011,"num_omap_keys":4,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"265'184","reported_seq":836,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:16.998554+0000","last_change":"2024-05-30T17:14:16.998697+0000","last_active":"2024-05-30T17:14:16.998554+0000","last_peered":"2024-05-30T17:14:16.998554+0000","last_clean":"2024-05-30T17:14:16.998554+0000","last_became_active":"2024-05-30T15:54:16.663360+0000","last_became_peered":"2024-05-30T15:54:16.663360+0000","last_unstale":"2024-05-30T17:14:16.998554+0000","last_undegraded":"2024-05-30T17:14:16.998554+0000","last_fullsized":"2024-05-30T17:14:16.998554+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'184","last_scrub_stamp":"2024-05-30T17:14:16.998338+0000","last_deep_scrub":"265'184","last_deep_scrub_stamp":"2024-05-30T17:14:16.998338+0000","last_clean_scrub_stamp":"2024-05-30T17:14:16.998338+0000","objects_scrubbed":6,"log_size":84,"log_dups_size":100,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:35:16.806460+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":661325,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":407,"num_read_kb":30298,"num_write":1063,"num_write_kb":42840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":531,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.604977+0000","last_change":"2024-05-30T15:54:08.577379+0000","last_active":"2024-05-30T17:13:41.604977+0000","last_peered":"2024-05-30T17:13:41.604977+0000","last_clean":"2024-05-30T17:13:41.604977+0000","last_became_active":"2024-05-30T15:54:08.576779+0000","last_became_peered":"2024-05-30T15:54:08.576779+0000","last_unstale":"2024-05-30T17:13:41.604977+0000","last_undegraded":"2024-05-30T17:13:41.604977+0000","last_fullsized":"2024-05-30T17:13:41.604977+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:07.550093+0000","last_clean_scrub_stamp":"2024-05-30T15:54:07.550093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":520,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:55.817272+0000","last_change":"2024-05-30T17:13:55.817326+0000","last_active":"2024-05-30T17:13:55.817272+0000","last_peered":"2024-05-30T17:13:55.817272+0000","last_clean":"2024-05-30T17:13:55.817272+0000","last_became_active":"2024-05-30T15:54:20.655220+0000","last_became_peered":"2024-05-30T15:54:20.655220+0000","last_unstale":"2024-05-30T17:13:55.817272+0000","last_undegraded":"2024-05-30T17:13:55.817272+0000","last_fullsized":"2024-05-30T17:13:55.817272+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:55.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:55.817064+0000","last_clean_scrub_stamp":"2024-05-30T17:13:55.817064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:12:59.140457+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"265'204","reported_seq":810,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:06.066922+0000","last_change":"2024-05-30T17:14:06.067009+0000","last_active":"2024-05-30T17:14:06.066922+0000","last_peered":"2024-05-30T17:14:06.066922+0000","last_clean":"2024-05-30T17:14:06.066922+0000","last_became_active":"2024-05-30T15:54:22.851338+0000","last_became_peered":"2024-05-30T15:54:22.851338+0000","last_unstale":"2024-05-30T17:14:06.066922+0000","last_undegraded":"2024-05-30T17:14:06.066922+0000","last_fullsized":"2024-05-30T17:14:06.066922+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'204","last_scrub_stamp":"2024-05-30T17:14:06.066726+0000","last_deep_scrub":"265'204","last_deep_scrub_stamp":"2024-05-30T17:14:06.066726+0000","last_clean_scrub_stamp":"2024-05-30T17:14:06.066726+0000","objects_scrubbed":7,"log_size":104,"log_dups_size":100,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:29:42.457191+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":256,"num_read_kb":249,"num_write":168,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"265'557","reported_seq":2774,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:57.243711+0000","last_change":"2024-05-30T17:13:57.243776+0000","last_active":"2024-05-30T17:13:57.243711+0000","last_peered":"2024-05-30T17:13:57.243711+0000","last_clean":"2024-05-30T17:13:57.243711+0000","last_became_active":"2024-05-30T15:54:24.677176+0000","last_became_peered":"2024-05-30T15:54:24.677176+0000","last_unstale":"2024-05-30T17:13:57.243711+0000","last_undegraded":"2024-05-30T17:13:57.243711+0000","last_fullsized":"2024-05-30T17:13:57.243711+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'557","last_scrub_stamp":"2024-05-30T17:13:57.243509+0000","last_deep_scrub":"265'557","last_deep_scrub_stamp":"2024-05-30T17:13:57.243509+0000","last_clean_scrub_stamp":"2024-05-30T17:13:57.243509+0000","objects_scrubbed":1,"log_size":57,"log_dups_size":500,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:04:49.675199+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"265'131","reported_seq":647,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.355450+0000","last_change":"2024-05-30T17:13:58.355552+0000","last_active":"2024-05-30T17:13:58.355450+0000","last_peered":"2024-05-30T17:13:58.355450+0000","last_clean":"2024-05-30T17:13:58.355450+0000","last_became_active":"2024-05-30T15:54:26.697294+0000","last_became_peered":"2024-05-30T15:54:26.697294+0000","last_unstale":"2024-05-30T17:13:58.355450+0000","last_undegraded":"2024-05-30T17:13:58.355450+0000","last_fullsized":"2024-05-30T17:13:58.355450+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'131","last_scrub_stamp":"2024-05-30T17:13:58.355257+0000","last_deep_scrub":"265'131","last_deep_scrub_stamp":"2024-05-30T17:13:58.355257+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.355257+0000","objects_scrubbed":7,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:43:23.516003+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1933,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":112,"num_read_kb":106,"num_write":288,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":64,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"265'716","reported_seq":2111,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:12.349843+0000","last_change":"2024-05-30T15:54:18.653261+0000","last_active":"2024-05-30T17:13:12.349843+0000","last_peered":"2024-05-30T17:13:12.349843+0000","last_clean":"2024-05-30T17:13:12.349843+0000","last_became_active":"2024-05-30T15:54:18.652981+0000","last_became_peered":"2024-05-30T15:54:18.652981+0000","last_unstale":"2024-05-30T17:13:12.349843+0000","last_undegraded":"2024-05-30T17:13:12.349843+0000","last_fullsized":"2024-05-30T17:13:12.349843+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":700,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3098,"num_read_kb":2766,"num_write":954,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"265'200","reported_seq":863,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.403305+0000","last_change":"2024-05-30T15:54:16.652512+0000","last_active":"2024-05-30T17:13:38.403305+0000","last_peered":"2024-05-30T17:13:38.403305+0000","last_clean":"2024-05-30T17:13:38.403305+0000","last_became_active":"2024-05-30T15:54:16.652033+0000","last_became_peered":"2024-05-30T15:54:16.652033+0000","last_unstale":"2024-05-30T17:13:38.403305+0000","last_undegraded":"2024-05-30T17:13:38.403305+0000","last_fullsized":"2024-05-30T17:13:38.403305+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":100,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":287689,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":515,"num_read_kb":15333,"num_write":1052,"num_write_kb":22470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":539,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:13.005555+0000","last_change":"2024-05-30T17:14:13.005555+0000","last_active":"2024-05-30T17:14:13.005555+0000","last_peered":"2024-05-30T17:14:13.005555+0000","last_clean":"2024-05-30T17:14:13.005555+0000","last_became_active":"2024-05-30T15:54:08.573754+0000","last_became_peered":"2024-05-30T15:54:08.573754+0000","last_unstale":"2024-05-30T17:14:13.005555+0000","last_undegraded":"2024-05-30T17:14:13.005555+0000","last_fullsized":"2024-05-30T17:14:13.005555+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:13.005314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:13.005314+0000","last_clean_scrub_stamp":"2024-05-30T17:14:13.005314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:57:00.677681+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":530,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:09.067533+0000","last_change":"2024-05-30T17:14:09.067590+0000","last_active":"2024-05-30T17:14:09.067533+0000","last_peered":"2024-05-30T17:14:09.067533+0000","last_clean":"2024-05-30T17:14:09.067533+0000","last_became_active":"2024-05-30T15:54:20.656941+0000","last_became_peered":"2024-05-30T15:54:20.656941+0000","last_unstale":"2024-05-30T17:14:09.067533+0000","last_undegraded":"2024-05-30T17:14:09.067533+0000","last_fullsized":"2024-05-30T17:14:09.067533+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:09.067331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:09.067331+0000","last_clean_scrub_stamp":"2024-05-30T17:14:09.067331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:25:27.763720+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"265'135","reported_seq":717,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:57.901626+0000","last_change":"2024-05-30T17:13:57.901801+0000","last_active":"2024-05-30T17:13:57.901626+0000","last_peered":"2024-05-30T17:13:57.901626+0000","last_clean":"2024-05-30T17:13:57.901626+0000","last_became_active":"2024-05-30T15:54:22.674027+0000","last_became_peered":"2024-05-30T15:54:22.674027+0000","last_unstale":"2024-05-30T17:13:57.901626+0000","last_undegraded":"2024-05-30T17:13:57.901626+0000","last_fullsized":"2024-05-30T17:13:57.901626+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'135","last_scrub_stamp":"2024-05-30T17:13:57.901361+0000","last_deep_scrub":"265'135","last_deep_scrub_stamp":"2024-05-30T17:13:57.901361+0000","last_clean_scrub_stamp":"2024-05-30T17:13:57.901361+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":100,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:02:52.230529+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":164,"num_read_kb":159,"num_write":108,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"265'550","reported_seq":2642,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:08.404324+0000","last_change":"2024-05-30T17:14:08.404392+0000","last_active":"2024-05-30T17:14:08.404324+0000","last_peered":"2024-05-30T17:14:08.404324+0000","last_clean":"2024-05-30T17:14:08.404324+0000","last_became_active":"2024-05-30T15:54:24.676608+0000","last_became_peered":"2024-05-30T15:54:24.676608+0000","last_unstale":"2024-05-30T17:14:08.404324+0000","last_undegraded":"2024-05-30T17:14:08.404324+0000","last_fullsized":"2024-05-30T17:14:08.404324+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'550","last_scrub_stamp":"2024-05-30T17:14:08.404133+0000","last_deep_scrub":"265'550","last_deep_scrub_stamp":"2024-05-30T17:14:08.404133+0000","last_clean_scrub_stamp":"2024-05-30T17:14:08.404133+0000","objects_scrubbed":1,"log_size":50,"log_dups_size":500,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:32:14.717679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"265'233","reported_seq":789,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.655070+0000","last_change":"2024-05-30T17:13:50.655156+0000","last_active":"2024-05-30T17:13:50.655070+0000","last_peered":"2024-05-30T17:13:50.655070+0000","last_clean":"2024-05-30T17:13:50.655070+0000","last_became_active":"2024-05-30T15:54:26.699148+0000","last_became_peered":"2024-05-30T15:54:26.699148+0000","last_unstale":"2024-05-30T17:13:50.655070+0000","last_undegraded":"2024-05-30T17:13:50.655070+0000","last_fullsized":"2024-05-30T17:13:50.655070+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'233","last_scrub_stamp":"2024-05-30T17:13:50.654846+0000","last_deep_scrub":"265'233","last_deep_scrub_stamp":"2024-05-30T17:13:50.654846+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.654846+0000","objects_scrubbed":6,"log_size":33,"log_dups_size":200,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:23:30.288859+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":326,"num_read_kb":298,"num_write":504,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":178,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"265'748","reported_seq":2349,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:08.073027+0000","last_change":"2024-05-30T17:14:08.073095+0000","last_active":"2024-05-30T17:14:08.073027+0000","last_peered":"2024-05-30T17:14:08.073027+0000","last_clean":"2024-05-30T17:14:08.073027+0000","last_became_active":"2024-05-30T15:54:18.657842+0000","last_became_peered":"2024-05-30T15:54:18.657842+0000","last_unstale":"2024-05-30T17:14:08.073027+0000","last_undegraded":"2024-05-30T17:14:08.073027+0000","last_fullsized":"2024-05-30T17:14:08.073027+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'748","last_scrub_stamp":"2024-05-30T17:14:08.072843+0000","last_deep_scrub":"265'748","last_deep_scrub_stamp":"2024-05-30T17:14:08.072843+0000","last_clean_scrub_stamp":"2024-05-30T17:14:08.072843+0000","objects_scrubbed":1,"log_size":48,"log_dups_size":700,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:49:34.177372+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3632,"num_read_kb":3285,"num_write":997,"num_write_kb":370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1532,"num_omap_keys":6,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"265'238","reported_seq":995,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.415399+0000","last_change":"2024-05-30T15:54:16.657515+0000","last_active":"2024-05-30T17:13:38.415399+0000","last_peered":"2024-05-30T17:13:38.415399+0000","last_clean":"2024-05-30T17:13:38.415399+0000","last_became_active":"2024-05-30T15:54:16.657067+0000","last_became_peered":"2024-05-30T15:54:16.657067+0000","last_unstale":"2024-05-30T17:13:38.415399+0000","last_undegraded":"2024-05-30T17:13:38.415399+0000","last_fullsized":"2024-05-30T17:13:38.415399+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":200,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5418111,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":685,"num_read_kb":72437,"num_write":1157,"num_write_kb":30547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":537,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:06.733343+0000","last_change":"2024-05-30T17:14:06.733343+0000","last_active":"2024-05-30T17:14:06.733343+0000","last_peered":"2024-05-30T17:14:06.733343+0000","last_clean":"2024-05-30T17:14:06.733343+0000","last_became_active":"2024-05-30T15:54:08.577191+0000","last_became_peered":"2024-05-30T15:54:08.577191+0000","last_unstale":"2024-05-30T17:14:06.733343+0000","last_undegraded":"2024-05-30T17:14:06.733343+0000","last_fullsized":"2024-05-30T17:14:06.733343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:06.733080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:06.733080+0000","last_clean_scrub_stamp":"2024-05-30T17:14:06.733080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:02:40.927625+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":524,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:17.373241+0000","last_change":"2024-05-30T17:14:17.373292+0000","last_active":"2024-05-30T17:14:17.373241+0000","last_peered":"2024-05-30T17:14:17.373241+0000","last_clean":"2024-05-30T17:14:17.373241+0000","last_became_active":"2024-05-30T15:54:20.653404+0000","last_became_peered":"2024-05-30T15:54:20.653404+0000","last_unstale":"2024-05-30T17:14:17.373241+0000","last_undegraded":"2024-05-30T17:14:17.373241+0000","last_fullsized":"2024-05-30T17:14:17.373241+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:17.373049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:17.373049+0000","last_clean_scrub_stamp":"2024-05-30T17:14:17.373049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:52:07.685494+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"265'346","reported_seq":1094,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.442403+0000","last_change":"2024-05-30T15:54:22.849490+0000","last_active":"2024-05-30T17:13:38.442403+0000","last_peered":"2024-05-30T17:13:38.442403+0000","last_clean":"2024-05-30T17:13:38.442403+0000","last_became_active":"2024-05-30T15:54:22.848761+0000","last_became_peered":"2024-05-30T15:54:22.848761+0000","last_unstale":"2024-05-30T17:13:38.442403+0000","last_undegraded":"2024-05-30T17:13:38.442403+0000","last_fullsized":"2024-05-30T17:13:38.442403+0000","mapping_epoch":28,"log_start":"265'300","ondisk_log_start":"265'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":300,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24415,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":525,"num_read_kb":761,"num_write":274,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":515,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:06.395596+0000","last_change":"2024-05-30T17:14:06.395648+0000","last_active":"2024-05-30T17:14:06.395596+0000","last_peered":"2024-05-30T17:14:06.395596+0000","last_clean":"2024-05-30T17:14:06.395596+0000","last_became_active":"2024-05-30T15:54:24.686801+0000","last_became_peered":"2024-05-30T15:54:24.686801+0000","last_unstale":"2024-05-30T17:14:06.395596+0000","last_undegraded":"2024-05-30T17:14:06.395596+0000","last_fullsized":"2024-05-30T17:14:06.395596+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:06.395403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:06.395403+0000","last_clean_scrub_stamp":"2024-05-30T17:14:06.395403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:26:52.769715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"265'152","reported_seq":662,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.241219+0000","last_change":"2024-05-30T17:13:51.241295+0000","last_active":"2024-05-30T17:13:51.241219+0000","last_peered":"2024-05-30T17:13:51.241219+0000","last_clean":"2024-05-30T17:13:51.241219+0000","last_became_active":"2024-05-30T15:54:26.700325+0000","last_became_peered":"2024-05-30T15:54:26.700325+0000","last_unstale":"2024-05-30T17:13:51.241219+0000","last_undegraded":"2024-05-30T17:13:51.241219+0000","last_fullsized":"2024-05-30T17:13:51.241219+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'152","last_scrub_stamp":"2024-05-30T17:13:51.241024+0000","last_deep_scrub":"265'152","last_deep_scrub_stamp":"2024-05-30T17:13:51.241024+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.241024+0000","objects_scrubbed":2,"log_size":52,"log_dups_size":100,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:14:44.872712+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":204,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":115,"num_write":342,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"265'150","reported_seq":732,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.796700+0000","last_change":"2024-05-30T17:13:56.796775+0000","last_active":"2024-05-30T17:13:56.796700+0000","last_peered":"2024-05-30T17:13:56.796700+0000","last_clean":"2024-05-30T17:13:56.796700+0000","last_became_active":"2024-05-30T15:54:26.686310+0000","last_became_peered":"2024-05-30T15:54:26.686310+0000","last_unstale":"2024-05-30T17:13:56.796700+0000","last_undegraded":"2024-05-30T17:13:56.796700+0000","last_fullsized":"2024-05-30T17:13:56.796700+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'150","last_scrub_stamp":"2024-05-30T17:13:56.796467+0000","last_deep_scrub":"265'150","last_deep_scrub_stamp":"2024-05-30T17:13:56.796467+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.796467+0000","objects_scrubbed":1,"log_size":50,"log_dups_size":100,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:01:08.998802+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":227,"num_read_kb":222,"num_write":347,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"265'747","reported_seq":2056,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:05.009608+0000","last_change":"2024-05-30T17:14:05.009756+0000","last_active":"2024-05-30T17:14:05.009608+0000","last_peered":"2024-05-30T17:14:05.009608+0000","last_clean":"2024-05-30T17:14:05.009608+0000","last_became_active":"2024-05-30T15:54:18.673418+0000","last_became_peered":"2024-05-30T15:54:18.673418+0000","last_unstale":"2024-05-30T17:14:05.009608+0000","last_undegraded":"2024-05-30T17:14:05.009608+0000","last_fullsized":"2024-05-30T17:14:05.009608+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'747","last_scrub_stamp":"2024-05-30T17:14:05.009388+0000","last_deep_scrub":"265'747","last_deep_scrub_stamp":"2024-05-30T17:14:05.009388+0000","last_clean_scrub_stamp":"2024-05-30T17:14:05.009388+0000","objects_scrubbed":1,"log_size":47,"log_dups_size":700,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:56:33.977718+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3258,"num_read_kb":2851,"num_write":1044,"num_write_kb":425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":556,"num_omap_keys":3,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"265'202","reported_seq":830,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.152930+0000","last_change":"2024-05-30T17:13:49.153045+0000","last_active":"2024-05-30T17:13:49.152930+0000","last_peered":"2024-05-30T17:13:49.152930+0000","last_clean":"2024-05-30T17:13:49.152930+0000","last_became_active":"2024-05-30T15:54:54.450966+0000","last_became_peered":"2024-05-30T15:54:54.450966+0000","last_unstale":"2024-05-30T17:13:49.152930+0000","last_undegraded":"2024-05-30T17:13:49.152930+0000","last_fullsized":"2024-05-30T17:13:49.152930+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'202","last_scrub_stamp":"2024-05-30T17:13:49.152709+0000","last_deep_scrub":"265'202","last_deep_scrub_stamp":"2024-05-30T17:13:49.152709+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.152709+0000","objects_scrubbed":6,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:19:50.214056+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262272,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":401,"num_read_kb":38159,"num_write":1164,"num_write_kb":35962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":518,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:00.085249+0000","last_change":"2024-05-30T17:14:00.085304+0000","last_active":"2024-05-30T17:14:00.085249+0000","last_peered":"2024-05-30T17:14:00.085249+0000","last_clean":"2024-05-30T17:14:00.085249+0000","last_became_active":"2024-05-30T15:54:20.664882+0000","last_became_peered":"2024-05-30T15:54:20.664882+0000","last_unstale":"2024-05-30T17:14:00.085249+0000","last_undegraded":"2024-05-30T17:14:00.085249+0000","last_fullsized":"2024-05-30T17:14:00.085249+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:00.085062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:00.085062+0000","last_clean_scrub_stamp":"2024-05-30T17:14:00.085062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:36:44.188047+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"265'153","reported_seq":753,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.074575+0000","last_change":"2024-05-30T15:54:22.672900+0000","last_active":"2024-05-30T17:13:43.074575+0000","last_peered":"2024-05-30T17:13:43.074575+0000","last_clean":"2024-05-30T17:13:43.074575+0000","last_became_active":"2024-05-30T15:54:22.672170+0000","last_became_peered":"2024-05-30T15:54:22.672170+0000","last_unstale":"2024-05-30T17:13:43.074575+0000","last_undegraded":"2024-05-30T17:13:43.074575+0000","last_fullsized":"2024-05-30T17:13:43.074575+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":100,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5933,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":205,"num_read_kb":259,"num_write":110,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":511,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:04.400786+0000","last_change":"2024-05-30T17:14:04.400840+0000","last_active":"2024-05-30T17:14:04.400786+0000","last_peered":"2024-05-30T17:14:04.400786+0000","last_clean":"2024-05-30T17:14:04.400786+0000","last_became_active":"2024-05-30T15:54:24.688053+0000","last_became_peered":"2024-05-30T15:54:24.688053+0000","last_unstale":"2024-05-30T17:14:04.400786+0000","last_undegraded":"2024-05-30T17:14:04.400786+0000","last_fullsized":"2024-05-30T17:14:04.400786+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:04.400563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:04.400563+0000","last_clean_scrub_stamp":"2024-05-30T17:14:04.400563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:58:36.746394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"265'120","reported_seq":654,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.374741+0000","last_change":"2024-05-30T17:13:54.374814+0000","last_active":"2024-05-30T17:13:54.374741+0000","last_peered":"2024-05-30T17:13:54.374741+0000","last_clean":"2024-05-30T17:13:54.374741+0000","last_became_active":"2024-05-30T15:54:26.701588+0000","last_became_peered":"2024-05-30T15:54:26.701588+0000","last_unstale":"2024-05-30T17:13:54.374741+0000","last_undegraded":"2024-05-30T17:13:54.374741+0000","last_fullsized":"2024-05-30T17:13:54.374741+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'120","last_scrub_stamp":"2024-05-30T17:13:54.374529+0000","last_deep_scrub":"265'120","last_deep_scrub_stamp":"2024-05-30T17:13:54.374529+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.374529+0000","objects_scrubbed":3,"log_size":20,"log_dups_size":100,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:54:34.109921+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":826,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":172,"num_read_kb":153,"num_write":293,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"265'323","reported_seq":1770,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:01.069483+0000","last_change":"2024-05-30T17:14:01.069557+0000","last_active":"2024-05-30T17:14:01.069483+0000","last_peered":"2024-05-30T17:14:01.069483+0000","last_clean":"2024-05-30T17:14:01.069483+0000","last_became_active":"2024-05-30T15:54:18.670707+0000","last_became_peered":"2024-05-30T15:54:18.670707+0000","last_unstale":"2024-05-30T17:14:01.069483+0000","last_undegraded":"2024-05-30T17:14:01.069483+0000","last_fullsized":"2024-05-30T17:14:01.069483+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'323","last_scrub_stamp":"2024-05-30T17:14:01.069299+0000","last_deep_scrub":"265'323","last_deep_scrub_stamp":"2024-05-30T17:14:01.069299+0000","last_clean_scrub_stamp":"2024-05-30T17:14:01.069299+0000","objects_scrubbed":2,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:28:00.081754+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1910,"num_read_kb":1828,"num_write":376,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"265'219","reported_seq":885,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.051242+0000","last_change":"2024-05-30T15:54:16.663769+0000","last_active":"2024-05-30T17:13:45.051242+0000","last_peered":"2024-05-30T17:13:45.051242+0000","last_clean":"2024-05-30T17:13:45.051242+0000","last_became_active":"2024-05-30T15:54:16.660427+0000","last_became_peered":"2024-05-30T15:54:16.660427+0000","last_unstale":"2024-05-30T17:13:45.051242+0000","last_undegraded":"2024-05-30T17:13:45.051242+0000","last_fullsized":"2024-05-30T17:13:45.051242+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":200,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262181,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":518,"num_read_kb":33224,"num_write":1148,"num_write_kb":46290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":519,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:15.367435+0000","last_change":"2024-05-30T17:14:15.367493+0000","last_active":"2024-05-30T17:14:15.367435+0000","last_peered":"2024-05-30T17:14:15.367435+0000","last_clean":"2024-05-30T17:14:15.367435+0000","last_became_active":"2024-05-30T15:54:20.649330+0000","last_became_peered":"2024-05-30T15:54:20.649330+0000","last_unstale":"2024-05-30T17:14:15.367435+0000","last_undegraded":"2024-05-30T17:14:15.367435+0000","last_fullsized":"2024-05-30T17:14:15.367435+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:15.367236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:15.367236+0000","last_clean_scrub_stamp":"2024-05-30T17:14:15.367236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:47:00.342990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"265'71","reported_seq":597,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.647459+0000","last_change":"2024-05-30T15:54:22.663453+0000","last_active":"2024-05-30T17:13:41.647459+0000","last_peered":"2024-05-30T17:13:41.647459+0000","last_clean":"2024-05-30T17:13:41.647459+0000","last_became_active":"2024-05-30T15:54:22.662738+0000","last_became_peered":"2024-05-30T15:54:22.662738+0000","last_unstale":"2024-05-30T17:13:41.647459+0000","last_undegraded":"2024-05-30T17:13:41.647459+0000","last_fullsized":"2024-05-30T17:13:41.647459+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":50,"num_write":40,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":509,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:05.756275+0000","last_change":"2024-05-30T17:14:05.756326+0000","last_active":"2024-05-30T17:14:05.756275+0000","last_peered":"2024-05-30T17:14:05.756275+0000","last_clean":"2024-05-30T17:14:05.756275+0000","last_became_active":"2024-05-30T15:54:24.680578+0000","last_became_peered":"2024-05-30T15:54:24.680578+0000","last_unstale":"2024-05-30T17:14:05.756275+0000","last_undegraded":"2024-05-30T17:14:05.756275+0000","last_fullsized":"2024-05-30T17:14:05.756275+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:05.756098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:05.756098+0000","last_clean_scrub_stamp":"2024-05-30T17:14:05.756098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:07:27.828294+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"265'215","reported_seq":784,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.750305+0000","last_change":"2024-05-30T17:13:54.750473+0000","last_active":"2024-05-30T17:13:54.750305+0000","last_peered":"2024-05-30T17:13:54.750305+0000","last_clean":"2024-05-30T17:13:54.750305+0000","last_became_active":"2024-05-30T15:54:26.686553+0000","last_became_peered":"2024-05-30T15:54:26.686553+0000","last_unstale":"2024-05-30T17:13:54.750305+0000","last_undegraded":"2024-05-30T17:13:54.750305+0000","last_fullsized":"2024-05-30T17:13:54.750305+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'215","last_scrub_stamp":"2024-05-30T17:13:54.750033+0000","last_deep_scrub":"265'215","last_deep_scrub_stamp":"2024-05-30T17:13:54.750033+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.750033+0000","objects_scrubbed":3,"log_size":15,"log_dups_size":200,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:23:19.977063+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":816,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":414,"num_read_kb":383,"num_write":513,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":398,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"265'419","reported_seq":1805,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:14.364566+0000","last_change":"2024-05-30T17:14:14.364672+0000","last_active":"2024-05-30T17:14:14.364566+0000","last_peered":"2024-05-30T17:14:14.364566+0000","last_clean":"2024-05-30T17:14:14.364566+0000","last_became_active":"2024-05-30T15:54:18.660089+0000","last_became_peered":"2024-05-30T15:54:18.660089+0000","last_unstale":"2024-05-30T17:14:14.364566+0000","last_undegraded":"2024-05-30T17:14:14.364566+0000","last_fullsized":"2024-05-30T17:14:14.364566+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'419","last_scrub_stamp":"2024-05-30T17:14:14.364379+0000","last_deep_scrub":"265'419","last_deep_scrub_stamp":"2024-05-30T17:14:14.364379+0000","last_clean_scrub_stamp":"2024-05-30T17:14:14.364379+0000","objects_scrubbed":6,"log_size":19,"log_dups_size":400,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:24:09.477211+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2036,"num_read_kb":1890,"num_write":519,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"265'217","reported_seq":896,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:20.299717+0000","last_change":"2024-05-30T17:14:20.299925+0000","last_active":"2024-05-30T17:14:20.299717+0000","last_peered":"2024-05-30T17:14:20.299717+0000","last_clean":"2024-05-30T17:14:20.299717+0000","last_became_active":"2024-05-30T15:54:16.650025+0000","last_became_peered":"2024-05-30T15:54:16.650025+0000","last_unstale":"2024-05-30T17:14:20.299717+0000","last_undegraded":"2024-05-30T17:14:20.299717+0000","last_fullsized":"2024-05-30T17:14:20.299717+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'217","last_scrub_stamp":"2024-05-30T17:14:20.299487+0000","last_deep_scrub":"265'217","last_deep_scrub_stamp":"2024-05-30T17:14:20.299487+0000","last_clean_scrub_stamp":"2024-05-30T17:14:20.299487+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":200,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:30:25.801340+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5569939,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":500,"num_read_kb":102670,"num_write":1113,"num_write_kb":39714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":517,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:12.802057+0000","last_change":"2024-05-30T17:14:12.802110+0000","last_active":"2024-05-30T17:14:12.802057+0000","last_peered":"2024-05-30T17:14:12.802057+0000","last_clean":"2024-05-30T17:14:12.802057+0000","last_became_active":"2024-05-30T15:54:20.656663+0000","last_became_peered":"2024-05-30T15:54:20.656663+0000","last_unstale":"2024-05-30T17:14:12.802057+0000","last_undegraded":"2024-05-30T17:14:12.802057+0000","last_fullsized":"2024-05-30T17:14:12.802057+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:12.801868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:12.801868+0000","last_clean_scrub_stamp":"2024-05-30T17:14:12.801868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:29:15.815405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"265'186","reported_seq":802,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.074553+0000","last_change":"2024-05-30T15:54:22.676934+0000","last_active":"2024-05-30T17:13:43.074553+0000","last_peered":"2024-05-30T17:13:43.074553+0000","last_clean":"2024-05-30T17:13:43.074553+0000","last_became_active":"2024-05-30T15:54:22.673673+0000","last_became_peered":"2024-05-30T15:54:22.673673+0000","last_unstale":"2024-05-30T17:13:43.074553+0000","last_undegraded":"2024-05-30T17:13:43.074553+0000","last_fullsized":"2024-05-30T17:13:43.074553+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":100,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7114,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":259,"num_read_kb":332,"num_write":146,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":508,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.754848+0000","last_change":"2024-05-30T17:13:50.754954+0000","last_active":"2024-05-30T17:13:50.754848+0000","last_peered":"2024-05-30T17:13:50.754848+0000","last_clean":"2024-05-30T17:13:50.754848+0000","last_became_active":"2024-05-30T15:54:24.675289+0000","last_became_peered":"2024-05-30T15:54:24.675289+0000","last_unstale":"2024-05-30T17:13:50.754848+0000","last_undegraded":"2024-05-30T17:13:50.754848+0000","last_fullsized":"2024-05-30T17:13:50.754848+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:50.754554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:50.754554+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.754554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:47:56.533373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"265'141","reported_seq":646,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.256964+0000","last_change":"2024-05-30T17:13:50.257049+0000","last_active":"2024-05-30T17:13:50.256964+0000","last_peered":"2024-05-30T17:13:50.256964+0000","last_clean":"2024-05-30T17:13:50.256964+0000","last_became_active":"2024-05-30T15:54:26.703433+0000","last_became_peered":"2024-05-30T15:54:26.703433+0000","last_unstale":"2024-05-30T17:13:50.256964+0000","last_undegraded":"2024-05-30T17:13:50.256964+0000","last_fullsized":"2024-05-30T17:13:50.256964+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'141","last_scrub_stamp":"2024-05-30T17:13:50.256770+0000","last_deep_scrub":"265'141","last_deep_scrub_stamp":"2024-05-30T17:13:50.256770+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.256770+0000","objects_scrubbed":3,"log_size":41,"log_dups_size":100,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:08:34.180219+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":856,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":110,"num_write":329,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"265'863","reported_seq":2492,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.619000+0000","last_change":"2024-05-30T15:54:18.649560+0000","last_active":"2024-05-30T17:13:41.619000+0000","last_peered":"2024-05-30T17:13:41.619000+0000","last_clean":"2024-05-30T17:13:41.619000+0000","last_became_active":"2024-05-30T15:54:18.649273+0000","last_became_peered":"2024-05-30T15:54:18.649273+0000","last_unstale":"2024-05-30T17:13:41.619000+0000","last_undegraded":"2024-05-30T17:13:41.619000+0000","last_fullsized":"2024-05-30T17:13:41.619000+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":800,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3918,"num_read_kb":3498,"num_write":1187,"num_write_kb":458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"265'207","reported_seq":869,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.646953+0000","last_change":"2024-05-30T15:54:16.649854+0000","last_active":"2024-05-30T17:13:41.646953+0000","last_peered":"2024-05-30T17:13:41.646953+0000","last_clean":"2024-05-30T17:13:41.646953+0000","last_became_active":"2024-05-30T15:54:16.649121+0000","last_became_peered":"2024-05-30T15:54:16.649121+0000","last_unstale":"2024-05-30T17:13:41.646953+0000","last_undegraded":"2024-05-30T17:13:41.646953+0000","last_fullsized":"2024-05-30T17:13:41.646953+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":100,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1029196,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":493,"num_read_kb":65907,"num_write":1120,"num_write_kb":54558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":516,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.736994+0000","last_change":"2024-05-30T17:13:51.737047+0000","last_active":"2024-05-30T17:13:51.736994+0000","last_peered":"2024-05-30T17:13:51.736994+0000","last_clean":"2024-05-30T17:13:51.736994+0000","last_became_active":"2024-05-30T15:54:20.667863+0000","last_became_peered":"2024-05-30T15:54:20.667863+0000","last_unstale":"2024-05-30T17:13:51.736994+0000","last_undegraded":"2024-05-30T17:13:51.736994+0000","last_fullsized":"2024-05-30T17:13:51.736994+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:51.736796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:51.736796+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.736796+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:58:41.051038+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"265'143","reported_seq":734,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.052100+0000","last_change":"2024-05-30T15:54:22.665742+0000","last_active":"2024-05-30T17:13:45.052100+0000","last_peered":"2024-05-30T17:13:45.052100+0000","last_clean":"2024-05-30T17:13:45.052100+0000","last_became_active":"2024-05-30T15:54:22.664977+0000","last_became_peered":"2024-05-30T15:54:22.664977+0000","last_unstale":"2024-05-30T17:13:45.052100+0000","last_undegraded":"2024-05-30T17:13:45.052100+0000","last_fullsized":"2024-05-30T17:13:45.052100+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":100,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7290,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":187,"num_read_kb":260,"num_write":98,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":514,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:02.866942+0000","last_change":"2024-05-30T17:14:02.866997+0000","last_active":"2024-05-30T17:14:02.866942+0000","last_peered":"2024-05-30T17:14:02.866942+0000","last_clean":"2024-05-30T17:14:02.866942+0000","last_became_active":"2024-05-30T15:54:24.676707+0000","last_became_peered":"2024-05-30T15:54:24.676707+0000","last_unstale":"2024-05-30T17:14:02.866942+0000","last_undegraded":"2024-05-30T17:14:02.866942+0000","last_fullsized":"2024-05-30T17:14:02.866942+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:02.866743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:02.866743+0000","last_clean_scrub_stamp":"2024-05-30T17:14:02.866743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:19:34.249137+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"265'443","reported_seq":2109,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.771973+0000","last_change":"2024-05-30T17:13:59.772041+0000","last_active":"2024-05-30T17:13:59.771973+0000","last_peered":"2024-05-30T17:13:59.771973+0000","last_clean":"2024-05-30T17:13:59.771973+0000","last_became_active":"2024-05-30T15:54:18.670756+0000","last_became_peered":"2024-05-30T15:54:18.670756+0000","last_unstale":"2024-05-30T17:13:59.771973+0000","last_undegraded":"2024-05-30T17:13:59.771973+0000","last_fullsized":"2024-05-30T17:13:59.771973+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'443","last_scrub_stamp":"2024-05-30T17:13:59.771783+0000","last_deep_scrub":"265'443","last_deep_scrub_stamp":"2024-05-30T17:13:59.771783+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.771783+0000","objects_scrubbed":1,"log_size":43,"log_dups_size":400,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:50:25.853396+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2383,"num_read_kb":2277,"num_write":522,"num_write_kb":97,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"265'164","reported_seq":770,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.073839+0000","last_change":"2024-05-30T15:54:16.654636+0000","last_active":"2024-05-30T17:13:43.073839+0000","last_peered":"2024-05-30T17:13:43.073839+0000","last_clean":"2024-05-30T17:13:43.073839+0000","last_became_active":"2024-05-30T15:54:16.653560+0000","last_became_peered":"2024-05-30T15:54:16.653560+0000","last_unstale":"2024-05-30T17:13:43.073839+0000","last_undegraded":"2024-05-30T17:13:43.073839+0000","last_fullsized":"2024-05-30T17:13:43.073839+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":100,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5417712,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":326,"num_read_kb":31913,"num_write":888,"num_write_kb":20958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":527,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:17.827376+0000","last_change":"2024-05-30T17:14:17.827438+0000","last_active":"2024-05-30T17:14:17.827376+0000","last_peered":"2024-05-30T17:14:17.827376+0000","last_clean":"2024-05-30T17:14:17.827376+0000","last_became_active":"2024-05-30T15:54:20.662669+0000","last_became_peered":"2024-05-30T15:54:20.662669+0000","last_unstale":"2024-05-30T17:14:17.827376+0000","last_undegraded":"2024-05-30T17:14:17.827376+0000","last_fullsized":"2024-05-30T17:14:17.827376+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:17.827181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:17.827181+0000","last_clean_scrub_stamp":"2024-05-30T17:14:17.827181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:03:52.723170+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"265'228","reported_seq":886,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:02.038315+0000","last_change":"2024-05-30T17:14:02.038392+0000","last_active":"2024-05-30T17:14:02.038315+0000","last_peered":"2024-05-30T17:14:02.038315+0000","last_clean":"2024-05-30T17:14:02.038315+0000","last_became_active":"2024-05-30T15:54:22.663518+0000","last_became_peered":"2024-05-30T15:54:22.663518+0000","last_unstale":"2024-05-30T17:14:02.038315+0000","last_undegraded":"2024-05-30T17:14:02.038315+0000","last_fullsized":"2024-05-30T17:14:02.038315+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'228","last_scrub_stamp":"2024-05-30T17:14:02.038127+0000","last_deep_scrub":"265'228","last_deep_scrub_stamp":"2024-05-30T17:14:02.038127+0000","last_clean_scrub_stamp":"2024-05-30T17:14:02.038127+0000","objects_scrubbed":6,"log_size":28,"log_dups_size":200,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:44:55.851829+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11676,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":342,"num_read_kb":490,"num_write":222,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":24966,"num_omap_keys":48,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:55.195013+0000","last_change":"2024-05-30T17:13:55.195111+0000","last_active":"2024-05-30T17:13:55.195013+0000","last_peered":"2024-05-30T17:13:55.195013+0000","last_clean":"2024-05-30T17:13:55.195013+0000","last_became_active":"2024-05-30T15:54:24.682609+0000","last_became_peered":"2024-05-30T15:54:24.682609+0000","last_unstale":"2024-05-30T17:13:55.195013+0000","last_undegraded":"2024-05-30T17:13:55.195013+0000","last_fullsized":"2024-05-30T17:13:55.195013+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:55.194814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:55.194814+0000","last_clean_scrub_stamp":"2024-05-30T17:13:55.194814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:35:33.515747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"265'151","reported_seq":672,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:00.314401+0000","last_change":"2024-05-30T17:14:00.314480+0000","last_active":"2024-05-30T17:14:00.314401+0000","last_peered":"2024-05-30T17:14:00.314401+0000","last_clean":"2024-05-30T17:14:00.314401+0000","last_became_active":"2024-05-30T15:54:26.695210+0000","last_became_peered":"2024-05-30T15:54:26.695210+0000","last_unstale":"2024-05-30T17:14:00.314401+0000","last_undegraded":"2024-05-30T17:14:00.314401+0000","last_fullsized":"2024-05-30T17:14:00.314401+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'151","last_scrub_stamp":"2024-05-30T17:14:00.314210+0000","last_deep_scrub":"265'151","last_deep_scrub_stamp":"2024-05-30T17:14:00.314210+0000","last_clean_scrub_stamp":"2024-05-30T17:14:00.314210+0000","objects_scrubbed":2,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:57:02.003537+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":643,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":167,"num_read_kb":179,"num_write":338,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"265'854","reported_seq":2476,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:44.656545+0000","last_change":"2024-05-30T15:54:18.641464+0000","last_active":"2024-05-30T17:12:44.656545+0000","last_peered":"2024-05-30T17:12:44.656545+0000","last_clean":"2024-05-30T17:12:44.656545+0000","last_became_active":"2024-05-30T15:54:18.640739+0000","last_became_peered":"2024-05-30T15:54:18.640739+0000","last_unstale":"2024-05-30T17:12:44.656545+0000","last_undegraded":"2024-05-30T17:12:44.656545+0000","last_fullsized":"2024-05-30T17:12:44.656545+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":800,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4053,"num_read_kb":3626,"num_write":1150,"num_write_kb":447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"265'245","reported_seq":988,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.935811+0000","last_change":"2024-05-30T17:13:47.935811+0000","last_active":"2024-05-30T17:13:47.935811+0000","last_peered":"2024-05-30T17:13:47.935811+0000","last_clean":"2024-05-30T17:13:47.935811+0000","last_became_active":"2024-05-30T15:54:54.424825+0000","last_became_peered":"2024-05-30T15:54:54.424825+0000","last_unstale":"2024-05-30T17:13:47.935811+0000","last_undegraded":"2024-05-30T17:13:47.935811+0000","last_fullsized":"2024-05-30T17:13:47.935811+0000","mapping_epoch":35,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'245","last_scrub_stamp":"2024-05-30T17:13:47.935515+0000","last_deep_scrub":"265'245","last_deep_scrub_stamp":"2024-05-30T17:13:47.935515+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.935515+0000","objects_scrubbed":3,"log_size":45,"log_dups_size":200,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:50:12.821107+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5285497,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":887,"num_read_kb":12756,"num_write":1230,"num_write_kb":34615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"265'4","reported_seq":848,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.789874+0000","last_change":"2024-05-30T17:13:53.789936+0000","last_active":"2024-05-30T17:13:53.789874+0000","last_peered":"2024-05-30T17:13:53.789874+0000","last_clean":"2024-05-30T17:13:53.789874+0000","last_became_active":"2024-05-30T15:54:20.664095+0000","last_became_peered":"2024-05-30T15:54:20.664095+0000","last_unstale":"2024-05-30T17:13:53.789874+0000","last_undegraded":"2024-05-30T17:13:53.789874+0000","last_fullsized":"2024-05-30T17:13:53.789874+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'4","last_scrub_stamp":"2024-05-30T17:13:53.789681+0000","last_deep_scrub":"265'4","last_deep_scrub_stamp":"2024-05-30T17:13:53.789681+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.789681+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:25:01.268608+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":494,"num_read_kb":492,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"265'202","reported_seq":821,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:20.840621+0000","last_change":"2024-05-30T17:14:20.840701+0000","last_active":"2024-05-30T17:14:20.840621+0000","last_peered":"2024-05-30T17:14:20.840621+0000","last_clean":"2024-05-30T17:14:20.840621+0000","last_became_active":"2024-05-30T15:54:22.664997+0000","last_became_peered":"2024-05-30T15:54:22.664997+0000","last_unstale":"2024-05-30T17:14:20.840621+0000","last_undegraded":"2024-05-30T17:14:20.840621+0000","last_fullsized":"2024-05-30T17:14:20.840621+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'202","last_scrub_stamp":"2024-05-30T17:14:20.840427+0000","last_deep_scrub":"265'202","last_deep_scrub_stamp":"2024-05-30T17:14:20.840427+0000","last_clean_scrub_stamp":"2024-05-30T17:14:20.840427+0000","objects_scrubbed":8,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:09:02.158801+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25625,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":290,"num_read_kb":412,"num_write":192,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11460,"num_omap_keys":24,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":507,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:10.050301+0000","last_change":"2024-05-30T17:14:10.050358+0000","last_active":"2024-05-30T17:14:10.050301+0000","last_peered":"2024-05-30T17:14:10.050301+0000","last_clean":"2024-05-30T17:14:10.050301+0000","last_became_active":"2024-05-30T15:54:24.675583+0000","last_became_peered":"2024-05-30T15:54:24.675583+0000","last_unstale":"2024-05-30T17:14:10.050301+0000","last_undegraded":"2024-05-30T17:14:10.050301+0000","last_fullsized":"2024-05-30T17:14:10.050301+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:10.050095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:10.050095+0000","last_clean_scrub_stamp":"2024-05-30T17:14:10.050095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:05:37.764975+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"265'148","reported_seq":673,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:14.828641+0000","last_change":"2024-05-30T17:14:14.828728+0000","last_active":"2024-05-30T17:14:14.828641+0000","last_peered":"2024-05-30T17:14:14.828641+0000","last_clean":"2024-05-30T17:14:14.828641+0000","last_became_active":"2024-05-30T15:54:26.700392+0000","last_became_peered":"2024-05-30T15:54:26.700392+0000","last_unstale":"2024-05-30T17:14:14.828641+0000","last_undegraded":"2024-05-30T17:14:14.828641+0000","last_fullsized":"2024-05-30T17:14:14.828641+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'148","last_scrub_stamp":"2024-05-30T17:14:14.828458+0000","last_deep_scrub":"265'148","last_deep_scrub_stamp":"2024-05-30T17:14:14.828458+0000","last_clean_scrub_stamp":"2024-05-30T17:14:14.828458+0000","objects_scrubbed":8,"log_size":48,"log_dups_size":100,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:48:23.720296+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2627,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":177,"num_read_kb":174,"num_write":403,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"265'371","reported_seq":1751,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.352650+0000","last_change":"2024-05-30T17:13:53.352650+0000","last_active":"2024-05-30T17:13:53.352650+0000","last_peered":"2024-05-30T17:13:53.352650+0000","last_clean":"2024-05-30T17:13:53.352650+0000","last_became_active":"2024-05-30T15:54:54.419468+0000","last_became_peered":"2024-05-30T15:54:54.419468+0000","last_unstale":"2024-05-30T17:13:53.352650+0000","last_undegraded":"2024-05-30T17:13:53.352650+0000","last_fullsized":"2024-05-30T17:13:53.352650+0000","mapping_epoch":35,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'371","last_scrub_stamp":"2024-05-30T17:13:53.352335+0000","last_deep_scrub":"265'371","last_deep_scrub_stamp":"2024-05-30T17:13:53.352335+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.352335+0000","objects_scrubbed":3,"log_size":71,"log_dups_size":300,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:30:41.234405+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1898,"num_read_kb":1782,"num_write":455,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":310,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"265'159","reported_seq":894,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:19.091877+0000","last_change":"2024-05-30T17:14:19.092022+0000","last_active":"2024-05-30T17:14:19.091877+0000","last_peered":"2024-05-30T17:14:19.091877+0000","last_clean":"2024-05-30T17:14:19.091877+0000","last_became_active":"2024-05-30T15:54:16.676440+0000","last_became_peered":"2024-05-30T15:54:16.676440+0000","last_unstale":"2024-05-30T17:14:19.091877+0000","last_undegraded":"2024-05-30T17:14:19.091877+0000","last_fullsized":"2024-05-30T17:14:19.091877+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'159","last_scrub_stamp":"2024-05-30T17:14:19.091627+0000","last_deep_scrub":"265'159","last_deep_scrub_stamp":"2024-05-30T17:14:19.091627+0000","last_clean_scrub_stamp":"2024-05-30T17:14:19.091627+0000","objects_scrubbed":6,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:13:38.026022+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456485,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":869,"num_read_kb":45599,"num_write":925,"num_write_kb":34636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":524,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:07.042918+0000","last_change":"2024-05-30T17:14:07.042918+0000","last_active":"2024-05-30T17:14:07.042918+0000","last_peered":"2024-05-30T17:14:07.042918+0000","last_clean":"2024-05-30T17:14:07.042918+0000","last_became_active":"2024-05-30T15:54:20.655388+0000","last_became_peered":"2024-05-30T15:54:20.655388+0000","last_unstale":"2024-05-30T17:14:07.042918+0000","last_undegraded":"2024-05-30T17:14:07.042918+0000","last_fullsized":"2024-05-30T17:14:07.042918+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:07.042677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:07.042677+0000","last_clean_scrub_stamp":"2024-05-30T17:14:07.042677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:08:28.451745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"265'170","reported_seq":782,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:09.437645+0000","last_change":"2024-05-30T17:14:09.437817+0000","last_active":"2024-05-30T17:14:09.437645+0000","last_peered":"2024-05-30T17:14:09.437645+0000","last_clean":"2024-05-30T17:14:09.437645+0000","last_became_active":"2024-05-30T15:54:22.849193+0000","last_became_peered":"2024-05-30T15:54:22.849193+0000","last_unstale":"2024-05-30T17:14:09.437645+0000","last_undegraded":"2024-05-30T17:14:09.437645+0000","last_fullsized":"2024-05-30T17:14:09.437645+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'170","last_scrub_stamp":"2024-05-30T17:14:09.437396+0000","last_deep_scrub":"265'170","last_deep_scrub_stamp":"2024-05-30T17:14:09.437396+0000","last_clean_scrub_stamp":"2024-05-30T17:14:09.437396+0000","objects_scrubbed":5,"log_size":70,"log_dups_size":100,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:29:49.223193+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11086,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":221,"num_read_kb":302,"num_write":124,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":515,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:19.980498+0000","last_change":"2024-05-30T17:14:19.980627+0000","last_active":"2024-05-30T17:14:19.980498+0000","last_peered":"2024-05-30T17:14:19.980498+0000","last_clean":"2024-05-30T17:14:19.980498+0000","last_became_active":"2024-05-30T15:54:24.681609+0000","last_became_peered":"2024-05-30T15:54:24.681609+0000","last_unstale":"2024-05-30T17:14:19.980498+0000","last_undegraded":"2024-05-30T17:14:19.980498+0000","last_fullsized":"2024-05-30T17:14:19.980498+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:19.980274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:19.980274+0000","last_clean_scrub_stamp":"2024-05-30T17:14:19.980274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:01:43.948910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"265'132","reported_seq":644,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.247687+0000","last_change":"2024-05-30T17:13:47.247831+0000","last_active":"2024-05-30T17:13:47.247687+0000","last_peered":"2024-05-30T17:13:47.247687+0000","last_clean":"2024-05-30T17:13:47.247687+0000","last_became_active":"2024-05-30T15:54:26.704842+0000","last_became_peered":"2024-05-30T15:54:26.704842+0000","last_unstale":"2024-05-30T17:13:47.247687+0000","last_undegraded":"2024-05-30T17:13:47.247687+0000","last_fullsized":"2024-05-30T17:13:47.247687+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'132","last_scrub_stamp":"2024-05-30T17:13:47.247463+0000","last_deep_scrub":"265'132","last_deep_scrub_stamp":"2024-05-30T17:13:47.247463+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.247463+0000","objects_scrubbed":3,"log_size":32,"log_dups_size":100,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:58:57.717875+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":383,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":127,"num_read_kb":122,"num_write":322,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"265'761","reported_seq":2269,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:02.365540+0000","last_change":"2024-05-30T17:14:02.365540+0000","last_active":"2024-05-30T17:14:02.365540+0000","last_peered":"2024-05-30T17:14:02.365540+0000","last_clean":"2024-05-30T17:14:02.365540+0000","last_became_active":"2024-05-30T15:54:18.634803+0000","last_became_peered":"2024-05-30T15:54:18.634803+0000","last_unstale":"2024-05-30T17:14:02.365540+0000","last_undegraded":"2024-05-30T17:14:02.365540+0000","last_fullsized":"2024-05-30T17:14:02.365540+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'761","last_scrub_stamp":"2024-05-30T17:14:02.365235+0000","last_deep_scrub":"265'761","last_deep_scrub_stamp":"2024-05-30T17:14:02.365235+0000","last_clean_scrub_stamp":"2024-05-30T17:14:02.365235+0000","objects_scrubbed":8,"log_size":61,"log_dups_size":700,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:38:15.880298+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":3476,"num_read_kb":3109,"num_write":1006,"num_write_kb":389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1743,"num_omap_keys":8,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"265'190","reported_seq":823,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.153453+0000","last_change":"2024-05-30T17:13:47.153543+0000","last_active":"2024-05-30T17:13:47.153453+0000","last_peered":"2024-05-30T17:13:47.153453+0000","last_clean":"2024-05-30T17:13:47.153453+0000","last_became_active":"2024-05-30T15:54:54.430490+0000","last_became_peered":"2024-05-30T15:54:54.430490+0000","last_unstale":"2024-05-30T17:13:47.153453+0000","last_undegraded":"2024-05-30T17:13:47.153453+0000","last_fullsized":"2024-05-30T17:13:47.153453+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'190","last_scrub_stamp":"2024-05-30T17:13:47.153254+0000","last_deep_scrub":"265'190","last_deep_scrub_stamp":"2024-05-30T17:13:47.153254+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.153254+0000","objects_scrubbed":4,"log_size":90,"log_dups_size":100,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:50:08.663040+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":481,"num_read_kb":40323,"num_write":1075,"num_write_kb":31617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":516,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.000902+0000","last_change":"2024-05-30T17:13:56.000902+0000","last_active":"2024-05-30T17:13:56.000902+0000","last_peered":"2024-05-30T17:13:56.000902+0000","last_clean":"2024-05-30T17:13:56.000902+0000","last_became_active":"2024-05-30T15:54:20.650944+0000","last_became_peered":"2024-05-30T15:54:20.650944+0000","last_unstale":"2024-05-30T17:13:56.000902+0000","last_undegraded":"2024-05-30T17:13:56.000902+0000","last_fullsized":"2024-05-30T17:13:56.000902+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:56.000601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:56.000601+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.000601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:28:54.937679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"265'120","reported_seq":687,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.917769+0000","last_change":"2024-05-30T17:13:56.917856+0000","last_active":"2024-05-30T17:13:56.917769+0000","last_peered":"2024-05-30T17:13:56.917769+0000","last_clean":"2024-05-30T17:13:56.917769+0000","last_became_active":"2024-05-30T15:54:22.850865+0000","last_became_peered":"2024-05-30T15:54:22.850865+0000","last_unstale":"2024-05-30T17:13:56.917769+0000","last_undegraded":"2024-05-30T17:13:56.917769+0000","last_fullsized":"2024-05-30T17:13:56.917769+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'120","last_scrub_stamp":"2024-05-30T17:13:56.917584+0000","last_deep_scrub":"265'120","last_deep_scrub_stamp":"2024-05-30T17:13:56.917584+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.917584+0000","objects_scrubbed":7,"log_size":20,"log_dups_size":100,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:18:29.769236+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":127,"num_read_kb":121,"num_write":92,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1780,"num_omap_keys":6,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"265'541","reported_seq":4178,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.730062+0000","last_change":"2024-05-30T17:13:58.730127+0000","last_active":"2024-05-30T17:13:58.730062+0000","last_peered":"2024-05-30T17:13:58.730062+0000","last_clean":"2024-05-30T17:13:58.730062+0000","last_became_active":"2024-05-30T15:54:24.680695+0000","last_became_peered":"2024-05-30T15:54:24.680695+0000","last_unstale":"2024-05-30T17:13:58.730062+0000","last_undegraded":"2024-05-30T17:13:58.730062+0000","last_fullsized":"2024-05-30T17:13:58.730062+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'541","last_scrub_stamp":"2024-05-30T17:13:58.729875+0000","last_deep_scrub":"265'541","last_deep_scrub_stamp":"2024-05-30T17:13:58.729875+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.729875+0000","objects_scrubbed":1,"log_size":41,"log_dups_size":500,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:30:04.009452+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"265'135","reported_seq":652,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:01.631789+0000","last_change":"2024-05-30T17:14:01.631955+0000","last_active":"2024-05-30T17:14:01.631789+0000","last_peered":"2024-05-30T17:14:01.631789+0000","last_clean":"2024-05-30T17:14:01.631789+0000","last_became_active":"2024-05-30T15:54:26.695842+0000","last_became_peered":"2024-05-30T15:54:26.695842+0000","last_unstale":"2024-05-30T17:14:01.631789+0000","last_undegraded":"2024-05-30T17:14:01.631789+0000","last_fullsized":"2024-05-30T17:14:01.631789+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'135","last_scrub_stamp":"2024-05-30T17:14:01.631538+0000","last_deep_scrub":"265'135","last_deep_scrub_stamp":"2024-05-30T17:14:01.631538+0000","last_clean_scrub_stamp":"2024-05-30T17:14:01.631538+0000","objects_scrubbed":1,"log_size":35,"log_dups_size":100,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:51:24.578349+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":418,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":133,"num_read_kb":127,"num_write":334,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"265'429","reported_seq":1958,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:06.042514+0000","last_change":"2024-05-30T17:14:06.042514+0000","last_active":"2024-05-30T17:14:06.042514+0000","last_peered":"2024-05-30T17:14:06.042514+0000","last_clean":"2024-05-30T17:14:06.042514+0000","last_became_active":"2024-05-30T15:54:18.660567+0000","last_became_peered":"2024-05-30T15:54:18.660567+0000","last_unstale":"2024-05-30T17:14:06.042514+0000","last_undegraded":"2024-05-30T17:14:06.042514+0000","last_fullsized":"2024-05-30T17:14:06.042514+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'429","last_scrub_stamp":"2024-05-30T17:14:06.042078+0000","last_deep_scrub":"265'429","last_deep_scrub_stamp":"2024-05-30T17:14:06.042078+0000","last_clean_scrub_stamp":"2024-05-30T17:14:06.042078+0000","objects_scrubbed":6,"log_size":29,"log_dups_size":400,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:17:31.512141+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2226,"num_read_kb":2104,"num_write":504,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"265'306","reported_seq":1007,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:11.360417+0000","last_change":"2024-05-30T17:14:11.360417+0000","last_active":"2024-05-30T17:14:11.360417+0000","last_peered":"2024-05-30T17:14:11.360417+0000","last_clean":"2024-05-30T17:14:11.360417+0000","last_became_active":"2024-05-30T15:54:16.685691+0000","last_became_peered":"2024-05-30T15:54:16.685691+0000","last_unstale":"2024-05-30T17:14:11.360417+0000","last_undegraded":"2024-05-30T17:14:11.360417+0000","last_fullsized":"2024-05-30T17:14:11.360417+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'306","last_scrub_stamp":"2024-05-30T17:14:11.360138+0000","last_deep_scrub":"265'306","last_deep_scrub_stamp":"2024-05-30T17:14:11.360138+0000","last_clean_scrub_stamp":"2024-05-30T17:14:11.360138+0000","objects_scrubbed":1,"log_size":106,"log_dups_size":200,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:54:48.371531+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":764,"num_read_kb":1764,"num_write":1253,"num_write_kb":18095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":515,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:18.847018+0000","last_change":"2024-05-30T17:14:18.847018+0000","last_active":"2024-05-30T17:14:18.847018+0000","last_peered":"2024-05-30T17:14:18.847018+0000","last_clean":"2024-05-30T17:14:18.847018+0000","last_became_active":"2024-05-30T15:54:20.669751+0000","last_became_peered":"2024-05-30T15:54:20.669751+0000","last_unstale":"2024-05-30T17:14:18.847018+0000","last_undegraded":"2024-05-30T17:14:18.847018+0000","last_fullsized":"2024-05-30T17:14:18.847018+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:18.846755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:18.846755+0000","last_clean_scrub_stamp":"2024-05-30T17:14:18.846755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:48:09.585537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"265'225","reported_seq":846,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.220776+0000","last_change":"2024-05-30T17:13:59.220776+0000","last_active":"2024-05-30T17:13:59.220776+0000","last_peered":"2024-05-30T17:13:59.220776+0000","last_clean":"2024-05-30T17:13:59.220776+0000","last_became_active":"2024-05-30T15:54:22.672078+0000","last_became_peered":"2024-05-30T15:54:22.672078+0000","last_unstale":"2024-05-30T17:13:59.220776+0000","last_undegraded":"2024-05-30T17:13:59.220776+0000","last_fullsized":"2024-05-30T17:13:59.220776+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'225","last_scrub_stamp":"2024-05-30T17:13:59.220468+0000","last_deep_scrub":"265'225","last_deep_scrub_stamp":"2024-05-30T17:13:59.220468+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.220468+0000","objects_scrubbed":9,"log_size":25,"log_dups_size":200,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T05:11:49.982867+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10553,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":297,"num_read_kb":402,"num_write":186,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":513,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:05.864109+0000","last_change":"2024-05-30T17:14:05.864109+0000","last_active":"2024-05-30T17:14:05.864109+0000","last_peered":"2024-05-30T17:14:05.864109+0000","last_clean":"2024-05-30T17:14:05.864109+0000","last_became_active":"2024-05-30T15:54:24.675467+0000","last_became_peered":"2024-05-30T15:54:24.675467+0000","last_unstale":"2024-05-30T17:14:05.864109+0000","last_undegraded":"2024-05-30T17:14:05.864109+0000","last_fullsized":"2024-05-30T17:14:05.864109+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:05.863982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:05.863982+0000","last_clean_scrub_stamp":"2024-05-30T17:14:05.863982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:16:33.932497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"265'106","reported_seq":633,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.532471+0000","last_change":"2024-05-30T17:13:58.532471+0000","last_active":"2024-05-30T17:13:58.532471+0000","last_peered":"2024-05-30T17:13:58.532471+0000","last_clean":"2024-05-30T17:13:58.532471+0000","last_became_active":"2024-05-30T15:54:26.705264+0000","last_became_peered":"2024-05-30T15:54:26.705264+0000","last_unstale":"2024-05-30T17:13:58.532471+0000","last_undegraded":"2024-05-30T17:13:58.532471+0000","last_fullsized":"2024-05-30T17:13:58.532471+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'106","last_scrub_stamp":"2024-05-30T17:13:58.532222+0000","last_deep_scrub":"265'106","last_deep_scrub_stamp":"2024-05-30T17:13:58.532222+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.532222+0000","objects_scrubbed":4,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:07:52.286487+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":805,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":151,"num_read_kb":135,"num_write":247,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"265'700","reported_seq":2179,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.780482+0000","last_change":"2024-05-30T17:13:45.780482+0000","last_active":"2024-05-30T17:13:45.780482+0000","last_peered":"2024-05-30T17:13:45.780482+0000","last_clean":"2024-05-30T17:13:45.780482+0000","last_became_active":"2024-05-30T15:54:54.421604+0000","last_became_peered":"2024-05-30T15:54:54.421604+0000","last_unstale":"2024-05-30T17:13:45.780482+0000","last_undegraded":"2024-05-30T17:13:45.780482+0000","last_fullsized":"2024-05-30T17:13:45.780482+0000","mapping_epoch":35,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'700","last_scrub_stamp":"2024-05-30T17:13:45.780204+0000","last_deep_scrub":"265'700","last_deep_scrub_stamp":"2024-05-30T17:13:45.780204+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.780204+0000","objects_scrubbed":2,"log_size":100,"log_dups_size":600,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:24:04.304931+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3204,"num_read_kb":2893,"num_write":920,"num_write_kb":324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"265'173","reported_seq":824,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.218809+0000","last_change":"2024-05-30T17:13:46.218809+0000","last_active":"2024-05-30T17:13:46.218809+0000","last_peered":"2024-05-30T17:13:46.218809+0000","last_clean":"2024-05-30T17:13:46.218809+0000","last_became_active":"2024-05-30T15:54:54.422916+0000","last_became_peered":"2024-05-30T15:54:54.422916+0000","last_unstale":"2024-05-30T17:13:46.218809+0000","last_undegraded":"2024-05-30T17:13:46.218809+0000","last_fullsized":"2024-05-30T17:13:46.218809+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'173","last_scrub_stamp":"2024-05-30T17:13:46.218474+0000","last_deep_scrub":"265'173","last_deep_scrub_stamp":"2024-05-30T17:13:46.218474+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.218474+0000","objects_scrubbed":4,"log_size":73,"log_dups_size":100,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:17:58.546167+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505146,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":405,"num_read_kb":71111,"num_write":971,"num_write_kb":25767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"265'5","reported_seq":842,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:08.314165+0000","last_change":"2024-05-30T17:14:08.314165+0000","last_active":"2024-05-30T17:14:08.314165+0000","last_peered":"2024-05-30T17:14:08.314165+0000","last_clean":"2024-05-30T17:14:08.314165+0000","last_became_active":"2024-05-30T15:54:20.652134+0000","last_became_peered":"2024-05-30T15:54:20.652134+0000","last_unstale":"2024-05-30T17:14:08.314165+0000","last_undegraded":"2024-05-30T17:14:08.314165+0000","last_fullsized":"2024-05-30T17:14:08.314165+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'5","last_scrub_stamp":"2024-05-30T17:14:08.313909+0000","last_deep_scrub":"265'5","last_deep_scrub_stamp":"2024-05-30T17:14:08.313909+0000","last_clean_scrub_stamp":"2024-05-30T17:14:08.313909+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:56:03.138686+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1139,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":493,"num_read_kb":807,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"265'151","reported_seq":746,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.647594+0000","last_change":"2024-05-30T15:54:22.673262+0000","last_active":"2024-05-30T17:13:41.647594+0000","last_peered":"2024-05-30T17:13:41.647594+0000","last_clean":"2024-05-30T17:13:41.647594+0000","last_became_active":"2024-05-30T15:54:22.672169+0000","last_became_peered":"2024-05-30T15:54:22.672169+0000","last_unstale":"2024-05-30T17:13:41.647594+0000","last_undegraded":"2024-05-30T17:13:41.647594+0000","last_fullsized":"2024-05-30T17:13:41.647594+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6731,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":192,"num_read_kb":263,"num_write":102,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":506,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:52.740783+0000","last_change":"2024-05-30T17:13:52.740908+0000","last_active":"2024-05-30T17:13:52.740783+0000","last_peered":"2024-05-30T17:13:52.740783+0000","last_clean":"2024-05-30T17:13:52.740783+0000","last_became_active":"2024-05-30T15:54:24.675683+0000","last_became_peered":"2024-05-30T15:54:24.675683+0000","last_unstale":"2024-05-30T17:13:52.740783+0000","last_undegraded":"2024-05-30T17:13:52.740783+0000","last_fullsized":"2024-05-30T17:13:52.740783+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:52.740555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:52.740555+0000","last_clean_scrub_stamp":"2024-05-30T17:13:52.740555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:52:04.474974+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"265'128","reported_seq":666,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:01.346031+0000","last_change":"2024-05-30T17:14:01.346031+0000","last_active":"2024-05-30T17:14:01.346031+0000","last_peered":"2024-05-30T17:14:01.346031+0000","last_clean":"2024-05-30T17:14:01.346031+0000","last_became_active":"2024-05-30T15:54:26.689049+0000","last_became_peered":"2024-05-30T15:54:26.689049+0000","last_unstale":"2024-05-30T17:14:01.346031+0000","last_undegraded":"2024-05-30T17:14:01.346031+0000","last_fullsized":"2024-05-30T17:14:01.346031+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'128","last_scrub_stamp":"2024-05-30T17:14:01.345770+0000","last_deep_scrub":"265'128","last_deep_scrub_stamp":"2024-05-30T17:14:01.345770+0000","last_clean_scrub_stamp":"2024-05-30T17:14:01.345770+0000","objects_scrubbed":1,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:20:24.034017+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":539,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":167,"num_read_kb":166,"num_write":309,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"265'654","reported_seq":2080,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:20.410526+0000","last_change":"2024-05-30T17:14:20.410597+0000","last_active":"2024-05-30T17:14:20.410526+0000","last_peered":"2024-05-30T17:14:20.410526+0000","last_clean":"2024-05-30T17:14:20.410526+0000","last_became_active":"2024-05-30T15:54:18.651778+0000","last_became_peered":"2024-05-30T15:54:18.651778+0000","last_unstale":"2024-05-30T17:14:20.410526+0000","last_undegraded":"2024-05-30T17:14:20.410526+0000","last_fullsized":"2024-05-30T17:14:20.410526+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'654","last_scrub_stamp":"2024-05-30T17:14:20.410335+0000","last_deep_scrub":"265'654","last_deep_scrub_stamp":"2024-05-30T17:14:20.410335+0000","last_clean_scrub_stamp":"2024-05-30T17:14:20.410335+0000","objects_scrubbed":2,"log_size":54,"log_dups_size":600,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:59:38.502836+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3068,"num_read_kb":2749,"num_write":930,"num_write_kb":367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"265'151","reported_seq":766,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:36.605206+0000","last_change":"2024-05-30T15:54:16.656872+0000","last_active":"2024-05-30T17:12:36.605206+0000","last_peered":"2024-05-30T17:12:36.605206+0000","last_clean":"2024-05-30T17:12:36.605206+0000","last_became_active":"2024-05-30T15:54:16.656081+0000","last_became_peered":"2024-05-30T15:54:16.656081+0000","last_unstale":"2024-05-30T17:12:36.605206+0000","last_undegraded":"2024-05-30T17:12:36.605206+0000","last_fullsized":"2024-05-30T17:12:36.605206+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":369,"num_read_kb":8201,"num_write":842,"num_write_kb":22196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":525,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:11.063361+0000","last_change":"2024-05-30T17:14:11.063466+0000","last_active":"2024-05-30T17:14:11.063361+0000","last_peered":"2024-05-30T17:14:11.063361+0000","last_clean":"2024-05-30T17:14:11.063361+0000","last_became_active":"2024-05-30T15:54:20.661090+0000","last_became_peered":"2024-05-30T15:54:20.661090+0000","last_unstale":"2024-05-30T17:14:11.063361+0000","last_undegraded":"2024-05-30T17:14:11.063361+0000","last_fullsized":"2024-05-30T17:14:11.063361+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:11.063113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:11.063113+0000","last_clean_scrub_stamp":"2024-05-30T17:14:11.063113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:16:18.209002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"265'116","reported_seq":689,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.119113+0000","last_change":"2024-05-30T15:54:22.670767+0000","last_active":"2024-05-30T17:13:38.119113+0000","last_peered":"2024-05-30T17:13:38.119113+0000","last_clean":"2024-05-30T17:13:38.119113+0000","last_became_active":"2024-05-30T15:54:22.669073+0000","last_became_peered":"2024-05-30T15:54:22.669073+0000","last_unstale":"2024-05-30T17:13:38.119113+0000","last_undegraded":"2024-05-30T17:13:38.119113+0000","last_fullsized":"2024-05-30T17:13:38.119113+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:21.628423+0000","last_clean_scrub_stamp":"2024-05-30T15:54:21.628423+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":100,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6915,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":145,"num_read_kb":188,"num_write":74,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":515,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.881973+0000","last_change":"2024-05-30T17:13:53.882025+0000","last_active":"2024-05-30T17:13:53.881973+0000","last_peered":"2024-05-30T17:13:53.881973+0000","last_clean":"2024-05-30T17:13:53.881973+0000","last_became_active":"2024-05-30T15:54:24.678292+0000","last_became_peered":"2024-05-30T15:54:24.678292+0000","last_unstale":"2024-05-30T17:13:53.881973+0000","last_undegraded":"2024-05-30T17:13:53.881973+0000","last_fullsized":"2024-05-30T17:13:53.881973+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:53.881772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:53.881772+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.881772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:31:21.481218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"265'171","reported_seq":918,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.566327+0000","last_change":"2024-05-30T17:13:59.566327+0000","last_active":"2024-05-30T17:13:59.566327+0000","last_peered":"2024-05-30T17:13:59.566327+0000","last_clean":"2024-05-30T17:13:59.566327+0000","last_became_active":"2024-05-30T15:54:26.703440+0000","last_became_peered":"2024-05-30T15:54:26.703440+0000","last_unstale":"2024-05-30T17:13:59.566327+0000","last_undegraded":"2024-05-30T17:13:59.566327+0000","last_fullsized":"2024-05-30T17:13:59.566327+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'171","last_scrub_stamp":"2024-05-30T17:13:59.565988+0000","last_deep_scrub":"265'171","last_deep_scrub_stamp":"2024-05-30T17:13:59.565988+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.565988+0000","objects_scrubbed":7,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:51:56.619593+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1074,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":420,"num_read_kb":450,"num_write":403,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1200,"num_omap_keys":6,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"265'333","reported_seq":1697,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:02.992132+0000","last_change":"2024-05-30T17:14:02.992132+0000","last_active":"2024-05-30T17:14:02.992132+0000","last_peered":"2024-05-30T17:14:02.992132+0000","last_clean":"2024-05-30T17:14:02.992132+0000","last_became_active":"2024-05-30T15:54:18.671650+0000","last_became_peered":"2024-05-30T15:54:18.671650+0000","last_unstale":"2024-05-30T17:14:02.992132+0000","last_undegraded":"2024-05-30T17:14:02.992132+0000","last_fullsized":"2024-05-30T17:14:02.992132+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'333","last_scrub_stamp":"2024-05-30T17:14:02.991716+0000","last_deep_scrub":"265'333","last_deep_scrub_stamp":"2024-05-30T17:14:02.991716+0000","last_clean_scrub_stamp":"2024-05-30T17:14:02.991716+0000","objects_scrubbed":6,"log_size":33,"log_dups_size":300,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:01:06.796193+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1672,"num_read_kb":1605,"num_write":361,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"265'206","reported_seq":868,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.350185+0000","last_change":"2024-05-30T15:54:16.661703+0000","last_active":"2024-05-30T17:13:38.350185+0000","last_peered":"2024-05-30T17:13:38.350185+0000","last_clean":"2024-05-30T17:13:38.350185+0000","last_became_active":"2024-05-30T15:54:16.660393+0000","last_became_peered":"2024-05-30T15:54:16.660393+0000","last_unstale":"2024-05-30T17:13:38.350185+0000","last_undegraded":"2024-05-30T17:13:38.350185+0000","last_fullsized":"2024-05-30T17:13:38.350185+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":100,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":594502,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":519,"num_read_kb":9177,"num_write":1141,"num_write_kb":26248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"265'2","reported_seq":519,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.016076+0000","last_change":"2024-05-30T17:13:58.016076+0000","last_active":"2024-05-30T17:13:58.016076+0000","last_peered":"2024-05-30T17:13:58.016076+0000","last_clean":"2024-05-30T17:13:58.016076+0000","last_became_active":"2024-05-30T15:54:20.661311+0000","last_became_peered":"2024-05-30T15:54:20.661311+0000","last_unstale":"2024-05-30T17:13:58.016076+0000","last_undegraded":"2024-05-30T17:13:58.016076+0000","last_fullsized":"2024-05-30T17:13:58.016076+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:13:58.015783+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:13:58.015783+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.015783+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:05:29.698239+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"265'180","reported_seq":800,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:15.065144+0000","last_change":"2024-05-30T17:14:15.065220+0000","last_active":"2024-05-30T17:14:15.065144+0000","last_peered":"2024-05-30T17:14:15.065144+0000","last_clean":"2024-05-30T17:14:15.065144+0000","last_became_active":"2024-05-30T15:54:22.663231+0000","last_became_peered":"2024-05-30T15:54:22.663231+0000","last_unstale":"2024-05-30T17:14:15.065144+0000","last_undegraded":"2024-05-30T17:14:15.065144+0000","last_fullsized":"2024-05-30T17:14:15.065144+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'180","last_scrub_stamp":"2024-05-30T17:14:15.064955+0000","last_deep_scrub":"265'180","last_deep_scrub_stamp":"2024-05-30T17:14:15.064955+0000","last_clean_scrub_stamp":"2024-05-30T17:14:15.064955+0000","objects_scrubbed":5,"log_size":80,"log_dups_size":100,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:54:03.215677+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4459,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":234,"num_read_kb":297,"num_write":134,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":511,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:04.853511+0000","last_change":"2024-05-30T17:14:04.853635+0000","last_active":"2024-05-30T17:14:04.853511+0000","last_peered":"2024-05-30T17:14:04.853511+0000","last_clean":"2024-05-30T17:14:04.853511+0000","last_became_active":"2024-05-30T15:54:24.673674+0000","last_became_peered":"2024-05-30T15:54:24.673674+0000","last_unstale":"2024-05-30T17:14:04.853511+0000","last_undegraded":"2024-05-30T17:14:04.853511+0000","last_fullsized":"2024-05-30T17:14:04.853511+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:04.853272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:04.853272+0000","last_clean_scrub_stamp":"2024-05-30T17:14:04.853272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:21:26.208754+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"265'114","reported_seq":629,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.822855+0000","last_change":"2024-05-30T17:13:48.822855+0000","last_active":"2024-05-30T17:13:48.822855+0000","last_peered":"2024-05-30T17:13:48.822855+0000","last_clean":"2024-05-30T17:13:48.822855+0000","last_became_active":"2024-05-30T15:54:26.677418+0000","last_became_peered":"2024-05-30T15:54:26.677418+0000","last_unstale":"2024-05-30T17:13:48.822855+0000","last_undegraded":"2024-05-30T17:13:48.822855+0000","last_fullsized":"2024-05-30T17:13:48.822855+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'114","last_scrub_stamp":"2024-05-30T17:13:48.822470+0000","last_deep_scrub":"265'114","last_deep_scrub_stamp":"2024-05-30T17:13:48.822470+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.822470+0000","objects_scrubbed":3,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:49:43.113999+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":435,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":67,"num_write":244,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"265'308","reported_seq":1720,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.056309+0000","last_change":"2024-05-30T17:13:59.056454+0000","last_active":"2024-05-30T17:13:59.056309+0000","last_peered":"2024-05-30T17:13:59.056309+0000","last_clean":"2024-05-30T17:13:59.056309+0000","last_became_active":"2024-05-30T15:54:18.671335+0000","last_became_peered":"2024-05-30T15:54:18.671335+0000","last_unstale":"2024-05-30T17:13:59.056309+0000","last_undegraded":"2024-05-30T17:13:59.056309+0000","last_fullsized":"2024-05-30T17:13:59.056309+0000","mapping_epoch":24,"log_start":"265'200","ondisk_log_start":"265'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'308","last_scrub_stamp":"2024-05-30T17:13:59.056066+0000","last_deep_scrub":"265'308","last_deep_scrub_stamp":"2024-05-30T17:13:59.056066+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.056066+0000","objects_scrubbed":3,"log_size":108,"log_dups_size":200,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:17:28.319253+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1771,"num_read_kb":1705,"num_write":340,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"265'206","reported_seq":867,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:09.773002+0000","last_change":"2024-05-30T17:14:09.773202+0000","last_active":"2024-05-30T17:14:09.773002+0000","last_peered":"2024-05-30T17:14:09.773002+0000","last_clean":"2024-05-30T17:14:09.773002+0000","last_became_active":"2024-05-30T15:54:16.674678+0000","last_became_peered":"2024-05-30T15:54:16.674678+0000","last_unstale":"2024-05-30T17:14:09.773002+0000","last_undegraded":"2024-05-30T17:14:09.773002+0000","last_fullsized":"2024-05-30T17:14:09.773002+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'206","last_scrub_stamp":"2024-05-30T17:14:09.772714+0000","last_deep_scrub":"265'206","last_deep_scrub_stamp":"2024-05-30T17:14:09.772714+0000","last_clean_scrub_stamp":"2024-05-30T17:14:09.772714+0000","objects_scrubbed":6,"log_size":106,"log_dups_size":100,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:12:22.788923+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25713,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":493,"num_read_kb":10347,"num_write":1036,"num_write_kb":23027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"265'562","reported_seq":2287,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:18.965625+0000","last_change":"2024-05-30T17:14:18.965782+0000","last_active":"2024-05-30T17:14:18.965625+0000","last_peered":"2024-05-30T17:14:18.965625+0000","last_clean":"2024-05-30T17:14:18.965625+0000","last_became_active":"2024-05-30T15:54:18.635092+0000","last_became_peered":"2024-05-30T15:54:18.635092+0000","last_unstale":"2024-05-30T17:14:18.965625+0000","last_undegraded":"2024-05-30T17:14:18.965625+0000","last_fullsized":"2024-05-30T17:14:18.965625+0000","mapping_epoch":24,"log_start":"265'500","ondisk_log_start":"265'500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'562","last_scrub_stamp":"2024-05-30T17:14:18.965373+0000","last_deep_scrub":"265'562","last_deep_scrub_stamp":"2024-05-30T17:14:18.965373+0000","last_clean_scrub_stamp":"2024-05-30T17:14:18.965373+0000","objects_scrubbed":3,"log_size":62,"log_dups_size":500,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:13:33.470676+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2950,"num_read_kb":2739,"num_write":700,"num_write_kb":188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1511,"num_omap_keys":6,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"265'165","reported_seq":788,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:01.304037+0000","last_change":"2024-05-30T17:14:01.304121+0000","last_active":"2024-05-30T17:14:01.304037+0000","last_peered":"2024-05-30T17:14:01.304037+0000","last_clean":"2024-05-30T17:14:01.304037+0000","last_became_active":"2024-05-30T15:54:16.658183+0000","last_became_peered":"2024-05-30T15:54:16.658183+0000","last_unstale":"2024-05-30T17:14:01.304037+0000","last_undegraded":"2024-05-30T17:14:01.304037+0000","last_fullsized":"2024-05-30T17:14:01.304037+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'165","last_scrub_stamp":"2024-05-30T17:14:01.303807+0000","last_deep_scrub":"265'165","last_deep_scrub_stamp":"2024-05-30T17:14:01.303807+0000","last_clean_scrub_stamp":"2024-05-30T17:14:01.303807+0000","objects_scrubbed":1,"log_size":65,"log_dups_size":100,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:51:27.141500+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":308,"num_read_kb":6656,"num_write":926,"num_write_kb":27882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"265'629","reported_seq":2194,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:02.302121+0000","last_change":"2024-05-30T17:14:02.302193+0000","last_active":"2024-05-30T17:14:02.302121+0000","last_peered":"2024-05-30T17:14:02.302121+0000","last_clean":"2024-05-30T17:14:02.302121+0000","last_became_active":"2024-05-30T15:54:18.662159+0000","last_became_peered":"2024-05-30T15:54:18.662159+0000","last_unstale":"2024-05-30T17:14:02.302121+0000","last_undegraded":"2024-05-30T17:14:02.302121+0000","last_fullsized":"2024-05-30T17:14:02.302121+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'629","last_scrub_stamp":"2024-05-30T17:14:02.301919+0000","last_deep_scrub":"265'629","last_deep_scrub_stamp":"2024-05-30T17:14:02.301919+0000","last_clean_scrub_stamp":"2024-05-30T17:14:02.301919+0000","objects_scrubbed":4,"log_size":29,"log_dups_size":600,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:46:32.528521+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3191,"num_read_kb":2926,"num_write":899,"num_write_kb":342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":249,"num_omap_keys":1,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"265'210","reported_seq":872,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:17.310246+0000","last_change":"2024-05-30T17:14:17.310357+0000","last_active":"2024-05-30T17:14:17.310246+0000","last_peered":"2024-05-30T17:14:17.310246+0000","last_clean":"2024-05-30T17:14:17.310246+0000","last_became_active":"2024-05-30T15:54:16.652664+0000","last_became_peered":"2024-05-30T15:54:16.652664+0000","last_unstale":"2024-05-30T17:14:17.310246+0000","last_undegraded":"2024-05-30T17:14:17.310246+0000","last_fullsized":"2024-05-30T17:14:17.310246+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'210","last_scrub_stamp":"2024-05-30T17:14:17.310051+0000","last_deep_scrub":"265'210","last_deep_scrub_stamp":"2024-05-30T17:14:17.310051+0000","last_clean_scrub_stamp":"2024-05-30T17:14:17.310051+0000","objects_scrubbed":6,"log_size":110,"log_dups_size":100,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:39:16.444559+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524420,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":451,"num_read_kb":27404,"num_write":1156,"num_write_kb":29020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"265'1306","reported_seq":2950,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:09.025093+0000","last_change":"2024-05-30T17:14:09.025175+0000","last_active":"2024-05-30T17:14:09.025093+0000","last_peered":"2024-05-30T17:14:09.025093+0000","last_clean":"2024-05-30T17:14:09.025093+0000","last_became_active":"2024-05-30T15:54:18.642697+0000","last_became_peered":"2024-05-30T15:54:18.642697+0000","last_unstale":"2024-05-30T17:14:09.025093+0000","last_undegraded":"2024-05-30T17:14:09.025093+0000","last_fullsized":"2024-05-30T17:14:09.025093+0000","mapping_epoch":24,"log_start":"265'1200","ondisk_log_start":"265'1200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'1306","last_scrub_stamp":"2024-05-30T17:14:09.024879+0000","last_deep_scrub":"265'1306","last_deep_scrub_stamp":"2024-05-30T17:14:09.024879+0000","last_clean_scrub_stamp":"2024-05-30T17:14:09.024879+0000","objects_scrubbed":9,"log_size":106,"log_dups_size":1200,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:00:44.114885+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":5484,"num_read_kb":4752,"num_write":1832,"num_write_kb":807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1870,"num_omap_keys":8,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"265'256","reported_seq":990,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.646944+0000","last_change":"2024-05-30T15:54:16.660154+0000","last_active":"2024-05-30T17:13:41.646944+0000","last_peered":"2024-05-30T17:13:41.646944+0000","last_clean":"2024-05-30T17:13:41.646944+0000","last_became_active":"2024-05-30T15:54:16.659589+0000","last_became_peered":"2024-05-30T15:54:16.659589+0000","last_unstale":"2024-05-30T17:13:41.646944+0000","last_undegraded":"2024-05-30T17:13:41.646944+0000","last_fullsized":"2024-05-30T17:13:41.646944+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":200,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3222795,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":793,"num_read_kb":47588,"num_write":1564,"num_write_kb":36318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"265'1101","reported_seq":2682,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.074235+0000","last_change":"2024-05-30T15:54:18.653159+0000","last_active":"2024-05-30T17:13:43.074235+0000","last_peered":"2024-05-30T17:13:43.074235+0000","last_clean":"2024-05-30T17:13:43.074235+0000","last_became_active":"2024-05-30T15:54:18.652794+0000","last_became_peered":"2024-05-30T15:54:18.652794+0000","last_unstale":"2024-05-30T17:13:43.074235+0000","last_undegraded":"2024-05-30T17:13:43.074235+0000","last_fullsized":"2024-05-30T17:13:43.074235+0000","mapping_epoch":24,"log_start":"265'1000","ondisk_log_start":"265'1000","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":1000,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4733,"num_read_kb":4179,"num_write":1497,"num_write_kb":596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"265'225","reported_seq":902,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:52.361991+0000","last_change":"2024-05-30T17:13:52.362088+0000","last_active":"2024-05-30T17:13:52.361991+0000","last_peered":"2024-05-30T17:13:52.361991+0000","last_clean":"2024-05-30T17:13:52.361991+0000","last_became_active":"2024-05-30T15:54:54.451265+0000","last_became_peered":"2024-05-30T15:54:54.451265+0000","last_unstale":"2024-05-30T17:13:52.361991+0000","last_undegraded":"2024-05-30T17:13:52.361991+0000","last_fullsized":"2024-05-30T17:13:52.361991+0000","mapping_epoch":35,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'225","last_scrub_stamp":"2024-05-30T17:13:52.361791+0000","last_deep_scrub":"265'225","last_deep_scrub_stamp":"2024-05-30T17:13:52.361791+0000","last_clean_scrub_stamp":"2024-05-30T17:13:52.361791+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":200,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:41:32.517363+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524357,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":480,"num_read_kb":2675,"num_write":1165,"num_write_kb":43073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"265'389","reported_seq":1961,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:04.004662+0000","last_change":"2024-05-30T17:14:04.004748+0000","last_active":"2024-05-30T17:14:04.004662+0000","last_peered":"2024-05-30T17:14:04.004662+0000","last_clean":"2024-05-30T17:14:04.004662+0000","last_became_active":"2024-05-30T15:54:18.670801+0000","last_became_peered":"2024-05-30T15:54:18.670801+0000","last_unstale":"2024-05-30T17:14:04.004662+0000","last_undegraded":"2024-05-30T17:14:04.004662+0000","last_fullsized":"2024-05-30T17:14:04.004662+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'389","last_scrub_stamp":"2024-05-30T17:14:04.004452+0000","last_deep_scrub":"265'389","last_deep_scrub_stamp":"2024-05-30T17:14:04.004452+0000","last_clean_scrub_stamp":"2024-05-30T17:14:04.004452+0000","objects_scrubbed":4,"log_size":89,"log_dups_size":300,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:27:27.990698+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2215,"num_read_kb":2115,"num_write":443,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2553,"num_omap_keys":10,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"265'222","reported_seq":882,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:14.943886+0000","last_change":"2024-05-30T17:14:14.943979+0000","last_active":"2024-05-30T17:14:14.943886+0000","last_peered":"2024-05-30T17:14:14.943886+0000","last_clean":"2024-05-30T17:14:14.943886+0000","last_became_active":"2024-05-30T15:54:16.649854+0000","last_became_peered":"2024-05-30T15:54:16.649854+0000","last_unstale":"2024-05-30T17:14:14.943886+0000","last_undegraded":"2024-05-30T17:14:14.943886+0000","last_fullsized":"2024-05-30T17:14:14.943886+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'222","last_scrub_stamp":"2024-05-30T17:14:14.943688+0000","last_deep_scrub":"265'222","last_deep_scrub_stamp":"2024-05-30T17:14:14.943688+0000","last_clean_scrub_stamp":"2024-05-30T17:14:14.943688+0000","objects_scrubbed":2,"log_size":22,"log_dups_size":200,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:30:00.639949+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306443,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":499,"num_read_kb":6249,"num_write":1151,"num_write_kb":30962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"265'405","reported_seq":1926,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:31.616767+0000","last_change":"2024-05-30T15:54:18.653103+0000","last_active":"2024-05-30T17:13:31.616767+0000","last_peered":"2024-05-30T17:13:31.616767+0000","last_clean":"2024-05-30T17:13:31.616767+0000","last_became_active":"2024-05-30T15:54:18.650832+0000","last_became_peered":"2024-05-30T15:54:18.650832+0000","last_unstale":"2024-05-30T17:13:31.616767+0000","last_undegraded":"2024-05-30T17:13:31.616767+0000","last_fullsized":"2024-05-30T17:13:31.616767+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:17.613765+0000","last_clean_scrub_stamp":"2024-05-30T15:54:17.613765+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":300,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2240,"num_read_kb":2125,"num_write":507,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"265'187","reported_seq":820,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:10.743922+0000","last_change":"2024-05-30T17:14:10.744038+0000","last_active":"2024-05-30T17:14:10.743922+0000","last_peered":"2024-05-30T17:14:10.743922+0000","last_clean":"2024-05-30T17:14:10.743922+0000","last_became_active":"2024-05-30T15:54:16.684670+0000","last_became_peered":"2024-05-30T15:54:16.684670+0000","last_unstale":"2024-05-30T17:14:10.743922+0000","last_undegraded":"2024-05-30T17:14:10.743922+0000","last_fullsized":"2024-05-30T17:14:10.743922+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'187","last_scrub_stamp":"2024-05-30T17:14:10.743692+0000","last_deep_scrub":"265'187","last_deep_scrub_stamp":"2024-05-30T17:14:10.743692+0000","last_clean_scrub_stamp":"2024-05-30T17:14:10.743692+0000","objects_scrubbed":7,"log_size":87,"log_dups_size":100,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:32:19.992966+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":254692,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":373,"num_read_kb":34719,"num_write":1090,"num_write_kb":29777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"265'823","reported_seq":2424,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:03.753084+0000","last_change":"2024-05-30T17:14:03.753161+0000","last_active":"2024-05-30T17:14:03.753084+0000","last_peered":"2024-05-30T17:14:03.753084+0000","last_clean":"2024-05-30T17:14:03.753084+0000","last_became_active":"2024-05-30T15:54:18.669105+0000","last_became_peered":"2024-05-30T15:54:18.669105+0000","last_unstale":"2024-05-30T17:14:03.753084+0000","last_undegraded":"2024-05-30T17:14:03.753084+0000","last_fullsized":"2024-05-30T17:14:03.753084+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'823","last_scrub_stamp":"2024-05-30T17:14:03.752908+0000","last_deep_scrub":"265'823","last_deep_scrub_stamp":"2024-05-30T17:14:03.752908+0000","last_clean_scrub_stamp":"2024-05-30T17:14:03.752908+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":800,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:05:28.270822+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3660,"num_read_kb":3297,"num_write":1114,"num_write_kb":434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":252,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"265'220","reported_seq":927,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.975897+0000","last_change":"2024-05-30T17:13:48.976145+0000","last_active":"2024-05-30T17:13:48.975897+0000","last_peered":"2024-05-30T17:13:48.975897+0000","last_clean":"2024-05-30T17:13:48.975897+0000","last_became_active":"2024-05-30T15:54:54.416482+0000","last_became_peered":"2024-05-30T15:54:54.416482+0000","last_unstale":"2024-05-30T17:13:48.975897+0000","last_undegraded":"2024-05-30T17:13:48.975897+0000","last_fullsized":"2024-05-30T17:13:48.975897+0000","mapping_epoch":35,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'220","last_scrub_stamp":"2024-05-30T17:13:48.975457+0000","last_deep_scrub":"265'220","last_deep_scrub_stamp":"2024-05-30T17:13:48.975457+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.975457+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":200,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:44:46.059246+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5708449,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":729,"num_read_kb":137610,"num_write":1205,"num_write_kb":64841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2f","version":"265'389","reported_seq":1986,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:10.889831+0000","last_change":"2024-05-30T17:14:10.890006+0000","last_active":"2024-05-30T17:14:10.889831+0000","last_peered":"2024-05-30T17:14:10.889831+0000","last_clean":"2024-05-30T17:14:10.889831+0000","last_became_active":"2024-05-30T15:54:18.657236+0000","last_became_peered":"2024-05-30T15:54:18.657236+0000","last_unstale":"2024-05-30T17:14:10.889831+0000","last_undegraded":"2024-05-30T17:14:10.889831+0000","last_fullsized":"2024-05-30T17:14:10.889831+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'389","last_scrub_stamp":"2024-05-30T17:14:10.889524+0000","last_deep_scrub":"265'389","last_deep_scrub_stamp":"2024-05-30T17:14:10.889524+0000","last_clean_scrub_stamp":"2024-05-30T17:14:10.889524+0000","objects_scrubbed":4,"log_size":89,"log_dups_size":300,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:08:38.176469+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2258,"num_read_kb":2156,"num_write":442,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2808,"num_omap_keys":11,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"265'203","reported_seq":883,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.051695+0000","last_change":"2024-05-30T15:54:16.661242+0000","last_active":"2024-05-30T17:13:38.051695+0000","last_peered":"2024-05-30T17:13:38.051695+0000","last_clean":"2024-05-30T17:13:38.051695+0000","last_became_active":"2024-05-30T15:54:16.653037+0000","last_became_peered":"2024-05-30T15:54:16.653037+0000","last_unstale":"2024-05-30T17:13:38.051695+0000","last_undegraded":"2024-05-30T17:13:38.051695+0000","last_fullsized":"2024-05-30T17:13:38.051695+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":100,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2108965,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":543,"num_read_kb":29183,"num_write":977,"num_write_kb":32330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"265'334","reported_seq":1841,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:21.115099+0000","last_change":"2024-05-30T17:14:21.115173+0000","last_active":"2024-05-30T17:14:21.115099+0000","last_peered":"2024-05-30T17:14:21.115099+0000","last_clean":"2024-05-30T17:14:21.115099+0000","last_became_active":"2024-05-30T15:54:18.635382+0000","last_became_peered":"2024-05-30T15:54:18.635382+0000","last_unstale":"2024-05-30T17:14:21.115099+0000","last_undegraded":"2024-05-30T17:14:21.115099+0000","last_fullsized":"2024-05-30T17:14:21.115099+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'334","last_scrub_stamp":"2024-05-30T17:14:21.114886+0000","last_deep_scrub":"265'334","last_deep_scrub_stamp":"2024-05-30T17:14:21.114886+0000","last_clean_scrub_stamp":"2024-05-30T17:14:21.114886+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:17:15.854111+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2071,"num_read_kb":1981,"num_write":382,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2453,"num_omap_keys":10,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"265'159","reported_seq":777,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.307596+0000","last_change":"2024-05-30T15:54:16.683072+0000","last_active":"2024-05-30T17:13:38.307596+0000","last_peered":"2024-05-30T17:13:38.307596+0000","last_clean":"2024-05-30T17:13:38.307596+0000","last_became_active":"2024-05-30T15:54:16.682629+0000","last_became_peered":"2024-05-30T15:54:16.682629+0000","last_unstale":"2024-05-30T17:13:38.307596+0000","last_undegraded":"2024-05-30T17:13:38.307596+0000","last_fullsized":"2024-05-30T17:13:38.307596+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310739,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":22431,"num_write":822,"num_write_kb":23734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"265'747","reported_seq":2325,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:06.362930+0000","last_change":"2024-05-30T17:14:06.362999+0000","last_active":"2024-05-30T17:14:06.362930+0000","last_peered":"2024-05-30T17:14:06.362930+0000","last_clean":"2024-05-30T17:14:06.362930+0000","last_became_active":"2024-05-30T15:54:18.668748+0000","last_became_peered":"2024-05-30T15:54:18.668748+0000","last_unstale":"2024-05-30T17:14:06.362930+0000","last_undegraded":"2024-05-30T17:14:06.362930+0000","last_fullsized":"2024-05-30T17:14:06.362930+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'747","last_scrub_stamp":"2024-05-30T17:14:06.362744+0000","last_deep_scrub":"265'747","last_deep_scrub_stamp":"2024-05-30T17:14:06.362744+0000","last_clean_scrub_stamp":"2024-05-30T17:14:06.362744+0000","objects_scrubbed":4,"log_size":47,"log_dups_size":700,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:14:16.405343+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3385,"num_read_kb":3066,"num_write":980,"num_write_kb":340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"265'165","reported_seq":909,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:13.340658+0000","last_change":"2024-05-30T17:14:13.340762+0000","last_active":"2024-05-30T17:14:13.340658+0000","last_peered":"2024-05-30T17:14:13.340658+0000","last_clean":"2024-05-30T17:14:13.340658+0000","last_became_active":"2024-05-30T15:54:16.674993+0000","last_became_peered":"2024-05-30T15:54:16.674993+0000","last_unstale":"2024-05-30T17:14:13.340658+0000","last_undegraded":"2024-05-30T17:14:13.340658+0000","last_fullsized":"2024-05-30T17:14:13.340658+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'165","last_scrub_stamp":"2024-05-30T17:14:13.340433+0000","last_deep_scrub":"265'165","last_deep_scrub_stamp":"2024-05-30T17:14:13.340433+0000","last_clean_scrub_stamp":"2024-05-30T17:14:13.340433+0000","objects_scrubbed":5,"log_size":65,"log_dups_size":100,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:25:29.735661+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6553619,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":567,"num_read_kb":112844,"num_write":925,"num_write_kb":45704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"265'472","reported_seq":2122,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:01.015512+0000","last_change":"2024-05-30T17:14:01.015577+0000","last_active":"2024-05-30T17:14:01.015512+0000","last_peered":"2024-05-30T17:14:01.015512+0000","last_clean":"2024-05-30T17:14:01.015512+0000","last_became_active":"2024-05-30T15:54:18.671434+0000","last_became_peered":"2024-05-30T15:54:18.671434+0000","last_unstale":"2024-05-30T17:14:01.015512+0000","last_undegraded":"2024-05-30T17:14:01.015512+0000","last_fullsized":"2024-05-30T17:14:01.015512+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'472","last_scrub_stamp":"2024-05-30T17:14:01.015328+0000","last_deep_scrub":"265'472","last_deep_scrub_stamp":"2024-05-30T17:14:01.015328+0000","last_clean_scrub_stamp":"2024-05-30T17:14:01.015328+0000","objects_scrubbed":1,"log_size":72,"log_dups_size":400,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:21:04.896268+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2525,"num_read_kb":2390,"num_write":596,"num_write_kb":183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"265'204","reported_seq":859,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.397498+0000","last_change":"2024-05-30T15:54:16.630459+0000","last_active":"2024-05-30T17:13:38.397498+0000","last_peered":"2024-05-30T17:13:38.397498+0000","last_clean":"2024-05-30T17:13:38.397498+0000","last_became_active":"2024-05-30T15:54:16.629843+0000","last_became_peered":"2024-05-30T15:54:16.629843+0000","last_unstale":"2024-05-30T17:13:38.397498+0000","last_undegraded":"2024-05-30T17:13:38.397498+0000","last_fullsized":"2024-05-30T17:13:38.397498+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":100,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524465,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":434,"num_read_kb":42078,"num_write":1152,"num_write_kb":26102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"265'516","reported_seq":2150,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:11.810946+0000","last_change":"2024-05-30T17:14:11.811023+0000","last_active":"2024-05-30T17:14:11.810946+0000","last_peered":"2024-05-30T17:14:11.810946+0000","last_clean":"2024-05-30T17:14:11.810946+0000","last_became_active":"2024-05-30T15:54:18.660101+0000","last_became_peered":"2024-05-30T15:54:18.660101+0000","last_unstale":"2024-05-30T17:14:11.810946+0000","last_undegraded":"2024-05-30T17:14:11.810946+0000","last_fullsized":"2024-05-30T17:14:11.810946+0000","mapping_epoch":24,"log_start":"265'500","ondisk_log_start":"265'500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'516","last_scrub_stamp":"2024-05-30T17:14:11.810772+0000","last_deep_scrub":"265'516","last_deep_scrub_stamp":"2024-05-30T17:14:11.810772+0000","last_clean_scrub_stamp":"2024-05-30T17:14:11.810772+0000","objects_scrubbed":7,"log_size":16,"log_dups_size":500,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:11:32.790447+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2760,"num_read_kb":2546,"num_write":661,"num_write_kb":201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2440,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"265'210","reported_seq":854,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:11.793065+0000","last_change":"2024-05-30T17:14:11.793335+0000","last_active":"2024-05-30T17:14:11.793065+0000","last_peered":"2024-05-30T17:14:11.793065+0000","last_clean":"2024-05-30T17:14:11.793065+0000","last_became_active":"2024-05-30T15:54:16.685969+0000","last_became_peered":"2024-05-30T15:54:16.685969+0000","last_unstale":"2024-05-30T17:14:11.793065+0000","last_undegraded":"2024-05-30T17:14:11.793065+0000","last_fullsized":"2024-05-30T17:14:11.793065+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'210","last_scrub_stamp":"2024-05-30T17:14:11.792805+0000","last_deep_scrub":"265'210","last_deep_scrub_stamp":"2024-05-30T17:14:11.792805+0000","last_clean_scrub_stamp":"2024-05-30T17:14:11.792805+0000","objects_scrubbed":5,"log_size":110,"log_dups_size":100,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:18:58.482461+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1342159,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":419,"num_read_kb":18996,"num_write":1148,"num_write_kb":34757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"265'671","reported_seq":2437,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:18.831077+0000","last_change":"2024-05-30T17:14:18.831145+0000","last_active":"2024-05-30T17:14:18.831077+0000","last_peered":"2024-05-30T17:14:18.831077+0000","last_clean":"2024-05-30T17:14:18.831077+0000","last_became_active":"2024-05-30T15:54:18.645400+0000","last_became_peered":"2024-05-30T15:54:18.645400+0000","last_unstale":"2024-05-30T17:14:18.831077+0000","last_undegraded":"2024-05-30T17:14:18.831077+0000","last_fullsized":"2024-05-30T17:14:18.831077+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'671","last_scrub_stamp":"2024-05-30T17:14:18.830880+0000","last_deep_scrub":"265'671","last_deep_scrub_stamp":"2024-05-30T17:14:18.830880+0000","last_clean_scrub_stamp":"2024-05-30T17:14:18.830880+0000","objects_scrubbed":1,"log_size":71,"log_dups_size":600,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:12:16.729926+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3447,"num_read_kb":3120,"num_write":968,"num_write_kb":434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"265'215","reported_seq":850,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.413515+0000","last_change":"2024-05-30T15:54:16.664235+0000","last_active":"2024-05-30T17:13:38.413515+0000","last_peered":"2024-05-30T17:13:38.413515+0000","last_clean":"2024-05-30T17:13:38.413515+0000","last_became_active":"2024-05-30T15:54:16.663255+0000","last_became_peered":"2024-05-30T15:54:16.663255+0000","last_unstale":"2024-05-30T17:13:38.413515+0000","last_undegraded":"2024-05-30T17:13:38.413515+0000","last_fullsized":"2024-05-30T17:13:38.413515+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":200,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767340,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":429,"num_read_kb":27123,"num_write":1161,"num_write_kb":19484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"265'365","reported_seq":1793,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:16.421825+0000","last_change":"2024-05-30T17:14:16.421904+0000","last_active":"2024-05-30T17:14:16.421825+0000","last_peered":"2024-05-30T17:14:16.421825+0000","last_clean":"2024-05-30T17:14:16.421825+0000","last_became_active":"2024-05-30T15:54:18.658230+0000","last_became_peered":"2024-05-30T15:54:18.658230+0000","last_unstale":"2024-05-30T17:14:16.421825+0000","last_undegraded":"2024-05-30T17:14:16.421825+0000","last_fullsized":"2024-05-30T17:14:16.421825+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'365","last_scrub_stamp":"2024-05-30T17:14:16.421610+0000","last_deep_scrub":"265'365","last_deep_scrub_stamp":"2024-05-30T17:14:16.421610+0000","last_clean_scrub_stamp":"2024-05-30T17:14:16.421610+0000","objects_scrubbed":5,"log_size":65,"log_dups_size":300,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:52:59.746921+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1934,"num_read_kb":1839,"num_write":414,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":472,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"265'170","reported_seq":792,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:00.278603+0000","last_change":"2024-05-30T17:14:00.278671+0000","last_active":"2024-05-30T17:14:00.278603+0000","last_peered":"2024-05-30T17:14:00.278603+0000","last_clean":"2024-05-30T17:14:00.278603+0000","last_became_active":"2024-05-30T15:54:16.654365+0000","last_became_peered":"2024-05-30T15:54:16.654365+0000","last_unstale":"2024-05-30T17:14:00.278603+0000","last_undegraded":"2024-05-30T17:14:00.278603+0000","last_fullsized":"2024-05-30T17:14:00.278603+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'170","last_scrub_stamp":"2024-05-30T17:14:00.278386+0000","last_deep_scrub":"265'170","last_deep_scrub_stamp":"2024-05-30T17:14:00.278386+0000","last_clean_scrub_stamp":"2024-05-30T17:14:00.278386+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":100,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:04:58.427288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":358,"num_read_kb":31154,"num_write":915,"num_write_kb":41557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"265'704","reported_seq":2143,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:10.773727+0000","last_change":"2024-05-30T17:14:10.773805+0000","last_active":"2024-05-30T17:14:10.773727+0000","last_peered":"2024-05-30T17:14:10.773727+0000","last_clean":"2024-05-30T17:14:10.773727+0000","last_became_active":"2024-05-30T15:54:18.660543+0000","last_became_peered":"2024-05-30T15:54:18.660543+0000","last_unstale":"2024-05-30T17:14:10.773727+0000","last_undegraded":"2024-05-30T17:14:10.773727+0000","last_fullsized":"2024-05-30T17:14:10.773727+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'704","last_scrub_stamp":"2024-05-30T17:14:10.773554+0000","last_deep_scrub":"265'704","last_deep_scrub_stamp":"2024-05-30T17:14:10.773554+0000","last_clean_scrub_stamp":"2024-05-30T17:14:10.773554+0000","objects_scrubbed":6,"log_size":104,"log_dups_size":600,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:36:12.916424+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3130,"num_read_kb":2812,"num_write":919,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"265'202","reported_seq":845,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:38.368900+0000","last_change":"2024-05-30T15:54:16.666386+0000","last_active":"2024-05-30T17:13:38.368900+0000","last_peered":"2024-05-30T17:13:38.368900+0000","last_clean":"2024-05-30T17:13:38.368900+0000","last_became_active":"2024-05-30T15:54:16.661983+0000","last_became_peered":"2024-05-30T15:54:16.661983+0000","last_unstale":"2024-05-30T17:13:38.368900+0000","last_undegraded":"2024-05-30T17:13:38.368900+0000","last_fullsized":"2024-05-30T17:13:38.368900+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T15:54:15.606343+0000","last_clean_scrub_stamp":"2024-05-30T15:54:15.606343+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718827,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":420,"num_read_kb":7620,"num_write":1155,"num_write_kb":9531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"265'154","reported_seq":793,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.164212+0000","last_change":"2024-05-30T17:13:45.164368+0000","last_active":"2024-05-30T17:13:45.164212+0000","last_peered":"2024-05-30T17:13:45.164212+0000","last_clean":"2024-05-30T17:13:45.164212+0000","last_became_active":"2024-05-30T15:55:54.767665+0000","last_became_peered":"2024-05-30T15:55:54.767665+0000","last_unstale":"2024-05-30T17:13:45.164212+0000","last_undegraded":"2024-05-30T17:13:45.164212+0000","last_fullsized":"2024-05-30T17:13:45.164212+0000","mapping_epoch":37,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'154","last_scrub_stamp":"2024-05-30T17:13:45.163967+0000","last_deep_scrub":"265'154","last_deep_scrub_stamp":"2024-05-30T17:13:45.163967+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.163967+0000","objects_scrubbed":2,"log_size":54,"log_dups_size":100,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:10:48.513574+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5682939,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":360,"num_read_kb":14674,"num_write":890,"num_write_kb":17371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,0],"acting":[5,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"265'771","reported_seq":2434,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:11.854663+0000","last_change":"2024-05-30T17:14:11.854742+0000","last_active":"2024-05-30T17:14:11.854663+0000","last_peered":"2024-05-30T17:14:11.854663+0000","last_clean":"2024-05-30T17:14:11.854663+0000","last_became_active":"2024-05-30T15:54:18.660501+0000","last_became_peered":"2024-05-30T15:54:18.660501+0000","last_unstale":"2024-05-30T17:14:11.854663+0000","last_undegraded":"2024-05-30T17:14:11.854663+0000","last_fullsized":"2024-05-30T17:14:11.854663+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'771","last_scrub_stamp":"2024-05-30T17:14:11.854443+0000","last_deep_scrub":"265'771","last_deep_scrub_stamp":"2024-05-30T17:14:11.854443+0000","last_clean_scrub_stamp":"2024-05-30T17:14:11.854443+0000","objects_scrubbed":6,"log_size":71,"log_dups_size":700,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:04:01.491788+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3837,"num_read_kb":3483,"num_write":1021,"num_write_kb":383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2811,"num_omap_keys":11,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"265'183","reported_seq":843,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:17.991730+0000","last_change":"2024-05-30T17:14:17.991836+0000","last_active":"2024-05-30T17:14:17.991730+0000","last_peered":"2024-05-30T17:14:17.991730+0000","last_clean":"2024-05-30T17:14:17.991730+0000","last_became_active":"2024-05-30T15:54:16.668178+0000","last_became_peered":"2024-05-30T15:54:16.668178+0000","last_unstale":"2024-05-30T17:14:17.991730+0000","last_undegraded":"2024-05-30T17:14:17.991730+0000","last_fullsized":"2024-05-30T17:14:17.991730+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'183","last_scrub_stamp":"2024-05-30T17:14:17.991533+0000","last_deep_scrub":"265'183","last_deep_scrub_stamp":"2024-05-30T17:14:17.991533+0000","last_clean_scrub_stamp":"2024-05-30T17:14:17.991533+0000","objects_scrubbed":6,"log_size":83,"log_dups_size":100,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:17:31.041725+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3976850,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":427,"num_read_kb":57498,"num_write":1000,"num_write_kb":21705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"265'477","reported_seq":2157,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:09.363466+0000","last_change":"2024-05-30T17:14:09.363603+0000","last_active":"2024-05-30T17:14:09.363466+0000","last_peered":"2024-05-30T17:14:09.363466+0000","last_clean":"2024-05-30T17:14:09.363466+0000","last_became_active":"2024-05-30T15:54:18.655872+0000","last_became_peered":"2024-05-30T15:54:18.655872+0000","last_unstale":"2024-05-30T17:14:09.363466+0000","last_undegraded":"2024-05-30T17:14:09.363466+0000","last_fullsized":"2024-05-30T17:14:09.363466+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'477","last_scrub_stamp":"2024-05-30T17:14:09.363234+0000","last_deep_scrub":"265'477","last_deep_scrub_stamp":"2024-05-30T17:14:09.363234+0000","last_clean_scrub_stamp":"2024-05-30T17:14:09.363234+0000","objects_scrubbed":3,"log_size":77,"log_dups_size":400,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:01:10.979098+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2571,"num_read_kb":2440,"num_write":587,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":497,"num_omap_keys":2,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5999,"num_read_kb":4309,"num_write":2135,"num_write_kb":880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6954,"num_omap_keys":14,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1142,"ondisk_log_size":1142,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19930,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":13226,"num_read_kb":13032,"num_write":15987,"num_write_kb":4690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4334,"num_omap_keys":26,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":704512,"data_stored":39860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1671,"ondisk_log_size":1671,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":331,"ondisk_log_size":331,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":279892,"num_objects":197,"num_object_clones":0,"num_object_copies":394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":197,"num_whiteouts":0,"num_read":9562,"num_read_kb":11753,"num_write":6685,"num_write_kb":2848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":548174,"num_omap_keys":1248,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1849,"ondisk_log_size":1849,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1634,"num_read_kb":1946,"num_write":20,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":242,"num_object_clones":0,"num_object_copies":484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":242,"num_whiteouts":0,"num_read":191181,"num_read_kb":174247,"num_write":51900,"num_write_kb":17923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":242,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":39650,"num_omap_keys":164,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4128,"ondisk_log_size":4128,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":376,"num_omap_keys":4,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":157354006,"num_objects":261,"num_object_clones":0,"num_object_copies":783,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":261,"num_whiteouts":0,"num_read":32072,"num_read_kb":2124808,"num_write":69910,"num_write_kb":2069422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":238202880,"data_stored":238202880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4424,"ondisk_log_size":4424,"up":192,"acting":192,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":135,"num_read_kb":182367,"num_write":151,"num_write_kb":122722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":133,"num_read_kb":181098,"num_write":151,"num_write_kb":122722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575819,"num_pgs":106,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300188,"kb_used_data":38548,"kb_used_omap":1,"kb_used_meta":261630,"kb_avail":94071652,"statfs":{"total":96636764160,"available":96329371648,"internally_reserved":0,"allocated":39473152,"data_stored":37452492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":267909578},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575818,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":289728,"kb_used_data":37816,"kb_used_omap":1,"kb_used_meta":251902,"kb_avail":94082112,"statfs":{"total":96636764160,"available":96340082688,"internally_reserved":0,"allocated":38723584,"data_stored":36776739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":257948102},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575818,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":133816,"kb_used_data":48496,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94238024,"statfs":{"total":96636764160,"available":96499736576,"internally_reserved":0,"allocated":49659904,"data_stored":47630081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87357893},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575818,"num_pgs":104,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":142452,"kb_used_data":44652,"kb_used_omap":1,"kb_used_meta":97790,"kb_avail":94229388,"statfs":{"total":96636764160,"available":96490893312,"internally_reserved":0,"allocated":45723648,"data_stored":43786666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":100137414},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575816,"num_pgs":124,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":269128,"kb_used_data":43584,"kb_used_omap":1,"kb_used_meta":225534,"kb_avail":94102712,"statfs":{"total":96636764160,"available":96361177088,"internally_reserved":0,"allocated":44630016,"data_stored":42616238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":230947271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575816,"num_pgs":91,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":107084,"kb_used_data":35652,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94264756,"statfs":{"total":96636764160,"available":96527110144,"internally_reserved":0,"allocated":36507648,"data_stored":34568865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575820,"num_pgs":112,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":164064,"kb_used_data":46872,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94207776,"statfs":{"total":96636764160,"available":96468762624,"internally_reserved":0,"allocated":47996928,"data_stored":46030586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575815,"num_pgs":109,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266824,"kb_used_data":37120,"kb_used_omap":1,"kb_used_meta":229694,"kb_avail":94105016,"statfs":{"total":96636764160,"available":96363536384,"internally_reserved":0,"allocated":38010880,"data_stored":36012935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":235207109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":27881472,"data_stored":27881472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":37441536,"data_stored":37441536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27029504,"data_stored":27029504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24588288,"data_stored":24588288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":31006720,"data_stored":31006720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":33796096,"data_stored":33796096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":32382976,"data_stored":32382976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24076288,"data_stored":24076288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":4787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":5121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":4696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":6691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":5459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":3918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":7193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T17:14:26.365 INFO:tasks.ceph:pgid 68.1a last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.365 INFO:tasks.ceph:pgid 67.1d last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.365 INFO:tasks.ceph:pgid 67.1c last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.365 INFO:tasks.ceph:pgid 68.18 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.365 INFO:tasks.ceph:pgid 67.1f last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.366 INFO:tasks.ceph:pgid 68.19 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.366 INFO:tasks.ceph:pgid 67.1e last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.366 INFO:tasks.ceph:pgid 68.1e last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.366 INFO:tasks.ceph:pgid 67.19 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.366 INFO:tasks.ceph:pgid 68.1f last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.367 INFO:tasks.ceph:pgid 68.1c last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.367 INFO:tasks.ceph:pgid 66.1a last_scrub_stamp 2024-05-30T16:17:30.145887+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=30, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.367 INFO:tasks.ceph:pgid 68.1d last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.367 INFO:tasks.ceph:pgid 68.12 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.367 INFO:tasks.ceph:pgid 67.17 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.367 INFO:tasks.ceph:pgid 68.11 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.368 INFO:tasks.ceph:pgid 67.16 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.368 INFO:tasks.ceph:pgid 68.16 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.368 INFO:tasks.ceph:pgid 67.11 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.368 INFO:tasks.ceph:pgid 68.17 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.368 INFO:tasks.ceph:pgid 68.14 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.369 INFO:tasks.ceph:pgid 68.15 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.369 INFO:tasks.ceph:pgid 68.b last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.369 INFO:tasks.ceph:pgid 67.c last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.369 INFO:tasks.ceph:pgid 67.f last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.369 INFO:tasks.ceph:pgid 68.e last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.370 INFO:tasks.ceph:pgid 67.9 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.370 INFO:tasks.ceph:pgid 68.f last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.370 INFO:tasks.ceph:pgid 68.c last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.370 INFO:tasks.ceph:pgid 67.b last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.370 INFO:tasks.ceph:pgid 68.d last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.371 INFO:tasks.ceph:pgid 68.2 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.371 INFO:tasks.ceph:pgid 68.3 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.371 INFO:tasks.ceph:pgid 67.4 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.371 INFO:tasks.ceph:pgid 68.0 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.371 INFO:tasks.ceph:pgid 68.1 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.371 INFO:tasks.ceph:pgid 67.6 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.372 INFO:tasks.ceph:pgid 68.7 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.372 INFO:tasks.ceph:pgid 67.0 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.372 INFO:tasks.ceph:pgid 68.4 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.372 INFO:tasks.ceph:pgid 68.5 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.372 INFO:tasks.ceph:pgid 67.2 last_scrub_stamp 2024-05-30T16:17:34.184545+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=17, tm_sec=34, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.373 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.373 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.373 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.373 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.373 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.373 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.374 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.374 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.374 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.374 INFO:tasks.ceph:pgid 6.16 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.375 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.375 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.375 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.375 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.375 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.376 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.376 INFO:tasks.ceph:pgid 6.8 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.376 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.376 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.376 INFO:tasks.ceph:pgid 6.e last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.377 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.377 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-30T15:54:01.512251+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=1, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.377 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.377 INFO:tasks.ceph:pgid 6.7 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.378 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-05-30T15:54:07.550093+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=7, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.378 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.378 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-05-30T15:54:07.550093+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=7, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.378 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.379 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.379 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.379 INFO:tasks.ceph:pgid 6.3 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.379 INFO:tasks.ceph:pgid 6.11 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.380 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.380 INFO:tasks.ceph:pgid 6.10 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.380 INFO:tasks.ceph:pgid 6.13 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.380 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.381 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.381 INFO:tasks.ceph:pgid 6.12 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.381 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.381 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.382 INFO:tasks.ceph:pgid 6.18 last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.382 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.382 INFO:tasks.ceph:pgid 6.1b last_scrub_stamp 2024-05-30T15:54:21.628423+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=21, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.382 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.383 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.383 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.383 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2024-05-30T15:54:17.613765+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.383 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.383 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.384 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.384 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.384 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-05-30T15:54:15.606343+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=15, tm_min=54, tm_sec=15, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:26.384 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-30T17:14:46.385 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T17:14:47.583 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:14:47.599 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-05-30T17:14:47.618 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":2679,"stamp":"2024-05-30T17:14:46.488552+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":191800421,"num_objects":837,"num_object_clones":0,"num_object_copies":1935,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":837,"num_whiteouts":0,"num_read":254020,"num_read_kb":2693627,"num_write":147058,"num_write_kb":2342860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":289,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":720471,"num_omap_keys":1763,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13934,"ondisk_log_size":13934,"up":850,"acting":850,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":850,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1660804,"kb_used_data":332740,"kb_used_omap":12,"kb_used_meta":1327987,"kb_avail":753313916,"statfs":{"total":773094113280,"available":771393449984,"internally_reserved":0,"allocated":340725760,"data_stored":324874602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":1359859249},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":752,"num_omap_keys":8,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008056"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":36,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:31.432419+0000","last_change":"2024-05-30T17:14:31.432480+0000","last_active":"2024-05-30T17:14:31.432419+0000","last_peered":"2024-05-30T17:14:31.432419+0000","last_clean":"2024-05-30T17:14:31.432419+0000","last_became_active":"2024-05-30T16:34:55.869226+0000","last_became_peered":"2024-05-30T16:34:55.869226+0000","last_unstale":"2024-05-30T17:14:31.432419+0000","last_undegraded":"2024-05-30T17:14:31.432419+0000","last_fullsized":"2024-05-30T17:14:31.432419+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:31.432202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:31.432202+0000","last_clean_scrub_stamp":"2024-05-30T17:14:31.432202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:29:56.470555+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"265'2","reported_seq":57,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:29.402904+0000","last_change":"2024-05-30T17:14:29.402967+0000","last_active":"2024-05-30T17:14:29.402904+0000","last_peered":"2024-05-30T17:14:29.402904+0000","last_clean":"2024-05-30T17:14:29.402904+0000","last_became_active":"2024-05-30T16:17:35.248740+0000","last_became_peered":"2024-05-30T16:17:35.248740+0000","last_unstale":"2024-05-30T17:14:29.402904+0000","last_undegraded":"2024-05-30T17:14:29.402904+0000","last_fullsized":"2024-05-30T17:14:29.402904+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:14:29.402717+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:14:29.402717+0000","last_clean_scrub_stamp":"2024-05-30T17:14:29.402717+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:19:16.475399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"265'8","reported_seq":75,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.443531+0000","last_change":"2024-05-30T17:13:43.443594+0000","last_active":"2024-05-30T17:13:43.443531+0000","last_peered":"2024-05-30T17:13:43.443531+0000","last_clean":"2024-05-30T17:13:43.443531+0000","last_became_active":"2024-05-30T16:17:31.183349+0000","last_became_peered":"2024-05-30T16:17:31.183349+0000","last_unstale":"2024-05-30T17:13:43.443531+0000","last_undegraded":"2024-05-30T17:13:43.443531+0000","last_fullsized":"2024-05-30T17:13:43.443531+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'8","last_scrub_stamp":"2024-05-30T17:13:43.443352+0000","last_deep_scrub":"265'8","last_deep_scrub_stamp":"2024-05-30T17:13:43.443352+0000","last_clean_scrub_stamp":"2024-05-30T17:13:43.443352+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:57:10.662914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":16384,"num_write":8,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"265'45","reported_seq":193,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.684557+0000","last_change":"2024-05-30T17:13:48.684645+0000","last_active":"2024-05-30T17:13:48.684557+0000","last_peered":"2024-05-30T17:13:48.684557+0000","last_clean":"2024-05-30T17:13:48.684557+0000","last_became_active":"2024-05-30T16:12:50.803352+0000","last_became_peered":"2024-05-30T16:12:50.803352+0000","last_unstale":"2024-05-30T17:13:48.684557+0000","last_undegraded":"2024-05-30T17:13:48.684557+0000","last_fullsized":"2024-05-30T17:13:48.684557+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'45","last_scrub_stamp":"2024-05-30T17:13:48.684344+0000","last_deep_scrub":"265'45","last_deep_scrub_stamp":"2024-05-30T17:13:48.684344+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.684344+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:56:59.286905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":225,"num_read_kb":156,"num_write":101,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":36,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:14.491141+0000","last_change":"2024-05-30T17:14:14.491199+0000","last_active":"2024-05-30T17:14:14.491141+0000","last_peered":"2024-05-30T17:14:14.491141+0000","last_clean":"2024-05-30T17:14:14.491141+0000","last_became_active":"2024-05-30T16:34:55.886731+0000","last_became_peered":"2024-05-30T16:34:55.886731+0000","last_unstale":"2024-05-30T17:14:14.491141+0000","last_undegraded":"2024-05-30T17:14:14.491141+0000","last_fullsized":"2024-05-30T17:14:14.491141+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:14.490948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:14.490948+0000","last_clean_scrub_stamp":"2024-05-30T17:14:14.490948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:54:47.154526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"0'0","reported_seq":53,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:30.002191+0000","last_change":"2024-05-30T17:14:30.002246+0000","last_active":"2024-05-30T17:14:30.002191+0000","last_peered":"2024-05-30T17:14:30.002191+0000","last_clean":"2024-05-30T17:14:30.002191+0000","last_became_active":"2024-05-30T16:17:35.247810+0000","last_became_peered":"2024-05-30T16:17:35.247810+0000","last_unstale":"2024-05-30T17:14:30.002191+0000","last_undegraded":"2024-05-30T17:14:30.002191+0000","last_fullsized":"2024-05-30T17:14:30.002191+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:30.001987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:30.001987+0000","last_clean_scrub_stamp":"2024-05-30T17:14:30.001987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:47:05.599792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"0'0","reported_seq":59,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.953095+0000","last_change":"2024-05-30T17:13:45.953152+0000","last_active":"2024-05-30T17:13:45.953095+0000","last_peered":"2024-05-30T17:13:45.953095+0000","last_clean":"2024-05-30T17:13:45.953095+0000","last_became_active":"2024-05-30T16:17:31.212337+0000","last_became_peered":"2024-05-30T16:17:31.212337+0000","last_unstale":"2024-05-30T17:13:45.953095+0000","last_undegraded":"2024-05-30T17:13:45.953095+0000","last_fullsized":"2024-05-30T17:13:45.953095+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:45.952903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:45.952903+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.952903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:31:16.835383+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"265'55","reported_seq":219,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:55.177804+0000","last_change":"2024-05-30T17:13:55.177870+0000","last_active":"2024-05-30T17:13:55.177804+0000","last_peered":"2024-05-30T17:13:55.177804+0000","last_clean":"2024-05-30T17:13:55.177804+0000","last_became_active":"2024-05-30T16:12:50.810207+0000","last_became_peered":"2024-05-30T16:12:50.810207+0000","last_unstale":"2024-05-30T17:13:55.177804+0000","last_undegraded":"2024-05-30T17:13:55.177804+0000","last_fullsized":"2024-05-30T17:13:55.177804+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'55","last_scrub_stamp":"2024-05-30T17:13:55.177616+0000","last_deep_scrub":"265'55","last_deep_scrub_stamp":"2024-05-30T17:13:55.177616+0000","last_clean_scrub_stamp":"2024-05-30T17:13:55.177616+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:10:07.203727+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":277,"num_read_kb":197,"num_write":99,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":37,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:21.829390+0000","last_change":"2024-05-30T17:14:21.829444+0000","last_active":"2024-05-30T17:14:21.829390+0000","last_peered":"2024-05-30T17:14:21.829390+0000","last_clean":"2024-05-30T17:14:21.829390+0000","last_became_active":"2024-05-30T16:34:55.874359+0000","last_became_peered":"2024-05-30T16:34:55.874359+0000","last_unstale":"2024-05-30T17:14:21.829390+0000","last_undegraded":"2024-05-30T17:14:21.829390+0000","last_fullsized":"2024-05-30T17:14:21.829390+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:21.829201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:21.829201+0000","last_clean_scrub_stamp":"2024-05-30T17:14:21.829201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:41:35.913572+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"0'0","reported_seq":58,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:35.919291+0000","last_change":"2024-05-30T17:14:35.919350+0000","last_active":"2024-05-30T17:14:35.919291+0000","last_peered":"2024-05-30T17:14:35.919291+0000","last_clean":"2024-05-30T17:14:35.919291+0000","last_became_active":"2024-05-30T16:17:35.743231+0000","last_became_peered":"2024-05-30T16:17:35.743231+0000","last_unstale":"2024-05-30T17:14:35.919291+0000","last_undegraded":"2024-05-30T17:14:35.919291+0000","last_fullsized":"2024-05-30T17:14:35.919291+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:35.919084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:35.919084+0000","last_clean_scrub_stamp":"2024-05-30T17:14:35.919084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:01:37.439385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"265'3","reported_seq":70,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:09.442558+0000","last_change":"2024-05-30T17:14:09.442626+0000","last_active":"2024-05-30T17:14:09.442558+0000","last_peered":"2024-05-30T17:14:09.442558+0000","last_clean":"2024-05-30T17:14:09.442558+0000","last_became_active":"2024-05-30T16:17:31.204806+0000","last_became_peered":"2024-05-30T16:17:31.204806+0000","last_unstale":"2024-05-30T17:14:09.442558+0000","last_undegraded":"2024-05-30T17:14:09.442558+0000","last_fullsized":"2024-05-30T17:14:09.442558+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:14:09.442363+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:14:09.442363+0000","last_clean_scrub_stamp":"2024-05-30T17:14:09.442363+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:38:40.295126+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"265'29","reported_seq":143,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.662503+0000","last_change":"2024-05-30T17:13:46.662582+0000","last_active":"2024-05-30T17:13:46.662503+0000","last_peered":"2024-05-30T17:13:46.662503+0000","last_clean":"2024-05-30T17:13:46.662503+0000","last_became_active":"2024-05-30T16:12:50.810062+0000","last_became_peered":"2024-05-30T16:12:50.810062+0000","last_unstale":"2024-05-30T17:13:46.662503+0000","last_undegraded":"2024-05-30T17:13:46.662503+0000","last_fullsized":"2024-05-30T17:13:46.662503+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'29","last_scrub_stamp":"2024-05-30T17:13:46.662271+0000","last_deep_scrub":"265'29","last_deep_scrub_stamp":"2024-05-30T17:13:46.662271+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.662271+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:14:37.786974+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":98,"num_write":65,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":37,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:40.496075+0000","last_change":"2024-05-30T17:14:40.496132+0000","last_active":"2024-05-30T17:14:40.496075+0000","last_peered":"2024-05-30T17:14:40.496075+0000","last_clean":"2024-05-30T17:14:40.496075+0000","last_became_active":"2024-05-30T16:34:55.887532+0000","last_became_peered":"2024-05-30T16:34:55.887532+0000","last_unstale":"2024-05-30T17:14:40.496075+0000","last_undegraded":"2024-05-30T17:14:40.496075+0000","last_fullsized":"2024-05-30T17:14:40.496075+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:40.495878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:40.495878+0000","last_clean_scrub_stamp":"2024-05-30T17:14:40.495878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:51:19.603432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"265'2","reported_seq":63,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:39.485135+0000","last_change":"2024-05-30T17:14:39.485206+0000","last_active":"2024-05-30T17:14:39.485135+0000","last_peered":"2024-05-30T17:14:39.485135+0000","last_clean":"2024-05-30T17:14:39.485135+0000","last_became_active":"2024-05-30T16:17:35.745709+0000","last_became_peered":"2024-05-30T16:17:35.745709+0000","last_unstale":"2024-05-30T17:14:39.485135+0000","last_undegraded":"2024-05-30T17:14:39.485135+0000","last_fullsized":"2024-05-30T17:14:39.485135+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:14:39.484943+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:14:39.484943+0000","last_clean_scrub_stamp":"2024-05-30T17:14:39.484943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:55:17.462949+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1142,"num_write":2,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1f","version":"265'3","reported_seq":67,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.831674+0000","last_change":"2024-05-30T17:13:50.831741+0000","last_active":"2024-05-30T17:13:50.831674+0000","last_peered":"2024-05-30T17:13:50.831674+0000","last_clean":"2024-05-30T17:13:50.831674+0000","last_became_active":"2024-05-30T16:17:31.182344+0000","last_became_peered":"2024-05-30T16:17:31.182344+0000","last_unstale":"2024-05-30T17:13:50.831674+0000","last_undegraded":"2024-05-30T17:13:50.831674+0000","last_fullsized":"2024-05-30T17:13:50.831674+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:13:50.831396+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:13:50.831396+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.831396+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:36:02.576103+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"260'6","reported_seq":85,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.454253+0000","last_change":"2024-05-30T17:13:46.454380+0000","last_active":"2024-05-30T17:13:46.454253+0000","last_peered":"2024-05-30T17:13:46.454253+0000","last_clean":"2024-05-30T17:13:46.454253+0000","last_became_active":"2024-05-30T16:12:50.826974+0000","last_became_peered":"2024-05-30T16:12:50.826974+0000","last_unstale":"2024-05-30T17:13:46.454253+0000","last_undegraded":"2024-05-30T17:13:46.454253+0000","last_fullsized":"2024-05-30T17:13:46.454253+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'6","last_scrub_stamp":"2024-05-30T17:13:46.453944+0000","last_deep_scrub":"260'6","last_deep_scrub_stamp":"2024-05-30T17:13:46.453944+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.453944+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:05:54.138613+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":24,"num_write":11,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":32,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:06.611451+0000","last_change":"2024-05-30T16:34:55.899516+0000","last_active":"2024-05-30T17:13:06.611451+0000","last_peered":"2024-05-30T17:13:06.611451+0000","last_clean":"2024-05-30T17:13:06.611451+0000","last_became_active":"2024-05-30T16:34:55.898864+0000","last_became_peered":"2024-05-30T16:34:55.898864+0000","last_unstale":"2024-05-30T17:13:06.611451+0000","last_undegraded":"2024-05-30T17:13:06.611451+0000","last_fullsized":"2024-05-30T17:13:06.611451+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"265'8","reported_seq":73,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:28.408420+0000","last_change":"2024-05-30T17:14:28.408568+0000","last_active":"2024-05-30T17:14:28.408420+0000","last_peered":"2024-05-30T17:14:28.408420+0000","last_clean":"2024-05-30T17:14:28.408420+0000","last_became_active":"2024-05-30T16:17:35.232744+0000","last_became_peered":"2024-05-30T16:17:35.232744+0000","last_unstale":"2024-05-30T17:14:28.408420+0000","last_undegraded":"2024-05-30T17:14:28.408420+0000","last_fullsized":"2024-05-30T17:14:28.408420+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'8","last_scrub_stamp":"2024-05-30T17:14:28.408191+0000","last_deep_scrub":"265'8","last_deep_scrub_stamp":"2024-05-30T17:14:28.408191+0000","last_clean_scrub_stamp":"2024-05-30T17:14:28.408191+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:24:09.808156+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":23554,"num_write":6,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"0'0","reported_seq":60,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.680121+0000","last_change":"2024-05-30T17:13:43.680163+0000","last_active":"2024-05-30T17:13:43.680121+0000","last_peered":"2024-05-30T17:13:43.680121+0000","last_clean":"2024-05-30T17:13:43.680121+0000","last_became_active":"2024-05-30T16:17:31.186229+0000","last_became_peered":"2024-05-30T16:17:31.186229+0000","last_unstale":"2024-05-30T17:13:43.680121+0000","last_undegraded":"2024-05-30T17:13:43.680121+0000","last_fullsized":"2024-05-30T17:13:43.680121+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:43.679999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:43.679999+0000","last_clean_scrub_stamp":"2024-05-30T17:13:43.679999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:30:30.079637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"265'27","reported_seq":143,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.491839+0000","last_change":"2024-05-30T17:13:45.491914+0000","last_active":"2024-05-30T17:13:45.491839+0000","last_peered":"2024-05-30T17:13:45.491839+0000","last_clean":"2024-05-30T17:13:45.491839+0000","last_became_active":"2024-05-30T16:12:50.827008+0000","last_became_peered":"2024-05-30T16:12:50.827008+0000","last_unstale":"2024-05-30T17:13:45.491839+0000","last_undegraded":"2024-05-30T17:13:45.491839+0000","last_fullsized":"2024-05-30T17:13:45.491839+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'27","last_scrub_stamp":"2024-05-30T17:13:45.491648+0000","last_deep_scrub":"265'27","last_deep_scrub_stamp":"2024-05-30T17:13:45.491648+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.491648+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:27:36.651992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":100,"num_write":57,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":40,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:24.924709+0000","last_change":"2024-05-30T17:14:24.924709+0000","last_active":"2024-05-30T17:14:24.924709+0000","last_peered":"2024-05-30T17:14:24.924709+0000","last_clean":"2024-05-30T17:14:24.924709+0000","last_became_active":"2024-05-30T16:34:55.870139+0000","last_became_peered":"2024-05-30T16:34:55.870139+0000","last_unstale":"2024-05-30T17:14:24.924709+0000","last_undegraded":"2024-05-30T17:14:24.924709+0000","last_fullsized":"2024-05-30T17:14:24.924709+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:24.924431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:24.924431+0000","last_clean_scrub_stamp":"2024-05-30T17:14:24.924431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:15:42.392670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"265'8","reported_seq":67,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:16.849562+0000","last_change":"2024-05-30T17:14:16.849722+0000","last_active":"2024-05-30T17:14:16.849562+0000","last_peered":"2024-05-30T17:14:16.849562+0000","last_clean":"2024-05-30T17:14:16.849562+0000","last_became_active":"2024-05-30T16:17:35.233040+0000","last_became_peered":"2024-05-30T16:17:35.233040+0000","last_unstale":"2024-05-30T17:14:16.849562+0000","last_undegraded":"2024-05-30T17:14:16.849562+0000","last_fullsized":"2024-05-30T17:14:16.849562+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'8","last_scrub_stamp":"2024-05-30T17:14:16.849340+0000","last_deep_scrub":"265'8","last_deep_scrub_stamp":"2024-05-30T17:14:16.849340+0000","last_clean_scrub_stamp":"2024-05-30T17:14:16.849340+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:56:30.841879+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":12336,"num_write":13,"num_write_kb":11312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"265'3","reported_seq":65,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.804589+0000","last_change":"2024-05-30T17:13:46.804678+0000","last_active":"2024-05-30T17:13:46.804589+0000","last_peered":"2024-05-30T17:13:46.804589+0000","last_clean":"2024-05-30T17:13:46.804589+0000","last_became_active":"2024-05-30T16:17:31.173900+0000","last_became_peered":"2024-05-30T16:17:31.173900+0000","last_unstale":"2024-05-30T17:13:46.804589+0000","last_undegraded":"2024-05-30T17:13:46.804589+0000","last_fullsized":"2024-05-30T17:13:46.804589+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:13:46.804390+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:13:46.804390+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.804390+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:37:20.477514+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"265'75","reported_seq":275,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.777415+0000","last_change":"2024-05-30T17:13:49.777540+0000","last_active":"2024-05-30T17:13:49.777415+0000","last_peered":"2024-05-30T17:13:49.777415+0000","last_clean":"2024-05-30T17:13:49.777415+0000","last_became_active":"2024-05-30T16:12:50.815032+0000","last_became_peered":"2024-05-30T16:12:50.815032+0000","last_unstale":"2024-05-30T17:13:49.777415+0000","last_undegraded":"2024-05-30T17:13:49.777415+0000","last_fullsized":"2024-05-30T17:13:49.777415+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'75","last_scrub_stamp":"2024-05-30T17:13:49.777169+0000","last_deep_scrub":"265'75","last_deep_scrub_stamp":"2024-05-30T17:13:49.777169+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.777169+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:42:13.290083+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":396,"num_read_kb":277,"num_write":137,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"265'2","reported_seq":38,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:36.716471+0000","last_change":"2024-05-30T17:14:36.716564+0000","last_active":"2024-05-30T17:14:36.716471+0000","last_peered":"2024-05-30T17:14:36.716471+0000","last_clean":"2024-05-30T17:14:36.716471+0000","last_became_active":"2024-05-30T16:34:55.881414+0000","last_became_peered":"2024-05-30T16:34:55.881414+0000","last_unstale":"2024-05-30T17:14:36.716471+0000","last_undegraded":"2024-05-30T17:14:36.716471+0000","last_fullsized":"2024-05-30T17:14:36.716471+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:14:36.716234+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:14:36.716234+0000","last_clean_scrub_stamp":"2024-05-30T17:14:36.716234+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:25:24.591710+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"262'2","reported_seq":57,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:17.859737+0000","last_change":"2024-05-30T17:14:17.859865+0000","last_active":"2024-05-30T17:14:17.859737+0000","last_peered":"2024-05-30T17:14:17.859737+0000","last_clean":"2024-05-30T17:14:17.859737+0000","last_became_active":"2024-05-30T16:17:35.237424+0000","last_became_peered":"2024-05-30T16:17:35.237424+0000","last_unstale":"2024-05-30T17:14:17.859737+0000","last_undegraded":"2024-05-30T17:14:17.859737+0000","last_fullsized":"2024-05-30T17:14:17.859737+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'2","last_scrub_stamp":"2024-05-30T17:14:17.859457+0000","last_deep_scrub":"262'2","last_deep_scrub_stamp":"2024-05-30T17:14:17.859457+0000","last_clean_scrub_stamp":"2024-05-30T17:14:17.859457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:38:38.589711+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"0'0","reported_seq":59,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:26.056055+0000","last_change":"2024-05-30T17:14:26.056112+0000","last_active":"2024-05-30T17:14:26.056055+0000","last_peered":"2024-05-30T17:14:26.056055+0000","last_clean":"2024-05-30T17:14:26.056055+0000","last_became_active":"2024-05-30T16:17:31.192269+0000","last_became_peered":"2024-05-30T16:17:31.192269+0000","last_unstale":"2024-05-30T17:14:26.056055+0000","last_undegraded":"2024-05-30T17:14:26.056055+0000","last_fullsized":"2024-05-30T17:14:26.056055+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:26.055851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:26.055851+0000","last_clean_scrub_stamp":"2024-05-30T17:14:26.055851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:16:52.090387+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"265'25","reported_seq":134,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.688810+0000","last_change":"2024-05-30T17:13:45.688906+0000","last_active":"2024-05-30T17:13:45.688810+0000","last_peered":"2024-05-30T17:13:45.688810+0000","last_clean":"2024-05-30T17:13:45.688810+0000","last_became_active":"2024-05-30T16:12:50.811732+0000","last_became_peered":"2024-05-30T16:12:50.811732+0000","last_unstale":"2024-05-30T17:13:45.688810+0000","last_undegraded":"2024-05-30T17:13:45.688810+0000","last_fullsized":"2024-05-30T17:13:45.688810+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'25","last_scrub_stamp":"2024-05-30T17:13:45.688602+0000","last_deep_scrub":"265'25","last_deep_scrub_stamp":"2024-05-30T17:13:45.688602+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.688602+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:20:25.477039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":85,"num_write":53,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":28,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:12:26.585664+0000","last_change":"2024-05-30T16:34:55.899801+0000","last_active":"2024-05-30T17:12:26.585664+0000","last_peered":"2024-05-30T17:12:26.585664+0000","last_clean":"2024-05-30T17:12:26.585664+0000","last_became_active":"2024-05-30T16:34:55.899241+0000","last_became_peered":"2024-05-30T16:34:55.899241+0000","last_unstale":"2024-05-30T17:12:26.585664+0000","last_undegraded":"2024-05-30T17:12:26.585664+0000","last_fullsized":"2024-05-30T17:12:26.585664+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"0'0","reported_seq":53,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:55.376659+0000","last_change":"2024-05-30T17:13:55.376721+0000","last_active":"2024-05-30T17:13:55.376659+0000","last_peered":"2024-05-30T17:13:55.376659+0000","last_clean":"2024-05-30T17:13:55.376659+0000","last_became_active":"2024-05-30T16:17:35.229034+0000","last_became_peered":"2024-05-30T16:17:35.229034+0000","last_unstale":"2024-05-30T17:13:55.376659+0000","last_undegraded":"2024-05-30T17:13:55.376659+0000","last_fullsized":"2024-05-30T17:13:55.376659+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:55.376450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:55.376450+0000","last_clean_scrub_stamp":"2024-05-30T17:13:55.376450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:44:03.105415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"265'9","reported_seq":82,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.700949+0000","last_change":"2024-05-30T17:13:47.701025+0000","last_active":"2024-05-30T17:13:47.700949+0000","last_peered":"2024-05-30T17:13:47.700949+0000","last_clean":"2024-05-30T17:13:47.700949+0000","last_became_active":"2024-05-30T16:17:31.190742+0000","last_became_peered":"2024-05-30T16:17:31.190742+0000","last_unstale":"2024-05-30T17:13:47.700949+0000","last_undegraded":"2024-05-30T17:13:47.700949+0000","last_fullsized":"2024-05-30T17:13:47.700949+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'9","last_scrub_stamp":"2024-05-30T17:13:47.700729+0000","last_deep_scrub":"265'9","last_deep_scrub_stamp":"2024-05-30T17:13:47.700729+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.700729+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:50:23.027430+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":30722,"num_write":17,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"265'33","reported_seq":152,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.992243+0000","last_change":"2024-05-30T17:13:44.992374+0000","last_active":"2024-05-30T17:13:44.992243+0000","last_peered":"2024-05-30T17:13:44.992243+0000","last_clean":"2024-05-30T17:13:44.992243+0000","last_became_active":"2024-05-30T16:12:50.820343+0000","last_became_peered":"2024-05-30T16:12:50.820343+0000","last_unstale":"2024-05-30T17:13:44.992243+0000","last_undegraded":"2024-05-30T17:13:44.992243+0000","last_fullsized":"2024-05-30T17:13:44.992243+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'33","last_scrub_stamp":"2024-05-30T17:13:44.992004+0000","last_deep_scrub":"265'33","last_deep_scrub_stamp":"2024-05-30T17:13:44.992004+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.992004+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:18:13.774581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":154,"num_read_kb":114,"num_write":68,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":38,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:26.916385+0000","last_change":"2024-05-30T17:14:26.916385+0000","last_active":"2024-05-30T17:14:26.916385+0000","last_peered":"2024-05-30T17:14:26.916385+0000","last_clean":"2024-05-30T17:14:26.916385+0000","last_became_active":"2024-05-30T16:34:55.888238+0000","last_became_peered":"2024-05-30T16:34:55.888238+0000","last_unstale":"2024-05-30T17:14:26.916385+0000","last_undegraded":"2024-05-30T17:14:26.916385+0000","last_fullsized":"2024-05-30T17:14:26.916385+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:26.916120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:26.916120+0000","last_clean_scrub_stamp":"2024-05-30T17:14:26.916120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:41:44.053419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"262'6","reported_seq":65,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:11.410160+0000","last_change":"2024-05-30T17:14:11.410224+0000","last_active":"2024-05-30T17:14:11.410160+0000","last_peered":"2024-05-30T17:14:11.410160+0000","last_clean":"2024-05-30T17:14:11.410160+0000","last_became_active":"2024-05-30T16:17:35.738420+0000","last_became_peered":"2024-05-30T16:17:35.738420+0000","last_unstale":"2024-05-30T17:14:11.410160+0000","last_undegraded":"2024-05-30T17:14:11.410160+0000","last_fullsized":"2024-05-30T17:14:11.410160+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'6","last_scrub_stamp":"2024-05-30T17:14:11.409961+0000","last_deep_scrub":"262'6","last_deep_scrub_stamp":"2024-05-30T17:14:11.409961+0000","last_clean_scrub_stamp":"2024-05-30T17:14:11.409961+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:59:36.378123+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6144,"num_write":16,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"265'5","reported_seq":70,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.196417+0000","last_change":"2024-05-30T17:13:56.196484+0000","last_active":"2024-05-30T17:13:56.196417+0000","last_peered":"2024-05-30T17:13:56.196417+0000","last_clean":"2024-05-30T17:13:56.196417+0000","last_became_active":"2024-05-30T16:17:31.195031+0000","last_became_peered":"2024-05-30T16:17:31.195031+0000","last_unstale":"2024-05-30T17:13:56.196417+0000","last_undegraded":"2024-05-30T17:13:56.196417+0000","last_fullsized":"2024-05-30T17:13:56.196417+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'5","last_scrub_stamp":"2024-05-30T17:13:56.196222+0000","last_deep_scrub":"265'5","last_deep_scrub_stamp":"2024-05-30T17:13:56.196222+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.196222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:04:32.713858+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6715,"num_write":5,"num_write_kb":6715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"265'65","reported_seq":246,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:42.646313+0000","last_change":"2024-05-30T17:13:42.646346+0000","last_active":"2024-05-30T17:13:42.646313+0000","last_peered":"2024-05-30T17:13:42.646313+0000","last_clean":"2024-05-30T17:13:42.646313+0000","last_became_active":"2024-05-30T16:12:50.807771+0000","last_became_peered":"2024-05-30T16:12:50.807771+0000","last_unstale":"2024-05-30T17:13:42.646313+0000","last_undegraded":"2024-05-30T17:13:42.646313+0000","last_fullsized":"2024-05-30T17:13:42.646313+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'65","last_scrub_stamp":"2024-05-30T17:13:42.646208+0000","last_deep_scrub":"265'65","last_deep_scrub_stamp":"2024-05-30T17:13:42.646208+0000","last_clean_scrub_stamp":"2024-05-30T17:13:42.646208+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:19:09.986519+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":343,"num_read_kb":242,"num_write":116,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"265'2","reported_seq":40,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:13.464224+0000","last_change":"2024-05-30T17:14:13.464224+0000","last_active":"2024-05-30T17:14:13.464224+0000","last_peered":"2024-05-30T17:14:13.464224+0000","last_clean":"2024-05-30T17:14:13.464224+0000","last_became_active":"2024-05-30T16:34:55.872581+0000","last_became_peered":"2024-05-30T16:34:55.872581+0000","last_unstale":"2024-05-30T17:14:13.464224+0000","last_undegraded":"2024-05-30T17:14:13.464224+0000","last_fullsized":"2024-05-30T17:14:13.464224+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:14:13.463958+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:14:13.463958+0000","last_clean_scrub_stamp":"2024-05-30T17:14:13.463958+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:33:38.750355+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"0'0","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:07.429700+0000","last_change":"2024-05-30T17:14:07.429850+0000","last_active":"2024-05-30T17:14:07.429700+0000","last_peered":"2024-05-30T17:14:07.429700+0000","last_clean":"2024-05-30T17:14:07.429700+0000","last_became_active":"2024-05-30T16:17:35.738397+0000","last_became_peered":"2024-05-30T16:17:35.738397+0000","last_unstale":"2024-05-30T17:14:07.429700+0000","last_undegraded":"2024-05-30T17:14:07.429700+0000","last_fullsized":"2024-05-30T17:14:07.429700+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:07.429474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:07.429474+0000","last_clean_scrub_stamp":"2024-05-30T17:14:07.429474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:01:13.797785+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"265'3","reported_seq":65,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.455165+0000","last_change":"2024-05-30T17:13:49.455225+0000","last_active":"2024-05-30T17:13:49.455165+0000","last_peered":"2024-05-30T17:13:49.455165+0000","last_clean":"2024-05-30T17:13:49.455165+0000","last_became_active":"2024-05-30T16:17:31.196345+0000","last_became_peered":"2024-05-30T16:17:31.196345+0000","last_unstale":"2024-05-30T17:13:49.455165+0000","last_undegraded":"2024-05-30T17:13:49.455165+0000","last_fullsized":"2024-05-30T17:13:49.455165+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:13:49.454958+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:13:49.454958+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.454958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:19:39.781936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"265'38","reported_seq":168,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.437795+0000","last_change":"2024-05-30T17:13:49.437862+0000","last_active":"2024-05-30T17:13:49.437795+0000","last_peered":"2024-05-30T17:13:49.437795+0000","last_clean":"2024-05-30T17:13:49.437795+0000","last_became_active":"2024-05-30T16:12:50.826729+0000","last_became_peered":"2024-05-30T16:12:50.826729+0000","last_unstale":"2024-05-30T17:13:49.437795+0000","last_undegraded":"2024-05-30T17:13:49.437795+0000","last_fullsized":"2024-05-30T17:13:49.437795+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'38","last_scrub_stamp":"2024-05-30T17:13:49.437536+0000","last_deep_scrub":"265'38","last_deep_scrub_stamp":"2024-05-30T17:13:49.437536+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.437536+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:58:24.201516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":187,"num_read_kb":132,"num_write":81,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":38,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:04.725681+0000","last_change":"2024-05-30T17:14:04.725681+0000","last_active":"2024-05-30T17:14:04.725681+0000","last_peered":"2024-05-30T17:14:04.725681+0000","last_clean":"2024-05-30T17:14:04.725681+0000","last_became_active":"2024-05-30T16:34:55.875788+0000","last_became_peered":"2024-05-30T16:34:55.875788+0000","last_unstale":"2024-05-30T17:14:04.725681+0000","last_undegraded":"2024-05-30T17:14:04.725681+0000","last_fullsized":"2024-05-30T17:14:04.725681+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:04.725336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:04.725336+0000","last_clean_scrub_stamp":"2024-05-30T17:14:04.725336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:32:51.275635+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"0'0","reported_seq":54,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:38.529799+0000","last_change":"2024-05-30T17:14:38.529904+0000","last_active":"2024-05-30T17:14:38.529799+0000","last_peered":"2024-05-30T17:14:38.529799+0000","last_clean":"2024-05-30T17:14:38.529799+0000","last_became_active":"2024-05-30T16:17:35.745769+0000","last_became_peered":"2024-05-30T16:17:35.745769+0000","last_unstale":"2024-05-30T17:14:38.529799+0000","last_undegraded":"2024-05-30T17:14:38.529799+0000","last_fullsized":"2024-05-30T17:14:38.529799+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:38.529559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:38.529559+0000","last_clean_scrub_stamp":"2024-05-30T17:14:38.529559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:21:37.195453+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"265'2","reported_seq":63,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:10.398735+0000","last_change":"2024-05-30T17:14:10.398878+0000","last_active":"2024-05-30T17:14:10.398735+0000","last_peered":"2024-05-30T17:14:10.398735+0000","last_clean":"2024-05-30T17:14:10.398735+0000","last_became_active":"2024-05-30T16:17:31.204434+0000","last_became_peered":"2024-05-30T16:17:31.204434+0000","last_unstale":"2024-05-30T17:14:10.398735+0000","last_undegraded":"2024-05-30T17:14:10.398735+0000","last_fullsized":"2024-05-30T17:14:10.398735+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:14:10.398504+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:14:10.398504+0000","last_clean_scrub_stamp":"2024-05-30T17:14:10.398504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:25:37.977228+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"265'26","reported_seq":134,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:57.206156+0000","last_change":"2024-05-30T17:13:57.206337+0000","last_active":"2024-05-30T17:13:57.206156+0000","last_peered":"2024-05-30T17:13:57.206156+0000","last_clean":"2024-05-30T17:13:57.206156+0000","last_became_active":"2024-05-30T16:12:50.813176+0000","last_became_peered":"2024-05-30T16:12:50.813176+0000","last_unstale":"2024-05-30T17:13:57.206156+0000","last_undegraded":"2024-05-30T17:13:57.206156+0000","last_fullsized":"2024-05-30T17:13:57.206156+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'26","last_scrub_stamp":"2024-05-30T17:13:57.205944+0000","last_deep_scrub":"265'26","last_deep_scrub_stamp":"2024-05-30T17:13:57.205944+0000","last_clean_scrub_stamp":"2024-05-30T17:13:57.205944+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:09:07.523880+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":85,"num_write":57,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":38,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:32.401279+0000","last_change":"2024-05-30T17:14:32.401279+0000","last_active":"2024-05-30T17:14:32.401279+0000","last_peered":"2024-05-30T17:14:32.401279+0000","last_clean":"2024-05-30T17:14:32.401279+0000","last_became_active":"2024-05-30T16:34:55.871800+0000","last_became_peered":"2024-05-30T16:34:55.871800+0000","last_unstale":"2024-05-30T17:14:32.401279+0000","last_undegraded":"2024-05-30T17:14:32.401279+0000","last_fullsized":"2024-05-30T17:14:32.401279+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:32.400922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:32.400922+0000","last_clean_scrub_stamp":"2024-05-30T17:14:32.400922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:58:22.647600+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"265'2","reported_seq":57,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:27.047209+0000","last_change":"2024-05-30T17:14:27.047335+0000","last_active":"2024-05-30T17:14:27.047209+0000","last_peered":"2024-05-30T17:14:27.047209+0000","last_clean":"2024-05-30T17:14:27.047209+0000","last_became_active":"2024-05-30T16:17:35.236867+0000","last_became_peered":"2024-05-30T16:17:35.236867+0000","last_unstale":"2024-05-30T17:14:27.047209+0000","last_undegraded":"2024-05-30T17:14:27.047209+0000","last_fullsized":"2024-05-30T17:14:27.047209+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:14:27.046910+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:14:27.046910+0000","last_clean_scrub_stamp":"2024-05-30T17:14:27.046910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:43:54.102925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":571,"num_write":2,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"0'0","reported_seq":60,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.914149+0000","last_change":"2024-05-30T17:13:59.914262+0000","last_active":"2024-05-30T17:13:59.914149+0000","last_peered":"2024-05-30T17:13:59.914149+0000","last_clean":"2024-05-30T17:13:59.914149+0000","last_became_active":"2024-05-30T16:17:31.209882+0000","last_became_peered":"2024-05-30T16:17:31.209882+0000","last_unstale":"2024-05-30T17:13:59.914149+0000","last_undegraded":"2024-05-30T17:13:59.914149+0000","last_fullsized":"2024-05-30T17:13:59.914149+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:59.913913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:59.913913+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.913913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:14:43.780738+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"265'56","reported_seq":228,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.549397+0000","last_change":"2024-05-30T17:13:54.549516+0000","last_active":"2024-05-30T17:13:54.549397+0000","last_peered":"2024-05-30T17:13:54.549397+0000","last_clean":"2024-05-30T17:13:54.549397+0000","last_became_active":"2024-05-30T16:12:50.824043+0000","last_became_peered":"2024-05-30T16:12:50.824043+0000","last_unstale":"2024-05-30T17:13:54.549397+0000","last_undegraded":"2024-05-30T17:13:54.549397+0000","last_fullsized":"2024-05-30T17:13:54.549397+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'56","last_scrub_stamp":"2024-05-30T17:13:54.549107+0000","last_deep_scrub":"265'56","last_deep_scrub_stamp":"2024-05-30T17:13:54.549107+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.549107+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:55:26.307978+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":308,"num_read_kb":221,"num_write":92,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":29,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:36.618123+0000","last_change":"2024-05-30T16:34:55.893966+0000","last_active":"2024-05-30T17:13:36.618123+0000","last_peered":"2024-05-30T17:13:36.618123+0000","last_clean":"2024-05-30T17:13:36.618123+0000","last_became_active":"2024-05-30T16:34:55.893040+0000","last_became_peered":"2024-05-30T16:34:55.893040+0000","last_unstale":"2024-05-30T17:13:36.618123+0000","last_undegraded":"2024-05-30T17:13:36.618123+0000","last_fullsized":"2024-05-30T17:13:36.618123+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"0'0","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:23.020982+0000","last_change":"2024-05-30T17:14:23.021037+0000","last_active":"2024-05-30T17:14:23.020982+0000","last_peered":"2024-05-30T17:14:23.020982+0000","last_clean":"2024-05-30T17:14:23.020982+0000","last_became_active":"2024-05-30T16:17:35.246187+0000","last_became_peered":"2024-05-30T16:17:35.246187+0000","last_unstale":"2024-05-30T17:14:23.020982+0000","last_undegraded":"2024-05-30T17:14:23.020982+0000","last_fullsized":"2024-05-30T17:14:23.020982+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:23.020786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:23.020786+0000","last_clean_scrub_stamp":"2024-05-30T17:14:23.020786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:21:12.644609+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"265'8","reported_seq":80,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.895849+0000","last_change":"2024-05-30T17:13:54.895979+0000","last_active":"2024-05-30T17:13:54.895849+0000","last_peered":"2024-05-30T17:13:54.895849+0000","last_clean":"2024-05-30T17:13:54.895849+0000","last_became_active":"2024-05-30T16:17:31.189091+0000","last_became_peered":"2024-05-30T16:17:31.189091+0000","last_unstale":"2024-05-30T17:13:54.895849+0000","last_undegraded":"2024-05-30T17:13:54.895849+0000","last_fullsized":"2024-05-30T17:13:54.895849+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'8","last_scrub_stamp":"2024-05-30T17:13:54.895517+0000","last_deep_scrub":"265'8","last_deep_scrub_stamp":"2024-05-30T17:13:54.895517+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.895517+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:48:19.231935+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":21053,"num_write":6,"num_write_kb":5691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"265'9","reported_seq":93,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:55.595931+0000","last_change":"2024-05-30T17:13:55.595991+0000","last_active":"2024-05-30T17:13:55.595931+0000","last_peered":"2024-05-30T17:13:55.595931+0000","last_clean":"2024-05-30T17:13:55.595931+0000","last_became_active":"2024-05-30T16:12:50.823759+0000","last_became_peered":"2024-05-30T16:12:50.823759+0000","last_unstale":"2024-05-30T17:13:55.595931+0000","last_undegraded":"2024-05-30T17:13:55.595931+0000","last_fullsized":"2024-05-30T17:13:55.595931+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'9","last_scrub_stamp":"2024-05-30T17:13:55.595737+0000","last_deep_scrub":"265'9","last_deep_scrub_stamp":"2024-05-30T17:13:55.595737+0000","last_clean_scrub_stamp":"2024-05-30T17:13:55.595737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:06:46.930141+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":33,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":37,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:37.946612+0000","last_change":"2024-05-30T17:14:37.946707+0000","last_active":"2024-05-30T17:14:37.946612+0000","last_peered":"2024-05-30T17:14:37.946612+0000","last_clean":"2024-05-30T17:14:37.946612+0000","last_became_active":"2024-05-30T16:34:55.860695+0000","last_became_peered":"2024-05-30T16:34:55.860695+0000","last_unstale":"2024-05-30T17:14:37.946612+0000","last_undegraded":"2024-05-30T17:14:37.946612+0000","last_fullsized":"2024-05-30T17:14:37.946612+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:37.946410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:37.946410+0000","last_clean_scrub_stamp":"2024-05-30T17:14:37.946410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:00:22.393141+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"0'0","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.981887+0000","last_change":"2024-05-30T17:13:53.981997+0000","last_active":"2024-05-30T17:13:53.981887+0000","last_peered":"2024-05-30T17:13:53.981887+0000","last_clean":"2024-05-30T17:13:53.981887+0000","last_became_active":"2024-05-30T16:17:35.239157+0000","last_became_peered":"2024-05-30T16:17:35.239157+0000","last_unstale":"2024-05-30T17:13:53.981887+0000","last_undegraded":"2024-05-30T17:13:53.981887+0000","last_fullsized":"2024-05-30T17:13:53.981887+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:53.981644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:53.981644+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.981644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:55:41.438700+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"265'2","reported_seq":64,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.438167+0000","last_change":"2024-05-30T17:13:47.438232+0000","last_active":"2024-05-30T17:13:47.438167+0000","last_peered":"2024-05-30T17:13:47.438167+0000","last_clean":"2024-05-30T17:13:47.438167+0000","last_became_active":"2024-05-30T16:17:31.184675+0000","last_became_peered":"2024-05-30T16:17:31.184675+0000","last_unstale":"2024-05-30T17:13:47.438167+0000","last_undegraded":"2024-05-30T17:13:47.438167+0000","last_fullsized":"2024-05-30T17:13:47.438167+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:13:47.437969+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:13:47.437969+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.437969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:58:28.886391+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":698,"num_write":2,"num_write_kb":698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"265'97","reported_seq":367,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.658941+0000","last_change":"2024-05-30T17:13:49.659011+0000","last_active":"2024-05-30T17:13:49.658941+0000","last_peered":"2024-05-30T17:13:49.658941+0000","last_clean":"2024-05-30T17:13:49.658941+0000","last_became_active":"2024-05-30T16:12:50.801791+0000","last_became_peered":"2024-05-30T16:12:50.801791+0000","last_unstale":"2024-05-30T17:13:49.658941+0000","last_undegraded":"2024-05-30T17:13:49.658941+0000","last_fullsized":"2024-05-30T17:13:49.658941+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'97","last_scrub_stamp":"2024-05-30T17:13:49.658745+0000","last_deep_scrub":"265'97","last_deep_scrub_stamp":"2024-05-30T17:13:49.658745+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.658745+0000","objects_scrubbed":1,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:10:57.618551+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":591,"num_read_kb":446,"num_write":104,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2994,"num_omap_keys":6,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"265'2","reported_seq":39,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:40.016895+0000","last_change":"2024-05-30T17:14:40.017063+0000","last_active":"2024-05-30T17:14:40.016895+0000","last_peered":"2024-05-30T17:14:40.016895+0000","last_clean":"2024-05-30T17:14:40.016895+0000","last_became_active":"2024-05-30T16:34:55.863483+0000","last_became_peered":"2024-05-30T16:34:55.863483+0000","last_unstale":"2024-05-30T17:14:40.016895+0000","last_undegraded":"2024-05-30T17:14:40.016895+0000","last_fullsized":"2024-05-30T17:14:40.016895+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:14:40.016641+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:14:40.016641+0000","last_clean_scrub_stamp":"2024-05-30T17:14:40.016641+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:24:13.343103+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"0'0","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:03.388657+0000","last_change":"2024-05-30T17:14:03.388717+0000","last_active":"2024-05-30T17:14:03.388657+0000","last_peered":"2024-05-30T17:14:03.388657+0000","last_clean":"2024-05-30T17:14:03.388657+0000","last_became_active":"2024-05-30T16:17:35.235283+0000","last_became_peered":"2024-05-30T16:17:35.235283+0000","last_unstale":"2024-05-30T17:14:03.388657+0000","last_undegraded":"2024-05-30T17:14:03.388657+0000","last_fullsized":"2024-05-30T17:14:03.388657+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:03.388441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:03.388441+0000","last_clean_scrub_stamp":"2024-05-30T17:14:03.388441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:18:19.298873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"0'0","reported_seq":61,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:15.820628+0000","last_change":"2024-05-30T17:14:15.820726+0000","last_active":"2024-05-30T17:14:15.820628+0000","last_peered":"2024-05-30T17:14:15.820628+0000","last_clean":"2024-05-30T17:14:15.820628+0000","last_became_active":"2024-05-30T16:17:31.199499+0000","last_became_peered":"2024-05-30T16:17:31.199499+0000","last_unstale":"2024-05-30T17:14:15.820628+0000","last_undegraded":"2024-05-30T17:14:15.820628+0000","last_fullsized":"2024-05-30T17:14:15.820628+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:15.820409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:15.820409+0000","last_clean_scrub_stamp":"2024-05-30T17:14:15.820409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:58:15.762012+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"265'22","reported_seq":127,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.758712+0000","last_change":"2024-05-30T17:13:44.758825+0000","last_active":"2024-05-30T17:13:44.758712+0000","last_peered":"2024-05-30T17:13:44.758712+0000","last_clean":"2024-05-30T17:13:44.758712+0000","last_became_active":"2024-05-30T16:12:50.807156+0000","last_became_peered":"2024-05-30T16:12:50.807156+0000","last_unstale":"2024-05-30T17:13:44.758712+0000","last_undegraded":"2024-05-30T17:13:44.758712+0000","last_fullsized":"2024-05-30T17:13:44.758712+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'22","last_scrub_stamp":"2024-05-30T17:13:44.758408+0000","last_deep_scrub":"265'22","last_deep_scrub_stamp":"2024-05-30T17:13:44.758408+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.758408+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:20:24.643496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":76,"num_write":50,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"265'2","reported_seq":39,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:41.506569+0000","last_change":"2024-05-30T17:14:41.506641+0000","last_active":"2024-05-30T17:14:41.506569+0000","last_peered":"2024-05-30T17:14:41.506569+0000","last_clean":"2024-05-30T17:14:41.506569+0000","last_became_active":"2024-05-30T16:34:55.887845+0000","last_became_peered":"2024-05-30T16:34:55.887845+0000","last_unstale":"2024-05-30T17:14:41.506569+0000","last_undegraded":"2024-05-30T17:14:41.506569+0000","last_fullsized":"2024-05-30T17:14:41.506569+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:14:41.506353+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:14:41.506353+0000","last_clean_scrub_stamp":"2024-05-30T17:14:41.506353+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:13:17.505411+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"0'0","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:19.832379+0000","last_change":"2024-05-30T17:14:19.832435+0000","last_active":"2024-05-30T17:14:19.832379+0000","last_peered":"2024-05-30T17:14:19.832379+0000","last_clean":"2024-05-30T17:14:19.832379+0000","last_became_active":"2024-05-30T16:17:35.245852+0000","last_became_peered":"2024-05-30T16:17:35.245852+0000","last_unstale":"2024-05-30T17:14:19.832379+0000","last_undegraded":"2024-05-30T17:14:19.832379+0000","last_fullsized":"2024-05-30T17:14:19.832379+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:19.832182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:19.832182+0000","last_clean_scrub_stamp":"2024-05-30T17:14:19.832182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:18:45.511193+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"265'6","reported_seq":77,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:00.249278+0000","last_change":"2024-05-30T17:14:00.249347+0000","last_active":"2024-05-30T17:14:00.249278+0000","last_peered":"2024-05-30T17:14:00.249278+0000","last_clean":"2024-05-30T17:14:00.249278+0000","last_became_active":"2024-05-30T16:17:31.218752+0000","last_became_peered":"2024-05-30T16:17:31.218752+0000","last_unstale":"2024-05-30T17:14:00.249278+0000","last_undegraded":"2024-05-30T17:14:00.249278+0000","last_fullsized":"2024-05-30T17:14:00.249278+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'6","last_scrub_stamp":"2024-05-30T17:14:00.249094+0000","last_deep_scrub":"265'6","last_deep_scrub_stamp":"2024-05-30T17:14:00.249094+0000","last_clean_scrub_stamp":"2024-05-30T17:14:00.249094+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:10:46.604501+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"265'17","reported_seq":112,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.434952+0000","last_change":"2024-05-30T17:13:48.435015+0000","last_active":"2024-05-30T17:13:48.434952+0000","last_peered":"2024-05-30T17:13:48.434952+0000","last_clean":"2024-05-30T17:13:48.434952+0000","last_became_active":"2024-05-30T16:12:50.817114+0000","last_became_peered":"2024-05-30T16:12:50.817114+0000","last_unstale":"2024-05-30T17:13:48.434952+0000","last_undegraded":"2024-05-30T17:13:48.434952+0000","last_fullsized":"2024-05-30T17:13:48.434952+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'17","last_scrub_stamp":"2024-05-30T17:13:48.434761+0000","last_deep_scrub":"265'17","last_deep_scrub_stamp":"2024-05-30T17:13:48.434761+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.434761+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:55.692466+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":55,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":39,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:12.439006+0000","last_change":"2024-05-30T17:14:12.439066+0000","last_active":"2024-05-30T17:14:12.439006+0000","last_peered":"2024-05-30T17:14:12.439006+0000","last_clean":"2024-05-30T17:14:12.439006+0000","last_became_active":"2024-05-30T16:34:55.866415+0000","last_became_peered":"2024-05-30T16:34:55.866415+0000","last_unstale":"2024-05-30T17:14:12.439006+0000","last_undegraded":"2024-05-30T17:14:12.439006+0000","last_fullsized":"2024-05-30T17:14:12.439006+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:12.438804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:12.438804+0000","last_clean_scrub_stamp":"2024-05-30T17:14:12.438804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:30:46.541635+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"0'0","reported_seq":56,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.361463+0000","last_change":"2024-05-30T17:13:51.361463+0000","last_active":"2024-05-30T17:13:51.361463+0000","last_peered":"2024-05-30T17:13:51.361463+0000","last_clean":"2024-05-30T17:13:51.361463+0000","last_became_active":"2024-05-30T16:17:35.227270+0000","last_became_peered":"2024-05-30T16:17:35.227270+0000","last_unstale":"2024-05-30T17:13:51.361463+0000","last_undegraded":"2024-05-30T17:13:51.361463+0000","last_fullsized":"2024-05-30T17:13:51.361463+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:51.361210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:51.361210+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.361210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:13:33.987563+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"0'0","reported_seq":62,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.503842+0000","last_change":"2024-05-30T17:13:45.503842+0000","last_active":"2024-05-30T17:13:45.503842+0000","last_peered":"2024-05-30T17:13:45.503842+0000","last_clean":"2024-05-30T17:13:45.503842+0000","last_became_active":"2024-05-30T16:17:31.203731+0000","last_became_peered":"2024-05-30T16:17:31.203731+0000","last_unstale":"2024-05-30T17:13:45.503842+0000","last_undegraded":"2024-05-30T17:13:45.503842+0000","last_fullsized":"2024-05-30T17:13:45.503842+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:45.503591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:45.503591+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.503591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:19:48.974417+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"265'29","reported_seq":146,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.392583+0000","last_change":"2024-05-30T17:13:50.392583+0000","last_active":"2024-05-30T17:13:50.392583+0000","last_peered":"2024-05-30T17:13:50.392583+0000","last_clean":"2024-05-30T17:13:50.392583+0000","last_became_active":"2024-05-30T16:12:50.824809+0000","last_became_peered":"2024-05-30T16:12:50.824809+0000","last_unstale":"2024-05-30T17:13:50.392583+0000","last_undegraded":"2024-05-30T17:13:50.392583+0000","last_fullsized":"2024-05-30T17:13:50.392583+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'29","last_scrub_stamp":"2024-05-30T17:13:50.392236+0000","last_deep_scrub":"265'29","last_deep_scrub_stamp":"2024-05-30T17:13:50.392236+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.392236+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:32:06.254235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":98,"num_write":77,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":38,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:40.671898+0000","last_change":"2024-05-30T17:14:40.671962+0000","last_active":"2024-05-30T17:14:40.671898+0000","last_peered":"2024-05-30T17:14:40.671898+0000","last_clean":"2024-05-30T17:14:40.671898+0000","last_became_active":"2024-05-30T16:34:55.896357+0000","last_became_peered":"2024-05-30T16:34:55.896357+0000","last_unstale":"2024-05-30T17:14:40.671898+0000","last_undegraded":"2024-05-30T17:14:40.671898+0000","last_fullsized":"2024-05-30T17:14:40.671898+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:40.671697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:40.671697+0000","last_clean_scrub_stamp":"2024-05-30T17:14:40.671697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:17:06.227011+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"265'3","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:30.431628+0000","last_change":"2024-05-30T17:14:30.431690+0000","last_active":"2024-05-30T17:14:30.431628+0000","last_peered":"2024-05-30T17:14:30.431628+0000","last_clean":"2024-05-30T17:14:30.431628+0000","last_became_active":"2024-05-30T16:17:55.550782+0000","last_became_peered":"2024-05-30T16:17:55.550782+0000","last_unstale":"2024-05-30T17:14:30.431628+0000","last_undegraded":"2024-05-30T17:14:30.431628+0000","last_fullsized":"2024-05-30T17:14:30.431628+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:14:30.431421+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:14:30.431421+0000","last_clean_scrub_stamp":"2024-05-30T17:14:30.431421+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:11:57.762683+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2048,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.d","version":"265'7","reported_seq":74,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:07.405375+0000","last_change":"2024-05-30T17:14:07.405503+0000","last_active":"2024-05-30T17:14:07.405375+0000","last_peered":"2024-05-30T17:14:07.405375+0000","last_clean":"2024-05-30T17:14:07.405375+0000","last_became_active":"2024-05-30T16:17:31.202042+0000","last_became_peered":"2024-05-30T16:17:31.202042+0000","last_unstale":"2024-05-30T17:14:07.405375+0000","last_undegraded":"2024-05-30T17:14:07.405375+0000","last_fullsized":"2024-05-30T17:14:07.405375+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'7","last_scrub_stamp":"2024-05-30T17:14:07.405148+0000","last_deep_scrub":"265'7","last_deep_scrub_stamp":"2024-05-30T17:14:07.405148+0000","last_clean_scrub_stamp":"2024-05-30T17:14:07.405148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:09:19.710358+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":6194,"num_write":7,"num_write_kb":6194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"265'24","reported_seq":139,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.172777+0000","last_change":"2024-05-30T17:13:44.172845+0000","last_active":"2024-05-30T17:13:44.172777+0000","last_peered":"2024-05-30T17:13:44.172777+0000","last_clean":"2024-05-30T17:13:44.172777+0000","last_became_active":"2024-05-30T16:12:50.810095+0000","last_became_peered":"2024-05-30T16:12:50.810095+0000","last_unstale":"2024-05-30T17:13:44.172777+0000","last_undegraded":"2024-05-30T17:13:44.172777+0000","last_fullsized":"2024-05-30T17:13:44.172777+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'24","last_scrub_stamp":"2024-05-30T17:13:44.172579+0000","last_deep_scrub":"265'24","last_deep_scrub_stamp":"2024-05-30T17:13:44.172579+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.172579+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:00:02.104351+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":90,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":39,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:20.793663+0000","last_change":"2024-05-30T17:14:20.793717+0000","last_active":"2024-05-30T17:14:20.793663+0000","last_peered":"2024-05-30T17:14:20.793663+0000","last_clean":"2024-05-30T17:14:20.793663+0000","last_became_active":"2024-05-30T16:34:55.862478+0000","last_became_peered":"2024-05-30T16:34:55.862478+0000","last_unstale":"2024-05-30T17:14:20.793663+0000","last_undegraded":"2024-05-30T17:14:20.793663+0000","last_fullsized":"2024-05-30T17:14:20.793663+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:20.793471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:20.793471+0000","last_clean_scrub_stamp":"2024-05-30T17:14:20.793471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:12:26.521954+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"262'3","reported_seq":62,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:23.971862+0000","last_change":"2024-05-30T17:14:23.971939+0000","last_active":"2024-05-30T17:14:23.971862+0000","last_peered":"2024-05-30T17:14:23.971862+0000","last_clean":"2024-05-30T17:14:23.971862+0000","last_became_active":"2024-05-30T16:17:35.742255+0000","last_became_peered":"2024-05-30T16:17:35.742255+0000","last_unstale":"2024-05-30T17:14:23.971862+0000","last_undegraded":"2024-05-30T17:14:23.971862+0000","last_fullsized":"2024-05-30T17:14:23.971862+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'3","last_scrub_stamp":"2024-05-30T17:14:23.971656+0000","last_deep_scrub":"262'3","last_deep_scrub_stamp":"2024-05-30T17:14:23.971656+0000","last_clean_scrub_stamp":"2024-05-30T17:14:23.971656+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:02:04.479467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"265'2","reported_seq":65,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:55.774110+0000","last_change":"2024-05-30T17:13:55.774249+0000","last_active":"2024-05-30T17:13:55.774110+0000","last_peered":"2024-05-30T17:13:55.774110+0000","last_clean":"2024-05-30T17:13:55.774110+0000","last_became_active":"2024-05-30T16:17:31.198744+0000","last_became_peered":"2024-05-30T16:17:31.198744+0000","last_unstale":"2024-05-30T17:13:55.774110+0000","last_undegraded":"2024-05-30T17:13:55.774110+0000","last_fullsized":"2024-05-30T17:13:55.774110+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:13:55.773876+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:13:55.773876+0000","last_clean_scrub_stamp":"2024-05-30T17:13:55.773876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:52:06.016908+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"265'36","reported_seq":172,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.883144+0000","last_change":"2024-05-30T17:13:45.883208+0000","last_active":"2024-05-30T17:13:45.883144+0000","last_peered":"2024-05-30T17:13:45.883144+0000","last_clean":"2024-05-30T17:13:45.883144+0000","last_became_active":"2024-05-30T16:12:50.820936+0000","last_became_peered":"2024-05-30T16:12:50.820936+0000","last_unstale":"2024-05-30T17:13:45.883144+0000","last_undegraded":"2024-05-30T17:13:45.883144+0000","last_fullsized":"2024-05-30T17:13:45.883144+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'36","last_scrub_stamp":"2024-05-30T17:13:45.882931+0000","last_deep_scrub":"265'36","last_deep_scrub_stamp":"2024-05-30T17:13:45.882931+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.882931+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:49:53.101567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":194,"num_read_kb":136,"num_write":73,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":39,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:08.715919+0000","last_change":"2024-05-30T17:14:08.715975+0000","last_active":"2024-05-30T17:14:08.715919+0000","last_peered":"2024-05-30T17:14:08.715919+0000","last_clean":"2024-05-30T17:14:08.715919+0000","last_became_active":"2024-05-30T16:34:55.875965+0000","last_became_peered":"2024-05-30T16:34:55.875965+0000","last_unstale":"2024-05-30T17:14:08.715919+0000","last_undegraded":"2024-05-30T17:14:08.715919+0000","last_fullsized":"2024-05-30T17:14:08.715919+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:08.715715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:08.715715+0000","last_clean_scrub_stamp":"2024-05-30T17:14:08.715715+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:29:25.928545+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"265'3","reported_seq":61,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.354852+0000","last_change":"2024-05-30T17:13:59.354913+0000","last_active":"2024-05-30T17:13:59.354852+0000","last_peered":"2024-05-30T17:13:59.354852+0000","last_clean":"2024-05-30T17:13:59.354852+0000","last_became_active":"2024-05-30T16:17:35.229643+0000","last_became_peered":"2024-05-30T16:17:35.229643+0000","last_unstale":"2024-05-30T17:13:59.354852+0000","last_undegraded":"2024-05-30T17:13:59.354852+0000","last_fullsized":"2024-05-30T17:13:59.354852+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:13:59.354655+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:13:59.354655+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.354655+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:40:27.564311+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.f","version":"262'6","reported_seq":73,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.021324+0000","last_change":"2024-05-30T17:13:43.021324+0000","last_active":"2024-05-30T17:13:43.021324+0000","last_peered":"2024-05-30T17:13:43.021324+0000","last_clean":"2024-05-30T17:13:43.021324+0000","last_became_active":"2024-05-30T16:17:31.212894+0000","last_became_peered":"2024-05-30T16:17:31.212894+0000","last_unstale":"2024-05-30T17:13:43.021324+0000","last_undegraded":"2024-05-30T17:13:43.021324+0000","last_fullsized":"2024-05-30T17:13:43.021324+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'6","last_scrub_stamp":"2024-05-30T17:13:43.021049+0000","last_deep_scrub":"262'6","last_deep_scrub_stamp":"2024-05-30T17:13:43.021049+0000","last_clean_scrub_stamp":"2024-05-30T17:13:43.021049+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:27:51.042980+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6144,"num_write":11,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"262'21","reported_seq":130,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.118605+0000","last_change":"2024-05-30T17:13:53.118667+0000","last_active":"2024-05-30T17:13:53.118605+0000","last_peered":"2024-05-30T17:13:53.118605+0000","last_clean":"2024-05-30T17:13:53.118605+0000","last_became_active":"2024-05-30T16:12:50.805772+0000","last_became_peered":"2024-05-30T16:12:50.805772+0000","last_unstale":"2024-05-30T17:13:53.118605+0000","last_undegraded":"2024-05-30T17:13:53.118605+0000","last_fullsized":"2024-05-30T17:13:53.118605+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'21","last_scrub_stamp":"2024-05-30T17:13:53.118396+0000","last_deep_scrub":"262'21","last_deep_scrub_stamp":"2024-05-30T17:13:53.118396+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.118396+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:21:33.139209+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":79,"num_write":45,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":30,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:11.612685+0000","last_change":"2024-05-30T16:34:55.891074+0000","last_active":"2024-05-30T17:13:11.612685+0000","last_peered":"2024-05-30T17:13:11.612685+0000","last_clean":"2024-05-30T17:13:11.612685+0000","last_became_active":"2024-05-30T16:34:55.890313+0000","last_became_peered":"2024-05-30T16:34:55.890313+0000","last_unstale":"2024-05-30T17:13:11.612685+0000","last_undegraded":"2024-05-30T17:13:11.612685+0000","last_fullsized":"2024-05-30T17:13:11.612685+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:34:54.830789+0000","last_clean_scrub_stamp":"2024-05-30T16:34:54.830789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"265'3","reported_seq":64,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:29.017317+0000","last_change":"2024-05-30T17:14:29.017378+0000","last_active":"2024-05-30T17:14:29.017317+0000","last_peered":"2024-05-30T17:14:29.017317+0000","last_clean":"2024-05-30T17:14:29.017317+0000","last_became_active":"2024-05-30T16:17:35.242271+0000","last_became_peered":"2024-05-30T16:17:35.242271+0000","last_unstale":"2024-05-30T17:14:29.017317+0000","last_undegraded":"2024-05-30T17:14:29.017317+0000","last_fullsized":"2024-05-30T17:14:29.017317+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:14:29.017118+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:14:29.017118+0000","last_clean_scrub_stamp":"2024-05-30T17:14:29.017118+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:44:42.366902+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10240,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"265'3","reported_seq":70,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.656221+0000","last_change":"2024-05-30T17:13:51.656292+0000","last_active":"2024-05-30T17:13:51.656221+0000","last_peered":"2024-05-30T17:13:51.656221+0000","last_clean":"2024-05-30T17:13:51.656221+0000","last_became_active":"2024-05-30T16:17:31.194774+0000","last_became_peered":"2024-05-30T16:17:31.194774+0000","last_unstale":"2024-05-30T17:13:51.656221+0000","last_undegraded":"2024-05-30T17:13:51.656221+0000","last_fullsized":"2024-05-30T17:13:51.656221+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:13:51.656016+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:13:51.656016+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.656016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:08:09.027576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":12288,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"265'18","reported_seq":116,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.659792+0000","last_change":"2024-05-30T17:13:44.659853+0000","last_active":"2024-05-30T17:13:44.659792+0000","last_peered":"2024-05-30T17:13:44.659792+0000","last_clean":"2024-05-30T17:13:44.659792+0000","last_became_active":"2024-05-30T16:12:50.820526+0000","last_became_peered":"2024-05-30T16:12:50.820526+0000","last_unstale":"2024-05-30T17:13:44.659792+0000","last_undegraded":"2024-05-30T17:13:44.659792+0000","last_fullsized":"2024-05-30T17:13:44.659792+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'18","last_scrub_stamp":"2024-05-30T17:13:44.659583+0000","last_deep_scrub":"265'18","last_deep_scrub_stamp":"2024-05-30T17:13:44.659583+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.659583+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:01:55.866157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":60,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":38,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:38.697474+0000","last_change":"2024-05-30T17:14:38.697534+0000","last_active":"2024-05-30T17:14:38.697474+0000","last_peered":"2024-05-30T17:14:38.697474+0000","last_clean":"2024-05-30T17:14:38.697474+0000","last_became_active":"2024-05-30T16:34:55.884065+0000","last_became_peered":"2024-05-30T16:34:55.884065+0000","last_unstale":"2024-05-30T17:14:38.697474+0000","last_undegraded":"2024-05-30T17:14:38.697474+0000","last_fullsized":"2024-05-30T17:14:38.697474+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:38.697264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:38.697264+0000","last_clean_scrub_stamp":"2024-05-30T17:14:38.697264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:45:29.551653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"265'11","reported_seq":83,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.610393+0000","last_change":"2024-05-30T17:13:56.610471+0000","last_active":"2024-05-30T17:13:56.610393+0000","last_peered":"2024-05-30T17:13:56.610393+0000","last_clean":"2024-05-30T17:13:56.610393+0000","last_became_active":"2024-05-30T16:17:35.738991+0000","last_became_peered":"2024-05-30T16:17:35.738991+0000","last_unstale":"2024-05-30T17:13:56.610393+0000","last_undegraded":"2024-05-30T17:13:56.610393+0000","last_fullsized":"2024-05-30T17:13:56.610393+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'11","last_scrub_stamp":"2024-05-30T17:13:56.610167+0000","last_deep_scrub":"265'11","last_deep_scrub_stamp":"2024-05-30T17:13:56.610167+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.610167+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:07:32.799346+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":26998,"num_write":14,"num_write_kb":10938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"262'6","reported_seq":75,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.948567+0000","last_change":"2024-05-30T17:13:49.948567+0000","last_active":"2024-05-30T17:13:49.948567+0000","last_peered":"2024-05-30T17:13:49.948567+0000","last_clean":"2024-05-30T17:13:49.948567+0000","last_became_active":"2024-05-30T16:17:31.212954+0000","last_became_peered":"2024-05-30T16:17:31.212954+0000","last_unstale":"2024-05-30T17:13:49.948567+0000","last_undegraded":"2024-05-30T17:13:49.948567+0000","last_fullsized":"2024-05-30T17:13:49.948567+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'6","last_scrub_stamp":"2024-05-30T17:13:49.948291+0000","last_deep_scrub":"262'6","last_deep_scrub_stamp":"2024-05-30T17:13:49.948291+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.948291+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:46:37.622512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":11264,"num_write":11,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"265'103","reported_seq":396,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:57.554791+0000","last_change":"2024-05-30T17:13:57.554869+0000","last_active":"2024-05-30T17:13:57.554791+0000","last_peered":"2024-05-30T17:13:57.554791+0000","last_clean":"2024-05-30T17:13:57.554791+0000","last_became_active":"2024-05-30T16:12:50.820250+0000","last_became_peered":"2024-05-30T16:12:50.820250+0000","last_unstale":"2024-05-30T17:13:57.554791+0000","last_undegraded":"2024-05-30T17:13:57.554791+0000","last_fullsized":"2024-05-30T17:13:57.554791+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'103","last_scrub_stamp":"2024-05-30T17:13:57.554576+0000","last_deep_scrub":"265'103","last_deep_scrub_stamp":"2024-05-30T17:13:57.554576+0000","last_clean_scrub_stamp":"2024-05-30T17:13:57.554576+0000","objects_scrubbed":1,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:49:22.258431+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":642,"num_read_kb":499,"num_write":114,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3960,"num_omap_keys":8,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"265'2","reported_seq":40,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:35.730656+0000","last_change":"2024-05-30T17:14:35.730732+0000","last_active":"2024-05-30T17:14:35.730656+0000","last_peered":"2024-05-30T17:14:35.730656+0000","last_clean":"2024-05-30T17:14:35.730656+0000","last_became_active":"2024-05-30T16:34:55.878171+0000","last_became_peered":"2024-05-30T16:34:55.878171+0000","last_unstale":"2024-05-30T17:14:35.730656+0000","last_undegraded":"2024-05-30T17:14:35.730656+0000","last_fullsized":"2024-05-30T17:14:35.730656+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:14:35.730439+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:14:35.730439+0000","last_clean_scrub_stamp":"2024-05-30T17:14:35.730439+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:15:42.676263+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"265'15","reported_seq":85,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:21.995914+0000","last_change":"2024-05-30T17:14:21.995983+0000","last_active":"2024-05-30T17:14:21.995914+0000","last_peered":"2024-05-30T17:14:21.995914+0000","last_clean":"2024-05-30T17:14:21.995914+0000","last_became_active":"2024-05-30T16:17:35.245008+0000","last_became_peered":"2024-05-30T16:17:35.245008+0000","last_unstale":"2024-05-30T17:14:21.995914+0000","last_undegraded":"2024-05-30T17:14:21.995914+0000","last_fullsized":"2024-05-30T17:14:21.995914+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'15","last_scrub_stamp":"2024-05-30T17:14:21.995693+0000","last_deep_scrub":"265'15","last_deep_scrub_stamp":"2024-05-30T17:14:21.995693+0000","last_clean_scrub_stamp":"2024-05-30T17:14:21.995693+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:59:57.620982+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":25602,"num_write":23,"num_write_kb":17408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"262'3","reported_seq":69,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:03.872899+0000","last_change":"2024-05-30T17:14:03.872899+0000","last_active":"2024-05-30T17:14:03.872899+0000","last_peered":"2024-05-30T17:14:03.872899+0000","last_clean":"2024-05-30T17:14:03.872899+0000","last_became_active":"2024-05-30T16:17:31.210195+0000","last_became_peered":"2024-05-30T16:17:31.210195+0000","last_unstale":"2024-05-30T17:14:03.872899+0000","last_undegraded":"2024-05-30T17:14:03.872899+0000","last_fullsized":"2024-05-30T17:14:03.872899+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'3","last_scrub_stamp":"2024-05-30T17:14:03.872614+0000","last_deep_scrub":"262'3","last_deep_scrub_stamp":"2024-05-30T17:14:03.872614+0000","last_clean_scrub_stamp":"2024-05-30T17:14:03.872614+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:58:00.016236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"265'43","reported_seq":185,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.468823+0000","last_change":"2024-05-30T17:13:44.468886+0000","last_active":"2024-05-30T17:13:44.468823+0000","last_peered":"2024-05-30T17:13:44.468823+0000","last_clean":"2024-05-30T17:13:44.468823+0000","last_became_active":"2024-05-30T16:12:50.830770+0000","last_became_peered":"2024-05-30T16:12:50.830770+0000","last_unstale":"2024-05-30T17:13:44.468823+0000","last_undegraded":"2024-05-30T17:13:44.468823+0000","last_fullsized":"2024-05-30T17:13:44.468823+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'43","last_scrub_stamp":"2024-05-30T17:13:44.468612+0000","last_deep_scrub":"265'43","last_deep_scrub_stamp":"2024-05-30T17:13:44.468612+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.468612+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:26:30.609648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":214,"num_read_kb":148,"num_write":89,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":38,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:40.995719+0000","last_change":"2024-05-30T17:14:40.995772+0000","last_active":"2024-05-30T17:14:40.995719+0000","last_peered":"2024-05-30T17:14:40.995719+0000","last_clean":"2024-05-30T17:14:40.995719+0000","last_became_active":"2024-05-30T16:34:55.887101+0000","last_became_peered":"2024-05-30T16:34:55.887101+0000","last_unstale":"2024-05-30T17:14:40.995719+0000","last_undegraded":"2024-05-30T17:14:40.995719+0000","last_fullsized":"2024-05-30T17:14:40.995719+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:40.995523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:40.995523+0000","last_clean_scrub_stamp":"2024-05-30T17:14:40.995523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:24:17.122800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"262'3","reported_seq":63,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:52.503977+0000","last_change":"2024-05-30T17:13:52.503977+0000","last_active":"2024-05-30T17:13:52.503977+0000","last_peered":"2024-05-30T17:13:52.503977+0000","last_clean":"2024-05-30T17:13:52.503977+0000","last_became_active":"2024-05-30T16:17:35.212403+0000","last_became_peered":"2024-05-30T16:17:35.212403+0000","last_unstale":"2024-05-30T17:13:52.503977+0000","last_undegraded":"2024-05-30T17:13:52.503977+0000","last_fullsized":"2024-05-30T17:13:52.503977+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'3","last_scrub_stamp":"2024-05-30T17:13:52.503699+0000","last_deep_scrub":"262'3","last_deep_scrub_stamp":"2024-05-30T17:13:52.503699+0000","last_clean_scrub_stamp":"2024-05-30T17:13:52.503699+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:51:26.227521+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"0'0","reported_seq":62,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.933818+0000","last_change":"2024-05-30T17:13:58.933876+0000","last_active":"2024-05-30T17:13:58.933818+0000","last_peered":"2024-05-30T17:13:58.933818+0000","last_clean":"2024-05-30T17:13:58.933818+0000","last_became_active":"2024-05-30T16:17:31.191955+0000","last_became_peered":"2024-05-30T16:17:31.191955+0000","last_unstale":"2024-05-30T17:13:58.933818+0000","last_undegraded":"2024-05-30T17:13:58.933818+0000","last_fullsized":"2024-05-30T17:13:58.933818+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:58.933611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:58.933611+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.933611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:34:51.660228+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"262'8","reported_seq":92,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.839264+0000","last_change":"2024-05-30T17:13:46.839335+0000","last_active":"2024-05-30T17:13:46.839264+0000","last_peered":"2024-05-30T17:13:46.839264+0000","last_clean":"2024-05-30T17:13:46.839264+0000","last_became_active":"2024-05-30T16:12:50.819871+0000","last_became_peered":"2024-05-30T16:12:50.819871+0000","last_unstale":"2024-05-30T17:13:46.839264+0000","last_undegraded":"2024-05-30T17:13:46.839264+0000","last_fullsized":"2024-05-30T17:13:46.839264+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'8","last_scrub_stamp":"2024-05-30T17:13:46.839029+0000","last_deep_scrub":"262'8","last_deep_scrub_stamp":"2024-05-30T17:13:46.839029+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.839029+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:29:46.097404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":28,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":40,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:37.742935+0000","last_change":"2024-05-30T17:14:37.742995+0000","last_active":"2024-05-30T17:14:37.742935+0000","last_peered":"2024-05-30T17:14:37.742935+0000","last_clean":"2024-05-30T17:14:37.742935+0000","last_became_active":"2024-05-30T16:34:55.882443+0000","last_became_peered":"2024-05-30T16:34:55.882443+0000","last_unstale":"2024-05-30T17:14:37.742935+0000","last_undegraded":"2024-05-30T17:14:37.742935+0000","last_fullsized":"2024-05-30T17:14:37.742935+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:37.742728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:37.742728+0000","last_clean_scrub_stamp":"2024-05-30T17:14:37.742728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:41:14.282969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"262'3","reported_seq":64,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:07.870394+0000","last_change":"2024-05-30T17:14:07.870537+0000","last_active":"2024-05-30T17:14:07.870394+0000","last_peered":"2024-05-30T17:14:07.870394+0000","last_clean":"2024-05-30T17:14:07.870394+0000","last_became_active":"2024-05-30T16:17:35.247419+0000","last_became_peered":"2024-05-30T16:17:35.247419+0000","last_unstale":"2024-05-30T17:14:07.870394+0000","last_undegraded":"2024-05-30T17:14:07.870394+0000","last_fullsized":"2024-05-30T17:14:07.870394+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'3","last_scrub_stamp":"2024-05-30T17:14:07.870141+0000","last_deep_scrub":"262'3","last_deep_scrub_stamp":"2024-05-30T17:14:07.870141+0000","last_clean_scrub_stamp":"2024-05-30T17:14:07.870141+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:18:01.008022+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"262'2","reported_seq":68,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.502634+0000","last_change":"2024-05-30T17:13:50.502697+0000","last_active":"2024-05-30T17:13:50.502634+0000","last_peered":"2024-05-30T17:13:50.502634+0000","last_clean":"2024-05-30T17:13:50.502634+0000","last_became_active":"2024-05-30T16:17:31.189122+0000","last_became_peered":"2024-05-30T16:17:31.189122+0000","last_unstale":"2024-05-30T17:13:50.502634+0000","last_undegraded":"2024-05-30T17:13:50.502634+0000","last_fullsized":"2024-05-30T17:13:50.502634+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'2","last_scrub_stamp":"2024-05-30T17:13:50.502442+0000","last_deep_scrub":"262'2","last_deep_scrub_stamp":"2024-05-30T17:13:50.502442+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.502442+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:48:21.318011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"265'53","reported_seq":210,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.655149+0000","last_change":"2024-05-30T17:13:41.655171+0000","last_active":"2024-05-30T17:13:41.655149+0000","last_peered":"2024-05-30T17:13:41.655149+0000","last_clean":"2024-05-30T17:13:41.655149+0000","last_became_active":"2024-05-30T16:12:50.809372+0000","last_became_peered":"2024-05-30T16:12:50.809372+0000","last_unstale":"2024-05-30T17:13:41.655149+0000","last_undegraded":"2024-05-30T17:13:41.655149+0000","last_fullsized":"2024-05-30T17:13:41.655149+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'53","last_scrub_stamp":"2024-05-30T17:13:41.655071+0000","last_deep_scrub":"265'53","last_deep_scrub_stamp":"2024-05-30T17:13:41.655071+0000","last_clean_scrub_stamp":"2024-05-30T17:13:41.655071+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:39:32.196268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":186,"num_write":103,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":40,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:33.403502+0000","last_change":"2024-05-30T17:14:33.403558+0000","last_active":"2024-05-30T17:14:33.403502+0000","last_peered":"2024-05-30T17:14:33.403502+0000","last_clean":"2024-05-30T17:14:33.403502+0000","last_became_active":"2024-05-30T16:34:55.880627+0000","last_became_peered":"2024-05-30T16:34:55.880627+0000","last_unstale":"2024-05-30T17:14:33.403502+0000","last_undegraded":"2024-05-30T17:14:33.403502+0000","last_fullsized":"2024-05-30T17:14:33.403502+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:33.403309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:33.403309+0000","last_clean_scrub_stamp":"2024-05-30T17:14:33.403309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:35:32.928002+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"0'0","reported_seq":58,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:34.878116+0000","last_change":"2024-05-30T17:14:34.878176+0000","last_active":"2024-05-30T17:14:34.878116+0000","last_peered":"2024-05-30T17:14:34.878116+0000","last_clean":"2024-05-30T17:14:34.878116+0000","last_became_active":"2024-05-30T16:17:35.743036+0000","last_became_peered":"2024-05-30T16:17:35.743036+0000","last_unstale":"2024-05-30T17:14:34.878116+0000","last_undegraded":"2024-05-30T17:14:34.878116+0000","last_fullsized":"2024-05-30T17:14:34.878116+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:34.877900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:34.877900+0000","last_clean_scrub_stamp":"2024-05-30T17:14:34.877900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:17:31.425313+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"265'3","reported_seq":69,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:08.435518+0000","last_change":"2024-05-30T17:14:08.435583+0000","last_active":"2024-05-30T17:14:08.435518+0000","last_peered":"2024-05-30T17:14:08.435518+0000","last_clean":"2024-05-30T17:14:08.435518+0000","last_became_active":"2024-05-30T16:17:31.202064+0000","last_became_peered":"2024-05-30T16:17:31.202064+0000","last_unstale":"2024-05-30T17:14:08.435518+0000","last_undegraded":"2024-05-30T17:14:08.435518+0000","last_fullsized":"2024-05-30T17:14:08.435518+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:14:08.435314+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:14:08.435314+0000","last_clean_scrub_stamp":"2024-05-30T17:14:08.435314+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:58:19.406401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2048,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"265'34","reported_seq":158,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.862253+0000","last_change":"2024-05-30T17:13:47.862374+0000","last_active":"2024-05-30T17:13:47.862253+0000","last_peered":"2024-05-30T17:13:47.862253+0000","last_clean":"2024-05-30T17:13:47.862253+0000","last_became_active":"2024-05-30T16:12:50.815895+0000","last_became_peered":"2024-05-30T16:12:50.815895+0000","last_unstale":"2024-05-30T17:13:47.862253+0000","last_undegraded":"2024-05-30T17:13:47.862253+0000","last_fullsized":"2024-05-30T17:13:47.862253+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'34","last_scrub_stamp":"2024-05-30T17:13:47.861959+0000","last_deep_scrub":"265'34","last_deep_scrub_stamp":"2024-05-30T17:13:47.861959+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.861959+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:37:52.262386+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":160,"num_read_kb":111,"num_write":71,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":40,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:25.906808+0000","last_change":"2024-05-30T17:14:25.906869+0000","last_active":"2024-05-30T17:14:25.906808+0000","last_peered":"2024-05-30T17:14:25.906808+0000","last_clean":"2024-05-30T17:14:25.906808+0000","last_became_active":"2024-05-30T16:34:55.883112+0000","last_became_peered":"2024-05-30T16:34:55.883112+0000","last_unstale":"2024-05-30T17:14:25.906808+0000","last_undegraded":"2024-05-30T17:14:25.906808+0000","last_fullsized":"2024-05-30T17:14:25.906808+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:25.906607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:25.906607+0000","last_clean_scrub_stamp":"2024-05-30T17:14:25.906607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:04:15.646212+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"265'3","reported_seq":62,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:00.593884+0000","last_change":"2024-05-30T17:14:00.593945+0000","last_active":"2024-05-30T17:14:00.593884+0000","last_peered":"2024-05-30T17:14:00.593884+0000","last_clean":"2024-05-30T17:14:00.593884+0000","last_became_active":"2024-05-30T16:17:35.743608+0000","last_became_peered":"2024-05-30T16:17:35.743608+0000","last_unstale":"2024-05-30T17:14:00.593884+0000","last_undegraded":"2024-05-30T17:14:00.593884+0000","last_fullsized":"2024-05-30T17:14:00.593884+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:14:00.593691+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:14:00.593691+0000","last_clean_scrub_stamp":"2024-05-30T17:14:00.593691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:51:53.188748+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"0'0","reported_seq":63,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:52.225937+0000","last_change":"2024-05-30T17:13:52.225991+0000","last_active":"2024-05-30T17:13:52.225937+0000","last_peered":"2024-05-30T17:13:52.225937+0000","last_clean":"2024-05-30T17:13:52.225937+0000","last_became_active":"2024-05-30T16:17:31.188810+0000","last_became_peered":"2024-05-30T16:17:31.188810+0000","last_unstale":"2024-05-30T17:13:52.225937+0000","last_undegraded":"2024-05-30T17:13:52.225937+0000","last_fullsized":"2024-05-30T17:13:52.225937+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:52.225736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:52.225736+0000","last_clean_scrub_stamp":"2024-05-30T17:13:52.225736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:31:29.155002+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"265'18","reported_seq":120,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.502424+0000","last_change":"2024-05-30T17:13:53.502489+0000","last_active":"2024-05-30T17:13:53.502424+0000","last_peered":"2024-05-30T17:13:53.502424+0000","last_clean":"2024-05-30T17:13:53.502424+0000","last_became_active":"2024-05-30T16:12:50.826880+0000","last_became_peered":"2024-05-30T16:12:50.826880+0000","last_unstale":"2024-05-30T17:13:53.502424+0000","last_undegraded":"2024-05-30T17:13:53.502424+0000","last_fullsized":"2024-05-30T17:13:53.502424+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'18","last_scrub_stamp":"2024-05-30T17:13:53.502233+0000","last_deep_scrub":"265'18","last_deep_scrub_stamp":"2024-05-30T17:13:53.502233+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.502233+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:07:36.967499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":61,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":40,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:32.064770+0000","last_change":"2024-05-30T17:14:32.064905+0000","last_active":"2024-05-30T17:14:32.064770+0000","last_peered":"2024-05-30T17:14:32.064770+0000","last_clean":"2024-05-30T17:14:32.064770+0000","last_became_active":"2024-05-30T16:34:55.885643+0000","last_became_peered":"2024-05-30T16:34:55.885643+0000","last_unstale":"2024-05-30T17:14:32.064770+0000","last_undegraded":"2024-05-30T17:14:32.064770+0000","last_fullsized":"2024-05-30T17:14:32.064770+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:32.064467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:32.064467+0000","last_clean_scrub_stamp":"2024-05-30T17:14:32.064467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:55:02.356439+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"262'2","reported_seq":61,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:31.036894+0000","last_change":"2024-05-30T17:14:31.036955+0000","last_active":"2024-05-30T17:14:31.036894+0000","last_peered":"2024-05-30T17:14:31.036894+0000","last_clean":"2024-05-30T17:14:31.036894+0000","last_became_active":"2024-05-30T16:17:35.247757+0000","last_became_peered":"2024-05-30T16:17:35.247757+0000","last_unstale":"2024-05-30T17:14:31.036894+0000","last_undegraded":"2024-05-30T17:14:31.036894+0000","last_fullsized":"2024-05-30T17:14:31.036894+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'2","last_scrub_stamp":"2024-05-30T17:14:31.036688+0000","last_deep_scrub":"262'2","last_deep_scrub_stamp":"2024-05-30T17:14:31.036688+0000","last_clean_scrub_stamp":"2024-05-30T17:14:31.036688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:54:00.790665+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"0'0","reported_seq":63,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.456500+0000","last_change":"2024-05-30T17:13:46.456500+0000","last_active":"2024-05-30T17:13:46.456500+0000","last_peered":"2024-05-30T17:13:46.456500+0000","last_clean":"2024-05-30T17:13:46.456500+0000","last_became_active":"2024-05-30T16:17:31.203452+0000","last_became_peered":"2024-05-30T16:17:31.203452+0000","last_unstale":"2024-05-30T17:13:46.456500+0000","last_undegraded":"2024-05-30T17:13:46.456500+0000","last_fullsized":"2024-05-30T17:13:46.456500+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:46.456245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:46.456245+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.456245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:07:15.159368+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"265'37","reported_seq":171,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.919132+0000","last_change":"2024-05-30T17:13:46.919198+0000","last_active":"2024-05-30T17:13:46.919132+0000","last_peered":"2024-05-30T17:13:46.919132+0000","last_clean":"2024-05-30T17:13:46.919132+0000","last_became_active":"2024-05-30T16:12:50.811626+0000","last_became_peered":"2024-05-30T16:12:50.811626+0000","last_unstale":"2024-05-30T17:13:46.919132+0000","last_undegraded":"2024-05-30T17:13:46.919132+0000","last_fullsized":"2024-05-30T17:13:46.919132+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'37","last_scrub_stamp":"2024-05-30T17:13:46.918931+0000","last_deep_scrub":"265'37","last_deep_scrub_stamp":"2024-05-30T17:13:46.918931+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.918931+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:38:14.857169+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":184,"num_read_kb":132,"num_write":81,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"265'2","reported_seq":41,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:11.395523+0000","last_change":"2024-05-30T17:14:11.395649+0000","last_active":"2024-05-30T17:14:11.395523+0000","last_peered":"2024-05-30T17:14:11.395523+0000","last_clean":"2024-05-30T17:14:11.395523+0000","last_became_active":"2024-05-30T16:34:55.861767+0000","last_became_peered":"2024-05-30T16:34:55.861767+0000","last_unstale":"2024-05-30T17:14:11.395523+0000","last_undegraded":"2024-05-30T17:14:11.395523+0000","last_fullsized":"2024-05-30T17:14:11.395523+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:14:11.395245+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:14:11.395245+0000","last_clean_scrub_stamp":"2024-05-30T17:14:11.395245+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:31:20.732589+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"265'8","reported_seq":73,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:20.954829+0000","last_change":"2024-05-30T17:14:20.954890+0000","last_active":"2024-05-30T17:14:20.954829+0000","last_peered":"2024-05-30T17:14:20.954829+0000","last_clean":"2024-05-30T17:14:20.954829+0000","last_became_active":"2024-05-30T16:17:35.243435+0000","last_became_peered":"2024-05-30T16:17:35.243435+0000","last_unstale":"2024-05-30T17:14:20.954829+0000","last_undegraded":"2024-05-30T17:14:20.954829+0000","last_fullsized":"2024-05-30T17:14:20.954829+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'8","last_scrub_stamp":"2024-05-30T17:14:20.954635+0000","last_deep_scrub":"265'8","last_deep_scrub_stamp":"2024-05-30T17:14:20.954635+0000","last_clean_scrub_stamp":"2024-05-30T17:14:20.954635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:57:35.836726+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":13312,"num_write":13,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"265'3","reported_seq":68,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.458042+0000","last_change":"2024-05-30T17:13:47.458106+0000","last_active":"2024-05-30T17:13:47.458042+0000","last_peered":"2024-05-30T17:13:47.458042+0000","last_clean":"2024-05-30T17:13:47.458042+0000","last_became_active":"2024-05-30T16:17:31.200663+0000","last_became_peered":"2024-05-30T16:17:31.200663+0000","last_unstale":"2024-05-30T17:13:47.458042+0000","last_undegraded":"2024-05-30T17:13:47.458042+0000","last_fullsized":"2024-05-30T17:13:47.458042+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:13:47.457841+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:13:47.457841+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.457841+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:37:54.600403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"265'15","reported_seq":112,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.234861+0000","last_change":"2024-05-30T17:13:48.234982+0000","last_active":"2024-05-30T17:13:48.234861+0000","last_peered":"2024-05-30T17:13:48.234861+0000","last_clean":"2024-05-30T17:13:48.234861+0000","last_became_active":"2024-05-30T16:12:50.813741+0000","last_became_peered":"2024-05-30T16:12:50.813741+0000","last_unstale":"2024-05-30T17:13:48.234861+0000","last_undegraded":"2024-05-30T17:13:48.234861+0000","last_fullsized":"2024-05-30T17:13:48.234861+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'15","last_scrub_stamp":"2024-05-30T17:13:48.234638+0000","last_deep_scrub":"265'15","last_deep_scrub_stamp":"2024-05-30T17:13:48.234638+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.234638+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:15:02.442194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":53,"num_write":33,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":39,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:36.967495+0000","last_change":"2024-05-30T17:14:36.967555+0000","last_active":"2024-05-30T17:14:36.967495+0000","last_peered":"2024-05-30T17:14:36.967495+0000","last_clean":"2024-05-30T17:14:36.967495+0000","last_became_active":"2024-05-30T16:34:55.858872+0000","last_became_peered":"2024-05-30T16:34:55.858872+0000","last_unstale":"2024-05-30T17:14:36.967495+0000","last_undegraded":"2024-05-30T17:14:36.967495+0000","last_fullsized":"2024-05-30T17:14:36.967495+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:36.967285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:36.967285+0000","last_clean_scrub_stamp":"2024-05-30T17:14:36.967285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:33:23.656589+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"0'0","reported_seq":59,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:27.365550+0000","last_change":"2024-05-30T17:14:27.365614+0000","last_active":"2024-05-30T17:14:27.365550+0000","last_peered":"2024-05-30T17:14:27.365550+0000","last_clean":"2024-05-30T17:14:27.365550+0000","last_became_active":"2024-05-30T16:17:35.226829+0000","last_became_peered":"2024-05-30T16:17:35.226829+0000","last_unstale":"2024-05-30T17:14:27.365550+0000","last_undegraded":"2024-05-30T17:14:27.365550+0000","last_fullsized":"2024-05-30T17:14:27.365550+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:27.365359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:27.365359+0000","last_clean_scrub_stamp":"2024-05-30T17:14:27.365359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:59:10.172045+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"0'0","reported_seq":64,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.486126+0000","last_change":"2024-05-30T17:13:48.486182+0000","last_active":"2024-05-30T17:13:48.486126+0000","last_peered":"2024-05-30T17:13:48.486126+0000","last_clean":"2024-05-30T17:13:48.486126+0000","last_became_active":"2024-05-30T16:17:31.203535+0000","last_became_peered":"2024-05-30T16:17:31.203535+0000","last_unstale":"2024-05-30T17:13:48.486126+0000","last_undegraded":"2024-05-30T17:13:48.486126+0000","last_fullsized":"2024-05-30T17:13:48.486126+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:48.485936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:48.485936+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.485936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:42:58.343268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"265'36","reported_seq":166,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.750202+0000","last_change":"2024-05-30T17:13:43.750267+0000","last_active":"2024-05-30T17:13:43.750202+0000","last_peered":"2024-05-30T17:13:43.750202+0000","last_clean":"2024-05-30T17:13:43.750202+0000","last_became_active":"2024-05-30T16:12:50.831842+0000","last_became_peered":"2024-05-30T16:12:50.831842+0000","last_unstale":"2024-05-30T17:13:43.750202+0000","last_undegraded":"2024-05-30T17:13:43.750202+0000","last_fullsized":"2024-05-30T17:13:43.750202+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'36","last_scrub_stamp":"2024-05-30T17:13:43.750005+0000","last_deep_scrub":"265'36","last_deep_scrub_stamp":"2024-05-30T17:13:43.750005+0000","last_clean_scrub_stamp":"2024-05-30T17:13:43.750005+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:20:47.390125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":177,"num_read_kb":125,"num_write":75,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":40,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:38.985535+0000","last_change":"2024-05-30T17:14:38.985674+0000","last_active":"2024-05-30T17:14:38.985535+0000","last_peered":"2024-05-30T17:14:38.985535+0000","last_clean":"2024-05-30T17:14:38.985535+0000","last_became_active":"2024-05-30T16:34:55.862709+0000","last_became_peered":"2024-05-30T16:34:55.862709+0000","last_unstale":"2024-05-30T17:14:38.985535+0000","last_undegraded":"2024-05-30T17:14:38.985535+0000","last_fullsized":"2024-05-30T17:14:38.985535+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:38.985285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:38.985285+0000","last_clean_scrub_stamp":"2024-05-30T17:14:38.985285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T05:10:37.092995+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"265'3","reported_seq":61,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:18.813164+0000","last_change":"2024-05-30T17:14:18.813250+0000","last_active":"2024-05-30T17:14:18.813164+0000","last_peered":"2024-05-30T17:14:18.813164+0000","last_clean":"2024-05-30T17:14:18.813164+0000","last_became_active":"2024-05-30T16:17:35.240472+0000","last_became_peered":"2024-05-30T16:17:35.240472+0000","last_unstale":"2024-05-30T17:14:18.813164+0000","last_undegraded":"2024-05-30T17:14:18.813164+0000","last_fullsized":"2024-05-30T17:14:18.813164+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:14:18.812962+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:14:18.812962+0000","last_clean_scrub_stamp":"2024-05-30T17:14:18.812962+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:06:37.629144+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"265'6","reported_seq":74,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.031484+0000","last_change":"2024-05-30T17:13:44.031484+0000","last_active":"2024-05-30T17:13:44.031484+0000","last_peered":"2024-05-30T17:13:44.031484+0000","last_clean":"2024-05-30T17:13:44.031484+0000","last_became_active":"2024-05-30T16:17:31.187958+0000","last_became_peered":"2024-05-30T16:17:31.187958+0000","last_unstale":"2024-05-30T17:13:44.031484+0000","last_undegraded":"2024-05-30T17:13:44.031484+0000","last_fullsized":"2024-05-30T17:13:44.031484+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'6","last_scrub_stamp":"2024-05-30T17:13:44.031239+0000","last_deep_scrub":"265'6","last_deep_scrub_stamp":"2024-05-30T17:13:44.031239+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.031239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:29:30.262819+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":10240,"num_write":16,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"265'18","reported_seq":121,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.918092+0000","last_change":"2024-05-30T17:13:44.918156+0000","last_active":"2024-05-30T17:13:44.918092+0000","last_peered":"2024-05-30T17:13:44.918092+0000","last_clean":"2024-05-30T17:13:44.918092+0000","last_became_active":"2024-05-30T16:12:50.823090+0000","last_became_peered":"2024-05-30T16:12:50.823090+0000","last_unstale":"2024-05-30T17:13:44.918092+0000","last_undegraded":"2024-05-30T17:13:44.918092+0000","last_fullsized":"2024-05-30T17:13:44.918092+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'18","last_scrub_stamp":"2024-05-30T17:13:44.917823+0000","last_deep_scrub":"265'18","last_deep_scrub_stamp":"2024-05-30T17:13:44.917823+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.917823+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:31:54.618562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":69,"num_write":35,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":40,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:39.690982+0000","last_change":"2024-05-30T17:14:39.690982+0000","last_active":"2024-05-30T17:14:39.690982+0000","last_peered":"2024-05-30T17:14:39.690982+0000","last_clean":"2024-05-30T17:14:39.690982+0000","last_became_active":"2024-05-30T16:34:55.890688+0000","last_became_peered":"2024-05-30T16:34:55.890688+0000","last_unstale":"2024-05-30T17:14:39.690982+0000","last_undegraded":"2024-05-30T17:14:39.690982+0000","last_fullsized":"2024-05-30T17:14:39.690982+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:39.690715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:39.690715+0000","last_clean_scrub_stamp":"2024-05-30T17:14:39.690715+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:31:19.278084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"265'9","reported_seq":77,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:28.055219+0000","last_change":"2024-05-30T17:14:28.055289+0000","last_active":"2024-05-30T17:14:28.055219+0000","last_peered":"2024-05-30T17:14:28.055219+0000","last_clean":"2024-05-30T17:14:28.055219+0000","last_became_active":"2024-05-30T16:17:35.237031+0000","last_became_peered":"2024-05-30T16:17:35.237031+0000","last_unstale":"2024-05-30T17:14:28.055219+0000","last_undegraded":"2024-05-30T17:14:28.055219+0000","last_fullsized":"2024-05-30T17:14:28.055219+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'9","last_scrub_stamp":"2024-05-30T17:14:28.055000+0000","last_deep_scrub":"265'9","last_deep_scrub_stamp":"2024-05-30T17:14:28.055000+0000","last_clean_scrub_stamp":"2024-05-30T17:14:28.055000+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:27:54.245671+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":20482,"num_write":17,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"265'6","reported_seq":76,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.818137+0000","last_change":"2024-05-30T17:13:48.818281+0000","last_active":"2024-05-30T17:13:48.818137+0000","last_peered":"2024-05-30T17:13:48.818137+0000","last_clean":"2024-05-30T17:13:48.818137+0000","last_became_active":"2024-05-30T16:17:31.203871+0000","last_became_peered":"2024-05-30T16:17:31.203871+0000","last_unstale":"2024-05-30T17:13:48.818137+0000","last_undegraded":"2024-05-30T17:13:48.818137+0000","last_fullsized":"2024-05-30T17:13:48.818137+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'6","last_scrub_stamp":"2024-05-30T17:13:48.817847+0000","last_deep_scrub":"265'6","last_deep_scrub_stamp":"2024-05-30T17:13:48.817847+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.817847+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:26:36.664287+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":4098,"num_write":9,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"265'24","reported_seq":142,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.471273+0000","last_change":"2024-05-30T17:13:51.471404+0000","last_active":"2024-05-30T17:13:51.471273+0000","last_peered":"2024-05-30T17:13:51.471273+0000","last_clean":"2024-05-30T17:13:51.471273+0000","last_became_active":"2024-05-30T16:12:50.827078+0000","last_became_peered":"2024-05-30T16:12:50.827078+0000","last_unstale":"2024-05-30T17:13:51.471273+0000","last_undegraded":"2024-05-30T17:13:51.471273+0000","last_fullsized":"2024-05-30T17:13:51.471273+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'24","last_scrub_stamp":"2024-05-30T17:13:51.471027+0000","last_deep_scrub":"265'24","last_deep_scrub_stamp":"2024-05-30T17:13:51.471027+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.471027+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:41:56.836293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":93,"num_write":46,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"265'165","reported_seq":785,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:25.765745+0000","last_change":"2024-05-30T17:14:25.765895+0000","last_active":"2024-05-30T17:14:25.765745+0000","last_peered":"2024-05-30T17:14:25.765745+0000","last_clean":"2024-05-30T17:14:25.765745+0000","last_became_active":"2024-05-30T15:54:16.661334+0000","last_became_peered":"2024-05-30T15:54:16.661334+0000","last_unstale":"2024-05-30T17:14:25.765745+0000","last_undegraded":"2024-05-30T17:14:25.765745+0000","last_fullsized":"2024-05-30T17:14:25.765745+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'165","last_scrub_stamp":"2024-05-30T17:14:25.765539+0000","last_deep_scrub":"265'165","last_deep_scrub_stamp":"2024-05-30T17:14:25.765539+0000","last_clean_scrub_stamp":"2024-05-30T17:14:25.765539+0000","objects_scrubbed":4,"log_size":65,"log_dups_size":100,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:26:13.701660+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2159792,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":347,"num_read_kb":7638,"num_write":1029,"num_write_kb":24692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"265'441","reported_seq":2084,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:06.288519+0000","last_change":"2024-05-30T17:14:06.288519+0000","last_active":"2024-05-30T17:14:06.288519+0000","last_peered":"2024-05-30T17:14:06.288519+0000","last_clean":"2024-05-30T17:14:06.288519+0000","last_became_active":"2024-05-30T15:54:18.669016+0000","last_became_peered":"2024-05-30T15:54:18.669016+0000","last_unstale":"2024-05-30T17:14:06.288519+0000","last_undegraded":"2024-05-30T17:14:06.288519+0000","last_fullsized":"2024-05-30T17:14:06.288519+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'441","last_scrub_stamp":"2024-05-30T17:14:06.288225+0000","last_deep_scrub":"265'441","last_deep_scrub_stamp":"2024-05-30T17:14:06.288225+0000","last_clean_scrub_stamp":"2024-05-30T17:14:06.288225+0000","objects_scrubbed":5,"log_size":41,"log_dups_size":400,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:34:29.289562+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2479,"num_read_kb":2366,"num_write":506,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"265'234","reported_seq":898,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:15.986498+0000","last_change":"2024-05-30T17:14:15.986759+0000","last_active":"2024-05-30T17:14:15.986498+0000","last_peered":"2024-05-30T17:14:15.986498+0000","last_clean":"2024-05-30T17:14:15.986498+0000","last_became_active":"2024-05-30T15:54:16.652724+0000","last_became_peered":"2024-05-30T15:54:16.652724+0000","last_unstale":"2024-05-30T17:14:15.986498+0000","last_undegraded":"2024-05-30T17:14:15.986498+0000","last_fullsized":"2024-05-30T17:14:15.986498+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'234","last_scrub_stamp":"2024-05-30T17:14:15.986234+0000","last_deep_scrub":"265'234","last_deep_scrub_stamp":"2024-05-30T17:14:15.986234+0000","last_clean_scrub_stamp":"2024-05-30T17:14:15.986234+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":200,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:24:51.351668+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6513950,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":554,"num_read_kb":8631,"num_write":1281,"num_write_kb":24636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"265'762","reported_seq":2127,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:15.778412+0000","last_change":"2024-05-30T17:14:15.778412+0000","last_active":"2024-05-30T17:14:15.778412+0000","last_peered":"2024-05-30T17:14:15.778412+0000","last_clean":"2024-05-30T17:14:15.778412+0000","last_became_active":"2024-05-30T15:54:18.648675+0000","last_became_peered":"2024-05-30T15:54:18.648675+0000","last_unstale":"2024-05-30T17:14:15.778412+0000","last_undegraded":"2024-05-30T17:14:15.778412+0000","last_fullsized":"2024-05-30T17:14:15.778412+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'762","last_scrub_stamp":"2024-05-30T17:14:15.778159+0000","last_deep_scrub":"265'762","last_deep_scrub_stamp":"2024-05-30T17:14:15.778159+0000","last_clean_scrub_stamp":"2024-05-30T17:14:15.778159+0000","objects_scrubbed":4,"log_size":62,"log_dups_size":700,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:29:08.620212+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3292,"num_read_kb":2905,"num_write":1031,"num_write_kb":415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"265'208","reported_seq":849,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:26.856702+0000","last_change":"2024-05-30T17:14:26.856841+0000","last_active":"2024-05-30T17:14:26.856702+0000","last_peered":"2024-05-30T17:14:26.856702+0000","last_clean":"2024-05-30T17:14:26.856702+0000","last_became_active":"2024-05-30T15:54:16.633011+0000","last_became_peered":"2024-05-30T15:54:16.633011+0000","last_unstale":"2024-05-30T17:14:26.856702+0000","last_undegraded":"2024-05-30T17:14:26.856702+0000","last_fullsized":"2024-05-30T17:14:26.856702+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'208","last_scrub_stamp":"2024-05-30T17:14:26.856421+0000","last_deep_scrub":"265'208","last_deep_scrub_stamp":"2024-05-30T17:14:26.856421+0000","last_clean_scrub_stamp":"2024-05-30T17:14:26.856421+0000","objects_scrubbed":9,"log_size":108,"log_dups_size":100,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:28:47.270416+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11855586,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":404,"num_read_kb":43860,"num_write":998,"num_write_kb":74051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"265'747","reported_seq":2155,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:14.074411+0000","last_change":"2024-05-30T17:14:14.074411+0000","last_active":"2024-05-30T17:14:14.074411+0000","last_peered":"2024-05-30T17:14:14.074411+0000","last_clean":"2024-05-30T17:14:14.074411+0000","last_became_active":"2024-05-30T15:54:18.649608+0000","last_became_peered":"2024-05-30T15:54:18.649608+0000","last_unstale":"2024-05-30T17:14:14.074411+0000","last_undegraded":"2024-05-30T17:14:14.074411+0000","last_fullsized":"2024-05-30T17:14:14.074411+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'747","last_scrub_stamp":"2024-05-30T17:14:14.074153+0000","last_deep_scrub":"265'747","last_deep_scrub_stamp":"2024-05-30T17:14:14.074153+0000","last_clean_scrub_stamp":"2024-05-30T17:14:14.074153+0000","objects_scrubbed":1,"log_size":47,"log_dups_size":700,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:05:55.305461+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3275,"num_read_kb":2909,"num_write":996,"num_write_kb":378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"265'154","reported_seq":802,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:22.300200+0000","last_change":"2024-05-30T17:14:22.300200+0000","last_active":"2024-05-30T17:14:22.300200+0000","last_peered":"2024-05-30T17:14:22.300200+0000","last_clean":"2024-05-30T17:14:22.300200+0000","last_became_active":"2024-05-30T15:54:16.652315+0000","last_became_peered":"2024-05-30T15:54:16.652315+0000","last_unstale":"2024-05-30T17:14:22.300200+0000","last_undegraded":"2024-05-30T17:14:22.300200+0000","last_fullsized":"2024-05-30T17:14:22.300200+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'154","last_scrub_stamp":"2024-05-30T17:14:22.299903+0000","last_deep_scrub":"265'154","last_deep_scrub_stamp":"2024-05-30T17:14:22.299903+0000","last_clean_scrub_stamp":"2024-05-30T17:14:22.299903+0000","objects_scrubbed":3,"log_size":54,"log_dups_size":100,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:34:02.246778+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1250479,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":437,"num_read_kb":3484,"num_write":911,"num_write_kb":6490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"265'490","reported_seq":1975,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:12.850064+0000","last_change":"2024-05-30T17:14:12.850064+0000","last_active":"2024-05-30T17:14:12.850064+0000","last_peered":"2024-05-30T17:14:12.850064+0000","last_clean":"2024-05-30T17:14:12.850064+0000","last_became_active":"2024-05-30T15:54:18.652451+0000","last_became_peered":"2024-05-30T15:54:18.652451+0000","last_unstale":"2024-05-30T17:14:12.850064+0000","last_undegraded":"2024-05-30T17:14:12.850064+0000","last_fullsized":"2024-05-30T17:14:12.850064+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'490","last_scrub_stamp":"2024-05-30T17:14:12.849644+0000","last_deep_scrub":"265'490","last_deep_scrub_stamp":"2024-05-30T17:14:12.849644+0000","last_clean_scrub_stamp":"2024-05-30T17:14:12.849644+0000","objects_scrubbed":3,"log_size":90,"log_dups_size":400,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:33:13.009731+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2387,"num_read_kb":2188,"num_write":616,"num_write_kb":168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":505,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"265'245","reported_seq":969,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:31.797437+0000","last_change":"2024-05-30T17:14:31.797533+0000","last_active":"2024-05-30T17:14:31.797437+0000","last_peered":"2024-05-30T17:14:31.797437+0000","last_clean":"2024-05-30T17:14:31.797437+0000","last_became_active":"2024-05-30T15:54:16.658338+0000","last_became_peered":"2024-05-30T15:54:16.658338+0000","last_unstale":"2024-05-30T17:14:31.797437+0000","last_undegraded":"2024-05-30T17:14:31.797437+0000","last_fullsized":"2024-05-30T17:14:31.797437+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'245","last_scrub_stamp":"2024-05-30T17:14:31.797211+0000","last_deep_scrub":"265'245","last_deep_scrub_stamp":"2024-05-30T17:14:31.797211+0000","last_clean_scrub_stamp":"2024-05-30T17:14:31.797211+0000","objects_scrubbed":5,"log_size":45,"log_dups_size":200,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:20:44.257811+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291491,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":830,"num_read_kb":24681,"num_write":1246,"num_write_kb":34031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"265'378","reported_seq":1880,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:05.719014+0000","last_change":"2024-05-30T17:14:05.719014+0000","last_active":"2024-05-30T17:14:05.719014+0000","last_peered":"2024-05-30T17:14:05.719014+0000","last_clean":"2024-05-30T17:14:05.719014+0000","last_became_active":"2024-05-30T15:54:18.669017+0000","last_became_peered":"2024-05-30T15:54:18.669017+0000","last_unstale":"2024-05-30T17:14:05.719014+0000","last_undegraded":"2024-05-30T17:14:05.719014+0000","last_fullsized":"2024-05-30T17:14:05.719014+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'378","last_scrub_stamp":"2024-05-30T17:14:05.718728+0000","last_deep_scrub":"265'378","last_deep_scrub_stamp":"2024-05-30T17:14:05.718728+0000","last_clean_scrub_stamp":"2024-05-30T17:14:05.718728+0000","objects_scrubbed":8,"log_size":78,"log_dups_size":300,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:31:23.767712+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1986,"num_read_kb":1906,"num_write":419,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":465,"num_omap_keys":3,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"265'196","reported_seq":831,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:15.302437+0000","last_change":"2024-05-30T17:14:15.302437+0000","last_active":"2024-05-30T17:14:15.302437+0000","last_peered":"2024-05-30T17:14:15.302437+0000","last_clean":"2024-05-30T17:14:15.302437+0000","last_became_active":"2024-05-30T15:54:16.671120+0000","last_became_peered":"2024-05-30T15:54:16.671120+0000","last_unstale":"2024-05-30T17:14:15.302437+0000","last_undegraded":"2024-05-30T17:14:15.302437+0000","last_fullsized":"2024-05-30T17:14:15.302437+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'196","last_scrub_stamp":"2024-05-30T17:14:15.302156+0000","last_deep_scrub":"265'196","last_deep_scrub_stamp":"2024-05-30T17:14:15.302156+0000","last_clean_scrub_stamp":"2024-05-30T17:14:15.302156+0000","objects_scrubbed":2,"log_size":96,"log_dups_size":100,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:02:53.975403+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":81,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":438,"num_read_kb":4718,"num_write":1101,"num_write_kb":20583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"265'727","reported_seq":2212,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:16.031885+0000","last_change":"2024-05-30T17:14:16.032031+0000","last_active":"2024-05-30T17:14:16.031885+0000","last_peered":"2024-05-30T17:14:16.031885+0000","last_clean":"2024-05-30T17:14:16.031885+0000","last_became_active":"2024-05-30T15:54:18.650543+0000","last_became_peered":"2024-05-30T15:54:18.650543+0000","last_unstale":"2024-05-30T17:14:16.031885+0000","last_undegraded":"2024-05-30T17:14:16.031885+0000","last_fullsized":"2024-05-30T17:14:16.031885+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'727","last_scrub_stamp":"2024-05-30T17:14:16.031640+0000","last_deep_scrub":"265'727","last_deep_scrub_stamp":"2024-05-30T17:14:16.031640+0000","last_clean_scrub_stamp":"2024-05-30T17:14:16.031640+0000","objects_scrubbed":2,"log_size":27,"log_dups_size":700,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:30:02.474759+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3282,"num_read_kb":2952,"num_write":973,"num_write_kb":356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"265'192","reported_seq":842,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:17.822847+0000","last_change":"2024-05-30T17:14:17.822942+0000","last_active":"2024-05-30T17:14:17.822847+0000","last_peered":"2024-05-30T17:14:17.822847+0000","last_clean":"2024-05-30T17:14:17.822847+0000","last_became_active":"2024-05-30T15:54:16.678799+0000","last_became_peered":"2024-05-30T15:54:16.678799+0000","last_unstale":"2024-05-30T17:14:17.822847+0000","last_undegraded":"2024-05-30T17:14:17.822847+0000","last_fullsized":"2024-05-30T17:14:17.822847+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'192","last_scrub_stamp":"2024-05-30T17:14:17.822619+0000","last_deep_scrub":"265'192","last_deep_scrub_stamp":"2024-05-30T17:14:17.822619+0000","last_clean_scrub_stamp":"2024-05-30T17:14:17.822619+0000","objects_scrubbed":4,"log_size":92,"log_dups_size":100,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:01:36.730473+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":491377,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":467,"num_read_kb":33008,"num_write":1090,"num_write_kb":27021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"265'851","reported_seq":2549,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:07.288471+0000","last_change":"2024-05-30T17:14:07.288543+0000","last_active":"2024-05-30T17:14:07.288471+0000","last_peered":"2024-05-30T17:14:07.288471+0000","last_clean":"2024-05-30T17:14:07.288471+0000","last_became_active":"2024-05-30T15:54:18.656475+0000","last_became_peered":"2024-05-30T15:54:18.656475+0000","last_unstale":"2024-05-30T17:14:07.288471+0000","last_undegraded":"2024-05-30T17:14:07.288471+0000","last_fullsized":"2024-05-30T17:14:07.288471+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'851","last_scrub_stamp":"2024-05-30T17:14:07.288276+0000","last_deep_scrub":"265'851","last_deep_scrub_stamp":"2024-05-30T17:14:07.288276+0000","last_clean_scrub_stamp":"2024-05-30T17:14:07.288276+0000","objects_scrubbed":1,"log_size":51,"log_dups_size":800,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:53:16.054837+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4088,"num_read_kb":3692,"num_write":1166,"num_write_kb":472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"265'140","reported_seq":746,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:34.449595+0000","last_change":"2024-05-30T17:14:34.449675+0000","last_active":"2024-05-30T17:14:34.449595+0000","last_peered":"2024-05-30T17:14:34.449595+0000","last_clean":"2024-05-30T17:14:34.449595+0000","last_became_active":"2024-05-30T15:54:16.669095+0000","last_became_peered":"2024-05-30T15:54:16.669095+0000","last_unstale":"2024-05-30T17:14:34.449595+0000","last_undegraded":"2024-05-30T17:14:34.449595+0000","last_fullsized":"2024-05-30T17:14:34.449595+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'140","last_scrub_stamp":"2024-05-30T17:14:34.449388+0000","last_deep_scrub":"265'140","last_deep_scrub_stamp":"2024-05-30T17:14:34.449388+0000","last_clean_scrub_stamp":"2024-05-30T17:14:34.449388+0000","objects_scrubbed":2,"log_size":40,"log_dups_size":100,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:07:00.707956+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5243043,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":303,"num_read_kb":17751,"num_write":817,"num_write_kb":30378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"265'373","reported_seq":1803,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:05.355282+0000","last_change":"2024-05-30T17:14:05.355364+0000","last_active":"2024-05-30T17:14:05.355282+0000","last_peered":"2024-05-30T17:14:05.355282+0000","last_clean":"2024-05-30T17:14:05.355282+0000","last_became_active":"2024-05-30T15:54:18.653189+0000","last_became_peered":"2024-05-30T15:54:18.653189+0000","last_unstale":"2024-05-30T17:14:05.355282+0000","last_undegraded":"2024-05-30T17:14:05.355282+0000","last_fullsized":"2024-05-30T17:14:05.355282+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'373","last_scrub_stamp":"2024-05-30T17:14:05.355099+0000","last_deep_scrub":"265'373","last_deep_scrub_stamp":"2024-05-30T17:14:05.355099+0000","last_clean_scrub_stamp":"2024-05-30T17:14:05.355099+0000","objects_scrubbed":4,"log_size":73,"log_dups_size":300,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:26:58.416740+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1969,"num_read_kb":1855,"num_write":435,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"265'340","reported_seq":1778,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:12.082673+0000","last_change":"2024-05-30T17:14:12.082843+0000","last_active":"2024-05-30T17:14:12.082673+0000","last_peered":"2024-05-30T17:14:12.082673+0000","last_clean":"2024-05-30T17:14:12.082673+0000","last_became_active":"2024-05-30T15:54:18.649994+0000","last_became_peered":"2024-05-30T15:54:18.649994+0000","last_unstale":"2024-05-30T17:14:12.082673+0000","last_undegraded":"2024-05-30T17:14:12.082673+0000","last_fullsized":"2024-05-30T17:14:12.082673+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'340","last_scrub_stamp":"2024-05-30T17:14:12.082448+0000","last_deep_scrub":"265'340","last_deep_scrub_stamp":"2024-05-30T17:14:12.082448+0000","last_clean_scrub_stamp":"2024-05-30T17:14:12.082448+0000","objects_scrubbed":4,"log_size":40,"log_dups_size":300,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:35:49.760153+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1976,"num_read_kb":1893,"num_write":388,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":249,"num_omap_keys":1,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"265'212","reported_seq":887,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:30.794777+0000","last_change":"2024-05-30T17:14:30.794874+0000","last_active":"2024-05-30T17:14:30.794777+0000","last_peered":"2024-05-30T17:14:30.794777+0000","last_clean":"2024-05-30T17:14:30.794777+0000","last_became_active":"2024-05-30T15:54:16.649872+0000","last_became_peered":"2024-05-30T15:54:16.649872+0000","last_unstale":"2024-05-30T17:14:30.794777+0000","last_undegraded":"2024-05-30T17:14:30.794777+0000","last_fullsized":"2024-05-30T17:14:30.794777+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'212","last_scrub_stamp":"2024-05-30T17:14:30.794580+0000","last_deep_scrub":"265'212","last_deep_scrub_stamp":"2024-05-30T17:14:30.794580+0000","last_clean_scrub_stamp":"2024-05-30T17:14:30.794580+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:09:42.032254+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33349,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":510,"num_read_kb":59018,"num_write":1166,"num_write_kb":23092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"265'1526","reported_seq":3141,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:13.760005+0000","last_change":"2024-05-30T17:14:13.760080+0000","last_active":"2024-05-30T17:14:13.760005+0000","last_peered":"2024-05-30T17:14:13.760005+0000","last_clean":"2024-05-30T17:14:13.760005+0000","last_became_active":"2024-05-30T15:54:18.648860+0000","last_became_peered":"2024-05-30T15:54:18.648860+0000","last_unstale":"2024-05-30T17:14:13.760005+0000","last_undegraded":"2024-05-30T17:14:13.760005+0000","last_fullsized":"2024-05-30T17:14:13.760005+0000","mapping_epoch":24,"log_start":"265'1500","ondisk_log_start":"265'1500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'1526","last_scrub_stamp":"2024-05-30T17:14:13.759812+0000","last_deep_scrub":"265'1526","last_deep_scrub_stamp":"2024-05-30T17:14:13.759812+0000","last_clean_scrub_stamp":"2024-05-30T17:14:13.759812+0000","objects_scrubbed":1,"log_size":26,"log_dups_size":1500,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:47:16.863579+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6132,"num_read_kb":5297,"num_write":2138,"num_write_kb":912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"265'221","reported_seq":1093,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:13.810620+0000","last_change":"2024-05-30T17:14:13.810876+0000","last_active":"2024-05-30T17:14:13.810620+0000","last_peered":"2024-05-30T17:14:13.810620+0000","last_clean":"2024-05-30T17:14:13.810620+0000","last_became_active":"2024-05-30T15:54:16.686847+0000","last_became_peered":"2024-05-30T15:54:16.686847+0000","last_unstale":"2024-05-30T17:14:13.810620+0000","last_undegraded":"2024-05-30T17:14:13.810620+0000","last_fullsized":"2024-05-30T17:14:13.810620+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'221","last_scrub_stamp":"2024-05-30T17:14:13.810368+0000","last_deep_scrub":"265'221","last_deep_scrub_stamp":"2024-05-30T17:14:13.810368+0000","last_clean_scrub_stamp":"2024-05-30T17:14:13.810368+0000","objects_scrubbed":5,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:53:55.581906+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2728232,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":925,"num_read_kb":24537,"num_write":1233,"num_write_kb":54861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"265'1169","reported_seq":3018,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:04.292081+0000","last_change":"2024-05-30T17:14:04.292155+0000","last_active":"2024-05-30T17:14:04.292081+0000","last_peered":"2024-05-30T17:14:04.292081+0000","last_clean":"2024-05-30T17:14:04.292081+0000","last_became_active":"2024-05-30T15:54:18.665302+0000","last_became_peered":"2024-05-30T15:54:18.665302+0000","last_unstale":"2024-05-30T17:14:04.292081+0000","last_undegraded":"2024-05-30T17:14:04.292081+0000","last_fullsized":"2024-05-30T17:14:04.292081+0000","mapping_epoch":24,"log_start":"265'1100","ondisk_log_start":"265'1100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'1169","last_scrub_stamp":"2024-05-30T17:14:04.291893+0000","last_deep_scrub":"265'1169","last_deep_scrub_stamp":"2024-05-30T17:14:04.291893+0000","last_clean_scrub_stamp":"2024-05-30T17:14:04.291893+0000","objects_scrubbed":5,"log_size":69,"log_dups_size":1100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:33:21.886845+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5292,"num_read_kb":4714,"num_write":1593,"num_write_kb":642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3544,"num_omap_keys":14,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"265'197","reported_seq":928,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:08.780887+0000","last_change":"2024-05-30T17:14:08.781004+0000","last_active":"2024-05-30T17:14:08.780887+0000","last_peered":"2024-05-30T17:14:08.780887+0000","last_clean":"2024-05-30T17:14:08.780887+0000","last_became_active":"2024-05-30T15:54:16.668438+0000","last_became_peered":"2024-05-30T15:54:16.668438+0000","last_unstale":"2024-05-30T17:14:08.780887+0000","last_undegraded":"2024-05-30T17:14:08.780887+0000","last_fullsized":"2024-05-30T17:14:08.780887+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'197","last_scrub_stamp":"2024-05-30T17:14:08.780668+0000","last_deep_scrub":"265'197","last_deep_scrub_stamp":"2024-05-30T17:14:08.780668+0000","last_clean_scrub_stamp":"2024-05-30T17:14:08.780668+0000","objects_scrubbed":5,"log_size":97,"log_dups_size":100,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:59:06.022425+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":918,"num_read_kb":8276,"num_write":1173,"num_write_kb":35310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"265'772","reported_seq":2296,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:04.332160+0000","last_change":"2024-05-30T17:14:04.332160+0000","last_active":"2024-05-30T17:14:04.332160+0000","last_peered":"2024-05-30T17:14:04.332160+0000","last_clean":"2024-05-30T17:14:04.332160+0000","last_became_active":"2024-05-30T15:54:18.651197+0000","last_became_peered":"2024-05-30T15:54:18.651197+0000","last_unstale":"2024-05-30T17:14:04.332160+0000","last_undegraded":"2024-05-30T17:14:04.332160+0000","last_fullsized":"2024-05-30T17:14:04.332160+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'772","last_scrub_stamp":"2024-05-30T17:14:04.331893+0000","last_deep_scrub":"265'772","last_deep_scrub_stamp":"2024-05-30T17:14:04.331893+0000","last_clean_scrub_stamp":"2024-05-30T17:14:04.331893+0000","objects_scrubbed":4,"log_size":72,"log_dups_size":700,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:54:58.149438+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3544,"num_read_kb":3171,"num_write":1051,"num_write_kb":404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"265'190","reported_seq":852,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:20.131165+0000","last_change":"2024-05-30T17:14:20.131261+0000","last_active":"2024-05-30T17:14:20.131165+0000","last_peered":"2024-05-30T17:14:20.131165+0000","last_clean":"2024-05-30T17:14:20.131165+0000","last_became_active":"2024-05-30T15:54:16.676703+0000","last_became_peered":"2024-05-30T15:54:16.676703+0000","last_unstale":"2024-05-30T17:14:20.131165+0000","last_undegraded":"2024-05-30T17:14:20.131165+0000","last_fullsized":"2024-05-30T17:14:20.131165+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'190","last_scrub_stamp":"2024-05-30T17:14:20.130939+0000","last_deep_scrub":"265'190","last_deep_scrub_stamp":"2024-05-30T17:14:20.130939+0000","last_clean_scrub_stamp":"2024-05-30T17:14:20.130939+0000","objects_scrubbed":3,"log_size":90,"log_dups_size":100,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:16:27.631887+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":366,"num_read_kb":73313,"num_write":1062,"num_write_kb":47694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"265'376","reported_seq":1838,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:03.319819+0000","last_change":"2024-05-30T17:14:03.319819+0000","last_active":"2024-05-30T17:14:03.319819+0000","last_peered":"2024-05-30T17:14:03.319819+0000","last_clean":"2024-05-30T17:14:03.319819+0000","last_became_active":"2024-05-30T15:54:18.650255+0000","last_became_peered":"2024-05-30T15:54:18.650255+0000","last_unstale":"2024-05-30T17:14:03.319819+0000","last_undegraded":"2024-05-30T17:14:03.319819+0000","last_fullsized":"2024-05-30T17:14:03.319819+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'376","last_scrub_stamp":"2024-05-30T17:14:03.319457+0000","last_deep_scrub":"265'376","last_deep_scrub_stamp":"2024-05-30T17:14:03.319457+0000","last_clean_scrub_stamp":"2024-05-30T17:14:03.319457+0000","objects_scrubbed":4,"log_size":76,"log_dups_size":300,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:24:42.352764+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1969,"num_read_kb":1871,"num_write":435,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1010,"num_omap_keys":4,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"265'203","reported_seq":864,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:32.402616+0000","last_change":"2024-05-30T17:14:32.402720+0000","last_active":"2024-05-30T17:14:32.402616+0000","last_peered":"2024-05-30T17:14:32.402616+0000","last_clean":"2024-05-30T17:14:32.402616+0000","last_became_active":"2024-05-30T15:54:16.665344+0000","last_became_peered":"2024-05-30T15:54:16.665344+0000","last_unstale":"2024-05-30T17:14:32.402616+0000","last_undegraded":"2024-05-30T17:14:32.402616+0000","last_fullsized":"2024-05-30T17:14:32.402616+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'203","last_scrub_stamp":"2024-05-30T17:14:32.402407+0000","last_deep_scrub":"265'203","last_deep_scrub_stamp":"2024-05-30T17:14:32.402407+0000","last_clean_scrub_stamp":"2024-05-30T17:14:32.402407+0000","objects_scrubbed":5,"log_size":103,"log_dups_size":100,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:47:10.414479+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6847332,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":477,"num_read_kb":7750,"num_write":1077,"num_write_kb":36544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"265'376","reported_seq":1811,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:24.134733+0000","last_change":"2024-05-30T17:14:24.134802+0000","last_active":"2024-05-30T17:14:24.134733+0000","last_peered":"2024-05-30T17:14:24.134733+0000","last_clean":"2024-05-30T17:14:24.134733+0000","last_became_active":"2024-05-30T15:54:18.643825+0000","last_became_peered":"2024-05-30T15:54:18.643825+0000","last_unstale":"2024-05-30T17:14:24.134733+0000","last_undegraded":"2024-05-30T17:14:24.134733+0000","last_fullsized":"2024-05-30T17:14:24.134733+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'376","last_scrub_stamp":"2024-05-30T17:14:24.134539+0000","last_deep_scrub":"265'376","last_deep_scrub_stamp":"2024-05-30T17:14:24.134539+0000","last_clean_scrub_stamp":"2024-05-30T17:14:24.134539+0000","objects_scrubbed":1,"log_size":76,"log_dups_size":300,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:48:05.206064+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1936,"num_read_kb":1833,"num_write":434,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"265'238","reported_seq":919,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:27.802082+0000","last_change":"2024-05-30T17:14:27.802182+0000","last_active":"2024-05-30T17:14:27.802082+0000","last_peered":"2024-05-30T17:14:27.802082+0000","last_clean":"2024-05-30T17:14:27.802082+0000","last_became_active":"2024-05-30T15:54:16.637928+0000","last_became_peered":"2024-05-30T15:54:16.637928+0000","last_unstale":"2024-05-30T17:14:27.802082+0000","last_undegraded":"2024-05-30T17:14:27.802082+0000","last_fullsized":"2024-05-30T17:14:27.802082+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'238","last_scrub_stamp":"2024-05-30T17:14:27.801850+0000","last_deep_scrub":"265'238","last_deep_scrub_stamp":"2024-05-30T17:14:27.801850+0000","last_clean_scrub_stamp":"2024-05-30T17:14:27.801850+0000","objects_scrubbed":3,"log_size":38,"log_dups_size":200,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:29:08.005106+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079922,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":526,"num_read_kb":65823,"num_write":1181,"num_write_kb":32044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"265'180","reported_seq":714,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.837308+0000","last_change":"2024-05-30T17:13:47.837308+0000","last_active":"2024-05-30T17:13:47.837308+0000","last_peered":"2024-05-30T17:13:47.837308+0000","last_clean":"2024-05-30T17:13:47.837308+0000","last_became_active":"2024-05-30T15:54:26.700957+0000","last_became_peered":"2024-05-30T15:54:26.700957+0000","last_unstale":"2024-05-30T17:13:47.837308+0000","last_undegraded":"2024-05-30T17:13:47.837308+0000","last_fullsized":"2024-05-30T17:13:47.837308+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'180","last_scrub_stamp":"2024-05-30T17:13:47.837040+0000","last_deep_scrub":"265'180","last_deep_scrub_stamp":"2024-05-30T17:13:47.837040+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.837040+0000","objects_scrubbed":5,"log_size":80,"log_dups_size":100,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:49:37.253715+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":270,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":132,"num_read_kb":126,"num_write":386,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"265'496","reported_seq":2033,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:18.368033+0000","last_change":"2024-05-30T17:14:18.368097+0000","last_active":"2024-05-30T17:14:18.368033+0000","last_peered":"2024-05-30T17:14:18.368033+0000","last_clean":"2024-05-30T17:14:18.368033+0000","last_became_active":"2024-05-30T15:54:18.652712+0000","last_became_peered":"2024-05-30T15:54:18.652712+0000","last_unstale":"2024-05-30T17:14:18.368033+0000","last_undegraded":"2024-05-30T17:14:18.368033+0000","last_fullsized":"2024-05-30T17:14:18.368033+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'496","last_scrub_stamp":"2024-05-30T17:14:18.367832+0000","last_deep_scrub":"265'496","last_deep_scrub_stamp":"2024-05-30T17:14:18.367832+0000","last_clean_scrub_stamp":"2024-05-30T17:14:18.367832+0000","objects_scrubbed":2,"log_size":96,"log_dups_size":400,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:39:11.375061+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2579,"num_read_kb":2377,"num_write":668,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":148,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"265'190","reported_seq":803,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.106668+0000","last_change":"2024-05-30T17:13:51.106756+0000","last_active":"2024-05-30T17:13:51.106668+0000","last_peered":"2024-05-30T17:13:51.106668+0000","last_clean":"2024-05-30T17:13:51.106668+0000","last_became_active":"2024-05-30T15:55:54.563874+0000","last_became_peered":"2024-05-30T15:55:54.563874+0000","last_unstale":"2024-05-30T17:13:51.106668+0000","last_undegraded":"2024-05-30T17:13:51.106668+0000","last_fullsized":"2024-05-30T17:13:51.106668+0000","mapping_epoch":37,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'190","last_scrub_stamp":"2024-05-30T17:13:51.106465+0000","last_deep_scrub":"265'190","last_deep_scrub_stamp":"2024-05-30T17:13:51.106465+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.106465+0000","objects_scrubbed":2,"log_size":90,"log_dups_size":100,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:21:06.302905+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079905,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":470,"num_read_kb":1589,"num_write":1089,"num_write_kb":27979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":516,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:12.375774+0000","last_change":"2024-05-30T17:14:12.375835+0000","last_active":"2024-05-30T17:14:12.375774+0000","last_peered":"2024-05-30T17:14:12.375774+0000","last_clean":"2024-05-30T17:14:12.375774+0000","last_became_active":"2024-05-30T15:54:20.658937+0000","last_became_peered":"2024-05-30T15:54:20.658937+0000","last_unstale":"2024-05-30T17:14:12.375774+0000","last_undegraded":"2024-05-30T17:14:12.375774+0000","last_fullsized":"2024-05-30T17:14:12.375774+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:12.375577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:12.375577+0000","last_clean_scrub_stamp":"2024-05-30T17:14:12.375577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:14:24.728898+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"265'206","reported_seq":842,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:23.889148+0000","last_change":"2024-05-30T17:14:23.889303+0000","last_active":"2024-05-30T17:14:23.889148+0000","last_peered":"2024-05-30T17:14:23.889148+0000","last_clean":"2024-05-30T17:14:23.889148+0000","last_became_active":"2024-05-30T15:54:22.667496+0000","last_became_peered":"2024-05-30T15:54:22.667496+0000","last_unstale":"2024-05-30T17:14:23.889148+0000","last_undegraded":"2024-05-30T17:14:23.889148+0000","last_fullsized":"2024-05-30T17:14:23.889148+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'206","last_scrub_stamp":"2024-05-30T17:14:23.888907+0000","last_deep_scrub":"265'206","last_deep_scrub_stamp":"2024-05-30T17:14:23.888907+0000","last_clean_scrub_stamp":"2024-05-30T17:14:23.888907+0000","objects_scrubbed":4,"log_size":106,"log_dups_size":100,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:36:09.311361+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6140,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":285,"num_read_kb":355,"num_write":168,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":508,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:06.781850+0000","last_change":"2024-05-30T17:14:06.781850+0000","last_active":"2024-05-30T17:14:06.781850+0000","last_peered":"2024-05-30T17:14:06.781850+0000","last_clean":"2024-05-30T17:14:06.781850+0000","last_became_active":"2024-05-30T15:54:24.674491+0000","last_became_peered":"2024-05-30T15:54:24.674491+0000","last_unstale":"2024-05-30T17:14:06.781850+0000","last_undegraded":"2024-05-30T17:14:06.781850+0000","last_fullsized":"2024-05-30T17:14:06.781850+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:06.781591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:06.781591+0000","last_clean_scrub_stamp":"2024-05-30T17:14:06.781591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:22:20.251425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"265'199","reported_seq":784,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.234943+0000","last_change":"2024-05-30T17:13:53.234943+0000","last_active":"2024-05-30T17:13:53.234943+0000","last_peered":"2024-05-30T17:13:53.234943+0000","last_clean":"2024-05-30T17:13:53.234943+0000","last_became_active":"2024-05-30T15:54:26.686430+0000","last_became_peered":"2024-05-30T15:54:26.686430+0000","last_unstale":"2024-05-30T17:13:53.234943+0000","last_undegraded":"2024-05-30T17:13:53.234943+0000","last_fullsized":"2024-05-30T17:13:53.234943+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'199","last_scrub_stamp":"2024-05-30T17:13:53.234506+0000","last_deep_scrub":"265'199","last_deep_scrub_stamp":"2024-05-30T17:13:53.234506+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.234506+0000","objects_scrubbed":5,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:07:57.530934+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":824,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":238,"num_read_kb":223,"num_write":274,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"265'512","reported_seq":2172,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:03.288156+0000","last_change":"2024-05-30T17:14:03.288156+0000","last_active":"2024-05-30T17:14:03.288156+0000","last_peered":"2024-05-30T17:14:03.288156+0000","last_clean":"2024-05-30T17:14:03.288156+0000","last_became_active":"2024-05-30T15:54:18.661762+0000","last_became_peered":"2024-05-30T15:54:18.661762+0000","last_unstale":"2024-05-30T17:14:03.288156+0000","last_undegraded":"2024-05-30T17:14:03.288156+0000","last_fullsized":"2024-05-30T17:14:03.288156+0000","mapping_epoch":24,"log_start":"265'501","ondisk_log_start":"265'501","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'512","last_scrub_stamp":"2024-05-30T17:14:03.287889+0000","last_deep_scrub":"265'512","last_deep_scrub_stamp":"2024-05-30T17:14:03.287889+0000","last_clean_scrub_stamp":"2024-05-30T17:14:03.287889+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":501,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:21:44.477336+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2657,"num_read_kb":2481,"num_write":627,"num_write_kb":155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"265'210","reported_seq":875,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:28.788740+0000","last_change":"2024-05-30T17:14:28.788833+0000","last_active":"2024-05-30T17:14:28.788740+0000","last_peered":"2024-05-30T17:14:28.788740+0000","last_clean":"2024-05-30T17:14:28.788740+0000","last_became_active":"2024-05-30T15:54:16.639284+0000","last_became_peered":"2024-05-30T15:54:16.639284+0000","last_unstale":"2024-05-30T17:14:28.788740+0000","last_undegraded":"2024-05-30T17:14:28.788740+0000","last_fullsized":"2024-05-30T17:14:28.788740+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'210","last_scrub_stamp":"2024-05-30T17:14:28.788504+0000","last_deep_scrub":"265'210","last_deep_scrub_stamp":"2024-05-30T17:14:28.788504+0000","last_clean_scrub_stamp":"2024-05-30T17:14:28.788504+0000","objects_scrubbed":2,"log_size":110,"log_dups_size":100,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:56:40.804738+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262178,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":445,"num_read_kb":15653,"num_write":1159,"num_write_kb":64581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":519,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:21.409846+0000","last_change":"2024-05-30T17:14:21.409904+0000","last_active":"2024-05-30T17:14:21.409846+0000","last_peered":"2024-05-30T17:14:21.409846+0000","last_clean":"2024-05-30T17:14:21.409846+0000","last_became_active":"2024-05-30T15:54:20.665488+0000","last_became_peered":"2024-05-30T15:54:20.665488+0000","last_unstale":"2024-05-30T17:14:21.409846+0000","last_undegraded":"2024-05-30T17:14:21.409846+0000","last_fullsized":"2024-05-30T17:14:21.409846+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:21.409644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:21.409644+0000","last_clean_scrub_stamp":"2024-05-30T17:14:21.409644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:41:07.021077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"265'105","reported_seq":666,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.970319+0000","last_change":"2024-05-30T17:13:59.970319+0000","last_active":"2024-05-30T17:13:59.970319+0000","last_peered":"2024-05-30T17:13:59.970319+0000","last_clean":"2024-05-30T17:13:59.970319+0000","last_became_active":"2024-05-30T15:54:22.666057+0000","last_became_peered":"2024-05-30T15:54:22.666057+0000","last_unstale":"2024-05-30T17:13:59.970319+0000","last_undegraded":"2024-05-30T17:13:59.970319+0000","last_fullsized":"2024-05-30T17:13:59.970319+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'105","last_scrub_stamp":"2024-05-30T17:13:59.969943+0000","last_deep_scrub":"265'105","last_deep_scrub_stamp":"2024-05-30T17:13:59.969943+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.969943+0000","objects_scrubbed":3,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:45:56.754937+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":113,"num_write":76,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":517,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:52.906758+0000","last_change":"2024-05-30T17:13:52.906758+0000","last_active":"2024-05-30T17:13:52.906758+0000","last_peered":"2024-05-30T17:13:52.906758+0000","last_clean":"2024-05-30T17:13:52.906758+0000","last_became_active":"2024-05-30T15:54:24.686595+0000","last_became_peered":"2024-05-30T15:54:24.686595+0000","last_unstale":"2024-05-30T17:13:52.906758+0000","last_undegraded":"2024-05-30T17:13:52.906758+0000","last_fullsized":"2024-05-30T17:13:52.906758+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:52.906486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:52.906486+0000","last_clean_scrub_stamp":"2024-05-30T17:13:52.906486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:22:22.141892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"265'103","reported_seq":626,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.717354+0000","last_change":"2024-05-30T17:13:53.717354+0000","last_active":"2024-05-30T17:13:53.717354+0000","last_peered":"2024-05-30T17:13:53.717354+0000","last_clean":"2024-05-30T17:13:53.717354+0000","last_became_active":"2024-05-30T15:54:26.688376+0000","last_became_peered":"2024-05-30T15:54:26.688376+0000","last_unstale":"2024-05-30T17:13:53.717354+0000","last_undegraded":"2024-05-30T17:13:53.717354+0000","last_fullsized":"2024-05-30T17:13:53.717354+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'103","last_scrub_stamp":"2024-05-30T17:13:53.717082+0000","last_deep_scrub":"265'103","last_deep_scrub_stamp":"2024-05-30T17:13:53.717082+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.717082+0000","objects_scrubbed":4,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:37:15.362960+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":271,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":103,"num_read_kb":102,"num_write":235,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"265'469","reported_seq":2028,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:09.932800+0000","last_change":"2024-05-30T17:14:09.932883+0000","last_active":"2024-05-30T17:14:09.932800+0000","last_peered":"2024-05-30T17:14:09.932800+0000","last_clean":"2024-05-30T17:14:09.932800+0000","last_became_active":"2024-05-30T15:54:18.657540+0000","last_became_peered":"2024-05-30T15:54:18.657540+0000","last_unstale":"2024-05-30T17:14:09.932800+0000","last_undegraded":"2024-05-30T17:14:09.932800+0000","last_fullsized":"2024-05-30T17:14:09.932800+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'469","last_scrub_stamp":"2024-05-30T17:14:09.932582+0000","last_deep_scrub":"265'469","last_deep_scrub_stamp":"2024-05-30T17:14:09.932582+0000","last_clean_scrub_stamp":"2024-05-30T17:14:09.932582+0000","objects_scrubbed":7,"log_size":69,"log_dups_size":400,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:37:05.257095+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2554,"num_read_kb":2362,"num_write":592,"num_write_kb":196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3556,"num_omap_keys":14,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"265'216","reported_seq":886,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:19.855125+0000","last_change":"2024-05-30T17:14:19.855235+0000","last_active":"2024-05-30T17:14:19.855125+0000","last_peered":"2024-05-30T17:14:19.855125+0000","last_clean":"2024-05-30T17:14:19.855125+0000","last_became_active":"2024-05-30T15:54:16.678659+0000","last_became_peered":"2024-05-30T15:54:16.678659+0000","last_unstale":"2024-05-30T17:14:19.855125+0000","last_undegraded":"2024-05-30T17:14:19.855125+0000","last_fullsized":"2024-05-30T17:14:19.855125+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'216","last_scrub_stamp":"2024-05-30T17:14:19.854896+0000","last_deep_scrub":"265'216","last_deep_scrub_stamp":"2024-05-30T17:14:19.854896+0000","last_clean_scrub_stamp":"2024-05-30T17:14:19.854896+0000","objects_scrubbed":4,"log_size":16,"log_dups_size":200,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:20:48.665014+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291507,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":524,"num_read_kb":29194,"num_write":1289,"num_write_kb":17953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":518,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:19.407506+0000","last_change":"2024-05-30T17:14:19.407560+0000","last_active":"2024-05-30T17:14:19.407506+0000","last_peered":"2024-05-30T17:14:19.407506+0000","last_clean":"2024-05-30T17:14:19.407506+0000","last_became_active":"2024-05-30T15:54:20.653316+0000","last_became_peered":"2024-05-30T15:54:20.653316+0000","last_unstale":"2024-05-30T17:14:19.407506+0000","last_undegraded":"2024-05-30T17:14:19.407506+0000","last_fullsized":"2024-05-30T17:14:19.407506+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:19.407301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:19.407301+0000","last_clean_scrub_stamp":"2024-05-30T17:14:19.407301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:47:55.159952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"265'155","reported_seq":727,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.161878+0000","last_change":"2024-05-30T17:13:48.161878+0000","last_active":"2024-05-30T17:13:48.161878+0000","last_peered":"2024-05-30T17:13:48.161878+0000","last_clean":"2024-05-30T17:13:48.161878+0000","last_became_active":"2024-05-30T15:54:22.664353+0000","last_became_peered":"2024-05-30T15:54:22.664353+0000","last_unstale":"2024-05-30T17:13:48.161878+0000","last_undegraded":"2024-05-30T17:13:48.161878+0000","last_fullsized":"2024-05-30T17:13:48.161878+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'155","last_scrub_stamp":"2024-05-30T17:13:48.161621+0000","last_deep_scrub":"265'155","last_deep_scrub_stamp":"2024-05-30T17:13:48.161621+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.161621+0000","objects_scrubbed":6,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:30:12.200451+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":173,"num_read_kb":165,"num_write":120,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"265'532","reported_seq":2492,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:03.691837+0000","last_change":"2024-05-30T17:14:03.691837+0000","last_active":"2024-05-30T17:14:03.691837+0000","last_peered":"2024-05-30T17:14:03.691837+0000","last_clean":"2024-05-30T17:14:03.691837+0000","last_became_active":"2024-05-30T15:54:24.681503+0000","last_became_peered":"2024-05-30T15:54:24.681503+0000","last_unstale":"2024-05-30T17:14:03.691837+0000","last_undegraded":"2024-05-30T17:14:03.691837+0000","last_fullsized":"2024-05-30T17:14:03.691837+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'532","last_scrub_stamp":"2024-05-30T17:14:03.691579+0000","last_deep_scrub":"265'532","last_deep_scrub_stamp":"2024-05-30T17:14:03.691579+0000","last_clean_scrub_stamp":"2024-05-30T17:14:03.691579+0000","objects_scrubbed":1,"log_size":32,"log_dups_size":500,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:45:49.735882+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"265'237","reported_seq":1088,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.884697+0000","last_change":"2024-05-30T17:13:51.884697+0000","last_active":"2024-05-30T17:13:51.884697+0000","last_peered":"2024-05-30T17:13:51.884697+0000","last_clean":"2024-05-30T17:13:51.884697+0000","last_became_active":"2024-05-30T15:54:26.670257+0000","last_became_peered":"2024-05-30T15:54:26.670257+0000","last_unstale":"2024-05-30T17:13:51.884697+0000","last_undegraded":"2024-05-30T17:13:51.884697+0000","last_fullsized":"2024-05-30T17:13:51.884697+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'237","last_scrub_stamp":"2024-05-30T17:13:51.884401+0000","last_deep_scrub":"265'237","last_deep_scrub_stamp":"2024-05-30T17:13:51.884401+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.884401+0000","objects_scrubbed":4,"log_size":37,"log_dups_size":200,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:24:37.651399+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":661,"num_read_kb":640,"num_write":540,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":234,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"265'409","reported_seq":2001,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:13.365832+0000","last_change":"2024-05-30T17:14:13.365901+0000","last_active":"2024-05-30T17:14:13.365832+0000","last_peered":"2024-05-30T17:14:13.365832+0000","last_clean":"2024-05-30T17:14:13.365832+0000","last_became_active":"2024-05-30T15:54:18.660656+0000","last_became_peered":"2024-05-30T15:54:18.660656+0000","last_unstale":"2024-05-30T17:14:13.365832+0000","last_undegraded":"2024-05-30T17:14:13.365832+0000","last_fullsized":"2024-05-30T17:14:13.365832+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'409","last_scrub_stamp":"2024-05-30T17:14:13.365635+0000","last_deep_scrub":"265'409","last_deep_scrub_stamp":"2024-05-30T17:14:13.365635+0000","last_clean_scrub_stamp":"2024-05-30T17:14:13.365635+0000","objects_scrubbed":3,"log_size":109,"log_dups_size":300,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:12:07.620435+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2368,"num_read_kb":2240,"num_write":473,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3015,"num_omap_keys":13,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"265'191","reported_seq":829,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:07.755507+0000","last_change":"2024-05-30T17:14:07.755692+0000","last_active":"2024-05-30T17:14:07.755507+0000","last_peered":"2024-05-30T17:14:07.755507+0000","last_clean":"2024-05-30T17:14:07.755507+0000","last_became_active":"2024-05-30T15:54:16.668083+0000","last_became_peered":"2024-05-30T15:54:16.668083+0000","last_unstale":"2024-05-30T17:14:07.755507+0000","last_undegraded":"2024-05-30T17:14:07.755507+0000","last_fullsized":"2024-05-30T17:14:07.755507+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'191","last_scrub_stamp":"2024-05-30T17:14:07.755193+0000","last_deep_scrub":"265'191","last_deep_scrub_stamp":"2024-05-30T17:14:07.755193+0000","last_clean_scrub_stamp":"2024-05-30T17:14:07.755193+0000","objects_scrubbed":3,"log_size":91,"log_dups_size":100,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:28:20.087604+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5263487,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":406,"num_read_kb":13963,"num_write":1054,"num_write_kb":66137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":516,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:10.355088+0000","last_change":"2024-05-30T17:14:10.355160+0000","last_active":"2024-05-30T17:14:10.355088+0000","last_peered":"2024-05-30T17:14:10.355088+0000","last_clean":"2024-05-30T17:14:10.355088+0000","last_became_active":"2024-05-30T15:54:20.658870+0000","last_became_peered":"2024-05-30T15:54:20.658870+0000","last_unstale":"2024-05-30T17:14:10.355088+0000","last_undegraded":"2024-05-30T17:14:10.355088+0000","last_fullsized":"2024-05-30T17:14:10.355088+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:10.354868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:10.354868+0000","last_clean_scrub_stamp":"2024-05-30T17:14:10.354868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:31:51.606017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"265'146","reported_seq":743,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:09.748021+0000","last_change":"2024-05-30T17:14:09.748021+0000","last_active":"2024-05-30T17:14:09.748021+0000","last_peered":"2024-05-30T17:14:09.748021+0000","last_clean":"2024-05-30T17:14:09.748021+0000","last_became_active":"2024-05-30T15:54:22.849978+0000","last_became_peered":"2024-05-30T15:54:22.849978+0000","last_unstale":"2024-05-30T17:14:09.748021+0000","last_undegraded":"2024-05-30T17:14:09.748021+0000","last_fullsized":"2024-05-30T17:14:09.748021+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'146","last_scrub_stamp":"2024-05-30T17:14:09.747755+0000","last_deep_scrub":"265'146","last_deep_scrub_stamp":"2024-05-30T17:14:09.747755+0000","last_clean_scrub_stamp":"2024-05-30T17:14:09.747755+0000","objects_scrubbed":4,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:28:48.260787+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8153,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":189,"num_read_kb":241,"num_write":102,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"265'549","reported_seq":2582,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:05.388708+0000","last_change":"2024-05-30T17:14:05.388708+0000","last_active":"2024-05-30T17:14:05.388708+0000","last_peered":"2024-05-30T17:14:05.388708+0000","last_clean":"2024-05-30T17:14:05.388708+0000","last_became_active":"2024-05-30T15:54:24.686546+0000","last_became_peered":"2024-05-30T15:54:24.686546+0000","last_unstale":"2024-05-30T17:14:05.388708+0000","last_undegraded":"2024-05-30T17:14:05.388708+0000","last_fullsized":"2024-05-30T17:14:05.388708+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'549","last_scrub_stamp":"2024-05-30T17:14:05.388354+0000","last_deep_scrub":"265'549","last_deep_scrub_stamp":"2024-05-30T17:14:05.388354+0000","last_clean_scrub_stamp":"2024-05-30T17:14:05.388354+0000","objects_scrubbed":1,"log_size":49,"log_dups_size":500,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:59:56.299339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"265'901","reported_seq":2526,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:05.284034+0000","last_change":"2024-05-30T17:14:05.284034+0000","last_active":"2024-05-30T17:14:05.284034+0000","last_peered":"2024-05-30T17:14:05.284034+0000","last_clean":"2024-05-30T17:14:05.284034+0000","last_became_active":"2024-05-30T15:54:18.658124+0000","last_became_peered":"2024-05-30T15:54:18.658124+0000","last_unstale":"2024-05-30T17:14:05.284034+0000","last_undegraded":"2024-05-30T17:14:05.284034+0000","last_fullsized":"2024-05-30T17:14:05.284034+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'901","last_scrub_stamp":"2024-05-30T17:14:05.283757+0000","last_deep_scrub":"265'901","last_deep_scrub_stamp":"2024-05-30T17:14:05.283757+0000","last_clean_scrub_stamp":"2024-05-30T17:14:05.283757+0000","objects_scrubbed":6,"log_size":101,"log_dups_size":800,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:45:18.502796+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4127,"num_read_kb":3657,"num_write":1246,"num_write_kb":491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2810,"num_omap_keys":11,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"265'208","reported_seq":897,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:32.804008+0000","last_change":"2024-05-30T17:14:32.804223+0000","last_active":"2024-05-30T17:14:32.804008+0000","last_peered":"2024-05-30T17:14:32.804008+0000","last_clean":"2024-05-30T17:14:32.804008+0000","last_became_active":"2024-05-30T15:54:16.658279+0000","last_became_peered":"2024-05-30T15:54:16.658279+0000","last_unstale":"2024-05-30T17:14:32.804008+0000","last_undegraded":"2024-05-30T17:14:32.804008+0000","last_fullsized":"2024-05-30T17:14:32.804008+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'208","last_scrub_stamp":"2024-05-30T17:14:32.803764+0000","last_deep_scrub":"265'208","last_deep_scrub_stamp":"2024-05-30T17:14:32.803764+0000","last_clean_scrub_stamp":"2024-05-30T17:14:32.803764+0000","objects_scrubbed":3,"log_size":108,"log_dups_size":100,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:31:04.878530+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1254547,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":480,"num_read_kb":111643,"num_write":1134,"num_write_kb":26225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":520,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:14.742349+0000","last_change":"2024-05-30T17:14:14.742349+0000","last_active":"2024-05-30T17:14:14.742349+0000","last_peered":"2024-05-30T17:14:14.742349+0000","last_clean":"2024-05-30T17:14:14.742349+0000","last_became_active":"2024-05-30T15:54:20.661583+0000","last_became_peered":"2024-05-30T15:54:20.661583+0000","last_unstale":"2024-05-30T17:14:14.742349+0000","last_undegraded":"2024-05-30T17:14:14.742349+0000","last_fullsized":"2024-05-30T17:14:14.742349+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:14.742060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:14.742060+0000","last_clean_scrub_stamp":"2024-05-30T17:14:14.742060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:08:49.700663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"265'265","reported_seq":921,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.257602+0000","last_change":"2024-05-30T17:13:58.257702+0000","last_active":"2024-05-30T17:13:58.257602+0000","last_peered":"2024-05-30T17:13:58.257602+0000","last_clean":"2024-05-30T17:13:58.257602+0000","last_became_active":"2024-05-30T15:54:22.849126+0000","last_became_peered":"2024-05-30T15:54:22.849126+0000","last_unstale":"2024-05-30T17:13:58.257602+0000","last_undegraded":"2024-05-30T17:13:58.257602+0000","last_fullsized":"2024-05-30T17:13:58.257602+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'265","last_scrub_stamp":"2024-05-30T17:13:58.257369+0000","last_deep_scrub":"265'265","last_deep_scrub_stamp":"2024-05-30T17:13:58.257369+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.257369+0000","objects_scrubbed":10,"log_size":65,"log_dups_size":200,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:15:48.980020+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6773,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":352,"num_read_kb":376,"num_write":210,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":511,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:07.996809+0000","last_change":"2024-05-30T17:14:07.996872+0000","last_active":"2024-05-30T17:14:07.996809+0000","last_peered":"2024-05-30T17:14:07.996809+0000","last_clean":"2024-05-30T17:14:07.996809+0000","last_became_active":"2024-05-30T15:54:24.673506+0000","last_became_peered":"2024-05-30T15:54:24.673506+0000","last_unstale":"2024-05-30T17:14:07.996809+0000","last_undegraded":"2024-05-30T17:14:07.996809+0000","last_fullsized":"2024-05-30T17:14:07.996809+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:07.996581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:07.996581+0000","last_clean_scrub_stamp":"2024-05-30T17:14:07.996581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:59:09.003040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"265'546","reported_seq":1385,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.226322+0000","last_change":"2024-05-30T17:13:54.226466+0000","last_active":"2024-05-30T17:13:54.226322+0000","last_peered":"2024-05-30T17:13:54.226322+0000","last_clean":"2024-05-30T17:13:54.226322+0000","last_became_active":"2024-05-30T15:54:26.686389+0000","last_became_peered":"2024-05-30T15:54:26.686389+0000","last_unstale":"2024-05-30T17:13:54.226322+0000","last_undegraded":"2024-05-30T17:13:54.226322+0000","last_fullsized":"2024-05-30T17:13:54.226322+0000","mapping_epoch":32,"log_start":"265'500","ondisk_log_start":"265'500","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'546","last_scrub_stamp":"2024-05-30T17:13:54.226029+0000","last_deep_scrub":"265'546","last_deep_scrub_stamp":"2024-05-30T17:13:54.226029+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.226029+0000","objects_scrubbed":3,"log_size":46,"log_dups_size":500,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:20:00.749166+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1297,"num_read_kb":1288,"num_write":1137,"num_write_kb":385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":381,"num_omap_keys":2,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"265'1403","reported_seq":3120,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:23.135183+0000","last_change":"2024-05-30T17:14:23.135183+0000","last_active":"2024-05-30T17:14:23.135183+0000","last_peered":"2024-05-30T17:14:23.135183+0000","last_clean":"2024-05-30T17:14:23.135183+0000","last_became_active":"2024-05-30T15:54:18.639667+0000","last_became_peered":"2024-05-30T15:54:18.639667+0000","last_unstale":"2024-05-30T17:14:23.135183+0000","last_undegraded":"2024-05-30T17:14:23.135183+0000","last_fullsized":"2024-05-30T17:14:23.135183+0000","mapping_epoch":24,"log_start":"265'1300","ondisk_log_start":"265'1300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'1403","last_scrub_stamp":"2024-05-30T17:14:23.134899+0000","last_deep_scrub":"265'1403","last_deep_scrub_stamp":"2024-05-30T17:14:23.134899+0000","last_clean_scrub_stamp":"2024-05-30T17:14:23.134899+0000","objects_scrubbed":5,"log_size":103,"log_dups_size":1300,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:49:41.516827+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5970,"num_read_kb":5130,"num_write":2044,"num_write_kb":884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":763,"num_omap_keys":3,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"265'213","reported_seq":876,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:33.753735+0000","last_change":"2024-05-30T17:14:33.753798+0000","last_active":"2024-05-30T17:14:33.753735+0000","last_peered":"2024-05-30T17:14:33.753735+0000","last_clean":"2024-05-30T17:14:33.753735+0000","last_became_active":"2024-05-30T15:54:16.659427+0000","last_became_peered":"2024-05-30T15:54:16.659427+0000","last_unstale":"2024-05-30T17:14:33.753735+0000","last_undegraded":"2024-05-30T17:14:33.753735+0000","last_fullsized":"2024-05-30T17:14:33.753735+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'213","last_scrub_stamp":"2024-05-30T17:14:33.753534+0000","last_deep_scrub":"265'213","last_deep_scrub_stamp":"2024-05-30T17:14:33.753534+0000","last_clean_scrub_stamp":"2024-05-30T17:14:33.753534+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":200,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:07:20.080063+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":467,"num_read_kb":34119,"num_write":1157,"num_write_kb":42367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":523,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.231474+0000","last_change":"2024-05-30T17:13:58.231474+0000","last_active":"2024-05-30T17:13:58.231474+0000","last_peered":"2024-05-30T17:13:58.231474+0000","last_clean":"2024-05-30T17:13:58.231474+0000","last_became_active":"2024-05-30T15:54:20.647565+0000","last_became_peered":"2024-05-30T15:54:20.647565+0000","last_unstale":"2024-05-30T17:13:58.231474+0000","last_undegraded":"2024-05-30T17:13:58.231474+0000","last_fullsized":"2024-05-30T17:13:58.231474+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:58.231223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:58.231223+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.231223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:53:45.579167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"265'150","reported_seq":751,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:25.359652+0000","last_change":"2024-05-30T17:14:25.359726+0000","last_active":"2024-05-30T17:14:25.359652+0000","last_peered":"2024-05-30T17:14:25.359652+0000","last_clean":"2024-05-30T17:14:25.359652+0000","last_became_active":"2024-05-30T15:54:22.670064+0000","last_became_peered":"2024-05-30T15:54:22.670064+0000","last_unstale":"2024-05-30T17:14:25.359652+0000","last_undegraded":"2024-05-30T17:14:25.359652+0000","last_fullsized":"2024-05-30T17:14:25.359652+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'150","last_scrub_stamp":"2024-05-30T17:14:25.359472+0000","last_deep_scrub":"265'150","last_deep_scrub_stamp":"2024-05-30T17:14:25.359472+0000","last_clean_scrub_stamp":"2024-05-30T17:14:25.359472+0000","objects_scrubbed":4,"log_size":50,"log_dups_size":100,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:25:32.352668+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12270,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":197,"num_read_kb":284,"num_write":108,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":513,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:02.359550+0000","last_change":"2024-05-30T17:14:02.359602+0000","last_active":"2024-05-30T17:14:02.359550+0000","last_peered":"2024-05-30T17:14:02.359550+0000","last_clean":"2024-05-30T17:14:02.359550+0000","last_became_active":"2024-05-30T15:54:24.688046+0000","last_became_peered":"2024-05-30T15:54:24.688046+0000","last_unstale":"2024-05-30T17:14:02.359550+0000","last_undegraded":"2024-05-30T17:14:02.359550+0000","last_fullsized":"2024-05-30T17:14:02.359550+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:02.359352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:02.359352+0000","last_clean_scrub_stamp":"2024-05-30T17:14:02.359352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:10:48.308830+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"265'2370","reported_seq":4200,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:04.738831+0000","last_change":"2024-05-30T17:14:04.738949+0000","last_active":"2024-05-30T17:14:04.738831+0000","last_peered":"2024-05-30T17:14:04.738831+0000","last_clean":"2024-05-30T17:14:04.738831+0000","last_became_active":"2024-05-30T15:54:26.691574+0000","last_became_peered":"2024-05-30T15:54:26.691574+0000","last_unstale":"2024-05-30T17:14:04.738831+0000","last_undegraded":"2024-05-30T17:14:04.738831+0000","last_fullsized":"2024-05-30T17:14:04.738831+0000","mapping_epoch":32,"log_start":"265'2300","ondisk_log_start":"265'2300","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2370","last_scrub_stamp":"2024-05-30T17:14:04.738577+0000","last_deep_scrub":"265'2370","last_deep_scrub_stamp":"2024-05-30T17:14:04.738577+0000","last_clean_scrub_stamp":"2024-05-30T17:14:04.738577+0000","objects_scrubbed":1,"log_size":70,"log_dups_size":2300,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:55:13.090443+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5943,"num_read_kb":5935,"num_write":4785,"num_write_kb":1767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":547,"num_omap_keys":3,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"265'373","reported_seq":1803,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:01.825393+0000","last_change":"2024-05-30T17:14:01.825479+0000","last_active":"2024-05-30T17:14:01.825393+0000","last_peered":"2024-05-30T17:14:01.825393+0000","last_clean":"2024-05-30T17:14:01.825393+0000","last_became_active":"2024-05-30T15:54:18.668963+0000","last_became_peered":"2024-05-30T15:54:18.668963+0000","last_unstale":"2024-05-30T17:14:01.825393+0000","last_undegraded":"2024-05-30T17:14:01.825393+0000","last_fullsized":"2024-05-30T17:14:01.825393+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'373","last_scrub_stamp":"2024-05-30T17:14:01.825217+0000","last_deep_scrub":"265'373","last_deep_scrub_stamp":"2024-05-30T17:14:01.825217+0000","last_clean_scrub_stamp":"2024-05-30T17:14:01.825217+0000","objects_scrubbed":2,"log_size":73,"log_dups_size":300,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:14:41.634577+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2011,"num_read_kb":1890,"num_write":455,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":225,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"265'240","reported_seq":889,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:29.763421+0000","last_change":"2024-05-30T17:14:29.763528+0000","last_active":"2024-05-30T17:14:29.763421+0000","last_peered":"2024-05-30T17:14:29.763421+0000","last_clean":"2024-05-30T17:14:29.763421+0000","last_became_active":"2024-05-30T15:54:16.649605+0000","last_became_peered":"2024-05-30T15:54:16.649605+0000","last_unstale":"2024-05-30T17:14:29.763421+0000","last_undegraded":"2024-05-30T17:14:29.763421+0000","last_fullsized":"2024-05-30T17:14:29.763421+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'240","last_scrub_stamp":"2024-05-30T17:14:29.763223+0000","last_deep_scrub":"265'240","last_deep_scrub_stamp":"2024-05-30T17:14:29.763223+0000","last_clean_scrub_stamp":"2024-05-30T17:14:29.763223+0000","objects_scrubbed":3,"log_size":40,"log_dups_size":200,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:14.911288+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242911,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":469,"num_read_kb":32402,"num_write":1238,"num_write_kb":40071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":521,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.170085+0000","last_change":"2024-05-30T17:13:56.170146+0000","last_active":"2024-05-30T17:13:56.170085+0000","last_peered":"2024-05-30T17:13:56.170085+0000","last_clean":"2024-05-30T17:13:56.170085+0000","last_became_active":"2024-05-30T15:54:20.651369+0000","last_became_peered":"2024-05-30T15:54:20.651369+0000","last_unstale":"2024-05-30T17:13:56.170085+0000","last_undegraded":"2024-05-30T17:13:56.170085+0000","last_fullsized":"2024-05-30T17:13:56.170085+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:56.169880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:56.169880+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.169880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:32:45.358384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"265'199","reported_seq":824,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:24.727401+0000","last_change":"2024-05-30T17:14:24.727401+0000","last_active":"2024-05-30T17:14:24.727401+0000","last_peered":"2024-05-30T17:14:24.727401+0000","last_clean":"2024-05-30T17:14:24.727401+0000","last_became_active":"2024-05-30T15:54:22.664874+0000","last_became_peered":"2024-05-30T15:54:22.664874+0000","last_unstale":"2024-05-30T17:14:24.727401+0000","last_undegraded":"2024-05-30T17:14:24.727401+0000","last_fullsized":"2024-05-30T17:14:24.727401+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'199","last_scrub_stamp":"2024-05-30T17:14:24.727125+0000","last_deep_scrub":"265'199","last_deep_scrub_stamp":"2024-05-30T17:14:24.727125+0000","last_clean_scrub_stamp":"2024-05-30T17:14:24.727125+0000","objects_scrubbed":8,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:38:48.594467+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6309,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":259,"num_read_kb":308,"num_write":148,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"265'549","reported_seq":2492,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:06.842942+0000","last_change":"2024-05-30T17:14:06.842942+0000","last_active":"2024-05-30T17:14:06.842942+0000","last_peered":"2024-05-30T17:14:06.842942+0000","last_clean":"2024-05-30T17:14:06.842942+0000","last_became_active":"2024-05-30T15:54:24.670141+0000","last_became_peered":"2024-05-30T15:54:24.670141+0000","last_unstale":"2024-05-30T17:14:06.842942+0000","last_undegraded":"2024-05-30T17:14:06.842942+0000","last_fullsized":"2024-05-30T17:14:06.842942+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'549","last_scrub_stamp":"2024-05-30T17:14:06.842673+0000","last_deep_scrub":"265'549","last_deep_scrub_stamp":"2024-05-30T17:14:06.842673+0000","last_clean_scrub_stamp":"2024-05-30T17:14:06.842673+0000","objects_scrubbed":1,"log_size":49,"log_dups_size":500,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:16:17.488221+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"265'107","reported_seq":621,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:52.677276+0000","last_change":"2024-05-30T17:13:52.677276+0000","last_active":"2024-05-30T17:13:52.677276+0000","last_peered":"2024-05-30T17:13:52.677276+0000","last_clean":"2024-05-30T17:13:52.677276+0000","last_became_active":"2024-05-30T15:54:26.689987+0000","last_became_peered":"2024-05-30T15:54:26.689987+0000","last_unstale":"2024-05-30T17:13:52.677276+0000","last_undegraded":"2024-05-30T17:13:52.677276+0000","last_fullsized":"2024-05-30T17:13:52.677276+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'107","last_scrub_stamp":"2024-05-30T17:13:52.676848+0000","last_deep_scrub":"265'107","last_deep_scrub_stamp":"2024-05-30T17:13:52.676848+0000","last_clean_scrub_stamp":"2024-05-30T17:13:52.676848+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:48:23.065085+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":80,"num_write":254,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"265'1335","reported_seq":2979,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:16.822597+0000","last_change":"2024-05-30T17:14:16.822667+0000","last_active":"2024-05-30T17:14:16.822597+0000","last_peered":"2024-05-30T17:14:16.822597+0000","last_clean":"2024-05-30T17:14:16.822597+0000","last_became_active":"2024-05-30T15:54:18.647634+0000","last_became_peered":"2024-05-30T15:54:18.647634+0000","last_unstale":"2024-05-30T17:14:16.822597+0000","last_undegraded":"2024-05-30T17:14:16.822597+0000","last_fullsized":"2024-05-30T17:14:16.822597+0000","mapping_epoch":24,"log_start":"265'1300","ondisk_log_start":"265'1300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'1335","last_scrub_stamp":"2024-05-30T17:14:16.822401+0000","last_deep_scrub":"265'1335","last_deep_scrub_stamp":"2024-05-30T17:14:16.822401+0000","last_clean_scrub_stamp":"2024-05-30T17:14:16.822401+0000","objects_scrubbed":4,"log_size":35,"log_dups_size":1300,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:44:49.543711+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5717,"num_read_kb":4967,"num_write":1912,"num_write_kb":881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"265'206","reported_seq":830,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:25.336089+0000","last_change":"2024-05-30T17:14:25.336288+0000","last_active":"2024-05-30T17:14:25.336089+0000","last_peered":"2024-05-30T17:14:25.336089+0000","last_clean":"2024-05-30T17:14:25.336089+0000","last_became_active":"2024-05-30T15:54:16.668336+0000","last_became_peered":"2024-05-30T15:54:16.668336+0000","last_unstale":"2024-05-30T17:14:25.336089+0000","last_undegraded":"2024-05-30T17:14:25.336089+0000","last_fullsized":"2024-05-30T17:14:25.336089+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'206","last_scrub_stamp":"2024-05-30T17:14:25.335853+0000","last_deep_scrub":"265'206","last_deep_scrub_stamp":"2024-05-30T17:14:25.335853+0000","last_clean_scrub_stamp":"2024-05-30T17:14:25.335853+0000","objects_scrubbed":4,"log_size":106,"log_dups_size":100,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:00:44.689923+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":403,"num_read_kb":1534,"num_write":1058,"num_write_kb":21253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":522,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.159988+0000","last_change":"2024-05-30T17:13:54.160046+0000","last_active":"2024-05-30T17:13:54.159988+0000","last_peered":"2024-05-30T17:13:54.159988+0000","last_clean":"2024-05-30T17:13:54.159988+0000","last_became_active":"2024-05-30T15:54:20.655573+0000","last_became_peered":"2024-05-30T15:54:20.655573+0000","last_unstale":"2024-05-30T17:13:54.159988+0000","last_undegraded":"2024-05-30T17:13:54.159988+0000","last_fullsized":"2024-05-30T17:13:54.159988+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:54.159789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:54.159789+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.159789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:27:43.050145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"265'212","reported_seq":880,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:19.794572+0000","last_change":"2024-05-30T17:14:19.794572+0000","last_active":"2024-05-30T17:14:19.794572+0000","last_peered":"2024-05-30T17:14:19.794572+0000","last_clean":"2024-05-30T17:14:19.794572+0000","last_became_active":"2024-05-30T15:54:22.658700+0000","last_became_peered":"2024-05-30T15:54:22.658700+0000","last_unstale":"2024-05-30T17:14:19.794572+0000","last_undegraded":"2024-05-30T17:14:19.794572+0000","last_fullsized":"2024-05-30T17:14:19.794572+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'212","last_scrub_stamp":"2024-05-30T17:14:19.794300+0000","last_deep_scrub":"265'212","last_deep_scrub_stamp":"2024-05-30T17:14:19.794300+0000","last_clean_scrub_stamp":"2024-05-30T17:14:19.794300+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:25:02.091631+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14119,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":309,"num_read_kb":443,"num_write":154,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":515,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.048221+0000","last_change":"2024-05-30T17:13:43.048221+0000","last_active":"2024-05-30T17:13:43.048221+0000","last_peered":"2024-05-30T17:13:43.048221+0000","last_clean":"2024-05-30T17:13:43.048221+0000","last_became_active":"2024-05-30T15:54:24.673505+0000","last_became_peered":"2024-05-30T15:54:24.673505+0000","last_unstale":"2024-05-30T17:13:43.048221+0000","last_undegraded":"2024-05-30T17:13:43.048221+0000","last_fullsized":"2024-05-30T17:13:43.048221+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:43.047974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:43.047974+0000","last_clean_scrub_stamp":"2024-05-30T17:13:43.047974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:04:22.180317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"265'101","reported_seq":628,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.955055+0000","last_change":"2024-05-30T17:13:50.955055+0000","last_active":"2024-05-30T17:13:50.955055+0000","last_peered":"2024-05-30T17:13:50.955055+0000","last_clean":"2024-05-30T17:13:50.955055+0000","last_became_active":"2024-05-30T15:54:26.700608+0000","last_became_peered":"2024-05-30T15:54:26.700608+0000","last_unstale":"2024-05-30T17:13:50.955055+0000","last_undegraded":"2024-05-30T17:13:50.955055+0000","last_fullsized":"2024-05-30T17:13:50.955055+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'101","last_scrub_stamp":"2024-05-30T17:13:50.954735+0000","last_deep_scrub":"265'101","last_deep_scrub_stamp":"2024-05-30T17:13:50.954735+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.954735+0000","objects_scrubbed":2,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:48:33.984457+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":918,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":103,"num_write":225,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"265'544","reported_seq":2181,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:10.057108+0000","last_change":"2024-05-30T17:14:10.057194+0000","last_active":"2024-05-30T17:14:10.057108+0000","last_peered":"2024-05-30T17:14:10.057108+0000","last_clean":"2024-05-30T17:14:10.057108+0000","last_became_active":"2024-05-30T15:54:18.657522+0000","last_became_peered":"2024-05-30T15:54:18.657522+0000","last_unstale":"2024-05-30T17:14:10.057108+0000","last_undegraded":"2024-05-30T17:14:10.057108+0000","last_fullsized":"2024-05-30T17:14:10.057108+0000","mapping_epoch":24,"log_start":"265'500","ondisk_log_start":"265'500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'544","last_scrub_stamp":"2024-05-30T17:14:10.056929+0000","last_deep_scrub":"265'544","last_deep_scrub_stamp":"2024-05-30T17:14:10.056929+0000","last_clean_scrub_stamp":"2024-05-30T17:14:10.056929+0000","objects_scrubbed":5,"log_size":44,"log_dups_size":500,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:06:50.095596+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2751,"num_read_kb":2537,"num_write":708,"num_write_kb":252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":303,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"265'234","reported_seq":924,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:33.434213+0000","last_change":"2024-05-30T17:14:33.434317+0000","last_active":"2024-05-30T17:14:33.434213+0000","last_peered":"2024-05-30T17:14:33.434213+0000","last_clean":"2024-05-30T17:14:33.434213+0000","last_became_active":"2024-05-30T15:54:16.666244+0000","last_became_peered":"2024-05-30T15:54:16.666244+0000","last_unstale":"2024-05-30T17:14:33.434213+0000","last_undegraded":"2024-05-30T17:14:33.434213+0000","last_fullsized":"2024-05-30T17:14:33.434213+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'234","last_scrub_stamp":"2024-05-30T17:14:33.433990+0000","last_deep_scrub":"265'234","last_deep_scrub_stamp":"2024-05-30T17:14:33.433990+0000","last_clean_scrub_stamp":"2024-05-30T17:14:33.433990+0000","objects_scrubbed":5,"log_size":34,"log_dups_size":200,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:55:15.577076+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":549856,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":638,"num_read_kb":32957,"num_write":1225,"num_write_kb":40334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":521,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:52.077511+0000","last_change":"2024-05-30T17:13:52.077571+0000","last_active":"2024-05-30T17:13:52.077511+0000","last_peered":"2024-05-30T17:13:52.077511+0000","last_clean":"2024-05-30T17:13:52.077511+0000","last_became_active":"2024-05-30T15:54:20.661967+0000","last_became_peered":"2024-05-30T15:54:20.661967+0000","last_unstale":"2024-05-30T17:13:52.077511+0000","last_undegraded":"2024-05-30T17:13:52.077511+0000","last_fullsized":"2024-05-30T17:13:52.077511+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:52.077316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:52.077316+0000","last_clean_scrub_stamp":"2024-05-30T17:13:52.077316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:55:45.987890+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"265'187","reported_seq":803,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:25.109166+0000","last_change":"2024-05-30T17:14:25.109258+0000","last_active":"2024-05-30T17:14:25.109166+0000","last_peered":"2024-05-30T17:14:25.109166+0000","last_clean":"2024-05-30T17:14:25.109166+0000","last_became_active":"2024-05-30T15:54:22.664732+0000","last_became_peered":"2024-05-30T15:54:22.664732+0000","last_unstale":"2024-05-30T17:14:25.109166+0000","last_undegraded":"2024-05-30T17:14:25.109166+0000","last_fullsized":"2024-05-30T17:14:25.109166+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'187","last_scrub_stamp":"2024-05-30T17:14:25.108975+0000","last_deep_scrub":"265'187","last_deep_scrub_stamp":"2024-05-30T17:14:25.108975+0000","last_clean_scrub_stamp":"2024-05-30T17:14:25.108975+0000","objects_scrubbed":6,"log_size":87,"log_dups_size":100,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:53:56.466888+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13534,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":265,"num_read_kb":366,"num_write":202,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15964,"num_omap_keys":38,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":515,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:01.839426+0000","last_change":"2024-05-30T17:14:01.839579+0000","last_active":"2024-05-30T17:14:01.839426+0000","last_peered":"2024-05-30T17:14:01.839426+0000","last_clean":"2024-05-30T17:14:01.839426+0000","last_became_active":"2024-05-30T15:54:24.679254+0000","last_became_peered":"2024-05-30T15:54:24.679254+0000","last_unstale":"2024-05-30T17:14:01.839426+0000","last_undegraded":"2024-05-30T17:14:01.839426+0000","last_fullsized":"2024-05-30T17:14:01.839426+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:01.839200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:01.839200+0000","last_clean_scrub_stamp":"2024-05-30T17:14:01.839200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:13:39.135342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"265'109","reported_seq":636,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:57.389981+0000","last_change":"2024-05-30T17:13:57.390054+0000","last_active":"2024-05-30T17:13:57.389981+0000","last_peered":"2024-05-30T17:13:57.389981+0000","last_clean":"2024-05-30T17:13:57.389981+0000","last_became_active":"2024-05-30T15:54:26.697333+0000","last_became_peered":"2024-05-30T15:54:26.697333+0000","last_unstale":"2024-05-30T17:13:57.389981+0000","last_undegraded":"2024-05-30T17:13:57.389981+0000","last_fullsized":"2024-05-30T17:13:57.389981+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'109","last_scrub_stamp":"2024-05-30T17:13:57.389793+0000","last_deep_scrub":"265'109","last_deep_scrub_stamp":"2024-05-30T17:13:57.389793+0000","last_clean_scrub_stamp":"2024-05-30T17:13:57.389793+0000","objects_scrubbed":4,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:34:55.402057+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":270,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":107,"num_read_kb":90,"num_write":242,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"265'402","reported_seq":2073,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:22.126305+0000","last_change":"2024-05-30T17:14:22.126392+0000","last_active":"2024-05-30T17:14:22.126305+0000","last_peered":"2024-05-30T17:14:22.126305+0000","last_clean":"2024-05-30T17:14:22.126305+0000","last_became_active":"2024-05-30T15:54:18.635519+0000","last_became_peered":"2024-05-30T15:54:18.635519+0000","last_unstale":"2024-05-30T17:14:22.126305+0000","last_undegraded":"2024-05-30T17:14:22.126305+0000","last_fullsized":"2024-05-30T17:14:22.126305+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'402","last_scrub_stamp":"2024-05-30T17:14:22.126084+0000","last_deep_scrub":"265'402","last_deep_scrub_stamp":"2024-05-30T17:14:22.126084+0000","last_clean_scrub_stamp":"2024-05-30T17:14:22.126084+0000","objects_scrubbed":5,"log_size":102,"log_dups_size":300,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:18:22.194175+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2332,"num_read_kb":2242,"num_write":450,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2297,"num_omap_keys":9,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"265'175","reported_seq":804,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:12.798118+0000","last_change":"2024-05-30T17:14:12.798266+0000","last_active":"2024-05-30T17:14:12.798118+0000","last_peered":"2024-05-30T17:14:12.798118+0000","last_clean":"2024-05-30T17:14:12.798118+0000","last_became_active":"2024-05-30T15:54:16.685162+0000","last_became_peered":"2024-05-30T15:54:16.685162+0000","last_unstale":"2024-05-30T17:14:12.798118+0000","last_undegraded":"2024-05-30T17:14:12.798118+0000","last_fullsized":"2024-05-30T17:14:12.798118+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'175","last_scrub_stamp":"2024-05-30T17:14:12.797857+0000","last_deep_scrub":"265'175","last_deep_scrub_stamp":"2024-05-30T17:14:12.797857+0000","last_clean_scrub_stamp":"2024-05-30T17:14:12.797857+0000","objects_scrubbed":6,"log_size":75,"log_dups_size":100,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:43:33.318612+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5904316,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":401,"num_read_kb":10586,"num_write":1027,"num_write_kb":21029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":519,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.991573+0000","last_change":"2024-05-30T17:13:56.991631+0000","last_active":"2024-05-30T17:13:56.991573+0000","last_peered":"2024-05-30T17:13:56.991573+0000","last_clean":"2024-05-30T17:13:56.991573+0000","last_became_active":"2024-05-30T15:54:20.646646+0000","last_became_peered":"2024-05-30T15:54:20.646646+0000","last_unstale":"2024-05-30T17:13:56.991573+0000","last_undegraded":"2024-05-30T17:13:56.991573+0000","last_fullsized":"2024-05-30T17:13:56.991573+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:56.991372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:56.991372+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.991372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:25:19.485498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"265'189","reported_seq":815,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.157684+0000","last_change":"2024-05-30T17:13:46.157858+0000","last_active":"2024-05-30T17:13:46.157684+0000","last_peered":"2024-05-30T17:13:46.157684+0000","last_clean":"2024-05-30T17:13:46.157684+0000","last_became_active":"2024-05-30T15:54:22.669660+0000","last_became_peered":"2024-05-30T15:54:22.669660+0000","last_unstale":"2024-05-30T17:13:46.157684+0000","last_undegraded":"2024-05-30T17:13:46.157684+0000","last_fullsized":"2024-05-30T17:13:46.157684+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'189","last_scrub_stamp":"2024-05-30T17:13:46.157487+0000","last_deep_scrub":"265'189","last_deep_scrub_stamp":"2024-05-30T17:13:46.157487+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.157487+0000","objects_scrubbed":7,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:49:02.360024+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6014,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":258,"num_read_kb":320,"num_write":146,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"265'512","reported_seq":4604,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.085253+0000","last_change":"2024-05-30T17:13:44.085280+0000","last_active":"2024-05-30T17:13:44.085253+0000","last_peered":"2024-05-30T17:13:44.085253+0000","last_clean":"2024-05-30T17:13:44.085253+0000","last_became_active":"2024-05-30T15:54:24.672377+0000","last_became_peered":"2024-05-30T15:54:24.672377+0000","last_unstale":"2024-05-30T17:13:44.085253+0000","last_undegraded":"2024-05-30T17:13:44.085253+0000","last_fullsized":"2024-05-30T17:13:44.085253+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'512","last_scrub_stamp":"2024-05-30T17:13:44.085165+0000","last_deep_scrub":"265'512","last_deep_scrub_stamp":"2024-05-30T17:13:44.085165+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.085165+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":500,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:13:21.462886+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"265'120","reported_seq":645,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.014771+0000","last_change":"2024-05-30T17:13:53.014854+0000","last_active":"2024-05-30T17:13:53.014771+0000","last_peered":"2024-05-30T17:13:53.014771+0000","last_clean":"2024-05-30T17:13:53.014771+0000","last_became_active":"2024-05-30T15:54:26.679866+0000","last_became_peered":"2024-05-30T15:54:26.679866+0000","last_unstale":"2024-05-30T17:13:53.014771+0000","last_undegraded":"2024-05-30T17:13:53.014771+0000","last_fullsized":"2024-05-30T17:13:53.014771+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'120","last_scrub_stamp":"2024-05-30T17:13:53.014561+0000","last_deep_scrub":"265'120","last_deep_scrub_stamp":"2024-05-30T17:13:53.014561+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.014561+0000","objects_scrubbed":3,"log_size":20,"log_dups_size":100,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:47:46.937703+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1256,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":144,"num_read_kb":134,"num_write":293,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"265'392","reported_seq":1808,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:02.030303+0000","last_change":"2024-05-30T17:14:02.030382+0000","last_active":"2024-05-30T17:14:02.030303+0000","last_peered":"2024-05-30T17:14:02.030303+0000","last_clean":"2024-05-30T17:14:02.030303+0000","last_became_active":"2024-05-30T15:54:18.671088+0000","last_became_peered":"2024-05-30T15:54:18.671088+0000","last_unstale":"2024-05-30T17:14:02.030303+0000","last_undegraded":"2024-05-30T17:14:02.030303+0000","last_fullsized":"2024-05-30T17:14:02.030303+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'392","last_scrub_stamp":"2024-05-30T17:14:02.030076+0000","last_deep_scrub":"265'392","last_deep_scrub_stamp":"2024-05-30T17:14:02.030076+0000","last_clean_scrub_stamp":"2024-05-30T17:14:02.030076+0000","objects_scrubbed":3,"log_size":92,"log_dups_size":300,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:37:05.922490+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2029,"num_read_kb":1894,"num_write":481,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"265'180","reported_seq":933,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:18.060426+0000","last_change":"2024-05-30T17:14:18.060426+0000","last_active":"2024-05-30T17:14:18.060426+0000","last_peered":"2024-05-30T17:14:18.060426+0000","last_clean":"2024-05-30T17:14:18.060426+0000","last_became_active":"2024-05-30T15:54:16.671477+0000","last_became_peered":"2024-05-30T15:54:16.671477+0000","last_unstale":"2024-05-30T17:14:18.060426+0000","last_undegraded":"2024-05-30T17:14:18.060426+0000","last_fullsized":"2024-05-30T17:14:18.060426+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'180","last_scrub_stamp":"2024-05-30T17:14:18.060120+0000","last_deep_scrub":"265'180","last_deep_scrub_stamp":"2024-05-30T17:14:18.060120+0000","last_clean_scrub_stamp":"2024-05-30T17:14:18.060120+0000","objects_scrubbed":3,"log_size":80,"log_dups_size":100,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:36:39.055061+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110807,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":679,"num_read_kb":167118,"num_write":1082,"num_write_kb":34693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"265'4","reported_seq":1169,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:16.307268+0000","last_change":"2024-05-30T17:14:16.307355+0000","last_active":"2024-05-30T17:14:16.307268+0000","last_peered":"2024-05-30T17:14:16.307268+0000","last_clean":"2024-05-30T17:14:16.307268+0000","last_became_active":"2024-05-30T15:54:20.662371+0000","last_became_peered":"2024-05-30T15:54:20.662371+0000","last_unstale":"2024-05-30T17:14:16.307268+0000","last_undegraded":"2024-05-30T17:14:16.307268+0000","last_fullsized":"2024-05-30T17:14:16.307268+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'4","last_scrub_stamp":"2024-05-30T17:14:16.307053+0000","last_deep_scrub":"265'4","last_deep_scrub_stamp":"2024-05-30T17:14:16.307053+0000","last_clean_scrub_stamp":"2024-05-30T17:14:16.307053+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:31:23.411741+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":647,"num_read_kb":647,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"265'114","reported_seq":670,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:10.427586+0000","last_change":"2024-05-30T17:14:10.427657+0000","last_active":"2024-05-30T17:14:10.427586+0000","last_peered":"2024-05-30T17:14:10.427586+0000","last_clean":"2024-05-30T17:14:10.427586+0000","last_became_active":"2024-05-30T15:54:22.849105+0000","last_became_peered":"2024-05-30T15:54:22.849105+0000","last_unstale":"2024-05-30T17:14:10.427586+0000","last_undegraded":"2024-05-30T17:14:10.427586+0000","last_fullsized":"2024-05-30T17:14:10.427586+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'114","last_scrub_stamp":"2024-05-30T17:14:10.427409+0000","last_deep_scrub":"265'114","last_deep_scrub_stamp":"2024-05-30T17:14:10.427409+0000","last_clean_scrub_stamp":"2024-05-30T17:14:10.427409+0000","objects_scrubbed":5,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:10:36.524011+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":120,"num_read_kb":115,"num_write":80,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"265'541","reported_seq":2326,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.806417+0000","last_change":"2024-05-30T17:13:54.806572+0000","last_active":"2024-05-30T17:13:54.806417+0000","last_peered":"2024-05-30T17:13:54.806417+0000","last_clean":"2024-05-30T17:13:54.806417+0000","last_became_active":"2024-05-30T15:54:24.677291+0000","last_became_peered":"2024-05-30T15:54:24.677291+0000","last_unstale":"2024-05-30T17:13:54.806417+0000","last_undegraded":"2024-05-30T17:13:54.806417+0000","last_fullsized":"2024-05-30T17:13:54.806417+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'541","last_scrub_stamp":"2024-05-30T17:13:54.806190+0000","last_deep_scrub":"265'541","last_deep_scrub_stamp":"2024-05-30T17:13:54.806190+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.806190+0000","objects_scrubbed":1,"log_size":41,"log_dups_size":500,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T05:09:18.353958+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"265'136","reported_seq":666,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.859668+0000","last_change":"2024-05-30T17:13:49.859798+0000","last_active":"2024-05-30T17:13:49.859668+0000","last_peered":"2024-05-30T17:13:49.859668+0000","last_clean":"2024-05-30T17:13:49.859668+0000","last_became_active":"2024-05-30T15:54:26.674517+0000","last_became_peered":"2024-05-30T15:54:26.674517+0000","last_unstale":"2024-05-30T17:13:49.859668+0000","last_undegraded":"2024-05-30T17:13:49.859668+0000","last_fullsized":"2024-05-30T17:13:49.859668+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'136","last_scrub_stamp":"2024-05-30T17:13:49.859451+0000","last_deep_scrub":"265'136","last_deep_scrub_stamp":"2024-05-30T17:13:49.859451+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.859451+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:25:15.350422+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":141,"num_read_kb":129,"num_write":332,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"265'410","reported_seq":1957,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:05.044019+0000","last_change":"2024-05-30T17:14:05.044110+0000","last_active":"2024-05-30T17:14:05.044019+0000","last_peered":"2024-05-30T17:14:05.044019+0000","last_clean":"2024-05-30T17:14:05.044019+0000","last_became_active":"2024-05-30T15:54:18.657247+0000","last_became_peered":"2024-05-30T15:54:18.657247+0000","last_unstale":"2024-05-30T17:14:05.044019+0000","last_undegraded":"2024-05-30T17:14:05.044019+0000","last_fullsized":"2024-05-30T17:14:05.044019+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'410","last_scrub_stamp":"2024-05-30T17:14:05.043803+0000","last_deep_scrub":"265'410","last_deep_scrub_stamp":"2024-05-30T17:14:05.043803+0000","last_clean_scrub_stamp":"2024-05-30T17:14:05.043803+0000","objects_scrubbed":8,"log_size":110,"log_dups_size":300,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:16:53.264752+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2106,"num_read_kb":2008,"num_write":461,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":502,"num_omap_keys":2,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"265'185","reported_seq":804,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.106642+0000","last_change":"2024-05-30T17:13:45.106682+0000","last_active":"2024-05-30T17:13:45.106642+0000","last_peered":"2024-05-30T17:13:45.106642+0000","last_clean":"2024-05-30T17:13:45.106642+0000","last_became_active":"2024-05-30T15:54:54.426538+0000","last_became_peered":"2024-05-30T15:54:54.426538+0000","last_unstale":"2024-05-30T17:13:45.106642+0000","last_undegraded":"2024-05-30T17:13:45.106642+0000","last_fullsized":"2024-05-30T17:13:45.106642+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'185","last_scrub_stamp":"2024-05-30T17:13:45.106515+0000","last_deep_scrub":"265'185","last_deep_scrub_stamp":"2024-05-30T17:13:45.106515+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.106515+0000","objects_scrubbed":1,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:52:29.468974+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":443,"num_read_kb":549,"num_write":1012,"num_write_kb":15096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,7],"acting":[2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":527,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:13.044644+0000","last_change":"2024-05-30T17:14:13.044703+0000","last_active":"2024-05-30T17:14:13.044644+0000","last_peered":"2024-05-30T17:14:13.044644+0000","last_clean":"2024-05-30T17:14:13.044644+0000","last_became_active":"2024-05-30T15:54:20.669492+0000","last_became_peered":"2024-05-30T15:54:20.669492+0000","last_unstale":"2024-05-30T17:14:13.044644+0000","last_undegraded":"2024-05-30T17:14:13.044644+0000","last_fullsized":"2024-05-30T17:14:13.044644+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:13.044456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:13.044456+0000","last_clean_scrub_stamp":"2024-05-30T17:14:13.044456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:52:25.523369+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"265'243","reported_seq":890,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:01.276657+0000","last_change":"2024-05-30T17:14:01.276754+0000","last_active":"2024-05-30T17:14:01.276657+0000","last_peered":"2024-05-30T17:14:01.276657+0000","last_clean":"2024-05-30T17:14:01.276657+0000","last_became_active":"2024-05-30T15:54:22.671943+0000","last_became_peered":"2024-05-30T15:54:22.671943+0000","last_unstale":"2024-05-30T17:14:01.276657+0000","last_undegraded":"2024-05-30T17:14:01.276657+0000","last_fullsized":"2024-05-30T17:14:01.276657+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'243","last_scrub_stamp":"2024-05-30T17:14:01.276461+0000","last_deep_scrub":"265'243","last_deep_scrub_stamp":"2024-05-30T17:14:01.276461+0000","last_clean_scrub_stamp":"2024-05-30T17:14:01.276461+0000","objects_scrubbed":8,"log_size":43,"log_dups_size":200,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:31:23.133344+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":313,"num_read_kb":312,"num_write":178,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":509,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:00.785558+0000","last_change":"2024-05-30T17:14:00.785659+0000","last_active":"2024-05-30T17:14:00.785558+0000","last_peered":"2024-05-30T17:14:00.785558+0000","last_clean":"2024-05-30T17:14:00.785558+0000","last_became_active":"2024-05-30T15:54:24.681595+0000","last_became_peered":"2024-05-30T15:54:24.681595+0000","last_unstale":"2024-05-30T17:14:00.785558+0000","last_undegraded":"2024-05-30T17:14:00.785558+0000","last_fullsized":"2024-05-30T17:14:00.785558+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:00.785264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:00.785264+0000","last_clean_scrub_stamp":"2024-05-30T17:14:00.785264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:38:41.393882+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"265'119","reported_seq":635,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.989353+0000","last_change":"2024-05-30T17:13:51.989431+0000","last_active":"2024-05-30T17:13:51.989353+0000","last_peered":"2024-05-30T17:13:51.989353+0000","last_clean":"2024-05-30T17:13:51.989353+0000","last_became_active":"2024-05-30T15:54:26.688518+0000","last_became_peered":"2024-05-30T15:54:26.688518+0000","last_unstale":"2024-05-30T17:13:51.989353+0000","last_undegraded":"2024-05-30T17:13:51.989353+0000","last_fullsized":"2024-05-30T17:13:51.989353+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'119","last_scrub_stamp":"2024-05-30T17:13:51.989161+0000","last_deep_scrub":"265'119","last_deep_scrub_stamp":"2024-05-30T17:13:51.989161+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.989161+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":100,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:46:17.674786+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":619,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":113,"num_read_kb":112,"num_write":261,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"265'758","reported_seq":2277,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:57.780923+0000","last_change":"2024-05-30T17:13:57.781083+0000","last_active":"2024-05-30T17:13:57.780923+0000","last_peered":"2024-05-30T17:13:57.780923+0000","last_clean":"2024-05-30T17:13:57.780923+0000","last_became_active":"2024-05-30T15:54:18.677957+0000","last_became_peered":"2024-05-30T15:54:18.677957+0000","last_unstale":"2024-05-30T17:13:57.780923+0000","last_undegraded":"2024-05-30T17:13:57.780923+0000","last_fullsized":"2024-05-30T17:13:57.780923+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'758","last_scrub_stamp":"2024-05-30T17:13:57.780707+0000","last_deep_scrub":"265'758","last_deep_scrub_stamp":"2024-05-30T17:13:57.780707+0000","last_clean_scrub_stamp":"2024-05-30T17:13:57.780707+0000","objects_scrubbed":2,"log_size":58,"log_dups_size":700,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:16:34.692139+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3440,"num_read_kb":3078,"num_write":1010,"num_write_kb":372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"265'74","reported_seq":639,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:26.767705+0000","last_change":"2024-05-30T17:14:26.767789+0000","last_active":"2024-05-30T17:14:26.767705+0000","last_peered":"2024-05-30T17:14:26.767705+0000","last_clean":"2024-05-30T17:14:26.767705+0000","last_became_active":"2024-05-30T15:54:03.029214+0000","last_became_peered":"2024-05-30T15:54:03.029214+0000","last_unstale":"2024-05-30T17:14:26.767705+0000","last_undegraded":"2024-05-30T17:14:26.767705+0000","last_fullsized":"2024-05-30T17:14:26.767705+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'74","last_scrub_stamp":"2024-05-30T17:14:26.767514+0000","last_deep_scrub":"265'74","last_deep_scrub_stamp":"2024-05-30T17:14:26.767514+0000","last_clean_scrub_stamp":"2024-05-30T17:14:26.767514+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:51:45.091266+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":539,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:19.247044+0000","last_change":"2024-05-30T17:14:19.247196+0000","last_active":"2024-05-30T17:14:19.247044+0000","last_peered":"2024-05-30T17:14:19.247044+0000","last_clean":"2024-05-30T17:14:19.247044+0000","last_became_active":"2024-05-30T15:54:08.577556+0000","last_became_peered":"2024-05-30T15:54:08.577556+0000","last_unstale":"2024-05-30T17:14:19.247044+0000","last_undegraded":"2024-05-30T17:14:19.247044+0000","last_fullsized":"2024-05-30T17:14:19.247044+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:19.246805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:19.246805+0000","last_clean_scrub_stamp":"2024-05-30T17:14:19.246805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:51:24.222735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"265'189","reported_seq":853,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:35.464631+0000","last_change":"2024-05-30T17:14:35.464721+0000","last_active":"2024-05-30T17:14:35.464631+0000","last_peered":"2024-05-30T17:14:35.464631+0000","last_clean":"2024-05-30T17:14:35.464631+0000","last_became_active":"2024-05-30T15:54:16.674229+0000","last_became_peered":"2024-05-30T15:54:16.674229+0000","last_unstale":"2024-05-30T17:14:35.464631+0000","last_undegraded":"2024-05-30T17:14:35.464631+0000","last_fullsized":"2024-05-30T17:14:35.464631+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'189","last_scrub_stamp":"2024-05-30T17:14:35.464439+0000","last_deep_scrub":"265'189","last_deep_scrub_stamp":"2024-05-30T17:14:35.464439+0000","last_clean_scrub_stamp":"2024-05-30T17:14:35.464439+0000","objects_scrubbed":3,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:34:15.501216+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5525719,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":447,"num_read_kb":11470,"num_write":1074,"num_write_kb":25445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":521,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:16.814789+0000","last_change":"2024-05-30T17:14:16.814843+0000","last_active":"2024-05-30T17:14:16.814789+0000","last_peered":"2024-05-30T17:14:16.814789+0000","last_clean":"2024-05-30T17:14:16.814789+0000","last_became_active":"2024-05-30T15:54:20.664317+0000","last_became_peered":"2024-05-30T15:54:20.664317+0000","last_unstale":"2024-05-30T17:14:16.814789+0000","last_undegraded":"2024-05-30T17:14:16.814789+0000","last_fullsized":"2024-05-30T17:14:16.814789+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:16.814584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:16.814584+0000","last_clean_scrub_stamp":"2024-05-30T17:14:16.814584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:59:43.265365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"265'340","reported_seq":1071,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:26.372042+0000","last_change":"2024-05-30T17:14:26.372042+0000","last_active":"2024-05-30T17:14:26.372042+0000","last_peered":"2024-05-30T17:14:26.372042+0000","last_clean":"2024-05-30T17:14:26.372042+0000","last_became_active":"2024-05-30T15:54:22.669871+0000","last_became_peered":"2024-05-30T15:54:22.669871+0000","last_unstale":"2024-05-30T17:14:26.372042+0000","last_undegraded":"2024-05-30T17:14:26.372042+0000","last_fullsized":"2024-05-30T17:14:26.372042+0000","mapping_epoch":28,"log_start":"265'300","ondisk_log_start":"265'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'340","last_scrub_stamp":"2024-05-30T17:14:26.371771+0000","last_deep_scrub":"265'340","last_deep_scrub_stamp":"2024-05-30T17:14:26.371771+0000","last_clean_scrub_stamp":"2024-05-30T17:14:26.371771+0000","objects_scrubbed":6,"log_size":40,"log_dups_size":300,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:33:55.213752+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49272,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":648,"num_read_kb":960,"num_write":598,"num_write_kb":460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":99452,"num_omap_keys":240,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":517,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.868833+0000","last_change":"2024-05-30T17:13:58.868833+0000","last_active":"2024-05-30T17:13:58.868833+0000","last_peered":"2024-05-30T17:13:58.868833+0000","last_clean":"2024-05-30T17:13:58.868833+0000","last_became_active":"2024-05-30T15:54:24.684709+0000","last_became_peered":"2024-05-30T15:54:24.684709+0000","last_unstale":"2024-05-30T17:13:58.868833+0000","last_undegraded":"2024-05-30T17:13:58.868833+0000","last_fullsized":"2024-05-30T17:13:58.868833+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:58.868550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:58.868550+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.868550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:45:56.978968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"265'178","reported_seq":892,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:02.796551+0000","last_change":"2024-05-30T17:14:02.796551+0000","last_active":"2024-05-30T17:14:02.796551+0000","last_peered":"2024-05-30T17:14:02.796551+0000","last_clean":"2024-05-30T17:14:02.796551+0000","last_became_active":"2024-05-30T15:54:26.686647+0000","last_became_peered":"2024-05-30T15:54:26.686647+0000","last_unstale":"2024-05-30T17:14:02.796551+0000","last_undegraded":"2024-05-30T17:14:02.796551+0000","last_fullsized":"2024-05-30T17:14:02.796551+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'178","last_scrub_stamp":"2024-05-30T17:14:02.796290+0000","last_deep_scrub":"265'178","last_deep_scrub_stamp":"2024-05-30T17:14:02.796290+0000","last_clean_scrub_stamp":"2024-05-30T17:14:02.796290+0000","objects_scrubbed":4,"log_size":78,"log_dups_size":100,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:33:47.206690+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1020,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":416,"num_read_kb":404,"num_write":359,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":66,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"265'389","reported_seq":1873,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:14.815492+0000","last_change":"2024-05-30T17:14:14.815492+0000","last_active":"2024-05-30T17:14:14.815492+0000","last_peered":"2024-05-30T17:14:14.815492+0000","last_clean":"2024-05-30T17:14:14.815492+0000","last_became_active":"2024-05-30T15:54:18.648529+0000","last_became_peered":"2024-05-30T15:54:18.648529+0000","last_unstale":"2024-05-30T17:14:14.815492+0000","last_undegraded":"2024-05-30T17:14:14.815492+0000","last_fullsized":"2024-05-30T17:14:14.815492+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'389","last_scrub_stamp":"2024-05-30T17:14:14.815228+0000","last_deep_scrub":"265'389","last_deep_scrub_stamp":"2024-05-30T17:14:14.815228+0000","last_clean_scrub_stamp":"2024-05-30T17:14:14.815228+0000","objects_scrubbed":1,"log_size":89,"log_dups_size":300,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:17:35.526601+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2082,"num_read_kb":1967,"num_write":465,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":541,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:27.733635+0000","last_change":"2024-05-30T17:14:27.733635+0000","last_active":"2024-05-30T17:14:27.733635+0000","last_peered":"2024-05-30T17:14:27.733635+0000","last_clean":"2024-05-30T17:14:27.733635+0000","last_became_active":"2024-05-30T15:54:08.574645+0000","last_became_peered":"2024-05-30T15:54:08.574645+0000","last_unstale":"2024-05-30T17:14:27.733635+0000","last_undegraded":"2024-05-30T17:14:27.733635+0000","last_fullsized":"2024-05-30T17:14:27.733635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:27.733385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:27.733385+0000","last_clean_scrub_stamp":"2024-05-30T17:14:27.733385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:08:57.081739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"265'174","reported_seq":818,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:24.863029+0000","last_change":"2024-05-30T17:14:24.863126+0000","last_active":"2024-05-30T17:14:24.863029+0000","last_peered":"2024-05-30T17:14:24.863029+0000","last_clean":"2024-05-30T17:14:24.863029+0000","last_became_active":"2024-05-30T15:54:16.678386+0000","last_became_peered":"2024-05-30T15:54:16.678386+0000","last_unstale":"2024-05-30T17:14:24.863029+0000","last_undegraded":"2024-05-30T17:14:24.863029+0000","last_fullsized":"2024-05-30T17:14:24.863029+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'174","last_scrub_stamp":"2024-05-30T17:14:24.862823+0000","last_deep_scrub":"265'174","last_deep_scrub_stamp":"2024-05-30T17:14:24.862823+0000","last_clean_scrub_stamp":"2024-05-30T17:14:24.862823+0000","objects_scrubbed":4,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:59:03.844221+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079916,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":364,"num_read_kb":7665,"num_write":1082,"num_write_kb":33234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":520,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:14.305625+0000","last_change":"2024-05-30T17:14:14.305625+0000","last_active":"2024-05-30T17:14:14.305625+0000","last_peered":"2024-05-30T17:14:14.305625+0000","last_clean":"2024-05-30T17:14:14.305625+0000","last_became_active":"2024-05-30T15:54:20.661502+0000","last_became_peered":"2024-05-30T15:54:20.661502+0000","last_unstale":"2024-05-30T17:14:14.305625+0000","last_undegraded":"2024-05-30T17:14:14.305625+0000","last_fullsized":"2024-05-30T17:14:14.305625+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:14.305372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:14.305372+0000","last_clean_scrub_stamp":"2024-05-30T17:14:14.305372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T05:04:18.082051+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"265'730","reported_seq":2569,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:04.002916+0000","last_change":"2024-05-30T17:14:04.003007+0000","last_active":"2024-05-30T17:14:04.002916+0000","last_peered":"2024-05-30T17:14:04.002916+0000","last_clean":"2024-05-30T17:14:04.002916+0000","last_became_active":"2024-05-30T15:54:22.671588+0000","last_became_peered":"2024-05-30T15:54:22.671588+0000","last_unstale":"2024-05-30T17:14:04.002916+0000","last_undegraded":"2024-05-30T17:14:04.002916+0000","last_fullsized":"2024-05-30T17:14:04.002916+0000","mapping_epoch":28,"log_start":"265'700","ondisk_log_start":"265'700","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'730","last_scrub_stamp":"2024-05-30T17:14:04.002697+0000","last_deep_scrub":"265'730","last_deep_scrub_stamp":"2024-05-30T17:14:04.002697+0000","last_clean_scrub_stamp":"2024-05-30T17:14:04.002697+0000","objects_scrubbed":12,"log_size":30,"log_dups_size":700,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:14:57.807605+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19819,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":1008,"num_read_kb":1208,"num_write":324,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6022,"num_omap_keys":18,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":513,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:02.668077+0000","last_change":"2024-05-30T17:14:02.668129+0000","last_active":"2024-05-30T17:14:02.668077+0000","last_peered":"2024-05-30T17:14:02.668077+0000","last_clean":"2024-05-30T17:14:02.668077+0000","last_became_active":"2024-05-30T15:54:24.686641+0000","last_became_peered":"2024-05-30T15:54:24.686641+0000","last_unstale":"2024-05-30T17:14:02.668077+0000","last_undegraded":"2024-05-30T17:14:02.668077+0000","last_fullsized":"2024-05-30T17:14:02.668077+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:02.667880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:02.667880+0000","last_clean_scrub_stamp":"2024-05-30T17:14:02.667880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:55:41.680344+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"265'125","reported_seq":645,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.985096+0000","last_change":"2024-05-30T17:13:54.985175+0000","last_active":"2024-05-30T17:13:54.985096+0000","last_peered":"2024-05-30T17:13:54.985096+0000","last_clean":"2024-05-30T17:13:54.985096+0000","last_became_active":"2024-05-30T15:54:26.667751+0000","last_became_peered":"2024-05-30T15:54:26.667751+0000","last_unstale":"2024-05-30T17:13:54.985096+0000","last_undegraded":"2024-05-30T17:13:54.985096+0000","last_fullsized":"2024-05-30T17:13:54.985096+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'125","last_scrub_stamp":"2024-05-30T17:13:54.984877+0000","last_deep_scrub":"265'125","last_deep_scrub_stamp":"2024-05-30T17:13:54.984877+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.984877+0000","objects_scrubbed":2,"log_size":25,"log_dups_size":100,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:40:14.213968+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":438,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":115,"num_write":291,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"265'328","reported_seq":1658,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:07.012769+0000","last_change":"2024-05-30T17:14:07.012769+0000","last_active":"2024-05-30T17:14:07.012769+0000","last_peered":"2024-05-30T17:14:07.012769+0000","last_clean":"2024-05-30T17:14:07.012769+0000","last_became_active":"2024-05-30T15:54:18.651853+0000","last_became_peered":"2024-05-30T15:54:18.651853+0000","last_unstale":"2024-05-30T17:14:07.012769+0000","last_undegraded":"2024-05-30T17:14:07.012769+0000","last_fullsized":"2024-05-30T17:14:07.012769+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'328","last_scrub_stamp":"2024-05-30T17:14:07.012480+0000","last_deep_scrub":"265'328","last_deep_scrub_stamp":"2024-05-30T17:14:07.012480+0000","last_clean_scrub_stamp":"2024-05-30T17:14:07.012480+0000","objects_scrubbed":2,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:54:27.007726+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1727,"num_read_kb":1636,"num_write":373,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":542,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.261872+0000","last_change":"2024-05-30T17:13:59.261974+0000","last_active":"2024-05-30T17:13:59.261872+0000","last_peered":"2024-05-30T17:13:59.261872+0000","last_clean":"2024-05-30T17:13:59.261872+0000","last_became_active":"2024-05-30T15:54:08.744898+0000","last_became_peered":"2024-05-30T15:54:08.744898+0000","last_unstale":"2024-05-30T17:13:59.261872+0000","last_undegraded":"2024-05-30T17:13:59.261872+0000","last_fullsized":"2024-05-30T17:13:59.261872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:59.261650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:59.261650+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.261650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:03:48.512476+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"265'185","reported_seq":823,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:13.997191+0000","last_change":"2024-05-30T17:14:13.997191+0000","last_active":"2024-05-30T17:14:13.997191+0000","last_peered":"2024-05-30T17:14:13.997191+0000","last_clean":"2024-05-30T17:14:13.997191+0000","last_became_active":"2024-05-30T15:54:16.650214+0000","last_became_peered":"2024-05-30T15:54:16.650214+0000","last_unstale":"2024-05-30T17:14:13.997191+0000","last_undegraded":"2024-05-30T17:14:13.997191+0000","last_fullsized":"2024-05-30T17:14:13.997191+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'185","last_scrub_stamp":"2024-05-30T17:14:13.996657+0000","last_deep_scrub":"265'185","last_deep_scrub_stamp":"2024-05-30T17:14:13.996657+0000","last_clean_scrub_stamp":"2024-05-30T17:14:13.996657+0000","objects_scrubbed":9,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:48:29.511347+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1911070,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":412,"num_read_kb":8332,"num_write":1154,"num_write_kb":25808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"265'2","reported_seq":525,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.146675+0000","last_change":"2024-05-30T17:13:50.146675+0000","last_active":"2024-05-30T17:13:50.146675+0000","last_peered":"2024-05-30T17:13:50.146675+0000","last_clean":"2024-05-30T17:13:50.146675+0000","last_became_active":"2024-05-30T15:54:20.666715+0000","last_became_peered":"2024-05-30T15:54:20.666715+0000","last_unstale":"2024-05-30T17:13:50.146675+0000","last_undegraded":"2024-05-30T17:13:50.146675+0000","last_fullsized":"2024-05-30T17:13:50.146675+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:13:50.146258+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:13:50.146258+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.146258+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:41:50.441025+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"265'268","reported_seq":1790,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:07.764156+0000","last_change":"2024-05-30T17:14:07.764244+0000","last_active":"2024-05-30T17:14:07.764156+0000","last_peered":"2024-05-30T17:14:07.764156+0000","last_clean":"2024-05-30T17:14:07.764156+0000","last_became_active":"2024-05-30T15:54:22.852604+0000","last_became_peered":"2024-05-30T15:54:22.852604+0000","last_unstale":"2024-05-30T17:14:07.764156+0000","last_undegraded":"2024-05-30T17:14:07.764156+0000","last_fullsized":"2024-05-30T17:14:07.764156+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'268","last_scrub_stamp":"2024-05-30T17:14:07.763964+0000","last_deep_scrub":"265'268","last_deep_scrub_stamp":"2024-05-30T17:14:07.763964+0000","last_clean_scrub_stamp":"2024-05-30T17:14:07.763964+0000","objects_scrubbed":9,"log_size":68,"log_dups_size":200,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:30:31.072037+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":232,"num_read_kb":224,"num_write":183,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5824,"num_omap_keys":14,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":518,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:00.884506+0000","last_change":"2024-05-30T17:14:00.884643+0000","last_active":"2024-05-30T17:14:00.884506+0000","last_peered":"2024-05-30T17:14:00.884506+0000","last_clean":"2024-05-30T17:14:00.884506+0000","last_became_active":"2024-05-30T15:54:24.679830+0000","last_became_peered":"2024-05-30T15:54:24.679830+0000","last_unstale":"2024-05-30T17:14:00.884506+0000","last_undegraded":"2024-05-30T17:14:00.884506+0000","last_fullsized":"2024-05-30T17:14:00.884506+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:00.884270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:00.884270+0000","last_clean_scrub_stamp":"2024-05-30T17:14:00.884270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:18:06.001324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"265'253","reported_seq":871,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.356811+0000","last_change":"2024-05-30T17:13:56.356879+0000","last_active":"2024-05-30T17:13:56.356811+0000","last_peered":"2024-05-30T17:13:56.356811+0000","last_clean":"2024-05-30T17:13:56.356811+0000","last_became_active":"2024-05-30T15:54:26.696477+0000","last_became_peered":"2024-05-30T15:54:26.696477+0000","last_unstale":"2024-05-30T17:13:56.356811+0000","last_undegraded":"2024-05-30T17:13:56.356811+0000","last_fullsized":"2024-05-30T17:13:56.356811+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'253","last_scrub_stamp":"2024-05-30T17:13:56.356623+0000","last_deep_scrub":"265'253","last_deep_scrub_stamp":"2024-05-30T17:13:56.356623+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.356623+0000","objects_scrubbed":2,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:15:54.649548+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":476,"num_read_kb":500,"num_write":542,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1193,"num_omap_keys":7,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"265'766","reported_seq":2335,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:11.094921+0000","last_change":"2024-05-30T17:14:11.095077+0000","last_active":"2024-05-30T17:14:11.094921+0000","last_peered":"2024-05-30T17:14:11.094921+0000","last_clean":"2024-05-30T17:14:11.094921+0000","last_became_active":"2024-05-30T15:54:18.639830+0000","last_became_peered":"2024-05-30T15:54:18.639830+0000","last_unstale":"2024-05-30T17:14:11.094921+0000","last_undegraded":"2024-05-30T17:14:11.094921+0000","last_fullsized":"2024-05-30T17:14:11.094921+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'766","last_scrub_stamp":"2024-05-30T17:14:11.094660+0000","last_deep_scrub":"265'766","last_deep_scrub_stamp":"2024-05-30T17:14:11.094660+0000","last_clean_scrub_stamp":"2024-05-30T17:14:11.094660+0000","objects_scrubbed":5,"log_size":66,"log_dups_size":700,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:57:27.825458+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3481,"num_read_kb":3132,"num_write":1014,"num_write_kb":367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":539,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:18.271712+0000","last_change":"2024-05-30T17:14:18.271770+0000","last_active":"2024-05-30T17:14:18.271712+0000","last_peered":"2024-05-30T17:14:18.271712+0000","last_clean":"2024-05-30T17:14:18.271712+0000","last_became_active":"2024-05-30T15:54:08.574817+0000","last_became_peered":"2024-05-30T15:54:08.574817+0000","last_unstale":"2024-05-30T17:14:18.271712+0000","last_undegraded":"2024-05-30T17:14:18.271712+0000","last_fullsized":"2024-05-30T17:14:18.271712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:18.271517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:18.271517+0000","last_clean_scrub_stamp":"2024-05-30T17:14:18.271517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:45:29.263529+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"265'199","reported_seq":923,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:17.027327+0000","last_change":"2024-05-30T17:14:17.027406+0000","last_active":"2024-05-30T17:14:17.027327+0000","last_peered":"2024-05-30T17:14:17.027327+0000","last_clean":"2024-05-30T17:14:17.027327+0000","last_became_active":"2024-05-30T15:54:16.655347+0000","last_became_peered":"2024-05-30T15:54:16.655347+0000","last_unstale":"2024-05-30T17:14:17.027327+0000","last_undegraded":"2024-05-30T17:14:17.027327+0000","last_fullsized":"2024-05-30T17:14:17.027327+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'199","last_scrub_stamp":"2024-05-30T17:14:17.027124+0000","last_deep_scrub":"265'199","last_deep_scrub_stamp":"2024-05-30T17:14:17.027124+0000","last_clean_scrub_stamp":"2024-05-30T17:14:17.027124+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:00:55.705228+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":572,"num_read_kb":42208,"num_write":1148,"num_write_kb":36370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":526,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:13.829376+0000","last_change":"2024-05-30T17:14:13.829430+0000","last_active":"2024-05-30T17:14:13.829376+0000","last_peered":"2024-05-30T17:14:13.829376+0000","last_clean":"2024-05-30T17:14:13.829376+0000","last_became_active":"2024-05-30T15:54:20.651449+0000","last_became_peered":"2024-05-30T15:54:20.651449+0000","last_unstale":"2024-05-30T17:14:13.829376+0000","last_undegraded":"2024-05-30T17:14:13.829376+0000","last_fullsized":"2024-05-30T17:14:13.829376+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:13.829176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:13.829176+0000","last_clean_scrub_stamp":"2024-05-30T17:14:13.829176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:22:24.775288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"265'167","reported_seq":749,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:12.373071+0000","last_change":"2024-05-30T17:14:12.373071+0000","last_active":"2024-05-30T17:14:12.373071+0000","last_peered":"2024-05-30T17:14:12.373071+0000","last_clean":"2024-05-30T17:14:12.373071+0000","last_became_active":"2024-05-30T15:54:22.673881+0000","last_became_peered":"2024-05-30T15:54:22.673881+0000","last_unstale":"2024-05-30T17:14:12.373071+0000","last_undegraded":"2024-05-30T17:14:12.373071+0000","last_fullsized":"2024-05-30T17:14:12.373071+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'167","last_scrub_stamp":"2024-05-30T17:14:12.372906+0000","last_deep_scrub":"265'167","last_deep_scrub_stamp":"2024-05-30T17:14:12.372906+0000","last_clean_scrub_stamp":"2024-05-30T17:14:12.372906+0000","objects_scrubbed":7,"log_size":67,"log_dups_size":100,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:29:18.523344+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":200,"num_read_kb":192,"num_write":144,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1630,"num_omap_keys":4,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":511,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.801223+0000","last_change":"2024-05-30T17:13:56.801223+0000","last_active":"2024-05-30T17:13:56.801223+0000","last_peered":"2024-05-30T17:13:56.801223+0000","last_clean":"2024-05-30T17:13:56.801223+0000","last_became_active":"2024-05-30T15:54:24.675916+0000","last_became_peered":"2024-05-30T15:54:24.675916+0000","last_unstale":"2024-05-30T17:13:56.801223+0000","last_undegraded":"2024-05-30T17:13:56.801223+0000","last_fullsized":"2024-05-30T17:13:56.801223+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:56.800964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:56.800964+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.800964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:50:34.435231+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"265'139","reported_seq":667,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.262568+0000","last_change":"2024-05-30T17:13:49.262568+0000","last_active":"2024-05-30T17:13:49.262568+0000","last_peered":"2024-05-30T17:13:49.262568+0000","last_clean":"2024-05-30T17:13:49.262568+0000","last_became_active":"2024-05-30T15:54:26.704217+0000","last_became_peered":"2024-05-30T15:54:26.704217+0000","last_unstale":"2024-05-30T17:13:49.262568+0000","last_undegraded":"2024-05-30T17:13:49.262568+0000","last_fullsized":"2024-05-30T17:13:49.262568+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'139","last_scrub_stamp":"2024-05-30T17:13:49.262162+0000","last_deep_scrub":"265'139","last_deep_scrub_stamp":"2024-05-30T17:13:49.262162+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.262162+0000","objects_scrubbed":2,"log_size":39,"log_dups_size":100,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:30:17.555273+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":834,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":138,"num_read_kb":136,"num_write":341,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"265'398","reported_seq":1838,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:03.013211+0000","last_change":"2024-05-30T17:14:03.013307+0000","last_active":"2024-05-30T17:14:03.013211+0000","last_peered":"2024-05-30T17:14:03.013211+0000","last_clean":"2024-05-30T17:14:03.013211+0000","last_became_active":"2024-05-30T15:54:18.665034+0000","last_became_peered":"2024-05-30T15:54:18.665034+0000","last_unstale":"2024-05-30T17:14:03.013211+0000","last_undegraded":"2024-05-30T17:14:03.013211+0000","last_fullsized":"2024-05-30T17:14:03.013211+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'398","last_scrub_stamp":"2024-05-30T17:14:03.013020+0000","last_deep_scrub":"265'398","last_deep_scrub_stamp":"2024-05-30T17:14:03.013020+0000","last_clean_scrub_stamp":"2024-05-30T17:14:03.013020+0000","objects_scrubbed":2,"log_size":98,"log_dups_size":300,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:58:48.490896+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2053,"num_read_kb":1920,"num_write":476,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"265'217","reported_seq":867,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:21.298652+0000","last_change":"2024-05-30T17:14:21.298756+0000","last_active":"2024-05-30T17:14:21.298652+0000","last_peered":"2024-05-30T17:14:21.298652+0000","last_clean":"2024-05-30T17:14:21.298652+0000","last_became_active":"2024-05-30T15:54:16.650735+0000","last_became_peered":"2024-05-30T15:54:16.650735+0000","last_unstale":"2024-05-30T17:14:21.298652+0000","last_undegraded":"2024-05-30T17:14:21.298652+0000","last_fullsized":"2024-05-30T17:14:21.298652+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'217","last_scrub_stamp":"2024-05-30T17:14:21.298453+0000","last_deep_scrub":"265'217","last_deep_scrub_stamp":"2024-05-30T17:14:21.298453+0000","last_clean_scrub_stamp":"2024-05-30T17:14:21.298453+0000","objects_scrubbed":6,"log_size":17,"log_dups_size":200,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:06:06.658095+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079967,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":420,"num_read_kb":5617,"num_write":983,"num_write_kb":15732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":540,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:12.043129+0000","last_change":"2024-05-30T17:14:12.043183+0000","last_active":"2024-05-30T17:14:12.043129+0000","last_peered":"2024-05-30T17:14:12.043129+0000","last_clean":"2024-05-30T17:14:12.043129+0000","last_became_active":"2024-05-30T15:54:08.567900+0000","last_became_peered":"2024-05-30T15:54:08.567900+0000","last_unstale":"2024-05-30T17:14:12.043129+0000","last_undegraded":"2024-05-30T17:14:12.043129+0000","last_fullsized":"2024-05-30T17:14:12.043129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:12.042933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:12.042933+0000","last_clean_scrub_stamp":"2024-05-30T17:14:12.042933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:36:17.591809+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":526,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:15.771566+0000","last_change":"2024-05-30T17:14:15.771620+0000","last_active":"2024-05-30T17:14:15.771566+0000","last_peered":"2024-05-30T17:14:15.771566+0000","last_clean":"2024-05-30T17:14:15.771566+0000","last_became_active":"2024-05-30T15:54:20.653790+0000","last_became_peered":"2024-05-30T15:54:20.653790+0000","last_unstale":"2024-05-30T17:14:15.771566+0000","last_undegraded":"2024-05-30T17:14:15.771566+0000","last_fullsized":"2024-05-30T17:14:15.771566+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:15.771368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:15.771368+0000","last_clean_scrub_stamp":"2024-05-30T17:14:15.771368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:07:25.961584+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"265'371","reported_seq":1080,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:08.931637+0000","last_change":"2024-05-30T17:14:08.931785+0000","last_active":"2024-05-30T17:14:08.931637+0000","last_peered":"2024-05-30T17:14:08.931637+0000","last_clean":"2024-05-30T17:14:08.931637+0000","last_became_active":"2024-05-30T15:54:22.673903+0000","last_became_peered":"2024-05-30T15:54:22.673903+0000","last_unstale":"2024-05-30T17:14:08.931637+0000","last_undegraded":"2024-05-30T17:14:08.931637+0000","last_fullsized":"2024-05-30T17:14:08.931637+0000","mapping_epoch":28,"log_start":"265'300","ondisk_log_start":"265'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'371","last_scrub_stamp":"2024-05-30T17:14:08.931442+0000","last_deep_scrub":"265'371","last_deep_scrub_stamp":"2024-05-30T17:14:08.931442+0000","last_clean_scrub_stamp":"2024-05-30T17:14:08.931442+0000","objects_scrubbed":10,"log_size":71,"log_dups_size":300,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:46:35.284674+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11442,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1140,"num_read_kb":1226,"num_write":1674,"num_write_kb":1475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":496492,"num_omap_keys":1134,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":522,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:55.903746+0000","last_change":"2024-05-30T17:13:55.903798+0000","last_active":"2024-05-30T17:13:55.903746+0000","last_peered":"2024-05-30T17:13:55.903746+0000","last_clean":"2024-05-30T17:13:55.903746+0000","last_became_active":"2024-05-30T15:54:24.677198+0000","last_became_peered":"2024-05-30T15:54:24.677198+0000","last_unstale":"2024-05-30T17:13:55.903746+0000","last_undegraded":"2024-05-30T17:13:55.903746+0000","last_fullsized":"2024-05-30T17:13:55.903746+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:55.903559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:55.903559+0000","last_clean_scrub_stamp":"2024-05-30T17:13:55.903559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:20:32.475018+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"265'122","reported_seq":649,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:57.847760+0000","last_change":"2024-05-30T17:13:57.847843+0000","last_active":"2024-05-30T17:13:57.847760+0000","last_peered":"2024-05-30T17:13:57.847760+0000","last_clean":"2024-05-30T17:13:57.847760+0000","last_became_active":"2024-05-30T15:54:26.676078+0000","last_became_peered":"2024-05-30T15:54:26.676078+0000","last_unstale":"2024-05-30T17:13:57.847760+0000","last_undegraded":"2024-05-30T17:13:57.847760+0000","last_fullsized":"2024-05-30T17:13:57.847760+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'122","last_scrub_stamp":"2024-05-30T17:13:57.847569+0000","last_deep_scrub":"265'122","last_deep_scrub_stamp":"2024-05-30T17:13:57.847569+0000","last_clean_scrub_stamp":"2024-05-30T17:13:57.847569+0000","objects_scrubbed":5,"log_size":22,"log_dups_size":100,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:27:31.982550+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":727,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":114,"num_read_kb":108,"num_write":274,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":73,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"265'797","reported_seq":2438,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:20.820346+0000","last_change":"2024-05-30T17:14:20.820437+0000","last_active":"2024-05-30T17:14:20.820346+0000","last_peered":"2024-05-30T17:14:20.820346+0000","last_clean":"2024-05-30T17:14:20.820346+0000","last_became_active":"2024-05-30T15:54:18.642683+0000","last_became_peered":"2024-05-30T15:54:18.642683+0000","last_unstale":"2024-05-30T17:14:20.820346+0000","last_undegraded":"2024-05-30T17:14:20.820346+0000","last_fullsized":"2024-05-30T17:14:20.820346+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'797","last_scrub_stamp":"2024-05-30T17:14:20.820153+0000","last_deep_scrub":"265'797","last_deep_scrub_stamp":"2024-05-30T17:14:20.820153+0000","last_clean_scrub_stamp":"2024-05-30T17:14:20.820153+0000","objects_scrubbed":6,"log_size":97,"log_dups_size":700,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:01:34.756808+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3596,"num_read_kb":3243,"num_write":1037,"num_write_kb":360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1011,"num_omap_keys":4,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"265'184","reported_seq":836,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:16.998554+0000","last_change":"2024-05-30T17:14:16.998697+0000","last_active":"2024-05-30T17:14:16.998554+0000","last_peered":"2024-05-30T17:14:16.998554+0000","last_clean":"2024-05-30T17:14:16.998554+0000","last_became_active":"2024-05-30T15:54:16.663360+0000","last_became_peered":"2024-05-30T15:54:16.663360+0000","last_unstale":"2024-05-30T17:14:16.998554+0000","last_undegraded":"2024-05-30T17:14:16.998554+0000","last_fullsized":"2024-05-30T17:14:16.998554+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'184","last_scrub_stamp":"2024-05-30T17:14:16.998338+0000","last_deep_scrub":"265'184","last_deep_scrub_stamp":"2024-05-30T17:14:16.998338+0000","last_clean_scrub_stamp":"2024-05-30T17:14:16.998338+0000","objects_scrubbed":6,"log_size":84,"log_dups_size":100,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:35:16.806460+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":661325,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":407,"num_read_kb":30298,"num_write":1063,"num_write_kb":42840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":539,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:28.740933+0000","last_change":"2024-05-30T17:14:28.740986+0000","last_active":"2024-05-30T17:14:28.740933+0000","last_peered":"2024-05-30T17:14:28.740933+0000","last_clean":"2024-05-30T17:14:28.740933+0000","last_became_active":"2024-05-30T15:54:08.576779+0000","last_became_peered":"2024-05-30T15:54:08.576779+0000","last_unstale":"2024-05-30T17:14:28.740933+0000","last_undegraded":"2024-05-30T17:14:28.740933+0000","last_fullsized":"2024-05-30T17:14:28.740933+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:28.740742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:28.740742+0000","last_clean_scrub_stamp":"2024-05-30T17:14:28.740742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:21:53.610258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":520,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:55.817272+0000","last_change":"2024-05-30T17:13:55.817326+0000","last_active":"2024-05-30T17:13:55.817272+0000","last_peered":"2024-05-30T17:13:55.817272+0000","last_clean":"2024-05-30T17:13:55.817272+0000","last_became_active":"2024-05-30T15:54:20.655220+0000","last_became_peered":"2024-05-30T15:54:20.655220+0000","last_unstale":"2024-05-30T17:13:55.817272+0000","last_undegraded":"2024-05-30T17:13:55.817272+0000","last_fullsized":"2024-05-30T17:13:55.817272+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:55.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:55.817064+0000","last_clean_scrub_stamp":"2024-05-30T17:13:55.817064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:12:59.140457+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"265'204","reported_seq":810,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:06.066922+0000","last_change":"2024-05-30T17:14:06.067009+0000","last_active":"2024-05-30T17:14:06.066922+0000","last_peered":"2024-05-30T17:14:06.066922+0000","last_clean":"2024-05-30T17:14:06.066922+0000","last_became_active":"2024-05-30T15:54:22.851338+0000","last_became_peered":"2024-05-30T15:54:22.851338+0000","last_unstale":"2024-05-30T17:14:06.066922+0000","last_undegraded":"2024-05-30T17:14:06.066922+0000","last_fullsized":"2024-05-30T17:14:06.066922+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'204","last_scrub_stamp":"2024-05-30T17:14:06.066726+0000","last_deep_scrub":"265'204","last_deep_scrub_stamp":"2024-05-30T17:14:06.066726+0000","last_clean_scrub_stamp":"2024-05-30T17:14:06.066726+0000","objects_scrubbed":7,"log_size":104,"log_dups_size":100,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:29:42.457191+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":256,"num_read_kb":249,"num_write":168,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"265'557","reported_seq":2775,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:57.243776+0000","last_change":"2024-05-30T17:13:57.243776+0000","last_active":"2024-05-30T17:13:57.243776+0000","last_peered":"2024-05-30T17:13:57.243776+0000","last_clean":"2024-05-30T17:13:57.243776+0000","last_became_active":"2024-05-30T15:54:24.677176+0000","last_became_peered":"2024-05-30T15:54:24.677176+0000","last_unstale":"2024-05-30T17:13:57.243776+0000","last_undegraded":"2024-05-30T17:13:57.243776+0000","last_fullsized":"2024-05-30T17:13:57.243776+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'557","last_scrub_stamp":"2024-05-30T17:13:57.243509+0000","last_deep_scrub":"265'557","last_deep_scrub_stamp":"2024-05-30T17:13:57.243509+0000","last_clean_scrub_stamp":"2024-05-30T17:13:57.243509+0000","objects_scrubbed":1,"log_size":57,"log_dups_size":500,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:04:49.675199+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"265'131","reported_seq":647,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.355450+0000","last_change":"2024-05-30T17:13:58.355552+0000","last_active":"2024-05-30T17:13:58.355450+0000","last_peered":"2024-05-30T17:13:58.355450+0000","last_clean":"2024-05-30T17:13:58.355450+0000","last_became_active":"2024-05-30T15:54:26.697294+0000","last_became_peered":"2024-05-30T15:54:26.697294+0000","last_unstale":"2024-05-30T17:13:58.355450+0000","last_undegraded":"2024-05-30T17:13:58.355450+0000","last_fullsized":"2024-05-30T17:13:58.355450+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'131","last_scrub_stamp":"2024-05-30T17:13:58.355257+0000","last_deep_scrub":"265'131","last_deep_scrub_stamp":"2024-05-30T17:13:58.355257+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.355257+0000","objects_scrubbed":7,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:43:23.516003+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1933,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":112,"num_read_kb":106,"num_write":288,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":64,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"265'720","reported_seq":2124,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:22.405228+0000","last_change":"2024-05-30T17:14:22.405228+0000","last_active":"2024-05-30T17:14:22.405228+0000","last_peered":"2024-05-30T17:14:22.405228+0000","last_clean":"2024-05-30T17:14:22.405228+0000","last_became_active":"2024-05-30T15:54:18.652981+0000","last_became_peered":"2024-05-30T15:54:18.652981+0000","last_unstale":"2024-05-30T17:14:22.405228+0000","last_undegraded":"2024-05-30T17:14:22.405228+0000","last_fullsized":"2024-05-30T17:14:22.405228+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'720","last_scrub_stamp":"2024-05-30T17:14:22.404963+0000","last_deep_scrub":"265'720","last_deep_scrub_stamp":"2024-05-30T17:14:22.404963+0000","last_clean_scrub_stamp":"2024-05-30T17:14:22.404963+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":700,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T05:09:41.233665+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3098,"num_read_kb":2766,"num_write":954,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":272,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"265'200","reported_seq":872,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:29.436665+0000","last_change":"2024-05-30T17:14:29.436784+0000","last_active":"2024-05-30T17:14:29.436665+0000","last_peered":"2024-05-30T17:14:29.436665+0000","last_clean":"2024-05-30T17:14:29.436665+0000","last_became_active":"2024-05-30T15:54:16.652033+0000","last_became_peered":"2024-05-30T15:54:16.652033+0000","last_unstale":"2024-05-30T17:14:29.436665+0000","last_undegraded":"2024-05-30T17:14:29.436665+0000","last_fullsized":"2024-05-30T17:14:29.436665+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'200","last_scrub_stamp":"2024-05-30T17:14:29.436451+0000","last_deep_scrub":"265'200","last_deep_scrub_stamp":"2024-05-30T17:14:29.436451+0000","last_clean_scrub_stamp":"2024-05-30T17:14:29.436451+0000","objects_scrubbed":5,"log_size":100,"log_dups_size":100,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:03:07.624018+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":287689,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":515,"num_read_kb":15333,"num_write":1052,"num_write_kb":22470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":539,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:13.005555+0000","last_change":"2024-05-30T17:14:13.005555+0000","last_active":"2024-05-30T17:14:13.005555+0000","last_peered":"2024-05-30T17:14:13.005555+0000","last_clean":"2024-05-30T17:14:13.005555+0000","last_became_active":"2024-05-30T15:54:08.573754+0000","last_became_peered":"2024-05-30T15:54:08.573754+0000","last_unstale":"2024-05-30T17:14:13.005555+0000","last_undegraded":"2024-05-30T17:14:13.005555+0000","last_fullsized":"2024-05-30T17:14:13.005555+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:13.005314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:13.005314+0000","last_clean_scrub_stamp":"2024-05-30T17:14:13.005314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:57:00.677681+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":530,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:09.067533+0000","last_change":"2024-05-30T17:14:09.067590+0000","last_active":"2024-05-30T17:14:09.067533+0000","last_peered":"2024-05-30T17:14:09.067533+0000","last_clean":"2024-05-30T17:14:09.067533+0000","last_became_active":"2024-05-30T15:54:20.656941+0000","last_became_peered":"2024-05-30T15:54:20.656941+0000","last_unstale":"2024-05-30T17:14:09.067533+0000","last_undegraded":"2024-05-30T17:14:09.067533+0000","last_fullsized":"2024-05-30T17:14:09.067533+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:09.067331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:09.067331+0000","last_clean_scrub_stamp":"2024-05-30T17:14:09.067331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:25:27.763720+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"265'135","reported_seq":717,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:57.901626+0000","last_change":"2024-05-30T17:13:57.901801+0000","last_active":"2024-05-30T17:13:57.901626+0000","last_peered":"2024-05-30T17:13:57.901626+0000","last_clean":"2024-05-30T17:13:57.901626+0000","last_became_active":"2024-05-30T15:54:22.674027+0000","last_became_peered":"2024-05-30T15:54:22.674027+0000","last_unstale":"2024-05-30T17:13:57.901626+0000","last_undegraded":"2024-05-30T17:13:57.901626+0000","last_fullsized":"2024-05-30T17:13:57.901626+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'135","last_scrub_stamp":"2024-05-30T17:13:57.901361+0000","last_deep_scrub":"265'135","last_deep_scrub_stamp":"2024-05-30T17:13:57.901361+0000","last_clean_scrub_stamp":"2024-05-30T17:13:57.901361+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":100,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:02:52.230529+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":164,"num_read_kb":159,"num_write":108,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"265'550","reported_seq":2642,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:08.404324+0000","last_change":"2024-05-30T17:14:08.404392+0000","last_active":"2024-05-30T17:14:08.404324+0000","last_peered":"2024-05-30T17:14:08.404324+0000","last_clean":"2024-05-30T17:14:08.404324+0000","last_became_active":"2024-05-30T15:54:24.676608+0000","last_became_peered":"2024-05-30T15:54:24.676608+0000","last_unstale":"2024-05-30T17:14:08.404324+0000","last_undegraded":"2024-05-30T17:14:08.404324+0000","last_fullsized":"2024-05-30T17:14:08.404324+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'550","last_scrub_stamp":"2024-05-30T17:14:08.404133+0000","last_deep_scrub":"265'550","last_deep_scrub_stamp":"2024-05-30T17:14:08.404133+0000","last_clean_scrub_stamp":"2024-05-30T17:14:08.404133+0000","objects_scrubbed":1,"log_size":50,"log_dups_size":500,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:32:14.717679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"265'233","reported_seq":789,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.655070+0000","last_change":"2024-05-30T17:13:50.655156+0000","last_active":"2024-05-30T17:13:50.655070+0000","last_peered":"2024-05-30T17:13:50.655070+0000","last_clean":"2024-05-30T17:13:50.655070+0000","last_became_active":"2024-05-30T15:54:26.699148+0000","last_became_peered":"2024-05-30T15:54:26.699148+0000","last_unstale":"2024-05-30T17:13:50.655070+0000","last_undegraded":"2024-05-30T17:13:50.655070+0000","last_fullsized":"2024-05-30T17:13:50.655070+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'233","last_scrub_stamp":"2024-05-30T17:13:50.654846+0000","last_deep_scrub":"265'233","last_deep_scrub_stamp":"2024-05-30T17:13:50.654846+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.654846+0000","objects_scrubbed":6,"log_size":33,"log_dups_size":200,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:23:30.288859+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":326,"num_read_kb":298,"num_write":504,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":178,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"265'748","reported_seq":2349,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:08.073027+0000","last_change":"2024-05-30T17:14:08.073095+0000","last_active":"2024-05-30T17:14:08.073027+0000","last_peered":"2024-05-30T17:14:08.073027+0000","last_clean":"2024-05-30T17:14:08.073027+0000","last_became_active":"2024-05-30T15:54:18.657842+0000","last_became_peered":"2024-05-30T15:54:18.657842+0000","last_unstale":"2024-05-30T17:14:08.073027+0000","last_undegraded":"2024-05-30T17:14:08.073027+0000","last_fullsized":"2024-05-30T17:14:08.073027+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'748","last_scrub_stamp":"2024-05-30T17:14:08.072843+0000","last_deep_scrub":"265'748","last_deep_scrub_stamp":"2024-05-30T17:14:08.072843+0000","last_clean_scrub_stamp":"2024-05-30T17:14:08.072843+0000","objects_scrubbed":1,"log_size":48,"log_dups_size":700,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:49:34.177372+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3632,"num_read_kb":3285,"num_write":997,"num_write_kb":370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1532,"num_omap_keys":6,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"265'238","reported_seq":1004,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:30.760818+0000","last_change":"2024-05-30T17:14:30.760939+0000","last_active":"2024-05-30T17:14:30.760818+0000","last_peered":"2024-05-30T17:14:30.760818+0000","last_clean":"2024-05-30T17:14:30.760818+0000","last_became_active":"2024-05-30T15:54:16.657067+0000","last_became_peered":"2024-05-30T15:54:16.657067+0000","last_unstale":"2024-05-30T17:14:30.760818+0000","last_undegraded":"2024-05-30T17:14:30.760818+0000","last_fullsized":"2024-05-30T17:14:30.760818+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'238","last_scrub_stamp":"2024-05-30T17:14:30.760570+0000","last_deep_scrub":"265'238","last_deep_scrub_stamp":"2024-05-30T17:14:30.760570+0000","last_clean_scrub_stamp":"2024-05-30T17:14:30.760570+0000","objects_scrubbed":4,"log_size":38,"log_dups_size":200,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:47:18.792673+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5418111,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":685,"num_read_kb":72437,"num_write":1157,"num_write_kb":30547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":537,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:06.733343+0000","last_change":"2024-05-30T17:14:06.733343+0000","last_active":"2024-05-30T17:14:06.733343+0000","last_peered":"2024-05-30T17:14:06.733343+0000","last_clean":"2024-05-30T17:14:06.733343+0000","last_became_active":"2024-05-30T15:54:08.577191+0000","last_became_peered":"2024-05-30T15:54:08.577191+0000","last_unstale":"2024-05-30T17:14:06.733343+0000","last_undegraded":"2024-05-30T17:14:06.733343+0000","last_fullsized":"2024-05-30T17:14:06.733343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:06.733080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:06.733080+0000","last_clean_scrub_stamp":"2024-05-30T17:14:06.733080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:02:40.927625+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":525,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:17.373292+0000","last_change":"2024-05-30T17:14:17.373292+0000","last_active":"2024-05-30T17:14:17.373292+0000","last_peered":"2024-05-30T17:14:17.373292+0000","last_clean":"2024-05-30T17:14:17.373292+0000","last_became_active":"2024-05-30T15:54:20.653404+0000","last_became_peered":"2024-05-30T15:54:20.653404+0000","last_unstale":"2024-05-30T17:14:17.373292+0000","last_undegraded":"2024-05-30T17:14:17.373292+0000","last_fullsized":"2024-05-30T17:14:17.373292+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:17.373049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:17.373049+0000","last_clean_scrub_stamp":"2024-05-30T17:14:17.373049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:52:07.685494+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"265'354","reported_seq":1110,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:34.766591+0000","last_change":"2024-05-30T17:14:34.766705+0000","last_active":"2024-05-30T17:14:34.766591+0000","last_peered":"2024-05-30T17:14:34.766591+0000","last_clean":"2024-05-30T17:14:34.766591+0000","last_became_active":"2024-05-30T15:54:22.848761+0000","last_became_peered":"2024-05-30T15:54:22.848761+0000","last_unstale":"2024-05-30T17:14:34.766591+0000","last_undegraded":"2024-05-30T17:14:34.766591+0000","last_fullsized":"2024-05-30T17:14:34.766591+0000","mapping_epoch":28,"log_start":"265'300","ondisk_log_start":"265'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'354","last_scrub_stamp":"2024-05-30T17:14:34.765964+0000","last_deep_scrub":"265'354","last_deep_scrub_stamp":"2024-05-30T17:14:34.765964+0000","last_clean_scrub_stamp":"2024-05-30T17:14:34.765964+0000","objects_scrubbed":8,"log_size":54,"log_dups_size":300,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:37:18.193004+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24415,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":525,"num_read_kb":761,"num_write":274,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":515,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:06.395596+0000","last_change":"2024-05-30T17:14:06.395648+0000","last_active":"2024-05-30T17:14:06.395596+0000","last_peered":"2024-05-30T17:14:06.395596+0000","last_clean":"2024-05-30T17:14:06.395596+0000","last_became_active":"2024-05-30T15:54:24.686801+0000","last_became_peered":"2024-05-30T15:54:24.686801+0000","last_unstale":"2024-05-30T17:14:06.395596+0000","last_undegraded":"2024-05-30T17:14:06.395596+0000","last_fullsized":"2024-05-30T17:14:06.395596+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:06.395403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:06.395403+0000","last_clean_scrub_stamp":"2024-05-30T17:14:06.395403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:26:52.769715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"265'152","reported_seq":662,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.241219+0000","last_change":"2024-05-30T17:13:51.241295+0000","last_active":"2024-05-30T17:13:51.241219+0000","last_peered":"2024-05-30T17:13:51.241219+0000","last_clean":"2024-05-30T17:13:51.241219+0000","last_became_active":"2024-05-30T15:54:26.700325+0000","last_became_peered":"2024-05-30T15:54:26.700325+0000","last_unstale":"2024-05-30T17:13:51.241219+0000","last_undegraded":"2024-05-30T17:13:51.241219+0000","last_fullsized":"2024-05-30T17:13:51.241219+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'152","last_scrub_stamp":"2024-05-30T17:13:51.241024+0000","last_deep_scrub":"265'152","last_deep_scrub_stamp":"2024-05-30T17:13:51.241024+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.241024+0000","objects_scrubbed":2,"log_size":52,"log_dups_size":100,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:14:44.872712+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":204,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":115,"num_write":342,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"265'150","reported_seq":732,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.796700+0000","last_change":"2024-05-30T17:13:56.796775+0000","last_active":"2024-05-30T17:13:56.796700+0000","last_peered":"2024-05-30T17:13:56.796700+0000","last_clean":"2024-05-30T17:13:56.796700+0000","last_became_active":"2024-05-30T15:54:26.686310+0000","last_became_peered":"2024-05-30T15:54:26.686310+0000","last_unstale":"2024-05-30T17:13:56.796700+0000","last_undegraded":"2024-05-30T17:13:56.796700+0000","last_fullsized":"2024-05-30T17:13:56.796700+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'150","last_scrub_stamp":"2024-05-30T17:13:56.796467+0000","last_deep_scrub":"265'150","last_deep_scrub_stamp":"2024-05-30T17:13:56.796467+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.796467+0000","objects_scrubbed":1,"log_size":50,"log_dups_size":100,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:01:08.998802+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":227,"num_read_kb":222,"num_write":347,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"265'747","reported_seq":2056,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:05.009608+0000","last_change":"2024-05-30T17:14:05.009756+0000","last_active":"2024-05-30T17:14:05.009608+0000","last_peered":"2024-05-30T17:14:05.009608+0000","last_clean":"2024-05-30T17:14:05.009608+0000","last_became_active":"2024-05-30T15:54:18.673418+0000","last_became_peered":"2024-05-30T15:54:18.673418+0000","last_unstale":"2024-05-30T17:14:05.009608+0000","last_undegraded":"2024-05-30T17:14:05.009608+0000","last_fullsized":"2024-05-30T17:14:05.009608+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'747","last_scrub_stamp":"2024-05-30T17:14:05.009388+0000","last_deep_scrub":"265'747","last_deep_scrub_stamp":"2024-05-30T17:14:05.009388+0000","last_clean_scrub_stamp":"2024-05-30T17:14:05.009388+0000","objects_scrubbed":1,"log_size":47,"log_dups_size":700,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:56:33.977718+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3258,"num_read_kb":2851,"num_write":1044,"num_write_kb":425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":556,"num_omap_keys":3,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"265'202","reported_seq":831,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.153045+0000","last_change":"2024-05-30T17:13:49.153045+0000","last_active":"2024-05-30T17:13:49.153045+0000","last_peered":"2024-05-30T17:13:49.153045+0000","last_clean":"2024-05-30T17:13:49.153045+0000","last_became_active":"2024-05-30T15:54:54.450966+0000","last_became_peered":"2024-05-30T15:54:54.450966+0000","last_unstale":"2024-05-30T17:13:49.153045+0000","last_undegraded":"2024-05-30T17:13:49.153045+0000","last_fullsized":"2024-05-30T17:13:49.153045+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'202","last_scrub_stamp":"2024-05-30T17:13:49.152709+0000","last_deep_scrub":"265'202","last_deep_scrub_stamp":"2024-05-30T17:13:49.152709+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.152709+0000","objects_scrubbed":6,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:19:50.214056+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262272,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":401,"num_read_kb":38159,"num_write":1164,"num_write_kb":35962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":518,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:00.085249+0000","last_change":"2024-05-30T17:14:00.085304+0000","last_active":"2024-05-30T17:14:00.085249+0000","last_peered":"2024-05-30T17:14:00.085249+0000","last_clean":"2024-05-30T17:14:00.085249+0000","last_became_active":"2024-05-30T15:54:20.664882+0000","last_became_peered":"2024-05-30T15:54:20.664882+0000","last_unstale":"2024-05-30T17:14:00.085249+0000","last_undegraded":"2024-05-30T17:14:00.085249+0000","last_fullsized":"2024-05-30T17:14:00.085249+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:00.085062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:00.085062+0000","last_clean_scrub_stamp":"2024-05-30T17:14:00.085062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:36:44.188047+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"265'158","reported_seq":766,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:27.374925+0000","last_change":"2024-05-30T17:14:27.375001+0000","last_active":"2024-05-30T17:14:27.374925+0000","last_peered":"2024-05-30T17:14:27.374925+0000","last_clean":"2024-05-30T17:14:27.374925+0000","last_became_active":"2024-05-30T15:54:22.672170+0000","last_became_peered":"2024-05-30T15:54:22.672170+0000","last_unstale":"2024-05-30T17:14:27.374925+0000","last_undegraded":"2024-05-30T17:14:27.374925+0000","last_fullsized":"2024-05-30T17:14:27.374925+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'158","last_scrub_stamp":"2024-05-30T17:14:27.374727+0000","last_deep_scrub":"265'158","last_deep_scrub_stamp":"2024-05-30T17:14:27.374727+0000","last_clean_scrub_stamp":"2024-05-30T17:14:27.374727+0000","objects_scrubbed":5,"log_size":58,"log_dups_size":100,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:50:38.280039+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5933,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":205,"num_read_kb":259,"num_write":110,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":511,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:04.400786+0000","last_change":"2024-05-30T17:14:04.400840+0000","last_active":"2024-05-30T17:14:04.400786+0000","last_peered":"2024-05-30T17:14:04.400786+0000","last_clean":"2024-05-30T17:14:04.400786+0000","last_became_active":"2024-05-30T15:54:24.688053+0000","last_became_peered":"2024-05-30T15:54:24.688053+0000","last_unstale":"2024-05-30T17:14:04.400786+0000","last_undegraded":"2024-05-30T17:14:04.400786+0000","last_fullsized":"2024-05-30T17:14:04.400786+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:04.400563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:04.400563+0000","last_clean_scrub_stamp":"2024-05-30T17:14:04.400563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:58:36.746394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"265'120","reported_seq":654,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.374741+0000","last_change":"2024-05-30T17:13:54.374814+0000","last_active":"2024-05-30T17:13:54.374741+0000","last_peered":"2024-05-30T17:13:54.374741+0000","last_clean":"2024-05-30T17:13:54.374741+0000","last_became_active":"2024-05-30T15:54:26.701588+0000","last_became_peered":"2024-05-30T15:54:26.701588+0000","last_unstale":"2024-05-30T17:13:54.374741+0000","last_undegraded":"2024-05-30T17:13:54.374741+0000","last_fullsized":"2024-05-30T17:13:54.374741+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'120","last_scrub_stamp":"2024-05-30T17:13:54.374529+0000","last_deep_scrub":"265'120","last_deep_scrub_stamp":"2024-05-30T17:13:54.374529+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.374529+0000","objects_scrubbed":3,"log_size":20,"log_dups_size":100,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:54:34.109921+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":826,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":172,"num_read_kb":153,"num_write":293,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"265'323","reported_seq":1770,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:01.069483+0000","last_change":"2024-05-30T17:14:01.069557+0000","last_active":"2024-05-30T17:14:01.069483+0000","last_peered":"2024-05-30T17:14:01.069483+0000","last_clean":"2024-05-30T17:14:01.069483+0000","last_became_active":"2024-05-30T15:54:18.670707+0000","last_became_peered":"2024-05-30T15:54:18.670707+0000","last_unstale":"2024-05-30T17:14:01.069483+0000","last_undegraded":"2024-05-30T17:14:01.069483+0000","last_fullsized":"2024-05-30T17:14:01.069483+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'323","last_scrub_stamp":"2024-05-30T17:14:01.069299+0000","last_deep_scrub":"265'323","last_deep_scrub_stamp":"2024-05-30T17:14:01.069299+0000","last_clean_scrub_stamp":"2024-05-30T17:14:01.069299+0000","objects_scrubbed":2,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:28:00.081754+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1910,"num_read_kb":1828,"num_write":376,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"265'219","reported_seq":893,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:23.735734+0000","last_change":"2024-05-30T17:14:23.735830+0000","last_active":"2024-05-30T17:14:23.735734+0000","last_peered":"2024-05-30T17:14:23.735734+0000","last_clean":"2024-05-30T17:14:23.735734+0000","last_became_active":"2024-05-30T15:54:16.660427+0000","last_became_peered":"2024-05-30T15:54:16.660427+0000","last_unstale":"2024-05-30T17:14:23.735734+0000","last_undegraded":"2024-05-30T17:14:23.735734+0000","last_fullsized":"2024-05-30T17:14:23.735734+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'219","last_scrub_stamp":"2024-05-30T17:14:23.735541+0000","last_deep_scrub":"265'219","last_deep_scrub_stamp":"2024-05-30T17:14:23.735541+0000","last_clean_scrub_stamp":"2024-05-30T17:14:23.735541+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":200,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:18:24.875280+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262181,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":518,"num_read_kb":33224,"num_write":1148,"num_write_kb":46290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":519,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:15.367435+0000","last_change":"2024-05-30T17:14:15.367493+0000","last_active":"2024-05-30T17:14:15.367435+0000","last_peered":"2024-05-30T17:14:15.367435+0000","last_clean":"2024-05-30T17:14:15.367435+0000","last_became_active":"2024-05-30T15:54:20.649330+0000","last_became_peered":"2024-05-30T15:54:20.649330+0000","last_unstale":"2024-05-30T17:14:15.367435+0000","last_undegraded":"2024-05-30T17:14:15.367435+0000","last_fullsized":"2024-05-30T17:14:15.367435+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:15.367236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:15.367236+0000","last_clean_scrub_stamp":"2024-05-30T17:14:15.367236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:47:00.342990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"265'74","reported_seq":608,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:21.824735+0000","last_change":"2024-05-30T17:14:21.824807+0000","last_active":"2024-05-30T17:14:21.824735+0000","last_peered":"2024-05-30T17:14:21.824735+0000","last_clean":"2024-05-30T17:14:21.824735+0000","last_became_active":"2024-05-30T15:54:22.662738+0000","last_became_peered":"2024-05-30T15:54:22.662738+0000","last_unstale":"2024-05-30T17:14:21.824735+0000","last_undegraded":"2024-05-30T17:14:21.824735+0000","last_fullsized":"2024-05-30T17:14:21.824735+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'74","last_scrub_stamp":"2024-05-30T17:14:21.824479+0000","last_deep_scrub":"265'74","last_deep_scrub_stamp":"2024-05-30T17:14:21.824479+0000","last_clean_scrub_stamp":"2024-05-30T17:14:21.824479+0000","objects_scrubbed":3,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:23:35.198468+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":50,"num_write":40,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":588,"num_omap_keys":4,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":509,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:05.756275+0000","last_change":"2024-05-30T17:14:05.756326+0000","last_active":"2024-05-30T17:14:05.756275+0000","last_peered":"2024-05-30T17:14:05.756275+0000","last_clean":"2024-05-30T17:14:05.756275+0000","last_became_active":"2024-05-30T15:54:24.680578+0000","last_became_peered":"2024-05-30T15:54:24.680578+0000","last_unstale":"2024-05-30T17:14:05.756275+0000","last_undegraded":"2024-05-30T17:14:05.756275+0000","last_fullsized":"2024-05-30T17:14:05.756275+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:05.756098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:05.756098+0000","last_clean_scrub_stamp":"2024-05-30T17:14:05.756098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:07:27.828294+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"265'215","reported_seq":785,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.750473+0000","last_change":"2024-05-30T17:13:54.750473+0000","last_active":"2024-05-30T17:13:54.750473+0000","last_peered":"2024-05-30T17:13:54.750473+0000","last_clean":"2024-05-30T17:13:54.750473+0000","last_became_active":"2024-05-30T15:54:26.686553+0000","last_became_peered":"2024-05-30T15:54:26.686553+0000","last_unstale":"2024-05-30T17:13:54.750473+0000","last_undegraded":"2024-05-30T17:13:54.750473+0000","last_fullsized":"2024-05-30T17:13:54.750473+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'215","last_scrub_stamp":"2024-05-30T17:13:54.750033+0000","last_deep_scrub":"265'215","last_deep_scrub_stamp":"2024-05-30T17:13:54.750033+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.750033+0000","objects_scrubbed":3,"log_size":15,"log_dups_size":200,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:23:19.977063+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":816,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":414,"num_read_kb":383,"num_write":513,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":398,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"265'419","reported_seq":1805,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:14.364566+0000","last_change":"2024-05-30T17:14:14.364672+0000","last_active":"2024-05-30T17:14:14.364566+0000","last_peered":"2024-05-30T17:14:14.364566+0000","last_clean":"2024-05-30T17:14:14.364566+0000","last_became_active":"2024-05-30T15:54:18.660089+0000","last_became_peered":"2024-05-30T15:54:18.660089+0000","last_unstale":"2024-05-30T17:14:14.364566+0000","last_undegraded":"2024-05-30T17:14:14.364566+0000","last_fullsized":"2024-05-30T17:14:14.364566+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'419","last_scrub_stamp":"2024-05-30T17:14:14.364379+0000","last_deep_scrub":"265'419","last_deep_scrub_stamp":"2024-05-30T17:14:14.364379+0000","last_clean_scrub_stamp":"2024-05-30T17:14:14.364379+0000","objects_scrubbed":6,"log_size":19,"log_dups_size":400,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:24:09.477211+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2036,"num_read_kb":1890,"num_write":519,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"265'217","reported_seq":896,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:20.299717+0000","last_change":"2024-05-30T17:14:20.299925+0000","last_active":"2024-05-30T17:14:20.299717+0000","last_peered":"2024-05-30T17:14:20.299717+0000","last_clean":"2024-05-30T17:14:20.299717+0000","last_became_active":"2024-05-30T15:54:16.650025+0000","last_became_peered":"2024-05-30T15:54:16.650025+0000","last_unstale":"2024-05-30T17:14:20.299717+0000","last_undegraded":"2024-05-30T17:14:20.299717+0000","last_fullsized":"2024-05-30T17:14:20.299717+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'217","last_scrub_stamp":"2024-05-30T17:14:20.299487+0000","last_deep_scrub":"265'217","last_deep_scrub_stamp":"2024-05-30T17:14:20.299487+0000","last_clean_scrub_stamp":"2024-05-30T17:14:20.299487+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":200,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:30:25.801340+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5569939,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":500,"num_read_kb":102670,"num_write":1113,"num_write_kb":39714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":517,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:12.802057+0000","last_change":"2024-05-30T17:14:12.802110+0000","last_active":"2024-05-30T17:14:12.802057+0000","last_peered":"2024-05-30T17:14:12.802057+0000","last_clean":"2024-05-30T17:14:12.802057+0000","last_became_active":"2024-05-30T15:54:20.656663+0000","last_became_peered":"2024-05-30T15:54:20.656663+0000","last_unstale":"2024-05-30T17:14:12.802057+0000","last_undegraded":"2024-05-30T17:14:12.802057+0000","last_fullsized":"2024-05-30T17:14:12.802057+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:12.801868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:12.801868+0000","last_clean_scrub_stamp":"2024-05-30T17:14:12.801868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:29:15.815405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"265'191","reported_seq":815,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:37.548757+0000","last_change":"2024-05-30T17:14:37.548845+0000","last_active":"2024-05-30T17:14:37.548757+0000","last_peered":"2024-05-30T17:14:37.548757+0000","last_clean":"2024-05-30T17:14:37.548757+0000","last_became_active":"2024-05-30T15:54:22.673673+0000","last_became_peered":"2024-05-30T15:54:22.673673+0000","last_unstale":"2024-05-30T17:14:37.548757+0000","last_undegraded":"2024-05-30T17:14:37.548757+0000","last_fullsized":"2024-05-30T17:14:37.548757+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'191","last_scrub_stamp":"2024-05-30T17:14:37.548524+0000","last_deep_scrub":"265'191","last_deep_scrub_stamp":"2024-05-30T17:14:37.548524+0000","last_clean_scrub_stamp":"2024-05-30T17:14:37.548524+0000","objects_scrubbed":5,"log_size":91,"log_dups_size":100,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:03:19.783707+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7114,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":259,"num_read_kb":332,"num_write":146,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":508,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.754848+0000","last_change":"2024-05-30T17:13:50.754954+0000","last_active":"2024-05-30T17:13:50.754848+0000","last_peered":"2024-05-30T17:13:50.754848+0000","last_clean":"2024-05-30T17:13:50.754848+0000","last_became_active":"2024-05-30T15:54:24.675289+0000","last_became_peered":"2024-05-30T15:54:24.675289+0000","last_unstale":"2024-05-30T17:13:50.754848+0000","last_undegraded":"2024-05-30T17:13:50.754848+0000","last_fullsized":"2024-05-30T17:13:50.754848+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:50.754554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:50.754554+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.754554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:47:56.533373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"265'141","reported_seq":647,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.257049+0000","last_change":"2024-05-30T17:13:50.257049+0000","last_active":"2024-05-30T17:13:50.257049+0000","last_peered":"2024-05-30T17:13:50.257049+0000","last_clean":"2024-05-30T17:13:50.257049+0000","last_became_active":"2024-05-30T15:54:26.703433+0000","last_became_peered":"2024-05-30T15:54:26.703433+0000","last_unstale":"2024-05-30T17:13:50.257049+0000","last_undegraded":"2024-05-30T17:13:50.257049+0000","last_fullsized":"2024-05-30T17:13:50.257049+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'141","last_scrub_stamp":"2024-05-30T17:13:50.256770+0000","last_deep_scrub":"265'141","last_deep_scrub_stamp":"2024-05-30T17:13:50.256770+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.256770+0000","objects_scrubbed":3,"log_size":41,"log_dups_size":100,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:08:34.180219+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":856,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":110,"num_write":329,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"265'867","reported_seq":2504,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:28.423699+0000","last_change":"2024-05-30T17:14:28.423766+0000","last_active":"2024-05-30T17:14:28.423699+0000","last_peered":"2024-05-30T17:14:28.423699+0000","last_clean":"2024-05-30T17:14:28.423699+0000","last_became_active":"2024-05-30T15:54:18.649273+0000","last_became_peered":"2024-05-30T15:54:18.649273+0000","last_unstale":"2024-05-30T17:14:28.423699+0000","last_undegraded":"2024-05-30T17:14:28.423699+0000","last_fullsized":"2024-05-30T17:14:28.423699+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'867","last_scrub_stamp":"2024-05-30T17:14:28.423514+0000","last_deep_scrub":"265'867","last_deep_scrub_stamp":"2024-05-30T17:14:28.423514+0000","last_clean_scrub_stamp":"2024-05-30T17:14:28.423514+0000","objects_scrubbed":4,"log_size":67,"log_dups_size":800,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:34:11.095216+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3918,"num_read_kb":3498,"num_write":1187,"num_write_kb":458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":401,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"265'207","reported_seq":877,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:29.742800+0000","last_change":"2024-05-30T17:14:29.742916+0000","last_active":"2024-05-30T17:14:29.742800+0000","last_peered":"2024-05-30T17:14:29.742800+0000","last_clean":"2024-05-30T17:14:29.742800+0000","last_became_active":"2024-05-30T15:54:16.649121+0000","last_became_peered":"2024-05-30T15:54:16.649121+0000","last_unstale":"2024-05-30T17:14:29.742800+0000","last_undegraded":"2024-05-30T17:14:29.742800+0000","last_fullsized":"2024-05-30T17:14:29.742800+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'207","last_scrub_stamp":"2024-05-30T17:14:29.742576+0000","last_deep_scrub":"265'207","last_deep_scrub_stamp":"2024-05-30T17:14:29.742576+0000","last_clean_scrub_stamp":"2024-05-30T17:14:29.742576+0000","objects_scrubbed":6,"log_size":107,"log_dups_size":100,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:05:54.402014+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1029196,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":493,"num_read_kb":65907,"num_write":1120,"num_write_kb":54558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":516,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.736994+0000","last_change":"2024-05-30T17:13:51.737047+0000","last_active":"2024-05-30T17:13:51.736994+0000","last_peered":"2024-05-30T17:13:51.736994+0000","last_clean":"2024-05-30T17:13:51.736994+0000","last_became_active":"2024-05-30T15:54:20.667863+0000","last_became_peered":"2024-05-30T15:54:20.667863+0000","last_unstale":"2024-05-30T17:13:51.736994+0000","last_undegraded":"2024-05-30T17:13:51.736994+0000","last_fullsized":"2024-05-30T17:13:51.736994+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:51.736796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:51.736796+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.736796+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:58:41.051038+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"265'147","reported_seq":746,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:22.771867+0000","last_change":"2024-05-30T17:14:22.772012+0000","last_active":"2024-05-30T17:14:22.771867+0000","last_peered":"2024-05-30T17:14:22.771867+0000","last_clean":"2024-05-30T17:14:22.771867+0000","last_became_active":"2024-05-30T15:54:22.664977+0000","last_became_peered":"2024-05-30T15:54:22.664977+0000","last_unstale":"2024-05-30T17:14:22.771867+0000","last_undegraded":"2024-05-30T17:14:22.771867+0000","last_fullsized":"2024-05-30T17:14:22.771867+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'147","last_scrub_stamp":"2024-05-30T17:14:22.771618+0000","last_deep_scrub":"265'147","last_deep_scrub_stamp":"2024-05-30T17:14:22.771618+0000","last_clean_scrub_stamp":"2024-05-30T17:14:22.771618+0000","objects_scrubbed":4,"log_size":47,"log_dups_size":100,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:06:27.800101+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7290,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":187,"num_read_kb":260,"num_write":98,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":514,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:02.866942+0000","last_change":"2024-05-30T17:14:02.866997+0000","last_active":"2024-05-30T17:14:02.866942+0000","last_peered":"2024-05-30T17:14:02.866942+0000","last_clean":"2024-05-30T17:14:02.866942+0000","last_became_active":"2024-05-30T15:54:24.676707+0000","last_became_peered":"2024-05-30T15:54:24.676707+0000","last_unstale":"2024-05-30T17:14:02.866942+0000","last_undegraded":"2024-05-30T17:14:02.866942+0000","last_fullsized":"2024-05-30T17:14:02.866942+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:02.866743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:02.866743+0000","last_clean_scrub_stamp":"2024-05-30T17:14:02.866743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:19:34.249137+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"265'443","reported_seq":2109,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.771973+0000","last_change":"2024-05-30T17:13:59.772041+0000","last_active":"2024-05-30T17:13:59.771973+0000","last_peered":"2024-05-30T17:13:59.771973+0000","last_clean":"2024-05-30T17:13:59.771973+0000","last_became_active":"2024-05-30T15:54:18.670756+0000","last_became_peered":"2024-05-30T15:54:18.670756+0000","last_unstale":"2024-05-30T17:13:59.771973+0000","last_undegraded":"2024-05-30T17:13:59.771973+0000","last_fullsized":"2024-05-30T17:13:59.771973+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'443","last_scrub_stamp":"2024-05-30T17:13:59.771783+0000","last_deep_scrub":"265'443","last_deep_scrub_stamp":"2024-05-30T17:13:59.771783+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.771783+0000","objects_scrubbed":1,"log_size":43,"log_dups_size":400,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:50:25.853396+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2383,"num_read_kb":2277,"num_write":522,"num_write_kb":97,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"265'164","reported_seq":778,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:30.419107+0000","last_change":"2024-05-30T17:14:30.419198+0000","last_active":"2024-05-30T17:14:30.419107+0000","last_peered":"2024-05-30T17:14:30.419107+0000","last_clean":"2024-05-30T17:14:30.419107+0000","last_became_active":"2024-05-30T15:54:16.653560+0000","last_became_peered":"2024-05-30T15:54:16.653560+0000","last_unstale":"2024-05-30T17:14:30.419107+0000","last_undegraded":"2024-05-30T17:14:30.419107+0000","last_fullsized":"2024-05-30T17:14:30.419107+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'164","last_scrub_stamp":"2024-05-30T17:14:30.418894+0000","last_deep_scrub":"265'164","last_deep_scrub_stamp":"2024-05-30T17:14:30.418894+0000","last_clean_scrub_stamp":"2024-05-30T17:14:30.418894+0000","objects_scrubbed":3,"log_size":64,"log_dups_size":100,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:31:08.828045+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5417712,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":326,"num_read_kb":31913,"num_write":888,"num_write_kb":20958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":528,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:17.827438+0000","last_change":"2024-05-30T17:14:17.827438+0000","last_active":"2024-05-30T17:14:17.827438+0000","last_peered":"2024-05-30T17:14:17.827438+0000","last_clean":"2024-05-30T17:14:17.827438+0000","last_became_active":"2024-05-30T15:54:20.662669+0000","last_became_peered":"2024-05-30T15:54:20.662669+0000","last_unstale":"2024-05-30T17:14:17.827438+0000","last_undegraded":"2024-05-30T17:14:17.827438+0000","last_fullsized":"2024-05-30T17:14:17.827438+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:17.827181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:17.827181+0000","last_clean_scrub_stamp":"2024-05-30T17:14:17.827181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:03:52.723170+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"265'228","reported_seq":887,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:02.038392+0000","last_change":"2024-05-30T17:14:02.038392+0000","last_active":"2024-05-30T17:14:02.038392+0000","last_peered":"2024-05-30T17:14:02.038392+0000","last_clean":"2024-05-30T17:14:02.038392+0000","last_became_active":"2024-05-30T15:54:22.663518+0000","last_became_peered":"2024-05-30T15:54:22.663518+0000","last_unstale":"2024-05-30T17:14:02.038392+0000","last_undegraded":"2024-05-30T17:14:02.038392+0000","last_fullsized":"2024-05-30T17:14:02.038392+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'228","last_scrub_stamp":"2024-05-30T17:14:02.038127+0000","last_deep_scrub":"265'228","last_deep_scrub_stamp":"2024-05-30T17:14:02.038127+0000","last_clean_scrub_stamp":"2024-05-30T17:14:02.038127+0000","objects_scrubbed":6,"log_size":28,"log_dups_size":200,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:44:55.851829+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11676,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":342,"num_read_kb":490,"num_write":222,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":24966,"num_omap_keys":48,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":507,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:55.195111+0000","last_change":"2024-05-30T17:13:55.195111+0000","last_active":"2024-05-30T17:13:55.195111+0000","last_peered":"2024-05-30T17:13:55.195111+0000","last_clean":"2024-05-30T17:13:55.195111+0000","last_became_active":"2024-05-30T15:54:24.682609+0000","last_became_peered":"2024-05-30T15:54:24.682609+0000","last_unstale":"2024-05-30T17:13:55.195111+0000","last_undegraded":"2024-05-30T17:13:55.195111+0000","last_fullsized":"2024-05-30T17:13:55.195111+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:55.194814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:55.194814+0000","last_clean_scrub_stamp":"2024-05-30T17:13:55.194814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:35:33.515747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"265'151","reported_seq":672,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:00.314401+0000","last_change":"2024-05-30T17:14:00.314480+0000","last_active":"2024-05-30T17:14:00.314401+0000","last_peered":"2024-05-30T17:14:00.314401+0000","last_clean":"2024-05-30T17:14:00.314401+0000","last_became_active":"2024-05-30T15:54:26.695210+0000","last_became_peered":"2024-05-30T15:54:26.695210+0000","last_unstale":"2024-05-30T17:14:00.314401+0000","last_undegraded":"2024-05-30T17:14:00.314401+0000","last_fullsized":"2024-05-30T17:14:00.314401+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'151","last_scrub_stamp":"2024-05-30T17:14:00.314210+0000","last_deep_scrub":"265'151","last_deep_scrub_stamp":"2024-05-30T17:14:00.314210+0000","last_clean_scrub_stamp":"2024-05-30T17:14:00.314210+0000","objects_scrubbed":2,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:57:02.003537+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":643,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":167,"num_read_kb":179,"num_write":338,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"265'857","reported_seq":2488,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:22.852426+0000","last_change":"2024-05-30T17:14:22.852426+0000","last_active":"2024-05-30T17:14:22.852426+0000","last_peered":"2024-05-30T17:14:22.852426+0000","last_clean":"2024-05-30T17:14:22.852426+0000","last_became_active":"2024-05-30T15:54:18.640739+0000","last_became_peered":"2024-05-30T15:54:18.640739+0000","last_unstale":"2024-05-30T17:14:22.852426+0000","last_undegraded":"2024-05-30T17:14:22.852426+0000","last_fullsized":"2024-05-30T17:14:22.852426+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'857","last_scrub_stamp":"2024-05-30T17:14:22.852147+0000","last_deep_scrub":"265'857","last_deep_scrub_stamp":"2024-05-30T17:14:22.852147+0000","last_clean_scrub_stamp":"2024-05-30T17:14:22.852147+0000","objects_scrubbed":3,"log_size":57,"log_dups_size":800,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:22:06.520996+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4053,"num_read_kb":3626,"num_write":1150,"num_write_kb":447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"265'245","reported_seq":988,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.935811+0000","last_change":"2024-05-30T17:13:47.935811+0000","last_active":"2024-05-30T17:13:47.935811+0000","last_peered":"2024-05-30T17:13:47.935811+0000","last_clean":"2024-05-30T17:13:47.935811+0000","last_became_active":"2024-05-30T15:54:54.424825+0000","last_became_peered":"2024-05-30T15:54:54.424825+0000","last_unstale":"2024-05-30T17:13:47.935811+0000","last_undegraded":"2024-05-30T17:13:47.935811+0000","last_fullsized":"2024-05-30T17:13:47.935811+0000","mapping_epoch":35,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'245","last_scrub_stamp":"2024-05-30T17:13:47.935515+0000","last_deep_scrub":"265'245","last_deep_scrub_stamp":"2024-05-30T17:13:47.935515+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.935515+0000","objects_scrubbed":3,"log_size":45,"log_dups_size":200,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:50:12.821107+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5285497,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":887,"num_read_kb":12756,"num_write":1230,"num_write_kb":34615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"265'4","reported_seq":848,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.789874+0000","last_change":"2024-05-30T17:13:53.789936+0000","last_active":"2024-05-30T17:13:53.789874+0000","last_peered":"2024-05-30T17:13:53.789874+0000","last_clean":"2024-05-30T17:13:53.789874+0000","last_became_active":"2024-05-30T15:54:20.664095+0000","last_became_peered":"2024-05-30T15:54:20.664095+0000","last_unstale":"2024-05-30T17:13:53.789874+0000","last_undegraded":"2024-05-30T17:13:53.789874+0000","last_fullsized":"2024-05-30T17:13:53.789874+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'4","last_scrub_stamp":"2024-05-30T17:13:53.789681+0000","last_deep_scrub":"265'4","last_deep_scrub_stamp":"2024-05-30T17:13:53.789681+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.789681+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:25:01.268608+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":494,"num_read_kb":492,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"265'202","reported_seq":822,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:20.840701+0000","last_change":"2024-05-30T17:14:20.840701+0000","last_active":"2024-05-30T17:14:20.840701+0000","last_peered":"2024-05-30T17:14:20.840701+0000","last_clean":"2024-05-30T17:14:20.840701+0000","last_became_active":"2024-05-30T15:54:22.664997+0000","last_became_peered":"2024-05-30T15:54:22.664997+0000","last_unstale":"2024-05-30T17:14:20.840701+0000","last_undegraded":"2024-05-30T17:14:20.840701+0000","last_fullsized":"2024-05-30T17:14:20.840701+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'202","last_scrub_stamp":"2024-05-30T17:14:20.840427+0000","last_deep_scrub":"265'202","last_deep_scrub_stamp":"2024-05-30T17:14:20.840427+0000","last_clean_scrub_stamp":"2024-05-30T17:14:20.840427+0000","objects_scrubbed":8,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:09:02.158801+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25625,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":290,"num_read_kb":412,"num_write":192,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11460,"num_omap_keys":24,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":508,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:10.050358+0000","last_change":"2024-05-30T17:14:10.050358+0000","last_active":"2024-05-30T17:14:10.050358+0000","last_peered":"2024-05-30T17:14:10.050358+0000","last_clean":"2024-05-30T17:14:10.050358+0000","last_became_active":"2024-05-30T15:54:24.675583+0000","last_became_peered":"2024-05-30T15:54:24.675583+0000","last_unstale":"2024-05-30T17:14:10.050358+0000","last_undegraded":"2024-05-30T17:14:10.050358+0000","last_fullsized":"2024-05-30T17:14:10.050358+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:10.050095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:10.050095+0000","last_clean_scrub_stamp":"2024-05-30T17:14:10.050095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:05:37.764975+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"265'148","reported_seq":673,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:14.828641+0000","last_change":"2024-05-30T17:14:14.828728+0000","last_active":"2024-05-30T17:14:14.828641+0000","last_peered":"2024-05-30T17:14:14.828641+0000","last_clean":"2024-05-30T17:14:14.828641+0000","last_became_active":"2024-05-30T15:54:26.700392+0000","last_became_peered":"2024-05-30T15:54:26.700392+0000","last_unstale":"2024-05-30T17:14:14.828641+0000","last_undegraded":"2024-05-30T17:14:14.828641+0000","last_fullsized":"2024-05-30T17:14:14.828641+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'148","last_scrub_stamp":"2024-05-30T17:14:14.828458+0000","last_deep_scrub":"265'148","last_deep_scrub_stamp":"2024-05-30T17:14:14.828458+0000","last_clean_scrub_stamp":"2024-05-30T17:14:14.828458+0000","objects_scrubbed":8,"log_size":48,"log_dups_size":100,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:48:23.720296+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2627,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":177,"num_read_kb":174,"num_write":403,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"265'371","reported_seq":1751,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.352650+0000","last_change":"2024-05-30T17:13:53.352650+0000","last_active":"2024-05-30T17:13:53.352650+0000","last_peered":"2024-05-30T17:13:53.352650+0000","last_clean":"2024-05-30T17:13:53.352650+0000","last_became_active":"2024-05-30T15:54:54.419468+0000","last_became_peered":"2024-05-30T15:54:54.419468+0000","last_unstale":"2024-05-30T17:13:53.352650+0000","last_undegraded":"2024-05-30T17:13:53.352650+0000","last_fullsized":"2024-05-30T17:13:53.352650+0000","mapping_epoch":35,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'371","last_scrub_stamp":"2024-05-30T17:13:53.352335+0000","last_deep_scrub":"265'371","last_deep_scrub_stamp":"2024-05-30T17:13:53.352335+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.352335+0000","objects_scrubbed":3,"log_size":71,"log_dups_size":300,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:30:41.234405+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1898,"num_read_kb":1782,"num_write":455,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":310,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"265'159","reported_seq":894,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:19.091877+0000","last_change":"2024-05-30T17:14:19.092022+0000","last_active":"2024-05-30T17:14:19.091877+0000","last_peered":"2024-05-30T17:14:19.091877+0000","last_clean":"2024-05-30T17:14:19.091877+0000","last_became_active":"2024-05-30T15:54:16.676440+0000","last_became_peered":"2024-05-30T15:54:16.676440+0000","last_unstale":"2024-05-30T17:14:19.091877+0000","last_undegraded":"2024-05-30T17:14:19.091877+0000","last_fullsized":"2024-05-30T17:14:19.091877+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'159","last_scrub_stamp":"2024-05-30T17:14:19.091627+0000","last_deep_scrub":"265'159","last_deep_scrub_stamp":"2024-05-30T17:14:19.091627+0000","last_clean_scrub_stamp":"2024-05-30T17:14:19.091627+0000","objects_scrubbed":6,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:13:38.026022+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456485,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":869,"num_read_kb":45599,"num_write":925,"num_write_kb":34636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":524,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:07.042918+0000","last_change":"2024-05-30T17:14:07.042918+0000","last_active":"2024-05-30T17:14:07.042918+0000","last_peered":"2024-05-30T17:14:07.042918+0000","last_clean":"2024-05-30T17:14:07.042918+0000","last_became_active":"2024-05-30T15:54:20.655388+0000","last_became_peered":"2024-05-30T15:54:20.655388+0000","last_unstale":"2024-05-30T17:14:07.042918+0000","last_undegraded":"2024-05-30T17:14:07.042918+0000","last_fullsized":"2024-05-30T17:14:07.042918+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:07.042677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:07.042677+0000","last_clean_scrub_stamp":"2024-05-30T17:14:07.042677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:08:28.451745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"265'170","reported_seq":782,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:09.437645+0000","last_change":"2024-05-30T17:14:09.437817+0000","last_active":"2024-05-30T17:14:09.437645+0000","last_peered":"2024-05-30T17:14:09.437645+0000","last_clean":"2024-05-30T17:14:09.437645+0000","last_became_active":"2024-05-30T15:54:22.849193+0000","last_became_peered":"2024-05-30T15:54:22.849193+0000","last_unstale":"2024-05-30T17:14:09.437645+0000","last_undegraded":"2024-05-30T17:14:09.437645+0000","last_fullsized":"2024-05-30T17:14:09.437645+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'170","last_scrub_stamp":"2024-05-30T17:14:09.437396+0000","last_deep_scrub":"265'170","last_deep_scrub_stamp":"2024-05-30T17:14:09.437396+0000","last_clean_scrub_stamp":"2024-05-30T17:14:09.437396+0000","objects_scrubbed":5,"log_size":70,"log_dups_size":100,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:29:49.223193+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11086,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":221,"num_read_kb":302,"num_write":124,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":516,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:19.980627+0000","last_change":"2024-05-30T17:14:19.980627+0000","last_active":"2024-05-30T17:14:19.980627+0000","last_peered":"2024-05-30T17:14:19.980627+0000","last_clean":"2024-05-30T17:14:19.980627+0000","last_became_active":"2024-05-30T15:54:24.681609+0000","last_became_peered":"2024-05-30T15:54:24.681609+0000","last_unstale":"2024-05-30T17:14:19.980627+0000","last_undegraded":"2024-05-30T17:14:19.980627+0000","last_fullsized":"2024-05-30T17:14:19.980627+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:19.980274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:19.980274+0000","last_clean_scrub_stamp":"2024-05-30T17:14:19.980274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:01:43.948910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"265'132","reported_seq":644,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.247687+0000","last_change":"2024-05-30T17:13:47.247831+0000","last_active":"2024-05-30T17:13:47.247687+0000","last_peered":"2024-05-30T17:13:47.247687+0000","last_clean":"2024-05-30T17:13:47.247687+0000","last_became_active":"2024-05-30T15:54:26.704842+0000","last_became_peered":"2024-05-30T15:54:26.704842+0000","last_unstale":"2024-05-30T17:13:47.247687+0000","last_undegraded":"2024-05-30T17:13:47.247687+0000","last_fullsized":"2024-05-30T17:13:47.247687+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'132","last_scrub_stamp":"2024-05-30T17:13:47.247463+0000","last_deep_scrub":"265'132","last_deep_scrub_stamp":"2024-05-30T17:13:47.247463+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.247463+0000","objects_scrubbed":3,"log_size":32,"log_dups_size":100,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:58:57.717875+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":383,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":127,"num_read_kb":122,"num_write":322,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"265'761","reported_seq":2269,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:02.365540+0000","last_change":"2024-05-30T17:14:02.365540+0000","last_active":"2024-05-30T17:14:02.365540+0000","last_peered":"2024-05-30T17:14:02.365540+0000","last_clean":"2024-05-30T17:14:02.365540+0000","last_became_active":"2024-05-30T15:54:18.634803+0000","last_became_peered":"2024-05-30T15:54:18.634803+0000","last_unstale":"2024-05-30T17:14:02.365540+0000","last_undegraded":"2024-05-30T17:14:02.365540+0000","last_fullsized":"2024-05-30T17:14:02.365540+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'761","last_scrub_stamp":"2024-05-30T17:14:02.365235+0000","last_deep_scrub":"265'761","last_deep_scrub_stamp":"2024-05-30T17:14:02.365235+0000","last_clean_scrub_stamp":"2024-05-30T17:14:02.365235+0000","objects_scrubbed":8,"log_size":61,"log_dups_size":700,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:38:15.880298+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":3476,"num_read_kb":3109,"num_write":1006,"num_write_kb":389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1743,"num_omap_keys":8,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"265'190","reported_seq":823,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.153453+0000","last_change":"2024-05-30T17:13:47.153543+0000","last_active":"2024-05-30T17:13:47.153453+0000","last_peered":"2024-05-30T17:13:47.153453+0000","last_clean":"2024-05-30T17:13:47.153453+0000","last_became_active":"2024-05-30T15:54:54.430490+0000","last_became_peered":"2024-05-30T15:54:54.430490+0000","last_unstale":"2024-05-30T17:13:47.153453+0000","last_undegraded":"2024-05-30T17:13:47.153453+0000","last_fullsized":"2024-05-30T17:13:47.153453+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'190","last_scrub_stamp":"2024-05-30T17:13:47.153254+0000","last_deep_scrub":"265'190","last_deep_scrub_stamp":"2024-05-30T17:13:47.153254+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.153254+0000","objects_scrubbed":4,"log_size":90,"log_dups_size":100,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:50:08.663040+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":481,"num_read_kb":40323,"num_write":1075,"num_write_kb":31617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":516,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.000902+0000","last_change":"2024-05-30T17:13:56.000902+0000","last_active":"2024-05-30T17:13:56.000902+0000","last_peered":"2024-05-30T17:13:56.000902+0000","last_clean":"2024-05-30T17:13:56.000902+0000","last_became_active":"2024-05-30T15:54:20.650944+0000","last_became_peered":"2024-05-30T15:54:20.650944+0000","last_unstale":"2024-05-30T17:13:56.000902+0000","last_undegraded":"2024-05-30T17:13:56.000902+0000","last_fullsized":"2024-05-30T17:13:56.000902+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:56.000601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:56.000601+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.000601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:28:54.937679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"265'120","reported_seq":688,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.917856+0000","last_change":"2024-05-30T17:13:56.917856+0000","last_active":"2024-05-30T17:13:56.917856+0000","last_peered":"2024-05-30T17:13:56.917856+0000","last_clean":"2024-05-30T17:13:56.917856+0000","last_became_active":"2024-05-30T15:54:22.850865+0000","last_became_peered":"2024-05-30T15:54:22.850865+0000","last_unstale":"2024-05-30T17:13:56.917856+0000","last_undegraded":"2024-05-30T17:13:56.917856+0000","last_fullsized":"2024-05-30T17:13:56.917856+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'120","last_scrub_stamp":"2024-05-30T17:13:56.917584+0000","last_deep_scrub":"265'120","last_deep_scrub_stamp":"2024-05-30T17:13:56.917584+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.917584+0000","objects_scrubbed":7,"log_size":20,"log_dups_size":100,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:18:29.769236+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":127,"num_read_kb":121,"num_write":92,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1780,"num_omap_keys":6,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"265'541","reported_seq":4178,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.730062+0000","last_change":"2024-05-30T17:13:58.730127+0000","last_active":"2024-05-30T17:13:58.730062+0000","last_peered":"2024-05-30T17:13:58.730062+0000","last_clean":"2024-05-30T17:13:58.730062+0000","last_became_active":"2024-05-30T15:54:24.680695+0000","last_became_peered":"2024-05-30T15:54:24.680695+0000","last_unstale":"2024-05-30T17:13:58.730062+0000","last_undegraded":"2024-05-30T17:13:58.730062+0000","last_fullsized":"2024-05-30T17:13:58.730062+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'541","last_scrub_stamp":"2024-05-30T17:13:58.729875+0000","last_deep_scrub":"265'541","last_deep_scrub_stamp":"2024-05-30T17:13:58.729875+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.729875+0000","objects_scrubbed":1,"log_size":41,"log_dups_size":500,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:30:04.009452+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"265'135","reported_seq":652,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:01.631789+0000","last_change":"2024-05-30T17:14:01.631955+0000","last_active":"2024-05-30T17:14:01.631789+0000","last_peered":"2024-05-30T17:14:01.631789+0000","last_clean":"2024-05-30T17:14:01.631789+0000","last_became_active":"2024-05-30T15:54:26.695842+0000","last_became_peered":"2024-05-30T15:54:26.695842+0000","last_unstale":"2024-05-30T17:14:01.631789+0000","last_undegraded":"2024-05-30T17:14:01.631789+0000","last_fullsized":"2024-05-30T17:14:01.631789+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'135","last_scrub_stamp":"2024-05-30T17:14:01.631538+0000","last_deep_scrub":"265'135","last_deep_scrub_stamp":"2024-05-30T17:14:01.631538+0000","last_clean_scrub_stamp":"2024-05-30T17:14:01.631538+0000","objects_scrubbed":1,"log_size":35,"log_dups_size":100,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:51:24.578349+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":418,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":133,"num_read_kb":127,"num_write":334,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"265'429","reported_seq":1958,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:06.042514+0000","last_change":"2024-05-30T17:14:06.042514+0000","last_active":"2024-05-30T17:14:06.042514+0000","last_peered":"2024-05-30T17:14:06.042514+0000","last_clean":"2024-05-30T17:14:06.042514+0000","last_became_active":"2024-05-30T15:54:18.660567+0000","last_became_peered":"2024-05-30T15:54:18.660567+0000","last_unstale":"2024-05-30T17:14:06.042514+0000","last_undegraded":"2024-05-30T17:14:06.042514+0000","last_fullsized":"2024-05-30T17:14:06.042514+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'429","last_scrub_stamp":"2024-05-30T17:14:06.042078+0000","last_deep_scrub":"265'429","last_deep_scrub_stamp":"2024-05-30T17:14:06.042078+0000","last_clean_scrub_stamp":"2024-05-30T17:14:06.042078+0000","objects_scrubbed":6,"log_size":29,"log_dups_size":400,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:17:31.512141+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2226,"num_read_kb":2104,"num_write":504,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"265'306","reported_seq":1007,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:11.360417+0000","last_change":"2024-05-30T17:14:11.360417+0000","last_active":"2024-05-30T17:14:11.360417+0000","last_peered":"2024-05-30T17:14:11.360417+0000","last_clean":"2024-05-30T17:14:11.360417+0000","last_became_active":"2024-05-30T15:54:16.685691+0000","last_became_peered":"2024-05-30T15:54:16.685691+0000","last_unstale":"2024-05-30T17:14:11.360417+0000","last_undegraded":"2024-05-30T17:14:11.360417+0000","last_fullsized":"2024-05-30T17:14:11.360417+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'306","last_scrub_stamp":"2024-05-30T17:14:11.360138+0000","last_deep_scrub":"265'306","last_deep_scrub_stamp":"2024-05-30T17:14:11.360138+0000","last_clean_scrub_stamp":"2024-05-30T17:14:11.360138+0000","objects_scrubbed":1,"log_size":106,"log_dups_size":200,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:54:48.371531+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":764,"num_read_kb":1764,"num_write":1253,"num_write_kb":18095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":515,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:18.847018+0000","last_change":"2024-05-30T17:14:18.847018+0000","last_active":"2024-05-30T17:14:18.847018+0000","last_peered":"2024-05-30T17:14:18.847018+0000","last_clean":"2024-05-30T17:14:18.847018+0000","last_became_active":"2024-05-30T15:54:20.669751+0000","last_became_peered":"2024-05-30T15:54:20.669751+0000","last_unstale":"2024-05-30T17:14:18.847018+0000","last_undegraded":"2024-05-30T17:14:18.847018+0000","last_fullsized":"2024-05-30T17:14:18.847018+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:18.846755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:18.846755+0000","last_clean_scrub_stamp":"2024-05-30T17:14:18.846755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:48:09.585537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"265'225","reported_seq":846,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.220776+0000","last_change":"2024-05-30T17:13:59.220776+0000","last_active":"2024-05-30T17:13:59.220776+0000","last_peered":"2024-05-30T17:13:59.220776+0000","last_clean":"2024-05-30T17:13:59.220776+0000","last_became_active":"2024-05-30T15:54:22.672078+0000","last_became_peered":"2024-05-30T15:54:22.672078+0000","last_unstale":"2024-05-30T17:13:59.220776+0000","last_undegraded":"2024-05-30T17:13:59.220776+0000","last_fullsized":"2024-05-30T17:13:59.220776+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'225","last_scrub_stamp":"2024-05-30T17:13:59.220468+0000","last_deep_scrub":"265'225","last_deep_scrub_stamp":"2024-05-30T17:13:59.220468+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.220468+0000","objects_scrubbed":9,"log_size":25,"log_dups_size":200,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T05:11:49.982867+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10553,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":297,"num_read_kb":402,"num_write":186,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":513,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:05.864109+0000","last_change":"2024-05-30T17:14:05.864109+0000","last_active":"2024-05-30T17:14:05.864109+0000","last_peered":"2024-05-30T17:14:05.864109+0000","last_clean":"2024-05-30T17:14:05.864109+0000","last_became_active":"2024-05-30T15:54:24.675467+0000","last_became_peered":"2024-05-30T15:54:24.675467+0000","last_unstale":"2024-05-30T17:14:05.864109+0000","last_undegraded":"2024-05-30T17:14:05.864109+0000","last_fullsized":"2024-05-30T17:14:05.864109+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:05.863982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:05.863982+0000","last_clean_scrub_stamp":"2024-05-30T17:14:05.863982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:16:33.932497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"265'106","reported_seq":633,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.532471+0000","last_change":"2024-05-30T17:13:58.532471+0000","last_active":"2024-05-30T17:13:58.532471+0000","last_peered":"2024-05-30T17:13:58.532471+0000","last_clean":"2024-05-30T17:13:58.532471+0000","last_became_active":"2024-05-30T15:54:26.705264+0000","last_became_peered":"2024-05-30T15:54:26.705264+0000","last_unstale":"2024-05-30T17:13:58.532471+0000","last_undegraded":"2024-05-30T17:13:58.532471+0000","last_fullsized":"2024-05-30T17:13:58.532471+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'106","last_scrub_stamp":"2024-05-30T17:13:58.532222+0000","last_deep_scrub":"265'106","last_deep_scrub_stamp":"2024-05-30T17:13:58.532222+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.532222+0000","objects_scrubbed":4,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:07:52.286487+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":805,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":151,"num_read_kb":135,"num_write":247,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"265'700","reported_seq":2179,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.780482+0000","last_change":"2024-05-30T17:13:45.780482+0000","last_active":"2024-05-30T17:13:45.780482+0000","last_peered":"2024-05-30T17:13:45.780482+0000","last_clean":"2024-05-30T17:13:45.780482+0000","last_became_active":"2024-05-30T15:54:54.421604+0000","last_became_peered":"2024-05-30T15:54:54.421604+0000","last_unstale":"2024-05-30T17:13:45.780482+0000","last_undegraded":"2024-05-30T17:13:45.780482+0000","last_fullsized":"2024-05-30T17:13:45.780482+0000","mapping_epoch":35,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'700","last_scrub_stamp":"2024-05-30T17:13:45.780204+0000","last_deep_scrub":"265'700","last_deep_scrub_stamp":"2024-05-30T17:13:45.780204+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.780204+0000","objects_scrubbed":2,"log_size":100,"log_dups_size":600,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:24:04.304931+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3204,"num_read_kb":2893,"num_write":920,"num_write_kb":324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"265'173","reported_seq":824,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.218809+0000","last_change":"2024-05-30T17:13:46.218809+0000","last_active":"2024-05-30T17:13:46.218809+0000","last_peered":"2024-05-30T17:13:46.218809+0000","last_clean":"2024-05-30T17:13:46.218809+0000","last_became_active":"2024-05-30T15:54:54.422916+0000","last_became_peered":"2024-05-30T15:54:54.422916+0000","last_unstale":"2024-05-30T17:13:46.218809+0000","last_undegraded":"2024-05-30T17:13:46.218809+0000","last_fullsized":"2024-05-30T17:13:46.218809+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'173","last_scrub_stamp":"2024-05-30T17:13:46.218474+0000","last_deep_scrub":"265'173","last_deep_scrub_stamp":"2024-05-30T17:13:46.218474+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.218474+0000","objects_scrubbed":4,"log_size":73,"log_dups_size":100,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:17:58.546167+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505146,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":405,"num_read_kb":71111,"num_write":971,"num_write_kb":25767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"265'5","reported_seq":842,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:08.314165+0000","last_change":"2024-05-30T17:14:08.314165+0000","last_active":"2024-05-30T17:14:08.314165+0000","last_peered":"2024-05-30T17:14:08.314165+0000","last_clean":"2024-05-30T17:14:08.314165+0000","last_became_active":"2024-05-30T15:54:20.652134+0000","last_became_peered":"2024-05-30T15:54:20.652134+0000","last_unstale":"2024-05-30T17:14:08.314165+0000","last_undegraded":"2024-05-30T17:14:08.314165+0000","last_fullsized":"2024-05-30T17:14:08.314165+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'5","last_scrub_stamp":"2024-05-30T17:14:08.313909+0000","last_deep_scrub":"265'5","last_deep_scrub_stamp":"2024-05-30T17:14:08.313909+0000","last_clean_scrub_stamp":"2024-05-30T17:14:08.313909+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:56:03.138686+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1139,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":493,"num_read_kb":807,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"265'155","reported_seq":758,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:33.860511+0000","last_change":"2024-05-30T17:14:33.860704+0000","last_active":"2024-05-30T17:14:33.860511+0000","last_peered":"2024-05-30T17:14:33.860511+0000","last_clean":"2024-05-30T17:14:33.860511+0000","last_became_active":"2024-05-30T15:54:22.672169+0000","last_became_peered":"2024-05-30T15:54:22.672169+0000","last_unstale":"2024-05-30T17:14:33.860511+0000","last_undegraded":"2024-05-30T17:14:33.860511+0000","last_fullsized":"2024-05-30T17:14:33.860511+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'155","last_scrub_stamp":"2024-05-30T17:14:33.860288+0000","last_deep_scrub":"265'155","last_deep_scrub_stamp":"2024-05-30T17:14:33.860288+0000","last_clean_scrub_stamp":"2024-05-30T17:14:33.860288+0000","objects_scrubbed":4,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:55:28.475193+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6731,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":192,"num_read_kb":263,"num_write":102,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":507,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:52.740908+0000","last_change":"2024-05-30T17:13:52.740908+0000","last_active":"2024-05-30T17:13:52.740908+0000","last_peered":"2024-05-30T17:13:52.740908+0000","last_clean":"2024-05-30T17:13:52.740908+0000","last_became_active":"2024-05-30T15:54:24.675683+0000","last_became_peered":"2024-05-30T15:54:24.675683+0000","last_unstale":"2024-05-30T17:13:52.740908+0000","last_undegraded":"2024-05-30T17:13:52.740908+0000","last_fullsized":"2024-05-30T17:13:52.740908+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:52.740555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:52.740555+0000","last_clean_scrub_stamp":"2024-05-30T17:13:52.740555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:52:04.474974+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"265'128","reported_seq":666,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:01.346031+0000","last_change":"2024-05-30T17:14:01.346031+0000","last_active":"2024-05-30T17:14:01.346031+0000","last_peered":"2024-05-30T17:14:01.346031+0000","last_clean":"2024-05-30T17:14:01.346031+0000","last_became_active":"2024-05-30T15:54:26.689049+0000","last_became_peered":"2024-05-30T15:54:26.689049+0000","last_unstale":"2024-05-30T17:14:01.346031+0000","last_undegraded":"2024-05-30T17:14:01.346031+0000","last_fullsized":"2024-05-30T17:14:01.346031+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'128","last_scrub_stamp":"2024-05-30T17:14:01.345770+0000","last_deep_scrub":"265'128","last_deep_scrub_stamp":"2024-05-30T17:14:01.345770+0000","last_clean_scrub_stamp":"2024-05-30T17:14:01.345770+0000","objects_scrubbed":1,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:20:24.034017+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":539,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":167,"num_read_kb":166,"num_write":309,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"265'654","reported_seq":2081,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:20.410597+0000","last_change":"2024-05-30T17:14:20.410597+0000","last_active":"2024-05-30T17:14:20.410597+0000","last_peered":"2024-05-30T17:14:20.410597+0000","last_clean":"2024-05-30T17:14:20.410597+0000","last_became_active":"2024-05-30T15:54:18.651778+0000","last_became_peered":"2024-05-30T15:54:18.651778+0000","last_unstale":"2024-05-30T17:14:20.410597+0000","last_undegraded":"2024-05-30T17:14:20.410597+0000","last_fullsized":"2024-05-30T17:14:20.410597+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'654","last_scrub_stamp":"2024-05-30T17:14:20.410335+0000","last_deep_scrub":"265'654","last_deep_scrub_stamp":"2024-05-30T17:14:20.410335+0000","last_clean_scrub_stamp":"2024-05-30T17:14:20.410335+0000","objects_scrubbed":2,"log_size":54,"log_dups_size":600,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:59:38.502836+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3068,"num_read_kb":2749,"num_write":930,"num_write_kb":367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"265'151","reported_seq":775,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:31.369696+0000","last_change":"2024-05-30T17:14:31.369696+0000","last_active":"2024-05-30T17:14:31.369696+0000","last_peered":"2024-05-30T17:14:31.369696+0000","last_clean":"2024-05-30T17:14:31.369696+0000","last_became_active":"2024-05-30T15:54:16.656081+0000","last_became_peered":"2024-05-30T15:54:16.656081+0000","last_unstale":"2024-05-30T17:14:31.369696+0000","last_undegraded":"2024-05-30T17:14:31.369696+0000","last_fullsized":"2024-05-30T17:14:31.369696+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'151","last_scrub_stamp":"2024-05-30T17:14:31.369421+0000","last_deep_scrub":"265'151","last_deep_scrub_stamp":"2024-05-30T17:14:31.369421+0000","last_clean_scrub_stamp":"2024-05-30T17:14:31.369421+0000","objects_scrubbed":1,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:35:41.015817+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":369,"num_read_kb":8201,"num_write":842,"num_write_kb":22196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":525,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:11.063361+0000","last_change":"2024-05-30T17:14:11.063466+0000","last_active":"2024-05-30T17:14:11.063361+0000","last_peered":"2024-05-30T17:14:11.063361+0000","last_clean":"2024-05-30T17:14:11.063361+0000","last_became_active":"2024-05-30T15:54:20.661090+0000","last_became_peered":"2024-05-30T15:54:20.661090+0000","last_unstale":"2024-05-30T17:14:11.063361+0000","last_undegraded":"2024-05-30T17:14:11.063361+0000","last_fullsized":"2024-05-30T17:14:11.063361+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:11.063113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:11.063113+0000","last_clean_scrub_stamp":"2024-05-30T17:14:11.063113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:16:18.209002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"265'118","reported_seq":700,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:23.403551+0000","last_change":"2024-05-30T17:14:23.403551+0000","last_active":"2024-05-30T17:14:23.403551+0000","last_peered":"2024-05-30T17:14:23.403551+0000","last_clean":"2024-05-30T17:14:23.403551+0000","last_became_active":"2024-05-30T15:54:22.669073+0000","last_became_peered":"2024-05-30T15:54:22.669073+0000","last_unstale":"2024-05-30T17:14:23.403551+0000","last_undegraded":"2024-05-30T17:14:23.403551+0000","last_fullsized":"2024-05-30T17:14:23.403551+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'118","last_scrub_stamp":"2024-05-30T17:14:23.403251+0000","last_deep_scrub":"265'118","last_deep_scrub_stamp":"2024-05-30T17:14:23.403251+0000","last_clean_scrub_stamp":"2024-05-30T17:14:23.403251+0000","objects_scrubbed":2,"log_size":18,"log_dups_size":100,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:07:50.837787+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6915,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":145,"num_read_kb":188,"num_write":74,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":516,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.882025+0000","last_change":"2024-05-30T17:13:53.882025+0000","last_active":"2024-05-30T17:13:53.882025+0000","last_peered":"2024-05-30T17:13:53.882025+0000","last_clean":"2024-05-30T17:13:53.882025+0000","last_became_active":"2024-05-30T15:54:24.678292+0000","last_became_peered":"2024-05-30T15:54:24.678292+0000","last_unstale":"2024-05-30T17:13:53.882025+0000","last_undegraded":"2024-05-30T17:13:53.882025+0000","last_fullsized":"2024-05-30T17:13:53.882025+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:53.881772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:53.881772+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.881772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:31:21.481218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"265'171","reported_seq":918,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.566327+0000","last_change":"2024-05-30T17:13:59.566327+0000","last_active":"2024-05-30T17:13:59.566327+0000","last_peered":"2024-05-30T17:13:59.566327+0000","last_clean":"2024-05-30T17:13:59.566327+0000","last_became_active":"2024-05-30T15:54:26.703440+0000","last_became_peered":"2024-05-30T15:54:26.703440+0000","last_unstale":"2024-05-30T17:13:59.566327+0000","last_undegraded":"2024-05-30T17:13:59.566327+0000","last_fullsized":"2024-05-30T17:13:59.566327+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'171","last_scrub_stamp":"2024-05-30T17:13:59.565988+0000","last_deep_scrub":"265'171","last_deep_scrub_stamp":"2024-05-30T17:13:59.565988+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.565988+0000","objects_scrubbed":7,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:51:56.619593+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1074,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":420,"num_read_kb":450,"num_write":403,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1200,"num_omap_keys":6,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"265'333","reported_seq":1697,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:02.992132+0000","last_change":"2024-05-30T17:14:02.992132+0000","last_active":"2024-05-30T17:14:02.992132+0000","last_peered":"2024-05-30T17:14:02.992132+0000","last_clean":"2024-05-30T17:14:02.992132+0000","last_became_active":"2024-05-30T15:54:18.671650+0000","last_became_peered":"2024-05-30T15:54:18.671650+0000","last_unstale":"2024-05-30T17:14:02.992132+0000","last_undegraded":"2024-05-30T17:14:02.992132+0000","last_fullsized":"2024-05-30T17:14:02.992132+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'333","last_scrub_stamp":"2024-05-30T17:14:02.991716+0000","last_deep_scrub":"265'333","last_deep_scrub_stamp":"2024-05-30T17:14:02.991716+0000","last_clean_scrub_stamp":"2024-05-30T17:14:02.991716+0000","objects_scrubbed":6,"log_size":33,"log_dups_size":300,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:01:06.796193+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1672,"num_read_kb":1605,"num_write":361,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"265'206","reported_seq":876,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:32.836244+0000","last_change":"2024-05-30T17:14:32.836326+0000","last_active":"2024-05-30T17:14:32.836244+0000","last_peered":"2024-05-30T17:14:32.836244+0000","last_clean":"2024-05-30T17:14:32.836244+0000","last_became_active":"2024-05-30T15:54:16.660393+0000","last_became_peered":"2024-05-30T15:54:16.660393+0000","last_unstale":"2024-05-30T17:14:32.836244+0000","last_undegraded":"2024-05-30T17:14:32.836244+0000","last_fullsized":"2024-05-30T17:14:32.836244+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'206","last_scrub_stamp":"2024-05-30T17:14:32.836035+0000","last_deep_scrub":"265'206","last_deep_scrub_stamp":"2024-05-30T17:14:32.836035+0000","last_clean_scrub_stamp":"2024-05-30T17:14:32.836035+0000","objects_scrubbed":2,"log_size":106,"log_dups_size":100,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:37:35.758467+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":594502,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":519,"num_read_kb":9177,"num_write":1141,"num_write_kb":26248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"265'2","reported_seq":519,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.016076+0000","last_change":"2024-05-30T17:13:58.016076+0000","last_active":"2024-05-30T17:13:58.016076+0000","last_peered":"2024-05-30T17:13:58.016076+0000","last_clean":"2024-05-30T17:13:58.016076+0000","last_became_active":"2024-05-30T15:54:20.661311+0000","last_became_peered":"2024-05-30T15:54:20.661311+0000","last_unstale":"2024-05-30T17:13:58.016076+0000","last_undegraded":"2024-05-30T17:13:58.016076+0000","last_fullsized":"2024-05-30T17:13:58.016076+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:13:58.015783+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:13:58.015783+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.015783+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:05:29.698239+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"265'180","reported_seq":801,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:15.065220+0000","last_change":"2024-05-30T17:14:15.065220+0000","last_active":"2024-05-30T17:14:15.065220+0000","last_peered":"2024-05-30T17:14:15.065220+0000","last_clean":"2024-05-30T17:14:15.065220+0000","last_became_active":"2024-05-30T15:54:22.663231+0000","last_became_peered":"2024-05-30T15:54:22.663231+0000","last_unstale":"2024-05-30T17:14:15.065220+0000","last_undegraded":"2024-05-30T17:14:15.065220+0000","last_fullsized":"2024-05-30T17:14:15.065220+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'180","last_scrub_stamp":"2024-05-30T17:14:15.064955+0000","last_deep_scrub":"265'180","last_deep_scrub_stamp":"2024-05-30T17:14:15.064955+0000","last_clean_scrub_stamp":"2024-05-30T17:14:15.064955+0000","objects_scrubbed":5,"log_size":80,"log_dups_size":100,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:54:03.215677+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4459,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":234,"num_read_kb":297,"num_write":134,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":512,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:04.853635+0000","last_change":"2024-05-30T17:14:04.853635+0000","last_active":"2024-05-30T17:14:04.853635+0000","last_peered":"2024-05-30T17:14:04.853635+0000","last_clean":"2024-05-30T17:14:04.853635+0000","last_became_active":"2024-05-30T15:54:24.673674+0000","last_became_peered":"2024-05-30T15:54:24.673674+0000","last_unstale":"2024-05-30T17:14:04.853635+0000","last_undegraded":"2024-05-30T17:14:04.853635+0000","last_fullsized":"2024-05-30T17:14:04.853635+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:04.853272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:04.853272+0000","last_clean_scrub_stamp":"2024-05-30T17:14:04.853272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:21:26.208754+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"265'114","reported_seq":629,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.822855+0000","last_change":"2024-05-30T17:13:48.822855+0000","last_active":"2024-05-30T17:13:48.822855+0000","last_peered":"2024-05-30T17:13:48.822855+0000","last_clean":"2024-05-30T17:13:48.822855+0000","last_became_active":"2024-05-30T15:54:26.677418+0000","last_became_peered":"2024-05-30T15:54:26.677418+0000","last_unstale":"2024-05-30T17:13:48.822855+0000","last_undegraded":"2024-05-30T17:13:48.822855+0000","last_fullsized":"2024-05-30T17:13:48.822855+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'114","last_scrub_stamp":"2024-05-30T17:13:48.822470+0000","last_deep_scrub":"265'114","last_deep_scrub_stamp":"2024-05-30T17:13:48.822470+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.822470+0000","objects_scrubbed":3,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:49:43.113999+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":435,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":67,"num_write":244,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"265'308","reported_seq":1721,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.056454+0000","last_change":"2024-05-30T17:13:59.056454+0000","last_active":"2024-05-30T17:13:59.056454+0000","last_peered":"2024-05-30T17:13:59.056454+0000","last_clean":"2024-05-30T17:13:59.056454+0000","last_became_active":"2024-05-30T15:54:18.671335+0000","last_became_peered":"2024-05-30T15:54:18.671335+0000","last_unstale":"2024-05-30T17:13:59.056454+0000","last_undegraded":"2024-05-30T17:13:59.056454+0000","last_fullsized":"2024-05-30T17:13:59.056454+0000","mapping_epoch":24,"log_start":"265'200","ondisk_log_start":"265'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'308","last_scrub_stamp":"2024-05-30T17:13:59.056066+0000","last_deep_scrub":"265'308","last_deep_scrub_stamp":"2024-05-30T17:13:59.056066+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.056066+0000","objects_scrubbed":3,"log_size":108,"log_dups_size":200,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:17:28.319253+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1771,"num_read_kb":1705,"num_write":340,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"265'206","reported_seq":867,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:09.773002+0000","last_change":"2024-05-30T17:14:09.773202+0000","last_active":"2024-05-30T17:14:09.773002+0000","last_peered":"2024-05-30T17:14:09.773002+0000","last_clean":"2024-05-30T17:14:09.773002+0000","last_became_active":"2024-05-30T15:54:16.674678+0000","last_became_peered":"2024-05-30T15:54:16.674678+0000","last_unstale":"2024-05-30T17:14:09.773002+0000","last_undegraded":"2024-05-30T17:14:09.773002+0000","last_fullsized":"2024-05-30T17:14:09.773002+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'206","last_scrub_stamp":"2024-05-30T17:14:09.772714+0000","last_deep_scrub":"265'206","last_deep_scrub_stamp":"2024-05-30T17:14:09.772714+0000","last_clean_scrub_stamp":"2024-05-30T17:14:09.772714+0000","objects_scrubbed":6,"log_size":106,"log_dups_size":100,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:12:22.788923+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25713,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":493,"num_read_kb":10347,"num_write":1036,"num_write_kb":23027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"265'562","reported_seq":2288,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:18.965782+0000","last_change":"2024-05-30T17:14:18.965782+0000","last_active":"2024-05-30T17:14:18.965782+0000","last_peered":"2024-05-30T17:14:18.965782+0000","last_clean":"2024-05-30T17:14:18.965782+0000","last_became_active":"2024-05-30T15:54:18.635092+0000","last_became_peered":"2024-05-30T15:54:18.635092+0000","last_unstale":"2024-05-30T17:14:18.965782+0000","last_undegraded":"2024-05-30T17:14:18.965782+0000","last_fullsized":"2024-05-30T17:14:18.965782+0000","mapping_epoch":24,"log_start":"265'500","ondisk_log_start":"265'500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'562","last_scrub_stamp":"2024-05-30T17:14:18.965373+0000","last_deep_scrub":"265'562","last_deep_scrub_stamp":"2024-05-30T17:14:18.965373+0000","last_clean_scrub_stamp":"2024-05-30T17:14:18.965373+0000","objects_scrubbed":3,"log_size":62,"log_dups_size":500,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:13:33.470676+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2950,"num_read_kb":2739,"num_write":700,"num_write_kb":188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1511,"num_omap_keys":6,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"265'165","reported_seq":788,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:01.304037+0000","last_change":"2024-05-30T17:14:01.304121+0000","last_active":"2024-05-30T17:14:01.304037+0000","last_peered":"2024-05-30T17:14:01.304037+0000","last_clean":"2024-05-30T17:14:01.304037+0000","last_became_active":"2024-05-30T15:54:16.658183+0000","last_became_peered":"2024-05-30T15:54:16.658183+0000","last_unstale":"2024-05-30T17:14:01.304037+0000","last_undegraded":"2024-05-30T17:14:01.304037+0000","last_fullsized":"2024-05-30T17:14:01.304037+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'165","last_scrub_stamp":"2024-05-30T17:14:01.303807+0000","last_deep_scrub":"265'165","last_deep_scrub_stamp":"2024-05-30T17:14:01.303807+0000","last_clean_scrub_stamp":"2024-05-30T17:14:01.303807+0000","objects_scrubbed":1,"log_size":65,"log_dups_size":100,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:51:27.141500+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":308,"num_read_kb":6656,"num_write":926,"num_write_kb":27882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"265'629","reported_seq":2195,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:02.302193+0000","last_change":"2024-05-30T17:14:02.302193+0000","last_active":"2024-05-30T17:14:02.302193+0000","last_peered":"2024-05-30T17:14:02.302193+0000","last_clean":"2024-05-30T17:14:02.302193+0000","last_became_active":"2024-05-30T15:54:18.662159+0000","last_became_peered":"2024-05-30T15:54:18.662159+0000","last_unstale":"2024-05-30T17:14:02.302193+0000","last_undegraded":"2024-05-30T17:14:02.302193+0000","last_fullsized":"2024-05-30T17:14:02.302193+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'629","last_scrub_stamp":"2024-05-30T17:14:02.301919+0000","last_deep_scrub":"265'629","last_deep_scrub_stamp":"2024-05-30T17:14:02.301919+0000","last_clean_scrub_stamp":"2024-05-30T17:14:02.301919+0000","objects_scrubbed":4,"log_size":29,"log_dups_size":600,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:46:32.528521+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3191,"num_read_kb":2926,"num_write":899,"num_write_kb":342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":249,"num_omap_keys":1,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"265'210","reported_seq":872,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:17.310246+0000","last_change":"2024-05-30T17:14:17.310357+0000","last_active":"2024-05-30T17:14:17.310246+0000","last_peered":"2024-05-30T17:14:17.310246+0000","last_clean":"2024-05-30T17:14:17.310246+0000","last_became_active":"2024-05-30T15:54:16.652664+0000","last_became_peered":"2024-05-30T15:54:16.652664+0000","last_unstale":"2024-05-30T17:14:17.310246+0000","last_undegraded":"2024-05-30T17:14:17.310246+0000","last_fullsized":"2024-05-30T17:14:17.310246+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'210","last_scrub_stamp":"2024-05-30T17:14:17.310051+0000","last_deep_scrub":"265'210","last_deep_scrub_stamp":"2024-05-30T17:14:17.310051+0000","last_clean_scrub_stamp":"2024-05-30T17:14:17.310051+0000","objects_scrubbed":6,"log_size":110,"log_dups_size":100,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:39:16.444559+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524420,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":451,"num_read_kb":27404,"num_write":1156,"num_write_kb":29020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"265'1306","reported_seq":2951,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:09.025175+0000","last_change":"2024-05-30T17:14:09.025175+0000","last_active":"2024-05-30T17:14:09.025175+0000","last_peered":"2024-05-30T17:14:09.025175+0000","last_clean":"2024-05-30T17:14:09.025175+0000","last_became_active":"2024-05-30T15:54:18.642697+0000","last_became_peered":"2024-05-30T15:54:18.642697+0000","last_unstale":"2024-05-30T17:14:09.025175+0000","last_undegraded":"2024-05-30T17:14:09.025175+0000","last_fullsized":"2024-05-30T17:14:09.025175+0000","mapping_epoch":24,"log_start":"265'1200","ondisk_log_start":"265'1200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'1306","last_scrub_stamp":"2024-05-30T17:14:09.024879+0000","last_deep_scrub":"265'1306","last_deep_scrub_stamp":"2024-05-30T17:14:09.024879+0000","last_clean_scrub_stamp":"2024-05-30T17:14:09.024879+0000","objects_scrubbed":9,"log_size":106,"log_dups_size":1200,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:00:44.114885+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":5484,"num_read_kb":4752,"num_write":1832,"num_write_kb":807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1870,"num_omap_keys":8,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"265'256","reported_seq":998,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:31.804410+0000","last_change":"2024-05-30T17:14:31.804556+0000","last_active":"2024-05-30T17:14:31.804410+0000","last_peered":"2024-05-30T17:14:31.804410+0000","last_clean":"2024-05-30T17:14:31.804410+0000","last_became_active":"2024-05-30T15:54:16.659589+0000","last_became_peered":"2024-05-30T15:54:16.659589+0000","last_unstale":"2024-05-30T17:14:31.804410+0000","last_undegraded":"2024-05-30T17:14:31.804410+0000","last_fullsized":"2024-05-30T17:14:31.804410+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'256","last_scrub_stamp":"2024-05-30T17:14:31.804205+0000","last_deep_scrub":"265'256","last_deep_scrub_stamp":"2024-05-30T17:14:31.804205+0000","last_clean_scrub_stamp":"2024-05-30T17:14:31.804205+0000","objects_scrubbed":6,"log_size":56,"log_dups_size":200,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:22:26.640703+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3222795,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":793,"num_read_kb":47588,"num_write":1564,"num_write_kb":36318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"265'1103","reported_seq":2692,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:24.360537+0000","last_change":"2024-05-30T17:14:24.360701+0000","last_active":"2024-05-30T17:14:24.360537+0000","last_peered":"2024-05-30T17:14:24.360537+0000","last_clean":"2024-05-30T17:14:24.360537+0000","last_became_active":"2024-05-30T15:54:18.652794+0000","last_became_peered":"2024-05-30T15:54:18.652794+0000","last_unstale":"2024-05-30T17:14:24.360537+0000","last_undegraded":"2024-05-30T17:14:24.360537+0000","last_fullsized":"2024-05-30T17:14:24.360537+0000","mapping_epoch":24,"log_start":"265'1000","ondisk_log_start":"265'1000","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'1103","last_scrub_stamp":"2024-05-30T17:14:24.360317+0000","last_deep_scrub":"265'1103","last_deep_scrub_stamp":"2024-05-30T17:14:24.360317+0000","last_clean_scrub_stamp":"2024-05-30T17:14:24.360317+0000","objects_scrubbed":2,"log_size":103,"log_dups_size":1000,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:53:29.381532+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4733,"num_read_kb":4179,"num_write":1497,"num_write_kb":596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":246,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"265'225","reported_seq":903,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:52.362088+0000","last_change":"2024-05-30T17:13:52.362088+0000","last_active":"2024-05-30T17:13:52.362088+0000","last_peered":"2024-05-30T17:13:52.362088+0000","last_clean":"2024-05-30T17:13:52.362088+0000","last_became_active":"2024-05-30T15:54:54.451265+0000","last_became_peered":"2024-05-30T15:54:54.451265+0000","last_unstale":"2024-05-30T17:13:52.362088+0000","last_undegraded":"2024-05-30T17:13:52.362088+0000","last_fullsized":"2024-05-30T17:13:52.362088+0000","mapping_epoch":35,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'225","last_scrub_stamp":"2024-05-30T17:13:52.361791+0000","last_deep_scrub":"265'225","last_deep_scrub_stamp":"2024-05-30T17:13:52.361791+0000","last_clean_scrub_stamp":"2024-05-30T17:13:52.361791+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":200,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:41:32.517363+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524357,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":480,"num_read_kb":2675,"num_write":1165,"num_write_kb":43073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"265'389","reported_seq":1961,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:04.004662+0000","last_change":"2024-05-30T17:14:04.004748+0000","last_active":"2024-05-30T17:14:04.004662+0000","last_peered":"2024-05-30T17:14:04.004662+0000","last_clean":"2024-05-30T17:14:04.004662+0000","last_became_active":"2024-05-30T15:54:18.670801+0000","last_became_peered":"2024-05-30T15:54:18.670801+0000","last_unstale":"2024-05-30T17:14:04.004662+0000","last_undegraded":"2024-05-30T17:14:04.004662+0000","last_fullsized":"2024-05-30T17:14:04.004662+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'389","last_scrub_stamp":"2024-05-30T17:14:04.004452+0000","last_deep_scrub":"265'389","last_deep_scrub_stamp":"2024-05-30T17:14:04.004452+0000","last_clean_scrub_stamp":"2024-05-30T17:14:04.004452+0000","objects_scrubbed":4,"log_size":89,"log_dups_size":300,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:27:27.990698+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2215,"num_read_kb":2115,"num_write":443,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2553,"num_omap_keys":10,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"265'222","reported_seq":883,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:14.943979+0000","last_change":"2024-05-30T17:14:14.943979+0000","last_active":"2024-05-30T17:14:14.943979+0000","last_peered":"2024-05-30T17:14:14.943979+0000","last_clean":"2024-05-30T17:14:14.943979+0000","last_became_active":"2024-05-30T15:54:16.649854+0000","last_became_peered":"2024-05-30T15:54:16.649854+0000","last_unstale":"2024-05-30T17:14:14.943979+0000","last_undegraded":"2024-05-30T17:14:14.943979+0000","last_fullsized":"2024-05-30T17:14:14.943979+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'222","last_scrub_stamp":"2024-05-30T17:14:14.943688+0000","last_deep_scrub":"265'222","last_deep_scrub_stamp":"2024-05-30T17:14:14.943688+0000","last_clean_scrub_stamp":"2024-05-30T17:14:14.943688+0000","objects_scrubbed":2,"log_size":22,"log_dups_size":200,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:30:00.639949+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306443,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":499,"num_read_kb":6249,"num_write":1151,"num_write_kb":30962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"265'410","reported_seq":1939,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:26.362322+0000","last_change":"2024-05-30T17:14:26.362393+0000","last_active":"2024-05-30T17:14:26.362322+0000","last_peered":"2024-05-30T17:14:26.362322+0000","last_clean":"2024-05-30T17:14:26.362322+0000","last_became_active":"2024-05-30T15:54:18.650832+0000","last_became_peered":"2024-05-30T15:54:18.650832+0000","last_unstale":"2024-05-30T17:14:26.362322+0000","last_undegraded":"2024-05-30T17:14:26.362322+0000","last_fullsized":"2024-05-30T17:14:26.362322+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'410","last_scrub_stamp":"2024-05-30T17:14:26.362163+0000","last_deep_scrub":"265'410","last_deep_scrub_stamp":"2024-05-30T17:14:26.362163+0000","last_clean_scrub_stamp":"2024-05-30T17:14:26.362163+0000","objects_scrubbed":5,"log_size":110,"log_dups_size":300,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:14:40.469181+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2240,"num_read_kb":2125,"num_write":507,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":248,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"265'187","reported_seq":821,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:10.744038+0000","last_change":"2024-05-30T17:14:10.744038+0000","last_active":"2024-05-30T17:14:10.744038+0000","last_peered":"2024-05-30T17:14:10.744038+0000","last_clean":"2024-05-30T17:14:10.744038+0000","last_became_active":"2024-05-30T15:54:16.684670+0000","last_became_peered":"2024-05-30T15:54:16.684670+0000","last_unstale":"2024-05-30T17:14:10.744038+0000","last_undegraded":"2024-05-30T17:14:10.744038+0000","last_fullsized":"2024-05-30T17:14:10.744038+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'187","last_scrub_stamp":"2024-05-30T17:14:10.743692+0000","last_deep_scrub":"265'187","last_deep_scrub_stamp":"2024-05-30T17:14:10.743692+0000","last_clean_scrub_stamp":"2024-05-30T17:14:10.743692+0000","objects_scrubbed":7,"log_size":87,"log_dups_size":100,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:32:19.992966+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":254692,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":373,"num_read_kb":34719,"num_write":1090,"num_write_kb":29777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"265'823","reported_seq":2424,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:03.753084+0000","last_change":"2024-05-30T17:14:03.753161+0000","last_active":"2024-05-30T17:14:03.753084+0000","last_peered":"2024-05-30T17:14:03.753084+0000","last_clean":"2024-05-30T17:14:03.753084+0000","last_became_active":"2024-05-30T15:54:18.669105+0000","last_became_peered":"2024-05-30T15:54:18.669105+0000","last_unstale":"2024-05-30T17:14:03.753084+0000","last_undegraded":"2024-05-30T17:14:03.753084+0000","last_fullsized":"2024-05-30T17:14:03.753084+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'823","last_scrub_stamp":"2024-05-30T17:14:03.752908+0000","last_deep_scrub":"265'823","last_deep_scrub_stamp":"2024-05-30T17:14:03.752908+0000","last_clean_scrub_stamp":"2024-05-30T17:14:03.752908+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":800,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:05:28.270822+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3660,"num_read_kb":3297,"num_write":1114,"num_write_kb":434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":252,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"265'220","reported_seq":927,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.975897+0000","last_change":"2024-05-30T17:13:48.976145+0000","last_active":"2024-05-30T17:13:48.975897+0000","last_peered":"2024-05-30T17:13:48.975897+0000","last_clean":"2024-05-30T17:13:48.975897+0000","last_became_active":"2024-05-30T15:54:54.416482+0000","last_became_peered":"2024-05-30T15:54:54.416482+0000","last_unstale":"2024-05-30T17:13:48.975897+0000","last_undegraded":"2024-05-30T17:13:48.975897+0000","last_fullsized":"2024-05-30T17:13:48.975897+0000","mapping_epoch":35,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'220","last_scrub_stamp":"2024-05-30T17:13:48.975457+0000","last_deep_scrub":"265'220","last_deep_scrub_stamp":"2024-05-30T17:13:48.975457+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.975457+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":200,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:44:46.059246+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5708449,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":729,"num_read_kb":137610,"num_write":1205,"num_write_kb":64841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2f","version":"265'389","reported_seq":1986,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:10.889831+0000","last_change":"2024-05-30T17:14:10.890006+0000","last_active":"2024-05-30T17:14:10.889831+0000","last_peered":"2024-05-30T17:14:10.889831+0000","last_clean":"2024-05-30T17:14:10.889831+0000","last_became_active":"2024-05-30T15:54:18.657236+0000","last_became_peered":"2024-05-30T15:54:18.657236+0000","last_unstale":"2024-05-30T17:14:10.889831+0000","last_undegraded":"2024-05-30T17:14:10.889831+0000","last_fullsized":"2024-05-30T17:14:10.889831+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'389","last_scrub_stamp":"2024-05-30T17:14:10.889524+0000","last_deep_scrub":"265'389","last_deep_scrub_stamp":"2024-05-30T17:14:10.889524+0000","last_clean_scrub_stamp":"2024-05-30T17:14:10.889524+0000","objects_scrubbed":4,"log_size":89,"log_dups_size":300,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:08:38.176469+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2258,"num_read_kb":2156,"num_write":442,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2808,"num_omap_keys":11,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"265'203","reported_seq":892,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:23.270071+0000","last_change":"2024-05-30T17:14:23.270071+0000","last_active":"2024-05-30T17:14:23.270071+0000","last_peered":"2024-05-30T17:14:23.270071+0000","last_clean":"2024-05-30T17:14:23.270071+0000","last_became_active":"2024-05-30T15:54:16.653037+0000","last_became_peered":"2024-05-30T15:54:16.653037+0000","last_unstale":"2024-05-30T17:14:23.270071+0000","last_undegraded":"2024-05-30T17:14:23.270071+0000","last_fullsized":"2024-05-30T17:14:23.270071+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'203","last_scrub_stamp":"2024-05-30T17:14:23.269779+0000","last_deep_scrub":"265'203","last_deep_scrub_stamp":"2024-05-30T17:14:23.269779+0000","last_clean_scrub_stamp":"2024-05-30T17:14:23.269779+0000","objects_scrubbed":5,"log_size":103,"log_dups_size":100,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:42:04.082214+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2108965,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":543,"num_read_kb":29183,"num_write":977,"num_write_kb":32330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"265'334","reported_seq":1841,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:21.115099+0000","last_change":"2024-05-30T17:14:21.115173+0000","last_active":"2024-05-30T17:14:21.115099+0000","last_peered":"2024-05-30T17:14:21.115099+0000","last_clean":"2024-05-30T17:14:21.115099+0000","last_became_active":"2024-05-30T15:54:18.635382+0000","last_became_peered":"2024-05-30T15:54:18.635382+0000","last_unstale":"2024-05-30T17:14:21.115099+0000","last_undegraded":"2024-05-30T17:14:21.115099+0000","last_fullsized":"2024-05-30T17:14:21.115099+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'334","last_scrub_stamp":"2024-05-30T17:14:21.114886+0000","last_deep_scrub":"265'334","last_deep_scrub_stamp":"2024-05-30T17:14:21.114886+0000","last_clean_scrub_stamp":"2024-05-30T17:14:21.114886+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:17:15.854111+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2071,"num_read_kb":1981,"num_write":382,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2453,"num_omap_keys":10,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"265'159","reported_seq":785,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:36.497334+0000","last_change":"2024-05-30T17:14:36.497413+0000","last_active":"2024-05-30T17:14:36.497334+0000","last_peered":"2024-05-30T17:14:36.497334+0000","last_clean":"2024-05-30T17:14:36.497334+0000","last_became_active":"2024-05-30T15:54:16.682629+0000","last_became_peered":"2024-05-30T15:54:16.682629+0000","last_unstale":"2024-05-30T17:14:36.497334+0000","last_undegraded":"2024-05-30T17:14:36.497334+0000","last_fullsized":"2024-05-30T17:14:36.497334+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'159","last_scrub_stamp":"2024-05-30T17:14:36.497105+0000","last_deep_scrub":"265'159","last_deep_scrub_stamp":"2024-05-30T17:14:36.497105+0000","last_clean_scrub_stamp":"2024-05-30T17:14:36.497105+0000","objects_scrubbed":2,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:16:58.514671+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310739,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":22431,"num_write":822,"num_write_kb":23734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"265'747","reported_seq":2325,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:06.362930+0000","last_change":"2024-05-30T17:14:06.362999+0000","last_active":"2024-05-30T17:14:06.362930+0000","last_peered":"2024-05-30T17:14:06.362930+0000","last_clean":"2024-05-30T17:14:06.362930+0000","last_became_active":"2024-05-30T15:54:18.668748+0000","last_became_peered":"2024-05-30T15:54:18.668748+0000","last_unstale":"2024-05-30T17:14:06.362930+0000","last_undegraded":"2024-05-30T17:14:06.362930+0000","last_fullsized":"2024-05-30T17:14:06.362930+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'747","last_scrub_stamp":"2024-05-30T17:14:06.362744+0000","last_deep_scrub":"265'747","last_deep_scrub_stamp":"2024-05-30T17:14:06.362744+0000","last_clean_scrub_stamp":"2024-05-30T17:14:06.362744+0000","objects_scrubbed":4,"log_size":47,"log_dups_size":700,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:14:16.405343+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3385,"num_read_kb":3066,"num_write":980,"num_write_kb":340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"265'165","reported_seq":909,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:13.340658+0000","last_change":"2024-05-30T17:14:13.340762+0000","last_active":"2024-05-30T17:14:13.340658+0000","last_peered":"2024-05-30T17:14:13.340658+0000","last_clean":"2024-05-30T17:14:13.340658+0000","last_became_active":"2024-05-30T15:54:16.674993+0000","last_became_peered":"2024-05-30T15:54:16.674993+0000","last_unstale":"2024-05-30T17:14:13.340658+0000","last_undegraded":"2024-05-30T17:14:13.340658+0000","last_fullsized":"2024-05-30T17:14:13.340658+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'165","last_scrub_stamp":"2024-05-30T17:14:13.340433+0000","last_deep_scrub":"265'165","last_deep_scrub_stamp":"2024-05-30T17:14:13.340433+0000","last_clean_scrub_stamp":"2024-05-30T17:14:13.340433+0000","objects_scrubbed":5,"log_size":65,"log_dups_size":100,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:25:29.735661+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6553619,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":567,"num_read_kb":112844,"num_write":925,"num_write_kb":45704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"265'472","reported_seq":2122,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:01.015512+0000","last_change":"2024-05-30T17:14:01.015577+0000","last_active":"2024-05-30T17:14:01.015512+0000","last_peered":"2024-05-30T17:14:01.015512+0000","last_clean":"2024-05-30T17:14:01.015512+0000","last_became_active":"2024-05-30T15:54:18.671434+0000","last_became_peered":"2024-05-30T15:54:18.671434+0000","last_unstale":"2024-05-30T17:14:01.015512+0000","last_undegraded":"2024-05-30T17:14:01.015512+0000","last_fullsized":"2024-05-30T17:14:01.015512+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'472","last_scrub_stamp":"2024-05-30T17:14:01.015328+0000","last_deep_scrub":"265'472","last_deep_scrub_stamp":"2024-05-30T17:14:01.015328+0000","last_clean_scrub_stamp":"2024-05-30T17:14:01.015328+0000","objects_scrubbed":1,"log_size":72,"log_dups_size":400,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:21:04.896268+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2525,"num_read_kb":2390,"num_write":596,"num_write_kb":183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"265'204","reported_seq":867,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:25.830042+0000","last_change":"2024-05-30T17:14:25.830133+0000","last_active":"2024-05-30T17:14:25.830042+0000","last_peered":"2024-05-30T17:14:25.830042+0000","last_clean":"2024-05-30T17:14:25.830042+0000","last_became_active":"2024-05-30T15:54:16.629843+0000","last_became_peered":"2024-05-30T15:54:16.629843+0000","last_unstale":"2024-05-30T17:14:25.830042+0000","last_undegraded":"2024-05-30T17:14:25.830042+0000","last_fullsized":"2024-05-30T17:14:25.830042+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'204","last_scrub_stamp":"2024-05-30T17:14:25.829810+0000","last_deep_scrub":"265'204","last_deep_scrub_stamp":"2024-05-30T17:14:25.829810+0000","last_clean_scrub_stamp":"2024-05-30T17:14:25.829810+0000","objects_scrubbed":3,"log_size":104,"log_dups_size":100,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:17:22.937587+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524465,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":434,"num_read_kb":42078,"num_write":1152,"num_write_kb":26102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"265'516","reported_seq":2150,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:11.810946+0000","last_change":"2024-05-30T17:14:11.811023+0000","last_active":"2024-05-30T17:14:11.810946+0000","last_peered":"2024-05-30T17:14:11.810946+0000","last_clean":"2024-05-30T17:14:11.810946+0000","last_became_active":"2024-05-30T15:54:18.660101+0000","last_became_peered":"2024-05-30T15:54:18.660101+0000","last_unstale":"2024-05-30T17:14:11.810946+0000","last_undegraded":"2024-05-30T17:14:11.810946+0000","last_fullsized":"2024-05-30T17:14:11.810946+0000","mapping_epoch":24,"log_start":"265'500","ondisk_log_start":"265'500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'516","last_scrub_stamp":"2024-05-30T17:14:11.810772+0000","last_deep_scrub":"265'516","last_deep_scrub_stamp":"2024-05-30T17:14:11.810772+0000","last_clean_scrub_stamp":"2024-05-30T17:14:11.810772+0000","objects_scrubbed":7,"log_size":16,"log_dups_size":500,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:11:32.790447+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2760,"num_read_kb":2546,"num_write":661,"num_write_kb":201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2440,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"265'210","reported_seq":854,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:11.793065+0000","last_change":"2024-05-30T17:14:11.793335+0000","last_active":"2024-05-30T17:14:11.793065+0000","last_peered":"2024-05-30T17:14:11.793065+0000","last_clean":"2024-05-30T17:14:11.793065+0000","last_became_active":"2024-05-30T15:54:16.685969+0000","last_became_peered":"2024-05-30T15:54:16.685969+0000","last_unstale":"2024-05-30T17:14:11.793065+0000","last_undegraded":"2024-05-30T17:14:11.793065+0000","last_fullsized":"2024-05-30T17:14:11.793065+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'210","last_scrub_stamp":"2024-05-30T17:14:11.792805+0000","last_deep_scrub":"265'210","last_deep_scrub_stamp":"2024-05-30T17:14:11.792805+0000","last_clean_scrub_stamp":"2024-05-30T17:14:11.792805+0000","objects_scrubbed":5,"log_size":110,"log_dups_size":100,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:18:58.482461+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1342159,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":419,"num_read_kb":18996,"num_write":1148,"num_write_kb":34757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"265'671","reported_seq":2437,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:18.831077+0000","last_change":"2024-05-30T17:14:18.831145+0000","last_active":"2024-05-30T17:14:18.831077+0000","last_peered":"2024-05-30T17:14:18.831077+0000","last_clean":"2024-05-30T17:14:18.831077+0000","last_became_active":"2024-05-30T15:54:18.645400+0000","last_became_peered":"2024-05-30T15:54:18.645400+0000","last_unstale":"2024-05-30T17:14:18.831077+0000","last_undegraded":"2024-05-30T17:14:18.831077+0000","last_fullsized":"2024-05-30T17:14:18.831077+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'671","last_scrub_stamp":"2024-05-30T17:14:18.830880+0000","last_deep_scrub":"265'671","last_deep_scrub_stamp":"2024-05-30T17:14:18.830880+0000","last_clean_scrub_stamp":"2024-05-30T17:14:18.830880+0000","objects_scrubbed":1,"log_size":71,"log_dups_size":600,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:12:16.729926+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3447,"num_read_kb":3120,"num_write":968,"num_write_kb":434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"265'215","reported_seq":858,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:21.805383+0000","last_change":"2024-05-30T17:14:21.805542+0000","last_active":"2024-05-30T17:14:21.805383+0000","last_peered":"2024-05-30T17:14:21.805383+0000","last_clean":"2024-05-30T17:14:21.805383+0000","last_became_active":"2024-05-30T15:54:16.663255+0000","last_became_peered":"2024-05-30T15:54:16.663255+0000","last_unstale":"2024-05-30T17:14:21.805383+0000","last_undegraded":"2024-05-30T17:14:21.805383+0000","last_fullsized":"2024-05-30T17:14:21.805383+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'215","last_scrub_stamp":"2024-05-30T17:14:21.805146+0000","last_deep_scrub":"265'215","last_deep_scrub_stamp":"2024-05-30T17:14:21.805146+0000","last_clean_scrub_stamp":"2024-05-30T17:14:21.805146+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":200,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:43:32.999785+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767340,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":429,"num_read_kb":27123,"num_write":1161,"num_write_kb":19484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"265'365","reported_seq":1793,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:16.421825+0000","last_change":"2024-05-30T17:14:16.421904+0000","last_active":"2024-05-30T17:14:16.421825+0000","last_peered":"2024-05-30T17:14:16.421825+0000","last_clean":"2024-05-30T17:14:16.421825+0000","last_became_active":"2024-05-30T15:54:18.658230+0000","last_became_peered":"2024-05-30T15:54:18.658230+0000","last_unstale":"2024-05-30T17:14:16.421825+0000","last_undegraded":"2024-05-30T17:14:16.421825+0000","last_fullsized":"2024-05-30T17:14:16.421825+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'365","last_scrub_stamp":"2024-05-30T17:14:16.421610+0000","last_deep_scrub":"265'365","last_deep_scrub_stamp":"2024-05-30T17:14:16.421610+0000","last_clean_scrub_stamp":"2024-05-30T17:14:16.421610+0000","objects_scrubbed":5,"log_size":65,"log_dups_size":300,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:52:59.746921+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1934,"num_read_kb":1839,"num_write":414,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":472,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"265'170","reported_seq":792,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:00.278603+0000","last_change":"2024-05-30T17:14:00.278671+0000","last_active":"2024-05-30T17:14:00.278603+0000","last_peered":"2024-05-30T17:14:00.278603+0000","last_clean":"2024-05-30T17:14:00.278603+0000","last_became_active":"2024-05-30T15:54:16.654365+0000","last_became_peered":"2024-05-30T15:54:16.654365+0000","last_unstale":"2024-05-30T17:14:00.278603+0000","last_undegraded":"2024-05-30T17:14:00.278603+0000","last_fullsized":"2024-05-30T17:14:00.278603+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'170","last_scrub_stamp":"2024-05-30T17:14:00.278386+0000","last_deep_scrub":"265'170","last_deep_scrub_stamp":"2024-05-30T17:14:00.278386+0000","last_clean_scrub_stamp":"2024-05-30T17:14:00.278386+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":100,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:04:58.427288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":358,"num_read_kb":31154,"num_write":915,"num_write_kb":41557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"265'704","reported_seq":2143,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:10.773727+0000","last_change":"2024-05-30T17:14:10.773805+0000","last_active":"2024-05-30T17:14:10.773727+0000","last_peered":"2024-05-30T17:14:10.773727+0000","last_clean":"2024-05-30T17:14:10.773727+0000","last_became_active":"2024-05-30T15:54:18.660543+0000","last_became_peered":"2024-05-30T15:54:18.660543+0000","last_unstale":"2024-05-30T17:14:10.773727+0000","last_undegraded":"2024-05-30T17:14:10.773727+0000","last_fullsized":"2024-05-30T17:14:10.773727+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'704","last_scrub_stamp":"2024-05-30T17:14:10.773554+0000","last_deep_scrub":"265'704","last_deep_scrub_stamp":"2024-05-30T17:14:10.773554+0000","last_clean_scrub_stamp":"2024-05-30T17:14:10.773554+0000","objects_scrubbed":6,"log_size":104,"log_dups_size":600,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:36:12.916424+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3130,"num_read_kb":2812,"num_write":919,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"265'202","reported_seq":853,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:24.321602+0000","last_change":"2024-05-30T17:14:24.321759+0000","last_active":"2024-05-30T17:14:24.321602+0000","last_peered":"2024-05-30T17:14:24.321602+0000","last_clean":"2024-05-30T17:14:24.321602+0000","last_became_active":"2024-05-30T15:54:16.661983+0000","last_became_peered":"2024-05-30T15:54:16.661983+0000","last_unstale":"2024-05-30T17:14:24.321602+0000","last_undegraded":"2024-05-30T17:14:24.321602+0000","last_fullsized":"2024-05-30T17:14:24.321602+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'202","last_scrub_stamp":"2024-05-30T17:14:24.321397+0000","last_deep_scrub":"265'202","last_deep_scrub_stamp":"2024-05-30T17:14:24.321397+0000","last_clean_scrub_stamp":"2024-05-30T17:14:24.321397+0000","objects_scrubbed":7,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:26:06.878260+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718827,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":420,"num_read_kb":7620,"num_write":1155,"num_write_kb":9531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"265'154","reported_seq":793,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.164212+0000","last_change":"2024-05-30T17:13:45.164368+0000","last_active":"2024-05-30T17:13:45.164212+0000","last_peered":"2024-05-30T17:13:45.164212+0000","last_clean":"2024-05-30T17:13:45.164212+0000","last_became_active":"2024-05-30T15:55:54.767665+0000","last_became_peered":"2024-05-30T15:55:54.767665+0000","last_unstale":"2024-05-30T17:13:45.164212+0000","last_undegraded":"2024-05-30T17:13:45.164212+0000","last_fullsized":"2024-05-30T17:13:45.164212+0000","mapping_epoch":37,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'154","last_scrub_stamp":"2024-05-30T17:13:45.163967+0000","last_deep_scrub":"265'154","last_deep_scrub_stamp":"2024-05-30T17:13:45.163967+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.163967+0000","objects_scrubbed":2,"log_size":54,"log_dups_size":100,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:10:48.513574+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5682939,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":360,"num_read_kb":14674,"num_write":890,"num_write_kb":17371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,0],"acting":[5,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"265'771","reported_seq":2435,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:11.854742+0000","last_change":"2024-05-30T17:14:11.854742+0000","last_active":"2024-05-30T17:14:11.854742+0000","last_peered":"2024-05-30T17:14:11.854742+0000","last_clean":"2024-05-30T17:14:11.854742+0000","last_became_active":"2024-05-30T15:54:18.660501+0000","last_became_peered":"2024-05-30T15:54:18.660501+0000","last_unstale":"2024-05-30T17:14:11.854742+0000","last_undegraded":"2024-05-30T17:14:11.854742+0000","last_fullsized":"2024-05-30T17:14:11.854742+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'771","last_scrub_stamp":"2024-05-30T17:14:11.854443+0000","last_deep_scrub":"265'771","last_deep_scrub_stamp":"2024-05-30T17:14:11.854443+0000","last_clean_scrub_stamp":"2024-05-30T17:14:11.854443+0000","objects_scrubbed":6,"log_size":71,"log_dups_size":700,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:04:01.491788+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3837,"num_read_kb":3483,"num_write":1021,"num_write_kb":383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2811,"num_omap_keys":11,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"265'183","reported_seq":843,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:17.991730+0000","last_change":"2024-05-30T17:14:17.991836+0000","last_active":"2024-05-30T17:14:17.991730+0000","last_peered":"2024-05-30T17:14:17.991730+0000","last_clean":"2024-05-30T17:14:17.991730+0000","last_became_active":"2024-05-30T15:54:16.668178+0000","last_became_peered":"2024-05-30T15:54:16.668178+0000","last_unstale":"2024-05-30T17:14:17.991730+0000","last_undegraded":"2024-05-30T17:14:17.991730+0000","last_fullsized":"2024-05-30T17:14:17.991730+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'183","last_scrub_stamp":"2024-05-30T17:14:17.991533+0000","last_deep_scrub":"265'183","last_deep_scrub_stamp":"2024-05-30T17:14:17.991533+0000","last_clean_scrub_stamp":"2024-05-30T17:14:17.991533+0000","objects_scrubbed":6,"log_size":83,"log_dups_size":100,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:17:31.041725+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3976850,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":427,"num_read_kb":57498,"num_write":1000,"num_write_kb":21705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"265'477","reported_seq":2157,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:09.363466+0000","last_change":"2024-05-30T17:14:09.363603+0000","last_active":"2024-05-30T17:14:09.363466+0000","last_peered":"2024-05-30T17:14:09.363466+0000","last_clean":"2024-05-30T17:14:09.363466+0000","last_became_active":"2024-05-30T15:54:18.655872+0000","last_became_peered":"2024-05-30T15:54:18.655872+0000","last_unstale":"2024-05-30T17:14:09.363466+0000","last_undegraded":"2024-05-30T17:14:09.363466+0000","last_fullsized":"2024-05-30T17:14:09.363466+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'477","last_scrub_stamp":"2024-05-30T17:14:09.363234+0000","last_deep_scrub":"265'477","last_deep_scrub_stamp":"2024-05-30T17:14:09.363234+0000","last_clean_scrub_stamp":"2024-05-30T17:14:09.363234+0000","objects_scrubbed":3,"log_size":77,"log_dups_size":400,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:01:10.979098+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2571,"num_read_kb":2440,"num_write":587,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":497,"num_omap_keys":2,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5999,"num_read_kb":4309,"num_write":2135,"num_write_kb":880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6954,"num_omap_keys":14,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1142,"ondisk_log_size":1142,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19930,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":13226,"num_read_kb":13032,"num_write":15987,"num_write_kb":4690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4334,"num_omap_keys":26,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":704512,"data_stored":39860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1671,"ondisk_log_size":1671,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":331,"ondisk_log_size":331,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":279892,"num_objects":197,"num_object_clones":0,"num_object_copies":394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":197,"num_whiteouts":0,"num_read":9562,"num_read_kb":11753,"num_write":6685,"num_write_kb":2848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":664178,"num_omap_keys":1530,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1908,"ondisk_log_size":1908,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1634,"num_read_kb":1946,"num_write":20,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":242,"num_object_clones":0,"num_object_copies":484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":242,"num_whiteouts":0,"num_read":191181,"num_read_kb":174247,"num_write":51900,"num_write_kb":17923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":242,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":43877,"num_omap_keys":181,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4157,"ondisk_log_size":4157,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1128,"num_omap_keys":12,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":157354006,"num_objects":261,"num_object_clones":0,"num_object_copies":783,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":261,"num_whiteouts":0,"num_read":32072,"num_read_kb":2124808,"num_write":69910,"num_write_kb":2069422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":238202880,"data_stored":238202880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4424,"ondisk_log_size":4424,"up":192,"acting":192,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":135,"num_read_kb":182367,"num_write":151,"num_write_kb":122722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":133,"num_read_kb":181098,"num_write":151,"num_write_kb":122722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575823,"num_pgs":106,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300188,"kb_used_data":38548,"kb_used_omap":1,"kb_used_meta":261630,"kb_avail":94071652,"statfs":{"total":96636764160,"available":96329371648,"internally_reserved":0,"allocated":39473152,"data_stored":37452492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":267909578},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575822,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":289728,"kb_used_data":37816,"kb_used_omap":1,"kb_used_meta":251902,"kb_avail":94082112,"statfs":{"total":96636764160,"available":96340082688,"internally_reserved":0,"allocated":38723584,"data_stored":36776739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":257948102},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575822,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137976,"kb_used_data":48496,"kb_used_omap":1,"kb_used_meta":89470,"kb_avail":94233864,"statfs":{"total":96636764160,"available":96495476736,"internally_reserved":0,"allocated":49659904,"data_stored":47630081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":91617733},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575822,"num_pgs":104,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":125812,"kb_used_data":44652,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94246028,"statfs":{"total":96636764160,"available":96507932672,"internally_reserved":0,"allocated":45723648,"data_stored":43786666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":83098054},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575820,"num_pgs":124,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":269128,"kb_used_data":43584,"kb_used_omap":1,"kb_used_meta":225534,"kb_avail":94102712,"statfs":{"total":96636764160,"available":96361177088,"internally_reserved":0,"allocated":44630016,"data_stored":42616238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":230947271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575820,"num_pgs":91,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":107084,"kb_used_data":35652,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94264756,"statfs":{"total":96636764160,"available":96527110144,"internally_reserved":0,"allocated":36507648,"data_stored":34568865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575825,"num_pgs":112,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":164064,"kb_used_data":46872,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94207776,"statfs":{"total":96636764160,"available":96468762624,"internally_reserved":0,"allocated":47996928,"data_stored":46030586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575819,"num_pgs":109,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266824,"kb_used_data":37120,"kb_used_omap":1,"kb_used_meta":229694,"kb_avail":94105016,"statfs":{"total":96636764160,"available":96363536384,"internally_reserved":0,"allocated":38010880,"data_stored":36012935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":235207109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":27881472,"data_stored":27881472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":37441536,"data_stored":37441536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27029504,"data_stored":27029504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24588288,"data_stored":24588288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":31006720,"data_stored":31006720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":33796096,"data_stored":33796096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":32382976,"data_stored":32382976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24076288,"data_stored":24076288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":4787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":5121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":4696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":6691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":5459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":3918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":7193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T17:14:47.632 INFO:tasks.ceph:pgid 68.1e last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:47.633 INFO:tasks.ceph:pgid 68.1d last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:47.633 INFO:tasks.ceph:pgid 68.16 last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:47.634 INFO:tasks.ceph:pgid 68.e last_scrub_stamp 2024-05-30T16:34:54.830789+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=34, tm_sec=54, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=13, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:14:47.637 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-30T17:15:07.638 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T17:15:07.998 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-05-30T17:15:08.010 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-05-30T17:15:08.019 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":2689,"stamp":"2024-05-30T17:15:06.503062+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":191800421,"num_objects":837,"num_object_clones":0,"num_object_copies":1935,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":837,"num_whiteouts":0,"num_read":254020,"num_read_kb":2693627,"num_write":147058,"num_write_kb":2342860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":289,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":720471,"num_omap_keys":1763,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13934,"ondisk_log_size":13934,"up":850,"acting":850,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":850,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1660804,"kb_used_data":332740,"kb_used_omap":12,"kb_used_meta":1327987,"kb_avail":753313916,"statfs":{"total":773094113280,"available":771393449984,"internally_reserved":0,"allocated":340725760,"data_stored":324874602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":1359859249},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008213"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":37,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:31.432480+0000","last_change":"2024-05-30T17:14:31.432480+0000","last_active":"2024-05-30T17:14:31.432480+0000","last_peered":"2024-05-30T17:14:31.432480+0000","last_clean":"2024-05-30T17:14:31.432480+0000","last_became_active":"2024-05-30T16:34:55.869226+0000","last_became_peered":"2024-05-30T16:34:55.869226+0000","last_unstale":"2024-05-30T17:14:31.432480+0000","last_undegraded":"2024-05-30T17:14:31.432480+0000","last_fullsized":"2024-05-30T17:14:31.432480+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:31.432202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:31.432202+0000","last_clean_scrub_stamp":"2024-05-30T17:14:31.432202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:29:56.470555+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"265'2","reported_seq":57,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:29.402904+0000","last_change":"2024-05-30T17:14:29.402967+0000","last_active":"2024-05-30T17:14:29.402904+0000","last_peered":"2024-05-30T17:14:29.402904+0000","last_clean":"2024-05-30T17:14:29.402904+0000","last_became_active":"2024-05-30T16:17:35.248740+0000","last_became_peered":"2024-05-30T16:17:35.248740+0000","last_unstale":"2024-05-30T17:14:29.402904+0000","last_undegraded":"2024-05-30T17:14:29.402904+0000","last_fullsized":"2024-05-30T17:14:29.402904+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:14:29.402717+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:14:29.402717+0000","last_clean_scrub_stamp":"2024-05-30T17:14:29.402717+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:19:16.475399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"265'8","reported_seq":76,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.443594+0000","last_change":"2024-05-30T17:13:43.443594+0000","last_active":"2024-05-30T17:13:43.443594+0000","last_peered":"2024-05-30T17:13:43.443594+0000","last_clean":"2024-05-30T17:13:43.443594+0000","last_became_active":"2024-05-30T16:17:31.183349+0000","last_became_peered":"2024-05-30T16:17:31.183349+0000","last_unstale":"2024-05-30T17:13:43.443594+0000","last_undegraded":"2024-05-30T17:13:43.443594+0000","last_fullsized":"2024-05-30T17:13:43.443594+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'8","last_scrub_stamp":"2024-05-30T17:13:43.443352+0000","last_deep_scrub":"265'8","last_deep_scrub_stamp":"2024-05-30T17:13:43.443352+0000","last_clean_scrub_stamp":"2024-05-30T17:13:43.443352+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:57:10.662914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":16384,"num_write":8,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"265'45","reported_seq":193,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.684557+0000","last_change":"2024-05-30T17:13:48.684645+0000","last_active":"2024-05-30T17:13:48.684557+0000","last_peered":"2024-05-30T17:13:48.684557+0000","last_clean":"2024-05-30T17:13:48.684557+0000","last_became_active":"2024-05-30T16:12:50.803352+0000","last_became_peered":"2024-05-30T16:12:50.803352+0000","last_unstale":"2024-05-30T17:13:48.684557+0000","last_undegraded":"2024-05-30T17:13:48.684557+0000","last_fullsized":"2024-05-30T17:13:48.684557+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'45","last_scrub_stamp":"2024-05-30T17:13:48.684344+0000","last_deep_scrub":"265'45","last_deep_scrub_stamp":"2024-05-30T17:13:48.684344+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.684344+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:56:59.286905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":225,"num_read_kb":156,"num_write":101,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":37,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:14.491199+0000","last_change":"2024-05-30T17:14:14.491199+0000","last_active":"2024-05-30T17:14:14.491199+0000","last_peered":"2024-05-30T17:14:14.491199+0000","last_clean":"2024-05-30T17:14:14.491199+0000","last_became_active":"2024-05-30T16:34:55.886731+0000","last_became_peered":"2024-05-30T16:34:55.886731+0000","last_unstale":"2024-05-30T17:14:14.491199+0000","last_undegraded":"2024-05-30T17:14:14.491199+0000","last_fullsized":"2024-05-30T17:14:14.491199+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:14.490948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:14.490948+0000","last_clean_scrub_stamp":"2024-05-30T17:14:14.490948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:54:47.154526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"0'0","reported_seq":53,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:30.002191+0000","last_change":"2024-05-30T17:14:30.002246+0000","last_active":"2024-05-30T17:14:30.002191+0000","last_peered":"2024-05-30T17:14:30.002191+0000","last_clean":"2024-05-30T17:14:30.002191+0000","last_became_active":"2024-05-30T16:17:35.247810+0000","last_became_peered":"2024-05-30T16:17:35.247810+0000","last_unstale":"2024-05-30T17:14:30.002191+0000","last_undegraded":"2024-05-30T17:14:30.002191+0000","last_fullsized":"2024-05-30T17:14:30.002191+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:30.001987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:30.001987+0000","last_clean_scrub_stamp":"2024-05-30T17:14:30.001987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:47:05.599792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"0'0","reported_seq":60,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.953152+0000","last_change":"2024-05-30T17:13:45.953152+0000","last_active":"2024-05-30T17:13:45.953152+0000","last_peered":"2024-05-30T17:13:45.953152+0000","last_clean":"2024-05-30T17:13:45.953152+0000","last_became_active":"2024-05-30T16:17:31.212337+0000","last_became_peered":"2024-05-30T16:17:31.212337+0000","last_unstale":"2024-05-30T17:13:45.953152+0000","last_undegraded":"2024-05-30T17:13:45.953152+0000","last_fullsized":"2024-05-30T17:13:45.953152+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:45.952903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:45.952903+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.952903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:31:16.835383+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"265'55","reported_seq":219,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:55.177804+0000","last_change":"2024-05-30T17:13:55.177870+0000","last_active":"2024-05-30T17:13:55.177804+0000","last_peered":"2024-05-30T17:13:55.177804+0000","last_clean":"2024-05-30T17:13:55.177804+0000","last_became_active":"2024-05-30T16:12:50.810207+0000","last_became_peered":"2024-05-30T16:12:50.810207+0000","last_unstale":"2024-05-30T17:13:55.177804+0000","last_undegraded":"2024-05-30T17:13:55.177804+0000","last_fullsized":"2024-05-30T17:13:55.177804+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'55","last_scrub_stamp":"2024-05-30T17:13:55.177616+0000","last_deep_scrub":"265'55","last_deep_scrub_stamp":"2024-05-30T17:13:55.177616+0000","last_clean_scrub_stamp":"2024-05-30T17:13:55.177616+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:10:07.203727+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":277,"num_read_kb":197,"num_write":99,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":37,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:21.829390+0000","last_change":"2024-05-30T17:14:21.829444+0000","last_active":"2024-05-30T17:14:21.829390+0000","last_peered":"2024-05-30T17:14:21.829390+0000","last_clean":"2024-05-30T17:14:21.829390+0000","last_became_active":"2024-05-30T16:34:55.874359+0000","last_became_peered":"2024-05-30T16:34:55.874359+0000","last_unstale":"2024-05-30T17:14:21.829390+0000","last_undegraded":"2024-05-30T17:14:21.829390+0000","last_fullsized":"2024-05-30T17:14:21.829390+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:21.829201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:21.829201+0000","last_clean_scrub_stamp":"2024-05-30T17:14:21.829201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:41:35.913572+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"0'0","reported_seq":58,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:35.919291+0000","last_change":"2024-05-30T17:14:35.919350+0000","last_active":"2024-05-30T17:14:35.919291+0000","last_peered":"2024-05-30T17:14:35.919291+0000","last_clean":"2024-05-30T17:14:35.919291+0000","last_became_active":"2024-05-30T16:17:35.743231+0000","last_became_peered":"2024-05-30T16:17:35.743231+0000","last_unstale":"2024-05-30T17:14:35.919291+0000","last_undegraded":"2024-05-30T17:14:35.919291+0000","last_fullsized":"2024-05-30T17:14:35.919291+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:35.919084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:35.919084+0000","last_clean_scrub_stamp":"2024-05-30T17:14:35.919084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:01:37.439385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"265'3","reported_seq":70,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:09.442558+0000","last_change":"2024-05-30T17:14:09.442626+0000","last_active":"2024-05-30T17:14:09.442558+0000","last_peered":"2024-05-30T17:14:09.442558+0000","last_clean":"2024-05-30T17:14:09.442558+0000","last_became_active":"2024-05-30T16:17:31.204806+0000","last_became_peered":"2024-05-30T16:17:31.204806+0000","last_unstale":"2024-05-30T17:14:09.442558+0000","last_undegraded":"2024-05-30T17:14:09.442558+0000","last_fullsized":"2024-05-30T17:14:09.442558+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:14:09.442363+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:14:09.442363+0000","last_clean_scrub_stamp":"2024-05-30T17:14:09.442363+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:38:40.295126+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"265'29","reported_seq":143,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.662503+0000","last_change":"2024-05-30T17:13:46.662582+0000","last_active":"2024-05-30T17:13:46.662503+0000","last_peered":"2024-05-30T17:13:46.662503+0000","last_clean":"2024-05-30T17:13:46.662503+0000","last_became_active":"2024-05-30T16:12:50.810062+0000","last_became_peered":"2024-05-30T16:12:50.810062+0000","last_unstale":"2024-05-30T17:13:46.662503+0000","last_undegraded":"2024-05-30T17:13:46.662503+0000","last_fullsized":"2024-05-30T17:13:46.662503+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'29","last_scrub_stamp":"2024-05-30T17:13:46.662271+0000","last_deep_scrub":"265'29","last_deep_scrub_stamp":"2024-05-30T17:13:46.662271+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.662271+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:14:37.786974+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":98,"num_write":65,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":37,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:40.496075+0000","last_change":"2024-05-30T17:14:40.496132+0000","last_active":"2024-05-30T17:14:40.496075+0000","last_peered":"2024-05-30T17:14:40.496075+0000","last_clean":"2024-05-30T17:14:40.496075+0000","last_became_active":"2024-05-30T16:34:55.887532+0000","last_became_peered":"2024-05-30T16:34:55.887532+0000","last_unstale":"2024-05-30T17:14:40.496075+0000","last_undegraded":"2024-05-30T17:14:40.496075+0000","last_fullsized":"2024-05-30T17:14:40.496075+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:40.495878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:40.495878+0000","last_clean_scrub_stamp":"2024-05-30T17:14:40.495878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:51:19.603432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"265'2","reported_seq":63,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:39.485135+0000","last_change":"2024-05-30T17:14:39.485206+0000","last_active":"2024-05-30T17:14:39.485135+0000","last_peered":"2024-05-30T17:14:39.485135+0000","last_clean":"2024-05-30T17:14:39.485135+0000","last_became_active":"2024-05-30T16:17:35.745709+0000","last_became_peered":"2024-05-30T16:17:35.745709+0000","last_unstale":"2024-05-30T17:14:39.485135+0000","last_undegraded":"2024-05-30T17:14:39.485135+0000","last_fullsized":"2024-05-30T17:14:39.485135+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:14:39.484943+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:14:39.484943+0000","last_clean_scrub_stamp":"2024-05-30T17:14:39.484943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:55:17.462949+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1142,"num_write":2,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1f","version":"265'3","reported_seq":67,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.831674+0000","last_change":"2024-05-30T17:13:50.831741+0000","last_active":"2024-05-30T17:13:50.831674+0000","last_peered":"2024-05-30T17:13:50.831674+0000","last_clean":"2024-05-30T17:13:50.831674+0000","last_became_active":"2024-05-30T16:17:31.182344+0000","last_became_peered":"2024-05-30T16:17:31.182344+0000","last_unstale":"2024-05-30T17:13:50.831674+0000","last_undegraded":"2024-05-30T17:13:50.831674+0000","last_fullsized":"2024-05-30T17:13:50.831674+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:13:50.831396+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:13:50.831396+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.831396+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:36:02.576103+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"260'6","reported_seq":85,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.454253+0000","last_change":"2024-05-30T17:13:46.454380+0000","last_active":"2024-05-30T17:13:46.454253+0000","last_peered":"2024-05-30T17:13:46.454253+0000","last_clean":"2024-05-30T17:13:46.454253+0000","last_became_active":"2024-05-30T16:12:50.826974+0000","last_became_peered":"2024-05-30T16:12:50.826974+0000","last_unstale":"2024-05-30T17:13:46.454253+0000","last_undegraded":"2024-05-30T17:13:46.454253+0000","last_fullsized":"2024-05-30T17:13:46.454253+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'6","last_scrub_stamp":"2024-05-30T17:13:46.453944+0000","last_deep_scrub":"260'6","last_deep_scrub_stamp":"2024-05-30T17:13:46.453944+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.453944+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:05:54.138613+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":24,"num_write":11,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":40,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:44.503138+0000","last_change":"2024-05-30T17:14:44.503191+0000","last_active":"2024-05-30T17:14:44.503138+0000","last_peered":"2024-05-30T17:14:44.503138+0000","last_clean":"2024-05-30T17:14:44.503138+0000","last_became_active":"2024-05-30T16:34:55.898864+0000","last_became_peered":"2024-05-30T16:34:55.898864+0000","last_unstale":"2024-05-30T17:14:44.503138+0000","last_undegraded":"2024-05-30T17:14:44.503138+0000","last_fullsized":"2024-05-30T17:14:44.503138+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:44.502949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:44.502949+0000","last_clean_scrub_stamp":"2024-05-30T17:14:44.502949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:26:24.613908+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"265'8","reported_seq":73,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:28.408420+0000","last_change":"2024-05-30T17:14:28.408568+0000","last_active":"2024-05-30T17:14:28.408420+0000","last_peered":"2024-05-30T17:14:28.408420+0000","last_clean":"2024-05-30T17:14:28.408420+0000","last_became_active":"2024-05-30T16:17:35.232744+0000","last_became_peered":"2024-05-30T16:17:35.232744+0000","last_unstale":"2024-05-30T17:14:28.408420+0000","last_undegraded":"2024-05-30T17:14:28.408420+0000","last_fullsized":"2024-05-30T17:14:28.408420+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'8","last_scrub_stamp":"2024-05-30T17:14:28.408191+0000","last_deep_scrub":"265'8","last_deep_scrub_stamp":"2024-05-30T17:14:28.408191+0000","last_clean_scrub_stamp":"2024-05-30T17:14:28.408191+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:24:09.808156+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":23554,"num_write":6,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"0'0","reported_seq":60,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.680121+0000","last_change":"2024-05-30T17:13:43.680163+0000","last_active":"2024-05-30T17:13:43.680121+0000","last_peered":"2024-05-30T17:13:43.680121+0000","last_clean":"2024-05-30T17:13:43.680121+0000","last_became_active":"2024-05-30T16:17:31.186229+0000","last_became_peered":"2024-05-30T16:17:31.186229+0000","last_unstale":"2024-05-30T17:13:43.680121+0000","last_undegraded":"2024-05-30T17:13:43.680121+0000","last_fullsized":"2024-05-30T17:13:43.680121+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:43.679999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:43.679999+0000","last_clean_scrub_stamp":"2024-05-30T17:13:43.679999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:30:30.079637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"265'27","reported_seq":143,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.491839+0000","last_change":"2024-05-30T17:13:45.491914+0000","last_active":"2024-05-30T17:13:45.491839+0000","last_peered":"2024-05-30T17:13:45.491839+0000","last_clean":"2024-05-30T17:13:45.491839+0000","last_became_active":"2024-05-30T16:12:50.827008+0000","last_became_peered":"2024-05-30T16:12:50.827008+0000","last_unstale":"2024-05-30T17:13:45.491839+0000","last_undegraded":"2024-05-30T17:13:45.491839+0000","last_fullsized":"2024-05-30T17:13:45.491839+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'27","last_scrub_stamp":"2024-05-30T17:13:45.491648+0000","last_deep_scrub":"265'27","last_deep_scrub_stamp":"2024-05-30T17:13:45.491648+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.491648+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:27:36.651992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":100,"num_write":57,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":40,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:24.924709+0000","last_change":"2024-05-30T17:14:24.924709+0000","last_active":"2024-05-30T17:14:24.924709+0000","last_peered":"2024-05-30T17:14:24.924709+0000","last_clean":"2024-05-30T17:14:24.924709+0000","last_became_active":"2024-05-30T16:34:55.870139+0000","last_became_peered":"2024-05-30T16:34:55.870139+0000","last_unstale":"2024-05-30T17:14:24.924709+0000","last_undegraded":"2024-05-30T17:14:24.924709+0000","last_fullsized":"2024-05-30T17:14:24.924709+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:24.924431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:24.924431+0000","last_clean_scrub_stamp":"2024-05-30T17:14:24.924431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:15:42.392670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"265'8","reported_seq":67,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:16.849562+0000","last_change":"2024-05-30T17:14:16.849722+0000","last_active":"2024-05-30T17:14:16.849562+0000","last_peered":"2024-05-30T17:14:16.849562+0000","last_clean":"2024-05-30T17:14:16.849562+0000","last_became_active":"2024-05-30T16:17:35.233040+0000","last_became_peered":"2024-05-30T16:17:35.233040+0000","last_unstale":"2024-05-30T17:14:16.849562+0000","last_undegraded":"2024-05-30T17:14:16.849562+0000","last_fullsized":"2024-05-30T17:14:16.849562+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'8","last_scrub_stamp":"2024-05-30T17:14:16.849340+0000","last_deep_scrub":"265'8","last_deep_scrub_stamp":"2024-05-30T17:14:16.849340+0000","last_clean_scrub_stamp":"2024-05-30T17:14:16.849340+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:56:30.841879+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":12336,"num_write":13,"num_write_kb":11312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"265'3","reported_seq":65,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.804589+0000","last_change":"2024-05-30T17:13:46.804678+0000","last_active":"2024-05-30T17:13:46.804589+0000","last_peered":"2024-05-30T17:13:46.804589+0000","last_clean":"2024-05-30T17:13:46.804589+0000","last_became_active":"2024-05-30T16:17:31.173900+0000","last_became_peered":"2024-05-30T16:17:31.173900+0000","last_unstale":"2024-05-30T17:13:46.804589+0000","last_undegraded":"2024-05-30T17:13:46.804589+0000","last_fullsized":"2024-05-30T17:13:46.804589+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:13:46.804390+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:13:46.804390+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.804390+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:37:20.477514+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"265'75","reported_seq":275,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.777415+0000","last_change":"2024-05-30T17:13:49.777540+0000","last_active":"2024-05-30T17:13:49.777415+0000","last_peered":"2024-05-30T17:13:49.777415+0000","last_clean":"2024-05-30T17:13:49.777415+0000","last_became_active":"2024-05-30T16:12:50.815032+0000","last_became_peered":"2024-05-30T16:12:50.815032+0000","last_unstale":"2024-05-30T17:13:49.777415+0000","last_undegraded":"2024-05-30T17:13:49.777415+0000","last_fullsized":"2024-05-30T17:13:49.777415+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'75","last_scrub_stamp":"2024-05-30T17:13:49.777169+0000","last_deep_scrub":"265'75","last_deep_scrub_stamp":"2024-05-30T17:13:49.777169+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.777169+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:42:13.290083+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":396,"num_read_kb":277,"num_write":137,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"265'2","reported_seq":39,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:36.716564+0000","last_change":"2024-05-30T17:14:36.716564+0000","last_active":"2024-05-30T17:14:36.716564+0000","last_peered":"2024-05-30T17:14:36.716564+0000","last_clean":"2024-05-30T17:14:36.716564+0000","last_became_active":"2024-05-30T16:34:55.881414+0000","last_became_peered":"2024-05-30T16:34:55.881414+0000","last_unstale":"2024-05-30T17:14:36.716564+0000","last_undegraded":"2024-05-30T17:14:36.716564+0000","last_fullsized":"2024-05-30T17:14:36.716564+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:14:36.716234+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:14:36.716234+0000","last_clean_scrub_stamp":"2024-05-30T17:14:36.716234+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:25:24.591710+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"262'2","reported_seq":57,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:17.859737+0000","last_change":"2024-05-30T17:14:17.859865+0000","last_active":"2024-05-30T17:14:17.859737+0000","last_peered":"2024-05-30T17:14:17.859737+0000","last_clean":"2024-05-30T17:14:17.859737+0000","last_became_active":"2024-05-30T16:17:35.237424+0000","last_became_peered":"2024-05-30T16:17:35.237424+0000","last_unstale":"2024-05-30T17:14:17.859737+0000","last_undegraded":"2024-05-30T17:14:17.859737+0000","last_fullsized":"2024-05-30T17:14:17.859737+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'2","last_scrub_stamp":"2024-05-30T17:14:17.859457+0000","last_deep_scrub":"262'2","last_deep_scrub_stamp":"2024-05-30T17:14:17.859457+0000","last_clean_scrub_stamp":"2024-05-30T17:14:17.859457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:38:38.589711+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"0'0","reported_seq":60,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:26.056112+0000","last_change":"2024-05-30T17:14:26.056112+0000","last_active":"2024-05-30T17:14:26.056112+0000","last_peered":"2024-05-30T17:14:26.056112+0000","last_clean":"2024-05-30T17:14:26.056112+0000","last_became_active":"2024-05-30T16:17:31.192269+0000","last_became_peered":"2024-05-30T16:17:31.192269+0000","last_unstale":"2024-05-30T17:14:26.056112+0000","last_undegraded":"2024-05-30T17:14:26.056112+0000","last_fullsized":"2024-05-30T17:14:26.056112+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:26.055851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:26.055851+0000","last_clean_scrub_stamp":"2024-05-30T17:14:26.055851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:16:52.090387+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"265'25","reported_seq":134,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.688810+0000","last_change":"2024-05-30T17:13:45.688906+0000","last_active":"2024-05-30T17:13:45.688810+0000","last_peered":"2024-05-30T17:13:45.688810+0000","last_clean":"2024-05-30T17:13:45.688810+0000","last_became_active":"2024-05-30T16:12:50.811732+0000","last_became_peered":"2024-05-30T16:12:50.811732+0000","last_unstale":"2024-05-30T17:13:45.688810+0000","last_undegraded":"2024-05-30T17:13:45.688810+0000","last_fullsized":"2024-05-30T17:13:45.688810+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'25","last_scrub_stamp":"2024-05-30T17:13:45.688602+0000","last_deep_scrub":"265'25","last_deep_scrub_stamp":"2024-05-30T17:13:45.688602+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.688602+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:20:25.477039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":85,"num_write":53,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":37,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:41.714497+0000","last_change":"2024-05-30T17:14:41.714497+0000","last_active":"2024-05-30T17:14:41.714497+0000","last_peered":"2024-05-30T17:14:41.714497+0000","last_clean":"2024-05-30T17:14:41.714497+0000","last_became_active":"2024-05-30T16:34:55.899241+0000","last_became_peered":"2024-05-30T16:34:55.899241+0000","last_unstale":"2024-05-30T17:14:41.714497+0000","last_undegraded":"2024-05-30T17:14:41.714497+0000","last_fullsized":"2024-05-30T17:14:41.714497+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:41.714213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:41.714213+0000","last_clean_scrub_stamp":"2024-05-30T17:14:41.714213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:21:59.944045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"0'0","reported_seq":53,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:55.376659+0000","last_change":"2024-05-30T17:13:55.376721+0000","last_active":"2024-05-30T17:13:55.376659+0000","last_peered":"2024-05-30T17:13:55.376659+0000","last_clean":"2024-05-30T17:13:55.376659+0000","last_became_active":"2024-05-30T16:17:35.229034+0000","last_became_peered":"2024-05-30T16:17:35.229034+0000","last_unstale":"2024-05-30T17:13:55.376659+0000","last_undegraded":"2024-05-30T17:13:55.376659+0000","last_fullsized":"2024-05-30T17:13:55.376659+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:55.376450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:55.376450+0000","last_clean_scrub_stamp":"2024-05-30T17:13:55.376450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:44:03.105415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"265'9","reported_seq":83,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.701025+0000","last_change":"2024-05-30T17:13:47.701025+0000","last_active":"2024-05-30T17:13:47.701025+0000","last_peered":"2024-05-30T17:13:47.701025+0000","last_clean":"2024-05-30T17:13:47.701025+0000","last_became_active":"2024-05-30T16:17:31.190742+0000","last_became_peered":"2024-05-30T16:17:31.190742+0000","last_unstale":"2024-05-30T17:13:47.701025+0000","last_undegraded":"2024-05-30T17:13:47.701025+0000","last_fullsized":"2024-05-30T17:13:47.701025+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'9","last_scrub_stamp":"2024-05-30T17:13:47.700729+0000","last_deep_scrub":"265'9","last_deep_scrub_stamp":"2024-05-30T17:13:47.700729+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.700729+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:50:23.027430+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":30722,"num_write":17,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"265'33","reported_seq":152,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.992243+0000","last_change":"2024-05-30T17:13:44.992374+0000","last_active":"2024-05-30T17:13:44.992243+0000","last_peered":"2024-05-30T17:13:44.992243+0000","last_clean":"2024-05-30T17:13:44.992243+0000","last_became_active":"2024-05-30T16:12:50.820343+0000","last_became_peered":"2024-05-30T16:12:50.820343+0000","last_unstale":"2024-05-30T17:13:44.992243+0000","last_undegraded":"2024-05-30T17:13:44.992243+0000","last_fullsized":"2024-05-30T17:13:44.992243+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'33","last_scrub_stamp":"2024-05-30T17:13:44.992004+0000","last_deep_scrub":"265'33","last_deep_scrub_stamp":"2024-05-30T17:13:44.992004+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.992004+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:18:13.774581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":154,"num_read_kb":114,"num_write":68,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":38,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:26.916385+0000","last_change":"2024-05-30T17:14:26.916385+0000","last_active":"2024-05-30T17:14:26.916385+0000","last_peered":"2024-05-30T17:14:26.916385+0000","last_clean":"2024-05-30T17:14:26.916385+0000","last_became_active":"2024-05-30T16:34:55.888238+0000","last_became_peered":"2024-05-30T16:34:55.888238+0000","last_unstale":"2024-05-30T17:14:26.916385+0000","last_undegraded":"2024-05-30T17:14:26.916385+0000","last_fullsized":"2024-05-30T17:14:26.916385+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:26.916120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:26.916120+0000","last_clean_scrub_stamp":"2024-05-30T17:14:26.916120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:41:44.053419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"262'6","reported_seq":65,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:11.410160+0000","last_change":"2024-05-30T17:14:11.410224+0000","last_active":"2024-05-30T17:14:11.410160+0000","last_peered":"2024-05-30T17:14:11.410160+0000","last_clean":"2024-05-30T17:14:11.410160+0000","last_became_active":"2024-05-30T16:17:35.738420+0000","last_became_peered":"2024-05-30T16:17:35.738420+0000","last_unstale":"2024-05-30T17:14:11.410160+0000","last_undegraded":"2024-05-30T17:14:11.410160+0000","last_fullsized":"2024-05-30T17:14:11.410160+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'6","last_scrub_stamp":"2024-05-30T17:14:11.409961+0000","last_deep_scrub":"262'6","last_deep_scrub_stamp":"2024-05-30T17:14:11.409961+0000","last_clean_scrub_stamp":"2024-05-30T17:14:11.409961+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:59:36.378123+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6144,"num_write":16,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"265'5","reported_seq":70,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.196417+0000","last_change":"2024-05-30T17:13:56.196484+0000","last_active":"2024-05-30T17:13:56.196417+0000","last_peered":"2024-05-30T17:13:56.196417+0000","last_clean":"2024-05-30T17:13:56.196417+0000","last_became_active":"2024-05-30T16:17:31.195031+0000","last_became_peered":"2024-05-30T16:17:31.195031+0000","last_unstale":"2024-05-30T17:13:56.196417+0000","last_undegraded":"2024-05-30T17:13:56.196417+0000","last_fullsized":"2024-05-30T17:13:56.196417+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'5","last_scrub_stamp":"2024-05-30T17:13:56.196222+0000","last_deep_scrub":"265'5","last_deep_scrub_stamp":"2024-05-30T17:13:56.196222+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.196222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:04:32.713858+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6715,"num_write":5,"num_write_kb":6715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"265'65","reported_seq":246,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:42.646313+0000","last_change":"2024-05-30T17:13:42.646346+0000","last_active":"2024-05-30T17:13:42.646313+0000","last_peered":"2024-05-30T17:13:42.646313+0000","last_clean":"2024-05-30T17:13:42.646313+0000","last_became_active":"2024-05-30T16:12:50.807771+0000","last_became_peered":"2024-05-30T16:12:50.807771+0000","last_unstale":"2024-05-30T17:13:42.646313+0000","last_undegraded":"2024-05-30T17:13:42.646313+0000","last_fullsized":"2024-05-30T17:13:42.646313+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'65","last_scrub_stamp":"2024-05-30T17:13:42.646208+0000","last_deep_scrub":"265'65","last_deep_scrub_stamp":"2024-05-30T17:13:42.646208+0000","last_clean_scrub_stamp":"2024-05-30T17:13:42.646208+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:19:09.986519+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":343,"num_read_kb":242,"num_write":116,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"265'2","reported_seq":40,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:13.464224+0000","last_change":"2024-05-30T17:14:13.464224+0000","last_active":"2024-05-30T17:14:13.464224+0000","last_peered":"2024-05-30T17:14:13.464224+0000","last_clean":"2024-05-30T17:14:13.464224+0000","last_became_active":"2024-05-30T16:34:55.872581+0000","last_became_peered":"2024-05-30T16:34:55.872581+0000","last_unstale":"2024-05-30T17:14:13.464224+0000","last_undegraded":"2024-05-30T17:14:13.464224+0000","last_fullsized":"2024-05-30T17:14:13.464224+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:14:13.463958+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:14:13.463958+0000","last_clean_scrub_stamp":"2024-05-30T17:14:13.463958+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:33:38.750355+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"0'0","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:07.429700+0000","last_change":"2024-05-30T17:14:07.429850+0000","last_active":"2024-05-30T17:14:07.429700+0000","last_peered":"2024-05-30T17:14:07.429700+0000","last_clean":"2024-05-30T17:14:07.429700+0000","last_became_active":"2024-05-30T16:17:35.738397+0000","last_became_peered":"2024-05-30T16:17:35.738397+0000","last_unstale":"2024-05-30T17:14:07.429700+0000","last_undegraded":"2024-05-30T17:14:07.429700+0000","last_fullsized":"2024-05-30T17:14:07.429700+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:07.429474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:07.429474+0000","last_clean_scrub_stamp":"2024-05-30T17:14:07.429474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:01:13.797785+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"265'3","reported_seq":65,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.455165+0000","last_change":"2024-05-30T17:13:49.455225+0000","last_active":"2024-05-30T17:13:49.455165+0000","last_peered":"2024-05-30T17:13:49.455165+0000","last_clean":"2024-05-30T17:13:49.455165+0000","last_became_active":"2024-05-30T16:17:31.196345+0000","last_became_peered":"2024-05-30T16:17:31.196345+0000","last_unstale":"2024-05-30T17:13:49.455165+0000","last_undegraded":"2024-05-30T17:13:49.455165+0000","last_fullsized":"2024-05-30T17:13:49.455165+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:13:49.454958+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:13:49.454958+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.454958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:19:39.781936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"265'38","reported_seq":168,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.437795+0000","last_change":"2024-05-30T17:13:49.437862+0000","last_active":"2024-05-30T17:13:49.437795+0000","last_peered":"2024-05-30T17:13:49.437795+0000","last_clean":"2024-05-30T17:13:49.437795+0000","last_became_active":"2024-05-30T16:12:50.826729+0000","last_became_peered":"2024-05-30T16:12:50.826729+0000","last_unstale":"2024-05-30T17:13:49.437795+0000","last_undegraded":"2024-05-30T17:13:49.437795+0000","last_fullsized":"2024-05-30T17:13:49.437795+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'38","last_scrub_stamp":"2024-05-30T17:13:49.437536+0000","last_deep_scrub":"265'38","last_deep_scrub_stamp":"2024-05-30T17:13:49.437536+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.437536+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:58:24.201516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":187,"num_read_kb":132,"num_write":81,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":38,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:04.725681+0000","last_change":"2024-05-30T17:14:04.725681+0000","last_active":"2024-05-30T17:14:04.725681+0000","last_peered":"2024-05-30T17:14:04.725681+0000","last_clean":"2024-05-30T17:14:04.725681+0000","last_became_active":"2024-05-30T16:34:55.875788+0000","last_became_peered":"2024-05-30T16:34:55.875788+0000","last_unstale":"2024-05-30T17:14:04.725681+0000","last_undegraded":"2024-05-30T17:14:04.725681+0000","last_fullsized":"2024-05-30T17:14:04.725681+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:04.725336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:04.725336+0000","last_clean_scrub_stamp":"2024-05-30T17:14:04.725336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:32:51.275635+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"0'0","reported_seq":54,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:38.529799+0000","last_change":"2024-05-30T17:14:38.529904+0000","last_active":"2024-05-30T17:14:38.529799+0000","last_peered":"2024-05-30T17:14:38.529799+0000","last_clean":"2024-05-30T17:14:38.529799+0000","last_became_active":"2024-05-30T16:17:35.745769+0000","last_became_peered":"2024-05-30T16:17:35.745769+0000","last_unstale":"2024-05-30T17:14:38.529799+0000","last_undegraded":"2024-05-30T17:14:38.529799+0000","last_fullsized":"2024-05-30T17:14:38.529799+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:38.529559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:38.529559+0000","last_clean_scrub_stamp":"2024-05-30T17:14:38.529559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:21:37.195453+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"265'2","reported_seq":63,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:10.398735+0000","last_change":"2024-05-30T17:14:10.398878+0000","last_active":"2024-05-30T17:14:10.398735+0000","last_peered":"2024-05-30T17:14:10.398735+0000","last_clean":"2024-05-30T17:14:10.398735+0000","last_became_active":"2024-05-30T16:17:31.204434+0000","last_became_peered":"2024-05-30T16:17:31.204434+0000","last_unstale":"2024-05-30T17:14:10.398735+0000","last_undegraded":"2024-05-30T17:14:10.398735+0000","last_fullsized":"2024-05-30T17:14:10.398735+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:14:10.398504+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:14:10.398504+0000","last_clean_scrub_stamp":"2024-05-30T17:14:10.398504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:25:37.977228+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"265'26","reported_seq":134,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:57.206156+0000","last_change":"2024-05-30T17:13:57.206337+0000","last_active":"2024-05-30T17:13:57.206156+0000","last_peered":"2024-05-30T17:13:57.206156+0000","last_clean":"2024-05-30T17:13:57.206156+0000","last_became_active":"2024-05-30T16:12:50.813176+0000","last_became_peered":"2024-05-30T16:12:50.813176+0000","last_unstale":"2024-05-30T17:13:57.206156+0000","last_undegraded":"2024-05-30T17:13:57.206156+0000","last_fullsized":"2024-05-30T17:13:57.206156+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'26","last_scrub_stamp":"2024-05-30T17:13:57.205944+0000","last_deep_scrub":"265'26","last_deep_scrub_stamp":"2024-05-30T17:13:57.205944+0000","last_clean_scrub_stamp":"2024-05-30T17:13:57.205944+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:09:07.523880+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":85,"num_write":57,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":38,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:32.401279+0000","last_change":"2024-05-30T17:14:32.401279+0000","last_active":"2024-05-30T17:14:32.401279+0000","last_peered":"2024-05-30T17:14:32.401279+0000","last_clean":"2024-05-30T17:14:32.401279+0000","last_became_active":"2024-05-30T16:34:55.871800+0000","last_became_peered":"2024-05-30T16:34:55.871800+0000","last_unstale":"2024-05-30T17:14:32.401279+0000","last_undegraded":"2024-05-30T17:14:32.401279+0000","last_fullsized":"2024-05-30T17:14:32.401279+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:32.400922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:32.400922+0000","last_clean_scrub_stamp":"2024-05-30T17:14:32.400922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:58:22.647600+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"265'2","reported_seq":57,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:27.047209+0000","last_change":"2024-05-30T17:14:27.047335+0000","last_active":"2024-05-30T17:14:27.047209+0000","last_peered":"2024-05-30T17:14:27.047209+0000","last_clean":"2024-05-30T17:14:27.047209+0000","last_became_active":"2024-05-30T16:17:35.236867+0000","last_became_peered":"2024-05-30T16:17:35.236867+0000","last_unstale":"2024-05-30T17:14:27.047209+0000","last_undegraded":"2024-05-30T17:14:27.047209+0000","last_fullsized":"2024-05-30T17:14:27.047209+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:14:27.046910+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:14:27.046910+0000","last_clean_scrub_stamp":"2024-05-30T17:14:27.046910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:43:54.102925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":571,"num_write":2,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"0'0","reported_seq":60,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.914149+0000","last_change":"2024-05-30T17:13:59.914262+0000","last_active":"2024-05-30T17:13:59.914149+0000","last_peered":"2024-05-30T17:13:59.914149+0000","last_clean":"2024-05-30T17:13:59.914149+0000","last_became_active":"2024-05-30T16:17:31.209882+0000","last_became_peered":"2024-05-30T16:17:31.209882+0000","last_unstale":"2024-05-30T17:13:59.914149+0000","last_undegraded":"2024-05-30T17:13:59.914149+0000","last_fullsized":"2024-05-30T17:13:59.914149+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:59.913913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:59.913913+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.913913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:14:43.780738+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"265'56","reported_seq":228,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.549397+0000","last_change":"2024-05-30T17:13:54.549516+0000","last_active":"2024-05-30T17:13:54.549397+0000","last_peered":"2024-05-30T17:13:54.549397+0000","last_clean":"2024-05-30T17:13:54.549397+0000","last_became_active":"2024-05-30T16:12:50.824043+0000","last_became_peered":"2024-05-30T16:12:50.824043+0000","last_unstale":"2024-05-30T17:13:54.549397+0000","last_undegraded":"2024-05-30T17:13:54.549397+0000","last_fullsized":"2024-05-30T17:13:54.549397+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'56","last_scrub_stamp":"2024-05-30T17:13:54.549107+0000","last_deep_scrub":"265'56","last_deep_scrub_stamp":"2024-05-30T17:13:54.549107+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.549107+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:55:26.307978+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":308,"num_read_kb":221,"num_write":92,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":37,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:43.486978+0000","last_change":"2024-05-30T17:14:43.487037+0000","last_active":"2024-05-30T17:14:43.486978+0000","last_peered":"2024-05-30T17:14:43.486978+0000","last_clean":"2024-05-30T17:14:43.486978+0000","last_became_active":"2024-05-30T16:34:55.893040+0000","last_became_peered":"2024-05-30T16:34:55.893040+0000","last_unstale":"2024-05-30T17:14:43.486978+0000","last_undegraded":"2024-05-30T17:14:43.486978+0000","last_fullsized":"2024-05-30T17:14:43.486978+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:43.486776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:43.486776+0000","last_clean_scrub_stamp":"2024-05-30T17:14:43.486776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:06:17.663331+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"0'0","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:23.020982+0000","last_change":"2024-05-30T17:14:23.021037+0000","last_active":"2024-05-30T17:14:23.020982+0000","last_peered":"2024-05-30T17:14:23.020982+0000","last_clean":"2024-05-30T17:14:23.020982+0000","last_became_active":"2024-05-30T16:17:35.246187+0000","last_became_peered":"2024-05-30T16:17:35.246187+0000","last_unstale":"2024-05-30T17:14:23.020982+0000","last_undegraded":"2024-05-30T17:14:23.020982+0000","last_fullsized":"2024-05-30T17:14:23.020982+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:23.020786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:23.020786+0000","last_clean_scrub_stamp":"2024-05-30T17:14:23.020786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:21:12.644609+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"265'8","reported_seq":80,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.895849+0000","last_change":"2024-05-30T17:13:54.895979+0000","last_active":"2024-05-30T17:13:54.895849+0000","last_peered":"2024-05-30T17:13:54.895849+0000","last_clean":"2024-05-30T17:13:54.895849+0000","last_became_active":"2024-05-30T16:17:31.189091+0000","last_became_peered":"2024-05-30T16:17:31.189091+0000","last_unstale":"2024-05-30T17:13:54.895849+0000","last_undegraded":"2024-05-30T17:13:54.895849+0000","last_fullsized":"2024-05-30T17:13:54.895849+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'8","last_scrub_stamp":"2024-05-30T17:13:54.895517+0000","last_deep_scrub":"265'8","last_deep_scrub_stamp":"2024-05-30T17:13:54.895517+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.895517+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:48:19.231935+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":21053,"num_write":6,"num_write_kb":5691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"265'9","reported_seq":93,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:55.595931+0000","last_change":"2024-05-30T17:13:55.595991+0000","last_active":"2024-05-30T17:13:55.595931+0000","last_peered":"2024-05-30T17:13:55.595931+0000","last_clean":"2024-05-30T17:13:55.595931+0000","last_became_active":"2024-05-30T16:12:50.823759+0000","last_became_peered":"2024-05-30T16:12:50.823759+0000","last_unstale":"2024-05-30T17:13:55.595931+0000","last_undegraded":"2024-05-30T17:13:55.595931+0000","last_fullsized":"2024-05-30T17:13:55.595931+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'9","last_scrub_stamp":"2024-05-30T17:13:55.595737+0000","last_deep_scrub":"265'9","last_deep_scrub_stamp":"2024-05-30T17:13:55.595737+0000","last_clean_scrub_stamp":"2024-05-30T17:13:55.595737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:06:46.930141+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":33,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":37,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:37.946612+0000","last_change":"2024-05-30T17:14:37.946707+0000","last_active":"2024-05-30T17:14:37.946612+0000","last_peered":"2024-05-30T17:14:37.946612+0000","last_clean":"2024-05-30T17:14:37.946612+0000","last_became_active":"2024-05-30T16:34:55.860695+0000","last_became_peered":"2024-05-30T16:34:55.860695+0000","last_unstale":"2024-05-30T17:14:37.946612+0000","last_undegraded":"2024-05-30T17:14:37.946612+0000","last_fullsized":"2024-05-30T17:14:37.946612+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:37.946410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:37.946410+0000","last_clean_scrub_stamp":"2024-05-30T17:14:37.946410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:00:22.393141+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"0'0","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.981887+0000","last_change":"2024-05-30T17:13:53.981997+0000","last_active":"2024-05-30T17:13:53.981887+0000","last_peered":"2024-05-30T17:13:53.981887+0000","last_clean":"2024-05-30T17:13:53.981887+0000","last_became_active":"2024-05-30T16:17:35.239157+0000","last_became_peered":"2024-05-30T16:17:35.239157+0000","last_unstale":"2024-05-30T17:13:53.981887+0000","last_undegraded":"2024-05-30T17:13:53.981887+0000","last_fullsized":"2024-05-30T17:13:53.981887+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:53.981644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:53.981644+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.981644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:55:41.438700+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"265'2","reported_seq":64,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.438167+0000","last_change":"2024-05-30T17:13:47.438232+0000","last_active":"2024-05-30T17:13:47.438167+0000","last_peered":"2024-05-30T17:13:47.438167+0000","last_clean":"2024-05-30T17:13:47.438167+0000","last_became_active":"2024-05-30T16:17:31.184675+0000","last_became_peered":"2024-05-30T16:17:31.184675+0000","last_unstale":"2024-05-30T17:13:47.438167+0000","last_undegraded":"2024-05-30T17:13:47.438167+0000","last_fullsized":"2024-05-30T17:13:47.438167+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:13:47.437969+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:13:47.437969+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.437969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:58:28.886391+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":698,"num_write":2,"num_write_kb":698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"265'97","reported_seq":367,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.658941+0000","last_change":"2024-05-30T17:13:49.659011+0000","last_active":"2024-05-30T17:13:49.658941+0000","last_peered":"2024-05-30T17:13:49.658941+0000","last_clean":"2024-05-30T17:13:49.658941+0000","last_became_active":"2024-05-30T16:12:50.801791+0000","last_became_peered":"2024-05-30T16:12:50.801791+0000","last_unstale":"2024-05-30T17:13:49.658941+0000","last_undegraded":"2024-05-30T17:13:49.658941+0000","last_fullsized":"2024-05-30T17:13:49.658941+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'97","last_scrub_stamp":"2024-05-30T17:13:49.658745+0000","last_deep_scrub":"265'97","last_deep_scrub_stamp":"2024-05-30T17:13:49.658745+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.658745+0000","objects_scrubbed":1,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:10:57.618551+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":591,"num_read_kb":446,"num_write":104,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2994,"num_omap_keys":6,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"265'2","reported_seq":40,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:40.017063+0000","last_change":"2024-05-30T17:14:40.017063+0000","last_active":"2024-05-30T17:14:40.017063+0000","last_peered":"2024-05-30T17:14:40.017063+0000","last_clean":"2024-05-30T17:14:40.017063+0000","last_became_active":"2024-05-30T16:34:55.863483+0000","last_became_peered":"2024-05-30T16:34:55.863483+0000","last_unstale":"2024-05-30T17:14:40.017063+0000","last_undegraded":"2024-05-30T17:14:40.017063+0000","last_fullsized":"2024-05-30T17:14:40.017063+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:14:40.016641+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:14:40.016641+0000","last_clean_scrub_stamp":"2024-05-30T17:14:40.016641+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:24:13.343103+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"0'0","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:03.388657+0000","last_change":"2024-05-30T17:14:03.388717+0000","last_active":"2024-05-30T17:14:03.388657+0000","last_peered":"2024-05-30T17:14:03.388657+0000","last_clean":"2024-05-30T17:14:03.388657+0000","last_became_active":"2024-05-30T16:17:35.235283+0000","last_became_peered":"2024-05-30T16:17:35.235283+0000","last_unstale":"2024-05-30T17:14:03.388657+0000","last_undegraded":"2024-05-30T17:14:03.388657+0000","last_fullsized":"2024-05-30T17:14:03.388657+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:03.388441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:03.388441+0000","last_clean_scrub_stamp":"2024-05-30T17:14:03.388441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:18:19.298873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"0'0","reported_seq":61,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:15.820628+0000","last_change":"2024-05-30T17:14:15.820726+0000","last_active":"2024-05-30T17:14:15.820628+0000","last_peered":"2024-05-30T17:14:15.820628+0000","last_clean":"2024-05-30T17:14:15.820628+0000","last_became_active":"2024-05-30T16:17:31.199499+0000","last_became_peered":"2024-05-30T16:17:31.199499+0000","last_unstale":"2024-05-30T17:14:15.820628+0000","last_undegraded":"2024-05-30T17:14:15.820628+0000","last_fullsized":"2024-05-30T17:14:15.820628+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:15.820409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:15.820409+0000","last_clean_scrub_stamp":"2024-05-30T17:14:15.820409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:58:15.762012+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"265'22","reported_seq":127,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.758712+0000","last_change":"2024-05-30T17:13:44.758825+0000","last_active":"2024-05-30T17:13:44.758712+0000","last_peered":"2024-05-30T17:13:44.758712+0000","last_clean":"2024-05-30T17:13:44.758712+0000","last_became_active":"2024-05-30T16:12:50.807156+0000","last_became_peered":"2024-05-30T16:12:50.807156+0000","last_unstale":"2024-05-30T17:13:44.758712+0000","last_undegraded":"2024-05-30T17:13:44.758712+0000","last_fullsized":"2024-05-30T17:13:44.758712+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'22","last_scrub_stamp":"2024-05-30T17:13:44.758408+0000","last_deep_scrub":"265'22","last_deep_scrub_stamp":"2024-05-30T17:13:44.758408+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.758408+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:20:24.643496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":76,"num_write":50,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"265'2","reported_seq":39,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:41.506569+0000","last_change":"2024-05-30T17:14:41.506641+0000","last_active":"2024-05-30T17:14:41.506569+0000","last_peered":"2024-05-30T17:14:41.506569+0000","last_clean":"2024-05-30T17:14:41.506569+0000","last_became_active":"2024-05-30T16:34:55.887845+0000","last_became_peered":"2024-05-30T16:34:55.887845+0000","last_unstale":"2024-05-30T17:14:41.506569+0000","last_undegraded":"2024-05-30T17:14:41.506569+0000","last_fullsized":"2024-05-30T17:14:41.506569+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:14:41.506353+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:14:41.506353+0000","last_clean_scrub_stamp":"2024-05-30T17:14:41.506353+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:13:17.505411+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"0'0","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:19.832379+0000","last_change":"2024-05-30T17:14:19.832435+0000","last_active":"2024-05-30T17:14:19.832379+0000","last_peered":"2024-05-30T17:14:19.832379+0000","last_clean":"2024-05-30T17:14:19.832379+0000","last_became_active":"2024-05-30T16:17:35.245852+0000","last_became_peered":"2024-05-30T16:17:35.245852+0000","last_unstale":"2024-05-30T17:14:19.832379+0000","last_undegraded":"2024-05-30T17:14:19.832379+0000","last_fullsized":"2024-05-30T17:14:19.832379+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:19.832182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:19.832182+0000","last_clean_scrub_stamp":"2024-05-30T17:14:19.832182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:18:45.511193+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"265'6","reported_seq":77,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:00.249278+0000","last_change":"2024-05-30T17:14:00.249347+0000","last_active":"2024-05-30T17:14:00.249278+0000","last_peered":"2024-05-30T17:14:00.249278+0000","last_clean":"2024-05-30T17:14:00.249278+0000","last_became_active":"2024-05-30T16:17:31.218752+0000","last_became_peered":"2024-05-30T16:17:31.218752+0000","last_unstale":"2024-05-30T17:14:00.249278+0000","last_undegraded":"2024-05-30T17:14:00.249278+0000","last_fullsized":"2024-05-30T17:14:00.249278+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'6","last_scrub_stamp":"2024-05-30T17:14:00.249094+0000","last_deep_scrub":"265'6","last_deep_scrub_stamp":"2024-05-30T17:14:00.249094+0000","last_clean_scrub_stamp":"2024-05-30T17:14:00.249094+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:10:46.604501+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"265'17","reported_seq":112,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.434952+0000","last_change":"2024-05-30T17:13:48.435015+0000","last_active":"2024-05-30T17:13:48.434952+0000","last_peered":"2024-05-30T17:13:48.434952+0000","last_clean":"2024-05-30T17:13:48.434952+0000","last_became_active":"2024-05-30T16:12:50.817114+0000","last_became_peered":"2024-05-30T16:12:50.817114+0000","last_unstale":"2024-05-30T17:13:48.434952+0000","last_undegraded":"2024-05-30T17:13:48.434952+0000","last_fullsized":"2024-05-30T17:13:48.434952+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'17","last_scrub_stamp":"2024-05-30T17:13:48.434761+0000","last_deep_scrub":"265'17","last_deep_scrub_stamp":"2024-05-30T17:13:48.434761+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.434761+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:55.692466+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":55,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":39,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:12.439006+0000","last_change":"2024-05-30T17:14:12.439066+0000","last_active":"2024-05-30T17:14:12.439006+0000","last_peered":"2024-05-30T17:14:12.439006+0000","last_clean":"2024-05-30T17:14:12.439006+0000","last_became_active":"2024-05-30T16:34:55.866415+0000","last_became_peered":"2024-05-30T16:34:55.866415+0000","last_unstale":"2024-05-30T17:14:12.439006+0000","last_undegraded":"2024-05-30T17:14:12.439006+0000","last_fullsized":"2024-05-30T17:14:12.439006+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:12.438804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:12.438804+0000","last_clean_scrub_stamp":"2024-05-30T17:14:12.438804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:30:46.541635+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"0'0","reported_seq":56,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.361463+0000","last_change":"2024-05-30T17:13:51.361463+0000","last_active":"2024-05-30T17:13:51.361463+0000","last_peered":"2024-05-30T17:13:51.361463+0000","last_clean":"2024-05-30T17:13:51.361463+0000","last_became_active":"2024-05-30T16:17:35.227270+0000","last_became_peered":"2024-05-30T16:17:35.227270+0000","last_unstale":"2024-05-30T17:13:51.361463+0000","last_undegraded":"2024-05-30T17:13:51.361463+0000","last_fullsized":"2024-05-30T17:13:51.361463+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:51.361210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:51.361210+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.361210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:13:33.987563+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"0'0","reported_seq":62,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.503842+0000","last_change":"2024-05-30T17:13:45.503842+0000","last_active":"2024-05-30T17:13:45.503842+0000","last_peered":"2024-05-30T17:13:45.503842+0000","last_clean":"2024-05-30T17:13:45.503842+0000","last_became_active":"2024-05-30T16:17:31.203731+0000","last_became_peered":"2024-05-30T16:17:31.203731+0000","last_unstale":"2024-05-30T17:13:45.503842+0000","last_undegraded":"2024-05-30T17:13:45.503842+0000","last_fullsized":"2024-05-30T17:13:45.503842+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:45.503591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:45.503591+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.503591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:19:48.974417+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"265'29","reported_seq":146,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.392583+0000","last_change":"2024-05-30T17:13:50.392583+0000","last_active":"2024-05-30T17:13:50.392583+0000","last_peered":"2024-05-30T17:13:50.392583+0000","last_clean":"2024-05-30T17:13:50.392583+0000","last_became_active":"2024-05-30T16:12:50.824809+0000","last_became_peered":"2024-05-30T16:12:50.824809+0000","last_unstale":"2024-05-30T17:13:50.392583+0000","last_undegraded":"2024-05-30T17:13:50.392583+0000","last_fullsized":"2024-05-30T17:13:50.392583+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'29","last_scrub_stamp":"2024-05-30T17:13:50.392236+0000","last_deep_scrub":"265'29","last_deep_scrub_stamp":"2024-05-30T17:13:50.392236+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.392236+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:32:06.254235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":98,"num_write":77,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":38,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:40.671898+0000","last_change":"2024-05-30T17:14:40.671962+0000","last_active":"2024-05-30T17:14:40.671898+0000","last_peered":"2024-05-30T17:14:40.671898+0000","last_clean":"2024-05-30T17:14:40.671898+0000","last_became_active":"2024-05-30T16:34:55.896357+0000","last_became_peered":"2024-05-30T16:34:55.896357+0000","last_unstale":"2024-05-30T17:14:40.671898+0000","last_undegraded":"2024-05-30T17:14:40.671898+0000","last_fullsized":"2024-05-30T17:14:40.671898+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:40.671697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:40.671697+0000","last_clean_scrub_stamp":"2024-05-30T17:14:40.671697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:17:06.227011+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"265'3","reported_seq":55,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:30.431628+0000","last_change":"2024-05-30T17:14:30.431690+0000","last_active":"2024-05-30T17:14:30.431628+0000","last_peered":"2024-05-30T17:14:30.431628+0000","last_clean":"2024-05-30T17:14:30.431628+0000","last_became_active":"2024-05-30T16:17:55.550782+0000","last_became_peered":"2024-05-30T16:17:55.550782+0000","last_unstale":"2024-05-30T17:14:30.431628+0000","last_undegraded":"2024-05-30T17:14:30.431628+0000","last_fullsized":"2024-05-30T17:14:30.431628+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:14:30.431421+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:14:30.431421+0000","last_clean_scrub_stamp":"2024-05-30T17:14:30.431421+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:11:57.762683+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2048,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.d","version":"265'7","reported_seq":74,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:07.405375+0000","last_change":"2024-05-30T17:14:07.405503+0000","last_active":"2024-05-30T17:14:07.405375+0000","last_peered":"2024-05-30T17:14:07.405375+0000","last_clean":"2024-05-30T17:14:07.405375+0000","last_became_active":"2024-05-30T16:17:31.202042+0000","last_became_peered":"2024-05-30T16:17:31.202042+0000","last_unstale":"2024-05-30T17:14:07.405375+0000","last_undegraded":"2024-05-30T17:14:07.405375+0000","last_fullsized":"2024-05-30T17:14:07.405375+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'7","last_scrub_stamp":"2024-05-30T17:14:07.405148+0000","last_deep_scrub":"265'7","last_deep_scrub_stamp":"2024-05-30T17:14:07.405148+0000","last_clean_scrub_stamp":"2024-05-30T17:14:07.405148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:09:19.710358+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":6194,"num_write":7,"num_write_kb":6194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"265'24","reported_seq":139,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.172777+0000","last_change":"2024-05-30T17:13:44.172845+0000","last_active":"2024-05-30T17:13:44.172777+0000","last_peered":"2024-05-30T17:13:44.172777+0000","last_clean":"2024-05-30T17:13:44.172777+0000","last_became_active":"2024-05-30T16:12:50.810095+0000","last_became_peered":"2024-05-30T16:12:50.810095+0000","last_unstale":"2024-05-30T17:13:44.172777+0000","last_undegraded":"2024-05-30T17:13:44.172777+0000","last_fullsized":"2024-05-30T17:13:44.172777+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'24","last_scrub_stamp":"2024-05-30T17:13:44.172579+0000","last_deep_scrub":"265'24","last_deep_scrub_stamp":"2024-05-30T17:13:44.172579+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.172579+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:00:02.104351+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":90,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":39,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:20.793663+0000","last_change":"2024-05-30T17:14:20.793717+0000","last_active":"2024-05-30T17:14:20.793663+0000","last_peered":"2024-05-30T17:14:20.793663+0000","last_clean":"2024-05-30T17:14:20.793663+0000","last_became_active":"2024-05-30T16:34:55.862478+0000","last_became_peered":"2024-05-30T16:34:55.862478+0000","last_unstale":"2024-05-30T17:14:20.793663+0000","last_undegraded":"2024-05-30T17:14:20.793663+0000","last_fullsized":"2024-05-30T17:14:20.793663+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:20.793471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:20.793471+0000","last_clean_scrub_stamp":"2024-05-30T17:14:20.793471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:12:26.521954+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"262'3","reported_seq":62,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:23.971862+0000","last_change":"2024-05-30T17:14:23.971939+0000","last_active":"2024-05-30T17:14:23.971862+0000","last_peered":"2024-05-30T17:14:23.971862+0000","last_clean":"2024-05-30T17:14:23.971862+0000","last_became_active":"2024-05-30T16:17:35.742255+0000","last_became_peered":"2024-05-30T16:17:35.742255+0000","last_unstale":"2024-05-30T17:14:23.971862+0000","last_undegraded":"2024-05-30T17:14:23.971862+0000","last_fullsized":"2024-05-30T17:14:23.971862+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'3","last_scrub_stamp":"2024-05-30T17:14:23.971656+0000","last_deep_scrub":"262'3","last_deep_scrub_stamp":"2024-05-30T17:14:23.971656+0000","last_clean_scrub_stamp":"2024-05-30T17:14:23.971656+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:02:04.479467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"265'2","reported_seq":65,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:55.774110+0000","last_change":"2024-05-30T17:13:55.774249+0000","last_active":"2024-05-30T17:13:55.774110+0000","last_peered":"2024-05-30T17:13:55.774110+0000","last_clean":"2024-05-30T17:13:55.774110+0000","last_became_active":"2024-05-30T16:17:31.198744+0000","last_became_peered":"2024-05-30T16:17:31.198744+0000","last_unstale":"2024-05-30T17:13:55.774110+0000","last_undegraded":"2024-05-30T17:13:55.774110+0000","last_fullsized":"2024-05-30T17:13:55.774110+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:13:55.773876+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:13:55.773876+0000","last_clean_scrub_stamp":"2024-05-30T17:13:55.773876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:52:06.016908+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"265'36","reported_seq":172,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.883144+0000","last_change":"2024-05-30T17:13:45.883208+0000","last_active":"2024-05-30T17:13:45.883144+0000","last_peered":"2024-05-30T17:13:45.883144+0000","last_clean":"2024-05-30T17:13:45.883144+0000","last_became_active":"2024-05-30T16:12:50.820936+0000","last_became_peered":"2024-05-30T16:12:50.820936+0000","last_unstale":"2024-05-30T17:13:45.883144+0000","last_undegraded":"2024-05-30T17:13:45.883144+0000","last_fullsized":"2024-05-30T17:13:45.883144+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'36","last_scrub_stamp":"2024-05-30T17:13:45.882931+0000","last_deep_scrub":"265'36","last_deep_scrub_stamp":"2024-05-30T17:13:45.882931+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.882931+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:49:53.101567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":194,"num_read_kb":136,"num_write":73,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":39,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:08.715919+0000","last_change":"2024-05-30T17:14:08.715975+0000","last_active":"2024-05-30T17:14:08.715919+0000","last_peered":"2024-05-30T17:14:08.715919+0000","last_clean":"2024-05-30T17:14:08.715919+0000","last_became_active":"2024-05-30T16:34:55.875965+0000","last_became_peered":"2024-05-30T16:34:55.875965+0000","last_unstale":"2024-05-30T17:14:08.715919+0000","last_undegraded":"2024-05-30T17:14:08.715919+0000","last_fullsized":"2024-05-30T17:14:08.715919+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:08.715715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:08.715715+0000","last_clean_scrub_stamp":"2024-05-30T17:14:08.715715+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:29:25.928545+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"265'3","reported_seq":61,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.354852+0000","last_change":"2024-05-30T17:13:59.354913+0000","last_active":"2024-05-30T17:13:59.354852+0000","last_peered":"2024-05-30T17:13:59.354852+0000","last_clean":"2024-05-30T17:13:59.354852+0000","last_became_active":"2024-05-30T16:17:35.229643+0000","last_became_peered":"2024-05-30T16:17:35.229643+0000","last_unstale":"2024-05-30T17:13:59.354852+0000","last_undegraded":"2024-05-30T17:13:59.354852+0000","last_fullsized":"2024-05-30T17:13:59.354852+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:13:59.354655+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:13:59.354655+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.354655+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:40:27.564311+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.f","version":"262'6","reported_seq":73,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.021324+0000","last_change":"2024-05-30T17:13:43.021324+0000","last_active":"2024-05-30T17:13:43.021324+0000","last_peered":"2024-05-30T17:13:43.021324+0000","last_clean":"2024-05-30T17:13:43.021324+0000","last_became_active":"2024-05-30T16:17:31.212894+0000","last_became_peered":"2024-05-30T16:17:31.212894+0000","last_unstale":"2024-05-30T17:13:43.021324+0000","last_undegraded":"2024-05-30T17:13:43.021324+0000","last_fullsized":"2024-05-30T17:13:43.021324+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'6","last_scrub_stamp":"2024-05-30T17:13:43.021049+0000","last_deep_scrub":"262'6","last_deep_scrub_stamp":"2024-05-30T17:13:43.021049+0000","last_clean_scrub_stamp":"2024-05-30T17:13:43.021049+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:27:51.042980+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6144,"num_write":11,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"262'21","reported_seq":130,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.118605+0000","last_change":"2024-05-30T17:13:53.118667+0000","last_active":"2024-05-30T17:13:53.118605+0000","last_peered":"2024-05-30T17:13:53.118605+0000","last_clean":"2024-05-30T17:13:53.118605+0000","last_became_active":"2024-05-30T16:12:50.805772+0000","last_became_peered":"2024-05-30T16:12:50.805772+0000","last_unstale":"2024-05-30T17:13:53.118605+0000","last_undegraded":"2024-05-30T17:13:53.118605+0000","last_fullsized":"2024-05-30T17:13:53.118605+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'21","last_scrub_stamp":"2024-05-30T17:13:53.118396+0000","last_deep_scrub":"262'21","last_deep_scrub_stamp":"2024-05-30T17:13:53.118396+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.118396+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:21:33.139209+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":79,"num_write":45,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":38,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:42.526238+0000","last_change":"2024-05-30T17:14:42.526293+0000","last_active":"2024-05-30T17:14:42.526238+0000","last_peered":"2024-05-30T17:14:42.526238+0000","last_clean":"2024-05-30T17:14:42.526238+0000","last_became_active":"2024-05-30T16:34:55.890313+0000","last_became_peered":"2024-05-30T16:34:55.890313+0000","last_unstale":"2024-05-30T17:14:42.526238+0000","last_undegraded":"2024-05-30T17:14:42.526238+0000","last_fullsized":"2024-05-30T17:14:42.526238+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:42.526043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:42.526043+0000","last_clean_scrub_stamp":"2024-05-30T17:14:42.526043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:54:48.585284+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"265'3","reported_seq":64,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:29.017317+0000","last_change":"2024-05-30T17:14:29.017378+0000","last_active":"2024-05-30T17:14:29.017317+0000","last_peered":"2024-05-30T17:14:29.017317+0000","last_clean":"2024-05-30T17:14:29.017317+0000","last_became_active":"2024-05-30T16:17:35.242271+0000","last_became_peered":"2024-05-30T16:17:35.242271+0000","last_unstale":"2024-05-30T17:14:29.017317+0000","last_undegraded":"2024-05-30T17:14:29.017317+0000","last_fullsized":"2024-05-30T17:14:29.017317+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:14:29.017118+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:14:29.017118+0000","last_clean_scrub_stamp":"2024-05-30T17:14:29.017118+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:44:42.366902+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10240,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"265'3","reported_seq":70,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.656221+0000","last_change":"2024-05-30T17:13:51.656292+0000","last_active":"2024-05-30T17:13:51.656221+0000","last_peered":"2024-05-30T17:13:51.656221+0000","last_clean":"2024-05-30T17:13:51.656221+0000","last_became_active":"2024-05-30T16:17:31.194774+0000","last_became_peered":"2024-05-30T16:17:31.194774+0000","last_unstale":"2024-05-30T17:13:51.656221+0000","last_undegraded":"2024-05-30T17:13:51.656221+0000","last_fullsized":"2024-05-30T17:13:51.656221+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:13:51.656016+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:13:51.656016+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.656016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:08:09.027576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":12288,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"265'18","reported_seq":116,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.659792+0000","last_change":"2024-05-30T17:13:44.659853+0000","last_active":"2024-05-30T17:13:44.659792+0000","last_peered":"2024-05-30T17:13:44.659792+0000","last_clean":"2024-05-30T17:13:44.659792+0000","last_became_active":"2024-05-30T16:12:50.820526+0000","last_became_peered":"2024-05-30T16:12:50.820526+0000","last_unstale":"2024-05-30T17:13:44.659792+0000","last_undegraded":"2024-05-30T17:13:44.659792+0000","last_fullsized":"2024-05-30T17:13:44.659792+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'18","last_scrub_stamp":"2024-05-30T17:13:44.659583+0000","last_deep_scrub":"265'18","last_deep_scrub_stamp":"2024-05-30T17:13:44.659583+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.659583+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:01:55.866157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":60,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":38,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:38.697474+0000","last_change":"2024-05-30T17:14:38.697534+0000","last_active":"2024-05-30T17:14:38.697474+0000","last_peered":"2024-05-30T17:14:38.697474+0000","last_clean":"2024-05-30T17:14:38.697474+0000","last_became_active":"2024-05-30T16:34:55.884065+0000","last_became_peered":"2024-05-30T16:34:55.884065+0000","last_unstale":"2024-05-30T17:14:38.697474+0000","last_undegraded":"2024-05-30T17:14:38.697474+0000","last_fullsized":"2024-05-30T17:14:38.697474+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:38.697264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:38.697264+0000","last_clean_scrub_stamp":"2024-05-30T17:14:38.697264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:45:29.551653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"265'11","reported_seq":83,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.610393+0000","last_change":"2024-05-30T17:13:56.610471+0000","last_active":"2024-05-30T17:13:56.610393+0000","last_peered":"2024-05-30T17:13:56.610393+0000","last_clean":"2024-05-30T17:13:56.610393+0000","last_became_active":"2024-05-30T16:17:35.738991+0000","last_became_peered":"2024-05-30T16:17:35.738991+0000","last_unstale":"2024-05-30T17:13:56.610393+0000","last_undegraded":"2024-05-30T17:13:56.610393+0000","last_fullsized":"2024-05-30T17:13:56.610393+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'11","last_scrub_stamp":"2024-05-30T17:13:56.610167+0000","last_deep_scrub":"265'11","last_deep_scrub_stamp":"2024-05-30T17:13:56.610167+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.610167+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:07:32.799346+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":26998,"num_write":14,"num_write_kb":10938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"262'6","reported_seq":75,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.948567+0000","last_change":"2024-05-30T17:13:49.948567+0000","last_active":"2024-05-30T17:13:49.948567+0000","last_peered":"2024-05-30T17:13:49.948567+0000","last_clean":"2024-05-30T17:13:49.948567+0000","last_became_active":"2024-05-30T16:17:31.212954+0000","last_became_peered":"2024-05-30T16:17:31.212954+0000","last_unstale":"2024-05-30T17:13:49.948567+0000","last_undegraded":"2024-05-30T17:13:49.948567+0000","last_fullsized":"2024-05-30T17:13:49.948567+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'6","last_scrub_stamp":"2024-05-30T17:13:49.948291+0000","last_deep_scrub":"262'6","last_deep_scrub_stamp":"2024-05-30T17:13:49.948291+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.948291+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:46:37.622512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":11264,"num_write":11,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"265'103","reported_seq":396,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:57.554791+0000","last_change":"2024-05-30T17:13:57.554869+0000","last_active":"2024-05-30T17:13:57.554791+0000","last_peered":"2024-05-30T17:13:57.554791+0000","last_clean":"2024-05-30T17:13:57.554791+0000","last_became_active":"2024-05-30T16:12:50.820250+0000","last_became_peered":"2024-05-30T16:12:50.820250+0000","last_unstale":"2024-05-30T17:13:57.554791+0000","last_undegraded":"2024-05-30T17:13:57.554791+0000","last_fullsized":"2024-05-30T17:13:57.554791+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'103","last_scrub_stamp":"2024-05-30T17:13:57.554576+0000","last_deep_scrub":"265'103","last_deep_scrub_stamp":"2024-05-30T17:13:57.554576+0000","last_clean_scrub_stamp":"2024-05-30T17:13:57.554576+0000","objects_scrubbed":1,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:49:22.258431+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":642,"num_read_kb":499,"num_write":114,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3960,"num_omap_keys":8,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"265'2","reported_seq":40,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:35.730656+0000","last_change":"2024-05-30T17:14:35.730732+0000","last_active":"2024-05-30T17:14:35.730656+0000","last_peered":"2024-05-30T17:14:35.730656+0000","last_clean":"2024-05-30T17:14:35.730656+0000","last_became_active":"2024-05-30T16:34:55.878171+0000","last_became_peered":"2024-05-30T16:34:55.878171+0000","last_unstale":"2024-05-30T17:14:35.730656+0000","last_undegraded":"2024-05-30T17:14:35.730656+0000","last_fullsized":"2024-05-30T17:14:35.730656+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:14:35.730439+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:14:35.730439+0000","last_clean_scrub_stamp":"2024-05-30T17:14:35.730439+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:15:42.676263+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"265'15","reported_seq":85,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:21.995914+0000","last_change":"2024-05-30T17:14:21.995983+0000","last_active":"2024-05-30T17:14:21.995914+0000","last_peered":"2024-05-30T17:14:21.995914+0000","last_clean":"2024-05-30T17:14:21.995914+0000","last_became_active":"2024-05-30T16:17:35.245008+0000","last_became_peered":"2024-05-30T16:17:35.245008+0000","last_unstale":"2024-05-30T17:14:21.995914+0000","last_undegraded":"2024-05-30T17:14:21.995914+0000","last_fullsized":"2024-05-30T17:14:21.995914+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'15","last_scrub_stamp":"2024-05-30T17:14:21.995693+0000","last_deep_scrub":"265'15","last_deep_scrub_stamp":"2024-05-30T17:14:21.995693+0000","last_clean_scrub_stamp":"2024-05-30T17:14:21.995693+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:59:57.620982+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":25602,"num_write":23,"num_write_kb":17408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"262'3","reported_seq":69,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:03.872899+0000","last_change":"2024-05-30T17:14:03.872899+0000","last_active":"2024-05-30T17:14:03.872899+0000","last_peered":"2024-05-30T17:14:03.872899+0000","last_clean":"2024-05-30T17:14:03.872899+0000","last_became_active":"2024-05-30T16:17:31.210195+0000","last_became_peered":"2024-05-30T16:17:31.210195+0000","last_unstale":"2024-05-30T17:14:03.872899+0000","last_undegraded":"2024-05-30T17:14:03.872899+0000","last_fullsized":"2024-05-30T17:14:03.872899+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'3","last_scrub_stamp":"2024-05-30T17:14:03.872614+0000","last_deep_scrub":"262'3","last_deep_scrub_stamp":"2024-05-30T17:14:03.872614+0000","last_clean_scrub_stamp":"2024-05-30T17:14:03.872614+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:58:00.016236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"265'43","reported_seq":185,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.468823+0000","last_change":"2024-05-30T17:13:44.468886+0000","last_active":"2024-05-30T17:13:44.468823+0000","last_peered":"2024-05-30T17:13:44.468823+0000","last_clean":"2024-05-30T17:13:44.468823+0000","last_became_active":"2024-05-30T16:12:50.830770+0000","last_became_peered":"2024-05-30T16:12:50.830770+0000","last_unstale":"2024-05-30T17:13:44.468823+0000","last_undegraded":"2024-05-30T17:13:44.468823+0000","last_fullsized":"2024-05-30T17:13:44.468823+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'43","last_scrub_stamp":"2024-05-30T17:13:44.468612+0000","last_deep_scrub":"265'43","last_deep_scrub_stamp":"2024-05-30T17:13:44.468612+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.468612+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:26:30.609648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":214,"num_read_kb":148,"num_write":89,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":38,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:40.995719+0000","last_change":"2024-05-30T17:14:40.995772+0000","last_active":"2024-05-30T17:14:40.995719+0000","last_peered":"2024-05-30T17:14:40.995719+0000","last_clean":"2024-05-30T17:14:40.995719+0000","last_became_active":"2024-05-30T16:34:55.887101+0000","last_became_peered":"2024-05-30T16:34:55.887101+0000","last_unstale":"2024-05-30T17:14:40.995719+0000","last_undegraded":"2024-05-30T17:14:40.995719+0000","last_fullsized":"2024-05-30T17:14:40.995719+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:40.995523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:40.995523+0000","last_clean_scrub_stamp":"2024-05-30T17:14:40.995523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:24:17.122800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"262'3","reported_seq":63,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:52.503977+0000","last_change":"2024-05-30T17:13:52.503977+0000","last_active":"2024-05-30T17:13:52.503977+0000","last_peered":"2024-05-30T17:13:52.503977+0000","last_clean":"2024-05-30T17:13:52.503977+0000","last_became_active":"2024-05-30T16:17:35.212403+0000","last_became_peered":"2024-05-30T16:17:35.212403+0000","last_unstale":"2024-05-30T17:13:52.503977+0000","last_undegraded":"2024-05-30T17:13:52.503977+0000","last_fullsized":"2024-05-30T17:13:52.503977+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'3","last_scrub_stamp":"2024-05-30T17:13:52.503699+0000","last_deep_scrub":"262'3","last_deep_scrub_stamp":"2024-05-30T17:13:52.503699+0000","last_clean_scrub_stamp":"2024-05-30T17:13:52.503699+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:51:26.227521+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"0'0","reported_seq":62,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.933818+0000","last_change":"2024-05-30T17:13:58.933876+0000","last_active":"2024-05-30T17:13:58.933818+0000","last_peered":"2024-05-30T17:13:58.933818+0000","last_clean":"2024-05-30T17:13:58.933818+0000","last_became_active":"2024-05-30T16:17:31.191955+0000","last_became_peered":"2024-05-30T16:17:31.191955+0000","last_unstale":"2024-05-30T17:13:58.933818+0000","last_undegraded":"2024-05-30T17:13:58.933818+0000","last_fullsized":"2024-05-30T17:13:58.933818+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:58.933611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:58.933611+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.933611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:34:51.660228+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"262'8","reported_seq":92,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.839264+0000","last_change":"2024-05-30T17:13:46.839335+0000","last_active":"2024-05-30T17:13:46.839264+0000","last_peered":"2024-05-30T17:13:46.839264+0000","last_clean":"2024-05-30T17:13:46.839264+0000","last_became_active":"2024-05-30T16:12:50.819871+0000","last_became_peered":"2024-05-30T16:12:50.819871+0000","last_unstale":"2024-05-30T17:13:46.839264+0000","last_undegraded":"2024-05-30T17:13:46.839264+0000","last_fullsized":"2024-05-30T17:13:46.839264+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'8","last_scrub_stamp":"2024-05-30T17:13:46.839029+0000","last_deep_scrub":"262'8","last_deep_scrub_stamp":"2024-05-30T17:13:46.839029+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.839029+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:29:46.097404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":28,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":40,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:37.742935+0000","last_change":"2024-05-30T17:14:37.742995+0000","last_active":"2024-05-30T17:14:37.742935+0000","last_peered":"2024-05-30T17:14:37.742935+0000","last_clean":"2024-05-30T17:14:37.742935+0000","last_became_active":"2024-05-30T16:34:55.882443+0000","last_became_peered":"2024-05-30T16:34:55.882443+0000","last_unstale":"2024-05-30T17:14:37.742935+0000","last_undegraded":"2024-05-30T17:14:37.742935+0000","last_fullsized":"2024-05-30T17:14:37.742935+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:37.742728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:37.742728+0000","last_clean_scrub_stamp":"2024-05-30T17:14:37.742728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:41:14.282969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"262'3","reported_seq":64,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:07.870394+0000","last_change":"2024-05-30T17:14:07.870537+0000","last_active":"2024-05-30T17:14:07.870394+0000","last_peered":"2024-05-30T17:14:07.870394+0000","last_clean":"2024-05-30T17:14:07.870394+0000","last_became_active":"2024-05-30T16:17:35.247419+0000","last_became_peered":"2024-05-30T16:17:35.247419+0000","last_unstale":"2024-05-30T17:14:07.870394+0000","last_undegraded":"2024-05-30T17:14:07.870394+0000","last_fullsized":"2024-05-30T17:14:07.870394+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'3","last_scrub_stamp":"2024-05-30T17:14:07.870141+0000","last_deep_scrub":"262'3","last_deep_scrub_stamp":"2024-05-30T17:14:07.870141+0000","last_clean_scrub_stamp":"2024-05-30T17:14:07.870141+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:18:01.008022+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"262'2","reported_seq":68,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.502634+0000","last_change":"2024-05-30T17:13:50.502697+0000","last_active":"2024-05-30T17:13:50.502634+0000","last_peered":"2024-05-30T17:13:50.502634+0000","last_clean":"2024-05-30T17:13:50.502634+0000","last_became_active":"2024-05-30T16:17:31.189122+0000","last_became_peered":"2024-05-30T16:17:31.189122+0000","last_unstale":"2024-05-30T17:13:50.502634+0000","last_undegraded":"2024-05-30T17:13:50.502634+0000","last_fullsized":"2024-05-30T17:13:50.502634+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'2","last_scrub_stamp":"2024-05-30T17:13:50.502442+0000","last_deep_scrub":"262'2","last_deep_scrub_stamp":"2024-05-30T17:13:50.502442+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.502442+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:48:21.318011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"265'53","reported_seq":210,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:41.655149+0000","last_change":"2024-05-30T17:13:41.655171+0000","last_active":"2024-05-30T17:13:41.655149+0000","last_peered":"2024-05-30T17:13:41.655149+0000","last_clean":"2024-05-30T17:13:41.655149+0000","last_became_active":"2024-05-30T16:12:50.809372+0000","last_became_peered":"2024-05-30T16:12:50.809372+0000","last_unstale":"2024-05-30T17:13:41.655149+0000","last_undegraded":"2024-05-30T17:13:41.655149+0000","last_fullsized":"2024-05-30T17:13:41.655149+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'53","last_scrub_stamp":"2024-05-30T17:13:41.655071+0000","last_deep_scrub":"265'53","last_deep_scrub_stamp":"2024-05-30T17:13:41.655071+0000","last_clean_scrub_stamp":"2024-05-30T17:13:41.655071+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:39:32.196268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":186,"num_write":103,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":40,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:33.403502+0000","last_change":"2024-05-30T17:14:33.403558+0000","last_active":"2024-05-30T17:14:33.403502+0000","last_peered":"2024-05-30T17:14:33.403502+0000","last_clean":"2024-05-30T17:14:33.403502+0000","last_became_active":"2024-05-30T16:34:55.880627+0000","last_became_peered":"2024-05-30T16:34:55.880627+0000","last_unstale":"2024-05-30T17:14:33.403502+0000","last_undegraded":"2024-05-30T17:14:33.403502+0000","last_fullsized":"2024-05-30T17:14:33.403502+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:33.403309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:33.403309+0000","last_clean_scrub_stamp":"2024-05-30T17:14:33.403309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:35:32.928002+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"0'0","reported_seq":58,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:34.878116+0000","last_change":"2024-05-30T17:14:34.878176+0000","last_active":"2024-05-30T17:14:34.878116+0000","last_peered":"2024-05-30T17:14:34.878116+0000","last_clean":"2024-05-30T17:14:34.878116+0000","last_became_active":"2024-05-30T16:17:35.743036+0000","last_became_peered":"2024-05-30T16:17:35.743036+0000","last_unstale":"2024-05-30T17:14:34.878116+0000","last_undegraded":"2024-05-30T17:14:34.878116+0000","last_fullsized":"2024-05-30T17:14:34.878116+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:34.877900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:34.877900+0000","last_clean_scrub_stamp":"2024-05-30T17:14:34.877900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:17:31.425313+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"265'3","reported_seq":69,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:08.435518+0000","last_change":"2024-05-30T17:14:08.435583+0000","last_active":"2024-05-30T17:14:08.435518+0000","last_peered":"2024-05-30T17:14:08.435518+0000","last_clean":"2024-05-30T17:14:08.435518+0000","last_became_active":"2024-05-30T16:17:31.202064+0000","last_became_peered":"2024-05-30T16:17:31.202064+0000","last_unstale":"2024-05-30T17:14:08.435518+0000","last_undegraded":"2024-05-30T17:14:08.435518+0000","last_fullsized":"2024-05-30T17:14:08.435518+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:14:08.435314+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:14:08.435314+0000","last_clean_scrub_stamp":"2024-05-30T17:14:08.435314+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:58:19.406401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2048,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"265'34","reported_seq":158,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.862253+0000","last_change":"2024-05-30T17:13:47.862374+0000","last_active":"2024-05-30T17:13:47.862253+0000","last_peered":"2024-05-30T17:13:47.862253+0000","last_clean":"2024-05-30T17:13:47.862253+0000","last_became_active":"2024-05-30T16:12:50.815895+0000","last_became_peered":"2024-05-30T16:12:50.815895+0000","last_unstale":"2024-05-30T17:13:47.862253+0000","last_undegraded":"2024-05-30T17:13:47.862253+0000","last_fullsized":"2024-05-30T17:13:47.862253+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'34","last_scrub_stamp":"2024-05-30T17:13:47.861959+0000","last_deep_scrub":"265'34","last_deep_scrub_stamp":"2024-05-30T17:13:47.861959+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.861959+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:37:52.262386+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":160,"num_read_kb":111,"num_write":71,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":41,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:25.906869+0000","last_change":"2024-05-30T17:14:25.906869+0000","last_active":"2024-05-30T17:14:25.906869+0000","last_peered":"2024-05-30T17:14:25.906869+0000","last_clean":"2024-05-30T17:14:25.906869+0000","last_became_active":"2024-05-30T16:34:55.883112+0000","last_became_peered":"2024-05-30T16:34:55.883112+0000","last_unstale":"2024-05-30T17:14:25.906869+0000","last_undegraded":"2024-05-30T17:14:25.906869+0000","last_fullsized":"2024-05-30T17:14:25.906869+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:25.906607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:25.906607+0000","last_clean_scrub_stamp":"2024-05-30T17:14:25.906607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:04:15.646212+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"265'3","reported_seq":62,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:00.593884+0000","last_change":"2024-05-30T17:14:00.593945+0000","last_active":"2024-05-30T17:14:00.593884+0000","last_peered":"2024-05-30T17:14:00.593884+0000","last_clean":"2024-05-30T17:14:00.593884+0000","last_became_active":"2024-05-30T16:17:35.743608+0000","last_became_peered":"2024-05-30T16:17:35.743608+0000","last_unstale":"2024-05-30T17:14:00.593884+0000","last_undegraded":"2024-05-30T17:14:00.593884+0000","last_fullsized":"2024-05-30T17:14:00.593884+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:14:00.593691+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:14:00.593691+0000","last_clean_scrub_stamp":"2024-05-30T17:14:00.593691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:51:53.188748+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"0'0","reported_seq":63,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:52.225937+0000","last_change":"2024-05-30T17:13:52.225991+0000","last_active":"2024-05-30T17:13:52.225937+0000","last_peered":"2024-05-30T17:13:52.225937+0000","last_clean":"2024-05-30T17:13:52.225937+0000","last_became_active":"2024-05-30T16:17:31.188810+0000","last_became_peered":"2024-05-30T16:17:31.188810+0000","last_unstale":"2024-05-30T17:13:52.225937+0000","last_undegraded":"2024-05-30T17:13:52.225937+0000","last_fullsized":"2024-05-30T17:13:52.225937+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:52.225736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:52.225736+0000","last_clean_scrub_stamp":"2024-05-30T17:13:52.225736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:31:29.155002+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"265'18","reported_seq":120,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.502424+0000","last_change":"2024-05-30T17:13:53.502489+0000","last_active":"2024-05-30T17:13:53.502424+0000","last_peered":"2024-05-30T17:13:53.502424+0000","last_clean":"2024-05-30T17:13:53.502424+0000","last_became_active":"2024-05-30T16:12:50.826880+0000","last_became_peered":"2024-05-30T16:12:50.826880+0000","last_unstale":"2024-05-30T17:13:53.502424+0000","last_undegraded":"2024-05-30T17:13:53.502424+0000","last_fullsized":"2024-05-30T17:13:53.502424+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'18","last_scrub_stamp":"2024-05-30T17:13:53.502233+0000","last_deep_scrub":"265'18","last_deep_scrub_stamp":"2024-05-30T17:13:53.502233+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.502233+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:07:36.967499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":61,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":40,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:32.064770+0000","last_change":"2024-05-30T17:14:32.064905+0000","last_active":"2024-05-30T17:14:32.064770+0000","last_peered":"2024-05-30T17:14:32.064770+0000","last_clean":"2024-05-30T17:14:32.064770+0000","last_became_active":"2024-05-30T16:34:55.885643+0000","last_became_peered":"2024-05-30T16:34:55.885643+0000","last_unstale":"2024-05-30T17:14:32.064770+0000","last_undegraded":"2024-05-30T17:14:32.064770+0000","last_fullsized":"2024-05-30T17:14:32.064770+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:32.064467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:32.064467+0000","last_clean_scrub_stamp":"2024-05-30T17:14:32.064467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:55:02.356439+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"262'2","reported_seq":61,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:31.036894+0000","last_change":"2024-05-30T17:14:31.036955+0000","last_active":"2024-05-30T17:14:31.036894+0000","last_peered":"2024-05-30T17:14:31.036894+0000","last_clean":"2024-05-30T17:14:31.036894+0000","last_became_active":"2024-05-30T16:17:35.247757+0000","last_became_peered":"2024-05-30T16:17:35.247757+0000","last_unstale":"2024-05-30T17:14:31.036894+0000","last_undegraded":"2024-05-30T17:14:31.036894+0000","last_fullsized":"2024-05-30T17:14:31.036894+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'2","last_scrub_stamp":"2024-05-30T17:14:31.036688+0000","last_deep_scrub":"262'2","last_deep_scrub_stamp":"2024-05-30T17:14:31.036688+0000","last_clean_scrub_stamp":"2024-05-30T17:14:31.036688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:54:00.790665+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"0'0","reported_seq":63,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.456500+0000","last_change":"2024-05-30T17:13:46.456500+0000","last_active":"2024-05-30T17:13:46.456500+0000","last_peered":"2024-05-30T17:13:46.456500+0000","last_clean":"2024-05-30T17:13:46.456500+0000","last_became_active":"2024-05-30T16:17:31.203452+0000","last_became_peered":"2024-05-30T16:17:31.203452+0000","last_unstale":"2024-05-30T17:13:46.456500+0000","last_undegraded":"2024-05-30T17:13:46.456500+0000","last_fullsized":"2024-05-30T17:13:46.456500+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:46.456245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:46.456245+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.456245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:07:15.159368+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"265'37","reported_seq":171,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.919132+0000","last_change":"2024-05-30T17:13:46.919198+0000","last_active":"2024-05-30T17:13:46.919132+0000","last_peered":"2024-05-30T17:13:46.919132+0000","last_clean":"2024-05-30T17:13:46.919132+0000","last_became_active":"2024-05-30T16:12:50.811626+0000","last_became_peered":"2024-05-30T16:12:50.811626+0000","last_unstale":"2024-05-30T17:13:46.919132+0000","last_undegraded":"2024-05-30T17:13:46.919132+0000","last_fullsized":"2024-05-30T17:13:46.919132+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'37","last_scrub_stamp":"2024-05-30T17:13:46.918931+0000","last_deep_scrub":"265'37","last_deep_scrub_stamp":"2024-05-30T17:13:46.918931+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.918931+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:38:14.857169+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":184,"num_read_kb":132,"num_write":81,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"265'2","reported_seq":41,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:11.395523+0000","last_change":"2024-05-30T17:14:11.395649+0000","last_active":"2024-05-30T17:14:11.395523+0000","last_peered":"2024-05-30T17:14:11.395523+0000","last_clean":"2024-05-30T17:14:11.395523+0000","last_became_active":"2024-05-30T16:34:55.861767+0000","last_became_peered":"2024-05-30T16:34:55.861767+0000","last_unstale":"2024-05-30T17:14:11.395523+0000","last_undegraded":"2024-05-30T17:14:11.395523+0000","last_fullsized":"2024-05-30T17:14:11.395523+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:14:11.395245+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:14:11.395245+0000","last_clean_scrub_stamp":"2024-05-30T17:14:11.395245+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:31:20.732589+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"265'8","reported_seq":73,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:20.954829+0000","last_change":"2024-05-30T17:14:20.954890+0000","last_active":"2024-05-30T17:14:20.954829+0000","last_peered":"2024-05-30T17:14:20.954829+0000","last_clean":"2024-05-30T17:14:20.954829+0000","last_became_active":"2024-05-30T16:17:35.243435+0000","last_became_peered":"2024-05-30T16:17:35.243435+0000","last_unstale":"2024-05-30T17:14:20.954829+0000","last_undegraded":"2024-05-30T17:14:20.954829+0000","last_fullsized":"2024-05-30T17:14:20.954829+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'8","last_scrub_stamp":"2024-05-30T17:14:20.954635+0000","last_deep_scrub":"265'8","last_deep_scrub_stamp":"2024-05-30T17:14:20.954635+0000","last_clean_scrub_stamp":"2024-05-30T17:14:20.954635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:57:35.836726+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":13312,"num_write":13,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"265'3","reported_seq":68,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.458042+0000","last_change":"2024-05-30T17:13:47.458106+0000","last_active":"2024-05-30T17:13:47.458042+0000","last_peered":"2024-05-30T17:13:47.458042+0000","last_clean":"2024-05-30T17:13:47.458042+0000","last_became_active":"2024-05-30T16:17:31.200663+0000","last_became_peered":"2024-05-30T16:17:31.200663+0000","last_unstale":"2024-05-30T17:13:47.458042+0000","last_undegraded":"2024-05-30T17:13:47.458042+0000","last_fullsized":"2024-05-30T17:13:47.458042+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:13:47.457841+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:13:47.457841+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.457841+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:37:54.600403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"265'15","reported_seq":112,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.234861+0000","last_change":"2024-05-30T17:13:48.234982+0000","last_active":"2024-05-30T17:13:48.234861+0000","last_peered":"2024-05-30T17:13:48.234861+0000","last_clean":"2024-05-30T17:13:48.234861+0000","last_became_active":"2024-05-30T16:12:50.813741+0000","last_became_peered":"2024-05-30T16:12:50.813741+0000","last_unstale":"2024-05-30T17:13:48.234861+0000","last_undegraded":"2024-05-30T17:13:48.234861+0000","last_fullsized":"2024-05-30T17:13:48.234861+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'15","last_scrub_stamp":"2024-05-30T17:13:48.234638+0000","last_deep_scrub":"265'15","last_deep_scrub_stamp":"2024-05-30T17:13:48.234638+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.234638+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:15:02.442194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":53,"num_write":33,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":39,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:36.967495+0000","last_change":"2024-05-30T17:14:36.967555+0000","last_active":"2024-05-30T17:14:36.967495+0000","last_peered":"2024-05-30T17:14:36.967495+0000","last_clean":"2024-05-30T17:14:36.967495+0000","last_became_active":"2024-05-30T16:34:55.858872+0000","last_became_peered":"2024-05-30T16:34:55.858872+0000","last_unstale":"2024-05-30T17:14:36.967495+0000","last_undegraded":"2024-05-30T17:14:36.967495+0000","last_fullsized":"2024-05-30T17:14:36.967495+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:36.967285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:36.967285+0000","last_clean_scrub_stamp":"2024-05-30T17:14:36.967285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:33:23.656589+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"0'0","reported_seq":59,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:27.365550+0000","last_change":"2024-05-30T17:14:27.365614+0000","last_active":"2024-05-30T17:14:27.365550+0000","last_peered":"2024-05-30T17:14:27.365550+0000","last_clean":"2024-05-30T17:14:27.365550+0000","last_became_active":"2024-05-30T16:17:35.226829+0000","last_became_peered":"2024-05-30T16:17:35.226829+0000","last_unstale":"2024-05-30T17:14:27.365550+0000","last_undegraded":"2024-05-30T17:14:27.365550+0000","last_fullsized":"2024-05-30T17:14:27.365550+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:27.365359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:27.365359+0000","last_clean_scrub_stamp":"2024-05-30T17:14:27.365359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:59:10.172045+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"0'0","reported_seq":64,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.486126+0000","last_change":"2024-05-30T17:13:48.486182+0000","last_active":"2024-05-30T17:13:48.486126+0000","last_peered":"2024-05-30T17:13:48.486126+0000","last_clean":"2024-05-30T17:13:48.486126+0000","last_became_active":"2024-05-30T16:17:31.203535+0000","last_became_peered":"2024-05-30T16:17:31.203535+0000","last_unstale":"2024-05-30T17:13:48.486126+0000","last_undegraded":"2024-05-30T17:13:48.486126+0000","last_fullsized":"2024-05-30T17:13:48.486126+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:48.485936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:48.485936+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.485936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:42:58.343268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"265'36","reported_seq":166,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.750202+0000","last_change":"2024-05-30T17:13:43.750267+0000","last_active":"2024-05-30T17:13:43.750202+0000","last_peered":"2024-05-30T17:13:43.750202+0000","last_clean":"2024-05-30T17:13:43.750202+0000","last_became_active":"2024-05-30T16:12:50.831842+0000","last_became_peered":"2024-05-30T16:12:50.831842+0000","last_unstale":"2024-05-30T17:13:43.750202+0000","last_undegraded":"2024-05-30T17:13:43.750202+0000","last_fullsized":"2024-05-30T17:13:43.750202+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'36","last_scrub_stamp":"2024-05-30T17:13:43.750005+0000","last_deep_scrub":"265'36","last_deep_scrub_stamp":"2024-05-30T17:13:43.750005+0000","last_clean_scrub_stamp":"2024-05-30T17:13:43.750005+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:20:47.390125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":177,"num_read_kb":125,"num_write":75,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":40,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:38.985535+0000","last_change":"2024-05-30T17:14:38.985674+0000","last_active":"2024-05-30T17:14:38.985535+0000","last_peered":"2024-05-30T17:14:38.985535+0000","last_clean":"2024-05-30T17:14:38.985535+0000","last_became_active":"2024-05-30T16:34:55.862709+0000","last_became_peered":"2024-05-30T16:34:55.862709+0000","last_unstale":"2024-05-30T17:14:38.985535+0000","last_undegraded":"2024-05-30T17:14:38.985535+0000","last_fullsized":"2024-05-30T17:14:38.985535+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:38.985285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:38.985285+0000","last_clean_scrub_stamp":"2024-05-30T17:14:38.985285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T05:10:37.092995+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"265'3","reported_seq":61,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:18.813164+0000","last_change":"2024-05-30T17:14:18.813250+0000","last_active":"2024-05-30T17:14:18.813164+0000","last_peered":"2024-05-30T17:14:18.813164+0000","last_clean":"2024-05-30T17:14:18.813164+0000","last_became_active":"2024-05-30T16:17:35.240472+0000","last_became_peered":"2024-05-30T16:17:35.240472+0000","last_unstale":"2024-05-30T17:14:18.813164+0000","last_undegraded":"2024-05-30T17:14:18.813164+0000","last_fullsized":"2024-05-30T17:14:18.813164+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-05-30T17:14:18.812962+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-05-30T17:14:18.812962+0000","last_clean_scrub_stamp":"2024-05-30T17:14:18.812962+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:06:37.629144+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"265'6","reported_seq":74,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.031484+0000","last_change":"2024-05-30T17:13:44.031484+0000","last_active":"2024-05-30T17:13:44.031484+0000","last_peered":"2024-05-30T17:13:44.031484+0000","last_clean":"2024-05-30T17:13:44.031484+0000","last_became_active":"2024-05-30T16:17:31.187958+0000","last_became_peered":"2024-05-30T16:17:31.187958+0000","last_unstale":"2024-05-30T17:13:44.031484+0000","last_undegraded":"2024-05-30T17:13:44.031484+0000","last_fullsized":"2024-05-30T17:13:44.031484+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'6","last_scrub_stamp":"2024-05-30T17:13:44.031239+0000","last_deep_scrub":"265'6","last_deep_scrub_stamp":"2024-05-30T17:13:44.031239+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.031239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:29:30.262819+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":10240,"num_write":16,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"265'18","reported_seq":121,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.918092+0000","last_change":"2024-05-30T17:13:44.918156+0000","last_active":"2024-05-30T17:13:44.918092+0000","last_peered":"2024-05-30T17:13:44.918092+0000","last_clean":"2024-05-30T17:13:44.918092+0000","last_became_active":"2024-05-30T16:12:50.823090+0000","last_became_peered":"2024-05-30T16:12:50.823090+0000","last_unstale":"2024-05-30T17:13:44.918092+0000","last_undegraded":"2024-05-30T17:13:44.918092+0000","last_fullsized":"2024-05-30T17:13:44.918092+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'18","last_scrub_stamp":"2024-05-30T17:13:44.917823+0000","last_deep_scrub":"265'18","last_deep_scrub_stamp":"2024-05-30T17:13:44.917823+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.917823+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:31:54.618562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":69,"num_write":35,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":40,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:39.690982+0000","last_change":"2024-05-30T17:14:39.690982+0000","last_active":"2024-05-30T17:14:39.690982+0000","last_peered":"2024-05-30T17:14:39.690982+0000","last_clean":"2024-05-30T17:14:39.690982+0000","last_became_active":"2024-05-30T16:34:55.890688+0000","last_became_peered":"2024-05-30T16:34:55.890688+0000","last_unstale":"2024-05-30T17:14:39.690982+0000","last_undegraded":"2024-05-30T17:14:39.690982+0000","last_fullsized":"2024-05-30T17:14:39.690982+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:39.690715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:39.690715+0000","last_clean_scrub_stamp":"2024-05-30T17:14:39.690715+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:31:19.278084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"265'9","reported_seq":77,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:28.055219+0000","last_change":"2024-05-30T17:14:28.055289+0000","last_active":"2024-05-30T17:14:28.055219+0000","last_peered":"2024-05-30T17:14:28.055219+0000","last_clean":"2024-05-30T17:14:28.055219+0000","last_became_active":"2024-05-30T16:17:35.237031+0000","last_became_peered":"2024-05-30T16:17:35.237031+0000","last_unstale":"2024-05-30T17:14:28.055219+0000","last_undegraded":"2024-05-30T17:14:28.055219+0000","last_fullsized":"2024-05-30T17:14:28.055219+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'9","last_scrub_stamp":"2024-05-30T17:14:28.055000+0000","last_deep_scrub":"265'9","last_deep_scrub_stamp":"2024-05-30T17:14:28.055000+0000","last_clean_scrub_stamp":"2024-05-30T17:14:28.055000+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:27:54.245671+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":20482,"num_write":17,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"265'6","reported_seq":76,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.818137+0000","last_change":"2024-05-30T17:13:48.818281+0000","last_active":"2024-05-30T17:13:48.818137+0000","last_peered":"2024-05-30T17:13:48.818137+0000","last_clean":"2024-05-30T17:13:48.818137+0000","last_became_active":"2024-05-30T16:17:31.203871+0000","last_became_peered":"2024-05-30T16:17:31.203871+0000","last_unstale":"2024-05-30T17:13:48.818137+0000","last_undegraded":"2024-05-30T17:13:48.818137+0000","last_fullsized":"2024-05-30T17:13:48.818137+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'6","last_scrub_stamp":"2024-05-30T17:13:48.817847+0000","last_deep_scrub":"265'6","last_deep_scrub_stamp":"2024-05-30T17:13:48.817847+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.817847+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:26:36.664287+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":4098,"num_write":9,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"265'24","reported_seq":142,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.471273+0000","last_change":"2024-05-30T17:13:51.471404+0000","last_active":"2024-05-30T17:13:51.471273+0000","last_peered":"2024-05-30T17:13:51.471273+0000","last_clean":"2024-05-30T17:13:51.471273+0000","last_became_active":"2024-05-30T16:12:50.827078+0000","last_became_peered":"2024-05-30T16:12:50.827078+0000","last_unstale":"2024-05-30T17:13:51.471273+0000","last_undegraded":"2024-05-30T17:13:51.471273+0000","last_fullsized":"2024-05-30T17:13:51.471273+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'24","last_scrub_stamp":"2024-05-30T17:13:51.471027+0000","last_deep_scrub":"265'24","last_deep_scrub_stamp":"2024-05-30T17:13:51.471027+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.471027+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:41:56.836293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":93,"num_write":46,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"265'165","reported_seq":785,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:25.765745+0000","last_change":"2024-05-30T17:14:25.765895+0000","last_active":"2024-05-30T17:14:25.765745+0000","last_peered":"2024-05-30T17:14:25.765745+0000","last_clean":"2024-05-30T17:14:25.765745+0000","last_became_active":"2024-05-30T15:54:16.661334+0000","last_became_peered":"2024-05-30T15:54:16.661334+0000","last_unstale":"2024-05-30T17:14:25.765745+0000","last_undegraded":"2024-05-30T17:14:25.765745+0000","last_fullsized":"2024-05-30T17:14:25.765745+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'165","last_scrub_stamp":"2024-05-30T17:14:25.765539+0000","last_deep_scrub":"265'165","last_deep_scrub_stamp":"2024-05-30T17:14:25.765539+0000","last_clean_scrub_stamp":"2024-05-30T17:14:25.765539+0000","objects_scrubbed":4,"log_size":65,"log_dups_size":100,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:26:13.701660+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2159792,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":347,"num_read_kb":7638,"num_write":1029,"num_write_kb":24692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"265'441","reported_seq":2084,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:06.288519+0000","last_change":"2024-05-30T17:14:06.288519+0000","last_active":"2024-05-30T17:14:06.288519+0000","last_peered":"2024-05-30T17:14:06.288519+0000","last_clean":"2024-05-30T17:14:06.288519+0000","last_became_active":"2024-05-30T15:54:18.669016+0000","last_became_peered":"2024-05-30T15:54:18.669016+0000","last_unstale":"2024-05-30T17:14:06.288519+0000","last_undegraded":"2024-05-30T17:14:06.288519+0000","last_fullsized":"2024-05-30T17:14:06.288519+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'441","last_scrub_stamp":"2024-05-30T17:14:06.288225+0000","last_deep_scrub":"265'441","last_deep_scrub_stamp":"2024-05-30T17:14:06.288225+0000","last_clean_scrub_stamp":"2024-05-30T17:14:06.288225+0000","objects_scrubbed":5,"log_size":41,"log_dups_size":400,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:34:29.289562+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2479,"num_read_kb":2366,"num_write":506,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"265'234","reported_seq":898,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:15.986498+0000","last_change":"2024-05-30T17:14:15.986759+0000","last_active":"2024-05-30T17:14:15.986498+0000","last_peered":"2024-05-30T17:14:15.986498+0000","last_clean":"2024-05-30T17:14:15.986498+0000","last_became_active":"2024-05-30T15:54:16.652724+0000","last_became_peered":"2024-05-30T15:54:16.652724+0000","last_unstale":"2024-05-30T17:14:15.986498+0000","last_undegraded":"2024-05-30T17:14:15.986498+0000","last_fullsized":"2024-05-30T17:14:15.986498+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'234","last_scrub_stamp":"2024-05-30T17:14:15.986234+0000","last_deep_scrub":"265'234","last_deep_scrub_stamp":"2024-05-30T17:14:15.986234+0000","last_clean_scrub_stamp":"2024-05-30T17:14:15.986234+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":200,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:24:51.351668+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6513950,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":554,"num_read_kb":8631,"num_write":1281,"num_write_kb":24636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"265'762","reported_seq":2127,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:15.778412+0000","last_change":"2024-05-30T17:14:15.778412+0000","last_active":"2024-05-30T17:14:15.778412+0000","last_peered":"2024-05-30T17:14:15.778412+0000","last_clean":"2024-05-30T17:14:15.778412+0000","last_became_active":"2024-05-30T15:54:18.648675+0000","last_became_peered":"2024-05-30T15:54:18.648675+0000","last_unstale":"2024-05-30T17:14:15.778412+0000","last_undegraded":"2024-05-30T17:14:15.778412+0000","last_fullsized":"2024-05-30T17:14:15.778412+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'762","last_scrub_stamp":"2024-05-30T17:14:15.778159+0000","last_deep_scrub":"265'762","last_deep_scrub_stamp":"2024-05-30T17:14:15.778159+0000","last_clean_scrub_stamp":"2024-05-30T17:14:15.778159+0000","objects_scrubbed":4,"log_size":62,"log_dups_size":700,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:29:08.620212+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3292,"num_read_kb":2905,"num_write":1031,"num_write_kb":415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"265'208","reported_seq":849,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:26.856702+0000","last_change":"2024-05-30T17:14:26.856841+0000","last_active":"2024-05-30T17:14:26.856702+0000","last_peered":"2024-05-30T17:14:26.856702+0000","last_clean":"2024-05-30T17:14:26.856702+0000","last_became_active":"2024-05-30T15:54:16.633011+0000","last_became_peered":"2024-05-30T15:54:16.633011+0000","last_unstale":"2024-05-30T17:14:26.856702+0000","last_undegraded":"2024-05-30T17:14:26.856702+0000","last_fullsized":"2024-05-30T17:14:26.856702+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'208","last_scrub_stamp":"2024-05-30T17:14:26.856421+0000","last_deep_scrub":"265'208","last_deep_scrub_stamp":"2024-05-30T17:14:26.856421+0000","last_clean_scrub_stamp":"2024-05-30T17:14:26.856421+0000","objects_scrubbed":9,"log_size":108,"log_dups_size":100,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:28:47.270416+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11855586,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":404,"num_read_kb":43860,"num_write":998,"num_write_kb":74051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"265'747","reported_seq":2155,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:14.074411+0000","last_change":"2024-05-30T17:14:14.074411+0000","last_active":"2024-05-30T17:14:14.074411+0000","last_peered":"2024-05-30T17:14:14.074411+0000","last_clean":"2024-05-30T17:14:14.074411+0000","last_became_active":"2024-05-30T15:54:18.649608+0000","last_became_peered":"2024-05-30T15:54:18.649608+0000","last_unstale":"2024-05-30T17:14:14.074411+0000","last_undegraded":"2024-05-30T17:14:14.074411+0000","last_fullsized":"2024-05-30T17:14:14.074411+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'747","last_scrub_stamp":"2024-05-30T17:14:14.074153+0000","last_deep_scrub":"265'747","last_deep_scrub_stamp":"2024-05-30T17:14:14.074153+0000","last_clean_scrub_stamp":"2024-05-30T17:14:14.074153+0000","objects_scrubbed":1,"log_size":47,"log_dups_size":700,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:05:55.305461+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3275,"num_read_kb":2909,"num_write":996,"num_write_kb":378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"265'154","reported_seq":802,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:22.300200+0000","last_change":"2024-05-30T17:14:22.300200+0000","last_active":"2024-05-30T17:14:22.300200+0000","last_peered":"2024-05-30T17:14:22.300200+0000","last_clean":"2024-05-30T17:14:22.300200+0000","last_became_active":"2024-05-30T15:54:16.652315+0000","last_became_peered":"2024-05-30T15:54:16.652315+0000","last_unstale":"2024-05-30T17:14:22.300200+0000","last_undegraded":"2024-05-30T17:14:22.300200+0000","last_fullsized":"2024-05-30T17:14:22.300200+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'154","last_scrub_stamp":"2024-05-30T17:14:22.299903+0000","last_deep_scrub":"265'154","last_deep_scrub_stamp":"2024-05-30T17:14:22.299903+0000","last_clean_scrub_stamp":"2024-05-30T17:14:22.299903+0000","objects_scrubbed":3,"log_size":54,"log_dups_size":100,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:34:02.246778+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1250479,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":437,"num_read_kb":3484,"num_write":911,"num_write_kb":6490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"265'490","reported_seq":1975,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:12.850064+0000","last_change":"2024-05-30T17:14:12.850064+0000","last_active":"2024-05-30T17:14:12.850064+0000","last_peered":"2024-05-30T17:14:12.850064+0000","last_clean":"2024-05-30T17:14:12.850064+0000","last_became_active":"2024-05-30T15:54:18.652451+0000","last_became_peered":"2024-05-30T15:54:18.652451+0000","last_unstale":"2024-05-30T17:14:12.850064+0000","last_undegraded":"2024-05-30T17:14:12.850064+0000","last_fullsized":"2024-05-30T17:14:12.850064+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'490","last_scrub_stamp":"2024-05-30T17:14:12.849644+0000","last_deep_scrub":"265'490","last_deep_scrub_stamp":"2024-05-30T17:14:12.849644+0000","last_clean_scrub_stamp":"2024-05-30T17:14:12.849644+0000","objects_scrubbed":3,"log_size":90,"log_dups_size":400,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:33:13.009731+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2387,"num_read_kb":2188,"num_write":616,"num_write_kb":168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":505,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"265'245","reported_seq":969,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:31.797437+0000","last_change":"2024-05-30T17:14:31.797533+0000","last_active":"2024-05-30T17:14:31.797437+0000","last_peered":"2024-05-30T17:14:31.797437+0000","last_clean":"2024-05-30T17:14:31.797437+0000","last_became_active":"2024-05-30T15:54:16.658338+0000","last_became_peered":"2024-05-30T15:54:16.658338+0000","last_unstale":"2024-05-30T17:14:31.797437+0000","last_undegraded":"2024-05-30T17:14:31.797437+0000","last_fullsized":"2024-05-30T17:14:31.797437+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'245","last_scrub_stamp":"2024-05-30T17:14:31.797211+0000","last_deep_scrub":"265'245","last_deep_scrub_stamp":"2024-05-30T17:14:31.797211+0000","last_clean_scrub_stamp":"2024-05-30T17:14:31.797211+0000","objects_scrubbed":5,"log_size":45,"log_dups_size":200,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:20:44.257811+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291491,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":830,"num_read_kb":24681,"num_write":1246,"num_write_kb":34031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"265'378","reported_seq":1880,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:05.719014+0000","last_change":"2024-05-30T17:14:05.719014+0000","last_active":"2024-05-30T17:14:05.719014+0000","last_peered":"2024-05-30T17:14:05.719014+0000","last_clean":"2024-05-30T17:14:05.719014+0000","last_became_active":"2024-05-30T15:54:18.669017+0000","last_became_peered":"2024-05-30T15:54:18.669017+0000","last_unstale":"2024-05-30T17:14:05.719014+0000","last_undegraded":"2024-05-30T17:14:05.719014+0000","last_fullsized":"2024-05-30T17:14:05.719014+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'378","last_scrub_stamp":"2024-05-30T17:14:05.718728+0000","last_deep_scrub":"265'378","last_deep_scrub_stamp":"2024-05-30T17:14:05.718728+0000","last_clean_scrub_stamp":"2024-05-30T17:14:05.718728+0000","objects_scrubbed":8,"log_size":78,"log_dups_size":300,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:31:23.767712+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1986,"num_read_kb":1906,"num_write":419,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":465,"num_omap_keys":3,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"265'196","reported_seq":831,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:15.302437+0000","last_change":"2024-05-30T17:14:15.302437+0000","last_active":"2024-05-30T17:14:15.302437+0000","last_peered":"2024-05-30T17:14:15.302437+0000","last_clean":"2024-05-30T17:14:15.302437+0000","last_became_active":"2024-05-30T15:54:16.671120+0000","last_became_peered":"2024-05-30T15:54:16.671120+0000","last_unstale":"2024-05-30T17:14:15.302437+0000","last_undegraded":"2024-05-30T17:14:15.302437+0000","last_fullsized":"2024-05-30T17:14:15.302437+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'196","last_scrub_stamp":"2024-05-30T17:14:15.302156+0000","last_deep_scrub":"265'196","last_deep_scrub_stamp":"2024-05-30T17:14:15.302156+0000","last_clean_scrub_stamp":"2024-05-30T17:14:15.302156+0000","objects_scrubbed":2,"log_size":96,"log_dups_size":100,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:02:53.975403+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":81,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":438,"num_read_kb":4718,"num_write":1101,"num_write_kb":20583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"265'727","reported_seq":2212,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:16.031885+0000","last_change":"2024-05-30T17:14:16.032031+0000","last_active":"2024-05-30T17:14:16.031885+0000","last_peered":"2024-05-30T17:14:16.031885+0000","last_clean":"2024-05-30T17:14:16.031885+0000","last_became_active":"2024-05-30T15:54:18.650543+0000","last_became_peered":"2024-05-30T15:54:18.650543+0000","last_unstale":"2024-05-30T17:14:16.031885+0000","last_undegraded":"2024-05-30T17:14:16.031885+0000","last_fullsized":"2024-05-30T17:14:16.031885+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'727","last_scrub_stamp":"2024-05-30T17:14:16.031640+0000","last_deep_scrub":"265'727","last_deep_scrub_stamp":"2024-05-30T17:14:16.031640+0000","last_clean_scrub_stamp":"2024-05-30T17:14:16.031640+0000","objects_scrubbed":2,"log_size":27,"log_dups_size":700,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:30:02.474759+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3282,"num_read_kb":2952,"num_write":973,"num_write_kb":356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"265'192","reported_seq":842,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:17.822847+0000","last_change":"2024-05-30T17:14:17.822942+0000","last_active":"2024-05-30T17:14:17.822847+0000","last_peered":"2024-05-30T17:14:17.822847+0000","last_clean":"2024-05-30T17:14:17.822847+0000","last_became_active":"2024-05-30T15:54:16.678799+0000","last_became_peered":"2024-05-30T15:54:16.678799+0000","last_unstale":"2024-05-30T17:14:17.822847+0000","last_undegraded":"2024-05-30T17:14:17.822847+0000","last_fullsized":"2024-05-30T17:14:17.822847+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'192","last_scrub_stamp":"2024-05-30T17:14:17.822619+0000","last_deep_scrub":"265'192","last_deep_scrub_stamp":"2024-05-30T17:14:17.822619+0000","last_clean_scrub_stamp":"2024-05-30T17:14:17.822619+0000","objects_scrubbed":4,"log_size":92,"log_dups_size":100,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:01:36.730473+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":491377,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":467,"num_read_kb":33008,"num_write":1090,"num_write_kb":27021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"265'851","reported_seq":2549,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:07.288471+0000","last_change":"2024-05-30T17:14:07.288543+0000","last_active":"2024-05-30T17:14:07.288471+0000","last_peered":"2024-05-30T17:14:07.288471+0000","last_clean":"2024-05-30T17:14:07.288471+0000","last_became_active":"2024-05-30T15:54:18.656475+0000","last_became_peered":"2024-05-30T15:54:18.656475+0000","last_unstale":"2024-05-30T17:14:07.288471+0000","last_undegraded":"2024-05-30T17:14:07.288471+0000","last_fullsized":"2024-05-30T17:14:07.288471+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'851","last_scrub_stamp":"2024-05-30T17:14:07.288276+0000","last_deep_scrub":"265'851","last_deep_scrub_stamp":"2024-05-30T17:14:07.288276+0000","last_clean_scrub_stamp":"2024-05-30T17:14:07.288276+0000","objects_scrubbed":1,"log_size":51,"log_dups_size":800,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:53:16.054837+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4088,"num_read_kb":3692,"num_write":1166,"num_write_kb":472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"265'140","reported_seq":746,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:34.449595+0000","last_change":"2024-05-30T17:14:34.449675+0000","last_active":"2024-05-30T17:14:34.449595+0000","last_peered":"2024-05-30T17:14:34.449595+0000","last_clean":"2024-05-30T17:14:34.449595+0000","last_became_active":"2024-05-30T15:54:16.669095+0000","last_became_peered":"2024-05-30T15:54:16.669095+0000","last_unstale":"2024-05-30T17:14:34.449595+0000","last_undegraded":"2024-05-30T17:14:34.449595+0000","last_fullsized":"2024-05-30T17:14:34.449595+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'140","last_scrub_stamp":"2024-05-30T17:14:34.449388+0000","last_deep_scrub":"265'140","last_deep_scrub_stamp":"2024-05-30T17:14:34.449388+0000","last_clean_scrub_stamp":"2024-05-30T17:14:34.449388+0000","objects_scrubbed":2,"log_size":40,"log_dups_size":100,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:07:00.707956+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5243043,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":303,"num_read_kb":17751,"num_write":817,"num_write_kb":30378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"265'373","reported_seq":1803,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:05.355282+0000","last_change":"2024-05-30T17:14:05.355364+0000","last_active":"2024-05-30T17:14:05.355282+0000","last_peered":"2024-05-30T17:14:05.355282+0000","last_clean":"2024-05-30T17:14:05.355282+0000","last_became_active":"2024-05-30T15:54:18.653189+0000","last_became_peered":"2024-05-30T15:54:18.653189+0000","last_unstale":"2024-05-30T17:14:05.355282+0000","last_undegraded":"2024-05-30T17:14:05.355282+0000","last_fullsized":"2024-05-30T17:14:05.355282+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'373","last_scrub_stamp":"2024-05-30T17:14:05.355099+0000","last_deep_scrub":"265'373","last_deep_scrub_stamp":"2024-05-30T17:14:05.355099+0000","last_clean_scrub_stamp":"2024-05-30T17:14:05.355099+0000","objects_scrubbed":4,"log_size":73,"log_dups_size":300,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:26:58.416740+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1969,"num_read_kb":1855,"num_write":435,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"265'340","reported_seq":1778,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:12.082673+0000","last_change":"2024-05-30T17:14:12.082843+0000","last_active":"2024-05-30T17:14:12.082673+0000","last_peered":"2024-05-30T17:14:12.082673+0000","last_clean":"2024-05-30T17:14:12.082673+0000","last_became_active":"2024-05-30T15:54:18.649994+0000","last_became_peered":"2024-05-30T15:54:18.649994+0000","last_unstale":"2024-05-30T17:14:12.082673+0000","last_undegraded":"2024-05-30T17:14:12.082673+0000","last_fullsized":"2024-05-30T17:14:12.082673+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'340","last_scrub_stamp":"2024-05-30T17:14:12.082448+0000","last_deep_scrub":"265'340","last_deep_scrub_stamp":"2024-05-30T17:14:12.082448+0000","last_clean_scrub_stamp":"2024-05-30T17:14:12.082448+0000","objects_scrubbed":4,"log_size":40,"log_dups_size":300,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:35:49.760153+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1976,"num_read_kb":1893,"num_write":388,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":249,"num_omap_keys":1,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"265'212","reported_seq":887,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:30.794777+0000","last_change":"2024-05-30T17:14:30.794874+0000","last_active":"2024-05-30T17:14:30.794777+0000","last_peered":"2024-05-30T17:14:30.794777+0000","last_clean":"2024-05-30T17:14:30.794777+0000","last_became_active":"2024-05-30T15:54:16.649872+0000","last_became_peered":"2024-05-30T15:54:16.649872+0000","last_unstale":"2024-05-30T17:14:30.794777+0000","last_undegraded":"2024-05-30T17:14:30.794777+0000","last_fullsized":"2024-05-30T17:14:30.794777+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'212","last_scrub_stamp":"2024-05-30T17:14:30.794580+0000","last_deep_scrub":"265'212","last_deep_scrub_stamp":"2024-05-30T17:14:30.794580+0000","last_clean_scrub_stamp":"2024-05-30T17:14:30.794580+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:09:42.032254+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33349,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":510,"num_read_kb":59018,"num_write":1166,"num_write_kb":23092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"265'1526","reported_seq":3141,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:13.760005+0000","last_change":"2024-05-30T17:14:13.760080+0000","last_active":"2024-05-30T17:14:13.760005+0000","last_peered":"2024-05-30T17:14:13.760005+0000","last_clean":"2024-05-30T17:14:13.760005+0000","last_became_active":"2024-05-30T15:54:18.648860+0000","last_became_peered":"2024-05-30T15:54:18.648860+0000","last_unstale":"2024-05-30T17:14:13.760005+0000","last_undegraded":"2024-05-30T17:14:13.760005+0000","last_fullsized":"2024-05-30T17:14:13.760005+0000","mapping_epoch":24,"log_start":"265'1500","ondisk_log_start":"265'1500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'1526","last_scrub_stamp":"2024-05-30T17:14:13.759812+0000","last_deep_scrub":"265'1526","last_deep_scrub_stamp":"2024-05-30T17:14:13.759812+0000","last_clean_scrub_stamp":"2024-05-30T17:14:13.759812+0000","objects_scrubbed":1,"log_size":26,"log_dups_size":1500,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:47:16.863579+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6132,"num_read_kb":5297,"num_write":2138,"num_write_kb":912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"265'221","reported_seq":1093,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:13.810620+0000","last_change":"2024-05-30T17:14:13.810876+0000","last_active":"2024-05-30T17:14:13.810620+0000","last_peered":"2024-05-30T17:14:13.810620+0000","last_clean":"2024-05-30T17:14:13.810620+0000","last_became_active":"2024-05-30T15:54:16.686847+0000","last_became_peered":"2024-05-30T15:54:16.686847+0000","last_unstale":"2024-05-30T17:14:13.810620+0000","last_undegraded":"2024-05-30T17:14:13.810620+0000","last_fullsized":"2024-05-30T17:14:13.810620+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'221","last_scrub_stamp":"2024-05-30T17:14:13.810368+0000","last_deep_scrub":"265'221","last_deep_scrub_stamp":"2024-05-30T17:14:13.810368+0000","last_clean_scrub_stamp":"2024-05-30T17:14:13.810368+0000","objects_scrubbed":5,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:53:55.581906+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2728232,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":925,"num_read_kb":24537,"num_write":1233,"num_write_kb":54861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"265'1169","reported_seq":3018,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:04.292081+0000","last_change":"2024-05-30T17:14:04.292155+0000","last_active":"2024-05-30T17:14:04.292081+0000","last_peered":"2024-05-30T17:14:04.292081+0000","last_clean":"2024-05-30T17:14:04.292081+0000","last_became_active":"2024-05-30T15:54:18.665302+0000","last_became_peered":"2024-05-30T15:54:18.665302+0000","last_unstale":"2024-05-30T17:14:04.292081+0000","last_undegraded":"2024-05-30T17:14:04.292081+0000","last_fullsized":"2024-05-30T17:14:04.292081+0000","mapping_epoch":24,"log_start":"265'1100","ondisk_log_start":"265'1100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'1169","last_scrub_stamp":"2024-05-30T17:14:04.291893+0000","last_deep_scrub":"265'1169","last_deep_scrub_stamp":"2024-05-30T17:14:04.291893+0000","last_clean_scrub_stamp":"2024-05-30T17:14:04.291893+0000","objects_scrubbed":5,"log_size":69,"log_dups_size":1100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:33:21.886845+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5292,"num_read_kb":4714,"num_write":1593,"num_write_kb":642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3544,"num_omap_keys":14,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"265'197","reported_seq":928,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:08.780887+0000","last_change":"2024-05-30T17:14:08.781004+0000","last_active":"2024-05-30T17:14:08.780887+0000","last_peered":"2024-05-30T17:14:08.780887+0000","last_clean":"2024-05-30T17:14:08.780887+0000","last_became_active":"2024-05-30T15:54:16.668438+0000","last_became_peered":"2024-05-30T15:54:16.668438+0000","last_unstale":"2024-05-30T17:14:08.780887+0000","last_undegraded":"2024-05-30T17:14:08.780887+0000","last_fullsized":"2024-05-30T17:14:08.780887+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'197","last_scrub_stamp":"2024-05-30T17:14:08.780668+0000","last_deep_scrub":"265'197","last_deep_scrub_stamp":"2024-05-30T17:14:08.780668+0000","last_clean_scrub_stamp":"2024-05-30T17:14:08.780668+0000","objects_scrubbed":5,"log_size":97,"log_dups_size":100,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:59:06.022425+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":918,"num_read_kb":8276,"num_write":1173,"num_write_kb":35310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"265'772","reported_seq":2296,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:04.332160+0000","last_change":"2024-05-30T17:14:04.332160+0000","last_active":"2024-05-30T17:14:04.332160+0000","last_peered":"2024-05-30T17:14:04.332160+0000","last_clean":"2024-05-30T17:14:04.332160+0000","last_became_active":"2024-05-30T15:54:18.651197+0000","last_became_peered":"2024-05-30T15:54:18.651197+0000","last_unstale":"2024-05-30T17:14:04.332160+0000","last_undegraded":"2024-05-30T17:14:04.332160+0000","last_fullsized":"2024-05-30T17:14:04.332160+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'772","last_scrub_stamp":"2024-05-30T17:14:04.331893+0000","last_deep_scrub":"265'772","last_deep_scrub_stamp":"2024-05-30T17:14:04.331893+0000","last_clean_scrub_stamp":"2024-05-30T17:14:04.331893+0000","objects_scrubbed":4,"log_size":72,"log_dups_size":700,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:54:58.149438+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3544,"num_read_kb":3171,"num_write":1051,"num_write_kb":404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"265'190","reported_seq":852,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:20.131165+0000","last_change":"2024-05-30T17:14:20.131261+0000","last_active":"2024-05-30T17:14:20.131165+0000","last_peered":"2024-05-30T17:14:20.131165+0000","last_clean":"2024-05-30T17:14:20.131165+0000","last_became_active":"2024-05-30T15:54:16.676703+0000","last_became_peered":"2024-05-30T15:54:16.676703+0000","last_unstale":"2024-05-30T17:14:20.131165+0000","last_undegraded":"2024-05-30T17:14:20.131165+0000","last_fullsized":"2024-05-30T17:14:20.131165+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'190","last_scrub_stamp":"2024-05-30T17:14:20.130939+0000","last_deep_scrub":"265'190","last_deep_scrub_stamp":"2024-05-30T17:14:20.130939+0000","last_clean_scrub_stamp":"2024-05-30T17:14:20.130939+0000","objects_scrubbed":3,"log_size":90,"log_dups_size":100,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:16:27.631887+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":366,"num_read_kb":73313,"num_write":1062,"num_write_kb":47694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"265'376","reported_seq":1838,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:03.319819+0000","last_change":"2024-05-30T17:14:03.319819+0000","last_active":"2024-05-30T17:14:03.319819+0000","last_peered":"2024-05-30T17:14:03.319819+0000","last_clean":"2024-05-30T17:14:03.319819+0000","last_became_active":"2024-05-30T15:54:18.650255+0000","last_became_peered":"2024-05-30T15:54:18.650255+0000","last_unstale":"2024-05-30T17:14:03.319819+0000","last_undegraded":"2024-05-30T17:14:03.319819+0000","last_fullsized":"2024-05-30T17:14:03.319819+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'376","last_scrub_stamp":"2024-05-30T17:14:03.319457+0000","last_deep_scrub":"265'376","last_deep_scrub_stamp":"2024-05-30T17:14:03.319457+0000","last_clean_scrub_stamp":"2024-05-30T17:14:03.319457+0000","objects_scrubbed":4,"log_size":76,"log_dups_size":300,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:24:42.352764+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1969,"num_read_kb":1871,"num_write":435,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1010,"num_omap_keys":4,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"265'203","reported_seq":864,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:32.402616+0000","last_change":"2024-05-30T17:14:32.402720+0000","last_active":"2024-05-30T17:14:32.402616+0000","last_peered":"2024-05-30T17:14:32.402616+0000","last_clean":"2024-05-30T17:14:32.402616+0000","last_became_active":"2024-05-30T15:54:16.665344+0000","last_became_peered":"2024-05-30T15:54:16.665344+0000","last_unstale":"2024-05-30T17:14:32.402616+0000","last_undegraded":"2024-05-30T17:14:32.402616+0000","last_fullsized":"2024-05-30T17:14:32.402616+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'203","last_scrub_stamp":"2024-05-30T17:14:32.402407+0000","last_deep_scrub":"265'203","last_deep_scrub_stamp":"2024-05-30T17:14:32.402407+0000","last_clean_scrub_stamp":"2024-05-30T17:14:32.402407+0000","objects_scrubbed":5,"log_size":103,"log_dups_size":100,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:47:10.414479+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6847332,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":477,"num_read_kb":7750,"num_write":1077,"num_write_kb":36544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"265'376","reported_seq":1811,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:24.134733+0000","last_change":"2024-05-30T17:14:24.134802+0000","last_active":"2024-05-30T17:14:24.134733+0000","last_peered":"2024-05-30T17:14:24.134733+0000","last_clean":"2024-05-30T17:14:24.134733+0000","last_became_active":"2024-05-30T15:54:18.643825+0000","last_became_peered":"2024-05-30T15:54:18.643825+0000","last_unstale":"2024-05-30T17:14:24.134733+0000","last_undegraded":"2024-05-30T17:14:24.134733+0000","last_fullsized":"2024-05-30T17:14:24.134733+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'376","last_scrub_stamp":"2024-05-30T17:14:24.134539+0000","last_deep_scrub":"265'376","last_deep_scrub_stamp":"2024-05-30T17:14:24.134539+0000","last_clean_scrub_stamp":"2024-05-30T17:14:24.134539+0000","objects_scrubbed":1,"log_size":76,"log_dups_size":300,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:48:05.206064+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1936,"num_read_kb":1833,"num_write":434,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"265'238","reported_seq":919,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:27.802082+0000","last_change":"2024-05-30T17:14:27.802182+0000","last_active":"2024-05-30T17:14:27.802082+0000","last_peered":"2024-05-30T17:14:27.802082+0000","last_clean":"2024-05-30T17:14:27.802082+0000","last_became_active":"2024-05-30T15:54:16.637928+0000","last_became_peered":"2024-05-30T15:54:16.637928+0000","last_unstale":"2024-05-30T17:14:27.802082+0000","last_undegraded":"2024-05-30T17:14:27.802082+0000","last_fullsized":"2024-05-30T17:14:27.802082+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'238","last_scrub_stamp":"2024-05-30T17:14:27.801850+0000","last_deep_scrub":"265'238","last_deep_scrub_stamp":"2024-05-30T17:14:27.801850+0000","last_clean_scrub_stamp":"2024-05-30T17:14:27.801850+0000","objects_scrubbed":3,"log_size":38,"log_dups_size":200,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:29:08.005106+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079922,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":526,"num_read_kb":65823,"num_write":1181,"num_write_kb":32044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"265'180","reported_seq":714,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.837308+0000","last_change":"2024-05-30T17:13:47.837308+0000","last_active":"2024-05-30T17:13:47.837308+0000","last_peered":"2024-05-30T17:13:47.837308+0000","last_clean":"2024-05-30T17:13:47.837308+0000","last_became_active":"2024-05-30T15:54:26.700957+0000","last_became_peered":"2024-05-30T15:54:26.700957+0000","last_unstale":"2024-05-30T17:13:47.837308+0000","last_undegraded":"2024-05-30T17:13:47.837308+0000","last_fullsized":"2024-05-30T17:13:47.837308+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'180","last_scrub_stamp":"2024-05-30T17:13:47.837040+0000","last_deep_scrub":"265'180","last_deep_scrub_stamp":"2024-05-30T17:13:47.837040+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.837040+0000","objects_scrubbed":5,"log_size":80,"log_dups_size":100,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:49:37.253715+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":270,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":132,"num_read_kb":126,"num_write":386,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"265'496","reported_seq":2033,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:18.368033+0000","last_change":"2024-05-30T17:14:18.368097+0000","last_active":"2024-05-30T17:14:18.368033+0000","last_peered":"2024-05-30T17:14:18.368033+0000","last_clean":"2024-05-30T17:14:18.368033+0000","last_became_active":"2024-05-30T15:54:18.652712+0000","last_became_peered":"2024-05-30T15:54:18.652712+0000","last_unstale":"2024-05-30T17:14:18.368033+0000","last_undegraded":"2024-05-30T17:14:18.368033+0000","last_fullsized":"2024-05-30T17:14:18.368033+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'496","last_scrub_stamp":"2024-05-30T17:14:18.367832+0000","last_deep_scrub":"265'496","last_deep_scrub_stamp":"2024-05-30T17:14:18.367832+0000","last_clean_scrub_stamp":"2024-05-30T17:14:18.367832+0000","objects_scrubbed":2,"log_size":96,"log_dups_size":400,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:39:11.375061+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2579,"num_read_kb":2377,"num_write":668,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":148,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"265'190","reported_seq":803,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.106668+0000","last_change":"2024-05-30T17:13:51.106756+0000","last_active":"2024-05-30T17:13:51.106668+0000","last_peered":"2024-05-30T17:13:51.106668+0000","last_clean":"2024-05-30T17:13:51.106668+0000","last_became_active":"2024-05-30T15:55:54.563874+0000","last_became_peered":"2024-05-30T15:55:54.563874+0000","last_unstale":"2024-05-30T17:13:51.106668+0000","last_undegraded":"2024-05-30T17:13:51.106668+0000","last_fullsized":"2024-05-30T17:13:51.106668+0000","mapping_epoch":37,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'190","last_scrub_stamp":"2024-05-30T17:13:51.106465+0000","last_deep_scrub":"265'190","last_deep_scrub_stamp":"2024-05-30T17:13:51.106465+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.106465+0000","objects_scrubbed":2,"log_size":90,"log_dups_size":100,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:21:06.302905+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079905,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":470,"num_read_kb":1589,"num_write":1089,"num_write_kb":27979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":516,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:12.375774+0000","last_change":"2024-05-30T17:14:12.375835+0000","last_active":"2024-05-30T17:14:12.375774+0000","last_peered":"2024-05-30T17:14:12.375774+0000","last_clean":"2024-05-30T17:14:12.375774+0000","last_became_active":"2024-05-30T15:54:20.658937+0000","last_became_peered":"2024-05-30T15:54:20.658937+0000","last_unstale":"2024-05-30T17:14:12.375774+0000","last_undegraded":"2024-05-30T17:14:12.375774+0000","last_fullsized":"2024-05-30T17:14:12.375774+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:12.375577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:12.375577+0000","last_clean_scrub_stamp":"2024-05-30T17:14:12.375577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:14:24.728898+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"265'206","reported_seq":842,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:23.889148+0000","last_change":"2024-05-30T17:14:23.889303+0000","last_active":"2024-05-30T17:14:23.889148+0000","last_peered":"2024-05-30T17:14:23.889148+0000","last_clean":"2024-05-30T17:14:23.889148+0000","last_became_active":"2024-05-30T15:54:22.667496+0000","last_became_peered":"2024-05-30T15:54:22.667496+0000","last_unstale":"2024-05-30T17:14:23.889148+0000","last_undegraded":"2024-05-30T17:14:23.889148+0000","last_fullsized":"2024-05-30T17:14:23.889148+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'206","last_scrub_stamp":"2024-05-30T17:14:23.888907+0000","last_deep_scrub":"265'206","last_deep_scrub_stamp":"2024-05-30T17:14:23.888907+0000","last_clean_scrub_stamp":"2024-05-30T17:14:23.888907+0000","objects_scrubbed":4,"log_size":106,"log_dups_size":100,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:36:09.311361+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6140,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":285,"num_read_kb":355,"num_write":168,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":508,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:06.781850+0000","last_change":"2024-05-30T17:14:06.781850+0000","last_active":"2024-05-30T17:14:06.781850+0000","last_peered":"2024-05-30T17:14:06.781850+0000","last_clean":"2024-05-30T17:14:06.781850+0000","last_became_active":"2024-05-30T15:54:24.674491+0000","last_became_peered":"2024-05-30T15:54:24.674491+0000","last_unstale":"2024-05-30T17:14:06.781850+0000","last_undegraded":"2024-05-30T17:14:06.781850+0000","last_fullsized":"2024-05-30T17:14:06.781850+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:06.781591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:06.781591+0000","last_clean_scrub_stamp":"2024-05-30T17:14:06.781591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:22:20.251425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"265'199","reported_seq":784,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.234943+0000","last_change":"2024-05-30T17:13:53.234943+0000","last_active":"2024-05-30T17:13:53.234943+0000","last_peered":"2024-05-30T17:13:53.234943+0000","last_clean":"2024-05-30T17:13:53.234943+0000","last_became_active":"2024-05-30T15:54:26.686430+0000","last_became_peered":"2024-05-30T15:54:26.686430+0000","last_unstale":"2024-05-30T17:13:53.234943+0000","last_undegraded":"2024-05-30T17:13:53.234943+0000","last_fullsized":"2024-05-30T17:13:53.234943+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'199","last_scrub_stamp":"2024-05-30T17:13:53.234506+0000","last_deep_scrub":"265'199","last_deep_scrub_stamp":"2024-05-30T17:13:53.234506+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.234506+0000","objects_scrubbed":5,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:07:57.530934+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":824,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":238,"num_read_kb":223,"num_write":274,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"265'512","reported_seq":2172,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:03.288156+0000","last_change":"2024-05-30T17:14:03.288156+0000","last_active":"2024-05-30T17:14:03.288156+0000","last_peered":"2024-05-30T17:14:03.288156+0000","last_clean":"2024-05-30T17:14:03.288156+0000","last_became_active":"2024-05-30T15:54:18.661762+0000","last_became_peered":"2024-05-30T15:54:18.661762+0000","last_unstale":"2024-05-30T17:14:03.288156+0000","last_undegraded":"2024-05-30T17:14:03.288156+0000","last_fullsized":"2024-05-30T17:14:03.288156+0000","mapping_epoch":24,"log_start":"265'501","ondisk_log_start":"265'501","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'512","last_scrub_stamp":"2024-05-30T17:14:03.287889+0000","last_deep_scrub":"265'512","last_deep_scrub_stamp":"2024-05-30T17:14:03.287889+0000","last_clean_scrub_stamp":"2024-05-30T17:14:03.287889+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":501,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:21:44.477336+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2657,"num_read_kb":2481,"num_write":627,"num_write_kb":155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"265'210","reported_seq":875,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:28.788740+0000","last_change":"2024-05-30T17:14:28.788833+0000","last_active":"2024-05-30T17:14:28.788740+0000","last_peered":"2024-05-30T17:14:28.788740+0000","last_clean":"2024-05-30T17:14:28.788740+0000","last_became_active":"2024-05-30T15:54:16.639284+0000","last_became_peered":"2024-05-30T15:54:16.639284+0000","last_unstale":"2024-05-30T17:14:28.788740+0000","last_undegraded":"2024-05-30T17:14:28.788740+0000","last_fullsized":"2024-05-30T17:14:28.788740+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'210","last_scrub_stamp":"2024-05-30T17:14:28.788504+0000","last_deep_scrub":"265'210","last_deep_scrub_stamp":"2024-05-30T17:14:28.788504+0000","last_clean_scrub_stamp":"2024-05-30T17:14:28.788504+0000","objects_scrubbed":2,"log_size":110,"log_dups_size":100,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:56:40.804738+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262178,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":445,"num_read_kb":15653,"num_write":1159,"num_write_kb":64581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":519,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:21.409846+0000","last_change":"2024-05-30T17:14:21.409904+0000","last_active":"2024-05-30T17:14:21.409846+0000","last_peered":"2024-05-30T17:14:21.409846+0000","last_clean":"2024-05-30T17:14:21.409846+0000","last_became_active":"2024-05-30T15:54:20.665488+0000","last_became_peered":"2024-05-30T15:54:20.665488+0000","last_unstale":"2024-05-30T17:14:21.409846+0000","last_undegraded":"2024-05-30T17:14:21.409846+0000","last_fullsized":"2024-05-30T17:14:21.409846+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:21.409644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:21.409644+0000","last_clean_scrub_stamp":"2024-05-30T17:14:21.409644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:41:07.021077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"265'105","reported_seq":666,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.970319+0000","last_change":"2024-05-30T17:13:59.970319+0000","last_active":"2024-05-30T17:13:59.970319+0000","last_peered":"2024-05-30T17:13:59.970319+0000","last_clean":"2024-05-30T17:13:59.970319+0000","last_became_active":"2024-05-30T15:54:22.666057+0000","last_became_peered":"2024-05-30T15:54:22.666057+0000","last_unstale":"2024-05-30T17:13:59.970319+0000","last_undegraded":"2024-05-30T17:13:59.970319+0000","last_fullsized":"2024-05-30T17:13:59.970319+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'105","last_scrub_stamp":"2024-05-30T17:13:59.969943+0000","last_deep_scrub":"265'105","last_deep_scrub_stamp":"2024-05-30T17:13:59.969943+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.969943+0000","objects_scrubbed":3,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:45:56.754937+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":113,"num_write":76,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":517,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:52.906758+0000","last_change":"2024-05-30T17:13:52.906758+0000","last_active":"2024-05-30T17:13:52.906758+0000","last_peered":"2024-05-30T17:13:52.906758+0000","last_clean":"2024-05-30T17:13:52.906758+0000","last_became_active":"2024-05-30T15:54:24.686595+0000","last_became_peered":"2024-05-30T15:54:24.686595+0000","last_unstale":"2024-05-30T17:13:52.906758+0000","last_undegraded":"2024-05-30T17:13:52.906758+0000","last_fullsized":"2024-05-30T17:13:52.906758+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:52.906486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:52.906486+0000","last_clean_scrub_stamp":"2024-05-30T17:13:52.906486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:22:22.141892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"265'103","reported_seq":626,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.717354+0000","last_change":"2024-05-30T17:13:53.717354+0000","last_active":"2024-05-30T17:13:53.717354+0000","last_peered":"2024-05-30T17:13:53.717354+0000","last_clean":"2024-05-30T17:13:53.717354+0000","last_became_active":"2024-05-30T15:54:26.688376+0000","last_became_peered":"2024-05-30T15:54:26.688376+0000","last_unstale":"2024-05-30T17:13:53.717354+0000","last_undegraded":"2024-05-30T17:13:53.717354+0000","last_fullsized":"2024-05-30T17:13:53.717354+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'103","last_scrub_stamp":"2024-05-30T17:13:53.717082+0000","last_deep_scrub":"265'103","last_deep_scrub_stamp":"2024-05-30T17:13:53.717082+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.717082+0000","objects_scrubbed":4,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:37:15.362960+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":271,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":103,"num_read_kb":102,"num_write":235,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"265'469","reported_seq":2028,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:09.932800+0000","last_change":"2024-05-30T17:14:09.932883+0000","last_active":"2024-05-30T17:14:09.932800+0000","last_peered":"2024-05-30T17:14:09.932800+0000","last_clean":"2024-05-30T17:14:09.932800+0000","last_became_active":"2024-05-30T15:54:18.657540+0000","last_became_peered":"2024-05-30T15:54:18.657540+0000","last_unstale":"2024-05-30T17:14:09.932800+0000","last_undegraded":"2024-05-30T17:14:09.932800+0000","last_fullsized":"2024-05-30T17:14:09.932800+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'469","last_scrub_stamp":"2024-05-30T17:14:09.932582+0000","last_deep_scrub":"265'469","last_deep_scrub_stamp":"2024-05-30T17:14:09.932582+0000","last_clean_scrub_stamp":"2024-05-30T17:14:09.932582+0000","objects_scrubbed":7,"log_size":69,"log_dups_size":400,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:37:05.257095+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2554,"num_read_kb":2362,"num_write":592,"num_write_kb":196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3556,"num_omap_keys":14,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"265'216","reported_seq":886,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:19.855125+0000","last_change":"2024-05-30T17:14:19.855235+0000","last_active":"2024-05-30T17:14:19.855125+0000","last_peered":"2024-05-30T17:14:19.855125+0000","last_clean":"2024-05-30T17:14:19.855125+0000","last_became_active":"2024-05-30T15:54:16.678659+0000","last_became_peered":"2024-05-30T15:54:16.678659+0000","last_unstale":"2024-05-30T17:14:19.855125+0000","last_undegraded":"2024-05-30T17:14:19.855125+0000","last_fullsized":"2024-05-30T17:14:19.855125+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'216","last_scrub_stamp":"2024-05-30T17:14:19.854896+0000","last_deep_scrub":"265'216","last_deep_scrub_stamp":"2024-05-30T17:14:19.854896+0000","last_clean_scrub_stamp":"2024-05-30T17:14:19.854896+0000","objects_scrubbed":4,"log_size":16,"log_dups_size":200,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:20:48.665014+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291507,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":524,"num_read_kb":29194,"num_write":1289,"num_write_kb":17953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":518,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:19.407506+0000","last_change":"2024-05-30T17:14:19.407560+0000","last_active":"2024-05-30T17:14:19.407506+0000","last_peered":"2024-05-30T17:14:19.407506+0000","last_clean":"2024-05-30T17:14:19.407506+0000","last_became_active":"2024-05-30T15:54:20.653316+0000","last_became_peered":"2024-05-30T15:54:20.653316+0000","last_unstale":"2024-05-30T17:14:19.407506+0000","last_undegraded":"2024-05-30T17:14:19.407506+0000","last_fullsized":"2024-05-30T17:14:19.407506+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:19.407301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:19.407301+0000","last_clean_scrub_stamp":"2024-05-30T17:14:19.407301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:47:55.159952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"265'155","reported_seq":727,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.161878+0000","last_change":"2024-05-30T17:13:48.161878+0000","last_active":"2024-05-30T17:13:48.161878+0000","last_peered":"2024-05-30T17:13:48.161878+0000","last_clean":"2024-05-30T17:13:48.161878+0000","last_became_active":"2024-05-30T15:54:22.664353+0000","last_became_peered":"2024-05-30T15:54:22.664353+0000","last_unstale":"2024-05-30T17:13:48.161878+0000","last_undegraded":"2024-05-30T17:13:48.161878+0000","last_fullsized":"2024-05-30T17:13:48.161878+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'155","last_scrub_stamp":"2024-05-30T17:13:48.161621+0000","last_deep_scrub":"265'155","last_deep_scrub_stamp":"2024-05-30T17:13:48.161621+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.161621+0000","objects_scrubbed":6,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:30:12.200451+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":173,"num_read_kb":165,"num_write":120,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"265'532","reported_seq":2492,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:03.691837+0000","last_change":"2024-05-30T17:14:03.691837+0000","last_active":"2024-05-30T17:14:03.691837+0000","last_peered":"2024-05-30T17:14:03.691837+0000","last_clean":"2024-05-30T17:14:03.691837+0000","last_became_active":"2024-05-30T15:54:24.681503+0000","last_became_peered":"2024-05-30T15:54:24.681503+0000","last_unstale":"2024-05-30T17:14:03.691837+0000","last_undegraded":"2024-05-30T17:14:03.691837+0000","last_fullsized":"2024-05-30T17:14:03.691837+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'532","last_scrub_stamp":"2024-05-30T17:14:03.691579+0000","last_deep_scrub":"265'532","last_deep_scrub_stamp":"2024-05-30T17:14:03.691579+0000","last_clean_scrub_stamp":"2024-05-30T17:14:03.691579+0000","objects_scrubbed":1,"log_size":32,"log_dups_size":500,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:45:49.735882+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"265'237","reported_seq":1088,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.884697+0000","last_change":"2024-05-30T17:13:51.884697+0000","last_active":"2024-05-30T17:13:51.884697+0000","last_peered":"2024-05-30T17:13:51.884697+0000","last_clean":"2024-05-30T17:13:51.884697+0000","last_became_active":"2024-05-30T15:54:26.670257+0000","last_became_peered":"2024-05-30T15:54:26.670257+0000","last_unstale":"2024-05-30T17:13:51.884697+0000","last_undegraded":"2024-05-30T17:13:51.884697+0000","last_fullsized":"2024-05-30T17:13:51.884697+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'237","last_scrub_stamp":"2024-05-30T17:13:51.884401+0000","last_deep_scrub":"265'237","last_deep_scrub_stamp":"2024-05-30T17:13:51.884401+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.884401+0000","objects_scrubbed":4,"log_size":37,"log_dups_size":200,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:24:37.651399+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":661,"num_read_kb":640,"num_write":540,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":234,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"265'409","reported_seq":2001,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:13.365832+0000","last_change":"2024-05-30T17:14:13.365901+0000","last_active":"2024-05-30T17:14:13.365832+0000","last_peered":"2024-05-30T17:14:13.365832+0000","last_clean":"2024-05-30T17:14:13.365832+0000","last_became_active":"2024-05-30T15:54:18.660656+0000","last_became_peered":"2024-05-30T15:54:18.660656+0000","last_unstale":"2024-05-30T17:14:13.365832+0000","last_undegraded":"2024-05-30T17:14:13.365832+0000","last_fullsized":"2024-05-30T17:14:13.365832+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'409","last_scrub_stamp":"2024-05-30T17:14:13.365635+0000","last_deep_scrub":"265'409","last_deep_scrub_stamp":"2024-05-30T17:14:13.365635+0000","last_clean_scrub_stamp":"2024-05-30T17:14:13.365635+0000","objects_scrubbed":3,"log_size":109,"log_dups_size":300,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:12:07.620435+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2368,"num_read_kb":2240,"num_write":473,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3015,"num_omap_keys":13,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"265'191","reported_seq":829,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:07.755507+0000","last_change":"2024-05-30T17:14:07.755692+0000","last_active":"2024-05-30T17:14:07.755507+0000","last_peered":"2024-05-30T17:14:07.755507+0000","last_clean":"2024-05-30T17:14:07.755507+0000","last_became_active":"2024-05-30T15:54:16.668083+0000","last_became_peered":"2024-05-30T15:54:16.668083+0000","last_unstale":"2024-05-30T17:14:07.755507+0000","last_undegraded":"2024-05-30T17:14:07.755507+0000","last_fullsized":"2024-05-30T17:14:07.755507+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'191","last_scrub_stamp":"2024-05-30T17:14:07.755193+0000","last_deep_scrub":"265'191","last_deep_scrub_stamp":"2024-05-30T17:14:07.755193+0000","last_clean_scrub_stamp":"2024-05-30T17:14:07.755193+0000","objects_scrubbed":3,"log_size":91,"log_dups_size":100,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:28:20.087604+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5263487,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":406,"num_read_kb":13963,"num_write":1054,"num_write_kb":66137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":516,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:10.355088+0000","last_change":"2024-05-30T17:14:10.355160+0000","last_active":"2024-05-30T17:14:10.355088+0000","last_peered":"2024-05-30T17:14:10.355088+0000","last_clean":"2024-05-30T17:14:10.355088+0000","last_became_active":"2024-05-30T15:54:20.658870+0000","last_became_peered":"2024-05-30T15:54:20.658870+0000","last_unstale":"2024-05-30T17:14:10.355088+0000","last_undegraded":"2024-05-30T17:14:10.355088+0000","last_fullsized":"2024-05-30T17:14:10.355088+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:10.354868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:10.354868+0000","last_clean_scrub_stamp":"2024-05-30T17:14:10.354868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:31:51.606017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"265'146","reported_seq":743,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:09.748021+0000","last_change":"2024-05-30T17:14:09.748021+0000","last_active":"2024-05-30T17:14:09.748021+0000","last_peered":"2024-05-30T17:14:09.748021+0000","last_clean":"2024-05-30T17:14:09.748021+0000","last_became_active":"2024-05-30T15:54:22.849978+0000","last_became_peered":"2024-05-30T15:54:22.849978+0000","last_unstale":"2024-05-30T17:14:09.748021+0000","last_undegraded":"2024-05-30T17:14:09.748021+0000","last_fullsized":"2024-05-30T17:14:09.748021+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'146","last_scrub_stamp":"2024-05-30T17:14:09.747755+0000","last_deep_scrub":"265'146","last_deep_scrub_stamp":"2024-05-30T17:14:09.747755+0000","last_clean_scrub_stamp":"2024-05-30T17:14:09.747755+0000","objects_scrubbed":4,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:28:48.260787+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8153,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":189,"num_read_kb":241,"num_write":102,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"265'549","reported_seq":2582,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:05.388708+0000","last_change":"2024-05-30T17:14:05.388708+0000","last_active":"2024-05-30T17:14:05.388708+0000","last_peered":"2024-05-30T17:14:05.388708+0000","last_clean":"2024-05-30T17:14:05.388708+0000","last_became_active":"2024-05-30T15:54:24.686546+0000","last_became_peered":"2024-05-30T15:54:24.686546+0000","last_unstale":"2024-05-30T17:14:05.388708+0000","last_undegraded":"2024-05-30T17:14:05.388708+0000","last_fullsized":"2024-05-30T17:14:05.388708+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'549","last_scrub_stamp":"2024-05-30T17:14:05.388354+0000","last_deep_scrub":"265'549","last_deep_scrub_stamp":"2024-05-30T17:14:05.388354+0000","last_clean_scrub_stamp":"2024-05-30T17:14:05.388354+0000","objects_scrubbed":1,"log_size":49,"log_dups_size":500,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:59:56.299339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"265'901","reported_seq":2526,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:05.284034+0000","last_change":"2024-05-30T17:14:05.284034+0000","last_active":"2024-05-30T17:14:05.284034+0000","last_peered":"2024-05-30T17:14:05.284034+0000","last_clean":"2024-05-30T17:14:05.284034+0000","last_became_active":"2024-05-30T15:54:18.658124+0000","last_became_peered":"2024-05-30T15:54:18.658124+0000","last_unstale":"2024-05-30T17:14:05.284034+0000","last_undegraded":"2024-05-30T17:14:05.284034+0000","last_fullsized":"2024-05-30T17:14:05.284034+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'901","last_scrub_stamp":"2024-05-30T17:14:05.283757+0000","last_deep_scrub":"265'901","last_deep_scrub_stamp":"2024-05-30T17:14:05.283757+0000","last_clean_scrub_stamp":"2024-05-30T17:14:05.283757+0000","objects_scrubbed":6,"log_size":101,"log_dups_size":800,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:45:18.502796+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4127,"num_read_kb":3657,"num_write":1246,"num_write_kb":491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2810,"num_omap_keys":11,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"265'208","reported_seq":897,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:32.804008+0000","last_change":"2024-05-30T17:14:32.804223+0000","last_active":"2024-05-30T17:14:32.804008+0000","last_peered":"2024-05-30T17:14:32.804008+0000","last_clean":"2024-05-30T17:14:32.804008+0000","last_became_active":"2024-05-30T15:54:16.658279+0000","last_became_peered":"2024-05-30T15:54:16.658279+0000","last_unstale":"2024-05-30T17:14:32.804008+0000","last_undegraded":"2024-05-30T17:14:32.804008+0000","last_fullsized":"2024-05-30T17:14:32.804008+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'208","last_scrub_stamp":"2024-05-30T17:14:32.803764+0000","last_deep_scrub":"265'208","last_deep_scrub_stamp":"2024-05-30T17:14:32.803764+0000","last_clean_scrub_stamp":"2024-05-30T17:14:32.803764+0000","objects_scrubbed":3,"log_size":108,"log_dups_size":100,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:31:04.878530+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1254547,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":480,"num_read_kb":111643,"num_write":1134,"num_write_kb":26225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":520,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:14.742349+0000","last_change":"2024-05-30T17:14:14.742349+0000","last_active":"2024-05-30T17:14:14.742349+0000","last_peered":"2024-05-30T17:14:14.742349+0000","last_clean":"2024-05-30T17:14:14.742349+0000","last_became_active":"2024-05-30T15:54:20.661583+0000","last_became_peered":"2024-05-30T15:54:20.661583+0000","last_unstale":"2024-05-30T17:14:14.742349+0000","last_undegraded":"2024-05-30T17:14:14.742349+0000","last_fullsized":"2024-05-30T17:14:14.742349+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:14.742060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:14.742060+0000","last_clean_scrub_stamp":"2024-05-30T17:14:14.742060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:08:49.700663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"265'265","reported_seq":922,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.257702+0000","last_change":"2024-05-30T17:13:58.257702+0000","last_active":"2024-05-30T17:13:58.257702+0000","last_peered":"2024-05-30T17:13:58.257702+0000","last_clean":"2024-05-30T17:13:58.257702+0000","last_became_active":"2024-05-30T15:54:22.849126+0000","last_became_peered":"2024-05-30T15:54:22.849126+0000","last_unstale":"2024-05-30T17:13:58.257702+0000","last_undegraded":"2024-05-30T17:13:58.257702+0000","last_fullsized":"2024-05-30T17:13:58.257702+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'265","last_scrub_stamp":"2024-05-30T17:13:58.257369+0000","last_deep_scrub":"265'265","last_deep_scrub_stamp":"2024-05-30T17:13:58.257369+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.257369+0000","objects_scrubbed":10,"log_size":65,"log_dups_size":200,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:15:48.980020+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6773,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":352,"num_read_kb":376,"num_write":210,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":511,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:07.996809+0000","last_change":"2024-05-30T17:14:07.996872+0000","last_active":"2024-05-30T17:14:07.996809+0000","last_peered":"2024-05-30T17:14:07.996809+0000","last_clean":"2024-05-30T17:14:07.996809+0000","last_became_active":"2024-05-30T15:54:24.673506+0000","last_became_peered":"2024-05-30T15:54:24.673506+0000","last_unstale":"2024-05-30T17:14:07.996809+0000","last_undegraded":"2024-05-30T17:14:07.996809+0000","last_fullsized":"2024-05-30T17:14:07.996809+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:07.996581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:07.996581+0000","last_clean_scrub_stamp":"2024-05-30T17:14:07.996581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:59:09.003040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"265'546","reported_seq":1385,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.226322+0000","last_change":"2024-05-30T17:13:54.226466+0000","last_active":"2024-05-30T17:13:54.226322+0000","last_peered":"2024-05-30T17:13:54.226322+0000","last_clean":"2024-05-30T17:13:54.226322+0000","last_became_active":"2024-05-30T15:54:26.686389+0000","last_became_peered":"2024-05-30T15:54:26.686389+0000","last_unstale":"2024-05-30T17:13:54.226322+0000","last_undegraded":"2024-05-30T17:13:54.226322+0000","last_fullsized":"2024-05-30T17:13:54.226322+0000","mapping_epoch":32,"log_start":"265'500","ondisk_log_start":"265'500","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'546","last_scrub_stamp":"2024-05-30T17:13:54.226029+0000","last_deep_scrub":"265'546","last_deep_scrub_stamp":"2024-05-30T17:13:54.226029+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.226029+0000","objects_scrubbed":3,"log_size":46,"log_dups_size":500,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:20:00.749166+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1297,"num_read_kb":1288,"num_write":1137,"num_write_kb":385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":381,"num_omap_keys":2,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"265'1403","reported_seq":3120,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:23.135183+0000","last_change":"2024-05-30T17:14:23.135183+0000","last_active":"2024-05-30T17:14:23.135183+0000","last_peered":"2024-05-30T17:14:23.135183+0000","last_clean":"2024-05-30T17:14:23.135183+0000","last_became_active":"2024-05-30T15:54:18.639667+0000","last_became_peered":"2024-05-30T15:54:18.639667+0000","last_unstale":"2024-05-30T17:14:23.135183+0000","last_undegraded":"2024-05-30T17:14:23.135183+0000","last_fullsized":"2024-05-30T17:14:23.135183+0000","mapping_epoch":24,"log_start":"265'1300","ondisk_log_start":"265'1300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'1403","last_scrub_stamp":"2024-05-30T17:14:23.134899+0000","last_deep_scrub":"265'1403","last_deep_scrub_stamp":"2024-05-30T17:14:23.134899+0000","last_clean_scrub_stamp":"2024-05-30T17:14:23.134899+0000","objects_scrubbed":5,"log_size":103,"log_dups_size":1300,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:49:41.516827+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5970,"num_read_kb":5130,"num_write":2044,"num_write_kb":884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":763,"num_omap_keys":3,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"265'213","reported_seq":876,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:33.753735+0000","last_change":"2024-05-30T17:14:33.753798+0000","last_active":"2024-05-30T17:14:33.753735+0000","last_peered":"2024-05-30T17:14:33.753735+0000","last_clean":"2024-05-30T17:14:33.753735+0000","last_became_active":"2024-05-30T15:54:16.659427+0000","last_became_peered":"2024-05-30T15:54:16.659427+0000","last_unstale":"2024-05-30T17:14:33.753735+0000","last_undegraded":"2024-05-30T17:14:33.753735+0000","last_fullsized":"2024-05-30T17:14:33.753735+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'213","last_scrub_stamp":"2024-05-30T17:14:33.753534+0000","last_deep_scrub":"265'213","last_deep_scrub_stamp":"2024-05-30T17:14:33.753534+0000","last_clean_scrub_stamp":"2024-05-30T17:14:33.753534+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":200,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:07:20.080063+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":467,"num_read_kb":34119,"num_write":1157,"num_write_kb":42367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":523,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.231474+0000","last_change":"2024-05-30T17:13:58.231474+0000","last_active":"2024-05-30T17:13:58.231474+0000","last_peered":"2024-05-30T17:13:58.231474+0000","last_clean":"2024-05-30T17:13:58.231474+0000","last_became_active":"2024-05-30T15:54:20.647565+0000","last_became_peered":"2024-05-30T15:54:20.647565+0000","last_unstale":"2024-05-30T17:13:58.231474+0000","last_undegraded":"2024-05-30T17:13:58.231474+0000","last_fullsized":"2024-05-30T17:13:58.231474+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:58.231223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:58.231223+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.231223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:53:45.579167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"265'150","reported_seq":751,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:25.359652+0000","last_change":"2024-05-30T17:14:25.359726+0000","last_active":"2024-05-30T17:14:25.359652+0000","last_peered":"2024-05-30T17:14:25.359652+0000","last_clean":"2024-05-30T17:14:25.359652+0000","last_became_active":"2024-05-30T15:54:22.670064+0000","last_became_peered":"2024-05-30T15:54:22.670064+0000","last_unstale":"2024-05-30T17:14:25.359652+0000","last_undegraded":"2024-05-30T17:14:25.359652+0000","last_fullsized":"2024-05-30T17:14:25.359652+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'150","last_scrub_stamp":"2024-05-30T17:14:25.359472+0000","last_deep_scrub":"265'150","last_deep_scrub_stamp":"2024-05-30T17:14:25.359472+0000","last_clean_scrub_stamp":"2024-05-30T17:14:25.359472+0000","objects_scrubbed":4,"log_size":50,"log_dups_size":100,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:25:32.352668+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12270,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":197,"num_read_kb":284,"num_write":108,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":514,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:02.359602+0000","last_change":"2024-05-30T17:14:02.359602+0000","last_active":"2024-05-30T17:14:02.359602+0000","last_peered":"2024-05-30T17:14:02.359602+0000","last_clean":"2024-05-30T17:14:02.359602+0000","last_became_active":"2024-05-30T15:54:24.688046+0000","last_became_peered":"2024-05-30T15:54:24.688046+0000","last_unstale":"2024-05-30T17:14:02.359602+0000","last_undegraded":"2024-05-30T17:14:02.359602+0000","last_fullsized":"2024-05-30T17:14:02.359602+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:02.359352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:02.359352+0000","last_clean_scrub_stamp":"2024-05-30T17:14:02.359352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:10:48.308830+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"265'2370","reported_seq":4200,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:04.738831+0000","last_change":"2024-05-30T17:14:04.738949+0000","last_active":"2024-05-30T17:14:04.738831+0000","last_peered":"2024-05-30T17:14:04.738831+0000","last_clean":"2024-05-30T17:14:04.738831+0000","last_became_active":"2024-05-30T15:54:26.691574+0000","last_became_peered":"2024-05-30T15:54:26.691574+0000","last_unstale":"2024-05-30T17:14:04.738831+0000","last_undegraded":"2024-05-30T17:14:04.738831+0000","last_fullsized":"2024-05-30T17:14:04.738831+0000","mapping_epoch":32,"log_start":"265'2300","ondisk_log_start":"265'2300","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2370","last_scrub_stamp":"2024-05-30T17:14:04.738577+0000","last_deep_scrub":"265'2370","last_deep_scrub_stamp":"2024-05-30T17:14:04.738577+0000","last_clean_scrub_stamp":"2024-05-30T17:14:04.738577+0000","objects_scrubbed":1,"log_size":70,"log_dups_size":2300,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:55:13.090443+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5943,"num_read_kb":5935,"num_write":4785,"num_write_kb":1767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":547,"num_omap_keys":3,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"265'373","reported_seq":1804,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:01.825479+0000","last_change":"2024-05-30T17:14:01.825479+0000","last_active":"2024-05-30T17:14:01.825479+0000","last_peered":"2024-05-30T17:14:01.825479+0000","last_clean":"2024-05-30T17:14:01.825479+0000","last_became_active":"2024-05-30T15:54:18.668963+0000","last_became_peered":"2024-05-30T15:54:18.668963+0000","last_unstale":"2024-05-30T17:14:01.825479+0000","last_undegraded":"2024-05-30T17:14:01.825479+0000","last_fullsized":"2024-05-30T17:14:01.825479+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'373","last_scrub_stamp":"2024-05-30T17:14:01.825217+0000","last_deep_scrub":"265'373","last_deep_scrub_stamp":"2024-05-30T17:14:01.825217+0000","last_clean_scrub_stamp":"2024-05-30T17:14:01.825217+0000","objects_scrubbed":2,"log_size":73,"log_dups_size":300,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:14:41.634577+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2011,"num_read_kb":1890,"num_write":455,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":225,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"265'240","reported_seq":889,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:29.763421+0000","last_change":"2024-05-30T17:14:29.763528+0000","last_active":"2024-05-30T17:14:29.763421+0000","last_peered":"2024-05-30T17:14:29.763421+0000","last_clean":"2024-05-30T17:14:29.763421+0000","last_became_active":"2024-05-30T15:54:16.649605+0000","last_became_peered":"2024-05-30T15:54:16.649605+0000","last_unstale":"2024-05-30T17:14:29.763421+0000","last_undegraded":"2024-05-30T17:14:29.763421+0000","last_fullsized":"2024-05-30T17:14:29.763421+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'240","last_scrub_stamp":"2024-05-30T17:14:29.763223+0000","last_deep_scrub":"265'240","last_deep_scrub_stamp":"2024-05-30T17:14:29.763223+0000","last_clean_scrub_stamp":"2024-05-30T17:14:29.763223+0000","objects_scrubbed":3,"log_size":40,"log_dups_size":200,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:14.911288+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242911,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":469,"num_read_kb":32402,"num_write":1238,"num_write_kb":40071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":521,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.170085+0000","last_change":"2024-05-30T17:13:56.170146+0000","last_active":"2024-05-30T17:13:56.170085+0000","last_peered":"2024-05-30T17:13:56.170085+0000","last_clean":"2024-05-30T17:13:56.170085+0000","last_became_active":"2024-05-30T15:54:20.651369+0000","last_became_peered":"2024-05-30T15:54:20.651369+0000","last_unstale":"2024-05-30T17:13:56.170085+0000","last_undegraded":"2024-05-30T17:13:56.170085+0000","last_fullsized":"2024-05-30T17:13:56.170085+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:56.169880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:56.169880+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.169880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:32:45.358384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"265'199","reported_seq":824,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:24.727401+0000","last_change":"2024-05-30T17:14:24.727401+0000","last_active":"2024-05-30T17:14:24.727401+0000","last_peered":"2024-05-30T17:14:24.727401+0000","last_clean":"2024-05-30T17:14:24.727401+0000","last_became_active":"2024-05-30T15:54:22.664874+0000","last_became_peered":"2024-05-30T15:54:22.664874+0000","last_unstale":"2024-05-30T17:14:24.727401+0000","last_undegraded":"2024-05-30T17:14:24.727401+0000","last_fullsized":"2024-05-30T17:14:24.727401+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'199","last_scrub_stamp":"2024-05-30T17:14:24.727125+0000","last_deep_scrub":"265'199","last_deep_scrub_stamp":"2024-05-30T17:14:24.727125+0000","last_clean_scrub_stamp":"2024-05-30T17:14:24.727125+0000","objects_scrubbed":8,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:38:48.594467+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6309,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":259,"num_read_kb":308,"num_write":148,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"265'549","reported_seq":2492,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:06.842942+0000","last_change":"2024-05-30T17:14:06.842942+0000","last_active":"2024-05-30T17:14:06.842942+0000","last_peered":"2024-05-30T17:14:06.842942+0000","last_clean":"2024-05-30T17:14:06.842942+0000","last_became_active":"2024-05-30T15:54:24.670141+0000","last_became_peered":"2024-05-30T15:54:24.670141+0000","last_unstale":"2024-05-30T17:14:06.842942+0000","last_undegraded":"2024-05-30T17:14:06.842942+0000","last_fullsized":"2024-05-30T17:14:06.842942+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'549","last_scrub_stamp":"2024-05-30T17:14:06.842673+0000","last_deep_scrub":"265'549","last_deep_scrub_stamp":"2024-05-30T17:14:06.842673+0000","last_clean_scrub_stamp":"2024-05-30T17:14:06.842673+0000","objects_scrubbed":1,"log_size":49,"log_dups_size":500,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:16:17.488221+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"265'107","reported_seq":621,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:52.677276+0000","last_change":"2024-05-30T17:13:52.677276+0000","last_active":"2024-05-30T17:13:52.677276+0000","last_peered":"2024-05-30T17:13:52.677276+0000","last_clean":"2024-05-30T17:13:52.677276+0000","last_became_active":"2024-05-30T15:54:26.689987+0000","last_became_peered":"2024-05-30T15:54:26.689987+0000","last_unstale":"2024-05-30T17:13:52.677276+0000","last_undegraded":"2024-05-30T17:13:52.677276+0000","last_fullsized":"2024-05-30T17:13:52.677276+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'107","last_scrub_stamp":"2024-05-30T17:13:52.676848+0000","last_deep_scrub":"265'107","last_deep_scrub_stamp":"2024-05-30T17:13:52.676848+0000","last_clean_scrub_stamp":"2024-05-30T17:13:52.676848+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:48:23.065085+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":80,"num_write":254,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"265'1335","reported_seq":2979,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:16.822597+0000","last_change":"2024-05-30T17:14:16.822667+0000","last_active":"2024-05-30T17:14:16.822597+0000","last_peered":"2024-05-30T17:14:16.822597+0000","last_clean":"2024-05-30T17:14:16.822597+0000","last_became_active":"2024-05-30T15:54:18.647634+0000","last_became_peered":"2024-05-30T15:54:18.647634+0000","last_unstale":"2024-05-30T17:14:16.822597+0000","last_undegraded":"2024-05-30T17:14:16.822597+0000","last_fullsized":"2024-05-30T17:14:16.822597+0000","mapping_epoch":24,"log_start":"265'1300","ondisk_log_start":"265'1300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'1335","last_scrub_stamp":"2024-05-30T17:14:16.822401+0000","last_deep_scrub":"265'1335","last_deep_scrub_stamp":"2024-05-30T17:14:16.822401+0000","last_clean_scrub_stamp":"2024-05-30T17:14:16.822401+0000","objects_scrubbed":4,"log_size":35,"log_dups_size":1300,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:44:49.543711+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5717,"num_read_kb":4967,"num_write":1912,"num_write_kb":881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"265'206","reported_seq":830,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:25.336089+0000","last_change":"2024-05-30T17:14:25.336288+0000","last_active":"2024-05-30T17:14:25.336089+0000","last_peered":"2024-05-30T17:14:25.336089+0000","last_clean":"2024-05-30T17:14:25.336089+0000","last_became_active":"2024-05-30T15:54:16.668336+0000","last_became_peered":"2024-05-30T15:54:16.668336+0000","last_unstale":"2024-05-30T17:14:25.336089+0000","last_undegraded":"2024-05-30T17:14:25.336089+0000","last_fullsized":"2024-05-30T17:14:25.336089+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'206","last_scrub_stamp":"2024-05-30T17:14:25.335853+0000","last_deep_scrub":"265'206","last_deep_scrub_stamp":"2024-05-30T17:14:25.335853+0000","last_clean_scrub_stamp":"2024-05-30T17:14:25.335853+0000","objects_scrubbed":4,"log_size":106,"log_dups_size":100,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:00:44.689923+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":403,"num_read_kb":1534,"num_write":1058,"num_write_kb":21253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":523,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.160046+0000","last_change":"2024-05-30T17:13:54.160046+0000","last_active":"2024-05-30T17:13:54.160046+0000","last_peered":"2024-05-30T17:13:54.160046+0000","last_clean":"2024-05-30T17:13:54.160046+0000","last_became_active":"2024-05-30T15:54:20.655573+0000","last_became_peered":"2024-05-30T15:54:20.655573+0000","last_unstale":"2024-05-30T17:13:54.160046+0000","last_undegraded":"2024-05-30T17:13:54.160046+0000","last_fullsized":"2024-05-30T17:13:54.160046+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:54.159789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:54.159789+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.159789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:27:43.050145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"265'212","reported_seq":880,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:19.794572+0000","last_change":"2024-05-30T17:14:19.794572+0000","last_active":"2024-05-30T17:14:19.794572+0000","last_peered":"2024-05-30T17:14:19.794572+0000","last_clean":"2024-05-30T17:14:19.794572+0000","last_became_active":"2024-05-30T15:54:22.658700+0000","last_became_peered":"2024-05-30T15:54:22.658700+0000","last_unstale":"2024-05-30T17:14:19.794572+0000","last_undegraded":"2024-05-30T17:14:19.794572+0000","last_fullsized":"2024-05-30T17:14:19.794572+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'212","last_scrub_stamp":"2024-05-30T17:14:19.794300+0000","last_deep_scrub":"265'212","last_deep_scrub_stamp":"2024-05-30T17:14:19.794300+0000","last_clean_scrub_stamp":"2024-05-30T17:14:19.794300+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:25:02.091631+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14119,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":309,"num_read_kb":443,"num_write":154,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":515,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:43.048221+0000","last_change":"2024-05-30T17:13:43.048221+0000","last_active":"2024-05-30T17:13:43.048221+0000","last_peered":"2024-05-30T17:13:43.048221+0000","last_clean":"2024-05-30T17:13:43.048221+0000","last_became_active":"2024-05-30T15:54:24.673505+0000","last_became_peered":"2024-05-30T15:54:24.673505+0000","last_unstale":"2024-05-30T17:13:43.048221+0000","last_undegraded":"2024-05-30T17:13:43.048221+0000","last_fullsized":"2024-05-30T17:13:43.048221+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:43.047974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:43.047974+0000","last_clean_scrub_stamp":"2024-05-30T17:13:43.047974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:04:22.180317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"265'101","reported_seq":628,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.955055+0000","last_change":"2024-05-30T17:13:50.955055+0000","last_active":"2024-05-30T17:13:50.955055+0000","last_peered":"2024-05-30T17:13:50.955055+0000","last_clean":"2024-05-30T17:13:50.955055+0000","last_became_active":"2024-05-30T15:54:26.700608+0000","last_became_peered":"2024-05-30T15:54:26.700608+0000","last_unstale":"2024-05-30T17:13:50.955055+0000","last_undegraded":"2024-05-30T17:13:50.955055+0000","last_fullsized":"2024-05-30T17:13:50.955055+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'101","last_scrub_stamp":"2024-05-30T17:13:50.954735+0000","last_deep_scrub":"265'101","last_deep_scrub_stamp":"2024-05-30T17:13:50.954735+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.954735+0000","objects_scrubbed":2,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:48:33.984457+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":918,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":103,"num_write":225,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"265'544","reported_seq":2181,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:10.057108+0000","last_change":"2024-05-30T17:14:10.057194+0000","last_active":"2024-05-30T17:14:10.057108+0000","last_peered":"2024-05-30T17:14:10.057108+0000","last_clean":"2024-05-30T17:14:10.057108+0000","last_became_active":"2024-05-30T15:54:18.657522+0000","last_became_peered":"2024-05-30T15:54:18.657522+0000","last_unstale":"2024-05-30T17:14:10.057108+0000","last_undegraded":"2024-05-30T17:14:10.057108+0000","last_fullsized":"2024-05-30T17:14:10.057108+0000","mapping_epoch":24,"log_start":"265'500","ondisk_log_start":"265'500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'544","last_scrub_stamp":"2024-05-30T17:14:10.056929+0000","last_deep_scrub":"265'544","last_deep_scrub_stamp":"2024-05-30T17:14:10.056929+0000","last_clean_scrub_stamp":"2024-05-30T17:14:10.056929+0000","objects_scrubbed":5,"log_size":44,"log_dups_size":500,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:06:50.095596+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2751,"num_read_kb":2537,"num_write":708,"num_write_kb":252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":303,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"265'234","reported_seq":924,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:33.434213+0000","last_change":"2024-05-30T17:14:33.434317+0000","last_active":"2024-05-30T17:14:33.434213+0000","last_peered":"2024-05-30T17:14:33.434213+0000","last_clean":"2024-05-30T17:14:33.434213+0000","last_became_active":"2024-05-30T15:54:16.666244+0000","last_became_peered":"2024-05-30T15:54:16.666244+0000","last_unstale":"2024-05-30T17:14:33.434213+0000","last_undegraded":"2024-05-30T17:14:33.434213+0000","last_fullsized":"2024-05-30T17:14:33.434213+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'234","last_scrub_stamp":"2024-05-30T17:14:33.433990+0000","last_deep_scrub":"265'234","last_deep_scrub_stamp":"2024-05-30T17:14:33.433990+0000","last_clean_scrub_stamp":"2024-05-30T17:14:33.433990+0000","objects_scrubbed":5,"log_size":34,"log_dups_size":200,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:55:15.577076+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":549856,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":638,"num_read_kb":32957,"num_write":1225,"num_write_kb":40334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":521,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:52.077511+0000","last_change":"2024-05-30T17:13:52.077571+0000","last_active":"2024-05-30T17:13:52.077511+0000","last_peered":"2024-05-30T17:13:52.077511+0000","last_clean":"2024-05-30T17:13:52.077511+0000","last_became_active":"2024-05-30T15:54:20.661967+0000","last_became_peered":"2024-05-30T15:54:20.661967+0000","last_unstale":"2024-05-30T17:13:52.077511+0000","last_undegraded":"2024-05-30T17:13:52.077511+0000","last_fullsized":"2024-05-30T17:13:52.077511+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:52.077316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:52.077316+0000","last_clean_scrub_stamp":"2024-05-30T17:13:52.077316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:55:45.987890+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"265'187","reported_seq":803,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:25.109166+0000","last_change":"2024-05-30T17:14:25.109258+0000","last_active":"2024-05-30T17:14:25.109166+0000","last_peered":"2024-05-30T17:14:25.109166+0000","last_clean":"2024-05-30T17:14:25.109166+0000","last_became_active":"2024-05-30T15:54:22.664732+0000","last_became_peered":"2024-05-30T15:54:22.664732+0000","last_unstale":"2024-05-30T17:14:25.109166+0000","last_undegraded":"2024-05-30T17:14:25.109166+0000","last_fullsized":"2024-05-30T17:14:25.109166+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'187","last_scrub_stamp":"2024-05-30T17:14:25.108975+0000","last_deep_scrub":"265'187","last_deep_scrub_stamp":"2024-05-30T17:14:25.108975+0000","last_clean_scrub_stamp":"2024-05-30T17:14:25.108975+0000","objects_scrubbed":6,"log_size":87,"log_dups_size":100,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:53:56.466888+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13534,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":265,"num_read_kb":366,"num_write":202,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15964,"num_omap_keys":38,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":515,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:01.839426+0000","last_change":"2024-05-30T17:14:01.839579+0000","last_active":"2024-05-30T17:14:01.839426+0000","last_peered":"2024-05-30T17:14:01.839426+0000","last_clean":"2024-05-30T17:14:01.839426+0000","last_became_active":"2024-05-30T15:54:24.679254+0000","last_became_peered":"2024-05-30T15:54:24.679254+0000","last_unstale":"2024-05-30T17:14:01.839426+0000","last_undegraded":"2024-05-30T17:14:01.839426+0000","last_fullsized":"2024-05-30T17:14:01.839426+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:01.839200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:01.839200+0000","last_clean_scrub_stamp":"2024-05-30T17:14:01.839200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:13:39.135342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"265'109","reported_seq":637,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:57.390054+0000","last_change":"2024-05-30T17:13:57.390054+0000","last_active":"2024-05-30T17:13:57.390054+0000","last_peered":"2024-05-30T17:13:57.390054+0000","last_clean":"2024-05-30T17:13:57.390054+0000","last_became_active":"2024-05-30T15:54:26.697333+0000","last_became_peered":"2024-05-30T15:54:26.697333+0000","last_unstale":"2024-05-30T17:13:57.390054+0000","last_undegraded":"2024-05-30T17:13:57.390054+0000","last_fullsized":"2024-05-30T17:13:57.390054+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'109","last_scrub_stamp":"2024-05-30T17:13:57.389793+0000","last_deep_scrub":"265'109","last_deep_scrub_stamp":"2024-05-30T17:13:57.389793+0000","last_clean_scrub_stamp":"2024-05-30T17:13:57.389793+0000","objects_scrubbed":4,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:34:55.402057+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":270,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":107,"num_read_kb":90,"num_write":242,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"265'402","reported_seq":2073,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:22.126305+0000","last_change":"2024-05-30T17:14:22.126392+0000","last_active":"2024-05-30T17:14:22.126305+0000","last_peered":"2024-05-30T17:14:22.126305+0000","last_clean":"2024-05-30T17:14:22.126305+0000","last_became_active":"2024-05-30T15:54:18.635519+0000","last_became_peered":"2024-05-30T15:54:18.635519+0000","last_unstale":"2024-05-30T17:14:22.126305+0000","last_undegraded":"2024-05-30T17:14:22.126305+0000","last_fullsized":"2024-05-30T17:14:22.126305+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'402","last_scrub_stamp":"2024-05-30T17:14:22.126084+0000","last_deep_scrub":"265'402","last_deep_scrub_stamp":"2024-05-30T17:14:22.126084+0000","last_clean_scrub_stamp":"2024-05-30T17:14:22.126084+0000","objects_scrubbed":5,"log_size":102,"log_dups_size":300,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:18:22.194175+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2332,"num_read_kb":2242,"num_write":450,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2297,"num_omap_keys":9,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"265'175","reported_seq":805,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:12.798266+0000","last_change":"2024-05-30T17:14:12.798266+0000","last_active":"2024-05-30T17:14:12.798266+0000","last_peered":"2024-05-30T17:14:12.798266+0000","last_clean":"2024-05-30T17:14:12.798266+0000","last_became_active":"2024-05-30T15:54:16.685162+0000","last_became_peered":"2024-05-30T15:54:16.685162+0000","last_unstale":"2024-05-30T17:14:12.798266+0000","last_undegraded":"2024-05-30T17:14:12.798266+0000","last_fullsized":"2024-05-30T17:14:12.798266+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'175","last_scrub_stamp":"2024-05-30T17:14:12.797857+0000","last_deep_scrub":"265'175","last_deep_scrub_stamp":"2024-05-30T17:14:12.797857+0000","last_clean_scrub_stamp":"2024-05-30T17:14:12.797857+0000","objects_scrubbed":6,"log_size":75,"log_dups_size":100,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:43:33.318612+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5904316,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":401,"num_read_kb":10586,"num_write":1027,"num_write_kb":21029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":519,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.991573+0000","last_change":"2024-05-30T17:13:56.991631+0000","last_active":"2024-05-30T17:13:56.991573+0000","last_peered":"2024-05-30T17:13:56.991573+0000","last_clean":"2024-05-30T17:13:56.991573+0000","last_became_active":"2024-05-30T15:54:20.646646+0000","last_became_peered":"2024-05-30T15:54:20.646646+0000","last_unstale":"2024-05-30T17:13:56.991573+0000","last_undegraded":"2024-05-30T17:13:56.991573+0000","last_fullsized":"2024-05-30T17:13:56.991573+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:56.991372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:56.991372+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.991372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:25:19.485498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"265'189","reported_seq":815,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.157684+0000","last_change":"2024-05-30T17:13:46.157858+0000","last_active":"2024-05-30T17:13:46.157684+0000","last_peered":"2024-05-30T17:13:46.157684+0000","last_clean":"2024-05-30T17:13:46.157684+0000","last_became_active":"2024-05-30T15:54:22.669660+0000","last_became_peered":"2024-05-30T15:54:22.669660+0000","last_unstale":"2024-05-30T17:13:46.157684+0000","last_undegraded":"2024-05-30T17:13:46.157684+0000","last_fullsized":"2024-05-30T17:13:46.157684+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'189","last_scrub_stamp":"2024-05-30T17:13:46.157487+0000","last_deep_scrub":"265'189","last_deep_scrub_stamp":"2024-05-30T17:13:46.157487+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.157487+0000","objects_scrubbed":7,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:49:02.360024+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6014,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":258,"num_read_kb":320,"num_write":146,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"265'512","reported_seq":4604,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:44.085253+0000","last_change":"2024-05-30T17:13:44.085280+0000","last_active":"2024-05-30T17:13:44.085253+0000","last_peered":"2024-05-30T17:13:44.085253+0000","last_clean":"2024-05-30T17:13:44.085253+0000","last_became_active":"2024-05-30T15:54:24.672377+0000","last_became_peered":"2024-05-30T15:54:24.672377+0000","last_unstale":"2024-05-30T17:13:44.085253+0000","last_undegraded":"2024-05-30T17:13:44.085253+0000","last_fullsized":"2024-05-30T17:13:44.085253+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'512","last_scrub_stamp":"2024-05-30T17:13:44.085165+0000","last_deep_scrub":"265'512","last_deep_scrub_stamp":"2024-05-30T17:13:44.085165+0000","last_clean_scrub_stamp":"2024-05-30T17:13:44.085165+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":500,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:13:21.462886+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"265'120","reported_seq":645,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.014771+0000","last_change":"2024-05-30T17:13:53.014854+0000","last_active":"2024-05-30T17:13:53.014771+0000","last_peered":"2024-05-30T17:13:53.014771+0000","last_clean":"2024-05-30T17:13:53.014771+0000","last_became_active":"2024-05-30T15:54:26.679866+0000","last_became_peered":"2024-05-30T15:54:26.679866+0000","last_unstale":"2024-05-30T17:13:53.014771+0000","last_undegraded":"2024-05-30T17:13:53.014771+0000","last_fullsized":"2024-05-30T17:13:53.014771+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'120","last_scrub_stamp":"2024-05-30T17:13:53.014561+0000","last_deep_scrub":"265'120","last_deep_scrub_stamp":"2024-05-30T17:13:53.014561+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.014561+0000","objects_scrubbed":3,"log_size":20,"log_dups_size":100,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:47:46.937703+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1256,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":144,"num_read_kb":134,"num_write":293,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"265'392","reported_seq":1808,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:02.030303+0000","last_change":"2024-05-30T17:14:02.030382+0000","last_active":"2024-05-30T17:14:02.030303+0000","last_peered":"2024-05-30T17:14:02.030303+0000","last_clean":"2024-05-30T17:14:02.030303+0000","last_became_active":"2024-05-30T15:54:18.671088+0000","last_became_peered":"2024-05-30T15:54:18.671088+0000","last_unstale":"2024-05-30T17:14:02.030303+0000","last_undegraded":"2024-05-30T17:14:02.030303+0000","last_fullsized":"2024-05-30T17:14:02.030303+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'392","last_scrub_stamp":"2024-05-30T17:14:02.030076+0000","last_deep_scrub":"265'392","last_deep_scrub_stamp":"2024-05-30T17:14:02.030076+0000","last_clean_scrub_stamp":"2024-05-30T17:14:02.030076+0000","objects_scrubbed":3,"log_size":92,"log_dups_size":300,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:37:05.922490+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2029,"num_read_kb":1894,"num_write":481,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"265'180","reported_seq":933,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:18.060426+0000","last_change":"2024-05-30T17:14:18.060426+0000","last_active":"2024-05-30T17:14:18.060426+0000","last_peered":"2024-05-30T17:14:18.060426+0000","last_clean":"2024-05-30T17:14:18.060426+0000","last_became_active":"2024-05-30T15:54:16.671477+0000","last_became_peered":"2024-05-30T15:54:16.671477+0000","last_unstale":"2024-05-30T17:14:18.060426+0000","last_undegraded":"2024-05-30T17:14:18.060426+0000","last_fullsized":"2024-05-30T17:14:18.060426+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'180","last_scrub_stamp":"2024-05-30T17:14:18.060120+0000","last_deep_scrub":"265'180","last_deep_scrub_stamp":"2024-05-30T17:14:18.060120+0000","last_clean_scrub_stamp":"2024-05-30T17:14:18.060120+0000","objects_scrubbed":3,"log_size":80,"log_dups_size":100,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:36:39.055061+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110807,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":679,"num_read_kb":167118,"num_write":1082,"num_write_kb":34693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"265'4","reported_seq":1169,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:16.307268+0000","last_change":"2024-05-30T17:14:16.307355+0000","last_active":"2024-05-30T17:14:16.307268+0000","last_peered":"2024-05-30T17:14:16.307268+0000","last_clean":"2024-05-30T17:14:16.307268+0000","last_became_active":"2024-05-30T15:54:20.662371+0000","last_became_peered":"2024-05-30T15:54:20.662371+0000","last_unstale":"2024-05-30T17:14:16.307268+0000","last_undegraded":"2024-05-30T17:14:16.307268+0000","last_fullsized":"2024-05-30T17:14:16.307268+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'4","last_scrub_stamp":"2024-05-30T17:14:16.307053+0000","last_deep_scrub":"265'4","last_deep_scrub_stamp":"2024-05-30T17:14:16.307053+0000","last_clean_scrub_stamp":"2024-05-30T17:14:16.307053+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:31:23.411741+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":647,"num_read_kb":647,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"265'114","reported_seq":670,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:10.427586+0000","last_change":"2024-05-30T17:14:10.427657+0000","last_active":"2024-05-30T17:14:10.427586+0000","last_peered":"2024-05-30T17:14:10.427586+0000","last_clean":"2024-05-30T17:14:10.427586+0000","last_became_active":"2024-05-30T15:54:22.849105+0000","last_became_peered":"2024-05-30T15:54:22.849105+0000","last_unstale":"2024-05-30T17:14:10.427586+0000","last_undegraded":"2024-05-30T17:14:10.427586+0000","last_fullsized":"2024-05-30T17:14:10.427586+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'114","last_scrub_stamp":"2024-05-30T17:14:10.427409+0000","last_deep_scrub":"265'114","last_deep_scrub_stamp":"2024-05-30T17:14:10.427409+0000","last_clean_scrub_stamp":"2024-05-30T17:14:10.427409+0000","objects_scrubbed":5,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:10:36.524011+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":120,"num_read_kb":115,"num_write":80,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"265'541","reported_seq":2326,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.806417+0000","last_change":"2024-05-30T17:13:54.806572+0000","last_active":"2024-05-30T17:13:54.806417+0000","last_peered":"2024-05-30T17:13:54.806417+0000","last_clean":"2024-05-30T17:13:54.806417+0000","last_became_active":"2024-05-30T15:54:24.677291+0000","last_became_peered":"2024-05-30T15:54:24.677291+0000","last_unstale":"2024-05-30T17:13:54.806417+0000","last_undegraded":"2024-05-30T17:13:54.806417+0000","last_fullsized":"2024-05-30T17:13:54.806417+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'541","last_scrub_stamp":"2024-05-30T17:13:54.806190+0000","last_deep_scrub":"265'541","last_deep_scrub_stamp":"2024-05-30T17:13:54.806190+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.806190+0000","objects_scrubbed":1,"log_size":41,"log_dups_size":500,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T05:09:18.353958+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"265'136","reported_seq":666,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.859668+0000","last_change":"2024-05-30T17:13:49.859798+0000","last_active":"2024-05-30T17:13:49.859668+0000","last_peered":"2024-05-30T17:13:49.859668+0000","last_clean":"2024-05-30T17:13:49.859668+0000","last_became_active":"2024-05-30T15:54:26.674517+0000","last_became_peered":"2024-05-30T15:54:26.674517+0000","last_unstale":"2024-05-30T17:13:49.859668+0000","last_undegraded":"2024-05-30T17:13:49.859668+0000","last_fullsized":"2024-05-30T17:13:49.859668+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'136","last_scrub_stamp":"2024-05-30T17:13:49.859451+0000","last_deep_scrub":"265'136","last_deep_scrub_stamp":"2024-05-30T17:13:49.859451+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.859451+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:25:15.350422+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":141,"num_read_kb":129,"num_write":332,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"265'410","reported_seq":1957,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:05.044019+0000","last_change":"2024-05-30T17:14:05.044110+0000","last_active":"2024-05-30T17:14:05.044019+0000","last_peered":"2024-05-30T17:14:05.044019+0000","last_clean":"2024-05-30T17:14:05.044019+0000","last_became_active":"2024-05-30T15:54:18.657247+0000","last_became_peered":"2024-05-30T15:54:18.657247+0000","last_unstale":"2024-05-30T17:14:05.044019+0000","last_undegraded":"2024-05-30T17:14:05.044019+0000","last_fullsized":"2024-05-30T17:14:05.044019+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'410","last_scrub_stamp":"2024-05-30T17:14:05.043803+0000","last_deep_scrub":"265'410","last_deep_scrub_stamp":"2024-05-30T17:14:05.043803+0000","last_clean_scrub_stamp":"2024-05-30T17:14:05.043803+0000","objects_scrubbed":8,"log_size":110,"log_dups_size":300,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:16:53.264752+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2106,"num_read_kb":2008,"num_write":461,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":502,"num_omap_keys":2,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"265'185","reported_seq":804,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.106642+0000","last_change":"2024-05-30T17:13:45.106682+0000","last_active":"2024-05-30T17:13:45.106642+0000","last_peered":"2024-05-30T17:13:45.106642+0000","last_clean":"2024-05-30T17:13:45.106642+0000","last_became_active":"2024-05-30T15:54:54.426538+0000","last_became_peered":"2024-05-30T15:54:54.426538+0000","last_unstale":"2024-05-30T17:13:45.106642+0000","last_undegraded":"2024-05-30T17:13:45.106642+0000","last_fullsized":"2024-05-30T17:13:45.106642+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'185","last_scrub_stamp":"2024-05-30T17:13:45.106515+0000","last_deep_scrub":"265'185","last_deep_scrub_stamp":"2024-05-30T17:13:45.106515+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.106515+0000","objects_scrubbed":1,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:52:29.468974+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":443,"num_read_kb":549,"num_write":1012,"num_write_kb":15096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,7],"acting":[2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":527,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:13.044644+0000","last_change":"2024-05-30T17:14:13.044703+0000","last_active":"2024-05-30T17:14:13.044644+0000","last_peered":"2024-05-30T17:14:13.044644+0000","last_clean":"2024-05-30T17:14:13.044644+0000","last_became_active":"2024-05-30T15:54:20.669492+0000","last_became_peered":"2024-05-30T15:54:20.669492+0000","last_unstale":"2024-05-30T17:14:13.044644+0000","last_undegraded":"2024-05-30T17:14:13.044644+0000","last_fullsized":"2024-05-30T17:14:13.044644+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:13.044456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:13.044456+0000","last_clean_scrub_stamp":"2024-05-30T17:14:13.044456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:52:25.523369+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"265'243","reported_seq":890,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:01.276657+0000","last_change":"2024-05-30T17:14:01.276754+0000","last_active":"2024-05-30T17:14:01.276657+0000","last_peered":"2024-05-30T17:14:01.276657+0000","last_clean":"2024-05-30T17:14:01.276657+0000","last_became_active":"2024-05-30T15:54:22.671943+0000","last_became_peered":"2024-05-30T15:54:22.671943+0000","last_unstale":"2024-05-30T17:14:01.276657+0000","last_undegraded":"2024-05-30T17:14:01.276657+0000","last_fullsized":"2024-05-30T17:14:01.276657+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'243","last_scrub_stamp":"2024-05-30T17:14:01.276461+0000","last_deep_scrub":"265'243","last_deep_scrub_stamp":"2024-05-30T17:14:01.276461+0000","last_clean_scrub_stamp":"2024-05-30T17:14:01.276461+0000","objects_scrubbed":8,"log_size":43,"log_dups_size":200,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:31:23.133344+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":313,"num_read_kb":312,"num_write":178,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":509,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:00.785558+0000","last_change":"2024-05-30T17:14:00.785659+0000","last_active":"2024-05-30T17:14:00.785558+0000","last_peered":"2024-05-30T17:14:00.785558+0000","last_clean":"2024-05-30T17:14:00.785558+0000","last_became_active":"2024-05-30T15:54:24.681595+0000","last_became_peered":"2024-05-30T15:54:24.681595+0000","last_unstale":"2024-05-30T17:14:00.785558+0000","last_undegraded":"2024-05-30T17:14:00.785558+0000","last_fullsized":"2024-05-30T17:14:00.785558+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:00.785264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:00.785264+0000","last_clean_scrub_stamp":"2024-05-30T17:14:00.785264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:38:41.393882+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"265'119","reported_seq":635,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.989353+0000","last_change":"2024-05-30T17:13:51.989431+0000","last_active":"2024-05-30T17:13:51.989353+0000","last_peered":"2024-05-30T17:13:51.989353+0000","last_clean":"2024-05-30T17:13:51.989353+0000","last_became_active":"2024-05-30T15:54:26.688518+0000","last_became_peered":"2024-05-30T15:54:26.688518+0000","last_unstale":"2024-05-30T17:13:51.989353+0000","last_undegraded":"2024-05-30T17:13:51.989353+0000","last_fullsized":"2024-05-30T17:13:51.989353+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'119","last_scrub_stamp":"2024-05-30T17:13:51.989161+0000","last_deep_scrub":"265'119","last_deep_scrub_stamp":"2024-05-30T17:13:51.989161+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.989161+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":100,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:46:17.674786+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":619,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":113,"num_read_kb":112,"num_write":261,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"265'758","reported_seq":2277,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:57.780923+0000","last_change":"2024-05-30T17:13:57.781083+0000","last_active":"2024-05-30T17:13:57.780923+0000","last_peered":"2024-05-30T17:13:57.780923+0000","last_clean":"2024-05-30T17:13:57.780923+0000","last_became_active":"2024-05-30T15:54:18.677957+0000","last_became_peered":"2024-05-30T15:54:18.677957+0000","last_unstale":"2024-05-30T17:13:57.780923+0000","last_undegraded":"2024-05-30T17:13:57.780923+0000","last_fullsized":"2024-05-30T17:13:57.780923+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'758","last_scrub_stamp":"2024-05-30T17:13:57.780707+0000","last_deep_scrub":"265'758","last_deep_scrub_stamp":"2024-05-30T17:13:57.780707+0000","last_clean_scrub_stamp":"2024-05-30T17:13:57.780707+0000","objects_scrubbed":2,"log_size":58,"log_dups_size":700,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:16:34.692139+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3440,"num_read_kb":3078,"num_write":1010,"num_write_kb":372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"265'74","reported_seq":639,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:26.767705+0000","last_change":"2024-05-30T17:14:26.767789+0000","last_active":"2024-05-30T17:14:26.767705+0000","last_peered":"2024-05-30T17:14:26.767705+0000","last_clean":"2024-05-30T17:14:26.767705+0000","last_became_active":"2024-05-30T15:54:03.029214+0000","last_became_peered":"2024-05-30T15:54:03.029214+0000","last_unstale":"2024-05-30T17:14:26.767705+0000","last_undegraded":"2024-05-30T17:14:26.767705+0000","last_fullsized":"2024-05-30T17:14:26.767705+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'74","last_scrub_stamp":"2024-05-30T17:14:26.767514+0000","last_deep_scrub":"265'74","last_deep_scrub_stamp":"2024-05-30T17:14:26.767514+0000","last_clean_scrub_stamp":"2024-05-30T17:14:26.767514+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:51:45.091266+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":539,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:19.247044+0000","last_change":"2024-05-30T17:14:19.247196+0000","last_active":"2024-05-30T17:14:19.247044+0000","last_peered":"2024-05-30T17:14:19.247044+0000","last_clean":"2024-05-30T17:14:19.247044+0000","last_became_active":"2024-05-30T15:54:08.577556+0000","last_became_peered":"2024-05-30T15:54:08.577556+0000","last_unstale":"2024-05-30T17:14:19.247044+0000","last_undegraded":"2024-05-30T17:14:19.247044+0000","last_fullsized":"2024-05-30T17:14:19.247044+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:19.246805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:19.246805+0000","last_clean_scrub_stamp":"2024-05-30T17:14:19.246805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:51:24.222735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"265'189","reported_seq":853,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:35.464631+0000","last_change":"2024-05-30T17:14:35.464721+0000","last_active":"2024-05-30T17:14:35.464631+0000","last_peered":"2024-05-30T17:14:35.464631+0000","last_clean":"2024-05-30T17:14:35.464631+0000","last_became_active":"2024-05-30T15:54:16.674229+0000","last_became_peered":"2024-05-30T15:54:16.674229+0000","last_unstale":"2024-05-30T17:14:35.464631+0000","last_undegraded":"2024-05-30T17:14:35.464631+0000","last_fullsized":"2024-05-30T17:14:35.464631+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'189","last_scrub_stamp":"2024-05-30T17:14:35.464439+0000","last_deep_scrub":"265'189","last_deep_scrub_stamp":"2024-05-30T17:14:35.464439+0000","last_clean_scrub_stamp":"2024-05-30T17:14:35.464439+0000","objects_scrubbed":3,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:34:15.501216+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5525719,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":447,"num_read_kb":11470,"num_write":1074,"num_write_kb":25445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":521,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:16.814789+0000","last_change":"2024-05-30T17:14:16.814843+0000","last_active":"2024-05-30T17:14:16.814789+0000","last_peered":"2024-05-30T17:14:16.814789+0000","last_clean":"2024-05-30T17:14:16.814789+0000","last_became_active":"2024-05-30T15:54:20.664317+0000","last_became_peered":"2024-05-30T15:54:20.664317+0000","last_unstale":"2024-05-30T17:14:16.814789+0000","last_undegraded":"2024-05-30T17:14:16.814789+0000","last_fullsized":"2024-05-30T17:14:16.814789+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:16.814584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:16.814584+0000","last_clean_scrub_stamp":"2024-05-30T17:14:16.814584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:59:43.265365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"265'340","reported_seq":1071,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:26.372042+0000","last_change":"2024-05-30T17:14:26.372042+0000","last_active":"2024-05-30T17:14:26.372042+0000","last_peered":"2024-05-30T17:14:26.372042+0000","last_clean":"2024-05-30T17:14:26.372042+0000","last_became_active":"2024-05-30T15:54:22.669871+0000","last_became_peered":"2024-05-30T15:54:22.669871+0000","last_unstale":"2024-05-30T17:14:26.372042+0000","last_undegraded":"2024-05-30T17:14:26.372042+0000","last_fullsized":"2024-05-30T17:14:26.372042+0000","mapping_epoch":28,"log_start":"265'300","ondisk_log_start":"265'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'340","last_scrub_stamp":"2024-05-30T17:14:26.371771+0000","last_deep_scrub":"265'340","last_deep_scrub_stamp":"2024-05-30T17:14:26.371771+0000","last_clean_scrub_stamp":"2024-05-30T17:14:26.371771+0000","objects_scrubbed":6,"log_size":40,"log_dups_size":300,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:33:55.213752+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49272,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":648,"num_read_kb":960,"num_write":598,"num_write_kb":460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":99452,"num_omap_keys":240,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":517,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.868833+0000","last_change":"2024-05-30T17:13:58.868833+0000","last_active":"2024-05-30T17:13:58.868833+0000","last_peered":"2024-05-30T17:13:58.868833+0000","last_clean":"2024-05-30T17:13:58.868833+0000","last_became_active":"2024-05-30T15:54:24.684709+0000","last_became_peered":"2024-05-30T15:54:24.684709+0000","last_unstale":"2024-05-30T17:13:58.868833+0000","last_undegraded":"2024-05-30T17:13:58.868833+0000","last_fullsized":"2024-05-30T17:13:58.868833+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:58.868550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:58.868550+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.868550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:45:56.978968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"265'178","reported_seq":892,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:02.796551+0000","last_change":"2024-05-30T17:14:02.796551+0000","last_active":"2024-05-30T17:14:02.796551+0000","last_peered":"2024-05-30T17:14:02.796551+0000","last_clean":"2024-05-30T17:14:02.796551+0000","last_became_active":"2024-05-30T15:54:26.686647+0000","last_became_peered":"2024-05-30T15:54:26.686647+0000","last_unstale":"2024-05-30T17:14:02.796551+0000","last_undegraded":"2024-05-30T17:14:02.796551+0000","last_fullsized":"2024-05-30T17:14:02.796551+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'178","last_scrub_stamp":"2024-05-30T17:14:02.796290+0000","last_deep_scrub":"265'178","last_deep_scrub_stamp":"2024-05-30T17:14:02.796290+0000","last_clean_scrub_stamp":"2024-05-30T17:14:02.796290+0000","objects_scrubbed":4,"log_size":78,"log_dups_size":100,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:33:47.206690+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1020,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":416,"num_read_kb":404,"num_write":359,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":66,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"265'389","reported_seq":1873,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:14.815492+0000","last_change":"2024-05-30T17:14:14.815492+0000","last_active":"2024-05-30T17:14:14.815492+0000","last_peered":"2024-05-30T17:14:14.815492+0000","last_clean":"2024-05-30T17:14:14.815492+0000","last_became_active":"2024-05-30T15:54:18.648529+0000","last_became_peered":"2024-05-30T15:54:18.648529+0000","last_unstale":"2024-05-30T17:14:14.815492+0000","last_undegraded":"2024-05-30T17:14:14.815492+0000","last_fullsized":"2024-05-30T17:14:14.815492+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'389","last_scrub_stamp":"2024-05-30T17:14:14.815228+0000","last_deep_scrub":"265'389","last_deep_scrub_stamp":"2024-05-30T17:14:14.815228+0000","last_clean_scrub_stamp":"2024-05-30T17:14:14.815228+0000","objects_scrubbed":1,"log_size":89,"log_dups_size":300,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:17:35.526601+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2082,"num_read_kb":1967,"num_write":465,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":541,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:27.733635+0000","last_change":"2024-05-30T17:14:27.733635+0000","last_active":"2024-05-30T17:14:27.733635+0000","last_peered":"2024-05-30T17:14:27.733635+0000","last_clean":"2024-05-30T17:14:27.733635+0000","last_became_active":"2024-05-30T15:54:08.574645+0000","last_became_peered":"2024-05-30T15:54:08.574645+0000","last_unstale":"2024-05-30T17:14:27.733635+0000","last_undegraded":"2024-05-30T17:14:27.733635+0000","last_fullsized":"2024-05-30T17:14:27.733635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:27.733385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:27.733385+0000","last_clean_scrub_stamp":"2024-05-30T17:14:27.733385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:08:57.081739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"265'174","reported_seq":818,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:24.863029+0000","last_change":"2024-05-30T17:14:24.863126+0000","last_active":"2024-05-30T17:14:24.863029+0000","last_peered":"2024-05-30T17:14:24.863029+0000","last_clean":"2024-05-30T17:14:24.863029+0000","last_became_active":"2024-05-30T15:54:16.678386+0000","last_became_peered":"2024-05-30T15:54:16.678386+0000","last_unstale":"2024-05-30T17:14:24.863029+0000","last_undegraded":"2024-05-30T17:14:24.863029+0000","last_fullsized":"2024-05-30T17:14:24.863029+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'174","last_scrub_stamp":"2024-05-30T17:14:24.862823+0000","last_deep_scrub":"265'174","last_deep_scrub_stamp":"2024-05-30T17:14:24.862823+0000","last_clean_scrub_stamp":"2024-05-30T17:14:24.862823+0000","objects_scrubbed":4,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:59:03.844221+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079916,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":364,"num_read_kb":7665,"num_write":1082,"num_write_kb":33234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":520,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:14.305625+0000","last_change":"2024-05-30T17:14:14.305625+0000","last_active":"2024-05-30T17:14:14.305625+0000","last_peered":"2024-05-30T17:14:14.305625+0000","last_clean":"2024-05-30T17:14:14.305625+0000","last_became_active":"2024-05-30T15:54:20.661502+0000","last_became_peered":"2024-05-30T15:54:20.661502+0000","last_unstale":"2024-05-30T17:14:14.305625+0000","last_undegraded":"2024-05-30T17:14:14.305625+0000","last_fullsized":"2024-05-30T17:14:14.305625+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:14.305372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:14.305372+0000","last_clean_scrub_stamp":"2024-05-30T17:14:14.305372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T05:04:18.082051+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"265'730","reported_seq":2569,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:04.002916+0000","last_change":"2024-05-30T17:14:04.003007+0000","last_active":"2024-05-30T17:14:04.002916+0000","last_peered":"2024-05-30T17:14:04.002916+0000","last_clean":"2024-05-30T17:14:04.002916+0000","last_became_active":"2024-05-30T15:54:22.671588+0000","last_became_peered":"2024-05-30T15:54:22.671588+0000","last_unstale":"2024-05-30T17:14:04.002916+0000","last_undegraded":"2024-05-30T17:14:04.002916+0000","last_fullsized":"2024-05-30T17:14:04.002916+0000","mapping_epoch":28,"log_start":"265'700","ondisk_log_start":"265'700","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'730","last_scrub_stamp":"2024-05-30T17:14:04.002697+0000","last_deep_scrub":"265'730","last_deep_scrub_stamp":"2024-05-30T17:14:04.002697+0000","last_clean_scrub_stamp":"2024-05-30T17:14:04.002697+0000","objects_scrubbed":12,"log_size":30,"log_dups_size":700,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:14:57.807605+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19819,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":1008,"num_read_kb":1208,"num_write":324,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6022,"num_omap_keys":18,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":513,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:02.668077+0000","last_change":"2024-05-30T17:14:02.668129+0000","last_active":"2024-05-30T17:14:02.668077+0000","last_peered":"2024-05-30T17:14:02.668077+0000","last_clean":"2024-05-30T17:14:02.668077+0000","last_became_active":"2024-05-30T15:54:24.686641+0000","last_became_peered":"2024-05-30T15:54:24.686641+0000","last_unstale":"2024-05-30T17:14:02.668077+0000","last_undegraded":"2024-05-30T17:14:02.668077+0000","last_fullsized":"2024-05-30T17:14:02.668077+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:02.667880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:02.667880+0000","last_clean_scrub_stamp":"2024-05-30T17:14:02.667880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:55:41.680344+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"265'125","reported_seq":646,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.985175+0000","last_change":"2024-05-30T17:13:54.985175+0000","last_active":"2024-05-30T17:13:54.985175+0000","last_peered":"2024-05-30T17:13:54.985175+0000","last_clean":"2024-05-30T17:13:54.985175+0000","last_became_active":"2024-05-30T15:54:26.667751+0000","last_became_peered":"2024-05-30T15:54:26.667751+0000","last_unstale":"2024-05-30T17:13:54.985175+0000","last_undegraded":"2024-05-30T17:13:54.985175+0000","last_fullsized":"2024-05-30T17:13:54.985175+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'125","last_scrub_stamp":"2024-05-30T17:13:54.984877+0000","last_deep_scrub":"265'125","last_deep_scrub_stamp":"2024-05-30T17:13:54.984877+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.984877+0000","objects_scrubbed":2,"log_size":25,"log_dups_size":100,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:40:14.213968+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":438,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":115,"num_write":291,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"265'328","reported_seq":1658,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:07.012769+0000","last_change":"2024-05-30T17:14:07.012769+0000","last_active":"2024-05-30T17:14:07.012769+0000","last_peered":"2024-05-30T17:14:07.012769+0000","last_clean":"2024-05-30T17:14:07.012769+0000","last_became_active":"2024-05-30T15:54:18.651853+0000","last_became_peered":"2024-05-30T15:54:18.651853+0000","last_unstale":"2024-05-30T17:14:07.012769+0000","last_undegraded":"2024-05-30T17:14:07.012769+0000","last_fullsized":"2024-05-30T17:14:07.012769+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'328","last_scrub_stamp":"2024-05-30T17:14:07.012480+0000","last_deep_scrub":"265'328","last_deep_scrub_stamp":"2024-05-30T17:14:07.012480+0000","last_clean_scrub_stamp":"2024-05-30T17:14:07.012480+0000","objects_scrubbed":2,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:54:27.007726+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1727,"num_read_kb":1636,"num_write":373,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":542,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.261872+0000","last_change":"2024-05-30T17:13:59.261974+0000","last_active":"2024-05-30T17:13:59.261872+0000","last_peered":"2024-05-30T17:13:59.261872+0000","last_clean":"2024-05-30T17:13:59.261872+0000","last_became_active":"2024-05-30T15:54:08.744898+0000","last_became_peered":"2024-05-30T15:54:08.744898+0000","last_unstale":"2024-05-30T17:13:59.261872+0000","last_undegraded":"2024-05-30T17:13:59.261872+0000","last_fullsized":"2024-05-30T17:13:59.261872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:59.261650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:59.261650+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.261650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:03:48.512476+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"265'185","reported_seq":823,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:13.997191+0000","last_change":"2024-05-30T17:14:13.997191+0000","last_active":"2024-05-30T17:14:13.997191+0000","last_peered":"2024-05-30T17:14:13.997191+0000","last_clean":"2024-05-30T17:14:13.997191+0000","last_became_active":"2024-05-30T15:54:16.650214+0000","last_became_peered":"2024-05-30T15:54:16.650214+0000","last_unstale":"2024-05-30T17:14:13.997191+0000","last_undegraded":"2024-05-30T17:14:13.997191+0000","last_fullsized":"2024-05-30T17:14:13.997191+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'185","last_scrub_stamp":"2024-05-30T17:14:13.996657+0000","last_deep_scrub":"265'185","last_deep_scrub_stamp":"2024-05-30T17:14:13.996657+0000","last_clean_scrub_stamp":"2024-05-30T17:14:13.996657+0000","objects_scrubbed":9,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:48:29.511347+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1911070,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":412,"num_read_kb":8332,"num_write":1154,"num_write_kb":25808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"265'2","reported_seq":525,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.146675+0000","last_change":"2024-05-30T17:13:50.146675+0000","last_active":"2024-05-30T17:13:50.146675+0000","last_peered":"2024-05-30T17:13:50.146675+0000","last_clean":"2024-05-30T17:13:50.146675+0000","last_became_active":"2024-05-30T15:54:20.666715+0000","last_became_peered":"2024-05-30T15:54:20.666715+0000","last_unstale":"2024-05-30T17:13:50.146675+0000","last_undegraded":"2024-05-30T17:13:50.146675+0000","last_fullsized":"2024-05-30T17:13:50.146675+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:13:50.146258+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:13:50.146258+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.146258+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:41:50.441025+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"265'268","reported_seq":1790,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:07.764156+0000","last_change":"2024-05-30T17:14:07.764244+0000","last_active":"2024-05-30T17:14:07.764156+0000","last_peered":"2024-05-30T17:14:07.764156+0000","last_clean":"2024-05-30T17:14:07.764156+0000","last_became_active":"2024-05-30T15:54:22.852604+0000","last_became_peered":"2024-05-30T15:54:22.852604+0000","last_unstale":"2024-05-30T17:14:07.764156+0000","last_undegraded":"2024-05-30T17:14:07.764156+0000","last_fullsized":"2024-05-30T17:14:07.764156+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'268","last_scrub_stamp":"2024-05-30T17:14:07.763964+0000","last_deep_scrub":"265'268","last_deep_scrub_stamp":"2024-05-30T17:14:07.763964+0000","last_clean_scrub_stamp":"2024-05-30T17:14:07.763964+0000","objects_scrubbed":9,"log_size":68,"log_dups_size":200,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:30:31.072037+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":232,"num_read_kb":224,"num_write":183,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5824,"num_omap_keys":14,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":518,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:00.884506+0000","last_change":"2024-05-30T17:14:00.884643+0000","last_active":"2024-05-30T17:14:00.884506+0000","last_peered":"2024-05-30T17:14:00.884506+0000","last_clean":"2024-05-30T17:14:00.884506+0000","last_became_active":"2024-05-30T15:54:24.679830+0000","last_became_peered":"2024-05-30T15:54:24.679830+0000","last_unstale":"2024-05-30T17:14:00.884506+0000","last_undegraded":"2024-05-30T17:14:00.884506+0000","last_fullsized":"2024-05-30T17:14:00.884506+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:00.884270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:00.884270+0000","last_clean_scrub_stamp":"2024-05-30T17:14:00.884270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:18:06.001324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"265'253","reported_seq":871,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.356811+0000","last_change":"2024-05-30T17:13:56.356879+0000","last_active":"2024-05-30T17:13:56.356811+0000","last_peered":"2024-05-30T17:13:56.356811+0000","last_clean":"2024-05-30T17:13:56.356811+0000","last_became_active":"2024-05-30T15:54:26.696477+0000","last_became_peered":"2024-05-30T15:54:26.696477+0000","last_unstale":"2024-05-30T17:13:56.356811+0000","last_undegraded":"2024-05-30T17:13:56.356811+0000","last_fullsized":"2024-05-30T17:13:56.356811+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'253","last_scrub_stamp":"2024-05-30T17:13:56.356623+0000","last_deep_scrub":"265'253","last_deep_scrub_stamp":"2024-05-30T17:13:56.356623+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.356623+0000","objects_scrubbed":2,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:15:54.649548+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":476,"num_read_kb":500,"num_write":542,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1193,"num_omap_keys":7,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"265'766","reported_seq":2335,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:11.094921+0000","last_change":"2024-05-30T17:14:11.095077+0000","last_active":"2024-05-30T17:14:11.094921+0000","last_peered":"2024-05-30T17:14:11.094921+0000","last_clean":"2024-05-30T17:14:11.094921+0000","last_became_active":"2024-05-30T15:54:18.639830+0000","last_became_peered":"2024-05-30T15:54:18.639830+0000","last_unstale":"2024-05-30T17:14:11.094921+0000","last_undegraded":"2024-05-30T17:14:11.094921+0000","last_fullsized":"2024-05-30T17:14:11.094921+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'766","last_scrub_stamp":"2024-05-30T17:14:11.094660+0000","last_deep_scrub":"265'766","last_deep_scrub_stamp":"2024-05-30T17:14:11.094660+0000","last_clean_scrub_stamp":"2024-05-30T17:14:11.094660+0000","objects_scrubbed":5,"log_size":66,"log_dups_size":700,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:57:27.825458+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3481,"num_read_kb":3132,"num_write":1014,"num_write_kb":367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":539,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:18.271712+0000","last_change":"2024-05-30T17:14:18.271770+0000","last_active":"2024-05-30T17:14:18.271712+0000","last_peered":"2024-05-30T17:14:18.271712+0000","last_clean":"2024-05-30T17:14:18.271712+0000","last_became_active":"2024-05-30T15:54:08.574817+0000","last_became_peered":"2024-05-30T15:54:08.574817+0000","last_unstale":"2024-05-30T17:14:18.271712+0000","last_undegraded":"2024-05-30T17:14:18.271712+0000","last_fullsized":"2024-05-30T17:14:18.271712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:18.271517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:18.271517+0000","last_clean_scrub_stamp":"2024-05-30T17:14:18.271517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:45:29.263529+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"265'199","reported_seq":923,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:17.027327+0000","last_change":"2024-05-30T17:14:17.027406+0000","last_active":"2024-05-30T17:14:17.027327+0000","last_peered":"2024-05-30T17:14:17.027327+0000","last_clean":"2024-05-30T17:14:17.027327+0000","last_became_active":"2024-05-30T15:54:16.655347+0000","last_became_peered":"2024-05-30T15:54:16.655347+0000","last_unstale":"2024-05-30T17:14:17.027327+0000","last_undegraded":"2024-05-30T17:14:17.027327+0000","last_fullsized":"2024-05-30T17:14:17.027327+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'199","last_scrub_stamp":"2024-05-30T17:14:17.027124+0000","last_deep_scrub":"265'199","last_deep_scrub_stamp":"2024-05-30T17:14:17.027124+0000","last_clean_scrub_stamp":"2024-05-30T17:14:17.027124+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:00:55.705228+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":572,"num_read_kb":42208,"num_write":1148,"num_write_kb":36370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":526,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:13.829376+0000","last_change":"2024-05-30T17:14:13.829430+0000","last_active":"2024-05-30T17:14:13.829376+0000","last_peered":"2024-05-30T17:14:13.829376+0000","last_clean":"2024-05-30T17:14:13.829376+0000","last_became_active":"2024-05-30T15:54:20.651449+0000","last_became_peered":"2024-05-30T15:54:20.651449+0000","last_unstale":"2024-05-30T17:14:13.829376+0000","last_undegraded":"2024-05-30T17:14:13.829376+0000","last_fullsized":"2024-05-30T17:14:13.829376+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:13.829176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:13.829176+0000","last_clean_scrub_stamp":"2024-05-30T17:14:13.829176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:22:24.775288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"265'167","reported_seq":749,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:12.373071+0000","last_change":"2024-05-30T17:14:12.373071+0000","last_active":"2024-05-30T17:14:12.373071+0000","last_peered":"2024-05-30T17:14:12.373071+0000","last_clean":"2024-05-30T17:14:12.373071+0000","last_became_active":"2024-05-30T15:54:22.673881+0000","last_became_peered":"2024-05-30T15:54:22.673881+0000","last_unstale":"2024-05-30T17:14:12.373071+0000","last_undegraded":"2024-05-30T17:14:12.373071+0000","last_fullsized":"2024-05-30T17:14:12.373071+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'167","last_scrub_stamp":"2024-05-30T17:14:12.372906+0000","last_deep_scrub":"265'167","last_deep_scrub_stamp":"2024-05-30T17:14:12.372906+0000","last_clean_scrub_stamp":"2024-05-30T17:14:12.372906+0000","objects_scrubbed":7,"log_size":67,"log_dups_size":100,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:29:18.523344+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":200,"num_read_kb":192,"num_write":144,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1630,"num_omap_keys":4,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":511,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.801223+0000","last_change":"2024-05-30T17:13:56.801223+0000","last_active":"2024-05-30T17:13:56.801223+0000","last_peered":"2024-05-30T17:13:56.801223+0000","last_clean":"2024-05-30T17:13:56.801223+0000","last_became_active":"2024-05-30T15:54:24.675916+0000","last_became_peered":"2024-05-30T15:54:24.675916+0000","last_unstale":"2024-05-30T17:13:56.801223+0000","last_undegraded":"2024-05-30T17:13:56.801223+0000","last_fullsized":"2024-05-30T17:13:56.801223+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:56.800964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:56.800964+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.800964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:50:34.435231+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"265'139","reported_seq":667,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.262568+0000","last_change":"2024-05-30T17:13:49.262568+0000","last_active":"2024-05-30T17:13:49.262568+0000","last_peered":"2024-05-30T17:13:49.262568+0000","last_clean":"2024-05-30T17:13:49.262568+0000","last_became_active":"2024-05-30T15:54:26.704217+0000","last_became_peered":"2024-05-30T15:54:26.704217+0000","last_unstale":"2024-05-30T17:13:49.262568+0000","last_undegraded":"2024-05-30T17:13:49.262568+0000","last_fullsized":"2024-05-30T17:13:49.262568+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'139","last_scrub_stamp":"2024-05-30T17:13:49.262162+0000","last_deep_scrub":"265'139","last_deep_scrub_stamp":"2024-05-30T17:13:49.262162+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.262162+0000","objects_scrubbed":2,"log_size":39,"log_dups_size":100,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:30:17.555273+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":834,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":138,"num_read_kb":136,"num_write":341,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"265'398","reported_seq":1838,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:03.013211+0000","last_change":"2024-05-30T17:14:03.013307+0000","last_active":"2024-05-30T17:14:03.013211+0000","last_peered":"2024-05-30T17:14:03.013211+0000","last_clean":"2024-05-30T17:14:03.013211+0000","last_became_active":"2024-05-30T15:54:18.665034+0000","last_became_peered":"2024-05-30T15:54:18.665034+0000","last_unstale":"2024-05-30T17:14:03.013211+0000","last_undegraded":"2024-05-30T17:14:03.013211+0000","last_fullsized":"2024-05-30T17:14:03.013211+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'398","last_scrub_stamp":"2024-05-30T17:14:03.013020+0000","last_deep_scrub":"265'398","last_deep_scrub_stamp":"2024-05-30T17:14:03.013020+0000","last_clean_scrub_stamp":"2024-05-30T17:14:03.013020+0000","objects_scrubbed":2,"log_size":98,"log_dups_size":300,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:58:48.490896+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2053,"num_read_kb":1920,"num_write":476,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"265'217","reported_seq":867,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:21.298652+0000","last_change":"2024-05-30T17:14:21.298756+0000","last_active":"2024-05-30T17:14:21.298652+0000","last_peered":"2024-05-30T17:14:21.298652+0000","last_clean":"2024-05-30T17:14:21.298652+0000","last_became_active":"2024-05-30T15:54:16.650735+0000","last_became_peered":"2024-05-30T15:54:16.650735+0000","last_unstale":"2024-05-30T17:14:21.298652+0000","last_undegraded":"2024-05-30T17:14:21.298652+0000","last_fullsized":"2024-05-30T17:14:21.298652+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'217","last_scrub_stamp":"2024-05-30T17:14:21.298453+0000","last_deep_scrub":"265'217","last_deep_scrub_stamp":"2024-05-30T17:14:21.298453+0000","last_clean_scrub_stamp":"2024-05-30T17:14:21.298453+0000","objects_scrubbed":6,"log_size":17,"log_dups_size":200,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:06:06.658095+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079967,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":420,"num_read_kb":5617,"num_write":983,"num_write_kb":15732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":540,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:12.043129+0000","last_change":"2024-05-30T17:14:12.043183+0000","last_active":"2024-05-30T17:14:12.043129+0000","last_peered":"2024-05-30T17:14:12.043129+0000","last_clean":"2024-05-30T17:14:12.043129+0000","last_became_active":"2024-05-30T15:54:08.567900+0000","last_became_peered":"2024-05-30T15:54:08.567900+0000","last_unstale":"2024-05-30T17:14:12.043129+0000","last_undegraded":"2024-05-30T17:14:12.043129+0000","last_fullsized":"2024-05-30T17:14:12.043129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:12.042933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:12.042933+0000","last_clean_scrub_stamp":"2024-05-30T17:14:12.042933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:36:17.591809+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":526,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:15.771566+0000","last_change":"2024-05-30T17:14:15.771620+0000","last_active":"2024-05-30T17:14:15.771566+0000","last_peered":"2024-05-30T17:14:15.771566+0000","last_clean":"2024-05-30T17:14:15.771566+0000","last_became_active":"2024-05-30T15:54:20.653790+0000","last_became_peered":"2024-05-30T15:54:20.653790+0000","last_unstale":"2024-05-30T17:14:15.771566+0000","last_undegraded":"2024-05-30T17:14:15.771566+0000","last_fullsized":"2024-05-30T17:14:15.771566+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:15.771368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:15.771368+0000","last_clean_scrub_stamp":"2024-05-30T17:14:15.771368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:07:25.961584+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"265'371","reported_seq":1081,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:08.931785+0000","last_change":"2024-05-30T17:14:08.931785+0000","last_active":"2024-05-30T17:14:08.931785+0000","last_peered":"2024-05-30T17:14:08.931785+0000","last_clean":"2024-05-30T17:14:08.931785+0000","last_became_active":"2024-05-30T15:54:22.673903+0000","last_became_peered":"2024-05-30T15:54:22.673903+0000","last_unstale":"2024-05-30T17:14:08.931785+0000","last_undegraded":"2024-05-30T17:14:08.931785+0000","last_fullsized":"2024-05-30T17:14:08.931785+0000","mapping_epoch":28,"log_start":"265'300","ondisk_log_start":"265'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'371","last_scrub_stamp":"2024-05-30T17:14:08.931442+0000","last_deep_scrub":"265'371","last_deep_scrub_stamp":"2024-05-30T17:14:08.931442+0000","last_clean_scrub_stamp":"2024-05-30T17:14:08.931442+0000","objects_scrubbed":10,"log_size":71,"log_dups_size":300,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:46:35.284674+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11442,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1140,"num_read_kb":1226,"num_write":1674,"num_write_kb":1475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":496492,"num_omap_keys":1134,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":522,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:55.903746+0000","last_change":"2024-05-30T17:13:55.903798+0000","last_active":"2024-05-30T17:13:55.903746+0000","last_peered":"2024-05-30T17:13:55.903746+0000","last_clean":"2024-05-30T17:13:55.903746+0000","last_became_active":"2024-05-30T15:54:24.677198+0000","last_became_peered":"2024-05-30T15:54:24.677198+0000","last_unstale":"2024-05-30T17:13:55.903746+0000","last_undegraded":"2024-05-30T17:13:55.903746+0000","last_fullsized":"2024-05-30T17:13:55.903746+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:55.903559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:55.903559+0000","last_clean_scrub_stamp":"2024-05-30T17:13:55.903559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:20:32.475018+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"265'122","reported_seq":649,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:57.847760+0000","last_change":"2024-05-30T17:13:57.847843+0000","last_active":"2024-05-30T17:13:57.847760+0000","last_peered":"2024-05-30T17:13:57.847760+0000","last_clean":"2024-05-30T17:13:57.847760+0000","last_became_active":"2024-05-30T15:54:26.676078+0000","last_became_peered":"2024-05-30T15:54:26.676078+0000","last_unstale":"2024-05-30T17:13:57.847760+0000","last_undegraded":"2024-05-30T17:13:57.847760+0000","last_fullsized":"2024-05-30T17:13:57.847760+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'122","last_scrub_stamp":"2024-05-30T17:13:57.847569+0000","last_deep_scrub":"265'122","last_deep_scrub_stamp":"2024-05-30T17:13:57.847569+0000","last_clean_scrub_stamp":"2024-05-30T17:13:57.847569+0000","objects_scrubbed":5,"log_size":22,"log_dups_size":100,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:27:31.982550+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":727,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":114,"num_read_kb":108,"num_write":274,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":73,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"265'797","reported_seq":2438,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:20.820346+0000","last_change":"2024-05-30T17:14:20.820437+0000","last_active":"2024-05-30T17:14:20.820346+0000","last_peered":"2024-05-30T17:14:20.820346+0000","last_clean":"2024-05-30T17:14:20.820346+0000","last_became_active":"2024-05-30T15:54:18.642683+0000","last_became_peered":"2024-05-30T15:54:18.642683+0000","last_unstale":"2024-05-30T17:14:20.820346+0000","last_undegraded":"2024-05-30T17:14:20.820346+0000","last_fullsized":"2024-05-30T17:14:20.820346+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'797","last_scrub_stamp":"2024-05-30T17:14:20.820153+0000","last_deep_scrub":"265'797","last_deep_scrub_stamp":"2024-05-30T17:14:20.820153+0000","last_clean_scrub_stamp":"2024-05-30T17:14:20.820153+0000","objects_scrubbed":6,"log_size":97,"log_dups_size":700,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:01:34.756808+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3596,"num_read_kb":3243,"num_write":1037,"num_write_kb":360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1011,"num_omap_keys":4,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"265'184","reported_seq":836,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:16.998554+0000","last_change":"2024-05-30T17:14:16.998697+0000","last_active":"2024-05-30T17:14:16.998554+0000","last_peered":"2024-05-30T17:14:16.998554+0000","last_clean":"2024-05-30T17:14:16.998554+0000","last_became_active":"2024-05-30T15:54:16.663360+0000","last_became_peered":"2024-05-30T15:54:16.663360+0000","last_unstale":"2024-05-30T17:14:16.998554+0000","last_undegraded":"2024-05-30T17:14:16.998554+0000","last_fullsized":"2024-05-30T17:14:16.998554+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'184","last_scrub_stamp":"2024-05-30T17:14:16.998338+0000","last_deep_scrub":"265'184","last_deep_scrub_stamp":"2024-05-30T17:14:16.998338+0000","last_clean_scrub_stamp":"2024-05-30T17:14:16.998338+0000","objects_scrubbed":6,"log_size":84,"log_dups_size":100,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:35:16.806460+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":661325,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":407,"num_read_kb":30298,"num_write":1063,"num_write_kb":42840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":539,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:28.740933+0000","last_change":"2024-05-30T17:14:28.740986+0000","last_active":"2024-05-30T17:14:28.740933+0000","last_peered":"2024-05-30T17:14:28.740933+0000","last_clean":"2024-05-30T17:14:28.740933+0000","last_became_active":"2024-05-30T15:54:08.576779+0000","last_became_peered":"2024-05-30T15:54:08.576779+0000","last_unstale":"2024-05-30T17:14:28.740933+0000","last_undegraded":"2024-05-30T17:14:28.740933+0000","last_fullsized":"2024-05-30T17:14:28.740933+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:28.740742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:28.740742+0000","last_clean_scrub_stamp":"2024-05-30T17:14:28.740742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:21:53.610258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":520,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:55.817272+0000","last_change":"2024-05-30T17:13:55.817326+0000","last_active":"2024-05-30T17:13:55.817272+0000","last_peered":"2024-05-30T17:13:55.817272+0000","last_clean":"2024-05-30T17:13:55.817272+0000","last_became_active":"2024-05-30T15:54:20.655220+0000","last_became_peered":"2024-05-30T15:54:20.655220+0000","last_unstale":"2024-05-30T17:13:55.817272+0000","last_undegraded":"2024-05-30T17:13:55.817272+0000","last_fullsized":"2024-05-30T17:13:55.817272+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:55.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:55.817064+0000","last_clean_scrub_stamp":"2024-05-30T17:13:55.817064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:12:59.140457+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"265'204","reported_seq":810,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:06.066922+0000","last_change":"2024-05-30T17:14:06.067009+0000","last_active":"2024-05-30T17:14:06.066922+0000","last_peered":"2024-05-30T17:14:06.066922+0000","last_clean":"2024-05-30T17:14:06.066922+0000","last_became_active":"2024-05-30T15:54:22.851338+0000","last_became_peered":"2024-05-30T15:54:22.851338+0000","last_unstale":"2024-05-30T17:14:06.066922+0000","last_undegraded":"2024-05-30T17:14:06.066922+0000","last_fullsized":"2024-05-30T17:14:06.066922+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'204","last_scrub_stamp":"2024-05-30T17:14:06.066726+0000","last_deep_scrub":"265'204","last_deep_scrub_stamp":"2024-05-30T17:14:06.066726+0000","last_clean_scrub_stamp":"2024-05-30T17:14:06.066726+0000","objects_scrubbed":7,"log_size":104,"log_dups_size":100,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:29:42.457191+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":256,"num_read_kb":249,"num_write":168,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"265'557","reported_seq":2775,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:57.243776+0000","last_change":"2024-05-30T17:13:57.243776+0000","last_active":"2024-05-30T17:13:57.243776+0000","last_peered":"2024-05-30T17:13:57.243776+0000","last_clean":"2024-05-30T17:13:57.243776+0000","last_became_active":"2024-05-30T15:54:24.677176+0000","last_became_peered":"2024-05-30T15:54:24.677176+0000","last_unstale":"2024-05-30T17:13:57.243776+0000","last_undegraded":"2024-05-30T17:13:57.243776+0000","last_fullsized":"2024-05-30T17:13:57.243776+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'557","last_scrub_stamp":"2024-05-30T17:13:57.243509+0000","last_deep_scrub":"265'557","last_deep_scrub_stamp":"2024-05-30T17:13:57.243509+0000","last_clean_scrub_stamp":"2024-05-30T17:13:57.243509+0000","objects_scrubbed":1,"log_size":57,"log_dups_size":500,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:04:49.675199+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"265'131","reported_seq":647,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.355450+0000","last_change":"2024-05-30T17:13:58.355552+0000","last_active":"2024-05-30T17:13:58.355450+0000","last_peered":"2024-05-30T17:13:58.355450+0000","last_clean":"2024-05-30T17:13:58.355450+0000","last_became_active":"2024-05-30T15:54:26.697294+0000","last_became_peered":"2024-05-30T15:54:26.697294+0000","last_unstale":"2024-05-30T17:13:58.355450+0000","last_undegraded":"2024-05-30T17:13:58.355450+0000","last_fullsized":"2024-05-30T17:13:58.355450+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'131","last_scrub_stamp":"2024-05-30T17:13:58.355257+0000","last_deep_scrub":"265'131","last_deep_scrub_stamp":"2024-05-30T17:13:58.355257+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.355257+0000","objects_scrubbed":7,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:43:23.516003+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1933,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":112,"num_read_kb":106,"num_write":288,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":64,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"265'720","reported_seq":2124,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:22.405228+0000","last_change":"2024-05-30T17:14:22.405228+0000","last_active":"2024-05-30T17:14:22.405228+0000","last_peered":"2024-05-30T17:14:22.405228+0000","last_clean":"2024-05-30T17:14:22.405228+0000","last_became_active":"2024-05-30T15:54:18.652981+0000","last_became_peered":"2024-05-30T15:54:18.652981+0000","last_unstale":"2024-05-30T17:14:22.405228+0000","last_undegraded":"2024-05-30T17:14:22.405228+0000","last_fullsized":"2024-05-30T17:14:22.405228+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'720","last_scrub_stamp":"2024-05-30T17:14:22.404963+0000","last_deep_scrub":"265'720","last_deep_scrub_stamp":"2024-05-30T17:14:22.404963+0000","last_clean_scrub_stamp":"2024-05-30T17:14:22.404963+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":700,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T05:09:41.233665+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3098,"num_read_kb":2766,"num_write":954,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":272,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"265'200","reported_seq":872,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:29.436665+0000","last_change":"2024-05-30T17:14:29.436784+0000","last_active":"2024-05-30T17:14:29.436665+0000","last_peered":"2024-05-30T17:14:29.436665+0000","last_clean":"2024-05-30T17:14:29.436665+0000","last_became_active":"2024-05-30T15:54:16.652033+0000","last_became_peered":"2024-05-30T15:54:16.652033+0000","last_unstale":"2024-05-30T17:14:29.436665+0000","last_undegraded":"2024-05-30T17:14:29.436665+0000","last_fullsized":"2024-05-30T17:14:29.436665+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'200","last_scrub_stamp":"2024-05-30T17:14:29.436451+0000","last_deep_scrub":"265'200","last_deep_scrub_stamp":"2024-05-30T17:14:29.436451+0000","last_clean_scrub_stamp":"2024-05-30T17:14:29.436451+0000","objects_scrubbed":5,"log_size":100,"log_dups_size":100,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:03:07.624018+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":287689,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":515,"num_read_kb":15333,"num_write":1052,"num_write_kb":22470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":539,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:13.005555+0000","last_change":"2024-05-30T17:14:13.005555+0000","last_active":"2024-05-30T17:14:13.005555+0000","last_peered":"2024-05-30T17:14:13.005555+0000","last_clean":"2024-05-30T17:14:13.005555+0000","last_became_active":"2024-05-30T15:54:08.573754+0000","last_became_peered":"2024-05-30T15:54:08.573754+0000","last_unstale":"2024-05-30T17:14:13.005555+0000","last_undegraded":"2024-05-30T17:14:13.005555+0000","last_fullsized":"2024-05-30T17:14:13.005555+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:13.005314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:13.005314+0000","last_clean_scrub_stamp":"2024-05-30T17:14:13.005314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:57:00.677681+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":530,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:09.067533+0000","last_change":"2024-05-30T17:14:09.067590+0000","last_active":"2024-05-30T17:14:09.067533+0000","last_peered":"2024-05-30T17:14:09.067533+0000","last_clean":"2024-05-30T17:14:09.067533+0000","last_became_active":"2024-05-30T15:54:20.656941+0000","last_became_peered":"2024-05-30T15:54:20.656941+0000","last_unstale":"2024-05-30T17:14:09.067533+0000","last_undegraded":"2024-05-30T17:14:09.067533+0000","last_fullsized":"2024-05-30T17:14:09.067533+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:09.067331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:09.067331+0000","last_clean_scrub_stamp":"2024-05-30T17:14:09.067331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:25:27.763720+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"265'135","reported_seq":717,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:57.901626+0000","last_change":"2024-05-30T17:13:57.901801+0000","last_active":"2024-05-30T17:13:57.901626+0000","last_peered":"2024-05-30T17:13:57.901626+0000","last_clean":"2024-05-30T17:13:57.901626+0000","last_became_active":"2024-05-30T15:54:22.674027+0000","last_became_peered":"2024-05-30T15:54:22.674027+0000","last_unstale":"2024-05-30T17:13:57.901626+0000","last_undegraded":"2024-05-30T17:13:57.901626+0000","last_fullsized":"2024-05-30T17:13:57.901626+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'135","last_scrub_stamp":"2024-05-30T17:13:57.901361+0000","last_deep_scrub":"265'135","last_deep_scrub_stamp":"2024-05-30T17:13:57.901361+0000","last_clean_scrub_stamp":"2024-05-30T17:13:57.901361+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":100,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:02:52.230529+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":164,"num_read_kb":159,"num_write":108,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"265'550","reported_seq":2642,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:08.404324+0000","last_change":"2024-05-30T17:14:08.404392+0000","last_active":"2024-05-30T17:14:08.404324+0000","last_peered":"2024-05-30T17:14:08.404324+0000","last_clean":"2024-05-30T17:14:08.404324+0000","last_became_active":"2024-05-30T15:54:24.676608+0000","last_became_peered":"2024-05-30T15:54:24.676608+0000","last_unstale":"2024-05-30T17:14:08.404324+0000","last_undegraded":"2024-05-30T17:14:08.404324+0000","last_fullsized":"2024-05-30T17:14:08.404324+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'550","last_scrub_stamp":"2024-05-30T17:14:08.404133+0000","last_deep_scrub":"265'550","last_deep_scrub_stamp":"2024-05-30T17:14:08.404133+0000","last_clean_scrub_stamp":"2024-05-30T17:14:08.404133+0000","objects_scrubbed":1,"log_size":50,"log_dups_size":500,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:32:14.717679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"265'233","reported_seq":789,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.655070+0000","last_change":"2024-05-30T17:13:50.655156+0000","last_active":"2024-05-30T17:13:50.655070+0000","last_peered":"2024-05-30T17:13:50.655070+0000","last_clean":"2024-05-30T17:13:50.655070+0000","last_became_active":"2024-05-30T15:54:26.699148+0000","last_became_peered":"2024-05-30T15:54:26.699148+0000","last_unstale":"2024-05-30T17:13:50.655070+0000","last_undegraded":"2024-05-30T17:13:50.655070+0000","last_fullsized":"2024-05-30T17:13:50.655070+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'233","last_scrub_stamp":"2024-05-30T17:13:50.654846+0000","last_deep_scrub":"265'233","last_deep_scrub_stamp":"2024-05-30T17:13:50.654846+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.654846+0000","objects_scrubbed":6,"log_size":33,"log_dups_size":200,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:23:30.288859+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":326,"num_read_kb":298,"num_write":504,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":178,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"265'748","reported_seq":2349,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:08.073027+0000","last_change":"2024-05-30T17:14:08.073095+0000","last_active":"2024-05-30T17:14:08.073027+0000","last_peered":"2024-05-30T17:14:08.073027+0000","last_clean":"2024-05-30T17:14:08.073027+0000","last_became_active":"2024-05-30T15:54:18.657842+0000","last_became_peered":"2024-05-30T15:54:18.657842+0000","last_unstale":"2024-05-30T17:14:08.073027+0000","last_undegraded":"2024-05-30T17:14:08.073027+0000","last_fullsized":"2024-05-30T17:14:08.073027+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'748","last_scrub_stamp":"2024-05-30T17:14:08.072843+0000","last_deep_scrub":"265'748","last_deep_scrub_stamp":"2024-05-30T17:14:08.072843+0000","last_clean_scrub_stamp":"2024-05-30T17:14:08.072843+0000","objects_scrubbed":1,"log_size":48,"log_dups_size":700,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:49:34.177372+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3632,"num_read_kb":3285,"num_write":997,"num_write_kb":370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1532,"num_omap_keys":6,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"265'238","reported_seq":1004,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:30.760818+0000","last_change":"2024-05-30T17:14:30.760939+0000","last_active":"2024-05-30T17:14:30.760818+0000","last_peered":"2024-05-30T17:14:30.760818+0000","last_clean":"2024-05-30T17:14:30.760818+0000","last_became_active":"2024-05-30T15:54:16.657067+0000","last_became_peered":"2024-05-30T15:54:16.657067+0000","last_unstale":"2024-05-30T17:14:30.760818+0000","last_undegraded":"2024-05-30T17:14:30.760818+0000","last_fullsized":"2024-05-30T17:14:30.760818+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'238","last_scrub_stamp":"2024-05-30T17:14:30.760570+0000","last_deep_scrub":"265'238","last_deep_scrub_stamp":"2024-05-30T17:14:30.760570+0000","last_clean_scrub_stamp":"2024-05-30T17:14:30.760570+0000","objects_scrubbed":4,"log_size":38,"log_dups_size":200,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:47:18.792673+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5418111,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":685,"num_read_kb":72437,"num_write":1157,"num_write_kb":30547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":537,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:06.733343+0000","last_change":"2024-05-30T17:14:06.733343+0000","last_active":"2024-05-30T17:14:06.733343+0000","last_peered":"2024-05-30T17:14:06.733343+0000","last_clean":"2024-05-30T17:14:06.733343+0000","last_became_active":"2024-05-30T15:54:08.577191+0000","last_became_peered":"2024-05-30T15:54:08.577191+0000","last_unstale":"2024-05-30T17:14:06.733343+0000","last_undegraded":"2024-05-30T17:14:06.733343+0000","last_fullsized":"2024-05-30T17:14:06.733343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:06.733080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:06.733080+0000","last_clean_scrub_stamp":"2024-05-30T17:14:06.733080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:02:40.927625+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":525,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:17.373292+0000","last_change":"2024-05-30T17:14:17.373292+0000","last_active":"2024-05-30T17:14:17.373292+0000","last_peered":"2024-05-30T17:14:17.373292+0000","last_clean":"2024-05-30T17:14:17.373292+0000","last_became_active":"2024-05-30T15:54:20.653404+0000","last_became_peered":"2024-05-30T15:54:20.653404+0000","last_unstale":"2024-05-30T17:14:17.373292+0000","last_undegraded":"2024-05-30T17:14:17.373292+0000","last_fullsized":"2024-05-30T17:14:17.373292+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:17.373049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:17.373049+0000","last_clean_scrub_stamp":"2024-05-30T17:14:17.373049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:52:07.685494+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"265'354","reported_seq":1110,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:34.766591+0000","last_change":"2024-05-30T17:14:34.766705+0000","last_active":"2024-05-30T17:14:34.766591+0000","last_peered":"2024-05-30T17:14:34.766591+0000","last_clean":"2024-05-30T17:14:34.766591+0000","last_became_active":"2024-05-30T15:54:22.848761+0000","last_became_peered":"2024-05-30T15:54:22.848761+0000","last_unstale":"2024-05-30T17:14:34.766591+0000","last_undegraded":"2024-05-30T17:14:34.766591+0000","last_fullsized":"2024-05-30T17:14:34.766591+0000","mapping_epoch":28,"log_start":"265'300","ondisk_log_start":"265'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'354","last_scrub_stamp":"2024-05-30T17:14:34.765964+0000","last_deep_scrub":"265'354","last_deep_scrub_stamp":"2024-05-30T17:14:34.765964+0000","last_clean_scrub_stamp":"2024-05-30T17:14:34.765964+0000","objects_scrubbed":8,"log_size":54,"log_dups_size":300,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:37:18.193004+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24415,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":525,"num_read_kb":761,"num_write":274,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":515,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:06.395596+0000","last_change":"2024-05-30T17:14:06.395648+0000","last_active":"2024-05-30T17:14:06.395596+0000","last_peered":"2024-05-30T17:14:06.395596+0000","last_clean":"2024-05-30T17:14:06.395596+0000","last_became_active":"2024-05-30T15:54:24.686801+0000","last_became_peered":"2024-05-30T15:54:24.686801+0000","last_unstale":"2024-05-30T17:14:06.395596+0000","last_undegraded":"2024-05-30T17:14:06.395596+0000","last_fullsized":"2024-05-30T17:14:06.395596+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:06.395403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:06.395403+0000","last_clean_scrub_stamp":"2024-05-30T17:14:06.395403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:26:52.769715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"265'152","reported_seq":662,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.241219+0000","last_change":"2024-05-30T17:13:51.241295+0000","last_active":"2024-05-30T17:13:51.241219+0000","last_peered":"2024-05-30T17:13:51.241219+0000","last_clean":"2024-05-30T17:13:51.241219+0000","last_became_active":"2024-05-30T15:54:26.700325+0000","last_became_peered":"2024-05-30T15:54:26.700325+0000","last_unstale":"2024-05-30T17:13:51.241219+0000","last_undegraded":"2024-05-30T17:13:51.241219+0000","last_fullsized":"2024-05-30T17:13:51.241219+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'152","last_scrub_stamp":"2024-05-30T17:13:51.241024+0000","last_deep_scrub":"265'152","last_deep_scrub_stamp":"2024-05-30T17:13:51.241024+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.241024+0000","objects_scrubbed":2,"log_size":52,"log_dups_size":100,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:14:44.872712+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":204,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":115,"num_write":342,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"265'150","reported_seq":732,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.796700+0000","last_change":"2024-05-30T17:13:56.796775+0000","last_active":"2024-05-30T17:13:56.796700+0000","last_peered":"2024-05-30T17:13:56.796700+0000","last_clean":"2024-05-30T17:13:56.796700+0000","last_became_active":"2024-05-30T15:54:26.686310+0000","last_became_peered":"2024-05-30T15:54:26.686310+0000","last_unstale":"2024-05-30T17:13:56.796700+0000","last_undegraded":"2024-05-30T17:13:56.796700+0000","last_fullsized":"2024-05-30T17:13:56.796700+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'150","last_scrub_stamp":"2024-05-30T17:13:56.796467+0000","last_deep_scrub":"265'150","last_deep_scrub_stamp":"2024-05-30T17:13:56.796467+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.796467+0000","objects_scrubbed":1,"log_size":50,"log_dups_size":100,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:01:08.998802+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":227,"num_read_kb":222,"num_write":347,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"265'747","reported_seq":2056,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:05.009608+0000","last_change":"2024-05-30T17:14:05.009756+0000","last_active":"2024-05-30T17:14:05.009608+0000","last_peered":"2024-05-30T17:14:05.009608+0000","last_clean":"2024-05-30T17:14:05.009608+0000","last_became_active":"2024-05-30T15:54:18.673418+0000","last_became_peered":"2024-05-30T15:54:18.673418+0000","last_unstale":"2024-05-30T17:14:05.009608+0000","last_undegraded":"2024-05-30T17:14:05.009608+0000","last_fullsized":"2024-05-30T17:14:05.009608+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'747","last_scrub_stamp":"2024-05-30T17:14:05.009388+0000","last_deep_scrub":"265'747","last_deep_scrub_stamp":"2024-05-30T17:14:05.009388+0000","last_clean_scrub_stamp":"2024-05-30T17:14:05.009388+0000","objects_scrubbed":1,"log_size":47,"log_dups_size":700,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:56:33.977718+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3258,"num_read_kb":2851,"num_write":1044,"num_write_kb":425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":556,"num_omap_keys":3,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"265'202","reported_seq":831,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:49.153045+0000","last_change":"2024-05-30T17:13:49.153045+0000","last_active":"2024-05-30T17:13:49.153045+0000","last_peered":"2024-05-30T17:13:49.153045+0000","last_clean":"2024-05-30T17:13:49.153045+0000","last_became_active":"2024-05-30T15:54:54.450966+0000","last_became_peered":"2024-05-30T15:54:54.450966+0000","last_unstale":"2024-05-30T17:13:49.153045+0000","last_undegraded":"2024-05-30T17:13:49.153045+0000","last_fullsized":"2024-05-30T17:13:49.153045+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'202","last_scrub_stamp":"2024-05-30T17:13:49.152709+0000","last_deep_scrub":"265'202","last_deep_scrub_stamp":"2024-05-30T17:13:49.152709+0000","last_clean_scrub_stamp":"2024-05-30T17:13:49.152709+0000","objects_scrubbed":6,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:19:50.214056+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262272,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":401,"num_read_kb":38159,"num_write":1164,"num_write_kb":35962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":518,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:00.085249+0000","last_change":"2024-05-30T17:14:00.085304+0000","last_active":"2024-05-30T17:14:00.085249+0000","last_peered":"2024-05-30T17:14:00.085249+0000","last_clean":"2024-05-30T17:14:00.085249+0000","last_became_active":"2024-05-30T15:54:20.664882+0000","last_became_peered":"2024-05-30T15:54:20.664882+0000","last_unstale":"2024-05-30T17:14:00.085249+0000","last_undegraded":"2024-05-30T17:14:00.085249+0000","last_fullsized":"2024-05-30T17:14:00.085249+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:00.085062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:00.085062+0000","last_clean_scrub_stamp":"2024-05-30T17:14:00.085062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:36:44.188047+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"265'158","reported_seq":766,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:27.374925+0000","last_change":"2024-05-30T17:14:27.375001+0000","last_active":"2024-05-30T17:14:27.374925+0000","last_peered":"2024-05-30T17:14:27.374925+0000","last_clean":"2024-05-30T17:14:27.374925+0000","last_became_active":"2024-05-30T15:54:22.672170+0000","last_became_peered":"2024-05-30T15:54:22.672170+0000","last_unstale":"2024-05-30T17:14:27.374925+0000","last_undegraded":"2024-05-30T17:14:27.374925+0000","last_fullsized":"2024-05-30T17:14:27.374925+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'158","last_scrub_stamp":"2024-05-30T17:14:27.374727+0000","last_deep_scrub":"265'158","last_deep_scrub_stamp":"2024-05-30T17:14:27.374727+0000","last_clean_scrub_stamp":"2024-05-30T17:14:27.374727+0000","objects_scrubbed":5,"log_size":58,"log_dups_size":100,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:50:38.280039+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5933,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":205,"num_read_kb":259,"num_write":110,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":511,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:04.400786+0000","last_change":"2024-05-30T17:14:04.400840+0000","last_active":"2024-05-30T17:14:04.400786+0000","last_peered":"2024-05-30T17:14:04.400786+0000","last_clean":"2024-05-30T17:14:04.400786+0000","last_became_active":"2024-05-30T15:54:24.688053+0000","last_became_peered":"2024-05-30T15:54:24.688053+0000","last_unstale":"2024-05-30T17:14:04.400786+0000","last_undegraded":"2024-05-30T17:14:04.400786+0000","last_fullsized":"2024-05-30T17:14:04.400786+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:04.400563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:04.400563+0000","last_clean_scrub_stamp":"2024-05-30T17:14:04.400563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:58:36.746394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"265'120","reported_seq":655,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.374814+0000","last_change":"2024-05-30T17:13:54.374814+0000","last_active":"2024-05-30T17:13:54.374814+0000","last_peered":"2024-05-30T17:13:54.374814+0000","last_clean":"2024-05-30T17:13:54.374814+0000","last_became_active":"2024-05-30T15:54:26.701588+0000","last_became_peered":"2024-05-30T15:54:26.701588+0000","last_unstale":"2024-05-30T17:13:54.374814+0000","last_undegraded":"2024-05-30T17:13:54.374814+0000","last_fullsized":"2024-05-30T17:13:54.374814+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'120","last_scrub_stamp":"2024-05-30T17:13:54.374529+0000","last_deep_scrub":"265'120","last_deep_scrub_stamp":"2024-05-30T17:13:54.374529+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.374529+0000","objects_scrubbed":3,"log_size":20,"log_dups_size":100,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:54:34.109921+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":826,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":172,"num_read_kb":153,"num_write":293,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"265'323","reported_seq":1770,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:01.069483+0000","last_change":"2024-05-30T17:14:01.069557+0000","last_active":"2024-05-30T17:14:01.069483+0000","last_peered":"2024-05-30T17:14:01.069483+0000","last_clean":"2024-05-30T17:14:01.069483+0000","last_became_active":"2024-05-30T15:54:18.670707+0000","last_became_peered":"2024-05-30T15:54:18.670707+0000","last_unstale":"2024-05-30T17:14:01.069483+0000","last_undegraded":"2024-05-30T17:14:01.069483+0000","last_fullsized":"2024-05-30T17:14:01.069483+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'323","last_scrub_stamp":"2024-05-30T17:14:01.069299+0000","last_deep_scrub":"265'323","last_deep_scrub_stamp":"2024-05-30T17:14:01.069299+0000","last_clean_scrub_stamp":"2024-05-30T17:14:01.069299+0000","objects_scrubbed":2,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:28:00.081754+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1910,"num_read_kb":1828,"num_write":376,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"265'219","reported_seq":893,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:23.735734+0000","last_change":"2024-05-30T17:14:23.735830+0000","last_active":"2024-05-30T17:14:23.735734+0000","last_peered":"2024-05-30T17:14:23.735734+0000","last_clean":"2024-05-30T17:14:23.735734+0000","last_became_active":"2024-05-30T15:54:16.660427+0000","last_became_peered":"2024-05-30T15:54:16.660427+0000","last_unstale":"2024-05-30T17:14:23.735734+0000","last_undegraded":"2024-05-30T17:14:23.735734+0000","last_fullsized":"2024-05-30T17:14:23.735734+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'219","last_scrub_stamp":"2024-05-30T17:14:23.735541+0000","last_deep_scrub":"265'219","last_deep_scrub_stamp":"2024-05-30T17:14:23.735541+0000","last_clean_scrub_stamp":"2024-05-30T17:14:23.735541+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":200,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:18:24.875280+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262181,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":518,"num_read_kb":33224,"num_write":1148,"num_write_kb":46290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":519,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:15.367435+0000","last_change":"2024-05-30T17:14:15.367493+0000","last_active":"2024-05-30T17:14:15.367435+0000","last_peered":"2024-05-30T17:14:15.367435+0000","last_clean":"2024-05-30T17:14:15.367435+0000","last_became_active":"2024-05-30T15:54:20.649330+0000","last_became_peered":"2024-05-30T15:54:20.649330+0000","last_unstale":"2024-05-30T17:14:15.367435+0000","last_undegraded":"2024-05-30T17:14:15.367435+0000","last_fullsized":"2024-05-30T17:14:15.367435+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:15.367236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:15.367236+0000","last_clean_scrub_stamp":"2024-05-30T17:14:15.367236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:47:00.342990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"265'74","reported_seq":608,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:21.824735+0000","last_change":"2024-05-30T17:14:21.824807+0000","last_active":"2024-05-30T17:14:21.824735+0000","last_peered":"2024-05-30T17:14:21.824735+0000","last_clean":"2024-05-30T17:14:21.824735+0000","last_became_active":"2024-05-30T15:54:22.662738+0000","last_became_peered":"2024-05-30T15:54:22.662738+0000","last_unstale":"2024-05-30T17:14:21.824735+0000","last_undegraded":"2024-05-30T17:14:21.824735+0000","last_fullsized":"2024-05-30T17:14:21.824735+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'74","last_scrub_stamp":"2024-05-30T17:14:21.824479+0000","last_deep_scrub":"265'74","last_deep_scrub_stamp":"2024-05-30T17:14:21.824479+0000","last_clean_scrub_stamp":"2024-05-30T17:14:21.824479+0000","objects_scrubbed":3,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:23:35.198468+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":50,"num_write":40,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":588,"num_omap_keys":4,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":509,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:05.756275+0000","last_change":"2024-05-30T17:14:05.756326+0000","last_active":"2024-05-30T17:14:05.756275+0000","last_peered":"2024-05-30T17:14:05.756275+0000","last_clean":"2024-05-30T17:14:05.756275+0000","last_became_active":"2024-05-30T15:54:24.680578+0000","last_became_peered":"2024-05-30T15:54:24.680578+0000","last_unstale":"2024-05-30T17:14:05.756275+0000","last_undegraded":"2024-05-30T17:14:05.756275+0000","last_fullsized":"2024-05-30T17:14:05.756275+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:05.756098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:05.756098+0000","last_clean_scrub_stamp":"2024-05-30T17:14:05.756098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:07:27.828294+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"265'215","reported_seq":785,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:54.750473+0000","last_change":"2024-05-30T17:13:54.750473+0000","last_active":"2024-05-30T17:13:54.750473+0000","last_peered":"2024-05-30T17:13:54.750473+0000","last_clean":"2024-05-30T17:13:54.750473+0000","last_became_active":"2024-05-30T15:54:26.686553+0000","last_became_peered":"2024-05-30T15:54:26.686553+0000","last_unstale":"2024-05-30T17:13:54.750473+0000","last_undegraded":"2024-05-30T17:13:54.750473+0000","last_fullsized":"2024-05-30T17:13:54.750473+0000","mapping_epoch":32,"log_start":"265'200","ondisk_log_start":"265'200","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'215","last_scrub_stamp":"2024-05-30T17:13:54.750033+0000","last_deep_scrub":"265'215","last_deep_scrub_stamp":"2024-05-30T17:13:54.750033+0000","last_clean_scrub_stamp":"2024-05-30T17:13:54.750033+0000","objects_scrubbed":3,"log_size":15,"log_dups_size":200,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:23:19.977063+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":816,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":414,"num_read_kb":383,"num_write":513,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":398,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"265'419","reported_seq":1805,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:14.364566+0000","last_change":"2024-05-30T17:14:14.364672+0000","last_active":"2024-05-30T17:14:14.364566+0000","last_peered":"2024-05-30T17:14:14.364566+0000","last_clean":"2024-05-30T17:14:14.364566+0000","last_became_active":"2024-05-30T15:54:18.660089+0000","last_became_peered":"2024-05-30T15:54:18.660089+0000","last_unstale":"2024-05-30T17:14:14.364566+0000","last_undegraded":"2024-05-30T17:14:14.364566+0000","last_fullsized":"2024-05-30T17:14:14.364566+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'419","last_scrub_stamp":"2024-05-30T17:14:14.364379+0000","last_deep_scrub":"265'419","last_deep_scrub_stamp":"2024-05-30T17:14:14.364379+0000","last_clean_scrub_stamp":"2024-05-30T17:14:14.364379+0000","objects_scrubbed":6,"log_size":19,"log_dups_size":400,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:24:09.477211+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2036,"num_read_kb":1890,"num_write":519,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"265'217","reported_seq":896,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:20.299717+0000","last_change":"2024-05-30T17:14:20.299925+0000","last_active":"2024-05-30T17:14:20.299717+0000","last_peered":"2024-05-30T17:14:20.299717+0000","last_clean":"2024-05-30T17:14:20.299717+0000","last_became_active":"2024-05-30T15:54:16.650025+0000","last_became_peered":"2024-05-30T15:54:16.650025+0000","last_unstale":"2024-05-30T17:14:20.299717+0000","last_undegraded":"2024-05-30T17:14:20.299717+0000","last_fullsized":"2024-05-30T17:14:20.299717+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'217","last_scrub_stamp":"2024-05-30T17:14:20.299487+0000","last_deep_scrub":"265'217","last_deep_scrub_stamp":"2024-05-30T17:14:20.299487+0000","last_clean_scrub_stamp":"2024-05-30T17:14:20.299487+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":200,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:30:25.801340+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5569939,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":500,"num_read_kb":102670,"num_write":1113,"num_write_kb":39714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":517,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:12.802057+0000","last_change":"2024-05-30T17:14:12.802110+0000","last_active":"2024-05-30T17:14:12.802057+0000","last_peered":"2024-05-30T17:14:12.802057+0000","last_clean":"2024-05-30T17:14:12.802057+0000","last_became_active":"2024-05-30T15:54:20.656663+0000","last_became_peered":"2024-05-30T15:54:20.656663+0000","last_unstale":"2024-05-30T17:14:12.802057+0000","last_undegraded":"2024-05-30T17:14:12.802057+0000","last_fullsized":"2024-05-30T17:14:12.802057+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:12.801868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:12.801868+0000","last_clean_scrub_stamp":"2024-05-30T17:14:12.801868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:29:15.815405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"265'191","reported_seq":815,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:37.548757+0000","last_change":"2024-05-30T17:14:37.548845+0000","last_active":"2024-05-30T17:14:37.548757+0000","last_peered":"2024-05-30T17:14:37.548757+0000","last_clean":"2024-05-30T17:14:37.548757+0000","last_became_active":"2024-05-30T15:54:22.673673+0000","last_became_peered":"2024-05-30T15:54:22.673673+0000","last_unstale":"2024-05-30T17:14:37.548757+0000","last_undegraded":"2024-05-30T17:14:37.548757+0000","last_fullsized":"2024-05-30T17:14:37.548757+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'191","last_scrub_stamp":"2024-05-30T17:14:37.548524+0000","last_deep_scrub":"265'191","last_deep_scrub_stamp":"2024-05-30T17:14:37.548524+0000","last_clean_scrub_stamp":"2024-05-30T17:14:37.548524+0000","objects_scrubbed":5,"log_size":91,"log_dups_size":100,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:03:19.783707+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7114,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":259,"num_read_kb":332,"num_write":146,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":508,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.754848+0000","last_change":"2024-05-30T17:13:50.754954+0000","last_active":"2024-05-30T17:13:50.754848+0000","last_peered":"2024-05-30T17:13:50.754848+0000","last_clean":"2024-05-30T17:13:50.754848+0000","last_became_active":"2024-05-30T15:54:24.675289+0000","last_became_peered":"2024-05-30T15:54:24.675289+0000","last_unstale":"2024-05-30T17:13:50.754848+0000","last_undegraded":"2024-05-30T17:13:50.754848+0000","last_fullsized":"2024-05-30T17:13:50.754848+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:50.754554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:50.754554+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.754554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:47:56.533373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"265'141","reported_seq":647,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:50.257049+0000","last_change":"2024-05-30T17:13:50.257049+0000","last_active":"2024-05-30T17:13:50.257049+0000","last_peered":"2024-05-30T17:13:50.257049+0000","last_clean":"2024-05-30T17:13:50.257049+0000","last_became_active":"2024-05-30T15:54:26.703433+0000","last_became_peered":"2024-05-30T15:54:26.703433+0000","last_unstale":"2024-05-30T17:13:50.257049+0000","last_undegraded":"2024-05-30T17:13:50.257049+0000","last_fullsized":"2024-05-30T17:13:50.257049+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'141","last_scrub_stamp":"2024-05-30T17:13:50.256770+0000","last_deep_scrub":"265'141","last_deep_scrub_stamp":"2024-05-30T17:13:50.256770+0000","last_clean_scrub_stamp":"2024-05-30T17:13:50.256770+0000","objects_scrubbed":3,"log_size":41,"log_dups_size":100,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:08:34.180219+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":856,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":110,"num_write":329,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"265'867","reported_seq":2504,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:28.423699+0000","last_change":"2024-05-30T17:14:28.423766+0000","last_active":"2024-05-30T17:14:28.423699+0000","last_peered":"2024-05-30T17:14:28.423699+0000","last_clean":"2024-05-30T17:14:28.423699+0000","last_became_active":"2024-05-30T15:54:18.649273+0000","last_became_peered":"2024-05-30T15:54:18.649273+0000","last_unstale":"2024-05-30T17:14:28.423699+0000","last_undegraded":"2024-05-30T17:14:28.423699+0000","last_fullsized":"2024-05-30T17:14:28.423699+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'867","last_scrub_stamp":"2024-05-30T17:14:28.423514+0000","last_deep_scrub":"265'867","last_deep_scrub_stamp":"2024-05-30T17:14:28.423514+0000","last_clean_scrub_stamp":"2024-05-30T17:14:28.423514+0000","objects_scrubbed":4,"log_size":67,"log_dups_size":800,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:34:11.095216+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3918,"num_read_kb":3498,"num_write":1187,"num_write_kb":458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":401,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"265'207","reported_seq":877,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:29.742800+0000","last_change":"2024-05-30T17:14:29.742916+0000","last_active":"2024-05-30T17:14:29.742800+0000","last_peered":"2024-05-30T17:14:29.742800+0000","last_clean":"2024-05-30T17:14:29.742800+0000","last_became_active":"2024-05-30T15:54:16.649121+0000","last_became_peered":"2024-05-30T15:54:16.649121+0000","last_unstale":"2024-05-30T17:14:29.742800+0000","last_undegraded":"2024-05-30T17:14:29.742800+0000","last_fullsized":"2024-05-30T17:14:29.742800+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'207","last_scrub_stamp":"2024-05-30T17:14:29.742576+0000","last_deep_scrub":"265'207","last_deep_scrub_stamp":"2024-05-30T17:14:29.742576+0000","last_clean_scrub_stamp":"2024-05-30T17:14:29.742576+0000","objects_scrubbed":6,"log_size":107,"log_dups_size":100,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:05:54.402014+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1029196,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":493,"num_read_kb":65907,"num_write":1120,"num_write_kb":54558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":516,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:51.736994+0000","last_change":"2024-05-30T17:13:51.737047+0000","last_active":"2024-05-30T17:13:51.736994+0000","last_peered":"2024-05-30T17:13:51.736994+0000","last_clean":"2024-05-30T17:13:51.736994+0000","last_became_active":"2024-05-30T15:54:20.667863+0000","last_became_peered":"2024-05-30T15:54:20.667863+0000","last_unstale":"2024-05-30T17:13:51.736994+0000","last_undegraded":"2024-05-30T17:13:51.736994+0000","last_fullsized":"2024-05-30T17:13:51.736994+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:51.736796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:51.736796+0000","last_clean_scrub_stamp":"2024-05-30T17:13:51.736796+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:58:41.051038+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"265'147","reported_seq":746,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:22.771867+0000","last_change":"2024-05-30T17:14:22.772012+0000","last_active":"2024-05-30T17:14:22.771867+0000","last_peered":"2024-05-30T17:14:22.771867+0000","last_clean":"2024-05-30T17:14:22.771867+0000","last_became_active":"2024-05-30T15:54:22.664977+0000","last_became_peered":"2024-05-30T15:54:22.664977+0000","last_unstale":"2024-05-30T17:14:22.771867+0000","last_undegraded":"2024-05-30T17:14:22.771867+0000","last_fullsized":"2024-05-30T17:14:22.771867+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'147","last_scrub_stamp":"2024-05-30T17:14:22.771618+0000","last_deep_scrub":"265'147","last_deep_scrub_stamp":"2024-05-30T17:14:22.771618+0000","last_clean_scrub_stamp":"2024-05-30T17:14:22.771618+0000","objects_scrubbed":4,"log_size":47,"log_dups_size":100,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:06:27.800101+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7290,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":187,"num_read_kb":260,"num_write":98,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":514,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:02.866942+0000","last_change":"2024-05-30T17:14:02.866997+0000","last_active":"2024-05-30T17:14:02.866942+0000","last_peered":"2024-05-30T17:14:02.866942+0000","last_clean":"2024-05-30T17:14:02.866942+0000","last_became_active":"2024-05-30T15:54:24.676707+0000","last_became_peered":"2024-05-30T15:54:24.676707+0000","last_unstale":"2024-05-30T17:14:02.866942+0000","last_undegraded":"2024-05-30T17:14:02.866942+0000","last_fullsized":"2024-05-30T17:14:02.866942+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:02.866743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:02.866743+0000","last_clean_scrub_stamp":"2024-05-30T17:14:02.866743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:19:34.249137+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"265'443","reported_seq":2109,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.771973+0000","last_change":"2024-05-30T17:13:59.772041+0000","last_active":"2024-05-30T17:13:59.771973+0000","last_peered":"2024-05-30T17:13:59.771973+0000","last_clean":"2024-05-30T17:13:59.771973+0000","last_became_active":"2024-05-30T15:54:18.670756+0000","last_became_peered":"2024-05-30T15:54:18.670756+0000","last_unstale":"2024-05-30T17:13:59.771973+0000","last_undegraded":"2024-05-30T17:13:59.771973+0000","last_fullsized":"2024-05-30T17:13:59.771973+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'443","last_scrub_stamp":"2024-05-30T17:13:59.771783+0000","last_deep_scrub":"265'443","last_deep_scrub_stamp":"2024-05-30T17:13:59.771783+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.771783+0000","objects_scrubbed":1,"log_size":43,"log_dups_size":400,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:50:25.853396+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2383,"num_read_kb":2277,"num_write":522,"num_write_kb":97,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"265'164","reported_seq":778,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:30.419107+0000","last_change":"2024-05-30T17:14:30.419198+0000","last_active":"2024-05-30T17:14:30.419107+0000","last_peered":"2024-05-30T17:14:30.419107+0000","last_clean":"2024-05-30T17:14:30.419107+0000","last_became_active":"2024-05-30T15:54:16.653560+0000","last_became_peered":"2024-05-30T15:54:16.653560+0000","last_unstale":"2024-05-30T17:14:30.419107+0000","last_undegraded":"2024-05-30T17:14:30.419107+0000","last_fullsized":"2024-05-30T17:14:30.419107+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'164","last_scrub_stamp":"2024-05-30T17:14:30.418894+0000","last_deep_scrub":"265'164","last_deep_scrub_stamp":"2024-05-30T17:14:30.418894+0000","last_clean_scrub_stamp":"2024-05-30T17:14:30.418894+0000","objects_scrubbed":3,"log_size":64,"log_dups_size":100,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:31:08.828045+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5417712,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":326,"num_read_kb":31913,"num_write":888,"num_write_kb":20958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":528,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:17.827438+0000","last_change":"2024-05-30T17:14:17.827438+0000","last_active":"2024-05-30T17:14:17.827438+0000","last_peered":"2024-05-30T17:14:17.827438+0000","last_clean":"2024-05-30T17:14:17.827438+0000","last_became_active":"2024-05-30T15:54:20.662669+0000","last_became_peered":"2024-05-30T15:54:20.662669+0000","last_unstale":"2024-05-30T17:14:17.827438+0000","last_undegraded":"2024-05-30T17:14:17.827438+0000","last_fullsized":"2024-05-30T17:14:17.827438+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:17.827181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:17.827181+0000","last_clean_scrub_stamp":"2024-05-30T17:14:17.827181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:03:52.723170+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"265'228","reported_seq":887,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:02.038392+0000","last_change":"2024-05-30T17:14:02.038392+0000","last_active":"2024-05-30T17:14:02.038392+0000","last_peered":"2024-05-30T17:14:02.038392+0000","last_clean":"2024-05-30T17:14:02.038392+0000","last_became_active":"2024-05-30T15:54:22.663518+0000","last_became_peered":"2024-05-30T15:54:22.663518+0000","last_unstale":"2024-05-30T17:14:02.038392+0000","last_undegraded":"2024-05-30T17:14:02.038392+0000","last_fullsized":"2024-05-30T17:14:02.038392+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'228","last_scrub_stamp":"2024-05-30T17:14:02.038127+0000","last_deep_scrub":"265'228","last_deep_scrub_stamp":"2024-05-30T17:14:02.038127+0000","last_clean_scrub_stamp":"2024-05-30T17:14:02.038127+0000","objects_scrubbed":6,"log_size":28,"log_dups_size":200,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:44:55.851829+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11676,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":342,"num_read_kb":490,"num_write":222,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":24966,"num_omap_keys":48,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":507,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:55.195111+0000","last_change":"2024-05-30T17:13:55.195111+0000","last_active":"2024-05-30T17:13:55.195111+0000","last_peered":"2024-05-30T17:13:55.195111+0000","last_clean":"2024-05-30T17:13:55.195111+0000","last_became_active":"2024-05-30T15:54:24.682609+0000","last_became_peered":"2024-05-30T15:54:24.682609+0000","last_unstale":"2024-05-30T17:13:55.195111+0000","last_undegraded":"2024-05-30T17:13:55.195111+0000","last_fullsized":"2024-05-30T17:13:55.195111+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:55.194814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:55.194814+0000","last_clean_scrub_stamp":"2024-05-30T17:13:55.194814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:35:33.515747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"265'151","reported_seq":672,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:00.314401+0000","last_change":"2024-05-30T17:14:00.314480+0000","last_active":"2024-05-30T17:14:00.314401+0000","last_peered":"2024-05-30T17:14:00.314401+0000","last_clean":"2024-05-30T17:14:00.314401+0000","last_became_active":"2024-05-30T15:54:26.695210+0000","last_became_peered":"2024-05-30T15:54:26.695210+0000","last_unstale":"2024-05-30T17:14:00.314401+0000","last_undegraded":"2024-05-30T17:14:00.314401+0000","last_fullsized":"2024-05-30T17:14:00.314401+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'151","last_scrub_stamp":"2024-05-30T17:14:00.314210+0000","last_deep_scrub":"265'151","last_deep_scrub_stamp":"2024-05-30T17:14:00.314210+0000","last_clean_scrub_stamp":"2024-05-30T17:14:00.314210+0000","objects_scrubbed":2,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:57:02.003537+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":643,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":167,"num_read_kb":179,"num_write":338,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"265'857","reported_seq":2488,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:22.852426+0000","last_change":"2024-05-30T17:14:22.852426+0000","last_active":"2024-05-30T17:14:22.852426+0000","last_peered":"2024-05-30T17:14:22.852426+0000","last_clean":"2024-05-30T17:14:22.852426+0000","last_became_active":"2024-05-30T15:54:18.640739+0000","last_became_peered":"2024-05-30T15:54:18.640739+0000","last_unstale":"2024-05-30T17:14:22.852426+0000","last_undegraded":"2024-05-30T17:14:22.852426+0000","last_fullsized":"2024-05-30T17:14:22.852426+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'857","last_scrub_stamp":"2024-05-30T17:14:22.852147+0000","last_deep_scrub":"265'857","last_deep_scrub_stamp":"2024-05-30T17:14:22.852147+0000","last_clean_scrub_stamp":"2024-05-30T17:14:22.852147+0000","objects_scrubbed":3,"log_size":57,"log_dups_size":800,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:22:06.520996+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4053,"num_read_kb":3626,"num_write":1150,"num_write_kb":447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"265'245","reported_seq":988,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.935811+0000","last_change":"2024-05-30T17:13:47.935811+0000","last_active":"2024-05-30T17:13:47.935811+0000","last_peered":"2024-05-30T17:13:47.935811+0000","last_clean":"2024-05-30T17:13:47.935811+0000","last_became_active":"2024-05-30T15:54:54.424825+0000","last_became_peered":"2024-05-30T15:54:54.424825+0000","last_unstale":"2024-05-30T17:13:47.935811+0000","last_undegraded":"2024-05-30T17:13:47.935811+0000","last_fullsized":"2024-05-30T17:13:47.935811+0000","mapping_epoch":35,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'245","last_scrub_stamp":"2024-05-30T17:13:47.935515+0000","last_deep_scrub":"265'245","last_deep_scrub_stamp":"2024-05-30T17:13:47.935515+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.935515+0000","objects_scrubbed":3,"log_size":45,"log_dups_size":200,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:50:12.821107+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5285497,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":887,"num_read_kb":12756,"num_write":1230,"num_write_kb":34615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"265'4","reported_seq":848,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.789874+0000","last_change":"2024-05-30T17:13:53.789936+0000","last_active":"2024-05-30T17:13:53.789874+0000","last_peered":"2024-05-30T17:13:53.789874+0000","last_clean":"2024-05-30T17:13:53.789874+0000","last_became_active":"2024-05-30T15:54:20.664095+0000","last_became_peered":"2024-05-30T15:54:20.664095+0000","last_unstale":"2024-05-30T17:13:53.789874+0000","last_undegraded":"2024-05-30T17:13:53.789874+0000","last_fullsized":"2024-05-30T17:13:53.789874+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'4","last_scrub_stamp":"2024-05-30T17:13:53.789681+0000","last_deep_scrub":"265'4","last_deep_scrub_stamp":"2024-05-30T17:13:53.789681+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.789681+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:25:01.268608+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":494,"num_read_kb":492,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"265'202","reported_seq":822,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:20.840701+0000","last_change":"2024-05-30T17:14:20.840701+0000","last_active":"2024-05-30T17:14:20.840701+0000","last_peered":"2024-05-30T17:14:20.840701+0000","last_clean":"2024-05-30T17:14:20.840701+0000","last_became_active":"2024-05-30T15:54:22.664997+0000","last_became_peered":"2024-05-30T15:54:22.664997+0000","last_unstale":"2024-05-30T17:14:20.840701+0000","last_undegraded":"2024-05-30T17:14:20.840701+0000","last_fullsized":"2024-05-30T17:14:20.840701+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'202","last_scrub_stamp":"2024-05-30T17:14:20.840427+0000","last_deep_scrub":"265'202","last_deep_scrub_stamp":"2024-05-30T17:14:20.840427+0000","last_clean_scrub_stamp":"2024-05-30T17:14:20.840427+0000","objects_scrubbed":8,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:09:02.158801+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25625,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":290,"num_read_kb":412,"num_write":192,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11460,"num_omap_keys":24,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":508,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:10.050358+0000","last_change":"2024-05-30T17:14:10.050358+0000","last_active":"2024-05-30T17:14:10.050358+0000","last_peered":"2024-05-30T17:14:10.050358+0000","last_clean":"2024-05-30T17:14:10.050358+0000","last_became_active":"2024-05-30T15:54:24.675583+0000","last_became_peered":"2024-05-30T15:54:24.675583+0000","last_unstale":"2024-05-30T17:14:10.050358+0000","last_undegraded":"2024-05-30T17:14:10.050358+0000","last_fullsized":"2024-05-30T17:14:10.050358+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:10.050095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:10.050095+0000","last_clean_scrub_stamp":"2024-05-30T17:14:10.050095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:05:37.764975+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"265'148","reported_seq":673,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:14.828641+0000","last_change":"2024-05-30T17:14:14.828728+0000","last_active":"2024-05-30T17:14:14.828641+0000","last_peered":"2024-05-30T17:14:14.828641+0000","last_clean":"2024-05-30T17:14:14.828641+0000","last_became_active":"2024-05-30T15:54:26.700392+0000","last_became_peered":"2024-05-30T15:54:26.700392+0000","last_unstale":"2024-05-30T17:14:14.828641+0000","last_undegraded":"2024-05-30T17:14:14.828641+0000","last_fullsized":"2024-05-30T17:14:14.828641+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'148","last_scrub_stamp":"2024-05-30T17:14:14.828458+0000","last_deep_scrub":"265'148","last_deep_scrub_stamp":"2024-05-30T17:14:14.828458+0000","last_clean_scrub_stamp":"2024-05-30T17:14:14.828458+0000","objects_scrubbed":8,"log_size":48,"log_dups_size":100,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:48:23.720296+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2627,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":177,"num_read_kb":174,"num_write":403,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"265'371","reported_seq":1751,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.352650+0000","last_change":"2024-05-30T17:13:53.352650+0000","last_active":"2024-05-30T17:13:53.352650+0000","last_peered":"2024-05-30T17:13:53.352650+0000","last_clean":"2024-05-30T17:13:53.352650+0000","last_became_active":"2024-05-30T15:54:54.419468+0000","last_became_peered":"2024-05-30T15:54:54.419468+0000","last_unstale":"2024-05-30T17:13:53.352650+0000","last_undegraded":"2024-05-30T17:13:53.352650+0000","last_fullsized":"2024-05-30T17:13:53.352650+0000","mapping_epoch":35,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'371","last_scrub_stamp":"2024-05-30T17:13:53.352335+0000","last_deep_scrub":"265'371","last_deep_scrub_stamp":"2024-05-30T17:13:53.352335+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.352335+0000","objects_scrubbed":3,"log_size":71,"log_dups_size":300,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:30:41.234405+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1898,"num_read_kb":1782,"num_write":455,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":310,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"265'159","reported_seq":894,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:19.091877+0000","last_change":"2024-05-30T17:14:19.092022+0000","last_active":"2024-05-30T17:14:19.091877+0000","last_peered":"2024-05-30T17:14:19.091877+0000","last_clean":"2024-05-30T17:14:19.091877+0000","last_became_active":"2024-05-30T15:54:16.676440+0000","last_became_peered":"2024-05-30T15:54:16.676440+0000","last_unstale":"2024-05-30T17:14:19.091877+0000","last_undegraded":"2024-05-30T17:14:19.091877+0000","last_fullsized":"2024-05-30T17:14:19.091877+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'159","last_scrub_stamp":"2024-05-30T17:14:19.091627+0000","last_deep_scrub":"265'159","last_deep_scrub_stamp":"2024-05-30T17:14:19.091627+0000","last_clean_scrub_stamp":"2024-05-30T17:14:19.091627+0000","objects_scrubbed":6,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:13:38.026022+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456485,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":869,"num_read_kb":45599,"num_write":925,"num_write_kb":34636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":524,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:07.042918+0000","last_change":"2024-05-30T17:14:07.042918+0000","last_active":"2024-05-30T17:14:07.042918+0000","last_peered":"2024-05-30T17:14:07.042918+0000","last_clean":"2024-05-30T17:14:07.042918+0000","last_became_active":"2024-05-30T15:54:20.655388+0000","last_became_peered":"2024-05-30T15:54:20.655388+0000","last_unstale":"2024-05-30T17:14:07.042918+0000","last_undegraded":"2024-05-30T17:14:07.042918+0000","last_fullsized":"2024-05-30T17:14:07.042918+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:07.042677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:07.042677+0000","last_clean_scrub_stamp":"2024-05-30T17:14:07.042677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:08:28.451745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"265'170","reported_seq":782,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:09.437645+0000","last_change":"2024-05-30T17:14:09.437817+0000","last_active":"2024-05-30T17:14:09.437645+0000","last_peered":"2024-05-30T17:14:09.437645+0000","last_clean":"2024-05-30T17:14:09.437645+0000","last_became_active":"2024-05-30T15:54:22.849193+0000","last_became_peered":"2024-05-30T15:54:22.849193+0000","last_unstale":"2024-05-30T17:14:09.437645+0000","last_undegraded":"2024-05-30T17:14:09.437645+0000","last_fullsized":"2024-05-30T17:14:09.437645+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'170","last_scrub_stamp":"2024-05-30T17:14:09.437396+0000","last_deep_scrub":"265'170","last_deep_scrub_stamp":"2024-05-30T17:14:09.437396+0000","last_clean_scrub_stamp":"2024-05-30T17:14:09.437396+0000","objects_scrubbed":5,"log_size":70,"log_dups_size":100,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:29:49.223193+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11086,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":221,"num_read_kb":302,"num_write":124,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":516,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:19.980627+0000","last_change":"2024-05-30T17:14:19.980627+0000","last_active":"2024-05-30T17:14:19.980627+0000","last_peered":"2024-05-30T17:14:19.980627+0000","last_clean":"2024-05-30T17:14:19.980627+0000","last_became_active":"2024-05-30T15:54:24.681609+0000","last_became_peered":"2024-05-30T15:54:24.681609+0000","last_unstale":"2024-05-30T17:14:19.980627+0000","last_undegraded":"2024-05-30T17:14:19.980627+0000","last_fullsized":"2024-05-30T17:14:19.980627+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:19.980274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:19.980274+0000","last_clean_scrub_stamp":"2024-05-30T17:14:19.980274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:01:43.948910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"265'132","reported_seq":644,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.247687+0000","last_change":"2024-05-30T17:13:47.247831+0000","last_active":"2024-05-30T17:13:47.247687+0000","last_peered":"2024-05-30T17:13:47.247687+0000","last_clean":"2024-05-30T17:13:47.247687+0000","last_became_active":"2024-05-30T15:54:26.704842+0000","last_became_peered":"2024-05-30T15:54:26.704842+0000","last_unstale":"2024-05-30T17:13:47.247687+0000","last_undegraded":"2024-05-30T17:13:47.247687+0000","last_fullsized":"2024-05-30T17:13:47.247687+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'132","last_scrub_stamp":"2024-05-30T17:13:47.247463+0000","last_deep_scrub":"265'132","last_deep_scrub_stamp":"2024-05-30T17:13:47.247463+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.247463+0000","objects_scrubbed":3,"log_size":32,"log_dups_size":100,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:58:57.717875+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":383,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":127,"num_read_kb":122,"num_write":322,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"265'761","reported_seq":2269,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:02.365540+0000","last_change":"2024-05-30T17:14:02.365540+0000","last_active":"2024-05-30T17:14:02.365540+0000","last_peered":"2024-05-30T17:14:02.365540+0000","last_clean":"2024-05-30T17:14:02.365540+0000","last_became_active":"2024-05-30T15:54:18.634803+0000","last_became_peered":"2024-05-30T15:54:18.634803+0000","last_unstale":"2024-05-30T17:14:02.365540+0000","last_undegraded":"2024-05-30T17:14:02.365540+0000","last_fullsized":"2024-05-30T17:14:02.365540+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'761","last_scrub_stamp":"2024-05-30T17:14:02.365235+0000","last_deep_scrub":"265'761","last_deep_scrub_stamp":"2024-05-30T17:14:02.365235+0000","last_clean_scrub_stamp":"2024-05-30T17:14:02.365235+0000","objects_scrubbed":8,"log_size":61,"log_dups_size":700,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:38:15.880298+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":3476,"num_read_kb":3109,"num_write":1006,"num_write_kb":389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1743,"num_omap_keys":8,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"265'190","reported_seq":824,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:47.153543+0000","last_change":"2024-05-30T17:13:47.153543+0000","last_active":"2024-05-30T17:13:47.153543+0000","last_peered":"2024-05-30T17:13:47.153543+0000","last_clean":"2024-05-30T17:13:47.153543+0000","last_became_active":"2024-05-30T15:54:54.430490+0000","last_became_peered":"2024-05-30T15:54:54.430490+0000","last_unstale":"2024-05-30T17:13:47.153543+0000","last_undegraded":"2024-05-30T17:13:47.153543+0000","last_fullsized":"2024-05-30T17:13:47.153543+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'190","last_scrub_stamp":"2024-05-30T17:13:47.153254+0000","last_deep_scrub":"265'190","last_deep_scrub_stamp":"2024-05-30T17:13:47.153254+0000","last_clean_scrub_stamp":"2024-05-30T17:13:47.153254+0000","objects_scrubbed":4,"log_size":90,"log_dups_size":100,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:50:08.663040+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":481,"num_read_kb":40323,"num_write":1075,"num_write_kb":31617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":516,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.000902+0000","last_change":"2024-05-30T17:13:56.000902+0000","last_active":"2024-05-30T17:13:56.000902+0000","last_peered":"2024-05-30T17:13:56.000902+0000","last_clean":"2024-05-30T17:13:56.000902+0000","last_became_active":"2024-05-30T15:54:20.650944+0000","last_became_peered":"2024-05-30T15:54:20.650944+0000","last_unstale":"2024-05-30T17:13:56.000902+0000","last_undegraded":"2024-05-30T17:13:56.000902+0000","last_fullsized":"2024-05-30T17:13:56.000902+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:56.000601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:56.000601+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.000601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:28:54.937679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"265'120","reported_seq":688,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:56.917856+0000","last_change":"2024-05-30T17:13:56.917856+0000","last_active":"2024-05-30T17:13:56.917856+0000","last_peered":"2024-05-30T17:13:56.917856+0000","last_clean":"2024-05-30T17:13:56.917856+0000","last_became_active":"2024-05-30T15:54:22.850865+0000","last_became_peered":"2024-05-30T15:54:22.850865+0000","last_unstale":"2024-05-30T17:13:56.917856+0000","last_undegraded":"2024-05-30T17:13:56.917856+0000","last_fullsized":"2024-05-30T17:13:56.917856+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'120","last_scrub_stamp":"2024-05-30T17:13:56.917584+0000","last_deep_scrub":"265'120","last_deep_scrub_stamp":"2024-05-30T17:13:56.917584+0000","last_clean_scrub_stamp":"2024-05-30T17:13:56.917584+0000","objects_scrubbed":7,"log_size":20,"log_dups_size":100,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:18:29.769236+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":127,"num_read_kb":121,"num_write":92,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1780,"num_omap_keys":6,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"265'541","reported_seq":4178,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.730062+0000","last_change":"2024-05-30T17:13:58.730127+0000","last_active":"2024-05-30T17:13:58.730062+0000","last_peered":"2024-05-30T17:13:58.730062+0000","last_clean":"2024-05-30T17:13:58.730062+0000","last_became_active":"2024-05-30T15:54:24.680695+0000","last_became_peered":"2024-05-30T15:54:24.680695+0000","last_unstale":"2024-05-30T17:13:58.730062+0000","last_undegraded":"2024-05-30T17:13:58.730062+0000","last_fullsized":"2024-05-30T17:13:58.730062+0000","mapping_epoch":30,"log_start":"265'500","ondisk_log_start":"265'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'541","last_scrub_stamp":"2024-05-30T17:13:58.729875+0000","last_deep_scrub":"265'541","last_deep_scrub_stamp":"2024-05-30T17:13:58.729875+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.729875+0000","objects_scrubbed":1,"log_size":41,"log_dups_size":500,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:30:04.009452+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"265'135","reported_seq":652,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:01.631789+0000","last_change":"2024-05-30T17:14:01.631955+0000","last_active":"2024-05-30T17:14:01.631789+0000","last_peered":"2024-05-30T17:14:01.631789+0000","last_clean":"2024-05-30T17:14:01.631789+0000","last_became_active":"2024-05-30T15:54:26.695842+0000","last_became_peered":"2024-05-30T15:54:26.695842+0000","last_unstale":"2024-05-30T17:14:01.631789+0000","last_undegraded":"2024-05-30T17:14:01.631789+0000","last_fullsized":"2024-05-30T17:14:01.631789+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'135","last_scrub_stamp":"2024-05-30T17:14:01.631538+0000","last_deep_scrub":"265'135","last_deep_scrub_stamp":"2024-05-30T17:14:01.631538+0000","last_clean_scrub_stamp":"2024-05-30T17:14:01.631538+0000","objects_scrubbed":1,"log_size":35,"log_dups_size":100,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:51:24.578349+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":418,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":133,"num_read_kb":127,"num_write":334,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"265'429","reported_seq":1958,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:06.042514+0000","last_change":"2024-05-30T17:14:06.042514+0000","last_active":"2024-05-30T17:14:06.042514+0000","last_peered":"2024-05-30T17:14:06.042514+0000","last_clean":"2024-05-30T17:14:06.042514+0000","last_became_active":"2024-05-30T15:54:18.660567+0000","last_became_peered":"2024-05-30T15:54:18.660567+0000","last_unstale":"2024-05-30T17:14:06.042514+0000","last_undegraded":"2024-05-30T17:14:06.042514+0000","last_fullsized":"2024-05-30T17:14:06.042514+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'429","last_scrub_stamp":"2024-05-30T17:14:06.042078+0000","last_deep_scrub":"265'429","last_deep_scrub_stamp":"2024-05-30T17:14:06.042078+0000","last_clean_scrub_stamp":"2024-05-30T17:14:06.042078+0000","objects_scrubbed":6,"log_size":29,"log_dups_size":400,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:17:31.512141+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2226,"num_read_kb":2104,"num_write":504,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"265'306","reported_seq":1007,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:11.360417+0000","last_change":"2024-05-30T17:14:11.360417+0000","last_active":"2024-05-30T17:14:11.360417+0000","last_peered":"2024-05-30T17:14:11.360417+0000","last_clean":"2024-05-30T17:14:11.360417+0000","last_became_active":"2024-05-30T15:54:16.685691+0000","last_became_peered":"2024-05-30T15:54:16.685691+0000","last_unstale":"2024-05-30T17:14:11.360417+0000","last_undegraded":"2024-05-30T17:14:11.360417+0000","last_fullsized":"2024-05-30T17:14:11.360417+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'306","last_scrub_stamp":"2024-05-30T17:14:11.360138+0000","last_deep_scrub":"265'306","last_deep_scrub_stamp":"2024-05-30T17:14:11.360138+0000","last_clean_scrub_stamp":"2024-05-30T17:14:11.360138+0000","objects_scrubbed":1,"log_size":106,"log_dups_size":200,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:54:48.371531+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":764,"num_read_kb":1764,"num_write":1253,"num_write_kb":18095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":515,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:18.847018+0000","last_change":"2024-05-30T17:14:18.847018+0000","last_active":"2024-05-30T17:14:18.847018+0000","last_peered":"2024-05-30T17:14:18.847018+0000","last_clean":"2024-05-30T17:14:18.847018+0000","last_became_active":"2024-05-30T15:54:20.669751+0000","last_became_peered":"2024-05-30T15:54:20.669751+0000","last_unstale":"2024-05-30T17:14:18.847018+0000","last_undegraded":"2024-05-30T17:14:18.847018+0000","last_fullsized":"2024-05-30T17:14:18.847018+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:18.846755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:18.846755+0000","last_clean_scrub_stamp":"2024-05-30T17:14:18.846755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:48:09.585537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"265'225","reported_seq":846,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.220776+0000","last_change":"2024-05-30T17:13:59.220776+0000","last_active":"2024-05-30T17:13:59.220776+0000","last_peered":"2024-05-30T17:13:59.220776+0000","last_clean":"2024-05-30T17:13:59.220776+0000","last_became_active":"2024-05-30T15:54:22.672078+0000","last_became_peered":"2024-05-30T15:54:22.672078+0000","last_unstale":"2024-05-30T17:13:59.220776+0000","last_undegraded":"2024-05-30T17:13:59.220776+0000","last_fullsized":"2024-05-30T17:13:59.220776+0000","mapping_epoch":28,"log_start":"265'200","ondisk_log_start":"265'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'225","last_scrub_stamp":"2024-05-30T17:13:59.220468+0000","last_deep_scrub":"265'225","last_deep_scrub_stamp":"2024-05-30T17:13:59.220468+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.220468+0000","objects_scrubbed":9,"log_size":25,"log_dups_size":200,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T05:11:49.982867+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10553,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":297,"num_read_kb":402,"num_write":186,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":513,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:05.864109+0000","last_change":"2024-05-30T17:14:05.864109+0000","last_active":"2024-05-30T17:14:05.864109+0000","last_peered":"2024-05-30T17:14:05.864109+0000","last_clean":"2024-05-30T17:14:05.864109+0000","last_became_active":"2024-05-30T15:54:24.675467+0000","last_became_peered":"2024-05-30T15:54:24.675467+0000","last_unstale":"2024-05-30T17:14:05.864109+0000","last_undegraded":"2024-05-30T17:14:05.864109+0000","last_fullsized":"2024-05-30T17:14:05.864109+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:05.863982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:05.863982+0000","last_clean_scrub_stamp":"2024-05-30T17:14:05.863982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:16:33.932497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"265'106","reported_seq":633,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.532471+0000","last_change":"2024-05-30T17:13:58.532471+0000","last_active":"2024-05-30T17:13:58.532471+0000","last_peered":"2024-05-30T17:13:58.532471+0000","last_clean":"2024-05-30T17:13:58.532471+0000","last_became_active":"2024-05-30T15:54:26.705264+0000","last_became_peered":"2024-05-30T15:54:26.705264+0000","last_unstale":"2024-05-30T17:13:58.532471+0000","last_undegraded":"2024-05-30T17:13:58.532471+0000","last_fullsized":"2024-05-30T17:13:58.532471+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'106","last_scrub_stamp":"2024-05-30T17:13:58.532222+0000","last_deep_scrub":"265'106","last_deep_scrub_stamp":"2024-05-30T17:13:58.532222+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.532222+0000","objects_scrubbed":4,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:07:52.286487+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":805,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":151,"num_read_kb":135,"num_write":247,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"265'700","reported_seq":2179,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.780482+0000","last_change":"2024-05-30T17:13:45.780482+0000","last_active":"2024-05-30T17:13:45.780482+0000","last_peered":"2024-05-30T17:13:45.780482+0000","last_clean":"2024-05-30T17:13:45.780482+0000","last_became_active":"2024-05-30T15:54:54.421604+0000","last_became_peered":"2024-05-30T15:54:54.421604+0000","last_unstale":"2024-05-30T17:13:45.780482+0000","last_undegraded":"2024-05-30T17:13:45.780482+0000","last_fullsized":"2024-05-30T17:13:45.780482+0000","mapping_epoch":35,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'700","last_scrub_stamp":"2024-05-30T17:13:45.780204+0000","last_deep_scrub":"265'700","last_deep_scrub_stamp":"2024-05-30T17:13:45.780204+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.780204+0000","objects_scrubbed":2,"log_size":100,"log_dups_size":600,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:24:04.304931+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3204,"num_read_kb":2893,"num_write":920,"num_write_kb":324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"265'173","reported_seq":824,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:46.218809+0000","last_change":"2024-05-30T17:13:46.218809+0000","last_active":"2024-05-30T17:13:46.218809+0000","last_peered":"2024-05-30T17:13:46.218809+0000","last_clean":"2024-05-30T17:13:46.218809+0000","last_became_active":"2024-05-30T15:54:54.422916+0000","last_became_peered":"2024-05-30T15:54:54.422916+0000","last_unstale":"2024-05-30T17:13:46.218809+0000","last_undegraded":"2024-05-30T17:13:46.218809+0000","last_fullsized":"2024-05-30T17:13:46.218809+0000","mapping_epoch":35,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'173","last_scrub_stamp":"2024-05-30T17:13:46.218474+0000","last_deep_scrub":"265'173","last_deep_scrub_stamp":"2024-05-30T17:13:46.218474+0000","last_clean_scrub_stamp":"2024-05-30T17:13:46.218474+0000","objects_scrubbed":4,"log_size":73,"log_dups_size":100,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:17:58.546167+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505146,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":405,"num_read_kb":71111,"num_write":971,"num_write_kb":25767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"265'5","reported_seq":842,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:08.314165+0000","last_change":"2024-05-30T17:14:08.314165+0000","last_active":"2024-05-30T17:14:08.314165+0000","last_peered":"2024-05-30T17:14:08.314165+0000","last_clean":"2024-05-30T17:14:08.314165+0000","last_became_active":"2024-05-30T15:54:20.652134+0000","last_became_peered":"2024-05-30T15:54:20.652134+0000","last_unstale":"2024-05-30T17:14:08.314165+0000","last_undegraded":"2024-05-30T17:14:08.314165+0000","last_fullsized":"2024-05-30T17:14:08.314165+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'5","last_scrub_stamp":"2024-05-30T17:14:08.313909+0000","last_deep_scrub":"265'5","last_deep_scrub_stamp":"2024-05-30T17:14:08.313909+0000","last_clean_scrub_stamp":"2024-05-30T17:14:08.313909+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:56:03.138686+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1139,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":493,"num_read_kb":807,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"265'155","reported_seq":758,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:33.860511+0000","last_change":"2024-05-30T17:14:33.860704+0000","last_active":"2024-05-30T17:14:33.860511+0000","last_peered":"2024-05-30T17:14:33.860511+0000","last_clean":"2024-05-30T17:14:33.860511+0000","last_became_active":"2024-05-30T15:54:22.672169+0000","last_became_peered":"2024-05-30T15:54:22.672169+0000","last_unstale":"2024-05-30T17:14:33.860511+0000","last_undegraded":"2024-05-30T17:14:33.860511+0000","last_fullsized":"2024-05-30T17:14:33.860511+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'155","last_scrub_stamp":"2024-05-30T17:14:33.860288+0000","last_deep_scrub":"265'155","last_deep_scrub_stamp":"2024-05-30T17:14:33.860288+0000","last_clean_scrub_stamp":"2024-05-30T17:14:33.860288+0000","objects_scrubbed":4,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:55:28.475193+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6731,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":192,"num_read_kb":263,"num_write":102,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":507,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:52.740908+0000","last_change":"2024-05-30T17:13:52.740908+0000","last_active":"2024-05-30T17:13:52.740908+0000","last_peered":"2024-05-30T17:13:52.740908+0000","last_clean":"2024-05-30T17:13:52.740908+0000","last_became_active":"2024-05-30T15:54:24.675683+0000","last_became_peered":"2024-05-30T15:54:24.675683+0000","last_unstale":"2024-05-30T17:13:52.740908+0000","last_undegraded":"2024-05-30T17:13:52.740908+0000","last_fullsized":"2024-05-30T17:13:52.740908+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:52.740555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:52.740555+0000","last_clean_scrub_stamp":"2024-05-30T17:13:52.740555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:52:04.474974+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"265'128","reported_seq":666,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:01.346031+0000","last_change":"2024-05-30T17:14:01.346031+0000","last_active":"2024-05-30T17:14:01.346031+0000","last_peered":"2024-05-30T17:14:01.346031+0000","last_clean":"2024-05-30T17:14:01.346031+0000","last_became_active":"2024-05-30T15:54:26.689049+0000","last_became_peered":"2024-05-30T15:54:26.689049+0000","last_unstale":"2024-05-30T17:14:01.346031+0000","last_undegraded":"2024-05-30T17:14:01.346031+0000","last_fullsized":"2024-05-30T17:14:01.346031+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'128","last_scrub_stamp":"2024-05-30T17:14:01.345770+0000","last_deep_scrub":"265'128","last_deep_scrub_stamp":"2024-05-30T17:14:01.345770+0000","last_clean_scrub_stamp":"2024-05-30T17:14:01.345770+0000","objects_scrubbed":1,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:20:24.034017+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":539,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":167,"num_read_kb":166,"num_write":309,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"265'654","reported_seq":2081,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:20.410597+0000","last_change":"2024-05-30T17:14:20.410597+0000","last_active":"2024-05-30T17:14:20.410597+0000","last_peered":"2024-05-30T17:14:20.410597+0000","last_clean":"2024-05-30T17:14:20.410597+0000","last_became_active":"2024-05-30T15:54:18.651778+0000","last_became_peered":"2024-05-30T15:54:18.651778+0000","last_unstale":"2024-05-30T17:14:20.410597+0000","last_undegraded":"2024-05-30T17:14:20.410597+0000","last_fullsized":"2024-05-30T17:14:20.410597+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'654","last_scrub_stamp":"2024-05-30T17:14:20.410335+0000","last_deep_scrub":"265'654","last_deep_scrub_stamp":"2024-05-30T17:14:20.410335+0000","last_clean_scrub_stamp":"2024-05-30T17:14:20.410335+0000","objects_scrubbed":2,"log_size":54,"log_dups_size":600,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:59:38.502836+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3068,"num_read_kb":2749,"num_write":930,"num_write_kb":367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"265'151","reported_seq":775,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:31.369696+0000","last_change":"2024-05-30T17:14:31.369696+0000","last_active":"2024-05-30T17:14:31.369696+0000","last_peered":"2024-05-30T17:14:31.369696+0000","last_clean":"2024-05-30T17:14:31.369696+0000","last_became_active":"2024-05-30T15:54:16.656081+0000","last_became_peered":"2024-05-30T15:54:16.656081+0000","last_unstale":"2024-05-30T17:14:31.369696+0000","last_undegraded":"2024-05-30T17:14:31.369696+0000","last_fullsized":"2024-05-30T17:14:31.369696+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'151","last_scrub_stamp":"2024-05-30T17:14:31.369421+0000","last_deep_scrub":"265'151","last_deep_scrub_stamp":"2024-05-30T17:14:31.369421+0000","last_clean_scrub_stamp":"2024-05-30T17:14:31.369421+0000","objects_scrubbed":1,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:35:41.015817+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":369,"num_read_kb":8201,"num_write":842,"num_write_kb":22196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":525,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:11.063361+0000","last_change":"2024-05-30T17:14:11.063466+0000","last_active":"2024-05-30T17:14:11.063361+0000","last_peered":"2024-05-30T17:14:11.063361+0000","last_clean":"2024-05-30T17:14:11.063361+0000","last_became_active":"2024-05-30T15:54:20.661090+0000","last_became_peered":"2024-05-30T15:54:20.661090+0000","last_unstale":"2024-05-30T17:14:11.063361+0000","last_undegraded":"2024-05-30T17:14:11.063361+0000","last_fullsized":"2024-05-30T17:14:11.063361+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:11.063113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:11.063113+0000","last_clean_scrub_stamp":"2024-05-30T17:14:11.063113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:16:18.209002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"265'118","reported_seq":700,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:23.403551+0000","last_change":"2024-05-30T17:14:23.403551+0000","last_active":"2024-05-30T17:14:23.403551+0000","last_peered":"2024-05-30T17:14:23.403551+0000","last_clean":"2024-05-30T17:14:23.403551+0000","last_became_active":"2024-05-30T15:54:22.669073+0000","last_became_peered":"2024-05-30T15:54:22.669073+0000","last_unstale":"2024-05-30T17:14:23.403551+0000","last_undegraded":"2024-05-30T17:14:23.403551+0000","last_fullsized":"2024-05-30T17:14:23.403551+0000","mapping_epoch":28,"log_start":"265'100","ondisk_log_start":"265'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'118","last_scrub_stamp":"2024-05-30T17:14:23.403251+0000","last_deep_scrub":"265'118","last_deep_scrub_stamp":"2024-05-30T17:14:23.403251+0000","last_clean_scrub_stamp":"2024-05-30T17:14:23.403251+0000","objects_scrubbed":2,"log_size":18,"log_dups_size":100,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:07:50.837787+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6915,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":145,"num_read_kb":188,"num_write":74,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":516,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:53.882025+0000","last_change":"2024-05-30T17:13:53.882025+0000","last_active":"2024-05-30T17:13:53.882025+0000","last_peered":"2024-05-30T17:13:53.882025+0000","last_clean":"2024-05-30T17:13:53.882025+0000","last_became_active":"2024-05-30T15:54:24.678292+0000","last_became_peered":"2024-05-30T15:54:24.678292+0000","last_unstale":"2024-05-30T17:13:53.882025+0000","last_undegraded":"2024-05-30T17:13:53.882025+0000","last_fullsized":"2024-05-30T17:13:53.882025+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:13:53.881772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:13:53.881772+0000","last_clean_scrub_stamp":"2024-05-30T17:13:53.881772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:31:21.481218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"265'171","reported_seq":918,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.566327+0000","last_change":"2024-05-30T17:13:59.566327+0000","last_active":"2024-05-30T17:13:59.566327+0000","last_peered":"2024-05-30T17:13:59.566327+0000","last_clean":"2024-05-30T17:13:59.566327+0000","last_became_active":"2024-05-30T15:54:26.703440+0000","last_became_peered":"2024-05-30T15:54:26.703440+0000","last_unstale":"2024-05-30T17:13:59.566327+0000","last_undegraded":"2024-05-30T17:13:59.566327+0000","last_fullsized":"2024-05-30T17:13:59.566327+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'171","last_scrub_stamp":"2024-05-30T17:13:59.565988+0000","last_deep_scrub":"265'171","last_deep_scrub_stamp":"2024-05-30T17:13:59.565988+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.565988+0000","objects_scrubbed":7,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:51:56.619593+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1074,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":420,"num_read_kb":450,"num_write":403,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1200,"num_omap_keys":6,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"265'333","reported_seq":1697,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:02.992132+0000","last_change":"2024-05-30T17:14:02.992132+0000","last_active":"2024-05-30T17:14:02.992132+0000","last_peered":"2024-05-30T17:14:02.992132+0000","last_clean":"2024-05-30T17:14:02.992132+0000","last_became_active":"2024-05-30T15:54:18.671650+0000","last_became_peered":"2024-05-30T15:54:18.671650+0000","last_unstale":"2024-05-30T17:14:02.992132+0000","last_undegraded":"2024-05-30T17:14:02.992132+0000","last_fullsized":"2024-05-30T17:14:02.992132+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'333","last_scrub_stamp":"2024-05-30T17:14:02.991716+0000","last_deep_scrub":"265'333","last_deep_scrub_stamp":"2024-05-30T17:14:02.991716+0000","last_clean_scrub_stamp":"2024-05-30T17:14:02.991716+0000","objects_scrubbed":6,"log_size":33,"log_dups_size":300,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:01:06.796193+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1672,"num_read_kb":1605,"num_write":361,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"265'206","reported_seq":876,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:32.836244+0000","last_change":"2024-05-30T17:14:32.836326+0000","last_active":"2024-05-30T17:14:32.836244+0000","last_peered":"2024-05-30T17:14:32.836244+0000","last_clean":"2024-05-30T17:14:32.836244+0000","last_became_active":"2024-05-30T15:54:16.660393+0000","last_became_peered":"2024-05-30T15:54:16.660393+0000","last_unstale":"2024-05-30T17:14:32.836244+0000","last_undegraded":"2024-05-30T17:14:32.836244+0000","last_fullsized":"2024-05-30T17:14:32.836244+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'206","last_scrub_stamp":"2024-05-30T17:14:32.836035+0000","last_deep_scrub":"265'206","last_deep_scrub_stamp":"2024-05-30T17:14:32.836035+0000","last_clean_scrub_stamp":"2024-05-30T17:14:32.836035+0000","objects_scrubbed":2,"log_size":106,"log_dups_size":100,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:37:35.758467+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":594502,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":519,"num_read_kb":9177,"num_write":1141,"num_write_kb":26248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"265'2","reported_seq":519,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:58.016076+0000","last_change":"2024-05-30T17:13:58.016076+0000","last_active":"2024-05-30T17:13:58.016076+0000","last_peered":"2024-05-30T17:13:58.016076+0000","last_clean":"2024-05-30T17:13:58.016076+0000","last_became_active":"2024-05-30T15:54:20.661311+0000","last_became_peered":"2024-05-30T15:54:20.661311+0000","last_unstale":"2024-05-30T17:13:58.016076+0000","last_undegraded":"2024-05-30T17:13:58.016076+0000","last_fullsized":"2024-05-30T17:13:58.016076+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-05-30T17:13:58.015783+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-05-30T17:13:58.015783+0000","last_clean_scrub_stamp":"2024-05-30T17:13:58.015783+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:05:29.698239+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"265'180","reported_seq":801,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:15.065220+0000","last_change":"2024-05-30T17:14:15.065220+0000","last_active":"2024-05-30T17:14:15.065220+0000","last_peered":"2024-05-30T17:14:15.065220+0000","last_clean":"2024-05-30T17:14:15.065220+0000","last_became_active":"2024-05-30T15:54:22.663231+0000","last_became_peered":"2024-05-30T15:54:22.663231+0000","last_unstale":"2024-05-30T17:14:15.065220+0000","last_undegraded":"2024-05-30T17:14:15.065220+0000","last_fullsized":"2024-05-30T17:14:15.065220+0000","mapping_epoch":28,"log_start":"262'100","ondisk_log_start":"262'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'180","last_scrub_stamp":"2024-05-30T17:14:15.064955+0000","last_deep_scrub":"265'180","last_deep_scrub_stamp":"2024-05-30T17:14:15.064955+0000","last_clean_scrub_stamp":"2024-05-30T17:14:15.064955+0000","objects_scrubbed":5,"log_size":80,"log_dups_size":100,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:54:03.215677+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4459,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":234,"num_read_kb":297,"num_write":134,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":512,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:04.853635+0000","last_change":"2024-05-30T17:14:04.853635+0000","last_active":"2024-05-30T17:14:04.853635+0000","last_peered":"2024-05-30T17:14:04.853635+0000","last_clean":"2024-05-30T17:14:04.853635+0000","last_became_active":"2024-05-30T15:54:24.673674+0000","last_became_peered":"2024-05-30T15:54:24.673674+0000","last_unstale":"2024-05-30T17:14:04.853635+0000","last_undegraded":"2024-05-30T17:14:04.853635+0000","last_fullsized":"2024-05-30T17:14:04.853635+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:14:04.853272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:14:04.853272+0000","last_clean_scrub_stamp":"2024-05-30T17:14:04.853272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:21:26.208754+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"265'114","reported_seq":629,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.822855+0000","last_change":"2024-05-30T17:13:48.822855+0000","last_active":"2024-05-30T17:13:48.822855+0000","last_peered":"2024-05-30T17:13:48.822855+0000","last_clean":"2024-05-30T17:13:48.822855+0000","last_became_active":"2024-05-30T15:54:26.677418+0000","last_became_peered":"2024-05-30T15:54:26.677418+0000","last_unstale":"2024-05-30T17:13:48.822855+0000","last_undegraded":"2024-05-30T17:13:48.822855+0000","last_fullsized":"2024-05-30T17:13:48.822855+0000","mapping_epoch":32,"log_start":"265'100","ondisk_log_start":"265'100","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'114","last_scrub_stamp":"2024-05-30T17:13:48.822470+0000","last_deep_scrub":"265'114","last_deep_scrub_stamp":"2024-05-30T17:13:48.822470+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.822470+0000","objects_scrubbed":3,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:49:43.113999+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":435,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":67,"num_write":244,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"265'308","reported_seq":1721,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:59.056454+0000","last_change":"2024-05-30T17:13:59.056454+0000","last_active":"2024-05-30T17:13:59.056454+0000","last_peered":"2024-05-30T17:13:59.056454+0000","last_clean":"2024-05-30T17:13:59.056454+0000","last_became_active":"2024-05-30T15:54:18.671335+0000","last_became_peered":"2024-05-30T15:54:18.671335+0000","last_unstale":"2024-05-30T17:13:59.056454+0000","last_undegraded":"2024-05-30T17:13:59.056454+0000","last_fullsized":"2024-05-30T17:13:59.056454+0000","mapping_epoch":24,"log_start":"265'200","ondisk_log_start":"265'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'308","last_scrub_stamp":"2024-05-30T17:13:59.056066+0000","last_deep_scrub":"265'308","last_deep_scrub_stamp":"2024-05-30T17:13:59.056066+0000","last_clean_scrub_stamp":"2024-05-30T17:13:59.056066+0000","objects_scrubbed":3,"log_size":108,"log_dups_size":200,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:17:28.319253+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1771,"num_read_kb":1705,"num_write":340,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"265'206","reported_seq":867,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:09.773002+0000","last_change":"2024-05-30T17:14:09.773202+0000","last_active":"2024-05-30T17:14:09.773002+0000","last_peered":"2024-05-30T17:14:09.773002+0000","last_clean":"2024-05-30T17:14:09.773002+0000","last_became_active":"2024-05-30T15:54:16.674678+0000","last_became_peered":"2024-05-30T15:54:16.674678+0000","last_unstale":"2024-05-30T17:14:09.773002+0000","last_undegraded":"2024-05-30T17:14:09.773002+0000","last_fullsized":"2024-05-30T17:14:09.773002+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'206","last_scrub_stamp":"2024-05-30T17:14:09.772714+0000","last_deep_scrub":"265'206","last_deep_scrub_stamp":"2024-05-30T17:14:09.772714+0000","last_clean_scrub_stamp":"2024-05-30T17:14:09.772714+0000","objects_scrubbed":6,"log_size":106,"log_dups_size":100,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:12:22.788923+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25713,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":493,"num_read_kb":10347,"num_write":1036,"num_write_kb":23027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"265'562","reported_seq":2288,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:18.965782+0000","last_change":"2024-05-30T17:14:18.965782+0000","last_active":"2024-05-30T17:14:18.965782+0000","last_peered":"2024-05-30T17:14:18.965782+0000","last_clean":"2024-05-30T17:14:18.965782+0000","last_became_active":"2024-05-30T15:54:18.635092+0000","last_became_peered":"2024-05-30T15:54:18.635092+0000","last_unstale":"2024-05-30T17:14:18.965782+0000","last_undegraded":"2024-05-30T17:14:18.965782+0000","last_fullsized":"2024-05-30T17:14:18.965782+0000","mapping_epoch":24,"log_start":"265'500","ondisk_log_start":"265'500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'562","last_scrub_stamp":"2024-05-30T17:14:18.965373+0000","last_deep_scrub":"265'562","last_deep_scrub_stamp":"2024-05-30T17:14:18.965373+0000","last_clean_scrub_stamp":"2024-05-30T17:14:18.965373+0000","objects_scrubbed":3,"log_size":62,"log_dups_size":500,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:13:33.470676+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2950,"num_read_kb":2739,"num_write":700,"num_write_kb":188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1511,"num_omap_keys":6,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"265'165","reported_seq":788,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:01.304037+0000","last_change":"2024-05-30T17:14:01.304121+0000","last_active":"2024-05-30T17:14:01.304037+0000","last_peered":"2024-05-30T17:14:01.304037+0000","last_clean":"2024-05-30T17:14:01.304037+0000","last_became_active":"2024-05-30T15:54:16.658183+0000","last_became_peered":"2024-05-30T15:54:16.658183+0000","last_unstale":"2024-05-30T17:14:01.304037+0000","last_undegraded":"2024-05-30T17:14:01.304037+0000","last_fullsized":"2024-05-30T17:14:01.304037+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'165","last_scrub_stamp":"2024-05-30T17:14:01.303807+0000","last_deep_scrub":"265'165","last_deep_scrub_stamp":"2024-05-30T17:14:01.303807+0000","last_clean_scrub_stamp":"2024-05-30T17:14:01.303807+0000","objects_scrubbed":1,"log_size":65,"log_dups_size":100,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:51:27.141500+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":308,"num_read_kb":6656,"num_write":926,"num_write_kb":27882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"265'629","reported_seq":2195,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:02.302193+0000","last_change":"2024-05-30T17:14:02.302193+0000","last_active":"2024-05-30T17:14:02.302193+0000","last_peered":"2024-05-30T17:14:02.302193+0000","last_clean":"2024-05-30T17:14:02.302193+0000","last_became_active":"2024-05-30T15:54:18.662159+0000","last_became_peered":"2024-05-30T15:54:18.662159+0000","last_unstale":"2024-05-30T17:14:02.302193+0000","last_undegraded":"2024-05-30T17:14:02.302193+0000","last_fullsized":"2024-05-30T17:14:02.302193+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'629","last_scrub_stamp":"2024-05-30T17:14:02.301919+0000","last_deep_scrub":"265'629","last_deep_scrub_stamp":"2024-05-30T17:14:02.301919+0000","last_clean_scrub_stamp":"2024-05-30T17:14:02.301919+0000","objects_scrubbed":4,"log_size":29,"log_dups_size":600,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:46:32.528521+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3191,"num_read_kb":2926,"num_write":899,"num_write_kb":342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":249,"num_omap_keys":1,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"265'210","reported_seq":872,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:17.310246+0000","last_change":"2024-05-30T17:14:17.310357+0000","last_active":"2024-05-30T17:14:17.310246+0000","last_peered":"2024-05-30T17:14:17.310246+0000","last_clean":"2024-05-30T17:14:17.310246+0000","last_became_active":"2024-05-30T15:54:16.652664+0000","last_became_peered":"2024-05-30T15:54:16.652664+0000","last_unstale":"2024-05-30T17:14:17.310246+0000","last_undegraded":"2024-05-30T17:14:17.310246+0000","last_fullsized":"2024-05-30T17:14:17.310246+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'210","last_scrub_stamp":"2024-05-30T17:14:17.310051+0000","last_deep_scrub":"265'210","last_deep_scrub_stamp":"2024-05-30T17:14:17.310051+0000","last_clean_scrub_stamp":"2024-05-30T17:14:17.310051+0000","objects_scrubbed":6,"log_size":110,"log_dups_size":100,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:39:16.444559+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524420,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":451,"num_read_kb":27404,"num_write":1156,"num_write_kb":29020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"265'1306","reported_seq":2951,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:09.025175+0000","last_change":"2024-05-30T17:14:09.025175+0000","last_active":"2024-05-30T17:14:09.025175+0000","last_peered":"2024-05-30T17:14:09.025175+0000","last_clean":"2024-05-30T17:14:09.025175+0000","last_became_active":"2024-05-30T15:54:18.642697+0000","last_became_peered":"2024-05-30T15:54:18.642697+0000","last_unstale":"2024-05-30T17:14:09.025175+0000","last_undegraded":"2024-05-30T17:14:09.025175+0000","last_fullsized":"2024-05-30T17:14:09.025175+0000","mapping_epoch":24,"log_start":"265'1200","ondisk_log_start":"265'1200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'1306","last_scrub_stamp":"2024-05-30T17:14:09.024879+0000","last_deep_scrub":"265'1306","last_deep_scrub_stamp":"2024-05-30T17:14:09.024879+0000","last_clean_scrub_stamp":"2024-05-30T17:14:09.024879+0000","objects_scrubbed":9,"log_size":106,"log_dups_size":1200,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:00:44.114885+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":5484,"num_read_kb":4752,"num_write":1832,"num_write_kb":807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1870,"num_omap_keys":8,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"265'256","reported_seq":998,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:31.804410+0000","last_change":"2024-05-30T17:14:31.804556+0000","last_active":"2024-05-30T17:14:31.804410+0000","last_peered":"2024-05-30T17:14:31.804410+0000","last_clean":"2024-05-30T17:14:31.804410+0000","last_became_active":"2024-05-30T15:54:16.659589+0000","last_became_peered":"2024-05-30T15:54:16.659589+0000","last_unstale":"2024-05-30T17:14:31.804410+0000","last_undegraded":"2024-05-30T17:14:31.804410+0000","last_fullsized":"2024-05-30T17:14:31.804410+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'256","last_scrub_stamp":"2024-05-30T17:14:31.804205+0000","last_deep_scrub":"265'256","last_deep_scrub_stamp":"2024-05-30T17:14:31.804205+0000","last_clean_scrub_stamp":"2024-05-30T17:14:31.804205+0000","objects_scrubbed":6,"log_size":56,"log_dups_size":200,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:22:26.640703+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3222795,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":793,"num_read_kb":47588,"num_write":1564,"num_write_kb":36318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"265'1103","reported_seq":2692,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:24.360537+0000","last_change":"2024-05-30T17:14:24.360701+0000","last_active":"2024-05-30T17:14:24.360537+0000","last_peered":"2024-05-30T17:14:24.360537+0000","last_clean":"2024-05-30T17:14:24.360537+0000","last_became_active":"2024-05-30T15:54:18.652794+0000","last_became_peered":"2024-05-30T15:54:18.652794+0000","last_unstale":"2024-05-30T17:14:24.360537+0000","last_undegraded":"2024-05-30T17:14:24.360537+0000","last_fullsized":"2024-05-30T17:14:24.360537+0000","mapping_epoch":24,"log_start":"265'1000","ondisk_log_start":"265'1000","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'1103","last_scrub_stamp":"2024-05-30T17:14:24.360317+0000","last_deep_scrub":"265'1103","last_deep_scrub_stamp":"2024-05-30T17:14:24.360317+0000","last_clean_scrub_stamp":"2024-05-30T17:14:24.360317+0000","objects_scrubbed":2,"log_size":103,"log_dups_size":1000,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:53:29.381532+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4733,"num_read_kb":4179,"num_write":1497,"num_write_kb":596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":246,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"265'225","reported_seq":903,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:52.362088+0000","last_change":"2024-05-30T17:13:52.362088+0000","last_active":"2024-05-30T17:13:52.362088+0000","last_peered":"2024-05-30T17:13:52.362088+0000","last_clean":"2024-05-30T17:13:52.362088+0000","last_became_active":"2024-05-30T15:54:54.451265+0000","last_became_peered":"2024-05-30T15:54:54.451265+0000","last_unstale":"2024-05-30T17:13:52.362088+0000","last_undegraded":"2024-05-30T17:13:52.362088+0000","last_fullsized":"2024-05-30T17:13:52.362088+0000","mapping_epoch":35,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'225","last_scrub_stamp":"2024-05-30T17:13:52.361791+0000","last_deep_scrub":"265'225","last_deep_scrub_stamp":"2024-05-30T17:13:52.361791+0000","last_clean_scrub_stamp":"2024-05-30T17:13:52.361791+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":200,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:41:32.517363+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524357,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":480,"num_read_kb":2675,"num_write":1165,"num_write_kb":43073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"265'389","reported_seq":1961,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:04.004662+0000","last_change":"2024-05-30T17:14:04.004748+0000","last_active":"2024-05-30T17:14:04.004662+0000","last_peered":"2024-05-30T17:14:04.004662+0000","last_clean":"2024-05-30T17:14:04.004662+0000","last_became_active":"2024-05-30T15:54:18.670801+0000","last_became_peered":"2024-05-30T15:54:18.670801+0000","last_unstale":"2024-05-30T17:14:04.004662+0000","last_undegraded":"2024-05-30T17:14:04.004662+0000","last_fullsized":"2024-05-30T17:14:04.004662+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'389","last_scrub_stamp":"2024-05-30T17:14:04.004452+0000","last_deep_scrub":"265'389","last_deep_scrub_stamp":"2024-05-30T17:14:04.004452+0000","last_clean_scrub_stamp":"2024-05-30T17:14:04.004452+0000","objects_scrubbed":4,"log_size":89,"log_dups_size":300,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:27:27.990698+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2215,"num_read_kb":2115,"num_write":443,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2553,"num_omap_keys":10,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"265'222","reported_seq":883,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:14.943979+0000","last_change":"2024-05-30T17:14:14.943979+0000","last_active":"2024-05-30T17:14:14.943979+0000","last_peered":"2024-05-30T17:14:14.943979+0000","last_clean":"2024-05-30T17:14:14.943979+0000","last_became_active":"2024-05-30T15:54:16.649854+0000","last_became_peered":"2024-05-30T15:54:16.649854+0000","last_unstale":"2024-05-30T17:14:14.943979+0000","last_undegraded":"2024-05-30T17:14:14.943979+0000","last_fullsized":"2024-05-30T17:14:14.943979+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'222","last_scrub_stamp":"2024-05-30T17:14:14.943688+0000","last_deep_scrub":"265'222","last_deep_scrub_stamp":"2024-05-30T17:14:14.943688+0000","last_clean_scrub_stamp":"2024-05-30T17:14:14.943688+0000","objects_scrubbed":2,"log_size":22,"log_dups_size":200,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:30:00.639949+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306443,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":499,"num_read_kb":6249,"num_write":1151,"num_write_kb":30962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"265'410","reported_seq":1939,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:26.362322+0000","last_change":"2024-05-30T17:14:26.362393+0000","last_active":"2024-05-30T17:14:26.362322+0000","last_peered":"2024-05-30T17:14:26.362322+0000","last_clean":"2024-05-30T17:14:26.362322+0000","last_became_active":"2024-05-30T15:54:18.650832+0000","last_became_peered":"2024-05-30T15:54:18.650832+0000","last_unstale":"2024-05-30T17:14:26.362322+0000","last_undegraded":"2024-05-30T17:14:26.362322+0000","last_fullsized":"2024-05-30T17:14:26.362322+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'410","last_scrub_stamp":"2024-05-30T17:14:26.362163+0000","last_deep_scrub":"265'410","last_deep_scrub_stamp":"2024-05-30T17:14:26.362163+0000","last_clean_scrub_stamp":"2024-05-30T17:14:26.362163+0000","objects_scrubbed":5,"log_size":110,"log_dups_size":300,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:14:40.469181+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2240,"num_read_kb":2125,"num_write":507,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":248,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"265'187","reported_seq":821,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:10.744038+0000","last_change":"2024-05-30T17:14:10.744038+0000","last_active":"2024-05-30T17:14:10.744038+0000","last_peered":"2024-05-30T17:14:10.744038+0000","last_clean":"2024-05-30T17:14:10.744038+0000","last_became_active":"2024-05-30T15:54:16.684670+0000","last_became_peered":"2024-05-30T15:54:16.684670+0000","last_unstale":"2024-05-30T17:14:10.744038+0000","last_undegraded":"2024-05-30T17:14:10.744038+0000","last_fullsized":"2024-05-30T17:14:10.744038+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'187","last_scrub_stamp":"2024-05-30T17:14:10.743692+0000","last_deep_scrub":"265'187","last_deep_scrub_stamp":"2024-05-30T17:14:10.743692+0000","last_clean_scrub_stamp":"2024-05-30T17:14:10.743692+0000","objects_scrubbed":7,"log_size":87,"log_dups_size":100,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:32:19.992966+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":254692,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":373,"num_read_kb":34719,"num_write":1090,"num_write_kb":29777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"265'823","reported_seq":2424,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:03.753084+0000","last_change":"2024-05-30T17:14:03.753161+0000","last_active":"2024-05-30T17:14:03.753084+0000","last_peered":"2024-05-30T17:14:03.753084+0000","last_clean":"2024-05-30T17:14:03.753084+0000","last_became_active":"2024-05-30T15:54:18.669105+0000","last_became_peered":"2024-05-30T15:54:18.669105+0000","last_unstale":"2024-05-30T17:14:03.753084+0000","last_undegraded":"2024-05-30T17:14:03.753084+0000","last_fullsized":"2024-05-30T17:14:03.753084+0000","mapping_epoch":24,"log_start":"265'800","ondisk_log_start":"265'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'823","last_scrub_stamp":"2024-05-30T17:14:03.752908+0000","last_deep_scrub":"265'823","last_deep_scrub_stamp":"2024-05-30T17:14:03.752908+0000","last_clean_scrub_stamp":"2024-05-30T17:14:03.752908+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":800,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:05:28.270822+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3660,"num_read_kb":3297,"num_write":1114,"num_write_kb":434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":252,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"265'220","reported_seq":927,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:48.975897+0000","last_change":"2024-05-30T17:13:48.976145+0000","last_active":"2024-05-30T17:13:48.975897+0000","last_peered":"2024-05-30T17:13:48.975897+0000","last_clean":"2024-05-30T17:13:48.975897+0000","last_became_active":"2024-05-30T15:54:54.416482+0000","last_became_peered":"2024-05-30T15:54:54.416482+0000","last_unstale":"2024-05-30T17:13:48.975897+0000","last_undegraded":"2024-05-30T17:13:48.975897+0000","last_fullsized":"2024-05-30T17:13:48.975897+0000","mapping_epoch":35,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'220","last_scrub_stamp":"2024-05-30T17:13:48.975457+0000","last_deep_scrub":"265'220","last_deep_scrub_stamp":"2024-05-30T17:13:48.975457+0000","last_clean_scrub_stamp":"2024-05-30T17:13:48.975457+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":200,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:44:46.059246+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5708449,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":729,"num_read_kb":137610,"num_write":1205,"num_write_kb":64841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2f","version":"265'389","reported_seq":1986,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:10.889831+0000","last_change":"2024-05-30T17:14:10.890006+0000","last_active":"2024-05-30T17:14:10.889831+0000","last_peered":"2024-05-30T17:14:10.889831+0000","last_clean":"2024-05-30T17:14:10.889831+0000","last_became_active":"2024-05-30T15:54:18.657236+0000","last_became_peered":"2024-05-30T15:54:18.657236+0000","last_unstale":"2024-05-30T17:14:10.889831+0000","last_undegraded":"2024-05-30T17:14:10.889831+0000","last_fullsized":"2024-05-30T17:14:10.889831+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'389","last_scrub_stamp":"2024-05-30T17:14:10.889524+0000","last_deep_scrub":"265'389","last_deep_scrub_stamp":"2024-05-30T17:14:10.889524+0000","last_clean_scrub_stamp":"2024-05-30T17:14:10.889524+0000","objects_scrubbed":4,"log_size":89,"log_dups_size":300,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:08:38.176469+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2258,"num_read_kb":2156,"num_write":442,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2808,"num_omap_keys":11,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"265'203","reported_seq":892,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:23.270071+0000","last_change":"2024-05-30T17:14:23.270071+0000","last_active":"2024-05-30T17:14:23.270071+0000","last_peered":"2024-05-30T17:14:23.270071+0000","last_clean":"2024-05-30T17:14:23.270071+0000","last_became_active":"2024-05-30T15:54:16.653037+0000","last_became_peered":"2024-05-30T15:54:16.653037+0000","last_unstale":"2024-05-30T17:14:23.270071+0000","last_undegraded":"2024-05-30T17:14:23.270071+0000","last_fullsized":"2024-05-30T17:14:23.270071+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'203","last_scrub_stamp":"2024-05-30T17:14:23.269779+0000","last_deep_scrub":"265'203","last_deep_scrub_stamp":"2024-05-30T17:14:23.269779+0000","last_clean_scrub_stamp":"2024-05-30T17:14:23.269779+0000","objects_scrubbed":5,"log_size":103,"log_dups_size":100,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:42:04.082214+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2108965,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":543,"num_read_kb":29183,"num_write":977,"num_write_kb":32330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"265'334","reported_seq":1841,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:21.115099+0000","last_change":"2024-05-30T17:14:21.115173+0000","last_active":"2024-05-30T17:14:21.115099+0000","last_peered":"2024-05-30T17:14:21.115099+0000","last_clean":"2024-05-30T17:14:21.115099+0000","last_became_active":"2024-05-30T15:54:18.635382+0000","last_became_peered":"2024-05-30T15:54:18.635382+0000","last_unstale":"2024-05-30T17:14:21.115099+0000","last_undegraded":"2024-05-30T17:14:21.115099+0000","last_fullsized":"2024-05-30T17:14:21.115099+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'334","last_scrub_stamp":"2024-05-30T17:14:21.114886+0000","last_deep_scrub":"265'334","last_deep_scrub_stamp":"2024-05-30T17:14:21.114886+0000","last_clean_scrub_stamp":"2024-05-30T17:14:21.114886+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:17:15.854111+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2071,"num_read_kb":1981,"num_write":382,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2453,"num_omap_keys":10,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"265'159","reported_seq":785,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:36.497334+0000","last_change":"2024-05-30T17:14:36.497413+0000","last_active":"2024-05-30T17:14:36.497334+0000","last_peered":"2024-05-30T17:14:36.497334+0000","last_clean":"2024-05-30T17:14:36.497334+0000","last_became_active":"2024-05-30T15:54:16.682629+0000","last_became_peered":"2024-05-30T15:54:16.682629+0000","last_unstale":"2024-05-30T17:14:36.497334+0000","last_undegraded":"2024-05-30T17:14:36.497334+0000","last_fullsized":"2024-05-30T17:14:36.497334+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'159","last_scrub_stamp":"2024-05-30T17:14:36.497105+0000","last_deep_scrub":"265'159","last_deep_scrub_stamp":"2024-05-30T17:14:36.497105+0000","last_clean_scrub_stamp":"2024-05-30T17:14:36.497105+0000","objects_scrubbed":2,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:16:58.514671+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310739,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":22431,"num_write":822,"num_write_kb":23734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"265'747","reported_seq":2325,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:06.362930+0000","last_change":"2024-05-30T17:14:06.362999+0000","last_active":"2024-05-30T17:14:06.362930+0000","last_peered":"2024-05-30T17:14:06.362930+0000","last_clean":"2024-05-30T17:14:06.362930+0000","last_became_active":"2024-05-30T15:54:18.668748+0000","last_became_peered":"2024-05-30T15:54:18.668748+0000","last_unstale":"2024-05-30T17:14:06.362930+0000","last_undegraded":"2024-05-30T17:14:06.362930+0000","last_fullsized":"2024-05-30T17:14:06.362930+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'747","last_scrub_stamp":"2024-05-30T17:14:06.362744+0000","last_deep_scrub":"265'747","last_deep_scrub_stamp":"2024-05-30T17:14:06.362744+0000","last_clean_scrub_stamp":"2024-05-30T17:14:06.362744+0000","objects_scrubbed":4,"log_size":47,"log_dups_size":700,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:14:16.405343+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3385,"num_read_kb":3066,"num_write":980,"num_write_kb":340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"265'165","reported_seq":909,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:13.340658+0000","last_change":"2024-05-30T17:14:13.340762+0000","last_active":"2024-05-30T17:14:13.340658+0000","last_peered":"2024-05-30T17:14:13.340658+0000","last_clean":"2024-05-30T17:14:13.340658+0000","last_became_active":"2024-05-30T15:54:16.674993+0000","last_became_peered":"2024-05-30T15:54:16.674993+0000","last_unstale":"2024-05-30T17:14:13.340658+0000","last_undegraded":"2024-05-30T17:14:13.340658+0000","last_fullsized":"2024-05-30T17:14:13.340658+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'165","last_scrub_stamp":"2024-05-30T17:14:13.340433+0000","last_deep_scrub":"265'165","last_deep_scrub_stamp":"2024-05-30T17:14:13.340433+0000","last_clean_scrub_stamp":"2024-05-30T17:14:13.340433+0000","objects_scrubbed":5,"log_size":65,"log_dups_size":100,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:25:29.735661+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6553619,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":567,"num_read_kb":112844,"num_write":925,"num_write_kb":45704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"265'472","reported_seq":2122,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:01.015512+0000","last_change":"2024-05-30T17:14:01.015577+0000","last_active":"2024-05-30T17:14:01.015512+0000","last_peered":"2024-05-30T17:14:01.015512+0000","last_clean":"2024-05-30T17:14:01.015512+0000","last_became_active":"2024-05-30T15:54:18.671434+0000","last_became_peered":"2024-05-30T15:54:18.671434+0000","last_unstale":"2024-05-30T17:14:01.015512+0000","last_undegraded":"2024-05-30T17:14:01.015512+0000","last_fullsized":"2024-05-30T17:14:01.015512+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'472","last_scrub_stamp":"2024-05-30T17:14:01.015328+0000","last_deep_scrub":"265'472","last_deep_scrub_stamp":"2024-05-30T17:14:01.015328+0000","last_clean_scrub_stamp":"2024-05-30T17:14:01.015328+0000","objects_scrubbed":1,"log_size":72,"log_dups_size":400,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:21:04.896268+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2525,"num_read_kb":2390,"num_write":596,"num_write_kb":183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"265'204","reported_seq":867,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:25.830042+0000","last_change":"2024-05-30T17:14:25.830133+0000","last_active":"2024-05-30T17:14:25.830042+0000","last_peered":"2024-05-30T17:14:25.830042+0000","last_clean":"2024-05-30T17:14:25.830042+0000","last_became_active":"2024-05-30T15:54:16.629843+0000","last_became_peered":"2024-05-30T15:54:16.629843+0000","last_unstale":"2024-05-30T17:14:25.830042+0000","last_undegraded":"2024-05-30T17:14:25.830042+0000","last_fullsized":"2024-05-30T17:14:25.830042+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'204","last_scrub_stamp":"2024-05-30T17:14:25.829810+0000","last_deep_scrub":"265'204","last_deep_scrub_stamp":"2024-05-30T17:14:25.829810+0000","last_clean_scrub_stamp":"2024-05-30T17:14:25.829810+0000","objects_scrubbed":3,"log_size":104,"log_dups_size":100,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:17:22.937587+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524465,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":434,"num_read_kb":42078,"num_write":1152,"num_write_kb":26102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"265'516","reported_seq":2150,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:11.810946+0000","last_change":"2024-05-30T17:14:11.811023+0000","last_active":"2024-05-30T17:14:11.810946+0000","last_peered":"2024-05-30T17:14:11.810946+0000","last_clean":"2024-05-30T17:14:11.810946+0000","last_became_active":"2024-05-30T15:54:18.660101+0000","last_became_peered":"2024-05-30T15:54:18.660101+0000","last_unstale":"2024-05-30T17:14:11.810946+0000","last_undegraded":"2024-05-30T17:14:11.810946+0000","last_fullsized":"2024-05-30T17:14:11.810946+0000","mapping_epoch":24,"log_start":"265'500","ondisk_log_start":"265'500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'516","last_scrub_stamp":"2024-05-30T17:14:11.810772+0000","last_deep_scrub":"265'516","last_deep_scrub_stamp":"2024-05-30T17:14:11.810772+0000","last_clean_scrub_stamp":"2024-05-30T17:14:11.810772+0000","objects_scrubbed":7,"log_size":16,"log_dups_size":500,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:11:32.790447+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2760,"num_read_kb":2546,"num_write":661,"num_write_kb":201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2440,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"265'210","reported_seq":854,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:11.793065+0000","last_change":"2024-05-30T17:14:11.793335+0000","last_active":"2024-05-30T17:14:11.793065+0000","last_peered":"2024-05-30T17:14:11.793065+0000","last_clean":"2024-05-30T17:14:11.793065+0000","last_became_active":"2024-05-30T15:54:16.685969+0000","last_became_peered":"2024-05-30T15:54:16.685969+0000","last_unstale":"2024-05-30T17:14:11.793065+0000","last_undegraded":"2024-05-30T17:14:11.793065+0000","last_fullsized":"2024-05-30T17:14:11.793065+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'210","last_scrub_stamp":"2024-05-30T17:14:11.792805+0000","last_deep_scrub":"265'210","last_deep_scrub_stamp":"2024-05-30T17:14:11.792805+0000","last_clean_scrub_stamp":"2024-05-30T17:14:11.792805+0000","objects_scrubbed":5,"log_size":110,"log_dups_size":100,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:18:58.482461+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1342159,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":419,"num_read_kb":18996,"num_write":1148,"num_write_kb":34757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"265'671","reported_seq":2437,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:18.831077+0000","last_change":"2024-05-30T17:14:18.831145+0000","last_active":"2024-05-30T17:14:18.831077+0000","last_peered":"2024-05-30T17:14:18.831077+0000","last_clean":"2024-05-30T17:14:18.831077+0000","last_became_active":"2024-05-30T15:54:18.645400+0000","last_became_peered":"2024-05-30T15:54:18.645400+0000","last_unstale":"2024-05-30T17:14:18.831077+0000","last_undegraded":"2024-05-30T17:14:18.831077+0000","last_fullsized":"2024-05-30T17:14:18.831077+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'671","last_scrub_stamp":"2024-05-30T17:14:18.830880+0000","last_deep_scrub":"265'671","last_deep_scrub_stamp":"2024-05-30T17:14:18.830880+0000","last_clean_scrub_stamp":"2024-05-30T17:14:18.830880+0000","objects_scrubbed":1,"log_size":71,"log_dups_size":600,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:12:16.729926+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3447,"num_read_kb":3120,"num_write":968,"num_write_kb":434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"265'215","reported_seq":858,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:21.805383+0000","last_change":"2024-05-30T17:14:21.805542+0000","last_active":"2024-05-30T17:14:21.805383+0000","last_peered":"2024-05-30T17:14:21.805383+0000","last_clean":"2024-05-30T17:14:21.805383+0000","last_became_active":"2024-05-30T15:54:16.663255+0000","last_became_peered":"2024-05-30T15:54:16.663255+0000","last_unstale":"2024-05-30T17:14:21.805383+0000","last_undegraded":"2024-05-30T17:14:21.805383+0000","last_fullsized":"2024-05-30T17:14:21.805383+0000","mapping_epoch":22,"log_start":"265'200","ondisk_log_start":"265'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'215","last_scrub_stamp":"2024-05-30T17:14:21.805146+0000","last_deep_scrub":"265'215","last_deep_scrub_stamp":"2024-05-30T17:14:21.805146+0000","last_clean_scrub_stamp":"2024-05-30T17:14:21.805146+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":200,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:43:32.999785+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767340,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":429,"num_read_kb":27123,"num_write":1161,"num_write_kb":19484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"265'365","reported_seq":1793,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:16.421825+0000","last_change":"2024-05-30T17:14:16.421904+0000","last_active":"2024-05-30T17:14:16.421825+0000","last_peered":"2024-05-30T17:14:16.421825+0000","last_clean":"2024-05-30T17:14:16.421825+0000","last_became_active":"2024-05-30T15:54:18.658230+0000","last_became_peered":"2024-05-30T15:54:18.658230+0000","last_unstale":"2024-05-30T17:14:16.421825+0000","last_undegraded":"2024-05-30T17:14:16.421825+0000","last_fullsized":"2024-05-30T17:14:16.421825+0000","mapping_epoch":24,"log_start":"265'300","ondisk_log_start":"265'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'365","last_scrub_stamp":"2024-05-30T17:14:16.421610+0000","last_deep_scrub":"265'365","last_deep_scrub_stamp":"2024-05-30T17:14:16.421610+0000","last_clean_scrub_stamp":"2024-05-30T17:14:16.421610+0000","objects_scrubbed":5,"log_size":65,"log_dups_size":300,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:52:59.746921+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1934,"num_read_kb":1839,"num_write":414,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":472,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"265'170","reported_seq":792,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:00.278603+0000","last_change":"2024-05-30T17:14:00.278671+0000","last_active":"2024-05-30T17:14:00.278603+0000","last_peered":"2024-05-30T17:14:00.278603+0000","last_clean":"2024-05-30T17:14:00.278603+0000","last_became_active":"2024-05-30T15:54:16.654365+0000","last_became_peered":"2024-05-30T15:54:16.654365+0000","last_unstale":"2024-05-30T17:14:00.278603+0000","last_undegraded":"2024-05-30T17:14:00.278603+0000","last_fullsized":"2024-05-30T17:14:00.278603+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'170","last_scrub_stamp":"2024-05-30T17:14:00.278386+0000","last_deep_scrub":"265'170","last_deep_scrub_stamp":"2024-05-30T17:14:00.278386+0000","last_clean_scrub_stamp":"2024-05-30T17:14:00.278386+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":100,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:04:58.427288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":358,"num_read_kb":31154,"num_write":915,"num_write_kb":41557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"265'704","reported_seq":2144,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:10.773805+0000","last_change":"2024-05-30T17:14:10.773805+0000","last_active":"2024-05-30T17:14:10.773805+0000","last_peered":"2024-05-30T17:14:10.773805+0000","last_clean":"2024-05-30T17:14:10.773805+0000","last_became_active":"2024-05-30T15:54:18.660543+0000","last_became_peered":"2024-05-30T15:54:18.660543+0000","last_unstale":"2024-05-30T17:14:10.773805+0000","last_undegraded":"2024-05-30T17:14:10.773805+0000","last_fullsized":"2024-05-30T17:14:10.773805+0000","mapping_epoch":24,"log_start":"265'600","ondisk_log_start":"265'600","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'704","last_scrub_stamp":"2024-05-30T17:14:10.773554+0000","last_deep_scrub":"265'704","last_deep_scrub_stamp":"2024-05-30T17:14:10.773554+0000","last_clean_scrub_stamp":"2024-05-30T17:14:10.773554+0000","objects_scrubbed":6,"log_size":104,"log_dups_size":600,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:36:12.916424+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3130,"num_read_kb":2812,"num_write":919,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"265'202","reported_seq":853,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:24.321602+0000","last_change":"2024-05-30T17:14:24.321759+0000","last_active":"2024-05-30T17:14:24.321602+0000","last_peered":"2024-05-30T17:14:24.321602+0000","last_clean":"2024-05-30T17:14:24.321602+0000","last_became_active":"2024-05-30T15:54:16.661983+0000","last_became_peered":"2024-05-30T15:54:16.661983+0000","last_unstale":"2024-05-30T17:14:24.321602+0000","last_undegraded":"2024-05-30T17:14:24.321602+0000","last_fullsized":"2024-05-30T17:14:24.321602+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'202","last_scrub_stamp":"2024-05-30T17:14:24.321397+0000","last_deep_scrub":"265'202","last_deep_scrub_stamp":"2024-05-30T17:14:24.321397+0000","last_clean_scrub_stamp":"2024-05-30T17:14:24.321397+0000","objects_scrubbed":7,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:26:06.878260+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718827,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":420,"num_read_kb":7620,"num_write":1155,"num_write_kb":9531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"265'154","reported_seq":793,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:13:45.164212+0000","last_change":"2024-05-30T17:13:45.164368+0000","last_active":"2024-05-30T17:13:45.164212+0000","last_peered":"2024-05-30T17:13:45.164212+0000","last_clean":"2024-05-30T17:13:45.164212+0000","last_became_active":"2024-05-30T15:55:54.767665+0000","last_became_peered":"2024-05-30T15:55:54.767665+0000","last_unstale":"2024-05-30T17:13:45.164212+0000","last_undegraded":"2024-05-30T17:13:45.164212+0000","last_fullsized":"2024-05-30T17:13:45.164212+0000","mapping_epoch":37,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'154","last_scrub_stamp":"2024-05-30T17:13:45.163967+0000","last_deep_scrub":"265'154","last_deep_scrub_stamp":"2024-05-30T17:13:45.163967+0000","last_clean_scrub_stamp":"2024-05-30T17:13:45.163967+0000","objects_scrubbed":2,"log_size":54,"log_dups_size":100,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:10:48.513574+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5682939,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":360,"num_read_kb":14674,"num_write":890,"num_write_kb":17371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,0],"acting":[5,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"265'771","reported_seq":2435,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:11.854742+0000","last_change":"2024-05-30T17:14:11.854742+0000","last_active":"2024-05-30T17:14:11.854742+0000","last_peered":"2024-05-30T17:14:11.854742+0000","last_clean":"2024-05-30T17:14:11.854742+0000","last_became_active":"2024-05-30T15:54:18.660501+0000","last_became_peered":"2024-05-30T15:54:18.660501+0000","last_unstale":"2024-05-30T17:14:11.854742+0000","last_undegraded":"2024-05-30T17:14:11.854742+0000","last_fullsized":"2024-05-30T17:14:11.854742+0000","mapping_epoch":24,"log_start":"265'700","ondisk_log_start":"265'700","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'771","last_scrub_stamp":"2024-05-30T17:14:11.854443+0000","last_deep_scrub":"265'771","last_deep_scrub_stamp":"2024-05-30T17:14:11.854443+0000","last_clean_scrub_stamp":"2024-05-30T17:14:11.854443+0000","objects_scrubbed":6,"log_size":71,"log_dups_size":700,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:04:01.491788+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3837,"num_read_kb":3483,"num_write":1021,"num_write_kb":383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2811,"num_omap_keys":11,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"265'183","reported_seq":843,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:17.991730+0000","last_change":"2024-05-30T17:14:17.991836+0000","last_active":"2024-05-30T17:14:17.991730+0000","last_peered":"2024-05-30T17:14:17.991730+0000","last_clean":"2024-05-30T17:14:17.991730+0000","last_became_active":"2024-05-30T15:54:16.668178+0000","last_became_peered":"2024-05-30T15:54:16.668178+0000","last_unstale":"2024-05-30T17:14:17.991730+0000","last_undegraded":"2024-05-30T17:14:17.991730+0000","last_fullsized":"2024-05-30T17:14:17.991730+0000","mapping_epoch":22,"log_start":"265'100","ondisk_log_start":"265'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'183","last_scrub_stamp":"2024-05-30T17:14:17.991533+0000","last_deep_scrub":"265'183","last_deep_scrub_stamp":"2024-05-30T17:14:17.991533+0000","last_clean_scrub_stamp":"2024-05-30T17:14:17.991533+0000","objects_scrubbed":6,"log_size":83,"log_dups_size":100,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:17:31.041725+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3976850,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":427,"num_read_kb":57498,"num_write":1000,"num_write_kb":21705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"265'477","reported_seq":2157,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-05-30T17:14:09.363466+0000","last_change":"2024-05-30T17:14:09.363603+0000","last_active":"2024-05-30T17:14:09.363466+0000","last_peered":"2024-05-30T17:14:09.363466+0000","last_clean":"2024-05-30T17:14:09.363466+0000","last_became_active":"2024-05-30T15:54:18.655872+0000","last_became_peered":"2024-05-30T15:54:18.655872+0000","last_unstale":"2024-05-30T17:14:09.363466+0000","last_undegraded":"2024-05-30T17:14:09.363466+0000","last_fullsized":"2024-05-30T17:14:09.363466+0000","mapping_epoch":24,"log_start":"265'400","ondisk_log_start":"265'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'477","last_scrub_stamp":"2024-05-30T17:14:09.363234+0000","last_deep_scrub":"265'477","last_deep_scrub_stamp":"2024-05-30T17:14:09.363234+0000","last_clean_scrub_stamp":"2024-05-30T17:14:09.363234+0000","objects_scrubbed":3,"log_size":77,"log_dups_size":400,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:01:10.979098+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2571,"num_read_kb":2440,"num_write":587,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":497,"num_omap_keys":2,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5999,"num_read_kb":4309,"num_write":2135,"num_write_kb":880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6954,"num_omap_keys":14,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1142,"ondisk_log_size":1142,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19930,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":13226,"num_read_kb":13032,"num_write":15987,"num_write_kb":4690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4334,"num_omap_keys":26,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":704512,"data_stored":39860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1671,"ondisk_log_size":1671,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":331,"ondisk_log_size":331,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":279892,"num_objects":197,"num_object_clones":0,"num_object_copies":394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":197,"num_whiteouts":0,"num_read":9562,"num_read_kb":11753,"num_write":6685,"num_write_kb":2848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":664178,"num_omap_keys":1530,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1908,"ondisk_log_size":1908,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1634,"num_read_kb":1946,"num_write":20,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":242,"num_object_clones":0,"num_object_copies":484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":242,"num_whiteouts":0,"num_read":191181,"num_read_kb":174247,"num_write":51900,"num_write_kb":17923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":242,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":43877,"num_omap_keys":181,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4157,"ondisk_log_size":4157,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1128,"num_omap_keys":12,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":157354006,"num_objects":261,"num_object_clones":0,"num_object_copies":783,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":261,"num_whiteouts":0,"num_read":32072,"num_read_kb":2124808,"num_write":69910,"num_write_kb":2069422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":238202880,"data_stored":238202880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4424,"ondisk_log_size":4424,"up":192,"acting":192,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":135,"num_read_kb":182367,"num_write":151,"num_write_kb":122722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":133,"num_read_kb":181098,"num_write":151,"num_write_kb":122722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575827,"num_pgs":106,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300188,"kb_used_data":38548,"kb_used_omap":1,"kb_used_meta":261630,"kb_avail":94071652,"statfs":{"total":96636764160,"available":96329371648,"internally_reserved":0,"allocated":39473152,"data_stored":37452492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":267909578},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575826,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":289728,"kb_used_data":37816,"kb_used_omap":1,"kb_used_meta":251902,"kb_avail":94082112,"statfs":{"total":96636764160,"available":96340082688,"internally_reserved":0,"allocated":38723584,"data_stored":36776739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":257948102},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575826,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137976,"kb_used_data":48496,"kb_used_omap":1,"kb_used_meta":89470,"kb_avail":94233864,"statfs":{"total":96636764160,"available":96495476736,"internally_reserved":0,"allocated":49659904,"data_stored":47630081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":91617733},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575826,"num_pgs":104,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":125812,"kb_used_data":44652,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94246028,"statfs":{"total":96636764160,"available":96507932672,"internally_reserved":0,"allocated":45723648,"data_stored":43786666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":83098054},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575824,"num_pgs":124,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":269128,"kb_used_data":43584,"kb_used_omap":1,"kb_used_meta":225534,"kb_avail":94102712,"statfs":{"total":96636764160,"available":96361177088,"internally_reserved":0,"allocated":44630016,"data_stored":42616238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":230947271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575824,"num_pgs":91,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":107084,"kb_used_data":35652,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94264756,"statfs":{"total":96636764160,"available":96527110144,"internally_reserved":0,"allocated":36507648,"data_stored":34568865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575829,"num_pgs":112,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":164064,"kb_used_data":46872,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94207776,"statfs":{"total":96636764160,"available":96468762624,"internally_reserved":0,"allocated":47996928,"data_stored":46030586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575823,"num_pgs":109,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266824,"kb_used_data":37120,"kb_used_omap":1,"kb_used_meta":229694,"kb_avail":94105016,"statfs":{"total":96636764160,"available":96363536384,"internally_reserved":0,"allocated":38010880,"data_stored":36012935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":235207109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":27881472,"data_stored":27881472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":37441536,"data_stored":37441536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27029504,"data_stored":27029504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24588288,"data_stored":24588288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":31006720,"data_stored":31006720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":33796096,"data_stored":33796096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":32382976,"data_stored":32382976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24076288,"data_stored":24076288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":4787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":5121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":4696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":6691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":5459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":3918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":7193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T17:15:08.041 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-30T17:15:08.398 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-30T17:15:08.398 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-30T17:15:08.399 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-30T17:15:08.399 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:15:08.487 INFO:tasks.ceph.osd.0:Stopped 2024-05-30T17:15:08.487 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-30T17:15:08.488 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:15:08.580 INFO:tasks.ceph.osd.1:Stopped 2024-05-30T17:15:08.581 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-30T17:15:08.581 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:15:13.557 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-05-30T17:15:14.683 INFO:tasks.ceph.osd.2:Stopped 2024-05-30T17:15:14.683 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-30T17:15:14.684 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:15:14.777 INFO:tasks.ceph.osd.3:Stopped 2024-05-30T17:15:14.778 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-30T17:15:14.778 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:15:14.859 INFO:tasks.ceph.osd.4:Stopped 2024-05-30T17:15:14.860 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-30T17:15:14.860 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:15:19.264 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-05-30T17:15:19.264 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-05-30T17:15:20.962 INFO:tasks.ceph.osd.5:Stopped 2024-05-30T17:15:20.963 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-30T17:15:20.963 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:15:21.041 INFO:tasks.ceph.osd.6:Stopped 2024-05-30T17:15:21.041 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-30T17:15:21.041 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:15:21.133 INFO:tasks.ceph.osd.7:Stopped 2024-05-30T17:15:21.134 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-30T17:15:21.134 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-30T17:15:21.134 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:15:21.162 INFO:tasks.ceph.mgr.y:Stopped 2024-05-30T17:15:21.162 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-30T17:15:21.163 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:15:21.193 INFO:tasks.ceph.mgr.x:Stopped 2024-05-30T17:15:21.193 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-30T17:15:21.193 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-30T17:15:21.194 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:15:21.224 INFO:tasks.ceph.mon.a:Stopped 2024-05-30T17:15:21.224 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-30T17:15:21.225 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:15:21.285 INFO:tasks.ceph.mon.c:Stopped 2024-05-30T17:15:21.286 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-30T17:15:21.286 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:15:21.314 INFO:tasks.ceph.mon.b:Stopped 2024-05-30T17:15:21.314 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-30T17:15:21.315 DEBUG:teuthology.orchestra.run.smithi006:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'not have an application enabled' | egrep -v '\(OSD_SLOW_PING_TIME' | head -n 1 2024-05-30T17:15:21.341 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi006.front.sepia.ceph.com 2024-05-30T17:15:21.342 DEBUG:teuthology.orchestra.run.smithi006:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-30T17:15:21.707 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi006.front.sepia.ceph.com 2024-05-30T17:15:21.708 DEBUG:teuthology.orchestra.run.smithi006:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-30T17:15:22.033 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi006.front.sepia.ceph.com 2024-05-30T17:15:22.033 DEBUG:teuthology.orchestra.run.smithi006:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-30T17:15:22.230 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi006.front.sepia.ceph.com 2024-05-30T17:15:22.230 DEBUG:teuthology.orchestra.run.smithi006:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-30T17:15:22.421 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi204.front.sepia.ceph.com 2024-05-30T17:15:22.421 DEBUG:teuthology.orchestra.run.smithi204:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-30T17:15:22.856 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi204.front.sepia.ceph.com 2024-05-30T17:15:22.856 DEBUG:teuthology.orchestra.run.smithi204:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-30T17:15:23.049 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi204.front.sepia.ceph.com 2024-05-30T17:15:23.050 DEBUG:teuthology.orchestra.run.smithi204:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-30T17:15:23.210 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi204.front.sepia.ceph.com 2024-05-30T17:15:23.210 DEBUG:teuthology.orchestra.run.smithi204:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-30T17:15:23.352 INFO:tasks.ceph:Archiving mon data... 2024-05-30T17:15:23.353 DEBUG:teuthology.misc:Transferring archived files from smithi006:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/cbodley-2024-05-30_15:09:56-rgw-wip-cbodley-testing-distro-default-smithi/7734120/data/mon.a.tgz 2024-05-30T17:15:23.353 DEBUG:teuthology.orchestra.run.smithi006:> mktemp 2024-05-30T17:15:23.359 INFO:teuthology.orchestra.run.smithi006.stdout:/tmp/tmp.FJJP3TmNXB 2024-05-30T17:15:23.360 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.FJJP3TmNXB 2024-05-30T17:15:23.548 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 0666 /tmp/tmp.FJJP3TmNXB 2024-05-30T17:15:23.612 DEBUG:teuthology.orchestra.remote:smithi006:/tmp/tmp.FJJP3TmNXB is 523KB 2024-05-30T17:15:23.673 DEBUG:teuthology.orchestra.run.smithi006:> rm -fr /tmp/tmp.FJJP3TmNXB 2024-05-30T17:15:23.679 DEBUG:teuthology.misc:Transferring archived files from smithi006:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/cbodley-2024-05-30_15:09:56-rgw-wip-cbodley-testing-distro-default-smithi/7734120/data/mon.c.tgz 2024-05-30T17:15:23.679 DEBUG:teuthology.orchestra.run.smithi006:> mktemp 2024-05-30T17:15:23.728 INFO:teuthology.orchestra.run.smithi006.stdout:/tmp/tmp.u4EWBSGWYq 2024-05-30T17:15:23.728 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.u4EWBSGWYq 2024-05-30T17:15:23.923 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 0666 /tmp/tmp.u4EWBSGWYq 2024-05-30T17:15:23.988 DEBUG:teuthology.orchestra.remote:smithi006:/tmp/tmp.u4EWBSGWYq is 541KB 2024-05-30T17:15:24.049 DEBUG:teuthology.orchestra.run.smithi006:> rm -fr /tmp/tmp.u4EWBSGWYq 2024-05-30T17:15:24.055 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/cbodley-2024-05-30_15:09:56-rgw-wip-cbodley-testing-distro-default-smithi/7734120/data/mon.b.tgz 2024-05-30T17:15:24.055 DEBUG:teuthology.orchestra.run.smithi204:> mktemp 2024-05-30T17:15:24.061 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.jNZQALUKYz 2024-05-30T17:15:24.061 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.jNZQALUKYz 2024-05-30T17:15:24.242 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.jNZQALUKYz 2024-05-30T17:15:24.264 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-05-30T17:15:24.306 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.jNZQALUKYz is 541KB 2024-05-30T17:15:24.368 DEBUG:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.jNZQALUKYz 2024-05-30T17:15:24.373 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-30T17:15:24.374 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-30T17:15:24.376 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-30T17:15:24.501 DEBUG:tasks.ceph:valgrind exception message: valgrind error: MismatchedFree operator delete[](void*, unsigned long) MonClient::get_monmap_and_config() 2024-05-30T17:15:24.507 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-30T17:15:24.507 INFO:tasks.ceph:Archiving crash dumps... 2024-05-30T17:15:24.509 DEBUG:teuthology.misc:Transferring archived files from smithi006:/var/lib/ceph/crash to /home/teuthworker/archive/cbodley-2024-05-30_15:09:56-rgw-wip-cbodley-testing-distro-default-smithi/7734120/remote/smithi006/crash 2024-05-30T17:15:24.509 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-30T17:15:24.524 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/crash to /home/teuthworker/archive/cbodley-2024-05-30_15:09:56-rgw-wip-cbodley-testing-distro-default-smithi/7734120/remote/smithi204/crash 2024-05-30T17:15:24.525 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-30T17:15:24.563 INFO:tasks.ceph:Compressing logs... 2024-05-30T17:15:24.563 DEBUG:teuthology.orchestra.run.smithi006:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-30T17:15:24.568 DEBUG:teuthology.orchestra.run.smithi204:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-30T17:15:24.588 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.133801.log 2024-05-30T17:15:24.589 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98132.log 2024-05-30T17:15:24.590 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.0.133801.log: gzip -5 --verbose -- /var/log/ceph/rgw.ceph.client.0.log 2024-05-30T17:15:24.590 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.98132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31489.log 2024-05-30T17:15:24.591 INFO:teuthology.orchestra.run.smithi006.stderr: 82.8% -- replaced with /var/log/ceph/ceph-client.admin.98132.log.gz 2024-05-30T17:15:24.591 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/rgw.ceph.client.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.131782.log 2024-05-30T17:15:24.593 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31959.log 2024-05-30T17:15:24.593 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.31489.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31489.log.gz 2024-05-30T17:15:24.593 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.0.131782.log: 90.6% -- replaced with /var/log/ceph/ceph-client.0.133801.log.gz 2024-05-30T17:15:24.593 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32031.log 2024-05-30T17:15:24.593 INFO:teuthology.orchestra.run.smithi006.stderr: 84.4% -- replaced with /var/log/ceph/ceph-client.0.131782.log.gz 2024-05-30T17:15:24.594 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.31959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53863.log 2024-05-30T17:15:24.594 INFO:teuthology.orchestra.run.smithi006.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31959.log.gz 2024-05-30T17:15:24.594 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.32031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31466.log 2024-05-30T17:15:24.594 INFO:teuthology.orchestra.run.smithi006.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32031.log.gz 2024-05-30T17:15:24.594 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.53863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31155.log 2024-05-30T17:15:24.594 INFO:teuthology.orchestra.run.smithi006.stderr: 54.4% -- replaced with /var/log/ceph/ceph-client.admin.53863.log.gz 2024-05-30T17:15:24.594 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.31466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134576.log 2024-05-30T17:15:24.594 INFO:teuthology.orchestra.run.smithi006.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31466.log.gz 2024-05-30T17:15:24.597 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.31155.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-05-30T17:15:24.597 INFO:teuthology.orchestra.run.smithi006.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31155.log.gz 2024-05-30T17:15:24.598 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53837.log 2024-05-30T17:15:24.598 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.134576.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134576.log.gz 2024-05-30T17:15:24.598 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43069.log 2024-05-30T17:15:24.598 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.53837.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.20900.log 2024-05-30T17:15:24.598 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.43069.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43069.log.gz 2024-05-30T17:15:24.598 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82201.log 2024-05-30T17:15:24.598 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40044.log 2024-05-30T17:15:24.598 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph.tmp-client.admin.20900.log: /var/log/ceph/ceph-client.admin.82201.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.20900.log.gz 2024-05-30T17:15:24.599 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-30T17:15:24.599 INFO:teuthology.orchestra.run.smithi006.stderr: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.82201.log.gz 2024-05-30T17:15:24.599 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80430.log 2024-05-30T17:15:24.599 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.40044.log: /var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- 84.8% /var/log/ceph/ceph-client.admin.112414.log 2024-05-30T17:15:24.599 INFO:teuthology.orchestra.run.smithi006.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40044.log.gz 2024-05-30T17:15:24.599 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75120.log 2024-05-30T17:15:24.602 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.80430.log: /var/log/ceph/ceph-client.admin.112414.log: 82.3% -- replaced with /var/log/ceph/ceph-client.admin.80430.log.gzgzip 2024-05-30T17:15:24.602 INFO:teuthology.orchestra.run.smithi006.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.54440.log 2024-05-30T17:15:24.602 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.75120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72760.log 2024-05-30T17:15:24.602 INFO:teuthology.orchestra.run.smithi006.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.112414.log.gz 2024-05-30T17:15:24.602 INFO:teuthology.orchestra.run.smithi006.stderr: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.75120.log.gz 2024-05-30T17:15:24.602 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.54440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135045.log 2024-05-30T17:15:24.602 INFO:teuthology.orchestra.run.smithi006.stderr: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.54440.log.gz 2024-05-30T17:15:24.602 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.72760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32055.log 2024-05-30T17:15:24.603 INFO:teuthology.orchestra.run.smithi006.stderr: 82.1% -- replaced with /var/log/ceph/ceph-client.admin.72760.log.gz 2024-05-30T17:15:24.603 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.135045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56965.log 2024-05-30T17:15:24.603 INFO:teuthology.orchestra.run.smithi006.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135045.log.gz 2024-05-30T17:15:24.603 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.32055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134810.log 2024-05-30T17:15:24.603 INFO:teuthology.orchestra.run.smithi006.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32055.log.gz 2024-05-30T17:15:24.603 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.56965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102852.log 2024-05-30T17:15:24.603 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.134810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41274.log 2024-05-30T17:15:24.603 INFO:teuthology.orchestra.run.smithi006.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134810.log.gz 2024-05-30T17:15:24.604 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.102852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31495.log 2024-05-30T17:15:24.608 INFO:teuthology.orchestra.run.smithi006.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.102852.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120594.log 2024-05-30T17:15:24.608 INFO:teuthology.orchestra.run.smithi006.stderr: 2024-05-30T17:15:24.608 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.41274.log: /var/log/ceph/ceph-client.admin.31495.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31495.log.gz 2024-05-30T17:15:24.608 INFO:teuthology.orchestra.run.smithi006.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.41274.log.gz 2024-05-30T17:15:24.609 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90462.log 2024-05-30T17:15:24.609 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.120594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84561.log 2024-05-30T17:15:24.609 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.90462.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.120594.log.gz 2024-05-30T17:15:24.609 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.33255.log 2024-05-30T17:15:24.609 INFO:teuthology.orchestra.run.smithi006.stderr: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.90462.log.gz 2024-05-30T17:15:24.609 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25847.log 2024-05-30T17:15:24.609 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.84561.log: /var/log/ceph/ceph-client.0.33255.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.84561.log.gz 2024-05-30T17:15:24.610 INFO:teuthology.orchestra.run.smithi006.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.0.33255.log.gz 2024-05-30T17:15:24.610 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57610.log 2024-05-30T17:15:24.610 INFO:teuthology.orchestra.run.smithi006.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.admin.56965.log.gz 2024-05-30T17:15:24.610 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.25847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68522.log 2024-05-30T17:15:24.610 INFO:teuthology.orchestra.run.smithi006.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25847.log.gz 2024-05-30T17:15:24.610 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.57610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38864.log 2024-05-30T17:15:24.610 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.0.68522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117511.log 2024-05-30T17:15:24.610 INFO:teuthology.orchestra.run.smithi006.stderr: 80.5% -- replaced with /var/log/ceph/ceph-client.0.68522.log.gz 2024-05-30T17:15:24.611 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.38864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92232.log 2024-05-30T17:15:24.611 INFO:teuthology.orchestra.run.smithi006.stderr: 83.8% -- replaced with /var/log/ceph/ceph-client.admin.38864.log.gz 2024-05-30T17:15:24.611 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.125292.log 2024-05-30T17:15:24.616 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.117511.log: /var/log/ceph/ceph-client.admin.92232.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.117511.log.gz 2024-05-30T17:15:24.617 INFO:teuthology.orchestra.run.smithi006.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.92232.log.gz 2024-05-30T17:15:24.617 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31286.log 2024-05-30T17:15:24.617 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134966.log 2024-05-30T17:15:24.617 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.0.125292.log: /var/log/ceph/ceph-client.admin.31286.log: 81.2% -- replaced with /var/log/ceph/ceph-client.0.125292.log.gz 2024-05-30T17:15:24.617 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99902.log 2024-05-30T17:15:24.617 INFO:teuthology.orchestra.run.smithi006.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31286.log.gz 2024-05-30T17:15:24.617 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53533.log 2024-05-30T17:15:24.617 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.134966.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134966.log.gz 2024-05-30T17:15:24.618 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.99902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.124702.log 2024-05-30T17:15:24.618 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.53533.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-30T17:15:24.618 INFO:teuthology.orchestra.run.smithi006.stderr: 57.5% 87.7% -- replaced with /var/log/ceph/ceph-client.admin.99902.log.gz 2024-05-30T17:15:24.618 INFO:teuthology.orchestra.run.smithi006.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53533.log.gz 2024-05-30T17:15:24.618 INFO:teuthology.orchestra.run.smithi006.stderr: 88.1%/var/log/ceph/ceph-client.0.124702.log: -- replaced with /var/log/ceph/ceph-client.admin.53837.log.gz 2024-05-30T17:15:24.618 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.33845.log 2024-05-30T17:15:24.618 INFO:teuthology.orchestra.run.smithi006.stderr: 84.4% -- replaced with /var/log/ceph/ceph-client.0.124702.log.gz 2024-05-30T17:15:24.618 INFO:teuthology.orchestra.run.smithi006.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.57610.log.gz 2024-05-30T17:15:24.619 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.129422.log 2024-05-30T17:15:24.619 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.0.33845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76300.log 2024-05-30T17:15:24.619 INFO:teuthology.orchestra.run.smithi006.stderr: 43.9% -- replaced with /var/log/ceph/ceph-client.0.33845.log.gz 2024-05-30T17:15:24.619 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.0.129422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96362.log 2024-05-30T17:15:24.619 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.76300.log: 84.3% -- replaced with /var/log/ceph/ceph-client.0.129422.log.gz 2024-05-30T17:15:24.619 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105212.log 2024-05-30T17:15:24.619 INFO:teuthology.orchestra.run.smithi006.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.76300.log.gz 2024-05-30T17:15:24.619 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31691.log 2024-05-30T17:15:24.620 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.96362.log: /var/log/ceph/ceph-client.admin.105212.log: 82.2% -- replaced with /var/log/ceph/ceph-client.admin.96362.log.gz 2024-05-30T17:15:24.620 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.126472.log 2024-05-30T17:15:24.620 INFO:teuthology.orchestra.run.smithi006.stderr: 82.1% -- replaced with /var/log/ceph/ceph-client.admin.105212.log.gz 2024-05-30T17:15:24.620 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46044.log 2024-05-30T17:15:24.620 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.31691.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31691.log.gz 2024-05-30T17:15:24.620 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.0.126472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111824.log 2024-05-30T17:15:24.620 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.46044.log: 84.6% -- replaced with /var/log/ceph/ceph-client.0.126472.log.gz 2024-05-30T17:15:24.620 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81021.log 2024-05-30T17:15:24.621 INFO:teuthology.orchestra.run.smithi006.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.46044.log.gz 2024-05-30T17:15:24.621 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.111824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96952.log 2024-05-30T17:15:24.621 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91642.log 2024-05-30T17:15:24.621 INFO:teuthology.orchestra.run.smithi006.stderr: 88.8%/var/log/ceph/ceph-client.admin.81021.log: -- replaced with /var/log/ceph/ceph-client.admin.111824.log.gz 2024-05-30T17:15:24.631 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.96952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31983.log 2024-05-30T17:15:24.631 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.91642.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.96952.log.gz 2024-05-30T17:15:24.631 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99312.log 2024-05-30T17:15:24.631 INFO:teuthology.orchestra.run.smithi006.stderr: 82.9%gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-30T17:15:24.631 INFO:teuthology.orchestra.run.smithi006.stderr: -- replaced with /var/log/ceph/ceph-client.admin.91642.log.gz 2024-05-30T17:15:24.631 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.31983.log: 86.4% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31983.log.gz 2024-05-30T17:15:24.631 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.99312.log: -- replaced with /var/log/ceph/ceph-client.admin.81021.log.gz 2024-05-30T17:15:24.631 INFO:teuthology.orchestra.run.smithi006.stderr: 82.1% -- replaced with /var/log/ceph/ceph-client.admin.99312.log.gz 2024-05-30T17:15:24.632 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.124112.log 2024-05-30T17:15:24.632 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35013.log 2024-05-30T17:15:24.632 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.0.124112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.132372.log 2024-05-30T17:15:24.632 INFO:teuthology.orchestra.run.smithi006.stderr: 80.3% -- replaced with /var/log/ceph/ceph-client.0.124112.log.gz/var/log/ceph/ceph-client.0.35013.log: 2024-05-30T17:15:24.632 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51429.log 2024-05-30T17:15:24.632 INFO:teuthology.orchestra.run.smithi006.stderr: 43.9% -- replaced with /var/log/ceph/ceph-client.0.35013.log.gz 2024-05-30T17:15:24.632 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134680.log 2024-05-30T17:15:24.633 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.0.132372.log: /var/log/ceph/ceph-client.admin.51429.log: 80.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73940.log 2024-05-30T17:15:24.633 INFO:teuthology.orchestra.run.smithi006.stderr: -- replaced with /var/log/ceph/ceph-client.0.132372.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51429.log.gz 2024-05-30T17:15:24.633 INFO:teuthology.orchestra.run.smithi006.stderr: 2024-05-30T17:15:24.633 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.134680.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134680.log.gz 2024-05-30T17:15:24.633 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48404.log 2024-05-30T17:15:24.633 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.73940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134628.log 2024-05-30T17:15:24.633 INFO:teuthology.orchestra.run.smithi006.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.73940.log.gz 2024-05-30T17:15:24.633 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.48404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31701.log 2024-05-30T17:15:24.634 INFO:teuthology.orchestra.run.smithi006.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.48404.log.gz 2024-05-30T17:15:24.634 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.134628.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134628.log.gz 2024-05-30T17:15:24.634 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42454.log 2024-05-30T17:15:24.634 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.31701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83971.log 2024-05-30T17:15:24.634 INFO:teuthology.orchestra.run.smithi006.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31701.log.gz 2024-05-30T17:15:24.634 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.42454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72170.log 2024-05-30T17:15:24.634 INFO:teuthology.orchestra.run.smithi006.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.42454.log.gz/var/log/ceph/ceph-client.admin.83971.log: 2024-05-30T17:15:24.634 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120005.log 2024-05-30T17:15:24.635 INFO:teuthology.orchestra.run.smithi006.stderr: 82.2% -- replaced with /var/log/ceph/ceph-client.admin.83971.log.gz 2024-05-30T17:15:24.635 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49044.log 2024-05-30T17:15:24.635 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.72170.log: /var/log/ceph/ceph-client.admin.120005.log: 82.4% -- replaced with /var/log/ceph/ceph-client.admin.72170.log.gz 2024-05-30T17:15:24.635 INFO:teuthology.orchestra.run.smithi006.stderr: 92.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122331.log 2024-05-30T17:15:24.635 INFO:teuthology.orchestra.run.smithi006.stderr: -- replaced with /var/log/ceph/ceph-client.admin.120005.log.gz 2024-05-30T17:15:24.635 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.49044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50224.log 2024-05-30T17:15:24.635 INFO:teuthology.orchestra.run.smithi006.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.49044.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60192.log 2024-05-30T17:15:24.635 INFO:teuthology.orchestra.run.smithi006.stderr: 2024-05-30T17:15:24.635 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.122331.log: /var/log/ceph/ceph-client.admin.50224.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.50224.log.gz 92.4% -- replaced with /var/log/ceph/ceph-client.admin.122331.log.gz 2024-05-30T17:15:24.636 INFO:teuthology.orchestra.run.smithi006.stderr: 2024-05-30T17:15:24.636 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98722.log 2024-05-30T17:15:24.636 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.60192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134732.log 2024-05-30T17:15:24.636 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.98722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36960.log 2024-05-30T17:15:24.636 INFO:teuthology.orchestra.run.smithi006.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.98722.log.gz/var/log/ceph/ceph-client.admin.134732.log: 2024-05-30T17:15:24.636 INFO:teuthology.orchestra.run.smithi006.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134732.log.gz 2024-05-30T17:15:24.636 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.125882.log 2024-05-30T17:15:24.647 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25862.log 2024-05-30T17:15:24.647 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-05-30T17:15:24.647 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.25862.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25862.log.gz -5 2024-05-30T17:15:24.647 INFO:teuthology.orchestra.run.smithi204.stderr: --verbose -- /var/log/ceph/ceph-osd.6.log 2024-05-30T17:15:24.647 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25838.log 2024-05-30T17:15:24.647 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-30T17:15:24.647 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.25838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25910.log 2024-05-30T17:15:24.647 INFO:teuthology.orchestra.run.smithi204.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25838.log.gz 2024-05-30T17:15:24.648 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25790.log 2024-05-30T17:15:24.648 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.25910.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25910.log.gz 2024-05-30T17:15:24.648 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-30T17:15:24.648 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-30T17:15:24.648 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.25790.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25790.log.gz 2024-05-30T17:15:24.648 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-30T17:15:24.648 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25941.log 2024-05-30T17:15:24.648 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25886.log 2024-05-30T17:15:24.649 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.25941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25814.log 2024-05-30T17:15:24.649 INFO:teuthology.orchestra.run.smithi204.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25941.log.gz 2024-05-30T17:15:24.649 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-30T17:15:24.649 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.25886.log: 0.0%/var/log/ceph/ceph-client.admin.25814.log: -- replaced with /var/log/ceph/ceph-client.admin.25886.log.gz 2024-05-30T17:15:24.649 INFO:teuthology.orchestra.run.smithi204.stderr: 90.9% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25814.log.gz 2024-05-30T17:15:24.649 INFO:teuthology.orchestra.run.smithi204.stderr: -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-30T17:15:24.649 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-30T17:15:24.649 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25965.log 2024-05-30T17:15:24.650 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.admin.25965.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25965.log.gz 2024-05-30T17:15:24.659 INFO:teuthology.orchestra.run.smithi204.stderr: 88.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-30T17:15:24.661 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106392.log 2024-05-30T17:15:24.661 INFO:teuthology.orchestra.run.smithi006.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.60192.log.gz 2024-05-30T17:15:24.661 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.36960.log: 82.9% -- replaced with /var/log/ceph/ceph-client.admin.36960.log.gz 2024-05-30T17:15:24.661 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.0.125882.log: 84.2% -- replaced with /var/log/ceph/ceph-client.0.125882.log.gz 2024-05-30T17:15:24.661 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31046.log 2024-05-30T17:15:24.662 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.106392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49634.log 2024-05-30T17:15:24.662 INFO:teuthology.orchestra.run.smithi006.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.106392.log.gz/var/log/ceph/ceph-client.admin.31046.log: 2024-05-30T17:15:24.662 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78070.log 2024-05-30T17:15:24.662 INFO:teuthology.orchestra.run.smithi006.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31046.log.gz 2024-05-30T17:15:24.662 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.49634.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.49634.log.gz 2024-05-30T17:15:24.662 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85741.log 2024-05-30T17:15:24.662 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.78070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66024.log 2024-05-30T17:15:24.662 INFO:teuthology.orchestra.run.smithi006.stderr: 82.8% -- replaced with /var/log/ceph/ceph-client.admin.78070.log.gz/var/log/ceph/ceph-client.admin.85741.log: 2024-05-30T17:15:24.663 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101082.log 2024-05-30T17:15:24.663 INFO:teuthology.orchestra.run.smithi006.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.85741.log.gz 2024-05-30T17:15:24.663 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.66024.log: 82.4% -- replaced with /var/log/ceph/ceph-client.admin.66024.log.gz 2024-05-30T17:15:24.663 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31273.log 2024-05-30T17:15:24.663 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.101082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31490.log 2024-05-30T17:15:24.663 INFO:teuthology.orchestra.run.smithi006.stderr: 82.2% -- replaced with /var/log/ceph/ceph-client.admin.101082.log.gz/var/log/ceph/ceph-client.admin.31273.log: 2024-05-30T17:15:24.663 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38274.log 2024-05-30T17:15:24.663 INFO:teuthology.orchestra.run.smithi006.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31273.log.gz 2024-05-30T17:15:24.663 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.31490.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31490.log.gz 2024-05-30T17:15:24.664 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25679.log 2024-05-30T17:15:24.664 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.38274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108184.log 2024-05-30T17:15:24.664 INFO:teuthology.orchestra.run.smithi006.stderr: 83.1% -- replaced with /var/log/ceph/ceph-client.admin.38274.log.gz/var/log/ceph/ceph-client.admin.25679.log: 2024-05-30T17:15:24.664 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.127652.log 2024-05-30T17:15:24.664 INFO:teuthology.orchestra.run.smithi006.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25679.log.gz 2024-05-30T17:15:24.664 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.108184.log: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.108184.log.gz 2024-05-30T17:15:24.664 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134888.log 2024-05-30T17:15:24.664 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.0.127652.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-30T17:15:24.664 INFO:teuthology.orchestra.run.smithi006.stderr: 84.2%/var/log/ceph/ceph-client.admin.134888.log: -- replaced with /var/log/ceph/ceph-client.0.127652.log.gz 2024-05-30T17:15:24.665 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65433.log 2024-05-30T17:15:24.665 INFO:teuthology.orchestra.run.smithi006.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134888.log.gz 2024-05-30T17:15:24.665 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31276.log 2024-05-30T17:15:24.665 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.65433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.32078.log 2024-05-30T17:15:24.665 INFO:teuthology.orchestra.run.smithi006.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.65433.log.gz 2024-05-30T17:15:24.665 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.31276.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67932.log 2024-05-30T17:15:24.665 INFO:teuthology.orchestra.run.smithi006.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31276.log.gz 2024-05-30T17:15:24.665 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.0.32078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56319.log 2024-05-30T17:15:24.665 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.0.67932.log: 96.0% -- replaced with /var/log/ceph/ceph-client.0.32078.log.gz 2024-05-30T17:15:24.666 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63662.log 2024-05-30T17:15:24.666 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.56319.log: 92.0% -- replaced with /var/log/ceph/ceph-client.0.67932.log.gz 2024-05-30T17:15:24.666 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66667.log 2024-05-30T17:15:24.666 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.63662.log: 95.1% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-05-30T17:15:24.666 INFO:teuthology.orchestra.run.smithi006.stderr: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.63662.log.gz 2024-05-30T17:15:24.666 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31690.log 2024-05-30T17:15:24.666 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.0.66667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69810.log 2024-05-30T17:15:24.666 INFO:teuthology.orchestra.run.smithi006.stderr: 84.3% -- replaced with /var/log/ceph/ceph-client.0.66667.log.gz 2024-05-30T17:15:24.667 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.31690.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31690.log.gz 2024-05-30T17:15:24.667 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91052.log 2024-05-30T17:15:24.667 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.69810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109364.log 2024-05-30T17:15:24.667 INFO:teuthology.orchestra.run.smithi006.stderr: 82.4% -- replaced with /var/log/ceph/ceph-client.admin.69810.log.gz 2024-05-30T17:15:24.667 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.91052.log: 86.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134836.log 2024-05-30T17:15:24.667 INFO:teuthology.orchestra.run.smithi006.stderr: -- replaced with /var/log/ceph/ceph-client.admin.91052.log.gz 2024-05-30T17:15:24.667 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.109364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53605.log 2024-05-30T17:15:24.667 INFO:teuthology.orchestra.run.smithi006.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.109364.log.gz 2024-05-30T17:15:24.667 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.134836.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134836.log.gz 2024-05-30T17:15:24.667 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115082.log 2024-05-30T17:15:24.668 INFO:teuthology.orchestra.run.smithi006.stderr: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.56319.log.gz/var/log/ceph/ceph-client.admin.53605.log: 2024-05-30T17:15:24.668 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58255.log 2024-05-30T17:15:24.668 INFO:teuthology.orchestra.run.smithi006.stderr: 51.3% -- replaced with /var/log/ceph/ceph-client.admin.53605.log.gz 2024-05-30T17:15:24.668 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31471.log 2024-05-30T17:15:24.668 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.115082.log: /var/log/ceph/ceph-client.admin.58255.log: gzip -5 --verbose -- /var/log/ceph/ops-log-ceph-client.admin.log 2024-05-30T17:15:24.668 INFO:teuthology.orchestra.run.smithi006.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.115082.log.gz 2024-05-30T17:15:24.668 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.31471.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31471.log.gz 2024-05-30T17:15:24.668 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134992.log 2024-05-30T17:15:24.668 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ops-log-ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48994.log 2024-05-30T17:15:24.669 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.134992.log: 89.9% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134992.log.gz 2024-05-30T17:15:24.669 INFO:teuthology.orchestra.run.smithi006.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58255.log.gz 2024-05-30T17:15:24.669 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79250.log 2024-05-30T17:15:24.669 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.48994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50814.log 2024-05-30T17:15:24.669 INFO:teuthology.orchestra.run.smithi006.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48994.log.gz 2024-05-30T17:15:24.669 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.79250.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79250.log.gz 2024-05-30T17:15:24.669 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-30T17:15:24.670 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.50814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103442.log 2024-05-30T17:15:24.670 INFO:teuthology.orchestra.run.smithi006.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.50814.log.gz 2024-05-30T17:15:24.670 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40684.log 2024-05-30T17:15:24.670 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.103442.log: 82.1% -- replaced with /var/log/ceph/ceph-client.admin.103442.log.gz 2024-05-30T17:15:24.680 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31283.log 2024-05-30T17:15:24.680 INFO:teuthology.orchestra.run.smithi006.stderr: 94.1% -- replaced with /var/log/ceph/ops-log-ceph-client.admin.log.gz 2024-05-30T17:15:24.681 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51454.log 2024-05-30T17:15:24.681 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.40684.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.40684.log.gz 2024-05-30T17:15:24.681 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.31283.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31283.log.gz 2024-05-30T17:15:24.681 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73350.log 2024-05-30T17:15:24.681 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.51454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118827.log 2024-05-30T17:15:24.681 INFO:teuthology.orchestra.run.smithi006.stderr: 83.9% -- replaced with /var/log/ceph/ceph-client.admin.51454.log.gz/var/log/ceph/ceph-client.admin.73350.log: 2024-05-30T17:15:24.681 INFO:teuthology.orchestra.run.smithi006.stderr: 82.3% -- replaced with /var/log/ceph/ceph-client.admin.73350.log.gz 2024-05-30T17:15:24.681 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134940.log 2024-05-30T17:15:24.682 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.118827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54406.log 2024-05-30T17:15:24.682 INFO:teuthology.orchestra.run.smithi006.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.118827.log.gz 2024-05-30T17:15:24.682 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.134940.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134940.log.gz 2024-05-30T17:15:24.682 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47814.log 2024-05-30T17:15:24.682 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.54406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31181.log 2024-05-30T17:15:24.682 INFO:teuthology.orchestra.run.smithi006.stderr: 20.3% -- replaced with /var/log/ceph/ceph-client.admin.54406.log.gz 2024-05-30T17:15:24.682 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.47814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107594.log 2024-05-30T17:15:24.682 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.31181.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.47814.log.gz 2024-05-30T17:15:24.682 INFO:teuthology.orchestra.run.smithi006.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31181.log.gz 2024-05-30T17:15:24.683 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106982.log 2024-05-30T17:15:24.683 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134550.log 2024-05-30T17:15:24.683 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.107594.log: /var/log/ceph/ceph-client.admin.106982.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106982.log.gz 2024-05-30T17:15:24.683 INFO:teuthology.orchestra.run.smithi006.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.107594.log.gz 2024-05-30T17:15:24.683 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102262.log 2024-05-30T17:15:24.683 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.134550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53774.log 2024-05-30T17:15:24.683 INFO:teuthology.orchestra.run.smithi006.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134550.log.gz 2024-05-30T17:15:24.683 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.102262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31477.log 2024-05-30T17:15:24.684 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.53774.log: 82.3% -- replaced with /var/log/ceph/ceph-client.admin.102262.log.gz 2024-05-30T17:15:24.684 INFO:teuthology.orchestra.run.smithi006.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53774.log.gz 2024-05-30T17:15:24.684 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78660.log 2024-05-30T17:15:24.684 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31285.log 2024-05-30T17:15:24.684 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.31477.log: /var/log/ceph/ceph-client.admin.78660.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31477.log.gz 2024-05-30T17:15:24.684 INFO:teuthology.orchestra.run.smithi006.stderr: 82.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31131.log 2024-05-30T17:15:24.684 INFO:teuthology.orchestra.run.smithi006.stderr: -- replaced with /var/log/ceph/ceph-client.admin.78660.log.gz 2024-05-30T17:15:24.684 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.31285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111799.log 2024-05-30T17:15:24.684 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.31131.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31285.log.gz 2024-05-30T17:15:24.685 INFO:teuthology.orchestra.run.smithi006.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31131.log.gz 2024-05-30T17:15:24.685 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62128.log 2024-05-30T17:15:24.685 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.128242.log 2024-05-30T17:15:24.685 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.111799.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111799.log.gz 2024-05-30T17:15:24.685 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.62128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104622.log 2024-05-30T17:15:24.685 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.0.128242.log: 80.8% -- replaced with /var/log/ceph/ceph-client.0.128242.log.gz 2024-05-30T17:15:24.685 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46634.log 2024-05-30T17:15:24.685 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.104622.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.104622.log.gz 2024-05-30T17:15:24.685 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76890.log 2024-05-30T17:15:24.686 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.46634.log: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.46634.log.gz 2024-05-30T17:15:24.686 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104032.log 2024-05-30T17:15:24.686 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.76890.log: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.62128.log.gz 2024-05-30T17:15:24.686 INFO:teuthology.orchestra.run.smithi006.stderr: 82.2% -- replaced with /var/log/ceph/ceph-client.admin.76890.log.gz 2024-05-30T17:15:24.686 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.130602.log 2024-05-30T17:15:24.686 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.104032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116922.log 2024-05-30T17:15:24.686 INFO:teuthology.orchestra.run.smithi006.stderr: 82.5% -- replaced with /var/log/ceph/ceph-client.admin.104032.log.gz 2024-05-30T17:15:24.686 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.0.130602.log: 80.6% -- replaced with /var/log/ceph/ceph-client.0.130602.log.gz 2024-05-30T17:15:24.686 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134602.log 2024-05-30T17:15:24.687 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.116922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31279.log 2024-05-30T17:15:24.687 INFO:teuthology.orchestra.run.smithi006.stderr: 92.4%/var/log/ceph/ceph-client.admin.134602.log: -- replaced with /var/log/ceph/ceph-client.admin.116922.log.gz 2024-05-30T17:15:24.687 INFO:teuthology.orchestra.run.smithi006.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134602.log.gz 2024-05-30T17:15:24.691 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134784.log 2024-05-30T17:15:24.691 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134524.log 2024-05-30T17:15:24.691 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.31279.log: /var/log/ceph/ceph-client.admin.134784.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31279.log.gz 2024-05-30T17:15:24.691 INFO:teuthology.orchestra.run.smithi006.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134784.log.gz 2024-05-30T17:15:24.691 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.34429.log 2024-05-30T17:15:24.692 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58902.log 2024-05-30T17:15:24.692 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.134524.log: /var/log/ceph/ceph-client.0.34429.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134524.log.gz 2024-05-30T17:15:24.692 INFO:teuthology.orchestra.run.smithi006.stderr: 43.9% -- replaced with /var/log/ceph/ceph-client.0.34429.log.gz 2024-05-30T17:15:24.692 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31703.log 2024-05-30T17:15:24.692 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119416.log 2024-05-30T17:15:24.692 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.58902.log: /var/log/ceph/ceph-client.admin.31703.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31703.log.gz 2024-05-30T17:15:24.692 INFO:teuthology.orchestra.run.smithi006.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58902.log.gz 2024-05-30T17:15:24.692 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53814.log 2024-05-30T17:15:24.693 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82791.log 2024-05-30T17:15:24.693 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.119416.log: /var/log/ceph/ceph-client.admin.53814.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53814.log.gz 2024-05-30T17:15:24.693 INFO:teuthology.orchestra.run.smithi006.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.119416.log.gz 2024-05-30T17:15:24.693 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43094.log 2024-05-30T17:15:24.693 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-30T17:15:24.693 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.82791.log: /var/log/ceph/ceph-client.admin.43094.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.82791.log.gz 2024-05-30T17:15:24.693 INFO:teuthology.orchestra.run.smithi006.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.43094.log.gz 2024-05-30T17:15:24.693 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116333.log 2024-05-30T17:15:24.694 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31106.log 2024-05-30T17:15:24.694 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.116333.log: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.116333.log.gz 2024-05-30T17:15:24.703 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49019.log 2024-05-30T17:15:24.704 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.31106.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31106.log.gz 2024-05-30T17:15:24.704 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52044.log 2024-05-30T17:15:24.705 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.49019.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49019.log.gz 2024-05-30T17:15:24.705 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55029.log 2024-05-30T17:15:24.706 INFO:teuthology.orchestra.run.smithi006.stderr: 93.7%/var/log/ceph/ceph-client.admin.52044.log: -- replaced with /var/log/ceph/ceph.log.gz 2024-05-30T17:15:24.706 INFO:teuthology.orchestra.run.smithi006.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.52044.log.gz 2024-05-30T17:15:24.707 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75710.log 2024-05-30T17:15:24.707 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.55029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37562.log 2024-05-30T17:15:24.708 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.75710.log: 82.4% -- replaced with /var/log/ceph/ceph-client.admin.75710.log.gz 2024-05-30T17:15:24.708 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39454.log 2024-05-30T17:15:24.709 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.37562.log: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.37562.log.gz 2024-05-30T17:15:24.709 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40634.log 2024-05-30T17:15:24.710 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.39454.log: 82.4% -- replaced with /var/log/ceph/ceph-client.admin.39454.log.gz 2024-05-30T17:15:24.710 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59547.log 2024-05-30T17:15:24.711 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.40634.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40634.log.gz 2024-05-30T17:15:24.711 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95182.log 2024-05-30T17:15:24.723 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.59547.log: 94.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31675.log 2024-05-30T17:15:24.724 INFO:teuthology.orchestra.run.smithi006.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55029.log.gz 2024-05-30T17:15:24.724 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.95182.log: 82.1% -- replaced with /var/log/ceph/ceph-client.admin.95182.log.gz 2024-05-30T17:15:24.724 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62779.log 2024-05-30T17:15:24.725 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.31675.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31675.log.gz 2024-05-30T17:15:24.725 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134498.log 2024-05-30T17:15:24.726 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.62779.log: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.62779.log.gz 2024-05-30T17:15:24.726 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69220.log 2024-05-30T17:15:24.727 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.134498.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134498.log.gz 2024-05-30T17:15:24.728 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.127062.log 2024-05-30T17:15:24.728 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.69220.log: 82.2% -- replaced with /var/log/ceph/ceph-client.admin.69220.log.gz 2024-05-30T17:15:24.729 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.133204.log 2024-05-30T17:15:24.730 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.0.127062.log: 80.9% -- replaced with /var/log/ceph/ceph-client.0.127062.log.gz 2024-05-30T17:15:24.730 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108774.log 2024-05-30T17:15:24.731 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.0.133204.log: 81.0% -- replaced with /var/log/ceph/ceph-client.0.133204.log.gz 2024-05-30T17:15:24.731 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45454.log 2024-05-30T17:15:24.732 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.108774.log: 82.2% -- replaced with /var/log/ceph/ceph-client.admin.108774.log.gz 2024-05-30T17:15:24.732 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110004.log 2024-05-30T17:15:24.733 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.45454.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.45454.log.gz 2024-05-30T17:15:24.733 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134862.log 2024-05-30T17:15:24.734 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.110004.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.110004.log.gz 2024-05-30T17:15:24.735 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52634.log 2024-05-30T17:15:24.735 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.134862.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134862.log.gz 2024-05-30T17:15:24.736 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31682.log 2024-05-30T17:15:24.737 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.52634.log: 82.4% -- replaced with /var/log/ceph/ceph-client.admin.52634.log.gz 2024-05-30T17:15:24.737 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107004.log 2024-05-30T17:15:24.738 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.31682.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31682.log.gz 2024-05-30T17:15:24.738 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-30T17:15:24.739 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.107004.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.107004.log.gz 2024-05-30T17:15:24.739 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87511.log 2024-05-30T17:15:24.740 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-mon.c.log: 95.2% -- replaced with /var/log/ceph/ceph-client.admin.59547.log.gz 2024-05-30T17:15:24.751 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25937.log 2024-05-30T17:15:24.753 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.87511.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.87511.log.gz 2024-05-30T17:15:24.754 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41864.log 2024-05-30T17:15:24.754 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.25937.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25937.log.gz 2024-05-30T17:15:24.755 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89282.log 2024-05-30T17:15:24.756 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.41864.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.41864.log.gz 2024-05-30T17:15:24.756 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31909.log 2024-05-30T17:15:24.757 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.89282.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.89282.log.gz 2024-05-30T17:15:24.757 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134706.log 2024-05-30T17:15:24.758 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.31909.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31909.log.gz 2024-05-30T17:15:24.758 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53319.log 2024-05-30T17:15:24.759 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.134706.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134706.log.gz 2024-05-30T17:15:24.759 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118100.log 2024-05-30T17:15:24.760 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.53319.log: 62.3% -- replaced with /var/log/ceph/ceph-client.admin.53319.log.gz 2024-05-30T17:15:24.761 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134394.log 2024-05-30T17:15:24.762 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.118100.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.118100.log.gz 2024-05-30T17:15:24.762 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83381.log 2024-05-30T17:15:24.763 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.134394.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134394.log.gz 2024-05-30T17:15:24.763 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ops-log-ceph-client.0.log 2024-05-30T17:15:24.764 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.83381.log: 82.7% -- replaced with /var/log/ceph/ceph-client.admin.83381.log.gz 2024-05-30T17:15:24.764 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64251.log 2024-05-30T17:15:24.776 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ops-log-ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70400.log 2024-05-30T17:15:24.776 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.64251.log: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.64251.log.gz 2024-05-30T17:15:24.787 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81611.log 2024-05-30T17:15:24.788 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.70400.log: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.70400.log.gz 2024-05-30T17:15:24.799 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97542.log 2024-05-30T17:15:24.800 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.81611.log: 82.3% -- replaced with /var/log/ceph/ceph-client.admin.81611.log.gz 2024-05-30T17:15:24.811 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134446.log 2024-05-30T17:15:24.812 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.97542.log: 82.3% -- replaced with /var/log/ceph/ceph-client.admin.97542.log.gz 2024-05-30T17:15:24.819 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-30T17:15:24.819 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.134446.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134446.log.gz 2024-05-30T17:15:24.827 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101672.log 2024-05-30T17:15:24.830 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph.audit.log: 94.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-30T17:15:24.835 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31480.log 2024-05-30T17:15:24.837 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.101672.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.101672.log.gz 2024-05-30T17:15:24.843 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71580.log 2024-05-30T17:15:24.843 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.31480.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31480.log.gz 2024-05-30T17:15:24.851 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32007.log 2024-05-30T17:15:24.852 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.71580.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.71580.log.gz 2024-05-30T17:15:24.859 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94002.log 2024-05-30T17:15:24.859 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.32007.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32007.log.gz 2024-05-30T17:15:24.867 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.32668.log 2024-05-30T17:15:24.867 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.94002.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.94002.log.gz 2024-05-30T17:15:24.875 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94592.log 2024-05-30T17:15:24.875 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.0.32668.log: 43.5% -- replaced with /var/log/ceph/ceph-client.0.32668.log.gz 2024-05-30T17:15:24.883 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109954.log 2024-05-30T17:15:24.883 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.94592.log: 82.4% -- replaced with /var/log/ceph/ceph-client.admin.94592.log.gz 2024-05-30T17:15:24.891 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31702.log 2024-05-30T17:15:24.892 INFO:teuthology.orchestra.run.smithi006.stderr: 94.4% -- replaced with /var/log/ceph/ops-log-ceph-client.0.log.gz 2024-05-30T17:15:24.892 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.109954.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109954.log.gz 2024-05-30T17:15:24.895 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86331.log 2024-05-30T17:15:24.895 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.31702.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31702.log.gz 2024-05-30T17:15:24.896 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89872.log 2024-05-30T17:15:24.896 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.86331.log: 82.1% -- replaced with /var/log/ceph/ceph-client.admin.86331.log.gz 2024-05-30T17:15:24.897 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79840.log 2024-05-30T17:15:24.898 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.89872.log: 82.7% -- replaced with /var/log/ceph/ceph-client.admin.89872.log.gz 2024-05-30T17:15:24.898 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43044.log 2024-05-30T17:15:24.899 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.79840.log: 82.1% -- replaced with /var/log/ceph/ceph-client.admin.79840.log.gz 2024-05-30T17:15:24.899 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44864.log 2024-05-30T17:15:24.900 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.43044.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43044.log.gz 2024-05-30T17:15:24.900 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.131192.log 2024-05-30T17:15:24.901 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.44864.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.44864.log.gz 2024-05-30T17:15:24.901 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/ceph.client.0.log 2024-05-30T17:15:24.902 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.0.131192.log: 84.0% -- replaced with /var/log/ceph/ceph-client.0.131192.log.gz 2024-05-30T17:15:24.903 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121742.log 2024-05-30T17:15:24.904 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/valgrind/ceph.client.0.log: 94.7% -- replaced with /var/log/ceph/valgrind/ceph.client.0.log.gz 2024-05-30T17:15:24.904 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134654.log 2024-05-30T17:15:24.906 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.121742.log: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.121742.log.gz 2024-05-30T17:15:24.906 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60838.log 2024-05-30T17:15:24.907 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.134654.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134654.log.gz 2024-05-30T17:15:24.907 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67257.log 2024-05-30T17:15:24.909 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.60838.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60838.log.gz 2024-05-30T17:15:24.910 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100492.log 2024-05-30T17:15:24.910 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.0.67257.log: 84.4% -- replaced with /var/log/ceph/ceph-client.0.67257.log.gz 2024-05-30T17:15:24.911 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88691.log 2024-05-30T17:15:24.912 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.100492.log: 82.5% -- replaced with /var/log/ceph/ceph-client.admin.100492.log.gz 2024-05-30T17:15:24.912 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54381.log 2024-05-30T17:15:24.913 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.88691.log: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.88691.log.gz 2024-05-30T17:15:24.913 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31883.log 2024-05-30T17:15:24.914 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.54381.log: 22.5% -- replaced with /var/log/ceph/ceph-client.admin.54381.log.gz 2024-05-30T17:15:24.914 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93412.log 2024-05-30T17:15:24.915 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.31883.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31883.log.gz 2024-05-30T17:15:24.916 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44274.log 2024-05-30T17:15:24.916 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.93412.log: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.93412.log.gz 2024-05-30T17:15:24.916 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114493.log 2024-05-30T17:15:24.917 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.44274.log: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.44274.log.gz 2024-05-30T17:15:24.918 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40659.log 2024-05-30T17:15:24.919 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.114493.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.114493.log.gz 2024-05-30T17:15:24.919 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.128832.log 2024-05-30T17:15:24.920 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.40659.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40659.log.gz 2024-05-30T17:15:24.920 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109979.log 2024-05-30T17:15:24.921 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.0.128832.log: 80.7% -- replaced with /var/log/ceph/ceph-client.0.128832.log.gz 2024-05-30T17:15:24.921 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70990.log 2024-05-30T17:15:24.922 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.109979.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109979.log.gz 2024-05-30T17:15:24.923 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.113901.log 2024-05-30T17:15:24.923 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.70990.log: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.70990.log.gz 2024-05-30T17:15:24.924 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113004.log 2024-05-30T17:15:24.924 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.0.113901.log: 84.9% -- replaced with /var/log/ceph/ceph-client.0.113901.log.gz 2024-05-30T17:15:24.925 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47224.log 2024-05-30T17:15:24.926 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.113004.log: 82.4% -- replaced with /var/log/ceph/ceph-client.admin.113004.log.gz 2024-05-30T17:15:24.926 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95772.log 2024-05-30T17:15:24.927 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.47224.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.47224.log.gz 2024-05-30T17:15:24.927 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92822.log 2024-05-30T17:15:24.929 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.95772.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.95772.log.gz 2024-05-30T17:15:24.931 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134472.log 2024-05-30T17:15:24.931 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.92822.log: 82.4% -- replaced with /var/log/ceph/ceph-client.admin.92822.log.gz 2024-05-30T17:15:24.931 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35597.log 2024-05-30T17:15:24.932 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.134472.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134472.log.gz 2024-05-30T17:15:24.932 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135021.log 2024-05-30T17:15:24.933 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.0.35597.log: 43.5% -- replaced with /var/log/ceph/ceph-client.0.35597.log.gz 2024-05-30T17:15:24.933 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111184.log 2024-05-30T17:15:24.934 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.135021.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135021.log.gz 2024-05-30T17:15:24.934 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74530.log 2024-05-30T17:15:24.935 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.111184.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.111184.log.gz 2024-05-30T17:15:24.935 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.123522.log 2024-05-30T17:15:24.936 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.74530.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74530.log.gz 2024-05-30T17:15:24.937 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85151.log 2024-05-30T17:15:24.937 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.0.123522.log: 84.7% -- replaced with /var/log/ceph/ceph-client.0.123522.log.gz 2024-05-30T17:15:24.938 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67897.log 2024-05-30T17:15:24.939 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.85151.log: 82.3% -- replaced with /var/log/ceph/ceph-client.admin.85151.log.gz 2024-05-30T17:15:24.939 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31233.log 2024-05-30T17:15:24.940 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.67897.log: 48.0% -- replaced with /var/log/ceph/ceph-client.admin.67897.log.gz 2024-05-30T17:15:24.940 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88101.log 2024-05-30T17:15:24.941 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.31233.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31233.log.gz 2024-05-30T17:15:24.941 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51404.log 2024-05-30T17:15:24.942 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.88101.log: 82.5% -- replaced with /var/log/ceph/ceph-client.admin.88101.log.gz 2024-05-30T17:15:24.942 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31269.log 2024-05-30T17:15:24.943 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.51404.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51404.log.gz 2024-05-30T17:15:24.943 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105802.log 2024-05-30T17:15:24.944 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.31269.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31269.log.gz 2024-05-30T17:15:24.944 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86921.log 2024-05-30T17:15:24.945 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.105802.log: 82.2% -- replaced with /var/log/ceph/ceph-client.admin.105802.log.gz 2024-05-30T17:15:24.946 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31207.log 2024-05-30T17:15:24.946 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.86921.log: 82.4% -- replaced with /var/log/ceph/ceph-client.admin.86921.log.gz 2024-05-30T17:15:24.947 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43684.log 2024-05-30T17:15:24.947 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.31207.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31207.log.gz 2024-05-30T17:15:24.948 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.130012.log 2024-05-30T17:15:24.949 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.43684.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.43684.log.gz 2024-05-30T17:15:24.949 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77480.log 2024-05-30T17:15:24.950 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.0.130012.log: 84.2% -- replaced with /var/log/ceph/ceph-client.0.130012.log.gz 2024-05-30T17:15:24.950 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111774.log 2024-05-30T17:15:24.951 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.77480.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.77480.log.gz 2024-05-30T17:15:24.952 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66615.log 2024-05-30T17:15:24.952 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.111774.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111774.log.gz 2024-05-30T17:15:24.953 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134914.log 2024-05-30T17:15:24.953 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.66615.log: 24.2% -- replaced with /var/log/ceph/ceph-client.admin.66615.log.gz 2024-05-30T17:15:24.954 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31935.log 2024-05-30T17:15:24.955 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.134914.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134914.log.gz 2024-05-30T17:15:24.955 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31287.log 2024-05-30T17:15:24.955 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.31935.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31935.log.gz 2024-05-30T17:15:24.956 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110594.log 2024-05-30T17:15:24.956 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.31287.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31287.log.gz 2024-05-30T17:15:24.957 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64843.log 2024-05-30T17:15:24.957 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.110594.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.110594.log.gz 2024-05-30T17:15:24.958 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61483.log 2024-05-30T17:15:24.959 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.64843.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.64843.log.gz 2024-05-30T17:15:24.960 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134758.log 2024-05-30T17:15:24.961 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.61483.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61483.log.gz 2024-05-30T17:15:24.962 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55674.log 2024-05-30T17:15:24.963 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.134758.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134758.log.gz 2024-05-30T17:15:24.963 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53886.log 2024-05-30T17:15:24.964 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.55674.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.55674.log.gz 2024-05-30T17:15:24.964 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31491.log 2024-05-30T17:15:24.965 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.53886.log: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.53886.log.gz 2024-05-30T17:15:24.965 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134420.log 2024-05-30T17:15:24.966 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.31491.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31491.log.gz 2024-05-30T17:15:24.967 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31688.log 2024-05-30T17:15:24.967 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.134420.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134420.log.gz 2024-05-30T17:15:24.968 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.122932.log 2024-05-30T17:15:24.968 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.31688.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31688.log.gz 2024-05-30T17:15:24.969 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.0.122932.log: 84.3% -- replaced with /var/log/ceph/ceph-client.0.122932.log.gz 2024-05-30T17:15:25.154 INFO:teuthology.orchestra.run.smithi204.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-30T17:15:26.019 INFO:teuthology.orchestra.run.smithi006.stderr: 93.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-30T17:15:26.154 INFO:teuthology.orchestra.run.smithi204.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-30T17:15:27.272 INFO:teuthology.orchestra.run.smithi006.stderr: 92.4% -- replaced with /var/log/ceph/rgw.ceph.client.0.log.gz 2024-05-30T17:15:28.030 INFO:teuthology.orchestra.run.smithi006.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-30T17:15:32.419 INFO:teuthology.orchestra.run.smithi006.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-30T17:15:34.357 INFO:teuthology.orchestra.run.smithi204.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-30T17:15:34.458 INFO:teuthology.orchestra.run.smithi006.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-30T17:15:34.501 INFO:teuthology.orchestra.run.smithi204.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-30T17:15:34.522 INFO:teuthology.orchestra.run.smithi204.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-05-30T17:15:37.118 INFO:teuthology.orchestra.run.smithi204.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-05-30T17:15:37.120 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-05-30T17:15:37.120 INFO:teuthology.orchestra.run.smithi204.stderr:real 0m12.505s 2024-05-30T17:15:37.121 INFO:teuthology.orchestra.run.smithi204.stderr:user 0m42.498s 2024-05-30T17:15:37.121 INFO:teuthology.orchestra.run.smithi204.stderr:sys 0m1.519s 2024-05-30T17:15:37.868 INFO:teuthology.orchestra.run.smithi006.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-30T17:15:44.745 INFO:teuthology.orchestra.run.smithi006.stderr: 94.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-30T17:15:44.748 INFO:teuthology.orchestra.run.smithi006.stderr: 2024-05-30T17:15:44.748 INFO:teuthology.orchestra.run.smithi006.stderr:real 0m20.176s 2024-05-30T17:15:44.748 INFO:teuthology.orchestra.run.smithi006.stderr:user 0m56.974s 2024-05-30T17:15:44.748 INFO:teuthology.orchestra.run.smithi006.stderr:sys 0m1.971s 2024-05-30T17:15:44.748 INFO:tasks.ceph:Archiving logs... 2024-05-30T17:15:44.749 DEBUG:teuthology.misc:Transferring archived files from smithi006:/var/log/ceph to /home/teuthworker/archive/cbodley-2024-05-30_15:09:56-rgw-wip-cbodley-testing-distro-default-smithi/7734120/remote/smithi006/log 2024-05-30T17:15:44.750 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-30T17:15:51.479 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/log/ceph to /home/teuthworker/archive/cbodley-2024-05-30_15:09:56-rgw-wip-cbodley-testing-distro-default-smithi/7734120/remote/smithi204/log 2024-05-30T17:15:51.480 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-30T17:15:56.470 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/qa/tasks/ceph.py", line 1913, in task with contextutil.nested(*subtasks): File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__ self.gen.throw(typ, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/qa/tasks/ceph.py", line 252, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/qa/tasks/ceph.py", line 346, in valgrind_post raise valgrind_exception Exception: valgrind error: MismatchedFree operator delete[](void*, unsigned long) MonClient::get_monmap_and_config() 2024-05-30T17:15:56.483 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-30T17:15:56.493 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/task/install/__init__.py", line 641, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/qa/tasks/ceph.py", line 1913, in task with contextutil.nested(*subtasks): File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__ self.gen.throw(typ, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/qa/tasks/ceph.py", line 252, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_da98024b78bbce5a0df00fc4fc8f11cdedc8a1e2/qa/tasks/ceph.py", line 346, in valgrind_post raise valgrind_exception Exception: valgrind error: MismatchedFree operator delete[](void*, unsigned long) MonClient::get_monmap_and_config() 2024-05-30T17:15:56.495 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-30T17:15:56.496 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-30T17:15:56.512 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-30T17:15:56.571 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-30T17:15:56.581 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-30T17:15:56.581 DEBUG:teuthology.orchestra.run.smithi006:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-30T17:15:56.584 DEBUG:teuthology.orchestra.run.smithi204:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-30T17:15:56.598 INFO:teuthology.orchestra.run.smithi006.stdout: remote refid st t when poll reach delay offset jitter 2024-05-30T17:15:56.598 INFO:teuthology.orchestra.run.smithi006.stdout:============================================================================== 2024-05-30T17:15:56.598 INFO:teuthology.orchestra.run.smithi006.stdout:+hv01.front.sepi 67.205.162.81 3 u 11 256 377 0.142 -2.429 0.296 2024-05-30T17:15:56.598 INFO:teuthology.orchestra.run.smithi006.stdout:+hv02.front.sepi 63.231.80.2 3 u 90 256 377 0.132 +0.313 0.296 2024-05-30T17:15:56.598 INFO:teuthology.orchestra.run.smithi006.stdout:*hv03.front.sepi 74.6.168.72 3 u 93 256 377 0.167 -0.483 0.138 2024-05-30T17:15:56.598 INFO:teuthology.orchestra.run.smithi006.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-05-30T17:15:56.600 INFO:teuthology.orchestra.run.smithi204.stdout: remote refid st t when poll reach delay offset jitter 2024-05-30T17:15:56.600 INFO:teuthology.orchestra.run.smithi204.stdout:============================================================================== 2024-05-30T17:15:56.600 INFO:teuthology.orchestra.run.smithi204.stdout:+hv01.front.sepi 67.205.162.81 3 u 92 128 377 0.102 -1.680 0.069 2024-05-30T17:15:56.601 INFO:teuthology.orchestra.run.smithi204.stdout:+hv02.front.sepi 63.231.80.2 3 u 47 128 377 0.117 +1.063 0.066 2024-05-30T17:15:56.601 INFO:teuthology.orchestra.run.smithi204.stdout:*hv03.front.sepi 74.6.168.72 3 u 38 128 377 0.108 -0.144 0.067 2024-05-30T17:15:56.601 INFO:teuthology.orchestra.run.smithi204.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-05-30T17:15:56.601 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-30T17:15:56.612 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-30T17:15:56.612 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-30T17:15:56.648 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-30T17:15:56.690 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-30T17:15:56.731 INFO:teuthology.task.internal:Duration was 5531.856380 seconds 2024-05-30T17:15:56.732 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-30T17:15:56.774 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-30T17:15:56.774 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-30T17:15:56.777 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-30T17:15:56.814 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-30T17:15:56.815 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi006.front.sepia.ceph.com 2024-05-30T17:15:56.815 DEBUG:teuthology.orchestra.run.smithi006:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-30T17:15:56.869 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi204.front.sepia.ceph.com 2024-05-30T17:15:56.869 DEBUG:teuthology.orchestra.run.smithi204:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-30T17:15:56.879 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-30T17:15:56.880 DEBUG:teuthology.orchestra.run.smithi006:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-30T17:15:56.913 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-30T17:15:56.949 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-30T17:15:56.949 DEBUG:teuthology.orchestra.run.smithi006:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-30T17:15:56.984 DEBUG:teuthology.orchestra.run.smithi204:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-30T17:15:57.046 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-30T17:15:57.058 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-30T17:15:57.058 DEBUG:teuthology.orchestra.run.smithi006:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-30T17:15:57.072 DEBUG:teuthology.orchestra.run.smithi204:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-30T17:15:57.084 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-30T17:15:57.107 DEBUG:teuthology.orchestra.run.smithi006:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-30T17:15:57.116 DEBUG:teuthology.orchestra.run.smithi204:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-30T17:15:57.126 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern = core 2024-05-30T17:15:57.137 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = core 2024-05-30T17:15:57.163 DEBUG:teuthology.orchestra.run.smithi006:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-30T17:15:57.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-30T17:15:57.187 DEBUG:teuthology.orchestra.run.smithi204:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-30T17:15:57.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-30T17:15:57.214 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-30T17:15:57.225 INFO:teuthology.task.internal:Transferring archived files... 2024-05-30T17:15:57.227 DEBUG:teuthology.misc:Transferring archived files from smithi006:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/cbodley-2024-05-30_15:09:56-rgw-wip-cbodley-testing-distro-default-smithi/7734120/remote/smithi006 2024-05-30T17:15:57.227 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-30T17:15:57.253 DEBUG:teuthology.misc:Transferring archived files from smithi204:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/cbodley-2024-05-30_15:09:56-rgw-wip-cbodley-testing-distro-default-smithi/7734120/remote/smithi204 2024-05-30T17:15:57.253 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-30T17:15:57.278 INFO:teuthology.task.internal:Removing archive directory... 2024-05-30T17:15:57.279 DEBUG:teuthology.orchestra.run.smithi006:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-30T17:15:57.289 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-30T17:15:57.323 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-30T17:15:57.335 INFO:teuthology.task.internal:Not uploading archives. 2024-05-30T17:15:57.335 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-30T17:15:57.399 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-30T17:15:57.400 DEBUG:teuthology.orchestra.run.smithi006:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-30T17:15:57.402 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-30T17:15:57.406 INFO:teuthology.orchestra.run.smithi006.stdout: 658464 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 30 17:15 /home/ubuntu/cephtest 2024-05-30T17:15:57.408 INFO:teuthology.orchestra.run.smithi204.stdout: 655475 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 30 17:15 /home/ubuntu/cephtest 2024-05-30T17:15:57.410 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-30T17:15:57.460 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-30T17:15:57.501 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: Exception: valgrind error: MismatchedFree operator delete[](void*, unsigned long) MonClient::get_monmap_and_config() 2024-05-30T17:15:57.505 INFO:teuthology.run:Summary data: description: rgw/verify/{0-install accounts$/{main-tenant} clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} duration: 5531.8563804626465 failure_reason: 'Command failed (s3 tests against rgw) on smithi006 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m ''not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3''"' flavor: default owner: scheduled_cbodley@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=816dc8dd342d46b79778613a2def831d status: fail success: false 2024-05-30T17:15:57.505 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-30T17:15:57.577 INFO:teuthology.run:FAIL